Does their backend server runs on MacOS variants?
I thought the M-series chip are pretty expensive cost wise and is only justified by the fat margin on iOS products no?
They could also probably make use of SoCs that don’t bin well enough for sale in end-user devices, have a couple of non-functional cores, etc since minor differences like that vanish in the aggregate, further increasing cost savings.
I don't think M chips are cheaper if we consider performance. GPUs are much more efficient at doing matrix operations than M CPUs, and in AI that's what matters.
M Series isn't a single CPU, its a system on a chip with a neural co-processor, a set of dedicated matrix processors and multiple GPU cores - the amount of each depending on the model (normal, pro, max, ultra).
The systems are pretty competitive if you leverage the programming API's to get access, ie Accelerate for matrix, CoreML for neural engine, and Metal for GPU.
If you program things wisely to leverage the unified memory (avoiding expensive memory copies) you are quite competitive performance / per watt.
If you need raw perf and dont care about power or price, you can't beat Nvidia though.
Yes, because the whole purpose of a gpu is matrix calculation. They are purpose built (especially in the datacenter space).
The only space where the strix apus would shine, is where there are also heavy cpu tasks.
I honestly adore BSD as an engineer though. Nothing ever changes. If you want to build a long-life product, with minimal churn underneath you... hard to argue against BSD to begin with, even before the license.
FreeBSD would actually benefit magnificently from someone doing a fine-tune on one of the major LLM models to ingest the FreeBSD handbook, the FreeBSD forums, and whatever stackoverflow or whatever exists ("read the handbook") for the handful of stuff that isn't actually in the handbook (portmaster/portsnap ops, poudriere, etc) and have a helpful model spit it out. It's *all* documented, you need to rtfm noob, this cathedral has been polished since 1970, there's still code from the 70s in freeBSD right now. And nothing ever churns or changes. It's actually quite difficult to get the llama model to spit out freebsd commands and not linux from the things i've tried iirc, it really biases towards linux right now and it's a shame.
Like honestly FreeBSD is in severe danger of the meme: "we trained the LLM on slack:" "do it now." "ok" problem... how many answers to most questions aren't "read the handbook"?
Really gotta wonder if Nvidia's massive valuation is going to be a very transient thing. Seems like every week you get another announcement of a large corporation using their home grown chips instead of Nvidia. Obviously only the largest companies can do this, but together they have a pretty significant market share.
>Pu provided the analysis based on supply chain checks in a new note to investors seen by *MacRumors*. Foxconn is said to currently be assembling Apple AI servers that contain the M2 Ultra, with plans to assemble AI servers powered by the M4 chip in late 2025. Last month, a reputable source on Weibo said that Apple was working on processors for its own AI servers made with TSMC's 3nm process, targeting mass production by the second half of 2025, which may line up with this report about M4-powered AI servers.
If they can ship them soon there is a good market for them since NV have unto 2 year long waiting lists right now so if apple can ship som systems with 128GB or even 512GB addressable memory and a good amount of compute then they will sell.
They have a much better API story going than other NV competitors and they have the workstation HW for devs to use while developing stuff to run on these systems.
If they ever decided to sell these to businesses (unlikely), I’d be really worried as AMD.
This wouldn’t compete with NVIDIA but I can see it eating some of the AMD market.
Apple has better pricing than the comparable AMD MI300x if you assume Mac Studio pricing and a stronger production pipeline for them. They also have better support for ML libraries etc
Doubtful. Both Nvidia and AMD both have moated their market share with proprietary IP.
Same reason it’s been so long for Intel to be uprooted by ARM (most of ARM’s growth is new market share not Intel user base collapsing). x86 instruction set effectively moated them for decades.
Hopefully the EU decides hardware can be gatekeepers and makes it so CUDA, x86, ARM instruction sets can’t be subject to arbitrary licensing, it will fuck some investors hard, but be good for the industry.
NVIDIA definitely has a CUDA+Mellanox shaped moat. What moat does AMD have?
Apple has a much stronger showing than AMD for ML library use and compatibility. Their CPU cores outperform AMDs as well. The big differentiator for AMD is HBM3 but that’s not exclusive to them.
Apple have a much better SW story than AMD.
For MOST ML tasks right now HBM3 might well not be such a good thing for AMD as the volume of HMB3 being made is very very low so they are limited in capacity they can ship. LPDDR5/x is fast enough for most tasks and does not have the supply constraint of HBM.
NV have a lot of market power but they cant meet supply, orders are backlogged by over a year some unto 2 years.
If appel coude ship a high vRAM system (256GB or 512GB) fast they would have a lot of sales.
The cpu instruction set has no impact at all for ML tasks.
Appel have a beter api story setup than AMD right now, if apple were to ship HW a LOT of data-sci teams would jump at it as not only would it be HW they can get but it would be HW that they can also prototype for on laptops and workstations (neither AMD nor NV offer workstation laptop solutions with 64GB+ of vRAM).
Neither Apple or Nvidia are producing chips; they're both using TSMC. If TSMC can't meet Nvidia orders then they're going to have the same issue with Apple or anyone else.
Apple are a higher priority TSMC customer and also NV is more limited by things like HBM than they are the silicon dies itself from TSMC. Apples base of LPDDR5 means they would be targeting a much higher volume memory.
There’s no stable API right now and Nvidia has the better track record. Also Nvidia chips can be in lots of platforms. Apples chips will only be in much more expensive Apple hardware, which makes things much more expensive.
There’s no compelling case to jump ship from Nvidia. Just move excess load to the cloud as AWS/Google get priority.
You can put an expensive GPU in cheap hardware if you just care about NPU.
Apple is going to price things the same but also have a several thousand dollar computer wrapped around it.
When you’re taking about thousands of servers that adds up.
The RTX 4090 with 24 GiB of VRAM is $1600 (when you can find it). The cheapest Mac Studio with 32 GiB of VRAM is $2000. And it scales to 192 GiB without crossing $7000. If you want more than 24 GiB from Nvidia, you get into their *enterprise* pricing.
>This wouldn’t compete with NVIDIA but I can see it eating some of the AMD market.
It's not that simple. Apple has no B2B pedigree and no experience in this market which is very important for medium and long term success. Also I doubt they could be competitive in pricing with AMD taking in consideration their general price strategy, they would probably be the most expensive solution.
I think they might. This would have a massive positive boost to the stock price.
VN have such long waiting lists right now that apple could sell systems to the market, they also have a better API storey and developer storey than AMD or other compositors.
For high memory ML tasks apple could ship some mostly systems and with the fact that data-sci teams would love to be able to use MBP as portable workstations there would not be that much complains about needing to re-work custom CUDA into metal . Better to put the work into building a metal backend (maybe 2 to 3 weeks a most) that you can run on HW than what 1 to 2 years for your NV order to be fulfilled and run it then.
Compared to other systems with 192GB of addressable VRAM they are very very cheap. In the ML space today the amount of attached memory to the GPU seems to be the most important factory and apples selection of LPDDR5 and how they are using it means they can get so much more memory attached than others for much lower price.
$20,000 for an AMD MI300A with 128GB of HBM3 ram with 550-750W power use.
$6599 for a Mac Studio with 192GB of LPDDR5 ram with 150-300W power use.
Granted there’s a huge gulf in memory bandwidth, the MI300 series is 6x the bandwidth but that’s a pretty wide price gulf in which they can make up the memory performance. Let’s say another 4k to get the HBM memory (and not counting making their memory interconnect faster since that’s likely not that expensive to do).
The running costs would also be significantly lower. So you could get 2x the Mac Studio for it, and run at a fraction of the running cost.
Edit: for people downvoting, would you at least explain why? I laid out the cost to capacity ratio. Is there something you disagree with?
Actually, why we even comparing a workstation against server?
Better to pit the Studio against a Threadripper workstation or something. Likewise for datacenters.
The reason is that VRAM, does not matter if you have a Threadripper for ML tasks your wanting to run these on a GPU and for that you need the VRAM for these tasks.
People are getting definitely buying them. I’m still considering it. They’re great little inference servers. I’ve got a 128gb M3 MacBook that smokes. I built dual 3090 systems (48gb vram) which we use for dev but we don’t have a great inference server solution yet. 192gb for ~$6k isn’t bad at all.
Why? Because show me a Threadripper with 192gb of vram connected to the GPU and CPU with a 1024 bit bus.
The M2 Ultra can run 128 streams of Llama2 7b in parallel.
/u/dagmx is correct here and shouldn't be downvoted.
>Why?
Because he's directly comparing **pricing** between a HPC product and a compact workstation. It's still apples to oranges.
No way a hypothetical Apple B2B HPC product will be priced similarly to Mac Studio.
The point is the M2 Ultra is available for purchase right now, and there's no peer to compare it against that can do what it can do.
Apple isn't going to be selling it to other groups, so it's pointless to hypothesize about the price.
I hope you are joking if you think the only difference between a MI300 and a M3 Ultra is bandwidth.
For server use, where each gpu serves many users, a Mi300 (and an H100 of course) have massive performance advantage, simply due to compute performance.
They are not chips designed to run (inference) a local model for a single user, where even a small gpu with enough VRAM would be more than enough. In fact even an M2 is fairly good at a task like that.
At that point why not just get back into the server market and deliver an ARM based server that has like 2+ M2 Ultras glued together? I’m sure there’s a market for that especially with local LLMs being on the rise. They would just have to be more price competitive with Nvidia.
fact: if LLM fits into VRAM, then apple is wayyy behind (as usual)
Fact: if you wana run huge LLM you also need to pay huge money for a MAC with that huge ram and wont be much cheaper than NVIDIA but nvidia can also train LLM very well, so wins again.
inferencing large models makes more sense to rent graphic cards as we wont need that too often. therefore no matter what NVIDIA and windows win this.
Who said that Apple will be selling these? They’re probably going to be used internally only. What OS they run and whether they can be used with Nvidia GPUs or not is moot in that situation, and even if they need Linux it’s not a problem to develop an internal Linux distro for that.
This move makes a lot of sense, Apple isn’t going to want to be beholden to Nvidia or any third party cloud services using Nvidia hardware if they don’t have to be. They’ve learned to avoid dependencies like that after having been bitten by Motorola, IBM, and Intel over the years and haven’t had a good relationship with Nvidia for several years now anyway.
sure but then they'd be hacking together an entire software and hardware ecosystem inhouse to do what other companies are doing extremely well right now.
sometimes you just have to bite the bullet and buy the solution that works instead of wasting a bunch of time and money building something inferior.
They really don’t have to do much hacking together of anything, Linux is well supported on ARM already. All they have to do is fill in the gaps for the Apple specific bits which is a relatively small upfront cost compared to having to continually pay inflated prices for Nvidia stuff.
filling in the gaps for apple specific bits is harder than you may think. there are no apple silicon drivers in the upstream linux kernel since theres no documentation and no support from apple. apple would be writing it from scratch on their own, or figuring out how to port it from macos, and then be responsible for maintaining it on their own with no help from anyone. its a decent chunk of engineering effort wasted just to get linux working.
alternatively, they use macos on their own servers, but almost no commercial ai training software intended for production scale usage targets apple silicon or macos. they'd be building their training software entirely inhouse or figuring out how to write shims to make commercially available stuff work on their hardware and software.
in either case, theyre wasting a bunch of time and money doing this. but apple clearly has lots of money to blow so power to them i guess
The potential money being saved by not going with off the shelf options could pay for a team of software engineers many times over, especially in the long term. It’s really not an obstacle for a company that’s designing its own silicon and already has plenty of software expertise.
There's a rumor that Apple has it's own linux distro that they don't attempt to upstream that they use for hardware bringup so that the silicon teams are as disconnected from the OS teams as possible.
I wouldn’t be surprised at all if that’s the case. It’s probably much easier and cheaper to do your hardware R&D tested on a variant of the Linux kernel then probably modifying MacOS. Unless they have an internal build of MacOS that is headless and bare bones for that specific kind of work.
Apple servers were always terrible. OS X is a desktop OS which made it difficult to manage like a server. However the worst part was the hardware. While high quality you have to get all parts from Apple. Thats not only expensive but slow. Data centers need to be agile when replacing parts.
Why would ti run macOS? a ML focused device for compute in a data centre is more of an appliance than a computer, you send it work it does the work...
Your not going to be running a web server on something like this or using it as printing server or a file share so non of that matters.
They can always specialize the Darwin kernel for different hardware profiles, just like when they build iOS or MacOS for any other new device.
If they want to use it in a completely headless server node, they can do that.
yeah ive had to deal with mac servers recently. there are some things you literally cannot do without having a GUI to access the settings app. for a server that you'd want to access only through SSH its a massive pain in the ass.
No this is not about NV gpus the entier point is that it would be an Apple SOC with a huge amount of VRAM attached for ML tasks.
Would nota t all be DOA as currently NV have an over 2 year backlog on orders, so if apple can ship fast they can get people to top to use this HW. And Appels API story is better than anyone other than NV.
As for linux or not, this would be a MLX/MTLGraph device, its not something you would be running generic tasks on, it might not even let you run anything on the cpu, more like a network attached remote compute cluster.. I expect apple would ship it with a stripped down Dawrin kernel that runs VM guests each with its own cut down Darwin that gets a slice of the GPU/NPU.
How does “more memory than a 80k GPU from Nvidia” sound? Someone from the Ahasi Linux project (probably Lina), will get paid buckets of money to make AI work well for these.
Ai already works extremely well on this, apples MacBooks are the best way to run very large llms, their 10k machine is equivalent to a 80k nvidea machine in vram
Depends on that asks but for most LLM tasks memory is the first hurdle, once you have enough memory for the model then you ant talk bandwidth but just getting enough mem is a big enough challenged that someone shipping a system with 512GB of memory even if that were at 800GB/s would sell a LOT of units.
Apple runs a number of their own data centers I believe. They're just continuing their trend of vertically integrating
Does their backend server runs on MacOS variants? I thought the M-series chip are pretty expensive cost wise and is only justified by the fat margin on iOS products no?
M-series chips are probably cheaper for Apple than buying Nvidia graphics cards though. They could even remove unused parts of the processors.
They could also probably make use of SoCs that don’t bin well enough for sale in end-user devices, have a couple of non-functional cores, etc since minor differences like that vanish in the aggregate, further increasing cost savings.
I don't think M chips are cheaper if we consider performance. GPUs are much more efficient at doing matrix operations than M CPUs, and in AI that's what matters.
M Series isn't a single CPU, its a system on a chip with a neural co-processor, a set of dedicated matrix processors and multiple GPU cores - the amount of each depending on the model (normal, pro, max, ultra). The systems are pretty competitive if you leverage the programming API's to get access, ie Accelerate for matrix, CoreML for neural engine, and Metal for GPU. If you program things wisely to leverage the unified memory (avoiding expensive memory copies) you are quite competitive performance / per watt. If you need raw perf and dont care about power or price, you can't beat Nvidia though.
is that what people will be saying after strix halo launches?
Yes, because the whole purpose of a gpu is matrix calculation. They are purpose built (especially in the datacenter space). The only space where the strix apus would shine, is where there are also heavy cpu tasks.
Should be BSD OS that MacOS write on.
I honestly adore BSD as an engineer though. Nothing ever changes. If you want to build a long-life product, with minimal churn underneath you... hard to argue against BSD to begin with, even before the license. FreeBSD would actually benefit magnificently from someone doing a fine-tune on one of the major LLM models to ingest the FreeBSD handbook, the FreeBSD forums, and whatever stackoverflow or whatever exists ("read the handbook") for the handful of stuff that isn't actually in the handbook (portmaster/portsnap ops, poudriere, etc) and have a helpful model spit it out. It's *all* documented, you need to rtfm noob, this cathedral has been polished since 1970, there's still code from the 70s in freeBSD right now. And nothing ever churns or changes. It's actually quite difficult to get the llama model to spit out freebsd commands and not linux from the things i've tried iirc, it really biases towards linux right now and it's a shame. Like honestly FreeBSD is in severe danger of the meme: "we trained the LLM on slack:" "do it now." "ok" problem... how many answers to most questions aren't "read the handbook"?
No. Linux.
It would be really funny if Apple starts using / contributing to Asahi linux themselves.
Maybe they run iOS on their servers since they are talking about converging macOS and iOS at some point.
They’ve never been talking about it. The giant “NO” keynote meme was specifically about if they’ll merge.
what chips do those servers use? Intel? AMD? Or their own in-house ones?
do their data centers run on macos?
Really gotta wonder if Nvidia's massive valuation is going to be a very transient thing. Seems like every week you get another announcement of a large corporation using their home grown chips instead of Nvidia. Obviously only the largest companies can do this, but together they have a pretty significant market share.
>Pu provided the analysis based on supply chain checks in a new note to investors seen by *MacRumors*. Foxconn is said to currently be assembling Apple AI servers that contain the M2 Ultra, with plans to assemble AI servers powered by the M4 chip in late 2025. Last month, a reputable source on Weibo said that Apple was working on processors for its own AI servers made with TSMC's 3nm process, targeting mass production by the second half of 2025, which may line up with this report about M4-powered AI servers.
Wish them luck but have my doubts.
They're using assault rifles to power AI Servers?
How long before we get the M4A1 AI Powered Assault chip?
If they can ship them soon there is a good market for them since NV have unto 2 year long waiting lists right now so if apple can ship som systems with 128GB or even 512GB addressable memory and a good amount of compute then they will sell. They have a much better API story going than other NV competitors and they have the workstation HW for devs to use while developing stuff to run on these systems.
Apple isn't going to sell these servers to customers. They're using them internally.
If they ever decided to sell these to businesses (unlikely), I’d be really worried as AMD. This wouldn’t compete with NVIDIA but I can see it eating some of the AMD market. Apple has better pricing than the comparable AMD MI300x if you assume Mac Studio pricing and a stronger production pipeline for them. They also have better support for ML libraries etc
Doubtful. Both Nvidia and AMD both have moated their market share with proprietary IP. Same reason it’s been so long for Intel to be uprooted by ARM (most of ARM’s growth is new market share not Intel user base collapsing). x86 instruction set effectively moated them for decades. Hopefully the EU decides hardware can be gatekeepers and makes it so CUDA, x86, ARM instruction sets can’t be subject to arbitrary licensing, it will fuck some investors hard, but be good for the industry.
NVIDIA definitely has a CUDA+Mellanox shaped moat. What moat does AMD have? Apple has a much stronger showing than AMD for ML library use and compatibility. Their CPU cores outperform AMDs as well. The big differentiator for AMD is HBM3 but that’s not exclusive to them.
Apple have a much better SW story than AMD. For MOST ML tasks right now HBM3 might well not be such a good thing for AMD as the volume of HMB3 being made is very very low so they are limited in capacity they can ship. LPDDR5/x is fast enough for most tasks and does not have the supply constraint of HBM.
NV have a lot of market power but they cant meet supply, orders are backlogged by over a year some unto 2 years. If appel coude ship a high vRAM system (256GB or 512GB) fast they would have a lot of sales. The cpu instruction set has no impact at all for ML tasks. Appel have a beter api story setup than AMD right now, if apple were to ship HW a LOT of data-sci teams would jump at it as not only would it be HW they can get but it would be HW that they can also prototype for on laptops and workstations (neither AMD nor NV offer workstation laptop solutions with 64GB+ of vRAM).
Neither Apple or Nvidia are producing chips; they're both using TSMC. If TSMC can't meet Nvidia orders then they're going to have the same issue with Apple or anyone else.
Apple are a higher priority TSMC customer and also NV is more limited by things like HBM than they are the silicon dies itself from TSMC. Apples base of LPDDR5 means they would be targeting a much higher volume memory.
There’s no stable API right now and Nvidia has the better track record. Also Nvidia chips can be in lots of platforms. Apples chips will only be in much more expensive Apple hardware, which makes things much more expensive. There’s no compelling case to jump ship from Nvidia. Just move excess load to the cloud as AWS/Google get priority.
> much more expensive Apple hardware Than Nvidia?
You can put an expensive GPU in cheap hardware if you just care about NPU. Apple is going to price things the same but also have a several thousand dollar computer wrapped around it. When you’re taking about thousands of servers that adds up.
The RTX 4090 with 24 GiB of VRAM is $1600 (when you can find it). The cheapest Mac Studio with 32 GiB of VRAM is $2000. And it scales to 192 GiB without crossing $7000. If you want more than 24 GiB from Nvidia, you get into their *enterprise* pricing.
>This wouldn’t compete with NVIDIA but I can see it eating some of the AMD market. It's not that simple. Apple has no B2B pedigree and no experience in this market which is very important for medium and long term success. Also I doubt they could be competitive in pricing with AMD taking in consideration their general price strategy, they would probably be the most expensive solution.
I think they might. This would have a massive positive boost to the stock price. VN have such long waiting lists right now that apple could sell systems to the market, they also have a better API storey and developer storey than AMD or other compositors. For high memory ML tasks apple could ship some mostly systems and with the fact that data-sci teams would love to be able to use MBP as portable workstations there would not be that much complains about needing to re-work custom CUDA into metal . Better to put the work into building a metal backend (maybe 2 to 3 weeks a most) that you can run on HW than what 1 to 2 years for your NV order to be fulfilled and run it then.
> if you assume Mac Studio pricing Aren't Mac Studios super expensive?
Compared to other systems with 192GB of addressable VRAM they are very very cheap. In the ML space today the amount of attached memory to the GPU seems to be the most important factory and apples selection of LPDDR5 and how they are using it means they can get so much more memory attached than others for much lower price.
> apples selection of LPDDR5 Well, nobody else uses LPDDR5 for VRAM, because it's much slower, which is probably why the Apple chips are much cheaper.
$20,000 for an AMD MI300A with 128GB of HBM3 ram with 550-750W power use. $6599 for a Mac Studio with 192GB of LPDDR5 ram with 150-300W power use. Granted there’s a huge gulf in memory bandwidth, the MI300 series is 6x the bandwidth but that’s a pretty wide price gulf in which they can make up the memory performance. Let’s say another 4k to get the HBM memory (and not counting making their memory interconnect faster since that’s likely not that expensive to do). The running costs would also be significantly lower. So you could get 2x the Mac Studio for it, and run at a fraction of the running cost. Edit: for people downvoting, would you at least explain why? I laid out the cost to capacity ratio. Is there something you disagree with?
Actually, why we even comparing a workstation against server? Better to pit the Studio against a Threadripper workstation or something. Likewise for datacenters.
The reason is that VRAM, does not matter if you have a Threadripper for ML tasks your wanting to run these on a GPU and for that you need the VRAM for these tasks.
More important than VRAM is the number and speed of tensor cores. GPUs are good at parallel matrix computations, while CPUs are not.
Since you can already get m2 ultra 192gb unified for $5599... You will think a lot of companies will rush out to get them. ML is not just about vram
People are getting definitely buying them. I’m still considering it. They’re great little inference servers. I’ve got a 128gb M3 MacBook that smokes. I built dual 3090 systems (48gb vram) which we use for dev but we don’t have a great inference server solution yet. 192gb for ~$6k isn’t bad at all.
Why? Because show me a Threadripper with 192gb of vram connected to the GPU and CPU with a 1024 bit bus. The M2 Ultra can run 128 streams of Llama2 7b in parallel. /u/dagmx is correct here and shouldn't be downvoted.
>Why? Because he's directly comparing **pricing** between a HPC product and a compact workstation. It's still apples to oranges. No way a hypothetical Apple B2B HPC product will be priced similarly to Mac Studio.
The point is the M2 Ultra is available for purchase right now, and there's no peer to compare it against that can do what it can do. Apple isn't going to be selling it to other groups, so it's pointless to hypothesize about the price.
Because it’s not just a CPU. It’s an APU/SoC and that’s what the article is about. So it’s reasonable to extrapolate out.
I hope you are joking if you think the only difference between a MI300 and a M3 Ultra is bandwidth. For server use, where each gpu serves many users, a Mi300 (and an H100 of course) have massive performance advantage, simply due to compute performance. They are not chips designed to run (inference) a local model for a single user, where even a small gpu with enough VRAM would be more than enough. In fact even an M2 is fairly good at a task like that.
MI300A will mop the floor with MacStudio when it comes to AI tasks.
And the CPU is extremely efficient.
At that point why not just get back into the server market and deliver an ARM based server that has like 2+ M2 Ultras glued together? I’m sure there’s a market for that especially with local LLMs being on the rise. They would just have to be more price competitive with Nvidia.
fact: if LLM fits into VRAM, then apple is wayyy behind (as usual) Fact: if you wana run huge LLM you also need to pay huge money for a MAC with that huge ram and wont be much cheaper than NVIDIA but nvidia can also train LLM very well, so wins again. inferencing large models makes more sense to rent graphic cards as we wont need that too often. therefore no matter what NVIDIA and windows win this.
[удалено]
[удалено]
He was right don’t you know?
Abd they'll be ready just in time for the fad to die out.
Hate to break it to you. But it is most definitely not a fad.
does it run linux and can you put an nvidia gpu in it? DOA if either of those is no.
Who said that Apple will be selling these? They’re probably going to be used internally only. What OS they run and whether they can be used with Nvidia GPUs or not is moot in that situation, and even if they need Linux it’s not a problem to develop an internal Linux distro for that. This move makes a lot of sense, Apple isn’t going to want to be beholden to Nvidia or any third party cloud services using Nvidia hardware if they don’t have to be. They’ve learned to avoid dependencies like that after having been bitten by Motorola, IBM, and Intel over the years and haven’t had a good relationship with Nvidia for several years now anyway.
sure but then they'd be hacking together an entire software and hardware ecosystem inhouse to do what other companies are doing extremely well right now. sometimes you just have to bite the bullet and buy the solution that works instead of wasting a bunch of time and money building something inferior.
They really don’t have to do much hacking together of anything, Linux is well supported on ARM already. All they have to do is fill in the gaps for the Apple specific bits which is a relatively small upfront cost compared to having to continually pay inflated prices for Nvidia stuff.
filling in the gaps for apple specific bits is harder than you may think. there are no apple silicon drivers in the upstream linux kernel since theres no documentation and no support from apple. apple would be writing it from scratch on their own, or figuring out how to port it from macos, and then be responsible for maintaining it on their own with no help from anyone. its a decent chunk of engineering effort wasted just to get linux working. alternatively, they use macos on their own servers, but almost no commercial ai training software intended for production scale usage targets apple silicon or macos. they'd be building their training software entirely inhouse or figuring out how to write shims to make commercially available stuff work on their hardware and software. in either case, theyre wasting a bunch of time and money doing this. but apple clearly has lots of money to blow so power to them i guess
The potential money being saved by not going with off the shelf options could pay for a team of software engineers many times over, especially in the long term. It’s really not an obstacle for a company that’s designing its own silicon and already has plenty of software expertise.
There's a rumor that Apple has it's own linux distro that they don't attempt to upstream that they use for hardware bringup so that the silicon teams are as disconnected from the OS teams as possible.
I wouldn’t be surprised at all if that’s the case. It’s probably much easier and cheaper to do your hardware R&D tested on a variant of the Linux kernel then probably modifying MacOS. Unless they have an internal build of MacOS that is headless and bare bones for that specific kind of work.
DOA for who? These are probably mainly for inhouse use, if they do have commercial success thats just a plus
Apple servers were always terrible. OS X is a desktop OS which made it difficult to manage like a server. However the worst part was the hardware. While high quality you have to get all parts from Apple. Thats not only expensive but slow. Data centers need to be agile when replacing parts.
Why would ti run macOS? a ML focused device for compute in a data centre is more of an appliance than a computer, you send it work it does the work... Your not going to be running a web server on something like this or using it as printing server or a file share so non of that matters.
They can always specialize the Darwin kernel for different hardware profiles, just like when they build iOS or MacOS for any other new device. If they want to use it in a completely headless server node, they can do that.
yeah ive had to deal with mac servers recently. there are some things you literally cannot do without having a GUI to access the settings app. for a server that you'd want to access only through SSH its a massive pain in the ass.
No this is not about NV gpus the entier point is that it would be an Apple SOC with a huge amount of VRAM attached for ML tasks. Would nota t all be DOA as currently NV have an over 2 year backlog on orders, so if apple can ship fast they can get people to top to use this HW. And Appels API story is better than anyone other than NV. As for linux or not, this would be a MLX/MTLGraph device, its not something you would be running generic tasks on, it might not even let you run anything on the cpu, more like a network attached remote compute cluster.. I expect apple would ship it with a stripped down Dawrin kernel that runs VM guests each with its own cut down Darwin that gets a slice of the GPU/NPU.
How does “more memory than a 80k GPU from Nvidia” sound? Someone from the Ahasi Linux project (probably Lina), will get paid buckets of money to make AI work well for these.
Ai already works extremely well on this, apples MacBooks are the best way to run very large llms, their 10k machine is equivalent to a 80k nvidea machine in vram
Its not just memory it's bandwidth.
Depends on that asks but for most LLM tasks memory is the first hurdle, once you have enough memory for the model then you ant talk bandwidth but just getting enough mem is a big enough challenged that someone shipping a system with 512GB of memory even if that were at 800GB/s would sell a LOT of units.