Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market.
That means they're actually incentivized at least short term, to benefit PCs becoming strong enough to do local LLMs. Which makes this play make even more sense. Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
> Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
"Death knell" is a touch hyperbolic. Hardware that can only run quantized models that take up GBs in VRAM falls short of even an A100 (by almost an order of magnitude[0]), which in turn falls short of what an 8xH100 cluster can do (also by another order of magnitude[0]).
I'm an avid believer in local LLMs, but I cannot deceive myself - data center accelerators will win on power dissipation numbers alone[1], even when giving generous allowances for higher efficiency on Apple chips - and assuming the Apple-efficiency advantage persists on the same TSMC process node.
0. Based on my unscientific fine-tuning training experiments across local and rented GPUs. YMMV for inference.
1. Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Doesn’t need to be a winner head to head. If it can do 90% of the tasks the big boys do, at 50% speed, for virtually no extra overhead cost save for the power consumed by a prompt - that’s gonna work for a lot of people. And that’s also basically where we’re at today. Qwen3.6 35b running quantized on 10 year old hardware solves basically all of my uses cases for agents except for coding.
The frontier models are faster, and better at coding, but not so much that i’ll pay $200/month for them.
Consider this. One of the smallest Qwen models (4B parameters) powers my home automation voice assistant, and runs on CPU alone at >20 tok/s. It is enough for that use case, and could be made even better/faster with a modest GPU. It isn't as smart as some cloud-connected thingamajig, but I would never allow a literal Google or Amazon bug in my home. Huge SOTA models aren't relevant everywhere. Most people use LLMs for rather trivial tasks such as finding typos or drafting text.
But with Apple's AFM 3 architecture, we might end up with huge SOTA adjacent on devices with limited RAM.
They use a technique where you only load between 1B and 4B of a 20B dense model for an entire prompt run, not token by token like a MoE, and use mostly the low power ANE instead of GPU cores.
Now, imagine if/when they scale up to 100B or more? On a chip using 2W?
I think we're also ignoring a potential innovative move in how models work.
If someone could splinter or fragment the models into more specific tasks i.e "spellchecker AI" and get these working as well as Sonnet 4.6-4.8 on those tasks on a personal laptop. You then question the $100 a month fee.
Bear in mind these laptops are likely to be $5000 or so because of the memory, HDD and M7 chip they likely need.
It feels to me like the beginning of the inflection point but software updates not hardware updates will be the accelerant.
"That’s where EMO comes in.
We show that EMO – a 1B-active, 14B-total-parameter (8-expert active, 128-expert total) MoE trained on 1 trillion tokens – supports selective expert use: for a given task or domain, we can use only a small subset of experts (just 12.5% of total experts) while retaining near full-model performance."
> If it can do 90% of the tasks the big boys do, at 50% speed
I want to live in this world too, but these numbers, as of today, are very aspirational and far removed from reality.
I'm no tokenmaxxer; I find my modest local setup useful, I also know the limitations, it's slow and it sucks (relatively) at high-level and/or long-context planning, compared to frontier models. Only a minority of my prompts are max-effort - its not all I do, but, it also means frontier labs aren't dying any time soon
Consider also that right now LLMs run slowly enough you can watch them think. I've seen a demo of an LLM running at an absurdly high speed and it reminds me of when I moved from a 2400 baud modem to a 14.4 - BBS screens that I could watch draw were all of a sudden nigh-interactive. Faster-than-realtime video generation is also coming, and will also continue to require huge hardware for a long while yet.
I love local models - I have a machine at home that runs a few for me and it's a lot of fun - but for the time being they are not super trustworthy on tool calls and staying on script. Another year or so might change all that!
Yea, is almost "scary fast" in a sense... the amount of compute you can do in parallel one one of those chips is amazing. hopefully they get their next chip completed as that will be a lot more useful for general workloads. I think the current one is based on llama3.
The weights they “etched” into the FPGA card that’s used for the ChatJimmy demo are that of a Llama 3-something 8b model.
The actually impressive and novel thing is that Taalas’ve managed to automate that process (clearly – nobody transforms 8 billion numbers into a physical representation by hand).
So now, they can work on scaling this process up, and with low enough lead times (I’ll be convinced they have inside connections to TSMC if they can actually deliver on the promised mere 3-4 months delay), will be able to offer 30-100b+ parameter models under half a year after they’re released, at thousands of tokens per second while probably drawing less wattage (per token, not sure about overall).
I’m sure you’re right, for the things you are asking of an llm, just as I am right about the things I am asking of an llm.
The real question is, what are 90% of people going to ask llms to do. I’d argue mostly it’s going to be stuff that works-now or almost-works on local models, but that’s just an opinion. It also depends on the frontier models hitting a wall of steeply diminishing returns, since they set the expectations for all of this stuff - my gut says that’s happened already they just won’t admit it for a while - but we’ll see.
This is what makes sense for me as well. All I need a local model is for playing with simple graphics: no gradients, at most ten colours which I can push through VTracer to get an SVG. Draw Things does the job, usually in 120 seconds or less.
Sometimes, I need a quick throwaway bit of python. That can take 30 minutes of my time.
We'll likely see a transformation in how frontier models are trained as a result of a push towards local inference. While it seems unlikely now, given current pricing for RAM, in 10-15 years it's not unthinkable to assume we could see individual machines with 10-12TB (and well beyond that) of RAM which are accessible to the GPU. Min/max system RAM increased a LOT from 2010-2025 and largely because it was cheap. Once the hyperscalers aren't generating revenue for the RAM manufacturers, I wouldn't be surprised to see a massive push towards consumers in order to maintain gross profit. Not to mention new players who enter the market because the margins are measurably absurd right now.
At some point there will be diminishing returns towards the "just throw more RAM at it" approach the current frontier models are taking. Commoditization is just as inevitable as it ever was... and in doing so will enable actual leaps of what AI/ML is capable of. That's not to say there won't be a place for 99.999999% accurate vs 99.99999% but those cases will be limited and likely prime to disruption based on real innovation vs access to capital.
10TB is about 80 times that, 200K in today’s money. A lot of capacity is coming online in the next 5 years and it’s reasonable to think we can get there with better process and stacking (the latter does little for pricing, but enables shorter latencies).
The established AI players have no financial interest to make LLM available locally. They aren't hardware companies and if running LLM requires paying them to host the models as well then they can naturally capture more of the value chain = more revenue.
Apple is the only player here where it would play into their natural hardware incentive to get you to pay more for better hardware. It would make sense for them to find a way to run LLM locally (eg, newer architectures that others here have pointed out).
Is it hyperbolic though? One of the best things about the compute and memory shortage is that people are going to insane lengths to optimize things to run on lower memory / lower compute devices. If we keep this up for a while and then ramp up memory and local compute production, that AI inflection point may actually come.
That’s about 4 years in hardware cadence alone. There is a lot of room to improve memory bandwidth, and performance is a given with every process node. IBM has shown yesterday they can do limited runs on 0.7nm (density equivalent).
The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
There is some signal that this is possible through both hardware innovation and training/data improvements.
Cloud models have their own constraints - I can’t have opus4.8 spend 4 hours on a deep research question I had in the shower without spending money. I can’t do real time video game upscaling and graphics work in the cloud period.
A laptop is about an order of magnitude cheaper than a cloud server thanks to economies of scale, uptime requirements, and other factors.
> The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
Benchmarks maybe? Real world, no.
You just need the context otherwise. There's no way around it.
Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc.
Whether such a model exists or not is a different question.
if you do the electricity math you'll see that you pay more on local models while getting less (local is more heavily quantized) compared with OpenRouter.
I'm not talking local Gemma/Qwen vs cloud Opus, but against OpenRouter same Gemma/Qwen
there are reasons to run local - privacy, availability, but cost is not one of them
I am allowed to plug in 800w of solar panels into a wall socket here in spain. That would more then cover my current computer with 16gb vram. Now if i went and built a LLM server, at full load i would probably be closer to 3600w (Dual Epyc CPUs that gives you 8 x16 PCI channels and up to 8 cards - Way overkill, i know). If i half that with 1 EPYC and 4 x16 PCI channels, and add the same amd 7800xt i currently have then i should in theory be able to run at around 1800w under full load. Now that could still be covered with a 2000w solar install (get a professional setup OR get a battery unit like a EcoFlow that can output 2000w and can input about the same amount of solar).
Now, this all brings the upfront costs way up, the solar panels are cheap, its all the rest around them that tends to cost money.
There has been a lot of market-subsidy in AI which is starting to fade away: e.g. the copilot quotas/pricing. When VC switches from investing to wanting a return, the price equation is likely to change.
for me it would be about $2 per day in electricity to generate 8 mil tok of Gemma4-26B at 4 bit quantization. this is excluding how much the GPU cost (no amortization)
ignoring the fact that I could get more free tokens per day for this model from Google/OpenRouter, it would cost $4 per day on OpenRouter if paid, but they would run it at full 16 bit precission
this would be the most "profitable" model for me
for Gemma4-31B I can generate only 1 mil tok per day, and so I pay more to get less quality than OpenRouter (ignoring that this model is also free on Google)
I know how many tokens per second my computer generates, and I have a wall power meter which measures how many watts my computer is using when generating.
The thing is, with the level of hard investment AI vendors have, even a small reduction of their addressable market is significant. They aren’t profitable, and inference is getting commoditized fast, so even if they eventually become profitable (not via financial engineering) they won’t be able to have good margin. The pressure of both open models AND local models is pretty bad imho
> Hardware that can only run quantized models that take up GBs in VRAM
That's the today hardware.
Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output" and then releases M7 devices with enough memory and compute to run bigger models.
> Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Laptops maybe. Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet.
> Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output"
It's revealing that they aren't doing this: no one wants to fund that gamble on the state of AI demand 12-18 months out, but ate happy to capitalize on their current product lines/capacity.
> Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet
100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
They are doing this. All three of the DRAM manufacturers are actively expanding capacity. It's just not an instantaneous process.
> 100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
A desktop with the performance and the 160GB of VRAM of two H100s would sell like hotcakes. The power consumption is also largely proportional to the amount of compute rather than memory, and can be significantly improved by using lower clock speeds and then either making up for it with wider silicon or just not caring that it can run the same model but is e.g. 25% slower than hardware that uses twice as much electricity.
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in terms of capability, there's no reason not to scale more, there's no incentive for billionaires not to grab all the fab capacity they can.
Enjoy paying $1000 or more for a little 4 GiB cloud terminal that connects you to all your online accounts where all your actual work gets done. This is the future.
There's a limit that won't be breached without a fundamental breakthrough in physics of computation, but we're not there yet by a long shot. You can train bigger models, faster, and infer with them faster and more precisely, by throwing more compute at the problem for the foreseeable.
At some point, and I can already see it, they’ll be better than us at writing code. We are still in the loop to coerce them into architecting well, but that’s nothing magical.
What’s frontier now is prosumer in a couple years and commonplace in a couple more.
It's plausible but is the Apple Tax for a 1TB memory machine on top of current memory prices really worth it? I paid around $4000 for 4090m laptop with 16GB VRAM back in 2023, it's great but DoA for even quantized LLMs. I can run SLMs and fine tune it but that's it.
We need one of those specialized inference chip startups to succeed and a PC manufacturer willing to bet on them against Nvidia for the local AI to find mass market appeal.
I recently bought a Mac mini M4 16 GB - mostly to run Immich. I assumed I needed a Linux box. After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax.
I'm not sure why you would need large AI models for Immich, the face detection is pretty cheap and will run on 10 year old hardware without a blip.
I think the decision comes primarily on how much data you would like to store for Immich, if you want to go cheaply, a 100 bucks used laptop will do the job, if you have too much data, a NAS will be more suitable (and you are certainly not going to get a mac where you can plug multiple internal hard drives for the price of a NAS)
>" After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax."
Apple has always been the most cost effective choice for the value you get going all the way back to the Apple II, it's just that the floor of that cost has always been high. Anyone who thinks otherwise is a just a fanboy one way or the other.
That's true only for the entry level macs. My M4 Mac Mini has the best Performance/value. But my workstation laptop with 32 cores, 96GB DDR5, Nvidia GPU costs lesser than Macs with lesser performance; not to mention I upgraded the RAM post purchase.
It really depends what you factor in as value, because wintel laptops like you described tend to require noise canceling headphones when working on them.
If you're happy with the base model, then Apple has a great value/price ratio, but there's nothing below it and any upgrade is really bad value for the money.
That's how much many developers currently spend on tokens - every day. Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
>Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
Current high-end Mac Studio with 32-core M3 Ultra chip and 96 GB of memory is $6800, 96GB is not enough to run GLM 5.2 without extreme quantization or stacking HW; but for the sake of discussion let's run quantized version on a single high end Mac Studio.
GLM 5.2 Max plan costs $ 112/m, so it would take ~60 months to recover the costs assuming the machine was bought just for AI. By then the current AI landscape would have changed drastically.
I use local AI on both Linux and Mac every single day, there's freedom, privacy and peace of mind in running the model locally. But I feel cost/value of local AI is overblown.
"Apple tax" is such a lazy and inaccurate accusation to level. Sure we've had expensive wheels on the cheese grater (ie Mac Pro) but we've also had:
1. When Apple came out with the real Macbook Air in 2010/2011 (not the silly 2008 one), nobody could compete with it with those specs at that price and they couldn't for years. And every competitor usually sucked in some major way, most often the trackpad;
2. The Mac Mini is an outstanding piece of hardware for $600. Or was;
3. I've generally found that "Apple tax" complaints levelled against the iPhone to be nothing more than Android cope;
4. The M-series silicon has been an absolute game-changer. I honestly thought the first-generation M1s would be not great but they came out swinging. And the price points for these Macbooks have all been great, much better than the last-gasp-of-Johnny-Ive touch bar butterfly keyboard series, which were objectively awful.
I didn't have a single Apple device in my house until a month ago when I bought a Neo. The last Apple devices I had before that were an iPod Nano and a PowerMac G5 many many years ago.
Apple has pretty good competition in every segment with the exception of maybe the iPad, but I'm not a tablet user.
Some folks like to have a computing environment free of proprietary influences and extremely strong vendor lock-in. I cannot claim to posses any apple devices.
I wasn't thinking of Asahi. Just pointing out that you can run all the standard unix/open source tools and apps on Mac OS (vi, git, qgis, blender, vsc, python, node, etc). With the advantage of higher quality hardware and generally less fiddling.
But if you don't like it, switch. I don't see vendor lock-in.
there a many people who don't own Apple. Why are you so surprised? I certainly don't and never will. What's it got that I can't get on a standard PC + Linux?
Tangential: About 8 years ago ex-Apple chip engineers left to design server-grade chips, this was Nuvia, and they got sued by Apple to the point that they had to get acquired by Qualcomm.
I worked at a hyperscaler when the M1 came out. A MacBook Air M1, running a Linux VM was faster and more energy efficient than anything we had in the data center.
Nope, across all benchmarks, Linux running in a VM on the M1 had higher perf (4 cores) than any instance type I launched across Intel, AMD and Arm. Triad, coremark, openfoam, verilator, etc. This may have changed, but my hunch is this is still probably true with M5 vs all the cloud providers.
That's not especially surprising for low thread counts. e.g. if you look at the current Geekbench single core charts, the fastest device across Android/iOS/Mac/PC is the M5 MacBook Pro. Second-fastest is the M5 iPad Pro. The fastest PC CPU (9950X3D) is about 80% the performance of the M5. (And the 9950X3D is about on par with the A18 in the iPhone 16 from 2024.)
(All the usual caveats about geekbench scores apply, but they're not nothing.)
>AND basically no play I'm the hyperscalar market.
I do wish they have Xserve back, or a Mac Pro that is Rack Based and support multiple node with M6 Ultra. The Hyperscaler market is so large along with AI their old business case of Xserve didn't make sense no long hold true.
They do stand in front of a great opportunity that would also benefit consumers, which seems rare in the llm era.
If people can get opus4.6/gpt5.5-like models locally, labs could raise their prices and sell token speed, better reasoning, mobile-focused improvements, you name it.
Not all consumers are power users and many will be happy to pay for flexibility.
The comparison is against your own computer, which is powerful enough, no need to add a box and set it up, and a very expensive cloud service. It’s nothing like setting up a NAS versus using iCloud (because we are talking Apple here).
LOL at the downvotes. I'm sure that the shareholders aka wealthy people at the top have a lot of patience for huge gambles that don't pay off. If OpenAI and Anthropic turn out to be the IT equivalent of mom and pop stores 5 years from now, their current shareholders will be ecstatic!
768GB RAM pipe dreams make no sense to Apple. By discontinuing 256GB / 512GB M3 Ultra and raising prices $5000 -> $7000 on Macbook pro with 128GB they basically confirmed how badly RAM shortage affecting them.
768GB is 64-times of 12GB which is rumored to be amount of RAM in new iPhones. Imagine what profit margin 768GB Mac Studio gonna need in order to justify making one instead of 64 iPhones.
Apple is the company that is okay about selling microfiber cloth for $100 and wheels for $700. Imagine how bad price hike for M3 Ultra 256GB / 512GB had to be in order for them to just discontinue them instead of getting free money out of desperate local AI folks.
They could treat the extreme spec machines separately from the prosumer ones, like they did with the Xserve. Let business customers spec up to 768GB (say) who are prepared for a $20-25k price tag, while keeping them away from the stores and usual consumer supply chains (Amazon et al). It may not be a big enough market segment for them to care about anymore, though.
They can do it, but that gonna need to be different SKU not Mac Studio. Otherwise news will be full of discussions about Apple price hike from $8000 to $24,000 or who the hell knows $48,000.
So yeah the only way I see them selling it is usual "call us" enterprise price tag.
But since its not what Apple usually do its easier to sell 4x Mac Studio 256GB RAM boxes with interconnect for lets say $12,000 - $15,000 each.
I don't think it's needs 'call us' just a separate SKU, I mean if they called it Xserve Ultra and it was just a studio in a 1u format with dual PSUs and extra RAM, it would fly off the shelves.
Isn't that the same thing you can get from ordinary 2S Epyc/Xeon servers at a similar price that have 24 memory channels (when the M3 Ultra has the equivalent of 16)?
And the reason people rarely use that for AI is that the enterprise GPUs from AMD and Nvidia are only moderately more expensive but are significantly faster because they use HBM instead of DDR5.
Yeah kind of, I think a 24 channels DDR5 works out approx 1TB/s, but the cost is astronomical, a M5 studio would probably beat that performance for around half the cost. You also get to use the GPU/NPU cores of the mac vs CPU only on the servers.
M5 ultra studio with 128GB RAM could probably beat out a sever with a RTX 6000 pro at half the price.
24 channels of DDR5-6400 is 1.2TB/s, M3 ultra is 0.8TB/s. They both use DDR5-6400.
> a M5 studio would probably beat that performance for around half the cost.
A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want. The price seems pretty comparable. The "problem" with doing this is actually that 128GB is too little memory, because you want to populate all the channels, but even using 16GB sticks, 24x16GB is already 384GB.
> You also get to use the GPU/NPU cores of the mac vs CPU only on the servers.
You only need enough cores to make sure the bottleneck is memory bandwidth.
M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now. Even if this was M4 Ultra it would still be ~comparable to any EPYC system in bandwidth, but get to use the GPU for compute so potentially faster than the EPYC. Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk. More RAM though, but again it's more about the ratio of RAM (size) to RAM (Memory Bandwith) to Compute and you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth and therefore the extra RAM (size) is wasted. But maybe not, different use cases will yeild different results I guess.
> A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want.
As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200, then you also need at least one NVME disk so you're looking at what $13,000?
> M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now.
M3 Ultra uses a 1024-bit memory bus, which is a major inconvenience to Apple because they're soldering everything. In ordinary systems if you have 16 memory slots and any one of the memory chips is bad, you replace that stick. If the processor is bad, you replace the processor. If the system board is bad, you transfer the processors and memory to another one.
If any of those has a defect after you solder thousands of dollars worth of memory onto the same board as a >$1000 CPU, you're not doing well. Worse, the more memory chips you have and the more pins the CPU needs for its memory bus, the higher the chances of one of them having a defect.
In addition to that, when you get to that number of pins it starts getting harder to run the memory at the highest speeds. The M3 uses DDR5-6400 but some of the M5 line is using DDR5-9600. It may or may not be possible to do that speed when using a 1024-bit bus -- not every existing M5 even does it. If it isn't then the Ultra wouldn't be much faster than the Max since it would have to use a lower memory speed. If it is then the tolerances would have to be even tighter and increase the defect rate even more.
Which is to say, I can see why they haven't released an Ultra since the M3.
> but get to use the GPU for compute so potentially faster than the EPYC.
Only if the bottleneck is compute rather than memory bandwidth, and for LLMs it's generally memory bandwidth. And if something significant was compute bound, there are also higher core count CPUs.
> Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk
The M3 Ultra has a 480W TDP. There are relevant EPYC SKUs on SP5 at 125-200W/socket.
> you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth
The extra RAM allows you to fit the larger model in memory to begin with, without which it's pretty hopeless. The bottleneck typically is memory bandwidth for LLMs, but that's true pretty much regardless of the model size. Moreover, mixture of experts models require significantly more RAM for the same amount of compute/bandwidth.
> you also need at least one NVME disk
That's ~$100.
> As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200
The chips don't cost a materially different amount based on whether you solder them. Apple presumably discontinued the 256GB and 512GB versions of the M3 Ultra because they'd have had to add a similar number to the price.
Using a server system designed for having hundreds of cores and TBs of RAM for LLMs because it has a medium-high amount of memory bandwidth is a hack for enthusiasts who want to take the price/performance trade off to run big models without paying for enterprise GPUs. The market for 8GB RDIMMs would be specifically that, since the more typical server workloads that need that amount of memory bandwidth also want the larger memory sticks (e.g. database servers, systems running hundreds of VMs), or aren't bounded by memory bandwidth to begin with and then don't need to populate all the channels.
And if you wanted that market segment then what you'd really do is produce a consumer GPU with 128GB of RAM.
> That would be the kind of thing you could sell as many as you could make.
People said that about the M1 Ultra Mac Pro, a few years before it was discontinued. I don't think there are many HPC customers looking at Apple hardware.
> Let business customers spec up to 768GB who are prepared for a $20-25k price tag
There is a clear difference between $25k and $100k.
64 iPhones at retail price is already around $64k. For something at 768GB to be profitable at Apple's terms, this has to retail at $100k for it to be profitable. That was the OP's point.
Honestly you're still looking at (from my understanding) ~3 minutes prefill (TTFT) even with architectural improvements and so on with a 32k context window (against a large model). How is this going to be competitive with Nvidia and all of the tricks massive scale get's you to parallelise context across many machines?
Is it supposed to be? I think the point with some of these Macs is you get the capability in something the size of a heatsink from Intel's Netburst architecture era, or a Macbook light enough to stick in a backpack and take with you to lunch.
If you're talking about chaining together multiple GPUs you're talking about a different game -- I suspect, anyway. Seems like a high-spec Mac would be good for development and testing. Arrays of GPUs, better aimed at production use.
The message I’m getting is that Apple will never compromise on its healthy margins. If something becomes basically unaffordable for their target market, they’d cut the production and even discontinue the product, than take a hit on margins. Their business model is refreshingly simple.
Apple is not gonna risk their iphones, as they are their flagship (aside even from giving them higher margins). My opinion is that, as we are talking about ram SHORTAGE (not just for ram price hike) they have to cut the more ram hungry models to be able to keep up with their projected production/demand (at reasonable ram prices). Getting iphones "sold out" is not a great thing for apple.
Once/if the ram shortage ends, they will continue increasing the ram caps as they were already doing, because then selling ram-heavy macs will not interfere with the rest of their products.
It's just a planned economy failing the way planned economies often do: the central planner failed to predict the demand correctly. Instead of trying to secure additional stock from the market at spot prices, they are simply waiting for the next batches they had planned for.
I don’t think that represents the scenario at all, not to mention the fact that it’s literally not a planned economy (but also not very analogous to one, either).
What’s really happening is that the effort of securing additional stock isn’t worth it because the price is so high that there aren’t enough buyers.
If ground beef were to suddenly cost $50/pound, McDonald’s doesn’t raise the price of the Big Mac to $25 and hope people buy it, because it makes zero sense for their business model. Sure, some fancy restaurant will still be selling hamburgers, but not your chain of thousands of working class fast food restaurants. McDonald’s would find some other alternative item to sell.
The truth is that nobody’s going to be buying Mac Studios that cost $25,000. Not even enterprises.
Businesses are usually planned economies, and supply chain management is literal central planning.
Apple failed to predict the demand for Mac Studios. Many other companies in its supply chain likely failed to predict that Apple would come back asking for more. There is no excess stock for some key components or the spare capacity to make them on demand. Apple would have to scour them from the market, likely paying much higher prices than it will pay for scheduled deliveries.
Hopefully I’m not being too pedantic, but I am saying that we can’t just use the words “planned economy” or “central planning” to describe a single company’s actions.
It’s just not what those words mean.
Apple is a large company but they are still just one company.
A single business can’t be doing “central planning,” it is by definition not in charge of the whole market.
Central planning would be if the government mandated that Micron to make ## of memory chips and distribute ## of them to Company A and ## of them to Company B.
If they cared about local AI market they would price hike M3 Ultra instead of discontinuing them. After all they conviniently introduced RDMA just few months before that.
Initially when it happened everyone expected they did it because they planned to announce M5 Ultra shortly, but its not looks like this is happening.
Now IMHO its indicates they simply run out of RAM supply.
If you’re truly serious about local AI you’re not using a Mac. A Mac is for people who want the best of both worlds, GPUs are much faster if memory is equal.
It's even worse than that. Demand is so high so every next GB will be sligly more expensive. There is a lot of smaller hardware manufacturers that unable to secure DRAM chips at any price, there just no free capacity on market.
Of course Apple is massive, but if they announce inference boxes that everyone wants they need to make 10,000s or even 100,000s of them.
And it's very much likely they'd rather sell 600,000 - 10,000,000 more iPhones or Macbooks Neo. End users bring Apple money with every OpenAI / Claude subscriprion sold through their platform.
And inference boxes is just one-off sale of hardware that will bring no further income.
They are assuming that they are able to get ram in the future, once the AI bubble either dissipates or pops. Its far easier to build something you planned for 3 years ago, than crash build it in 3 months.
Of course if RAM prices crash Apple might of bring high RAM options back. We just cant bet on it as consumers.
Right now RAM shortages are bad to the point where likely even Apple have to decide what products they make and what they discontinue.
There been short time where M3 Ultra with 256GB / 512GB been best offet on market because Apple lagged with price increase. Now HN crowd expect Apple of all companies jump into price war with Nvidia and to subsidize their inference hardware.
The article says base M7 memory bandwidth is targeted at 240GB/s.
M1 had 70 GB/s, M1 Pro: 200, M1 Max 400, M1 Ultra 800.
Modern RTX 6000: ~1,600 or so.
If we get a 1,200-1,500 GB/s bandwidth M7 variant in late 2027 with 512GB of RAM, that will be a very interesting chip. Tracking LLM size and performance improvements, I can imagine that being a sort of inflection point for local inference. I wonder what the power budget would be in desktop format.
A hypothetical M7 Ultra with LPDDR6 14.4Gbps memory would be 1.85 Tb/s.
You're look at about 100 tokens/s for a 1T MoE 37B active 4bit model.
It'd probably cost $30k or more I'm guessing if memory prices do not come down. Even at $30k, it could still be a relative bargain since an RTX Pro 6000 Blackwell 96GB card costs $12k today. The M3 Ultra with 512GB was around $8k before Apple discontinued it. I expect an M7 Ultra to have 768GB or 1024GB.
Apple Silicon Macs were on their way to becoming cheap local LLM machines relative to professional GPUs before this memory crisis. It may still emerge as such in a few years.
Here's some interesting math: At 512GB, an Ultra chip could make 42 pro iPhones. Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB. If they only have a limited supply of RAM for all their products, it makes no sense to produce an $8000 M3 Ultra 512GB when you can produce 42 pro iPhones. You can only configure an M3 Ultra up to 96GB today as of June 2026.
Apple would have to raise the price of a 512GB Ultra Mac to around $50k to match iPhone profits.
> Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB.
How would that work? They purchase 512GB from Samsung and then it doesn't matter if that's like 128x 4GB or 4x 128GB?
Note that this reserved capacity now has competition from OpenAI, Anthropic, xAI, Meta, Microsoft, Chinese data centers and so on, all willing to pay premium.
If comapnies keep spending half a macbook neo worth of subscription on AI plans monthly per person, Apple is going to have a hard time competing.
In British English the "an" is correct, even though most English dialects don't actually render the H as silent. It's a French-derived word that had a silent H originally, ergo we use "an".
I’d assume by next year the open weights models will be outlawed the way things are going nowadays :/
Edit: for those of you downvoting I don’t celebrate this prospect. I’m merely realistic about where things are going given the rapid vibe shift from the administration on AI since the start of June.
Apple is finally going to realize Jobs vision where sand comes into the factory, is turned into RAM and CPU chips, then installed in a Mac or iPhone then shipped to a customer.
Well yes. But similar to the Apple TSMC relationship, could Apple step in with large orders to established RAM makers such that the RAM makers can invest with stability?
No it isn't, DRAM is made with a different process and those are chiplets, perfectly possible to outsource, and the only possibility really as TSMC does not make DRAM.
192gb or 256gb of RAM would be enough ! We could run real time large MoE models, REAPed for our usage (e.g. english agentic coding), dynamic quant 2-4bits
Well yeah but NVidia just released a contender to their silicon and the M6 is probably already set in stone. Best to reshift resources to a great M7 than having a mediocre M6 and M7.
(This is assuming Apple will deliver, but this area is one of the biggest ones they have in AI, and they need the developer ecosystem to exist and survive)
Come to think of it, modern cars have a lot of electronics such as touchscreens, cameras, and sensors. It wouldn’t surprise me if new car prices are not immune to what’s happening with RAM and storage prices.
I'm pretty sure the vision/hwa reqs for cars is much less than an LLM/genai in general so that doesn't quite work out. But it would be nice to have an AI server with wheels :p
I think that people are still underestimating the technical merits of Intel's 18A fabrication process.
I haven't seen any competitor even try to address the backside power delivery of 18A. I suspect that Samsung,TSMC have something similar and doesn't talk about it.
The design rules for the standard cell (sort of corresponding to the die area required by a transistor) for the Intel 18A seem to target dense, high performance designs. That's not a particularly meaningful insight - of course Intel wants to have the highest performance of all the fabs.
Intel's packaging expertise used to be a generation ahead, and indeed their server chips currently use a mad mix of chiplets and through-silicon visas for direct stacking, all heaped onto a reticule-limited monster interposer die. All of this expensive complexity might be sustainable as long as Intel can keep its enterprise customers happy. That hasn't turned out too well for them.
AMD has found a mass-market winner with mainstream gaming CPU with extra level 3 cache die stacked on top. Compared to Intel servers, it's brutally simple. But extremely effective in its consumer market.
But the Intel chiplets and packaging could be a great toolbox for M7 generation of Apple Silicon. Now that the M5 Pro and Max are multi chip packages, they more resemble the Intel and AMD designs, with chiplets dedicated to I/O or GPU.
(Speculation and dreams. That's all I got, and I'm writing it in the face of an absolutely psychotic autocorrect on a tablet.)
> I suspect that Samsung,TSMC have something similar and doesn't talk about it.
They do, just not as hyped up as Intel. TSMC will have it after 20A. Either 18A or 14A. GAA was supposed to be in 3nm but didn't happen due to multiple reasons. So it is now delayed to 20A. Backside delivery was supposed to be 20A and also got pushed back as well.
As we all know, Intel used to be famous for their engineering and their ability to scale up a newer, smaller process with way earlier commercial viability. This all ended with the Sisyphean 10nm move that was years late and honestly Intel just don't seem to have recovered from it.
So Intel seemingly has underutilized fab capacity whereas the likes of TSMC and Samsung can probably produce every chip they make with demand to spare. Given the CHIPS Act that was passed under Biden, the Trump admin taking a stake in Intel and the environment of tariffs and a push for American manufacturing, everything seems to be lining up for someone to take advantage of Inte's physical fabs and American production and that could be Apple.
It's not so much ripping off the designs - nothing of what Apple Silicon is doing is particularly surprising and both x86 and Intel's microarchitectures are sufficiently different to Apple Silicon/ARM that knowledge of specific implementation approaches wouldn't be directly useful in most cases.
The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
> The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
While I'm sure some level of internal leakage does take place, at least on paper the fab's planning needs to be firewalled off from their own chip roadmap.
I'm also not sure how much Apple actually cares, tbh. Yes, they currently have an edge in silicon, but it's heavily due to being willing to outspend everyone else, and their real superpower is vertical integration - which Intel isn't in a position to compete with.
I think Apple doesn't really have a choice. They've been very strongly encouraged by the current US government to move as much chip manufacturing to the US as possible, and particularly to make Intel Foundry work, or face... problems.
Also the AI boom means NVIDIA et al. can afford to buy TSMC's best processes at scale, which means less available capacity for Apple.
I'm sure given no other forces at work, Apple would prefer to stick with what they were doing previously, buying the lion's share of TSMC's best process.
I mean in the opposite sense. If Intel can glean enough from Apple's roadmap to close the performance/watt gap, great, but they still can't match the vertical integration Apple has
M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth.
This has both a technical and human component.
On the human side, top x86 execs refused to see any threat coming. They must have thought Apple couldn't overcome the x86 software moat, thought the chips were for servers, consoles, or some other non-PC device, or perhaps they simply couldn't believe what their investigative teams told them.
At the same time, we're 6 years post-launch. The proof of ARM's capability is clear. x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86. Either all the x86 engineers are completely incompetent and incapable of learning from years of ARM designs or there are aspects of x86 that makes copying those designs infeasible.
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC.
> teams from Intel and AMD absolutely HAD to know the truth.
These people are professionals that acknowledge IPC is a stupid metric. If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM. Intel invested early on big.LITTLE, and AMD drilled down on denser nodes for their preexisting designs. As both businesses converge on each other's ideas, their SOCs have adapted most of ARMs' greatest mobile innovations. Even before that, x86 hardware was always usable - AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
All of this makes sense, nothing objectively prevents the x86 architecture from being power-efficient. Arm LTD. would have gouged any of those vendors for their IP, and even with an architectural license it's not like AMD or Intel would get usable core designs from Arm. There was no reason to pivot to ARM for either company, they both saw Qualcomm and could read the writing on the wall.
> x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86
That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market. Apple's early ARM hardware was outright ignored for server/HPC applications, leading to the discontinuation of the Mac Pro. Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving, and Grace for the high-end of "we need CUDA and enough bandwidth for Infiniband" that made trillions in the HPC market.
> These people are professionals that acknowledge IPC is a stupid metric.
Even if you think IPC is a stupid metric (it's not), ARM is still wiping the floor in total performance, performance per area, and performance per watt.
> If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
Fujitsu also makes a 512-bit SIMD that has very good performance, but that large SIMD eats area and power while giving very little back to consumers. Honestly, SME co-processors or 6-wide SVE are much closer to "space age" than simply making a wider packed SIMD.
> x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM.
AMD actually bought an architectural license and Intel used to have one too. They could (and should) undercut ARM entirely by making x86+RISC-V cores. You can talk about Intel's E-cores or AMD C-cores, but they still don't compare in the key metrics.
> AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
Which GPUs are you talking about? 4800U (Vega 8) was around 50% slower than M1's GPU. 5700G released in April 2021 and also had the same Vega 8 GPU design. They didn't see a bump until RDNA2 in Rembrandt in 2022 (by which point Apple was already on M2).
> nothing objectively prevents the x86 architecture from being power-efficient.
Citation needed. This is like saying nothing prevents branch delay slots from being good or VLIW/EPIC/Itanium is just as good as anything else. uop cache, extra pipeline stages for decoding, and memory ordering prediction are enough by themselves to use more power when under load. If you have actual proof that x86 workarounds have absolutely ZERO energy/area/latency, then show it.
> That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market.
Linus Torvalds pointed out that ARM server wouldn't take off until devs could run ARM natively on their laptops. Apple provided those laptops.
MS claims Cobalt 200 lowers carbon footprint by 40% vs x86 (which should mostly reflect power consumption per unit of work). ARM accounted for over a third of Azure deployments in 2025 and should increase as C200 becomes available.
> The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving
That hasn't been true for a long time now. Graviton5 in particular is a big deal. It removes the NUMA issues resulting in much lower latencies (and higher performance) for a lot of things. It gives 2.6x more cache per core and a general ~25% performance uplift (SAP said it was up to 60% for their workloads) which is more than the difference between graviton4 and Zen5 for most workloads. Graviton accounts for more than 50% of all new servers in AWS and basically all of their 1000 largest customers are using them.
> Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
Apple wasn't incapable -- they didn't want to because they aren't in the server market. That is the entire reason X-Elite/Nuvia exists. GPU stack is completely orthogonal to making server CPUs (you can make good CPUs without making good GPUs).
What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
Maybe this strategy works, even in that world.
Remember when we all thought (were told we thought) the world was heading to 3D views of our 2D lived experience like a solid Cube of GUI we could rotate around and live inside? Well Apple took the simple 2D square pane of virtual desktops and .. made it a SONY strip. One variable: sideways.
So here we are being told AI is the future. Apple seems to be saying "yes but it will run local" which might be a safe bet if AI comes true but I wonder how many of us want the AI outcome, which is morally speaking the 3D immersive GUI cube here: what if we don't want that?
I can't imagine any world where we put this AI stuff back in the box. It is simply too useful and too powerful. And as we start seeing all his upheaval where models are getting banned, etc, I can even see the appeal of on-device AI increasing for a lot of use cases.
So I think Apple has the right instinct. In fact, I've had the thought multiple times that I really want a lot of workflows just running on my device. Workflows like fast vector search (already fast on the m4, but I want it more common place), or realtime transcription and summarization to be even faster, on device, etc.
To me AI is on par with the internet and what made it so powerful was piracy and porn and just the wide spectrum of things that are possible when you connect machines together. We are going to need the same thing again. Freedom to use any model that does any thing we want.
Looking to China as a purveyor of laissez faire freedom to do whatever you want is so nonsensical as to be a joke. Both US government and the Chinese government want this newfound power to be on a tight leash.
They're still the only ones releasing any open model at all?
Plus, not like chatgpt is going to reply honestly about sensitive current hot topics in USA politics. Normally it has filters and refuses to respond as well.
The worst case scenario is that we're at a plateau and LLMs max out around here. And it'd stand to reason that if that happens we'd see local models catch up at least to some extent. Compared to 5 years ago, that's a pretty good world.
So I think it's fair to say that AI isn't going away. That doens't mean that SpaceX, OpenAI and Anthropic won't crash. But I've long believed that within 5 years we'll have access to relatively cheap hardware that can run sufficient but not cutting-edge models locally. You can buy a 5090 PC for <$5000 already so I guess it's already true but I think we'll do even better.
So what happens? Nothing. If Apple make M7 Max/Ultra computes with 128-768GB of RAM and nobody buys them then... nobody buys them. Apple isn't betting the entire company on AI just like Google isn't. The rest of the internals are the same Macbook, Mac Mini or Mac Studio. You're just selling something with less RAM.
Google sold (will sell?) about $70bn worth of Google shares to fund AI infrastructure build outs. It's also issued bonds (=debt; I forget the number, $30bn?) to pay for more infrastructure. Fairly sure it also has established a shadow company, a Special Purpose Vehicle (SPV) to stash away unpleasant financial things it doesn't want to show, also for the AI build out.
Amazon, Google, Meta, Oracle are overstretching at the moment. They are predicted to become cash flow negative (more money going out than coming in) if they keep going at this rate, some time in 2027 or 2028.
Now, they won't go bankrupt but it's possible they will be hit by huge restructuring waves once the dust settles.
Google has a market cap over $4 trillion and it's 2025 financials were $130 billion profit on $400 billion revenue, which was something like 15% Y/Y growth.
$100 billion in equity and bond offerings is not betting the farm on AI.
Additionally, Google's data centers are notoriously efficient and they build their own networking hardware as as their TPUs. Now their TPUs lag behind NVidia offerings obviously but they'll keep getting better.
Google is the company I'm least concerned about in the AI space.
As for Amazon, I'm not sure what their split is on in-house AI build out and usage vs AWS. I suspect the majority is for AWS and I also suspect their ROI is insanely short (ie less than 5 years) on any AI capex.
A lot of us on HN dismiss Oracle with good reason but it's also fair to say they've survived and thrived through the dot-com crash, the GFC and the pandemic until now. They're clearly doing something right.
Meta I think is the Sick Man of Tech. Their social media assets are of declining value (IMHO) and they seem completely unable to adapt to changing conditions. The whole Metaverse was a massive $70B+ boondoggle on something long before they had anything resembling a product-market fit. They seem unable to leverage their social media assets for building any competitive AI products or tech. Plus they seem unable to hire and retain key staff in this space.
For the purposes of most financial discussions, the market cap can be ignored. It tells almost nothing about the business fundamentals.
> and it's 2025 financials were $130 billion profit on $400 billion revenue, which was something like 15% Y/Y growth.
Most of that growth, which has re-accelerated, is AI driven. Aka eggs in a single basket.
> $100 billion in equity and bond offerings is not betting the farm on AI.
That's on top of capex of $185bn in 2026. In 2023 it was $32bn, 2024 $52bn, 2025 $91bn.
All your financial alarm bells should start ringing.
Amazon is doing similar things.
> A lot of us on HN dismiss Oracle with good reason but it's also fair to say they've survived and thrived through the dot-com crash, the GFC and the pandemic until now. They're clearly doing something right.
Oracle is the most likely to go bankrupt out of all these companies outside of AI labs. They've been very adept at financials until now, when they've bet the farm on explosive AI revenue and especially profit growth. If those don't materialize soon, bye bye Oracle! Which would probably make a great many engineer happy :-))
> For the purposes of most financial discussions, the market cap can be ignored.
If you issue $70B in stock, which was your point btw, market cap absolutely matters. $70B is less than 2% new shares issued. If it was a $100B company, it would be 70%. That's why market cap matters.
> All your financial alarm bells should start ringing.
Only if you start with the premise of "Google is imploding" and then go looking for evidence.
Oh, so we're using CDS rates as a proxy for alarm now? Ok, Google's rates are hovering under 50 basis points [1]. For comparison, this is only slightly higher than the sovereign debt of Canada [2].
I love how you completely side stepped the cash flow comment AND the graph.
It's all right, I spent years arguing against cryptocurrency boosters. I'll probably have to wait a few more years for AI boosters to be gone into the nether.
I find it very unlikely that nobody buys them, some people will definitely buy them to run giant open LLMS. But there's also not much risk to apple because those configs would probably be made to order
I doubt inference costs will scale up significantly, but even if they do, it simply strengthens the strategic case for Apple's focus on local inference.
Without AI everyone’s computing needs were pretty well satisfied with current phones and laptops. LLMs are the one thing that could drive new demand if they can run locally.
AI was the only reason I bought a new computer (a refurb M3 max with 64GB). Without AI, no idea what we should bother with, it depends on what application comes out to drive local computing power (maybe better games? Yawn).
this is the backup strategy. the "AI doesn't pan out" scenario is basically if claude and openai go bankrupt, we continue running local models on our hardware.
there isn't a future where we all just decide that nah, we don't want AI anymore. usefuly things don't disappear.
In between flat material-ish (hehe) design windows and 3d compiz cube with burn effects we have settled on transparency and blur effects with a bit of visual planes thrown in.
It's highly unlikely we will end up tokenmaxxing everything, it's highly unlikely the genie can lose enough weight to fit back into the bottle. We will end up somewhere in between that strikes a balance between nice, productive and cost effective.
> What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
I think reducing the die area dedicated to ai stuff is not going to be a problem.
And in fairness apple already has essentially ai-less hardware in the form of the MacBook neo and it’s been an astonishing success.
I have one and it’s a very good laptop, particularly for the price i paid it.
Anything AI focused in silicon is also valuable for a ton of other use cases. If LLMs and GenAI don’t pan out, that silicon just gets used for other processing. Then they scale back on the dedicated die space in subsequent generations.
Do we have a choice? It's being forced upon us by folks who have the power to distort any market they want. Energy prices are rising, and the PC industry is about to be destroyed by component prices. It will be dumb clients that run the software our feudal overlords of the data centers will have the grace to grant us. And the government lets it happen because it furthers their interests.
AI isn't going anywhere, this is akin to the .com bubble. It burst, but the internet didn't go anywhere. While companies can fail, this technology is with us for the long run now, short of societal collapse.
Seems like a made-up distinction that shouldn't be necessary since M6 has not even released. I suspect this is a marketing ploy to meant to drive up both interest while also increasing prices for the next generation of Mac hardware.
What it's saying is that the M6 will be released, but not the M6 Pro or M6 Max. Instead, Apple will wait to release new Max/Pro chips for a future generation.
It's not simply marketing since the Pro/Max chips of a generation use the same cores as the regular version, just more of them or different combinations of performance and efficiency cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
The claim is that M6 will be released, but the only variants will be lower end.
When they get to the M7 generation, they will make high end variants.
It's a real distinction because each generation of parts shares an architecture.
The article has an entire section speculating what the M6 parts will be, but says they'll top out around 200GB/s memory bandwidth and 12 graphics cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
Why would it? Each generation of the M series has an architectural improvement on their chipsets. The difference between an M1 and an M1 Pro is the allocation and arrangement not the architecture. M6 to M7 presumably will have architectural changes.
This is no different than them skipping the “Ultra” chips on some generations. The only real difference is it going all the way down to skipping the “Pro” line. So, only the MacBook Air, low end MBP, and maybe the iPad Pro and Mac Mini get the M6.
Made up how? They'll do a refresh of lower end devices, but not the high core count versions.
It's the same thing as how the Mac Studio got an M4 Max refresh, but they didn't make an M4 Ultra so if you want the 28+ core CPU or 60+ core GPU, that's still using an M3 Ultra.
This time it'll be across all the Pro, Max, and Ultra versions, if you want those they'll stay at the previous generation for the M6 cycle.
Not that weird - Apple has a huge set of chips and hardware and software products. Putting every single thing on a fixed identical update cycle together won't always make sense.
Except that is not what's happening. The article clarifies something that is misleading if you interpret the headline in isolation: "high-end M6" means "the high-end variants of the M6 line", not "the entire M6 line".
Whether it matters for the consumer (who only sees released and announced end results) or not is irrelevant.
It can still be a very real, not made-up distinction, if the actual facts on the ground are that Apple designed an M6 line, but then scrapped that design and asked the team to create a new design with emphasis on AI-focused specs.
It's not the name that's important (the M7 could still come out as M6), is them skipping a design, or cpu "Tick-Tock model" step.
Perhaps to support demand for the products with recent price hikes, and/or the upcoming Mac Studio with M5 Ultra, rather than have customers sit on the sidelines thinking they'll wait this generation out.
I am still skeptical that Apple intentionally leaked this because they normally are so tight-lipped, but there are reasons in favor of leaking this.
Well, I guess this is the silver lining to the price increases. I'd been thinking about an M5 128GB for local inference (eg DS4), probably off the table now given that it jumped $2k overnight. But I was on the fence about it for a long time given that even the M5 is not that good compared to even a 4090. It would have been good, but not "omg" good.
If they are pulling out all the stops to make the M7 more competitive.. guess I can wait for that?
The M7 Pro and M7 Max are scheduled for as early as the end of 2027, while the M7 Ultra is on track for 2028.
This means there won't be a redesigned MBP this year since there won't be M6 Pro/Max chips. People were expecting a redesigned slimmer MBP with OLED display later this year, myself included.
I was holding out for one until I decided to switch from an M1 Pro 16" MBP to an M5 Air 15" due to the expected price increase. I think many M1 Pro/Max generation people were waiting to upgrade this year.
Current MBPs are such a delight, I really don't want to think about a thinner MBP again, I just get shivers remembering the Ive butterfly keyboard models
I can see why people would want a more powerful machine but as someone who moves around a lot, the 16" MBP weight is a pain. The 14" MBP screen is not big enough.
Isn't that switch basically a downgrade? You get some more single core performance and some weight savings, but also a worse (and smaller) screen, less multicore performance, less GPU performance, less video encoding performance and a smaller battery? I'm on an M2 Max myself, and glad they introduced a larger form factor Air, but it seems like a long way from an upgrade.
M5 is faster than the M1 Pro in ST, MT, GPU. Not sure about video encoding as it's something I rarely use. It's a smaller battery but overall a battery life improvement since my 5 year old M1 Pro only had 79% battery capacity left.
The optics and marketing is already fucked, the MBP goes to M5 Max, the Mini has the M4, the Studio has M2 or M3, the iMac apparently has two different kinds of M4s, it's all fucked.
Well this kind of sucks. I've been waiting for the M6 MBPs because they're rumored (strong rumors, though) to finally remove the notch that has been a historic self-own. But it sounds like I might as well wait longer for the M7 lineup. Or maybe get a Framework Pro instead.
I'm not even convinced this is a bug. Menu icons proliferated over the last couple of years, this isn't a problem unless you enable more than you have room for.
There’s so many annoying bugs in Mac OS (like the screwed up window management and alt-tab not working properly), that the notch seems like an odd complaint at this point. The OS is fighting the user constantly, and there’s not much we can do…
I'm a huge fan of Star Trek, David Warner, and Patrick Stewart, and I'm really not. The notch isn't taking anything away, it's a way to give users more screen space than they could otherwise get. Normally the camera has to be above the screen, this was a way to get the menu bar off of the rest of the screen real estate and get you more usable pixels.
It’s a complete embarrassment. They added it for aesthetic alignment with the iPhone 13. And then the 14 removed the notch soon after. They’ve kept it for years since then. It has no functional purpose. It’s not there for face ID or because they couldn’t figure out how to do a hole punch camera.
I agree. It was very annoying to me to spend the money (and on the nano matte one too) and still have that stupid notch. But it never makes any difference at all which is good news.
Same, have a very old MBP. Not sure what to do because I don’t want to wait a year and a half. That coupled with today’s price increases make it a tougher decision.
In the long run I truly believe local AI will win and Apple will be the world's most important AI company because of these chips. Imagine something like today's Opus running for free and in complete privacy on your local machine with a beautiful Apple UX on top. For most tasks for most people, that's a much better proposition than a frontier model in the cloud you have to pay for and send all your data to and that only works when you're online.
>In the long run I truly believe local AI will win
What do you mean by 'win'?
For a normal coder/person's use cases, yes. But AI companies are becoming more specialised in different fields and these tailored models will be leagues ahead in those niches.
The way I see it - Opus 4.8 xhigh can do any programming task with a programmer instructing it. If Apple releases local model together with a device that can run said model it would render OpenAI/Anthropic useless for vast majority of usecases.
And if a local mcahine can run something like Opus 4.8, who is to say that those "specialized" models would just not come at a later date, or even loading open models wouldn't be an option with something like M7-verified flag from huggingface that would make it extremely easy for any consumer to just play around.
There is built-in demand for local LLMs. An obvious example is law firms where using remote AI tools may be breaking privilege [1]. Any medical applications may likewise run into legal issues.
The problem is basically that we can't have nice things. AI chat logs themselves become another commodity to sell and to train on. We recently had a story about how Chinese firms are reselling Claude tokens [2]. The chat logs are a commodity here.
The only way to avoid this is to run LLMs locally. Even if you trust someone like Anthropic or Google, case law simply hasn't been established that the chat logs aren't discoverable.
Add to that that a sub-$5000 PC with a 5090 can already run a 31B model at reasonable inference speeds. Not amazing but good enough for many applications. Obviously that can't compete with Mythos but it doesn't have to. It also shows where the trend line is going for hardware. A $10k Nvidia GPU from 10 years ago now sells for scrap. What a consumer-level computer in 5 years can run locally will probably shock a lot of people.
I would say local AI is very real. I use it but so many here am on other forums do so nowadays as well. This is the reason I just cannot fathom the valuations of the AI firms out there.
Mac mini Pro line is doomed, they never made enough of it; skipped M5 Pro, now skipping M6 Pro, it is like 2014-2018 again. Now ordering a custom M4 Pro build take 3 months+ to ship with an increased price.
I was waiting for a MacBook Pro M6 Max and now I don’t know what to do, especially with the price increase I feel like I really screwed up not just getting an MBP M5 Max a month ago
because America can't compete. Build a fab in the US, labor unions, labor costs, regulations, land, energy, taxes, government, water, etc all make this not economical. Everything would cost twice as much and you'd rather buy the cheaper product and it'll be bankrupt. There were reasons why all the manufacturing went overseas to Asia. You're right, the demand right now is HUGE but it won't always be huge. At this point, we don't have the talent or the knowledge to do it well anyway which is why we needed TSMC and Samsung to bring employees over to train people. https://www.cppionline.org/wp-content/uploads/2017/07/The-De...
I am waiting till apple copies the "allocation" concept from high end car manufacturers. "Sure, buy the 25 iphones ans we will gladly put you on the waitlist."
And it's not like Apple hasn't dabbled in the luxury space, with utterly predictable results. Anyone remember the 24 carat gold plated Apple Watch Series 0, sold at ultra-high-end luxury boutiques?
Apple to skip high-end versions of M6 Mac chips...
I read it as the M6 being "high-end" in general, and Apple skipping the whole generation, which made no sense to me. But they are going to use the M6 at all, just not bother to create Max and Ultra versions of it.
So the big question: Is this an excuse to save on memory costs and delaying stuff like the M5 Ultra, M6 Max, etc until 2027 when memory prices come back down?
Given that M6 will be on TSMC smaller 2nm node and the first smaller node size in 3-years, it seems like the oddest of all years for the high-end Macs to skip.
1. NVidia aggressively segments the market on VRAM and will continue to do so. A 5090 with 32GB of RAM, ~21k CUDA cores and 1800GB/s of memory bandwidth is $3-4k. An RTX 6000 Pro with 96GB of RAM, ~24k CUDA cores and 1800GB/s memory bandwidth is ~$11k;
2. The 5090 won't be replaced until late 2028 or even 2029. There has been no mid-cycle refresh (eg 4080 Super vs 4080) and likely won't be either at all or for at least a year. If there is in a year, it basically confirms that the 6000 series won't be until 2028/2029. Also, the x090 never got a mid-cycle refresh so the current consumer high-end is staying that way for years;
3. The 6090 whenever it comes will still have 32GB of VRAM unless the memory market drastically changes;
4. Many have anticipated an M5 Max/Ultra refresh of the Mac Studio line in Q3. Given that Apple chose to hike the prices on Studios rather than discontinue them, I now think this isn't going to happen. We may not see a Studio refresh for up to 2 years. Apple has done this before with the Mac Pro;
5. M7 Max/Ultra will probably go to a memory bandwidth of 1.2-1.8TB/s vs the current tops of M3 Ultra, M4 Max and M5 Max of 600-900GB/s. This simply needs to go up to boost inference speed;
6. You'll also see the number of GPU cores go up. All of this will add up to an M7 Max being 50-80%+ of the performance of a 5090. That's huge given the shared memory architecture;
7. We may see the return of Apple using its massive cash pile for vendor-financing of an exclusive memory supply. This was one of Tim Apple's [sic] big innovations.
Do you really think the average Apple user will use it when there’s already better AI provided by OpenAI and Anthropic which don’t require advanced local hardware?
I was just countering your argument with an equally compelling counter-argument.
As for how it helps: we're not talking about this year's AI ecosystem, or even next year's. This rumor, assuming it's true, is talking about two chip generations into the future — and probably at least three or four chip generations before it's a mature AI platform. What will AI be doing for us in five years from now? How does Apple plan for that future? Will concerns of privacy increase or decrease in that time?
>What will AI be doing for us in five years from now?
I don’t know, but the majority of it will be running in data centers, not high end consumer grade workstations.
>Will concerns of privacy increase or decrease in that time?
People have been more concerned about privacy than ever, it hasn’t seemed to stop people from using cloud AI services.
The fact that Apple missed the starting gun on AI to the point that they’re using Googles also inferior Gemini, I’m not compelled these Apple AI chips have a consumer.
Anyone savvy enough to do their own AI hosting is more likely to use Nvdia boxes, etc. Apple workstations always been more successful in Art/Graphic design. That’s why they cut off their server hardware long ago.
>How does Apple plan for that future?
Ideally by making attractive simple products regular users can buy. The iPhone and MacBook Neo seem a step in that direction. AI vision and Apple Vision scream 90s pre-iMac Apple.
Apple is very late to the AI party. By the time M7 is shipped, Nvidia will announce 6090 and people will be buying used (3|4|5)090 GPUs to run local models at much better performance than heat throttled M7.
Neural Accelerators in M5 are already 4x faster than M4 at prefill. With M7, especially if they focus on AI like this article claims, it likely will have excellent prefill compute.
What people? Are you seriously thinking the hundreds of millions of customers Apple have is going to be buying run-to-the-ground GPUs second hand and build local workstations for AI? Might as well ask them to self host email while you’re at it.
The difference between these two is that one of them is an unsolved research problem that we’ve all spent far too much time on, and the other is just running an LLM.
I would prefer a Studio if it does a decent enough job even if throttles a bit under load, way less power usage and noise than those GPUs plus the PC you need to put those in.
RAM is a commodity and nvidia will be paying the same prices. The used market will reflect the cost of RAM. nvidia owns the top of the market but many of us don't need that.
Apple isn't just transitioning to TSMC's 2nm node, they are also transitioning to a chiplet based design using TSMC's advanced packaging.
> What sets the A20 apart isn’t just the node shrink—it’s the revolution in packaging. Apple is transitioning to Wafer-Level Multi-Chip Module (WLCM) integration, meaning that RAM will no longer be situated beside the chip, but rather on the chip wafer itself, integrated alongside the CPU, GPU, and Neural Engine.
This shift eliminates the need for silicon interposers and substrates, thereby enhancing signal integrity, improving thermal dissipation, and facilitating faster memory access with lower latency. The benefits? Better multitasking, smoother AI processing (hello, Apple Intelligence), improved battery life, and potentially a smaller chip footprint—freeing up space for other components.
Do we have any explanations of what WLCM means that are more industry focused? I couldn't find anything that didn't look like blogspam. And that explanation of the DRAM being on the same wafer doesn't really make sense. For one, at that point there's no "multi chip" part if you're integrating more onto the same die rather than less.
And their explanation isn't really passing the smell test for me for other reasons, for instance the fact that DRAM processes are pretty radically different than bulk logic processes, which wouldn't really let you put it all on the same wafer, much less the same die. Even back in the day when you had eDRAM blocks (like the Xbox 360's eDRAM die), that was really a DRAM process with a bit of logic cells that wouldn't be competitive if they weren't sitting right next to the DRAM blocks.
I could be wrong here though, my examples are more than a bit long in the tooth.
The terms to search for are fan-out wafer level packaging (FOWLP) and TSMC InFO. The chiplets come from different wafers and are reconstituted into a molded plastic wafer, allowing multiple die side-by-side. Then multiple layers of wires are built on top, terminating in a BGA.
Ok, part of my confusion was that it was being presented in contrast to InFO-oS and InFO-PoP, but it appears to mostly be a modified version of InFO-PoP called InFO-M? Because Apple has been using InFO-PoP for almost a decade at this point, starting with the A10.
You can start by reading up on TSMC's name for the tech (although there are many versions at TSMC and TSMC isn't the only company packaging chiplets and memory on top of a silicon interposer).
InFO combines chiplets, but does not do so by bonding multiple chiplets directly to a silicon interposer as CoWoS does.
The article compares CoWoS to InFO.
> CoWoS uses a passive silicon interposer—etched with thousands of fine interconnects—that sits between the active dies and the package substrate. This interposer provides high-density connections between chiplets and supports memory integration, most notably High Bandwidth Memory (HBM) stacks.
It also mentions that CoWoS is the tech used to build Nvidia's high end AI accelerators.
> AI Accelerators: Nvidia H100, B100 (HBM3 via CoWoS)
Apple is taking a tech that has previously only been used in very high end enterprise applications and is using it to make consumer SOCs starting with this years iPhone.
You're calling it CoWoS, but also said "this shift eliminates the need for silicon interposers", but also said "CoWoS uses a passive silicon interposer"
Something's got to give here. I think it's your original article that's wrong, and it's poorly trying to describe InFO-M.
A kind request - please try to write HN replies without AI, but if you're going to, please at least edit out any "it's not X its Y" or "isn't just X, but also Y" AI tics. A lot of us come here to get away from talking to AIs all day.
‘isn’t just / it’s also’ AI-ism should at least turn your AI radar on, then you get this weirdly formal structure that sounds like a trying-to-be-relatable press release:
“This shift eliminates A, thereby enhancing B, improving C, and facilitating D. The benefits? Better U, smoother V (hello, W), improved X, and potentially Y—freeing up Z.”
Question to self interjection, chipper ‘(hello, W)’ aside, topped off with a zero spaced emdash. 10000% AI, stylistically this isn’t text tuned to an HN audience, comments never sound like this. What’s funny is the paragraph it’s quoting has nearly the same style, the LLM probably picked that up.
Not trying to call anyone out, just pointing out the stylistic tells we should all be aware of.
So far the only thing I've seen useful out of apple intelligence is running parakeet natively and effectively... which should have been their very first feature... given it's been on phones for 10+ years.
As someone who wants to run effective llms locally for many things their other big benefit has been the unified memory studios for a small bit.
Apple is actually interesting. They are one of the few companies with a chip / PC play with real power AND basically no play I'm the hyperscalar market.
That means they're actually incentivized at least short term, to benefit PCs becoming strong enough to do local LLMs. Which makes this play make even more sense. Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
> Though, I've been saying for a while that the local AI inflectiom point is the death knell for these frontier labs.
"Death knell" is a touch hyperbolic. Hardware that can only run quantized models that take up GBs in VRAM falls short of even an A100 (by almost an order of magnitude[0]), which in turn falls short of what an 8xH100 cluster can do (also by another order of magnitude[0]).
I'm an avid believer in local LLMs, but I cannot deceive myself - data center accelerators will win on power dissipation numbers alone[1], even when giving generous allowances for higher efficiency on Apple chips - and assuming the Apple-efficiency advantage persists on the same TSMC process node.
0. Based on my unscientific fine-tuning training experiments across local and rented GPUs. YMMV for inference.
1. Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Doesn’t need to be a winner head to head. If it can do 90% of the tasks the big boys do, at 50% speed, for virtually no extra overhead cost save for the power consumed by a prompt - that’s gonna work for a lot of people. And that’s also basically where we’re at today. Qwen3.6 35b running quantized on 10 year old hardware solves basically all of my uses cases for agents except for coding.
The frontier models are faster, and better at coding, but not so much that i’ll pay $200/month for them.
Consider this. One of the smallest Qwen models (4B parameters) powers my home automation voice assistant, and runs on CPU alone at >20 tok/s. It is enough for that use case, and could be made even better/faster with a modest GPU. It isn't as smart as some cloud-connected thingamajig, but I would never allow a literal Google or Amazon bug in my home. Huge SOTA models aren't relevant everywhere. Most people use LLMs for rather trivial tasks such as finding typos or drafting text.
But with Apple's AFM 3 architecture, we might end up with huge SOTA adjacent on devices with limited RAM.
They use a technique where you only load between 1B and 4B of a 20B dense model for an entire prompt run, not token by token like a MoE, and use mostly the low power ANE instead of GPU cores.
Now, imagine if/when they scale up to 100B or more? On a chip using 2W?
I think we're also ignoring a potential innovative move in how models work.
If someone could splinter or fragment the models into more specific tasks i.e "spellchecker AI" and get these working as well as Sonnet 4.6-4.8 on those tasks on a personal laptop. You then question the $100 a month fee.
Bear in mind these laptops are likely to be $5000 or so because of the memory, HDD and M7 chip they likely need.
It feels to me like the beginning of the inflection point but software updates not hardware updates will be the accelerant.
Curious, what exactly does it do for you? I has bad luck with these small models to do anything useful tbh.
It's a voice assistant that can respond to commands such as "turn on the light," or explain things (within the abilities of the small model).
> If it can do 90% of the tasks the big boys do, at 50% speed
I want to live in this world too, but these numbers, as of today, are very aspirational and far removed from reality.
I'm no tokenmaxxer; I find my modest local setup useful, I also know the limitations, it's slow and it sucks (relatively) at high-level and/or long-context planning, compared to frontier models. Only a minority of my prompts are max-effort - its not all I do, but, it also means frontier labs aren't dying any time soon
Consider also that right now LLMs run slowly enough you can watch them think. I've seen a demo of an LLM running at an absurdly high speed and it reminds me of when I moved from a 2400 baud modem to a 14.4 - BBS screens that I could watch draw were all of a sudden nigh-interactive. Faster-than-realtime video generation is also coming, and will also continue to require huge hardware for a long while yet.
I love local models - I have a machine at home that runs a few for me and it's a lot of fun - but for the time being they are not super trustworthy on tool calls and staying on script. Another year or so might change all that!
If anyone wishes to see the future. A fast LLM is quite eye-opening. I think chatjimmy uses Talaas' chips where models are hardcoded into the silicon.
https://chatjimmy.ai/
Yea, is almost "scary fast" in a sense... the amount of compute you can do in parallel one one of those chips is amazing. hopefully they get their next chip completed as that will be a lot more useful for general workloads. I think the current one is based on llama3.
*corrected llama version to 3
Thanks, I didn't know that one! Very impressive speed although quality seems very bad
> although quality seems very bad
The weights they “etched” into the FPGA card that’s used for the ChatJimmy demo are that of a Llama 3-something 8b model.
The actually impressive and novel thing is that Taalas’ve managed to automate that process (clearly – nobody transforms 8 billion numbers into a physical representation by hand).
So now, they can work on scaling this process up, and with low enough lead times (I’ll be convinced they have inside connections to TSMC if they can actually deliver on the promised mere 3-4 months delay), will be able to offer 30-100b+ parameter models under half a year after they’re released, at thousands of tokens per second while probably drawing less wattage (per token, not sure about overall).
Exciting times ahead, folks.
At some point we're going to have these chips for a couple bucks in kids toys
What does your local setup look like?
1650 watts of liquid cooled Risc-V
I’m sure you’re right, for the things you are asking of an llm, just as I am right about the things I am asking of an llm.
The real question is, what are 90% of people going to ask llms to do. I’d argue mostly it’s going to be stuff that works-now or almost-works on local models, but that’s just an opinion. It also depends on the frontier models hitting a wall of steeply diminishing returns, since they set the expectations for all of this stuff - my gut says that’s happened already they just won’t admit it for a while - but we’ll see.
This is what makes sense for me as well. All I need a local model is for playing with simple graphics: no gradients, at most ten colours which I can push through VTracer to get an SVG. Draw Things does the job, usually in 120 seconds or less.
Sometimes, I need a quick throwaway bit of python. That can take 30 minutes of my time.
We'll likely see a transformation in how frontier models are trained as a result of a push towards local inference. While it seems unlikely now, given current pricing for RAM, in 10-15 years it's not unthinkable to assume we could see individual machines with 10-12TB (and well beyond that) of RAM which are accessible to the GPU. Min/max system RAM increased a LOT from 2010-2025 and largely because it was cheap. Once the hyperscalers aren't generating revenue for the RAM manufacturers, I wouldn't be surprised to see a massive push towards consumers in order to maintain gross profit. Not to mention new players who enter the market because the margins are measurably absurd right now.
At some point there will be diminishing returns towards the "just throw more RAM at it" approach the current frontier models are taking. Commoditization is just as inevitable as it ever was... and in doing so will enable actual leaps of what AI/ML is capable of. That's not to say there won't be a place for 99.999999% accurate vs 99.99999% but those cases will be limited and likely prime to disruption based on real innovation vs access to capital.
The 1080ti is out there for almost 10 years now. It has 11GB of VRAM. A 5090 has 32GB.
SOCs with unified memory have shifted this a bit forward, but they're also expensive as shit.
10TB ram in a consumer device is simply not happening in the next 10 years.
Half a year ago you could get a AI max 395+ with 128GB ram in mobile form factor for ~$2200. The same thing costs $3700. Same SoC, same memory.
10TB is about 80 times that, 200K in today’s money. A lot of capacity is coming online in the next 5 years and it’s reasonable to think we can get there with better process and stacking (the latter does little for pricing, but enables shorter latencies).
> A lot of capacity is coming online in the next 5 years
Is this true though? I don't really have time to do the research and have no dog in the race but I'm sceptical...
But then again I'm not there one making our profiting from the claims that there is billions being invested into infra...
I agree with the general direction but I'm a little skeptical of the "just add a few more TB of RAM and the frontier moves local" version of it
The established AI players have no financial interest to make LLM available locally. They aren't hardware companies and if running LLM requires paying them to host the models as well then they can naturally capture more of the value chain = more revenue.
Apple is the only player here where it would play into their natural hardware incentive to get you to pay more for better hardware. It would make sense for them to find a way to run LLM locally (eg, newer architectures that others here have pointed out).
Interesting times.
Is it hyperbolic though? One of the best things about the compute and memory shortage is that people are going to insane lengths to optimize things to run on lower memory / lower compute devices. If we keep this up for a while and then ramp up memory and local compute production, that AI inflection point may actually come.
Of course, these are a lot of ifs.
If we advance just 2x in hardware plus 2x in software, all coding can be done on local hardware imho.
That’s about 4 years in hardware cadence alone. There is a lot of room to improve memory bandwidth, and performance is a given with every process node. IBM has shown yesterday they can do limited runs on 0.7nm (density equivalent).
The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
There is some signal that this is possible through both hardware innovation and training/data improvements.
Cloud models have their own constraints - I can’t have opus4.8 spend 4 hours on a deep research question I had in the shower without spending money. I can’t do real time video game upscaling and graphics work in the cloud period.
A laptop is about an order of magnitude cheaper than a cloud server thanks to economies of scale, uptime requirements, and other factors.
> The big question for local LLMs is whether there is a 100 tok/s model which requires less than 16 GB of memory and is competitive on most tasks with the cloud models.
Benchmarks maybe? Real world, no.
You just need the context otherwise. There's no way around it.
Context is more available locally. You can have the LLM operate for arbitrarily long periods, use your credentials to access services (if desired), store memory locally etc.
Whether such a model exists or not is a different question.
if you do the electricity math you'll see that you pay more on local models while getting less (local is more heavily quantized) compared with OpenRouter.
I'm not talking local Gemma/Qwen vs cloud Opus, but against OpenRouter same Gemma/Qwen
there are reasons to run local - privacy, availability, but cost is not one of them
I am allowed to plug in 800w of solar panels into a wall socket here in spain. That would more then cover my current computer with 16gb vram. Now if i went and built a LLM server, at full load i would probably be closer to 3600w (Dual Epyc CPUs that gives you 8 x16 PCI channels and up to 8 cards - Way overkill, i know). If i half that with 1 EPYC and 4 x16 PCI channels, and add the same amd 7800xt i currently have then i should in theory be able to run at around 1800w under full load. Now that could still be covered with a 2000w solar install (get a professional setup OR get a battery unit like a EcoFlow that can output 2000w and can input about the same amount of solar).
Now, this all brings the upfront costs way up, the solar panels are cheap, its all the rest around them that tends to cost money.
That's assuming consumption pricing remains as-is.
There has been a lot of market-subsidy in AI which is starting to fade away: e.g. the copilot quotas/pricing. When VC switches from investing to wanting a return, the price equation is likely to change.
There is no subsidy on most OpenRouter providers, they are profitable today.
You buy a big GPU, you serve LLMs, you print money.
And if you skip open router and go direct you save another 5%
> do the electricity math
Could you give an example with real figures?
obviously depends on your location and GPU
for me it would be about $2 per day in electricity to generate 8 mil tok of Gemma4-26B at 4 bit quantization. this is excluding how much the GPU cost (no amortization)
ignoring the fact that I could get more free tokens per day for this model from Google/OpenRouter, it would cost $4 per day on OpenRouter if paid, but they would run it at full 16 bit precission
this would be the most "profitable" model for me
for Gemma4-31B I can generate only 1 mil tok per day, and so I pay more to get less quality than OpenRouter (ignoring that this model is also free on Google)
This is very interesting to me. How do you come up with the kwh/token for your setup?
I know how many tokens per second my computer generates, and I have a wall power meter which measures how many watts my computer is using when generating.
The thing is, with the level of hard investment AI vendors have, even a small reduction of their addressable market is significant. They aren’t profitable, and inference is getting commoditized fast, so even if they eventually become profitable (not via financial engineering) they won’t be able to have good margin. The pressure of both open models AND local models is pretty bad imho
> Hardware that can only run quantized models that take up GBs in VRAM
That's the today hardware.
Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output" and then releases M7 devices with enough memory and compute to run bigger models.
> Unless Apple surprises everyone and brings back the XServe with M7, if not, then laptop and desktop for factors simply can't dump heat fast enough to compete head-to-head, and will be designed for lower input wattage.
Laptops maybe. Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet.
> Now suppose Apple goes to any of Samsung/Micron/Hynix and says "we'll pay you the entire cost of building another DRAM fab and in exchange we want its entire output"
It's revealing that they aren't doing this: no one wants to fund that gamble on the state of AI demand 12-18 months out, but ate happy to capitalize on their current product lines/capacity.
> Desktops can dissipate more heat than the amount of electricity you can draw from a typical household wall outlet
100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
> It's revealing that they aren't doing this
They are doing this. All three of the DRAM manufacturers are actively expanding capacity. It's just not an instantaneous process.
> 100% agree, but the data center power and cooling infra are not limited by home wiring, and go way beyond what a wall outlet can safely provide (1,440W max on a typical 15A circuit at 120V). A single H100 maxes out at 700W
A desktop with the performance and the 160GB of VRAM of two H100s would sell like hotcakes. The power consumption is also largely proportional to the amount of compute rather than memory, and can be significantly improved by using lower clock speeds and then either making up for it with wider silicon or just not caring that it can run the same model but is e.g. 25% slower than hardware that uses twice as much electricity.
I'm not paying for a super computer to do my taxes if a cheap pc can do it for free.
So yeah, commercially it might be a death knell. Yes there's still a market for super computers, but would your rather own Apple or Cray?
> would your rather own Apple or Cray?
I would consider an HPE tower server with a processor on the same league as an M6 or M7 under the Cray brand.
I think this is right but it also depends on what "compete" means
Indeed. Local models becoming available and halfway decent don't obviate the laws of scale. And because there's no ceiling to what scaling more will buy you in terms of capability, there's no reason not to scale more, there's no incentive for billionaires not to grab all the fab capacity they can.
Enjoy paying $1000 or more for a little 4 GiB cloud terminal that connects you to all your online accounts where all your actual work gets done. This is the future.
>there's no ceiling to what scaling more will buy you in terms of capability
This is highly doubtful.
Rule of thumb: everything people think is exponential is actually an S curve.
There's a limit that won't be breached without a fundamental breakthrough in physics of computation, but we're not there yet by a long shot. You can train bigger models, faster, and infer with them faster and more precisely, by throwing more compute at the problem for the foreseeable.
At some point, and I can already see it, they’ll be better than us at writing code. We are still in the loop to coerce them into architecting well, but that’s nothing magical.
What’s frontier now is prosumer in a couple years and commonplace in a couple more.
better rule: exponentials are overlapping S-curves
Indeed. If Apple makes it feasible to run models like GLM 5.2 at home, I will become their customer.
It's plausible but is the Apple Tax for a 1TB memory machine on top of current memory prices really worth it? I paid around $4000 for 4090m laptop with 16GB VRAM back in 2023, it's great but DoA for even quantized LLMs. I can run SLMs and fine tune it but that's it.
We need one of those specialized inference chip startups to succeed and a PC manufacturer willing to bet on them against Nvidia for the local AI to find mass market appeal.
I recently bought a Mac mini M4 16 GB - mostly to run Immich. I assumed I needed a Linux box. After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax.
> After a lot of researched I was quite surprised that the mac was the cheapest option
For Immich, the cheapest option will either be a NAS or a used laptop depending on the amount of data you need, I wouldn't buy a mac for that.
Maybe he wants really fast or large AI models inside immich?
(I just run the defaults on my CPU, works for me)
I'm not sure why you would need large AI models for Immich, the face detection is pretty cheap and will run on 10 year old hardware without a blip.
I think the decision comes primarily on how much data you would like to store for Immich, if you want to go cheaply, a 100 bucks used laptop will do the job, if you have too much data, a NAS will be more suitable (and you are certainly not going to get a mac where you can plug multiple internal hard drives for the price of a NAS)
Not for faces, but the CLIP model for the context search https://docs.immich.app/features/searching/
That needs to be in (v)ram for searches.
That can be yeah but personally I wouldn't pay 3 times the amount you would pay normally just for that feature in particular.
Yeah, me neither. Fast enough for me on CPU.
>" After a lot of researched I was quite surprised that the mac was the cheapest option. So not always an Apple tax."
Apple has always been the most cost effective choice for the value you get going all the way back to the Apple II, it's just that the floor of that cost has always been high. Anyone who thinks otherwise is a just a fanboy one way or the other.
That's true only for the entry level macs. My M4 Mac Mini has the best Performance/value. But my workstation laptop with 32 cores, 96GB DDR5, Nvidia GPU costs lesser than Macs with lesser performance; not to mention I upgraded the RAM post purchase.
It really depends what you factor in as value, because wintel laptops like you described tend to require noise canceling headphones when working on them.
True, can't beat current Apple chips on power efficiency.
RAM upgrade possibility has a downside though - very low RAM bandwidth, which is highly relevant today if you want to run local LLMs
If you're happy with the base model, then Apple has a great value/price ratio, but there's nothing below it and any upgrade is really bad value for the money.
Hmmm, not always. Between at least 1998 and 2005, PCs were just better. Better CPUs.
If you think the apple tax is high you should see the nvidia tax.
> I paid around $4000 for 4090m laptop
That's how much many developers currently spend on tokens - every day. Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
>Whatever "Apple Tax" applies to a device that can run a capable model offline will amortise itself in a blink.
Current high-end Mac Studio with 32-core M3 Ultra chip and 96 GB of memory is $6800, 96GB is not enough to run GLM 5.2 without extreme quantization or stacking HW; but for the sake of discussion let's run quantized version on a single high end Mac Studio.
GLM 5.2 Max plan costs $ 112/m, so it would take ~60 months to recover the costs assuming the machine was bought just for AI. By then the current AI landscape would have changed drastically.
I use local AI on both Linux and Mac every single day, there's freedom, privacy and peace of mind in running the model locally. But I feel cost/value of local AI is overblown.
In what sustainable world outside of Bay Area jobs do devs spend 120k on tokens monthly?
Nobody said anything about sustainable, or outside of the Bay Area really
Apple can't make devices they sell to 1000 people once, so yes, you did say that, implicitly.
"Apple tax" is such a lazy and inaccurate accusation to level. Sure we've had expensive wheels on the cheese grater (ie Mac Pro) but we've also had:
1. When Apple came out with the real Macbook Air in 2010/2011 (not the silly 2008 one), nobody could compete with it with those specs at that price and they couldn't for years. And every competitor usually sucked in some major way, most often the trackpad;
2. The Mac Mini is an outstanding piece of hardware for $600. Or was;
3. I've generally found that "Apple tax" complaints levelled against the iPhone to be nothing more than Android cope;
4. The M-series silicon has been an absolute game-changer. I honestly thought the first-generation M1s would be not great but they came out swinging. And the price points for these Macbooks have all been great, much better than the last-gasp-of-Johnny-Ive touch bar butterfly keyboard series, which were objectively awful.
you're not a customer of any of their products at all already? not a single apple device in your household?
I didn't have a single Apple device in my house until a month ago when I bought a Neo. The last Apple devices I had before that were an iPod Nano and a PowerMac G5 many many years ago.
Apple has pretty good competition in every segment with the exception of maybe the iPad, but I'm not a tablet user.
Some folks like to have a computing environment free of proprietary influences and extremely strong vendor lock-in. I cannot claim to posses any apple devices.
How does the Mac have extremely strong vendor lock-in?
Sure, you can use the App Store and use all the stuff that integrates with iPhone, iCloud, etc
But you can also just treat it as Linux for Laptops (that actually works), and roll with all the standard open source tools.
I don't disagree with you, but technically speaking MacOS is still proprietary and Asahi is not compatible with the latest and greatest Apple devices.
While they don't _prevent_ Asahi from doing what they're doing, they certainly don't go out of their way to make it easy for them.
I wasn't thinking of Asahi. Just pointing out that you can run all the standard unix/open source tools and apps on Mac OS (vi, git, qgis, blender, vsc, python, node, etc). With the advantage of higher quality hardware and generally less fiddling.
But if you don't like it, switch. I don't see vendor lock-in.
A lot of convenient things come with the lock-in if you have >1 device
Notes sync, Copy/Paste would be hard to give up and took zero effort
I don't now how the MacOS equivalent compares, but Linux/Windows has KDE Connect that I use for that https://kdeconnect.kde.org/
Apple is also iOS
Correct. Been using Linux and Android for over ten years. My household had no Apple devices until I got married.
No, I've never owned an Apple device in my life, neither has anyone in my family to my knowledge.
None. And I have a PC, a personal laptop, a work laptop, my current and my previous Android phone.
What a bizarre bubble you live in to even be asking this question... I've never owned a single apple product, and never will.
And in the rare occasions in which I have to use someone's MacBook, I'm completely lost - like some elderly person.
there a many people who don't own Apple. Why are you so surprised? I certainly don't and never will. What's it got that I can't get on a standard PC + Linux?
Not. A. Single. One.
Tangential: About 8 years ago ex-Apple chip engineers left to design server-grade chips, this was Nuvia, and they got sued by Apple to the point that they had to get acquired by Qualcomm.
Then, after getting acquired by Qualcomm they got sued by ARM.
So maybe they were assholes.
Or maybe corporations hate competition. We'll never know!
I worked at a hyperscaler when the M1 came out. A MacBook Air M1, running a Linux VM was faster and more energy efficient than anything we had in the data center.
I'm certainly willing to believe the M1 was the most energy efficient, but your data center didnt have anything faster than a laptop?
Nope, across all benchmarks, Linux running in a VM on the M1 had higher perf (4 cores) than any instance type I launched across Intel, AMD and Arm. Triad, coremark, openfoam, verilator, etc. This may have changed, but my hunch is this is still probably true with M5 vs all the cloud providers.
That's not especially surprising for low thread counts. e.g. if you look at the current Geekbench single core charts, the fastest device across Android/iOS/Mac/PC is the M5 MacBook Pro. Second-fastest is the M5 iPad Pro. The fastest PC CPU (9950X3D) is about 80% the performance of the M5. (And the 9950X3D is about on par with the A18 in the iPhone 16 from 2024.)
(All the usual caveats about geekbench scores apply, but they're not nothing.)
>AND basically no play I'm the hyperscalar market.
I do wish they have Xserve back, or a Mac Pro that is Rack Based and support multiple node with M6 Ultra. The Hyperscaler market is so large along with AI their old business case of Xserve didn't make sense no long hold true.
They do stand in front of a great opportunity that would also benefit consumers, which seems rare in the llm era.
If people can get opus4.6/gpt5.5-like models locally, labs could raise their prices and sell token speed, better reasoning, mobile-focused improvements, you name it.
Not all consumers are power users and many will be happy to pay for flexibility.
Most people don't actually want to manage models, updates, context limits, quantization, etc. They just want the thing to work everywhere
Once one person figures that out and writes a blog post, everybody else can do it.
Yes, just like 90% of regular users set up NASes instead of just using Dropbox or Google Drive.
https://xkcd.com/2501/
The comparison is against your own computer, which is powerful enough, no need to add a box and set it up, and a very expensive cloud service. It’s nothing like setting up a NAS versus using iCloud (because we are talking Apple here).
I really wish people stopped saying things "I've been saying that"
why not just say "I think that"
do you see yourself as some kind of visionary about this particular topic? literally EVERYONE is saying that, it's the most obvious fact about AI
I'm not sure it's a death knell for frontier labs so much as a narrowing of what people need them for
When you've raised hundreds of billions in funding, every result except "to the moon" is a death knell.
LOL at the downvotes. I'm sure that the shareholders aka wealthy people at the top have a lot of patience for huge gambles that don't pay off. If OpenAI and Anthropic turn out to be the IT equivalent of mom and pop stores 5 years from now, their current shareholders will be ecstatic!
768GB RAM pipe dreams make no sense to Apple. By discontinuing 256GB / 512GB M3 Ultra and raising prices $5000 -> $7000 on Macbook pro with 128GB they basically confirmed how badly RAM shortage affecting them.
768GB is 64-times of 12GB which is rumored to be amount of RAM in new iPhones. Imagine what profit margin 768GB Mac Studio gonna need in order to justify making one instead of 64 iPhones.
Apple is the company that is okay about selling microfiber cloth for $100 and wheels for $700. Imagine how bad price hike for M3 Ultra 256GB / 512GB had to be in order for them to just discontinue them instead of getting free money out of desperate local AI folks.
They could treat the extreme spec machines separately from the prosumer ones, like they did with the Xserve. Let business customers spec up to 768GB (say) who are prepared for a $20-25k price tag, while keeping them away from the stores and usual consumer supply chains (Amazon et al). It may not be a big enough market segment for them to care about anymore, though.
They can do it, but that gonna need to be different SKU not Mac Studio. Otherwise news will be full of discussions about Apple price hike from $8000 to $24,000 or who the hell knows $48,000.
So yeah the only way I see them selling it is usual "call us" enterprise price tag.
But since its not what Apple usually do its easier to sell 4x Mac Studio 256GB RAM boxes with interconnect for lets say $12,000 - $15,000 each.
I don't think it's needs 'call us' just a separate SKU, I mean if they called it Xserve Ultra and it was just a studio in a 1u format with dual PSUs and extra RAM, it would fly off the shelves.
Isn't that the same thing you can get from ordinary 2S Epyc/Xeon servers at a similar price that have 24 memory channels (when the M3 Ultra has the equivalent of 16)?
And the reason people rarely use that for AI is that the enterprise GPUs from AMD and Nvidia are only moderately more expensive but are significantly faster because they use HBM instead of DDR5.
Yeah kind of, I think a 24 channels DDR5 works out approx 1TB/s, but the cost is astronomical, a M5 studio would probably beat that performance for around half the cost. You also get to use the GPU/NPU cores of the mac vs CPU only on the servers. M5 ultra studio with 128GB RAM could probably beat out a sever with a RTX 6000 pro at half the price.
24 channels of DDR5-6400 is 1.2TB/s, M3 ultra is 0.8TB/s. They both use DDR5-6400.
> a M5 studio would probably beat that performance for around half the cost.
A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want. The price seems pretty comparable. The "problem" with doing this is actually that 128GB is too little memory, because you want to populate all the channels, but even using 16GB sticks, 24x16GB is already 384GB.
> You also get to use the GPU/NPU cores of the mac vs CPU only on the servers.
You only need enough cores to make sure the bottleneck is memory bandwidth.
M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now. Even if this was M4 Ultra it would still be ~comparable to any EPYC system in bandwidth, but get to use the GPU for compute so potentially faster than the EPYC. Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk. More RAM though, but again it's more about the ratio of RAM (size) to RAM (Memory Bandwith) to Compute and you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth and therefore the extra RAM (size) is wasted. But maybe not, different use cases will yeild different results I guess.
> A barebones 2S system with no CPUs or memory is ~$2000, a pair of 16 core CPUs another ~$1000 each, and then however much memory you want.
As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200, then you also need at least one NVME disk so you're looking at what $13,000?
> M3 ultra is obviously 1-2 generations behind and new the studio is expected 'any day now.
M3 Ultra uses a 1024-bit memory bus, which is a major inconvenience to Apple because they're soldering everything. In ordinary systems if you have 16 memory slots and any one of the memory chips is bad, you replace that stick. If the processor is bad, you replace the processor. If the system board is bad, you transfer the processors and memory to another one.
If any of those has a defect after you solder thousands of dollars worth of memory onto the same board as a >$1000 CPU, you're not doing well. Worse, the more memory chips you have and the more pins the CPU needs for its memory bus, the higher the chances of one of them having a defect.
In addition to that, when you get to that number of pins it starts getting harder to run the memory at the highest speeds. The M3 uses DDR5-6400 but some of the M5 line is using DDR5-9600. It may or may not be possible to do that speed when using a 1024-bit bus -- not every existing M5 even does it. If it isn't then the Ultra wouldn't be much faster than the Max since it would have to use a lower memory speed. If it is then the tolerances would have to be even tighter and increase the defect rate even more.
Which is to say, I can see why they haven't released an Ultra since the M3.
> but get to use the GPU for compute so potentially faster than the EPYC.
Only if the bottleneck is compute rather than memory bandwidth, and for LLMs it's generally memory bandwidth. And if something significant was compute bound, there are also higher core count CPUs.
> Total Cost of Ownership in the Epyc is going to be WAY higher because of electricity costs, the EYPC is going to be consuming probably 5X the electricity and is probably not going to sit quietly on your desk
The M3 Ultra has a 480W TDP. There are relevant EPYC SKUs on SP5 at 125-200W/socket.
> you may find a model bigger than e.g 70b suddenly is bottlenecked by the CPUs or memory bandwidth
The extra RAM allows you to fit the larger model in memory to begin with, without which it's pretty hopeless. The bottleneck typically is memory bandwidth for LLMs, but that's true pretty much regardless of the model size. Moreover, mixture of experts models require significantly more RAM for the same amount of compute/bandwidth.
> you also need at least one NVME disk
That's ~$100.
> As you say, the thing is it's not 'however much memory you want' it's 24 sticks which at $300 a stick for 16GB is $7200
The chips don't cost a materially different amount based on whether you solder them. Apple presumably discontinued the 256GB and 512GB versions of the M3 Ultra because they'd have had to add a similar number to the price.
> but even using 16GB sticks, 24x16GB is already 384GB.
Question: you need 16GB sticks because they're the smallest doublesided ones, which you need for maximum BW, right? Otherwise why not 8G?
If 8GB sticks of registered DDR5-6400 even exist, they're not common.
That was because memory used to be cheap. Will be curious if smaller capacities come back en vogue as a cost cutting mechanism.
Using a server system designed for having hundreds of cores and TBs of RAM for LLMs because it has a medium-high amount of memory bandwidth is a hack for enthusiasts who want to take the price/performance trade off to run big models without paying for enterprise GPUs. The market for 8GB RDIMMs would be specifically that, since the more typical server workloads that need that amount of memory bandwidth also want the larger memory sticks (e.g. database servers, systems running hundreds of VMs), or aren't bounded by memory bandwidth to begin with and then don't need to populate all the channels.
And if you wanted that market segment then what you'd really do is produce a consumer GPU with 128GB of RAM.
Imagine if they bring back the Mac Pro with 768GB of ram to compete with the $100k DGX Station.
A very good idea, the Mac Pro also had a rack mount option didn't it? That would be the kind of thing you could sell as many as you could make.
> That would be the kind of thing you could sell as many as you could make.
People said that about the M1 Ultra Mac Pro, a few years before it was discontinued. I don't think there are many HPC customers looking at Apple hardware.
The timing of the demise of the Mac Pro has been about as bad as the Steam Machine launch timing.
You mean the Mac Pro that was still on the M2 and hadn't been updated for 3+ years?
> Let business customers spec up to 768GB who are prepared for a $20-25k price tag
There is a clear difference between $25k and $100k.
64 iPhones at retail price is already around $64k. For something at 768GB to be profitable at Apple's terms, this has to retail at $100k for it to be profitable. That was the OP's point.
Honestly you're still looking at (from my understanding) ~3 minutes prefill (TTFT) even with architectural improvements and so on with a 32k context window (against a large model). How is this going to be competitive with Nvidia and all of the tricks massive scale get's you to parallelise context across many machines?
Is it supposed to be? I think the point with some of these Macs is you get the capability in something the size of a heatsink from Intel's Netburst architecture era, or a Macbook light enough to stick in a backpack and take with you to lunch.
If you're talking about chaining together multiple GPUs you're talking about a different game -- I suspect, anyway. Seems like a high-spec Mac would be good for development and testing. Arrays of GPUs, better aimed at production use.
They'll sell more $20-25k Macs when Goldman green lights Apple Card credit limits that will fit one of them.
Apple Card already offers credit limits high enough for it
The message I’m getting is that Apple will never compromise on its healthy margins. If something becomes basically unaffordable for their target market, they’d cut the production and even discontinue the product, than take a hit on margins. Their business model is refreshingly simple.
That works only if the product is the product.
Iphones/tablets drive app sales/apple subscription services, if they force a user to move to android they may never return.
Why do you think they sell the iPhone 17e/se? They need to maximise their user base as its ongoing recurring income stream.
Apple is not gonna risk their iphones, as they are their flagship (aside even from giving them higher margins). My opinion is that, as we are talking about ram SHORTAGE (not just for ram price hike) they have to cut the more ram hungry models to be able to keep up with their projected production/demand (at reasonable ram prices). Getting iphones "sold out" is not a great thing for apple.
Once/if the ram shortage ends, they will continue increasing the ram caps as they were already doing, because then selling ram-heavy macs will not interfere with the rest of their products.
It's just a planned economy failing the way planned economies often do: the central planner failed to predict the demand correctly. Instead of trying to secure additional stock from the market at spot prices, they are simply waiting for the next batches they had planned for.
I don’t think that represents the scenario at all, not to mention the fact that it’s literally not a planned economy (but also not very analogous to one, either).
What’s really happening is that the effort of securing additional stock isn’t worth it because the price is so high that there aren’t enough buyers.
If ground beef were to suddenly cost $50/pound, McDonald’s doesn’t raise the price of the Big Mac to $25 and hope people buy it, because it makes zero sense for their business model. Sure, some fancy restaurant will still be selling hamburgers, but not your chain of thousands of working class fast food restaurants. McDonald’s would find some other alternative item to sell.
The truth is that nobody’s going to be buying Mac Studios that cost $25,000. Not even enterprises.
Businesses are usually planned economies, and supply chain management is literal central planning.
Apple failed to predict the demand for Mac Studios. Many other companies in its supply chain likely failed to predict that Apple would come back asking for more. There is no excess stock for some key components or the spare capacity to make them on demand. Apple would have to scour them from the market, likely paying much higher prices than it will pay for scheduled deliveries.
Hopefully I’m not being too pedantic, but I am saying that we can’t just use the words “planned economy” or “central planning” to describe a single company’s actions.
It’s just not what those words mean.
Apple is a large company but they are still just one company.
A single business can’t be doing “central planning,” it is by definition not in charge of the whole market.
Central planning would be if the government mandated that Micron to make ## of memory chips and distribute ## of them to Company A and ## of them to Company B.
Apple has no such power.
https://en.wikipedia.org/wiki/Planned_economy
I actually think Apple is right in increasing prices, and in fact should have increased it more.
Their target market is composed of people that would pay for it nonetheless. They should have tightened the screws a bit more.
I think you have an excellent point and I'll bet someone at Apple has all this in spreadsheet and is making that case.
However, I think without the very high end machines Apple is also seeding a lot of professional middle market too.
If the choice is between, say, a Framework desktop vs nothing from Apple I'll obviously pick the Framework.
If I get used to a Framework desktop running Linux then I'd probably stop buying MacBook Pros.
Right now Apple has a chance at capturing local AI but that opportunity won't last forever.
If they cared about local AI market they would price hike M3 Ultra instead of discontinuing them. After all they conviniently introduced RDMA just few months before that.
Initially when it happened everyone expected they did it because they planned to announce M5 Ultra shortly, but its not looks like this is happening.
Now IMHO its indicates they simply run out of RAM supply.
If you’re truly serious about local AI you’re not using a Mac. A Mac is for people who want the best of both worlds, GPUs are much faster if memory is equal.
do you mean ceding that market?
Yes! Thanks for point that out as it's an important difference.
Mea culpa, dyslexic moment.
This is only true under the assumption that RAM price per GB scales linearly.
It's even worse than that. Demand is so high so every next GB will be sligly more expensive. There is a lot of smaller hardware manufacturers that unable to secure DRAM chips at any price, there just no free capacity on market.
Of course Apple is massive, but if they announce inference boxes that everyone wants they need to make 10,000s or even 100,000s of them.
And it's very much likely they'd rather sell 600,000 - 10,000,000 more iPhones or Macbooks Neo. End users bring Apple money with every OpenAI / Claude subscriprion sold through their platform.
And inference boxes is just one-off sale of hardware that will bring no further income.
Forward planning though.
They are assuming that they are able to get ram in the future, once the AI bubble either dissipates or pops. Its far easier to build something you planned for 3 years ago, than crash build it in 3 months.
Of course if RAM prices crash Apple might of bring high RAM options back. We just cant bet on it as consumers.
Right now RAM shortages are bad to the point where likely even Apple have to decide what products they make and what they discontinue.
There been short time where M3 Ultra with 256GB / 512GB been best offet on market because Apple lagged with price increase. Now HN crowd expect Apple of all companies jump into price war with Nvidia and to subsidize their inference hardware.
[dead]
The article says base M7 memory bandwidth is targeted at 240GB/s.
M1 had 70 GB/s, M1 Pro: 200, M1 Max 400, M1 Ultra 800.
Modern RTX 6000: ~1,600 or so.
If we get a 1,200-1,500 GB/s bandwidth M7 variant in late 2027 with 512GB of RAM, that will be a very interesting chip. Tracking LLM size and performance improvements, I can imagine that being a sort of inflection point for local inference. I wonder what the power budget would be in desktop format.
A hypothetical M7 Ultra with LPDDR6 14.4Gbps memory would be 1.85 Tb/s.
You're look at about 100 tokens/s for a 1T MoE 37B active 4bit model.
It'd probably cost $30k or more I'm guessing if memory prices do not come down. Even at $30k, it could still be a relative bargain since an RTX Pro 6000 Blackwell 96GB card costs $12k today. The M3 Ultra with 512GB was around $8k before Apple discontinued it. I expect an M7 Ultra to have 768GB or 1024GB.
Apple Silicon Macs were on their way to becoming cheap local LLM machines relative to professional GPUs before this memory crisis. It may still emerge as such in a few years.
Here's some interesting math: At 512GB, an Ultra chip could make 42 pro iPhones. Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB. If they only have a limited supply of RAM for all their products, it makes no sense to produce an $8000 M3 Ultra 512GB when you can produce 42 pro iPhones. You can only configure an M3 Ultra up to 96GB today as of June 2026.
Apple would have to raise the price of a 512GB Ultra Mac to around $50k to match iPhone profits.
> Assume a 55% profit margins, and $1200 ASP, you're looking at $28,160 in profit from making iPhones instead. No wonder Apple discontinued the M3 Ultra 512GB.
How would that work? They purchase 512GB from Samsung and then it doesn't matter if that's like 128x 4GB or 4x 128GB?
It's likely the capacity they have reserved can be in different combinations.
Note that this reserved capacity now has competition from OpenAI, Anthropic, xAI, Meta, Microsoft, Chinese data centers and so on, all willing to pay premium.
If comapnies keep spending half a macbook neo worth of subscription on AI plans monthly per person, Apple is going to have a hard time competing.
Companies are spending even more than that if they’re using the $200 subscription worth of tokens on the enterprise plans too.
That's a very big if, though. There's been extensive news coverage about companies increasingly trying to move away from tokenmaxxing
but they move from tokenmaxxing to tokenmidding, not to tokenzero
That's part of why Apple is building on 18A from Intel next year.
Where did you get 55% from? iPhone and Mac gross margins behave been 40% or so for years IIRC.
Quick internet search. Whether it’s 40% or 55%, the main points stay.
An ‘ypothetical!
In what neck of the woods? English pronunciation never gets boring.
In British English the "an" is correct, even though most English dialects don't actually render the H as silent. It's a French-derived word that had a silent H originally, ergo we use "an".
> A hypothetical M7 Ultra with LPDDR6
That’s indeed very hypothetical considering that Apple silicon uses on-package HBM.
Apple only uses LPDDR.
> The M3 Ultra with 512GB was around $8k before Apple discontinued it
The base model was $9k, that much RAM got you into $14k range.
512GB was around $9.5k. The $14k would be if you upgraded to 16TB SSD.
https://youtu.be/jSYobH9kr1E?si=hc1xUQ37_SEbkDkj&t=1242
I keep forgetting you fuckers show prices before taxes. Yeah, sure.
I’d assume by next year the open weights models will be outlawed the way things are going nowadays :/
Edit: for those of you downvoting I don’t celebrate this prospect. I’m merely realistic about where things are going given the rapid vibe shift from the administration on AI since the start of June.
[dead]
M5 is 153GB/s, M5 Pro 307GB/s, M5 Max 614GB/s.
The article didn't state the M5 Ultra won't be released. It will probably provide 1228GB/s of memory bandwidth this year.
Problem is affording the ram…
Apple is finally going to realize Jobs vision where sand comes into the factory, is turned into RAM and CPU chips, then installed in a Mac or iPhone then shipped to a customer.
Isnt the proper kind of sand in short supply aswell?
Just curious, is there a way to invest in sand?
https://www.theguardian.com/environment/2026/may/12/global-s...
As some like to call it, 'the last moat'.
Well yes. But similar to the Apple TSMC relationship, could Apple step in with large orders to established RAM makers such that the RAM makers can invest with stability?
M series chips are system-on-chip with RAM on the same wafer with CPU and GPU, so it impossible to outsource only part of the chip
No it isn't, DRAM is made with a different process and those are chiplets, perfectly possible to outsource, and the only possibility really as TSMC does not make DRAM.
Thanks for clarifying, then the suggestion above does makes sense
192gb or 256gb of RAM would be enough ! We could run real time large MoE models, REAPed for our usage (e.g. english agentic coding), dynamic quant 2-4bits
At this point and given the cost of memory, it will probably make sense to invest in faster SSD to allow for good performance with less memory
late 2027 is a very long time.
Well yeah but NVidia just released a contender to their silicon and the M6 is probably already set in stone. Best to reshift resources to a great M7 than having a mediocre M6 and M7.
(This is assuming Apple will deliver, but this area is one of the biggest ones they have in AI, and they need the developer ecosystem to exist and survive)
That would cost as much as a new car.
Don't worry, they'll just make cars more expensive
Come to think of it, modern cars have a lot of electronics such as touchscreens, cameras, and sensors. It wouldn’t surprise me if new car prices are not immune to what’s happening with RAM and storage prices.
Why don't people run their AI on their car? Two birds, one fragglerock.
I'm pretty sure the vision/hwa reqs for cars is much less than an LLM/genai in general so that doesn't quite work out. But it would be nice to have an AI server with wheels :p
Most cars with AI run their AI locally. Not LLMs though.
Maxed out 2019 Mac Pro was $50k+. The wheels on that thing were $400. This is a bargain compared to that.
Former AnandTech editor Gavin Bonshor had reports that the M7 would be manufactured on Intel's 18A node.
https://bontechlabs.com/news/apple-is-reportedly-using-intel...
Given the risks involved in establishing Apple Silicon designs with a new fab, I would expect early M7 parts to be in test production right now.
The fundamental M7 design is already set in stone.
Mark Gurman's Bloomberg article does not mention fabrication partners or processes.
I think that people are still underestimating the technical merits of Intel's 18A fabrication process.
I haven't seen any competitor even try to address the backside power delivery of 18A. I suspect that Samsung,TSMC have something similar and doesn't talk about it.
The design rules for the standard cell (sort of corresponding to the die area required by a transistor) for the Intel 18A seem to target dense, high performance designs. That's not a particularly meaningful insight - of course Intel wants to have the highest performance of all the fabs.
Intel's packaging expertise used to be a generation ahead, and indeed their server chips currently use a mad mix of chiplets and through-silicon visas for direct stacking, all heaped onto a reticule-limited monster interposer die. All of this expensive complexity might be sustainable as long as Intel can keep its enterprise customers happy. That hasn't turned out too well for them.
AMD has found a mass-market winner with mainstream gaming CPU with extra level 3 cache die stacked on top. Compared to Intel servers, it's brutally simple. But extremely effective in its consumer market.
But the Intel chiplets and packaging could be a great toolbox for M7 generation of Apple Silicon. Now that the M5 Pro and Max are multi chip packages, they more resemble the Intel and AMD designs, with chiplets dedicated to I/O or GPU.
(Speculation and dreams. That's all I got, and I'm writing it in the face of an absolutely psychotic autocorrect on a tablet.)
> I suspect that Samsung,TSMC have something similar and doesn't talk about it.
They do, just not as hyped up as Intel. TSMC will have it after 20A. Either 18A or 14A. GAA was supposed to be in 3nm but didn't happen due to multiple reasons. So it is now delayed to 20A. Backside delivery was supposed to be 20A and also got pushed back as well.
I find this rumor at least plausible.
As we all know, Intel used to be famous for their engineering and their ability to scale up a newer, smaller process with way earlier commercial viability. This all ended with the Sisyphean 10nm move that was years late and honestly Intel just don't seem to have recovered from it.
So Intel seemingly has underutilized fab capacity whereas the likes of TSMC and Samsung can probably produce every chip they make with demand to spare. Given the CHIPS Act that was passed under Biden, the Trump admin taking a stake in Intel and the environment of tariffs and a push for American manufacturing, everything seems to be lining up for someone to take advantage of Inte's physical fabs and American production and that could be Apple.
Wouldn't this help Intel compete?
If they have Apple's designs months prior to launch, rather than after launch.
Ripping off designs from your own fab customers is a pretty sure way to crater your fab business, and get sued into the ground at the same time.
It's not so much ripping off the designs - nothing of what Apple Silicon is doing is particularly surprising and both x86 and Intel's microarchitectures are sufficiently different to Apple Silicon/ARM that knowledge of specific implementation approaches wouldn't be directly useful in most cases.
The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
> The real advantage is knowing exactly what Apple is launching months or years in advance, because that can inform strategic planning.
While I'm sure some level of internal leakage does take place, at least on paper the fab's planning needs to be firewalled off from their own chip roadmap.
I'm also not sure how much Apple actually cares, tbh. Yes, they currently have an edge in silicon, but it's heavily due to being willing to outspend everyone else, and their real superpower is vertical integration - which Intel isn't in a position to compete with.
I think Apple doesn't really have a choice. They've been very strongly encouraged by the current US government to move as much chip manufacturing to the US as possible, and particularly to make Intel Foundry work, or face... problems.
Also the AI boom means NVIDIA et al. can afford to buy TSMC's best processes at scale, which means less available capacity for Apple.
I'm sure given no other forces at work, Apple would prefer to stick with what they were doing previously, buying the lion's share of TSMC's best process.
I mean in the opposite sense. If Intel can glean enough from Apple's roadmap to close the performance/watt gap, great, but they still can't match the vertical integration Apple has
M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC. Anandtech was observing that Apple had better IPC in their phone chips YEARS before M1. Lots of people discredited it as "apples and oranges" because the ISAs were different, but investigative teams from Intel and AMD absolutely HAD to know the truth.
This has both a technical and human component.
On the human side, top x86 execs refused to see any threat coming. They must have thought Apple couldn't overcome the x86 software moat, thought the chips were for servers, consoles, or some other non-PC device, or perhaps they simply couldn't believe what their investigative teams told them.
At the same time, we're 6 years post-launch. The proof of ARM's capability is clear. x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86. Either all the x86 engineers are completely incompetent and incapable of learning from years of ARM designs or there are aspects of x86 that makes copying those designs infeasible.
> M1 released 6 years ago, but AMD/Intel still can't get close to ARM cores in IPC.
> teams from Intel and AMD absolutely HAD to know the truth.
These people are professionals that acknowledge IPC is a stupid metric. If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM. Intel invested early on big.LITTLE, and AMD drilled down on denser nodes for their preexisting designs. As both businesses converge on each other's ideas, their SOCs have adapted most of ARMs' greatest mobile innovations. Even before that, x86 hardware was always usable - AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
All of this makes sense, nothing objectively prevents the x86 architecture from being power-efficient. Arm LTD. would have gouged any of those vendors for their IP, and even with an architectural license it's not like AMD or Intel would get usable core designs from Arm. There was no reason to pivot to ARM for either company, they both saw Qualcomm and could read the writing on the wall.
> x86 server marketshare is about to hit just 50% and Microsoft is pushing ARM hard as a replacement for x86
That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market. Apple's early ARM hardware was outright ignored for server/HPC applications, leading to the discontinuation of the Mac Pro. Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving, and Grace for the high-end of "we need CUDA and enough bandwidth for Infiniband" that made trillions in the HPC market.
>AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
At 30 watts TDP?
> These people are professionals that acknowledge IPC is a stupid metric.
Even if you think IPC is a stupid metric (it's not), ARM is still wiping the floor in total performance, performance per area, and performance per watt.
> If you switch your statement to SIMD throughput, now ARM NEON has the lower IPC and x86 looks like space age technology. They're optimized for different workloads.
Fujitsu also makes a 512-bit SIMD that has very good performance, but that large SIMD eats area and power while giving very little back to consumers. Honestly, SME co-processors or 6-wide SVE are much closer to "space age" than simply making a wider packed SIMD.
> x86 vendors recognized that they could recoup the majority of efficiency that Apple Silicon has without buying an architectural license for ARM.
AMD actually bought an architectural license and Intel used to have one too. They could (and should) undercut ARM entirely by making x86+RISC-V cores. You can talk about Intel's E-cores or AMD C-cores, but they still don't compare in the key metrics.
> AMD was shipping faster integrated GPUs than the M1 Pro before the M1 ever hit shelves.
Which GPUs are you talking about? 4800U (Vega 8) was around 50% slower than M1's GPU. 5700G released in April 2021 and also had the same Vega 8 GPU design. They didn't see a bump until RDNA2 in Rembrandt in 2022 (by which point Apple was already on M2).
> nothing objectively prevents the x86 architecture from being power-efficient.
Citation needed. This is like saying nothing prevents branch delay slots from being good or VLIW/EPIC/Itanium is just as good as anything else. uop cache, extra pipeline stages for decoding, and memory ordering prediction are enough by themselves to use more power when under load. If you have actual proof that x86 workarounds have absolutely ZERO energy/area/latency, then show it.
> That's Nvidia's work, no credit is due to Microsoft or Apple for reshaping the server market.
Linus Torvalds pointed out that ARM server wouldn't take off until devs could run ARM natively on their laptops. Apple provided those laptops.
MS claims Cobalt 200 lowers carbon footprint by 40% vs x86 (which should mostly reflect power consumption per unit of work). ARM accounted for over a third of Azure deployments in 2025 and should increase as C200 becomes available.
> The Graviton and Grace chips that displaced x86 servers did it because they are slower, cheaper and less feature-dense. Graviton for the bare minimum of Raspberry Pi-tier web serving
That hasn't been true for a long time now. Graviton5 in particular is a big deal. It removes the NUMA issues resulting in much lower latencies (and higher performance) for a lot of things. It gives 2.6x more cache per core and a general ~25% performance uplift (SAP said it was up to 60% for their workloads) which is more than the difference between graviton4 and Zen5 for most workloads. Graviton accounts for more than 50% of all new servers in AWS and basically all of their 1000 largest customers are using them.
> Apple was entirely incapable of pivoting their mobile chipsets to the server scale, surprising nobody that had paid attention to Apple's godawful raster/GPGPU acceleration stack. The Ultra hardware looked like a dog's dinner compared to x86 arches like CDNA.
Apple wasn't incapable -- they didn't want to because they aren't in the server market. That is the entire reason X-Elite/Nuvia exists. GPU stack is completely orthogonal to making server CPUs (you can make good CPUs without making good GPUs).
Just competing against their own FPGA customers was enough to crater Intel Fab 1.0
What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
Maybe this strategy works, even in that world.
Remember when we all thought (were told we thought) the world was heading to 3D views of our 2D lived experience like a solid Cube of GUI we could rotate around and live inside? Well Apple took the simple 2D square pane of virtual desktops and .. made it a SONY strip. One variable: sideways.
So here we are being told AI is the future. Apple seems to be saying "yes but it will run local" which might be a safe bet if AI comes true but I wonder how many of us want the AI outcome, which is morally speaking the 3D immersive GUI cube here: what if we don't want that?
I can't imagine any world where we put this AI stuff back in the box. It is simply too useful and too powerful. And as we start seeing all his upheaval where models are getting banned, etc, I can even see the appeal of on-device AI increasing for a lot of use cases.
So I think Apple has the right instinct. In fact, I've had the thought multiple times that I really want a lot of workflows just running on my device. Workflows like fast vector search (already fast on the m4, but I want it more common place), or realtime transcription and summarization to be even faster, on device, etc.
To me AI is on par with the internet and what made it so powerful was piracy and porn and just the wide spectrum of things that are possible when you connect machines together. We are going to need the same thing again. Freedom to use any model that does any thing we want.
If it happens it's because of china but it will be forbidden in USA.
Looking to China as a purveyor of laissez faire freedom to do whatever you want is so nonsensical as to be a joke. Both US government and the Chinese government want this newfound power to be on a tight leash.
[flagged]
https://www.aimadetools.com/blog/claude-fable-5-banned-us-ex...
Maybe two things can be true at the same time?
https://swelljoe.com/post/open-model-censorship/
They're still the only ones releasing any open model at all?
Plus, not like chatgpt is going to reply honestly about sensitive current hot topics in USA politics. Normally it has filters and refuses to respond as well.
[dead]
The worst case scenario is that we're at a plateau and LLMs max out around here. And it'd stand to reason that if that happens we'd see local models catch up at least to some extent. Compared to 5 years ago, that's a pretty good world.
So I think it's fair to say that AI isn't going away. That doens't mean that SpaceX, OpenAI and Anthropic won't crash. But I've long believed that within 5 years we'll have access to relatively cheap hardware that can run sufficient but not cutting-edge models locally. You can buy a 5090 PC for <$5000 already so I guess it's already true but I think we'll do even better.
So what happens? Nothing. If Apple make M7 Max/Ultra computes with 128-768GB of RAM and nobody buys them then... nobody buys them. Apple isn't betting the entire company on AI just like Google isn't. The rest of the internals are the same Macbook, Mac Mini or Mac Studio. You're just selling something with less RAM.
> Google isn't
Google sold (will sell?) about $70bn worth of Google shares to fund AI infrastructure build outs. It's also issued bonds (=debt; I forget the number, $30bn?) to pay for more infrastructure. Fairly sure it also has established a shadow company, a Special Purpose Vehicle (SPV) to stash away unpleasant financial things it doesn't want to show, also for the AI build out.
Amazon, Google, Meta, Oracle are overstretching at the moment. They are predicted to become cash flow negative (more money going out than coming in) if they keep going at this rate, some time in 2027 or 2028.
Now, they won't go bankrupt but it's possible they will be hit by huge restructuring waves once the dust settles.
Google has a market cap over $4 trillion and it's 2025 financials were $130 billion profit on $400 billion revenue, which was something like 15% Y/Y growth.
$100 billion in equity and bond offerings is not betting the farm on AI.
Additionally, Google's data centers are notoriously efficient and they build their own networking hardware as as their TPUs. Now their TPUs lag behind NVidia offerings obviously but they'll keep getting better.
Google is the company I'm least concerned about in the AI space.
As for Amazon, I'm not sure what their split is on in-house AI build out and usage vs AWS. I suspect the majority is for AWS and I also suspect their ROI is insanely short (ie less than 5 years) on any AI capex.
A lot of us on HN dismiss Oracle with good reason but it's also fair to say they've survived and thrived through the dot-com crash, the GFC and the pandemic until now. They're clearly doing something right.
Meta I think is the Sick Man of Tech. Their social media assets are of declining value (IMHO) and they seem completely unable to adapt to changing conditions. The whole Metaverse was a massive $70B+ boondoggle on something long before they had anything resembling a product-market fit. They seem unable to leverage their social media assets for building any competitive AI products or tech. Plus they seem unable to hire and retain key staff in this space.
https://s.yimg.com/lo/mysterio/api/619866B692DE6F8A91F783066...
> Google has a market cap over $4 trillion
For the purposes of most financial discussions, the market cap can be ignored. It tells almost nothing about the business fundamentals.
> and it's 2025 financials were $130 billion profit on $400 billion revenue, which was something like 15% Y/Y growth.
Most of that growth, which has re-accelerated, is AI driven. Aka eggs in a single basket.
> $100 billion in equity and bond offerings is not betting the farm on AI.
That's on top of capex of $185bn in 2026. In 2023 it was $32bn, 2024 $52bn, 2025 $91bn.
All your financial alarm bells should start ringing.
Amazon is doing similar things.
> A lot of us on HN dismiss Oracle with good reason but it's also fair to say they've survived and thrived through the dot-com crash, the GFC and the pandemic until now. They're clearly doing something right.
https://www.reuters.com/business/autos-transportation/cost-i...
Oracle is the most likely to go bankrupt out of all these companies outside of AI labs. They've been very adept at financials until now, when they've bet the farm on explosive AI revenue and especially profit growth. If those don't materialize soon, bye bye Oracle! Which would probably make a great many engineer happy :-))
> For the purposes of most financial discussions, the market cap can be ignored.
If you issue $70B in stock, which was your point btw, market cap absolutely matters. $70B is less than 2% new shares issued. If it was a $100B company, it would be 70%. That's why market cap matters.
> All your financial alarm bells should start ringing.
Only if you start with the premise of "Google is imploding" and then go looking for evidence.
Oh, so we're using CDS rates as a proxy for alarm now? Ok, Google's rates are hovering under 50 basis points [1]. For comparison, this is only slightly higher than the sovereign debt of Canada [2].
[1]: https://finance.yahoo.com/markets/stocks/articles/alphabet-v...
[2]: https://www.worldgovernmentbonds.com/sovereign-cds/
I love how you completely side stepped the cash flow comment AND the graph.
It's all right, I spent years arguing against cryptocurrency boosters. I'll probably have to wait a few more years for AI boosters to be gone into the nether.
I find it very unlikely that nobody buys them, some people will definitely buy them to run giant open LLMS. But there's also not much risk to apple because those configs would probably be made to order
AI already has massive, growing adoption, whereas "3D immersive GUI cubes" never really had any.
It has at the current subsidised prices.
I doubt inference costs will scale up significantly, but even if they do, it simply strengthens the strategic case for Apple's focus on local inference.
Inference costs wont increase but inference prices have to go up dramatically.
Investors are expecting trillions of dollars to come out of this play. That's only possible through monopoly pricing essentially.
"Being a profitable business", which selling tokens could absolutely be, is not sufficient. They need to own a large chunk of the world
Without AI everyone’s computing needs were pretty well satisfied with current phones and laptops. LLMs are the one thing that could drive new demand if they can run locally.
Not software developers, they were already flocking to Apple M series machines with more RAM and cores before LLMs.
AI was the only reason I bought a new computer (a refurb M3 max with 64GB). Without AI, no idea what we should bother with, it depends on what application comes out to drive local computing power (maybe better games? Yawn).
this is the backup strategy. the "AI doesn't pan out" scenario is basically if claude and openai go bankrupt, we continue running local models on our hardware.
there isn't a future where we all just decide that nah, we don't want AI anymore. usefuly things don't disappear.
"What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?"
Can't it do both? The M1 Pro with 16gb+ is still more than nearly everyone needs.
In between flat material-ish (hehe) design windows and 3d compiz cube with burn effects we have settled on transparency and blur effects with a bit of visual planes thrown in. It's highly unlikely we will end up tokenmaxxing everything, it's highly unlikely the genie can lose enough weight to fit back into the bottle. We will end up somewhere in between that strikes a balance between nice, productive and cost effective.
> What's their backup plan if the AI world doesn't pan out? What if it turns out people want base compute capability and lots of RAM for filestore cache and programs?
I think reducing the die area dedicated to ai stuff is not going to be a problem.
And in fairness apple already has essentially ai-less hardware in the form of the MacBook neo and it’s been an astonishing success.
I have one and it’s a very good laptop, particularly for the price i paid it.
Anything AI focused in silicon is also valuable for a ton of other use cases. If LLMs and GenAI don’t pan out, that silicon just gets used for other processing. Then they scale back on the dedicated die space in subsequent generations.
It’s all fairly easy bets to make and correct.
> what if we don't want that?
Do we have a choice? It's being forced upon us by folks who have the power to distort any market they want. Energy prices are rising, and the PC industry is about to be destroyed by component prices. It will be dumb clients that run the software our feudal overlords of the data centers will have the grace to grant us. And the government lets it happen because it furthers their interests.
China might be the only spanner in those works, so expect further action taken against them. Most likely sanctions on their hardware
those billion people which use LLMs every week, are they all being forced to use it?
AI isn't going anywhere, this is akin to the .com bubble. It burst, but the internet didn't go anywhere. While companies can fail, this technology is with us for the long run now, short of societal collapse.
Seems like a made-up distinction that shouldn't be necessary since M6 has not even released. I suspect this is a marketing ploy to meant to drive up both interest while also increasing prices for the next generation of Mac hardware.
What it's saying is that the M6 will be released, but not the M6 Pro or M6 Max. Instead, Apple will wait to release new Max/Pro chips for a future generation.
It's not simply marketing since the Pro/Max chips of a generation use the same cores as the regular version, just more of them or different combinations of performance and efficiency cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
The claim is that M6 will be released, but the only variants will be lower end.
When they get to the M7 generation, they will make high end variants.
It's a real distinction because each generation of parts shares an architecture.
The article has an entire section speculating what the M6 parts will be, but says they'll top out around 200GB/s memory bandwidth and 12 graphics cores.
> Seems like a made-up distinction that shouldn't be necessary since M6 has not even released.
Why would it? Each generation of the M series has an architectural improvement on their chipsets. The difference between an M1 and an M1 Pro is the allocation and arrangement not the architecture. M6 to M7 presumably will have architectural changes.
Is the point that M6 doesn't exist? What change are they making that justifies M5 to M7?
Or did this announcement also add an M6 chip, and they're just skipping pro?
This is no different than them skipping the “Ultra” chips on some generations. The only real difference is it going all the way down to skipping the “Pro” line. So, only the MacBook Air, low end MBP, and maybe the iPad Pro and Mac Mini get the M6.
The article says the 2nd.
Made up how? They'll do a refresh of lower end devices, but not the high core count versions.
It's the same thing as how the Mac Studio got an M4 Max refresh, but they didn't make an M4 Ultra so if you want the 28+ core CPU or 60+ core GPU, that's still using an M3 Ultra.
This time it'll be across all the Pro, Max, and Ultra versions, if you want those they'll stay at the previous generation for the M6 cycle.
Not that weird - Apple has a huge set of chips and hardware and software products. Putting every single thing on a fixed identical update cycle together won't always make sense.
Made up: “this one goes to 11”
Except that is not what's happening. The article clarifies something that is misleading if you interpret the headline in isolation: "high-end M6" means "the high-end variants of the M6 line", not "the entire M6 line".
Whether it matters for the consumer (who only sees released and announced end results) or not is irrelevant.
It can still be a very real, not made-up distinction, if the actual facts on the ground are that Apple designed an M6 line, but then scrapped that design and asked the team to create a new design with emphasis on AI-focused specs.
It's not the name that's important (the M7 could still come out as M6), is them skipping a design, or cpu "Tick-Tock model" step.
Why? The specs and benchmarks will show the differences, there’s no marketing around that.
It’s an amusing conspiracy theory, but I don’t get it.
Are you thinking Apple is leaking that there will be a long wait for much more expensive chips in order to… what?
Perhaps to support demand for the products with recent price hikes, and/or the upcoming Mac Studio with M5 Ultra, rather than have customers sit on the sidelines thinking they'll wait this generation out.
I am still skeptical that Apple intentionally leaked this because they normally are so tight-lipped, but there are reasons in favor of leaking this.
Well, I guess this is the silver lining to the price increases. I'd been thinking about an M5 128GB for local inference (eg DS4), probably off the table now given that it jumped $2k overnight. But I was on the fence about it for a long time given that even the M5 is not that good compared to even a 4090. It would have been good, but not "omg" good.
If they are pulling out all the stops to make the M7 more competitive.. guess I can wait for that?
I imagine they weren't planning for their plans to be leaked so publicly, as yes it will now mean some buyers will delay their purchases.
I was holding out for one until I decided to switch from an M1 Pro 16" MBP to an M5 Air 15" due to the expected price increase. I think many M1 Pro/Max generation people were waiting to upgrade this year.
Current MBPs are such a delight, I really don't want to think about a thinner MBP again, I just get shivers remembering the Ive butterfly keyboard models
I can see why people would want a more powerful machine but as someone who moves around a lot, the 16" MBP weight is a pain. The 14" MBP screen is not big enough.
any reason to go with the Pro versus Air if power is not a requirement? Screen quality / number of ports?
Definitely the screen. Huge difference going from the Pro to Air.
The extra ports are nice along with better speakers.
Get a 15" MBA...
I did!
The actual laptop I want is an Air 15" with 120hz OLED screen.
Isn't that switch basically a downgrade? You get some more single core performance and some weight savings, but also a worse (and smaller) screen, less multicore performance, less GPU performance, less video encoding performance and a smaller battery? I'm on an M2 Max myself, and glad they introduced a larger form factor Air, but it seems like a long way from an upgrade.
M5 is faster than the M1 Pro in ST, MT, GPU. Not sure about video encoding as it's something I rarely use. It's a smaller battery but overall a battery life improvement since my 5 year old M1 Pro only had 79% battery capacity left.
> This means there won't be a redesigned MBP this year since there won't be M6 Pro/Max chips.
They can release a redesigned MBP with the base M6 chip.
They can, it wouldn't make sense from a marketing and optics perspective.
They don't want to tell the world how the new redesigned MBP is the best laptop in the world but it's slower than the older MBPs.
The optics and marketing is already fucked, the MBP goes to M5 Max, the Mini has the M4, the Studio has M2 or M3, the iMac apparently has two different kinds of M4s, it's all fucked.
Air and Pros are the flagship products.
They released the M5 MacBook Pro last fall, but didn't release the M5 Pro and Max models until the spring.
It seems likely the MB Ultra will ship with the M5 Ultra.
Well this kind of sucks. I've been waiting for the M6 MBPs because they're rumored (strong rumors, though) to finally remove the notch that has been a historic self-own. But it sounds like I might as well wait longer for the M7 lineup. Or maybe get a Framework Pro instead.
The notch does not matter at all, you will forget it's there.
Well… your menu items will disappear behind it still in 2026. You would need an extra app for that…
But in terms of “noticing it” you are correct. You won’t pay attention after a day or two.
EDIT: this menu managing app will need permissios to make screen captures. So much for the privacy. Forgot to mention.
I think this is one of the things they're fixing in macOS 27 golden gate, took them long enough. https://www.reddit.com/r/MacOSBeta/comments/1u0lhyz/macos_27...
I'm not even convinced this is a bug. Menu icons proliferated over the last couple of years, this isn't a problem unless you enable more than you have room for.
I agree, I keep mine pretty minimal, most are basically there as ads to remind you they exist cough cough ms teams.
There’s so many annoying bugs in Mac OS (like the screwed up window management and alt-tab not working properly), that the notch seems like an odd complaint at this point. The OS is fighting the user constantly, and there’s not much we can do…
My menu items go around it...
Basically, you are saying "There are 4 lights"
I'm a huge fan of Star Trek, David Warner, and Patrick Stewart, and I'm really not. The notch isn't taking anything away, it's a way to give users more screen space than they could otherwise get. Normally the camera has to be above the screen, this was a way to get the menu bar off of the rest of the screen real estate and get you more usable pixels.
I was surprised to even see it mentioned after all these years, literally haven’t thought about it a single time since I got my first MBP that had it.
It’s a complete embarrassment. They added it for aesthetic alignment with the iPhone 13. And then the 14 removed the notch soon after. They’ve kept it for years since then. It has no functional purpose. It’s not there for face ID or because they couldn’t figure out how to do a hole punch camera.
It's there to give you more screen space...
Yes we got more vertical space by going into what used to be the bezel. But why is the notch so wide? Why not have a hole punch camera?
I agree. It was very annoying to me to spend the money (and on the nano matte one too) and still have that stupid notch. But it never makes any difference at all which is good news.
Except that it hides stuff...
If you have that much in your menubar you have bigger problems
Aerospace, Nvidia Sync, Dropbox, Espanso, Karabiner, displaylink manager, NTFS for mac, Maccy, Streamdeck, Tailscale, + 6 mac display/bat/wifi/search/cfg/date.
Elaborate on my problems... or help me fix them.
Blaming the user, nice to see apple traditions still respected.
It just doesn't seem like a problem. If you put too many icons somewhere than fit in that place, of course you're going to run out of space…
And if you have a hole in your screen you must make your OS aware of it and put the menus around it?
It's literally not an issue
And when it is, it's the user's fault. We're going in circles.
Same, have a very old MBP. Not sure what to do because I don’t want to wait a year and a half. That coupled with today’s price increases make it a tougher decision.
You can turn off the notch, I mean crop it out at least.
In the long run I truly believe local AI will win and Apple will be the world's most important AI company because of these chips. Imagine something like today's Opus running for free and in complete privacy on your local machine with a beautiful Apple UX on top. For most tasks for most people, that's a much better proposition than a frontier model in the cloud you have to pay for and send all your data to and that only works when you're online.
>In the long run I truly believe local AI will win
What do you mean by 'win'?
For a normal coder/person's use cases, yes. But AI companies are becoming more specialised in different fields and these tailored models will be leagues ahead in those niches.
The way I see it - Opus 4.8 xhigh can do any programming task with a programmer instructing it. If Apple releases local model together with a device that can run said model it would render OpenAI/Anthropic useless for vast majority of usecases.
And if a local mcahine can run something like Opus 4.8, who is to say that those "specialized" models would just not come at a later date, or even loading open models wouldn't be an option with something like M7-verified flag from huggingface that would make it extremely easy for any consumer to just play around.
But most private buyer and most business buyers don't get anything more than base models.
As of yet no indication that small models that can fit in 8gb/16gb can be fully relied upon?
There is built-in demand for local LLMs. An obvious example is law firms where using remote AI tools may be breaking privilege [1]. Any medical applications may likewise run into legal issues.
The problem is basically that we can't have nice things. AI chat logs themselves become another commodity to sell and to train on. We recently had a story about how Chinese firms are reselling Claude tokens [2]. The chat logs are a commodity here.
The only way to avoid this is to run LLMs locally. Even if you trust someone like Anthropic or Google, case law simply hasn't been established that the chat logs aren't discoverable.
Add to that that a sub-$5000 PC with a 5090 can already run a 31B model at reasonable inference speeds. Not amazing but good enough for many applications. Obviously that can't compete with Mythos but it doesn't have to. It also shows where the trend line is going for hardware. A $10k Nvidia GPU from 10 years ago now sells for scrap. What a consumer-level computer in 5 years can run locally will probably shock a lot of people.
[1]: https://www.williamsmullen.com/insights/news/legal-news/ai-t...
[2]: https://news.ycombinator.com/item?id=48667495
I would say local AI is very real. I use it but so many here am on other forums do so nowadays as well. This is the reason I just cannot fathom the valuations of the AI firms out there.
At this point can Apple profit from selling any contracts they have with TSMC?
If they make a deal with say google to delay their own chips, could they profit more than by selling their production?
Demand is so crazy idk if this would begin to make sense
Mac mini Pro line is doomed, they never made enough of it; skipped M5 Pro, now skipping M6 Pro, it is like 2014-2018 again. Now ordering a custom M4 Pro build take 3 months+ to ship with an increased price.
I was waiting for a MacBook Pro M6 Max and now I don’t know what to do, especially with the price increase I feel like I really screwed up not just getting an MBP M5 Max a month ago
Just buy now. DRAM prices are not coming down any time soon, and Apple may be forced to raise prices again.
The question is what will you be using if you don't.
Are you upgrading from a perfectly good machine? Then wait.
There’s not a lot of detail in the article but that doesn’t mean we have to link to a pay walled thing. https://www.macrumors.com/2026/06/25/2027-macs-m7-chips/
really stupid question, but why doesn't the US gov work with someone like apple to build a american fab with like 1 trillion dollars budget?
some kind of private-public partnership
sorry if thats already happening in some capacity, like i said - "stupid question"
because America can't compete. Build a fab in the US, labor unions, labor costs, regulations, land, energy, taxes, government, water, etc all make this not economical. Everything would cost twice as much and you'd rather buy the cheaper product and it'll be bankrupt. There were reasons why all the manufacturing went overseas to Asia. You're right, the demand right now is HUGE but it won't always be huge. At this point, we don't have the talent or the knowledge to do it well anyway which is why we needed TSMC and Samsung to bring employees over to train people. https://www.cppionline.org/wp-content/uploads/2017/07/The-De...
i wonder, what about the manhattan project? they were able to do that very secretly, i understand it was a different time.
but can the gov not just fast track this as a "national security" or something?
i think the usa should be the one who make 1nm or smaller chips on demand, even if it takes 5-10. years to do.
and yes i realize i might sound dumb here but i'm the one suffering from high hardware prices!!
they’re doing m7 on the intel 18a fab, which is exactly that
À fab takes a good decade to build, then another one to become profitable
Everyone seems to miss it but the article also says that M5 Ultra Mac Studio is coming out later this year. Yay!
I wonder how much the rumored 768GB RAM version will cost.
I am waiting till apple copies the "allocation" concept from high end car manufacturers. "Sure, buy the 25 iphones ans we will gladly put you on the waitlist."
This makes no sense. Apple doesn't need to generate artificial demand for their products. Apple doesn't need (or want) a perception of exclusivity.
It makes no sense as Hyper Cars are luxury status symbols and a store of wealth that appreciates in value.
A top of range Mac is a depreciating asset and looks exactly the same as the other models physically.
And it's not like Apple hasn't dabbled in the luxury space, with utterly predictable results. Anyone remember the 24 carat gold plated Apple Watch Series 0, sold at ultra-high-end luxury boutiques?
[dead]
Apple to skip high-end versions of M6 Mac chips...
I read it as the M6 being "high-end" in general, and Apple skipping the whole generation, which made no sense to me. But they are going to use the M6 at all, just not bother to create Max and Ultra versions of it.
So the big question: Is this an excuse to save on memory costs and delaying stuff like the M5 Ultra, M6 Max, etc until 2027 when memory prices come back down?
Is it safe to assume they will come down in 2027?
I think it is safe to assume they won't come down in 2027
Given that M6 will be on TSMC smaller 2nm node and the first smaller node size in 3-years, it seems like the oddest of all years for the high-end Macs to skip.
my 2 cents is that a new tech node is harder to produce variants on. it's easier to make new flavors of a mature tech node
2028 local and onprem agentic revolution. enterprises will pay for expensive mac line then
hyperscalers better all IPO in the next 8 quarters
I’m hoping they put a lot more silicon in the GPU so when I’m not running a local LLM the game I play runs smoother
Counter theory: the M6 is so good that they want to keep some oomph for the M7 line up.
My predictions:
1. NVidia aggressively segments the market on VRAM and will continue to do so. A 5090 with 32GB of RAM, ~21k CUDA cores and 1800GB/s of memory bandwidth is $3-4k. An RTX 6000 Pro with 96GB of RAM, ~24k CUDA cores and 1800GB/s memory bandwidth is ~$11k;
2. The 5090 won't be replaced until late 2028 or even 2029. There has been no mid-cycle refresh (eg 4080 Super vs 4080) and likely won't be either at all or for at least a year. If there is in a year, it basically confirms that the 6000 series won't be until 2028/2029. Also, the x090 never got a mid-cycle refresh so the current consumer high-end is staying that way for years;
3. The 6090 whenever it comes will still have 32GB of VRAM unless the memory market drastically changes;
4. Many have anticipated an M5 Max/Ultra refresh of the Mac Studio line in Q3. Given that Apple chose to hike the prices on Studios rather than discontinue them, I now think this isn't going to happen. We may not see a Studio refresh for up to 2 years. Apple has done this before with the Mac Pro;
5. M7 Max/Ultra will probably go to a memory bandwidth of 1.2-1.8TB/s vs the current tops of M3 Ultra, M4 Max and M5 Max of 600-900GB/s. This simply needs to go up to boost inference speed;
6. You'll also see the number of GPU cores go up. All of this will add up to an M7 Max being 50-80%+ of the performance of a 5090. That's huge given the shared memory architecture;
7. We may see the return of Apple using its massive cash pile for vendor-financing of an exclusive memory supply. This was one of Tim Apple's [sic] big innovations.
Local AI isn’t gonna help Apple, especially not with the rate hardware prices are increasing.
They need to pull out of this half assed bandwagon approach.
Local AI is going to help Apple, especially with a return to normal RAM pricing being an inevitability, even if it takes years.
They don't need to pull out of this approach.
How is it going to help them?
Do you really think the average Apple user will use it when there’s already better AI provided by OpenAI and Anthropic which don’t require advanced local hardware?
I was just countering your argument with an equally compelling counter-argument.
As for how it helps: we're not talking about this year's AI ecosystem, or even next year's. This rumor, assuming it's true, is talking about two chip generations into the future — and probably at least three or four chip generations before it's a mature AI platform. What will AI be doing for us in five years from now? How does Apple plan for that future? Will concerns of privacy increase or decrease in that time?
>What will AI be doing for us in five years from now?
I don’t know, but the majority of it will be running in data centers, not high end consumer grade workstations.
>Will concerns of privacy increase or decrease in that time?
People have been more concerned about privacy than ever, it hasn’t seemed to stop people from using cloud AI services.
The fact that Apple missed the starting gun on AI to the point that they’re using Googles also inferior Gemini, I’m not compelled these Apple AI chips have a consumer.
Anyone savvy enough to do their own AI hosting is more likely to use Nvdia boxes, etc. Apple workstations always been more successful in Art/Graphic design. That’s why they cut off their server hardware long ago.
>How does Apple plan for that future?
Ideally by making attractive simple products regular users can buy. The iPhone and MacBook Neo seem a step in that direction. AI vision and Apple Vision scream 90s pre-iMac Apple.
How about we release M5 Ultra first?
I'd even buy a M4 Ultra....
Bro just give me a new iPhone mini
You will have a massive phone and you will be happy.
No one buys the smaller phones because people care more about battery life than ergonomics
People care more about price. Price them decently and they will sell… which means they need to not be apple.
Yip but come on Apple it's time to solve that problem and sell some more phones. Each new iPhone is just a different color.
I will settle for a slightly larger iPhone that unfolds into a iPad mini.
Q
Come on Apple - just buy TSMC and fully kit out the RAM in all Mac Studios - you could even make (more of) a fortune selling the excess.
There's already reporting that M7 will be on Intel 18A - Apple's giving Nvidia and Broadcom and others more TSMC capacity back.
The URL above is wrong. At present it is https://www.bloomberg.com/news/articlehttps://www.bloomberg....
I guess it should be https://www.bloomberg.com/news/articles/2026-06-25/apple-to-...
Thanks for catching that - unfortunately I can't edit the submission URL, but I've emailed hn@ycombinator.com to see if the mods can fix it.
EDIT: gift link if paywalled (archive.is capture is truncated): https://www.bloomberg.com/news/articles/2026-06-25/apple-to-...
Fixed now. Thanks to you both!
[dead]
[flagged]
[dead]
[dead]
Apple is very late to the AI party. By the time M7 is shipped, Nvidia will announce 6090 and people will be buying used (3|4|5)090 GPUs to run local models at much better performance than heat throttled M7.
This a significant misunderstanding of which party it is Apple wants to attend.
And 6090 will have 48GB of RAM compared to something like an M7 Max that might have 192GB or an M7 Ultra that might have 768GB.
The M7 Max and M7 Ultra will likely prefill-bottlenecked at 100GB+ scale inference. Layered 6090s would not be.
Neural Accelerators in M5 are already 4x faster than M4 at prefill. With M7, especially if they focus on AI like this article claims, it likely will have excellent prefill compute.
What people? Are you seriously thinking the hundreds of millions of customers Apple have is going to be buying run-to-the-ground GPUs second hand and build local workstations for AI? Might as well ask them to self host email while you’re at it.
The difference between these two is that one of them is an unsolved research problem that we’ve all spent far too much time on, and the other is just running an LLM.
I would prefer a Studio if it does a decent enough job even if throttles a bit under load, way less power usage and noise than those GPUs plus the PC you need to put those in.
Yeah but you could heat your whole house!
If you're fine overpaying for a throttling computer, you could buy 40-series cards and underclock them to the same TDP of a Mac Studio.
You'd probably get faster prefill speeds, as well as better drivers for accelerated transcode and gaming applications.
RAM is a commodity and nvidia will be paying the same prices. The used market will reflect the cost of RAM. nvidia owns the top of the market but many of us don't need that.
same people who bought all mac mini for ai?
Theyve dropped the ball bigtime.
Apple isn't just transitioning to TSMC's 2nm node, they are also transitioning to a chiplet based design using TSMC's advanced packaging.
> What sets the A20 apart isn’t just the node shrink—it’s the revolution in packaging. Apple is transitioning to Wafer-Level Multi-Chip Module (WLCM) integration, meaning that RAM will no longer be situated beside the chip, but rather on the chip wafer itself, integrated alongside the CPU, GPU, and Neural Engine.
This shift eliminates the need for silicon interposers and substrates, thereby enhancing signal integrity, improving thermal dissipation, and facilitating faster memory access with lower latency. The benefits? Better multitasking, smoother AI processing (hello, Apple Intelligence), improved battery life, and potentially a smaller chip footprint—freeing up space for other components.
https://hwbusters.com/news/apples-a20-chip-ushers-in-a-new-e...
It's entirely possible that TSMC is ramping up more slowly than expected.
Do we have any explanations of what WLCM means that are more industry focused? I couldn't find anything that didn't look like blogspam. And that explanation of the DRAM being on the same wafer doesn't really make sense. For one, at that point there's no "multi chip" part if you're integrating more onto the same die rather than less.
And their explanation isn't really passing the smell test for me for other reasons, for instance the fact that DRAM processes are pretty radically different than bulk logic processes, which wouldn't really let you put it all on the same wafer, much less the same die. Even back in the day when you had eDRAM blocks (like the Xbox 360's eDRAM die), that was really a DRAM process with a bit of logic cells that wouldn't be competitive if they weren't sitting right next to the DRAM blocks.
I could be wrong here though, my examples are more than a bit long in the tooth.
The terms to search for are fan-out wafer level packaging (FOWLP) and TSMC InFO. The chiplets come from different wafers and are reconstituted into a molded plastic wafer, allowing multiple die side-by-side. Then multiple layers of wires are built on top, terminating in a BGA.
Ok, part of my confusion was that it was being presented in contrast to InFO-oS and InFO-PoP, but it appears to mostly be a modified version of InFO-PoP called InFO-M? Because Apple has been using InFO-PoP for almost a decade at this point, starting with the A10.
My astonishment at these manufacturing processes is never-ending.
You can start by reading up on TSMC's name for the tech (although there are many versions at TSMC and TSMC isn't the only company packaging chiplets and memory on top of a silicon interposer).
> CoWoS (Chip-on-Wafer-on-Substrate)
https://semiwiki.com/wikis/industry-wikis/cowos-chip-on-wafe...
It's a more advanced update from their older InFO tech.
As your article states, they're already using CoWoS; that's not a new technology for Apple to adopt.
Also, InFO-oS is a CoWoS technique.
InFO combines chiplets, but does not do so by bonding multiple chiplets directly to a silicon interposer as CoWoS does.
The article compares CoWoS to InFO.
> CoWoS uses a passive silicon interposer—etched with thousands of fine interconnects—that sits between the active dies and the package substrate. This interposer provides high-density connections between chiplets and supports memory integration, most notably High Bandwidth Memory (HBM) stacks.
It also mentions that CoWoS is the tech used to build Nvidia's high end AI accelerators.
> AI Accelerators: Nvidia H100, B100 (HBM3 via CoWoS)
Apple is taking a tech that has previously only been used in very high end enterprise applications and is using it to make consumer SOCs starting with this years iPhone.
You're calling it CoWoS, but also said "this shift eliminates the need for silicon interposers", but also said "CoWoS uses a passive silicon interposer"
Something's got to give here. I think it's your original article that's wrong, and it's poorly trying to describe InFO-M.
A kind request - please try to write HN replies without AI, but if you're going to, please at least edit out any "it's not X its Y" or "isn't just X, but also Y" AI tics. A lot of us come here to get away from talking to AIs all day.
The comment you're replying to doesn't sound like AI to me
It’s very obviously AI, several tells:
‘isn’t just / it’s also’ AI-ism should at least turn your AI radar on, then you get this weirdly formal structure that sounds like a trying-to-be-relatable press release:
“This shift eliminates A, thereby enhancing B, improving C, and facilitating D. The benefits? Better U, smoother V (hello, W), improved X, and potentially Y—freeing up Z.”
Question to self interjection, chipper ‘(hello, W)’ aside, topped off with a zero spaced emdash. 10000% AI, stylistically this isn’t text tuned to an HN audience, comments never sound like this. What’s funny is the paragraph it’s quoting has nearly the same style, the LLM probably picked that up.
Not trying to call anyone out, just pointing out the stylistic tells we should all be aware of.
Is there anything less interesting on this site than baseless claims that other people's posts are AI?
There are certainly more interesting things than you failing to defending your recent AI looking posts (c/w all the hallmarks).
So far the only thing I've seen useful out of apple intelligence is running parakeet natively and effectively... which should have been their very first feature... given it's been on phones for 10+ years.
As someone who wants to run effective llms locally for many things their other big benefit has been the unified memory studios for a small bit.
[flagged]
Please don't use HN primarily for battle over geopolitical disputes. The guidelines make it clear we're trying to avoid that here. https://news.ycombinator.com/newsguidelines.html