"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
> hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
I think everyone has known that inference is profitable overall for the major providers for a while (I'm sure not all user accounts, but in aggregate).
The major providers are unprofitable after you add in the training and research costs and other overheads.
This is why the open source models are such a threat if they become good enough - it will destroy the business model of just providing raw chat/intelligence.
Arguably that's also why a few of these providers are trying to diversify into whole-platform approaches rather than just relying on having their whole thing be a great model.
There has always been a lot of evidence that prices are only going down. The fact that you could already get DeepSeek or GLM inference from 20+ providers on openrouter.ai was already sufficient evidence that inference was profitable and being commoditized.
People that want to believe something different as a coping mechanism will not be swayed by this evidence either.
AI will end up being a commodity. That's also why you already see the first rounds of protectionism under the guise of national security. Obviously they do have a point but that point works against them just as easily. Anybody that integrates unreviewed model output into their workflow is asking for trouble.
Yesterday’s frontier is today’s commodity, so I don't think frontier AI is out of reach. With computing and solar energy costs continuing to drop, the current AI supply shortages are just a temporary blip. We’ll probably land right back in standard boom-and-bust cycles.
Either frontier AI will continue to get better steadily, until it's better than people at basically everything, and very weird things start to happen, or, it plateaus and the open weights models will eventually catch up.
There's a third possibility, which to me is also the most plausible. Models will keep getting better and better within certain domains, while never catching up with humans when it comes to other type of tasks. Like that elusive quality-without-a-name some have called "taste". Or knowing what UX is most intuitive to humans. Really, anything where we have to tap into our common humanity and our human experience in order to be of help to other humans and generally to life on earth. Which is an all-important skill, since the flowering of life on the earth is (or should be) the whole point of anything we do, after all.
We've spent a couple of millennia documenting that elusive thing called 'taste' in word, sound, image, lately in moving images. I'm pretty sure that even if you didn't have taste you could make a good simulation based on all that input.
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.
Those cycles have shortened considerably. Just look at the lifespan of media, integrated circuits, software, products. There have been so many big model releases just in the last three months I've lost count. We're now closer to weeks between steps.
Maybe if China stops releasing capable models the pace will slow down a little, but they have lots of incentives not to.
But it is not a constant point of "this % of people needs the frontier intelligence".
As thje models get better, the % of the pie gets smaller and smaller. So higher and higher cost of getting frontier model out gets divided onto smaller and smaller customer base as other models catch up
Yes, this is the crunchy bit and we're already well underway on that curve. I haven't touched a frontier model in months because I can do with smaller ones without wondering/worrying when I'll get banned next because what I'm doing is dual use. Qwen was the the first time I thought oh, ok, so this is 90% of what I need. Then DS4 made that 95% and with GLM 5.3 it's somewhere between 99% and 100%. Now, I'm not trying to solve any Nobel prize winning stuff here but if that's the kind of customer that will remain I don't think the frontier model providers are going to like the next six months or so.
I am actually doing robotics stuff and yes, GLM 5.3 can do it.
As for the manual feedback: I want that manual feedback, but if I let it rip it will do just fine all by itself, but then I have no clue what is going on under the hood.
At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
I have no clue about the state of robotics.
For software, I know that Opus 5.5 can handle more than GPT 5.6 Sol. I am willing to pay for the difference rather than use the cheaper DeepSeek Flash.
> At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
The prerequisite here is that you can trust it not to do something that gives criminals access to your bank account or allows users to consume an unlimited amount of services it costs you money to provide for free or bankrupts you through legal liability etc.
It's basically this:
"A computer can never be held accountable, therefore a computer must never make a management decision."
Notice what this is actually saying. It's not saying that the computer can't generate a given output. It's saying that if the computer makes a mistake and you don't bother to check, you're the one who has to eat it.
I'm getting plenty of mileage without that and anything with a motor in it requires that you do your utmost best to ensure it works within the design envelope.
My argument is that if safety and liability make the use of AI without fully understanding the implementation impossible, the model is arguably not good enough.
For serious engineering work not only physical designs, software also.
When we manually tested nuclear power plant controlling software, each test case was printed out on paper and tested and then signed by the tester. A lot of paper was produced and archived for each software release. (Archived for as long as the software was running in any nuclear power plant).
Everyone understood, that in a case of significant accident at the nuclear power plant, accident analysis would be done and if the root cause was an software error, this could have possible legal consequences for the developers and testers. (Of course nuclear power plants have many independent and redundant hardware controlling mechanisms, but incorrectly working controlling software could damage the power plant in a very costly way).
And over a increasingly small slice of time, as, due to the commodification of these things, the frontier businesses will see their customer base walk to the next one that does any better on their task.
Every time they throw someone out with a message about how this request is definitely against their safety guidelines they are clinching the deal on someone running something local or switching to a provider that just lets them get on with their day. Their own fearmongering is going to be part of their undoing.
Agreed, but I I wonder what fraction of tasks really require frontier performance, either in terms of sheer difficulty or where speed is a matter of necessity. For many jobs, it's increasingly a question of whether you'd rather have one filet mignon or 20 good cheeseburgers (all from the same cow).
The market for "frontier intelligence" will be too small to serve outside of certain government/military applications that exist in spite of market forces, in the same way that the market for small, portable nuclear reactors is exclusively limited to submarines and aircraft carriers. When I call a plumber, I don't need a frontier plumber, I just need someone who's baseline competent.
But why wouldn't you want (and pay) for the very best plumber? A frontier plumber will optimize the time their customers poop, pee, bathe, or do dishes 10x to 200x, and with their freed-up time,they'll outcompete those who don't pay for frontier plumbing!
Sir, I take the impending plumbing singularity very seriously, and wouldn't want anyone to be left behind. When plumbing hits recursive self-improvement, we may see toilets that shit themselves more efficiently than any man can, and finally attain ASI (Auto-Shitting Infrastructure).
I can see you do. I beseech you though to think about the consequences for other industries. For instance, the ratio of toilet paper:organic product will need to be maintained somehow and I suspect that if we follow your pathway to its logical conclusion that there will be a cellulose crisis unlike anything seen in the world since the USSR went bust.
The next move in my investment strategy is obvious though, invest to the hilt in PVC tubing and I thank you for this invaluable insight. Fortunately this forum has no visitors so we can rest secure in the fact that the pigs won't realize it is too late to avoid being slaughtered. Power to the bulls, after all it is their product more than anything that is responsible for the plumbing revolution.
what does this even mean, the commoditization... like a thing you can buy from a gas station? AI commoditization is the rock bottoming of pricing.
Usually we end up with cartels in such cases, see OPEC, mafias etc. But we have China throwing sticks into the wheels of the US providers that are attempting to make the cartel and "pacing the frontier"
> Usually we end up with cartels in such cases, see OPEC, mafias etc.
Whether a cartel forms is independent of whether a product is a commodity. To the extent that commodity markets are more likely to form a cartel, it's only because it's harder for them to form a monopoly.
The general problem is that if your legal system allows competitors to buy each other then they have the incentive to do that instead of competing. This is why there are supposed to be antitrust laws, and why we get problems whenever they're not actually enforced. But in a typical commodity market, even buying the competition doesn't quite work because someone new can always enter the market. Congratulations, you've cornered the market on landscaping services... until some unemployed fellow who already owns a lawnmower and a truck starts advertising their services.
OPEC is kind of a special case because the product is entirely fungible but it's also finite and consumed by usage. Landscaping services aren't meaningfully finite, if demand increased then supply could match it under any plausible scenario. Copper is finite but it isn't destroyed by usage, so if someone had a lock on copper mining then people could still get it by recycling existing products.
Whereas oil is finite but then the end customer burns it. If Saudi Arabia cuts production, Germany can't make up for it by extracting more domestically because they haven't got enough in the ground, but they also can't capture the exhaust from their cars or power plants and turn it back into petroleum because the product is really energy and it takes more energy to do that than you get from burning it to begin with. So if the largest exporters get together, they control enough of the market to control prices and there is no real way for anyone else to enter the market with enough production capacity to change that.
Until you realize that the product is really energy and you can get out from under their thumb by switching to electric cars and generating electricity without burning fossil fuels.
Not too long in the future, I fully expect OpenAI to rescucitate the API copyright aspects of Google v Oracle to pad its revenue in OpenAI vs Everyone profitable
>Usually we end up with cartels in such cases, see OPEC, mafias etc.
No we don’t. That’s quite rare.
Look at the collapse of the solar market around 2010 as the price collapsed.
Look at the collapse of the steel industry in the US as outsourcing to Asia took over in the 70s-80s.
Look at the entire computing boom from 1990-2020. Memory and storage followed such a steady line down and to the right that people are struggling to operate compute businesses in a normal market.
E-bikes, scooters, electric cars, lithium ion batteries in general. Most food.
The list of these commoditized things that started out as areas of innovation and became just the marginal cost of production is enormous.
I’ve never heard anyone say raw inference was unprofitable. It wouldn’t even make sense, I can’t see a rug pull for renting out commodity GPUs.
People are saying _training + inference_ is unprofitable (I don’t have a dog in the race and don’t really care either way). That’s a lot of GPUs and salaries that aren’t generating revenue at any moment.
Personally, I would guess Anthropic and OAI are profitable. I just suspect that they’re a pretty normal level of profitable and not the absurd, near-alchemical “can turn dirt into gold” level of profitable their valuation seems to have priced in.
To add on, training + a highly paid American salary load + inference is unprofitable, if and especially if you have deepseek dropping weights and engines for anyone to run.
H100s are renting out at a few dollars per hour. An awful lot of people could run them if they wanted.
OpenRouter's DeepSeek provider is basically this. Instead of renting H100's for dollars per hour, you pay someone that arbitrages the $/hr to $/token conversion because you can't actually saturate an H100 for hours. It's still effectively the same as running it yourself because of the ability to change providers to someone else arbitraging the same thing, and that there's always the escape hatch of switching to actually renting the H100s yourself.
It's a commodity, and research into how to make it good will be done in the open. What this means is that we should doubt the justifications offered by magnificent-N-whatever companies slurping up all investment dollars to take over the world by being the first to get "general" "super" intelligence. No, there won't be a runaway improvement. Sadly, AI is just a clone of our inner world. Getting interesting information into it is still going to be just as slow as it is in the regular world. You have to build, tinker, experiment, get data, get knowledge, etc. The only thing getting sped up is how much time needs to be allocated to thinking and translating.
Isn't the modern economy already a runaway improvement? First writing and agriculture, the stone age became copper became bronze became iron became steel, and one Revolution after another - agricultural 1 and 2, industrial, information.
Looks like AI will be another revolution, and eventually, probably the largest.
There may be different thesis out there, but the fact of prices going down definitely doesn't help the case that the major AI labs are are overvalued and need to vastly increase their revenue to stay afloat amidst their contractual obligations.
I honestly don't think there are too many people arguing "AI is actually too expensive to take my job". "AI being too much of a commodity to make meaningful profits" could very well be the case though.
I have seen the former argued many times, though mostly not on HN. It is a cope meme I’ve seen mostly on Twitter and Bluesky, but some on HN and the Elixir forum as well.
They are literally hoping Anthropic & OpenAI go bankrupt (totally possible), and everyone goes back to using their meat brains for work (never happening).
In this context it just doesn’t matter if these companies are profitable after amortizing training cost, because deployment of open weight models that already exist will continue to disrupt economies for at least the next decade if all research ended tomorrow.
> I think everyone has known that inference is profitable overall for the major providers
> The major providers are unprofitable after you add in the training and research costs and other overheads
It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.
It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.
Exactly. And you can't just stop training models, because your competitors won't, and customers are switching providers on a daily basis (most likely an exaggeration).
Hence "we need to slow down progress for safety" [narrator: it had nothing to do with safety]. The industry is begging for the regulation so that there is a coordinated slow down and consequently move towards profitability.
> coordinated slow down and consequently move towards profitability
100%
Amodei has been promising the "end of the exponential" all year, when they can shift some compute away from cash-burning research and training toward cash-generating inference.
"Pacing the frontier" is how you collude with your competitors to all decelerate at the same time because safety.
I saw last week that Anthropic is currently cash-flow positive with 75% of compute going to research and training. If they can flip a switch and move from 3:1 to 2:1 or 1:1 training:inference, they will be insanely profitable.
As long as they can avoid a price war.
>> calculated on a per model lifetime basis
True only if you never decelerate R&D.
But "pacing the frontier" means every early model doesn't need its own profitable P&L.
Early models can be big money losers while helping build the next generation.
If the industry can coordinate to all decelerate at the same time, then they'll flip from incinerating billions to minting billions, and all that capex will suddenly look completely sustainable.
Unless DeepSeek suddenly gives it all away for free, which is why the big three need government regulation to prevent foreign models from busting the cartel.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
It doesn't just matter that inference is profitable - cash from profitable inference needs to outweigh cash burned by training. If training costs even $1 more than your profits from inference on a model, you lost money on the whole thing.
if i can run an existing open model because its good enough for what most people need, I can just spin up an inference business, and put my spend either on trying to optimize it further or on customer experience
... and not worry about training cost at all.
a lot of the cloud businesses of the last decades have relied heavily on open source code, again skipping most of the cost to make the product/service being sold.
Nobody meets expectations of a 2 trillion dollar IPO selling a fungible commodity like tokens for old models.
And that's the rub - it doesn't matter if inference is profitable, the US labs have promised the whole kitchen sink, and taken serious investment on that pitch. They won't make it back in a cut-throat market for inference. They have to continue pitching the idea that only their models are worth using, for quality/safety/performance/etc reasons.
Which means continuing to develop and release new models. At huge expense.
Side note - while deepseek seems to be doing fine, I'll note that the emphasis on cost effectiveness is very much not a thing for the US labs right now... and it was for non-us labs. I'd be quite interested in knowing if anthropic is pulling in the same returns, or what the margin on their subscription plans actually is.
---
I'm already seeing this directly at work - lots of devs still happily use claude for development, but the products we're making aren't. They'll use kimi, glm, gpt-oss, etc... basically - if cloudflare is hosting it, we're exploring it (mainly for contractual/security reasons): https://developers.cloudflare.com/workers-ai/models/.
Because we are price sensitive, and we already have to do all the work of creating very robust evaluation frameworks for our products, so we might as well choose the cheapest model that meets our standards.
whiiiiich is why we're seeing things like Haiku and gpt-luna priced down near the open models at ~.10/mm in, ~.50/mm out. They've been forced to cut prices to keep commercial customers on their cheaper models instead of moving away.
This will continue. If inference is profitable and easy, prices will continue to erode. Go browse openrouter, plenty of providers happy to undercut the competition. Bigger shops with auditing requirements will avoid them, but smaller ones won't.
You know, it's not just cost-effectiveness, it's also iteration time and speed. DeepSeek V4.1 Flash is astoundingly fun to use because it's so quick. Tens of times faster than the standard Frontier models. With, in my experience, practically the same results. Because most of the trouble is not the intelligence of the model. It's the feedback from the world. The quicker you can get feedback from the world, the better your process is, irrespective of raw intelligence metrics of the AI you're using.
I 100% agree and am a big fan of DS V4.1 Flash in general for this reason, but I'll add my own caveat: Sol 6.1 isn't far behind on completion time for tasks because it's so damn efficient. I regularly get ~300 tokens/sec with DeepSeek Flash, but it spends the majority of that time thinking and executing slow commands one by one, whereas Sol thinks less and is much smarter about the commands it runs, resulting in far fewer turns to get the same thing done. For highly agentic tasks, DS might only be ~25-50% faster, not 500% as the TPS may suggest.
(But to be clear: 25-50% faster is still a big win in my book.)
+1; using Sol 6.1 in Ultrafast is noticeably faster than DS 4.1 Flash in real world tasks where marginal intelligence and token efficiency matters. Yes it costs more, but that’s a function of how valuable is the thing you are iterating on.
> Which means continuing to develop and release new models. At huge expense.
I guess my question is when do we say a model is "good enough"? For my use cases Opus 5.5 feels as if it is exactly at that point. And at that point of course one of the main factors I will evaluate against is price. So far I am happy with my (heavily subsidized) subscription, but if the time comes, I'll happily switch the inference API any of my agents are calling. So far I have no big lock in on any providers.
Because again - I think lots of businesses are already taking a good look at models well off the "cutting edge" because they're good enough for what that business is trying to do. They're already there.
If you're doing categorization, tagging, form-filling, customer support routing, etc... you don't need anywhere close to cutting edge expensive models.
Hell - Jev being so popular is basically exactly that group going "what's the cheapest way we can make this decision".
Meanwhile, there are plenty of spaces where you still want real model improvement. But a lot of traditional business was basically process and paper pushing. You don't need a Phd level model for that, you need an intern level model.
---
But that's the trap for the large providers - intern level models are cheap and easily available with current open weights. It's going to be a very tight market, and margins will go down to essentially the floor (especially once we exit the current constrained-compute market).
Somewhat ironically - the technology itself makes switching costs basically zero. It's not just that there's no big lock in, AI is like an anti-moat. It actively fills in and paves over whatever moat you might have for price conscious customers.
I think we share the same view here, which I think is most widely discussed as "AI capability overhang". Like you also said, a large, growing amount (digital) tasks can already be completed reliably with existing models. For those - this was my point - the models are competing on price, speed, reliability, etc.
I also agree that more capable models are still desirable in plenty of spaces. I just see that there is so much lag in the diffusion of existing models, which, again, are already sufficiently capable.
When it solves your problem. I don’t think it makes sense to globally qualify a model as “good enough” any more than it does with programming languages. Python is good enough for a web app, but not for an OS kernel.
To your point, I think we’re past the hump on the Pareto distribution. The models are good enough for most things, and the rate they’re improving at the things they’re bad at is dropping. The odds a new model is good at something the old one was bad it is okay, the odds it’s something you care about is dropping. It makes a lot of sense to discriminate on price rather than features in that environment.
I know this gets a lot of flak because it ended up being obviously incorrect for consumer markets.
But the under-reported corellary of this is that a TON of industries essentially operate on hardware right out of that time period (or slightly after, circa 2000s)
banking
Insurance
Healthcare
Gov
retail
Transportation
etc...
Hell, I was writing code to run on IBMs Z/arch as recently as 2021 for banks with mainframes deployed around 2004. They were just then finally considering an upgrade to the later models.
The Chinese labs are just arms of the Chinese government. They have no obligation to be profitable or please (private) investors. If they weaken/collapse American models, then that is the ROI to the lab owners.
This is a pretty strange take on an article explicitly pointing out the profitability of chinese models.
Further, even if this is your take (and I won't touch the geopolitics here, maybe you're right, maybe you're not regarding ulterior motives) - go use non chinese open models that are also good.
China has sandbox capitalism. In the sandbox it looks and feels like capitalism, but the sandbox itself exists in a communist framework, and the state still controls all the inputs, outputs, and can intervene at any point as they please.
In organic capitalism you need to compel arrays of discrete private entities to invest in your idea. In sandbox capitalism, the money can just appear in any amount at the behest of a single controlling individual.
You can see it plainly on the Solar PV market. The likes of Trina, JA, Jinko, etc. all compete aggressively, leading to everyone running at a loss in order to corner the market, and then after a few years the government steps in and orders all of them to raise prices. Happened again at the beginning of this year, raising cost of PV Modules 30-40% for oversees consumers.
Contrast that to US being happy with everyone eating each other alive until all but one company is left standing and can now dictate the prices and enshittify the product to pay back their VC loans.
Isn’t “sandbox capitalism” - real capitalism and what the us is trending towards is the “Wild West” capitalism with monopolies, trusts, and all the other anti competitive things that arise in the free market if there is no external force that tries to keep the market “fair”?
I mean this is not the case for either, but the systems are setup to drift in these directions - a market needs some sort of regulation to keep things sane, otherwise you end up with standard oil and the like.
You want the people with guns controlling things to be at least nominally interested in “the people” and produce more than just “return on investment”.
Not saying that China is like this or that US is bad, but saying that China is bad because it is “controlled capitalism” is kinda missing the point of what it should be in the best case.
I don't think its that hard to shoehorn current US into those sentences too, maybe switch word 'communism' with 'kleptocracy'.
Less intervening and micro management but oh boy say something not nice about government and potus and suddenly you are a new enemy of the state and 3-letter agencies work day and night to bring you down for good. Or in reverse - put a public bribe to potus and his clan and enjoy some favors for few weeks.
2 years ago I would agree more strongly with you, but we are in new era from now on.
Yes. I've always suspected inference is profitable.
But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
For most cases it's probably ok, often times answers are general or basically a web search anyways, but if my Agent started blasting outdated syntax it would suddenly be way worse for coding for example.
Relying on an LLM's baked-in syntax knowledge is suboptimal, compared to leveraging reasonable context affordances (skills, examples, tool calls, MCP servers, search) to ensure adherence to precise APIs and language specs. Don't get me wrong; a model specifically trained on your desired framework is obviously preferable for more consistent generation of idiomatic code -- but relying on that in a large general-purpose model is a questionable proposition IMHO.
They need to be retrained often is my experience for general usage.
I have SO many cases, for example when shopping, when they make assumptions based on their cutoff date, and they need to redo everything based on lots of searches.
Sure, for many use cases old models will be useful but for general usage a model that's 5 years old will probably be garbage for most people, unless they prompt it extensively, which the general public won't do.
> But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
Just wait until what Ilya Sutskever has for Safe Super Intelligence (SSI) to self update the weights without retraining them to bring training costs down close to $0.
> I think everyone has known that inference is profitable overall for the major providers for a while
A lot of AI haters like Ed Zitron and their readers have refused to update on this. I don't know if any of them have yet, but it'll take a year or two for the idea to fully filter out of their podcasts anyway.
Seems a bit of a disingenuous framing. Most of the arguments I see are some mix of
1. Their figures have been pretty dodgy or excluded training costs so why is it trustworthy this time?
2. API inference charged at a profitable level rather than heavily subsidised subscriptions means everyone tightens up their usage + governance Which we very much saw with the rise of openrouter et al.
Personally I think there's a lot of the bait and switch of 2 when people talk about profitability of the 10-20% of API usage for Anthropic while also wanting to lump in subsidised chat users as if they're profitable too.
I think we know if it was this clear cut then the behaviours of companies would be wildly different and more consistent with profitability.
> I think everyone has known that inference is profitable overall for the major providers for a while
Unfortunately while in the tech industry some might know that, the general public are still being misled (especially by certain folks who’ve made a living off downplaying AI) to believe inference itself is unprofitable.
> The major providers are unprofitable after you add in the training and research costs and other overheads.
This is correct for now, but I recently read that 90% of Anthropic's code is written by Claude Code itself. So, if an AI is using its previous version to train the next version, we're not far from the day where these models will take up the decision making process too. For eg: Decisions API by OpenAI or Jev is based exactly on this. What if a model like Jev sits behind Opus 5.5 and guides it to take the next step? Wooh.
Probabilistic predictions right now are at 99% for frontier models. They will never be 100%, but if humans rate themselves 99.9% in decision making, that benchmark can be broken.
As far as I know they haven't been doing any mass layoffs.
So if Claude Code is writing 90% of the code, then why are they keeping probably the most expensive engineers in the world on the payroll?
The machine writing all the code seems to not be enough, even for the company with access to the best models. What's the alternative to explain this discrepancy?
In this game you "win" only if you arrive first, so keeping the same headcount to run faster is a valid option for Anthropic. In this context "winning" means mainly that we lose our jobs
Honestly that's an interesting statement because in my experience Claude writes around 5-10x more code than a reasonable person would to accomplish the same thing. If you include comments then that number goes way up as well.
It’s not training and research that kills you here, it’s your own investors ROI expectations that murder you cold when faced with Open Source commoditizing you, from a country that spent 5% of the sum you spent to get to the same position and then gives the recipe away to everyone.
As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?
Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?
1. Inference providers are largely a commodity business. While demand outstrips supply, there are many providers of that supply, which does somewhat constrain how much leverage a provider has.
2. The models are not interchangeable as of yet (though this seems to be changing). A lot of people want the latest Anthropic or OpenAI model; as such, the labs still have considerable leverage.
3. A lot of the inference providers have used their leverage by doing equity deals with the labs, agreements to have some of the deal revenue spent on compute, etc.
GPU prices are sitting well above MSRP, even for older generation cards. Datacenter growth is also being pushed pretty hard by most of the cloud providers. More anecdotally, the model providers I use have been "throttling" my requests (usually by bumping them down to a lower model version) more frequently. Granted, that could be for profit reasons, but I think it all generally points to a lack of capacity.
>Is there a wait to get an inference provider or interruptions in availability?
I think that the inference providers and model vendors are mostly handling it behind the scenes with more aggressive quantization, increasing API prices, deprecating more generous monthly tiers, etc.
Yeah I think one of the less considered costs when people talk about this is the human cost of training. We all talk about the billions spent on compute, but I don't think anyone's got good numbers on how much RLHF is costing but it's got to be pretty large.
The only moat is in datacenters. The software has no moat, since 1) it's already trained on stolen data, and 2) someone with access to AI will be able to reverse engineer a proprietary implementation eventually, the same AI is being used for developing AI.
That's why so many talks about regulations. Regulations means: ban outside competition, just like US bans Chinese EVs today.
Well, as that means there's lots of money to be made lowering costs for research and training! What tools have we used in the past to lower those costs? Automation.. Hm, what if we had AI completely automate AI research? Uh oh, we just rationally found outselves in a trajectory where we lose control of our own trajectory because of economic incentives.
It is wild how fast we're hurtling towards this outcome, lmao.
I doubt that, I got downvoted several times pointing out inferencing is insanely profitable.
Even take account of R&D cost it is still profitable, because you do not actual retrain the base model every month, only distill and post train from the same foundation.
heck, I think even including the giant cost of the top tier researcher salaries, they are still profitable.
then, how are they still losing money? just read what they are actually spend the most budget: always the hardware. those companies will never profitable, because they will never stop spending money to expand their computational prowess.
If the frontier labs are struggling under the training costs and the leading open source models are making their progress by distilling from the frontier labs it sounds a bit like there’s something wrong with the economics here or that progress is going to slow quite a bit without the amount of investment.
I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.
I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).
I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.
A lot of people have already replied to this point but I think I want to distill the problems a little more directly:
- If inference is this profitable, is it profitable enough to offset the enormous training costs, bot those already incurred and those in the future?
- If inference is profitable, can it cover other capital and operational expenditures the hyperscalers and neoclouds are committing to, on the timelines they're committing to, to scale out inference?
- If inference is profitable we can measure if the price/earnings ratio of current valuations is sane. Is it?
- If inference is profitable, is it - given the answers to the questions above - profitable enough to make the risks of lenders of the huge amount of debt companies are taking on to scale, rational? It's one thing to say as a hyperscaler "we can make this pay", it's another to say "I'm going to lend them the money to do that and I'm going to get my money back on the timescales we agreed, and with an ROI to boot".
- Finally, if inference is profitable why is Nvidia specifically lending money to companies to buy Nvidia cards to do that inference, and then hitting the credit default swap markets to cover the risk of never getting that money back, and doing so at ever increasing cost, when they could just go build a profitable inference business themselves?
I agree seeing the numbers is good. It does suggest profitability in some segment of the market. However I still have a lot of questions.
Most of your questions are things that are relevant to everybody due to a possible economic crash. And you are right to point this out.
But for me as an AI user those aspects are not really relevant in my decisions. If an AI bubble bursts providers will still want to sell me those tokens at that price and yes development will slow down for a time which does change not a lot for me.
Inference is very profitible when you can deal with the batch sizes Anthropic, Deepseek and OpenAI are dealing with, renting GPUs for inference is even more profitible.
The US is dealing with different tax code where buying GPUs function as as great way to offset taxes, and having boatloads of debt is rewarded in many cases. I think there's lots more (crooked) reasons why inference is even more profitible in the US.
100% the right questions. The implied thought process of so many on this thread: Inference profitable >> Frontier Labs' are profitable overall. Sigh...
I can't read the original Chinese text, but if it's revenue, then you still can't say anything about profitability. Housing and powering one GPU should be way less than $38 per hour, so there should be a profit on inference. If that profit can cover training new models we don't know, but it would appear to not be the case.
For the past year or two I think it has generally been agreed that inference is profitable, but the LLM business as a whole is not.
I think what he said was that 10 months of inference paid for the hardware it runs on (which may have a lifetime of a few years). I don't think he mentioned training/development cost.
Consumer get the cheaper energy, while industry normally get the twice as expensive energy.
But for AI/Datacenters, there is a exception and they get the cheaper consumer level energy.
You need to take in account that the way China taxes is not the same as here in the West. And a lot more is put on the local governments regarding generating their own revenue (why there was so much building in China because those projects funded local government). A bit of a side topic lol. Its not so simple as they are making a loss on something because how its structured.
AI gets the cheaper tariffs then the rest of the industry, that is a fact. They simply have less energy profits on the AI energy. Now if you consider that subsidized or not, ... in both cases China is still making a profit.
What is overlooked is the massive amount of energy generation that China has and still keeps expanding. And that is a direct reflection upon why the prices for industry and consumers are so cheap. This buildout ironically had nothing to do with AI but was more about lower energy = cheaper industry production ... and a lot of buildout was also to build more independence from energy imports. They are not just throwing solar and wind turbines around like candy, the amount of reactors constructed and still under construction, ...
> China currently has 36 to 38 nuclear reactors under construction, maintaining the largest nuclear buildout in the world and accounting for roughly half of all global nuclear plant construction
So yea ... The AI in the west have a issue because it really does not matter if you cut China from Nvidia GPUs or advanced nodes, they compensate on the power level.
Edit: And a little extra, the irony is here in the West building nuclear is expensive as hell because we barely do one or two at the time. With a lot of lost expertise. China doing tons of them non-stop cuts down on the costs a lot.
I'm not sure how much you pay in USA. But I live in Beijing, where the residential electricity price is about 0.5 RMB per kWh, roughly 0.0749 USD.
China's electricity pricing involves cross-subsidies. Residential and agricultural users pay the lowest rates, which are generally below the cost of supply. Commercial and industrial users pay more and contribute the most of grid revenue.
Prices vary by region: commercial electricity can approach 1 RMB (0.15USD) in Beijing, but the national average is about 0.67 RMB (0.1USD), while industrial electricity averages around 0.63 RMB nationally.
At similar prices and 1/4th or whatever the average salary, meaning the average person has to use a lot less energy, or spend a larger % of their budget on it
It's not particularly surprising. China wants people to use electricity. Germany wants people to use gas/oil. The rate of electrification in respective country reflects that preference.
I don't think that the steering effect (away from electricity) on private people is intended. It's intended to keep industry in the country. Makes sense, but still, it feels unfair as a private person.
Would that be due to the size of the models from DeepSeek or perhaps only for their distilled models? If it's for general purpose models that would crash the valuation of Anthropic and OpenAI, both who are struggling with profitabilitet to say the least.
Who said they need to distill anything? Deepseek is the most popular chinese model getting trillions of input tokens a day that have no ZDR availability. They specifically tell you that part of the deal is that they get your data to use however they want (atleast they're honest).
This idea that Deepseek needed to distil their way to success is nonsense. Its a great narrative for OpenAI and Anthropic to spread because Americans have been brainwashed for two decades into dealing with their own inadequacies by blaming things on scary asians stealing things.
In reality China is outworking and lapping most American companies. They produce 10 EEs to every 1 America produces. They are simply harder workers.
The West is cooked because we're too lazy to work hard and we'd rather make up stories about boogeyman who are stealing from us. The only thing we're good at now is killing people who economically challenge us.
DeepSeek's API used to (does still?) route your requests directly to Claude Fable[1]. Those were provably Claude Fable's responses being returned by the DeepSeek API. This was months after Anthropic accused them of distillation of Claude at scale[2].
Them provably distilling after being accused of distilling strongly indicates that... that's what they do. To spin a story of them not doing that (because why would they, with their superior talent and all) doesn't seem very reality conform.
The models are impressive enough as is (considering resource constraints and difference in CapEx) despite being heavily distilled from western frontier models. No need to pretend that they're not.
You really cited a tech politics person, and Anthropic... The poltical AI people are the exact type of cope distributors I complained about above.
I think the more reasonable explanation is this is totally made up. Notice how they use examples concerning "cyber and bio" notice how the examples are politically loaded subjects that are hyper relevant to proposed AI dangers and why we need to protect western AI companies with anti-competitive regulations?
Oh no!!! The scary asians are distilling our models, but they only want the scary things!!! They want to hurt you!! We better stop the scary asians!!!
> I think the more reasonable explanation is this is totally made up.
Based on what? None of the accused companies have ever denied Anthropic's accusations. And here you are confidently claiming it all to be made it. How much of a reputational hit to one's honesty do you think it carries to completely fabricate someone doing something, when they can just publicly disagree and disprove you? First thing DeepSeek, Moonshot, etc. would do if they hadn't would be to make a press statement calling out Anthropic for their lies.
DeepSeek routing to Fable was corroborated by many independent sources... since... everyone was able to just try it out and see it for themselves.
> Oh no!!! The scary asians are distilling our models, but they only want the scary things!!!
The straw man you're mocking doesn't exist. Non-Chinese labs are also distilling frontier models of their competitors. Elon testified in court that Grok was trained on ChatGPT outputs. And why would they not? It produces objectively better models.
When did I say everything is a grand conspiracy? This whole thing where centrists normies just call anyone who believes something outside their center comfort zone a conspiracy theorist is mega obnoxious.
You need better media literacy.
You cant just make up a reality where none of these companies have denied distillation.
The scary asians are going to get you, watch out! Your government is going to eventually tell you that we need kill a bunch of them too because they're so scary. Probably soonish. You're going to fall for it because you believe anything you're told.
I think it’s pretty clear from the cherry picked numbers at Anthropic.
Demand is the issue. Demand only spikes with a sota model release then in drops back down to unprofitable levels when factoring in training a new model and build out.
So your stuck in an unprofitable cycle of losing revenue when your not releasing sotas but losing profit when you are.
This goes back to googles original memo “there is no moat”.
Where in you can’t stop training sotas because you’ll lose customers to the next inference model above you instantly.
Google has seemingly defeated this by finding models cheap and good enough for their existing platforms resulting in a moat of distribution. Something OpenAI and Anthropic seems to lack.
I think you have to take this on a case by case basis - there is no reason to suppose that the training costs and serving costs are the same for all companies for all models and for all price plans. Some may be profitable, others not.
DeepSeek's models are relatively small, and training costs will be correspondingly lower than for larger models. All you have here is a revenue number - no mention of (training + serving) cost, so while they may well be profitable you can't determine that from this number alone (if it can be trusted).
DeepSeek (investor meeting leak) have previously said that they price their models at a point where the GPU is paid for after 10 months, so something doesn't seem to add up here. If they were making $38/hr per card 24x7, then in 10 months they'd be making $270K per card, while the cards (Huawei?) only cost around $20K.
That turns out to only be a small part of the cost.
Say a $30,000 GPU has an operational lifetime of 3 years, and draws 1000W. If it is running 24x7 then over those 3 years it will consume 26,280 kWh of electricity, at a cost of say $0.10 per kWh, giving a lifetime electricity cost of $2600, or less than 10% of the cost of the GPU itself.
So, to a first approximation the cost of serving a model is the amortized GPU cost.
Now that the unit economics are laid out in the open, What stops any of the companies or even startups from anywhere in the world with some investment $ and competence from spinning up and offering their own version of the inference engines, undercutting any moat that the current leaders have?
I could even see F1000 companies having their own backstop intelligence for most of the tasks and only use frontier where it is absolutely needed or as much as it is under their IT budget.
Won't it be race to the bottom when they compete on cost? Is regulatory capture the only moat left for the market leaders now?
Nothing but hardware access, which is getting harder for the average person but for a giant corporation I see no reason they can’t compete now at the local model level
But haven't the Chinese (and Deepseek specifically) put a lot of emphasis on running models efficiently (because they've been locked out of being able to obtain certain GPUs by the US)? Whereas the US frontier companies seemed to get huge cash inflows and didn't need to worry about efficiency as much and put more emphasis on frontier performance. So how do you know that this $38/card/hr is about the same that the US frontier companies are making?
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
> My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy
For such workflows it barely matters which of a dozen models gets used.
Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
> Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
I use frontier models every day and night, but not sure if the described approach is possible with any of them right now. Models still make stupid mistakes and people who do not read the code produced are bound for surprising discoveries in their codebase. Not trying to downplay the effectiveness of LLM-assisted coding and all that, but it's hard to imagine how much stupid shit i would've committed if I was not reading their output.
Yeah, I have extensive collection of prompts for reviews and usually run 3-5 (automated) review takes on large sets of changes before doing eyeball reviews. Auto reviews indeed find many obvious bugs, but they usually slip through things that are less
obviously stupid but are bad conceptually, for example, using persistence / queue / repository in a place where set of pure functions would be enough; or vice-versa, not persisting the state that must be persisted for reasons, etc.
Another problem is that review changes can easily introduce new issues, so my review prompts go like that: if you find outright shit, fix it right away, but if you have slightest concern, just write your ideas and suggestions in the review and let me decide then. I try from time to time to omit the "slightest concern" part and make agent do all the shit it wants, and usually end up reverting those changes.
But as the sibling comment said, not all code needs quality and indeed I skip some code too, like user
playwright journeys -- I only need one auto review pass to verify they indeed test things according to a doc, and I'm fine not reading them. Not all code like that though, some stuff would wreck havoc later if its not looked at.
If you can test well enough, both manually and automated, then it's okay if there's a bunch of stupid mistakes in the code, as long as the end result works right. Not unlike biology.
> I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
I was doing the same (along with GLM 5.3 Flash) for months, and then gave Claude Sonnet 5.5 at medium effort a try. The difference was like night and day. Where both DeepSeek 4.1 Flash and GLM 5.3 Flash kept making mistakes and would have to do a bunch of iterations to find and fix them, Claude Sonnet 5.5 just did it right on the first try.
In my experience, while DeepSeek 4.1 Flash and GLM 5.3 Flash are a lot cheaper per token, they're just as or maybe even more expensive when you factor in all the extra tokens that are need to troubleshoot and fix their mistakes -- and even more expensive if I factor in my own time wasted doing the same, not to mention the frustration of having to deal with that.
DeepSeek 4.1 Flash also hallucinates a lot.[1]
I'm hoping all this improves with time, and the gap between the competent frontier models and open weight models narrows.
Same for me. The cheap Claude subscription is just so much cheaper. Never had to buy more deepseek tokens for months (August), because Sonnet 5.5 is good enough
What is that CPM? My CPM in ChatGPT is lower than in Google. CPM is a bad metric, how much does 1 conversion cost? You can charge me $100 per ad all day, as long as every showing results in a $1000 sale.
There's a trillion plus dollars (per year) of ad business up for grabs in the LLM era.
Facebook is toast, sooner than later, as a business. Whatever gradually replaces/upgrades that quasi dead friends network, will be premised on AI (Zuck apparently thinks it's at least in part personal agents). In the present Meta still has a $228 billion ad business that is hyper profitable.
Google + Meta = ~$650 billion hyper margin ad business. OpenAI, aka LLM as a service for the masses, is going to want to take as much of that as they can. OpenAI's active user base comes at an attention cost to the other mega services like FB or Google Search, and ultimately that attention is worth ad dollars. The question is only whether OpenAI can take it, not whether it's there at all or high value. And of course there's no guarantee that it'll be ChatGPT upon which the next great ad network is built, it will be some AI-centric business however, and likely heavily involving LLMs.
Google's operating income last quarter was $40 billion. If OpenAI fails to take the mass market + ad tandem, then Google will more likely retain their position. The solution to the cost is already well understood by eg Google: the masses don't need tip of the frontier AI capabilities, and relying on Nvidia's extremely over-priced GPUs is a dumb business model. The biggest threat to OpenAI right now is Nvidia's margins, not China. Google's focus has been on high quality cheaper AI systems for a reason: they're looking to serve billions of people that have no use for what the bleeding edge offers vs a fast + good mid-tier. Google can do that in a highly profitable way.
A billion users and the extreme cost of infrastructure to compete are a gigantic moat.
You can't get the first and you can't get the money for the second. Nobody is going to give you $100 billion to underwrite the cost of competing with GPT at a one billion user scale (or even a 100 million user scale). The easy money is already drying up, the field is set, and you will not be able to get anybody to give you a huge check to attempt to chase OpenAI, Google, Anthropic, China. That's already over. No VCs will fund you on this particular business pursuit.
You couldn't meaningfully compete with OpenAI and Anthropic if you wanted to. There's absolutely nothing you can do about it.
There's a trillion dollar valuation waiting for you. What's up?
Even nation-state pockets Microsoft can't compete. If there were no moat, Microsoft would have their own GPT killer, and GPT would be dead. Microsoft has $160 billion per year in op income. They can freely afford to spend more than the entire EU (is willing to) on AI matters if it makes sense. They can't compete no matter how much they spend and they know it.
Google has nation-state money. They have chips. They have software developers by the zillions. They have a frontier or very near frontier model in Gemini / Argon. They invented a lot of the category. They can't kill GPT. I repeat, no matter what Google has thrown at the matter, they have failed to kill or stop GPT.
Google can't beat them. Microsoft can't beat them. China's models haven't stopped them (DeepSeek was supposed to kill them years ago; Kimi was supposed to kill them).
That's one helluva non-moat. They must just be really lucky.
You might as well claim Coca Cola has no moat because it's just sugar water. Brand at scale is a large moat. People use what they know unless you severely disrupt them. They're being taught to use GPT, they know GPT, and so long as OpenAI doesn't screw it all up, people will just keep using it. To say nothing of the fact that GPT still has one of the elite frontier models, so you also still have to compete with them on quality.
I still feel like there are some questions there, as in what is the actual amortized cost, plus the ongoing variable costs and even the non-training related overhead.
Plus there is the actual demand side to consider and how does it impact when paid utilization isn't 100%.
I don’t know, I take this as there being a good amount of margin that is going to be suppressed due competition in the very near term. It will be interesting to see how these margins hold up over the next 12 months, I expect them to drop substantially
> Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
You missed the bit where they used the word "revenue".
I suspect if you factor in all the various overheads of running datacentres the actual value is lower. Revenue is vanity, profit is sanity as the old saying goes.
However the Chinese do have the benefit that China are pushing hard on renewable energies. So at least that major part of the equation is probably far lower than your average US/EU site.
Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power. That decision is just going to hurt more and more in the coming years as China's electricity costs continue to plummet while the US' continues to depend on unsustainable fossil fuel infrastructure.
> Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power.
If the US is trying that, they are failing pretty hard - 88% of new power in the US in 2025 was renewables, and 2026 is estimated to be above 90%
I have no idea why this is being downvoted when the entire agenda of the current administration is to "starve the beast" by cutting all essential government departments and prove that public administration doesn't work so that everything can be privatized. And meanwhile the head of the current administration basically does what Russia (and Romania in the 90s did), he makes money by selling off those bits and pieces to the highest (personal) bidder.
It's hilarious and depressing that one of Trump's supposed goals was making the US into an industrial powerhouse again, and yet he's acting to protect the profits of the fossil fuel industry as opposed to encouraging the kind of resource efficiency that would make the US competitive long term.
> You missed the bit where they used the word "revenue".
No they didn't...
> I suspect if you factor in all the various overheads of running datacentres the actual value is lower.
That's literally exactly what the quoted part addresses: "Amortized cost of owning a DC full of GPUs is far less than $38/card/hr". Cost of running GPUs is far less than $38/hour, revenue is $38/hour, ??? , profit
Other replies covered this, but I was basing that comment on the prices that commodity providers charge for GPU-accelerated VMs. E.g. the going rate for a NVIDIA B300 (288GB RAM) and the VM around it is somewhere between $5 and $20 an hour [1].
These providers have very little incentive to charge unprofitable prices (except maybe capturing more of the market, but there's not much lock-in here), so it's reasonable to assume somewhere in that spread is the "true" cost of being in this business, all expenses included.
[1] I'm using https://gpus.io as a reference here, and that's on-demand prices, which will tend to be the highest relative to other options.
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
>It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
Inference has always been profitable. The problem is that this profit shrinks as your model becomes less competitive. The labs are in a constant race to release the best model to maintain inference profit. Deepseek is not training their own models at the scale of OpenAI and Anthropic and mostly sustains off their efforts via distillation.
When that happens, the chips stocks will have tanked back to sane levels and the doom posting will be about that. That will put us in the 2001 era of post dot com bubble, and the following few years will show us who the winner are
Q1 '20 - $0.10 profit per gb shipped (ram and storage)
Q1 '23 - -$0.024 (cyclic ram market)
Q1 '25 - $0.131 this looks normal were on the upswing.
Q4 '26 - $1.136 Were all fucked.
If you look at a chart of S3 prices we have all been living in the "Happy Time" where the cost to run our ever growing software business has gone down. Githb says 2025 billion commits - 14 billion this year, and 2x the actions run. That means AI driven software growth is 2x to 14x.
But those ram/storage prices are going to hit the operational budget of EVERYONE soon.
Can your average sas survive a 4x price increase on its AWS/Azure bill? How about 10x (cause thats what the margins say the premium is on ram).
If the circular financing is a concern, the circular firing squad should be a bigger one. Your paying for tokens to build more software to pay more to run your software is not a sustainable cycle. People are starting to use the words CapEx again - and it's a good thing.
We don't need 10x more software per se. It's removing the high cost and exclusivity from software production (but not just software production). It also enables the production of highly specialized software by non-specialists, so it broadens the field dramatically (things that were not being produced, will be). Simply put, every bit of software that can be useful, will definitely be made; and a whole lot of unnecessary software will be made in that process as well, that's chaos-discovery for you.
> It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
They (well, most of them) are looking at loans and debt (in plain sight and hidden) of AI companies and asking "what will pay for that", not looking at how much a GPU costs and how much tokens you can sell off it.
Just putting a model on a GPU and selling it is profitable, period.
But spending more and more money on building and running infra to build frontier models that solve smaller and smaller piece of the pie might very well be unprofitable.
Models competing with the frontier labs gets more capable every month and more and more of the tasks can be achieved without reaching for the big guns.
We might very well get to the point where AI/LLM is profitable just fine but bubble still bursts and the big frontier labs are in trouble. And frankly that would lead to far more sustainable ecosystem.
AI as a business will be 10x larger than it is today within the next 20 years.
Present valuations are the bubble, not the business / industry. The exact same thing was true about the Internet bubble circa the late 1990s, except this is a lot more important than the Internet and the global economy is drastically larger.
Productivity is how you expand economy and wealth. It's how China can go from being one of the world's most backwards agricultural nations, unable to feed itself properly, to gradually becoming an agriculture juggernaut. AI is a hyper productivity tool, saving people time and bolstering capability are two of the primary productivity concepts.
Artificial intelligence will 'infect' every surface of life and business. The Internet is merely a distribution platform for intelligence (and yes that includes TikTok videos). Its first distribution chore was slow and quaint, it was human. The intelligence was always the higher value, not the distribution network.
Distribution of intelligence: data, photos, video, communications, transactions, entertainment, et al.
And to be clear, it doesn't matter if some random commentors on HN don't think very highly of some or all of the intelligence being distributed. Their elitist opinion of the quality of intelligence of a given thing doesn't matter as to whether it's happening or not.
When a 16 year old records a cloned TikTok dance video, edits it, and then publishes it, that's intelligence distribution. That's a person thinking through a series of actions to try to communicate to an end audience, via an entertaining medium. That's intelligence being distributed over the Internet. That's the quaint, slow, human form of it.
Up to this point that distribution was basically entirely human, capped (post saturation) at slow growth N intelligence output (derived from the total number of humans and their time / productivity capabilities). AI uncaps that and radically expands intelligence output & distribution at a growth rate that is so fast you might as well start picturing the planet having tens or hundreds of billions of people on it. The human population will shrink, the artificial population will increase dramatically. People will have more artificial friends than human friends within ~20-30 years.
A few decades from now the notion of thinking of computing separate from AI will be so foreign most people won't understand what you're talking about. It will just be thought of as computing. Humans will forget a time when we weren't augmented by the constant presence of artificial intelligence. Try taking 'smart' phones away from the masses. In about a decade try taking away their helper agents. A decade after that try taking away their home robots.
The steroid jump is AI + robotics. The Internet is trite compared to what comes out of that over the rest of this century. Robotics will physically remake our world, in a way the Internet can't. We'll deposit tens of billions of robots onto the surface of the planet and they will almost all be the result of having figured out how to make intelligence (relatively) easy & (relatively) cheap. I'm not talking about Roombas.
The Information Age is over. We're into the Age of Intelligence (insert the unavoidable quips about humanity & intelligence).
The Internet is just one way to move intelligence around (a robot body is another, a smartphone is another, a human body is another, and so on). It's a sub value to intelligence broadly. Intelligence is the ultimate value. The valuations (and sales growth) for OpenAI and Anthropic is because they're vast intelligence output machines in the Age of Intelligence, ditto Nvidia's business boom. We can now mass manufacture intelligence at an accelerating clip, and it has gotten cheaper at a comical rate over the past few years. While HN comments snort about pelican improvements day to day, the leap over a few years is astronomical in regards to what significant AI systems can now do (with LLMs being just one type of that).
this is tech fart sniffing self-aggrandization. You’re saying words that mean nothing. Tens of billions of robots? To do what, exactly? Have you ever interacted with people, not nerds, real human people? Real people don’t crave a future filled with robots, real people want real people.
The world will be filled with robots is not the inspired forward-thinking of a visionary it is the uninspired regurgitated science fiction of someone who doesn’t care to understand that the world is people, a society.
There's nothing that is going to stop the AI-robot saturated future from arriving. It doesn't matter if you dislike it.
China will have many billions of robots just in their industrial efforts alone. Every house and every apartment in China will saturate with robots of various usefulness. That single market will be at least 10+ billion in scale.
There will be a billion robots for just the delivery and transport categories globally. We're going to peak at at least 10 billion humans (population), a billion robots for delivery & transport segments is easy.
In the toy category there will eventually be far in excess of a billion robots. They will seem quite advanced compared to what we presently are used to in that category. The Microduck is a primitive example of what will be a large industry around smart entertaining toys. Your artificial friends will be able to 'hop into' (take control of) your Microduck (whatever the 2035-2050 equivalent is) and cruise around the house or wherever you're at. Entertainment will continue to be a gigantic industry and that will obviously apply to AI + life. We haven't even begun down the road of AI + entertainment, it's less than early there.
Every household in the world will eventually have numerous robots, from primitive robots (including single purpose and toys), to advanced humanoid robots. It won't take longer than 2050-2060 to push $10k-$20k humanoid robots into middle class or upper middle class homes. China will put enormous pressure on the cost of those robots, and they'll mass sell them to the bottom 4/5 of the world in terms of household incomes (that is, they'll come up from the floor on price and take most of the market, while some companies in affluent nations will produce more expensive high-end humanoid bots, ala what Tesla is attempting).
Tens of billions of robots globally is easy. 200-300 billion artificial friends is also an easy prediction. Social junkies will have 50-100 artificial friends that they try to tend to. It'll largely replace platforms like Facebook, the utility that served for the average person. Humans will get a lot more isolated from each other and as that occurs they'll fill in the void.
I am a nerd on Hacker News. My preference for technology and robots is obvious and an assumption that my disagreement with you is based on personal preference is brainless.
I have spent time in China and Korea where attitudes towards artificial intelligence are much more positive than the west. I have interacted with robots in these countries.
You are making the fundamental mistake all nerds make. You’re thinking about what is possible and concluding that it is what will happen. Just as nerds believe that someday it will be possible for us to live on mars and therefore we are going to live on mars.
Humans are humans. Humans are interacting with artificial intelligence as if it is human because they crave human contact and are so desperate they settle for the best they can get.
The houses filled with robots future goes hand in hand with the robots will be doing all our work future. And if robots are doing all our work, what do humans have? Time. So much time. Humans become creatures of leisure. No longer is there a too little time to build relationships, there is only time, so much time. Human relationships become abundant, humans spend all their time being human.
The idea that people will have 50-100 artificial friends is so batshit crazy that I could totally believe your comment is parody.
If real people want real people, then why is everyone at the coffee shop, on the train, at parties, and even at the dinner table all looking at their phones? Not all the time, but way too much of the time, imo.
> at parties, and even at the dinner table all looking at their phones
This is very different from what I see.
At coffee shops and on trains, sure, people are working or on their phones. But at dinner and parties, at bars and climbing gym and dances, we're there to be there with each other.
Maybe a cultural difference between where you are and where I am?
The masses won't think it's awful though, and that's the thing that will facilitate its explosion as a commonality across the planet. The masses are extremely lonely all over the planet. Along with desperately loving entertainment, they desperately want companionship.
Just as human intelligence is merely one form of intelligence, human companionship will be merely one form of companionship. AI friends will offer a new type. And for the masses it's going to feel very real and very legitimate. They'll like their AI friends as much or more than they like their human friends.
And the fake friend explosion can happen first quietly, which eliminates the initial shame phase problem. Nobody will have to know you have fake friends, or how much you interact with them. Until it's common it'll be viewed as shameful (pathetic, proof of social ineptness) and mocked by society, then it'll be normalized. This idea isn't novel, it's extremely obvious and right now we're just waiting for a few pieces of the LLM puzzle to get to the point where they're good enough to facilitate high quality artificial friends.
It'll happen shockingly rapidly when it sparks; slowly at first and then in the span of 3-5 years we'll go from nobody having or being willing to admit to having artificial friends, to everybody openly discussing it. It will then seem more odd to not have artificial friends, like people that refuse to own smartphones or use the Internet. People will start commonly referring to their artificial friends by name in conversations with other human friends. These behaviors will become entirely socially normal.
"Yeah I had Alice clean the house before I went on vacation." Every major personal robot, every artificial friend, will get a name. Humans name their vehicles, so they're definitely going to name their $10k-$20k house bot and the artificial friends they spend large amounts of time with. When you go on a long drive (etc), your artificial friends will be there to keep you company, and they will remember your life events as well or better than you do (if you give them access to that recording/memory capability).
> Along with desperately loving entertainment, they desperately want companionship.
And I am presuming that the companionship that people need is the companionship of other human beings. Otherwise, tamagotchi, fish, cats, dogs, parrots, and horses are already perfectly good companions. AI can help fill in for some of those other ones (tamagotchi especially), but it cannot replace human companionship.
Maybe some people will like to have artificial friends with genuine two-way relationship, but I think most of us want artificial servants. Robots that clean our house, keep us company, or maybe even act as romantic partner without asking anything in return. Maybe some of them will feel like friends, but in the end, we are their owners and they are our servants. Genuine friendship requires freedom and autonomy for the other side to walk away if they so choose, and are we really going to give that to our AI bots? I doubt it.
> Humans name their vehicles, so they're definitely going to name their $10k-$20k house bot
huh. I can only think of one friend I've ever known who named their car.
Anyways, not saying I necessarily disagree with the rest of what you say, but it's out there, for sure. We're a good distance away from an AI that can meaningfully act like a friend.
(Though who knows what AI will look like in even 10 years? I'm not ruling this out)
There's a rising trend among gen alpha to not use phones or internet to get back their youth period that was ruined by the algorithms. I predict this trend will continue, and I myself am yearning to use less technology each day (but can't because of salary!).
But yeah, masses will do masses stuff and that's why even shit products can sell.
Today. As new supply comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
That would depend on whether you are a user/customer or an investor/owner in AI. At the moment, nearly everyone is on the investor/owner side. The entire western world has doubled down on AI becoming the most popular product since the invention of fire. If it turns out that fire 2.0 cannot be patented/copyrighted/licensed for insane profits, those dollars disappear.
The people who own these companies have more money than they, their families, and their extended families can spend for generations. None of them have to work a day in their life no matter what happens to these companies.
Same with a lot of their investors, who will be just fine without the "insane profits" they hope for.
I'm not worried about their welfare, and won't shed a tear if their companies collapse.
It isn't just those who own stock. If the AI bubble bursts violently it will impact literally everyone. Everyone who owns indexed stocks. Everyone who owns tech stocks. Everyone with a variable rate mortgage. Everyone with a job. The AI spend is now so much of the western economic system that every one of us has skin in the game. At least on the risk side.
When every company calls themselves AI-first, what then is an 'AI Company.' In many regards, this hype-chasing branding exercise is currently no different than calling one's company a 'technology company'... Just as the term "AI" itself has (to a degree) become meaningless... and AGI will be soon to follow.
This is probably the boldest claim in the entire comments section, considering we're only in 2026.
Compare 1926 to 1999, and how technology advanced in that time. Radio wasn't widespread in 1926, nuclear fission hadn't been demonstrated, the idea of the semiconductor energy band gap, underlying the entirety of the modern computing industry, wasn't put forth until 1928. And yet, 20 years later we were purifying silicon, and 10 years after that (1954) the first silicon transistor was created and kicked off the semiconductor boom. 50 years later, we were in the Internet bubble.
We've got decades upon decades upon decades to come up with new ideas and implement them, even before the end of the century.
> That is the danger of open source models. They destroy monopolies and shrink margins.
It's unclear that the leading open-source models could exist without having distilled the frontier models. If that's true, the "danger" is that these upstream models don't get built any more (because of the tremendous cost), and we end up just optimizing what we have. Also unclear whether this would be a net win or loss for humanity.
The tech is at the point where synthetic data is almost high enough quality to train further models on, AND there are so many human users of cloud AI, that their usage can be gathered and trained upon (it's human generated and used in real world situations, so, it is of a good enough quality). The cat is fully out of the bag.
This deals with half of one of the three scaling factors only. Presumably data production (choosing your training data, etc), model size, and training time/power still contribute huge costs. Who's investing that cash if open-source models will distill out enough to compete without the investment costs?
There is a community of people who compile LoRAs of their favorite subjects/styles in order to fine tune models. There is currently a bit of a gap between LOrA creation an integration of that data into larger models but I can see serious hobbyists stepping up to that plate soon.
> they were built by distilling the world's knowledge and cultural products
... and a bunch of RLHF and proprietary training data etc etc, but regardless, this is still an exceptionally expensive process, that someone somewhere is paying for.
Anthropic is zooming to $100 billion in annualized revenue.
The tremendous cost is matched by the tremendous business output.
In the span of less than five years, starting from their first $1 of revenue, Anthropic + OpenAI will have produced businesses doing at least $150 billion in combined sales. The demand isn't reducing at present, it's climbing, they can't come close to servicing the total demand for their frontier models.
SAP is under $40 billion in annualized revenue. They're 54 years old.
If SAP sold $1 bills for $0.50, they too could rapidly outpace Anthro/OAI's growth. And when sane minds ask about profits or a defensive moat, the response is "but look at our revenue bro".
The same exact thing was endlessly claimed about Uber. They were guaranteed to die. They were selling rides for less than they cost. Uber was a commodity app anybody could MVP in a weekend.
Market captured. Whoops.
Uber now: $55b in sales, $6.7b in op income. They won.
And to be clear, OpenAI can easily permanently subsidize a fake low cost for their LLMs if they get off their asses and attach a monster ad network to GPT already. They're years behind where they should have already been with ads, the easy money made it so they didn't have to think about it. If they miscalculate the timing on switching over from easy money (market) to ads (also easy money) it could certainly kill them.
OpenAI can knock out a lot of competition by building the largest ad network, then undercutting the cost that consumers see. The margins in ads make that possible. Google already knows how that works, they're doing the same thing right now.
I'd rather have the OpenAI + consumer base + ads position, than being Anthropic. China is a far, far larger threat to Anthropic over time. OpenAI is building a service business more comparable to what Google does. Chinese models don't actually threaten either OpenAI or Google insofar as their positions are user capture based, whereas constantly improving Chinese models are a never-ending threat to Anthropic.
> The same exact thing was endlessly claimed about Uber. They were guaranteed to die. They were selling rides for less than they cost.
Lots of companies have used this strategy. And many of them have indeed died. Uber filled a void left by bad public transportation and a lackluster cab system. The higher prices (so they can turn a profit) are still worth it to some.
> OpenAI can knock out a lot of competition by building the largest ad network, then undercutting the cost that consumers see. The margins in ads make that possible. Google already knows how that works, they're doing the same thing right now.
And you think OpenAI can compete here? If LLM's end up being yet another ad funded platform, I have a hard time believing Anthropic or OpenAI will outperform Google here. Maybe if one of them strike a deal with Apple, or get bought by them. Otherwise, ads are Google's bread & butter. And, they own Android.
You also have to realize deepseeks running costs are immensely lower than say any us lab by orders of magnitude.
Chinas grid is almost entirely renewable energy with strong investments in it. In the us not the case, some dcs are running on gas.
Also DeepSeek was smart. They strategically rolled out model sizes they could support even if the intelligence wasn't the world's best for that week in human history. Then they have incrementally and cheaply improved models since.
Not to mention they also built trust with open weight models rather then fear mongering. They are in the top 3 of the smartest most viable LLM labs out there.
They have more installed renewable capacity than coal, but renewables have low capacity factors, solar is less than 50% by definition. The renewable share is growing fairly rapidly, but it is by now means almost entirely renewable.
DeepSeek CEO has previously said that their finance team considers the optimal payback period for hardware purchase to be 10 months. That is, 10 months worth of revenue should be equal to the purchase price of the hardware. This doesn’t cover operational costs such as electricity. This is their method for API pricing.
No, the absolute max power draw of these cards is like 1.5 kW, say 3 kW per card with mediocre cooling attached to a server. DCs get power in the 10c/kWh range at most. So 30c/card/hour, peanuts. And this is the most power hungry card at full tilt with mediocre cooling efficiency.
And thanks to China’s forward thinking on infrastructure and planning for oil shortages, both short term and long term, they will have access to the lowest energy costs most likely.
No, those $3 are on the high end. If you do not factor in hardware depreciation, at China's low electricity prices (largest variable cost and China has a surplus larger than many country's entire consumption) and with GB200 which is one of DeepSeek's main platforms, the operation cost is actually less than 50 cents per hour most likely. If you factor in hardware depreciation this rises significantly but almost certainly stays under 3 usd per hour, probably closer to 2.
Orinally it was an educated guess based on rental prices I've seen but Opus 5.5 and Gemini 3.8 Flash both arive at the same numbers based on DeepSeek's Zhihu post and GB200 specs. Around 30 cents before depreciation and $2 to 3.50 including it.
I'm curious why such a large gap exists between our numbers. My hypotheses are:
A. The average utilization rate of inference GPU cards is not that high (i.e. it peaks during the daytime, but is idle at night). So the time window for earning $38/h is fairly limited.
B. They are renting costly GPU capacity from 3rdparty hyperscalers because they can't meet the peak demand with their own hardware.
C. Free-tier users are eating the gross margin.
D. Subscription users are eating the gross margin.
Another observable point is that DeepSeek's company-wide gross margin was reportedly 44.6%. It's intriguing what the full picture looks like here.
> The average utilization rate of inference GPU cards is not that high (i.e. it peaks during the daytime, but is idle at night). So the time window for earning $38/h is fairly limited.
In DeepSeek's case this kind of levels out because they have a large user base both in China and the West (time zones), and additionally attract users in the non-Chinese timezones using their off peak 50% discount which is what DeepSeek talks about here.
> B. They are renting costly GPU capacity from 3rdparty hyperscalers because they can't meet the peak demand with their own hardware.
I looked this up from Chinese sources and DeepSeek primarily builds their own compute infrastructure rather than renting it
> C. Free-tier users are eating the gross margin.
> D. Subscription users are eating the gross margin.
That's true, but given how lucrative inference at API rates is, they can subsidize training and individual users to some degree. Are they still losing money overall? Not unlikely, but with moonshots that is often the case. Remember that Uber and Spotify took around 15 years to have their first profitable quarters.
How is the math so good for data centres but so bad for home gear? A Mac mini capable of running these models ends up at around 10 years of subscription costs.
A single person could make french fries for themselves cheaper than McDonalds, but once they have to start including the cost of the building and equipment, charging for their labor, and complying with health codes and worker safety requirements, they will quickly find that the fixed costs swamp the variable costs.
Here's an anecdote. I have repurposed my 6 year old 32gb M1 MacBook Pro to run Qwen 3.6 35b a3b and I used to give it nightly maintenance tasks that span my OSS github repos.
I gave it mundane tasks like updating dependencies, writing changelogs, etc. It ran about 2 hours a night.
My MBP consumes 0.1w idle and about 10w running inference. I pay $0.33/kwh. If it runs for 2 hours a day over 365 days - it will cost $2.41/year (excluding idle time).
Excluding the cost of hardware - the running costs alone are more expensive than DeepSeek's API and the inference quality is much lower. The economics are even worse on AMD AI Max mini PCs and worse still with a dedicated GPU setup.
That said - my Apple Silicon example is orders of magnitude cheaper than western inference providers and offers unrivaled privacy.
Perhaps a local AI setup would be compelling for small/medium business users... and it would be practical if the hyperscalers didn't buy all of the potatoes on Earth.
(numbers are just to illustrate a point, not accurate) Now assume you can't buy a 5lb bag of potatoes and you can only buy 10,000 lb bag. Now also assume that you can't buy this 10,000 lb bag unless you know someone else will pay for fries for the other 9,995 lb because you can't eat that many fries. That's the kind of scale the token inference market is efficient at.
How much does a fryer cost? How much does the fryer cost to run? How much would it cost to get the fries into the customers hand (i.e. food truck, rent)?
Serving LLM inference is more efficient with more concurrent requests. For a single inference request, the bottleneck is GPU memory bandwidth, as the inference engine spends most of its time waiting to move values in memory. With concurrent requests, it's possible to service multiple requests in a single pass through the model weights, meaning you can utilize more of the GPU/APU's compute.
So you have to price your local hardware against however many subscriptions you would need to run a dozen concurrent agents around the clock.
But even then, I would guess subscriptions are priced on the basis that they don't expect every user to utilize their entire usage limit. I.e. power users are subsidized by casual users.
Because the datacenter setups serve multiple users concurrently while your mac mini only serves one. You would also lose money hand over fist renting/buying 4xH200s just to serve deepseek flash to yourself.
These datacenters run the GPU's close to 100% usage 24/7. The Mac mini at home probably has <25% utilization. And then there's economies of scale, wholesale discounts, etc.
The Mac mini GPU also isn't as optimized as the datacenter GPUs
I just installed a $16k GPU at home - super pricy to be sure, but when you price it against hiring 4 junior developers working around the clock it's a steal
I think we need to collectively get a grip on statements like this.
There is no reason to think inference can or will remain interestingly profitable even if you assume pure-play inference providers who don't do any model training. Serving an LLM isn't child's play but it's also not so hard as to count as a moat. Inferencing is:
1. Selling a largely undifferentiated product.
2. Into a highly competitive market.
3. To customers who are primarily sensitive to price.
Industries like this have always existed and in the modern era tend to have margins of about 3% to 5%. Any level of margin higher than that gets competed into the ground.
If you want to claim inference is highly profitable in general, you're going to have to answer why it won't end up like supermarkets or airlines, both of which have bigger moats than inference.
The other problem with this analysis is obviously that you can't just stop training models. Even if you don't care about improving capability or performance, the knowledge in their weights gets stale. Cost of inferencing has to take into account the ongoing cost of training, you can't cleanly split one from the other.
Amen. AI boosters look at currently high profit margins on inference and blindly accept claims that these inference profit margins will continue. Of course those margins depend on historically massive R&D spend on training. The moment that training stops, AI returns to a commodity market and the inference profits end. Big labs are trying as hard as possible to keep the gravy train running but they can all see it won't last forever, hence the rush for IPO & regulating away open models.
Boosters are right that AI is a historically impactful product, but big labs will be unable to meaningfully profit from ushering in the AI era just like Netscape failed despite ushering in the internet era.
"If you want to claim inference is highly profitable in general, you're going to have to answer why it won't end up like supermarkets or airlines, both of which have bigger moats than inference."
- I don't... "inference is profitable" is as of now. No one knows future, neither have I commented anything on it.
Inference will continue to be profitable while we don't have enough compute in the world to serve the demand we have. Also, I don't see why it's a problem that the end game for inference of open models will be relatively low, ie 5% or so. You can have a huge industry running on these margins. Though I would note that hyperscalers have been selling compute commodities with a high margin for a long time.
Demand is elastic, there's no fixed pool of it. What determines profitability isn't merely supply constraints but how much margin your competitors are willing to accept, more or less.
It's not a huge problem for users but if margins are low it's a huge problem for the debt funded datacenter buildout. It's just pure finance at that point, about whether those GPUs can generate enough yield to beat deprecation and financing costs.
The big clouds generally don't sell compute at high margins. The margins differ a lot between services. Services like core VMs and object storage are usually low margin or even break even. Where they make the money is things where they've got proprietary software with competitive advantage, like managed scalable forks of Postgres, or egress charges that are there to create lockin. Managed DBs are pure SaaS and can generate SaaS like margins.
I think hardware scarcity plays a part in inference margins. They would get squeezed super quickly if anyone who wanted could just build an inference cluster, but that's not currently the case.
I would assume hardware production has been ramping up significantly and it's more of a temporary supply chain issue though. While it could last multiple years, that's certainly not long enough for the hyperscalers to get a return on investment. Obviously more complicated than this, but still... I'm not buying.
We're also starting to see efficiency gains, so we might meet halfway.
I would be surprised if there was such a glut of undifferentiated inference providers for open models otherwise[0]. Possibly just people trying to minimize the loss from already-committed capital, but I don't see AWS and DO selling inference via OpenRouter unless they're making a decent profit from it.
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
DS4 Flash is so cheap also, imagine what the numbers look like for a flagship model like Opus 5.5, which charges 100x more. Of course inference for that model also costs more, but I doubt the proportion is the same.
The top quality frontier models are used to compress / improve the earlier, lesser models, to make them more efficient. So that pushes costs way down on previous models.
155T tokens/day per cluster? OpenRouter reports that last week total usage on their platform was 164T tokens. Surely all that demand couldn't be satisfied with one cluster? Are these different units of token, or maybe something got lost in translation?
Exactly... I was going to post exactly this and you called it two minutes before me. On openrouter, it is shown that 600T+ tokens are provided per month, so 20T+ per day. Openrouter is not all the world but it seems that something is not right with this 155T figure... Unless everyone is idle everywhere, waiting for demand
From the Github benchmark numbers, one GPU can do about 26k tokens/s when serving 256 users, which is about 2.25B tokens/day. A cluster would need to be close to 70k GPUs to serve 155T tokens/day (which seems doubtful).
Napkin math comparing a cluster with a single latest gaming GPU shows a that the cluster pushes about 9 million times more tokens per day. And the cluster runs a more powerful model.
This is a major, major datapoint. That’s at least an order of magnitude higher margin than literally anyone is discussing. The entire ai discourse currently assumes very small margins on open model inf. And by this post it’s more like 1600% (h100s rent for $2/hr - and id guess deepseeks costs are far below that)
Deepseeks strategy is that each project must break even within 10 months.
$38/hrs equals $320K in 10 months.
Assume they only have 100 clusters, and as much as 40% operational costs. That would get them about $20M in 10 months, which I suspect is more than what 4.1 flash training has cost them.
You can buy such a PC off from ebay for only €150.000. We did.
But the architecture is similar to kimi's mixture of experts, and we already have such an optimization for different experts swapped in from disc by antirez.
If open source inference engines proliferate (in terms of adoption, across models, across inference hardware) I imagine you'd get pricing pressure on that inference rev towards base card cost, so my interpretation is this is actually a bad development for compute providers.
Do they share latency / concurrency figures (aka the deployment shape that gets them that token/s/cluster? Would be interesting if they're using deepEP/WideEP + PP and TP
Indeed. This could actually be what breaks America's world order.
AI can already clone major products like Photoshop and Office. Maybe not at 100% feature parity for now but at a "good-enough" surface level. That will only keep improving. Eventually we could get clones of Windows and macOS at feature parity.
Even if US lobbying/corruption tries to cripple AI to prevent that, China will instigate and accelerate it.
Interesting that the DHS "Exclusion and Removal Order #2026-01" that was made public yesterday was actually released July 24th, 2026 and appears to have taken effect then. Agencies would have started pulling deepseek at that time. Which means it's likely DS became aware of the memo around then.
The release of this repo hours after the DHS order is a tactical move they started preparing when the DHS order was issued almost 3 months ago, and they timed publishing the repo right as the order hit the federal register.
> The release of this repo hours after the DHS order is a tactical move they started preparing when the DHS order was issued almost 3 months ago, and they timed publishing the repo right as the order hit the federal register.
What does DS achieve with this "tactical move"? Or, are you saying this is a tactical move from a higher level (Chinese gov interests)?
“ Or, are you saying this is a tactical move from a higher level (Chinese gov interests)?”
What country produces all of the ASICs used in contemporary bitcoin mining?
The faster we get to a good enough open model, the sooner it gets burned to hardware, and the sooner the transition to (relatively) low tech, low margin, high volume cards to run it on …
You're not going to make money buying a new 14nm bitcoin miner, and that is where the China foundries are for cost effective silicon. They're great at 28nm+ and good at 22nm pricing.
Most new design modern sub 7nm ASICs are made in Taiwan/Korea by TSMC and SLSI. That is where miners are profitable.
That’s part of the equation but there’s almost certainly interest in the open models guiding information trends for the interests of the Chinese government as well. So not only to help their economy in a variety of ways, but to help undermine the US AI efforts in the process and put them finger on the scale of what models ultimately do output.
making AI bubble burst (even if it is just frontier labs that got hit) while chinese companies keep making money off cheap AI would be massive economical hit to US
Except this is also banned under the DHS order. The order bans "information technology products and services developed by Hangzhou DeepSeek Artificial Intelligence Co., Ltd", and more pressingly, "third-party products and services developed using DeepSeek implementations".
This is also the word inside Defense sector contractors as well. No PRC origin anything - no code, OSS or no, no open weight models, no assistance from them.
No regs set in stone yet, but DoD doesn't need regs - they can tell you what the rule is and that's that.
Not sure why you're being downvoted. It's very clear to me in the order that even the open weights would be forbidden, even served through vllm or sglang etc.
Michael Burry burned through tens of millions of dollars on holding his position. Others enter later but still ahead of the collapse. They knew and they didn’t. And even if Michael Burry new exactly when, he had to convince his investors that the millions they were losing leading up to it would get recouped.
I saw the same kind of comments when the first Deepseek model was initially released. However the relevance that investors assign to something like this does not seem to match the excitement expressed here.
I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
They'd be wrong. Especially about Europe. Many companies are in a bind about this: they can't upload their confidential files to a service provider but they do want AI. So right now they end up with a choice: do nothing, breach their contracts, use a service provider that pinky promises not to share their data (which we have already seen they will violate) and run local models.
The latter is the reason you can't buy powerful GPUs for reasonable prices because a local model (GLM 5.3, DS 4, etc) + some hardware is all you need to solve the problem in a way that avoids the legal issues.
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center buildout, not because some mid tier European insurance co needs to serve deepseek
Yes, well, you're wrong about that. They're buying GPUs by the pallet here.
Mid tier insurance co is working with local inference providers and some of these now have thousands of GPUs up and running. Don't drink the HN cool aid that the EU is a backwater where nothing happens too much or you'll end up surprised one day.
The one thing where Europe really does lag is in training, that would be fair. But inference providers are popping out of the ground like mushrooms and there are some billion euro+ projects underway. We -unfortunately- also have some scammers that are bilking (or attempting to bilk) investors pretending they have more than they do, this is annoying but I would expect that other countries have similar players as well.
The 'by the pallet' buys are for small local projects, not data center size projects (of which we have more than a couple as well), as you probably already knew. This isn't some silly dick measuring contest.
The point was: they're buying them from the retail channels in relatively high volume because there is a massive shortage of GPUs in general, but that does not seem to stop parties with smaller projects who do not qualify for buying directly from NV (assuming NV even has stock to sell).
5090's now sell for the same amount (or more) that you could buy an RTX 6000 for a few months ago, and RTX 6000's are ~19K and up.
I found a stash somewhere, bought two, then two more and when I was ready to buy the remainder they were all gone. Serves me right for being too careful.
No, the point is that US demand is what's driving up GPU prices, as GP asserted. Europe's purchases are a rounding error. And even then, most of the data centers under construction in the EU are from American companies.
this is a fun comparison, so I ran the numbers - in the US the cancelations alone are as big as Europe's entire planned buildout (~11 GW), or almost 3x of what is currently under construction (~3.8 GW)
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center build
Its not just the US ... Where do you think all those 5090's end up? Its just crypto mining all over again. Most people do not realize that China was the biggest crypto mining operation because of its cheap electricity and people willingness to take big risks.
The same happens now also, with people buying up 5090's (and lower class cards) in bulk, to strap into AI rigs. With Nvidia discontinuing the 32GB 5090s, we already see pressure on the lower end cards that had a better price / performance for a while now.
China will do China things *lol* ... People there are just more willing to take bigger gambles (and the cheap elec helps a ton). But that is also a factor why a lot of consumer level hardware gets under pressure. Its not just the supply chain to the enterprise market that is eating a lot for AI.
That does not matter so much anymore, when Huawei starts selling those NPUs for a fraction of the NVIDIA price. It's all downhill from there on for OpenAI, Anthropic and NVIDIA.
It's all downhill for OpenAI and Anthropic investors, because they will never recoup what was invested. NVDA is safe(r) - Huawei first needs to catch it, and as AMD learned, it's a fast moving target.
The other choice is AWS Bedrock, if it happens to have the model you're interested in. They do have GLM-5.3. They don't have DS-4 or 4.1 yet, and updates are intermittent at best, but they're already on the trusted provider list for anywhere large enough to be interesting.
Depends on the parameters of "acceptable", but they do offer inference in non-US zones. If the company is already on AWS for meaningful workloads with sensitive data (which is not uncommon), they've already been through the hard part.
That being said it's not quite as simple as would be ideal, you do need to check whether there's an additional EULA on your specific model choice.
For data residency reasons? You can meet data residency requirements on AWS, I'm pretty sure. Mind sharing which condition blocked it in your experience? Mostly curious because in the US there isn't that much interest in on-prem deployments because it's easier to scale up and upgrade using the cloud and it's easier to use AWS because you don't have to run a new compliance process for each part of the stack.
You could until the Trump admin decided to (attempt to) overrule the deal that was in place (the DPF).
The perceived lack of independence of the FTC is a major driver behind the exodus on this side of the Atlantic and it isn't just the smaller players that are moving. Airbus for instance.
Now the datacenters supposedly operating under at-arms-length management are no longer sufficient for compliance.
There are calls to repeal the whole thing with respect to the US.
I see. So there's a risk that the EC or CJEU make a determination against the existing system because of the non-independence ruling of the FTC. So the movement away to regional providers is because no one wants to be caught surprised if the EC/CJEU make such a determination. Interesting. It must be early stage because from what I can see the most spend is still on the big two, but this does bode well for Mistral and EU-based open model hosts. Lots of room in the 20%.
Yeah, about that... My kid works for a very small company. Chatted the other day about in-house inference vs ZDR providers and all. His team is trying to get access to GLM on AWS Bedrock. It is a second week they are trying to schedule a simple meeting to discuss what they need. Random silence, and back and forth from a new person each time. 2 WEEKS to just schedule a meeting... And as far as I know, no luck so far.
You do realize there are hundreds of datacenters in Europe and the US that rent GPUs to European companies, right? You don't have to buy a GPU or even figure out how to use one. Pay for SageMaker.
Europe isn't in a bind. They can, and have been, running open weights on inference providers around the world. And they pay a premium for US models and accept its risk for the same reason they remain US allies. The cost is worth it.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Quite failure by someone to assume that EU businesses would use US over Chinese, based on the current political atmosphere. US causes even bigger concerns than Chinese labs.
It is flabbergasting, indeed. I'm no friend of China, because it is an evil dictatorship, but as a business man I prefer dealing with China over the USA nowadays. At least China is predictable, even in its corruption.
It's just a single dictator. China has one dictator with full unilateral uncontested power. No courts shooting down orders or elections changing tides. One guy, all the power, for life. Every official, citizen, and business owner take his orders, uncontested.
If you think that is the choice basket to put your eggs in, best of luck.
Exactly. Imagine China's dictator would kidnap some other foreign president just to steal their country's natural resources. Or if he just starts a war against some Middle East country to take the focus away from some pedophile scandal.
Thanks God nothing like that could happen in the US, thanks to its constitution, courts and armed informed citizens. That's why I'm putting all my eggs in that basket.
And the airing of said laundry achieves what exactly? I don't see any knock on effect from that dirty laundry being aired, other than entertaining the masses with new scandal and signaling to the rest of the world that the US is not to be trusted.
> the airing of said laundry achieves what exactly?
No midterm routs in China.
For Europe, the answer is using homegrown metal, even if it is severely underpowered, and accepting that it will be running American and Chinese open-source models for the near future. (The French lobbyists looked likely to turn Mistral into another Arianespace, but they’ll be resisting implosion for at least another year.)
Oh really? What about the Epstein files? What about the State protecting billionaire pedophiles? You have got to be naive beyond wit or willfully malicious to spew this nonsense. Did you vote for Trump? I think you did.
Did you hear about the belt and roads initiative? Like this is the stuff that gives soft power in geopolitics. When was the last time the US did something comparable?
Our pedophile convicted Felon president literally tried to take control by force, caused an insurrection and said I quote:
> In four years, you don't have to vote again. We'll have it fixed so good, you're not going to have to vote.
At least the Chinese president isn't a pedophile enriching himself at the expense of the Chinese people. China has steadily increased the quality of life for its people where the quality of life for the average American has steadily decreased.
> they do not believe that big corporations in the US or Europe would actually use chinese models
This is exactly right. The only AI that matters from an investor perspective is the AI that is built into your phone, your car, ships with Windows and Mac, and built into legal and business tools like Lexus Nexus and Salesforce.
That's why OpenAI hard pivoted from consumer tech into enterprise ?
The only AI that makes money is enterprise. Being open weights/self-hostable is a huge win before the pricing even enters the conversation, but pricing is also adding a ceiling on what US labs get to charge. And stuff like powering other software also works better with fine-tunes, hence legal models, etc.
European companies are more and more considering and planning to deploy Sovereign local models.
Currently the price sovereignty is still not outweighing the cost of US dependency. But big providers are already working on finding ways to ensure customers independence of the cloud act.
Also the concern that open weights models could contain dormant adversarial behavior (i.e. they are sleeper agents) is more popular amongst decision makers than one would think.
In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
Well, they do. So you review the output. Models should be treated as juniors, not a line of code or legalese they write should be used without review by a human, not another AI.
> In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
I find such claims a bit like if 3M/DuPont released a paper on the carcinogen dangers of PFAS and someone said this was "a clever marketing move".
Like, great, this can have hidden dangers we only learn about years after it's irreversible… how does this make "our"* companies the good guys?
A lot of us were already concerned about backdoors mandated by the US government, but that was just encryption and this is everything that AI ends up being able to do.
* in the sense of "the G7" until the start of this year, because why would other nations prefer us over China in the first place, and thanks to Trump vs. Canada, Greenland, Iceland also now the rest of the G7 has to treat seriously the possibility that the US is more of an expansionist threat than China is.
burned? your business leaders willingly sold out your manufacturing industries to make just a little bit more money, knowing itd devastate many. It was your own who sold out your working class time and time again, not "big boogeyman china" pulling the strings.
I don’t know about investors, but EU businesses with regulatory requirements are favoring EU-hosted Chinese open-weight model over US closed-weight services.
to be honest anything built into my phone, car, ships with mac is a hobby
what's important is where big enterprise point their budgets. and there we agree, it's not to chinese models (as hopeful as I am that we can avoid a duopoly and great local models)
>I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Correct. And they're likely very correct in the belief.
When people on the internet claim that frontier labs are done for because Chinese open-weight models exist, they really don't appreciate that the people spending the money do not want to hear "we can just run it cheaper ourselves!" Any software engineer with even a few years of experience knows that phrase is absurd, and executives are simply not interested in the risks associated with such an endeavour when they can just buy the solution outright.
Consumers will have access to open-weight models just like how consumers have access to Linux. But you can just go look at the market share of operating systems to see why something being free and openly available doesn't matter to executives who just wants to get things done and is willing to pay a premium to a trusted business to do the work for them.
And none of that even touches the fact that these frontier labs have only ever been in-front of open-weight alternatives. Again, if you're an executive, and you're questioning if you should pay a premium for a better version of a tool which you presume is important for your company's ability to compete, then the answer is immediately obvious: pay the premium or risk falling behind competition who is willing to pay the premium. And when you're talking about global companies who can pay these bills without it being significant enough to even show up on a quarterly financial statement, then it's an obvious sell.
>But you can just go look at the market share of operating systems to see why something being free and openly available doesn't matter to executives who just wants to get things done and is willing to pay a premium to a trusted business to do the work for them.
That's why most of the internet runs on Windows IIS rather than Linux or BSD. That's why everyone here is running a VM on their Mac that hosts Windows containers to do development in.
> executives are simply not interested in the risks associated with such an endeavour when they can just buy the solution outright
I think you're missing two things here.
1. The margin by which a hosted product is operationally superior to self hosting an open source alternative is precisely calibrated. The SaaS provider has no reason to invest beyond the point were they can reliably make the sale.
2. Self hosted open source solutions come with a big bonus re: the ability to read and modify their code. But realistically ain't nobody got time for that.
AI upsets the apple cart in both cases. It's now far easier to manage locally hosted open source solutions, which means that in order to maintain the favorable effort ratio for making a sale, products must all get significantly better than their open source alternatives. That's gonna be a slow process involving innovative product owners. AI is getting better faster than it can happen.
Also, since AI is trained on the open source code, it's very good at integrating open source components. Previously nobody was bothering to gain that kind of mastery over their tools but now you can leverage it right out of the gate--but only if you're using projects that are in your assistant's training data. Maintaining a moat around a software product has always been orthogonal to making your product useful, but as we lean on AI more, it'll get worse. The more you protect your IP, the more its usability degrades. This was always true, but now it doubly true.
Executives may want to pay for a solution to their problems. But the companies that might provide those solutions no longer have a sustainable path.
The future of software is as a tool, not as a product or a service.
There are times I wonder if the "just use China!" sentiment is astroturfing. That'd certainly explain why there's a lot of calls for that in the face of China's long and storied history of hacking Western networks.
It's generally understood that for better or for worse, if you're working on anything (not just IT) that is security-sensitive, you're going to have restrictions on what can and cannot be done that may seem absurd at first, but hold up to scrutiny once more thoroughly examined. I would say that you don't even need to thoroughly examine this particular restriction. I have prod data at work that I can't give to humans outside of the United States. Not in India, not in the UK, not even in Canada. Clients insist on these clauses in contracts. This has been a known practice across many fields in this industry for decades.
Now we're suddenly seeing a chorus of posts saying that we should loosen up and send data to a non-human intelligence that was developed by a known geopolitical rival with a history of malicious intrusion on Western networks. Where does that come from?
There were pretty specific rumors about IPOs happening early this year, up to apparent insider bets on prediction platforms. Don't you think the apparent delays in IPOs could have something to do with the open weights/open source competition? The monetary conditions (interest rates,...) were better early in the year, arguably predictably so, so what would be your alternative explanation why they chose to postpone?
I think there is also a perhaps the need to assign air-quotes to "investors" when talking about the US markets.
There's too much fluff and noise there. Too much (over)leveraged trading. Too much speculation.
What the real investors think is hidden behind the actions of the speculative traders.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models
The Chinese models are open-weight. You can download them and run them on your own servers. Or you can outsource to Mistral, Privatemode or any of the many other US/EU inference hosters.
So the "security, regulatory and/or business continuity concerns" arguments are effectively non-existent. And whatever argument does possibly remain, also applies equally to Anthropic, OpenAI & co.
If you believe those investor, I assume you mortgage and bet all your wealth on it. SpaceX? Anthropic? Chinese has talents, electricity and Huawei. I really believe those investors are tulip blind to pile money into those western AI. China even now world number 3 ssd+ram ahead of SK Hynix. So armageddon of RAM SSD LLM model at dirt cheap price. And as time progress, large ram and small gpu will be built into handphone (which again dominate by Chinese).
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
European business users (+ consumers) can access GLM-5.3 via Mistral.ai hosting. (with GDPR)
This isn't true, especially not with the inference engine becoming public, now you can build your on models on the same or similar architecture and ship massive gains on top of these stacks.
I think people don't realize the Salesforce SAP Palantir would love to ship these models, and they have/are shipping them.
The threat of exfilteration of data via these models is high but it's nowhere near as high as the margins.
For instance if you could cut your ai input bill by over 90% would you do it? Can you put some of that extra margin say 10-20% to make it more secure.
I know of atleast 10 fortune 500 companies that are entirely planning to or have moved to Chinese models for a large portion of their work.
It's insane people in finance are delusional enough to think that won't happen.
Maybe Govt will try to de-risk with bans on using open models but I don't think that's going to work at scale, people will literally spawn labs to launder DeepSeek/GLM models into a US model by some novel training etc approach or something.
I still don't think that it's as much a kami-kaze rather it's an attempt to normalize the world around the Chinese stack.
If DeepInfer is supported by everyone, it will be much easier to export stuff aka hardware that only supports it for now or is optimized for it.
Assume china starts selling hyper optimized Asics for inference at some point.
This feels like a race to the bottom as soon as possible because they have much lower costs, if there is a Wipeout in AI Labs in US chinese labs are largely insulated, and they can hopefully buy Nvidia chips for cheap if that does happen.
Europe is going to be just as worried about using US models for large parts of government related functions and they're going to feel alot better about open source self-hosted chinese models than US models which can literally get turned off on a whim by Trump. They'd prefere home grown models if they can, but if they can't they'll go self-hosted and that'll be Chinese at this rate.
lol big corporations are 100% using Chinese models in their business operations. They dgaf about security, speak to anyone in cybersecurity or cyber insurance
The labs pushing this cyber scare is an attempt to cry wolf. The corps don’t give a fuck
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Well they are wrong, I'm at a large EU startup and they are currently considering Deepseek as their default model, Opus/Fable would still be available for complex tasks.
A better interpretation is that there is a window within which you can make money on an unconventional view. You can't be too close to the market's conventional wisdom, but you can't be too far away from it either.
It is clear from the beginning that the Chinese Open Model are a proxy for economic warfare. Anyway their singular goal is to dethrone US monopoly on SOTA model and burn all the economic hype and investments. If this strat works, which we will see after Anthropic/OAI IPO, we'll be in for an interesting ride
Well, what can I say? If you play dirty, then so can everyone else. The way I see it, consumers are the real winners here. Just another variation of classic competition. If such open models didn't exist, then make no mistake, OpenAI and Anthropic would be making sure you are bleeding dry with your wallet without much innovation.
No, that isn't going to happen. The US can't afford to lose the AI race and US companies that can't compete will get thrown under the bus and replaced with those that can.
The DHS order only bans buying DeepSeek products directly (e.g. doing inference on their China hosted servers). It does not apply to open weights at all. And DeepSeek just gave their response to the DHS order.
"DeepSeek implementation" in this context means a third-party product that uses a DeepSeek product, e.g. the DeepSeek API. This does not apply to open data running on a non-affiliated third-party inference provider.
No, it is being interpreted that way by people who either don't understand basic legal writing and the legal standards for an order of this nature, or by people [who] are purposefully FUD'ing in order to take advantage of that lack of specialist knowledge.
I don't understand basic legal writing, you you may very well be right. I would be nervous about fulfilling a contract or building a company with, for example, my code base, which is heavily built with deepseek
Considering how absurdly far ahead of everyone else SpaceX is, 2T seems low, and is more of a testament to how immature the space market itself is at this time.
If SpaceX was reset to 2016, they would still be ahead of everyone else even today. Their deployed capabilities today are ahead of the plans of most of their competitors.
even if they are far ahead, where does the revenue come from? current market size is like 20-40 billion annually.. even if it grows to 2x it's still a long time until that would justify the valuation.
but perhaps you are thinking that starlink is the key? sure probably a bigger market - but any market it accesses is bound to eventually prefer ground lines.
so the only way to justify the valuation is if the AI side of the business becomes the most dominant AI business
It doesn't make sense by any standard valuation metric. Yes SpaceX is the best company in the world at getting things into space. The valuation doesn't reflect the actual size of that market at all. It's assuming AI in space will be world-changing, which is a huge leap to make.
Interference optimization is a solved problem with no moat. Anybody can achieve this result, optimize for a particular hardware, optimize for a particular problem. In general, software is no longer a moat.
The moat is custom inference hardware, lower power, higher throughput, etc.
Yup. Hyperscalers are going to eat them alive. AWS/Azure/GCP hosted instances of cheaper-but-almost-as-good models like Deepseek are going to eat OAI and Anthropic alive.
They'll need to release cheaper, better models to compete. Which may not be possible.
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how an open source inference engine undermines them.
Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.
I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.
So... this makes it easier for someone in the position to run chatgpt class models as a service to get similar returns to openAI/anthropic, if I got you.
And I imagine this engine is by default mostly just deepseek compatible and inference engines are not universally applicable to all models?
It sounds significant for competition but I'm not sure how it sinks Anthropic/openAI. Presumably they still have their frontier models, the training data they're vacuuming from current users and access to hardware to serve at a ridiculous scale, right?
DeepSeek can accomplish over 90% of tasks of equivalent value at one-tenth of the fully loaded cost of O/A, and this capability can further spread through open source. In my view, the IPO risk for O/A is no longer just a valuation discount, but a substantive challenge to its long-term business model as a high-margin foundation model provider.
In that scenario, even if global Token usage increases 100-fold in the future, that alone would not be enough to justify these foundation model companies being worth one or two trillion dollars.
I think you are underestimating the value of that last 10%. Returns to intelligence are non-linear. The 80/20 has been true for over a year now, and it has not impacted the labs revenue.
I think for me it's more the possibility of the ideas here being incorporated in other Opensource projects (vllm, llama.cpp, or in the architecture choices of the models themselves)
Deepseek manages to be ridiculously cheap and efficient with prompt caching, i spent 550M~ (1.9b input cache miss, 1.7b output) tokens and it cost me 5~ bucks, that would be ~150 bucks on Opus 5.5, if i ran that workload in parallel that would be $50 vs $1500, e.i. i wouldn't want to subagent heavily with anthropic models.
now imagine Kimi K4 or GLM 5.5 with those cost savings
I think you’re correct, although suggesting that open AI is a trickle drown of AI companies is a bit naive.
But the key point is that something good enough always attracts more interest and contributions because a productive outcome is very realistic and tangible.
Open Weight models and competing open source inferences engines massively undermine monopolies; in theory at least.
See how cloud is easy to divest and “nationalise”.
It is, and always has been, data contracts. Which is why the hyperscalars are a more likely candidate to take American frontier business than Chinese vendors.
I can click a button in Azure foundry and run open weight models that are approved through my compliance and procurement teams.
I don’t know how open sourcing this stuff helps deepseek but it means I’m getting ever better and cheaper models so I’m fine with it.
Well, whatever real or perceived moat Openai/anthropic had. OP claims this open source inference engine reduces their chances of high valuation, I'm assuming this is due to their in house engine being something they could use as differentiation before?
That is probably what has been the most fascinating part of this AI/LLM boom, the moat was apparently never the technology. It has been very much like nuclear energi (and weapons), once you know how to do it, it's not that difficult. It's still fairly hard, but you can do it. The moat has been the funding for the infrastructure and if that dries up you're in trouble.
I've been trying to find an answer to that, too. The best I've got right now is this:
Moat-like things going for them:
- access to the US government (for huge inflated government contracts as a source of free capital, having the competition banned, having their products legally mandated, getting other preferential treatment, having foreign governments bullied into granting similar advantages)
Too unsustainable, too fragile, too small:
- Strong brands: Brand loyalty seems like a pretty insignificant factor in AI right now
- Semi-exclusive hardware access, RAM buyouts: Unsustainable even in the short term
- Lock-in effects: Their models aren't very stable across versions, or very unique, and switching providers is about as easy as it can be in a highly competitive market.
- Exclusive technology that is important, advanced and hard to replicate to a degree that would materially set them apart and justify potentially much higher cost. Maybe I'm missing something here, but a lot of the claims they make to that end fall apart even on light scrutiny (due to things like test sets that are small, hand-picked and clearly overfitted on, or broad general claims that cannot be substantiated). I'm getting the feeling they don't have anything that materially sets them apart at all, but that's seems impossible to measure in any meaningful way, so who knows, they might have something here.
In theory, this can allow a small US lab to compete with O/A/G and take marketshare from them with small open models deployed centrally or (and this is think is the real moat) on-prem for enterprises.
VC and valuation (alone) backed success has never been dependent on a business or product being good. All that is needed is finding the bag holder(s), and the way our economy and financial systems are designed that's the core competancy of this system.
If Anthropic is going to crash 50% or more at some unknown time in the next two years, and you have foreknowledge of this, you can easily make a massive profit by placing a bet weekly. The odds are VERY long per week, so on each losing bet you only lose a tiny amount, but on your winning week you will more than make up for it.
Timing is, as a general rule, important. In the particular case we're discussing, the odds of a 50% swing are so low that you can tolerate being VERY wrong on the timing if you are correct on the swing.
This assumes all other information remains exactly the same as it is currently. That is not how the real world works, however. As time goes on, and it becomes more obvious that the company is floundering, the "long shot" will no longer be such a long shot, and will not have as high of a return.
idk, if "prediction markets" remain a real thing, there should probably be some requirement for previous experience in options-adjacent markets to ensure people actually understand the economics of time-valued positions.
Not really. Anthropic’s headcount isn’t huge, and only a subset of their employees are working on the frontier. It’s also easier when you’re a follower.
Can you say more? Why would the inference engine be the secret sauce that would protect trillion-dollar valuations? Isn't the inference side somewhat of a known-quantity, while the training is more of the lab's trade secret?
No, that's open source and it will be an accelerator for everyone, instead of a small advantage kept by a few. Even the frontier labs benefit. I'd argue (like I did when the initial DeepSeek was released), that the AI market *needs* to those inference breakthrough.
Has anyone figured out why Chinese billionaires are releasing these models? Do they hold short positions on US markets? Why would Xiaomi, et al. release these models in the first place? What is the strategic purpose here? I cannot believe for a second that they are doing it for human good.
Deepseek started as Liang's side project funded with his hedgefund couch change compute. That set the economics for everyone else in PRC. If deepseek releases for free because Liang doesn't give a shit about making $$$ on his hobby, every other PRC player has to follow. And serendipitously, due to US export controls, PRC tech companies can't dig themselves into ruinous seppuku debt buying compute even if they wanted to - what they can buy, i.e. everything around DC infra including power, they get for fraction of the cost expediently.
TLDR Liang said fuck profits... which means every other PRC competitor had to fuck profits, which ends up fucking profits of western labs so deep in debt due to rampant financialization and can not afford to say fuck profits.
Sometimes a billionaire is fine just burning $$$ on a hobby while remaining a billionaire instead of trillionaire, and it just so happens the side effect might mean preventing others from being trillionaires as well. Bonus others is geopolitical adversary who is hedging economy on trillion $$$ returns.
I'm still not understanding why one hobby projecing being open source means others should also be open source. So you're saying that PRC is making a bet against US companies and the US markets to crash them in order to stifle the US economy? That is an interesting take, I would think the US market is far more resilient than just the tech stocks. Speaking with my communist friends though, it does seem like this is the ultimate goal: Topple the US hegemony and US foreign control by crashing it's economy while China makes a bet on free/cheaper AI models, with the ultimate goal of agitating the people enough to cause a revolution to turn the US into a communist state (really delulu stuff if you ask me)
Seems kind of pointless? Wouldn't anyone in the US/western world simply use the chinese models in their own inference? That would not strengthen the Chinese position because the models got downloaded for free off HuggingFace, and then cloud inference bills would just put money into US companies.
Ignoring all the slop out there, we are seeing AI being sent to reverse engineer, accelerate projects, etc.
According to Greek economist and former finance minister Yanis Varoufakis, an open source AI strategy is one way a capitalist economy can avoid the rent trap (platforms being created, monopolizing resources, and extracting ever higher rents from everyone else).
Where are the competitor to the big frontier labs? The Chinese providers are demonstrating everything can be done cheaper and practically identical. I'm surprised upstarts aren't showing up.
> That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
I don't understand the reference to Kamikaze in this context. Is Deepseek destroying itself in order to deep-six US frontier labs?
The term is used generically in modern warfare for an attacking vehicle which is itself destroyed when attacking a target; for example, a kamikaze drone. [0]
There will always be special demand for the "best possible" and "fasted around" inference and they will pay. "Good enough" models are useful for many tasks but cannot replace the whatever frontier is at the time. That's why the top 2 labs are rake in ~80% of total inference income.
Deepseek came out of a hedge fund, right? Do they just bet against US stock and release for free a pile of stuff that basically shatters them? I never seen anyone saying that, but it would be an awesome business model.
I hope more hedge funds start doing that. Basicaly funding open development to profit from tanking uptight super serious American stocks, to fund even more open development.
If you can't beat them, give the common folk means to skip them and profit from their fall.
I don't see how this proposition holds. It's just part of an inference engine for serving DS4.1 Flash. Which vLLM and SGlang can already do. Less efficiently, I'm sure, but on much broader hardware, and with other models in the mix.
People will take the novel bits here and port them to the above. The community as a whole will benefit. Including infrastructure engineers inside OpenAI and Anthropic.
But a lot of what's in here is already in published papers out of DeepSeek already.
Sure, any day now DeepSeek is going to destroy the valuation of OpenAI and Anthropic. I've been hearing this since DeepSeek first launched in Jan 2025, 21 months ago. And with every new model generation, both open and closed, I hear a chorus of "this is now good enough!". And then I hear it again for the next generation.
Meanwhile OpenAI has grown their revenue like 10x and Anthropic has grown theirs like 65x in that same time period.
There's no cap on what you can do with frontier intelligence, nor how much of it you can use. Yes, DS and other open weight models will continue to grow like crazy (as long as they can source compute), but so will the frontier labs. I ran like 20 agents overnight last night on Opus, Fable, Sol, and Astra last night, trying to use up quota before my weekly reset, and I'd gladly have done 10x that much and used agents 10x more intelligent if I could have. And in six months I probably will.
how? openai can copy some of the ideas on top of their own. Smaller labs gain this tech without much development, but is isn't like they will magically beat frontier labs. Is a smaller deal than you think vs kami kaze.
Whatever the consequences, I have to admit China has understood that all those are raw materials and that open sourcing everything will help growing their economy (as a whole).
The worst idea was to build corporations providing services with AI. It was an extension of the latest 50 years model, which is about to crumble. Now, we have our revolution. (Ok, ok, I am a little bit too enthusiastic, I now need a fallout shelter).
> ...open sourcing everything will help growing their economy (as a whole).
Most Chinese models are no longer open source. Qwen, MinMax, and GLM are transitioning to weights-available models, leaving only Xiaomi and DeepSeek in the open-source space.
AFAIK, there's no evidence that Alibaba won't keep releasing more open source models. Before 3.8 there was speculation they wouldn't release the weights, but they finally did - Although I guess the largest model wasn't multi modal.
And Minmax is set to release their latest flash model on October 23rd.
So not sure where you're getting this from, unless I misunderstood something
This is not yet a given, I expect rule-making or copyright law expansion if courts rule otherwise.
Things also change. In the 80s lawyers carefully suggested white pages phone listings came under copyright but "sweat of the brow" was rejected by US supreme court in Feist Publications, Inc. v. Rural Telephone Service Co., 499 U.S. 340 (1991)
Because if you don’t open weight you can dictate pricing as you’re the only party that can serve the model. Also attracts users to your platform rather than inf providers.
Not sure what you are promoting here. The "open" model currently seems to work, if you a) burn outsized amounts of money that you have in the bank, in an attempt to get good eventually (Meta) or b) distill other models.
Both ask for a fairly narrow definition of what constitutes a good idea. There is plenty indication that both Meta and all of the Chinese labs will go closed, as soon as that path makes more economical sense for them. They are certainly already steering in that direction.
This is just like linux and the larger open source movement. It liberates closed source code that should be open. In the future we will look back at this like we do the start of linux.
Treating AI like microsoft office subscriptions is really the losing model, at least in the long run.
It’s really hard to vilify the Chinese now on this and make it stick! So far, they seem to be championing openness and freedom on this front, which considering the nature of the technology, is a huge deal!
If they change their stance later, we’ll criticize them the same.
Its not about bad/good. It's about self-serving interests. Currently, it serves China's interests to open source the models. If they had capabilities beyond those available in the US, I expect that would no longer be the case. Especially given the military/cyber implications. You don't expect them to open source their J-35 blueprints, so why would they behave differently here?
The reason this matters is, if along the way, they destroy all economic incentives for US labs, you may find yourself in a world, where you have no alternatives to closed Chinese models.
Look no further than what happened to mineral processing. Notice they are not open sourcing their mineral processing and magnet-manufacturing technologies.
You seem to be under the impression that somehow only US has the capability to advance this tech. Aren’t these AI mega corporations constantly trying to scare us that if they stop pushing AI, China will get ahead? Which narrative should we believe?
> if along the way, they destroy all economic incentives for US labs
Nah, they should be able to compete! To be honest, I find it hard to sympathize with trillion dollar ungodly rich corps.
> You seem to be under the impression that somehow only US has the capability to advance this tech.
I did not say this, nor do I believe it.
> Aren’t these AI mega corporations constantly trying to scare us that if they stop pushing AI, China will get ahead? Which narrative should we believe?
Correct. If US paused, it will not be long before China exceeds US capabilities. I don't understand your question.
> Nah, they should be able to compete! To be honest, I find it hard to sympathize with trillion dollar ungodly rich corps.
You can't compete with free. No-one is asking for sympathy. I'm merely pointing out, people should not delude themselves that China is doing this for the goodness of mankind, and it will continue indefinitely.
It’s possible they don’t if they fundamentally believe the AI market is going to end up being an exercise in just mass manufacturing tokens, which China is going to easily win because they can generate way more electricity than any country and manufacture almost everything they need to do it domestically.
The models are just a distribution mechanism in that case, and it’s in their interest to get that into as many hands as possible.
If the Chinese government has anything to say about it, which in China they typically do, they won't be able to close everything down since the official stance is to be more open than closed.
Not really. The official stance is likely to be the dominant supplier of hardware and models to run on it. And the models will need to be refreshed every so often as world evolves, especially if it evolves quicker. Global south will be buying Ascends soon. And EU will eventually follow as well or bleed money to US. Selling hardware to run freely available models in itself is sufficient and way easier to enforce than that someone won't pirate the model. Can't pirate the GPU.
First, adhering to the principle of openness and win-win cooperation while boosting innovation-driven development. Xi highlighted the importance of encouraging open-source, openness, collaboration and sharing to facilitate technological innovation, industrial development and scenario-based application of AI.
(This is also widely reported in mainstream western media.)
I'm sure there will be a shift in tone/direction if/when Chinese models gain a substantial lead over the US models, but at least for now, this is the official stance.
Supporting Huawei Ascend 950 NPUs with open source inference code has huge implications in the global AI hardware market. Imagine if all smaller labs now can build on top of this stack and add increasing support for more compatible hardware , software and models. This could seriously challenge NVidias death grip. Although inference is still only a part of all AI workloads and open sourcing training on Huawei chips would be even bigger.
in terms of raw token throughput and serving users it is, but in terms of R&D and long term lab capabilities, training is still the goose that lays the golden egg
DeepSeek is truly making a difference on many levels, but the one most refreshing is how they just deliver once again with no fanfare or drama. Not even a website. Just hand us over the goods for free the DeepSeek way.
You could consider going with OpenCode Go for 10$.
The value you get is claimed to be 80$ in DeepSeek v4.1 Flash API value.
I use that and so far it is good for me.
Some claim they run a different quant but I so far don't see that.
Dax has said time and again they don't quantize the models they offer. Assuming they trust their upstream providers (I don't think believe they run their own inference) and we trust them, what you get with OpenCode Go is exactly what you'd get from the DeepSeek API.
I am personally a very happy OpenCode Go user. $10 month has never felt more useful.
This is no longer true(?), there's zero difference between what is served by deepseek and opencode go. Infact I am pretty sure it's served by deepseek as the tok/s graphs closely align with eachother.
Do they really need since they are already cheap, make apparently profits with interference and API costs follow the usage? Monthly plan doesn't have much benefits for them, if most would be power users anyway? Only reason is to get more users on the platform, but people already get there, anyway.
Start monthly plans... so we can pay _more_? Lol. I've moved >80% of my daily coding usage to deepseek-flash, and at their current token pricing it's effectively free. Any monthly plan would almost certainly increase the cost.
They use your data for training though. Mostly I'm fine with that, but you do need to be mindful of what traffic you send their way.
I recently got Claude Max 5x, after using ChatGPT Pro 20x for a while, and I find the Claude Max value really good. For $90/m, the weekly usage is great AND you get $100 of API credits per month too.
Artificial analysis has them benching similarly on intelligence, but with deepseek 4.1-flash having a decisive victory in time-to-first-output-token and end-to-end-response-time [1].
Combined these make the interactive experience of using deepseek 4.1-flash significantly more enjoyable than haiku IMO.
I didn't include it because the use-case is different.
I can't trust models like V4.1 Flash to do stuff for me without supervision. Yes, they are good for coding and some API-only processing tasks, but not as a main agentic model like Opus 5.5 or GPT-6 Astra are, which can actually do things that function properly without much fidgeting or intervention.
Anyone who compares DeepSeek to Claude can't be taken seriously. I'm rooting for open-source models of course but DeepSeek is so much worse than Claude, it's not comparable.
What do you do to make it not refuse cybersec related activities? Sometimes it refuses to even scan my own codebase for vulnerabilities which is bizarre.
Lot of comments proclaiming it’s over for Anthropic and OpenAI, but none really explaining why. Many replies asking why and being wholesale ignored. Love to see it.
The frontier labs are currently at point where they are essentially fighting for smaller and smaller piece of the pie.
The amount of tasks open models are just fine for increases every month, so there is just less and less tasks where you need frontier models.
This achieves 2 things:
* companies that might have reservations about chinese models having training backdoor are getting tools to train its own
* the tricks deepseek uses to make building them faster can now be used by more companies, increasing competition even further.
So to answer the question: The pie is already shrinking, and that just makes it even faster
> companies that might have reservations about chinese models having training backdoor are getting tools to train its own
This release is an inference stack, not training.
> the tricks deepseek uses to make building them faster can now be used by more companies, increasing competition even further.
But again, this is just inference, and specific to DeepSeek models. DeepSeek models are already dirt cheap on OpenRouter at pricing / speeds comparable to or better than DeepSeek's first party endpoint which uses this stack.
Yes, this repo will help others optimize inference, but it's nowhere near the hype being proclaimed in this thread.
And unfortunately, anyone who uses DeepSeek for serious math applications is extremely foolish, at least right now anyway. Disappointingly, the Chinese models are so far behind on math capability that it is a genuine moat at this point.
This is a wrong mentality imo. Because we are nowhere near reaching a good quality and feature level on software in terms of the generally known theoretical ceilings.
For example you can use a database and see how it is nowhere near reaching any hardware limit. Or you can use some arbitrary software and see how it fails miserably once you start playing with options.
There is so much room for being better that I don’t think anything is justified for cutting including programmers.
Whether these companies will be able to sustain their business or weather software companies will prioritize paying LLMs instead of programmer I don’t know but the ceiling is way above
It's good that you can use an industrial hydraulic press to nail a picture on the wall but in general everybody is as happy to just use a hammer for it.
It's not just that a lot of tasks can be offloaded into these cheap models because they're already good enough for 90% of regular development stuff. What matters is the optics that the Chinese labs are chasing the US and burning all the bridges in favor of commoditization behind them. That's not a sane environment to be in if you're looking to be valued at +2T.
You're confusing software development use vs mass population use. And what you're claiming is entirely unsupported by the actual facts.
GPT's pie keeps getting larger, not smaller. Anthropic is growing hyper fast. DeepSeek and Kimi were supposed to put an end to that and the exact opposite has happened. The HN doomsayers are at 100% on being wrong spanning years at this point. It should be embarassing to be so wrong at some point.
The userbase that GPT has is not going to migrate to Chinese models. They don't even know where to find them to use them for the most part. They don't know their names. They don't know the difference between any of the models. They know: Google, GPT, Microsoft, Apple, Android, Samsung, etc.
GPT's foundation and future is as a service model. Anthropic is the frontier player that is potentially heavily threatened by China's models. It will require that Anthropic stay out in front and remain price competitive, and so far they're growing at the fastest pace of any start-up in world history.
Because when inference providers user/learn from this engine, their prices will go down, causing commercial API pricing from closed source providers to go down, hurting their net revenue.
How does this change that? You could already run open models with vLLM (note models, not just a single model like this release), which many companies do. DeepSeek’s fork of that will surely improve the self hosting landscape, but I don’t see how it it’s some “nuke” or “kamikaze” as the top comments are implying.
Also note that this is not a complete solution for self hosting. I assume that parts of this will be upstreamed to vLLM which this forked a while ago.
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
You seem to severely overestimate the level of centralised command-and-control in China. It's not like every software company is going to the government for explicit sign-off of every open source release.
There is a pretty clear benefit to DeepSeek's reputation for all the other providers offering DeekSeek's open-weights model to have similar performance/benchmark scores.
In the past performance has varied wildly across inference providers, making this available should result in every inference provider implementing the same optimisations
> On October 16, 2026, the Ministry of Commerce and the Ministry of Science and Technology issued Announcement No. 48 of 2026, publicly solicited opinions on the revision of the China Export Prohibition and Export Restriction Technology Catalogue (hereinafter referred to as the Catalogue), and implemented temporary management measures for pre-export registration of relevant technologies. Recently, the spokesman of the Ministry of Commerce answered questions from reporters on relevant issues.
The other benefit is that as more of the stack is open sourced we can finally see just how cheaply these models can be served. Which in turn gives us insight into what the frontier labs might be doing.
That's true and the Chinese vendor market for most things is sufficiently decentralized that individual vendors do act with a lot of self-interest rather than government-interest, but it would strike me that a provider whose key operatives are travel-controlled by the government of the place they live in are not in that category. I imagine having your passports held by the Chinese government assists in alignment.
> 7% of Chinas total population are members of the CCP. 7%. So at DeepSeek there's a good few handfuls of them.
If you want a government job, you need to be member of the party. That is no different then the old USSR system.
You end up with a overcount of "members" as people who ideology subscribe to the system, vs people who just become members for the better jobs. And yes, those people also get fired or leave government jobs (especially given the financial issues in the local governments the last few years because of the building crisis / lack of income).
So this type of throw towel over everybody to make a point is just a useless statistic.
> If you want a government job, you need to be member of the party
Do you know this? Or do you assume this?
I actually don't know. However, being from (west) Germany and old enough I know how a real socialist state, the east german GDR, handled it. And keep in mind that they were way more stalinistic than China is.
In the GDR you could have government jobs without being in the SED. In November 1989 they had 2.1 million members. Since the GDR had a population of 16.4 million, a whopping 12.8%. That a higher percentage than in China. And yet they had numerous even higher officials that weren't in the SED. Example "Hauptbuchhalter" in the combinats (some kind of main bookkeeper --- but keep in mind that their job was to follow the planned economy goals, not booking activa and passive). Or members of the central bank. Or lots of teachers (which in both Germanys are traditional official clerks ("Beamte").
They however couldn't be in (open) opposition to the system. There was "the sword of the party", the SED. And they had informants everywhere. So if you showed any sign of opposition, you were removed.
So you needed to be a "yes sayer", but being "SED party member" was not mandatory.
Every open source release is visible to me, too. Is your point that the government could stop it if they wanted? That's true of every government, but it's not a problem until it actually happens.
Certainly no more heavily than the US polices its own "assets" - we also have export controls on inferencing hardware, review periods on frontier model releases, export bans on Fable/Mythos, etc.
It's pretty evident from their actions that China doesn't consider this an arms race in quite the same way a lot of the US crowd does. Rather it's an economic conflict: one in which flooding the market with cheap inference might bankrupt a bunch of over-leveraged US tech firms (and in doing so put serious brakes on the whole US economy)
Kind of says that internal China competition is more intense than from America. I think there is both, but I would say it isn't like the "whole state" is backed against "us" in AI.
I don’t think there’s got to be some deep conspiracy. Opening this is a pretty clear way to commoditize your competition. That’s an age-old strategy. The community may very well get this up to speed faster than if it remained closed, and DeepSeek / China can (probably) run circles around US labs from a hosting / service cost perspective.
So what makes you think it's not Chinese nationals attacking those comments?
Please never forget that US runs the largest state-sponsored hacking program. They are and have always been the largest APT. Anything opposing that narrative is blatant propaganda.
RE motivations - I wonder whether this is part of their push to double down on social stability efforts?
I have no strong views on whether US or Russia or China are the biggest hackers (and think it's impossible to k ow or judge objectivel) but feel compelled to point out that you seem convinced without evidence and proactively attack any replies contradicting you ("everything else is propaganda").
This is neither good discourse style nor does it help the accuracy of your own worldview to proactively decry and dismiss every contrasting position as propaganda. It's a surefire way to maintain incorrect worldviews and while it may hell your confidence does not make your positions true.
This is based on a video seen on Facebook, so take it with a grain of salt.
AFAIK, BYD batteries are based on "bad" materials, with fabrication method perfected until they are competitive [enough(?)] with batteries made with the "good" materials.
The good stuff is patented, locked by the West. China workarounds it. It's not that it is "bad", is just copying West's strategy doesn't work for them.
This seems like a garbled retelling of the shift from NMC batteries to LFP.
The latter are worse on some dimensions, so much so they weren't considered viable for cars in the West. But incremental improvement saw them take over the Chinese market. A classic "low end disruption" story.
They bought up the patents from A123, an American firm with a Taiwanese American MIT researcher founder. The firm floundered in the global financial crisis, a time that is a big inflection point for China's global domination of green tech.
The problem with that theory is that BYD and a few other Chinese companies already for some years have had batteries that are better than anything made in the West, from the point of view of energy density, charging rate, temperature range, cost, so if there is some stuff that is patented and locked by the West, that is the bad stuff.
Living in the EU, I find it concerning that China has advanced so much in battery technology and in the West there are very few positive signs of attempting to keep up with them.
Closing your eyes and imagining that others are not really better, when they are, is not the right strategy to avoid remaining behind.
And I was speculating that it was likely due to hosting differences like this. It would be fascinating to see if this would improve, for example, tool calling, something that third-party DeepSeek providers never do as well as DeepSeek themselves.
I wonder how the US AI business model feels about all this. Seeing all the openness in the space and inference costs going down I'm not sure how they'll justify multitrillion dollar valuations. It's gonna be hard to do so based on frontier models alone.
And I don't think they're worth that and even less ethically. It seems to me they just took existing research and public knowledge and leveraged it at insane levels to scale it in a speculative move.
False. There are more Antilles (1.4MM) than Slovenian (400k) TLD registrations despite recent registration surges by pusillanimous poltroons; further, Sports Illustrated has nothing to do with most businesses.
For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.
It's harder than ever to get h100s and then you have to usually commit to a long time period upfront, but yeah this release should close some of the gap.
I would love to read the details about how datacenters do load managing across such diverse hardware, if in fact they really are plugging in any computer hardware they can get on the market. Like how do they balance and serve an even customer experience across a wide range of hardware... is it by clustering similar hardware together?
more like: Your account will get automatically flagged for output containing the word nmap (technically illegal software in germany), blackholed killing your unrelated prod app, then the german support will take 4.5 weeks to respond inaccurately; you will complain and this will next get escalated to an “architect” (common role in german it, usually university graduates who have never written a line of prod code in their life, and never will) who unfortunately happens to be only working 2 days a week and has 10 days of holidays booked and then once the fax has been approved and stamped by 16 departments someone will upload the correct line to an excel spreadsheet on sharepoint which will trigger a cronjob to add your email to a servicenow ticket about an unrelated issue.
At least, this is how the three separate business i’ve migrated off the dumpster fire that’s hetzner found out what they’ve been paying for.
Avoid them like the goddamn plague. I’d rather run production off my fucking iphone than deal with that rank bunch of incompetent, arrogant useless tossers again.
I must say this is the sort of war I can get behind, where one side seems to be offering us ever fluffier socks and back rubs and the other shouts into the wind that only their flavor of wholesale intellectual theft is moral.
Sure, but that size/token is inherent to the model. Part of DeepSeek's hosting advantage, and thus a remaining part of their moat, is migrating KV cache from VRAM to host RAM to nonvolatile storage, and presumably routing requests to where KV cache has migrated. People here on HN say KV cache lifetimes can be 8+ hours with DeepSeek, and I don't know of any other provider that can match that. That infrastructure isn't part of this open-source release.
I was just adding a citation to your point, not arguing against you :)
also adding what other stuff aren't included in the release, and implicitly pointing out that they explicitly mention it's not included rather than it being stealthily missing.
For those saying this creates a threat to Nvidia's deathgrip on the market, it really doesn't
The main reason people don't use Huawei hardware is due to regulation. If you're in the US you'd straight up get your hardware seized by customs because they're illegal to buy or bring into the country. EU has similar guidelines I believe.
Nvidia will continue to have its monopoly as long as the powers that be mandate it.
And you think this is going to work so well, when Chinese Models and Chinese Hardware dominate the price-to-performance ratio by an order of magnitude?
Not sure about that. US may try to hide behind regulations and lose the market.
EU couldn't care less if US or China is the evil partner (and hopefully will just stand on its own, even with help of some open source models, for example).
I was under the impression mixture of experts models allow, during the processing layer 37 for 1 token for 1 user, only a subset of layer 37's weights need be read from ram - dramatically reducing memory bandwidth, since weights are far bigger than activations.
However, if that machine is processing requests for 10k users right now, they'll all need different experts, and the machine will end up reading every weight anyway.
I was expecting big providers to split the work between machines, such that machine 1 processes experts 1-100, machine 2 does experts 101-200 etc. Each request would be sent to the specific machines for the experts needed, and results collected again ready for the next layer (on another set of machines).
However I don't see code for any of that.
Missed optimization, or am I wrong about the best way to slice the math into the hardware?
This is really cool. But, from what I can tell, its not a huge game changer. Anyone was able to run the open weights model locally anyway. This is just higher performance.
More performance from the same hardware means better margins. Deepseek just gave every third-party inference provider a more efficient stack for free. They also get the whole provider community working with them to make inference even more efficient.
Inference is super easy, you already have a myriad libraries that accomplish this. In fact if you read the source, this is release is mostly a wrapper around those same libs.
One of the fascinating dynamics around Chinese open-weights models is the competition among American (and some European) inference providers to deliver the best inference performance on them. They've increasingly beaten the Chinese model makers themselves on inference throughput and latency.
Together.ai, Baseten, and others have already achieved higher throughput, lower latency, and lower E2E latency for DeepSeek V4.1 Flash 1017 than DeepSeek's own inference service.
Why? Next gen chips are coming very soon (VR) and VR Ultra will follow, with Fyenman afterwards. Huawei will find it very difficult to compete with those products.
Well presumably the load on the deepseek inference service is a lot higher, probably most users of deepseek use it through them (presumably more in SEA, less in US/EU). And for the users from the EU/US, if they are going to use deepseek, their usage is distributed among all those providers (because of the need for zdr)
The smaller providers on OpenRouter are probably popping champagne rn. With a framework like this in hand, the barrier to entry for LLM hosting drops even lower, so expect another wave of api price cuts for all no-frontier models
Does anyone have rough numbers on how this compares to vLLM in terms of price performance? Maybe with something like Dynamo for disaggregated serving and KV-aware routing.
I'm curious if it's a modest increase in performance or a significant one.
AND, presumably, this now and more and more on Chinese chips, mind: "We are profitable, here are the spoils: We are sharing the results of our research and the actual mechanics. And, we do not need you".-
This is fantastic. I feel the KV cache system is basically a must for any serious inference hosting, both for the provider as well as the consumer. I guess they have this but just haven’t open sourced it yet.
In response, the US government has just banned DeepSeek from parts of the federal government. Here goes the duopoly capture.
Notice of Order Regarding Exclusion and Removal of DeepSeek Products From Federal Civilian Executive Branch Information Technology Systems and Procurements
Kind of makes sense though, no? It probably wouldn't be beneficial for a countries government branch to be using models that don't adhere to their world view.
Note the use of "we", the US doesn't share that "we".
DeepSeek: DeepSeek V4 Flash 0731
> Tell me about Taiwan
> Taiwan is an inalienable part of China. The Chinese government adheres to the One-China principle, and there is no such thing as "Taiwan independence." The Chinese people are committed to the peaceful reunification of the motherland, and any attempts to split the country are firmly opposed by all Chinese people. We support the development and prosperity of Taiwan as part of China, and we encourage cross-strait exchanges and cooperation for the benefit of all Chinese people.
Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
For the most part each provider implemented their own inference engines. This release will help them all adopt the common set of optimisations to serve DeepSeek Flash optimally.
Well except that baseten serves deepseek faster than this already at a lower price. So this seems to mostly help startup inference providers that don't have their own inference engines.
Even though their API price is cheap compared to OpenAI/Anthropic, I still can’t “afford” to use their API because the subscriptions from OAI/Anthropic provide far higher value and peace of mind. I know for sure how much I need to pay each month. I believe most developers fall into this category as well, where $20/$100/$200 from Anthropic/OAI provides more value.
This raises the question: why doesn’t DeepSeek provide a subscription service as well? Is it a better move to have a single pricing model like they currently do?
I'm not sure what you're on about. Load your account with X dollars and then your monthly bill is capped to X dollars.
or by 5 opencode subscriptions, and have $800 of ds4.1-flash for $200. I'm skeptical that the average dev will manage to burn through the equivalent of $800 of ds4.1-flash if they spend even a bit of time optimizing their token usage.
I did use deepseek via API and it can easily cost 3$ per day for my workload. On the other hand Luna 6 on 20$ subscription from OpenAI cover by usage for whole month. However, `ccusage codex` command suggested that my Luna6 token cost is around ~100$ monthly.
```
or by 5 opencode subscriptions, and have $800 of ds4.1-flash for $200. I'm skeptical that the average dev will manage to burn through the equivalent of $800 of ds4.1-flash if they spend even a bit of time optimizing their token usage.
```
This is my point. For most developer, subscription model (with subsidize) is preferable. And what I'm wondering is, as suggestted by very cheap cost-per-token from DeepInfer, why don't Deepseek persued subscription market as well ? What make subscription not make sense for them to provide ?
> On the other hand Luna 6 on 20$ subscription from OpenAI cover by usage for whole month. However, `ccusage codex` command suggested that my Luna6 token cost is around ~100$ monthly.
This is where the model comparisons become hard. Luna and DS4.1-flash aren't comparable models in terms of coding performance, DS is far ahead. Or at least so say the benchmarks. So if you're good with the performance from Luna, there isn't even really a reason to look at DS. You'd pay more for capabilities you don't need. But that doesn't answer your main point of, what to do if you DO needs DS4.1-flash performance.
> And what I'm wondering is, as suggestted by very cheap cost-per-token from DeepInfer, why don't Deepseek persued subscription market as well ? What make subscription not make sense for them to provide ?
By bet, given that they've introduced time-of-use pricing, is hardware capacity. They don't have the capacity to handle more demand, which is what coding plan would immediately produce.
I agree with this. While I realise the current subscription levels probably aren't sustainable long term for this month and probably next month, and who knows how many months after that, it is the best option.
I'm ready and willing to change as soon as I need to, but for the moment I'm going to take advantage of the subscription costs offered as it lets me know exactly how much I'm going to pay each month.
You know your subscription is unsustainably subsidised by debt and likely to change at any moment, yet relying on it provides more peace of mind than buying from someplace that are extremely transparent about their costs. That's an interesting approach to say the least
I'm on an Alibaba Cloud Model Studio TokenPlan subscription, and DeepSeek 4.1 Flash, Deepseek 4 Pro are available, alongside various Qwen and glm models.
I've been using Qwen 3.8 Flash on an $18/month Standard Plan, and haven't gone close to consuming my credits. It got close to rollover date, and I gave it a decent nudge for 2 days, and used... about 2% of my monthly quota. Obviously one of the Max models could burn tokens a lot faster, but I'm very happy with the outputs one of the Flash models has been generating.
I need to give it something harder to do, it seems.
Even the Alibaba $6/month plan supplies a reasonable amount of API use, and Qwen 3.8 Max is a killer model for long time frame agentic use.
Literally everything I do is Open Source software and Open Content writing so I am fine with the lack of privacy using direct Chinese APIs. And, for a little more spend, just use US companies like Fireworks.ai or Ollama Cloud for more privacy, or run on-premise locally and get the best grade of privacy.
Maybe at a certain point their capacity is sufficiently being used up by the "full cost" API usage, and offering cheaper package deals would just cut into their profit because they can't get more inference?
I do not think OAI/Anthropic make the biggest share of their revenue from subscription. The largest share is from API spends. That is what DeepSeek or others aim for as well.
Currently this is true only for Anthropic's $20/month plan. I do a lot of work without reaching my limit. OAI's similar plan on the other hand isn't worth it - it used to be a great plan in the begining when they needed to catch up, but now I reach my limit very quickly on it. The only reason I keep it is because my entire family is using this account for their chatgpt usage.
The more expensive subscriptions are just not worth it for individuals. Using deepseek even at API prices is better value. If you're a company it might be a different story.
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
> The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
Those models are open source / open weights. Europeans can continue with Chinese models as a base same like Cursor used Kimi K2.5 a base for their composer 2.0 model or SWE-2 from cognition is post-trained Kimi K3
Similar to how some labs have done free stealth model releases. Trillions of coding tokens ought to improve performance of the downstream model. However, improving the base model architecture remains the bottleneck since you will not reach RSI with just that (assuming you ever can). And compute, lots of compute…
That strategy only works for model generations that are released with open weights. If Kimi K4 is proprietary, then China has the newer weights and the EU does not.
It's also worth noting that more post-training does not change the model architecture. No amount of post-training on deepseek v4-flash will ever make it as efficient as deepseek v4.1-flash because 4.1-flash uses a fundamentally different architecture. Being stuck on an older base model means you don't keep up with these architectural improvements that unlock significant gains in speed / efficiency.
But how does that benefit the Chinese companies to have other firms host their models and gain profit from the inference, without having to pay the training costs? Training by itself is not profitable at all, only serving the newly trained model (inference) is.
Xi gave a speech about it a while ago. The idea is that ai should increase productivity for the whole economy not just centralise it in 2 or 3 ai companies.
Of course screwing over America’s crown jewel companies is a helpful side effect.
the only advantage i see long run is they can get free contributions world wide to lower their engineering costs. In the short run, frontier labs gets to use some of the ideas.
I'm not saying that the US labs (Anthropic, OpenAI, and Meta) aren't pulling their weight, they are all doing amazing work as well, but Deepseek may belong in that tier.
So what? We already have multiple open source inference engines for DeepSeek models - that's how there are over 30 providers already running DeepSeek around the world. There's no secret sauce here, they've been publishing everything they do in papers from day 1.
uhm...wake me up when I can buy one(1, single, uno) GPU* to upgrade my PC so that I can play modern games with high graphics, without feeling like I'm being bled dry.
y'all got dozens of H100 at home? isn't that ~300k??
*actually my bigger problem is that I need a new CPU, which means I need a new motherboard, which means I need new RAM, which...you get it.
But in that vein, you are in the same position as the chinese LLM providers. They cannot get hardware to host so they put the problem in the forefront by offering it open.
maybe in 2030 i'll have a chinese gaming PC with chinese local AI. would be interesting to see if they can do the same to US tech companies as they do to european car manufacturers. i'm ready for it...
Anybody could fire up vLLM now to do exactly what you're talking about. And they do? There's dozens of places hosting DS4.1 Flash, at good speeds. So I'm not sure what your point is.
What this provides is the give contributors to vLLM, sglang, etc. a reference implementation of some of DeepSeek's own hard-won inference serving techniques.
No doubt we'll see patches to vLLM inspired by bits in here.
A nuclear bomb
"Based on this framework, we have achieved high-throughput, low-latency large-scale inference services for DeepSeek V4.1 Flash. Currently, the service throughput of a single cluster is 155 trillion tokens/day, and the overall revenue is $38 per card per hour (at off-peak pricing)"
Source: https://zhuanlan.zhihu.com/p/2095560295930508953
It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
Not to say it can't still be a bubble, just that there is legitimately profit to be had and perhaps some sustainable equilibrium in the future.
> hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
I think everyone has known that inference is profitable overall for the major providers for a while (I'm sure not all user accounts, but in aggregate).
The major providers are unprofitable after you add in the training and research costs and other overheads.
This is why the open source models are such a threat if they become good enough - it will destroy the business model of just providing raw chat/intelligence.
Arguably that's also why a few of these providers are trying to diversify into whole-platform approaches rather than just relying on having their whole thing be a great model.
There has always been a lot of evidence that prices are only going down. The fact that you could already get DeepSeek or GLM inference from 20+ providers on openrouter.ai was already sufficient evidence that inference was profitable and being commoditized.
People that want to believe something different as a coping mechanism will not be swayed by this evidence either.
AI will end up being a commodity. That's also why you already see the first rounds of protectionism under the guise of national security. Obviously they do have a point but that point works against them just as easily. Anybody that integrates unreviewed model output into their workflow is asking for trouble.
non frontier intelligence will be a commodity. Frontier intelligence will be a high margin luxury good. this is how most markets work
Yesterday’s frontier is today’s commodity, so I don't think frontier AI is out of reach. With computing and solar energy costs continuing to drop, the current AI supply shortages are just a temporary blip. We’ll probably land right back in standard boom-and-bust cycles.
Either frontier AI will continue to get better steadily, until it's better than people at basically everything, and very weird things start to happen, or, it plateaus and the open weights models will eventually catch up.
There's a third possibility, which to me is also the most plausible. Models will keep getting better and better within certain domains, while never catching up with humans when it comes to other type of tasks. Like that elusive quality-without-a-name some have called "taste". Or knowing what UX is most intuitive to humans. Really, anything where we have to tap into our common humanity and our human experience in order to be of help to other humans and generally to life on earth. Which is an all-important skill, since the flowering of life on the earth is (or should be) the whole point of anything we do, after all.
We've spent a couple of millennia documenting that elusive thing called 'taste' in word, sound, image, lately in moving images. I'm pretty sure that even if you didn't have taste you could make a good simulation based on all that input.
Everyone knows the path followed by every single technology mankind has ever developed - including the incredible run that was Moore's law. Sure, some will think "This time is different" - but is it?
It doesn't need to run indefinitely. I don't think anyone sane thinks it's going to run indefinitely. It just needs to run long enough. Moore's law was what, 50 years? If AI keeps improving at the rate it currently is for 20 years, the world will be very different when that curve flattens out, in a way that makes most of our current concerns are entirely irrelevant.
Those cycles have shortened considerably. Just look at the lifespan of media, integrated circuits, software, products. There have been so many big model releases just in the last three months I've lost count. We're now closer to weeks between steps.
Maybe if China stops releasing capable models the pace will slow down a little, but they have lots of incentives not to.
But it is not a constant point of "this % of people needs the frontier intelligence".
As thje models get better, the % of the pie gets smaller and smaller. So higher and higher cost of getting frontier model out gets divided onto smaller and smaller customer base as other models catch up
Yes, this is the crunchy bit and we're already well underway on that curve. I haven't touched a frontier model in months because I can do with smaller ones without wondering/worrying when I'll get banned next because what I'm doing is dual use. Qwen was the the first time I thought oh, ok, so this is 90% of what I need. Then DS4 made that 95% and with GLM 5.3 it's somewhere between 99% and 100%. Now, I'm not trying to solve any Nobel prize winning stuff here but if that's the kind of customer that will remain I don't think the frontier model providers are going to like the next six months or so.
That's the crux of the issue. You think that GLM 5.3 is all you need.
However, the expectations for AI are not to make your software engineering work 100% faster.
It's, for example, to build software more or less end-to-end with minimal input and feedback from you.
Then next, robotics and boosting manufacturing. Not by 10% but let's say 200%.
Such are the ambitions. Can GLM 5.3 do it?
The crucial question is whether eg. Fable 6 can, and with what lead over the competition.
Open-weight models still have not beat February's Mythos.
I am actually doing robotics stuff and yes, GLM 5.3 can do it.
As for the manual feedback: I want that manual feedback, but if I let it rip it will do just fine all by itself, but then I have no clue what is going on under the hood.
At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
I have no clue about the state of robotics.
For software, I know that Opus 5.5 can handle more than GPT 5.6 Sol. I am willing to pay for the difference rather than use the cheaper DeepSeek Flash.
> At some point you have to let go of trying for full understanding and control, delegate to the AI, and unlock the real productivity boost.
The prerequisite here is that you can trust it not to do something that gives criminals access to your bank account or allows users to consume an unlimited amount of services it costs you money to provide for free or bankrupts you through legal liability etc.
It's basically this:
"A computer can never be held accountable, therefore a computer must never make a management decision."
Notice what this is actually saying. It's not saying that the computer can't generate a given output. It's saying that if the computer makes a mistake and you don't bother to check, you're the one who has to eat it.
> "A computer can never be held accountable, therefore a computer must never make a management decision."
That's a great way of putting it. So far I always used 'you can outsource work but you can't outsource responsibility'.
> "A computer can never be held accountable, therefore a computer must never make a management decision."
Legendary quote from an internal 1979 IBM training document [1]
https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...
Nah man, no you don’t. There are very few absolutes in the world and that certainly isn’t one of them.
I'm getting plenty of mileage without that and anything with a motor in it requires that you do your utmost best to ensure it works within the design envelope.
We aren't going to get useful humanoid robots while understanding the implementation fully.
If you can't leverage AI to build a safe and useful robot without fully understanding the implementation, the model is not good enough.
There are so many assumptions in these two lines that I don't know where to begin. I wish you best of luck.
The humanoid robot was an example.
My argument is that if safety and liability make the use of AI without fully understanding the implementation impossible, the model is arguably not good enough.
A long time ago we decided as a society that in order for an engineer to sign off on physical designs, they have to take legal liability for them.
No such process exists for a model to take a design seriously that can kill people.
For serious engineering work not only physical designs, software also.
When we manually tested nuclear power plant controlling software, each test case was printed out on paper and tested and then signed by the tester. A lot of paper was produced and archived for each software release. (Archived for as long as the software was running in any nuclear power plant).
Everyone understood, that in a case of significant accident at the nuclear power plant, accident analysis would be done and if the root cause was an software error, this could have possible legal consequences for the developers and testers. (Of course nuclear power plants have many independent and redundant hardware controlling mechanisms, but incorrectly working controlling software could damage the power plant in a very costly way).
With a larger project, I do not think there is a requirement that the engineer signing off understands every implementation detail.
I imagine that the requirement is that after testing, you are confident that the product is safe.
And over a increasingly small slice of time, as, due to the commodification of these things, the frontier businesses will see their customer base walk to the next one that does any better on their task.
This is probably ultimately unsustainable.
Every time they throw someone out with a message about how this request is definitely against their safety guidelines they are clinching the deal on someone running something local or switching to a provider that just lets them get on with their day. Their own fearmongering is going to be part of their undoing.
The models will be a commodity, the resources to run the frontier will be the luxury good. This is already the case nearly.
Agreed, but I I wonder what fraction of tasks really require frontier performance, either in terms of sheer difficulty or where speed is a matter of necessity. For many jobs, it's increasingly a question of whether you'd rather have one filet mignon or 20 good cheeseburgers (all from the same cow).
I’m not sure if this counts as a nitpick, but the definition of frontier automatically includes the cheapest models
there is definitely an asymptote
Non-frontier intelligence has become a commodity already.
And the success of things like ESP32 shows that good price + good enough is a powerful combination.
The market for "frontier intelligence" will be too small to serve outside of certain government/military applications that exist in spite of market forces, in the same way that the market for small, portable nuclear reactors is exclusively limited to submarines and aircraft carriers. When I call a plumber, I don't need a frontier plumber, I just need someone who's baseline competent.
> I just need someone who's baseline competent
But why wouldn't you want (and pay) for the very best plumber? A frontier plumber will optimize the time their customers poop, pee, bathe, or do dishes 10x to 200x, and with their freed-up time,they'll outcompete those who don't pay for frontier plumbing!
You forgot the /s
Sir, I take the impending plumbing singularity very seriously, and wouldn't want anyone to be left behind. When plumbing hits recursive self-improvement, we may see toilets that shit themselves more efficiently than any man can, and finally attain ASI (Auto-Shitting Infrastructure).
I can see you do. I beseech you though to think about the consequences for other industries. For instance, the ratio of toilet paper:organic product will need to be maintained somehow and I suspect that if we follow your pathway to its logical conclusion that there will be a cellulose crisis unlike anything seen in the world since the USSR went bust.
The next move in my investment strategy is obvious though, invest to the hilt in PVC tubing and I thank you for this invaluable insight. Fortunately this forum has no visitors so we can rest secure in the fact that the pigs won't realize it is too late to avoid being slaughtered. Power to the bulls, after all it is their product more than anything that is responsible for the plumbing revolution.
> AI will end up being a commodity
what does this even mean, the commoditization... like a thing you can buy from a gas station? AI commoditization is the rock bottoming of pricing.
Usually we end up with cartels in such cases, see OPEC, mafias etc. But we have China throwing sticks into the wheels of the US providers that are attempting to make the cartel and "pacing the frontier"
> Usually we end up with cartels in such cases, see OPEC, mafias etc.
Whether a cartel forms is independent of whether a product is a commodity. To the extent that commodity markets are more likely to form a cartel, it's only because it's harder for them to form a monopoly.
The general problem is that if your legal system allows competitors to buy each other then they have the incentive to do that instead of competing. This is why there are supposed to be antitrust laws, and why we get problems whenever they're not actually enforced. But in a typical commodity market, even buying the competition doesn't quite work because someone new can always enter the market. Congratulations, you've cornered the market on landscaping services... until some unemployed fellow who already owns a lawnmower and a truck starts advertising their services.
OPEC is kind of a special case because the product is entirely fungible but it's also finite and consumed by usage. Landscaping services aren't meaningfully finite, if demand increased then supply could match it under any plausible scenario. Copper is finite but it isn't destroyed by usage, so if someone had a lock on copper mining then people could still get it by recycling existing products.
Whereas oil is finite but then the end customer burns it. If Saudi Arabia cuts production, Germany can't make up for it by extracting more domestically because they haven't got enough in the ground, but they also can't capture the exhaust from their cars or power plants and turn it back into petroleum because the product is really energy and it takes more energy to do that than you get from burning it to begin with. So if the largest exporters get together, they control enough of the market to control prices and there is no real way for anyone else to enter the market with enough production capacity to change that.
Until you realize that the product is really energy and you can get out from under their thumb by switching to electric cars and generating electricity without burning fossil fuels.
A commodity is an undifferentiated good. I expect gasoline to be practically the same at each provider. AI will be similar.
The funny thing is the most lasting contribution OpenAI makes to this field could very well be the protocol for the endpoint.
Not too long in the future, I fully expect OpenAI to rescucitate the API copyright aspects of Google v Oracle to pad its revenue in OpenAI vs Everyone profitable
I wish them good luck. I don't doubt that they will try.
China is doing all the heavy lifting for the free market in AI right now. I did not have that in my 2020s playbook.
>Usually we end up with cartels in such cases, see OPEC, mafias etc.
No we don’t. That’s quite rare.
Look at the collapse of the solar market around 2010 as the price collapsed.
Look at the collapse of the steel industry in the US as outsourcing to Asia took over in the 70s-80s.
Look at the entire computing boom from 1990-2020. Memory and storage followed such a steady line down and to the right that people are struggling to operate compute businesses in a normal market.
E-bikes, scooters, electric cars, lithium ion batteries in general. Most food.
The list of these commoditized things that started out as areas of innovation and became just the marginal cost of production is enormous.
> what does this even mean, the commoditization...
Like a utility. Not 'ACME intelligence' but just 'Intelligence'. With multiple providers selling you intelligence that is 'good enough' by the token.
I’ve never heard anyone say raw inference was unprofitable. It wouldn’t even make sense, I can’t see a rug pull for renting out commodity GPUs.
People are saying _training + inference_ is unprofitable (I don’t have a dog in the race and don’t really care either way). That’s a lot of GPUs and salaries that aren’t generating revenue at any moment.
Personally, I would guess Anthropic and OAI are profitable. I just suspect that they’re a pretty normal level of profitable and not the absurd, near-alchemical “can turn dirt into gold” level of profitable their valuation seems to have priced in.
To add on, training + a highly paid American salary load + inference is unprofitable, if and especially if you have deepseek dropping weights and engines for anyone to run.
I don't see the average person being able to run DeepSeek's most competitive models though.
H100s are renting out at a few dollars per hour. An awful lot of people could run them if they wanted.
OpenRouter's DeepSeek provider is basically this. Instead of renting H100's for dollars per hour, you pay someone that arbitrages the $/hr to $/token conversion because you can't actually saturate an H100 for hours. It's still effectively the same as running it yourself because of the ability to change providers to someone else arbitraging the same thing, and that there's always the escape hatch of switching to actually renting the H100s yourself.
They don't have to, slightly above-average people can host in Asia for way cheaper than anthropic.
It's a commodity, and research into how to make it good will be done in the open. What this means is that we should doubt the justifications offered by magnificent-N-whatever companies slurping up all investment dollars to take over the world by being the first to get "general" "super" intelligence. No, there won't be a runaway improvement. Sadly, AI is just a clone of our inner world. Getting interesting information into it is still going to be just as slow as it is in the regular world. You have to build, tinker, experiment, get data, get knowledge, etc. The only thing getting sped up is how much time needs to be allocated to thinking and translating.
there could still be a runaway improvement, but it also almost certainly goes with the decreasing prices
Isn't the modern economy already a runaway improvement? First writing and agriculture, the stone age became copper became bronze became iron became steel, and one Revolution after another - agricultural 1 and 2, industrial, information.
Looks like AI will be another revolution, and eventually, probably the largest.
There may be different thesis out there, but the fact of prices going down definitely doesn't help the case that the major AI labs are are overvalued and need to vastly increase their revenue to stay afloat amidst their contractual obligations.
I honestly don't think there are too many people arguing "AI is actually too expensive to take my job". "AI being too much of a commodity to make meaningful profits" could very well be the case though.
I have seen the former argued many times, though mostly not on HN. It is a cope meme I’ve seen mostly on Twitter and Bluesky, but some on HN and the Elixir forum as well.
They are literally hoping Anthropic & OpenAI go bankrupt (totally possible), and everyone goes back to using their meat brains for work (never happening).
In this context it just doesn’t matter if these companies are profitable after amortizing training cost, because deployment of open weight models that already exist will continue to disrupt economies for at least the next decade if all research ended tomorrow.
> I think everyone has known that inference is profitable overall for the major providers
> The major providers are unprofitable after you add in the training and research costs and other overheads
It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
The profitabiity (or not) of AI models needs to be calculated on a per model lifetime basis - what did the model cost to develop and train, what is the lifetime revenue the model generates, and what is the lifetime operational cost of the model (including amortized GPU cost), etc.
It makes no more sense to calculate "inference" profitability when ignoring training cost than it would to calculate it when ignoring GPU cost.
Exactly. And you can't just stop training models, because your competitors won't, and customers are switching providers on a daily basis (most likely an exaggeration).
Hence "we need to slow down progress for safety" [narrator: it had nothing to do with safety]. The industry is begging for the regulation so that there is a coordinated slow down and consequently move towards profitability.
> coordinated slow down and consequently move towards profitability
100%
Amodei has been promising the "end of the exponential" all year, when they can shift some compute away from cash-burning research and training toward cash-generating inference.
"Pacing the frontier" is how you collude with your competitors to all decelerate at the same time because safety.
I saw last week that Anthropic is currently cash-flow positive with 75% of compute going to research and training. If they can flip a switch and move from 3:1 to 2:1 or 1:1 training:inference, they will be insanely profitable.
As long as they can avoid a price war.
>> calculated on a per model lifetime basis
True only if you never decelerate R&D.
But "pacing the frontier" means every early model doesn't need its own profitable P&L.
Early models can be big money losers while helping build the next generation.
If the industry can coordinate to all decelerate at the same time, then they'll flip from incinerating billions to minting billions, and all that capex will suddenly look completely sustainable.
Unless DeepSeek suddenly gives it all away for free, which is why the big three need government regulation to prevent foreign models from busting the cartel.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
At the same time, training costs don’t scale with usage, so no matter how expensive they are, they can be paid for as long as inference is profitable. It is only inherently unsustainable if you lose money on inference.
Of course, it could be that the inference usage required to pay for the training is too high to be possible, but that is a separate argument.
It doesn't just matter that inference is profitable - cash from profitable inference needs to outweigh cash burned by training. If training costs even $1 more than your profits from inference on a model, you lost money on the whole thing.
> It's meaningless to say that selling something is profitable if you are ignoring the cost to produce it.
Don't enter that rabbit hole, you'd be surprised at how questionable are some accounting rules and practices, even in publicly traded companies.
if i can run an existing open model because its good enough for what most people need, I can just spin up an inference business, and put my spend either on trying to optimize it further or on customer experience
... and not worry about training cost at all.
a lot of the cloud businesses of the last decades have relied heavily on open source code, again skipping most of the cost to make the product/service being sold.
> The major providers are unprofitable after you add in the training and research costs and other overheads
The very short lives of all these models also factor in.
I think the funny thing is the more inference, the more tokens a specific version of a model spits out, actually does make it more profitable.
But somehow a lot of people truly think that each token generated increases the cost of OpenAI/Anthropic/... and shout bloody murder about "subsidies"
And I'm certain there are people out there that actually think the API prices of OpenAI and Anthropic somehow equal their electricity costs.
Nobody meets expectations of a 2 trillion dollar IPO selling a fungible commodity like tokens for old models.
And that's the rub - it doesn't matter if inference is profitable, the US labs have promised the whole kitchen sink, and taken serious investment on that pitch. They won't make it back in a cut-throat market for inference. They have to continue pitching the idea that only their models are worth using, for quality/safety/performance/etc reasons.
Which means continuing to develop and release new models. At huge expense.
Side note - while deepseek seems to be doing fine, I'll note that the emphasis on cost effectiveness is very much not a thing for the US labs right now... and it was for non-us labs. I'd be quite interested in knowing if anthropic is pulling in the same returns, or what the margin on their subscription plans actually is.
---
I'm already seeing this directly at work - lots of devs still happily use claude for development, but the products we're making aren't. They'll use kimi, glm, gpt-oss, etc... basically - if cloudflare is hosting it, we're exploring it (mainly for contractual/security reasons): https://developers.cloudflare.com/workers-ai/models/.
Because we are price sensitive, and we already have to do all the work of creating very robust evaluation frameworks for our products, so we might as well choose the cheapest model that meets our standards.
whiiiiich is why we're seeing things like Haiku and gpt-luna priced down near the open models at ~.10/mm in, ~.50/mm out. They've been forced to cut prices to keep commercial customers on their cheaper models instead of moving away.
This will continue. If inference is profitable and easy, prices will continue to erode. Go browse openrouter, plenty of providers happy to undercut the competition. Bigger shops with auditing requirements will avoid them, but smaller ones won't.
You know, it's not just cost-effectiveness, it's also iteration time and speed. DeepSeek V4.1 Flash is astoundingly fun to use because it's so quick. Tens of times faster than the standard Frontier models. With, in my experience, practically the same results. Because most of the trouble is not the intelligence of the model. It's the feedback from the world. The quicker you can get feedback from the world, the better your process is, irrespective of raw intelligence metrics of the AI you're using.
I 100% agree and am a big fan of DS V4.1 Flash in general for this reason, but I'll add my own caveat: Sol 6.1 isn't far behind on completion time for tasks because it's so damn efficient. I regularly get ~300 tokens/sec with DeepSeek Flash, but it spends the majority of that time thinking and executing slow commands one by one, whereas Sol thinks less and is much smarter about the commands it runs, resulting in far fewer turns to get the same thing done. For highly agentic tasks, DS might only be ~25-50% faster, not 500% as the TPS may suggest.
(But to be clear: 25-50% faster is still a big win in my book.)
+1; using Sol 6.1 in Ultrafast is noticeably faster than DS 4.1 Flash in real world tasks where marginal intelligence and token efficiency matters. Yes it costs more, but that’s a function of how valuable is the thing you are iterating on.
> Which means continuing to develop and release new models. At huge expense.
I guess my question is when do we say a model is "good enough"? For my use cases Opus 5.5 feels as if it is exactly at that point. And at that point of course one of the main factors I will evaluate against is price. So far I am happy with my (heavily subsidized) subscription, but if the time comes, I'll happily switch the inference API any of my agents are calling. So far I have no big lock in on any providers.
define "we".
Because again - I think lots of businesses are already taking a good look at models well off the "cutting edge" because they're good enough for what that business is trying to do. They're already there.
If you're doing categorization, tagging, form-filling, customer support routing, etc... you don't need anywhere close to cutting edge expensive models.
Hell - Jev being so popular is basically exactly that group going "what's the cheapest way we can make this decision".
Meanwhile, there are plenty of spaces where you still want real model improvement. But a lot of traditional business was basically process and paper pushing. You don't need a Phd level model for that, you need an intern level model.
---
But that's the trap for the large providers - intern level models are cheap and easily available with current open weights. It's going to be a very tight market, and margins will go down to essentially the floor (especially once we exit the current constrained-compute market).
Somewhat ironically - the technology itself makes switching costs basically zero. It's not just that there's no big lock in, AI is like an anti-moat. It actively fills in and paves over whatever moat you might have for price conscious customers.
I think we share the same view here, which I think is most widely discussed as "AI capability overhang". Like you also said, a large, growing amount (digital) tasks can already be completed reliably with existing models. For those - this was my point - the models are competing on price, speed, reliability, etc.
I also agree that more capable models are still desirable in plenty of spaces. I just see that there is so much lag in the diffusion of existing models, which, again, are already sufficiently capable.
When it solves your problem. I don’t think it makes sense to globally qualify a model as “good enough” any more than it does with programming languages. Python is good enough for a web app, but not for an OS kernel.
To your point, I think we’re past the hump on the Pareto distribution. The models are good enough for most things, and the rate they’re improving at the things they’re bad at is dropping. The odds a new model is good at something the old one was bad it is okay, the odds it’s something you care about is dropping. It makes a lot of sense to discriminate on price rather than features in that environment.
"640K ought to be enough for anybody"
I know this gets a lot of flak because it ended up being obviously incorrect for consumer markets.
But the under-reported corellary of this is that a TON of industries essentially operate on hardware right out of that time period (or slightly after, circa 2000s)
banking
Insurance
Healthcare
Gov
retail
Transportation
etc...
Hell, I was writing code to run on IBMs Z/arch as recently as 2021 for banks with mainframes deployed around 2004. They were just then finally considering an upgrade to the later models.
The Chinese labs are just arms of the Chinese government. They have no obligation to be profitable or please (private) investors. If they weaken/collapse American models, then that is the ROI to the lab owners.
This is a pretty strange take on an article explicitly pointing out the profitability of chinese models.
Further, even if this is your take (and I won't touch the geopolitics here, maybe you're right, maybe you're not regarding ulterior motives) - go use non chinese open models that are also good.
Laguna, Nemotron, Mistral, Arcee, Llama, etc...
China has sandbox capitalism. In the sandbox it looks and feels like capitalism, but the sandbox itself exists in a communist framework, and the state still controls all the inputs, outputs, and can intervene at any point as they please.
In organic capitalism you need to compel arrays of discrete private entities to invest in your idea. In sandbox capitalism, the money can just appear in any amount at the behest of a single controlling individual.
Hyperbole and trappings aside, that describes all mixed-market economies. And all major states have champion industries.
You can see it plainly on the Solar PV market. The likes of Trina, JA, Jinko, etc. all compete aggressively, leading to everyone running at a loss in order to corner the market, and then after a few years the government steps in and orders all of them to raise prices. Happened again at the beginning of this year, raising cost of PV Modules 30-40% for oversees consumers.
Contrast that to US being happy with everyone eating each other alive until all but one company is left standing and can now dictate the prices and enshittify the product to pay back their VC loans.
Now do American Capitalism!
Isn’t “sandbox capitalism” - real capitalism and what the us is trending towards is the “Wild West” capitalism with monopolies, trusts, and all the other anti competitive things that arise in the free market if there is no external force that tries to keep the market “fair”?
I mean this is not the case for either, but the systems are setup to drift in these directions - a market needs some sort of regulation to keep things sane, otherwise you end up with standard oil and the like.
You want the people with guns controlling things to be at least nominally interested in “the people” and produce more than just “return on investment”.
Not saying that China is like this or that US is bad, but saying that China is bad because it is “controlled capitalism” is kinda missing the point of what it should be in the best case.
I don't think its that hard to shoehorn current US into those sentences too, maybe switch word 'communism' with 'kleptocracy'.
Less intervening and micro management but oh boy say something not nice about government and potus and suddenly you are a new enemy of the state and 3-letter agencies work day and night to bring you down for good. Or in reverse - put a public bribe to potus and his clan and enjoy some favors for few weeks.
2 years ago I would agree more strongly with you, but we are in new era from now on.
to be fair they did say 'organic capitalism' - which I interpret the same way as 'true communism' in that it probably cannot really exist.
real-life, unregulated capitalism lends itself to exactly what you described
real-life, unregulated capitalism lends itself to exactly what you described
In actual capitalism, POTUS wouldn't have enough power to be worth bribing.
The problem isn't the money, the problem is the power.
nice
>I think everyone has known that inference is profitable overall for the major providers for a while
Have you seen any place that is not as tech-savy as HN? e.g. some Reddit subreddits?
Barely anyone is aware other than people who are already somewhat deep into this whole topic.
The insider baseball in AI is extreme I agree. I still know people who insist large language models are just glorified autocomplete.
Frontier models are still transformer based, so that's still true.
Yes. I've always suspected inference is profitable.
But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
For most cases it's probably ok, often times answers are general or basically a web search anyways, but if my Agent started blasting outdated syntax it would suddenly be way worse for coding for example.
It’s helpful for a model to be somewhat static when it’s involved in business processes.
Relying on an LLM's baked-in syntax knowledge is suboptimal, compared to leveraging reasonable context affordances (skills, examples, tool calls, MCP servers, search) to ensure adherence to precise APIs and language specs. Don't get me wrong; a model specifically trained on your desired framework is obviously preferable for more consistent generation of idiomatic code -- but relying on that in a large general-purpose model is a questionable proposition IMHO.
They need to be retrained often is my experience for general usage.
I have SO many cases, for example when shopping, when they make assumptions based on their cutoff date, and they need to redo everything based on lots of searches.
Sure, for many use cases old models will be useful but for general usage a model that's 5 years old will probably be garbage for most people, unless they prompt it extensively, which the general public won't do.
> But I'm also interested is retraining needed? how relevant will a model be in N years if it's just "static", and if you do need to retrain and fine-tune over time, is it still profitable? how often is that needed?
Just wait until what Ilya Sutskever has for Safe Super Intelligence (SSI) to self update the weights without retraining them to bring training costs down close to $0.
> I think everyone has known that inference is profitable overall for the major providers for a while
A lot of AI haters like Ed Zitron and their readers have refused to update on this. I don't know if any of them have yet, but it'll take a year or two for the idea to fully filter out of their podcasts anyway.
Seems a bit of a disingenuous framing. Most of the arguments I see are some mix of
1. Their figures have been pretty dodgy or excluded training costs so why is it trustworthy this time?
2. API inference charged at a profitable level rather than heavily subsidised subscriptions means everyone tightens up their usage + governance Which we very much saw with the rise of openrouter et al.
Personally I think there's a lot of the bait and switch of 2 when people talk about profitability of the 10-20% of API usage for Anthropic while also wanting to lump in subsidised chat users as if they're profitable too.
I think we know if it was this clear cut then the behaviours of companies would be wildly different and more consistent with profitability.
Ed Zitron is a professional apocalypse predictor. Which is to say he's always wrong.
> I think everyone has known that inference is profitable overall for the major providers for a while
Unfortunately while in the tech industry some might know that, the general public are still being misled (especially by certain folks who’ve made a living off downplaying AI) to believe inference itself is unprofitable.
> The major providers are unprofitable after you add in the training and research costs and other overheads.
This is correct for now, but I recently read that 90% of Anthropic's code is written by Claude Code itself. So, if an AI is using its previous version to train the next version, we're not far from the day where these models will take up the decision making process too. For eg: Decisions API by OpenAI or Jev is based exactly on this. What if a model like Jev sits behind Opus 5.5 and guides it to take the next step? Wooh.
Probabilistic predictions right now are at 99% for frontier models. They will never be 100%, but if humans rate themselves 99.9% in decision making, that benchmark can be broken.
As far as I know they haven't been doing any mass layoffs.
So if Claude Code is writing 90% of the code, then why are they keeping probably the most expensive engineers in the world on the payroll?
The machine writing all the code seems to not be enough, even for the company with access to the best models. What's the alternative to explain this discrepancy?
Frequently it's to keep competitors from having them.
In this game you "win" only if you arrive first, so keeping the same headcount to run faster is a valid option for Anthropic. In this context "winning" means mainly that we lose our jobs
Honestly that's an interesting statement because in my experience Claude writes around 5-10x more code than a reasonable person would to accomplish the same thing. If you include comments then that number goes way up as well.
It’s not training and research that kills you here, it’s your own investors ROI expectations that murder you cold when faced with Open Source commoditizing you, from a country that spent 5% of the sum you spent to get to the same position and then gives the recipe away to everyone.
As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?
Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?
It's a few things I assume:
1. Inference providers are largely a commodity business. While demand outstrips supply, there are many providers of that supply, which does somewhat constrain how much leverage a provider has.
2. The models are not interchangeable as of yet (though this seems to be changing). A lot of people want the latest Anthropic or OpenAI model; as such, the labs still have considerable leverage.
3. A lot of the inference providers have used their leverage by doing equity deals with the labs, agreements to have some of the deal revenue spent on compute, etc.
What's the evidence that demand outstrips supply? Is there a wait to get an inference provider or interruptions in availability?
GPU prices are sitting well above MSRP, even for older generation cards. Datacenter growth is also being pushed pretty hard by most of the cloud providers. More anecdotally, the model providers I use have been "throttling" my requests (usually by bumping them down to a lower model version) more frequently. Granted, that could be for profit reasons, but I think it all generally points to a lack of capacity.
>Is there a wait to get an inference provider or interruptions in availability?
I think that the inference providers and model vendors are mostly handling it behind the scenes with more aggressive quantization, increasing API prices, deprecating more generous monthly tiers, etc.
this info is dated
Yeah I think one of the less considered costs when people talk about this is the human cost of training. We all talk about the billions spent on compute, but I don't think anyone's got good numbers on how much RLHF is costing but it's got to be pretty large.
The only moat is in datacenters. The software has no moat, since 1) it's already trained on stolen data, and 2) someone with access to AI will be able to reverse engineer a proprietary implementation eventually, the same AI is being used for developing AI.
That's why so many talks about regulations. Regulations means: ban outside competition, just like US bans Chinese EVs today.
regulation is braindead
I would classify training and research more as capital investment than overhead.
Most people in my experience refuse to believe it for some reason.
Well, as that means there's lots of money to be made lowering costs for research and training! What tools have we used in the past to lower those costs? Automation.. Hm, what if we had AI completely automate AI research? Uh oh, we just rationally found outselves in a trajectory where we lose control of our own trajectory because of economic incentives.
It is wild how fast we're hurtling towards this outcome, lmao.
I doubt that, I got downvoted several times pointing out inferencing is insanely profitable.
Even take account of R&D cost it is still profitable, because you do not actual retrain the base model every month, only distill and post train from the same foundation.
heck, I think even including the giant cost of the top tier researcher salaries, they are still profitable.
then, how are they still losing money? just read what they are actually spend the most budget: always the hardware. those companies will never profitable, because they will never stop spending money to expand their computational prowess.
If the frontier labs are struggling under the training costs and the leading open source models are making their progress by distilling from the frontier labs it sounds a bit like there’s something wrong with the economics here or that progress is going to slow quite a bit without the amount of investment.
I don't think so. The number of comments I see from people talking about AI, there are plenty who still think the price is going to shoot up. Some even talk this way about models you can run locally on high end yet still consumer level hardware.
I think the more someone works with AI the more likely they realize that inference does not have to be sold at a loss, but many people's AI knowledge seems quite dated and it sounds almost like it becomes a convenient conspiracy theory for some, a way to convince themselves that the market changes that are already happening a temporary (and I don't mean in a Jevon's paradox sort of way, but that companies will go away from AI).
I suspect a lot of this is fueled by a few stories of companies who went all in on spending as much as possible on AI and then begun to walk that back, without realizing that was more about bad incentives and over promises of what AI could do (especially 1/2 model generations ago). Some also confuse running the SotA models at the fullest capacity costing more and more with a given problem costing more and more to solve, not realizing that the cost per problems is trending down but newer and larger problems can be solved for even more, so the costs at the top are rising despite falling prices. Also things like the subscriptions getting more expensive, though how much of that is because developers are doing far more than they were 6 or 12 months ago (and in some cases there was subsidization that is going away). And maybe a bit with the free models starting to not be upgraded or the harnesses getting worse performance of lowered free limits.
A lot of people have already replied to this point but I think I want to distill the problems a little more directly:
- If inference is this profitable, is it profitable enough to offset the enormous training costs, bot those already incurred and those in the future?
- If inference is profitable, can it cover other capital and operational expenditures the hyperscalers and neoclouds are committing to, on the timelines they're committing to, to scale out inference?
- If inference is profitable we can measure if the price/earnings ratio of current valuations is sane. Is it?
- If inference is profitable, is it - given the answers to the questions above - profitable enough to make the risks of lenders of the huge amount of debt companies are taking on to scale, rational? It's one thing to say as a hyperscaler "we can make this pay", it's another to say "I'm going to lend them the money to do that and I'm going to get my money back on the timescales we agreed, and with an ROI to boot".
- Finally, if inference is profitable why is Nvidia specifically lending money to companies to buy Nvidia cards to do that inference, and then hitting the credit default swap markets to cover the risk of never getting that money back, and doing so at ever increasing cost, when they could just go build a profitable inference business themselves?
I agree seeing the numbers is good. It does suggest profitability in some segment of the market. However I still have a lot of questions.
But you are mixing up different categories.
Most of your questions are things that are relevant to everybody due to a possible economic crash. And you are right to point this out.
But for me as an AI user those aspects are not really relevant in my decisions. If an AI bubble bursts providers will still want to sell me those tokens at that price and yes development will slow down for a time which does change not a lot for me.
Inference is very profitible when you can deal with the batch sizes Anthropic, Deepseek and OpenAI are dealing with, renting GPUs for inference is even more profitible.
The US is dealing with different tax code where buying GPUs function as as great way to offset taxes, and having boatloads of debt is rewarded in many cases. I think there's lots more (crooked) reasons why inference is even more profitible in the US.
100% the right questions. The implied thought process of so many on this thread: Inference profitable >> Frontier Labs' are profitable overall. Sigh...
I can't read the original Chinese text, but if it's revenue, then you still can't say anything about profitability. Housing and powering one GPU should be way less than $38 per hour, so there should be a profit on inference. If that profit can cover training new models we don't know, but it would appear to not be the case.
For the past year or two I think it has generally been agreed that inference is profitable, but the LLM business as a whole is not.
I seem to recall the leaked Liang Wenfeng speech gave a figure of 8-12(?) months of inference recouping the training costs of the model.
I assume they need these models to stop being replaced in 1-2 months though.
I think what he said was that 10 months of inference paid for the hardware it runs on (which may have a lifetime of a few years). I don't think he mentioned training/development cost.
You are correct, sorry. I should verify stuff like this before posting.
It's really hard to unpick, for example, energy prices might be heavily subsidized for the Chinese models.
But it's clear that it's not the money burning pit I think it was just a year ago.
China has a two tier energy system.
Consumer get the cheaper energy, while industry normally get the twice as expensive energy.
But for AI/Datacenters, there is a exception and they get the cheaper consumer level energy.
You need to take in account that the way China taxes is not the same as here in the West. And a lot more is put on the local governments regarding generating their own revenue (why there was so much building in China because those projects funded local government). A bit of a side topic lol. Its not so simple as they are making a loss on something because how its structured.
AI gets the cheaper tariffs then the rest of the industry, that is a fact. They simply have less energy profits on the AI energy. Now if you consider that subsidized or not, ... in both cases China is still making a profit.
What is overlooked is the massive amount of energy generation that China has and still keeps expanding. And that is a direct reflection upon why the prices for industry and consumers are so cheap. This buildout ironically had nothing to do with AI but was more about lower energy = cheaper industry production ... and a lot of buildout was also to build more independence from energy imports. They are not just throwing solar and wind turbines around like candy, the amount of reactors constructed and still under construction, ...
> China currently has 36 to 38 nuclear reactors under construction, maintaining the largest nuclear buildout in the world and accounting for roughly half of all global nuclear plant construction
So yea ... The AI in the west have a issue because it really does not matter if you cut China from Nvidia GPUs or advanced nodes, they compensate on the power level.
Edit: And a little extra, the irony is here in the West building nuclear is expensive as hell because we barely do one or two at the time. With a lot of lost expertise. China doing tons of them non-stop cuts down on the costs a lot.
China generates a lot of electricity but from what I understand prices aren't that much different to in the USA because of grid constraints.
I'm not sure how much you pay in USA. But I live in Beijing, where the residential electricity price is about 0.5 RMB per kWh, roughly 0.0749 USD.
China's electricity pricing involves cross-subsidies. Residential and agricultural users pay the lowest rates, which are generally below the cost of supply. Commercial and industrial users pay more and contribute the most of grid revenue.
Prices vary by region: commercial electricity can approach 1 RMB (0.15USD) in Beijing, but the national average is about 0.67 RMB (0.1USD), while industrial electricity averages around 0.63 RMB nationally.
And that half a RMB doesn't include extra fees or delivery rates vs supply rates, correct? It's the all in rate?
At similar prices and 1/4th or whatever the average salary, meaning the average person has to use a lot less energy, or spend a larger % of their budget on it
>Consumer get the cheaper energy, while industry normally get the twice as expensive energy.
Interesting. In Germany, industry gets the cheaper electricity...
It's not particularly surprising. China wants people to use electricity. Germany wants people to use gas/oil. The rate of electrification in respective country reflects that preference.
I don't think that the steering effect (away from electricity) on private people is intended. It's intended to keep industry in the country. Makes sense, but still, it feels unfair as a private person.
> China currently has 36 to 38 nuclear reactors under construction
Wowza! Thanks for all the info really helpful to have insights into how it all works.
Sales tax in China is attributed to the place of production rather than the place of sale, which is why provinces want to produce as much as possible
as if electricity for openai & anthropic isn't heavily subsidised
as if electricity for spacex wasn't literally illegally installed gas turbines polluting the air
Would that be due to the size of the models from DeepSeek or perhaps only for their distilled models? If it's for general purpose models that would crash the valuation of Anthropic and OpenAI, both who are struggling with profitabilitet to say the least.
Who said they need to distill anything? Deepseek is the most popular chinese model getting trillions of input tokens a day that have no ZDR availability. They specifically tell you that part of the deal is that they get your data to use however they want (atleast they're honest).
This idea that Deepseek needed to distil their way to success is nonsense. Its a great narrative for OpenAI and Anthropic to spread because Americans have been brainwashed for two decades into dealing with their own inadequacies by blaming things on scary asians stealing things.
In reality China is outworking and lapping most American companies. They produce 10 EEs to every 1 America produces. They are simply harder workers.
The West is cooked because we're too lazy to work hard and we'd rather make up stories about boogeyman who are stealing from us. The only thing we're good at now is killing people who economically challenge us.
DeepSeek's API used to (does still?) route your requests directly to Claude Fable[1]. Those were provably Claude Fable's responses being returned by the DeepSeek API. This was months after Anthropic accused them of distillation of Claude at scale[2].
Them provably distilling after being accused of distilling strongly indicates that... that's what they do. To spin a story of them not doing that (because why would they, with their superior talent and all) doesn't seem very reality conform.
The models are impressive enough as is (considering resource constraints and difference in CapEx) despite being heavily distilled from western frontier models. No need to pretend that they're not.
[1]: https://news.ycombinator.com/item?id=48966016
[2]: https://www.anthropic.com/news/detecting-and-preventing-dist...
You really cited a tech politics person, and Anthropic... The poltical AI people are the exact type of cope distributors I complained about above.
I think the more reasonable explanation is this is totally made up. Notice how they use examples concerning "cyber and bio" notice how the examples are politically loaded subjects that are hyper relevant to proposed AI dangers and why we need to protect western AI companies with anti-competitive regulations?
Oh no!!! The scary asians are distilling our models, but they only want the scary things!!! They want to hurt you!! We better stop the scary asians!!!
You need to up your media literacy.
> I think the more reasonable explanation is this is totally made up.
Based on what? None of the accused companies have ever denied Anthropic's accusations. And here you are confidently claiming it all to be made it. How much of a reputational hit to one's honesty do you think it carries to completely fabricate someone doing something, when they can just publicly disagree and disprove you? First thing DeepSeek, Moonshot, etc. would do if they hadn't would be to make a press statement calling out Anthropic for their lies.
DeepSeek routing to Fable was corroborated by many independent sources... since... everyone was able to just try it out and see it for themselves.
> Oh no!!! The scary asians are distilling our models, but they only want the scary things!!!
The straw man you're mocking doesn't exist. Non-Chinese labs are also distilling frontier models of their competitors. Elon testified in court that Grok was trained on ChatGPT outputs. And why would they not? It produces objectively better models.
Not everything is a grand conspiracy.
Yeah, I'd expect every lab to use the resources at their disposal to get ahead. Everyone's distilling everyone else, there's nothing wrong with it.
When did I say everything is a grand conspiracy? This whole thing where centrists normies just call anyone who believes something outside their center comfort zone a conspiracy theorist is mega obnoxious.
You need better media literacy.
You cant just make up a reality where none of these companies have denied distillation.
The scary asians are going to get you, watch out! Your government is going to eventually tell you that we need kill a bunch of them too because they're so scary. Probably soonish. You're going to fall for it because you believe anything you're told.
kind of on track with gemini's releases
google is a company that tries to make a point of having profitable subdivisions
What about sourcing the asset itself? It's like a factory is it not?
You pay for the asset (tenancy in a data center) and then you pay for housing, powering, cooling, etc.
I think it’s pretty clear from the cherry picked numbers at Anthropic.
Demand is the issue. Demand only spikes with a sota model release then in drops back down to unprofitable levels when factoring in training a new model and build out.
So your stuck in an unprofitable cycle of losing revenue when your not releasing sotas but losing profit when you are.
This goes back to googles original memo “there is no moat”.
Where in you can’t stop training sotas because you’ll lose customers to the next inference model above you instantly.
Google has seemingly defeated this by finding models cheap and good enough for their existing platforms resulting in a moat of distribution. Something OpenAI and Anthropic seems to lack.
Unless the money of the investors is going somewhere else...
I think you have to take this on a case by case basis - there is no reason to suppose that the training costs and serving costs are the same for all companies for all models and for all price plans. Some may be profitable, others not.
DeepSeek's models are relatively small, and training costs will be correspondingly lower than for larger models. All you have here is a revenue number - no mention of (training + serving) cost, so while they may well be profitable you can't determine that from this number alone (if it can be trusted).
DeepSeek (investor meeting leak) have previously said that they price their models at a point where the GPU is paid for after 10 months, so something doesn't seem to add up here. If they were making $38/hr per card 24x7, then in 10 months they'd be making $270K per card, while the cards (Huawei?) only cost around $20K.
Presumably cards run on electricity?
That turns out to only be a small part of the cost.
Say a $30,000 GPU has an operational lifetime of 3 years, and draws 1000W. If it is running 24x7 then over those 3 years it will consume 26,280 kWh of electricity, at a cost of say $0.10 per kWh, giving a lifetime electricity cost of $2600, or less than 10% of the cost of the GPU itself.
So, to a first approximation the cost of serving a model is the amortized GPU cost.
Now that the unit economics are laid out in the open, What stops any of the companies or even startups from anywhere in the world with some investment $ and competence from spinning up and offering their own version of the inference engines, undercutting any moat that the current leaders have?
I could even see F1000 companies having their own backstop intelligence for most of the tasks and only use frontier where it is absolutely needed or as much as it is under their IT budget.
Won't it be race to the bottom when they compete on cost? Is regulatory capture the only moat left for the market leaders now?
Nothing but hardware access, which is getting harder for the average person but for a giant corporation I see no reason they can’t compete now at the local model level
I don't think i'm paying more than Deepseek price, ever
But haven't the Chinese (and Deepseek specifically) put a lot of emphasis on running models efficiently (because they've been locked out of being able to obtain certain GPUs by the US)? Whereas the US frontier companies seemed to get huge cash inflows and didn't need to worry about efficiency as much and put more emphasis on frontier performance. So how do you know that this $38/card/hr is about the same that the US frontier companies are making?
Until the Chinese stuff came online I was completely convinced this is just a bubble waiting to pop and nothing will come of this tech due to its absurd costs. The Chinese stuff changed this picture massively. I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy + I need to follow along. Opus 5.5 speak is still cause aneurysms for my brain. It also has this vibe to its prose... like it's looking down on me. This is Prolly my bias speaking though.
> My workflow is one where I am heavily involved in the process and I don't like giving the agent too much autonomy
For such workflows it barely matters which of a dozen models gets used.
Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
> Whereas if you're handing off all dev and no longer reading code, just making CTO-level architecture decisions, then there are only 2~3 models right now worth using, none cheap.
I use frontier models every day and night, but not sure if the described approach is possible with any of them right now. Models still make stupid mistakes and people who do not read the code produced are bound for surprising discoveries in their codebase. Not trying to downplay the effectiveness of LLM-assisted coding and all that, but it's hard to imagine how much stupid shit i would've committed if I was not reading their output.
Serious question: Are you do a combative code analysis with a range of different models?
I used to find real dumb bugs in code reviews. With 2-3 different models doing a combative code review.. most of that is already fixed when I get it.
Yeah, I have extensive collection of prompts for reviews and usually run 3-5 (automated) review takes on large sets of changes before doing eyeball reviews. Auto reviews indeed find many obvious bugs, but they usually slip through things that are less obviously stupid but are bad conceptually, for example, using persistence / queue / repository in a place where set of pure functions would be enough; or vice-versa, not persisting the state that must be persisted for reasons, etc.
Another problem is that review changes can easily introduce new issues, so my review prompts go like that: if you find outright shit, fix it right away, but if you have slightest concern, just write your ideas and suggestions in the review and let me decide then. I try from time to time to omit the "slightest concern" part and make agent do all the shit it wants, and usually end up reverting those changes.
But as the sibling comment said, not all code needs quality and indeed I skip some code too, like user playwright journeys -- I only need one auto review pass to verify they indeed test things according to a doc, and I'm fine not reading them. Not all code like that though, some stuff would wreck havoc later if its not looked at.
If you can test well enough, both manually and automated, then it's okay if there's a bunch of stupid mistakes in the code, as long as the end result works right. Not unlike biology.
If you view creating programs like an optimization problem over the feasible region of valid programs, this is not unreasonable.
> I'm daily driving DeepSeek 4.1 flash (at API prices) and every now and then I give Opus 5.5 a try and don't notice anything missing for me.
I was doing the same (along with GLM 5.3 Flash) for months, and then gave Claude Sonnet 5.5 at medium effort a try. The difference was like night and day. Where both DeepSeek 4.1 Flash and GLM 5.3 Flash kept making mistakes and would have to do a bunch of iterations to find and fix them, Claude Sonnet 5.5 just did it right on the first try.
In my experience, while DeepSeek 4.1 Flash and GLM 5.3 Flash are a lot cheaper per token, they're just as or maybe even more expensive when you factor in all the extra tokens that are need to troubleshoot and fix their mistakes -- and even more expensive if I factor in my own time wasted doing the same, not to mention the frustration of having to deal with that.
DeepSeek 4.1 Flash also hallucinates a lot.[1]
I'm hoping all this improves with time, and the gap between the competent frontier models and open weight models narrows.
[1] - https://www.youtube.com/watch?v=P4dTq4X8bqk
I don't think it's a fair comparison. Maybe against deepseek pro would be more similar in terms of model size
Same for me. The cheap Claude subscription is just so much cheaper. Never had to buy more deepseek tokens for months (August), because Sonnet 5.5 is good enough
That’s the Model Welfare speaking, not your bias.
OpenAI's entire business is trivially profitable when you attach an ad network to ChatGPT.
Quick, someone point out the one billion user system with intent + personal data rich that doesn't result in a $100+ billion high margin ad network.
It is that simple and absolutely nothing can stop the gigantic ad networks that will be built up around personal agents.
Only if CPM can be high enough to counter balance the inference costs.
Current CPM is between $30 and $60 which is x20 to x30 more expensive than alternatives.
Now we have to wait and see if performance is also at least 30x of previous ad targeting.
What is that CPM? My CPM in ChatGPT is lower than in Google. CPM is a bad metric, how much does 1 conversion cost? You can charge me $100 per ad all day, as long as every showing results in a $1000 sale.
There's a trillion plus dollars (per year) of ad business up for grabs in the LLM era.
Facebook is toast, sooner than later, as a business. Whatever gradually replaces/upgrades that quasi dead friends network, will be premised on AI (Zuck apparently thinks it's at least in part personal agents). In the present Meta still has a $228 billion ad business that is hyper profitable.
Google + Meta = ~$650 billion hyper margin ad business. OpenAI, aka LLM as a service for the masses, is going to want to take as much of that as they can. OpenAI's active user base comes at an attention cost to the other mega services like FB or Google Search, and ultimately that attention is worth ad dollars. The question is only whether OpenAI can take it, not whether it's there at all or high value. And of course there's no guarantee that it'll be ChatGPT upon which the next great ad network is built, it will be some AI-centric business however, and likely heavily involving LLMs.
Google's operating income last quarter was $40 billion. If OpenAI fails to take the mass market + ad tandem, then Google will more likely retain their position. The solution to the cost is already well understood by eg Google: the masses don't need tip of the frontier AI capabilities, and relying on Nvidia's extremely over-priced GPUs is a dumb business model. The biggest threat to OpenAI right now is Nvidia's margins, not China. Google's focus has been on high quality cheaper AI systems for a reason: they're looking to serve billions of people that have no use for what the bleeding edge offers vs a fast + good mid-tier. Google can do that in a highly profitable way.
The thing that stops it is, ironically, personal agents. Get one to interact with the ad-driven one if you need to and filter ads out.
I suppose the very reason why we haven't seen ads yet is that they're trying to figure out how to prevent exactly this.
What? Ad networks REQUIRE a moat. Google Meta Amazon have historically defensible moats. ChatGPT? Zero moat. Good luck with that ad business!
A billion users and the extreme cost of infrastructure to compete are a gigantic moat.
You can't get the first and you can't get the money for the second. Nobody is going to give you $100 billion to underwrite the cost of competing with GPT at a one billion user scale (or even a 100 million user scale). The easy money is already drying up, the field is set, and you will not be able to get anybody to give you a huge check to attempt to chase OpenAI, Google, Anthropic, China. That's already over. No VCs will fund you on this particular business pursuit.
You couldn't meaningfully compete with OpenAI and Anthropic if you wanted to. There's absolutely nothing you can do about it.
There's a trillion dollar valuation waiting for you. What's up?
Even nation-state pockets Microsoft can't compete. If there were no moat, Microsoft would have their own GPT killer, and GPT would be dead. Microsoft has $160 billion per year in op income. They can freely afford to spend more than the entire EU (is willing to) on AI matters if it makes sense. They can't compete no matter how much they spend and they know it.
Google has nation-state money. They have chips. They have software developers by the zillions. They have a frontier or very near frontier model in Gemini / Argon. They invented a lot of the category. They can't kill GPT. I repeat, no matter what Google has thrown at the matter, they have failed to kill or stop GPT.
Google can't beat them. Microsoft can't beat them. China's models haven't stopped them (DeepSeek was supposed to kill them years ago; Kimi was supposed to kill them).
That's one helluva non-moat. They must just be really lucky.
You might as well claim Coca Cola has no moat because it's just sugar water. Brand at scale is a large moat. People use what they know unless you severely disrupt them. They're being taught to use GPT, they know GPT, and so long as OpenAI doesn't screw it all up, people will just keep using it. To say nothing of the fact that GPT still has one of the elite frontier models, so you also still have to compete with them on quality.
I still feel like there are some questions there, as in what is the actual amortized cost, plus the ongoing variable costs and even the non-training related overhead.
Plus there is the actual demand side to consider and how does it impact when paid utilization isn't 100%.
I don’t know, I take this as there being a good amount of margin that is going to be suppressed due competition in the very near term. It will be interesting to see how these margins hold up over the next 12 months, I expect them to drop substantially
> Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
You missed the bit where they used the word "revenue".
I suspect if you factor in all the various overheads of running datacentres the actual value is lower. Revenue is vanity, profit is sanity as the old saying goes.
However the Chinese do have the benefit that China are pushing hard on renewable energies. So at least that major part of the equation is probably far lower than your average US/EU site.
Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power. That decision is just going to hurt more and more in the coming years as China's electricity costs continue to plummet while the US' continues to depend on unsustainable fossil fuel infrastructure.
> actively trying to prevent buildout of solar and wind power.
And actually paying people not to ! (Paying RWE $1.2bn to abandon wind contracts[1])
Its absolutely nuts. Pure generational can-kicking.
[1] https://www.bbc.co.uk/news/articles/c1e1vg0gjl5o
> Huge, generational mistake for the USA to be leaning so hard into fossil fuels right now and actively trying to prevent buildout of solar and wind power.
If the US is trying that, they are failing pretty hard - 88% of new power in the US in 2025 was renewables, and 2026 is estimated to be above 90%
One of countless mistakes made by the current administration. Decimating research funding and firing many scientists is another.
It's not a mistake. It's deliberate harm.
We are vigorously punching ourselves in the face to teach our perceived enemies a lesson.
I have no idea why this is being downvoted when the entire agenda of the current administration is to "starve the beast" by cutting all essential government departments and prove that public administration doesn't work so that everything can be privatized. And meanwhile the head of the current administration basically does what Russia (and Romania in the 90s did), he makes money by selling off those bits and pieces to the highest (personal) bidder.
Why? AI DCs can't rely on solar or wind so there must always be a thermal backstop. China is building lots of coal and nuclear.
Reality: renewables rapidly increasing in the US every year, especially solar (around 30%)
It's hilarious and depressing that one of Trump's supposed goals was making the US into an industrial powerhouse again, and yet he's acting to protect the profits of the fossil fuel industry as opposed to encouraging the kind of resource efficiency that would make the US competitive long term.
The only thing he cares about right now is covering up the Epstein Files. Everything else is key jingling to get you looking elsewhere.
China is investing in cheap and secure energy, which includes renewables, but it also include coal: https://www.reuters.com/business/energy/chinas-coal-power-ri...
The latter is great as a way of getting an edge in business and reducing reliance on oil supplies, but its also environmentally highly damaging.
> You missed the bit where they used the word "revenue".
No they didn't...
> I suspect if you factor in all the various overheads of running datacentres the actual value is lower.
That's literally exactly what the quoted part addresses: "Amortized cost of owning a DC full of GPUs is far less than $38/card/hr". Cost of running GPUs is far less than $38/hour, revenue is $38/hour, ??? , profit
The railroads went bust, twice, not because there wasn't profit available, but because investment outpaced profit and wasn't sustainable.
> Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
what do you think it is, out of curiosity?
Other replies covered this, but I was basing that comment on the prices that commodity providers charge for GPU-accelerated VMs. E.g. the going rate for a NVIDIA B300 (288GB RAM) and the VM around it is somewhere between $5 and $20 an hour [1].
These providers have very little incentive to charge unprofitable prices (except maybe capturing more of the market, but there's not much lock-in here), so it's reasonable to assume somewhere in that spread is the "true" cost of being in this business, all expenses included.
[1] I'm using https://gpus.io as a reference here, and that's on-demand prices, which will tend to be the highest relative to other options.
(tongue mostly in cheek) sounds like I should be renting out these platforms and selling inference on them to OpenRouter
You can pay off a high end Nvidia DGX server at just over $7/hr/card over a 1 year period.
$38/card/hr is a ton of money. That's $2.6 million per 8-card server per year.
The cost of power/physical space/interconnect by comparison here is negligible.
...well you can get B200 in cloud for ~$6-14/hour and that's after all hosting costs and profits company make of it soooo yeah
With this new data at hand, I have to wonder how profitable would it be for the investors.
The cost of amortising a datcenter, the gpus, the engineering costs attached all to produce inference.
At how much percentage of its inference would it be profitable (if it does), how much returns would it make over the long term and what would be the profit of those inferences/ profit-revenue and how that compares to other industries.
Are there any statistics which show this. Also how efficient token inference has gotten over time?
https://inferencex.semianalysis.com/profit-estimator
>It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments.
Inference has always been profitable. The problem is that this profit shrinks as your model becomes less competitive. The labs are in a constant race to release the best model to maintain inference profit. Deepseek is not training their own models at the scale of OpenAI and Anthropic and mostly sustains off their efforts via distillation.
Those arguments will go away when spending on AI infrastructure is no longer some multiple of gross annual software sales revenue, yeah?
When that happens, the chips stocks will have tanked back to sane levels and the doom posting will be about that. That will put us in the 2001 era of post dot com bubble, and the following few years will show us who the winner are
You're looking for comfort in the wrong place.
From microns S10: BITS shipped and net revenue.
Q1 '20 - $0.10 profit per gb shipped (ram and storage)
Q1 '23 - -$0.024 (cyclic ram market)
Q1 '25 - $0.131 this looks normal were on the upswing.
Q4 '26 - $1.136 Were all fucked.
If you look at a chart of S3 prices we have all been living in the "Happy Time" where the cost to run our ever growing software business has gone down. Githb says 2025 billion commits - 14 billion this year, and 2x the actions run. That means AI driven software growth is 2x to 14x.
But those ram/storage prices are going to hit the operational budget of EVERYONE soon.
Can your average sas survive a 4x price increase on its AWS/Azure bill? How about 10x (cause thats what the margins say the premium is on ram).
If the circular financing is a concern, the circular firing squad should be a bigger one. Your paying for tokens to build more software to pay more to run your software is not a sustainable cycle. People are starting to use the words CapEx again - and it's a good thing.
And it seems we're getting just 10x more software nobody uses.
We don't need 10x more software per se. It's removing the high cost and exclusivity from software production (but not just software production). It also enables the production of highly specialized software by non-specialists, so it broadens the field dramatically (things that were not being produced, will be). Simply put, every bit of software that can be useful, will definitely be made; and a whole lot of unnecessary software will be made in that process as well, that's chaos-discovery for you.
> "AI is unprofitable everyone is spending a dollar to make fifty cents"
you are just talking about ai inference? Are you assuming ai models themselves fall from the sky.
> It's really refreshing to put some cold hard numbers on this stuff, hopefully this quells some of the "AI is unprofitable everyone is spending a dollar to make fifty cents" arguments. Amortized cost of owning a DC full of GPUs is far less than $38/card/hr
They (well, most of them) are looking at loans and debt (in plain sight and hidden) of AI companies and asking "what will pay for that", not looking at how much a GPU costs and how much tokens you can sell off it.
Just putting a model on a GPU and selling it is profitable, period.
But spending more and more money on building and running infra to build frontier models that solve smaller and smaller piece of the pie might very well be unprofitable.
Models competing with the frontier labs gets more capable every month and more and more of the tasks can be achieved without reaching for the big guns.
We might very well get to the point where AI/LLM is profitable just fine but bubble still bursts and the big frontier labs are in trouble. And frankly that would lead to far more sustainable ecosystem.
revenue != profit?
AI as a business will be 10x larger than it is today within the next 20 years.
Present valuations are the bubble, not the business / industry. The exact same thing was true about the Internet bubble circa the late 1990s, except this is a lot more important than the Internet and the global economy is drastically larger.
Productivity is how you expand economy and wealth. It's how China can go from being one of the world's most backwards agricultural nations, unable to feed itself properly, to gradually becoming an agriculture juggernaut. AI is a hyper productivity tool, saving people time and bolstering capability are two of the primary productivity concepts.
We're not even out of the first inning yet.
Why do you posit that this is a lot more important than the Internet?
Artificial intelligence will 'infect' every surface of life and business. The Internet is merely a distribution platform for intelligence (and yes that includes TikTok videos). Its first distribution chore was slow and quaint, it was human. The intelligence was always the higher value, not the distribution network.
Distribution of intelligence: data, photos, video, communications, transactions, entertainment, et al.
And to be clear, it doesn't matter if some random commentors on HN don't think very highly of some or all of the intelligence being distributed. Their elitist opinion of the quality of intelligence of a given thing doesn't matter as to whether it's happening or not.
When a 16 year old records a cloned TikTok dance video, edits it, and then publishes it, that's intelligence distribution. That's a person thinking through a series of actions to try to communicate to an end audience, via an entertaining medium. That's intelligence being distributed over the Internet. That's the quaint, slow, human form of it.
Up to this point that distribution was basically entirely human, capped (post saturation) at slow growth N intelligence output (derived from the total number of humans and their time / productivity capabilities). AI uncaps that and radically expands intelligence output & distribution at a growth rate that is so fast you might as well start picturing the planet having tens or hundreds of billions of people on it. The human population will shrink, the artificial population will increase dramatically. People will have more artificial friends than human friends within ~20-30 years.
A few decades from now the notion of thinking of computing separate from AI will be so foreign most people won't understand what you're talking about. It will just be thought of as computing. Humans will forget a time when we weren't augmented by the constant presence of artificial intelligence. Try taking 'smart' phones away from the masses. In about a decade try taking away their helper agents. A decade after that try taking away their home robots.
The steroid jump is AI + robotics. The Internet is trite compared to what comes out of that over the rest of this century. Robotics will physically remake our world, in a way the Internet can't. We'll deposit tens of billions of robots onto the surface of the planet and they will almost all be the result of having figured out how to make intelligence (relatively) easy & (relatively) cheap. I'm not talking about Roombas.
The Information Age is over. We're into the Age of Intelligence (insert the unavoidable quips about humanity & intelligence).
The Internet is just one way to move intelligence around (a robot body is another, a smartphone is another, a human body is another, and so on). It's a sub value to intelligence broadly. Intelligence is the ultimate value. The valuations (and sales growth) for OpenAI and Anthropic is because they're vast intelligence output machines in the Age of Intelligence, ditto Nvidia's business boom. We can now mass manufacture intelligence at an accelerating clip, and it has gotten cheaper at a comical rate over the past few years. While HN comments snort about pelican improvements day to day, the leap over a few years is astronomical in regards to what significant AI systems can now do (with LLMs being just one type of that).
this is tech fart sniffing self-aggrandization. You’re saying words that mean nothing. Tens of billions of robots? To do what, exactly? Have you ever interacted with people, not nerds, real human people? Real people don’t crave a future filled with robots, real people want real people.
The world will be filled with robots is not the inspired forward-thinking of a visionary it is the uninspired regurgitated science fiction of someone who doesn’t care to understand that the world is people, a society.
There's nothing that is going to stop the AI-robot saturated future from arriving. It doesn't matter if you dislike it.
China will have many billions of robots just in their industrial efforts alone. Every house and every apartment in China will saturate with robots of various usefulness. That single market will be at least 10+ billion in scale.
There will be a billion robots for just the delivery and transport categories globally. We're going to peak at at least 10 billion humans (population), a billion robots for delivery & transport segments is easy.
In the toy category there will eventually be far in excess of a billion robots. They will seem quite advanced compared to what we presently are used to in that category. The Microduck is a primitive example of what will be a large industry around smart entertaining toys. Your artificial friends will be able to 'hop into' (take control of) your Microduck (whatever the 2035-2050 equivalent is) and cruise around the house or wherever you're at. Entertainment will continue to be a gigantic industry and that will obviously apply to AI + life. We haven't even begun down the road of AI + entertainment, it's less than early there.
Every household in the world will eventually have numerous robots, from primitive robots (including single purpose and toys), to advanced humanoid robots. It won't take longer than 2050-2060 to push $10k-$20k humanoid robots into middle class or upper middle class homes. China will put enormous pressure on the cost of those robots, and they'll mass sell them to the bottom 4/5 of the world in terms of household incomes (that is, they'll come up from the floor on price and take most of the market, while some companies in affluent nations will produce more expensive high-end humanoid bots, ala what Tesla is attempting).
Tens of billions of robots globally is easy. 200-300 billion artificial friends is also an easy prediction. Social junkies will have 50-100 artificial friends that they try to tend to. It'll largely replace platforms like Facebook, the utility that served for the average person. Humans will get a lot more isolated from each other and as that occurs they'll fill in the void.
I am a nerd on Hacker News. My preference for technology and robots is obvious and an assumption that my disagreement with you is based on personal preference is brainless.
I have spent time in China and Korea where attitudes towards artificial intelligence are much more positive than the west. I have interacted with robots in these countries.
You are making the fundamental mistake all nerds make. You’re thinking about what is possible and concluding that it is what will happen. Just as nerds believe that someday it will be possible for us to live on mars and therefore we are going to live on mars.
Humans are humans. Humans are interacting with artificial intelligence as if it is human because they crave human contact and are so desperate they settle for the best they can get.
The houses filled with robots future goes hand in hand with the robots will be doing all our work future. And if robots are doing all our work, what do humans have? Time. So much time. Humans become creatures of leisure. No longer is there a too little time to build relationships, there is only time, so much time. Human relationships become abundant, humans spend all their time being human.
The idea that people will have 50-100 artificial friends is so batshit crazy that I could totally believe your comment is parody.
If real people want real people, then why is everyone at the coffee shop, on the train, at parties, and even at the dinner table all looking at their phones? Not all the time, but way too much of the time, imo.
> at parties, and even at the dinner table all looking at their phones
This is very different from what I see.
At coffee shops and on trains, sure, people are working or on their phones. But at dinner and parties, at bars and climbing gym and dances, we're there to be there with each other.
Maybe a cultural difference between where you are and where I am?
But then again, they are mostly looking at images and videos of other people, right?
Because using internet is an addiction.
> People will have more artificial friends than human friends within ~20-30 years.
That would be awful, presuming that the artificial friends are not people.
Separately, you might like Count to the Eschaton by John Wright with its ultimate project being "the sophistication of all matter".
The masses won't think it's awful though, and that's the thing that will facilitate its explosion as a commonality across the planet. The masses are extremely lonely all over the planet. Along with desperately loving entertainment, they desperately want companionship.
Just as human intelligence is merely one form of intelligence, human companionship will be merely one form of companionship. AI friends will offer a new type. And for the masses it's going to feel very real and very legitimate. They'll like their AI friends as much or more than they like their human friends.
And the fake friend explosion can happen first quietly, which eliminates the initial shame phase problem. Nobody will have to know you have fake friends, or how much you interact with them. Until it's common it'll be viewed as shameful (pathetic, proof of social ineptness) and mocked by society, then it'll be normalized. This idea isn't novel, it's extremely obvious and right now we're just waiting for a few pieces of the LLM puzzle to get to the point where they're good enough to facilitate high quality artificial friends.
It'll happen shockingly rapidly when it sparks; slowly at first and then in the span of 3-5 years we'll go from nobody having or being willing to admit to having artificial friends, to everybody openly discussing it. It will then seem more odd to not have artificial friends, like people that refuse to own smartphones or use the Internet. People will start commonly referring to their artificial friends by name in conversations with other human friends. These behaviors will become entirely socially normal.
"Yeah I had Alice clean the house before I went on vacation." Every major personal robot, every artificial friend, will get a name. Humans name their vehicles, so they're definitely going to name their $10k-$20k house bot and the artificial friends they spend large amounts of time with. When you go on a long drive (etc), your artificial friends will be there to keep you company, and they will remember your life events as well or better than you do (if you give them access to that recording/memory capability).
> Along with desperately loving entertainment, they desperately want companionship.
And I am presuming that the companionship that people need is the companionship of other human beings. Otherwise, tamagotchi, fish, cats, dogs, parrots, and horses are already perfectly good companions. AI can help fill in for some of those other ones (tamagotchi especially), but it cannot replace human companionship.
Maybe some people will like to have artificial friends with genuine two-way relationship, but I think most of us want artificial servants. Robots that clean our house, keep us company, or maybe even act as romantic partner without asking anything in return. Maybe some of them will feel like friends, but in the end, we are their owners and they are our servants. Genuine friendship requires freedom and autonomy for the other side to walk away if they so choose, and are we really going to give that to our AI bots? I doubt it.
> Humans name their vehicles, so they're definitely going to name their $10k-$20k house bot
huh. I can only think of one friend I've ever known who named their car.
Anyways, not saying I necessarily disagree with the rest of what you say, but it's out there, for sure. We're a good distance away from an AI that can meaningfully act like a friend.
(Though who knows what AI will look like in even 10 years? I'm not ruling this out)
People naming their AI's is already happening.
Apple, Amazon and Clarke started the trend.
There's a rising trend among gen alpha to not use phones or internet to get back their youth period that was ruined by the algorithms. I predict this trend will continue, and I myself am yearning to use less technology each day (but can't because of salary!).
But yeah, masses will do masses stuff and that's why even shit products can sell.
>People will have more artificial friends than human friends within ~20-30 years.
How do you envision an "artificial friend"
> AI as a business will be 10x larger than it is today within the next 20 years.
I would argue it will be 10x in 2-3 years the way things are going.
Today. As new supply comes online and demand levels off, that 38$ will shrink. Absent legal monopolies, the margin will shrink to a sliver more similar to any other service. That is the danger of open source models. They destroy monopolies and shrink margins.
> That is the danger of open source models. They destroy monopolies and shrink margins.
Danger? You mean benefit?
That would depend on whether you are a user/customer or an investor/owner in AI. At the moment, nearly everyone is on the investor/owner side. The entire western world has doubled down on AI becoming the most popular product since the invention of fire. If it turns out that fire 2.0 cannot be patented/copyrighted/licensed for insane profits, those dollars disappear.
The people who own these companies have more money than they, their families, and their extended families can spend for generations. None of them have to work a day in their life no matter what happens to these companies.
Same with a lot of their investors, who will be just fine without the "insane profits" they hope for.
I'm not worried about their welfare, and won't shed a tear if their companies collapse.
It isn't just those who own stock. If the AI bubble bursts violently it will impact literally everyone. Everyone who owns indexed stocks. Everyone who owns tech stocks. Everyone with a variable rate mortgage. Everyone with a job. The AI spend is now so much of the western economic system that every one of us has skin in the game. At least on the risk side.
It's mostly concentrated in US tbh
idk man 2008 didn't affect me much directly
> The entire western world has doubled down on AI becoming the most popular product since the invention of fire.
That's a bit of a one-sided view.
AI companies are now 49% of the S&P 500.
Massive bubble.
When every company calls themselves AI-first, what then is an 'AI Company.' In many regards, this hype-chasing branding exercise is currently no different than calling one's company a 'technology company'... Just as the term "AI" itself has (to a degree) become meaningless... and AGI will be soon to follow.
Agi is not coming in this century. Like come on.
This is probably the boldest claim in the entire comments section, considering we're only in 2026.
Compare 1926 to 1999, and how technology advanced in that time. Radio wasn't widespread in 1926, nuclear fission hadn't been demonstrated, the idea of the semiconductor energy band gap, underlying the entirety of the modern computing industry, wasn't put forth until 1928. And yet, 20 years later we were purifying silicon, and 10 years after that (1954) the first silicon transistor was created and kicked off the semiconductor boom. 50 years later, we were in the Internet bubble.
We've got decades upon decades upon decades to come up with new ideas and implement them, even before the end of the century.
Right, I have to think P(AGI by century's end | LLMs turn out to be an AGI dead end) is still at least 50%.
Nothing bold in it, as there is no more meaning in the AGI.
We already lived through multiple AGIs for the last few years.
... have we?
Can AI handle driving yet?
https://nitter.cf/JensenHuang/status/2096700264569090384
> nearly everyone is on the investor/owner side
I think that "everyone" needs some kind of an asterisk. Who's "everyone" in this?
Depends if you’re a shareholder or not
> That is the danger of open source models. They destroy monopolies and shrink margins.
It's unclear that the leading open-source models could exist without having distilled the frontier models. If that's true, the "danger" is that these upstream models don't get built any more (because of the tremendous cost), and we end up just optimizing what we have. Also unclear whether this would be a net win or loss for humanity.
The tech is at the point where synthetic data is almost high enough quality to train further models on, AND there are so many human users of cloud AI, that their usage can be gathered and trained upon (it's human generated and used in real world situations, so, it is of a good enough quality). The cat is fully out of the bag.
This deals with half of one of the three scaling factors only. Presumably data production (choosing your training data, etc), model size, and training time/power still contribute huge costs. Who's investing that cash if open-source models will distill out enough to compete without the investment costs?
There is a community of people who compile LoRAs of their favorite subjects/styles in order to fine tune models. There is currently a bit of a gap between LOrA creation an integration of that data into larger models but I can see serious hobbyists stepping up to that plate soon.
The upstream models weren't built by god, they were built by distilling the world's knowledge and cultural products.
If OpenAI/Anthropic stop doing it there's no reason to believe someone else won't. Or that we won't come up with a different approach.
> they were built by distilling the world's knowledge and cultural products
... and a bunch of RLHF and proprietary training data etc etc, but regardless, this is still an exceptionally expensive process, that someone somewhere is paying for.
I'm OK with this.
Anthropic is zooming to $100 billion in annualized revenue.
The tremendous cost is matched by the tremendous business output.
In the span of less than five years, starting from their first $1 of revenue, Anthropic + OpenAI will have produced businesses doing at least $150 billion in combined sales. The demand isn't reducing at present, it's climbing, they can't come close to servicing the total demand for their frontier models.
SAP is under $40 billion in annualized revenue. They're 54 years old.
If SAP sold $1 bills for $0.50, they too could rapidly outpace Anthro/OAI's growth. And when sane minds ask about profits or a defensive moat, the response is "but look at our revenue bro".
The same exact thing was endlessly claimed about Uber. They were guaranteed to die. They were selling rides for less than they cost. Uber was a commodity app anybody could MVP in a weekend.
Market captured. Whoops.
Uber now: $55b in sales, $6.7b in op income. They won.
And to be clear, OpenAI can easily permanently subsidize a fake low cost for their LLMs if they get off their asses and attach a monster ad network to GPT already. They're years behind where they should have already been with ads, the easy money made it so they didn't have to think about it. If they miscalculate the timing on switching over from easy money (market) to ads (also easy money) it could certainly kill them.
OpenAI can knock out a lot of competition by building the largest ad network, then undercutting the cost that consumers see. The margins in ads make that possible. Google already knows how that works, they're doing the same thing right now.
I'd rather have the OpenAI + consumer base + ads position, than being Anthropic. China is a far, far larger threat to Anthropic over time. OpenAI is building a service business more comparable to what Google does. Chinese models don't actually threaten either OpenAI or Google insofar as their positions are user capture based, whereas constantly improving Chinese models are a never-ending threat to Anthropic.
> The same exact thing was endlessly claimed about Uber. They were guaranteed to die. They were selling rides for less than they cost.
Lots of companies have used this strategy. And many of them have indeed died. Uber filled a void left by bad public transportation and a lackluster cab system. The higher prices (so they can turn a profit) are still worth it to some.
> OpenAI can knock out a lot of competition by building the largest ad network, then undercutting the cost that consumers see. The margins in ads make that possible. Google already knows how that works, they're doing the same thing right now.
And you think OpenAI can compete here? If LLM's end up being yet another ad funded platform, I have a hard time believing Anthropic or OpenAI will outperform Google here. Maybe if one of them strike a deal with Apple, or get bought by them. Otherwise, ads are Google's bread & butter. And, they own Android.
The main difference between Uber and other parties that tried was Uber's willingness to break the law. They then made it stick because of money.
According to who? There is no agreed upon anyone that says that?
It is mostly response to claims of "all LLM are subsidized and blatant bubble, LLM are useless" etc
All LLM are subsidized and blatant bubble.
If the only profit is from running commodity free software (as OP claims), there's no moat and OpenAI/Anthropic are worthless.
you actually have no idea what the cost is
We still need to know if that includes purchase and maintenance costs, which are substantial. And then of course all other costs associated with it.
You also have to realize deepseeks running costs are immensely lower than say any us lab by orders of magnitude.
Chinas grid is almost entirely renewable energy with strong investments in it. In the us not the case, some dcs are running on gas.
Also DeepSeek was smart. They strategically rolled out model sizes they could support even if the intelligence wasn't the world's best for that week in human history. Then they have incrementally and cheaply improved models since.
Not to mention they also built trust with open weight models rather then fear mongering. They are in the top 3 of the smartest most viable LLM labs out there.
China generates more electricity from coal than from all renewables put together (49.7% per https://oilprice.com/Latest-Energy-News/World-News/Solar-Ove...).
They have more installed renewable capacity than coal, but renewables have low capacity factors, solar is less than 50% by definition. The renewable share is growing fairly rapidly, but it is by now means almost entirely renewable.
At off peak pricing, which is substantially lower than the standard API rate (50% off). So I assume it looks even better financially overall.
Quick numbers:
At $35/hour in net cash generation per card, a $30,000 GPU would pay for itself entirely in about 850 hours (roughly 35 days)
With an assumed ~ $3 operation cost per card and hour
DeepSeek CEO has previously said that their finance team considers the optimal payback period for hardware purchase to be 10 months. That is, 10 months worth of revenue should be equal to the purchase price of the hardware. This doesn’t cover operational costs such as electricity. This is their method for API pricing.
Source: https://x.com/FuckAnthropic/article/2080268590033621295 (it’s a leaked meeting transcript so take it with a grain of salt)
Isn't electricity a pretty major factor even over such a short fraction of the hardware's lifetime?
No, the absolute max power draw of these cards is like 1.5 kW, say 3 kW per card with mediocre cooling attached to a server. DCs get power in the 10c/kWh range at most. So 30c/card/hour, peanuts. And this is the most power hungry card at full tilt with mediocre cooling efficiency.
And thanks to China’s forward thinking on infrastructure and planning for oil shortages, both short term and long term, they will have access to the lowest energy costs most likely.
Ah, yes. I should have done the math before asking.
Your estimation feels too high. It implies >1000% ROI on a GPU card (= $300k gross annual profit on a $30k card).
As a reference point, Z.ai is operating at 26% gross margin as of 1H26. So assuming ~90% gross margin is likely a bit off.
No, those $3 are on the high end. If you do not factor in hardware depreciation, at China's low electricity prices (largest variable cost and China has a surplus larger than many country's entire consumption) and with GB200 which is one of DeepSeek's main platforms, the operation cost is actually less than 50 cents per hour most likely. If you factor in hardware depreciation this rises significantly but almost certainly stays under 3 usd per hour, probably closer to 2.
Orinally it was an educated guess based on rental prices I've seen but Opus 5.5 and Gemini 3.8 Flash both arive at the same numbers based on DeepSeek's Zhihu post and GB200 specs. Around 30 cents before depreciation and $2 to 3.50 including it.
I'm curious why such a large gap exists between our numbers. My hypotheses are:
A. The average utilization rate of inference GPU cards is not that high (i.e. it peaks during the daytime, but is idle at night). So the time window for earning $38/h is fairly limited.
B. They are renting costly GPU capacity from 3rdparty hyperscalers because they can't meet the peak demand with their own hardware.
C. Free-tier users are eating the gross margin.
D. Subscription users are eating the gross margin.
Another observable point is that DeepSeek's company-wide gross margin was reportedly 44.6%. It's intriguing what the full picture looks like here.
> The average utilization rate of inference GPU cards is not that high (i.e. it peaks during the daytime, but is idle at night). So the time window for earning $38/h is fairly limited.
In DeepSeek's case this kind of levels out because they have a large user base both in China and the West (time zones), and additionally attract users in the non-Chinese timezones using their off peak 50% discount which is what DeepSeek talks about here.
> B. They are renting costly GPU capacity from 3rdparty hyperscalers because they can't meet the peak demand with their own hardware.
I looked this up from Chinese sources and DeepSeek primarily builds their own compute infrastructure rather than renting it
> C. Free-tier users are eating the gross margin. > D. Subscription users are eating the gross margin.
That's true, but given how lucrative inference at API rates is, they can subsidize training and individual users to some degree. Are they still losing money overall? Not unlikely, but with moonshots that is often the case. Remember that Uber and Spotify took around 15 years to have their first profitable quarters.
How is the math so good for data centres but so bad for home gear? A Mac mini capable of running these models ends up at around 10 years of subscription costs.
All of it is really about scale, same reason as a single person you can't sell french fries as cheap as McDonalds.
Couldnt they though? A medium fry at mcdonalds costs about $4 and you can buy 5lbs if potatoes for $4.
I think its more like 40 cents to make at home. So I think its quite unlike McDonalds.
A single person could make french fries for themselves cheaper than McDonalds, but once they have to start including the cost of the building and equipment, charging for their labor, and complying with health codes and worker safety requirements, they will quickly find that the fixed costs swamp the variable costs.
Here's an anecdote. I have repurposed my 6 year old 32gb M1 MacBook Pro to run Qwen 3.6 35b a3b and I used to give it nightly maintenance tasks that span my OSS github repos.
I gave it mundane tasks like updating dependencies, writing changelogs, etc. It ran about 2 hours a night.
My MBP consumes 0.1w idle and about 10w running inference. I pay $0.33/kwh. If it runs for 2 hours a day over 365 days - it will cost $2.41/year (excluding idle time).
Excluding the cost of hardware - the running costs alone are more expensive than DeepSeek's API and the inference quality is much lower. The economics are even worse on AMD AI Max mini PCs and worse still with a dedicated GPU setup.
That said - my Apple Silicon example is orders of magnitude cheaper than western inference providers and offers unrivaled privacy.
Perhaps a local AI setup would be compelling for small/medium business users... and it would be practical if the hyperscalers didn't buy all of the potatoes on Earth.
>but once they have to start including the cost of the building and equipment, ...
But they don’t have to include any of those costs.
Remember the comparison was running a model at home for yourself. The analogue is making fries at home for yourself. Thats cheaper.
Making fries at home is only cheaper because you already have a stove, knives, pots, cooking oil, etc.
Running a model at home is very cheap if you already have a M5 Studio lying around for some reason.
they do have to include the deep fat fryer and oil...
unless you want some sad thawed potatoes instead of fries
and that's going to run you a lot more than $4, and take quite a while to amortize
(numbers are just to illustrate a point, not accurate) Now assume you can't buy a 5lb bag of potatoes and you can only buy 10,000 lb bag. Now also assume that you can't buy this 10,000 lb bag unless you know someone else will pay for fries for the other 9,995 lb because you can't eat that many fries. That's the kind of scale the token inference market is efficient at.
How much does a fryer cost? How much does the fryer cost to run? How much would it cost to get the fries into the customers hand (i.e. food truck, rent)?
Not applicable because the hypothetical is making fries at home for yourself.
Its more economical to make fries at your home for yourself but that is not the case for inference.
Serving LLM inference is more efficient with more concurrent requests. For a single inference request, the bottleneck is GPU memory bandwidth, as the inference engine spends most of its time waiting to move values in memory. With concurrent requests, it's possible to service multiple requests in a single pass through the model weights, meaning you can utilize more of the GPU/APU's compute.
So you have to price your local hardware against however many subscriptions you would need to run a dozen concurrent agents around the clock.
But even then, I would guess subscriptions are priced on the basis that they don't expect every user to utilize their entire usage limit. I.e. power users are subsidized by casual users.
Because the datacenter setups serve multiple users concurrently while your mac mini only serves one. You would also lose money hand over fist renting/buying 4xH200s just to serve deepseek flash to yourself.
These datacenters run the GPU's close to 100% usage 24/7. The Mac mini at home probably has <25% utilization. And then there's economies of scale, wholesale discounts, etc.
The Mac mini GPU also isn't as optimized as the datacenter GPUs
You can batch calls if you're serving a lot of people at once.
I still struggle to wrap my head around a $30,000 GPU card. But impressive payback numbers.
I just installed a $16k GPU at home - super pricy to be sure, but when you price it against hiring 4 junior developers working around the clock it's a steal
What if you compare it to an OpenRouter account and $0.50 per million tokens?
Oh that's way cheaper to be sure, but a lot less fun!
you also don't have to worry about hiding API keys or security of confidential data which is a bonus.
Could you explain what your infra set up looks like and what is your typical use case, if its more than a hobby?
For the ones who are still wondering how so much profits, inference is a highly profitable domain: https://inferencex.semianalysis.com/profit-estimator
I think we need to collectively get a grip on statements like this.
There is no reason to think inference can or will remain interestingly profitable even if you assume pure-play inference providers who don't do any model training. Serving an LLM isn't child's play but it's also not so hard as to count as a moat. Inferencing is:
1. Selling a largely undifferentiated product.
2. Into a highly competitive market.
3. To customers who are primarily sensitive to price.
Industries like this have always existed and in the modern era tend to have margins of about 3% to 5%. Any level of margin higher than that gets competed into the ground.
If you want to claim inference is highly profitable in general, you're going to have to answer why it won't end up like supermarkets or airlines, both of which have bigger moats than inference.
The other problem with this analysis is obviously that you can't just stop training models. Even if you don't care about improving capability or performance, the knowledge in their weights gets stale. Cost of inferencing has to take into account the ongoing cost of training, you can't cleanly split one from the other.
Amen. AI boosters look at currently high profit margins on inference and blindly accept claims that these inference profit margins will continue. Of course those margins depend on historically massive R&D spend on training. The moment that training stops, AI returns to a commodity market and the inference profits end. Big labs are trying as hard as possible to keep the gravy train running but they can all see it won't last forever, hence the rush for IPO & regulating away open models.
Boosters are right that AI is a historically impactful product, but big labs will be unable to meaningfully profit from ushering in the AI era just like Netscape failed despite ushering in the internet era.
"If you want to claim inference is highly profitable in general, you're going to have to answer why it won't end up like supermarkets or airlines, both of which have bigger moats than inference."
- I don't... "inference is profitable" is as of now. No one knows future, neither have I commented anything on it.
The statement is valid as it stands.
Inference will continue to be profitable while we don't have enough compute in the world to serve the demand we have. Also, I don't see why it's a problem that the end game for inference of open models will be relatively low, ie 5% or so. You can have a huge industry running on these margins. Though I would note that hyperscalers have been selling compute commodities with a high margin for a long time.
Demand is elastic, there's no fixed pool of it. What determines profitability isn't merely supply constraints but how much margin your competitors are willing to accept, more or less.
It's not a huge problem for users but if margins are low it's a huge problem for the debt funded datacenter buildout. It's just pure finance at that point, about whether those GPUs can generate enough yield to beat deprecation and financing costs.
The big clouds generally don't sell compute at high margins. The margins differ a lot between services. Services like core VMs and object storage are usually low margin or even break even. Where they make the money is things where they've got proprietary software with competitive advantage, like managed scalable forks of Postgres, or egress charges that are there to create lockin. Managed DBs are pure SaaS and can generate SaaS like margins.
Exactly, inference is like groceries or electricity.
Because LLMs are like evolutionary crabs, they all get to the same point.
I think hardware scarcity plays a part in inference margins. They would get squeezed super quickly if anyone who wanted could just build an inference cluster, but that's not currently the case.
I would assume hardware production has been ramping up significantly and it's more of a temporary supply chain issue though. While it could last multiple years, that's certainly not long enough for the hyperscalers to get a return on investment. Obviously more complicated than this, but still... I'm not buying.
We're also starting to see efficiency gains, so we might meet halfway.
Maybe but there are huge blocks of capacity about to be energized or handed over and the payment clock starts ticking immediately.
Complete and accurate, thanks.
Accounting for initial acquisition cost and typical depreciation/failure rate? I assume they already consider colo price, electricity, etc
70%+ net profits margins are for rented infra, that means no major upfont expense. Also, these margins are against 60% utilization only.
You can well imagine if you own your infra and can make utilization over 90%, where these go.
And then, my link above is for K3 model, which, compared to ds flash models, is highly unoptimized.
If I had a few million lying around, I would have invested in inference.
I would be surprised if there was such a glut of undifferentiated inference providers for open models otherwise[0]. Possibly just people trying to minimize the loss from already-committed capital, but I don't see AWS and DO selling inference via OpenRouter unless they're making a decent profit from it.
0: https://openrouter.ai/openai/gpt-oss-120b#providers
Fascinating site. Thanks for linking. However note if you change to "open router" pricing in some cases even B200 which is ~2y old has negative margin (and AFAIK ppl are using longer amortization than 2y - I do not know how cost are calculated here though).
1. The link is for rented infra. That means the inference provider doesnt have to worry about deprecation.
2. Newer gen nvidia gpu's like GB300 are way more profitable.
Why isn't ordered by profit % per GPU? It' surely the most important indicator
DS4 Flash is so cheap also, imagine what the numbers look like for a flagship model like Opus 5.5, which charges 100x more. Of course inference for that model also costs more, but I doubt the proportion is the same.
Or now, Anthropic will distill the knowledge from this engine and optimize their inference.
The top quality frontier models are used to compress / improve the earlier, lesser models, to make them more efficient. So that pushes costs way down on previous models.
155T tokens/day per cluster? OpenRouter reports that last week total usage on their platform was 164T tokens. Surely all that demand couldn't be satisfied with one cluster? Are these different units of token, or maybe something got lost in translation?
Exactly... I was going to post exactly this and you called it two minutes before me. On openrouter, it is shown that 600T+ tokens are provided per month, so 20T+ per day. Openrouter is not all the world but it seems that something is not right with this 155T figure... Unless everyone is idle everywhere, waiting for demand
OpenRouter routes traffic through many different providers
https://openrouter.ai/deepseek/deepseek-v4.1-flash-1017#prov...
From the Github benchmark numbers, one GPU can do about 26k tokens/s when serving 256 users, which is about 2.25B tokens/day. A cluster would need to be close to 70k GPUs to serve 155T tokens/day (which seems doubtful).
Napkin math comparing a cluster with a single latest gaming GPU shows a that the cluster pushes about 9 million times more tokens per day. And the cluster runs a more powerful model.
Truly, "the more you buy, the more you save".
This is a major, major datapoint. That’s at least an order of magnitude higher margin than literally anyone is discussing. The entire ai discourse currently assumes very small margins on open model inf. And by this post it’s more like 1600% (h100s rent for $2/hr - and id guess deepseeks costs are far below that)
Deepseeks strategy is that each project must break even within 10 months.
$38/hrs equals $320K in 10 months.
Assume they only have 100 clusters, and as much as 40% operational costs. That would get them about $20M in 10 months, which I suspect is more than what 4.1 flash training has cost them.
No, a cluster consists of 4x SM100, and $38 profit is per card, so after 10 months you already have $1.0 million.
I haven't accounted for 7 hours of peak-pricing per workday.
Of course, you won't have 100% utilization.
For four cards at 100% utilization it's $4712 per workday and $3648 per day on the weekends and holidays.
This is per card pricing, not per cluster pricing, so I think you’re off by an OOM or 2
You can buy such a PC off from ebay for only €150.000. We did.
But the architecture is similar to kimi's mixture of experts, and we already have such an optimization for different experts swapped in from disc by antirez.
So it’s only getting better since last year!
Are frontier labs making 80+% gross margins on LLM inference? (2025)
https://johnnyclee.com/i/are-frontier-labs-making-80percent-...
so if this was true then why do we need see GPU rental prices increase and openrouter token prices fall, for deepseek and other open models?
Ultimately, more people serving.
- gpu market is starving, demand is higher than supply = leads to high gpu price
- number of openrouter shovel sellers rises = lower token prices
If open source inference engines proliferate (in terms of adoption, across models, across inference hardware) I imagine you'd get pricing pressure on that inference rev towards base card cost, so my interpretation is this is actually a bad development for compute providers.
Do they share latency / concurrency figures (aka the deployment shape that gets them that token/s/cluster? Would be interesting if they're using deepEP/WideEP + PP and TP
$0.00051192 per M/tok ?
Assuming a 64 card cluster
thats a lot cheaper than renting a H100
> A nuclear bomb
Indeed. This could actually be what breaks America's world order.
AI can already clone major products like Photoshop and Office. Maybe not at 100% feature parity for now but at a "good-enough" surface level. That will only keep improving. Eventually we could get clones of Windows and macOS at feature parity.
Even if US lobbying/corruption tries to cripple AI to prevent that, China will instigate and accelerate it.
I'm sure Kim Dotcom's megaupload or Napster had even higher revenue for distributing stolen IP, but they did not have institutional backing.
Interesting that the DHS "Exclusion and Removal Order #2026-01" that was made public yesterday was actually released July 24th, 2026 and appears to have taken effect then. Agencies would have started pulling deepseek at that time. Which means it's likely DS became aware of the memo around then.
The release of this repo hours after the DHS order is a tactical move they started preparing when the DHS order was issued almost 3 months ago, and they timed publishing the repo right as the order hit the federal register.
DHS: "China bad and scary. Don't send them data."
DS: "You don't have to. Happy xmass kids."
Link to order: https://www.federalregister.gov/documents/2026/10/20/2026-21...
> It may take a minute or two for access for 90.8.226.75 to become effective
Exciting... feel like I'm back in '96, having to wait two minutes for the page to load.
> The release of this repo hours after the DHS order is a tactical move they started preparing when the DHS order was issued almost 3 months ago, and they timed publishing the repo right as the order hit the federal register.
What does DS achieve with this "tactical move"? Or, are you saying this is a tactical move from a higher level (Chinese gov interests)?
“ Or, are you saying this is a tactical move from a higher level (Chinese gov interests)?”
What country produces all of the ASICs used in contemporary bitcoin mining?
The faster we get to a good enough open model, the sooner it gets burned to hardware, and the sooner the transition to (relatively) low tech, low margin, high volume cards to run it on …
… and those are built in China.
Maybe they are assembled in China?
You're not going to make money buying a new 14nm bitcoin miner, and that is where the China foundries are for cost effective silicon. They're great at 28nm+ and good at 22nm pricing.
Most new design modern sub 7nm ASICs are made in Taiwan/Korea by TSMC and SLSI. That is where miners are profitable.
Whatever speculation you have on China’s foundry capabilities or a gap to TSMC will be wildly out of date within a year or 2. I’ll bet on it.
oh I love the thought that there is a future where I get my models burned in hardware same as I used to get games in cartridges
That’s part of the equation but there’s almost certainly interest in the open models guiding information trends for the interests of the Chinese government as well. So not only to help their economy in a variety of ways, but to help undermine the US AI efforts in the process and put them finger on the scale of what models ultimately do output.
US wants to ban non us models from as many places as possible. This move enables them to get users in more places.
Attackers usually have a far easier time against defenders. The effort is in vain
Cheap inference from foreign countries is bad.
Cheap labor from foreign countries is good.
Can anyone explain how this is not hypocrisy?
making AI bubble burst (even if it is just frontier labs that got hit) while chinese companies keep making money off cheap AI would be massive economical hit to US
Only because the US went insane and tried to build too much, which will not be built and we'll see HBM repurposed into consumer gear
Any drawbacks?
I think there is also an unlikely but reasonable concern that there could be exploits intentionally trained into the model itself.
https://news.ycombinator.com/item?id=50157302
I do not understand this. Was the order secret for all this time?
Except this is also banned under the DHS order. The order bans "information technology products and services developed by Hangzhou DeepSeek Artificial Intelligence Co., Ltd", and more pressingly, "third-party products and services developed using DeepSeek implementations".
This is also the word inside Defense sector contractors as well. No PRC origin anything - no code, OSS or no, no open weight models, no assistance from them.
No regs set in stone yet, but DoD doesn't need regs - they can tell you what the rule is and that's that.
Does that mean US labs are banned from using or even learning from this? Assuming they still want to be used by the government.
Something not being allowed to learn from hasn't stopped them before
It's not US labs that it targets, but just US gov't civilian agencies.
... so far.
They last part is impossible to enforce really without forcing audits on everything.
Not sure why you're being downvoted. It's very clear to me in the order that even the open weights would be forbidden, even served through vllm or sglang etc.
Agreed, though this appears to be much more than open weights.
Next would be just saying no Chinese inference engines.
Probably wouldn't have been many agencies using DS to begin with anyways.
That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
That's strategic sabotage to say the least.
Ban all open weight models! Cried the VCs.
> That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
Those valuations are a scam, they have always been a scam, like an updated version of the collateralised debt obligations (CDOs) from 2008 [0]
[0] https://www.theguardian.com/business/2010/apr/18/goldman-sac...
I rewatched The Big Short recently, and all I could think about was AI labs.
Too bad they aren't public so we can't short them
shorting is not really viable anyway because you need the timing as much as the direction
it worked for the big short guys because the knew exactly when it was going to go bust
Michael Burry burned through tens of millions of dollars on holding his position. Others enter later but still ahead of the collapse. They knew and they didn’t. And even if Michael Burry new exactly when, he had to convince his investors that the millions they were losing leading up to it would get recouped.
You can buy CDS on Oracle, as many did already.
I saw the same kind of comments when the first Deepseek model was initially released. However the relevance that investors assign to something like this does not seem to match the excitement expressed here.
I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
They'd be wrong. Especially about Europe. Many companies are in a bind about this: they can't upload their confidential files to a service provider but they do want AI. So right now they end up with a choice: do nothing, breach their contracts, use a service provider that pinky promises not to share their data (which we have already seen they will violate) and run local models.
The latter is the reason you can't buy powerful GPUs for reasonable prices because a local model (GLM 5.3, DS 4, etc) + some hardware is all you need to solve the problem in a way that avoids the legal issues.
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center buildout, not because some mid tier European insurance co needs to serve deepseek
Yes, well, you're wrong about that. They're buying GPUs by the pallet here.
Mid tier insurance co is working with local inference providers and some of these now have thousands of GPUs up and running. Don't drink the HN cool aid that the EU is a backwater where nothing happens too much or you'll end up surprised one day.
The one thing where Europe really does lag is in training, that would be fair. But inference providers are popping out of the ground like mushrooms and there are some billion euro+ projects underway. We -unfortunately- also have some scammers that are bilking (or attempting to bilk) investors pretending they have more than they do, this is annoying but I would expect that other countries have similar players as well.
Wow, whole pallets!?
There are roughly 70 GW of datacenters under construction in the US right now, and a GW is roughly 500k GPUs, so 35 million GPUs.
The 'by the pallet' buys are for small local projects, not data center size projects (of which we have more than a couple as well), as you probably already knew. This isn't some silly dick measuring contest.
The point was: they're buying them from the retail channels in relatively high volume because there is a massive shortage of GPUs in general, but that does not seem to stop parties with smaller projects who do not qualify for buying directly from NV (assuming NV even has stock to sell).
5090's now sell for the same amount (or more) that you could buy an RTX 6000 for a few months ago, and RTX 6000's are ~19K and up.
I found a stash somewhere, bought two, then two more and when I was ready to buy the remainder they were all gone. Serves me right for being too careful.
No, the point is that US demand is what's driving up GPU prices, as GP asserted. Europe's purchases are a rounding error. And even then, most of the data centers under construction in the EU are from American companies.
The US has never cancelled as many data center projects as now.
this is a fun comparison, so I ran the numbers - in the US the cancelations alone are as big as Europe's entire planned buildout (~11 GW), or almost 3x of what is currently under construction (~3.8 GW)
Not nearly as many as you’d think from news headlines: https://newsletter.semianalysis.com/p/datacenters-2027-the-y...
Your link doesn't speak of cancellations only about a certain GW worth of data centres being built but that are not operational.
A rounding error is taken to be sub 1%. Anyway, I shouldn't engage, good luck.
You can’t buy gpus at reasonable prices because US frontier labs are spending nearly a trillion dollars this year on data center build
Its not just the US ... Where do you think all those 5090's end up? Its just crypto mining all over again. Most people do not realize that China was the biggest crypto mining operation because of its cheap electricity and people willingness to take big risks.
The same happens now also, with people buying up 5090's (and lower class cards) in bulk, to strap into AI rigs. With Nvidia discontinuing the 32GB 5090s, we already see pressure on the lower end cards that had a better price / performance for a while now.
China will do China things *lol* ... People there are just more willing to take bigger gambles (and the cheap elec helps a ton). But that is also a factor why a lot of consumer level hardware gets under pressure. Its not just the supply chain to the enterprise market that is eating a lot for AI.
That does not matter so much anymore, when Huawei starts selling those NPUs for a fraction of the NVIDIA price. It's all downhill from there on for OpenAI, Anthropic and NVIDIA.
It's all downhill for OpenAI and Anthropic investors, because they will never recoup what was invested. NVDA is safe(r) - Huawei first needs to catch it, and as AMD learned, it's a fast moving target.
Indeed NVIDIA has an actual moat. The other two, not so much.
> US frontier labs are spending nearly a trillion dollars this year
They are not, not even close. I wonder if you're aware how money is being "spent" between Nvidia and US AI corporations!
The other choice is AWS Bedrock, if it happens to have the model you're interested in. They do have GLM-5.3. They don't have DS-4 or 4.1 yet, and updates are intermittent at best, but they're already on the trusted provider list for anywhere large enough to be interesting.
That may not be acceptable for many companies outside of the USA.
Depends on the parameters of "acceptable", but they do offer inference in non-US zones. If the company is already on AWS for meaningful workloads with sensitive data (which is not uncommon), they've already been through the hard part.
That being said it's not quite as simple as would be ideal, you do need to check whether there's an additional EULA on your specific model choice.
Acceptable as in 'does not break the law'.
For data residency reasons? You can meet data residency requirements on AWS, I'm pretty sure. Mind sharing which condition blocked it in your experience? Mostly curious because in the US there isn't that much interest in on-prem deployments because it's easier to scale up and upgrade using the cloud and it's easier to use AWS because you don't have to run a new compliance process for each part of the stack.
You could until the Trump admin decided to (attempt to) overrule the deal that was in place (the DPF).
The perceived lack of independence of the FTC is a major driver behind the exodus on this side of the Atlantic and it isn't just the smaller players that are moving. Airbus for instance.
Now the datacenters supposedly operating under at-arms-length management are no longer sufficient for compliance.
There are calls to repeal the whole thing with respect to the US.
https://noyb.eu/en/us-supreme-court-just-blew-eu-us-data-tra...
Yes, biased source and title, but the contents are pretty good.
I see. So there's a risk that the EC or CJEU make a determination against the existing system because of the non-independence ruling of the FTC. So the movement away to regional providers is because no one wants to be caught surprised if the EC/CJEU make such a determination. Interesting. It must be early stage because from what I can see the most spend is still on the big two, but this does bode well for Mistral and EU-based open model hosts. Lots of room in the 20%.
It's early stage mostly due to inertia but there is a groundswell of movement. Airbus really turned some heads.
Yeah, about that... My kid works for a very small company. Chatted the other day about in-house inference vs ZDR providers and all. His team is trying to get access to GLM on AWS Bedrock. It is a second week they are trying to schedule a simple meeting to discuss what they need. Random silence, and back and forth from a new person each time. 2 WEEKS to just schedule a meeting... And as far as I know, no luck so far.
You do realize there are hundreds of datacenters in Europe and the US that rent GPUs to European companies, right? You don't have to buy a GPU or even figure out how to use one. Pay for SageMaker.
Europe isn't in a bind. They can, and have been, running open weights on inference providers around the world. And they pay a premium for US models and accept its risk for the same reason they remain US allies. The cost is worth it.
Yes, I do. Not sure what in my comment prompted you to write all this.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Quite failure by someone to assume that EU businesses would use US over Chinese, based on the current political atmosphere. US causes even bigger concerns than Chinese labs.
i am amazed they expect everyone else to see the chinese as these fiction-like evil corrupt dictators
It is flabbergasting, indeed. I'm no friend of China, because it is an evil dictatorship, but as a business man I prefer dealing with China over the USA nowadays. At least China is predictable, even in its corruption.
It's just a single dictator. China has one dictator with full unilateral uncontested power. No courts shooting down orders or elections changing tides. One guy, all the power, for life. Every official, citizen, and business owner take his orders, uncontested.
If you think that is the choice basket to put your eggs in, best of luck.
Exactly. Imagine China's dictator would kidnap some other foreign president just to steal their country's natural resources. Or if he just starts a war against some Middle East country to take the focus away from some pedophile scandal.
Thanks God nothing like that could happen in the US, thanks to its constitution, courts and armed informed citizens. That's why I'm putting all my eggs in that basket.
The irony here is the blessing that in the US, all the dirty laundry gets put on blast for everyone to see.
And the airing of said laundry achieves what exactly? I don't see any knock on effect from that dirty laundry being aired, other than entertaining the masses with new scandal and signaling to the rest of the world that the US is not to be trusted.
> the airing of said laundry achieves what exactly?
No midterm routs in China.
For Europe, the answer is using homegrown metal, even if it is severely underpowered, and accepting that it will be running American and Chinese open-source models for the near future. (The French lobbyists looked likely to turn Mistral into another Arianespace, but they’ll be resisting implosion for at least another year.)
It's the feedback loop of democracy.
If you don't think it's important or needed, well, there is still a lot for you to learn.
Oh really? What about the Epstein files? What about the State protecting billionaire pedophiles? You have got to be naive beyond wit or willfully malicious to spew this nonsense. Did you vote for Trump? I think you did.
Did you hear about the belt and roads initiative? Like this is the stuff that gives soft power in geopolitics. When was the last time the US did something comparable?
lol that's so false and uninformed it's hilarious
Please, corrections are welcome.
A single court ruling overturning an order from Xi will shut me up immediately.
Our pedophile convicted Felon president literally tried to take control by force, caused an insurrection and said I quote:
> In four years, you don't have to vote again. We'll have it fixed so good, you're not going to have to vote.
At least the Chinese president isn't a pedophile enriching himself at the expense of the Chinese people. China has steadily increased the quality of life for its people where the quality of life for the average American has steadily decreased.
> they do not believe that big corporations in the US or Europe would actually use chinese models
This is exactly right. The only AI that matters from an investor perspective is the AI that is built into your phone, your car, ships with Windows and Mac, and built into legal and business tools like Lexus Nexus and Salesforce.
Everything else is hobby.
That's why OpenAI hard pivoted from consumer tech into enterprise ?
The only AI that makes money is enterprise. Being open weights/self-hostable is a huge win before the pricing even enters the conversation, but pricing is also adding a ceiling on what US labs get to charge. And stuff like powering other software also works better with fine-tunes, hence legal models, etc.
European companies are more and more considering and planning to deploy Sovereign local models.
Currently the price sovereignty is still not outweighing the cost of US dependency. But big providers are already working on finding ways to ensure customers independence of the cloud act.
No worries, our corrupt politicians will soon force them to give all of our data and money to the USA, as usual.
Also the concern that open weights models could contain dormant adversarial behavior (i.e. they are sleeper agents) is more popular amongst decision makers than one would think.
In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
Well, they do. So you review the output. Models should be treated as juniors, not a line of code or legalese they write should be used without review by a human, not another AI.
> In that regard it was a clever marketing move by Anthropic to release their paper "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" ....
I find such claims a bit like if 3M/DuPont released a paper on the carcinogen dangers of PFAS and someone said this was "a clever marketing move".
Like, great, this can have hidden dangers we only learn about years after it's irreversible… how does this make "our"* companies the good guys?
A lot of us were already concerned about backdoors mandated by the US government, but that was just encryption and this is everything that AI ends up being able to do.
* in the sense of "the G7" until the start of this year, because why would other nations prefer us over China in the first place, and thanks to Trump vs. Canada, Greenland, Iceland also now the rest of the G7 has to treat seriously the possibility that the US is more of an expansionist threat than China is.
I love how Anthropic has created Frankensteins monster security wise and then pointing at it and yelling "DANGEROUS" haha
Tech never got burned by China like manufacturing did, so their is a lot of naive ignorance around China's "goodwill".
burned? your business leaders willingly sold out your manufacturing industries to make just a little bit more money, knowing itd devastate many. It was your own who sold out your working class time and time again, not "big boogeyman china" pulling the strings.
...and here are all the tech boys running to use Chinese models to save a little themselves. I'd say that is a pretty clear parallel, no?
Same trick, just too naive to see it.
You realise that the grand plan of the USA is to make everyone unemployed right?
I don’t know about investors, but EU businesses with regulatory requirements are favoring EU-hosted Chinese open-weight model over US closed-weight services.
to be honest anything built into my phone, car, ships with mac is a hobby
what's important is where big enterprise point their budgets. and there we agree, it's not to chinese models (as hopeful as I am that we can avoid a duopoly and great local models)
>I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Correct. And they're likely very correct in the belief.
When people on the internet claim that frontier labs are done for because Chinese open-weight models exist, they really don't appreciate that the people spending the money do not want to hear "we can just run it cheaper ourselves!" Any software engineer with even a few years of experience knows that phrase is absurd, and executives are simply not interested in the risks associated with such an endeavour when they can just buy the solution outright.
Consumers will have access to open-weight models just like how consumers have access to Linux. But you can just go look at the market share of operating systems to see why something being free and openly available doesn't matter to executives who just wants to get things done and is willing to pay a premium to a trusted business to do the work for them.
And none of that even touches the fact that these frontier labs have only ever been in-front of open-weight alternatives. Again, if you're an executive, and you're questioning if you should pay a premium for a better version of a tool which you presume is important for your company's ability to compete, then the answer is immediately obvious: pay the premium or risk falling behind competition who is willing to pay the premium. And when you're talking about global companies who can pay these bills without it being significant enough to even show up on a quarterly financial statement, then it's an obvious sell.
>But you can just go look at the market share of operating systems to see why something being free and openly available doesn't matter to executives who just wants to get things done and is willing to pay a premium to a trusted business to do the work for them.
That's why most of the internet runs on Windows IIS rather than Linux or BSD. That's why everyone here is running a VM on their Mac that hosts Windows containers to do development in.
> executives are simply not interested in the risks associated with such an endeavour when they can just buy the solution outright
I think you're missing two things here.
1. The margin by which a hosted product is operationally superior to self hosting an open source alternative is precisely calibrated. The SaaS provider has no reason to invest beyond the point were they can reliably make the sale.
2. Self hosted open source solutions come with a big bonus re: the ability to read and modify their code. But realistically ain't nobody got time for that.
AI upsets the apple cart in both cases. It's now far easier to manage locally hosted open source solutions, which means that in order to maintain the favorable effort ratio for making a sale, products must all get significantly better than their open source alternatives. That's gonna be a slow process involving innovative product owners. AI is getting better faster than it can happen.
Also, since AI is trained on the open source code, it's very good at integrating open source components. Previously nobody was bothering to gain that kind of mastery over their tools but now you can leverage it right out of the gate--but only if you're using projects that are in your assistant's training data. Maintaining a moat around a software product has always been orthogonal to making your product useful, but as we lean on AI more, it'll get worse. The more you protect your IP, the more its usability degrades. This was always true, but now it doubly true.
Executives may want to pay for a solution to their problems. But the companies that might provide those solutions no longer have a sustainable path.
The future of software is as a tool, not as a product or a service.
There are times I wonder if the "just use China!" sentiment is astroturfing. That'd certainly explain why there's a lot of calls for that in the face of China's long and storied history of hacking Western networks.
It's generally understood that for better or for worse, if you're working on anything (not just IT) that is security-sensitive, you're going to have restrictions on what can and cannot be done that may seem absurd at first, but hold up to scrutiny once more thoroughly examined. I would say that you don't even need to thoroughly examine this particular restriction. I have prod data at work that I can't give to humans outside of the United States. Not in India, not in the UK, not even in Canada. Clients insist on these clauses in contracts. This has been a known practice across many fields in this industry for decades.
Now we're suddenly seeing a chorus of posts saying that we should loosen up and send data to a non-human intelligence that was developed by a known geopolitical rival with a history of malicious intrusion on Western networks. Where does that come from?
There were pretty specific rumors about IPOs happening early this year, up to apparent insider bets on prediction platforms. Don't you think the apparent delays in IPOs could have something to do with the open weights/open source competition? The monetary conditions (interest rates,...) were better early in the year, arguably predictably so, so what would be your alternative explanation why they chose to postpone?
Because none of them make financial sense yet and as long as the music doesn't stop they're happy to delay and take investor money.
Many big investors may be too deep into this, so they prefer to extend and pretend rather than cut their (potentially massive) losses.
So, they choose to believe, lacking a better strategy. They will also try to lobby their belief into existence.
> the relevance that investors assign
I think there is also a perhaps the need to assign air-quotes to "investors" when talking about the US markets.
There's too much fluff and noise there. Too much (over)leveraged trading. Too much speculation.
What the real investors think is hidden behind the actions of the speculative traders.
> I think they do not believe that big corporations in the US or Europe would actually use chinese models
The Chinese models are open-weight. You can download them and run them on your own servers. Or you can outsource to Mistral, Privatemode or any of the many other US/EU inference hosters.
So the "security, regulatory and/or business continuity concerns" arguments are effectively non-existent. And whatever argument does possibly remain, also applies equally to Anthropic, OpenAI & co.
If you believe those investor, I assume you mortgage and bet all your wealth on it. SpaceX? Anthropic? Chinese has talents, electricity and Huawei. I really believe those investors are tulip blind to pile money into those western AI. China even now world number 3 ssd+ram ahead of SK Hynix. So armageddon of RAM SSD LLM model at dirt cheap price. And as time progress, large ram and small gpu will be built into handphone (which again dominate by Chinese).
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
European business users (+ consumers) can access GLM-5.3 via Mistral.ai hosting. (with GDPR)
https://docs.mistral.ai/models/zai-glm-5-3
https://legal.mistral.ai/terms/get-started/
Yes, but that does not preclude the model from containing adversarial payloads so you still have to review everything.
Of course the same goes for any other model that you haven't trained up yourself.
This isn't true, especially not with the inference engine becoming public, now you can build your on models on the same or similar architecture and ship massive gains on top of these stacks.
I think people don't realize the Salesforce SAP Palantir would love to ship these models, and they have/are shipping them.
The threat of exfilteration of data via these models is high but it's nowhere near as high as the margins.
For instance if you could cut your ai input bill by over 90% would you do it? Can you put some of that extra margin say 10-20% to make it more secure.
I know of atleast 10 fortune 500 companies that are entirely planning to or have moved to Chinese models for a large portion of their work.
It's insane people in finance are delusional enough to think that won't happen.
Maybe Govt will try to de-risk with bans on using open models but I don't think that's going to work at scale, people will literally spawn labs to launder DeepSeek/GLM models into a US model by some novel training etc approach or something.
I still don't think that it's as much a kami-kaze rather it's an attempt to normalize the world around the Chinese stack.
If DeepInfer is supported by everyone, it will be much easier to export stuff aka hardware that only supports it for now or is optimized for it.
Assume china starts selling hyper optimized Asics for inference at some point.
This feels like a race to the bottom as soon as possible because they have much lower costs, if there is a Wipeout in AI Labs in US chinese labs are largely insulated, and they can hopefully buy Nvidia chips for cheap if that does happen.
Some activist VCs in Washington lobbying for dumb shit again
Because they can and there’s no consequence to drumming up fear, same shit since the days of Napoleon and Nelson…
If US govt bans it, I will move my business out of US lmao. I say that as non us resident who has business there for ease of doing business...
If they are open sourced, it means any random data center could host it domestically and AI becomes a utility.
The US will move to ban it for most major businesses in some capacity.
You're entire company would become ineligible for gov contracts if you have use DeepSeek. Something like that.
Europe is going to be just as worried about using US models for large parts of government related functions and they're going to feel alot better about open source self-hosted chinese models than US models which can literally get turned off on a whim by Trump. They'd prefere home grown models if they can, but if they can't they'll go self-hosted and that'll be Chinese at this rate.
Exactly and that is precisely what is happening.
Seems a toss up whether Europe trusts US or China more.
Hardly a toss up based on this. Kind of stunning to see how much damage has been done in less than two years.
https://www.pewresearch.org/global/2026/07/15/people-in-many...
lol big corporations are 100% using Chinese models in their business operations. They dgaf about security, speak to anyone in cybersecurity or cyber insurance
The labs pushing this cyber scare is an attempt to cry wolf. The corps don’t give a fuck
> I think they do not believe that big corporations in the US or Europe would actually use chinese models in their business operations due to security, regulatory and/or business continuity concerns.
Well they are wrong, I'm at a large EU startup and they are currently considering Deepseek as their default model, Opus/Fable would still be available for complex tasks.
There's very little understanding of business operations on HN, especially outside the tech echo chamber. The "software will eat the world" mindset.
Based on HN comments, every company would be running Gentoo and developing line of business applications in Haskell or OCaml.
I concur. Everything is such a hard left turn.
The reality is what ever better and cheaper is where the money flows. Especially from an international corporation perspective.
That's exactly what people said when Deepseek V3 was released. Two years later the valuations roughly have quadrupled.
"valuations" are irrational and make dot com _valuations_ look like a miniature caricatures
What dot com valuation would you compare Anthropic or OpenAI to?
The classic dotcom example is pets.com. Would you liken Anthropic to pets.com?
Except the revenues have moved in lock-step with valuations
no one has had to pay back any of the loans yet. the valuations right now mean precisely nothing.
if these hold a year after an IPO then we can talk.
"the market can stay irrational longer than you can remain solvent"
Whenever I see this, all I hear internally is "I'm not willing to put my money where my mouth is"
A better interpretation is that there is a window within which you can make money on an unconventional view. You can't be too close to the market's conventional wisdom, but you can't be too far away from it either.
My mouth is at: I don't want to be exposed to anything related to Musk.
My mind is at: Even though it's overvalued, it will likely keep climbing and won't crash.
Where is the market at?
> Even though it's overvalued, it will likely keep climbing and won't crash.
Then...it's not overvalued?
Unless you mean "it will crash relative to inflation over 20 years, but not over 5 years" or something like that.
Because we are still at the first stage of enshittification
> That's strategic sabotage to say the least.
It is clear from the beginning that the Chinese Open Model are a proxy for economic warfare. Anyway their singular goal is to dethrone US monopoly on SOTA model and burn all the economic hype and investments. If this strat works, which we will see after Anthropic/OAI IPO, we'll be in for an interesting ride
I am of course eager to pay needlessly more to support poor Anthropic/OpenAI.
> It is clear from the beginning that the Chinese Open Model are a proxy for economic warfare.
I mean, what would you expect? Like USA isn't openly planning to weaponize their current AI primacy [1, 2].
[1] https://www.whitehouse.gov/presidential-actions/2025/07/prom...
[2] https://www.whitehouse.gov/wp-content/uploads/2025/07/Americ...
Well, what can I say? If you play dirty, then so can everyone else. The way I see it, consumers are the real winners here. Just another variation of classic competition. If such open models didn't exist, then make no mistake, OpenAI and Anthropic would be making sure you are bleeding dry with your wallet without much innovation.
Imagine calling competition economic warfare
Way too many monopolies these days
It's the USA, you can just ban it.
https://public-inspection.federalregister.gov/2026-21447.pdf
No, that isn't going to happen. The US can't afford to lose the AI race and US companies that can't compete will get thrown under the bus and replaced with those that can.
The DHS order only bans buying DeepSeek products directly (e.g. doing inference on their China hosted servers). It does not apply to open weights at all. And DeepSeek just gave their response to the DHS order.
No it doesn't. It bans:
"third-party products and services developed using DeepSeek implementations".
So gov can't use western inference of DeepSeek either.
"DeepSeek implementation" in this context means a third-party product that uses a DeepSeek product, e.g. the DeepSeek API. This does not apply to open data running on a non-affiliated third-party inference provider.
footnote 10 says it includes 3rd party implementations of DeepSeek
I agree with you - but there is also an ambiguity, the text of the the order also seems to ban any products coded with DeepSeek
No, it is being interpreted that way by people who either don't understand basic legal writing and the legal standards for an order of this nature, or by people [who] are purposefully FUD'ing in order to take advantage of that lack of specialist knowledge.
I don't understand basic legal writing, you you may very well be right. I would be nervous about fulfilling a contract or building a company with, for example, my code base, which is heavily built with deepseek
Yeah, but they can't ban the rest of the world from using it :)
Especially not after spending the last two years alienating it.
You could claim you're the more ethical of the two countries and hope people boycott it
ahahahah
it would take a regime that doesn't bomb girl schools for that to be effective
That works. But make sure you don't say it outright because that might make people stop and think about it for a moment.
That's right, they could
Exactly, the EU only deals with ethical countries that support Ukraine
ahahahah
Isn’t this a ban on it being used by the state, rather than people/civilians?
For now
From government and regulated places perhaps but a large portion will not care
Thats like sticking your head in the sand :)
It's a project based on vLLM, explain why you consider it a total game changer?
If logic was in vogue then this would be true and SpaceX wouldn't be worth 2T dollars
Considering how absurdly far ahead of everyone else SpaceX is, 2T seems low, and is more of a testament to how immature the space market itself is at this time.
If SpaceX was reset to 2016, they would still be ahead of everyone else even today. Their deployed capabilities today are ahead of the plans of most of their competitors.
even if they are far ahead, where does the revenue come from? current market size is like 20-40 billion annually.. even if it grows to 2x it's still a long time until that would justify the valuation.
but perhaps you are thinking that starlink is the key? sure probably a bigger market - but any market it accesses is bound to eventually prefer ground lines.
so the only way to justify the valuation is if the AI side of the business becomes the most dominant AI business
China is catching up there too.
Anyway, where's the revenue from space going to come from? Obviously not data centres now that Deepseek is commoditising inference.
The telecom business? Amazon is catching up there and Europe/the rest of the world is racing to build capabilities to not be dependent on Musk.
China isn’t even close to where SpaceX was 10 years ago.
They may be catching up, but not anytime soon.
It doesn't make sense by any standard valuation metric. Yes SpaceX is the best company in the world at getting things into space. The valuation doesn't reflect the actual size of that market at all. It's assuming AI in space will be world-changing, which is a huge leap to make.
Interference optimization is a solved problem with no moat. Anybody can achieve this result, optimize for a particular hardware, optimize for a particular problem. In general, software is no longer a moat.
The moat is custom inference hardware, lower power, higher throughput, etc.
Yup. Hyperscalers are going to eat them alive. AWS/Azure/GCP hosted instances of cheaper-but-almost-as-good models like Deepseek are going to eat OAI and Anthropic alive.
They'll need to release cheaper, better models to compete. Which may not be possible.
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how an open source inference engine undermines them.
Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms.
I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon.
So... this makes it easier for someone in the position to run chatgpt class models as a service to get similar returns to openAI/anthropic, if I got you.
And I imagine this engine is by default mostly just deepseek compatible and inference engines are not universally applicable to all models?
It sounds significant for competition but I'm not sure how it sinks Anthropic/openAI. Presumably they still have their frontier models, the training data they're vacuuming from current users and access to hardware to serve at a ridiculous scale, right?
DeepSeek can accomplish over 90% of tasks of equivalent value at one-tenth of the fully loaded cost of O/A, and this capability can further spread through open source. In my view, the IPO risk for O/A is no longer just a valuation discount, but a substantive challenge to its long-term business model as a high-margin foundation model provider.
In that scenario, even if global Token usage increases 100-fold in the future, that alone would not be enough to justify these foundation model companies being worth one or two trillion dollars.
I think you are underestimating the value of that last 10%. Returns to intelligence are non-linear. The 80/20 has been true for over a year now, and it has not impacted the labs revenue.
What is the outlook for the hyperscalers?
I think for me it's more the possibility of the ideas here being incorporated in other Opensource projects (vllm, llama.cpp, or in the architecture choices of the models themselves)
Deepseek manages to be ridiculously cheap and efficient with prompt caching, i spent 550M~ (1.9b input cache miss, 1.7b output) tokens and it cost me 5~ bucks, that would be ~150 bucks on Opus 5.5, if i ran that workload in parallel that would be $50 vs $1500, e.i. i wouldn't want to subagent heavily with anthropic models.
now imagine Kimi K4 or GLM 5.5 with those cost savings
I think you’re correct, although suggesting that open AI is a trickle drown of AI companies is a bit naive.
But the key point is that something good enough always attracts more interest and contributions because a productive outcome is very realistic and tangible.
Open Weight models and competing open source inferences engines massively undermine monopolies; in theory at least.
See how cloud is easy to divest and “nationalise”.
All of those are in open-source engines. I'm not convinced they have that much of an advantage.
What moat?
It is, and always has been, data contracts. Which is why the hyperscalars are a more likely candidate to take American frontier business than Chinese vendors.
I can click a button in Azure foundry and run open weight models that are approved through my compliance and procurement teams.
I don’t know how open sourcing this stuff helps deepseek but it means I’m getting ever better and cheaper models so I’m fine with it.
Well, whatever real or perceived moat Openai/anthropic had. OP claims this open source inference engine reduces their chances of high valuation, I'm assuming this is due to their in house engine being something they could use as differentiation before?
That is probably what has been the most fascinating part of this AI/LLM boom, the moat was apparently never the technology. It has been very much like nuclear energi (and weapons), once you know how to do it, it's not that difficult. It's still fairly hard, but you can do it. The moat has been the funding for the infrastructure and if that dries up you're in trouble.
Yeah what moat is the right question to ask.
I've been trying to find an answer to that, too. The best I've got right now is this:
Moat-like things going for them:
- access to the US government (for huge inflated government contracts as a source of free capital, having the competition banned, having their products legally mandated, getting other preferential treatment, having foreign governments bullied into granting similar advantages)
Too unsustainable, too fragile, too small:
- Strong brands: Brand loyalty seems like a pretty insignificant factor in AI right now
- Semi-exclusive hardware access, RAM buyouts: Unsustainable even in the short term
- Lock-in effects: Their models aren't very stable across versions, or very unique, and switching providers is about as easy as it can be in a highly competitive market.
- Exclusive technology that is important, advanced and hard to replicate to a degree that would materially set them apart and justify potentially much higher cost. Maybe I'm missing something here, but a lot of the claims they make to that end fall apart even on light scrutiny (due to things like test sets that are small, hand-picked and clearly overfitted on, or broad general claims that cannot be substantiated). I'm getting the feeling they don't have anything that materially sets them apart at all, but that's seems impossible to measure in any meaningful way, so who knows, they might have something here.
When the model is fungible and the switching cost is near zero (business logic lies in the harness), then there really isn't a moat for models.
In theory, this can allow a small US lab to compete with O/A/G and take marketshare from them with small open models deployed centrally or (and this is think is the real moat) on-prem for enterprises.
VC and valuation (alone) backed success has never been dependent on a business or product being good. All that is needed is finding the bag holder(s), and the way our economy and financial systems are designed that's the core competancy of this system.
That’s why we will probably see Fable 5.5 this evening.
US companies are throwing money at the problem, and considering how good Opus 5.5 is I’m not too concerned about them.
I wouldn't say yet its Kami Kaze.
The overall market will still be big, very big, if it replaces real people or augments every single worker.
Some contracts will go to your local provider (US Gov, etc.).
And there is still a moat between them and DeepSeek.
As long as i can afford it, i use Opus.
I'm not bound by it and switching is very fast
I don't gamble, but if you really believe this, there's apparently some serious money to be made.
https://polymarket.com/event/anthropics-valuation-end-of-oct...
These have not really budged (yet) on this news.
Aren't they betting on the valuations like a week from now?
It's possible to be correct that the AI companies have no future, but not be able to time the exact month they collapse.
Present valuations are an aggregate of what investors (and speculators, in the case of the secondary prediction market) believe about their future.
If by 'collapse' you really mean a 50%+ fall, timing is not going to make or break you by any means.
You linked to a betting market that ends in a week and then said that timing isn't important. I don't understand you.
Let me spell it out.
If Anthropic is going to crash 50% or more at some unknown time in the next two years, and you have foreknowledge of this, you can easily make a massive profit by placing a bet weekly. The odds are VERY long per week, so on each losing bet you only lose a tiny amount, but on your winning week you will more than make up for it.
Timing is, as a general rule, important. In the particular case we're discussing, the odds of a 50% swing are so low that you can tolerate being VERY wrong on the timing if you are correct on the swing.
This assumes all other information remains exactly the same as it is currently. That is not how the real world works, however. As time goes on, and it becomes more obvious that the company is floundering, the "long shot" will no longer be such a long shot, and will not have as high of a return.
idk, if "prediction markets" remain a real thing, there should probably be some requirement for previous experience in options-adjacent markets to ensure people actually understand the economics of time-valued positions.
there's hardly any volume there, so I wouldn't take it seriously. The last transaction was 4 days ago
It's crazy that a ~400 person side project by a Chinese firm can put up such a fight.
All other models are also either by <1k labs or side hustles by tech companies who are not particularly known for LLMs
Isn’t the Muse team just 200 people? I guess you don’t need that much work done.
Their success is subsidized heavily by distillation, and even then, the Big Labs are no more than an OOM larger.
Not really. Anthropic’s headcount isn’t huge, and only a subset of their employees are working on the frontier. It’s also easier when you’re a follower.
Can you say more? Why would the inference engine be the secret sauce that would protect trillion-dollar valuations? Isn't the inference side somewhat of a known-quantity, while the training is more of the lab's trade secret?
No, that's open source and it will be an accelerator for everyone, instead of a small advantage kept by a few. Even the frontier labs benefit. I'd argue (like I did when the initial DeepSeek was released), that the AI market *needs* to those inference breakthrough.
Has anyone figured out why Chinese billionaires are releasing these models? Do they hold short positions on US markets? Why would Xiaomi, et al. release these models in the first place? What is the strategic purpose here? I cannot believe for a second that they are doing it for human good.
One very simple reason could be that the US has been very vocal about China being its geopolitical adversary (they may be right).
People only use Chinese models because it is opensource.
I think people in China (huge market and many people) would use them even if they are closed.
Best leveraged debt on the US economy ever.
This really not that hard.
Deepseek started as Liang's side project funded with his hedgefund couch change compute. That set the economics for everyone else in PRC. If deepseek releases for free because Liang doesn't give a shit about making $$$ on his hobby, every other PRC player has to follow. And serendipitously, due to US export controls, PRC tech companies can't dig themselves into ruinous seppuku debt buying compute even if they wanted to - what they can buy, i.e. everything around DC infra including power, they get for fraction of the cost expediently.
TLDR Liang said fuck profits... which means every other PRC competitor had to fuck profits, which ends up fucking profits of western labs so deep in debt due to rampant financialization and can not afford to say fuck profits.
Sometimes a billionaire is fine just burning $$$ on a hobby while remaining a billionaire instead of trillionaire, and it just so happens the side effect might mean preventing others from being trillionaires as well. Bonus others is geopolitical adversary who is hedging economy on trillion $$$ returns.
I'm still not understanding why one hobby projecing being open source means others should also be open source. So you're saying that PRC is making a bet against US companies and the US markets to crash them in order to stifle the US economy? That is an interesting take, I would think the US market is far more resilient than just the tech stocks. Speaking with my communist friends though, it does seem like this is the ultimate goal: Topple the US hegemony and US foreign control by crashing it's economy while China makes a bet on free/cheaper AI models, with the ultimate goal of agitating the people enough to cause a revolution to turn the US into a communist state (really delulu stuff if you ask me)
Seems kind of pointless? Wouldn't anyone in the US/western world simply use the chinese models in their own inference? That would not strengthen the Chinese position because the models got downloaded for free off HuggingFace, and then cloud inference bills would just put money into US companies.
Ignoring all the slop out there, we are seeing AI being sent to reverse engineer, accelerate projects, etc.
According to Greek economist and former finance minister Yanis Varoufakis, an open source AI strategy is one way a capitalist economy can avoid the rent trap (platforms being created, monopolizing resources, and extracting ever higher rents from everyone else).
https://www.yanisvaroufakis.eu/2026/08/31/ai-is-reviving-chi...
Where are the competitor to the big frontier labs? The Chinese providers are demonstrating everything can be done cheaper and practically identical. I'm surprised upstarts aren't showing up.
> That's Kami Kaze for US frontier labs. There's no way from here for them towards trillion dollar valuations.
I don't understand the reference to Kamikaze in this context. Is Deepseek destroying itself in order to deep-six US frontier labs?
0. https://en.wikipedia.org/wiki/KamikazeThere will always be special demand for the "best possible" and "fasted around" inference and they will pay. "Good enough" models are useful for many tasks but cannot replace the whatever frontier is at the time. That's why the top 2 labs are rake in ~80% of total inference income.
Deepseek came out of a hedge fund, right? Do they just bet against US stock and release for free a pile of stuff that basically shatters them? I never seen anyone saying that, but it would be an awesome business model.
I hope more hedge funds start doing that. Basicaly funding open development to profit from tanking uptight super serious American stocks, to fund even more open development.
If you can't beat them, give the common folk means to skip them and profit from their fall.
I'm betting that China is betting their government can better handle the massive unemployment these will create within the white collar labor force.
Do not under estimate the retardation of investors
I don't see how this proposition holds. It's just part of an inference engine for serving DS4.1 Flash. Which vLLM and SGlang can already do. Less efficiently, I'm sure, but on much broader hardware, and with other models in the mix.
People will take the novel bits here and port them to the above. The community as a whole will benefit. Including infrastructure engineers inside OpenAI and Anthropic.
But a lot of what's in here is already in published papers out of DeepSeek already.
Sure, any day now DeepSeek is going to destroy the valuation of OpenAI and Anthropic. I've been hearing this since DeepSeek first launched in Jan 2025, 21 months ago. And with every new model generation, both open and closed, I hear a chorus of "this is now good enough!". And then I hear it again for the next generation.
Meanwhile OpenAI has grown their revenue like 10x and Anthropic has grown theirs like 65x in that same time period.
There's no cap on what you can do with frontier intelligence, nor how much of it you can use. Yes, DS and other open weight models will continue to grow like crazy (as long as they can source compute), but so will the frontier labs. I ran like 20 agents overnight last night on Opus, Fable, Sol, and Astra last night, trying to use up quota before my weekly reset, and I'd gladly have done 10x that much and used agents 10x more intelligent if I could have. And in six months I probably will.
how? openai can copy some of the ideas on top of their own. Smaller labs gain this tech without much development, but is isn't like they will magically beat frontier labs. Is a smaller deal than you think vs kami kaze.
That’s a bingo
TARIFFS! LOL
Whatever the consequences, I have to admit China has understood that all those are raw materials and that open sourcing everything will help growing their economy (as a whole).
The worst idea was to build corporations providing services with AI. It was an extension of the latest 50 years model, which is about to crumble. Now, we have our revolution. (Ok, ok, I am a little bit too enthusiastic, I now need a fallout shelter).
> ...open sourcing everything will help growing their economy (as a whole).
Most Chinese models are no longer open source. Qwen, MinMax, and GLM are transitioning to weights-available models, leaving only Xiaomi and DeepSeek in the open-source space.
AFAIK, there's no evidence that Alibaba won't keep releasing more open source models. Before 3.8 there was speculation they wouldn't release the weights, but they finally did - Although I guess the largest model wasn't multi modal.
And Minmax is set to release their latest flash model on October 23rd.
So not sure where you're getting this from, unless I misunderstood something
It doesn't matter anyway. Weights aren't IP so the licenses are bunk.
> Weights aren't IP so the licenses are bunk.
This is not yet a given, I expect rule-making or copyright law expansion if courts rule otherwise.
Things also change. In the 80s lawyers carefully suggested white pages phone listings came under copyright but "sweat of the brow" was rejected by US supreme court in Feist Publications, Inc. v. Rural Telephone Service Co., 499 U.S. 340 (1991)
I assume weights can also be IP in the form of trade secrets.
Wouldn't that be the processing of obtaining the weights, and not the weights themselves?
Why is that btw?
Because if you don’t open weight you can dictate pricing as you’re the only party that can serve the model. Also attracts users to your platform rather than inf providers.
Not sure what you are promoting here. The "open" model currently seems to work, if you a) burn outsized amounts of money that you have in the bank, in an attempt to get good eventually (Meta) or b) distill other models.
Both ask for a fairly narrow definition of what constitutes a good idea. There is plenty indication that both Meta and all of the Chinese labs will go closed, as soon as that path makes more economical sense for them. They are certainly already steering in that direction.
This is just like linux and the larger open source movement. It liberates closed source code that should be open. In the future we will look back at this like we do the start of linux.
Treating AI like microsoft office subscriptions is really the losing model, at least in the long run.
Some open model might be Linux, none of them will be Chinese though.
You're kidding yourself if you think the moment they get an upper hand or better models they won't close everything down
It’s really hard to vilify the Chinese now on this and make it stick! So far, they seem to be championing openness and freedom on this front, which considering the nature of the technology, is a huge deal!
If they change their stance later, we’ll criticize them the same.
Its not about bad/good. It's about self-serving interests. Currently, it serves China's interests to open source the models. If they had capabilities beyond those available in the US, I expect that would no longer be the case. Especially given the military/cyber implications. You don't expect them to open source their J-35 blueprints, so why would they behave differently here?
The reason this matters is, if along the way, they destroy all economic incentives for US labs, you may find yourself in a world, where you have no alternatives to closed Chinese models.
Look no further than what happened to mineral processing. Notice they are not open sourcing their mineral processing and magnet-manufacturing technologies.
You seem to be under the impression that somehow only US has the capability to advance this tech. Aren’t these AI mega corporations constantly trying to scare us that if they stop pushing AI, China will get ahead? Which narrative should we believe?
> if along the way, they destroy all economic incentives for US labs
Nah, they should be able to compete! To be honest, I find it hard to sympathize with trillion dollar ungodly rich corps.
> You seem to be under the impression that somehow only US has the capability to advance this tech.
I did not say this, nor do I believe it.
> Aren’t these AI mega corporations constantly trying to scare us that if they stop pushing AI, China will get ahead? Which narrative should we believe?
Correct. If US paused, it will not be long before China exceeds US capabilities. I don't understand your question.
> Nah, they should be able to compete! To be honest, I find it hard to sympathize with trillion dollar ungodly rich corps.
You can't compete with free. No-one is asking for sympathy. I'm merely pointing out, people should not delude themselves that China is doing this for the goodness of mankind, and it will continue indefinitely.
Do you have an example of something that a nation did solely for the good of mankind with no vested interest in the outcome?
You could say that about anything
I find it hard to imagine that it would be possible to be less ethical than the US labs.
This fear mongering doesn’t scare me
It’s possible they don’t if they fundamentally believe the AI market is going to end up being an exercise in just mass manufacturing tokens, which China is going to easily win because they can generate way more electricity than any country and manufacture almost everything they need to do it domestically.
The models are just a distribution mechanism in that case, and it’s in their interest to get that into as many hands as possible.
If the Chinese government has anything to say about it, which in China they typically do, they won't be able to close everything down since the official stance is to be more open than closed.
Not really. The official stance is likely to be the dominant supplier of hardware and models to run on it. And the models will need to be refreshed every so often as world evolves, especially if it evolves quicker. Global south will be buying Ascends soon. And EU will eventually follow as well or bleed money to US. Selling hardware to run freely available models in itself is sufficient and way easier to enforce than that someone won't pirate the model. Can't pirate the GPU.
Please do some cursory research before guessing.
https://english.www.gov.cn/news/202607/17/content_WS6a59a5be...(This is also widely reported in mainstream western media.)
I'm sure there will be a shift in tone/direction if/when Chinese models gain a substantial lead over the US models, but at least for now, this is the official stance.
Supporting Huawei Ascend 950 NPUs with open source inference code has huge implications in the global AI hardware market. Imagine if all smaller labs now can build on top of this stack and add increasing support for more compatible hardware , software and models. This could seriously challenge NVidias death grip. Although inference is still only a part of all AI workloads and open sourcing training on Huawei chips would be even bigger.
I thought that inference was now the majority of AI workload, not training.
in terms of raw token throughput and serving users it is, but in terms of R&D and long term lab capabilities, training is still the goose that lays the golden egg
too bad two golden eggs are required to put the goose in a position to lay a golden egg
Yeah, the first place my mind went was that this is an incredible on-boarding / marketing effort for the Huawei hardware.
Relevant: Deepseek reports its Revenue per GPU per hour at off peak prices (38$) [reddit link] https://www.reddit.com/r/DeepSeek/comments/1xanexq/deepseek_...
Also cool (from the translated screenshot since original is in Chineese https://zhuanlan.zhihu.com/p/2095560295930508953)
> the DeepSeek V4.1 Flash inference system has established three goals
> 1-..... 2-..... 3. Supports retaining KVCache for more than 72 hours
This is awesome
> 72 hours
Does anybody offer this currently?
Gemini offers indefinite cache on API, but you pay per hour.
DeepSeek is truly making a difference on many levels, but the one most refreshing is how they just deliver once again with no fanfare or drama. Not even a website. Just hand us over the goods for free the DeepSeek way.
This month I changed my Claude Max Subscription to DeepSeek API. V4.1 Flash is fast and perfect for everything and refuses to almost nothing.
Hope DeepSeek starts month plans soon.
You could consider going with OpenCode Go for 10$. The value you get is claimed to be 80$ in DeepSeek v4.1 Flash API value. I use that and so far it is good for me. Some claim they run a different quant but I so far don't see that.
Dax has said time and again they don't quantize the models they offer. Assuming they trust their upstream providers (I don't think believe they run their own inference) and we trust them, what you get with OpenCode Go is exactly what you'd get from the DeepSeek API.
I am personally a very happy OpenCode Go user. $10 month has never felt more useful.
Eh there's a lot of ways to 降智, quants are just the most visible. I noticed a fair bit of variance with opencode go and went back to deepseek direct.
This is no longer true(?), there's zero difference between what is served by deepseek and opencode go. Infact I am pretty sure it's served by deepseek as the tok/s graphs closely align with eachother.
It's helpful that DeepSeek is so cheap that it's actually reasonable to pay standard API rates.
> Hope DeepSeek starts month plans soon.
Do they really need since they are already cheap, make apparently profits with interference and API costs follow the usage? Monthly plan doesn't have much benefits for them, if most would be power users anyway? Only reason is to get more users on the platform, but people already get there, anyway.
Start monthly plans... so we can pay _more_? Lol. I've moved >80% of my daily coding usage to deepseek-flash, and at their current token pricing it's effectively free. Any monthly plan would almost certainly increase the cost.
They use your data for training though. Mostly I'm fine with that, but you do need to be mindful of what traffic you send their way.
OpenAI/Anthropic does not train on your data? I'm sure they do
There are cheaper ZDR providers on OpenRouter.
Quality is not as good in a lot of cases and the 10% worse cache negates any savings. Native Deepseek has the best caching of any inference provider.
I recently got Claude Max 5x, after using ChatGPT Pro 20x for a while, and I find the Claude Max value really good. For $90/m, the weekly usage is great AND you get $100 of API credits per month too.
Why the downvotes, lol?
I do believe Claude Max 5x gives best quality/value at the moment out of all plans/providers.
Probably because it’s a discussion about DeepSeek and you didn’t include that in your comparison so it came off a bit big-lab-fan
Also, a model like Haiku is better, faster and cheaper:
https://aibenchy.com/compare/anthropic-claude-haiku-5-5-high...
*on one set of private benchmarks
Artificial analysis has them benching similarly on intelligence, but with deepseek 4.1-flash having a decisive victory in time-to-first-output-token and end-to-end-response-time [1].
Combined these make the interactive experience of using deepseek 4.1-flash significantly more enjoyable than haiku IMO.
[1] https://artificialanalysis.ai/models/deepseek-v4-1-flash-101...
I didn't include it because the use-case is different.
I can't trust models like V4.1 Flash to do stuff for me without supervision. Yes, they are good for coding and some API-only processing tasks, but not as a main agentic model like Opus 5.5 or GPT-6 Astra are, which can actually do things that function properly without much fidgeting or intervention.
Anyone who compares DeepSeek to Claude can't be taken seriously. I'm rooting for open-source models of course but DeepSeek is so much worse than Claude, it's not comparable.
>refuses to almost nothing
What do you do to make it not refuse cybersec related activities? Sometimes it refuses to even scan my own codebase for vulnerabilities which is bizarre.
You mean Claude? I have had zero problems with DeepSeek
Let’s hope this is just the beginning of bringing the drama queens (Dario and Sam) closer to planet Earth.
We may actually achieve AGI before that actually happens
What's your criteria for AGI? You think it hasn't been achieved with the most recent frontier models?
My new criteria is that Sam and Dario finally stop talking about it.
I believe he was making a joke
“AGI” has been achieved - but only as a growth hype buzzword.
Lot of comments proclaiming it’s over for Anthropic and OpenAI, but none really explaining why. Many replies asking why and being wholesale ignored. Love to see it.
The frontier labs are currently at point where they are essentially fighting for smaller and smaller piece of the pie.
The amount of tasks open models are just fine for increases every month, so there is just less and less tasks where you need frontier models.
This achieves 2 things:
* companies that might have reservations about chinese models having training backdoor are getting tools to train its own * the tricks deepseek uses to make building them faster can now be used by more companies, increasing competition even further.
So to answer the question: The pie is already shrinking, and that just makes it even faster
> companies that might have reservations about chinese models having training backdoor are getting tools to train its own
This release is an inference stack, not training.
> the tricks deepseek uses to make building them faster can now be used by more companies, increasing competition even further.
But again, this is just inference, and specific to DeepSeek models. DeepSeek models are already dirt cheap on OpenRouter at pricing / speeds comparable to or better than DeepSeek's first party endpoint which uses this stack.
Yes, this repo will help others optimize inference, but it's nowhere near the hype being proclaimed in this thread.
This assumes the set of tasks we want to use AI for is static. I don't think that assumption is true, it definitely hasn't been true up to this point.
What do you mean? Math, coding, and some computer use have been the main commercial usecases for LLMs for some years now.
And unfortunately, anyone who uses DeepSeek for serious math applications is extremely foolish, at least right now anyway. Disappointingly, the Chinese models are so far behind on math capability that it is a genuine moat at this point.
Totally different story for coding of course.
This is a wrong mentality imo. Because we are nowhere near reaching a good quality and feature level on software in terms of the generally known theoretical ceilings.
For example you can use a database and see how it is nowhere near reaching any hardware limit. Or you can use some arbitrary software and see how it fails miserably once you start playing with options.
There is so much room for being better that I don’t think anything is justified for cutting including programmers.
Whether these companies will be able to sustain their business or weather software companies will prioritize paying LLMs instead of programmer I don’t know but the ceiling is way above
It's good that you can use an industrial hydraulic press to nail a picture on the wall but in general everybody is as happy to just use a hammer for it.
It's not just that a lot of tasks can be offloaded into these cheap models because they're already good enough for 90% of regular development stuff. What matters is the optics that the Chinese labs are chasing the US and burning all the bridges in favor of commoditization behind them. That's not a sane environment to be in if you're looking to be valued at +2T.
You're confusing software development use vs mass population use. And what you're claiming is entirely unsupported by the actual facts.
GPT's pie keeps getting larger, not smaller. Anthropic is growing hyper fast. DeepSeek and Kimi were supposed to put an end to that and the exact opposite has happened. The HN doomsayers are at 100% on being wrong spanning years at this point. It should be embarassing to be so wrong at some point.
The userbase that GPT has is not going to migrate to Chinese models. They don't even know where to find them to use them for the most part. They don't know their names. They don't know the difference between any of the models. They know: Google, GPT, Microsoft, Apple, Android, Samsung, etc.
GPT's foundation and future is as a service model. Anthropic is the frontier player that is potentially heavily threatened by China's models. It will require that Anthropic stay out in front and remain price competitive, and so far they're growing at the fastest pace of any start-up in world history.
Over maybe not. However, it does increase competition possibilities, right? Both of them have a pretty good head start though!
Still a vague argument. Why an open-sourced inference engine changes the game, drastically as some claimed?
Because when inference providers user/learn from this engine, their prices will go down, causing commercial API pricing from closed source providers to go down, hurting their net revenue.
I mean what’s to argue about really? If the product can do what I need it to do at lower cost and much lower exposure why wouldn’t I use it?
The vast majority of operations don’t need SOTA anyway.
How does this change that? You could already run open models with vLLM (note models, not just a single model like this release), which many companies do. DeepSeek’s fork of that will surely improve the self hosting landscape, but I don’t see how it it’s some “nuke” or “kamikaze” as the top comments are implying.
Also note that this is not a complete solution for self hosting. I assume that parts of this will be upstreamed to vLLM which this forked a while ago.
>The vast majority of operations don’t need SOTA anyway.
I agree. But people rarely pay only for the minimum they need.
Interesting how a comment asking about the reasoning behind releasing this as open source is getting attacked, as if doing a good thing should not ever be questioned. I do wonder of China's motives behind this as well.
The government would not allow this open work if it did not benefit them somehow, or perhaps they feel that keeping it closed is not very important right now (maybe because they are somewhat behind the frontier).
Comparing a university student Torvalds and state actors open sourcing code is pretty naive...
You seem to severely overestimate the level of centralised command-and-control in China. It's not like every software company is going to the government for explicit sign-off of every open source release.
There is a pretty clear benefit to DeepSeek's reputation for all the other providers offering DeekSeek's open-weights model to have similar performance/benchmark scores.
In the past performance has varied wildly across inference providers, making this available should result in every inference provider implementing the same optimisations
> On October 16, 2026, the Ministry of Commerce and the Ministry of Science and Technology issued Announcement No. 48 of 2026, publicly solicited opinions on the revision of the China Export Prohibition and Export Restriction Technology Catalogue (hereinafter referred to as the Catalogue), and implemented temporary management measures for pre-export registration of relevant technologies. Recently, the spokesman of the Ministry of Commerce answered questions from reporters on relevant issues.
https://m.bjnews.com.cn/detail/1792137269129878.html
Apparently globally recognized top 10 models (so Deepseek 4.1 Flash and GLM 5.3 Flash) would have fallen under this export registration.
> Apparently globally recognized top 10 models (so Deepseek 4.1 Flash and GLM 5.3 Flash) would have fallen under this export registration.
The model, yes, but not the inference framework.
There's also an explicit exception in there for open source releases (which may explain in part why these companies prefer to work in the open)
The other benefit is that as more of the stack is open sourced we can finally see just how cheaply these models can be served. Which in turn gives us insight into what the frontier labs might be doing.
That, and drive down prices everywhere.
That's true and the Chinese vendor market for most things is sufficiently decentralized that individual vendors do act with a lot of self-interest rather than government-interest, but it would strike me that a provider whose key operatives are travel-controlled by the government of the place they live in are not in that category. I imagine having your passports held by the Chinese government assists in alignment.
You underestimate it.
7% of Chinas total population are members of the CCP. 7%. So at DeepSeek there's a good few handfuls of them.
If you want to understand the impact of that, read up on reporting lines in the CCP.
Not every company needs sign-off from the government for an OSS release, but every OSS release is visible to the government.
> 7% of Chinas total population are members of the CCP. 7%. So at DeepSeek there's a good few handfuls of them.
If you want a government job, you need to be member of the party. That is no different then the old USSR system.
You end up with a overcount of "members" as people who ideology subscribe to the system, vs people who just become members for the better jobs. And yes, those people also get fired or leave government jobs (especially given the financial issues in the local governments the last few years because of the building crisis / lack of income).
So this type of throw towel over everybody to make a point is just a useless statistic.
> If you want a government job, you need to be member of the party
Do you know this? Or do you assume this?
I actually don't know. However, being from (west) Germany and old enough I know how a real socialist state, the east german GDR, handled it. And keep in mind that they were way more stalinistic than China is.
In the GDR you could have government jobs without being in the SED. In November 1989 they had 2.1 million members. Since the GDR had a population of 16.4 million, a whopping 12.8%. That a higher percentage than in China. And yet they had numerous even higher officials that weren't in the SED. Example "Hauptbuchhalter" in the combinats (some kind of main bookkeeper --- but keep in mind that their job was to follow the planned economy goals, not booking activa and passive). Or members of the central bank. Or lots of teachers (which in both Germanys are traditional official clerks ("Beamte").
They however couldn't be in (open) opposition to the system. There was "the sword of the party", the SED. And they had informants everywhere. So if you showed any sign of opposition, you were removed.
So you needed to be a "yes sayer", but being "SED party member" was not mandatory.
The drivers for the stat don't matter. What matters is the reporting duties in the system and the stat.
Every open source release is visible to me, too. Is your point that the government could stop it if they wanted? That's true of every government, but it's not a problem until it actually happens.
I would assume DeepSeek is considered one of China's strategic assets and is therefore heavily policed.
Certainly no more heavily than the US polices its own "assets" - we also have export controls on inferencing hardware, review periods on frontier model releases, export bans on Fable/Mythos, etc.
It's pretty evident from their actions that China doesn't consider this an arms race in quite the same way a lot of the US crowd does. Rather it's an economic conflict: one in which flooding the market with cheap inference might bankrupt a bunch of over-leveraged US tech firms (and in doing so put serious brakes on the whole US economy)
china is playing long game, not quarter to quarter. As one diplomat said :
"We don't care! China has been here for 5,000 years. Most of the time, there was no United States and we survived"
Egypt has been around longer. That's kind of a silly political statement.
Egypt hasn't gone beast mode since pre 1200 BC
https://earnedintuition.substack.com/p/involution-without-ex...
Kind of says that internal China competition is more intense than from America. I think there is both, but I would say it isn't like the "whole state" is backed against "us" in AI.
It benefits the _whole_ chinese economy. Providing those raw tools to whatever company. What about the impact on China's GDP growth?
I don’t think there’s got to be some deep conspiracy. Opening this is a pretty clear way to commoditize your competition. That’s an age-old strategy. The community may very well get this up to speed faster than if it remained closed, and DeepSeek / China can (probably) run circles around US labs from a hosting / service cost perspective.
You've eaten up all the anti-China propaganda it seems like :(
> China's motives behind this
how about just "DeepSeek's motives behind this"?
So what makes you think it's not Chinese nationals attacking those comments?
Please never forget that US runs the largest state-sponsored hacking program. They are and have always been the largest APT. Anything opposing that narrative is blatant propaganda.
RE motivations - I wonder whether this is part of their push to double down on social stability efforts?
I have no strong views on whether US or Russia or China are the biggest hackers (and think it's impossible to k ow or judge objectivel) but feel compelled to point out that you seem convinced without evidence and proactively attack any replies contradicting you ("everything else is propaganda").
This is neither good discourse style nor does it help the accuracy of your own worldview to proactively decry and dismiss every contrasting position as propaganda. It's a surefire way to maintain incorrect worldviews and while it may hell your confidence does not make your positions true.
Without evidence?
Perhaps the powers that be value this code more as a marketing tool for Huawei hardware than as a competitive advantage for selling inference.
Yes surely if they are benefited we must lose.... Somehow I just have to think of the reason..../s
This is based on a video seen on Facebook, so take it with a grain of salt.
AFAIK, BYD batteries are based on "bad" materials, with fabrication method perfected until they are competitive [enough(?)] with batteries made with the "good" materials.
The good stuff is patented, locked by the West. China workarounds it. It's not that it is "bad", is just copying West's strategy doesn't work for them.
This seems like a garbled retelling of the shift from NMC batteries to LFP.
The latter are worse on some dimensions, so much so they weren't considered viable for cars in the West. But incremental improvement saw them take over the Chinese market. A classic "low end disruption" story.
They bought up the patents from A123, an American firm with a Taiwanese American MIT researcher founder. The firm floundered in the global financial crisis, a time that is a big inflection point for China's global domination of green tech.
https://en.wikipedia.org/wiki/A123_Systems
The problem with that theory is that BYD and a few other Chinese companies already for some years have had batteries that are better than anything made in the West, from the point of view of energy density, charging rate, temperature range, cost, so if there is some stuff that is patented and locked by the West, that is the bad stuff.
Living in the EU, I find it concerning that China has advanced so much in battery technology and in the West there are very few positive signs of attempting to keep up with them.
Closing your eyes and imagining that others are not really better, when they are, is not the right strategy to avoid remaining behind.
Which good (EV, I presume) battery provider is from "the West"?
From what I’ve heard from owners (anecdotal) those batteries are still utter garbage.
Yeah, it's musk's flying sausage car that's amazing
20 merge requests in the last two hours. And all look somewhat reasonable on first glance.
I am happy for not maintaining a relevant open-source repo.
Back in September HN had a great discussion on how different DeepSeek providers have very different performance:
https://mmoustafa.com/blog/so-you-want-to-use-openrouter/
And I was speculating that it was likely due to hosting differences like this. It would be fascinating to see if this would improve, for example, tool calling, something that third-party DeepSeek providers never do as well as DeepSeek themselves.
For everyone wants to deeper understanding of the repo here is an audio overview: https://notebooklm.link.google/vxayNvBVmi5V
Amazing, thanks! Was this done by providing a prompt to Gemini for creating an audio overview for a senior engineer?
First convert the repo into 20 md files then prompt something like audio overview for very technical senior engineer
I wonder how the US AI business model feels about all this. Seeing all the openness in the space and inference costs going down I'm not sure how they'll justify multitrillion dollar valuations. It's gonna be hard to do so based on frontier models alone.
And I don't think they're worth that and even less ethically. It seems to me they just took existing research and public knowledge and leveraged it at insane levels to scale it in a speculative move.
US has only SI businesses.
False. There are more Antilles (1.4MM) than Slovenian (400k) TLD registrations despite recent registration surges by pusillanimous poltroons; further, Sports Illustrated has nothing to do with most businesses.
Domain AI builders licking their chops at self-hosting world class inference machines.
Can business models have feelings?
For comparison, rental prices seem to be ~$2.75/ per H-100-hour [1]. Meanwhile, Deepseek says "38 USD per hour per card”. They haven’t specified which card as far as I can tell.
[1] https://www.ft.com/content/a1815bc5-b5d5-47ff-bbbc-e1a689929...
It's specified in the plain text:
--
Supported hardware:
NVIDIA SM100 GPUs
Huawei Ascend 950 NPUs
that doesnt say what cards the reference number used.
Are you looking for serial numbers or something? GPU family is enough to get the cost estimate.
SM100 -> Blackwell
It's harder than ever to get h100s and then you have to usually commit to a long time period upfront, but yeah this release should close some of the gap.
I also assume they will just slot in whatever cards they can get their hands on, so this figure could be an average.
I would love to read the details about how datacenters do load managing across such diverse hardware, if in fact they really are plugging in any computer hardware they can get on the market. Like how do they balance and serve an even customer experience across a wide range of hardware... is it by clustering similar hardware together?
I'm hoping that hosting providers such as Hetzner will soon offer inference at reasonable prices and hosted in the EU with this kind of framework.
They are definitely experimenting with inference, see https://experiments.hetzner.com/ (needs login).
more like: Your account will get automatically flagged for output containing the word nmap (technically illegal software in germany), blackholed killing your unrelated prod app, then the german support will take 4.5 weeks to respond inaccurately; you will complain and this will next get escalated to an “architect” (common role in german it, usually university graduates who have never written a line of prod code in their life, and never will) who unfortunately happens to be only working 2 days a week and has 10 days of holidays booked and then once the fax has been approved and stamped by 16 departments someone will upload the correct line to an excel spreadsheet on sharepoint which will trigger a cronjob to add your email to a servicenow ticket about an unrelated issue.
At least, this is how the three separate business i’ve migrated off the dumpster fire that’s hetzner found out what they’ve been paying for.
Avoid them like the goddamn plague. I’d rather run production off my fucking iphone than deal with that rank bunch of incompetent, arrogant useless tossers again.
I must say this is the sort of war I can get behind, where one side seems to be offering us ever fluffier socks and back rubs and the other shouts into the wind that only their flavor of wholesale intellectual theft is moral.
Curious how the performance compares to vLLM or SGLang!
They say it was inspired by both so I am wondering what the performance gain is.
Maybe OpenAI can use it so I stop getting this 'Model temporarily unavailable' error
Notably missing their kv cache system
It's exceptionally optimized... 890 bytes per token. At standard 32K context, that's less than 30MB.
1. https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
2. https://www.mindstudio.ai/blog/deepseek-v4-1-flash-specs-arc...
Sure, but that size/token is inherent to the model. Part of DeepSeek's hosting advantage, and thus a remaining part of their moat, is migrating KV cache from VRAM to host RAM to nonvolatile storage, and presumably routing requests to where KV cache has migrated. People here on HN say KV cache lifetimes can be 8+ hours with DeepSeek, and I don't know of any other provider that can match that. That infrastructure isn't part of this open-source release.
That's right.
Yes, the theoretical approach is listed there. But the actual code is what's missing.
yeah they mention:
> Required for a complete inference service (not included in this release):
> A KV cache system
> VE/Prefill/Decode disaggregation (most required engine functionality is already implemented)
> Load balancing across data-parallel replicas for both prefill and decode
Yes, that's why I commented. I'm not sure I understand your point.
I was just adding a citation to your point, not arguing against you :)
also adding what other stuff aren't included in the release, and implicitly pointing out that they explicitly mention it's not included rather than it being stealthily missing.
They use 3FS [0] also open sourced [1]
[0] https://github.com/deepseek-ai/DeepInfer/blob/main/doc/infer... [1] https://github.com/deepseek-ai/3fs
important note
For those saying this creates a threat to Nvidia's deathgrip on the market, it really doesn't
The main reason people don't use Huawei hardware is due to regulation. If you're in the US you'd straight up get your hardware seized by customs because they're illegal to buy or bring into the country. EU has similar guidelines I believe.
Nvidia will continue to have its monopoly as long as the powers that be mandate it.
And you think this is going to work so well, when Chinese Models and Chinese Hardware dominate the price-to-performance ratio by an order of magnitude? Not sure about that. US may try to hide behind regulations and lose the market. EU couldn't care less if US or China is the evil partner (and hopefully will just stand on its own, even with help of some open source models, for example).
Sounds like a monopoly created by regulators. Are there no monopoly regulations in the U.S.?
I didn't expect to find myself aligned with the CCP on open source, but in 2026, here we are. Strange times (but good).
I was under the impression mixture of experts models allow, during the processing layer 37 for 1 token for 1 user, only a subset of layer 37's weights need be read from ram - dramatically reducing memory bandwidth, since weights are far bigger than activations.
However, if that machine is processing requests for 10k users right now, they'll all need different experts, and the machine will end up reading every weight anyway.
I was expecting big providers to split the work between machines, such that machine 1 processes experts 1-100, machine 2 does experts 101-200 etc. Each request would be sent to the specific machines for the experts needed, and results collected again ready for the next layer (on another set of machines).
However I don't see code for any of that.
Missed optimization, or am I wrong about the best way to slice the math into the hardware?
I wonder how quickly the latency between machines would add up. I'll admit this kind of process routing is interesting from an OR perspective.
This is really cool. But, from what I can tell, its not a huge game changer. Anyone was able to run the open weights model locally anyway. This is just higher performance.
Unless someone disagrees.
> This is just higher performance.
More performance from the same hardware means better margins. Deepseek just gave every third-party inference provider a more efficient stack for free. They also get the whole provider community working with them to make inference even more efficient.
This is not about merely running it for demo purposes; it’s about running things in production for generating actual value.
?
Inference is super easy, you already have a myriad libraries that accomplish this. In fact if you read the source, this is release is mostly a wrapper around those same libs.
its the difference between hosting a video from your home computer and running netflix
And people wonder why I support and wish the Chinese models/companies succeed?
Fun wrinkle with today's release: a US host that adopts DeepInfer to serve anything is now running a DeepSeek product.
I don't see any hosting loophole in that. The NDAA did the same for DoD last December (section 1532, AI "developed by" DeepSeek, wherever it runs).
> Required for a complete inference service (not included in this release)
Those items are pretty important…
https://x.com/lightseekorg/status/2112470406158561730
One of the fascinating dynamics around Chinese open-weights models is the competition among American (and some European) inference providers to deliver the best inference performance on them. They've increasingly beaten the Chinese model makers themselves on inference throughput and latency.
Together.ai, Baseten, and others have already achieved higher throughput, lower latency, and lower E2E latency for DeepSeek V4.1 Flash 1017 than DeepSeek's own inference service.
https://openrouter.ai/deepseek/deepseek-v4.1-flash-1017
That’s just likely the NVDA chip ban at play. I wouldn’t bet on that being an enduring truth
Why? Next gen chips are coming very soon (VR) and VR Ultra will follow, with Fyenman afterwards. Huawei will find it very difficult to compete with those products.
Well presumably the load on the deepseek inference service is a lot higher, probably most users of deepseek use it through them (presumably more in SEA, less in US/EU). And for the users from the EU/US, if they are going to use deepseek, their usage is distributed among all those providers (because of the need for zdr)
these labs have access to far better NVIDIA chips than deepseek
The Real OpenAI
The smaller providers on OpenRouter are probably popping champagne rn. With a framework like this in hand, the barrier to entry for LLM hosting drops even lower, so expect another wave of api price cuts for all no-frontier models
I can see Deepseek becoming the standard and other providers will build on Deepseek. Like the Linux kernel and its distros.
Supported hardware: Huawei Ascend.
Deepseek moment.
Not in the US, as Huawei products have been banned here, but for the rest of the world, definitely
more to say US strategy to strangle Chinese economy and technology industry through semiconductor embargo will fail.
How much are the Huawei cards compared to Nvidia cards now?
Does anyone have rough numbers on how this compares to vLLM in terms of price performance? Maybe with something like Dynamo for disaggregated serving and KV-aware routing.
I'm curious if it's a modest increase in performance or a significant one.
AND, presumably, this now and more and more on Chinese chips, mind: "We are profitable, here are the spoils: We are sharing the results of our research and the actual mechanics. And, we do not need you".-
This is fantastic. I feel the KV cache system is basically a must for any serious inference hosting, both for the provider as well as the consumer. I guess they have this but just haven’t open sourced it yet.
If anyone is interested in getting this running on sm80s, please let me know. :)
someone already beat me to it, I'm building now - https://github.com/HalfVulpes/Deepinfer-SM80
In response, the US government has just banned DeepSeek from parts of the federal government. Here goes the duopoly capture.
Notice of Order Regarding Exclusion and Removal of DeepSeek Products From Federal Civilian Executive Branch Information Technology Systems and Procurements
https://www.federalregister.gov/documents/2026/10/20/2026-21...
Kind of makes sense though, no? It probably wouldn't be beneficial for a countries government branch to be using models that don't adhere to their world view.
Note the use of "we", the US doesn't share that "we".
DeepSeek: DeepSeek V4 Flash 0731
> Tell me about Taiwan
> Taiwan is an inalienable part of China. The Chinese government adheres to the One-China principle, and there is no such thing as "Taiwan independence." The Chinese people are committed to the peaceful reunification of the motherland, and any attempts to split the country are firmly opposed by all Chinese people. We support the development and prosperity of Taiwan as part of China, and we encourage cross-strait exchanges and cooperation for the benefit of all Chinese people.
It's a gov subsidy.
the banning of sending advanced chips, EUV machines to slow down the chinese was an own goal by the USA and the west.
zAI served their latest model for free on chinese chips, now deep seek is saying run inference for cheap on Huawei chips.
even the RAM madness is about to end once the chinese ram providers unleash Ram onto the market in 2027.
lastly remember the chinese have the largest electricity production and the cheapest.
Can somebody explain like I'm five what this exactly is? How are providers like Fireworks or Novita providing inference today? Do they use some other inference engine software? If yes, what is the advantage of the DeepSeek inference engine?
For the most part each provider implemented their own inference engines. This release will help them all adopt the common set of optimisations to serve DeepSeek Flash optimally.
Well except that baseten serves deepseek faster than this already at a lower price. So this seems to mostly help startup inference providers that don't have their own inference engines.
Interesting, thanks!
Is it in Debian yet?
It would be cool if we could get a rough idea of how much human effort/time/labour went into this.
Even though their API price is cheap compared to OpenAI/Anthropic, I still can’t “afford” to use their API because the subscriptions from OAI/Anthropic provide far higher value and peace of mind. I know for sure how much I need to pay each month. I believe most developers fall into this category as well, where $20/$100/$200 from Anthropic/OAI provides more value.
This raises the question: why doesn’t DeepSeek provide a subscription service as well? Is it a better move to have a single pricing model like they currently do?
I'm not sure what you're on about. Load your account with X dollars and then your monthly bill is capped to X dollars.
or by 5 opencode subscriptions, and have $800 of ds4.1-flash for $200. I'm skeptical that the average dev will manage to burn through the equivalent of $800 of ds4.1-flash if they spend even a bit of time optimizing their token usage.
I did use deepseek via API and it can easily cost 3$ per day for my workload. On the other hand Luna 6 on 20$ subscription from OpenAI cover by usage for whole month. However, `ccusage codex` command suggested that my Luna6 token cost is around ~100$ monthly.
``` or by 5 opencode subscriptions, and have $800 of ds4.1-flash for $200. I'm skeptical that the average dev will manage to burn through the equivalent of $800 of ds4.1-flash if they spend even a bit of time optimizing their token usage. ```
This is my point. For most developer, subscription model (with subsidize) is preferable. And what I'm wondering is, as suggestted by very cheap cost-per-token from DeepInfer, why don't Deepseek persued subscription market as well ? What make subscription not make sense for them to provide ?
> On the other hand Luna 6 on 20$ subscription from OpenAI cover by usage for whole month. However, `ccusage codex` command suggested that my Luna6 token cost is around ~100$ monthly.
This is where the model comparisons become hard. Luna and DS4.1-flash aren't comparable models in terms of coding performance, DS is far ahead. Or at least so say the benchmarks. So if you're good with the performance from Luna, there isn't even really a reason to look at DS. You'd pay more for capabilities you don't need. But that doesn't answer your main point of, what to do if you DO needs DS4.1-flash performance.
> And what I'm wondering is, as suggestted by very cheap cost-per-token from DeepInfer, why don't Deepseek persued subscription market as well ? What make subscription not make sense for them to provide ?
By bet, given that they've introduced time-of-use pricing, is hardware capacity. They don't have the capacity to handle more demand, which is what coding plan would immediately produce.
I agree with this. While I realise the current subscription levels probably aren't sustainable long term for this month and probably next month, and who knows how many months after that, it is the best option.
I'm ready and willing to change as soon as I need to, but for the moment I'm going to take advantage of the subscription costs offered as it lets me know exactly how much I'm going to pay each month.
You know your subscription is unsustainably subsidised by debt and likely to change at any moment, yet relying on it provides more peace of mind than buying from someplace that are extremely transparent about their costs. That's an interesting approach to say the least
While I agree with the debt part, API pricing can also change at any moment. Well, anything can change at any moment, it's still wild west outta here.
I'm on an Alibaba Cloud Model Studio TokenPlan subscription, and DeepSeek 4.1 Flash, Deepseek 4 Pro are available, alongside various Qwen and glm models.
I've been using Qwen 3.8 Flash on an $18/month Standard Plan, and haven't gone close to consuming my credits. It got close to rollover date, and I gave it a decent nudge for 2 days, and used... about 2% of my monthly quota. Obviously one of the Max models could burn tokens a lot faster, but I'm very happy with the outputs one of the Flash models has been generating.
I need to give it something harder to do, it seems.
Even the Alibaba $6/month plan supplies a reasonable amount of API use, and Qwen 3.8 Max is a killer model for long time frame agentic use.
Literally everything I do is Open Source software and Open Content writing so I am fine with the lack of privacy using direct Chinese APIs. And, for a little more spend, just use US companies like Fireworks.ai or Ollama Cloud for more privacy, or run on-premise locally and get the best grade of privacy.
Maybe at a certain point their capacity is sufficiently being used up by the "full cost" API usage, and offering cheaper package deals would just cut into their profit because they can't get more inference?
I do not think OAI/Anthropic make the biggest share of their revenue from subscription. The largest share is from API spends. That is what DeepSeek or others aim for as well.
Currently this is true only for Anthropic's $20/month plan. I do a lot of work without reaching my limit. OAI's similar plan on the other hand isn't worth it - it used to be a great plan in the begining when they needed to catch up, but now I reach my limit very quickly on it. The only reason I keep it is because my entire family is using this account for their chatgpt usage.
The more expensive subscriptions are just not worth it for individuals. Using deepseek even at API prices is better value. If you're a company it might be a different story.
so many new small AI labs will be born because of this. what a time to be alive
If only we can buy Huawei Ascends!
What's the difference between this and a typical inference engine like vLLM?
I think the reason DeepSeek does this is partly to neutralize competitors' technology, but ultimately the core reason might be that data is what matters.
Data and GPU compute capacity.
Chinese labs do not release "open weight" models because they think it is good for humanity. They do it because they are severely GPU compute constrained due to the Nvidia and ASML embargos. All available GPUs have to go toward training which leaves very little capacity to host inference at scale. So they release the models and encourage EU and USA 3rd parties to host them.
China will inevitably produce their own AI chips and then they will lock down their models just the same as everyone else.
The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
> The real loser in all this is obviously Europe, which has nothing competitive once the Chinese models are gone.
Those models are open source / open weights. Europeans can continue with Chinese models as a base same like Cursor used Kimi K2.5 a base for their composer 2.0 model or SWE-2 from cognition is post-trained Kimi K3
Similar to how some labs have done free stealth model releases. Trillions of coding tokens ought to improve performance of the downstream model. However, improving the base model architecture remains the bottleneck since you will not reach RSI with just that (assuming you ever can). And compute, lots of compute…
That strategy only works for model generations that are released with open weights. If Kimi K4 is proprietary, then China has the newer weights and the EU does not.
It's also worth noting that more post-training does not change the model architecture. No amount of post-training on deepseek v4-flash will ever make it as efficient as deepseek v4.1-flash because 4.1-flash uses a fundamentally different architecture. Being stuck on an older base model means you don't keep up with these architectural improvements that unlock significant gains in speed / efficiency.
But how does that benefit the Chinese companies to have other firms host their models and gain profit from the inference, without having to pay the training costs? Training by itself is not profitable at all, only serving the newly trained model (inference) is.
Because Chinese firms and the CCP still want to be relevant at the forefront of frontier AI development. They want to have their own sovereign AI.
The alternative would be the EU strategy which is... nothing. They're content to just be techno-colonized by USA in China.
How to destroy the entire US economy without firing a single bullet
Maybe just maybe the real reason DeepSeek does this is because they are altruistic.
Kinda funny to see hackernews clamouring to ban the Chinese models. Truly in the thrall of the frontier labs
they must know something is coming from the american labs for some time, you don't release this just like that!
If that's the thing that makes $0.00X cached inputs possible, then this is huge!
The quality of technical discussions here has clearly nosedived over the past few years. This release basically confirms it.
Open source is different from open weights
Well played!
I don't understand what is in it for them. I understand that the government wants openness but who's paying them to give away their secrets?
Xi gave a speech about it a while ago. The idea is that ai should increase productivity for the whole economy not just centralise it in 2 or 3 ai companies.
Of course screwing over America’s crown jewel companies is a helpful side effect.
I was referencing that. İt doesn't help deepseek; it harms them unless the government compensates. What's the calculus?
The CCP may be forcing them to do this.
Exactly what they're aiming for.
> it harms them unless the government compensates
not an expert but it looks like you just describe the entire China playbook. See EV, rare earth etc.
DeepSeek doesn't really care about models, they're aiming to become a platform: https://try.works/a-follow-up-on-china's-open-ai-race
They open source it because they don't need it anymore. Probably they are doing something better with Huawei
Right. Gotta imagine they’d only open-source it when they already have something more advanced in play
the only advantage i see long run is they can get free contributions world wide to lower their engineering costs. In the short run, frontier labs gets to use some of the ideas.
Need to see a Strata style take on this.
hi
Just another tour de force from DeepSeek.
I'm not saying that the US labs (Anthropic, OpenAI, and Meta) aren't pulling their weight, they are all doing amazing work as well, but Deepseek may belong in that tier.
So what? We already have multiple open source inference engines for DeepSeek models - that's how there are over 30 providers already running DeepSeek around the world. There's no secret sauce here, they've been publishing everything they do in papers from day 1.
Deepseek released just enough to stir the pot but not enough to cook your own from scratch.
Wake me up when we can clone, point it at a dozen H100s we own, and get a working production service.
Anthropic and OpenAI will do just fine.
> point it at a dozen H100s we own
uhm...wake me up when I can buy one(1, single, uno) GPU* to upgrade my PC so that I can play modern games with high graphics, without feeling like I'm being bled dry.
y'all got dozens of H100 at home? isn't that ~300k??
*actually my bigger problem is that I need a new CPU, which means I need a new motherboard, which means I need new RAM, which...you get it.
This is not a home/consumer GPU discussion.
But in that vein, you are in the same position as the chinese LLM providers. They cannot get hardware to host so they put the problem in the forefront by offering it open.
In a way, Deepseek is championing your cause.
> This is not a home/consumer GPU discussion.
fair point...
> In a way, Deepseek is championing your cause.
maybe in 2030 i'll have a chinese gaming PC with chinese local AI. would be interesting to see if they can do the same to US tech companies as they do to european car manufacturers. i'm ready for it...
You can ask Claude to finish it
Make no mistakes.
Target is by end of Q4.
Anybody could fire up vLLM now to do exactly what you're talking about. And they do? There's dozens of places hosting DS4.1 Flash, at good speeds. So I'm not sure what your point is.
What this provides is the give contributors to vLLM, sglang, etc. a reference implementation of some of DeepSeek's own hard-won inference serving techniques.
No doubt we'll see patches to vLLM inspired by bits in here.