This is a good counter argument. But you have to note that this is after OpenAI cut Luna costs by 80%. If you compare launch pricing, Qwen probably comes out ahead on a cost-performance basis.
You assume that openai's inference is profitable and that they aren't just trying to bolster revenue before their IPO.
The only indication that openai is profitable comes from openai (whom I wouldn't trust with any statement, especially when it comes to profitability).
In fact there is evidence that inference is not profitable simply because the rate of losses doesn't seem to reduce as revenue increases: if inference had great margins, we would expect that as revenues increase, the amount of spend on training reduces as a fraction of total expenses.
Since the loss-making fixed costs shrink as a fraction compared to the profitable inference, we should expect profitability to rise with total revenue.
However, all leaks of openai's numbers seem to suggest the opposite: as revenues increase so do the losses.
The indication that OpenAI's inference is profitable is that 3rd party providers host large models for cheaper.
Given that OpenAI is ahead in intelligence, it's also reasonably likely that they are at the frontier of efficiency too.
Your "evidence" for OpenAI's inference not being profitable is apparently based on leaked financials supposedly showing growing losses for reasons entirely unknown.
With their research, training, data centers, chip development, and hardware product development, there seem to be a number of reasons that might explain growing losses.
They have an incentive to make their models efficient enough to serve demand and make a profit on it.
The incentive that is missing is passing on efficiency improvements as price savings to customers, when your model is still in demand because of its higher intelligence.
Agreed, efficiency is still important, but being at the "frontier of efficiency" is significantly more relevant to commodity model providers than state-of-the-art model providers. Frontier labs are incentivized to route their spend towards beating benchmarks because that's what enables them to charge a premium.
You can make the other argument that China subsidizes the price and that they can't be profitable at this pricing level. From an industrial strategy standpoint, they already do this for many other industries with huge subsidized state loans.
So we can go round and round on this, each with our made-up objections about how it's temporary or unrealistic or impossible or whatever, or we can just accept the prices as listed and use that to guide our economic decisions.
Just look at the prices that inference providers charge for small models. The argument that these unit economics are negative is trivial to disprove.
DeepInfra sells DS v4-flash at 0.08 in, $0.18 out. Gemma4 they sell for $0.07 in, $0.34 out. OpenAI's price for luna is $0.20 in, $1.20 out.
Why would you assume OpenAI is somehow uniquely incompetent at making small, fast models? And that they're worse at serving it than DeepInfra? Any observer can see they are making money here.
I never understand why people who are convinced there is a big con just don't check market prices and see if there's money to be made.
That doesn't mean their business is great -- they're losing tons of money, but it's because they spend too much on fixed costs, and they can't stop spending money on training next generation models with no end in sight, not because the inference is margin negative, which is a flimsy idea that just clouds the actual business issue.
There's a difference between the Deepseek.com provider lunch pricing and the pricing every other provider is doing now.
Right now DS4-Pro-0813 is available from multiple providers for $1.32/million input tokens[1].
It's pretty easy to work backwards from B200 and electricity prices and see this is profitable even without the heavy serving optimization these providers are doing[1.5].
The OpenCode CEO said: "inference is very profitable and probably a good opportunity to understand some basic business math"[2] and "the inference we do is already profitable and that's with some middlemen involved"[3]
If at this point people don't believe inference can be profitable, and providers can turn the prices up and down to choose exactly how profitable they make it I don't know what to say.
what if it was because of quantization and they haven't released the new benchmarks for it?
Anything which changes the model needs new benchmarks I guess to compare with other models, otherwise you can benchmark Fable, and distill it to student model and keep claiming this is the Fable model
ARC Prize has retested Luna after the discount and validated identical performance.
(Also, quantization isn't inherently bad or damaging when done properly, e.g. QAT).
These APIs are used heavily by enterprises at scale; with lots of performance telemetry, live evals, etc. You can't really silently nerf API models at scale without people noticing.
Of course, what I said doesn't apply to non-API consumer sub models; there's many documented and officially confirmed instances of under-the-hood "juice/effort" adjustments. (Juice = a number your effort tier maps to underneath the hood; much like Inkling's effort=0.00 to 0.99).
Given the timing, I think they A. shat their pants since Deepseek flash just came out with insane pricing before the price hikes, and B. Anthropic is really struggling in model tiers below opus.
It was smart for them to cut prices regardless of whether they had 80% efficiency gains or not
Because labs can learn to optimize inference post launch, plus can move to use bigger/better clusters depending on demand. It is not impossible to imagine Qwen cuts prices further with QAT/MTP-like improvements.
Those prices are just tokens? Since each model uses different amounts of tokens to do the same thing, it's a misleading price that often makes open-weights look more competitive than they are, since most open weights models use dramatically more tokens and time to complete tasks than many frontier models.
In Artifical Analysis's cost per task, Luna(max) costs $0.05 per task, and Qwen 3.8 27B costs $0.25 per task, a 5X increase. We'll see how 3.8-flash-next does.
It's not free. You're paying electricity and you're ignoring the cost of the hardware. Even on electricity alone, there are cloud providers who may beat your laptop on price per million tokens. Qwen 3.8 flash is interesting in this space.
Not to say that there aren't other benefits of running models locally, I loaded Qwen 3.8 27B 6bit MLX just yesterday.
Thats only important if running it locally is critical for privacy reasons or just as a hobby.
Time has a cost in business. If a model needs 30 million tokens to achieve a similar result as another that can do it in 10 million, that 60 tokens per second will take a long time.
Right now qwen 3.6 35b-a3b has a success rate of 92% and qwen 3.8 27b has a success rate of 96%. But the 35b moe does about 1080 tokens/s at concurrency 54, vs 480 tokens/s at concurrency 28. For our specific workflow on blackwell.
Of course enormous batch jobs are different. I was explicit when I said consumer laptop.
Curious, how are you running it and what quantization are you using? I've mostly been using MTPLX; 125B sort of looks like it'd be right at the limits of my 128GB MacBook once you factor in KV cache and context window.. wondering if it's worth it compared to the 27B model which gives me a lot of headroom or even a 72B model.
For world knowledge, you'd want it to find and reference the source material to be sure. At that point, it doesn't matter if the knowledge is embedded.
I think the big models have adequate recall, so tool use is probably unnecessary, but the user said the correctness of my response is important. Let me look up the data instead of relying on my memory.
I don’t think that’s the right way to think about LLM ‘knowledge’. They don’t have absolute recall of everything in the training set. They have been trained so that they have weights that can predict what those books might say - that is, if they read them they would find the contents unsurprising. That doesn’t mean it wouldn’t be helpful to pull relevant passages of text directly into context for a particular task.
Does it really matter? What about including all relevant and up-to-date literature as skills for local models? I have no experience with this but I am pretty sure someone has already thought about it.
Ex - nodejs natively supports a huge set of typescript with built-in type stripping these days. But ask most hosted models to build a typescript project and they default to a heavy compile step, or a tool like tsx, ts-node, etc.
Models with lots of "world knowledge" have a good chunk of that knowledge go stale, and there's no real way to refresh it without training a new model.
Another classic example of this back in the day was to ask who the president of the US was, and watch different models happily give different answers based on the date they were trained.
---
Personally, I'm really interested to see if we're headed towards a spot where the model is entirely distinct from the knowledge store.
We're vaguely there with the ability for models to go search the web, but I think the reliability of that path is going to continue declining (more and more spam content, less and less genuine value).
I kinda want a paradigm where I can pick and engine and a knowledge bank, and combine them as I please.
Ex - if I'm doing gardening, I can pick "gardening for models (version 32)" as my knowledge store.
If I'm doing auto-repair... "cars for dummies (version 3)". etc...
> Personally, I'm really interested to see if we're headed towards a spot where the model is entirely distinct from the knowledge store.
This is what I've been trying to focus on with local AI for now. I've been trying to build all new documentation so it's more AI friendly. It's been pretty interesting. Qwen-35BA3B with a small prompt does a good job of surfacing what I'd consider institutional knowledge.
I've been trying to silo the docs I write from the model with a prompt that tells it not to use general knowledge unless asked to. From the anecdotal testing I did, Qwen-35BA3B is great for it. It does a really good job of following the prompt and calling tools, so I've been able to play around a lot to see what seems to work best.
Ultimately, I think one of the most effective uses of AI will be having a distinct knowledge store combined with an opinionated agent (and sub-agent) setup along with different models for each task.
Who owns the knowledge store is going to be the big caveat. Right now I think the big online models are trying for generic, persistent memory and I'd be very hesitant to let that happen. Think of having someone with a perfect memory following you around forever, but someone else has the ability to make them disappear. That's not a good situation.
One of the consequences of encountering a lot of LLM generated text which includes things the model vaguely remembers from its training is that honestly I have grown less tolerant even of human comments and documents that are based on mostly ‘I seem to recall that…’ level sourcing.
In a discussion on economic history, say, someone will opine that Alexander Hamilton had some particular opinion about tariff policy… based on their having a vague memory of a blog post where someone quoted a passage in support of some point. But wait - you can search the federalist papers, the text’s right there to be read, before you commit to saying online ‘Hamilton thought tariffs were a great idea’ you could take your internal ‘I seem to recall reading something about hamilton’s opinion on tariffs’ thought and turn it into a little RAG query where you pull up a source and check before you put another factoid out onto the internet.
And so I feel absolutely the same way about LLMs. I don’t care how much factual information was in the training data, when the LLM wants to rely on something it vaguely recalls having been trained on, it owes it to me to dig up a source and vet it.
There are limits to this, of course. I don’t want it to be thinking ‘but wait, maybe my memory of Python syntax is faulty. Is = used for assignment? <web search>…’.
But in general some caution about repeating vaguely recalled easily checked facts is warranted.
At 125B + 51B I'd expect it to have some degree of world knowledge, clearly in the middle between small models like qwen 27B, and huge trillion parameter models.
My AMD strix halo box (haven’t benchmarked yet) should also run it reasonably well. It was $1400 at launch, and is $4K now.
Your mac is < $2K in Biden-era dollars. Presumably the economy will eventually recover; maybe in one Moore’s law doubling if the midterms go outrageously well. That’ll be two doublings since the halo launched. I’d expect this model to run on a sub $1K box by then. $2K ought to get you a 512b parameter model at that point. If we have to wait out the rest of the term, the cost cliff will be even more pronounced when it hits.
I believe you're underestimating the lag inherent in the economy. Even if we grant the idea that the political party controlling the US House/Senate has a significant impact on the economy, and that the current party is BAD and the next one would be GOOD, I would still expect that things will continue getting WORSE for a good 4 to 8 years before they get better again.
And that's even with assuming that we can continue to ignore the long-term problems like social security insolvency, the debt bomb, or climate change forever.
Opus 4.6 Max self-hosted at 30 tok/s on a 5k Macbook in Aug 2026. The LLM timelines are crazy.