The current commitment by hyperscalers is around 1.7T USD, reported liabilities 1.3T and this year global debt related to AI is 570B. So that’s around 3T total. For this to make sense AI must generate 2T in new revenue per year by the end of the decade. And that would be only a 10% ROIC. For context ROIC for big tech is around 35% so at 10% they will be barely breaking even. The SP500 gives 10-12%. With 10% ROIC from AI the only thing investors will be celebrating is that the whole thing didn’t trigger a financial crisis. Data centers are NOT real estate. Buildings and power lines usually last 30-50 years. GPUs become obsolete in 5 years. If hyperscalers need to refinance and their interest rate goes up there’s zero margin for error.
H100 is nearing five years and costs more to buy a used one now than a new one when it was released :)
You are completely missing the bet these companies are making.
They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.
If you haven't been paying attention, the cost is about 1/100th of what it was in 2024. This is the trajectory pretty much every technology has followed.
Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.
The fact that supply constrained GPUs holding value is negative indicator. It's like the tulips mania except bubble enthusiasts are buying wilted/dead tulips because live tulips supply constrained due to irrational demand. Now GPUs has more gross utility than tulips but seems like at current revenue/capex spend, every GPU is still negative net financial yield - they lose money - literally buying tulips and watching it wilt. Economically, better off simply not buying and losing more. That's the level of economic irrationality at place sustaining bubble, at least for hyperscaler/big tech balance sheet - small operators logic different and antagonistic to big operator demand/business model.
If cost of inference goes down 100x, would need 100x more demand. This makes overspending on GPU even more irrational. Jevons this, Jevons that but ultimately irrelevant. At end of day, leading players, hungergame winner candidates is saddling themselves with so much debt, even if they survive, post crash they are immediately uncompetitive against new entrant with blank slate and newer gen, more efficient GPUs that will be cheaper to buy/operate post crash when hardware prices will revert to mean.
It doesn't matter if some of the current players survive, they've basically stabbed and weakened themselves so much any healthy upstart in the future can wipe them out unless they lock in legislative protection... safety regulations, ban open source models etc.
That is the new bet, regulatory capture moat, because economic bet is entirely lost, especially with open models eroding mote.
You're making a classic philosophical error here by accidentally anthropomorphizing companies :)
A company losing out on a risky bet and failing, letting a new upstart rise up is pretty natural. A large fraction of experts from the failed companies continue at the new ones, business as usual. There are engineering teams at $BIGTECH now full of OS, database, or compiler experts from XP, Sun, HP etc.
This natural ability of companies to take risky bets is what made silicon valley successful.
Companies are run by people, 100 billion dollar companies are ran by people who want to stay billionaires and will street accordingly. But I'm not sure what we're disagreeing on, yes the talent will migrate, AI industry will eventually settle on some none bubble equilibrium, but that doesn't mean current AI economics is sensible, or inflated hardware costs beyond yield is not danger indicator. Like yeah, risky bets are burning, most people are going to move and carry their technical expertise on instead of unalive themselves or flip burgers, but that's independent of whether business models and broader economy is going to explode.
You're making a classic logic error by accidentally equating corner bodega with trillion dollar keystone industry.
Business models fail all the time, trillion business models that underpins entire macro economic growth engine exploding and causing massive contagion happens rarely, not even generationally. But when they do the leading bubble mathematic indicators are more clear.
You may be conflating magnitude of dollars invested in AI being at records with the fact that they're actually a lower percentage of GDP than other speculative bubbles have been ... this is important to understand because the magnitude of a slow down maybe much smaller than you seem to think
Direct DC/AI capex lower bound ~2.5% gdp already comparable to dotcom / dark fiber. Include down stream supply chain / infra and its ~5% - railway territory. Either way relative magnitude of AI financial footprint is systemic risk is within historic bubble territory. Also important to understand AI basically only major growth driver, other sectors underperforming, there's nothing to cushion, and fallout maybe much larger than you seem to think. Strip away IT 90s baseline economy was still growing 3%, strip out AI, 2026 baseline growth ~1%, negative once you factor in money printing and historic ~3% (and growing) burnt in debt servicing. Stack on hardware depreciation, and I wouldn't say contagion magnitude will be slow/small.
> They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically.
We are also within an arms race of training newer larger models with more speed while discontinuing older models.
Gemini/Chatgpt have already discontinued their models from 2024 (iirc) because they are using all their compute in serving/training newer models. Being quite frank, nobody is serving a model from 2024 as the intended use-case while having very little moat as open source models are catching up.
> Of course there will be market crashes and corrections and things like that and most companies won't survive, but the bet is that whoever survives ends up doing pretty well.
How so, by raising the prices? because the current prices aren't sustainable and I feel as if there would certainly be companies which will try for one reason or other to be cheaper to capture the market share because of the larger promise of whoever is able to get as market share. I had once thought about it and I don't think that even in an ideal world, they would end up doing pretty well given no moat.
Also even if a company survives and ends up being one of the survivors and makes profit in the ideal scenario you mention, then within some years other companies will try again and construct more datacenters and end up driving the prices down for everyone, so nobody knows how things might look down for 2-3 years let alone a decade, so I remain a bit skeptic currently so.
I had actually thought some on the economics of datacenters and I found it to be very related to power. The only ones which seems to be making money might be the power generators actually because power is the actual bottleneck rather than GPU's in datacenters from my understanding.
Though the power is raised at the cost of electricity bill increases for everybody including people living in houses. The job prospects are minimal as well, as a nation, aside from just getting investment just for the sake of it because AI's trendy right now, I feel like its a net negative deal for people living there.
> It worked out for Uber, they are wildly profitable now after spending a decade losing money.
Uber have a 10% margin, which is definitely not what I'd consider wildly profitable. (Their post tax numbers look better, because of accumulated losses).
Uber is a two sided marketplace and was still famously unprofitable for at the same time.
Over 14 years, Uber burnt ~31 Billion dollars and in nearly whereas the amount invested within AI seems to be within Trillions at this point and in near future with a product which doesn't have much moat and shaky financials on profit.
Is this true? Hardware costs have only gone up during this time. Are you referring to electricity cost to serve these models? (i.e. compute got more efficient?)
not to miss, future models will be more compute hungry too. Current hardware prices are still goin up and no it's not cheaper to run your AI for like %99 of the people because of lots of costs, it's not just hardware.
The per token costs plummet with more concurrents. A box that can do 100 tps at request depth 1 might be able to do 3000 tps at request depth 64. Less per thread, but massively more per GPU/joule/etc. That’s the economy of scale of running in a DC rather than locally that they were referring to.
I think this fails to take into account how many people are fine with "fast enough" vs "fastest".
I've seen people happily use AI that takes several minutes to generate text or edit an image because to them they already aren't using their computer when they tell it to start; they just grab their phone and walk away and come back only to check in on it.
I feel like people here and on other technology discussions -- although it's worse here -- don't seem to parse what being the minority means.
They know they're one of the few to have access to such incredible hardware -- whether it be rented or purchased for way too much cash -- but they only see their own kin; their own ilk. They only compare themselves to the best.
The reality is that nobody expects data centre speed nor power in their own home and are satisfied to just go "haha its thinking" and let their computer quietly tick in the background as opposed to paying outragious prices for subscriptions or hardware.
That is completely discounting future capabilities and new use cases. Sure in 10 years you will have current SOTA locally, but in no way is it obvious we are anywhere near the limit of marginal value from improved capability
Unless you're a developer or doing complicated research you do not need much to use an LLM at home with consumer-grade hardware.
I'm editing photos with 32GB of DDR4 RAM at 3200mhz alongside a 3050 with a nearly decade-old mobo and ryzen 5800XT and at most? It takes 30 seconds and that's allowing the gpu to use my actual RAM as 'fallback' memory.
There are photoshop filters -now- that take longer than that before the LLM craze even began. Hell; I can train a lora in an hour or two.
This obsession with "more specs more data faster and faster" isn't going to win; it already isn't winning.
The “contents” represent the majority of the cost and meaningful functionality of what we call a “datacenter”. Those contents will not last for “real estate” debt timelines.
Right but their payback period is insanely short - a $5M GB200 NVL72 cluster is expected to generate $75M in revenue over 3 years for inference providers. That's a 3 month payback period.
AND - they're operating well past their estimated service lifetime.
Chances they're planning on replacing personal, local compute with time-sharing on data center hardware that's too outmoded for AI...? You know, since they sunk the consumer component market for the next half-decade.
Not only that, but they're typically amortized over 5 years, where the actual lifespan usually falls far shorter (1-3 years), adding to the artificial subsidy conditions we see today. So they're gaming the lenders into deferring interest payments as much as possible today so that new competitors don't have the same cheap financing advantage.[0]
If they're deliberately inflating the likely useful economic life of their assets to get a lower interest rate, it's hard to see how that wouldn't be classed as fraud.
It's the sort of behaviour that really does end up with people going to prison.
You’re all getting some concepts mixed up here. That five year amortization rate is the IRS’ usual amortization rate for computers. GPUs are classed as computers for asset depreciation purposes. But GPUs are part of 168(k) so they’re eligible for a 100% bonus depreciation the year of purchase.
There’s nothing fraudulent at all here just people using terms they really aren’t comfortable with.
I would be more careful before assuming 5 years depreciation schedule. Currently price tags are attached to computing power, not the production cost. Computing is not getting meaningfully cheaper with newer GPUs but it only allows better scaling, which makes older hardware more relevant for many use cases. This is why A100 is still selling like hotcakes. I don't think this trend will change soon.
Apple can enjoy because it controls a significant fraction of consumer computing platform so they can simply collect tax from everyone else. This is not true for the rest of big tech. Only Google has Android but it cannot sit and enjoy because they still don't control hardware and AI is an existential problem for their search business.
The GPUs are far from worthless after 5 years. E.g. the A100 80GB PCIe version cost around $15k when it was introduced in 2021 and now sells for $10k used.
Things might be slightly worse for the data center servers, but I am sure they will find find buyers.
They only reason that they are retaining value is there was not so much demand for GPUs in 2021 as there is today. Once the demand drops you will find then in dumpsters across our barren, burning dystopia.
They hold value as there is insane demand. The same reason a consumer RTX4090 costs more today than bew in 2021. Once the tide drops enough for hardware lead times to shorten to weeks, they will go the way of other used DC hardware - written off after 5 years.
For what its worth, SK Hynix CEO has every incentive to show that RAM prices will remain high for as long as possible because that is the only thing which is floating their evaluation to such astronomical amounts.
Independent estimates sort of show around 2027-28 from what I remember.
I remember reading some article which said that RAM prices are already going down from its peak slowly (IIRC I can be wrong, I usually am but 3-5% month from its absolute peak) but the current RAM prices are still astronomical given past rates but the RAM prices will slow down hopefully sooner rather than later.
Also, I don't recall any era where "proof" is required for allegations. "Evidence" is probably prudent, but proof is usually produced when the accused gets ends up in court.
Unless you time travelled from a feudal era where they throw you in jail for slandering the nobleman or something.
> Back in the day, we used to have this thing called “proof” before making wild allegations of illegality
I understand that you might've wanted to know more sources but its better to ask them directly in good faith as I am more than happy to provide sources and have a good faith discussion! :)
> are you just playing pretend?
(This part feels a bit personally rude to me. I would prefer if you could omit this part of the messages in future or take care of that, Hackernews is a lovely community for the most part so lets keep it that way :-D)
I appreciated reading the comment of @hnfong. The DRAM Price fixing scandal and multiple criminal investigations are something that I didn't even know of!
The general consensus is that its late 27-28 and I would like to share some of the sources that you were asking.
I don't know if they are lying or not but the interview on reuters was conducted on the same day that they went on Nasdaq
> "We forecast that next year will be the worst year in the industry's history from the supply perspective," Kwak told Reuters in an interview on Friday, the day the South Korean memory chipmaker began trading on Nasdaq.[From (1)]
> Multiple analysts push meaningful relief to late 2027 at earliest — with some estimates stretching into 2028[0]
> UBS likewise expects the global DRAM industry to remain undersupplied until at least the second quarter of 2028.[1]
The above one [1] reference is literally from the reuter article itself which was referenced in the parent comment
> Across the U.S., China and Europe, DDR5 RAM prices fell sharply. For example, after reaching an all-time high of $500, Corsair’s Vengeance 32GB kit is now down to $369 on Amazon. Memory has fallen more than 20% from recent peak prices, and stock prices of major memory factories like Micron, Samsung and SK Hynix have seen drops of over 20%.[2]
No one knows. There's a known bullwhip effect in supply chain [0], and DRAM makers are pretty far out along the supply chain. Just like it ramped up wildly it will stop even faster.
That sounds reasonable, it's "just" $1k/yr for 2B workers (there are about 1.2B total "knowledge workers" in the world including gig drivers), or $10k/yr for 200M workers (there are 70M office and technical workers in the US). /s
In 4 years it better be 10x more important to have than a cell phone is today, or 10x more important than having internet/monitor/pc/printer is for an office worker today.
The math looks good on paper, but in reality enterprise AI is hard, most companies are realizing they are actually not seeing ROI from AI. One of my customers took about 8 months to rollout an AI initiative that by the time it launched and people got trained on it, it was already legacy. Also if you are 10x more productive with AI that doesn’t necessarily increases your billable output. There could be some super models like Mythos aimed at very specific hard tasks like drug development, but we have not seen any of that yet, and the clock is ticking.
Yes, I'm agreeing with you. There need to be 200 companies willing to pay $10B/yr for this. What is the ROI? That's the pay roll of ~half the work force of the largest 200 companies. Unless you can fire %50 your employees, everything else is a sunk cost you already own.
I just don’t understand this view. This is the most significant technology ever developed. The uncertainty currently is whether it 1) has massive impact, completely altering society and the making world significantly significantly better or 2) if we go into a fast takeoff/rsi loop. Personally I’ve always been highly skeptical of the later, but that seems like a genuine possibility now. It’s not ‘are we going to be able to generate 10% roic on compute’ the answer to that is yes.
It will obviously be a large part of the economy like online shopping is today. The companies that built up a lot of debt to be brand names in online shopping primarily went bankrupt because new companies had no debt (and perhaps no negative sentiment from early customer experiences.)
the difference is all the current capex is going to durable, hard to get physical assets + things like PPAs. In your online shopping analogy, the hyperscalars are acting like Amazon in 1998
What destroyed hardware manufacturers after the first bubble burst was all the old but overpowered server hardware ending up in resale after bankruptcies. Hard to get hardware is a bad investment that is also potentially obsolete after new optimizations make the next generation much more efficient. The companies that think their individual optimizations are going to outpace industry wide optimization are delusional, historically speaking.
