AI's $600B Question

According to Jensen it takes about 8000 H100s running for 90 days to train a 1.8 Trillion param MoE GPT-4 scale model.<p>Meta has about 350,000 of these GPUs and a whole bunch of A100s. This means the ability to train 50 GPT-4 scale models every 90 days or 200 such models per year.<p>This level of overkill suggests to me that the core models will be commoditized to oblivion, making the actual profit margins from AI-centric companies close to 0, especially if Microsoft and Meta keep giving away these models for free.<p>This is actually terrible for investors, but amazing for builders (ironically).<p>The real value methinks is actually over the control of proprietary data used for training which is the single most important factor for model output quality. And this is actually as much an issue for copyright lawyers rather than software engineers once the big regulatory hammers start dropping to protect American workers.

I think this is the correct take. My understanding of the article is that huge investments in hardware, mostly to NVIDIA, and spending by major tech companies is currently defining the market, even if we include OpenAI, Anthropic, etc. It is FAANG money they are running on.<p>I put this as equivalent to investing in Sun Microsystems and Netscape in the late 90s. We knew the internet was going to change the world, and we were right, but we were completely wrong as to how, and where the money would flow.

&gt; Founders and company builders will continue to build in AI—and they will be more likely to succeed, because they will benefit both from lower costs and from learnings accrued during this period of experimentation<p>Highly debatable.<p>When we look back during the internet and mobile waves it is overwhelmingly the companies that came in after the hype cycle had died that have been enduring.

Maybe because the revenue isn&#x27;t directly attributable to AI itself, but is realized in the cost savings and productivity improvements in already existing revenue streams? That&#x27;s where AI has been useful to me. I can&#x27;t put a number on how much AI has made me exactly, but it has certainly helped all aspects of my bootstrapped startup.

Others are saying this article is bearish, but then...<p>&gt; A huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely.<p>Such strong speculative predictions about the future, with no evidence. How can anyone be so certain about this? Do they have some kind of crystal ball? Later in the article they even admit that this is another one of tech&#x27;s all-too-familiar &quot;Speculative frenzies.&quot;<p>The whole AI thing just continues to baffle me. It&#x27;s like everyone is in the same trance and simply assuming and chanting over and over that This Will Change Everything, just like previous technology hype cycles were surely going to Change Everything. I mean, we&#x27;re seeing huge companies&#x27; entire product strategies changing overnight because We Must All Believe.<p>How can anyone speak definitively about what AI will do at this stage of the cycle?

&gt; In reality, the road ahead is going to be a long one. It will have ups and downs. But almost certainly it will be worthwhile.<p>The biggest takeaway from this piece is the stark realism of this article (maybe a bit too bearish, imo) compared to the usual Sequoia VC-speak. Maybe FTX <i>did</i> teach them something, after all.

The calculations here seem to depend on a few false assumptions:<p>1. That all datacenter GPUs being purchased are feeding AI. You might be able to argue that some are or a lot are, but you don&#x27;t know how many just looking at Nvidia sales numbers. I know of at least two projects deploying rows of cabinets in datacenters full of GPUs for non-AI workloads.<p>2. The assumption that pay-for-an-API is the only AI business model. What we now call &quot;AI&quot; has been driving Google&#x27;s search and ad businesses for nearly a decade, sooo AI is already doing $300B&#x2F;yr in revenue? There is no way for this guy to quantify how AI is solving problems that aren&#x27;t SaaS.<p>David Chan, if you are reading this feel free to email me if you want a fact check for what will surely be the third installment in the series.

38 OECD countries have combined population 1.38 billion, GDP $49.6 trillion<p>$600 billion is $434 per person, $36 per month per person, 1.2% percent of GDP.<p>If 75% of the spending goes to increasing productivity, I could see it. Get rid of $450 billion in labor costs (12 - 15 million work years). Cut worked hours in call centers, customer service, many services, menial programming jobs, ...<p>But I don&#x27;t see it happening fast enough to pay for current investments.

<i>&quot;Unlike the CPU cloud, which became an oligopoly, new entrants building dedicated AI clouds continue to flood the market. Without a monopoly or oligopoly, high fixed cost + low marginal cost businesses almost always see prices competed down to marginal cost (e.g., airlines).&quot;</i><p>So why aren&#x27;t there more entrants in the CPU cloud area? The technology is a commodity. Google and Amazon don&#x27;t make CPUs.

I think there is far too much focus on potential productivity gains, and not nearly enough on entertainment value, which still has a ways to go but much higher potential compared to serious problem solving, which may or may not be be feasible with these models. Nobody has really figured out yet a good way to integrate generative multimedia together in a way that makes a hit, but surely someone will in the next few years. Right now there are utterly half-assed apps like Linky making good money, and that&#x27;s just midjourney and CAI glued together awkwardly. I don&#x27;t know what a genuine hit product is going to look like, or if it&#x27;ll even be an app, but the potential for generative multimedia is so much higher that what we&#x27;re seeing now.

It interests me that the $200-600 billion number seems to be all-derived from GPUs. Are LLMs&#x2F;AIs totally dependent on GPUs? I read last week (<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40787349">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40787349</a>) that there is research ongoing to run LLMs on FPGAs at greater energy efficiency.<p>I&#x27;m reminded of Bitcoin&#x2F;crpyto, which in its early history was all operated on GPUs. And, then, almost overnight, the whole thing was run on ASICs.<p>Is there an intrinsic reason something similar couldn&#x27;t happen with LLMs? If so, the idea of a bubble seems even more concerning.

This article and most of these thesis assume one thing, you need to make a return of investment from AI while using OpenAI revenue as an anchor to measure it.<p>What if these AI investment were only to protect or strengthen their current business? AI in Windows, macOS, Adobe, iPhone, Facebook, Instagram may not bring in any additional revenue. But it add additional value to their current product line, making competition harder, further hardening their moat.<p>Nvidia or Jensen is also smart to play the national security card. Does the European want their model to be all US based. Are the answer culturally correct? Just like how every single country invested in their own Telecom or Internet infrastructure, if this pitch were even half as successful, do these numbers we are looking at even matter when it is spread out across G7 or G20?<p>While I believe we are still far, or at least 10+ years away from AGI, the current form of AI still has a lot of improvement incoming and are already bringing in real world benefits and value to a lot users. The adoption curve will accelerate once it is integrated into Windows, Office and Mac. So even if we are in a bubble, I still think we are very early in the curve before it burst.

As always, free analysis by VCs and investment banks is not necessarily for the reader&#x27;s benefit.<p>So we have this bearish piece and the previous bearish Goldman Sachs piece. While I agree with their analysis in this case, there is a lingering doubt that some banks might just want to tank Nvidia a little in order to go long. Or something like that.

&gt; I expect this will lead to a final surge in demand for NVDA chips<p>But other than FOMO why would someone buy better chips when they don&#x27;t actually know what to do with their old ones?

LLMs are undeniably remarkable, but their wheels start to fall off once you go beyond the basics and a few party tricks. The cynical part of me thinks this tracks pretty well with our society&#x27;s tendency to favor appearance over substance. Yet, if my burger order gets messed up, I don&#x27;t fault the restaurant; however, if an LLM messes up my order, the same cannot be said. As a seasoned human, I feel confident saying that we humans love blaming other humans, whether it&#x27;s justified or not.<p>So while I initially thought customer-facing roles would be front and center in the &quot;AI revolution.&quot; Today, I tend to think they&#x27;ll be bringing up the rear, with entertainment&#x2F;smut applications at the forefront along with a few unexpected applications where LLMs operate behind the scenes.

I think we are applying the wrong model here. We should not try to force-fit an app store model. A more applicable model is that of a decentralized internet and intranet. As in the case of intra-net, even though there were no visible benefits to the end-user, it did help the business itself improve margins by making information broadly available. Then just like the internet I would see these AI optimisations to work with one another Ex: It&#x27;s totally possible that a model from Amazon and DHL talk to each other to optimize package delivery given a constraint like a truck parked on a loading dock and unable to move.<p>#1 Just like the intranet there are a lot of productivity gains that firms like Tesla, Meta, Google and Amazon can gain internally by optimizing their own workflows. That in itself should justify the investment. Granted some of these optimisations will use their own chips instead of Nvidia, but Nvidia will get a lion-share of this.<p>#2. Then there are other verticals - pharma, oil and gas, logistics who can optimize their internal workflows to gain productivity. It just helps them improve their margins. No end user benefit may be realized and that’s fine.<p>#3. Nation states are buying GPUs too. Ex: Falcon2 was trained on a cluster owned by a middle eastern country. Nation states see something larger at stake here than just releasing an app. This does not have to even be a profitable endeavour.

AI is dangerous to VCs.<p>The returns are going to chip-makers and employers, including single founder startups, who don&#x27;t have to hire a lot of people, and additionally get productivity they never thought possible. A surgeon I know uses AI every day - to translate, to explain, to figure out problems, to write. They wouldn&#x27;t have paid someone to do that, but now they get that output in seconds. This is a time to solve all kinds of problems we didn&#x27;t think possible - because AI has made the enterprising among us instantly smarter.<p>All the fodder about AGI being a next step is smoke and mirrors - for everyone using OpenAI knows they don&#x27;t need any more niche tools as their one $20 subscription is doing more for them every day. AGI is here. Experts can correct AI generated mistakes, but those are getting less and less too. The real benchmark is: Name how many people you know who can out-do ChatGPT on a question. You won&#x27;t bother to check LinkedIn for that.<p>The gains are aggregating towards Chips, Clouds and Entrepreneurs. The VCs, since A16Z&#x27;s original AI blog post (all expense, little return, echoed this Sequoia post but did it but years ago), know they are not needed as much anymore. Fewer VCs will beat the market when founders can grow startups without raising too much money (they don&#x27;t need to hire as many people). Hiring needs lead to PR waves which require VC funding. Valuation is not a big deal for founders making money either, so they may not even disclose how successful their companies are. Bragging about your gains only invites competition. So other than ponzi-type ventures where you need to attract the dinner to serve dinner, you won&#x27;t hear much about the good ones.<p>A different era indeed. The tech giants are in for a lot of change as well. Those who have distribution may try to push their models to the masses to be the point of reference, but that can get expensive, especially for those who don&#x27;t charge. AI will help improve AI performance as well and that means cheaper better performance with time.<p>What&#x27;s most needed in this era are people who know what the world needs that hasn&#x27;t been invented yet. They need to be inventing and monetizing it. Little stopping you now.

&gt; Those who remain level-headed through this moment have the chance to build extremely important companies.<p>But in which sector will these extemely important companies be active? Adtech? Knowledge management &#x2F; productivity tools? Some completely new category?<p>What is an undeniable fact is the drastic commodization of the hardware &#x2F; software stack for certain classes of algorithms. How is this technological development going to be absorbed and internalized by the economy feels still rather uncertain.

Has anyone asked will AI actually very soon be commodity tech? With too many players competing with each other driving prices to bottom? Is there actually meaningful value to extract, after you have spend money on hardware and power?<p>Modern web is full of examples of platforms that really don&#x27;t make money...

This seems about as bearish as a VC is allowed to be. My takeaway is:<p>Sell! Sell! Sell now before it&#x27;s too late!

This is not an AI or ML problem. This is an Nvidia, Google, Microsoft, Meta, Tesla, etc. problem.<p>Most AI startups aren&#x27;t building massive data centers so they&#x27;re unaffected. Most money isn&#x27;t spent on compute in most startups. Only a few companies spend big.<p>It&#x27;s obviously a terrible idea to invest massive amounts into compute when Nvidia&#x27;s profit margins are so astronomical if you need ROI in the long term. The massive corporations won&#x27;t get their money back for these investments; but they don&#x27;t have to.

About a week ago, [Angela Collier posted a video on this topic](<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=dKmAg4S2KeE" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=dKmAg4S2KeE</a>) highlighting the &quot;malicious optimism&quot; of AI hype. She used comments by the Zoom CEO as a basis, but it generalizes.

Data center GPUs are not only used for LLMs. Meta makes hundreds of billions and invests tens of billions on GPUs, using them directly in its core business, not just for cute chatbots. OpenAI is not the clear winner in revenue generation from &quot;AI&quot; if we define AI as deep learning models trained on GPUs.

There&#x27;s a growing divide between inference GPUs and training GPUs.<p>Investors first need to ask what that ratio might look like in 10 years. 10-to-1? 100-to-1? Inference-to-Training<p>Assuming for each NVDA training GPU sold there are 100 open source &#x2F; commodity GPUs doing inference, who owns and supplies those data centers and hardware?

ELI5, why are GPUs not a commodity piece of hardware that any sophisticated chip manufacturer can&#x27;t also produce?<p>This is one of the first time in the tech industry where the value was fully reaped by the hardware itself and not by the differentiated software that ran on top of it

Context: I have been paid to work as an AI practitioner since 2982, nothing special about me except I have worked with amazing people. I have lived through at least two “AI winters.”<p>The author of this blog makes a great argument that there is a risk of AI investments not paying off because if a revenue gap: The author argues that the gap between the revenue expectations implied by the AI infrastructure build-out and actual revenue growth in the AI ecosystem has increased from $200B to $600B. This is due to factors such as the subsiding of the GPU supply shortage, growing GPU stockpiles, and the dominance of OpenAI in AI revenue. The author also notes that the $125B hole in AI revenue has now become a $500B hole.<p>However, my experience with previous AI winters is not relevant here because now is the first time in history that there is a possibility of what we used to call “real AI” and now call AGI. No investor wants to miss out completely on the creation of near limitless wealth.

Are there any case studies of companies&#x2F;cloud vendors using non-Nvidia GPUs for inference at scale?<p>Assuming startups like Etched (with its recent massive funding) could shrink CapEx quite a bit (and make it not such a large revenue shortfall)

The only thing AI needs to do to demonstrate its value is replacing jobs at scale. Until it happens, it’s all capex and opex bs and hobby projects.

Many of the winners haven&#x27;t even been formed yet. And most of the losers have already lost, they just don&#x27;t know it yet.

&gt; Investment incineration<p>Just two words to describe all the idiocy of the current wave of AI offerings.<p>&gt; A huge amount of economic value is going to be created by AI. Company builders focused on delivering value to end users will be rewarded handsomely. We are living through what has the potential to be a generation-defining technology wave.<p>No. It&#x27;s crap no matter how much money you throw at it. It will end in tears, because there is no standard, no operating system, no protocol, only a handful of APIs controlled by a couple of companies. They are not building networks, but hubs with a single point of failure--the API provider. The internet is such a world-changing force because it is built on top of TCP&#x2F;IP, which allows the rest of the internet to survive even if a part of it goes down. When an AI API provider shuts down, all those bullshitters repackaging LLMs will be left holding the bag, or rather their investors will be. In a way, the coming AI bubble burst is going to be a self-fulfilling prophecy--AI will make its creators redundant.<p>&gt; But we need to make sure not to believe in the delusion that has now spread from Silicon Valley to the rest of the country, and indeed the world. That delusion says that we’re all going to get rich quick, because AGI is coming tomorrow, and we all need to stockpile the only valuable resource, which is GPUs.<p>No, the delusion is that Gen AI is good for anything. It is not.<p>Soon, the $600B question is going to be, &quot;where&#x27;s the money gone?&quot;

The problem with AI is that it&#x27;s very much a gimmick and consumers don&#x27;t have much use for gimmicks. Certainly not when it works half the time and hallucinating the other half. All it takes is an NBER announcement to pop this mania.

&gt; AI&#x27;s $600B Question<p>What do you get if you multiply six by seven

The article doesn&#x27;t say anything. First is says AI is meh for profit and it ends with AI is the most important tech ever and will generate tons of money. I think if it wasn&#x27;t written in the sequoia blog it would have gotten very little attention.

<i>&quot;Consider how much value you get from Netflix for $15.49&#x2F;month or Spotify for $11.99. Long term, AI companies will need to deliver significant value for consumers to continue opening their wallets.&quot;</i>

Reminds me of the gnomes from South Park.

I recall Marvin Minsky saying that 1 GigaFlops would suffice for doing most of what humans do. Current AI is addiction to more data, more compute but no smarts. At least they gave us cheap compute, soon the right people will figure out how to produce real AI, not this travesty.