Gen AI: Too much spend, too little benefit [pdf]

My workstation runs a local LLM that allows me to ask deeply technical questions in plain English and receive useful, usually highly accurate (95%+ of the time) responses in a few seconds.<p>It has no memory and requires no Internet access, so I feel generally comfortable interacting with it in a way that I do not when using Google or Facebook - if I randomly ask about drill presses or brownie recipes I will not suddenly be deluged by ads for toolboxes and Ozempic.<p>This to me is a radically powerful tool, and one that makes me skeptical of the &quot;AI Bust&quot; memes I see on occasion. There may be a lot of dumb money being poured into applying AI to problems it&#x27;s ill suited for, but the things it is good at is such a positive step-function increase over the old tools that I cannot possibly imagine losing it.

&gt; Tech giants and beyond are set to spend over $1tn on AI capex in coming years, with so far little to show for it.<p>Regardless of whether or not the implicit claim here is true (the claim being &quot;all this spend won&#x27;t produce an ROI&quot;), the explicit claim here is nonsensical.<p>Of course the $1tn in capex has nothing to show for it! The spend has not happened yet! Of the spend that _has_ happened, most of the chips are not physically in data centers yet. Of the chips that _are_ in data centers, most of the models are not yet trained!<p>And of the models that _have_ been trained, many have clearly had a significant ROI. GPT-4 cost $100m, and OpenAI&#x27;s revenue is now reported to be $3.4 billion a year.<p>Saying there&#x27;s &quot;little to show for it&quot; is an absurd claim; the products are _printing_ cash! We beat the turing test! You can drive around in a self-driving car!<p>It&#x27;s perfectly reasonable to say &quot;where does the ROI come from when you spend $1tn on capex&quot;, but it&#x27;s hard to argue against the success of the spend of the last generation of models.

The technology has been around for about one year now. For people who have been working in the field for as long as I have (&gt; 30 years), the output is absolutely incredible. Really incredible... However, for the vast armies of academics and others who went on a different route, the result is bitter. We failed to deliver anything close to LLMs. I put myself in the list. The problem is that many of these technologies have been developped by the GAFAM, which very people have confidence in, for obvious reasons, however these were the only entities with enough computing power to do so. Nothing surprising here. Now we have many people who think that AI is a hoax and that it is going to burst soon. There is something pretty religious here, it reminds me of these people who get convinced by gurus that the world will end on a specific date for reasons... Even in its state, AI is already incredibly useful, and I don&#x27;t see anything to convince me that AI has reached a glass ceiling. Believe me, 5 years ago this glass ceiling was much.. much lower than today.

I think it&#x27;s a little weird to say there&#x27;s no killer app when millions of people already use ChatGPT and tools like GitHub Copilot are already quite embedded into professional workflows. It&#x27;s just not a given that these will truly take off into the stratosphere as the next Google or whatever. If people are expecting that to happen in the next year, well, I guess they spent too much money. Whoopsies.

When someone has a short position, they and the submission author will use all kinds of biased information to manipulate others to sell.

This is probably accurate.<p>But GenAI is a major upheaval. In most those, a massive amount of initial capital is invested, but the payoff happens slowly - only over decades. But the payoffs are huge.<p>Think about electricity. Building out the grid was HUGELY expensive. But then benefits are derived for decades. Same for the highway system, railway system, etc.

AI is useless for any kind of foresight from human intuition and serendipity, it answers without any clue of the impacts of its answers and when they filter responses out AI just gets more useless as censorship is known to reduce culture and learning.

Within large tech companies AI projects (often just features within existing products) are seen as the safe place to be while non-AI projects have higher layoff rates.<p>I wonder how much of the AI spend is driven by the above situation.

&gt;And he doesn’t take much comfort from history that shows technologies improving and becoming less costly over time, arguing that AI model advances likely won’t occur nearly as quickly—or be nearly as impressive—as many believe.<p>Currently we need cards with larger memory, then we&#x27;ll need faster cards, then cards with more memory again.<p>The only reason why we don&#x27;t have $1000 64gb cards is because, as the article points out, NVidia currently has a stranglehold in the market. Given that GPUs print money it won&#x27;t be more than a few years before we see those cards from other parties.<p>At the same time the improvement in small models has been phenomenal, there are now 8b models that consistently outperform the original gpt-3.5-turbo. In two years the state of the art model is now something that can run in your phone for free.

Lots of warning signs here, good on GS to call it out<p>[1] virtually one supplier in the mix (NVidia)<p>[2] Costly ops not returning good value<p>[3] Signal-to-noise is extremely low in the GenAI services space, where Blockchain grifters moved following the notional collapse

⠀