LLM discourse needs more nuance

Is this a good summary?<p>LLMs have solved the language processing problem. Its responses are fluent and rapidly becoming indistinguishable from human output. Or if anything, its responses are too good to be mistaken for an average human.<p>However, that’s different from producing accurate knowledge and insights. It’s often bullshitting, in the sense that it’s convincing prose often turns out to not reflect reality.<p>The reference problem seems to be one example of this.

It’s an interesting situation because on one level, the technology is at least somewhat misrepresented (best case results rising to the top of everyone’s feeds), but on the other side, we haven’t even begun to see what integrated Chat GPT will bring. If I could replace Siri with Chat GPT, I would do so immediately, as it’s objectively better. The author makes some points about the eye watering costs; however, if this is something that people truly want—and I think we do want it—the market will find a way to deliver it at scale in a cost effective way.

Is the author conflating the cost to train an AI model with the cost to use the resulting models, here?<p>I can certainly believe that model <i>training</i> cost is exponential as the # of parameters goes exponential. But that is a one time(-ish) cost relative to actually using those models.<p>Or am I completely off base here?

I&#x27;m not sure that there&#x27;s anything truly new in this article. Transformer-based models are very compute- and parameter-heavy, yes - but that&#x27;s because they&#x27;re optimized for generality and easily parallelizable training, even at a very large scale. The ML community has always been aware of this as an unaddressed issue. Once compute cost for both training and inference becomes a relevant metric, there&#x27;s lots of things you can do as far as model architecture goes to make smaller and leaner varieties more applicable.

In a sense, some people are getting ahead of themselves - which by the way is normal in technological cycles, with typically financial innovation going beyond what the underlying innovation can support.<p>If AI fails to move into high risk applications and instead stays largely confined to low risk applications, then the revenue pools will be stay limited to more consumer and small business facing services. Those are big pools but it will also not create the trust needed for certain high risk applications.

Why most people are blind to the most important point of those models: the progress is moving at astonishing pace.<p>IMO it is a useless discussion to debate about the current SOTA or economics. In two years we will certainly have some breakthrough as we have been seen for the past years. And things are speeding up.<p>edit: typo.

Investors don&#x27;t &quot;believe&quot; in AI anymore than they &quot;believed&quot; in crypto. But they can smell a bubble, and hope to bail out at the top.

<i>While personally, the experience lived up to some of the hype, social media claims of it implementing entire front ends turned out to be fake. Clearly, this models performs well on in-sample tasks, and is far off on anything else.</i><p>That wasn&#x27;t my experience. ChatGPT dealt with everything I threw at it, except when it refused to, and even then I could get round it by telling it to pretend etc. Every wild example that I saw on twitter I was able to reproduce

&gt;&gt; But only those that users liked and retweeted ended up circulating, which contributed to a strange spread between a users&#x27; expectations and experiences they had when first talking to chatGPT.<p>Which is exactly what makes this so dangerous. In our current culture, only those voices that are promoted and amplified by social media matter. So a tech than can produce material in industrial quantities, even if most of it is tripe, could be a powerful manipulation tool. It will certainly be cheaper than hiring flesh-and-blood people for man a troll farm.

A lot of what this takes are missing is that the unreliability of those models is not a hurdle for a tremendous amount of applications. If there is a human in the loop that can now write a couple of words in a fuzzy human language, check or select or edit the resulting response, and thus do their job twice as fast or god beware 10x as fast, that is absolutely a game changer.<p>My main criticism for LLMs are:<p>- the way they were rolled out was counterproductive. Unleashing a chatbot that pretends it knows everything, without any background and guardrails, is directly responsible for the hype untethered discourse that is prevalent in the mainstream. In the literature and among practitioners, every body is well aware that these things don&#x27;t &quot;think&quot;<p>- for the first time, it feels that a significant amount of what my value as a aprogrammer will be fully owned by a corporation and trickled out to me for $99.95 a month. It&#x27;s already the case with copilot. I can&#x27;t imagine going back to a world where I work without gpt3 and copilot, which gives me no choice but to fully embrace my corporate overlords. I fully feel what farmers feel wrt their tractors.<p>The best I can do for now is figure out what the real usecases are for me and how to leverage GPT3, and start looking heavily into open models, so that I can help out with whatever unix &lt;&gt; bsd situation we are going to end up with.<p>None of this has anything to do with the end of human culture, education, discourse, or the end of quality in software. If software quality could be any lower, under capitalism, it would be. It&#x27;s not like I can get really shitty code that pretends to do something for $5&#x2F;h on upwork.

I&#x27;ll stick to waiting this one out, because I know &quot;investor&quot; personality types are mostly comparable to an individual sitting on a pink cloud wearing sunglasses that won&#x27;t allow a singly light particle to enter.<p>I&#x27;m hearing quite little about the downfall of AI route that science fiction predicted at times. Always good to waddle into that middle outcome.

The part about compute cost ruining the economics of AI companies is somewhat hilarious and eye rolling to me for the following simple reason: users have computers right in front of them.<p>Some models are small enough to run locally. For the rest is it conceivable to ask the customer to contribute cycles? Especially for anything cheap or free?<p>There might be major technical challenges here but that’s not my point. My point is that I get the distinct impression this has barely occurred to most of these people as a possibility to even explore.<p>Are we so far into peak SaaS now that the industry has forgotten about local compute the same way we forgot about data centers during peak PC?

Not sure about ChatGPT, but Midjourney et al have definitely transformed many markets already. Why would I get any illustrations from fiverr.com, when I can get better turnaround times for $0 from Midjourney?

If chatgpt and AI friends is the solution, what is the problem? If we push the AI madness aside the problem definition seems to be: I want to query large amounts of textual data to find an &quot;optimal match&#x2F;answer&quot; to a given user input&#x2F;&quot;question&quot;.<p>What is the precise nature of this optimal match? It is unclear (as it is essentially encoded in a black box algorithm and its training data set). Different algorithms and different training data sets would provide different &quot;answers&quot;.

The tweet from Sam Altman in this article made me laugh out loud.

To show the cost of extra parameters is exponentional, they made a graph with price linear and parameter count logarithmic and then fit an exponential? It seems very misleading:<p><a href="https:&#x2F;&#x2F;proofinprogress.com&#x2F;assets&#x2F;images&#x2F;cost-llms.png" rel="nofollow">https:&#x2F;&#x2F;proofinprogress.com&#x2F;assets&#x2F;images&#x2F;cost-llms.png</a><p>With linear or polynomial increase per parameter it would also look exponential with that graph setup.

As long as these AI models are unable to transparently explain themselves then all of this is essentially a vacuum of hype to those who don&#x27;t understand that such black box models are essentially <i>&#x27;throw more money and data on it&#x27;</i> for the gamble on accuracy and the risk of overfitting and wasting money.<p>No different to the majority of proof-of-waste cryptocurrencies like Bitcoin.

I think everybody overestimates in the beginning, here it’s the same. AI will solve every problem, nope it won’t, but it will help.

I spent lots of time to think of the title „The AI Crowd is Mad“ because I wanted to hit all triggers for virality (belief, belonging, behavior). A bit disappointed it was moderated&#x2F;changed

Reading this article is like walking through deep mud. This is largely due to the author‘s excessive and baffling use of commas. But also due to myriad grammatical errors and ambiguous sentence constructions. Parsing some of his sentences is like looking at an optical illusion or an ambiguous painting. All things considered, I had a bad time attempting to read this article. I do not look forward to reading more of this author‘s writing in the future.

What a pompous style. I got annoyed trying to read it.<p>By the way, <a href="https:&#x2F;&#x2F;www.theatlantic.com&#x2F;technology&#x2F;archive&#x2F;2023&#x2F;01&#x2F;chatgpt-ai-language-human-computer-grammar-logic&#x2F;672902&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.theatlantic.com&#x2F;technology&#x2F;archive&#x2F;2023&#x2F;01&#x2F;chatg...</a> shows that at least some of the broader press gets what&#x27;s going on with &quot;AI.&quot;

Not the plane with the dots again!

content warning: crypto&#x2F;web3 bro

Imagine spending hundreds of thousands of dollars on infrastructure to serve out content that isn&#x27;t even yours to begin with, and then wonder why there&#x27;s such a blowback in its usage

I don&#x27;t think anyone is being &quot;deceived&quot; when it comes to ChatGPT. In fact I&#x27;ve found it to be just the opposite. People are still completely unaware of how game changing and revolutionary it is until they sit down and actually use it. It&#x27;s impossible to describe to someone just how different this is than anything else before, as the AI well has been so thouroughly poisoned by the false promises of prior tools.