The author does a good job of pointing out what may be the strongest skills of LLMs but the claim they aren't useful as a search engine didn't ring particularly true. For many questions I have ChatGPT is the best tool to use because I know the topics I'm asking about are mentioned hundreds of times in the web, and the LLM can distill down at knowledge to the specifics I'm asking about. If you treat it as a friend who has a ton of esoteric knowledge in many areas but is prone to making stuff up to sound like they know what they're talking about you can still get lots of use pulling facts and some basic reasoning out of the models.
As a long time LLM enjoyer I just want to mention <a href="https://generative.ink/posts/simulators/" rel="nofollow">https://generative.ink/posts/simulators/</a> as I think it's by far the most insightful take on the GPT LLMs even though it was from before ChatGPT. It's better than blurry jpeg and stochastic parrot etc.
I've seen this take a lot, and I find it frustrating as it flies in the face of the information theory underpinning how large neural networks learn information.<p>This is more than just a fancy zip file of Markov sequences. Someone has got to put a stop to this silly line of reasoning, I'm not sure why more people familiar with the math of deep learning aren't doing their best to dispel this particular belief (which people will then use as the foundation for other arguments, and so on, and so on, and this is how misconceptions somehow become canon in the larger body of work).
"The ChatGPT model is huge, but it’s not huge enough to retain every exact fact it’s encountered in its training set."<p>That's because there is no way for the model to take the internet and separate fact from fiction or truth from falsehood. So it should not even try to, unless it can somehow weigh options (or preform its own experiments). And that doesn't mean counting occurrences, it means figuring out a coherent worldview and using it as a prior to interpret information, and then still acknowledging that it could be wrong.
> Language models don’t—if you run the same prompt through a LLM several times you’ll get a slightly different reply every time.<p>You can get deterministic output (on a given machine) by setting temperature=0. The Chatgpt interface doesn't let you do that, but the playground API does.
The most telling thing about the state-of-the-art currently is that somewhere between all the marketecture in no-code and low-code, we can essentially create a recursive crawler on-the-fly on par with a large cloud provider (because it is) and ask it to do recursive crawls that at least begin to approach a sensemaking machine, i.e. it can provide consensus for subjective truth agreed upon by experts above and beyond simple objective truth. To me, that's the great innovation that truly embraces and extends average intelligence by providing a prosthetic device for the brain to make inferences that would have only been approachable by high functioning individuals previously, i.e. the kind of argument about consciousness that you see from low latency inhibition researchers like Peterson. What any individual does with this is really where prompt engineering becomes the human-computer agent collaboration that should be our default mode in computing - a kind of tortoise wins the race story where we lost our minds in the race to interaction via javascript. It's not terribly interesting to watch a computer type. There's a place for batch mode (queuing, etc) , if the tools built up around it handle long running job management well. Sadly, that seems rarer to me now than 30 years ago.
It makes me sad that the next time I enjoy a piece of writing, I'm going to have to wonder if it was "enhanced" or even written wholesale by ChatGPT. I don't feel the same with arithmetic at all.
Often it's a hint generator that tells you where to look and what you could try.<p>The hints are not calculated from the input, they're from the training set.
Another powerful use not mentioned in the article is the ability to convert unstructured data into structured date.<p>For example you can copy paste a page describing API documentation and ask an LLM to not only make an API call but then also interpret results. This is the most fascinating use of LLMs to me so far.
Good to see more people pointing out the problems using standalone llm (I.e. not connected to an external data source/the internet) for search. So many people I talk to dismiss gpt because it can’t answer some subject-specific question accurately.
I think the editorial capabilities of ChatGPT are fantastic, and the author provides a good list of examples. On top of that I would add that ChatGPT is really good as composing text. The meaning making is therefore still what we have to do.
Please stop linking to Ted Chiang's article. I like him as an imaginative writer, but his article is just wrong and gives readers incorrect intuitions. His claim that GPT models are not able to learn decimal addition has been known to be false for years and you can verify yourself that GPT-4 can do it.
Great analogy & breakdown.<p>My go-to explanation is to think of ChatGPT like a really intelligent friend who's always available to help you out – but they're also super autistic, and you need to learn the best way to interact with them over time.
I think a better analogy would be: "ChatGPT as a Crappy Primary School Teacher", has hard-coded traditionalistic morales, and can answer anything but information quality is strictly not guaranteed.
> if you run the same prompt through a LLM several times you’ll get a slightly different reply every time.<p>If it has the same seed, why would you get a different reply?