Insistence that LLMs aren't intelligence and are just parrots and glorified autocomplete make me think of the argument against evolution where there's this missing link between Australopithecus and Homo Sapiens (though that's in, itself, an outdated theory) and if only scientists would find that missing link, then they'd believe in evolution. Just in reverse. Obviously my time machine is as good as your time machine, so I have no idea if this will lead to AGI, but even as it stands, these things are useful and are such a good fascimile of intelligent that it's easier to call them an artificial intelligence. there's this notion that they can't create anything outside of their training data, which means they're not intelligent in the same way that humans are, but that's the whole point of calling them artificial, because they aren't! can humans create things outside of their training data? is Dali an LLM,
unable to create beyond the bounds of his training data because, even at the height of his creativity, clocks and melting are preexisting concepts? do humans create new things? of course. can an LLM create a new thing? hmm, that's less obvious, except that when it's writing or drawing, that particular arrangement of bits hasn't existed before, which means it's creating things on the face of it. Creating new things, then, is entirely within its capabilities. We had older models that created entirely new words, but we rejected them because all they created was giberrish. GPT-2 was great at making off-kilter poems. where do new ideas come from in humans? I don't know, but it's based on their training data. in predicting the next word, LLMs predict something from its training data, far more vast than any one human could hold in their mind. can it create an entirely brand new thing? that seems like the wrong question to ask when it does it every time it answers, like asking where's three missing link, every time a new link is found.