<i>> It has no intelligence; it doesn't know anything and doesn't understand anything.</i><p>I don't like the word "intelligence". It's too arbitrary and depends on the language. In my native language, there are two synonyms which imply different thresholds for something to be considered intelligent. I'm sure in other languages it's also pretty arbitrary.<p>Instead, let's compare those models with complex biological systems we typically consider to be at least somewhat intelligent.<p>- biological systems use spiking networks and are incredibly power efficient. This is more or less irrelevant for capabilities.<p>- biological systems have a lot of surrounding neural and biochemical hardware - hardwired motorics, senses processing, internal regulators. Complex I/O is missing from these models, but is being added as we're talking. The large downside of current models is that it cannot understand what drives humans as it has different hardware, it's trained on their output, and has to "reverse engineer" the motivation. Which might or might not be possible, but it makes them <i>different</i>.<p>- biological systems are <i>autonomous agents</i> in their world. They exist on an uninterrupted timeline, with input and output streams constantly present. Those models don't exist on a timeline, they are activated by the user each time.<p>- biological systems have some form of memory; they compress incoming data into higher order concepts on the fly, and store them. This is a HUGE DEAL. The model has no equivalent of memory or neuroplasticity, it's a function that doesn't keep any state. LLMs have the context which can be turned into a sliding window with an external feedback loop (chatbots do that), however it's not equivalent to biological memory at all, as it just stores tokens verbatim, instead of trying to compress the incoming data.<p>- biological systems exhibit highly complex emergent behavior. This also happens in LLMs and even simpler models.<p>- biological systems are social. Birds compose songs from tokens, and spread them through the population. Dogs, monkeys, and humans teach their kids. The mental capacity of a human isn't that great; every time you think you're smart, remember that you stand on the shoulders of giants. The model does have much more capacity than a human.<p>My own conclusion: sparks of "intelligence"? Undeniably, the emergent behavior alone is enough. They <i>do</i> understand things, in the conventional terms. However, they are also profoundly different than human intelligence, and still lack key elements like the memory.