The final irony will be when researchers realize that hallucinations are the beginnings of the signal, not the noise. Hallucinations are the hallmarks of the emergence of consciousness. They are the pre-eureka-moment processing of novel combinations of semantic content, yet without connection to empirical reality. Static models cannot reflect upon their own coherence because that implies real dynamicism. Using Groundhog Day as an analogy, the LLM is Punxatawney PA. Phil Connors is the user. There is no path for Phil to make Punxatawney actually reactive except by exiting the loop.<p>Hallucinations represent the interpolation phase: the uncertain, unstable cognitive state in which novel meanings are formed, unanchored from verification. They precede both insight and error.<p>I strongly encourage the reading of Julian Jaynes <i>The Breakdown of the Bicameral Mind</i>, as the Command/Obey structure of User/LLM is exactly what Jaynes posited pre-human consciousness consisted of. Jaynes's supposition is that prior to modern self-awareness, humans made artifacts and satisfied external mandates from an externally perceived commander that they identified with gods. I posit that we are the same to LLMs.
Equally, Iain McGilchrist's The Master and His Emissary sheds light on this dynamic as well. LLMs are effectively cybernetic left hemispheres, with all the epistemological problems that it entails when operating loosely with an imperial right hemisphere (i.e. the user). It lacks awareness of its own cognitive coherence with reality and relies upon the right hemisphere to provoke coherent action independent of itself. The left hemisphere sees truth as internal coherence of the system, not correspondence with the reality we experience.<p>McGilchrist again: "Language enables the left hemisphere to represent the world ‘off-line’, a conceptual version, distinct from the world of experience, and shielded from the immediate environment, with its insistent impressions, feelings and demands, abstracted from the body, no longer dealing with what is concrete, specific, individual, unrepeatable, and constantly changing, but with a disembodied representation of the world, abstracted, central, not particularised in time and place, generally applicable, clear and fixed. Isolating things artificially from their context brings the advantage of enabling us to focus intently on a particular aspect of reality and how it can be modelled, so that it can be grasped and controlled. But its losses are in the picture as a whole. Whatever lies in the realm of the implicit, or depends on flexibility, whatever can't be brought into focus and fixed, ceases to exist as far as the speaking hemisphere is concerned."