Mapping the semantic void: Strange goings-on in GPT embedding spaces

This is the first post in LessWrong that I found interesting and not insipid navel gazing.<p>My interpretation of the finding is that the LLM training has revealed the primordial nature of human language, akin to the primary function of genetic code predominately being to ensure the ongoing mechanical homeostasis of the organism.<p>Language’s evolutionary purpose is to drive group-fitness. Group management fundamentals are thus the predominant themes in the undefined latent space.<p>Who is in our tribe? Who should we trust? Who should we kill? Who leads us?<p>These matters likely dominated human speech for a thousand millennia.

So I am a bit confused about the part where you go k distance out from the centroids.<p>Since there are ~5000 dimensions, in which of those dimensions are we moving k out ?<p>Is the idea you just move, out, in all dimensions such that the final Euclidean distance is k ?<p>Seems that’s how they get multiple samples at those distances.<p>Either way I think it’s more interesting to go out in specific dimensions. Ideally there is a mapping between each dimension and something inherent about the token, like the part where a dimension corresponds with the first word of the token.<p>We went through this discovery phase when we were generating images using autoencoders, same idea, some of those dimensions would correspond to certain features of the image, so moving along them would change the image output in some predictable way.<p>Either way, I think the overall structure of those spaces says something about how the human brain works ( given we invented the language). I’m interested to see if anything neurologic can be derived from those vector embeddings.

I think that&#x27;s really interesting but one thing that I&#x27;m not sure about is that a lot of tokens are just fragments of words like &quot;ei&quot;, not whole words. Can we really makes conclusions about those embeddings?

Should try asking it to define things in French. Korean, etc.

That is _profoundly_ interesting, and I think a serious study of that is going to reveal a lot about human (or at least english speaking) psychology.

Based on the first three figures, it seems like the author isn&#x27;t familiar at all with probability in high-dimensional spaces, where these phenomena are exactly what you&#x27;d expect. Because of that, the figure with intersecting spheres is a big misinterpretation. I stopped reading there, but I think it&#x27;s good to generally ignore things you find on lesswrong!

Please can someone explain the gist of this article to a mere human with no PhD in AI?

Someone shoot me down if I&#x27;m wrong:<p>In this model (one of many infinitely many possible models) a word is a point in 4096-space.<p>This article is trying to tease out the structure of those points, and suggesting we might be able to conclude something about natural language from that structure - like looking at the large-scale structure of the Universe and deriving information about the Big Bang.<p>Obvious questions: is the large-scale structure conserved across languages?<p>What happens if we train on random tokens - a corpus of noise? Does structure still emerge?<p>It might be interesting, it might be an artifact. I&#x27;d be curious to know what happens when you only examine complete words.

Would be interesting to see how changing the tokenization strategy — whole words or even phrases instead of traditional tokens — changes the results.

Has anyone done this analysis for other llms like llama2?

Maybe a dumb question: Shouldn&#x27;t these be called immersions instead of embeddings?