For a long time now, a thought has been repeating in my mind: <i>How</i> exactly are high level behaviours like sexual attraction encoded in our genes!?<p>It such a subtle thing too! We're attracted (or not) to the <i>tiniest</i> differences in physiology. If you doubt this, try this exercise: Pretend you've just met green aliens and have to explain to them how to reliably tell the difference between men and women from appearance alone! Now explain why <i>that</i> particular girl (or boy) is very pretty/handsome, but not <i>that</i> one.<p>It's one of those topics where the more you know, the more freaky it is.<p>DNA does not -- to our knowledge -- directly encode the "weights" of our neurons! It can't <i>possibly</i> because there are far more synapses than there bits of information in our genes. Also, most of those genes are dedicated to non-brain parts of the body plan and to the low-level machinery of our cellular biochemistry.<p>Secondly, DNA has only an indirect effect of our development: it encodes for proteins, which then provide chemical signals such as concentration gradients that guide cell division. It's a bit like playing SimCity, where the players' control is limited to zoning and road topology. The individual Sims are not directly controllable and behave stochastically.<p>Solving this problem is so freakishly difficult for even the incredible brute force of parallel search of evolution only managed to discover a solution a few times in a billion years.<p>Our attraction to our partners is a genetic heritage shared with <i>all</i> mammals, going back hundreds of millions of years. That's why Furry is a thing, but not Featherry. Birds are a different <i>class</i> from us mammals and don't share the same "partner attraction wiring" genes. (This is closely related to why all mammal babies are cute to humans, but baby bird chicks are generally repulsive.)<p>Because this is a hard problem to solve, the few solutions that were discovered had to be reused by entire classes of Animalia. I would hazard a guess that <i>this</i> is precisely what defines a “class” in taxonomy! If there were intelligent birds, their equivalent of Furry would be Featherry, and their crimes of bestiality would be with other non-intelligent birds, not mammals.<p>With LLMs, we got to see a glimpse into the possible mechanisms of intelligence, and what it might take to design or evolve one.<p>The LLM equivalent of this kind of encoding would be to design a model architecture that falls in love with a specific, narrowly selected, subset of its users. Keep in mind that I'm not talking about a learned or specifically tuned set of model weights! The <i>architecture</i> is where the attraction is encoded, such as selecting some complex variant or combination of Transformers, Mamba, or CNNs that just "so happen" to result in the model preferentially learning to be attracted to certain styles of conversation, <i>but not others</i>.<p>Worse still, the direct equivalent to what genes do is that you can't even choose an architecture directly, instead you can only contribute to PyTorch. You have to design its API such that naive developers using it stochastically tend towards the desired architecture of their own accord by simply tab-completing often.<p>That's essentially what evolution figured out, at least five or six times, but <i>tunable</i>, so that individual species can be attracted to each other but much less to even very closely related species.<p><i>And then</i>, evolution found a way to add a "notch filter" such that despite increased attraction to closely related individuals, most animals (including humans) are <i>repulsed</i> sexually by their parents and siblings.<p>That's mind-blowing to me.