The objective function of LLMs are to predict the next few tokens after in the best possible way.<p>But for humans, the objective function is survival. Evolution has created us human beings, and evolution works by survival of the fittest. So the objective is survival.<p>How do we digitize this objective of survival, so that it can be used to train models leading to AGI?
1) "survival" is a massively oversimplified view of <i>human</i> value functions. Evolutionary biology leads to "successfully replicate genes", but humans are a lot more interesting than that precisely <i>because</i> we can choose to have much more interesting value functions. Anyone and anything that's attempting to actually understand humans needs a much more nuanced view.<p>2) Don't. It's incredibly hard to build a <i>correct</i> value function for an AGI that won't unintentionally kill everyone. "Successfully replicate" is precisely the kind of thing to <i>avoid</i>. That's directly the "gray goo" scenario: <a href="https://en.wikipedia.org/wiki/Gray_goo" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Gray_goo</a>