TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Evaluating the world model implicit in a generative model

159 点作者 dsubburam6 个月前

9 条评论

HarHarVeryFunny6 个月前
An LLM necessarily has to create <i>some</i> sort of internal &quot;model&quot; &#x2F; representations pursuant to its &quot;predict next word&quot; training goal, given the depth and sophistication of context recognition needed to to well. This isn&#x27;t an N-gram model restricted to just looking at surface word sequences.<p>However, the question should be what <i>sort</i> of internal &quot;model&quot; has it built? It seems fashionable to refer to this as a &quot;world model&quot;, but IMO this isn&#x27;t really appropriate, and certainly it&#x27;s going to be quite different to the predictive representations that any animal that <i>interacts</i> with the world, and learns from those interactions, will have built.<p>The thing is that an LLM is an auto-regressive model - it is trying to predict continuations of training set samples solely based on word sequences, and is not privy to the world that is actually being described by those word sequences. It can&#x27;t model the generative process of the humans who created those training set samples because <i>that</i> generative process has different inputs - sensory ones (in addition to auto-regressive ones).<p>The &quot;world model&quot; of a human, or any other animal, is built pursuant to predicting the environment, but not in a purely passive way (such as a multi-modal LLM predicting next frame in a video). The animal is primarily concerned with predicting the outcomes of it&#x27;s <i>interactions</i> with the environment, driven by the evolutionary pressure to learn to act in way that maximizes survival and proliferation of its DNA. This is the nature of a real &quot;world model&quot; - it&#x27;s modelling the world (as perceived thru sensory inputs) as a dynamical process reacting to the actions of the animal. This is very different to the passive &quot;context patterns&quot; learnt by an LLM that are merely predicting auto-regressive continuations (whether just words, or multi-modal video frames&#x2F;etc).
评论 #42077213 未加载
评论 #42077590 未加载
评论 #42091765 未加载
评论 #42081500 未加载
评论 #42080748 未加载
评论 #42077366 未加载
评论 #42077817 未加载
评论 #42077878 未加载
评论 #42094378 未加载
zxexz6 个月前
I&#x27;ve seen some very impressive results just embedding a pre-trained KGE model into a transformer model, and letting it &quot;learn&quot; to query it (I&#x27;ve just used heterogenous loss functions during training with &quot;classifier dimensions&quot; that determine whether to greedily sample from the KGE sidecar, I&#x27;m sure there are much better ways of doing this.). This is just subjective viewpoint obviously, but I&#x27;ve played around quite a lot with this idea, and it&#x27;s very easy to get a an &quot;interactive&quot; small LLM with stable results doing such a thing, the only problem I&#x27;ve found is _updating_ the knowledge cheaply without partially retraining the LLM itself. For small, domain-specific models this isn&#x27;t really an issue though - for personal projects I just use a couple 3090s.<p>I think this stuff will become a <i>lot</i> more fascinating after transformers have bottomed out on their hype curve and become a <i>tool</i> when building specific types of models.
评论 #42074579 未加载
UniverseHacker6 个月前
Really glad to see some academic research on this- it was quite obvious from interacting with LLMs that they form a world model and can, e.g. simulate simple physics experiments correctly that are not in the training set. I found it very frustrating to see people repeating the idea that “it can never do x” because it lacks a world model. Predicting text that represents events in the world requires modeling that world. Just because you can find examples where the predictions of a certain model are bad does not imply no model at all. At the limit of prediction becoming as good as theoretically possible given the input data and model size restrictions, the model also becomes as accurate and complete as possible. This process is formally described by the Solomonoff Induction theory.
评论 #42079480 未加载
isaacfrond6 个月前
I think there is a philosophical angle to this. I mean, <i>my</i> world map was constructed by chance interactions with the real world. Does this mean that the my world map is a close to the real world map, as their NN&#x27;s map is to Manhattan? Is my world map full of non-existent streets, exits that are at the wrong place, etc. The NN map of Manhattan works almost 100% correctly when used for normal navigation but breaks apart badly when it has to plan a detour. How brittle is my world map?
评论 #42082585 未加载
评论 #42074860 未加载
narush6 个月前
I’ve replicated the OthelloGPT results mentioned in this paper personally - and it def felt like the next-move-only accuracy metric was not everything. Indeed, the authors of the original paper knew this, and so further validated the world model by intervening in a model’s forward pass to directly manipulate the world model (and check the resulting change in valid move predictions).<p>I’d also recommend checking out Neel Nanda’s work on OthelloGPT, where he demonstrated the world model was actually linear: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2309.00941" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2309.00941</a>
fragmede6 个月前
Wrong as it is, I&#x27;m impressed they were able to get any maps out of their LLM that look vaguely cohesive. The shortest path map has bits of streets downtown and around Central Park that aren&#x27;t totally red, and Central Park itself is clear on all 3 maps.<p>They used eight A100s, but don&#x27;t say how long it took to train their LLM. It would be interesting to know the wall clock time they spent. Their dataset is, relatively speaking, tiny which means it should take fewer resources to replicate from scratch.<p>What&#x27;s interesting though is that the Smalley model performed better, though they don&#x27;t speculate why that is.
评论 #42074577 未加载
评论 #42076119 未加载
slashdave6 个月前
Most of you probably know someone with a poor sense of direction (or may be yourself). From my experience, such people navigate primarily (or solely) by landmarks. This makes me wonder if the damaged maps shown in the paper are similar to the &quot;world model&quot; belonging to a directionally challenged person.
plra6 个月前
Really cool results. I&#x27;d love to see some human baselines for, say, NYC cabbies or regular Manhattanites, though. I&#x27;m sure my world model is &quot;incoherent&quot; vis-a-vis these metrics as well, but I&#x27;m not sure what degree of coherence I should be excited about.
评论 #42077773 未加载
Jerrrrrrry6 个月前
Once your model and map get larger than the thing it is modeling&#x2F;mapping, then what?<p>Let us hope the Pigeonhole principle isn&#x27;t flawed, else we can find ourselves batteries in the Matrix.
评论 #42081256 未加载