> Genie is capable of converting a variety of different prompts into interactive, playable environments that can be easily created, stepped into, and explored<p>If these are generating a fully interactive environments, why are all the clips ~1 second long?<p>Based on the first sentence in your paper, I would have expected a playable example as a demo. Or 20.<p>But reading a bit further into the paper, it sounds like the model needs to be actively running inference and will generate the next frame on the fly as actions are taken- is that correct?
Firstly, do these models learn a good physics grounding for nonsense actions? Like keep pressing down even when you are in the ground? Or will they phase you through the ground?<p>Secondly, why are all videos like half a second long? I thought video generation came much farther than this. My guess would be that the world models unravel at any length longer than that, which is (and has always been) the problem with models such as these. Minus the video generation part, we had pretty good world models for games already, see Dreamer line of work: <a href="https://danijar.com/project/dreamerv3/" rel="nofollow">https://danijar.com/project/dreamerv3/</a>
The results seem quite bad. Compare the static image and "game" in this one example:
Static Image: <a href="https://lh3.googleusercontent.com/c0GV4hG0Xg0eqpsUS1z62v6aJ2qRGYKmmyJLbpfp1DbJ92elZ_lAvQpx4WSBVKbDO97OBvc0rhu_gNhXbzI_Uv121_QkX9Ur6m3MhMINg8kJ35RMNlA2lxQ2fIuennhBaQ=w1280" rel="nofollow">https://lh3.googleusercontent.com/c0GV4hG0Xg0eqpsUS1z62v6aJ2...</a>
"Game": <a href="https://lh5.googleusercontent.com/L_WsAa1saPmj29DSKda_fzk15y2Qk2Rvh3-b-4-n6EdXgaMJ6DrF5Chp3Oh9Nd6Pzz9IfmEwRJnM1t18zo2bsS9nNxpY1sr3pVJabbM-n1jf2bNbVAX9QEQiKlfcNYoQWQ=w1280" rel="nofollow">https://lh5.googleusercontent.com/L_WsAa1saPmj29DSKda_fzk15y...</a><p>In the video, the character becomes a pixelated mess. In the static image, the character is clearly on rocks in the foreground, but in the "game" we see the character magically jumping from the foreground rocks to the background structure which also contains significant distortions.<p>The extremely short demo videos make it slightly harder to catch these obvious issues.
I'm old an immediately assumed this would link to historical retrospective of GEnie<p><a href="https://en.wikipedia.org/wiki/GEnie" rel="nofollow">https://en.wikipedia.org/wiki/GEnie</a>
Seems very interesting, but as soon as I see "Google Research" or "Deepmind" now it's an instant turndown. Too much PR, not enough substance. Not targeting directly you guys with this research, but the company you work for.
Looking forward to following your progress. I've been wanting to see how we might replace polygons for gaming long term, this seems like a step in the right direction.