This research is all beyond me so maybe someone can explain: How does this compare to the state of the art in using simulators to train physical robots? Does using transformers help in any way or can this just as easily be done with other architectures?<p>To the uninitiated this looks cool as all heck and yet another step towards the Star Trek future where we do everything in a simulator first and it always kinda just works in the real world (plot requirements notwithstanding).<p>Although I can also hear the distant sounds of a hundred military R&D labs booting up Metalhead [1] simulators.<p>Edit: Looks like the previous SOTA was still a manual process where the user had to come up with a reward function that actually rewards the actions they wanted to the algorithm to learn. This research uses language models to do that tedious step instead.<p>[1] <a href="https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror)" rel="nofollow">https://en.wikipedia.org/wiki/Metalhead_(Black_Mirror)</a>
This looks very singularity-ish to me; LLMs can construct a reward function that can be trained in simulation, and the trained reward function works surprisingly well in the physical world.<p>Some of the videos raised questions in my mind as to whether or not the leash was doing stabilization work on the robot; it might be. But, if you watch the video where they deflate a yoga ball, you can see the difference when they're "saving" the robot and when they're just keeping it taut. The 'dextrous cube' manipulation video is also pretty compelling.<p>This sort of 'automate a grad student' type work, in this case, coming up with a reasonable reward function, all stacks up over time. And, I'd bet a lot of people's priors would be "this probably won't work," so it's good to see it can work in some circumstances -- will save time and effort down the road.
Every single second of every example has a handler holding a leash - and not just holding it, holding it without any slack.<p>Blindingly obvious interference from Ouija board effect.<p>I don't mean to denigrate the work, I believe the researchers are honest and I hope there's demoes outside the published one. Just, at best, an obvious unforced error that leaves open a big question.<p>EDIT: Replier below shared a gif with failures, tl;dr this looks like two different experiment protocols, one for success, one for failure. <a href="https://imgur.com/a/DmepBVU" rel="nofollow">https://imgur.com/a/DmepBVU</a>
Kind of like how a human visualizes before a sport.?<p>Like visualizing free throws in basketball, makes you measurably better, without actually doing free throws for real?