Solving the Rubik’s cube with a robot hand

&gt;We’ve trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The neural networks are trained entirely in simulation<p>Simulated training is so cool. Related, is anyone interested in a plugin for Blender that allows you to easily build physically-accurate simulation environments for robots and then apply reinforcement learning to the virtual robots? I have a hodge-podge amount of code for doing exactly this, and I&#x27;m curious if anyone else would be interested in it?

Gwern on the luck of the last mover:<p>&quot;Launching too early means failure, but being conservative &amp; launching later is just as bad because regardless of forecasting, a good idea will draw overly-optimistic researchers or entrepreneurs to it like moths to a flame: all get immolated but the one with the dumb luck to kiss the flame at the perfect instant, who then wins everything, at which point everyone can see that the optimal time is past.&quot;<p>Robotics has been a money pit for startups and corporations for a long time. Think of the billions Toytoa has spent on home robotics research, to little avail.<p>But at some point it won&#x27;t be. Some entity will &quot;kiss the flame&quot; at the right movement. The wealth they create will be beyond any company ever, by an almost incomparable margin.

Many caviates but impressive progress in manipulation, especially sim2real:<p>- Only 20% attempts successful on hardest configs with 26+ moves<p>- Solving steps are not generated by RL (but could be[1])<p>- Cube is modified internally to transmit additional state via bluetooth<p>- Highly calibrated and fine tuned environment+MuJoCo based sim to match simulation to reality as much as possible<p>- Open AI Five algorithm is pretty much reused as-is<p>- Cumulative training time = 13 thousand years, same order of magnitude as the 40 thousand years<p>- 32+64 V100 GPUs per training cycle<p>[1] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1805.07470" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1805.07470</a>

Some of Vernor Vinge&#x27;s books deal with the &#x27;alien&#x27; in alien intelligence in ways that were quite illuminating&#x2F;shocking for me at the time. They weren&#x27;t just humanoids with animal instincts. He created intelligent spiders that were believable. And the only sympathetic treatment of a hive mind I&#x27;ve yet encountered (Card&#x27;s are pale in comparison)<p>But one of my favorite inventions of his was a creature that had somehow evolved wheels. With veins and nerves and such there is hardly a creature on earth that can rotate a limb farther much farther than 200°, and the ones that can, like owls, we treat with a certain reverence.<p>Developing an artificial wrist that can spin arbitrarily would be, I&#x27;d think, a quite compelling compensation for someone having to use a prosthetic arm. It would also make for some wicked Rubix solving skills. I wonder how proprioception would deal with that though...

I appreciate the plushed giraffe perturbation. Reinforcement learning needs to account for all eventualities, including giraffes.

&quot;What people don&#x27;t appreciate, when they picture Terminator-style automatons striding triumphantly across a mountain of human skulls, is how hard it is to keep your footing on something as unstable as a mountain of human skulls.&quot;<p>...I&#x27;m not feeling so confident now.

It&#x27;s great that OpenAI has continued to exist as an technological organization with no clear revenue expectations. At the same time I am not sure how long they can sustain doing what they are doing OR whether there is this new found feasibility for private research organizations to exist in this space provided they produce clear high-quality output like OpenAI is doing.

As a roboticist, it&#x27;s really clear to me that this sort of transfer in controlled environment is hard but doable. I think it&#x27;s already been demonstrated many times and I&#x27;m not that convinced that there is anything new in there except more GPU + fancier robot.<p>I&#x27;ll be impressed by RL is a) they manage to do sim2real in open environments, think Doom -&gt; office building or b) they manage to get data efficient enough that sim2real is still necessary but you don&#x27;t have to do real data collection with 10 parallel robots for days on end.<p>As someone in mobile robotics as opposed to pure manipulation, I read these papers and I&#x27;m like: &quot;How the hell am I supposed to get this to work on a robot moving in the real world???&quot;. I don&#x27;t see anyone being close to this right now.

This is cool. I wonder about the hardware. Why does the mount for the hand have a fan? Does it contain the inference computer? Power transformers?

Does anyone have any experience with soft robotics? For example these guys: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=X6CRe2ieuYE" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=X6CRe2ieuYE</a> advertise their gripper as supposedly being able to handle weight&#x2F;size variety with no training at all, just with the use of different materials in the gripper

A few weeks back I was at a program synthesis conference and gave a short lightning talk where I said deep learning so far has been used to solve the easy computer chess, and the easy computer go, etc...not to take away from those accomplishments at all, I was just saying that having a robot beat grandmasters at real world physics chess where you have to move the pieces with many degrees of freedom is a harder problem, but trivial for a 7 year old.<p>I thought we were still a decade away from having machines beat humans at real chess and real go, but this makes me think maybes it’s just 5 years out. Very impressive.

This is a pretty pathetic result and is damning for the progress of AI. Instead of focusing on efficiency, AI researchers simply throw more resources at the problem with the hope that it&#x27;s enough. The end result after 13000 years of training is a robot hand that can do nothing but solve rubik&#x27;s cubes and fails 40% of the time.

world record one handed is 6&quot;:88 average of 5 is 9&quot;:48 <a href="https:&#x2F;&#x2F;www.worldcubeassociation.org&#x2F;results&#x2F;rankings&#x2F;333oh&#x2F;average" rel="nofollow">https:&#x2F;&#x2F;www.worldcubeassociation.org&#x2F;results&#x2F;rankings&#x2F;333oh&#x2F;...</a><p>non-world-class: doing &lt;30&quot; one handed is very doable, anyone can do less than 1 min (yes, if you know how to solve and you trained one handed. ofc not if you never solved a cube in your life)<p>that said... I really don&#x27;t understand how the hand keeps the cube &quot;floating&quot; around. In one handed the technique is pretty much to keep the cube fixed holding front&#x2F;back centers with thumb and index. Something like <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=mUF3aPDTO-4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=mUF3aPDTO-4</a><p>I understand the achievement, but wow, this solve is HORRIBLE. What did they train the network with to get this?!

Around 1988, I read a book that claimed the ideal robot hand would have fingers that repeatedly bifurcate until it has a digit so small it can manipulate matter at the atomic level. Implausible but fun to think about.

I think artificial muscles that are more biomimetic with better power-to-weight ratios are going to make a huge improvement in robot capabilities at some point. Especially for humanoids.

Any YC companies going after the robotics industry?

Would be cool to see it done with two hands (or one), solved faster than the human world record. It&#x27;s still pretty clumsy looking.