RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning (2023)

This was a nice demonstration of letting these robotic hands learn to play a keyboard. The technique is limited due to constraints of the hands, and is closer to parts of the technique of early keyboard instruments like an organ or a harpsichord, rather than that of a modern grand piano, which requires a lot more control of the body core, shoulders, elbows, arms, and wrists with the fingers doing as minimal a motion as possible. I suppose a similar algorithm learning to play on a grand piano using a full humanoid body could learn a technique that would be exciting to analyze.

Any way we can get the Westworld intro song into the demo list?

This was the basis of a project I did for my deep reinforcement learning class!<p><a href="https:&#x2F;&#x2F;ericye16.com&#x2F;stanford-cs224r" rel="nofollow">https:&#x2F;&#x2F;ericye16.com&#x2F;stanford-cs224r</a><p>We were able to make some improvements by tuning how the reward is distributed and also by first pretraining the agent on scales before fine-tuning them on the final pieces.<p>Thanks to Kevin Zakka for helping us get started with the RL environment!

&gt; UC Berkeley, Google DeepMind, Stanford University<p>I&#x27;ve personally experienced how research around this time was being shut down because of AI doomerism. People were getting laid off because of it. It&#x27;s clear to me that these institutions actively spread AI doomerism so that they have full control over it. They actively called for a stop in AI research so that their personal labs can leap forward ahead. It was a little too on the nose for Big Tech, but they don&#x27;t understand nuance.

Is this actually buildable? I&#x27;d like to hear it on a real piano because (IMHO) the one in the videos sounds bad.<p>I&#x27;d like to also hear how loud the mechanical noise of the machine playing the piano would be. Does the left hand work harder with the heavier keys? What would the hands be mounted to?

Nice work! And great interactive 3d application. My 6yo had a lot of fun annoying the robot while it&#x27;s playing by forcefully moving its hands around.

All the right notes but not necessarily at the right time.

Has anyone tried to get a robot to do oil paintings?

waiting for Robot Devil.

Funny enough I’m listening to Rhapsody in Blue while browsing HN. I’d like to see it do that for 17 minutes.

How does this research help us win naval battles? Should Elon and his DOGE friends cut it?

Imediatelly reminded of the anonymous quote: &quot;I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes.&quot;<p>I understand it&#x27;s a bad argument because 1) people may need assistance to do art&#x2F;writing and 2) the advancements gained from teaching AI to do art can be applied to other non-artistic endeavors (e.g. piano AI could be really good to operate machines with lots of buttons and no computerized interface).<p>However, the cost cutting side of the argument is the one that bothers me because companies&#x2F;people WILL use AI like that in place of actual humans because they&#x27;re likely to be cheaper in the future. So that pianist or musician playing in a local restaurant can be sure their job will be automated away by a subpar AI and real humans will be relegated to very expensive locations (an extension of replacing humans with recorded music, in a way).<p>My pessismist side thinks greed will be the downfall of humanity.

Wasted opportunity to not play the Westworld Theme

the fact that the failure modes come from limitations of the robotic hands is super interesting

Honestly,<p>this is really bad. It might be a breakthrough of what you are doing, but when I listen to the output all of the timing and phrasing is aweful.

I mean, keys are always in the place where you expect them to be, so what is the challenge here? (Yes, too lazy to read the article).

&quot;Player Piano&quot;

The thing needs a teacher, terrible technique! &#x2F;s