This was a nice demonstration of letting these robotic hands learn to play a keyboard. The technique is limited due to constraints of the hands, and is closer to parts of the technique of early keyboard instruments like an organ or a harpsichord, rather than that of a modern grand piano, which requires a lot more control of the body core, shoulders, elbows, arms, and wrists with the fingers doing as minimal a motion as possible. I suppose a similar algorithm learning to play on a grand piano using a full humanoid body could learn a technique that would be exciting to analyze.
This was the basis of a project I did for my deep reinforcement learning class!<p><a href="https://ericye16.com/stanford-cs224r" rel="nofollow">https://ericye16.com/stanford-cs224r</a><p>We were able to make some improvements by tuning how the reward is distributed and also by first pretraining the agent on scales before fine-tuning them on the final pieces.<p>Thanks to Kevin Zakka for helping us get started with the RL environment!
> UC Berkeley, Google DeepMind, Stanford University<p>I've personally experienced how research around this time was being shut down because of AI doomerism. People were getting laid off because of it. It's clear to me that these institutions actively spread AI doomerism so that they have full control over it. They actively called for a stop in AI research so that their personal labs can leap forward ahead. It was a little too on the nose for Big Tech, but they don't understand nuance.
Is this actually buildable? I'd like to hear it on a real piano because (IMHO) the one in the videos sounds bad.<p>I'd like to also hear how loud the mechanical noise of the machine playing the piano would be. Does the left hand work harder with the heavier keys? What would the hands be mounted to?
Nice work! And great interactive 3d application. My 6yo had a lot of fun annoying the robot while it's playing by forcefully moving its hands around.
Imediatelly reminded of the anonymous quote: "I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do laundry and dishes."<p>I understand it's a bad argument because 1) people may need assistance to do art/writing and 2) the advancements gained from teaching AI to do art can be applied to other non-artistic endeavors (e.g. piano AI could be really good to operate machines with lots of buttons and no computerized interface).<p>However, the cost cutting side of the argument is the one that bothers me because companies/people WILL use AI like that in place of actual humans because they're likely to be cheaper in the future. So that pianist or musician playing in a local restaurant can be sure their job will be automated away by a subpar AI and real humans will be relegated to very expensive locations (an extension of replacing humans with recorded music, in a way).<p>My pessismist side thinks greed will be the downfall of humanity.
Honestly,<p>this is really bad. It might be a breakthrough of what you are doing, but when I listen to the output all of the timing and phrasing is aweful.