Hi HN! Proud to share our open-source robot platform, Dobb·E, a home robot system that needs just 5 minutes of human teaching to learn new tasks. We've already taken Dobb·E to 10 different homes in New York, taught it 100+ tasks, and we are just getting started! I would love to hear your thoughts about this.<p>Here are some more details, below (or see a Twitter thread with attached media: <a href="https://twitter.com/i/status/1729515379892826211" rel="nofollow noreferrer">https://twitter.com/i/status/1729515379892826211</a> or <a href="https://nitter.net/i/status/1729515379892826211" rel="nofollow noreferrer">https://nitter.net/i/status/1729515379892826211</a>):<p>We engineered Dobb·E to maximize efficiency, safety, and user comfort. As a system, it is composed of four parts: a data collection tool, a home dataset, a pretrained vision model, and a policy fine-tuning recipe.<p>We teach our robots with imitation learning, and for data collection, we created the “Stick”, a tool made out of $25 of hardware and an iPhone.<p>Then, using the Stick, we collected a 13 hour dataset in 22 New York homes, called Homes of New York (HoNY). HoNY has 1.5M frames collected over 216 different "environments" which is an order of magnitude larger compared to similar open source datasets.<p>Then we trained a foundational vision model that we can fine-tune fast (15 minutes!) on a new task with only 5 minutes (human time)/ 90 seconds (demo time) of data. So from start to finish, it takes about 20 minutes to teach the robot a new task.<p>Over a month, we visited 10 homes, tried 109 tasks, and got 81% success rate in simple household tasks. We also found a line of challenges, from mirrors to heavy objects, that we must overcome if we are to get a general purpose home robot.<p>We open-sourced our entire system because our primary goal is to get more robotics and AI researchers, engineers, and enthusiasts to go beyond constrained lab environments and start getting into homes!<p>So here is how you can get started:<p>1. Code and STL files: <a href="https://github.com/notmahi/dobb-e/">https://github.com/notmahi/dobb-e/</a><p>2. Technical documentation: <a href="https://docs.dobb-e.com/" rel="nofollow noreferrer">https://docs.dobb-e.com/</a><p>3. Paper: <a href="https://arxiv.org/abs/2311.16098" rel="nofollow noreferrer">https://arxiv.org/abs/2311.16098</a><p>4. More videos and the dataset: <a href="https://dobb-e.com" rel="nofollow noreferrer">https://dobb-e.com</a><p>5. Robot we used: <a href="https://hello-robot.com" rel="nofollow noreferrer">https://hello-robot.com</a>