I am fond of saying there are only two hard problems in robotics: Perception and Funding. If you have a magical sensor that answers questions about the world, and have a magic box full of near-limitless money, you can easily build any robotic system you want. If perception is "processing data from sensors and users so we can make decisions about it", then there isn't much robotics left.<p>Got a controls problem? forward predict using the magic sensor.<p>Got a planning problem? just sense the world as a few matrices and plug it into an ILP or MDP.<p>What did the user mean? Ask the box.<p>etc etc. Distilling the world into the kind of input our computers require is immesnely difficult, but once that's done "My" problem (being a planning expert) is super easy. I'm often left holding the bag when things go wrong because "my" part is built last (the planning stack), and has the most visible "breaks" (the plan is bad). But it's 90% of the time traceable up to the perception, or a violated assumption about the world.<p>TFA is spot on - it's just not clear how to sense the world to make "programming" robotics a thing. In the way you'd "program" your computer to make lines appear on a screen or packets fly across the internet, we'd love to "program" a robot to pick up an object and put it away, but even a specious attempt to define generally what "object" and "put away" mean is still 100s of PhD theses away.So it's like we invent the entire ecosystem from scratch each time we build a new robot.
Loved the TFA.<p>I've been working on robotics pretty much my whole career and people usually miss how complicated it can get even for simple things once you consider what can go wrong AND it's a meeting place for a multitude of areas: hardware, software, mechanical, electrical, machine learning, computer vision, control, driver, database, etc. An issue can hide in between any of those for months before it shows up with bells and whistles.<p>What is sometimes difficult to get across to people is that building robots is not only difficult per se, but the base of comparison is unusually unfair: if you build an e-commerce website you benchmark it against other e-commerce websites, maybe Amazon, maybe ebay; for robots usually the benchmark is against people, the most adaptable and fault tolerant machine that exists, every robot will suck compared to a human doing the same task, but that's what we compare it to every time.
> once they are programming a robot, I feel they become roboticists<p>Yes! I am not a roboticist (or at least a good one in any sense) but I was having a similar discussion regarding enabling non-technical users do data analysis. Once they start doing anything more complicated than `SELECT COUNT(*) FROM blah WHERE foo=otherblah` it's going to get real real quick. You can't just give them some cheap point and click stuff because their questions will immediately overrun the extent of what's practicable. Asking interesting questions of data is roughly as difficult as phrasing the questions in SQL (or any other formal query language) and anyone who can do the first can do the latter easily enough.<p>(or the point and click stuff _is_ really powerful but it's some proprietary non-googleable voodoo that requires a month long training course that costs $5K/week to get a certificate and become middlingly powerful)
This is just a phase. The Internet went through this. It was criticized in the early days as requiring "too many PhDs per packet". Eventually, with standardization and automation, we got past that. Now anybody can connect.<p>Rethink Robotics went bust because they couldn't solve this usability problem.
It's a problem at a much higher level than the author is talking about. If you're driving your robot with positional data, that's easy to understand, but a huge pain to set up.
Usually, you have very rigid tooling and feeders, so that everything is where it is supposed to be. If it's not, you shut down and call for a human.<p>What you'd often like to do is an assembly task like this:<p>- Reach into bin and pull out a part.<p>- Manipulate part until part is in standard orientation.<p>- Place part against assembly so that holes align.<p>- Put in first bolt, leave loose.<p>- Put in other bolts, leave loose.<p>- Tighten all bolts to specified torque.<p>Each of those is a hard but possible robotic task at present. Doing all of those together is even harder. Designing a system where the end user can specify a task at that level of abstraction does not seem to have been done yet.<p>Somebody will probably crack that problem in the next five years.
The goal of computing is, and has always been, controlling the behavior of machines the same way or easier than we do with other agents in the world toward some measurable end<p>So, to what level of granularity do you have to specify a system task in order for it to do the thing you want it to do, at the level of accuracy that you wanted to operate in?<p>That all depends on how accurate you can specify what you want to do<p>which means you have a sense of all of the systems that interact with, and impede the successful task of the set of systems<p>We can build abstraction layers we can build filters, but at some point somebody has to map a set of actions with a set of inputs and outputs, in order to sequentially build this set of tasks, which rolls out into the function of a physical manifestation of some sort<p>Add to that the complexities of mobile actuation complex environments and just the general state of power, computing, routing, etc. and you have a 15 body problem simply to have anything that someone would look at as benefit to humanity<p>Only a couple of disciplines can totally encapsulate all that and none of them are available to study anymore primarily cybernetics, and all of the interactions necessary to fully build a human machine symbiotic system
I was super excited to take a robotics class in college. I'd fallen in love with programming and was excited to take all that magic into the real world.<p>We all had to buy roombas to program. The final exam was getting it to traverse a maze. It seemed so simple! They even gave us the exact dimensions and layout ahead of time. Just hard-code the path, right? Spin the wheels so many rotations, turn 90 degrees, spin some more.<p>Except the real world is messy, and tiny errors add up quickly. One of the wheels hits a bump, or slips a little on the tile, and suddenly you're way off course. Without some kind of feedback loop to self-correct, everything falls apart.<p>My excitement for robotics died quickly. I much prefer the perfectly constrained environment of a CPU.
> Design your APIs for someone as smart as you, but less tolerant of stupid bulls*t.<p>This is definitely applicable outside of robotics. For example, I work on a large-scale LLM training framework and tend to think this way when thinking about design decisions.
> That would be great, right? We should make a software framework so that non-roboticists can program robots.<p>Lol, I work in the field of test automation and this is exactly how no/low code frameworks get pushed as well. And, it rarely does play out in a way that people think it will.<p>In fact, having read the entire article, I feel like a lot of it can be applied more broadly. Basically any time people go "X sure is complex, we should make a simple to use framework for non-X folks to use". Not that it will always fail, but I have seen it happen enough to recognize a pattern.
My personal view, as an industrial control systems engineer, is that so much of the world's production software requires teams of software professionals to monitor it and keep it working. When these same software professionals look at systems which physically interact with the real world on a real time basis then a different dynamic comes into play.
As someone who runs a medialab at an art school it is fascinating how many people believe because they understood the general principle of a thing, it is therefore simple to just do it.<p>Many people seem to long for a magical technology that you could just pour over things and they will work out in the ways you wanted, while miraculously sensing the ways you didn't.<p>Those with the edge on the new tech will always be those who have a good understanding of it's limitations, because once a new thing comes around they immediately see the possibilities.
as someone that messes with every low cost robotics <i>thing</i>. this part stuck out as painfully true :<p>“Oh yeah, if you try to move the robot without calling enable() it segfaults. That's a safety feature… I guess? But also if you call it twice, that also segfaults. Just call it exactly once, ever.”
I work as a non-roboticist at a robot company. Most of my job is to enable people with PhDs to do work and to clean up after people who hastily stood up some sort of infrastructure (either actual infra, tooling, or libraries) so that they could go work on the thing they were actually interested in. Occasionally I'll get to work on vaguely actual robot things - drivers, communications frameworks, timing, etc.
I think the real takeaway from this article, which is applicable to pretty widely applicable, is that a lot of times when you get the requirement "make it simple" the actual requirement is "make it intuitive." Taking away functionality doesn't typically make things any more intuitive, indeed they often make things much less intuitive because a lot of things need to be coupled that a user would not naturally expect to be. Conversely giving the user tons of options but making sure they are distinct, clearly named, their interfaces are consistent, and the defaults are sensible allows someone who understands the problem they want to use the api to solve to jump right in.
This quote hits close to home. “So I got the arm to match the conveyor speed by monitoring the position and pre-empting the motion command, alternating ‘slow’ and ‘fast’ at 10Hz with a duty cycle depending on how we are tracking our target. The motion is pretty jerky but it works.”<p>I’ve been through exactly this scenario in two very popular robotics frameworks.<p>Countless hours were spent by well intentioned framework developers to abstract away the underlying PID loop from the end users. Then countless more hours were spent by the end users to work around this by implementing a PID loop on top of the abstraction.
I used to work with Benjie at X and he was one of my favorite people. Benjie if you see this it is great to see you doing this kind of writing and I love this article!
So if LLMs are so great, I would think binding black box robotics hardware with apis would lead to a revolution in robotics.<p>That sort of one step implementation seems to be a sweet spot for llm<p>Problem is a lack of available examples for training?
s/roboticist/programmer/g and you get an infinite bullshit generator for the world of business and enterprise software, without even firing up ChatGPT!<p>Having worked in both industries, I concur that robotics is much, much messier, as the system has to engage via hardware with the super-messy physical world as opposed to the comparatively modestly messy world of business transactions, data analysis, or whatever. But if we stop trying to solve for "programming for nonprogrammers" and assume that anyone who uses a language or API <i>is</i> a programmer (because once you start programming, that's what you become, irrespective of what's in your job title), we can remove a whole lot of wasted effort from the industry.
> Design your APIs for someone as smart as you, but less tolerant of stupid bullshit.<p>I feel like this has been the problem plaquing the ROS navigation stack since move_base and now nav2. They design the API for people a few standard deviations smarter than everyone else on the planet. Billions of parameters that affect each other in unpredictable ways and you're supposed to read the thesis on each one.<p>Or do what most everyone else does and use the defaults and hope for the best, lmao. You either make an API that the average user will understand or it'll inevitably be used as a black box.
Robotics has been completely transformed in the last six months, check out this Figure video from yesterday<p>Robotics is dead. Long live robotics.<p><a href="https://x.com/Figure_robot/status/1767913661253984474?s=20" rel="nofollow">https://x.com/Figure_robot/status/1767913661253984474?s=20</a>