GM's Cruise alleged to rely on human operators to achieve "autonomous" driving

Cruise CEO here. Some relevant context follows.<p>Cruise AVs are being remotely assisted (RA) 2-4% of the time on average, in complex urban environments. This is low enough already that there isn’t a huge cost benefit to optimizing much further, especially given how useful it is to have humans review things in certain situations.<p>The stat quoted by nyt is how frequently the AVs initiate an RA session. Of those, many are resolved by the AV itself before the human even looks at things, since we often have the AV initiate proactively and before it is certain it will need help. Many sessions are quick confirmation requests (it is ok to proceed?) that are resolved in seconds. There are some that take longer and involve guiding the AV through tricky situations. Again, in aggregate this is 2-4% of time in driverless mode.<p>In terms of staffing, we are intentionally over staffed given our small fleet size in order to handle localized bursts of RA demand. With a larger fleet we expect to handle bursts with a smaller ratio of RA operators to AVs. Lastly, I believe the staffing numbers quoted by nyt include several other functions involved in operating fleets of AVs beyond remote assistance (people who clean, charge, maintain, etc.) which are also something that improve significantly with scale and over time.

<a href="http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20231105193346&#x2F;https:&#x2F;&#x2F;www.nytimes.com&#x2F;2023&#x2F;11&#x2F;03&#x2F;technology&#x2F;cruise-general-motors-self-driving-cars.html" rel="nofollow noreferrer">http:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20231105193346&#x2F;https:&#x2F;&#x2F;www.nytime...</a>

Cruise is leveraging human-in-the-loop to expand faster than they otherwise would, with the hope that they will solve autonomy later to bring this down.<p>I don&#x27;t think this is a viable strategy though given the enormous costs and challenges involved.<p>There doesn&#x27;t exist a short-term timeline where Cruise makes money, and the window is rapidly closing. They needed to expand to show big revenues, even if they had to throw 1.5 bodies per car at the problem.<p>Prediction: GM will offload cruise, a buyer will replace leadership and layoff 40% of the company. The tech may live to see another day, but given the challenges that GM has generally (strikes, EVs, etc), they can no longer endlessly subsidize Cruise.

&gt; Two months ago, Kyle Vogt, the chief executive of Cruise, choked up as he recounted how a driver had killed a 4-year-old girl in a stroller at a San Francisco intersection. “It barely made the news,” he said, pausing to collect himself. “Sorry. I get emotional.”<p>...<p>&gt; Cruise’s board has hired the law firm Quinn Emanuel to investigate the company’s response to the incident, including its interactions with regulators, law enforcement and the media. &#x2F; The board plans to evaluate the findings and any recommended changes. Exponent, a consulting firm that evaluates complex software systems, is conducting a separate review of the crash, said two people who attended a companywide meeting at Cruise on Monday.<p>After the first [edit: the first performative charade, about little girl in a stroller], why should we trust the second isn&#x27;t also a performative charade? What independence or credibility does some hired law firm have, that the company itself does not? How about using an independent third party?

Having to be remotely operated every 2.5 to 5 miles seem to defeat most of the economics of self driving cars.<p>Back of the napkin math, cars drive at an average of 18mph in cities, so every 10-20min. Let’s assume it takes over for 1min, and that you need remote drivers not too far for ping purposes, so at the same hourly rate. To guarantee you’ll be able to take over all demands immediately, due to the birthday paradox, you end up needing like 30 drivers for 100 vehicles? It’s not that incredible of a tech…

Look, isn&#x27;t remotely assisted driving something unbelievably stupid? Why should I rely, when I am on my &quot;driverless&quot; car, rely on someone else who is remote, need to be updated at all times about the situation (when things can go wrong in a matter of tenth of seconds, while driving), and needs to react, and it&#x27;s not as much motivated as me (as I am risking my life, while he is sitting somewhere without having as much skin in the game as me)? It makes a lot more sense, then, to have just an assisted driving car, or a semi-autonomous car where the &quot;assistant&quot; to the AI it&#x27;s me and not someone else.

<a href="https:&#x2F;&#x2F;archive.ph&#x2F;2023.11.04-050448&#x2F;https:&#x2F;&#x2F;www.nytimes.com&#x2F;2023&#x2F;11&#x2F;03&#x2F;technology&#x2F;cruise-general-motors-self-driving-cars.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;archive.ph&#x2F;2023.11.04-050448&#x2F;https:&#x2F;&#x2F;www.nytimes.com...</a>

If humans need to remotely intervene for a car in motion, that implies it could impact safety.<p>If that&#x27;s correct, then the remote signaling of a problem and the human&#x27;s response and control must have flawless availability and low latency. How does Cruise achieve that?<p>Cellular isn&#x27;t that reliable. Maybe I misunderstand something.

I&#x27;ve said this here before and I will repeat it:<p>An overwhelming majority of Americans will choose 45,000 deaths in car crashes annually (last year&#x27;s number) in human-driven cars over 450 deaths&#x2F;year with all self-driving cars.<p>In the American (and probably ALL) mind(s), human agency trumps all.

&quot;That is a rare level of talent,&quot; said Sam Altman, head of the Y Combinator startup incubator. &quot;I can see Kyle being the next CEO of GM.&quot;<p><a href="https:&#x2F;&#x2F;www.vox.com&#x2F;2016&#x2F;3&#x2F;11&#x2F;11586898&#x2F;meet-kyle-vogt-the-robot-guru-who-just-sold-his-second-billion-dollar" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.vox.com&#x2F;2016&#x2F;3&#x2F;11&#x2F;11586898&#x2F;meet-kyle-vogt-the-ro...</a>

This was a plot point in Captain Laserhawk: All the self-driving cars and flying drones were actually being remotely piloted by prisoners in a massive VR facility.<p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Captain_Laserhawk:_A_Blood_Dragon_Remix" rel="nofollow noreferrer">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Captain_Laserhawk:_A_Blood_Dra...</a>

&gt; <i>Company insiders are putting the blame for what went wrong on a tech industry culture — led by the 38-year-old Mr. Vogt — that put a priority on the speed of the program over safety. [...] He named Louise Zhang, vice president of safety, as the company’s interim chief safety officer [...]</i><p>I hope Chief Safety Officer isn&#x27;t just a sacrificial lamb job, like CISO tends to be.<p>Is the &quot;interim&quot; part hinting at insufficient faith, and maybe future blame will be put on how the VP Safety performed previously (discovered after the non-interim person is hired)?<p>&gt; <i>[...] and said she would report directly to him.</i><p>Is the CSO nominally responsible for safety?<p>Does the CSO have any leverage to push back when their recommendations aren&#x27;t taken, other than resigning?

It sounds like there are a lot of people working at GM who don&#x27;t like Cruise and are willing to complain to the NYT about it. One of those frustrating &quot;we&#x27;re a startup inside a large company&quot; things.<p><i>Cruise employees worry that there is no easy way to fix the company’s problems.</i><p><i>Company insiders are putting the blame for what went wrong on a tech industry culture.</i><p>What, because car companies with car company culture are doing such a great job building self-driving cars?<p>I&#x27;m rooting for both Cruise and Waymo here. Self-driving cars would be great for humanity. Good luck to the teams working hard to make them happen.

Here we go again with a CEO who proclaims &quot;autonomous cars are safer than human-driven cars.&quot; And their definition of &quot;safer&quot; conveniently ignores that autonomous cars <i>create new failure modes</i> which do not exist in manually-driven cars.<p>It may be true that statistically fewer fatalities per mile happen with autonomous cars than with human-driven cars. But that&#x27;s irrelevant. If the car kills one person because it did something utterly stupid like driving under a semi crossing the highway or dragging a pedestrian along the ground, the public will not accept it.<p>This is another example of the uncanny valley problem: Most &quot;smart&quot; devices are merely dumb in new ways. If your &quot;smart&quot; gizmo is only smart in how it collects private information from people (e.g. smart TVs), or it&#x27;s merely smarter than a toggle switch, that&#x27;s not what the public considers smart. It has to be smarter than a reasonably competent human <i>along almost all dimensions</i>; otherwise you&#x27;re just using &quot;smart&quot; as a euphemism for &quot;idiot savant.&quot; Self-driving cars are a particularly difficult &quot;smart&quot; problem because lives are at stake, and the number of edge cases is astronomical.

&gt; Having to be remotely operated every 2.5 to 5 miles<p>Regarding Cruises&#x27; suspension, how likely is it that the backup driver restarted the car to drive again after the car stopped with the pedestrian below?

This is the same whacky theory I&#x27;ve been spreading about Tesla self-driving for a year or so. &quot;Imagine Tesla self-driving is like some dude driving your car via videogame on the other side of the world.&quot;<p>Most people are pretty sure my theory is wrong. I have absolutely no evidence this is true, it&#x27;s just some crazy idea that popped into my head one day.

I get a feeling Cruise is going to get sold off within the next 5 yrs. Waymo will likely be the leading provider for “autonomous vehicle” software&#x2F;hardware.<p>Government Motors can only sustain such a loss on their books for a short time. This is probably why Vogt has been pushing so hard for market dominance.

<a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20231104212102&#x2F;https:&#x2F;&#x2F;www.nytimes.com&#x2F;2023&#x2F;11&#x2F;03&#x2F;technology&#x2F;cruise-general-motors-self-driving-cars.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20231104212102&#x2F;https:&#x2F;&#x2F;www.nytim...</a>

It&#x27;s not an allegation. It&#x27;s the same as using human feedback for tuning large language models. There are no autonomous cars currently regardless of what is written on the marketing brochures. In various &quot;emergency&quot; situations the cars phone home and ask a human operator to take over the controls.

I thought being a social media moderator and being constantly exposed to violence, racism, and child pornography was bad. Having your whole day being a series of &quot;quick, don&#x27;t let these people die!&quot; moments seems like the worst tech job on earth.

Relevant XCKD <a href="https:&#x2F;&#x2F;xkcd.com&#x2F;1897&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;xkcd.com&#x2F;1897&#x2F;</a>

I wonder if there are rails to prevent a bad actor Cruise worker from remote driving erratically…

@dang title not the same as original

The title isn&#x27;t news at all as every single trustworthy autonomous driving solution MUST HAVE human operators somewhere to take over but the actual article is a good summary of Cruise&#x27;s current situation and I&#x27;d guess the competition as well.

Cruise came out of YC if I recall?

Can we just admit that this likely isn’t possible in our lifetime and put more money into early childhood education, better healthcare and geriatric care?