Self-driving vehicles against human drivers: Equal safety is far from enough

If it&#x27;s a subjective matter tied to perception of risk rather than actual, statistical risk, such perception can be swayed.<p>The challenge remains that people will be killed in accidents involving autonomous control. And we anticipate that the number of people killed will be fewer, hence &#x27;saving lives&#x27;. However, the lives lost in autonomous accidents will be a different set of people than those that would have died in human driven accidents. There will be cases where a court determines that the autonomous system was the cause. Families of those killed will want justice, while those separately saved by autonomous systems may never be heard from in the same case.<p>I expect that in the end it will come down to a business decision, and that decision will be informed by an actuarial exercise: Will profits and insurance be able to cover the costs of defending and settling such cases. Who knows, maybe the threshold is crossed at 5x safer.

Would this baseline include all the accidents from distracted drivers, drunk drivers, drug drivers etc.? Or is it referring to an average human driver who isn&#x27;t intentionally breaking the law?<p>If the baseline includes all these sorts of human error, I see no issue with holding robots to a higher standard. Imagine if we rolled out robot policemen who only executed black people for no reason at the same rate as humans do.

A lot of comments are focusing on safety via driving better. But with self-driving vehicles, can&#x27;t we make the layout of the car safer, and thus accidents cause less harm to the people inside?<p>For example, right now because we need to see the road, I assume there is significantly more danger from the windshield vs. a padded back on both sides of the car with passengers facing each other like in a train car.<p>It seems likely that we can make self-driving vehicles much safer, even with the same number of collisions, by just changing the layout.

This seems pretty reasonable and also very possible to achieve. It would be insane to allow a technology on the streets that makes as many mistakes as humans are making. I certainly wouldn’t use self driving cars if they killed 30000 people per year like humans are doing right now. How would you assign responsibility for crashes? Our current system is far from perfect but at least it’s something people understand and know how to navigate. And there are drivers that are better and more cautious than others. So it’s not just an illusion of control.

What so many autonomous car advocates seem to miss is that it is nearly impossible to meaningfully compare relative safety with current self driving cars, because we don&#x27;t have level 5 autonomy yet.<p>In order to compare them with current technology, you&#x27;d have to be able to answer the question: how safe would human drivers actually be if they didn&#x27;t have to perform their most difficult tasks? Because that is what current autonomy does.<p>I&#x27;m willing to believe that current tech is capable of being safer than human drivers, simply because they do so many things way better than humans do, like stopping for pedestrians and safely navigating around cyclists. But to compare them <i>in general</i>, that is left to be proven. You can&#x27;t just compare incidents per mile driven, because autonomous vehicles can conveniently opt out of driving whenever the task gets too hard.

5x safer isn&#x27;t an unreasonable threshold to ask for. All life-critical, engineered systems incorporate a safety margin.<p>To win buy-in from the population at large, you sure as heck better have overwhelming statistics. Marginal just won&#x27;t cut it.<p>When you&#x27;re talking about taking control away from the driver, all you need is one accident a human would have prevented to create political backlash and deteriorate trust in the system. Just because several other unvoiced lives may have been saved doesn&#x27;t offer consolation to the victims or deem the loss acceptable. I think of it as analogous to the legal doctrine that convicting one innocent is worse than letting 10 guilty individuals go free.<p>I&#x27;m always skeptical when developers claim a computer can do a better job than a human, as I&#x27;ve encountered so many edge cases the programming just never adequately accounted for. It will take time and a great deal of experience running these platforms in the wild until they become truly resilient. I would be quite pissed off to find the severed limb I suffered in a crash was due to programming not sufficiently distinguishing e.g. road paint from water streaks.<p>And the metric you choose to define &quot;safer&quot; will never be perfect. Extra buffer helps offset any bias or gaps in your methodology and capture more of the long tail of &quot;one-offs&quot; never accounted for in version 1.<p>I&#x27;m a big believer that driverless vehicles will provide huge improvements to our quality of life, I just feel there&#x27;s a lot of misrepresentation taking place out there today as to how far along these systems are.

This should be calibrated to the risk the top X% of cautious&#x2F;safe drivers, and exclude reckless, inexperienced, or intoxicated drivers. As a safe driver, you shouldn&#x27;t have to accept risk calibrated to &quot;average&quot; (i.e. drunk, reckless) driver.

Engendering trust and reducing materially regressive liability&#x2F;litigiousness is a good call - and something that SHOULD be set as a standard by an external body.<p>IMHO this is typically a good role for government regulation - setting a standard measurement of outcome for the public good, but not dictacting HOW that should be achieved.<p>Now, we&#x27;re just haggling over the price... <i></i><p>(<i></i> as not-churchill infamously didn&#x27;t say...)

I have an older friend whose driving terrifies me, but who lives in an area with effectively zero public transportation or reliable cab service. While I don&#x27;t want to see this person on the roads, the alternative is literally moving into a senior community (which would probably be the death of this person).<p>Frankly, if self-driving cars became .75x as safe as the average human driver, it would still a net safety improvement if got this person out from behind the wheel.

So to speed up the adoption of self-driving cars, we could simply make human-driven cars more prone to accidents :p

Here&#x27;s how I think autonomous cars could be introduced, once they reach human-level safety:<p>As a penalty for bad driving.<p>I believe people would be much more accepting of machines taking the wheel if it meant that at least &quot;this guy there&quot; isn&#x27;t driving.

This title is awful (but it was copied from the site). What it should say is &quot;Study finds that most people surveyed didn&#x27;t trust self driving cars until they were five times safer&quot;.

That might be their current stated preference, but I don&#x27;t think it will be most people&#x27;s actual choices. Imagine if self driving was available on every car right now with the press of a button and it was as safe or twice as safe as a normal driver. How many people would press that button, start texting, and just continuing to progress to paying less and less attention ? People already don&#x27;t pay the attention they should when driving or when using a driver assistant system.

Based on the abstract, it looks like this is an attempt to measure how safe self-driving cars need to be in order for people to prefer using them. It is not any sort of requirement from the NIH.

What&#x27;s so magical about 5? Why not 4x or 6x. 2x safer will be 500,000 lives saved yearly. We can see that even 1.25x safer is very significant. Just weird seeing that magic number 5x...

I rented a 2019 Mercedes last week and drove it for over 1200 miles, most of which was driven with the cars driving assist technologies enabled.<p>My guess is that because this car drives so &quot;carefully&quot;, such as automatically following at a safe distance (leaving maybe a 3 second gap between the car in front of it), human drivers will end up causing many more accidents. There must have been more than 50 drivers (with many annoyed stares into my window as they passed) that made unnecessary lane changes to go around me just to then closely follow the car in front of me.<p>This large gap may make it seem like the car is going slower than it is, as so many drivers tried to overtake me but failed as slower traffic in the other lanes blocked them.<p>Human drivers may just become worse over time as more law-abiding autonomous vehicles hit the road. &quot;5x&quot; might not be as much of an improvement in the future.

Problem is your the average for human drivers is brought down mostly by bad drivers. However, even the if we talking about averages 50% of people are better than average. I would not feel to confident in a self driving car that is only approximately as good as 1 out of 2 drivers. I would want the car to be better than high upper percentile. Especially knowing an automated system can have reaction times that can put any humans reaction times to shame.<p>So yes equal safety is far from enough. Especially considering 50% would be better drivers than a self driving car that was only as good as the average driver. Your asking 50% of people and some percentage of people who overestimate their abilities to trust a car that would perform worse than them.

Misleading title. The article is not saying we <i>need</i> more than equal safety; it&#x27;s saying that self-driven vehicles are <i>perceived</i> as more dangerous and people in the study wanted them to be 4-5x safer than human-driven vehicles to overcome this.

Human beings need a target for vengeance and hatred. If someone kills your child, you can hate them. If an automated car kills your child, you don&#x27;t have anyone to hate. Saying that this is about safety thresholds is a distraction from the true human problem exposed by automated cars:<p>&quot;Which individual will held accountable and risk jailtime if their car kills someone you love, and how can this individual be identified from the appropriate government registries within 24 hours of a death?&quot;<p>Until this is clearly defined in law, automated driving will continue to be resisted under any number of plausible justifications, and arguing with those justifications will have little effect.

What most people seem to misunderstand about &quot;driving&quot; is that it is not a sensory or stimulus response problem. It is a cognition problem.<p>Computers or computer based &quot;AI&quot; are good at solving bounded problems that do not require open ended or on the fly model building and judgement exercising in real time. At this biological intelligence, honed by millions of years of selective evolution still excels.<p>Computers can &quot;solve&quot; GO or Chess because fundamentally the rules are simple and the models required to play these games are subject to only a few, static constraints.<p>Driving, in all conditions on all roads, on the other hand, requires a flexible model of the real world that approaches that built by sentient biological intelligences.<p>The problem is not sensor or perception latency.<p>Sure LIDAR can blow human perception out of the water.<p>But that does not matter.<p>What matters is making the correct decision based on sensory inputs using a high fidelity model of the real world.<p>Computer does not understand the difference between two people playing catch by the roadside, parallel to the road and a situation where a child might be chasing after a soccer ball. This is not just combinatorics and probability...it is theory of mind. Thus until AGI is invented FSD will be a misnomer.<p>Doesn&#x27;t matter how much faster a computer can perceive if it does not know how to integrate the raw data it receives into a model of the world that yields correct decisions to the circumstances.

Costs aside, what about something like mag-lev tracks for cars, that can start&#x2F;go on a dime, and on freeways go faster, even switch lanes to get around slower traffic. Maybe even do away w&#x2F; speed limits just go as fast as you &#x27;feel safe&#x27; going with the only limit being the max. In cities you&#x27;d have sensors&#x2F;grids everywhere to detect non-car traffic, and regular cars could even drive over the mag-lev, or it could be a separate track, and you can go in&#x2F;out of mag-lev&#x2F;drive modes. Maybe it parks you, til you&#x27;re ready to take over control (say you&#x27;re napping on the commute). Alarm goes up, you wake up. Stretch, even get out and stand up for a minute, get back in. Buckle up - drive the final block to where you want to park at your job, or if it&#x27;s a country side location, up in the mountains, etc you might drive for longer then park where ever.<p>Essentially you could just cover cities and highways to nearest gas stations. Car&#x27;s running out of gas&#x2F;electricity it routes itself to nearest depot.<p>Going cross country and want to stop at lunch? Program the car, and it&#x27;ll pull to nearest gas station in tim-buk-two and let you figure out where to go from there.<p>Point: A&#x2F;I self-driving aren&#x27;t the only way to get autonomous cars. 50&#x2F;50 re-thinking infrastructure, sensors, car-to-car communications could get us a lot closer faster.

I want to know who is responsible when the AI makes the wrong decision and someone gets hurt. I want that to be fleshed out first.<p>Beyond that, if SDCs are even <i>just</i> as safe as HDCs, I&#x27;m good.

This is 100% just an artifact of a system going from human control to non-human control. Nobody bats an eye at systems which were never human controlled - or transitioned so long ago that nobody recalls human control.<p>I&#x27;ve never seen anyone hesitate when getting on a fully automated train system at an airport or an elevator. Even more-so with amusement park rides that literally put people in extreme situations.

How far do we think human drivers&#x27; safety level ranges? Like most drivers I&#x27;m falsely convinced I&#x27;m a safer driver than most, but still I expect quite a large range (say a factor 10 between 10th and 90th percentile?). It seems reasonable for self-driving cars to be expected to improve safety over human driving for the large majority of drivers, not just half of them.

An oft overlooked factor in acceptance of AVs is that the evolution of technology from driver-assist to autonomy will alter perceptions:<p>First, human drivers using driver assist will become safer &quot;drivers&quot; even though the added safety is properly the result of the technology that is evolving toward AVs. For example, it should become very difficult for a human driver to hit a pedestrian or cyclist. Not impossible. Just exceedingly unlikely to be the fault of the driver.<p>Secondly, driver assist will habituate drivers and other road users to the performance characteristics of AV technology. The upshot is that AV technology will not be benchmarked against the way drivers and other road users behave and perform today. In some ways the expectations for incrementally better safety will be higher. In other ways, the &quot;flavor&quot; or road risks will change in a way that converges on how AVs perform.

I think the insurance companies will have a different and much more financially based standard.<p>More importantly, I doubt NIH will trump that conglomerate and its influence on NHTSA<p>Also, you could argue that restricting a technology that will result in 20% less deaths on the road is the opposite of the public health.<p>To underline that, that is potentially 10,000 people in terms of death or major disfigurement. PER YEAR.<p>And self-driving could be, in a targeted&#x2F;situational manner FAR safer if it took drunk&#x2F;drugged&#x2F;tired drivers out of the equation, which are responsible for around 33% of deaths.<p>If someone is drunk, a technology 2x an alert driver will be 10x a drunk driver.

One issue that is often overlooked is that humans are pretty robust to unforeseen situations vs AI. Take for example the recent fires in California and the smokey skies. Many of the cell phone pictures did a horrible job of capturing the photos, because their AI had been trained that the daytime sky is blue.<p>And with such a failure, all the cars with similar software would be affected at once.

Is 5x safer a realistic goal? There are limits to how safe a car can be on a road full of human drivers, no matter what sensor suite it has and how fast its reactions are. A vehicle can only respond so quickly to control inputs. Making a computer that&#x27;s five times as safe as a human might be a thousand times more difficult than making one twice as safe.

The big problem with numbers like this is how do you measure it?<p>Tesla claims their system is vastly safer than human drivers, but currently it only engages in situations where it&#x27;s already fairly safe to use. So should that system be 5 times safer than all human-driving, or safer than human-driving under the conditions the Tesla is able to engage?

Why, though? Even in the case that it&#x27;s the same safety self driving cars would yield huge productivity gains as people can work or sleep while commuting and truckers can have 1 or 2 self driving trucks following them cross country. And transportation for the elderly or disabled who cannot drive themselves.

Maybe people want self-driving cars to be 5 times safer because they don&#x27;t trust the people saying the cars are X times safer to begin with.<p>Like you are testing how much they trust machines, and how much they trust the people telling them the machine is X% better.<p>Given 2020, think a little skepticism on both&#x2F;either is reasonable.

Doesn&#x27;t this 5x requirement hurt more people than (say) 1.00001x? What am I missing here...

I find it weird that nobody seems to talk about the national security implications of self-driving cars. Imagine the Russian cyber attack we just experienced happening on millions of self-driving cars...

I wonder what range in safety we tolerate in human drivers? How much worse than the average is a newly-licenced 17 year-old (or whatever age) or an 80 year-old?

There will always be a long tail where the machines fail in scenarios a human can handle. We&#x27;re just going to write off those deaths as an act of nature?

Algorithm aversion is real and shows we prefer humans even in the face of statistical evidence that humans are sub-optimal decision makers. [1]<p>I suspect it’s because we inherently dislike the idea of handing control over to a complex black box. Barring sociopaths, we can reasonably assume to interpret how a person thinks. This isn’t necessarily the case for algorithms, which leads to trust issues.<p>[1] <a href="https:&#x2F;&#x2F;repository.upenn.edu&#x2F;cgi&#x2F;viewcontent.cgi?article=1392&amp;context=fnce_papers" rel="nofollow">https:&#x2F;&#x2F;repository.upenn.edu&#x2F;cgi&#x2F;viewcontent.cgi?article=139...</a>

this is quite an academic exercise since a decade of intensive research hasn&#x27;t brought us close to working self-driving cars, much less 1x safe self-driving cars, much less 5x safe, nor is there any clear path to resolving this open research problem.

Economically, people will not buy a car that sacrifices them in favour of others.

How many people have the current SOTA systems killed?

I think this related to the &quot;illusion of control&quot;. People feel safer when they are driving, rather than a machine, even when they are not safer. I hope government regulators do not impose 5x safety requirements on self-driving cars.

&quot;We arbitrarily chose a number so we could feel like we were making improvements. Nothing justifies 5 over, say, 3, or 10. When cars are in fact 3x safer, all those saved lives won&#x27;t be saved, because our arbitrary 5 has yet to be reached.&quot;<p>This is meaningless and bad.