IIHS claims Vemos are safer than human drivers if you squint too hard while reading data

Waymo robotaxis are either extremely dangerous or a “public health breakthrough,” depending on who you ask. The Insurance Institute for Highway Safety has come out in favor of Waymo and announced that it Driverless cars crash less than people. However, the IIHS also freely admits that some statistical gymnastics have been performed to reach that conclusion, because it is hard to get data to make direct comparisons.

It’s easy to find data about crashes involving human drivers, but it’s much more difficult to find comparable data for autonomous vehicles. While California has required autonomous crash reporting since 2014, and NHTSA has required the same reporting since 2021, the standards are different for humans.

An important statistic in determining accident rates is the total vehicle miles traveled. Transport Department report on traffic volume trends List of annual vehicle miles traveled since 2000. Such mileage reporting is not required for autonomous vehicles, making it impossible to calculate the rate of accidents per mile traveled across the industry. Waymo voluntarily shares this data publiclyBut other companies don’t do this. That’s why the IIHS study focuses on Waymo rather than Tesla, Zoox or any other robotaxi company.

The biggest problem with this claim is the size of the different data sets. The IIHS had to do a lot of work to “clean up” the data on autonomous vehicles from 2021 to 2024 before any comparisons were possible. This meant eliminating duplicate records, as well as “situations in which automation was not involved, the vehicle was not on a public road, or no actual accident occurred.” It then applied the same standards for reporting accidents to the police as human drivers. “For example, if a person complains of pain but leaves the scene, the researchers coded that crash as unlikely to be reported to police. If an airbag deployed, they coded that crash as police-reportable because of the high impact speed and cost of airbag replacement – ​​regardless of other property damage or injury.

In the end, less than one-quarter of the autonomous crash data was usable.

Using that common-sense standard, they estimated that only 22% of the 736 public-road crashes that involved automation were potentially police-reportable – 89 crashes were related to Waymo vehicles, 50 were related to Cruise vehicles, 10 were related to Zoox vehicles and 10 were related to other companies. Because mileage numbers were only available to Waymo, they used its crashes to calculate self-driving vehicle accident rates. Of Waymo’s 89 police-reportable accidents, 64 were in driverless operation, and the others involved supervised use of automation.

After crunching all these numbers, IIHS concludes this:

Overall, Waymo’s crash involvement rate was 68% lower than that of human drivers – 76% lower in Phoenix, 35% lower in San Francisco, 71% lower in Los Angeles, and 4% higher in Austin, where the sample size was relatively small. Waymo vehicles were involved in 85% fewer single-vehicle crashes and 81% fewer injury crashes per (vehicle mile traveled) than human drivers.

However, how reliable are these statistics?

according to NHTSAThere were 6,103,213 crashes across the US in 2021, and 5,930,496 crashes in 2022. Apart from this, 6,138,474 accidents occurred in 2023, and 6,180,241 accidents occurred in 2024. NHTSA reportAnd that’s a total of 24,352,424 reported accidents involving human drivers, compared to only 89 for autonomous vehicles. The same disparity exists with the number of miles traveled. IIHS says, “Waymo vehicles traveled approximately 50 million miles in driverless operation over the study period, compared with approximately 222 billion miles traveled by human drivers over the same period and in the same locations.”

It is difficult to call this comparison statistically relevant. The autonomous vehicle sample size is orders of magnitude smaller than the number of accidents with human drivers. Also keep in mind that Waymo operates exclusively in big cities, where speeds are lower, and they don’t drive on highways, making this number even lower. I’m not saying the IIHS is lying, and I don’t believe that’s their intention. But I question how they can make a definitive statement that Waymos are safer than human drivers, when their data on autonomous vehicles was so limited, and needed so much cleaning, that only 22% of it was left. even IIHS The study concluded that:

This approach of coding narratives is not sustainable as the number of deployments and incidents continues to grow. Practical Application: This study identifies ways in which national crash and VMT data collection for L4 vehicles can be improved to allow for more timely and accurate safety assessments.