But the tests said it worked

Apple reprogrammed my iPod to steal my private files and send them to Apple’s servers. I wasn’t the only victim, but the news barely got mentioned online. Because nobody expects software to work right.

This happened shortly after I read a proposal to shield car makers from liability for bugs in their driving software.

The model was the National Vaccine Injury Compensation Program. If you have a bad reaction to a government-required vaccine, the government will pay for it. About one in a million people do.

The government wants everybody to be vaccinated. The VICP was motivated by a few facts on top of that one:

  • Vaccines are all supposed to be identical, with strict quality control.
  • Vaccines are simple by the standards of biological systems.
  • Vaccines are extensively tested before the FDA approves them.
  • A few people have severe reactions to vaccines regardless of quality control.

With drug makers worried about huge jury verdicts for injuries they couldn’t prevent, Congress stepped in to protect the industry and the public.

The policy considerations that require no-fault vaccine insurance do not apply to self-driving cars. Cars are not all the same and their computers are complicated. Manufacturers are under a lot of pressure not to test them properly.

A long time ago, one of the servers at work got a new login program. Testing was thought to be simple. Type your username and password, and it should log you in. It does. So ship it. Apparently nobody tested what would happen if you typed the wrong password. Turned out it let you in anyway. It was as secure as an unlocked door.

I assume Apple was so busy with positive testing — does iCloud work when enabled? — it didn’t bother with negative testing — does iCloud not work when not enabled? In fact it did work even though you turned it off. By “work” I mean it sends your files to Apple’s servers.

Testing error handling is hard and companies tend to skip it.

I trust car makers to build cars that can follow each other at a fixed distance along a well-marked freeway. That’s easy to build and easy to test. Everybody will score five stars on the lane-following test.

When that parked car on the shoulder starts moving, when a goose starts to walk across the road, when a snow squall hits, when a tire blows, when a sensor malfunctions, that’s when you get to be the first one to test how your car reacts. A certification test, once we have one, will not anticipate everything that can go wrong.

And what happens when you survive and the manufacturer decides to improve its cars?

When a drug company wants to try a new formula it has to spend years and millions of dollars convincing the FDA that it is safe.

When Tesla wants to try a new formula it just pushes a new program to your car. What if it doesn’t work? NHTSA says if your self-driving car crashes it’s your fault for letting it drive.

NHTSA doesn’t want to be the FDA of the highway.

The government is already being very generous letting car makers avoid liability.

We may need a better system than the lawsuit lottery, but we have to make poorly tested cars expensive for the manufacturer and not for the public.

The opinions expressed in this post belong to the author and do not necessarily represent those of the National Motorists Association or the NMA Foundation. This content is for informational purposes and is not intended as legal advice. No representations are made regarding the accuracy of this post or the included links.

Not an NMA Member yet?

Join today and get these great benefits!

Comments are closed.