Money, politics and cars collide in the blink of an eye.

Should a car drive off a cliff to avoid two jaywalking pedestrians or should it strike the hapless jaywalkers to save its single occupant? The moral questions behind scenarios such as this are important when designing the crash code that drives autonomous cars. This code will have the ability to calculate the risks, costs and consequences of an accident in milliseconds. A key assumption in the debate around these questions is that this code will be transparent, with the public having full access to it and engaged in its design. But will we?

Much like in the healthcare industry, it isn’t the doctors or patients that ultimately decide treatment programs; it’s the insurance companies. And with autonomous cars, it’s likely the insurance companies will be controlling the crash code (either directly or indirectly). Accountancy firm KPMG have predicted the auto insurance industry could decline from $US140bn to around $US50bn over the next couple of decades. With so much at stake, insurance companies will be increasingly engaged in the design and control of the crash code. So what will that look like and what role will users and policy makers have? Let’s start at the point of impact.

Not all accidents cost the same.

Scenario One: An entry level hatchback is driving through a wealthy suburb. Its occupants are two house cleaners from a poorer neighbourhood ride sharing an autonomous car on the way to work. A mother and child dart across the road in front of the car, the car can save the occupants or the pedestrians, not both. It calculates the cost of hitting the wealthy pedestrians to be higher than the cost of saving the occupants, and so off the cliff our cleaners go. Nasty, right? It gets nastier.

Scenario Two: A Ferrari being driven through a poor suburb, let’s say pulling off the motorway to buy gas. Its occupants are a wealthy couple on the way to their holiday house. Same scenario, two pedestrians dart in front. What does the car do? In this case the car calculates the pedestrians are likely poorer than its occupants (and its own self worth). It calculates the lowest cost to the insurance company is to strike the pedestrians and save its occupants.

In essence it’s the same, scenario but with two very different outcomes driven by balance sheet motives rather than morals. They’re extreme examples, but the insurance companies would have to program for scenarios such as these. It’s clearly immoral. So you say we monitor the insurance company software and call out immoral code? There’s a problem with that.

Independent researchers don’t have legal access to the algorithms.

Under the Digital Millennium Copyright Act, there is no legal means by which independent researchers have access to the code. It’s a piece of legislation written in 1998 to protect the ownership rights of movie makers. Basically preventing internet pirates from ripping off DVDs. Some believe it to be a masterful piece of legislation that spawned the internet into the success it is today. Others see it as an antiquated bill that now doesn’t even perform the original task to which it was intended.

“It’s like traffic rules for carriages with horses, and you have autonomous cars now.” Why Taylor Swift Is Asking Congress To Update Copyright Laws, by NPR

This was the problem with the case against VW and its ‘clever’ emissions software. VW was only caught out by researchers testing the actual emissions from the cars. This then allowed law enforcement to open an investigation and discover the sketchy code. With so many variables in a car accident, it would be very hard to prove that something untoward was going on. And without a smoking tailpipe, as in the VW case, it would be almost impossible to open a formal investigation.

But there is hope, an exemption to the legislation allows access to software by legislative researchers. That is, government. It could fall on law makers to legislate the algorithms. But there are a couple of problems with that.

The insurance industry is the second biggest lobby group.

Spending over $150m in 2015 lobbying politicians, second only to the healthcare industry. With so much at stake and very deep pockets, the insurance industry will lobby hard. The way congress is looking these days, it seems improbable they’ll take much of stand against the lobby groups. And even if they did…

Code, code and more code.

Getting access to the code and knowing what it does are two different things. Investigators in the VW case struggled to understand the code and who had written what. The insurance companies could well create code that is so enormous that it would either be too expensive or simply impossible to understand.

Consumers get to choose though, and they’ll choose transparent insurance, right?

Yes and no. When all cars on the road are connected and autonomous, your life is not only in the hands of the software in your car, but in all of the cars on the road. Hands up who believes man-made climate change is a thing. Now hands up who’s still driving a gas powered automobile. Mmmm, why the difference? We’re not exactly selfless when it comes to our buying behaviour. So if given the choice — expensive car insurance that’s open and transparent (but may result in your own death) or the same cheap insurance as every other car on the road — on balance, I suspect most people will choose the later.

And who’s car is it anyway?

Some have said that in the future we won’t own our own car anyway. Consumers could belong to a subscription service, a hybrid of Zipcar and Uber that summons a car on demand. Ownership of the cars could well fall to car manufacturers who will run fleets of their own cars. And it will be the providers of these services that would negotiate and choose the insurance for them. Car companies don’t have a great track record when it comes to putting human life over profit. It’s likely they’ll opt for the cheapest insurance, not the most human-friendly. And I don’t fault the individuals that work for these organisations. Malcolm Gladwell’s article in the New Yorker covering the Pinto scandal in the 1970’s, outlines how hard it is being an engineer in a car manufacturer business and how the organisation has an almost gravitational pull toward profit over morals.

What’s the solution?

Insurance companies aren’t inherently ‘bad’, but they have a responsibility to shareholders to do everything they can to maximise company value. Lobbying politicians is a tool they use to do this and managing the code that determines their cost of accidents will be another. If we want them to act under some form of moral guidance, then we need to create legislation that directs them in the direction we want.

Rather than spend our time debating the morality of 1,001 different accident scenarios, we need to think deeper about DRM and create a framework in which open and transparent crash code can be implemented. We need some way to monitor the crash code the insurance companies create. The Digital Millennium Copyright Act was designed to keep money flowing into film studios. It wasn’t designed to monitor life-and-death crash code. There is work being done by the likes of the Electronic Frontier Foundation, into opening up the software that lives inside cars. But they concede the road is long and hard. I’ll leave the last word to them.

“When you entrust your health, safety, or privacy to a device, the law shouldn’t punish you for trying to understand how that device works and whether it is trustworthy.”

Note: I’m focusing on the situation in the USA. It’s probable that it will play out differently across countries, but I’m based in the US so it makes sense to address the situation here.