How they did it: an analysis of emission defeat devices in modern automobiles Contag et al., IEEE Security and Privacy 2017

We’ll be looking at a selection of papers from the IEEE Security and Privacy 2017 conference over the next few days, starting with this wonderful tear down of the defeat devices used by Volkswagen Group (and Fiat Chrysler Automobiles as it turns out) in their ECUs. Through a static analysis tool the authors built, it is even possible to watch the evolution of defeat devices over a period of 8 years.

On September 18, 2015, the US Environmental Protection Agency (EPA) issued a notice of violation to the Volkswagen Group, accusing one of the world’s largest automakers of circumventing the EPA’s emissions tests, setting into motion the most expensive emissions scandal in history.

The use of so-called defeat devices is possible of course because pretty much everything that goes on in a modern car is controlled by software – over 70 electronic control units and about 100 million lines of codes in a premium class automobile. At the heart is the Engine Control Unit, or ECU, which operates a closed loop control system between engine sensors and actuators.

… while some emission control measures, like the catalytic converter or particulate filters, are passive, many others require active control by the ECU, which must sometimes sacrifice performance or efficiency for compliance. These tradeoffs are particularly challenging for diesel engines.

The authors find that both the Volkswagen group and Fiat Chrysler Automobiles used an EDC17 diesel ECU manufactured by Bosch:

Notably, we find strong evidence that both defeat devices were created by Bosch and then enabled by Volkswagen and Fiat for their respective vehicles.

(The evidence strongly suggests that Bosch builds the ECU hardware and software, and then manufacturers configure the ECU for each vehicle model using configuration variables documented by Bosch).

To understand how the defeat devices work, we first need to take a look at the means used by modern cars to control emissions. After that we’ll take a look at how the Volkswagen and Fiat devices work, and finally we’ll touch upon the CurveDiff tool the authors wrote to automatically analyse firmware and detect certain kinds of defeat devices.

Controlling emissions

I learned a lot more about diesel engines and how they control emissions in this CS paper than I ever knew before!

In a diesel engine, air is first drawn into the combustion cylinder, and then fuel is injected, which ignites in the compressed air. The mixing therefore happens at the time of ignition and so can never be perfect. “This is responsible for many of the diesel engine’s distinctive characteristics, including the black smoke and heavy knocking sound known as ‘diesel knock.'” The black smoke (particulate matter) is a result of the incomplete combustion of the fuel. The second major pollutants are nitrogen oxides (NO_x). The emissions standards limit the amount of particulate matter and NO_x that can be emitted.

There are four emission control devices that help to regulate pollution:

• Exhaust Gas Recirculation (EGR) recirculates exhaust gas back into the engine intake. This significantly reduces NO_x, but unfortunately it also increases the amount of particulate matter.
• An NO_x Storage Catalyst** (NSC) works by oxidizing NO to NO₂ and then storing NO₂ in the catalyst itself. After 30-300 seconds the catalyst capacity is exhausted and must be regenerated by switching the engine to a rich fuel-air mixture for 2 to 10 seconds. During this period, the engine is less efficient (reduced fuel economy).
• Selective Catalyst Reduction (SCR) is an NSC alternative that works by injecting urea (trade name AdBlue) into the exhaust stream to reduce NO_x emissions. SCR is more effective than NSC, and usually used in diesel engines of size 3 litres and above. “The drawback of SCR is its increased complexity and the need to carry and replenish the urea fluid. Several Volkswagen vehicles implicated in the emission cheating scandal are reported to limit urea injection levels outside of a test cycle.”
• Diesel Particulate Filters (DPF) trap particulates (soot), reducing the amount of black smoke leaving the tailpipe. A DPF needs to be purged of accumulated particulates approximately every 500 km, in a regeneration cycle that lasts 10 to 15 minutes. This requires high exhaust temperatures only achieved at full load. Since this may not occur in regular driving (track day anyone?), the ECU may need to perform active regeneration in which engine operation is adjusted to increase exhaust temperature. This can only work if the vehicle is driven for longer distances. The vehicle needs servicing if regeneration does not happen.

… according to the New York Attorney General’s complaint, at normal load Volkswagen’s DPF could only last 50,000 miles before needing replacement, far short of the 120,000 miles standard Volkswagen was required to meet, compelling Volkswagen to reduce wear on the DPF.

Emissions tests are carried out on a chassis dynamometer, which holds the car in place while its drive wheel turns with varying resistance. Emissions are measured during the test and compared to emissions standards (EPA in the US, the more stringent CARB in California, and Euro 1 through 6 in Europe). To ensure repeatable and comparable measurements of exhaust emissions to evaluate emission compliance, the test goes through exactly the same cycle each time, and these cycles are known to the manufacturers.

How Volkswagen and Fiat cheat the emissions tests

A defeat device is a mechanism that causes a vehicle to behave differently during an emission test than on the road.

Essentially the ECU will have two modes: ‘test’ mode and ‘real driving’ mode. By monitoring the observed conditions, the ECU can determine which mode to operate in. You don’t need perfect detection, so long as you tune things such that when you get it wrong, you err on the side of thinking you’re undergoing an emissions test when in fact you’re not.

There are loads of pieces of information you can use to guess at whether or not you’re in a test (I think it would be super easy to train a highly accurate classifier!). For example, from 1991-95 General Motors was caught using the fact that air conditioning was turned off (tests were done at the time with AC off). Whether the steering wheel is being turned seems to be another easy one. The defeat device found in the Fiat (as of the time of paper publication, Fiat Chrysler Automobiles have not acknowledged the device) uses a very simple test – if the engine has been running for more than 26 minutes and 40 seconds (the test duration) then it’s not in a test!

Volkswagen’s test detection mechanism is more sophisticated. Here’s the schematic from the Bosch function sheet:

(Click to enlarge)

The flip-flop (FF) labeled (1) signals whether or not the vehicle is could currently undergoing an emissions test. It is initially set to true if all of the following five conditions hold (2):

Since the pressure and temperature conditions are easily satisfied, the ECU essentially assumes it might be in a test whenever the engine is started normally. The ECU then monitors its sensors, and sets the car into normal driving mode (3) if any one of four deactivating conditions occur (4).

1. A configurable time period has elapsed since the acceleration pedal first exceeded a configurable threshold.
2. The rev counter exceeds a configurable threshold.
3. The ECU is configured not to perform emissions test detection (this would be a one-off factory setting as I understand it). This condition is never triggered.
4. As of May 2017 a fourth condition was added. This condition measures how long the vehicle has been driving at a certain speed and the distance traveled in the current driving cycle. There are seven pairs of test cycle curves, in each pair one curve represents minimum distance travelled over time, and the other curve a maximum distance travelled over time. Together they define a time-distance profile. If the car strays outside of this time-distance profile, a flip-flop is set.

If all seven flip-flops are set, then the vehicle has strayed outside the profile of each of the curves at least once since start. If this happens, the acoustic condition [indication that the car could be undergoing a test] is canceled. In the firmware we examined, these curves are used to define the precise time-distance profile of known emission tests.

(Emphasis mine).

The number of profiles checked has been increasing with time, from seven in 2009 with the version of the firmware discussed above, to ten today in VW part number 03L906012. The most recent three profiles also include a steering wheel angle check that resets the flip-flop for the given profile if the steering wheel angle deviates by more than 20 degrees from the neutral position.

When the vehicle detects it is not in a test, the fuel injection behaviour is modified by adding a value based on engine speed (under test conditions, engine speed is ignored).

Exhaust Gas Recirculation is also altered when not under test.

The following table summarises all of the subsystems whose behaviour can be altered based on whether or not the vehicle thinks it is under test:

In the Fiat 500X, NSC regeneration is configured to happen more frequently in the first 26 minutes and 40 seconds of engine operation (test duration).

The CurveDiff tool for analysing firmware

Based on the insights obtained in our case studies, we designed a static analysis tool that helps us to identify a defeat device in a given firmware image. We implemented a prototype of this approach in a tool called CurveDiff for EDC17 ECUs that enables us to track the evolution and behavior of such a device across a large number of firmware images.

In the Bosch firmware, curve definitions are one of four different configuration parameter types, identified by the suffix _CUR. CurveDiff disassembles a firmware image, creates a static single assignment (SSA) form of the logic contained within it, and then analyses curve function invocations to match curve checks against known test cycles.

The reference test cycles as used for emissions testing are available either free of charge or tied to a small subscription fee. In most cases, the cycles are given in the form of two-dimensional data points, containing information about the elapsed time in seconds and the speed at this point.

Using CurveDiff, the team analysed 963 firmware images, for which analysis completed successfully for 924. 406 of the analysed images contained a defeat device, out of which 333 contained at least one active profile. In at least 268 images, the test detection affects the EGR. Firmware images released on Dec 3rd 2014 are used in VW Passat cars, and include the refinement to the defeat device to detect steering wheel angle that we discussed previously.

This refinement of the defeat device is noteworthy given that at that point in time, the CARB hand already started to investigate emission abnormalities in Volkswagen cars.

Here’s a summary of the firmware images in which defeat devices were detected, and the cars that use them:

It will be interesting to see how the court case(s) progress!