Before BMW’s autonomous driving (AD) technology is mass production-ready, it will need to drive 240 million virtual kilometers. Learn how Unity is helping BMW put more on its odometer every day.
In our first post, we covered how a team at BMW Fully Autonomous Driving & Driver Assistance Systems has used Unity to develop custom tools for simulation visualization and scenario creation. With these tools, the BMW Group is well-equipped to tackle the most daunting daily challenges in AD development.
Let’s walk through a few areas where Unity is helping out in a major way.
By combining simulation with key performance indicators (e.g., continually maintaining a safe distance from traffic vehicles), BMW can assess how complete its features actually are. As individual components of its AD function master basic scenarios, BMW can conduct mass validation of its entire AD system in more complex situations.
These tests come in multiple forms:
Because driving situations can be repeated as often as required in simulation, BMW runs tests throughout each day, including “night drives.” After using the Unity-based scenario editor to set up tests, developers can review the results the next morning, leverage analytics to pinpoint exactly when functions failed, and bring up the exact point of failure in a frame rendered from Unity. The team can automatically extract those problematic situations as small-scale scenarios, and then visualize them to better understand why the situation was problematic.
For instance, in this scenario, a traffic vehicle cuts in suddenly, yet the host vehicle does not decelerate immediately, resulting in a near accident. The scenario can be subsequently replayed after each incremental code update until the AD function reacts properly.
After an initial failure in this scenario, improvements ensure that this vehicle brakes properly in response to a traffic vehicle merging into its lane aggressively.
To reach the high automation level for their vehicles, BMW’s developers need to identify the limitations of their AD functions in as many situations as possible. Yet scenarios like the ones simulated in the video below are too cost-prohibitive, difficult, or dangerous to replicate in the real world.
Using the Unity-based scenario editor, the developers can devise edge-case scenarios, such as a traffic vehicle running a stop sign. Uncovering these corner cases in the confines of a virtual world is much more cost-effective and safe while allowing for the opportunity to test reproducibly.
BMW uses simulation to test scenarios that are too unusual to occur or too risky to create in a real-world driving environment. Here three edge cases are shown: 1) A pedestrian unexpectedly appearing in the host vehicle’s lane in a high-speed, highway setting; 2) A traffic vehicle cutting in suddenly; 3) A stopped vehicle in the host vehicle’s lane.
Unity is used on a daily basis to help the 1,800 AD developers in the BMW Group continuously improve the code for which they are individually responsible. As they make changes to the codebase to add a new function or improve an existing one, they run integration tests before merging into the master.
For instance, a developer focused on perception can use the Unity-based scenario editor to design multiple scenarios in which the vehicle passes a speed limit sign. These small-scale tests are simulated on the developer’s PC and can be visualized with Unity live as they are being run.
The developer can visually validate their results as well as use evaluation metrics to identify improvements or confirm that the feature is ready to be merged into the master (i.e., the vehicle adjusts to the posted limit every time).
Developers can simultaneously test and visualize the results of their incremental code updates.
Post-merge, they can run acceptance tests to identify failures in other functions that arise as a result of their commit or vice versa. For instance, a merge from their peer could introduce a bug that affects the perception of speed limit signage. The developers can use Unity for visual debugging and easily inspect what is happening so they can fix things faster.
BMW’s system is set up in such a way that developers can set breakpoints within the driving function and within the simulation code. The AD function and simulation are always in sync with one another, so the team can step through the code line by line and swap between the two worlds as they debug.
The synchronicity is also mirrored by the visualization, allowing simultaneous inspection of the code and the simulated world. Because developers can still move around and inspect values in the Unity-based application, they can reduce the number of tools needing to be open at the same time, while still keeping the data as transparent as possible.
All these elements ensure that the production code that will ultimately power BMW’s autonomous vehicles meets its standards for safety and reliability.
Learn more about the ways that Unity is used for AD simulation in our whitepaper: Top 5 Ways Real-Time 3D Is Revolutionizing the Automotive Product Lifecycle.