Behind the scenes: Speeding up Unity workflows

Rapid prototyping is a key element of any creative work. In this blog post, learn what we’re doing to optimize the core of the Unity Editor so that you can iterate quickly through the entire lifetime of your productions, from importing assets to building and deploying a playable game.

When your project starts to scale up as you move from a prototype into production and start adding more content, Editor performance is an important component of making everyone on the team feel productive. That’s why performance and iteration speed is one of our core product priorities for 2021, as we shared in our roadmap post.

With most types of content, the Unity Editor needs to convert the data from the asset’s source file into a format that it can use in a game or real-time application. When Unity’s done importing them, it stores these converted files and the data associated with them in the Asset Database. If, after the import, a file that the importer used has changed on disk, Unity refreshes the Asset Database. If Unity detects any script changes, it reloads the C# domain. When you’re ready to deploy your project to the target platform, Unity’s build system helps you package it to binary files which can be distributed and run on the platform of your choice.

The Asset Database is the technology at the heart of enabling a reliable, performant, and scalable Asset Import Pipeline throughout all of these steps.

But many other puzzle pieces have an impact on your Editor iteration time, from importers and compression tools to the core Editor code and the build pipeline. For this blog post, I’ve talked to some of the people who work on these areas to get a detailed explanation of what they did to help you get the most out of Unity 2021.2.

Since the Asset Database is responsible for a lot of Unity processes during project startup, we wanted to see how impactful it would be to optimize the Asset Database code. Starting with a synthetic test project of 900,000 assets, Javier Abud and Jonas Drewsen, senior software engineers from the Asset Database team, collaborated on a benchmarking framework that led to a number of optimizations and improvements to Editor startup times. They’ve also collaborated on a suite of internal tools that automate the process of gathering metrics on how changes to the Asset Database impact the performance of the Unity Editor.

“Right away, there were some low-hanging fruit, but also there were a number of situations that required a bit more outside-the-box thinking. We would chip away around 15 seconds of startup time each week on the test project and would then verify these changes with 22 real-world projects. Just to make sure that optimizations for large-scale projects did not regress performance for smaller, real-world projects,” says Javier Abud. For example, the team reduced memory allocations on threads or during sorting and introduced parallel gathering of all project files and folders. For Unity 2021.2, they managed to reduce the startup time of the Editor from 3 mins 36s down to 1 min 17s for a project with 900,000 text assets, as well as helping to speed up project opening times for smaller projects by an average of 8.7%.

Javier has also worked on a new tool called the Import Activity Window, which gives you information on why assets are imported, how long the process took, and which assets are susceptible to being imported because of their dependencies. You can learn more about it in this forum post.

Importing models and textures typically takes up the biggest piece of the pie chart when we look at what exactly is going on while you’re waiting to start working in the Editor. Here’s a test importing all the textures, models, Prefabs and Scenes from Book of the Dead: Environment project.

This contains 2.25GB of source asset data (346 textures, 133 models, 214 Prefabs, six Scenes). Timings are on Windows, AMD ThreadRipper 1950X (16 threads), and SSD storage machine, and we’re comparing Unity 2020.3.10, 2021.1.9 and 2021.2 alpha (a19).

Out of the box, 2021.2 alpha imported these assets in 114 seconds (1.27x speedup compared to 2021.1). If we enable parallel asset import with four asset import worker processes, the assets import in 77 seconds. You can find the setting for this in Project Settings → Editor → Refresh.

If we also use the Force Fast Compressor option in the Build Settings window to reduce iteration time, all the assets import in 38 seconds – that’s a 3.8x speedup compared to the 2021.1 version! This doesn’t change any texture sizes or formats, but it spends less effort trying to come up with the best possible compressed texture pixels.

Richard Kettlewell from the optimization team has been focused on speeding up model importing throughout the 2021 cycle. “There were a lot of different problems and solutions, but a few of them stood out,” he explains. “For example, instead of allocating memory per vertex, which was very slow for large meshes, some algorithms could run a pre-pass to count the total required memory and then allocate it in one block. Also, lots of import tasks operate on every mesh, vertex, animation, or material in the file. Determining which parts were independent and running them in parallel gave huge wins.” 

“We also made use of a faster memory allocator for memory that was only needed temporarily during the import phase,” he continues, “which gave a small boost everywhere. We want to thank all the artists who took the time to share problematic models with us, we couldn’t have done this without some good content to test with!”

If a file contains multiple meshes, Unity 2021.2 can now generate normals, tangents and lightmap UVs in parallel. On one particular file that took 33 minutes to import (30 minutes of which were just generating lightmap UVs), we were able to reduce the total import time to only 11 minutes, a 66% reduction. Any model file that contains multiple meshes will benefit from this change, and, as these steps can be slow, we expect to see a significant reduction in import time for any applicable files.

We have also multithreaded other model import stages, such as:

• FBX animation importing
• Running the vertex cache optimizer
• Mesh triangulation (turning n-gons into triangles)
• SketchUp vertex processing

The Quality of Life team took the lead on the texture import part of the process, continuing on the work that came out during the 2021.1 cycle. For 2021.2, they’ve again updated the ASTC compressor, gaining a further 30% speedup and improving reliability across different CPUs. Also, BC7 compression is now circa 2–3x faster. 

The Force Fast Compressor option in Build Settings is also one of their initiatives. “If you don't care about texture quality much, you can tell texture compression to spend less effort (“Force Fast Compressor”), or globally clamp maximum texture import size,” explains Aras Pranckevičius. “When I’m importing two gigabytes of textures for Android ASTC format, I can do that in under two minutes compared to 20 minutes, with the only downside being slightly more compression artifacts. This is great for when you want to iterate quickly.”

Scene roots (GameObjects at the top level in the Hierarchy view) were expensive to load and merge because they were stored in order in a linked list. Previously, opening a scene in the Editor would add each root object to this list in turn. Items were added to the list as they were loaded from the Scene, and each new GameObject had to walk the linked list looking for the new item. Now, we store a dynamic_array of newly added root objects, only sorting it and turning it into a linked list when the list is required. As a result, a generated test scene could be opened in the Editor 90% less time – from 17 seconds down to just 1.45 seconds.

One of the main tasks of the Performance Optimization Team is working with our customers and large internal productions to spot opportunities to speed up Editor operations. Even a few seconds of waiting time can become a problem as the game gets bigger. 

Peter Hall was the main driver of this effort for Unity 2021.2. One of the many speedups he worked on is optimizing the workflows of dragging and selecting lots of items in a large Scene, which took 78 seconds in one large scene from an internal project that he used as his benchmark. “Capturing this in a profiler showed that the majority of the time was in BaseHierarchyProperty::Find, called from GameObjectTreeViewDataSource.RevealItems. This was working out all the parent objects in the hierarchy back to the root, but doing it one by one for each selected item. Making use of IHierarchyProperty.FindAllAncestors cut the cost for this operation to around 50ms for a 1400x speedup,” he says.

There are other places where in-Editor workflows have been made faster for large projects, including creating new materials (2x faster), selecting all objects in a Scene (1.6x faster) or copy/pasting GameObjects (50% faster).

Once you’re ready to start testing and optimizing, iteration speed on devices becomes critical. That’s why we’re investing a lot of time and effort to optimize our build pipeline across the board. For example, Unity 2021.2 brings speedups in the Scriptable Build Pipeline and Build Cache.

One of the major focus areas here is making sure the player build time scales with the size of the incremental changes instead of the total project size. We’ve recently improved the way we’re building Unity itself, and now we’re bringing the same tools to player builds and script compilation. As a result, Unity 2021.2 beta supports incremental player build support for Windows, macOS, Android, Linux, and WebGL with a solution that supports incremental C# script compilation. We’re working on adding this improvement to the remaining platforms in future versions of Unity.

“Unity does many things when building a player. First, we build the player data by serializing all the scenes and related assets and game managers into data files. This happens in native C++ code,” explains Jonas Echterhoff from the scripting team. “When the data build is done, we will call a post-processing method. This is platform-specific, managed code. What exactly happens in the post-processor varies by platform, but usually includes tasks like copying all the player binaries and related files to the final destination, compiling the IL2CPP-generated code into a native library and platform-specific compression and platform-specific packaging. This post-processing logic is what we have moved to the new build system.” This system understands the inputs and outputs of each build step and the dependencies between them, so Unity only reruns the steps it actually needs to rerun and runs things in parallel when possible.

Also, a new IL2CPP Code Generation option in the Build Settings menu produces up to 50% less code using full-generic sharing. This allows for both faster IL2CPP build times and smaller executable files. There may be a small runtime performance impact due to the different code generation method, so this option is best suited for temporarily improving team iteration times. Let us know how this impacts your project speed in this forum thread.

In 2020.3, IL2CPP saw a major overhaul which allowed it to leverage multiple cores to generate the C++ code for a player build. “We were working in an existing code base so this imposed some constraints on the design choices we could make,” says Michael Voorhees from the IL2CPP team. “We had to remove static fields that were used across the code base. Then we refactored into divide and conquer patterns that would parallelize well. Refactoring internal APIs allowed us to maintain parallel and serial conversion modes. This was important for a low-risk rollout. Still, we’ve created robust stress-testing scenes and waited to turn on parallel conversion as the default until we had about a month of green tests.” Conversion times further decreased throughout 2021.1, thanks to improved core utilization and optimizations.

In 2021.2, the team parallelized generics collection, an expensive stage of code conversion.  “In order to open up new optimization opportunities, we replaced Cecil with our own data model. These changes, along with many smaller optimizations, have made IL2CPP code conversion in 2021.2 roughly twice as fast as 2021.1,” says Michael.

The result is a huge change in build speeds, especially for mobile developers. For example, building Dragon Crashers, our latest 2D example project, for ARMv7 and ARM64 using non-incremental build in Unity 2020 LTS took 91 seconds following a simple script change. In Unity 2021.2, using the new incremental build tools took less than half of that, 44 seconds.

Dynamic content delivery is more important than ever for our mobile developers, which means a big dependency on AssetBundles for archiving and loading assets at runtime. However, there was a fixed memory cost for each separate AssetBundle loaded simultaneously, which could range from 14k to 128k depending on the runtime platform.

This spring, Kirsten Pilla and Joseph Scheinberg from our Asset Pipeline team tackled the problem. “Instead of having a dedicated cache for each AssetBundle, we introduced a shared pool of pages,” says Kirsten. “We wanted to validate it using a real-world project that makes a heavy use of AssetBundles. So we’ve worked with Patrick DeVarney from Unity Professional Services who was helping Unknown Worlds optimize Subnautica.” With 1,616 AssetBundles loaded, the shared page pool brought down the initial 215.7MB overhead to 12.9MB, a 94% reduction. 

The team also introduced a simple public API for managing this memory budget (AssetBundle.memoryBudgetKB).

We hear that a rapid iteration cycle is one of the things you like most about working in Unity. That’s why we’re keeping a close eye on reports about team productivity bottlenecks from customer projects and internal productions. We’re also building better tools for monitoring the impact of both built-in and package features that impact Editor iteration time. By staying on top of key performance metrics for Editor iteration time during the internal development cycle, we ensure that future releases improve the speed for projects of all sizes.

In the area of asset imports, our long-term strategy is loading assets on demand, so that Unity imports only what you need when you need it. We’re also working on making sure that incremental builds are possible on all the platforms we support, with iOS coming up next.

The best way to make sure we’re considering your specific needs for speed across all of these areas is adding your suggestions in the relevant areas of our pipelines and integrations roadmap. 

We’d love to hear how the Unity 2021.2 speedups perform in your specific project. This release is now available in beta, and you can get an in-depth overview of all of the improvements in it from our beta release blog post. Join us on the Unity 2021.2 forum to let us know what you think. The key areas we’d love to get your feedback on are model and texture import and the new IL2CPP Code Generation option. Huge thanks to alpha testers who have shared their numbers already, like Robert Wetzold, the developer of traVRsal!

If you’re interested in sharing your feedback about your Unity experience in general and want to influence the future of Unity, you should join Unity Pulse. This is our new research platform and community, where we conduct surveys, polls, roundtables, interviews, and group discussions that fuel how we prioritize our resources.