Looking for a way to get started writing safe multithreaded code? Learn the principles behind our Job System and how it works together with the Entity Component System (ECS) and the Burst compiler from this brief intro!
In Unity 2017.3 we exposed our Job System to C# code. Together with our new Burst compiler and Entity Component System (ECS), the job system makes up a high-performance multithreaded system, that will make it possible for games to fully utilize the multicore processors available today.
The purpose of the Job System is to allow the game simulation to use all the available CPU cores. Almost all modern CPUs have multiple cores and the trend is increasing. Yet many games and applications rely on using just a single core. When you split your processing into multiple smaller chunks and run them across multiple cores, you are able to process simultaneously in parallel, instead of one after another. This uses the capacity of the cores more efficiently and therefore brings massive performance improvements. Or to be more specific, using all available cores makes the simulation use less wall-time (time on the clock from starting the simulation until completing it), without optimizing the thread-time (the number of CPU instructions spent computing the result).
The easiest way to get started reducing wall-time with the Job System is to use ParallelFor jobs. A ParallelFor job is used when processing a large set of values in the same way. Essentially, the job system processes each item in the array individually using a job - which means all the items can be processed in parallel to each other utilizing multiple CPU cores, if available. In practice, the number of jobs is actually much lower than one per item in the array, there is one job per CPU core and they each get an even amount of items to process. Since some workers finish their work faster than others, we use something called work-stealing to even out the time spent on each core. When a worker has finished all its work, it looks at the other workers' queues and tries to process some of the items assigned to another worker.
If you have some very heavy systems containing many similar items, ParallelFor works great. But even if you only have a few things of each type, you can take advantage of the Job System. On a high level, the design of a Job System is to split the entire application into small self-contained units of work called jobs. Each CPU core has its own thread executing these jobs, which makes all jobs run in parallel to each other. So as long as the different items don’t depend on each other, all you have to do is schedule jobs for them without waiting for any other jobs, and they will run in parallel to other things.
Something we often suggest when talking about the Job System is the concept of scheduling early and waiting late. The purpose of this pattern is to make sure the main thread doesn’t have to wait for the job to complete. By the time the main thread needs the results of a job, it should ideally already have finished executing. A very common question which does not have a simple answer is: Which update pass is "early" and "late"? What we mean when we say schedule early and wait late is that you should give the job as much time as possible to run. It doesn’t matter much in which part of the frame you schedule and wait, as long as they’re as far apart as possible. If one frame latency is acceptable, you can even wait for the job in the next frame. Any time you see a "wait" on the main thread in the profiler, you should investigate what it’s waiting for and, if you can, schedule that job earlier or complete it later to get rid of the wait.
A Job System is not designed to be a solution for long running low priority tasks, and it is not designed for operations waiting instead of using CPU resources, like IO. It’s still possible to do these things, but it’s not the primary purpose of the Job System, which means they come with some limitations you need to be aware of.
Each worker thread in the JobSystem is tied to a physical or a virtual CPU core. Once one of these threads start executing a job, the job will run to completion without any interruptions. If you want to share a CPU core with something else, you need to manually yield, and the only way to do that is to split your job into two jobs with dependencies between them. Since the system is never doing any context switching for you, a running job will occupy one full core of the CPU, even if you aren’t actually doing anything important.
There are many implications for using the C# Job System, and generally speaking, this approach should lead to better performance across the board. This is particularly true as new Unity features like the Entity Component System and the Burst compiler technology come into play. The Entity Component System focuses on reducing the thread-time required to compute a result by organizing your data in a very cache-friendly way. Burst focuses on reducing the thread-time by optimizing your code better when it’s running within the job system. The goal of all these systems is to increase what is fundamentally possible in Unity in terms of performance, while still supporting existing workflows and making the transition easier.
Modern hardware architecture is equipped with and trending towards having multiple cores. Yet many processes rely on using just a single core. By running multiple processes across multiple cores, you’re able to run it simultaneously in parallel, instead of one after another, thus utilizing the capacity of the cores more efficiently and gaining massive performance improvements.
The new C# Job System takes advantage of multiple cores in a safe and easy way. Easy, as it’s designed to open this approach up to your scripts and allow you to write fast jobified code, and safe because it provides protection from some of the pitfalls of multi-threading, such as race conditions.
You can use the new multithreaded systems to create games that run on a variety of hardware. You can also take full advantage of the performance gains to create richer game worlds with more units and more complex simulations.
To learn more and grab resources to get started please visit https://unity3d.com/unity/features/job-system-ECS.
Got questions? Talk to us on the ECS and C# Job System forum.