Search Unity

Fixing Time.deltaTime in Unity 2020.2 for smoother gameplay: What did it take?

October 1, 2020 in Technology | 18 min. read
Blog header image
Blog header image
Topics covered
Share

Unity 2020.2 beta introduces a fix to an issue that afflicts many development platforms: inconsistent Time.deltaTime values, which lead to jerky, stuttering movements. Read this blog post to understand what was going on and how the upcoming version of Unity helps you create slightly smoother gameplay.

Since the dawn of gaming, achieving framerate-independent movement in video games meant taking frame delta time into account:

void Update()
{
transform.position += m_Velocity * Time.deltaTime;
}

This achieves the desired effect of an object moving at constant average velocity, regardless of the frame rate the game is running at. It should, in theory, also move the object at a steady pace if your frame rate is rock solid. In practice, the picture is quite different. If you looked at actual reported Time.deltaTime values, you might have seen this:

6.854 ms
7.423 ms
6.691 ms
6.707 ms
7.045 ms
7.346 ms
6.513 ms

This is an issue that affects many game engines, including Unity – and we’re thankful to our users for bringing it to our attention. Happily, Unity 2020.2 beta begins to address it.

So why does this happen? Why, when the frame rate is locked to constant 144 fps, is Time.deltaTime not equal to 1144 seconds (~6.94 ms) every time? In this blog post, I’ll take you on the journey of investigating and ultimately fixing this phenomenon.

What is delta time and why is it important?

In layman’s terms, delta time is the amount of time your last frame took to complete. It sounds simple, but it’s not as intuitive as you might think. In most game development books you’ll find this canonical definition of a game loop:

while (true)
{
ProcessInput();
Update();
Render();
}

With a game loop like this, it’s easy to calculate delta time:

var time = GetTime();
while (true)
{
var lastTime = time;
time = GetTime();
var deltaTime = time - lastTime;
ProcessInput();
Update(deltaTime);
Render(deltaTime);
}

While this model is simple and easy to understand, it’s highly inadequate for modern game engines. To achieve high performance, engines nowadays use a technique called “pipelining,” which allows an engine to work on more than one frame at any given time.

Compare this:

To this:

In both of these cases, individual parts of the game loop take the same amount of time, but the second case executes them in parallel, which allows it to push out more than twice as many frames in the same amount of time. Pipelining the engine changes the frame time from being equal to the sum of all pipeline stages to being equal to the longest one.

However, even that is a simplification of what actually happens every frame in the engine:

  • Each pipeline stage takes a different amount of time every frame. Perhaps this frame has more objects on the screen than the last, which would make rendering take longer. Or perhaps the player rolled their face on the keyboard, which made input processing take longer.
  • Since different pipeline stages take different amounts of time, we need to artificially halt the faster ones so they don’t get ahead too much. Most commonly, this is implemented by waiting until some previous frame is flipped to the front buffer (also known as the screen buffer). If VSync is enabled, this additionally synchronizes to the start of the display’s VBLANK period. I’ll touch more on this later.

With that knowledge in mind, let’s take a look at a typical frame timeline in Unity 2020.1. Since platform selection and various settings significantly affect it, this article will assume a Windows Standalone player with multithreaded rendering enabled, graphics jobs disabled, vsync enabled and QualitySettings.maxQueuedFrames set to 2 running on a 144 Hz monitor without dropping any frames. Click on the image to see it in full size:

Unity’s frame pipeline wasn’t implemented from scratch. Instead, it evolved over the last decade to become what it is today. If you go back to past versions of Unity, you will find that it changes every few releases.

You may immediately notice a couple things about it:

  • Once all the work is submitted to the GPU, Unity doesn’t wait for that frame to be flipped to the screen: instead, it waits for the previous one. This is controlled by the QualitySettings.maxQueuedFrames API. This setting describes how far the frame that is currently being displayed can be behind the frame that’s currently rendering. The minimum possible value is 1, since the best you can do is render framen+1 when framen is being displayed on the screen. Since it is set to 2 in this case (which is the default), Unity makes sure that framen gets displayed on the screen before it starts rendering framen+2 (for instance, before Unity starts rendering frame5, it waits for frame3 to appear on the screen).
  • Frame5 takes longer to render on the GPU than a single refresh interval of the monitor (7.22 ms vs 6.94 ms); however, none of the frames are dropped. This happens because QualitySettings.maxQueuedFrames with the value of 2 delays when the actual frame appears on the screen, which produces a buffer in the time that safeguards against dropping frames, as long as the “spike” doesn’t become the norm. If it were set to 1, Unity would have surely dropped the frame, as it would no longer overlap the work.

Even though screen refresh happens every 6.94 ms, Unity’s time sampling presents a different image:

tdeltaTime(5) = 1.4 + 3.19 + 1.51 + 0.5 + 0.67 = 7.27 ms
tdeltaTime(6) = 1.45 + 2.81 + 1.48 + 0.5 + 0.4 = 6.64 ms
tdeltaTime(7) = 1.43 + 3.13 + 1.61 + 0.51 + 0.35 = 7.03 ms

The delta time average in this case ((7.27 + 6.64 + 7.03)/3 = 6.98 ms) is very close to the actual monitor refresh rate (6.94 ms), and if you were to measure this for a longer period of time, it would eventually average out to exactly 6.94 ms. Unfortunately, if you use this delta time as it is to calculate visible object movement, you will introduce a very subtle jitter. To illustrate this, I created a simple Unity project. It contains three green squares moving across the world space:

The camera is attached to the top cube, so it appears perfectly still on the screen. If Time.deltaTime is accurate, the middle and bottom cubes would appear to be still as well. The cubes move twice the width of the display every second: the higher the velocity, the more visible the jitter becomes. To illustrate movement, I placed purple and pink non-moving cubes in fixed positions in the background so that you can tell how fast the cubes are actually moving.

In Unity 2020.1, the middle and the bottom cubes don’t quite match the top cube movement – they jitter slightly. Below is a video captured with a slow-motion camera (slowed down 20x):

Identifying the source of delta time variation

So where do these delta time inconsistencies come from? The display shows each frame for a fixed amount of time, changing the picture every 6.94 ms. This is the real delta time because that’s how much time it takes for a frame to appear on the screen and that’s the amount of time the player of your game will observe each frame for.

Each 6.94 ms interval consists of two parts: processing and sleeping. The example frame timeline shows that the delta time is calculated on the main thread, so it will be our main focus. The processing part of the main thread consists of pumping OS messages, processing input, calling Update and issuing rendering commands. “Wait for render thread” is the sleeping part. The sum of these two intervals is equal to the real frame time:

tprocessing + twaiting = 6.94 ms

Both of these timings fluctuate for various reasons every frame, but their sum remains constant. If the processing time increases, the waiting time will decrease and vice versa, so they always equal exactly 6.94 ms. In fact, the sum of all the parts leading up to the wait always equals 6.94 ms:

tissueGPUCommands(4) + tpumpOSMessages(5) + tprocessInput(5) + tUpdate(5) + twait(5) = 1.51 + 0.5 + 0.67 + 1.45 + 2.81 = 6.94 ms
tissueGPUCommands(5) + tpumpOSMessages(6) + tprocessInput(6) + tUpdate(6) + twait(6) = 1.48 + 0.5 + 0.4 + 1.43 + 3.13 = 6.94 ms
tissueGPUCommands(6) + tpumpOSMessages(7) + tprocessInput(7) + tUpdate(7) + twait(7) = 1.61 + 0.51 + 0.35 + 1.28 + 3.19 = 6.94 ms

However, Unity queries time at the beginning of Update. Because of that, any variation in time it takes to issue rendering commands, pump OS messages or process input events will throw off the result.

A simplified Unity main thread loop can be defined like this:

while (!ShouldQuit())
{
PumpOSMessages();
UpdateInput();
SampleTime(); // We sample time here!
Update();
WaitForRenderThread();
IssueRenderingCommands();
}

The solution to this problem seems to be straightforward: just move the time sampling to after the wait, so the game loop becomes this:

while (!ShouldQuit())
{
PumpOSMessages();
UpdateInput();
Update();
WaitForRenderThread();
SampleTime();
IssueRenderingCommands();
}

However, this change doesn’t work correctly: rendering has different time readings than Update(), which has adverse effects on all sorts of things. One option is to save the sampled time at this point and update engine time only at the beginning of the next frame. However, that would mean the engine would be using time from before rendering the latest frame.

Since moving SampleTime() to after the Update() is not effective, perhaps moving the wait to the beginning of the frame will be more successful:

while (!ShouldQuit())
{
PumpOSMessages();
UpdateInput();
WaitForRenderThread();
SampleTime();
Update();
IssueRenderingCommands();
}

Unfortunately, that causes another issue: now the render thread must finish rendering almost as soon as requested, which means that the rendering thread will benefit only minimally from doing work in parallel.

Let’s look back at the frame timeline:

October 1, 2020 in Technology | 18 min. read
Topics covered