It’s showtime! New tools for cinematic creators

Creators of film, TV, and animation need specialized tools to tell their stories. Read on to discover some of the latest workflows and features that make Unity easier and more powerful for cinematic artists.

[UPDATED November 2021 to reflect improvements and releases in 2021.2]

For a while now, our users have been out ahead of the real-time wave in the film and television space. From Dead Pixels to Fixi in Playland to Hero Dad, you’ve demonstrated how real-time 3D development in Unity adds power to the moviemaking process. And of course our internal teams haven’t been watching from the sidelines, working on one or two narrative examples along the way.

Our R&D team is making significant strides right now to support this important and growing segment of creators. Over the summer, we released an open beta of new tools for cinematic creators. Today we're pleased to announce that all these tools are in general release.

Broadly speaking, we've released two groups of tools: Cinematic Studio and the cinematic companion apps.

Cinematic Studio is a group of tools that, taken together, enable the creation of cinematic content as a first-class experience within the Unity Editor. It comes in three forms:

• Cinematic Studio Sample: this is what we released over the summer. It includes a short cartoon called Mich-L, made by our friends at Plip! Animation, and is suitable if you're using Unity 2020.3.
• Cinematic Studio Template: new in 2021.2, this is the same content as the Sample, but available directly as a template when creating a new Unity project from the Unity Hub.
• Cinematic Studio Feature Set: also new in 2021.2, the feature set includes only the tools, without the sample project. This is suitable if you just need the functionality. The feature set may be installed through the Package Manager.

Here are the specific features included in all versions of Cinematic Studio:

• Timeline: the backbone of any cinematic
• Sequences: a new workflow tool to keep your movie’s editorial content organized, collaborative, and flexible
• Recorder: a tool that captures and exports videos, stills, arbitrary output variables (AOVs), and more
• Alembic: a file format for optimized playback of complex animations
• FBX Exporter: a workflow that allows you to roundtrip content to DCCs
• Cinemachine: our camera system for framing and following all the action

The second group is comprised of two cinematic companion apps, new tools that leverage the capabilities of augmented reality (AR)-enabled iPads and iPhones to capture performances within Unity:

• Unity Virtual Camera: an app that allows you to use your mobile device to drive the Unity camera
• Unity Face Capture: an app that allows you to animate facial expressions

Both sets of tools are foundational for all storytellers across all industries and all levels of user experience. As such, these tools will be released to all users, regardless of license.

In this post, I want to highlight four of the key new workflows we’re supporting:

• Sequences
• Recorder
• Unity Virtual Camera
• Unity Face Capture

I don’t need to tell you that Unity is a great tool for making games and all sorts of interactive experiences. But that very interactivity comes at the cost of increased complexity: an interactive story can have many Timelines, triggered when needed. This requires some fairly complicated wiring in order to express exactly what needs to happen when.

A linear story is different. Everything happens in a predictable order, so a lot of the process can be automated and steps eliminated. And your project can be organized using industry-standard terms and folder structures. This is what Sequences does.

Check out the video below to see how Sequences accelerates the creation of your cinematic while simultaneously keeping you organized and flexible.

Key benefits Sequences provides:

• Create shots and sequences with just a few clicks: all the under-the-hood wiring is handled for you.
• Each cinematic element is an independently editable prefab asset, allowing you to work in parallel and avoid conflicts.
• That same prefab workflow allows you to try out changes nondestructively. 
• Recorder integration simplifies rendering output.
• Asset organization keeps your timelines and project structure tidy.
• A Timeline “Scene Activation Track” allows you to switch scenes on and off as the story progresses.

You’re telling a story, right? A movie or animation or cinematic? Presumably at some point you’ll want to show it to someone. That, in a nutshell, is what Recorder does, enabling you to export what’s on your Timeline and in your Game View. You can export content in a wide variety of formats, from still images to videos, from final pixel to intermediates for comp or effects work. And Recorder supports the most requested formats for release, including ProRes and H.264, so you can send your creations off for your audience to enjoy.

The following video details many of the great features in Recorder.

Key benefits with the Recorder:

• Export video (H.264, WebM, ProRes)
• Export frames (PNG, JPEG, EXR)
• Support for accumulation motion blur, High Definition Render Pipeline (HDRP) only
• Support for Arbitrary Output Variables (AOVs) for effects and comp work
• Export animation data (Anim, FBX, Alembic) for DCC round-tripping
• Export WAV audio

The new Unity Virtual Camera is an iOS app that leverages Apple’s ARKit to drive the movement of a camera in the Unity Editor using real-world AR tracked motion from your device. You can use this capability to shoot scenes in an organic, hand-held style with intuitive control over depth of field, zoom, motion damping, and more. Using the included take system, you can work directly in Timeline and even build up a shot over multiple takes – for example, pulling focus on a second pass.

Unity Virtual Camera is available on Apple's App Store. System requirements:

• iPad or iPhone with ARKit support
• Operating system: Windows 10 or macOS

Unity Virtual Camera’s key features:

• Use your iOS device to move and aim a Unity camera with intuitive, familiar camera controls.
• See exactly what you’re shooting on your device’s screen via a low-latency video stream.
• Preview what you just shot, right on the device.
• Build shots in iterative takes.
• See all your camera settings – sensor size (gate), depth of field, focal length, zoom, and more – via an informative heads-up display (HUD).
• Use motion scaling and damping to help you get the perfect shot.

Face tracking is an incredibly useful feature on modern mobile devices. iOS’s ARKit in particular does a great job of capturing expressions. For an animator, capturing this information and applying it to a character can turn the work of dozens of hours into a few minutes.

Unity Face Capture allows you to use your Face ID-enabled iPhone or iPad to capture, preview, and record performances and bind them to a model. We think the applications for movies and games will be immense, saving artists untold hours.

Unity Face Capture is available on Apple's App Store. System requirements:

• iPhone and iPad models that support Face ID
• Operating system: Windows 10 or macOS

Unity Face Capture’s key features include:

• Capture and animate facial expressions and head poses using ARKit.
• See the animation in real-time.
• Per-character mapping and overrides.
• Recorded animations can be applied to any compatible rig.
• Customizable settings help you manage animation values and limit jitter.

We really hope these new real-time animation tools resonate with you, our users. As I said at the outset, you’ve been leading the way in this area, and we’re delighted to be able to support you with tools that make Unity easier and more powerful for cinematic creators. 

We want to hear what you think. Are we working on the capabilities that will best help your animation projects succeed? What more can we do to answer your needs in this area? Please tell us either in the comments below or on our forum.