Search Unity

Stream high-quality real-time graphics through your browser with our new WebRTC framework

September 17, 2019 in Technology | 4 min. read
Topics covered

Now you can develop applications that take advantage of Unity’s rapidly advancing graphics capabilities without being constrained by device performance. Our new open-source WebRTC library for Unity and easy-to-use drop-in framework demonstrates how you can stream your projects through your browser.

The power of WebRTC technologies lets you run Unity projects with high-quality rendering through your browser. The framework can be used in runtime or in the Editor, so it is useful for a variety of purposes, like running a car configurator made with HDRP or viewing an architectural model – projects that use high-end graphics.

This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers.

WebRTC bridges the gap between browsers and real-time rendering

Developed by Google in 2011, WebRTC is open-source software that enables real-time peer-to-peer communication between browsers and mobile platforms. Any device can use Unity’s open-source framework for render streaming, so long as it’s equipped with the latest version of a browser that supports WebRTC. This includes all major browsers for iPad, iPhone, and Android.

WebRTC can be paired with Unity thanks to our app based on the Apache 2.0 license, which is publicly available through Github. This library is also available as a Preview release through Package Manager, to make it even easier to add it to your project.

Render Streaming with the WebRTC + Unity drop-in framework

We’ve used the library to develop a render streaming framework that allows you to run Unity applications through your browser.

After adding the Render Streaming package to your project, you can control your project in real-time from everyday browsers like Google Chrome and Safari, both on desktop and mobile devices.

The package is available to add to your project through the Package Manager today. The source code is available through this Github repository.

User input from the browser

The Render Streaming package instantly sends inputs from the browser to the Unity application. This includes keyboard input, mouse and touchpad input, and HTML button events. All you have to do is set up the input processing, just as you would for a regular Unity project, and the framework will pass the input to the new Input System using WebRTC’s DataChannel API. The result from browser inputs is the same quality of developer experience that you would get from working directly in Unity.

Simulcasting to multiple devices

With the Render Streaming package, you can broadcast video to multiple devices simultaneously. The maximum number of devices depends on the specs of the broadcasting machine and video quality – an average machine under normal circumstances can broadcast to about four devices at once.

One important consideration when using the Render Streaming package is the network environment. WebRTC handles video streaming, so a stable network is essential to maintaining video quality.

It is also important to remember that each additional device increases the burden on the broadcasting machine. It’s possible to create an intranet system to stream with the Render Streaming package, but setting up an internet-based broadcasting service requires a substantially more complicated system.

In general, if the goal is to broadcast to more than ten devices, we recommend incorporating an SFU decentralized server into the system structure along with a separate STUN/TURN server for WebRTC.

Video hardware encoding

To provide high-fidelity graphics and a solid streaming frame rate for high-quality user experience, we had to tackle two problems: performance and latency.

To reduce latency, the Render Streaming framework broadcasts applications to the browser using the NVIDIA Video Codec SDK to perform GPU hardware encoding on the frame buffer.

This means that to use the framework you currently need a computer with an NVIDIA GPU. Support for other GPUs is planned for future updates.

What’s the support like and how do I get started?

The WebRTC library is based on the Apache 2.0 license and managed as a community-supported technology. We encourage you to get involved in the continuing development of this technology. Feel free to fork the repository or expand any feature you like.

To get started, take a look at the tutorial document (English, Japanese) and consult the FAQ in the documentation for any additional questions. There is also a forum for WebRTC and Render Streaming, and we would love to get your feedback.

We’re excited to see what kind of amazing projects you’ll put together with the power of WebRTC and Unity.

September 17, 2019 in Technology | 4 min. read
Topics covered