On-demand testing with Unity Automated QA

[Update] We’re very excited about the future of DevOps at Unity, and we believe the best approach is to focus our attention for the next year on foundational parts of the pipeline, specifically version control, build, and artifact storage.

These areas are fundamental to automated testing and will help us provide a more streamlined, end-to-end experience. 

As a result, we’ve decided to put further development of Automated QA on hold, and hope to reassess in the second half of 2022. 

What does this mean for existing Automated QA users? 

Cloud testing via cloud device farm will be taken offline, but local testing features will remain active, and tests will still run in the editor; recorded playback, generated tests, and the Test Driver, will be unaffected and should continue to work as intended. 

The package will be unsupported and won't be updated with newer versions of Unity, and our Automated QA email and forum will no longer be monitored. 

Continue to follow our development plans on our new Unity DevOps roadmap page. 

---

Games are incredibly challenging to test. Game developers build games from components, yet the player interacts with a visual and dynamic world that is much more interesting and complex than a sum of components.  

Due to the complexity of modern games, even the most sophisticated QA teams have very limited options to scale the QA testing process.

We’re building Automated QA to give QA teams the power to test on-demand and at scale with recorded playback - and soon with many more types of game playing agents.

Announcing: Unity Automated QA [Experimental]

The Automated QA package enables users to record and playback touch or drag interactions with the UI of a Unity Project and optionally use recordings to drive Unity Tests -  in the editor, on an iOS or Android device.

Requirements and Limitations

• We support automation of click or touch on UI elements implemented with Unity UI
• Unity 2019+ required

Recorded Playback makes it easy to automate smoke testing

Authoring a smoke test is as easy as pressing the record button and performing the playthrough you would like to automate. When used to drive a test, the playback will only succeed if each object involved in the recorded playback was visible on-screen and clickable (or draggable).

Key Features of Recorded Playback

• Object based recording: recorded playbacks are stored as timestamped sequences of objects identified by name and tags
• Record taps or drag events
• Automatic Unity Test Framework test generation

Composite Recordings keep tests organized and maintainable

Sophisticated QA teams validate hundreds to thousands of different paths through a game before each release. We built the Composite Recording feature to enable QA testers to automate the gameplay common to many tests (think navigating through the main menu) without sacrificing the confidence that can only be achieved by testing the game as the player will experience it (e.g. by loading levels from the main menu).

Key Benefits of Composite Agents

• Create agents by mixing and matching other agents (like Recorded Playbacks)
• Less duplication of playthrough logic means easier maintainability. Menu system changes? No problem, just update the agent responsible for menu navigation and all downstream composite agents will be updated

Coming Soon: Composite Agents

We are adding support for the creation of game playing agents composed of recordings, C# scripts, and ML-Agents. Sign up for early access.

Run tests on real devices: local or in the cloud

Once you’ve authored an automated test, the Automated QA package makes it easy to test on a real device. Run tests on devices you already own, just by plugging them into your machine and pressing “Run” from the Unity Test Runner.

Coming Soon: Run tests on real devices in the cloud from the Unity Editor or CI

We’ll handle the infrastructure so that cloud on-device testing is even easier than local on-device testing. And we’ll make the resulting pass/fail status, logs, and screenshots available directly from both the Unity Editor and via API. Sign up for early access.

Try it today in minutes!

Installation

1. With your project open in the Unity Editor, open the Package Manager (Window > Package Manager).
2. Press the plus button (+) in the top left of Package Manager and then select "Add package from git URL...".
3. Enter com.unity.automated-testing in the text box and then press "Add".

Open the Recorded Playback window (Window > Automated QA > Recorded Playback) and start recording! See the package documentation for more instructions.

Please email us at AutomatedQA@unity3d.com with any questions!

Contributors: Dylan Scandinaro, Jason Greco, Justin Rempel, Lauryn Porte, Matt Sorg, Shuo Diao, Rambod Kermanizadeh, Souranil Sen, Tim Sibiski, Tom Thompson, Willis Kennedy