Search Unity

Take a deep dive into the verification process for Verified Solutions Partners

May 6, 2021 in Technology | 13 min. read
VSP blog header image
VSP blog header image
Share

Empowering our creators for success within the Unity ecosystem is a core tenet that drives our team here at Unity. We continuously think through ways to leverage Unity-owned and partnered solutions to make that vision a reality. By engaging in ongoing conversations with our community, we work to better understand your needs and challenges, and then harness those insights to update and inform the way we shape our offerings. 

The Verified Solutions Partners (VSP) program was created in response to common critiques voiced by our community of creators like you – namely, that it’s challenging to assess quality and performance when purchasing third-party products. To take some of the guesswork out of choosing the right third-party solutions for your aims, and alleviate the pressure surrounding this decision, we designed the VSP program, which vets and curates third-party solutions based on their ability to enhance and extend the usability of our core Unity products. From SDKs and plug-ins to Editor apps and cloud services, all verified partners are Technically Verified to ensure compatibility with the latest and most frequently used versions of Unity. So, what does this verification process involve exactly, and how does it all work? Let’s break it down.

Image of the unity verification process
VSP verification proces

Technical verification

For a company to be designated a Verified Solutions Partner, their software solution must undergo a stringent verification process to ensure that the solution is compliant with Unity’s technical standards. For this reason, our Verification team provides users both quality assurance and software compatibility checks. At the highest level, our Verification team follows four guiding principles:

  1. Stability Works well with official Unity releases: 2019.4 (LTS), 2020.3 (LTS), 2021.1 (Current Tech Stream)
  2. Productivity Seamless integration with, and easy to use within, Unity Editor
  3. Performance Doesn’t slow down the in-editor experience or bloat your Unity application or game
  4. Usability Comes equipped with up-to-date documentation and well-built demo projects or scenes to showcase the solution’s functionality

Stability

The Verification team tests on the most widely used versions of Unity, the LTS versions, as well as the Current Tech Stream to guarantee that our partner solutions are future proof. These versions are updated every six months, but the main ones supported right now are 2019.4 (LTS), 2020.3 (LTS), and 2021.1 (Current Tech Stream), as mentioned above.

Additionally, the Verification team tests partner solutions on all platforms supported by the VSP solution, including but not limited to Android, iOS, Windows, Nintendo Switch, PlayStation 5, and Xbox Series X|S.

As testing occurs, our team flags all critical or major bugs that need to be fixed before passing verification to become a VSP partner. We also make recommendations to help our partners rectify minor issues and improve their product and overall user experience. 

Productivity

Our Verification team approaches testing from an end-user perspective. In other words, we download the partner solution and set up a new project in the Editor, as any Unity developer would. It is important to us that the solution feels and behaves like a part of Unity within the Editor; it should integrate easily with other Unity features and existing projects. The goal of integrating third-party solutions is to increase productivity. The Verification team, essentially, looks for an intuitive UI experience that allows you to work in-Editor just minutes after the solution has been added to your project.

Performance

There are two key indicators under performance that the Verification team screens for: The first surrounds speed, as the solution should not slow down the Editor application, and the second concerns the quality of the experience, so that there are no crashes or glitches within the Editor. We also check for any negative effects on the runtime performance of your Unity app or game.

Usability

To carefully evaluate usability, our Verification team always considers the full spectrum of user levels within the Unity developer community, including beginners. As a general rule of thumb, we strive for partner solutions that are mostly plug and play and only require a minimal amount of coding and configuration experience. The goal is to get to work quickly, rather than toy around with a plug-in. The interface should look and feel familiar to Unity users from the start.

VSP self-help guides

Self-help materials

In addition to our four guiding principles, the Verification team also ensures that every partner solution comes with up-to-date self-help guides, such as documentation and demo projects or scenes, to showcase the full functionality of the solution.

Documentation

To stay in the VSP program, partners must provide up-to-date end-user documentation for their solution. We recommend web-based documentation for detailed product and technical information, whereas a short (one-page) document should suffice for quick offline references. Such quick references should include the following information: 

  • Version number
  • Brief overview of functionality
  • Platform and Unity version supported
  • Link to online documentation
  • Known issues
  • Link to demo project (if there’s no demo scene part of the Unity package)

Demo project or scene

VSP partners are required to provide either a demo scene within their Unity package or a standalone demo project. Our Verification team reviews all demos to ensure that we can help users understand the workflow and functionality of the proposed partner solution. We also recommend demos that focus on all available APIs for implementation. Ideally, partners should reference the demo scene or project in their documentation, to facilitate the user onboarding experience.

What the VSP process looks like

What does the technical verification process look like?

The entire verification process moves through seven phases over a period of four to eight weeks:

1. Inventory check

Our team checks whether the partner has submitted all of the necessary documentation to begin the verification process. This documentation should include the following:

  • Latest version of a partner’s solution (must be a release candidate or stable release)
  • Solution documentation
  • Demo scene
  • Licenses, test environment, etc. 

2. Exploration

Naturally, this is the most fun part for the Verification team, as we get to go through the demo project and documentation submitted. We also play around with the solution in-Editor for a few days to gain hands-on experience and better grasp the workflow.

3. Test case creation

Once we have a firm understanding of the proposed solution, we write test cases to guide the testing process. A test case is a set of actions executed to verify a particular feature or functionality of a software application. It typically looks like this:

VSP test case scenario

An individual test case contains distinct test steps and data, or specific test scenarios, to verify just about any requirement. Our test cases include variables like the different Unity versions or platforms that the software solution runs on.

4.Testing

Our Verification team is made up of test engineers who use the test cases to compare expected and actual results. This determines whether a solution is operating as anticipated. We test across all supported platforms (Windows, Mac, Android, iOS, Nintendo Switch, Playstation, Xbox, etc.) for all supported versions of Unity. We also test within the Unity Editor once the Unity project has been built. Any issues that we detect are documented within the Verification report, with concrete steps, screenshots, and video footage to recreate the issue, so that our partner can identify its root cause. 

5. Reporting

After running through all the test cases, the Verification team finalizes the Verification report and assesses the severity of each individual issue or bug. The bug severity is the measure of impact that an issue can have on the functionality of a software’s feature while it is in use. Depending on the impact that an issue can have on someone using the solution, we categorize the bug severity into one of four levels:

  • Low: The bug won’t result in any noticeable breakdown of the system.
  • Minor: The bug results in some unexpected behavior, but not enough to disrupt the software’s overall functionality.
  • Major: The bug is capable of collapsing large parts of the system.
  • Critical: The bug is capable of triggering a complete system shutdown.

All critical and major bugs must be fixed before becoming a VSP partner, whereas minor and trivial bugs are highlighted with suggestions to improve the user experience.

6. Revision

After sharing and discussing the findings of the Verification report with our partner, we then move on to review and fix the bugs. In most verification processes, our team finds at least one issue that needs to be fixed prior to certifying a Verified Solutions Partner. While our team does not participate in any active bug fixing, we generally try to leverage our knowledge of Unity and its third-party ecosystem to guide our partners in the right direction to resolve the issue. Once all open issues have been addressed by our partner, the Verification team restarts the process in phase four: Testing. This process is repeated until all critical and major issues have been resolved. 

7. Verification

Once the Verification team can attest that all open issues have been resolved, the partner is deemed Technically Verified. This is a major step in our partnership that unlocks the many benefits that come with this program, such as our partner’s ability to use the Verified Solutions Partner badge in their marketing materials. In addition, we share our Verified Solutions Partners on both Unity.com and the Asset Store, and even include them in Unity-run educational and marketing programs. 

The verification process does not end there. With every major partner solution release and feature update, our partner receives another re-verification request from the Verification team. The same holds true for new Unity releases; the Verification team works to ensure that the partner solution runs smoothly on the latest version of Unity as well. The re-verification process comprises the same seven phases of verification.

Takeaway

Our Verification team consists of developers, much like the Unity end-user community. As fellow creators, we understand the importance of compatibility and performance. That’s why the verification process is so meticulous; we want everyone working in Unity to feel confident and know that when we endorse a third-party solution in the Verified Solutions Partner program, we are attesting to their high quality. To learn more about the Verified Solutions Partners program, visit this page

May 6, 2021 in Technology | 13 min. read
Related Posts