Today marks three years since our first patch release (4.3.7p1) way back on 15th of May 2014 and the subsequent creation of the Sustained Engineering (SE) team!
The SE team was established to help improve both the quality and the stability of the latest shipped (and some legacy) versions of Unity. Weekly patches and rolled up monthly releases are the main way we try to achieve this.
As of today we have released over 220 patches, which include almost 6000 fixes from multiple teams and areas of R&D.
The first incarnation of SE was a single Release Manager, with the sole objective of getting fixes to the users from time to time, while ensuring those fixes were not lost in the process. We are now a fully-fledged team of 15 developers and 3 QA engineers, actively fixing and improving the Unity codebase every day. The official launch event of the SE team was held in Brighton in October 2015 and the SE-QA team was formed in November 2016.
In addition to fixing bugs, the team owns some of Unity’s mature systems such as Kernel and the Version Control System. We also work on improving the user experience for our source code customers. One of the major focus areas for us right now is helping to reduce technical debt - more about this will be communicated in the near future.
As well as maintaining all released versions of Unity, members of the SE team also help to alleviate the pressure on other teams in R&D by handling backports, assisting with bug fixes and writing new automated tests. This helps those teams to concentrate on new features and their own high priority bug fixes.
Members of the SE team also visit Enterprise Support customers, focusing on improving the understanding of our processes and hence increasing confidence in our product. The team participates in both customer project reviews and upgrade tests for the new releases, and has a collection of customer projects that could be used for testing future releases.
We have come a long way, so let's take a look at how we got here.
During Unity 5.0, the intention was to support a version for three months until the next major release. However, after user feedback we extended this to one year. This means we have multiple versions that we are supporting simultaneously, each version for a year since its initial release.
When possible we fix a bug on trunk first and then propagate that fix to each version, starting with the latest version and working towards the oldest - (a process known as ‘backporting’). This helps to ensure that you don’t see a bug get fixed in one version and then reappear when you upgrade. We also add new automated tests where possible, to reduce the risk of the problem being reintroduced in future versions.
During the last year, the quality of our patches became our priority; we improved our QA processes and scrutinised every fix that went in, sometimes rejecting fixes. When we first started patch releases, our process was to roll all the fixes into the patch and then QA would verify the reported bug fixes. We still verify each bug fix today, but we now have a dedicated SE QA team who also run a mini Release Acceptance Test (based on the areas touched, primarily) on every patch release. We also run a subset of our automated tests on every change we make, and the full collection of tests on each release.
We also don't backport every fix; we recognise that each change carries an inherent risk and so we must decide if the benefit outweighs the risk. We do this by examining several factors including the user pain. The more stable a version becomes, the stricter we become on accepting changes.
It is interesting to note that we can now apply fixes, run our QA process and generate the release all within a week, which was considered an impossible goal when we started patches. In addition, we do this for multiple versions simultaneously, thanks to the dedication of all teams involved, developers, QA and more importantly the build and the tools teams.
The Sustained Engineering team is continuing to invest in ways of improving the quality and stability of Unity even more and providing updates at regular intervals. We’re hard at work to see how much more we can achieve in the next three years - Happy Birthday to SE!