Search Unity

How Magnopus United the Physical and Digital Worlds for Expo 2020 Dubai

March 29, 2022 in Community | 7 min. read
Expo Dubai Xplorer
Expo Dubai Xplorer

This is a guest blog post collaboration by Daisy Leak, executive producer; David Swift, Director of Software Engineering; and Ben Grossmann, co-founder at Magnopus. Magnopus is a team of 170+ artists, designers, and engineers with studios in the US and UK that create, deploy, and operate, cross-reality experiences at scale. 

At Magnopus, we’re focused on bringing people together with great experiences across the physical and digital worlds. Over the past three years, we developed and launched the largest geospatial Metaverse experience in the world, and we leveraged our Magnopus technology stack as well as the Unity engine to build the experience.  

This incredible and challenging project started almost four years ago when we joined a team from Expo 2020 Dubai to develop a visitor experience for the Mobility Pavilion, one of the three signature pavilions reflecting the themes of the World Expo: Mobility, Sustainability, and Opportunity. During the discovery with the team, we were compelled by the mission statement conveyed by Her Excellency Reem Al Hashimi: “connecting minds, creating the future.”

The idea

Many pavilions espouse a narrative of “past, present, future,” but we proposed to take the visitors on a journey of human mobility through the physical world, into the digital world, arriving at a world where both are unified. This story empowered children to overcome the present limitations of mobility, to build the world they wanted to see, as their ancestors had for millennia.  

It was during this exploration that we all wondered: Could we build an “alpha” of this vision across the entirety of the Expo site? Could we share the experience of the physical site in the form of a social digital twin to people around the world, and connect them across both?

It would be much more complicated than creating a game and would require building on constantly shifting sands the entire time, but we gained the support of the leadership of Expo to pursue this ambitious vision. So, we assembled a diverse team of over 200 engineers, designers, and artists working across seven countries, and a network of companies. Together, we worked night and day to create a city-scale cross-reality space where on-site and remote visitors to Expo 2020 Dubai could connect in real-time in shared experiences. 

After 39 months of development, 6 months of live operations, and a global pandemic, that future-facing experience is now live. Expo Dubai Xplorer is accessible through mobile apps, web, cloud technologies, as well as digital signage on the site.

Expo Dubai 2020
Expo Dubai Xplorer is available to visitors on-site, and to remote visitors from around the world, on iOS and Android; all connected by the technology stack and cloud-hosted services written by Magnopus.

Features

The experience consists of two key components united by complex interoperable technologies—a digital layer of content to enhance the real-world Expo; and a digital twin of the 4.38km² site, filled with inspiring experiences and more than 200 buildings from the world’s leading architects, including 192 unique country pavilions. 

Virtual reality was used to collaboratively develop the designs and experiences so they could be tested before the physical site was even built. 

Digital activations 

We created a huge assortment of digital art installations and located them throughout the site to reveal insightful stories of the UAE, Expo 2020 themes, country pavilions, and the little details of the site itself that transcend what’s physically present. These activations include:

  • Approachable, interactive 3D scenes that allow the user to engage with bite-sized content that enhances their Expo journey. For example, users can build their own colorful coral colony by placing digital elements like polyps and sea sponges to discover local fish species such as the Hamour. This Activation brings visibility and fosters a personal investment in the world’s fragile coral reef ecosystems that are under threat, while highlighting the UAE’s commitment to cultivating 1.5 million new coral colonies in the Arabian Gulf by 2025.
  • Large augmented reality displays connected to prominent features and buildings of the Expo site create surprise and delight. For example, an AR Water Dragon is inspired by the mind-bending water feature in the heart of Expo and it makes an epic aerial appearance to provide a thrilling, up-close encounter for visitors. 
  • Magic portals lead users into fully immersive 360° environments, transporting them to far away destinations or providing a glimpse into real-world locations. The UAE is full of stunning sights, and remote locations are made easily accessible through the power of these portals, building an appreciation for the region’s beauty. 

More than just spectacles, our Activations invite users to engage with the World Expo in meaningful ways that will earn them progress towards “Seeds of Change”, which they can then pledge to real-world causes. This creates a positive cycle of play that rewards users for their continuous curiosity and in turn, empowers them to make the world a better place.  

Connectivity across the physical and digital worlds 

The multiplayer experience features real-time connectivity between on-site and remote users, allowing us to bridge the physical/digital divide, extending reach and engagement. Standing on the Expo site, a visitor may see a friend at home on the other side of the world (viewed through the lens of their phone) as an avatar they can share experiences with, and communicate with via group creation and messaging. 

Together, both types of visitors can explore the interactive digital twin. A “collect them all” capability incentivizes users to explore the expansive site, leading them to find additional content and site features they might have otherwise missed. We were able to anchor these specifically to low-traffic areas so that the game loop encourages and rewards people for “going off the beaten path.” Treasure artifacts are linked together as a set of related items with clues to find the next one.

Expo Dubai Xplorer Physical and Digital.

Precise spatial mapping 

One of the largest deployments in the world of persistent ARCore Cloud Anchors by Google allows millions of on-site visitors to enjoy augmented reality spectacles aligned accurately to real-world locations. Remote visitors can access the same AR content and experiences, over a connected digital twin of the site, via an interface similar to popular mobile games like Roblox.

Powered by the technology stack and cloud-hosted services written by Magnopus, content is automatically streamed down to the user's device when they approach a location without impacting the application’s install size. Geospatial authoring interfaces and global content delivery networks for updating and publishing new content in real-time, enable creators to release new interactive AR experiences relevant to a visitor’s position on the site. 

Augmented Reality Falcon
This activation showcases the inspiration of one of Expo’s most iconic structures, the UAE Pavilion, and celebrates architect Santiago Calatrava's inspiration: the falcon. Powered by a set of cloud-hosted services written by Magnopus, the content is automatically streamed down to the user's device when they approach a location without impacting the application’s initial install size.

A social, interactive digital twin of the 4.38km² Expo site 

Hundreds of artists working around the world spent more than two years creating the living digital replica, enhancing it with dynamic lighting, art installations, animated experiences, and stunning spectacles. 

The platform geolocates content on the site in real-time, so the digital twin was built to a high degree of accuracy—a 1:1 scale from the architect’s CAD or BIM files—and is streamed to visitors based on their location and proximity, without impacting the application’s install size. 

Personalized avatars 

Using advanced stylized avatar digitization technology, users can create a feature animation quality 3D character of themselves by uploading a single photo. The solution we created produces an animation-ready and personalized 3D head model including hair, which can be attached to a body model and fully customized from over 700 options. The avatars run completely on the cloud and can be accessed through a web browser or a mobile device. 

Accurate virtual humans offer a more engaging and personalized experience. While the selection system we custom-built reduces user friction to make the whole process as simple as possible. 

Unity features

To create this cross-reality experience, our teams leveraged our internal technology stack, a robust set of custom solutions, and Unity to solve a diverse set of problems, including:

  • Building a lightweight and scalable application to easily target the widest range of mobile devices possible.
  • Extending the AR functionality to create a consistent AR experience for all platforms. 
  • Packaging dynamic content to reduce the initial install size using Addressables packages.

Lightweight, extensible, and cross-platform

Unity is built to make cross-platform applications easy to create with minimal platform-specific code. This allowed the team to concentrate on implementing engaging features, custom Magnopus integrations, and eye-catching content without spending too much time catering to specific devices, particularly in the very diverse ecosystem of Android devices.

During the development process, we were able to generate a virtual reality version on desktop PCs, a simplified app to be embedded inside native mobile applications, and even a version built for the outdoor kiosks on the Expo site. The kiosk application, built for Windows 10 PCs with dedicated GPUs, used cameras to show a live, high-quality augmented reality feed of animated content and the same multiplayer avatars as in the mobile application.

Streamlining the AR development process with AR Foundation

Unity’s AR Foundation, which wraps ARKit and ARCore, provided a very straightforward base for building the AR application for Expo Dubai Xplorer, but also meant we could implement features in a generic fashion with Unity. This enabled the team to build both the full virtual world experience and the AR experience from the same codebase and source assets, rather than having to build multiple different applications. 

Dynamic content using Addressables

Our team created a custom over-the-air delivery system for users both in AR and digital. Instead of building the entire application every time a piece of content changed, we expanded our Magnopus CICD solution to leverage the Addressables build process to rebuild and deploy the content that had changed. For small changes, this could be the difference between a half-hour build for all platforms and a few minutes.

Since the content and code are decoupled, we could deliver updates to content even after the application had passed through the lengthy approval process for Android and iOS. We built a service that was able to provide Unity Addressable Bundles on demand, which were delivered through our custom location triggered content delivery network. 

Our tools for geospatial authoring allowed us to associate Addressable IDs with physical locations so designers could update the placement of content in real-time. The client application could then quickly download just the content it needed for nearby activities based on our cloud APIs.

Learnings

The personal growth we’ve all experienced as a result of facing the unique challenges of a project of this scale and ambition has been unbelievable. We’ve learned an insane amount… 

  • Digital/physical experiences are not the same as video games, or casual mobile games—there was no “one size fits all” or “off the shelf” solution—so being able to develop custom capabilities was critical. 
  • Start building quickly, be agile, and avoid over-designing on paper. When you start playing with it, you’ll realize that things that seemed like a great idea might not be, and you need flexibility to “find the fun” as you build.
  • Consider “cognitive load.” The physical world might be too much already for people on-site. If you want the content to be the same across both, people off-site will need “more experience” as a base layer.
  • Avoid loading too much computing on the device, or you’ll kill users' batteries before lunch, and make people sad for the rest of the day. 
  • People have a limited amount of time. They’re willing to hold up their phones to look through them at something exciting, but after a couple of minutes they’ll get tired and put their phones down, even if the experience isn’t done. Engaging AR content works best when it’s not too long. 
  • It’s complicated, and takes longer than you think, to develop the technology at the same time as the content that goes on top of it. 
  • It’s a challenge to build a digital twin at the same time as a physical site (especially when both need to match exactly!) integrating with the active BIM workflows is critical.
  • It’s hard to design experiences that work for “everyone, everywhere” instead of “someone specific on a specific device.”  Focus where you can.

What's next?

We created Expo Dubai Xplorer as an example of what the future offers; to unite people and places in a better way. From what we learned on this large-scale “alpha,” we’re refactoring for performance and flexibility, and designing new capabilities. We aim to help others create similar large-scale experiences with the ability to spend more of their effort on the content that’s unique to them, rather than developing technologies that should already be consistent across big experiences like this.

March 29, 2022 in Community | 7 min. read