AI is enjoying a heyday, although the path has yet to be paved in terms of how it can be leveraged for immersive experiences. AI is not the solution to every problem, but it does provide the missing link needed to create next-generation augmented experiences.
Considering this AI boom, I’d like to talk about the concept of “context” as it relates to creating augmented reality (AR) experiences. A context is a collection of parameters describing a domain that can be encoded into machine-processable data. A domain can be literally anything, but in the AR space, three critical domains exist: physical, virtual, and human.
The physical domain is the world we live in. The context for a specific location may include world coordinates (that is, a geographical position), environment scan data, object positions, weather, or images of the surroundings – whatever real-world parameters are relevant to support the generation of a solution to a specific need.
The virtual domain contains any data that has a useful correlation with a location in the real world. This is a broad definition, but that’s the idea: AR experiences don’t need complex 3D assets or models to provide value. Any kind of location metadata can be the basis for an experience, for example, rainfall data or the location of stock in a store.
Finally, the human domain is the body of human requirements, expressed in terms a machine can understand. This is the jumping-off point for AI, where natural language processing (NLP) and generative pre-trained transformer (GPT) models play a key role in converting the human context into machine language. The human domain also encompasses how machine-generated data is communicated.
Generating a domain context is a relatively straightforward task. Where it gets tricky is ensuring the relationships between components are usable: physical and virtual coordinate systems must align, digital twins must be up to date with the physical world, human descriptions must be mappable to trainable behaviors, etc.
Both software and hardware related to emerging technologies, including AI, robotics, and the Internet of Things, are evolving rapidly. Until standards (for interoperability, for example) and best practices are in place governing their implementation, effective usage and compatibility relies on the skilled design of networked components. But once this system is designed, you have the generalized foundation for creating augmented reality experiences for any application, be it industry, retail, or general productivity improvement.
An example of how AI enables cross-domain mapping would be an individual pointing at an object in the distance. This is a physical context that many technologies can provide, but the gesture in itself doesn’t have intrinsic meaning and is not sufficient for defining the problem that needs solving. It could be in reference to a direction of travel, or an inquiry about an object. The context when correlated with language like, “How do I get there?” now forms a complete query. Thus, AI can process the physical data guided by human context data to “understand” the natural interactions we all perform daily without thinking about them and generate an appropriate response. That transparency of request/response is elevating all forms of AR experience, with the ultimate goal of making our lives easier.
Let’s explore some examples of how context can be defined for different applications. AI’s primary role is in the human domain, processing user requests, anticipating user needs, drawing on relevant data, and facilitating communication between people and devices.
Room painting: A user wants to paint a space and would like to know the quantity of materials needed. Specifics: They have a device that can measure the space and issue a voice command asking how much paint is needed.
Fitness routing: A user requests a run variation from the route usually taken. Specifics: The user has a headset that’s able to determine the user’s location, has a record of previous routes, and can project visual information.
Airport optimization: Situational awareness and automation to improve operations management. A user needs just-in-time prompts for conducting airfield activities safely. Specifics: The user has a wrist wearable that can determine the user’s location, and has a data connection to a central operational digital twin.
As one can see in these examples, the value of real-time 3D extends well beyond the generation of impressive visuals. It’s the core engine for processing cross-domain contexts to generate spatial solutions to problems. Given that we live in a 3D world, it’s not surprising that real-time 3D plays a central role.
Unity as a core data engine has massive traction in the gaming market, and so its applicability to non-game use cases is often overlooked. As wearables, devices, and AI models advance technologically, capturing more and better data, ever richer contexts will be defined, generating more precise solutions. Unity will be the primary tool for collecting this data for creating the experiences that make our lives better at work and play.
We’re excited by what our developers will create, and hopefully this blog post provided some ideas on how to structure your next-generation experiences.