Search Unity

How ARCore enables you to create brand new types of user interaction

March 7, 2018 in Technology | 5 min. read
Topics covered

Is this article helpful for you?

Thank you for your feedback!

AR frameworks unlock new types of gameplay and user interaction. Here are some practical ideas to inspire you to start building. We encourage you to share your own creations with the community - we’ll be actively looking for great examples!

Back in 2003, Konami released the Game Boy Advance title, Boktai: The sun is in your hands. The physical cartridge included a light sensor that you needed to expose to sunlight in order to charge your in-game weapon. Now, using light estimation in mobile AR frameworks like ARCore 1.0 for Unity, it’s possible to create gameplay just like that without the need of additional hardware — giving you access to millions of users with AR-enabled devices!

Today, Unity developers can bring new forms of user interaction to life leveraging not only light-estimation, but features like motion tracking and environmental understanding.

ARCore 1.0 for Unity makes it so much easier for you to create mobile AR experiences for Android by leveraging Unity’s tools and community.

We want to share some functional ideas that demonstrate the capabilities of ARCore 1.0 for Unity SDK, which we hope will inspire you to create your own use-cases and share them with our community. Start building today to reach millions of users with Android AR-enabled devices and unleash the creative potential of AR gameplay and user interaction.

How will you leverage Unity and ARCore to unleash your creative potential? Join the Unity Connect Mobile AR channel and share what you’ve been working on with a gif or video. Mobile AR is still very new and there’s lots of room to explore and build unique applications. We’ll be watching for the best ideas!

The following are some of the features that help you get creative with AR.

Environmental Understanding

ARCore 1.0 for Unity provides a default grid that changes color for each new plane that it finds. You can easily modify the plane rendering by swapping out the TriGrid material texture.

Default plane with new tiling heart texture

If you want to modify the rendering further, you can change a single line of code in the PlaneGrid shader, you can also change the texture to one with transparency by returning the _GridColor * col. The colors are chosen randomly from an array in TrackedPlaneVisualizer.cs, so check that out if you have specific colors you want to use.

Modified shader and texture with transparency
fixed4 frag (v2f i) : SV_Target
    fixed4 col = tex2D(_MainTex, i.uv);
    //return fixed4(_GridColor.rgb, col.r * i.color.a);
    return _GridColor * col;


Don’t only superimpose digital objects, modify and change the planes to give your app a unique look!

Oriented points

With oriented points, objects can be stuck to all different types of surfaces at any angle. You can take ARCore outside and start to liven up the world.

Android mascot stuck to a tree using oriented feature points.

Another unique use-case takes advantage of the way we render AR by lining up objects on either side of a wall to create an x-ray effect.

Two virtual studs places on the wall then viewed from the opposite side

Light Estimation

ARCore returns an ambient light value based on overall intensity of the pixels in the current camera frame. The SDK has special shaders that allow you to easily make any object’s shading adjust to its current environment. You can use take advantage of this to create a unique effect in which characters react to their current environment, as shown below.

The flower is sad because the ambient light-value is low.
The flower becomes happy when the ambient light value increases.

Motion Tracking

Motion tracking will play an increasingly important role in mobile AR apps. Imagine using proximity and motion as key drivers for minor interactions in your application, as demonstrated below.

When the user gets close enough to a Cutie, it will trigger the wink animation.

Motion tracking is not limited to a small space, either. The device moves through the world understanding its own orientation and position. You can allows users to record motion paths simply drop markers behind them to retrace their path (as shown below).

Every meter a marker is dropped allowing a user to retrace their steps later.

ARCore resources and how to share your ideas

Share your ideas with the community and use ARCore 1.0 for Unity to create high quality AR apps for more than 100 million Android devices on Google Play! Here’s how.

  1. Setup ARCore 1.0 for Unity.
  2. Join the Unity Connect Mobile AR channel for an opportunity to meet, chat, and learn from other community creators working on AR apps.
  3. Share a short use-case video or a gif with a description on the channel.
  4. Unity will be actively engaging in the channel and watching for the most creative ideas!

In San Francisco during GDC?

Attend one of our ARCore for Unity free hands-on workshops to learn how to master the SDK (Pixel devices will be provided/BYO Laptop). For more information, visit Unity at GDC.

Article Contributors: Brad Weiers, XR Technical Product Manager, Mauricio Vergara, Partner Marketing Manager

March 7, 2018 in Technology | 5 min. read

Is this article helpful for you?

Thank you for your feedback!

Topics covered