Search Unity

Create realistic materials using Photometric Stereo via Chrome Ball in ArtEngine

December 2, 2021 in Engine & platform | 12 min. read
hero image
hero image
Topics covered
Share

Is this article helpful for you?

Thank you for your feedback!

Introducing Photometric Stereo via Chrome Ball, a handy new way to create original and incredibly photorealistic PBR materials from just about any surface.

What it is

This new ArtEngine node automatically generates detailed Albedo and Normal maps by analyzing multiple photos of a static surface – lit from various angles – upon which a chrome ball, or reflective sphere, has been placed.

With this Photometric Stereo technique, you can more quickly capture surfaces, both indoors and out, to create your own high-quality PBR material.

What you need

All you need is your camera, tripod, light source, a reflective ball, a dark setting, and of course, ArtEngine.

The technique in brief

Set up your scene in low-light conditions, place a chrome ball in the shot, and take four to 16 photos with your fixed camera, repositioning the light each time. Back at your computer, add a mask, import the images into ArtEngine, and execute the node. Crop out the ball, and you’re all done.

How it works

In each image, the reflection on the chrome ball is used to estimate the light direction. By analyzing all of the images, ArtEngine can generate a Normal and Albedo map with accurate surface characteristics.

What it adds to ArtEngine

Capturing the subtle and coarse details of real-world surfaces can be quite challenging for artists and technicians looking to increase realism. In fact, this typically requires custom equipment.

By observing an object under different lighting conditions, you can estimate the surface normals. This is known as the Photometric Stereo technique. ArtEngine’s Multi-angle to Texture node supports this technique but requires strict lighting angles, often achieved by using a custom scanning rig.

But now, Photometric Stereo via Chrome Ball offers similar professional results, while making the overall process quicker, cheaper, and more flexible than before.

This content is hosted by a third party provider that does not allow video views without acceptance of Targeting Cookies. Please set your cookie preferences for Targeting Cookies to yes if you wish to view videos from these providers.

How to get started

Here's a more detailed list of what you need to get started:

  • Camera: With remote or timer release
  • Tripod: Plus a bracket/extension if shooting the ground
  • Light: Bright handheld torch, flashlight, or flash (LED with no lens banding)
  • Chrome Ball: A reflective sphere (i.e., polished stainless steel or obsidian)
  • Software: ArtEngine

 

Choosing a Chrome Ball

Select a sphere that is polished and highly reflective. Some options include:

  • Stainless steel: Solid bearing with polished/chrome surface
  • Glass: Obsidian (must be solid black)
  • Snooker/pool ball: Phenolic resin or polyester (also solid black)
  • Recommended diameter: 2-5 cm
Black snooker balls
A black snooker ball (left, polyester) and obsidian sphere (right, glass), both with diameters of 5 cm, side by side. These should produce similar results in most scenarios.

How to scan

Your light can be handheld. After all, strict lighting angles are no longer expected. Just circuit the scene either clockwise or counterclockwise (or mixed).

Top view
Top view: A circuit with possible lighting positions

Nothing should move during your session, except for you and the light as you reposition it.

Point the light to the center of the surface for each shot and maintain a consistent distance.

Avoid casting shadows from the tripod onto your scene.

Side view
Side view with the recommended lighting angle

To simplify things, keep the light and camera roughly equidistant from the scan center.

During the circuit, maintain about a 45° vertical lighting angle from the scan center or within the illustrated ‘safe zone.’

Hold the light steady for each shutter release.

General guidelines

Lighting

The darker your unlit scene, the better. Aim for nighttime outdoors or a darkened room. Your own light must be the dominant light source, though some ambient lighting is generally okay (i.e., dusk or moonlight). 

Remember, your light must both illuminate the surface and create a distinct reflection point on the sphere for analysis. A bright LED torch without lens banding is recommended. Some torches even let the lens be unscrewed to expose the LED(s), offering a bright, unrefracted light source.

Camera

Once set up, the camera should remain rigid and static throughout the session. Use timed intervals or a remote control to release the shutter.

Exposure

Another best practice is to set your camera exposure so that it has a black response when your light is turned off. Then take photos with the light on.

Tripod

The camera should be rigidly fixed via tripod (or similar method), to offer enough freedom for illuminating the scene from various directions.

Scan surface area

Larger areas will need stronger illumination. A larger chrome ball might be necessary if the camera distance is increased.

Sphere size (relative)

The image of the sphere in your scans must be clear enough for analysis. Three variables will impact this: Camera distance, image resolution, and the physical size of the sphere. If the sphere itself becomes pixelated, accuracy will decline. For most purposes, the recommended diameter is 2-4 cm. On smooth surfaces, use a washer for stability. Polish off any dirt or smudges when placing your sphere.

Number of scans

A maximum of 16 images can be processed. It is recommended to take at least four, depending on your desired level of realism. It’s totally fine to overlap shots after a circuit.  

Processing in ArtEngine

Once your scanning session is complete and your images are ready, there’s only one more key step needed to process them in ArtEngine.

To perform the analysis, the algorithm requires a simple mask image to indicate the position of the ball. Because the camera, sphere, and surface itself remain static, any one of your input scans can be used to create a black mask with a white disc to indicate the sphere size and location. Your mask must be 100% black and white (no grey tones).

Create this mask in ArtEngine by importing any of your images. Then adjust the Mask Paint brush size (diameter) to cover the ball, ignoring any shadows.

If the sphere is not placed near the scan center, it can appear elongated due to lens distortion. In this case, shift it slightly toward the center. Note that it’s more important to create a simple round disc than to specify some kind of ellipse. Don’t worry about painting in every small detail.

Next, output this mask to the same folder as your scans, with the word ‘mask’ in the file name. ArtEngine will automatically save it as a black and white image. Now you’re prepared to process your images and create your PBR material.

Importing your images

Unlike the standard Multi-Angle to Texture node, where individual images are wired to input pins, the Chrome Ball node lets you input, or browse to your input folder via the node properties panel.

There are three settings available in the properties panel: Mirror Shape, Shadow Strength, and Normal Gradient Removal.

Proporties panel

Mirror Shape (drop-down)

This serves to specify whether a sphere or hemisphere was used, which in turn, tells the algorithm if the object was roughly one diameter or one radius high, respectively. Mirror Shape therefore ensures that calculations are accurate. Reflective hemispheres are harder to find/make, but cast less shadows.

Shadow Strength (slider)

When producing an Albedo image with the mirror ball technique, information about the accuracy of its computation is already available. This information can be used to compensate for errors in the Albedo image. Generally, this is a shadow/gloss trade-off. Increasing the shadow strength keeps more shadows, removes gloss – and vice versa. This option will not affect the computation of the Normal, or the basis for the Albedo, so recomputation after changing this value has been optimized (range zero to one).

Normal Gradient Removal (slider)

This applies normal-aware gradient removal to the resulting Normal map (range zero to one). The node can then output as either two separate maps or a single material.

Node with Material output (left) and with Bitmap output (right)
Node with Material output (left) and with Bitmap output (right)

Upon node execution, the first 16 images in the input folder (along with the mask) are processed in alphabetical order. If there are too few images or the mask image isn’t present, the node will trigger an error or produce no result.

Once your material is generated, all of ArtEngine’s power becomes available to you. Just remove the processed Chrome Ball from your material to make it tileable, add variations, adjust blends, and so on.

Removing the processed Chrome Ball from your material

The most convenient way to remove the processed Chrome Ball is to crop it out (i.e., if/when you crop your material to create a square).

To remove it manually, without cropping, use the Clone Stamp to first remove it from the Albedo map. Then execute the node to automatically remove it from the Normal map.

Alternatively, leverage ArtEngine’s AI-powered Mutation or Seam Removal nodes to replace the area containing the ball with new image data.

Examples of a scan set and mask

Chrome ball scan
One scan of a set (left) and the Mask for that set (right)

Please note that while your images do not need to be square, all of them (including the mask) must have identical dimensions.

Example of input folder with mask
An example input folder containing 11 scans and a mask; the mask file must include the word ‘mask.’

We hope that you enjoyed this post, and look forward to seeing the stunning materials you’ll make with Photometric Stereo via Chrome Ball in ArtEngine.

For more, join our community discussion and Q&A on our Discord channel. We’d love to hear your suggestions for refining this technique.

If you don’t have ArtEngine yet, get your license here.

December 2, 2021 in Engine & platform | 12 min. read

Is this article helpful for you?

Thank you for your feedback!

Topics covered