Behind the scenes: The magic of Ziva VFX in crafting The Sea Beast's characters

With its enchanting visuals, gripping narrative, and heartfelt characters, Netflix’s animated film The Sea Beast – a 2023 Academy Award^® nominee – is a tale of adventure, discovery, and the enduring power of friendship.

We talked to Andrew Anderson and Brian Cohen from Sony Pictures Imageworks about how they used Ziva VFX to create the characters Blue and Red. They shared valuable insights about the challenges they faced and how Ziva VFX helped streamline their production process and achieve their artistic vision.

What was the initial creative brief for both characters?

Brian Cohen, head of character setup/finaling (BC): The brief for Red was that she should appear majestic, yet powerful. Anatomically, she was to mostly resemble a seal, but due to her size, the creative team referenced whales quite frequently, too.

Initially, that meant seeing a lot of muscular detail, particularly when she was out of the water. We found that over the course of the show, the creative team wanted her to look smoother, so Ziva became more of a tool for providing really nice volume preservation than physically accurate anatomical detail.

Blue was a much more open exploration. The client always mentioned the adjective “cute” when referring to Blue, but what they meant exactly took some time to figure out and understand. He was supposed to be really innocent and playful, but they didn’t want him to appear too “jiggly.” This meant his properties ended up being more like a water balloon. There was an initial briefing that Blue would also be picked up and held in a lot of shots, so that impacted our setups and testing to anticipate that much collision work.

During that setup and testing phase for Blue, did that bring up any additional concerns with his animation?

Andrew Anderson, senior character setup/finaling TD (AA): Blue was a major concern, especially when trying to create the fat folds between the jaw and belly and the sides of the head and shoulders. Without an initial outward bias or an existing fat bulge, we had to create one from almost nothing.

Another concern was where normally the jaw would just press into the belly, and getting the skin and fat to push down with enough skin to fill in the deep recesses between the head and body, so we had to perform this through a simulation.

How many artists worked on the characters?

BC: Two artists were responsible for the setup of Blue – one artist for the animation facial rig and another for the animation body rig and Ziva build – while four artists were responsible for the setup of Red, including the animation body rig, the animation facial rig, and the Ziva build. In shots, as many as 10 artists on the character setup/finaling team worked together to create the final performances of Blue and Red in shots alongside the animation team.

You used a variety of tissue and bone styles on the characters, as well as some FX work. Can you talk a bit about how you used Ziva VFX for Red?

AA: Red was a good example of an anatomical setup for a feature animation. Since she is a unique creature that needed a full muscle simulation, we had to improvise the muscle setup by looking at seal, whale, and sea lion references. Without having a previous setup similar to this, we had to have modeling create a setup from scratch, which adds cost to the creature.

The advantage of this kind of setup, however, is that you can get good volume preservation on the shoulders and upper arm areas, but it took quite a bit of back and forth with the animation supervisor to nail down the look of the shoulders and arms. Another challenge occurs when the rigged character looks different than the simulated character – so to the animator, the character looks different once it comes out of the sim.

One other problem we ran into with Red was that while the musculature looks great in the initial tests to create a strong character, the artistic direction was to make the character as smooth as possible – which took out much of what Ziva could have added to the character.

With such specific art direction, what process did you use to determine the Ziva VFX setups for each case?

BC: There is always an anatomical basis when choosing a setup for characters, even in feature animation. If the character itself is not designed to match a particular real set of anatomy, identifying the correlation to a real creature and using parts of their anatomy to inform how the pushed character moves is still useful. I think where the setups move to be more stylized is for situations where we only use a subset of layers in the Ziva build, such as bone and fat with no muscle for a character like Blue.

AA: To be honest, the body type drove the type of Ziva VFX setup more than anything. Blue was a soft, fat setup, while Red was a strong, muscle-type setup, which was also based on feedback from the client.

How was it working with the scale of Red and the simulation process?

AA: Compensating for Red’s massive scale was easily handled by scaling the solver appropriately. By scaling the solver to roughly half the size of the character ( eyeballing it), we were able to use the initial muscle settings to create a good initial setup, which did not require much tweaking out of the box.

And for Blue; he’s so squishy, and that’s a tough case for deformation. How did you use stylized simulation to handle that?

AA: The simulation had to match the poses given by the art director. At the time, we had a calisthenic animation to set those poses, and in the beginning we had a hard time getting Blue to push out his stomach when his jaw was opened fully, to create a second chin effect under the jaw. Normally this would not have happened given his initial neck and jaw shape, so we had to make the bone driving the body and neck slightly thicker and installed a PSD so that when the jaw was opened, it would push out the belly more to hit the shape precisely.

Controlling the volume conservation was sometimes a bit tricky too – this is something Ziva helped us with. Where the volume would be conserved at, usually it seemed to want to push the butt and tail out larger! So we needed more control over that by making the inside body bone larger, then modifying that. We ran into a similar problem with the head tilting back, trying to create a good bulge or crease on the back.

We also used the Ziva sim for the head fins and body fins. They were very thick and needed just a bit of secondary motion, and with the tet meshes, we were able to achieve a good look of weight and thickness that the animation supervisor and client both liked. In any case, where you need flesh or thickness to an object to simulate, Ziva does a great job!

For the most part, we were able to get a good base to correct the shapes in finalling with the help of the Ziva sim and bone PSDs. Usually you want the fat to just do its thing naturally, but with the art direction, you have to find ways to cheat it a bit.

How did this work with character-to-character interactions?

AA: Most of the interaction was with the character Maisie holding Blue. We would start by  pushing Maisie’s body away from him so there were no initial collisions, and then in the preroll we had her body move into Blue and also wrap her arms around and onto him in a way as to not pass through Blue’s body.

Using collisions to fix the penetrations between her clothes and body with Blue, it added a little bit of time to the sim setup. I did use a script to help set up Maisie’s animation cache as a collision and made controls to help position the arms.

How did the number of creative iterations change when using Ziva VFX compared to other tools?

AA: Ziva definitely helped in the iteration part of the character finaling for Red and Blue. With the volume compensation and giving a better base to do shapes on top of it saved us around 30% of the finaling time on each character. This allowed the artist more time to focus on the art direction for particular parts on each character, rather than fixing the entire character.

What are your thoughts on using simulation for corrective generation vs simulating at the shot level – which direction do you see yourself going?

BC: It really depends on the character and art direction. For more naturalistic and anatomical characters that need to perform in complex ways difficult to achieve through animation, shot simulation still makes the most sense. For many cases on CG features, where the characters are pushed and less anatomical, using Ziva VFX to generate correctives as opposed to sculpting by hand can still be a huge time saver with great results, allowing the animation team to have full control in shots with great-looking deformations. We are hoping to use the corrective generation technique on all primary characters in the future.

From photorealistic to fantasy-based and stylized, populate your worlds with characters like Red and Blue, and discover how Ziva VFX can bring your characters to life.