How 'Coraline' studio Laika uses Intel's AI to perfect stop motion films

Laika, the stop motion animation studio behind films like Coraline and Kubo and the Two Strings, isn't afraid to embrace new technology. We chronicled how it used 3D printing to add a whole new level of detail to 2012 film Paranorman. Now, the studio has its sights set on something even more ambitious: Artificial intelligence. Laika has partnered with Intel to use machine learning tools to clean up artifacts that used to require thousands of hours of manual artist labor.

As Jeff Stringer, Laika’s director of production technology explains, the company 3D prints shapes for facial performances, which animators painstakingly tweak to create natural and emotive motion-capture characters. But that process also leaves a visible seam on the models, as well as noticeable artifacts like circles around mechanical eyelids. So far, the company has relied on manual digital rotopainting and wire removal techniques to deal with those imperfections. But, as you can imagine, that’s painstaking and time-consuming work for an entire department at Laika.

At one point, the studio even considered leaving those artifacts in the films to add a bit of character to the scene. It could have been a way to remind audiences that everything they’re watching is crafted by hand. But once Laika started clearing things up, the work scaled considerably.

“The largest subset of the visual effects team is rotopoaint, and nobody who goes to a film walks out and says, 'Oh my God, the rotopaint was so incredible in that!,” Steve Emerson, visual FX supervisor at Laika, told Engadget. “So we want to be able to to streamline some of these tasks. So then we can ship those resources over to really push the look and the visual experience of a Laika film. And then it also gives the people that are doing those types of tasks new opportunities to really stretch themselves as artists.”

The Intel-powered machine learning tool shows up as a plugin within Nuke, the 3D modeling program used by Laika artists. The company tells us it’s able to clear up artifacts in 70 frames (lasting around 3 seconds) in around five minutes and 25 seconds. Previously, it would take artists five to six hours to do the same with earlier tools. The AI solution isn’t completely hands-free: Laika still needs to train it to understand individual character models, a process which takes several weeks. And artists will still need to tell the plugin where it needs to work by setting bounding boxes. But it’s hard to argue with the massive reduction in tedious work.

Based on the labor involved with Missing Link, Laika’s most recent film, the studio estimates the Intel tool could lead to a 50 percent reduction in cleanup time, opening up 2,000 artist days. That gives it even more room to perfect every shot, as well as deal with all of the other time-consuming aspects stop-motion animation. And while it’s embracing computer tools, the studio made it clear that it’s also trying to solve plenty of issues within the camera as well. It’s looking at digital tools as a way to enhance the hand-crafted spirit of stop motion animation, not replace it.

Laika plans to use Intel’s machine learning plugin for its unannounced upcoming project. We can’t tell you what that’ll be just yet (and trust me, I asked many times), but you’ll be able to thank a bit of AI for making it possible.

Images: Laika