Disney's innovative AI can quickly make actors appear younger or older

The Disney researchers said that other systems currently available “typically suffer from facial identity loss, poor resolution and unstable results across subsequent video frames.” The research team mentioned that their method offers “the first practical, fully-automatic and production-ready method for re-aging faces in video images.”

The face re-aging network, or FRAN

The team calls their AI system FRAN, an acronym for face re-aging network.

FRAN is a neural network that incorporates neural face models from being pre-trained on thousands of images of different faces without any age pairing. Neural face models are created using deep neural networks and are used in digital work and facial animation.

Researchers mentioned that it would be impossible to train FRAN on datasets using real people because that would require numerous pairs of images showing people with the same facial expressions, on the same backgrounds at two known ages.  FRAN was trained using datasets from thousands of randomly generated faces to collect the necessary information for re-aging actors.

The new research conducted by Disney Research Studios shortens the aging process in video creation. Disney Research Studios focuses on innovations in filmmaking by using deep learning, artificial neural networks, and other subsets of artificial intelligence to use in videos.

The neural network can analyze an actor’s headshot and predict what part of the face would potentially be affected by aging. It then applies the aging properties, such as wrinkles for older effect or skin smoothing for a younger aesthetic. The process layers the AI- generated effect on top of the original face. The ability to use AI in the images speeds up the process and efficiently creates realistic images.

Re-aging actors in 2D images

The novel AI model FRAN is able to generate images by improving them on a 2D re-aging workflow, according to researchers at Disney. “FRAN currently only improves upon the predominant 2D re-aging workflow, whereas in particular cases a 3D re-aging solution may be preferable for more elaborate levels of control including, for instance, relighting and other physically based manipulations,” the researchers said in the study.

Any images that require enhancements in a 3D format are still being explored. The team hopes that they will one day be able to also use artificial intelligence on 3D re-aging as well. “We believe that these limitations also represent exciting opportunities for additional improvements in future work.”

Challenges using a U-NET architecture

Researchers mentioned a few limitations with the current AI tool. There are a few challenges, such as generating large changes on images. The team notes the complexities are often difficult in typical U-Net architectures, a form of deep learning. A U-Net architecture is a symmetric architecture used in image segmentation, created in 2015, specifically at the time to solve issues within biomedical image segmentation. It is now used in various artificial intelligence systems. It is named for the shape of its network, forming a U-shape and consisting of the usual AI elements of training the network using input datasets to produce output.

FRAN creates the images using a U-NET architecture incorporating a large number of datasets. The AI system can create realistic and re-aged images within a range of 18 and 85 years old.

Due to the limitation of creating large changes, stated by the researchers, there is difficulty in aging characters from extremely young ages to the present age of the actor. They recommend using other methods, such as LATS or DLFS, known as deep learning from scratch in AI, in these cases.

The research team also noted that another limitation when aging characters using AI, is the difficulty of graying scalp hair on characters, an effect that the AI system currently is unable to capture from the datasets. An additional limitation is re-aging characters while including possible changes in Body Mass Index (BMI) as they age. “Similarly, re-aging can also introduce variation in body mass index (BMI) whose effects on the face we currently cannot control,” the team stated.

The future of FRAN

Despite the challenges, the team is optimistic about using the AI to improve the re-aging process in films and create a simpler, quicker process they can continue to build upon through future advancements. “FRAN has the potential to improve existing re-aging workflows,” the researchers said, “reducing the time it takes to re-age complete shots from a matter of days to just a few hours or even minutes, facilitating the creation of high-quality visual effects at scale.”