Projection technology has made incredible improvements in recent years. Disney took advantage of these advances most notably in the company’s theme park projection-based castle shows. They’ve even tapped into using projection on their themed wedding cakes.
However, this might be the first for the company looking to project images onto an actor’s face. The results are pretty terrifying, especially if you’re afraid of clowns.
The facial tracking system “paints” the actors’ expressions given the overall theme. It’s effectively stage makeup without the makeup.
These Makeup Lamps use motion capture to track the movements. Then color and texture adjustments can be made with changing the lighting. It can also be used to create a depth effect, as seen in the video’s wrinkle/aging demonstration.
“We’ve seen astounding advances in recent years in capturing facial performances of actors and transferring those expressions to virtual characters,” said Markus Gross, vice president at Disney Research. “Leveraging these technologies to augment the appearance of live actors is the next step and could result in amazing transformations before our eyes of stage actors in theaters or other venues.”
The biggest issue with these live projections remains latency — or the time between having the image match the actor’s expression and that image being properly registered. Disney reduced this time by simplifying its algorithm. They used a system called Kalman filtering that can predict the facial expressions made by the actor with minor adjustments.
As with most of the Disney Research announcements, we don’t know exactly what these tools could be used for. The easiest guess could be to sample what cosmetics or facial prosthetics could look like on actors before applying the look. Given how many live action films the company has slated over the next five years, we wouldn’t be surprised if this technology was used to help try on particular looks for actors. This could also potentially be used in Disney parks to expand the versatility of animatronic systems. With the Kalman filtering, the projections could easily track a robotics system created to make the same expressions.
Disney debuted the software at the European Association for Computer Graphics conference in Lyon, France earlier this week.