New AI Can Make Your Favorite Actors Speak Fluently in Foreign Languages
We have all watched foreign films that have been dubbed and have laughed at how silly the actors look with the voice-over not matching their lips. Now, new deepfake technology has come to the surface that can dub films almost as if they were original.
According to Arstechnica, the new technology is based on artificial intelligence and it could revolutionize how films will be translated into foreign languages. The tech is developed by a UK company called Flawless co-founded by director Scott Mann who says he was fed up with poor dubbing.
"I remember just being devastated," he told Arstechnica. "You make a small change in a word or a performance, it can have a large change on a character in the story beat, and in turn on the film."
A quick visit to Flawless' website will show you an inspirational reel clip where scenes from different movies are shown translated almost to perfection. You can almost believe that the actors are indeed speaking French, German, or Japanese despite having seen these famous movies in English.
Mann achieves this perfection in translation by relying on the work of Christian Theobalt, a professor at the Max Planck Institute for Informatics in Germany and says his technique will indeed soon be flawless.
"It's going to be invisible pretty soon," he says. "People will be watching something and they won't realize it was originally shot in French or whatever."
Mann says they could even use the technology to put new lines in an actor's mouth. If that seems a little scary it's because it is. Deepfake technology has come a long way in the last few years and it has some people worried.
In 2019, a new report warned that AI might not be able to protect us from the potentially disruptive effects of deepfakes on society. Study authors Britt Paris and Joan Donovan claimed that deepfakes are part of a long history of media manipulation and would require a social as well as a technical fix to be resolved.
“The relationship between media and truth has never been stable,” the report said at the time. Flawless translation is just one step closer to generating completely fake videos of people saying things they might not ever say.
In the hands of the wrong people, this technology could be used for propaganda and fear-mongering. As deepfakes become ever more sophisticated, we have to wonder: what will keep us safe?