
Swedish Sci-Fi Thriller ‘Watch The Skies’ Will Use AI-Powered Visual Dubbing For Its International Release, But Should It?
Is this a triumphant confluence of art and technology, or an unnecessary solution to a non-existent problem?
By
Chris Catt
Recently, Variety ran a story about a Swedish movie titled Watch the Skies. The film is about a young woman who is searching for her father who she believes was abducted by aliens many years ago. But that’s not the big story here. Watch the Skies is making headlines because of its use of artificial intelligence in the filmmaking process.
More specifically, Watch the Skies is using a process called “visual dubbing” for its upcoming theatrical release in English-speaking countries. Touted as the first feature film of its kind, the movie—which was filmed in the actors’ native Swedish language—will appear as if its actors were filmed speaking English. At least, that’s the hope. How is this to be accomplished? Through everyone’s favorite hot-button topic, generative AI.
The “Flawless” Process

AI-powered filmmaking studio Flawless uses technology they call TrueSync for visual dubbing (aka vubbing). In short, TrueSync uses AI to visually alter an actor’s lip movements to match the words spoken in a dubbed audio track. For Watch the Skies, lead actors Inez Dahl Torhaug and Jesper Barkselius re-recorded all of their dialogue in English in a studio, and their new dialogue was synced to their original performances in the film.
This isn’t the first time the visual dubbing process has been used in a major film. Filmmaker Scott Mann is the co-CEO and a founder of Flawless, and he utilized vubbing in his movie Fall (2022). As Mann explains in a video on the Flawless YouTube channel, Fall was headed towards an R rating because of foul language. TrueSync was used to change any offending words (and lip movements) into something more agreeable to a PG-13.
Watch the Skies, however, will be the first film with all of its dialogue vubbed for release in international markets. That makes it notable, but “notable” doesn’t always mean “good.”
What Are The Positives?

Even though I am skeptical of the increasing use of generative AI in all forms of art, I can understand where tech like TrueSync could be beneficial for filmmakers. A hype video for Watch the Skies offers the point that visual dubbing could make the movie more appealing for certain audiences when compared to traditional dubbing. A wider reach is important, especially for smaller films trying to earn every dollar possible in a highly competitive and volatile industry.
Visual dubbing and Flawless’s related programs can also help save money by reducing the need for reshoots in some cases (like in the case of Fall mentioned above). Filmmaking is expensive, sometimes prohibitively so for smaller studios. Saving money on a production is good, but there could be a cost in other areas.
Is It Really Still The Actor’s Performance?
When given the choice of subtitles or dubbing when watching a movie or TV show made in a language I don’t understand, I choose subtitles every time. Why? Because I want to see and hear the actors’ original performances. If given the choice of subs, dubs, or vubs, I’m still going to choose subtitles. Every single time. Why? For the same reason. I want to experience 100% of an actor’s performance.
One of the claims made by Flawless is that “every nuance of the actor’s performance is preserved” when their visual dubbing process is used. But that’s not true. The performers didn’t say those words on set while they were acting out that scene. In the case of Watch the Skies, it is still the actor’s voice, but from a different performance. Vubbing is actually two separate performances linked with something generated by AI (i.e. their mouth movements).
Yes, the AI is trained on visual data from the actor, but it’s still a generative process that creates something new on top of the actual, in-camera performance. It isn’t 100% authentic.
Also, it’s important to note that different languages have different nuances. They have different rhythms, and different emphases on different parts of speech. If an actor is given the same scene to act out in two different languages, their performance choices will likely be different based on the structure of the language they’re speaking at the time. It’s not just about mouth movements. It also involves body and facial movements influenced by their understanding and emotional connection to the words that are coming out of their mouth.
And what about when the original actor can’t record dialogue in the language being dubbed? That’s not the case for Watch the Skies, but it’s a logical next step for a different actor’s voice to be synced with the original performer (which is something Flawless shows in their example videos). Would Flawless still claim that it preserves “every nuance” of the actor’s performance if they’re melding two performances from two different actors?
Traditional dubbing is good for people who need it, but it’s not a 100% authentic experience. And if the actor’s lips are altered by vubbing, then it is even less authentic. It’s a Ship of Theseus situation; how many “nuances” can be altered before the actor on screen is no longer the original actor?
There’s Nothing Wrong With Traditional Subs Or Dubs

I watch subtitled movies often. I feel that there is no discernible difference in enjoyment between a movie with dialogue originally performed in my native language and a subtitled movie in a different language. The experience is different, but it is no less enjoyable. I think subtitled is the best way to watch foreign-language films. As Bong Joon-ho once said (through a translator after hearing him say it in Korean), “once you overcome the one inch tall barrier of subtitles, you will be introduced to so many more amazing films.”
I understand that not everyone can read subtitles while watching a movie. I’m not advocating that dubbing be eliminated. What I am saying is that if you have the choice and the ability, subs are better than dubs. Watching and hearing a movie in its original language is the better choice in that situation. Part of the joy of watching movies is immersing yourself in an experience that is foreign to you. Language is a large part of that. But, dubbing is a perfectly viable option for someone who can’t read or who weirdly hates subtitles. There are many great voice actors out there who do phenomenal work.
Truthfully, I do enjoy dubs in certain circumstances. For example, older movies from Hong Kong and Italy are often completely dubbed, even if the dubbed audio is in Mandarin or Cantonese, or Italian respectively. There are plenty of additional examples of this, but my point is that these movies were dubbed in their originally released form, so it doesn’t bother me. I’d prefer to hear the original actors’ voices as they performed them in front of the camera, but since that isn’t an option in these cases, the dubs are as close as they come. And yes, I know what ADR is, and I feel similarly about that; Original is best, re-recorded in the same language by the same actor is second best.
The Uncanny Valley

Something that bothers me is when English dialogue is spoken by actors when the movie is set in a location where I know English wouldn’t be used. This extends beyond just dubbing. As I write this, I’m thinking of movies like Memoirs of a Geisha (2005) and the live-action version of Disney’s Mulan (2020). Both are primarily in English, yet, based on their settings, they shouldn’t be. And they both feel weird throughout.
To me, visual dubbing is going to create the same weirdness of mismatching language with setting, but with the additional weirdness of imperfect AI lip movements. The TrueSync tech is impressive, but it’s not anywhere close to perfect. Even just the trailer for Watch the Skies looks slightly odd at times. Is it more uncanny than a lot of the CGI overused in movies of all budgets these days? No, probably not, but it’s an additional factor that makes it feel less real. It’s also unnecessary. Subtitles are good, and traditional dubbing is a fine backup.
Watching people in modern-day Sweden speak English to each other in Watch the Skies might not be as egregious as seeing everyone speak English in mid-1900s Japan or in Disney’s version of historical China, but it’s still odd. And that kind of thing is definitely where visual dubbing is headed. Why would native speakers of a language exclusively talk to each other in a language foreign to them? It doesn’t make sense, and it’s going to be strange in other movies even after Flawless perfects visual dubbing.
The Future Of Filmmaking?

So, my gripe here isn’t with AI per se, it’s with the unnecessary use of generative AI to create a solution for a problem that doesn’t really exist. If someone has an issue of preference with watching foreign-language movies with subtitles (or dubbed normally), then their issues with foreign films might run deeper than what an artificial English performance can fix.
Still, if visual dubbing gets more eyes on smaller international films like Watch the Skies, then fine. Good. I’m happy to see more filmmakers have a better chance at success. But it’s part of the slippery slope of generative AI.
Flawless is already rolling out software to “tweak” actors’ performances in additional ways. Changing an actor’s delivery here, creating a new performance in a different location there. Every new possible tweak and alteration is another step towards the Ship of Theseus. Towards actors being little more than props for generative AI to “fix” in post. AI might be the inevitable future of filmmaking, but if this is where the future is headed, I hope it takes us a long, long time to get there.