8 deepfake examples that terrified the internet

[Rss Bot 1]

The most convincing deepfake examples show us how deep learning technology could transform pipelines for filmmakers and 3D artists and greatly reduce manual editing time. But there’s growing concern over the potential abuse of increasingly realistic doctored videos. 

Deepfakes have already made it into some of our favourite 3D movies,  and they are certainly cropping up more and more in the mainstream. Here, we look at eight landmarks in the rise of deepfakes. But first, a bit more info on what deepfakes actually are.

What are deepfakes?

Deepfakes are so-named because they use deep learning technology, a branch of machine learning that applies neural net simulation to massive data sets, to create a fake. Artificial intelligence effectively learns what a source face looks like at different angles in order to transpose the face onto a target, usually an actor, as if it were a mask. Huge advances came through the application of generative adversarial networks (GANS) to pit two AI algorithms against each other, one creating the fakes and the other grading its efforts, teaching the synthesis engine to make better forgeries.

Hollywood has transposed real or fictional faces onto other actors, for example, bringing Peter Cushing back to life in 2016’s Rogue One: A Star Wars Story, but the technique used complex, expensive pipelines and face-mounted cameras. 

Simple software tools such as FakeApp and DeepFaceLab have since made a comparable effect available to all. The technology offers interesting possibilities that range from dubbing, improving and repairing video to solving the uncanny valley effect in video games, avoiding actors having to repeat fluffed line, the creation of apps that allow us to try on clothes or hairstyles, and even to train doctors, but many fear it being used for nefarious ends. These are the fakes that have scared people the most so far.

01. Obama’s public service announcement

Many of the most convincing deepfakes have been produced using impersonators capable of mimicking the source’s voice and gestures, as seen in this warning produced by BuzzFeed and comedian Jordan Peele using After Effects CC and FakeApp. They pasted Peele’s mouth over Obama’s, replaced the former president’s jawline with one that followed Peele’s mouth movements, then used FakeApp to refine the footage through more than 50 hours of automatic processing. 

Politicians and celebrities are the most common victims of deepfakes. Less than a year before the above video, University of Washington computer scientists had used neural network AI to model the shape of Obama’s mouth and make it lip synch to audio input. 

High-profile figures are used because their public profiles provides ample source material for AI to learn from, but with the amount of selfies the average person takes in a lifetime and technological advances, perhaps soon anyone could be used as a source.

02. Nancy Pelosi slowed down

This deepfake example is not actually a deepfake but an example of why their potential misuse has become so feared in politics. This really was Nancy Pelosi, the speaker of the US House of Representatives, but the video was slowed down by 25 per cent and the pitch was altered to make it seem like she was slurring her words. 

The video was posted by a Facebook page called Politics Watchdog and was shared widely, including by former New York City mayor Rudy Giuliani, who tweeted: “What is wrong with Nancy Pelosi? Her speech pattern is bizarre.” 

Facebook initially refused to remove the clip but said it had reduced its distribution after it was fact checked as false. The post was later deleted but it’s unclear who by. The case illustrates the kind of misuse people fear could be made of tech presented by Stanford University in June that allows audio in a video to be edited as easily as a text document.

03. Zuckerberg speaks frankly

In response to Facebook’s refusal to remove the video of Nancy Pelosi, artist Bill Posters posted this on Facebook-owned Instagram in June, showing Mark Zuckerberg boasting of how the platform "owns" its users. Would Facebook react differently when its own founder was being manipulated? 

The film originally formed part of Posters’ and Daniel Howe’s Spectre piece, commissioned for Sheffield Doc Fest to draw attention to how people can be manipulated by social media. It was made using Israeli startup Canny AI’s VDR (video dialogue replacement) software, which it's promoted with a deepfake singalong starring various world leaders. 

Instagram didn’t take the Zuckerberg video down, but said it would, “treat this content the same way we treat all misinformation on Instagram. If third-party fact checkers mark it as false, we will filter it.” Posters had flagged it using the hashtag #deepfake. While the video is reasonably convincing on mute, the voice gives it away, showing that a good actor is still needed to make plausible fakes, but with AI voice synthesis already mooted by Lyrebird and Adobe VoCo, it may not be long until passable voices can easily be added to deepfakes.

04. Donald Trump lectures Belgium

In the first known case of a political party using a deepfake, Belgium’s Socialistische Partij Anders (sp.a) posted this video on Facebook back in May 2018 showing Trump taunting Belgium for remaining in the Paris climate agreement. With Trump’s hair looking even stranger than usual and the crude movement of the mouth, it’s very clearly fake, and the voiceover says as much, though the final line “We all know that climate change is fake, just like this video,” isn’t subtitled in Flemish, but it was still enough to provoke one user to comment “Trumpy needs to look at his own country with its crazy child killers,” and for sp.a to have to clarify it was fake. 

A more convincing Trump (below) was created this year by YouTuber Derpfakes, who trained DeepFaceLab to map a composite of Trump’s face over Alec Baldwin’s Saturday Night Live impersonation of the president, showing how far the technology has come in a year. The video has been blocked in the US and Canada.

05. Yang Mi travels in time

In February, a video pasting the face of Yang Mi, one of China’s best-known contemporary actors, into 1983 Hong Kong television drama The Legend Of The Condor Heroes went viral, racking up a reported 240 million views before it was removed by Chinese authorities. 

Its creator, a fan of Yang Mi, issued an apology on microblogging site Weibo and said he’d made the video as a warning to raise awareness of the technology. While it seems likely that there will be an initial backlash against deepfakes from the film and television industry, it’s also possible to see how the industry could eventually embrace the technology and turn it to profit by allowing viewers to play director on home releases through manipulating dialogue, incorporating alternative scenes or even inserting themselves as characters. Expect also to see video games chock-full of celebrity appearances.

06. Salvador Dalí comes back to life

Agency GS&P pulled off the kind of headline-grabbing stunt publicity-loving Dalí would surely have appreciated himself when they resurrected the Catalan artist as a charismatic host at the Dalí Museum in Florida. Billed as “art meets artificial intelligence”, Dalí Lives was created by pulling more than 6,000 frames from old video interviews and processing them through 1,000 hours of machine learning to be able to overlay the source onto an actor’s face. The text was composed from a mixture of quotes from interviews and letters with new commentary designed to help visitors emphasise with the artist and relate to his work. 

The novelty here is that this deepfake is interactive. A total 45 minutes of footage split over 125 videos allows more than 190,000 possible combinations depending on visitor responses and even includes comments on the weather. It finishes with Dalí turning around and snapping a selfie with his audience. Dalí claimed it was unlikely he would ever die, and maybe he was right, because he was brought to life a second time recently by Samsung’s AI lab in Moscow, this time by training AI on landmark facial features from just a handful of images rather than the usual thousands.

07. Gabon’s president shows prompts questions

For the moment at least, the majority of deepfakes online are clearly flagged as such and are not intended to fool anyone, usually being played for laughs, for example putting Nicolas Cage in everything ever produced, or for sordid fantasy – it was through fake celebrity porn that the technology first took root. But while there’s been no confirmed case of anyone trying to pass one off as real, one case has been questioned. 

No one knows quite what to make of this video of Gabon’s president Ali Bongo, who had been absent from the public eye for some time – leading to speculation about his health. This New Year's video address was supposed to lay doubts to rest but backfired due to opposition claims that it was actually a deepfake, allegations that possibly played a role in provoking an attempted military coup. 

The incident shows that the simple fact that deepfakes exist now means that any video that looks slightly odd could be called into question to sow doubt. It’s a fear that’s being taken seriously enough that the Pentagon’s Defense Advanced Research Projects Agency is investing in research into how to detect fakes, while both China and the US are discussing new legislation.

08. Bill Hader morphs into Pacino and Schwarzenegger

Ctrl Shift Face is one of the most prolific creators of deepfakes on YouTube. He’s put Jim Carrey into The Shining and Sylvester Stalone into Terminator 2, but this is one of his most eerie pieces yet, showing former Saturday Night Live star Bill Hader appear to seamlessly transform into the two actors he impersonates during this interview. The change is so subtle you almost don’t notice it happen. 

Currently, DeepFaceLab, the programme used, can only replace the target’s face below the forehead, but Stanford University has achieved a method of transferring an entire 3D head – the people of Reddit are desperate for the source code – and Heidelberg University has presented a technique that allows a source’s entire body to be mapped onto a target. Shimmer and distortion can be still be a giveaway for now, but with the technology improving all the time, for how much longer will we be able to tell fake from reality?

Read more:

• 35 greatest CGI moments of all time
• 4 inexplicable CGI decisions in the Cats movie trailer
• The best 3D modelling software

View the full article