Approaches to Generating Deep Fakes

• Download project files - 75.5 MB

Deep fakes - the use of deep learning to swap one person's face into another in video - are one of the most interesting and frightening ways that AI is being used today.

While deep fakes can be used for legitimate purposes, they can also be used in disinformation. With the ability to easily swap someone's face into any video, can we really trust what our eyes are telling us? A real-looking video of a politician or actor doing or saying something shocking might not be real at all.

In this article series, we're going to show how deep fakes work, and show how to implement them from scratch. We'll then take a look at DeepFaceLab, which is the all-in-one Tensorflow-powered tool often used for creating convincing deep fakes.

In the previous article I gave a general overview of deep fakes generation and the elements involved in the process. In this article I’ll give you a brief explanation of some of the approaches you could take to generate deep fakes: Autoencoders (DIY), DeepFaceLab (pre-packaged), and Faceswap (pre-packaged). There are multiple other options, but from my perspective these are the most relevant solutions. Since this series is focused on the nuts and bolts of DIY solutions, I’ll be focusing more on autoencoders.

Autoencoders Overview

I found autoencoders to be simple to understand and easy to use. This neural network is a self-supervised learning structure designed to rebuild its input from a lower dimensional representation. In other words, their main advantage is that they compress the input into a latent-space representation (the latent, or hidden, representation of compressed data) to then rebuild it from this reduced representation. The final output should be the exact thing that was ingested. This special network is made up of two elements:

• The encoder: This part of the network takes the input data and compresses it into the latent-space representation.
• The decoder: This part of the network outputs the reconstructed data, which it generates from the latent-space representation.

This is how a general autoencoder structure looks:

The process of making input data smaller without losing important information is called dimensionality reduction, and it’s key in deep fake generators.

DeepFaceLab Overview

This software has changed the way deep fakes are produced. As its creators say in the official repository, it’s "the leading software for creating deep fakes," and it’s mind blowing. DeepFaceLab is a simple and flexible face swapping, open-source framework very popular on GitHub. It can generate realistic deep fake videos with no coding required, yet remains flexible for users who don’t have an understanding of deep learning.

DeepFaceLab can be installed on Windows, Linux, or used in Google Colab notebooks, which is very useful when you don’t have a powerful local machine. It provides a good set of functionalities: you can replace partial or whole faces, de-age faces, replace the whole head or manipulate lips in videos. I’ve taken some images and videos from their official GitHub repository just to show you how fantastic this solution is. Take a look:

These recognizable faces look quite normal, until you realize they don’t belong in the scenes they’re featured in!

Here’s an direct comparison of an original and a face swapped version:

And there’s a video: https://www.youtube.com/watch?v=Ddx5B-84ebo

As you can see, this approach is brilliant! These guys have put a huge effort into it, right down to the documentation. I invite you to go and check it out!

Faceswap Overview

Faceswap, like DeepFaceLab, is brilliant as well! It’s compatible with Windows, and Linux, and MacOS, and it’s built on TensorFlow, Keras, and Python. They have a great GitHub repo and a fantastic forum where you can reach out for help on any issue that you have with the app itself.

In terms of training speed and final results, I would say the deep fakes it generates are very similar in quality to DeepFaceLab. You should go ahead and check it out, and if you have some time, play around with it!

In the next article we’ll start to focus on our DIY approach. I’ll talk about autoencoders applied to deep fakes. See you there!