By Marco Fontani, Forensics Director at Amped Software
Amped Software releases a new update to Amped Authenticate and adds deepfake detection capabilities, which increases the already broad coverage of Authenticate for localizing forgeries. In this article, you will learn more about deepfakes and synthesized faces, and discover with practical examples what other improvements are included in this release.
What are GAN Deepfakes?
Deepfakes are the latest evolution of image forgery, and they are getting more and more popular and dangerous. A deepfake is a fake image or video generated with a deep ANN, that is an artificial neural network (ANN) with many layers of neurons. ANNs are not something new, they’ve been an Artificial Intelligence (AI) tool for decades, but they have become effective for working with images only recently, due to the availability of powerful processors and GPUs.
Deepfakes may be used to create a completely synthetic face (“synthetization”), to change a person’s face with someone else’s face (so-called “face-swaps”), to change what a subject is saying (“lip-sync”), or even to change the words and movements of someone’s head so that they are like a puppet, or a guided actor (“re-enactment”). Each application requires a different AI architecture and workflow.
With this release of Amped Authenticate, Amped Software delivers a new filter called Face GAN Deepfake, which lets you analyze all faces in an image to determine whether they are real or synthetic faces obtained with a Generative Adversarial Network (GAN).
A GAN is a deep-learning architecture that employs two competing neural networks: a Generator network which tries to synthesize realistic content (usually faces), and a Discriminator network which is trained to distinguish real faces from synthesized ones. An iterative feedback process lets the Generator improve to the point that the Discriminator can no longer distinguish between real and forged faces. This process is visually depicted below.
Once the process is finished, the Discriminator is discarded, and the Generator can be used to create fake faces. Synthetized faces generated with modern GANs are very hard to recognize, to the point that researchers found most viewers are completely fooled by them. If you want an idea of how realistic GAN-generated faces are, look at https://thispersondoesnotexist.com (refresh the page to see more results).
All of this was about what face GAN deepfakes are; now let’s see how you can detect such deepfakes with Amped Authenticate!
The Face GAN Deepfake filter
The current state-of-the-art technologies for deepfake detection are also based on AI, specifically on deep learning. Indeed, GAN-generated images often contain most of the properties and statistics of authentic images, so detecting such forgeries based on some hand-crafted artifact analysis is very hard.
The Amped research team has thus developed and trained a deep neural network to detect GAN-generated images. To validate their detection system, Amped participated in an international challenge proposed by the University of Catania; their system was independently tested by the challenge organizers on a large and diverse dataset (5 different face synthesis GANs were used) and obtained an overall accuracy score of 90.05% (whereas the median score of participants was 72.62%). You can find details about the challenge and how their detection system works in the related scientific paper, which has been published by the prestigious Journal of Imaging.
And now, this powerful detection tool is available in Amped Authenticate, under the Local Analysis category!
The Face GAN Deepfake filter first localizes faces within the image and then uses AI to classify each face and provide you with a label and a confidence score between 0 and 1. When the confidence score is above 0.8, the label can either be Real (green color) or Fake (red color).
When the score is lower, Authenticate will play it safer and write Uncertain Real or Uncertain Fake. For example, if you strongly downscale, filter and compress the original image shown above, here’s what you get:
There are two ways of viewing the results, available under the Output Mode parameter: “Overlay” on the original image, or “Collage” of faces. While the “Overlay” method is visually effective for images with large faces, it may generate too small boxes and labels when there are many faces, and some of which are small.
That is where the “Collage” method is preferable.
The collage can fit up to 30 faces in a single image; if your image contains a larger number of faces, you may consider creating two cropped versions of it, save them to a lossless format, and analyze them separately.
Hint: if the resulting image looks too tiny when generating the report, raise the “Report Max Image Width” parameter either in the Program Options or in the Project Properties from the default value (600) to a higher one.
The filter has one more parameter called Face Detection, which tunes the sensitivity of the, you guessed it, face detection system. There are three options: “Medium” (default), “Sensitive”, and “Robust”. If the filter didn’t detect the face you hoped to analyze, try setting the Face Detection parameter to “Sensitive”. On the other hand, if the filter happened to analyze tiny faces in the background or it even picked up something which is not a face, then try using the “Robust” value. For example, changing the Face Detection to “Sensitive” in the previous example will pick up and analyze one more face.
Most importantly, the filter also works when a synthesized image is pasted into an original picture to replace someone’s face.
Nevertheless, Authenticate’s Face GAN Deepfake does its job!
Before moving to the next topic, a remark must be done: as you know, Amped policy is to be very cautious with using AI for forensics. As you can read in the popular article “Can AI Be Used for Forensics and Investigations?“, it is believe that currently, deep learning-based AI techniques should not be used for image enhancement for evidentiary purposes; however, they can be considered for image analysis, which is what Authenticate is mostly about. Amped’s AI-based deepfake detector does not create new data; it classifies existing data. Moreover, it is important to remark that their technique is fully disclosed in a scientific paper, and the training dataset has been made available to the scientific community.
As to the produced results, it’s important to keep in mind that the confidence score provided by the system is just a way to measure how much the network feels confident about the result; it does not guarantee anything: if a face is marked as “Real 1.00”, that doesn’t mean it is undoubtedly authentic, it simply means that the tool is unable to detect any sign of a synthesized generation.
Annotate Gets Alignment and Snapping Features
In its previous update, Amped introduced powerful annotation tools to Authenticate. With this release, they’re getting even better: when you have multiple annotation objects and drag one of them over the image, magenta-dashed lines will appear to help you line up the dragged object with existing ones. Lines will also allow you to align objects to the image center and borders.
There is also a snapping feature that will help you position objects without any pixel-hunting effort. You can temporarily disable snapping by holding the ALT key while dragging the object: magenta lines will still appear, but they won’t capture the object.
Major update to the Social Media Identification filter database
Nowadays, it is increasingly common for an investigation to involve imagery from Social Media Platforms (SMPs), such as Facebook, Instagram, Twitter, etc., or from messaging apps, such as Messenger and WhatsApp.
Most SMPs will strip the original metadata, downscale the image, and recompress it as part of the uploading process. Therefore, uploading to SMPs acts as a “counter-forensic” or “laundering” process when detecting manipulations. Indeed, many image forgery detection algorithms that are very effective on full-sized manipulated images become way less reliable when applied to social media images, because of the downscale-and-recompress process.
For the above reasons, a decent image authenticity analysis cannot be oblivious to whether the investigated image comes from an SMP. That’s why Amped Authenticate has featured the Social Media Identification filter for several years. The filter is based on a nearest-neighbor classifier that extracts some of the features from the analyzed image and compares them against a dataset of thousands of pictures downloaded from SMPs; when a match is found, the corresponding SMP is marked as compatible with the questioned image.
Since SMPs continuously update their processing rules, it is crucial to update the reference database. For this reason, Amped include a significant update in this release: they have gathered original images from iOS and Android devices and uploaded them to SMPs to include the most recent signatures in the database. Moreover, they have added the “Facebook Messenger” platform to the database, which was not present before.
Overall, this means that the number of entries in the database increased by more than 900 elements, and they now cover the following SMPs: Facebook, Flickr, Google+, Instagram, Messenger, Telegram, Tinypic, Tumblr, Twitter, WhatsApp.
Besides the main novelties above, this release contains improvements to other features, such as PRNU Identification and Double Encoding Detection (VPF).
Read here the full blog post from the Amped Software blog.
Contact Amped Software to learn more.