The deepfake detection module is a tool that helps the user detect facial manipulations in images and videos. The tool can detect manipulations that are generated by many state-of-the-art deepfake generation approaches.
For an input image, the tool applies a face detection process and calculates a prediction for each detected face. Predictions closer to 0% mean that the corresponding faces are considered real by the model, on the contrary predictions closer to 100% indicate a manipulated face. The uncertainty of the model is increased for predictions closer to 50%.
In its current version, the tool supports JPEG, PNG, and TIF image formats and MP4, MOV, MKV, WEBM video formats. For an input image, the tool detects all faces on the frame and clusters them to face components. These components correspond to different people in the frame. For each face component, the deepfake detection model calculates a prediction. The final image prediction is an average among face component predictions. Below, there are two examples that show the results of deepfake analysis for two images. In the first case we demonstrate an example with predictions closer to 0% which means that the corresponding faces are considered real by the model, on the contrary in the second we illustrate another image with predictions closer to 100% which indicates a manipulated face.
Deepfake detection prediction closer to 0%
Deepfake detection prediction closer to 100%
For an input video, the tool applies a shot segmentation process that segments the video into parts that correspond to a continuous view filmed without interruption. For each frame in a shot, the tool detects all faces on the shot frames and clusters them to face components. These components correspond to different people in the shot. For each face component, the deepfake detection model calculates a prediction. The shot prediction is the maximum among face component predictions. The final video prediction is an average of the shot predictions.
Video example with Putin having an interview
Deepfake detection in video and shot segmentation – one person
Video example with more than one person depicted in the video
Deepfake detection in video and shot segmentation – more than one person
