← Back to project catalogue
GP-CY-0W0GOCJCybersecurityOpen for request

Deepfake authenticity evaluation studio

A controlled evaluation studio for comparing deepfake detectors across prepared image and video datasets, compression levels, and manipulation types.

  • Nuxt
  • Vue
  • FastAPI
  • PostgreSQL
  • Python
  • PyTorch
  • OpenCV
  • FFmpeg
  • Docker

Project definition

Problem statement

Deepfake detectors can perform well on the dataset used for training but fail on a different generator, compression level, resolution, or editing pipeline. A confidence score alone cannot prove that media is authentic or manipulated.

The engineering problem is to compare detectors under controlled conditions, measure cross-condition robustness and calibration, and show the evidence and limits behind each result.

Project objectives

  • Prepare versioned real and manipulated media splits with documented source and consent status.
  • Run selected detectors through a common frame, clip, and media-level interface.
  • Compare performance across manipulation type, compression, resolution, and unseen-source splits.
  • Measure confidence calibration and define an uncertain range for human review.
  • Display frame-level evidence and common failure cases without claiming certainty.

System design

System modules

01

Media registry

Stores source, label, consent or licence, manipulation type, quality, split, and file hashes.

02

Preprocessing pipeline

Extracts metadata, samples frames, detects faces where required, and creates fixed media inputs.

03

Detector runner

Executes selected model adapters and records frame, clip, and media-level scores and runtime.

04

Robustness laboratory

Applies prepared compression, resizing, and transcoding conditions and reruns the same evaluation.

05

Evidence dashboard

Shows scores, calibration, frame timelines, explanations, errors, and review status.

Methodology

System workflow

  1. 01
    Prepare dataset

    Permitted media is hashed, labelled, documented, and assigned to fixed source-separated splits.

  2. 02
    Configure experiment

    The student selects detector versions, preprocessing, quality conditions, and metrics.

  3. 03
    Run detection

    Each detector processes the same prepared media and records predictions and performance.

  4. 04
    Test robustness

    Controlled compression and resizing variants are evaluated against the original results.

  5. 05
    Review failures

    The dashboard groups false results and uncertain cases by source and media condition.

Demonstration scenario

Two detectors process a prepared set containing real and manipulated videos from separated sources. The dashboard compares their media-level results, then repeats the test after compression. One detector loses accuracy, and the student inspects the frame timeline and places an uncertain case into human review.

Engineering

Technical architecture

Web application
Nuxt and Vue for dataset control, experiment setup, metric comparison, evidence timelines, and review.
Evaluation API
FastAPI for media jobs, detector adapters, robustness variants, predictions, metrics, and exports.
Data layer
PostgreSQL for media metadata, dataset versions, models, experiments, predictions, metrics, and reviews.
Media pipeline
Python, OpenCV, and FFmpeg for controlled decoding, frame selection, transformations, and metadata capture.
Model evaluation
PyTorch detector adapters with source-separated tests, threshold analysis, calibration, and saved model versions.

Testing

Evaluation

Evaluation measures

  • Area under the ROC and precision-recall curves
  • False-positive and false-negative rates at selected thresholds
  • Performance on unseen manipulation sources
  • Accuracy loss under compression, resizing, and transcoding
  • Calibration error and selective accuracy for reviewed cases
  • Inference time and failure rate by media duration and quality

System boundaries

  • Only licensed, consented, public research, or safely generated media is included.
  • The result is an experimental detector score and not proof that media is authentic or fake.
  • The system does not identify the person shown or infer intent, authorship, or criminal activity.
  • Real decisions require provenance checks, source investigation, and qualified human review.

Included

  1. 01Media dataset and experiment-management interface
  2. 02Detector comparison, robustness, and calibration pipelines
  3. 03Frame evidence and error-analysis dashboard
  4. 04Prepared licensed media, experiments, and evaluation results
  5. 05Complete source code in a private GitHub repository
  6. 06Complete project documentation with synopsis, abstract, methodology, model diagrams, experiment results, screenshots, and conclusion
  7. 07Setup and usage guide

Project record

No buyer information is collected on this page.

Permanent project ID
GP-CY-0W0GOCJ
Catalogued
21 Aug 2026
Completed
Pending
Verified
Pending
Demonstration
Added when ready

Handover

After purchase

  1. 01
    Payment is confirmed

    The project is marked unavailable and cannot be purchased again.

  2. 02
    Repository access is granted

    The buyer's submitted GitHub account receives access to the private repository.

  3. 03
    The purchase record is delivered

    The certification sheet is prepared from the reviewed buyer details and sent privately by email.