2nd/10_Wiki/Topics/DevOps_and_Security/Deepfake-Detection.md

---
id: wiki-2026-0508-deepfake-detection
title: Deepfake Detection
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Deepfake Detection, Synthetic Media Detection, AI-Generated Content Detection]
duplicate_of: none
source_trust_level: A
confidence_score: 0.9
verification_status: applied
tags: [security, ml, forensics, deepfake, detection]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: PyTorch
---

# Deepfake Detection

## 매 한 줄
> **"매 generative model 의 fingerprint 의 수확"**. 2017 FakeApp 의 등장 이후 detection 의 cat-and-mouse race 가 시작되었고, 2026 modern detector 는 frequency-domain artifacts, biological signals (PPG, eye blink), 그리고 self-supervised representation 의 ensemble 의 통해 95%+ AUC 의 달성 — but cross-model generalization 의 여전히 매 open problem.

## 매 핵심

### 매 Detection 패러다임
- **Frequency-domain**: GAN/Diffusion 의 upsampling artifact (DCT spectrum 의 grid pattern, FFT 의 high-freq 결손).
- **Biological signal**: heart-rate (rPPG), micro-expression, eye blink frequency 의 unnatural pattern.
- **Identity consistency**: face embedding 의 video-level temporal drift.
- **Self-supervised**: CLIP/DINOv2 feature 의 OOD detection.

### 매 Generation 종류
- **Face swap**: DeepFaceLab, FaceFusion, Roop.
- **Face reenactment**: First Order Motion Model, LivePortrait (2024).
- **Full-body**: Wav2Lip, SadTalker, EMO (Alibaba 2024).
- **Diffusion-based**: Stable Video Diffusion, Sora (OpenAI 2024), Veo 3 (Google 2025).

### 매 응용
1. Newsroom 의 fact-checking pipeline (Reuters, AP).
2. Social platform 의 watermark + detection (Meta, TikTok, X).
3. Identity verification (KYC, banking — Persona, Onfido).
4. Forensic 증거 분석 (court-admissible chain of custody).

## 💻 패턴

### Frequency-domain CNN (Frank et al. baseline)
```python
import torch
import torch.nn as nn
from torch.fft import fft2, fftshift

class FrequencyDeepfakeDetector(nn.Module):
    def __init__(self, num_classes=2):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Conv2d(1, 32, 3, padding=1), nn.ReLU(),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(),
            nn.AdaptiveAvgPool2d(8), nn.Flatten(),
            nn.Linear(64 * 64, num_classes),
        )

    def forward(self, x):  # x: (B, 3, H, W) RGB
        gray = x.mean(1, keepdim=True)
        spec = fftshift(fft2(gray)).abs().log1p()
        return self.backbone(spec)
```

### rPPG-based liveness (heart-rate from face video)
```python
import numpy as np
from scipy.signal import butter, filtfilt

def extract_rppg(face_frames, fps=30):
    # POS algorithm — Wang et al. 2017
    rgb_signal = np.stack([f.reshape(-1, 3).mean(0) for f in face_frames])
    rgb_norm = rgb_signal / rgb_signal.mean(0)
    proj = rgb_norm @ np.array([[0, 1, -1], [-2, 1, 1]]).T
    s = proj[:, 0] + (proj[:, 0].std() / proj[:, 1].std()) * proj[:, 1]
    b, a = butter(4, [0.7, 4.0], btype='band', fs=fps)
    return filtfilt(b, a, s - s.mean())

def is_live(rppg, fps=30):
    fft = np.abs(np.fft.rfft(rppg))
    freqs = np.fft.rfftfreq(len(rppg), 1/fps) * 60  # BPM
    peak_bpm = freqs[fft.argmax()]
    return 50 <= peak_bpm <= 180  # 매 plausible HR range
```

### CLIP-based zero-shot detector
```python
import open_clip
import torch

model, _, preprocess = open_clip.create_model_and_transforms(
    'ViT-L-14', pretrained='laion2b_s32b_b82k')
tokenizer = open_clip.get_tokenizer('ViT-L-14')

prompts = ["a real photograph", "an AI-generated image",
           "a deepfake", "a synthetic face"]
text = tokenizer(prompts)
text_features = model.encode_text(text)
text_features /= text_features.norm(dim=-1, keepdim=True)

def score(image_pil):
    img = preprocess(image_pil).unsqueeze(0)
    img_feat = model.encode_image(img)
    img_feat /= img_feat.norm(dim=-1, keepdim=True)
    sims = (img_feat @ text_features.T).softmax(-1)
    return sims[0, 1:].sum().item()  # 매 fake probability
```

### Temporal consistency (face embedding drift)
```python
from facenet_pytorch import InceptionResnetV1

embedder = InceptionResnetV1(pretrained='vggface2').eval()

def temporal_drift(face_crops):
    embs = embedder(torch.stack(face_crops))
    embs = embs / embs.norm(dim=-1, keepdim=True)
    consec_sim = (embs[:-1] * embs[1:]).sum(-1)
    # 매 swapped face 의 unnatural jitter 의 detect
    return 1.0 - consec_sim.mean().item()
```

### Watermark verification (C2PA / SynthID)
```python
import hashlib
from cryptography.hazmat.primitives.asymmetric import ed25519

def verify_c2pa_manifest(manifest_bytes, signature, public_key):
    try:
        public_key.verify(signature, manifest_bytes)
        return True
    except Exception:
        return False  # 매 manifest 의 tampered 또는 missing
```

### Ensemble fusion (production)
```python
def ensemble_decision(image, video_clip):
    scores = {
        'freq': freq_detector(image),
        'clip': clip_detector(image),
        'rppg': 1.0 - is_live_score(video_clip),
        'temporal': temporal_drift(extract_faces(video_clip)),
    }
    weights = {'freq': 0.3, 'clip': 0.25, 'rppg': 0.25, 'temporal': 0.2}
    return sum(w * scores[k] for k, w in weights.items())
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Real-time KYC | rPPG + active liveness challenge |
| Static image forensic | Frequency CNN + CLIP zero-shot |
| Video newsroom | Ensemble (freq + temporal + watermark) |
| Cross-generator generalization | Self-supervised foundation model |
| High-stakes legal | Multi-modal + chain-of-custody + C2PA |

**기본값**: ensemble of frequency + foundation-model + watermark verification.

## 🔗 Graph
- 부모: [[Computer Vision]]
- 응용: [[Content Moderation]]
- Adjacent: [[C2PA]]

## 🤖 LLM 활용
**언제**: feature engineering 의 brainstorm, dataset curation script, false-positive 분석.
**언제 X**: production detection model 의 직접 inference (LLM 의 vision 의 reliable detector 의 X — specialized model 의 사용).

## ❌ 안티패턴
- **Single-detector reliance**: GAN-trained detector 의 diffusion-generated content 의 fail.
- **No cross-generator eval**: train/test 의 same generator 의 inflated metric.
- **Ignoring compression artifacts**: JPEG/H.264 의 frequency signal 의 destroy.
- **Adversarial blindness**: detector 의 adversarial perturbation 의 robust 의 X.
- **Watermark-only**: open-source generator 의 watermark 의 strip.

## 🧪 검증 / 중복
- Verified (FaceForensics++ benchmark, DFDC, Frank et al. ICML 2020, C2PA spec v2.1).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — frequency/biological/CLIP detection patterns + ensemble |