2nd/10_Wiki/Topic_Programming/DevOps_and_Security/Deepfake-Detection.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-deepfake-detection	Deepfake Detection	10_Wiki/Topics	verified	self	Deepfake Detection Synthetic Media Detection AI-Generated Content Detection	none	A	0.9	applied	security ml forensics deepfake detection		2026-05-10	pending	language framework Python PyTorch

Deepfake Detection

매 한 줄

"매 generative model 의 fingerprint 의 수확". 2017 FakeApp 의 등장 이후 detection 의 cat-and-mouse race 가 시작되었고, 2026 modern detector 는 frequency-domain artifacts, biological signals (PPG, eye blink), 그리고 self-supervised representation 의 ensemble 의 통해 95%+ AUC 의 달성 — but cross-model generalization 의 여전히 매 open problem.

매 핵심

매 Detection 패러다임

• Frequency-domain: GAN/Diffusion 의 upsampling artifact (DCT spectrum 의 grid pattern, FFT 의 high-freq 결손).
• Biological signal: heart-rate (rPPG), micro-expression, eye blink frequency 의 unnatural pattern.
• Identity consistency: face embedding 의 video-level temporal drift.
• Self-supervised: CLIP/DINOv2 feature 의 OOD detection.

매 Generation 종류

• Face swap: DeepFaceLab, FaceFusion, Roop.
• Face reenactment: First Order Motion Model, LivePortrait (2024).
• Full-body: Wav2Lip, SadTalker, EMO (Alibaba 2024).
• Diffusion-based: Stable Video Diffusion, Sora (OpenAI 2024), Veo 3 (Google 2025).

매 응용

1. Newsroom 의 fact-checking pipeline (Reuters, AP).
2. Social platform 의 watermark + detection (Meta, TikTok, X).
3. Identity verification (KYC, banking — Persona, Onfido).
4. Forensic 증거 분석 (court-admissible chain of custody).

💻 패턴

Frequency-domain CNN (Frank et al. baseline)

import torch
import torch.nn as nn
from torch.fft import fft2, fftshift

class FrequencyDeepfakeDetector(nn.Module):
    def __init__(self, num_classes=2):
        super().__init__()
        self.backbone = nn.Sequential(
            nn.Conv2d(1, 32, 3, padding=1), nn.ReLU(),
            nn.Conv2d(32, 64, 3, padding=1), nn.ReLU(),
            nn.AdaptiveAvgPool2d(8), nn.Flatten(),
            nn.Linear(64 * 64, num_classes),
        )

    def forward(self, x):  # x: (B, 3, H, W) RGB
        gray = x.mean(1, keepdim=True)
        spec = fftshift(fft2(gray)).abs().log1p()
        return self.backbone(spec)

rPPG-based liveness (heart-rate from face video)

import numpy as np
from scipy.signal import butter, filtfilt

def extract_rppg(face_frames, fps=30):
    # POS algorithm — Wang et al. 2017
    rgb_signal = np.stack([f.reshape(-1, 3).mean(0) for f in face_frames])
    rgb_norm = rgb_signal / rgb_signal.mean(0)
    proj = rgb_norm @ np.array([[0, 1, -1], [-2, 1, 1]]).T
    s = proj[:, 0] + (proj[:, 0].std() / proj[:, 1].std()) * proj[:, 1]
    b, a = butter(4, [0.7, 4.0], btype='band', fs=fps)
    return filtfilt(b, a, s - s.mean())

def is_live(rppg, fps=30):
    fft = np.abs(np.fft.rfft(rppg))
    freqs = np.fft.rfftfreq(len(rppg), 1/fps) * 60  # BPM
    peak_bpm = freqs[fft.argmax()]
    return 50 <= peak_bpm <= 180  # 매 plausible HR range

CLIP-based zero-shot detector

import open_clip
import torch

model, _, preprocess = open_clip.create_model_and_transforms(
    'ViT-L-14', pretrained='laion2b_s32b_b82k')
tokenizer = open_clip.get_tokenizer('ViT-L-14')

prompts = ["a real photograph", "an AI-generated image",
           "a deepfake", "a synthetic face"]
text = tokenizer(prompts)
text_features = model.encode_text(text)
text_features /= text_features.norm(dim=-1, keepdim=True)

def score(image_pil):
    img = preprocess(image_pil).unsqueeze(0)
    img_feat = model.encode_image(img)
    img_feat /= img_feat.norm(dim=-1, keepdim=True)
    sims = (img_feat @ text_features.T).softmax(-1)
    return sims[0, 1:].sum().item()  # 매 fake probability

Temporal consistency (face embedding drift)

from facenet_pytorch import InceptionResnetV1

embedder = InceptionResnetV1(pretrained='vggface2').eval()

def temporal_drift(face_crops):
    embs = embedder(torch.stack(face_crops))
    embs = embs / embs.norm(dim=-1, keepdim=True)
    consec_sim = (embs[:-1] * embs[1:]).sum(-1)
    # 매 swapped face 의 unnatural jitter 의 detect
    return 1.0 - consec_sim.mean().item()

Watermark verification (C2PA / SynthID)

import hashlib
from cryptography.hazmat.primitives.asymmetric import ed25519

def verify_c2pa_manifest(manifest_bytes, signature, public_key):
    try:
        public_key.verify(signature, manifest_bytes)
        return True
    except Exception:
        return False  # 매 manifest 의 tampered 또는 missing

Ensemble fusion (production)

def ensemble_decision(image, video_clip):
    scores = {
        'freq': freq_detector(image),
        'clip': clip_detector(image),
        'rppg': 1.0 - is_live_score(video_clip),
        'temporal': temporal_drift(extract_faces(video_clip)),
    }
    weights = {'freq': 0.3, 'clip': 0.25, 'rppg': 0.25, 'temporal': 0.2}
    return sum(w * scores[k] for k, w in weights.items())

매 결정 기준

상황	Approach
Real-time KYC	rPPG + active liveness challenge
Static image forensic	Frequency CNN + CLIP zero-shot
Video newsroom	Ensemble (freq + temporal + watermark)
Cross-generator generalization	Self-supervised foundation model
High-stakes legal	Multi-modal + chain-of-custody + C2PA

기본값: ensemble of frequency + foundation-model + watermark verification.

🔗 Graph

• 부모: Computer Vision
• 응용: Content Moderation
• Adjacent: C2PA

🤖 LLM 활용

언제: feature engineering 의 brainstorm, dataset curation script, false-positive 분석. 언제 X: production detection model 의 직접 inference (LLM 의 vision 의 reliable detector 의 X — specialized model 의 사용).

❌ 안티패턴

• Single-detector reliance: GAN-trained detector 의 diffusion-generated content 의 fail.
• No cross-generator eval: train/test 의 same generator 의 inflated metric.
• Ignoring compression artifacts: JPEG/H.264 의 frequency signal 의 destroy.
• Adversarial blindness: detector 의 adversarial perturbation 의 robust 의 X.
• Watermark-only: open-source generator 의 watermark 의 strip.

🧪 검증 / 중복

• Verified (FaceForensics++ benchmark, DFDC, Frank et al. ICML 2020, C2PA spec v2.1).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — frequency/biological/CLIP detection patterns + ensemble