2nd/10_Wiki/Topics/AI_and_ML/3D Gaussian Splatting (3DGS).md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, inferred_by, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	inferred_by	tech_stack
wiki-2026-0508-3d-gaussian-splatting-3dgs	3D Gaussian Splatting (3DGS)	10_Wiki/Topics	verified	self	3DGS Gaussian Splatting 3D-GS splatting NeRF alternative	none	B	0.85	conceptual	3d-rendering neural-rendering gaussian-splatting computer-graphics real-time webgpu novel-view-synthesis		2026-05-09	pending	Claude Opus 4.7 (manual cleanup 2026-05-09)	language framework CUDA / Python / WGSL PyTorch / WebGPU / Three.js

3D Gaussian Splatting (3DGS)

📌 한 줄 통찰 (The Karpathy Summary)

Implicit field (NeRF) 가 아닌 explicit primitive (millions of 3D Gaussian) 으로 scene 표현. Real-time rendering (60+ FPS) + 고품질 + differentiable 학습. NeRF 의 modern 후계자.

📖 구조화된 지식 (Synthesized Content)

핵심 idea

• 매 scene 가 수백만 개 의 anisotropic 3D Gaussian 으로 표현.
• 매 Gaussian = (position, covariance, opacity, color via spherical harmonics).
• Camera 시점 의 projection → 2D ellipse → alpha-blend.
• Differentiable → photogrammetry image 로 train.

NeRF 와 비교

	NeRF	3DGS
표현	Implicit (MLP)	Explicit (primitive)
Train 시간	수 시간-day	수십 분
Render 시간	수 sec / frame	< 17 ms / frame
메모리	작은 (MLP)	큰 (primitive 별 byte)
편집	어려움	Per-primitive 가능
GPU	A100	RTX 3090 충분

→ 2023+ 의 favorite (Real-time 가 큰 win).

Pipeline

1. SfM (Structure from Motion): Image → camera pose + sparse point cloud (COLMAP).
2. Initialization: Sparse point → Gaussian (position 의 init).
3. Differentiable Rasterization: 2D project + alpha blend.
4. Optimization: Gradient descent on (pos, scale, rot, opacity, color).
5. Densification: 큰 gradient 가 split / clone (detail ↑).
6. Pruning: 작은 opacity = remove.

Math (간단)

매 Gaussian:

• Mean μ ∈ ℝ³
• Covariance Σ = R S Sᵀ Rᵀ (R = quaternion, S = scale).
• Opacity α ∈ [0, 1]
• Color = SH coefficient (view-dependent).

2D projection:

• Σ' = J W Σ Wᵀ Jᵀ (W = view, J = projection Jacobian).
• 2D Gaussian → tile → per-pixel α blend.

Render: C = Σᵢ cᵢ αᵢ Πⱼ<ᵢ (1 - αⱼ)

→ Order-dependent (depth sort).

Implementation: WebGPU

WebGL 의 한계:

• Compute shader X.
• 매 frame 의 sort 가 CPU (JS / WASM) → 느림.

WebGPU:

• Compute shader 가 sort GPU.
• Wait-free radix sort.
• Atomics + storage buffer.

WebSplatter (2024+):

• 매 frame 의 sort + render = GPU only.
• 4.5x faster 보다 WebGL-based.

응용

• Photogrammetry / 3D scan: drone capture → 3DGS 모델.
• VR / AR: 실제 환경 의 immersive view (Meta Reality Labs).
• Game engine: Unity / Unreal 의 plugin.
• Self-driving simulation: 실제 거리 의 train environment.
• Cultural heritage: 박물관 의 360 view.
• Real-time video: dynamic 3DGS (4D scene).

💻 코드 패턴 (Code Patterns)

Train (gsplat / official)

# Install gsplat (NeRF Studio 의 backend)
pip install gsplat

# Run nerfstudio
ns-train splatfacto --data ./images
ns-render --load-config outputs/.../config.yml

PyTorch (개념)

class GaussianModel(torch.nn.Module):
    def __init__(self, num_points):
        super().__init__()
        # Trainable parameters
        self._xyz = nn.Parameter(torch.randn(num_points, 3))
        self._scales = nn.Parameter(torch.ones(num_points, 3))   # log scale
        self._rotations = nn.Parameter(torch.zeros(num_points, 4))  # quaternion
        self._opacity = nn.Parameter(torch.zeros(num_points, 1))    # logit
        self._features_dc = nn.Parameter(torch.zeros(num_points, 3))   # SH 0
        self._features_rest = nn.Parameter(torch.zeros(num_points, 15, 3))  # SH 1-3
    
    def get_covariance(self):
        S = torch.diag_embed(torch.exp(self._scales))
        R = quaternion_to_matrix(F.normalize(self._rotations, dim=-1))
        return R @ S @ S.transpose(-2, -1) @ R.transpose(-2, -1)

# Train loop
def train_step(gaussians, image_gt, camera):
    rendered = differentiable_rasterize(gaussians, camera)
    loss = (rendered - image_gt).abs().mean()
    
    # Densification heuristic
    if step > 500 and step % 100 == 0:
        densify(gaussians, gradient_threshold=2e-4)
        prune(gaussians, opacity_threshold=0.005)
    
    return loss

Differentiable rasterization (CUDA kernel)

__global__ void rasterize_kernel(
    const float3* means_2d, const float* cov_2d, const float* alphas, const float3* colors,
    int W, int H, float* output_color
) {
    int tile_x = blockIdx.x;
    int tile_y = blockIdx.y;
    int px = threadIdx.x + tile_x * TILE_W;
    int py = threadIdx.y + tile_y * TILE_H;
    
    float T = 1.0;
    float3 C = make_float3(0, 0, 0);
    
    for (int i = 0; i < N_GAUSSIANS; i++) {
        if (T < 1e-4) break;   // saturate
        
        float2 d = make_float2(px - means_2d[i].x, py - means_2d[i].y);
        float power = -0.5 * (d.x * d.x * cov_2d[i*4+0] + d.y * d.y * cov_2d[i*4+3] + 2 * d.x * d.y * cov_2d[i*4+1]);
        float alpha = min(0.99, alphas[i] * exp(power));
        
        if (alpha < 1.0/255) continue;
        
        C += T * alpha * colors[i];
        T *= (1 - alpha);
    }
    
    output_color[py * W + px] = C;
}

WebGPU (real-time)

const sortPipeline = device.createComputePipeline({
  layout: 'auto',
  compute: { module: sortShaderModule, entryPoint: 'main' },
});

// Per-frame
const pass = encoder.beginComputePass();
pass.setPipeline(sortPipeline);
pass.dispatchWorkgroups(numTiles);
pass.end();

// Render
const renderPass = encoder.beginRenderPass({...});
renderPass.draw(numGaussians * 4);   // quad per gaussian
renderPass.end();

Three.js / Babylon.js plugin

npm i @mkkellogg/gaussian-splats-3d

import { GaussianSplats3D } from '@mkkellogg/gaussian-splats-3d';

const viewer = new GaussianSplats3D.Viewer({
  splatRenderMode: GaussianSplats3D.SplatRenderMode.ThreeD,
});
viewer.addSplatScene('./scene.ply').then(() => {
  viewer.start();
});

→ Drop-in WebGL viewer.

🤔 의사결정 기준 (Decision Criteria)

상황	추천
실시간 web viewer	3DGS + WebGPU
고품질 photogrammetry	3DGS (NeRF 보다 빠름)
매우 큰 scene	3DGS + culling
Editing / animation	3DGS (per-primitive)
Implicit field 필요	NeRF
작은 메모리	NeRF (MLP)
Dynamic scene	4DGS / dynamic 3DGS
Mobile / AR	Compressed 3DGS

기본값: 3DGS (real-time + 고품질). NeRF 는 specific (작은 메모리, implicit query) case.

⚠️ 모순 및 업데이트 (Contradictions & Updates)

• 저장 size: 1 scene = 100MB-1GB. Compression 가 active research (codec for splats).
• Dynamic scene: 옛 = static 만. 모던 = 4DGS, Dynamic-Gaussian (시간 차원 추가).
• Editing: NeRF 보다 좋음. 하지만 매 primitive 의 manual edit = 어려움. AI editor (segment + manipulate).
• License: 원래 paper 의 code = non-commercial. gsplat / 다른 implementation 가 MIT.
• Mobile performance: 매 platform 의 GPU 차이. iPhone 가 OK, low-end Android 가 부족.

🔗 지식 연결 (Graph)

• Adjacent: Point-Cloud
• Tools: gsplat · NeRF Studio · Brush · Splatfacto · Polycam · Luma AI
• Web: WebGPU · WebGL · Three.js · Babylonjs

🤖 LLM 활용 힌트 (How to Use This Knowledge)

언제 이 지식을 쓰는가:

• 실시간 3D web viewer 디자인 (NeRF 의 alternative).
• Photogrammetry pipeline 의 modern (drone → 3D model).
• VR / AR 의 실제 환경 reconstruction.
• 게임 의 background environment (LOD 의 modern).
• Self-driving 의 simulation environment.

언제 쓰면 안 되는가:

• 정확한 mesh / triangle 필요 (CAD, 3D printing) — explicit mesh.
• 매우 작은 메모리 budget (mobile, embedded) — implicit / compressed.
• Animation / rigging — traditional skeletal animation.
• Procedural generation — primitive-based 가 비효율.
• Light simulation (path tracing) — radiance field 가 더 적절.

❌ 안티패턴 (Anti-Patterns)

• WebGL 만 + JS sort: production 의 frame budget 깨짐.
• No densification heuristic: 매 area 의 detail 부족.
• Pruning 안 함: 메모리 폭발 (거의 invisible primitive 누적).
• 고정 SH degree: low-frequency light 가 충분 가, high-frequency 가 손실.
• Camera pose 가 부정확 (SfM 약): 학습 의 quality 깨짐.
• Train data 의 view 가 적은 area: hole / artifact.
• Compression 의 quality eval 없음: silent quality loss.

🧪 검증 상태 (Validation)

• 정보 상태: verified (concept-level)
• 출처 신뢰도: B (SIGGRAPH 2023 paper, gsplat documentation, web 의 다양한 implementation).
• 검토 이유: Manual cleanup. 매 specific number / benchmark 가 implementation / hardware 의존.

🧬 중복 검사 (Duplicate Check)

• 기존 유사 문서: NeRF-Neural-Radiance-Fields (alternative tech), Neural-Rendering (parent), WebGPU (deployment).
• 처리 방식: KEEP (distinct technique).
• 처리 이유: 3DGS 가 NeRF 의 explicit alternative. 매 own document.

🕓 변경 이력 (Changelog)

날짜	변경 내용	처리 방식	신뢰도
2026-05-08	P-Reinforce Phase 1 정규화	UPDATE	A
2026-05-09	Manual cleanup — code pattern + math + 결정 기준 + 안티패턴 추가, tag 정리	UPDATE	B

🔗 관련 문서 (자동 연결)

• Aerospace Flight Simulation