2nd/10_Wiki/Topics/AI_and_ML/3D Gaussian Splatting (3DGS).md

---
id: wiki-2026-0508-3d-gaussian-splatting-3dgs
title: 3D Gaussian Splatting (3DGS)
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [3DGS, Gaussian Splatting, 3D-GS, splatting, NeRF alternative]
duplicate_of: none
source_trust_level: B
confidence_score: 0.85
verification_status: conceptual
tags: [3d-rendering, neural-rendering, gaussian-splatting, computer-graphics, real-time, webgpu, novel-view-synthesis]
raw_sources: []
last_reinforced: 2026-05-09
github_commit: pending
inferred_by: Claude Opus 4.7 (manual cleanup 2026-05-09)
tech_stack:
  language: CUDA / Python / WGSL
  framework: PyTorch / WebGPU / Three.js
---

# 3D Gaussian Splatting (3DGS)

## 📌 한 줄 통찰 (The Karpathy Summary)
> **Implicit field (NeRF) 가 아닌 explicit primitive (millions of 3D Gaussian) 으로 scene 표현**. Real-time rendering (60+ FPS) + 고품질 + differentiable 학습. NeRF 의 modern 후계자.

## 📖 구조화된 지식 (Synthesized Content)

### 핵심 idea
- 매 scene 가 수백만 개 의 **anisotropic 3D Gaussian** 으로 표현.
- 매 Gaussian = (position, covariance, opacity, color via spherical harmonics).
- Camera 시점 의 projection → 2D ellipse → alpha-blend.
- Differentiable → photogrammetry image 로 train.

### NeRF 와 비교
| | NeRF | 3DGS |
|---|---|---|
| 표현 | Implicit (MLP) | Explicit (primitive) |
| Train 시간 | 수 시간-day | 수십 분 |
| Render 시간 | 수 sec / frame | < 17 ms / frame |
| 메모리 | 작은 (MLP) | 큰 (primitive 별 byte) |
| 편집 | 어려움 | Per-primitive 가능 |
| GPU | A100 | RTX 3090 충분 |

→ 2023+ 의 favorite (Real-time 가 큰 win).

### Pipeline
1. **SfM (Structure from Motion)**: Image → camera pose + sparse point cloud (COLMAP).
2. **Initialization**: Sparse point → Gaussian (position 의 init).
3. **Differentiable Rasterization**: 2D project + alpha blend.
4. **Optimization**: Gradient descent on (pos, scale, rot, opacity, color).
5. **Densification**: 큰 gradient 가 split / clone (detail ↑).
6. **Pruning**: 작은 opacity = remove.

### Math (간단)
매 Gaussian:
- Mean μ ∈ ℝ³
- Covariance Σ = R S Sᵀ Rᵀ (R = quaternion, S = scale).
- Opacity α ∈ [0, 1]
- Color = SH coefficient (view-dependent).

2D projection:
- Σ' = J W Σ Wᵀ Jᵀ (W = view, J = projection Jacobian).
- 2D Gaussian → tile → per-pixel α blend.

Render:
C = Σᵢ cᵢ αᵢ Πⱼ<ᵢ (1 - αⱼ)

→ Order-dependent (depth sort).

### Implementation: WebGPU
WebGL 의 한계:
- Compute shader X.
- 매 frame 의 sort 가 CPU (JS / WASM) → 느림.

WebGPU:
- Compute shader 가 sort GPU.
- Wait-free radix sort.
- Atomics + storage buffer.

WebSplatter (2024+):
- 매 frame 의 sort + render = GPU only.
- 4.5x faster 보다 WebGL-based.

### 응용
- **Photogrammetry / 3D scan**: drone capture → 3DGS 모델.
- **VR / AR**: 실제 환경 의 immersive view (Meta Reality Labs).
- **Game engine**: Unity / Unreal 의 plugin.
- **Self-driving simulation**: 실제 거리 의 train environment.
- **Cultural heritage**: 박물관 의 360 view.
- **Real-time video**: dynamic 3DGS (4D scene).

## 💻 코드 패턴 (Code Patterns)

### Train (gsplat / official)
```bash
# Install gsplat (NeRF Studio 의 backend)
pip install gsplat

# Run nerfstudio
ns-train splatfacto --data ./images
ns-render --load-config outputs/.../config.yml
```

### PyTorch (개념)
```python
class GaussianModel(torch.nn.Module):
    def __init__(self, num_points):
        super().__init__()
        # Trainable parameters
        self._xyz = nn.Parameter(torch.randn(num_points, 3))
        self._scales = nn.Parameter(torch.ones(num_points, 3))   # log scale
        self._rotations = nn.Parameter(torch.zeros(num_points, 4))  # quaternion
        self._opacity = nn.Parameter(torch.zeros(num_points, 1))    # logit
        self._features_dc = nn.Parameter(torch.zeros(num_points, 3))   # SH 0
        self._features_rest = nn.Parameter(torch.zeros(num_points, 15, 3))  # SH 1-3
    
    def get_covariance(self):
        S = torch.diag_embed(torch.exp(self._scales))
        R = quaternion_to_matrix(F.normalize(self._rotations, dim=-1))
        return R @ S @ S.transpose(-2, -1) @ R.transpose(-2, -1)

# Train loop
def train_step(gaussians, image_gt, camera):
    rendered = differentiable_rasterize(gaussians, camera)
    loss = (rendered - image_gt).abs().mean()
    
    # Densification heuristic
    if step > 500 and step % 100 == 0:
        densify(gaussians, gradient_threshold=2e-4)
        prune(gaussians, opacity_threshold=0.005)
    
    return loss
```

### Differentiable rasterization (CUDA kernel)
```cuda
__global__ void rasterize_kernel(
    const float3* means_2d, const float* cov_2d, const float* alphas, const float3* colors,
    int W, int H, float* output_color
) {
    int tile_x = blockIdx.x;
    int tile_y = blockIdx.y;
    int px = threadIdx.x + tile_x * TILE_W;
    int py = threadIdx.y + tile_y * TILE_H;
    
    float T = 1.0;
    float3 C = make_float3(0, 0, 0);
    
    for (int i = 0; i < N_GAUSSIANS; i++) {
        if (T < 1e-4) break;   // saturate
        
        float2 d = make_float2(px - means_2d[i].x, py - means_2d[i].y);
        float power = -0.5 * (d.x * d.x * cov_2d[i*4+0] + d.y * d.y * cov_2d[i*4+3] + 2 * d.x * d.y * cov_2d[i*4+1]);
        float alpha = min(0.99, alphas[i] * exp(power));
        
        if (alpha < 1.0/255) continue;
        
        C += T * alpha * colors[i];
        T *= (1 - alpha);
    }
    
    output_color[py * W + px] = C;
}
```

### WebGPU (real-time)
```typescript
const sortPipeline = device.createComputePipeline({
  layout: 'auto',
  compute: { module: sortShaderModule, entryPoint: 'main' },
});

// Per-frame
const pass = encoder.beginComputePass();
pass.setPipeline(sortPipeline);
pass.dispatchWorkgroups(numTiles);
pass.end();

// Render
const renderPass = encoder.beginRenderPass({...});
renderPass.draw(numGaussians * 4);   // quad per gaussian
renderPass.end();
```

### Three.js / Babylon.js plugin
```bash
npm i @mkkellogg/gaussian-splats-3d
```

```typescript
import { GaussianSplats3D } from '@mkkellogg/gaussian-splats-3d';

const viewer = new GaussianSplats3D.Viewer({
  splatRenderMode: GaussianSplats3D.SplatRenderMode.ThreeD,
});
viewer.addSplatScene('./scene.ply').then(() => {
  viewer.start();
});
```

→ Drop-in WebGL viewer.

## 🤔 의사결정 기준 (Decision Criteria)

| 상황 | 추천 |
|---|---|
| 실시간 web viewer | 3DGS + WebGPU |
| 고품질 photogrammetry | 3DGS (NeRF 보다 빠름) |
| 매우 큰 scene | 3DGS + culling |
| Editing / animation | 3DGS (per-primitive) |
| Implicit field 필요 | NeRF |
| 작은 메모리 | NeRF (MLP) |
| Dynamic scene | 4DGS / dynamic 3DGS |
| Mobile / AR | Compressed 3DGS |

**기본값**: 3DGS (real-time + 고품질). NeRF 는 specific (작은 메모리, implicit query) case.

## ⚠️ 모순 및 업데이트 (Contradictions & Updates)
- **저장 size**: 1 scene = 100MB-1GB. Compression 가 active research (codec for splats).
- **Dynamic scene**: 옛 = static 만. 모던 = 4DGS, Dynamic-Gaussian (시간 차원 추가).
- **Editing**: NeRF 보다 좋음. 하지만 매 primitive 의 manual edit = 어려움. AI editor (segment + manipulate).
- **License**: 원래 paper 의 code = non-commercial. gsplat / 다른 implementation 가 MIT.
- **Mobile performance**: 매 platform 의 GPU 차이. iPhone 가 OK, low-end Android 가 부족.

## 🔗 지식 연결 (Graph)
- Adjacent: [[Point-Cloud]]
- Tools: gsplat · NeRF Studio · Brush · Splatfacto · Polycam · Luma AI
- Web: [[WebGPU]] · [[WebGL]] · [[Three.js]] · [[Babylonjs]]

## 🤖 LLM 활용 힌트 (How to Use This Knowledge)

**언제 이 지식을 쓰는가:**
- 실시간 3D web viewer 디자인 (NeRF 의 alternative).
- Photogrammetry pipeline 의 modern (drone → 3D model).
- VR / AR 의 실제 환경 reconstruction.
- 게임 의 background environment (LOD 의 modern).
- Self-driving 의 simulation environment.

**언제 쓰면 안 되는가:**
- 정확한 mesh / triangle 필요 (CAD, 3D printing) — explicit mesh.
- 매우 작은 메모리 budget (mobile, embedded) — implicit / compressed.
- Animation / rigging — traditional skeletal animation.
- Procedural generation — primitive-based 가 비효율.
- Light simulation (path tracing) — radiance field 가 더 적절.

## ❌ 안티패턴 (Anti-Patterns)
- **WebGL 만 + JS sort**: production 의 frame budget 깨짐.
- **No densification heuristic**: 매 area 의 detail 부족.
- **Pruning 안 함**: 메모리 폭발 (거의 invisible primitive 누적).
- **고정 SH degree**: low-frequency light 가 충분 가, high-frequency 가 손실.
- **Camera pose 가 부정확 (SfM 약)**: 학습 의 quality 깨짐.
- **Train data 의 view 가 적은 area**: hole / artifact.
- **Compression 의 quality eval 없음**: silent quality loss.

## 🧪 검증 상태 (Validation)
- **정보 상태:** verified (concept-level)
- **출처 신뢰도:** B (SIGGRAPH 2023 paper, gsplat documentation, web 의 다양한 implementation).
- **검토 이유:** Manual cleanup. 매 specific number / benchmark 가 implementation / hardware 의존.

## 🧬 중복 검사 (Duplicate Check)
- **기존 유사 문서:** [[NeRF-Neural-Radiance-Fields]] (alternative tech), [[Neural-Rendering]] (parent), [[WebGPU]] (deployment).
- **처리 방식:** KEEP (distinct technique).
- **처리 이유:** 3DGS 가 NeRF 의 explicit alternative. 매 own document.

## 🕓 변경 이력 (Changelog)
| 날짜 | 변경 내용 | 처리 방식 | 신뢰도 |
|------|-----------|-----------|--------|
| 2026-05-08 | P-Reinforce Phase 1 정규화 | UPDATE | A |
| 2026-05-09 | Manual cleanup — code pattern + math + 결정 기준 + 안티패턴 추가, tag 정리 | UPDATE | B |

<!-- AUTO-CONNECT 2026-06-10 -->
## 🔗 관련 문서 (자동 연결)
- [[Aerospace Flight Simulation]]