2nd/10_Wiki/Topics/AI_and_ML/Extended-Reality-XR.md

---
id: wiki-2026-0508-extended-reality-xr
title: Extended Reality (XR)
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [XR, VR, AR, MR, spatial computing, Vision Pro, Quest, WebXR]
duplicate_of: none
source_trust_level: A
confidence_score: 0.96
verification_status: applied
tags: [xr, vr, ar, mr, spatial-computing, vision-pro, quest, webxr]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Unity / Swift / WebXR
  framework: Unity XR / RealityKit / WebXR / OpenXR
---

# Extended Reality (XR)

## 매 한 줄
> **"매 VR + AR + MR 의 umbrella"**. 매 modern: 매 Apple Vision Pro (2024), Meta Quest 3, Magic Leap 2. 매 Apple "spatial computing" rebrand. 매 ML / AI integration: 매 hand tracking, scene understanding, foundation 3D model.

## 매 핵심

### 매 spectrum
- **VR** (immersive): 매 fully virtual.
- **AR** (overlay): 매 real + digital.
- **MR** (mixed): 매 anchored interaction.
- **Spatial computing** (Apple): 매 ambient.

### 매 modern device
- **Apple Vision Pro** (2024): 매 4K micro-OLED, R1 chip.
- **Meta Quest 3 / Pro**: 매 standalone, color passthrough.
- **Magic Leap 2** (enterprise).
- **HoloLens 2** (Microsoft).
- **PSVR2**.

### 매 standard
- **OpenXR**: 매 cross-vendor.
- **WebXR**: 매 browser-native.
- **GLTF / USDZ**: 매 3D format.
- **Spatial anchor**.
- **Hand tracking**.

### 매 AI integration
- **Hand tracking** (MediaPipe, Apple ARKit).
- **Scene understanding** (semantic).
- **Eye tracking** (foveated rendering).
- **Avatars** (Codec, persona).
- **Foundation 3D** (NeRF, Gaussian Splatting).
- **AI agents in space**.

### 매 응용
1. **Gaming**: 매 immersive.
2. **Training**: 매 surgery, military.
3. **Productivity**: 매 multi-monitor in space.
4. **Telepresence**: 매 avatar meeting.
5. **Therapy**: 매 exposure, PTSD.
6. **Design**: 매 CAD review.
7. **Education**: 매 anatomy, history.

## 💻 패턴

### Unity XR (cross-platform)
```csharp
using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;

public class GrabSpawn : MonoBehaviour {
    public XRGrabInteractable interactable;
    void Start() {
        interactable.selectEntered.AddListener(OnGrab);
    }
    void OnGrab(SelectEnterEventArgs e) { ... }
}
```

### WebXR (browser)
```javascript
import * as THREE from 'three';
import { ARButton } from 'three/addons/webxr/ARButton.js';

const renderer = new THREE.WebGLRenderer({ alpha: true });
renderer.xr.enabled = true;
document.body.appendChild(ARButton.createButton(renderer));
renderer.setAnimationLoop(frame => {
  renderer.render(scene, camera);
});
```

### visionOS (Swift)
```swift
import RealityKit
import SwiftUI

struct ContentView: View {
    var body: some View {
        RealityView { content in
            let model = try await ModelEntity(named: "scene.usdz")
            content.add(model)
        }
        .gesture(SpatialTapGesture().onEnded { _ in
            handleTap()
        })
    }
}
```

### Hand tracking (Quest, Unity)
```csharp
using Oculus.Interaction.Input;

public class HandTrack : MonoBehaviour {
    public IHand hand;
    void Update() {
        if (hand.GetFingerIsPinching(HandFinger.Index)) {
            performAction();
        }
    }
}
```

### Spatial anchor (persistent)
```csharp
async Task<bool> SaveAnchor(OVRSpatialAnchor anchor) {
    var saved = await anchor.SaveAsync();
    return saved.Success;
}

async Task LoadAnchors(IEnumerable<Guid> uuids) {
    var loadOp = await OVRSpatialAnchor.LoadUnboundAnchorsAsync(uuids);
    foreach (var unbound in loadOp.Value) {
        unbound.LocalizeAsync(new GameObject().AddComponent<OVRSpatialAnchor>());
    }
}
```

### Foveated rendering
```csharp
// 매 eye-tracked foveated rendering on Quest Pro
OVRPlugin.foveatedRenderingLevel = OVRPlugin.FoveatedRenderingLevel.High;
OVRPlugin.useDynamicFoveatedRendering = true;
```

### Gaussian Splatting (NeRF alternative)
```python
# 매 INRIA SIBR-style
import gaussian_splatting as gs
scene = gs.Scene.from_images('captures/')
scene.train(iterations=30000)
scene.export('output.splat')
# 매 viewer 의 60fps WebGL
```

### Scene understanding (Apple ARKit)
```swift
let config = ARWorldTrackingConfiguration()
config.sceneReconstruction = .meshWithClassification
config.planeDetection = [.horizontal, .vertical]
arSession.run(config)

// 매 ARMeshAnchor 의 classifications: floor, wall, table, ceiling, ...
```

### Avatar (Codec-style)
```python
class CodecAvatar:
    def __init__(self):
        self.expression = ExpressionEncoder()
        self.audio = AudioFeatureExtractor()
    
    def render(self, video_frame, audio_chunk):
        expr = self.expression(video_frame)
        speech = self.audio(audio_chunk)
        return decode_avatar(expr, speech)
```

### Spatial UI (visionOS)
```swift
WindowGroup { ContentView() }
    .windowStyle(.volumetric)
    .defaultSize(width: 1, height: 1, depth: 1, in: .meters)

ImmersiveSpace(id: "ImmersiveSpace") {
    ImmersiveView()
}
```

### Locomotion (comfort)
```csharp
// 매 teleport 의 nausea-friendly
public class Teleport : MonoBehaviour {
    void OnSelect() {
        var hit = RaycastFromController();
        if (hit.collider) {
            FadeToBlack(0.2f);
            transform.position = hit.point;
            FadeFromBlack(0.2f);
        }
    }
}
```

### Performance (90fps target)
```csharp
// 매 PC VR: 90fps target. 매 standalone: 72-120fps
[ExecuteAlways]
public class FrameMonitor : MonoBehaviour {
    void Update() {
        if (Time.deltaTime > 1f / 80f) {
            Debug.LogWarning($"Frame drop: {1/Time.deltaTime}fps");
        }
    }
}
```

### MR passthrough toggle
```csharp
OVRPassthroughLayer passthrough;
void OnAR() { passthrough.enabled = true; }
void OnVR() { passthrough.enabled = false; }
```

## 매 결정 기준
| 상황 | Platform |
|---|---|
| Mass market mobile AR | WebXR + ARKit/ARCore |
| Premium spatial | visionOS |
| Gaming / fitness | Quest 3 |
| Enterprise | Magic Leap / HoloLens |
| Cross-platform | OpenXR + Unity |
| Web | WebXR + Three.js |

**기본값**: 매 Unity OpenXR + 매 hand tracking + 매 spatial anchor + 매 60+fps + 매 comfort first.

## 🔗 Graph
- 부모: [[Computer-Graphics]] · [[HCI]]
- 변형: [[VR]] · [[AR]] · [[MR]] · [[Spatial Computing]]
- 응용: [[Vision-Pro]] · [[Quest]] · [[WebXR]]
- Adjacent: [[Gaussian-Splatting]] · [[NeRF]] · [[Embodied-AI]]

## 🤖 LLM 활용
**언제**: 매 immersive product. 매 training simulation. 매 design.
**언제 X**: 매 simple 2D suffices.

## ❌ 안티패턴
- **Below 60fps**: 매 nausea.
- **Tiny text**: 매 readability fail.
- **No comfort options**: 매 motion sickness.
- **Ignore battery**: 매 30min limit.
- **Heavy passthrough**: 매 thermal.

## 🧪 검증 / 중복
- Verified (Apple visionOS, Meta Quest docs, OpenXR spec).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-04-26 | XR auto |
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — VR/AR/MR + 매 Unity / WebXR / visionOS / Quest code |