2nd/10_Wiki/Topic_Programming/AI_and_ML/Extended-Reality-XR.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-extended-reality-xr	Extended Reality (XR)	10_Wiki/Topics	verified	self	XR VR AR MR spatial computing Vision Pro Quest WebXR	none	A	0.96	applied	xr vr ar mr spatial-computing vision-pro quest webxr		2026-05-10	pending	language framework Unity / Swift / WebXR Unity XR / RealityKit / WebXR / OpenXR

Extended Reality (XR)

매 한 줄

"매 VR + AR + MR 의 umbrella". 매 modern: 매 Apple Vision Pro (2024), Meta Quest 3, Magic Leap 2. 매 Apple "spatial computing" rebrand. 매 ML / AI integration: 매 hand tracking, scene understanding, foundation 3D model.

매 핵심

매 spectrum

• VR (immersive): 매 fully virtual.
• AR (overlay): 매 real + digital.
• MR (mixed): 매 anchored interaction.
• Spatial computing (Apple): 매 ambient.

매 modern device

• Apple Vision Pro (2024): 매 4K micro-OLED, R1 chip.
• Meta Quest 3 / Pro: 매 standalone, color passthrough.
• Magic Leap 2 (enterprise).
• HoloLens 2 (Microsoft).
• PSVR2.

매 standard

• OpenXR: 매 cross-vendor.
• WebXR: 매 browser-native.
• GLTF / USDZ: 매 3D format.
• Spatial anchor.
• Hand tracking.

매 AI integration

• Hand tracking (MediaPipe, Apple ARKit).
• Scene understanding (semantic).
• Eye tracking (foveated rendering).
• Avatars (Codec, persona).
• Foundation 3D (NeRF, Gaussian Splatting).
• AI agents in space.

매 응용

1. Gaming: 매 immersive.
2. Training: 매 surgery, military.
3. Productivity: 매 multi-monitor in space.
4. Telepresence: 매 avatar meeting.
5. Therapy: 매 exposure, PTSD.
6. Design: 매 CAD review.
7. Education: 매 anatomy, history.

💻 패턴

Unity XR (cross-platform)

using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;

public class GrabSpawn : MonoBehaviour {
    public XRGrabInteractable interactable;
    void Start() {
        interactable.selectEntered.AddListener(OnGrab);
    }
    void OnGrab(SelectEnterEventArgs e) { ... }
}

WebXR (browser)

import * as THREE from 'three';
import { ARButton } from 'three/addons/webxr/ARButton.js';

const renderer = new THREE.WebGLRenderer({ alpha: true });
renderer.xr.enabled = true;
document.body.appendChild(ARButton.createButton(renderer));
renderer.setAnimationLoop(frame => {
  renderer.render(scene, camera);
});

visionOS (Swift)

import RealityKit
import SwiftUI

struct ContentView: View {
    var body: some View {
        RealityView { content in
            let model = try await ModelEntity(named: "scene.usdz")
            content.add(model)
        }
        .gesture(SpatialTapGesture().onEnded { _ in
            handleTap()
        })
    }
}

Hand tracking (Quest, Unity)

using Oculus.Interaction.Input;

public class HandTrack : MonoBehaviour {
    public IHand hand;
    void Update() {
        if (hand.GetFingerIsPinching(HandFinger.Index)) {
            performAction();
        }
    }
}

Spatial anchor (persistent)

async Task<bool> SaveAnchor(OVRSpatialAnchor anchor) {
    var saved = await anchor.SaveAsync();
    return saved.Success;
}

async Task LoadAnchors(IEnumerable<Guid> uuids) {
    var loadOp = await OVRSpatialAnchor.LoadUnboundAnchorsAsync(uuids);
    foreach (var unbound in loadOp.Value) {
        unbound.LocalizeAsync(new GameObject().AddComponent<OVRSpatialAnchor>());
    }
}

Foveated rendering

// 매 eye-tracked foveated rendering on Quest Pro
OVRPlugin.foveatedRenderingLevel = OVRPlugin.FoveatedRenderingLevel.High;
OVRPlugin.useDynamicFoveatedRendering = true;

Gaussian Splatting (NeRF alternative)

# 매 INRIA SIBR-style
import gaussian_splatting as gs
scene = gs.Scene.from_images('captures/')
scene.train(iterations=30000)
scene.export('output.splat')
# 매 viewer 의 60fps WebGL

Scene understanding (Apple ARKit)

let config = ARWorldTrackingConfiguration()
config.sceneReconstruction = .meshWithClassification
config.planeDetection = [.horizontal, .vertical]
arSession.run(config)

// 매 ARMeshAnchor 의 classifications: floor, wall, table, ceiling, ...

Avatar (Codec-style)

class CodecAvatar:
    def __init__(self):
        self.expression = ExpressionEncoder()
        self.audio = AudioFeatureExtractor()
    
    def render(self, video_frame, audio_chunk):
        expr = self.expression(video_frame)
        speech = self.audio(audio_chunk)
        return decode_avatar(expr, speech)

Spatial UI (visionOS)

WindowGroup { ContentView() }
    .windowStyle(.volumetric)
    .defaultSize(width: 1, height: 1, depth: 1, in: .meters)

ImmersiveSpace(id: "ImmersiveSpace") {
    ImmersiveView()
}

Locomotion (comfort)

// 매 teleport 의 nausea-friendly
public class Teleport : MonoBehaviour {
    void OnSelect() {
        var hit = RaycastFromController();
        if (hit.collider) {
            FadeToBlack(0.2f);
            transform.position = hit.point;
            FadeFromBlack(0.2f);
        }
    }
}

Performance (90fps target)

// 매 PC VR: 90fps target. 매 standalone: 72-120fps
[ExecuteAlways]
public class FrameMonitor : MonoBehaviour {
    void Update() {
        if (Time.deltaTime > 1f / 80f) {
            Debug.LogWarning($"Frame drop: {1/Time.deltaTime}fps");
        }
    }
}

MR passthrough toggle

OVRPassthroughLayer passthrough;
void OnAR() { passthrough.enabled = true; }
void OnVR() { passthrough.enabled = false; }

매 결정 기준

상황	Platform
Mass market mobile AR	WebXR + ARKit/ARCore
Premium spatial	visionOS
Gaming / fitness	Quest 3
Enterprise	Magic Leap / HoloLens
Cross-platform	OpenXR + Unity
Web	WebXR + Three.js

기본값: 매 Unity OpenXR + 매 hand tracking + 매 spatial anchor + 매 60+fps + 매 comfort first.

🔗 Graph

• 부모: Computer-Graphics · HCI
• 변형: VR · AR · MR · Spatial Computing
• 응용: Vision-Pro · Quest · WebXR
• Adjacent: Gaussian-Splatting · NeRF · Embodied-AI

🤖 LLM 활용

언제: 매 immersive product. 매 training simulation. 매 design. 언제 X: 매 simple 2D suffices.

❌ 안티패턴

• Below 60fps: 매 nausea.
• Tiny text: 매 readability fail.
• No comfort options: 매 motion sickness.
• Ignore battery: 매 30min limit.
• Heavy passthrough: 매 thermal.

🧪 검증 / 중복

• Verified (Apple visionOS, Meta Quest docs, OpenXR spec).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-04-26	XR auto
2026-05-08	Phase 1
2026-05-10	Manual cleanup — VR/AR/MR + 매 Unity / WebXR / visionOS / Quest code