2nd/10_Wiki/Topic_Programming/Frontend/Eye-Tracking.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-eye-tracking	Eye Tracking	10_Wiki/Topics	verified	self	Gaze Tracking Eye Gaze Foveated Rendering Input	none	A	0.85	applied	eye-tracking xr vr ar accessibility ux		2026-05-10	pending	language framework typescript webxr

Eye Tracking

매 한 줄

"매 eye tracking 은 user gaze direction 의 measure — pupil/cornea reflection 또는 ML model 의 사용". 2026 의 mainstream: Apple Vision Pro, Meta Quest 3S Pro, PSVR2. 매 foveated rendering, gaze-based UI selection, accessibility (ALS), UX research 의 enables.

매 핵심

매 Sensing Methods

• PCCR (Pupil Center Corneal Reflection): IR LED illumination + IR camera. Sub-degree accuracy, dedicated HW (Tobii, Vision Pro).
• Webcam-based ML: WebGazer.js, MediaPipe Iris. ~3-5° accuracy, no special HW.
• Electrooculography (EOG): skin electrodes; medical/lab use.
• Scleral search coil: highest precision, invasive (research only).

매 Output Data

• Gaze point: 2D screen coord or 3D ray.
• Pupil diameter: cognitive load proxy.
• Fixations: stable gaze (>100ms) — what user actually reads.
• Saccades: rapid eye movement between fixations.
• Smooth pursuit: tracking moving target.

매 응용

1. Foveated rendering (high-res only at gaze point — VR perf 4-10x).
2. Gaze + pinch UI selection (Apple Vision Pro paradigm).
3. Accessibility: gaze-typing for ALS/locked-in patients.
4. UX research: heatmaps, attention analysis.
5. Driver monitoring (drowsiness detection).

💻 패턴

WebGazer.js (browser, no HW)

import webgazer from 'webgazer';

await webgazer.setRegression('ridge')
  .setGazeListener((data, timestamp) => {
    if (!data) return;
    console.log('gaze:', data.x, data.y);  // screen px
  })
  .begin();

// User must calibrate by clicking around

MediaPipe Iris (TF.js)

import * as faceLandmarksDetection from '@tensorflow-models/face-landmarks-detection';

const detector = await faceLandmarksDetection.createDetector(
  faceLandmarksDetection.SupportedModels.MediaPipeFaceMesh,
  { runtime: 'tfjs', refineLandmarks: true }  // refine = iris landmarks
);

const faces = await detector.estimateFaces(video);
const irisLeft = faces[0].keypoints.filter(k => k.name?.includes('leftIris'));
const irisRight = faces[0].keypoints.filter(k => k.name?.includes('rightIris'));

WebXR Eye Tracking (Vision Pro Safari, Quest)

const session = await navigator.xr.requestSession('immersive-vr', {
  requiredFeatures: ['eye-tracking'],
});

session.requestAnimationFrame(function onFrame(t, frame) {
  const eyeSpace = session.requestReferenceSpace('viewer');
  const gaze = frame.getViewerPose(eyeSpace);
  // gaze.transform.position / orientation = gaze ray
  session.requestAnimationFrame(onFrame);
});

Fixation Detection (I-DT algorithm)

// Identification by Dispersion Threshold
function detectFixations(gazePoints: {x:number, y:number, t:number}[],
                        dispersionPx = 50, minDurationMs = 100) {
  const fixations = [];
  let window: typeof gazePoints = [];
  for (const p of gazePoints) {
    window.push(p);
    const xs = window.map(g => g.x), ys = window.map(g => g.y);
    const disp = (Math.max(...xs) - Math.min(...xs))
               + (Math.max(...ys) - Math.min(...ys));
    if (disp > dispersionPx) {
      const dur = window[window.length-2].t - window[0].t;
      if (dur >= minDurationMs) fixations.push(window.slice(0, -1));
      window = [p];
    }
  }
  return fixations;
}

Foveated Rendering Hint (WebGPU)

// VRS / foveated rendering via GPU extension (browser-dependent)
const passEncoder = encoder.beginRenderPass({
  colorAttachments: [...],
  // implementation-specific: shading rate texture pointing at gaze
});

Heatmap from Fixations

function buildHeatmap(fixations, w, h, sigma = 30) {
  const map = new Float32Array(w * h);
  for (const f of fixations) {
    for (let y = 0; y < h; y++) for (let x = 0; x < w; x++) {
      const d2 = (x-f.x)**2 + (y-f.y)**2;
      map[y*w+x] += Math.exp(-d2 / (2*sigma*sigma)) * f.duration;
    }
  }
  return map;
}

매 결정 기준

상황	Approach
VR/AR headset (2026)	Native eye-tracking API (visionOS, OpenXR ext)
Browser, accuracy critical	Tobii / dedicated HW + SDK
Browser, no HW	WebGazer.js or MediaPipe Iris (lower accuracy)
Accessibility input	Tobii Eye Tracker 5 + Windows Eye Control
UX research	Tobii Pro / GazePoint + heatmap tool

기본값: 매 platform 의 native eye-tracking API. Browser fallback: MediaPipe Iris.

🔗 Graph

• 부모: Human Computer Interaction · XR
• 응용: Accessibility (A11y)
• Adjacent: WebXR · MediaPipe

🤖 LLM 활용

언제: gaze-based UI 의 design, foveated rendering question, accessibility input. 언제 X: physiological eye-disease diagnosis (clinical 의 ophthalmologist).

❌ 안티패턴

• No calibration: webcam-based 의 every-user calibration 의 require.
• Click-on-gaze without dwell time: Midas touch problem — every glance triggers.
• Storing raw gaze without consent: PII / biometric (GDPR Art. 9).
• Webcam ML for safety-critical: insufficient accuracy (driver assist 의 dedicated HW).
• Ignoring head movement: gaze ≠ pupil position; head pose 의 compensate.

🧪 검증 / 중복

• Verified (Apple visionOS docs, OpenXR EXT_eye_gaze_interaction, Tobii Developer Zone).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — eye tracking full content