docs(10_Wiki): Topic_Business/General/Graphic/Programming을 Topics/ 하위로 이동

최상위 10_Wiki/Topic_*였던 4개 카테고리 폴더를 10_Wiki/Topics/Topic_* 로 재배치. 콘텐츠 변경 없음(순수 폴더 이동) — Topics/ 하위 나머지 폴더는 이미 지난 커밋에서 전부 정리된 상태(잔존 항목은 에이전트 운영 상태 및 사용자가 보존을 요청한 업데이트0615/무제 3.canvas 뿐).
2026-07-05 00:44:01 +09:00
parent e9cbf23ab5
commit 9a135bd19d
6127 changed files with 0 additions and 0 deletions
@@ -0,0 +1,159 @@
+---
+id: wiki-2026-0508-automation-paradox
+title: Automation Paradox
+category: 10_Wiki/Topics
+status: verified
+canonical_id: self
+aliases: [Paradox of Automation, Bainbridge's Ironies, Lights-Out Fallacy]
+duplicate_of: none
+source_trust_level: A
+confidence_score: 0.92
+verification_status: applied
+tags: [human-factors, automation, ai-safety, ergonomics, system-design]
+raw_sources: []
+last_reinforced: 2026-05-10
+github_commit: pending
+tech_stack:
+  language: n/a
+  framework: human-factors / HRO
+---
+
+# Automation Paradox
+
+## 매 한 줄
+> **"매 자동화 의 better 일수록, human operator 의 role 의 critical 의 increase — skill atrophy 와 vigilance decrement 의 통한 catastrophic edge-case 의 amplification"**. Lisanne Bainbridge (1983) "Ironies of Automation" 의 origin, 2026 의 LLM agent + autonomous vehicle + algorithmic trading 의 era 의 acute relevance.
+
+## 매 핵심
+
+### 매 4 ironies (Bainbridge)
+- **Designer 의 error**: automation 의 design 의 bug 의 operator 의 inherit
+- **Skill atrophy**: routine-task 의 takeover 의 인해 human skill 의 decay → emergency 의 unable
+- **Monitoring task**: vigilance 의 인해 의 unsuited 의 task 의 human 의 assign
+- **Trust calibration**: under-trust (rejection) 또는 over-trust (complacency) 의 binary failure
+
+### 매 mechanisms
+- **Out-of-the-loop unfamiliarity (OOTLUF)** — automation handover 의 시 의 operator 의 context 의 lack
+- **Mode confusion** — automation 의 current state 의 mismatch (Air France 447, Tesla autopilot)
+- **Skill decay curve** — manual skill 의 disuse 의 인해 의 exponential degradation (~6 months)
+- **Calibration drift** — automation 의 reliability 의 over-extrapolation
+
+### 매 응용
+1. Autonomous vehicle handover — Level 2/3 의 6-second take-over budget 의 unrealistic.
+2. LLM coding agent — generated code 의 review 의 automation bias 의 인해 의 bug 의 miss.
+3. Algorithmic trading kill-switch — flash-crash 의 인간 의 intervention 의 too late.
+4. Aviation glass cockpit — Air France 447 (2009) 의 stall 의 mode confusion.
+
+## 💻 패턴
+
+### Trust calibration metric (TLX-derived)
+```python
+from dataclasses import dataclass
+
+@dataclass
+class TrustCalibration:
+    perceived_reliability: float  # 0-1, operator estimate
+    actual_reliability: float     # 0-1, measured
+    
+    @property
+    def miscalibration(self) -> float:
+        """Positive => over-trust, negative => under-trust."""
+        return self.perceived_reliability - self.actual_reliability
+    
+    def risk_class(self) -> str:
+        gap = self.miscalibration
+        if gap > 0.15:  return "OVER_TRUST_DANGER"
+        if gap < -0.15: return "REJECTION_DANGER"
+        return "CALIBRATED"
+```
+
+### Vigilance decrement model
+```python
+import numpy as np
+
+def vigilance_curve(t_minutes: np.ndarray, base_hit_rate: float = 0.95) -> np.ndarray:
+    """Mackworth clock — 30-min decrement of ~0.2 in detection."""
+    decay = 0.2 * (1 - np.exp(-t_minutes / 15))
+    return np.clip(base_hit_rate - decay, 0, 1)
+
+# Recommendation: rotate operators every 20 min on monitoring tasks
+```
+
+### Handover protocol (autonomous vehicle)
+```python
+class L3HandoverManager:
+    def __init__(self, min_takeover_seconds: float = 12.0):
+        self.budget = min_takeover_seconds
+    
+    def request_handover(self, driver_state: dict) -> dict:
+        # 2026 SAE J3016 update: budget grew from 6s to 10-15s
+        if not driver_state["eyes_on_road"]:
+            return {"action": "MRM_pull_over", "reason": "OOTLUF"}
+        if driver_state["secondary_task"] == "phone":
+            return {"action": "MRM_pull_over", "reason": "high_OOTLUF_risk"}
+        return {"action": "alert_takeover", "budget_s": self.budget}
+```
+
+### LLM agent guardrail (skill-atrophy aware)
+```python
+class CodeReviewWithAutomationParadox:
+    """Force human active review on high-stakes diff to prevent atrophy."""
+    
+    def __init__(self, llm_client):
+        self.llm = llm_client
+    
+    def review(self, diff: str, stakes: str) -> str:
+        ai_review = self.llm.review(diff)
+        if stakes == "high":
+            # require human to manually annotate before showing AI review
+            human = prompt_for_independent_review(diff)
+            return reconcile(human, ai_review)
+        return ai_review
+```
+
+### Skill maintenance schedule
+```python
+def manual_practice_schedule(automation_uptime_pct: float) -> dict:
+    """Recommend periodic manual mode to combat skill decay."""
+    if automation_uptime_pct > 0.9:
+        return {"frequency": "weekly", "duration_min": 30}
+    if automation_uptime_pct > 0.7:
+        return {"frequency": "biweekly", "duration_min": 20}
+    return {"frequency": "monthly", "duration_min": 15}
+```
+
+## 매 결정 기준
+| 상황 | Approach |
+|---|---|
+| High-stakes + rare edge case | Keep human in loop, force periodic manual mode |
+| Routine + low risk | Full automation OK |
+| L2/L3 autonomy | Long handover budget (12s+), DMS (driver monitoring) |
+| LLM agent | Active review on critical paths, not passive accept |
+| HRO (high-reliability) | Multiple redundant operators, rotation |
+
+**기본값**: high-stakes automation 의 default — human-in-the-loop + periodic manual practice + miscalibration monitoring.
+
+## 🔗 Graph
+- 부모: [[Neuroergonomics]] · [[Complex Systems]]
+- 변형: [[Neuromuscular-Control]]
+- 응용: [[AI_Safety_and_Alignment|AI Safety]] · [[Multi-agent-System]]
+- Adjacent: [[Burnout]] · [[Continuous Obsolescence]]
+
+## 🤖 LLM 활용
+**언제**: AI agent system design, code review automation, autonomous system handover protocol.
+**언제 X**: stateless automation 의 인간 의 involvement 의 unnecessary 인 case.
+
+## ❌ 안티패턴
+- **"Lights-out" fallacy**: full automation 의 human 의 unnecessary 의 assume.
+- **6-second handover budget**: empirically insufficient — 12-15s baseline.
+- **Automation bias**: AI suggestion 의 default-accept — independent verification 의 missing.
+- **Skill decay 의 ignore**: emergency-only manual training 의 too late.
+
+## 🧪 검증 / 중복
+- Verified (Bainbridge 1983 *Ironies of Automation*; Parasuraman & Manzey 2010 *Complacency and Bias*).
+- 신뢰도 A.
+
+## 🕓 Changelog
+| 날짜 | 변경 |
+|---|---|
+| 2026-05-08 | Phase 1 |
+| 2026-05-10 | Manual cleanup — Bainbridge ironies, trust calibration, L3 handover, LLM agent guardrail |