[G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00
parent 21ac3ed255
commit 504fd5fb42
3011 changed files with 380280 additions and 206977 deletions
@@ -0,0 +1,356 @@
+---
+id: ai-self-reflection-deep
+title: Self-Reflection — agent 가 자기 답 평가
+category: Coding
+status: draft
+source_trust_level: B
+verification_status: conceptual
+created_at: 2026-05-09
+updated_at: 2026-05-09
+tags: [ai, agents, reflection, vibe-coding]
+tech_stack: { language: "TS / Python", applicable_to: ["AI"] }
+applied_in: []
+aliases: [self-reflection, critique, ReAct, Reflexion, chain-of-thought, self-consistency]
+---
+
+# Self-Reflection
+
+> Agent 가 출력 후 "이게 좋아?" 자기 검토. **Critique → revise loop. Reflexion / self-consistency / verifier**. Cost ↑, quality ↑.
+
+## 📖 핵심 개념
+- 1 shot answer = bug.
+- Critique 가 자기 자신.
+- Iterate (revise) 가 quality ↑.
+- Cost (N x LLM) 가 가치 가져야.
+
+## 💻 코드 패턴
+
+### Self-critique
+```ts
+async function withReflection(task: string, maxRounds = 3) {
+  let solution = await llm.complete({ prompt: task });
+  
+  for (let i = 0; i < maxRounds; i++) {
+    const critique = await llm.complete({
+      system: 'Find issues with this answer. If perfect, say "OK".',
+      prompt: `Task: ${task}\n\nAnswer: ${solution}`,
+    });
+    
+    if (critique.toLowerCase().startsWith('ok')) return solution;
+    
+    solution = await llm.complete({
+      system: 'Improve based on feedback.',
+      prompt: `Task: ${task}\n\nAnswer: ${solution}\n\nIssues: ${critique}`,
+    });
+  }
+  
+  return solution;
+}
+```
+
+→ 매 round = 2x LLM call.
+
+### Reflexion (memory + reflection)
+```python
+class Reflexion:
+    def __init__(self):
+        self.memory = []  # 과거 reflection
+    
+    def solve(self, task):
+        for attempt in range(3):
+            solution = self.llm(task, context=self.memory)
+            
+            # Test
+            ok = self.test(solution)
+            if ok:
+                return solution
+            
+            # Reflect on failure
+            reflection = self.llm(f'Why did this fail? task={task}, solution={solution}')
+            self.memory.append(reflection)
+        
+        return solution
+```
+
+→ 다음 attempt 가 reflection 활용.
+
+### Self-consistency (voting)
+```ts
+async function selfConsistency(task: string, n: number = 5) {
+  const answers = await Promise.all(
+    Array(n).fill(0).map(() => llm.complete({ prompt: task, temperature: 0.7 }))
+  );
+  
+  // Majority vote
+  const counts = new Map<string, number>();
+  for (const a of answers) {
+    counts.set(a, (counts.get(a) ?? 0) + 1);
+  }
+  
+  return [...counts.entries()].sort((a, b) => b[1] - a[1])[0][0];
+}
+```
+
+→ 5 answer + vote. Hallucination 줄임. 5x cost.
+
+### Chain of thought (CoT)
+```ts
+const prompt = `
+Question: ${q}
+
+Think step by step:
+1. ...
+2. ...
+3. ...
+
+Answer: ...
+`;
+```
+
+→ Implicit reflection. Reasoning 가 visible.
+
+### Tree of Thought (ToT)
+```
+Q: 24 game (사칙 연산 → 24)
+
+Tree:
+- Branch 1: 8 + 16 = 24? — 8 어디서?
+  - Sub: 6 + 2 = 8 → 8 + 16 = 24 ✓
+- Branch 2: 12 × 2 = 24
+  - ...
+
+→ 매 step 가 가능 path 다 탐색 + 평가 + 가장 좋은 선택.
+```
+
+### Verifier (separate model)
+```ts
+async function verify(solution: string, task: string) {
+  const r = await verifierLLM.complete({
+    system: 'Rate solution 0-10. Return JSON: {score, reason}.',
+    prompt: `Task: ${task}\nSolution: ${solution}`,
+  });
+  return JSON.parse(r.text);
+}
+
+// 5 candidates → verifier 가 best 선택
+const candidates = await Promise.all([...]);
+const scores = await Promise.all(candidates.map(c => verify(c, task)));
+const best = candidates[indexOfMax(scores.map(s => s.score))];
+```
+
+### Reward model (RLHF 식)
+```python
+# Trained model 가 score
+reward_model = load_model('reward')
+
+candidates = [llm(task) for _ in range(8)]
+scores = reward_model.predict(candidates)
+best = candidates[scores.argmax()]
+```
+
+→ OpenAI o1 / DeepSeek R1 가 RL 식.
+
+### Test execution (code)
+```ts
+async function withTest(prompt: string) {
+  for (let i = 0; i < 3; i++) {
+    const code = await llm.complete({ prompt });
+    
+    const result = await runInSandbox(code);
+    if (result.ok) return code;
+    
+    prompt = `${prompt}\n\nPrevious attempt:\n${code}\n\nError: ${result.error}\nFix it.`;
+  }
+}
+```
+
+→ Test 가 ground truth. LLM critique 보다 강.
+
+### LLM-as-judge
+```ts
+async function judge(answerA: string, answerB: string, task: string) {
+  const r = await llm.complete({
+    system: 'Compare. Return "A", "B", or "tie".',
+    prompt: `Task: ${task}\nA: ${answerA}\nB: ${answerB}`,
+  });
+  return r.text;
+}
+```
+
+→ Eval framework 가 사용.
+
+### Self-refine (iter improve)
+```
+Loop:
+  Generate → Feedback → Refine
+
+Stop:
+- N round
+- "OK" feedback
+- Quality plateau
+```
+
+```ts
+async function selfRefine(task: string) {
+  let answer = await llm.complete({ prompt: task });
+  let prevQuality = -1;
+  
+  while (true) {
+    const feedback = await llm.complete({
+      prompt: `Task: ${task}\nAnswer: ${answer}\n\nFeedback (concrete improvements):`,
+    });
+    
+    const refined = await llm.complete({
+      prompt: `Task: ${task}\nAnswer: ${answer}\nFeedback: ${feedback}\n\nRefined:`,
+    });
+    
+    const quality = await rate(refined);
+    if (quality <= prevQuality) break;  // plateau
+    
+    answer = refined;
+    prevQuality = quality;
+  }
+  
+  return answer;
+}
+```
+
+### CRITIC (with tool)
+```python
+# Self-critique 가 fact verify 와 결합
+def critique_with_tool(answer):
+    claims = extract_claims(answer)
+    for c in claims:
+        result = web_search(c)
+        if not consistent(c, result):
+            return False, c
+    return True, None
+```
+
+→ "내가 한 답 가 Internet 와 다름" 감지.
+
+### Confidence calibration
+```ts
+async function answerWithConfidence(q: string) {
+  const r = await llm.complete({
+    system: 'Answer + 0-1 confidence (calibrated).',
+    prompt: q,
+  });
+  return JSON.parse(r.text);
+  // { answer: '...', confidence: 0.7 }
+}
+
+// Low confidence → search / ask user
+if (r.confidence < 0.5) {
+  // Search / clarify
+}
+```
+
+→ 자기 모름 = visible.
+
+### Cost
+```
+1 shot:        $0.05
+Critique:      $0.10 (2x)
+Reflexion 3 round: $0.30 (6x)
+Self-consistency 5: $0.25 (5x)
+ToT 3-deep:    $0.50+ (10x)
+
+→ Quality 가 cost 정당화 ?
+```
+
+### Latency
+```
+Sequential (critique + revise): 매 round 가 2 LLM = 5 sec.
+Parallel (self-consistency): 1 round.
+
+→ User-facing = parallel.
+```
+
+### When 사용?
+```
+✓ Math / 정확 task
+✓ Code generation
+✓ Research / multi-step
+✓ Critical (legal, medical)
+✗ Casual chat
+✗ Speed-sensitive
+✗ Cost-sensitive
+```
+
+### Evaluation
+```python
+# Reflection 가 정말 도움?
+without = [llm(q) for q in tasks]
+with_ = [reflection_loop(q) for q in tasks]
+
+acc_without = evaluate(without)
+acc_with = evaluate(with_)
+print(f'Improvement: {acc_with - acc_without:.2%}')
+```
+
+### 함정
+```
+- Critique 가 hallucinate (가짜 issue)
+- "OK" 빨리 끝남 (sycophancy)
+- 무한 loop (matter 없는 변경)
+- Cost 폭발
+- Slower
+```
+
+### OpenAI o1 / DeepSeek R1
+```
+RL trained 가 chain-of-thought.
+Auto reflection / verify.
+
+→ Single LLM call 안에서 자체 reflection.
+```
+
+### Anthropic 의 권장
+```
+"Strong base model + simple prompt > complex reflection on weak model".
+
+→ Reflection 의 가치 가 model size 에 비례 작아질 수.
+```
+
+### Multi-step verify
+```
+Q: "이 medical claim 가 정확?"
+
+Step 1: Identify claims.
+Step 2: Search recent literature.
+Step 3: Check consistency.
+Step 4: Verdict + citation.
+
+→ 명시적 step 가 implicit 보다 좋음.
+```
+
+## 🤔 의사결정 기준
+| 상황 | 추천 |
+|---|---|
+| 정확 critical | Reflection + verifier |
+| Math / code | Test execution |
+| Research | CoT + critique |
+| User-facing chat | 1 shot + retry on flag |
+| Voting | Self-consistency |
+| Search / decompose | ToT |
+| Modern LLM (o1/R1) | 자체 reflection |
+
+## ❌ 안티패턴
+- **Reflection 가 모든 task**: cost.
+- **무한 loop**: stop 조건 없음.
+- **Critique 신뢰 100%**: critic 도 fallible.
+- **Cost / latency 무시**: prod 못 씀.
+- **Eval 없음**: improvement 모름.
+- **약한 model + reflection**: 강 모델 1 shot 나음.
+
+## 🤖 LLM 활용 힌트
+- Reflection = quality ↑ + cost ↑.
+- Test (code) > LLM critique.
+- Self-consistency 가 voting 의 답.
+- 강 model (o1) 가 자체 reflection.
+
+## 🔗 관련 문서
+- [[AI_Agentic_Patterns]]
+- [[AI_LLM_Eval_Patterns]]
+- [[AI_Multi_Agent_Coordination]]