[G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00
parent 21ac3ed255
commit 504fd5fb42
3011 changed files with 380280 additions and 206977 deletions
@@ -0,0 +1,324 @@
+---
+id: testing-stryker-mutation
+title: Mutation Testing — Stryker / PIT / Mutmut
+category: Coding
+status: draft
+source_trust_level: B
+verification_status: conceptual
+created_at: 2026-05-09
+updated_at: 2026-05-09
+tags: [testing, mutation, vibe-coding]
+tech_stack: { language: "TS / Java / Python", applicable_to: ["Testing"] }
+applied_in: []
+aliases: [mutation testing, Stryker, PIT, Mutmut, mutation score, killed mutant, survived mutant]
+---
+
+# Mutation Testing
+
+> Code coverage 100% ≠ test 가 좋다. **Mutation: 코드 의 작은 변경 → test 가 잡으면 OK**. Stryker (JS/TS), PIT (Java), Mutmut (Python).
+
+## 📖 핵심 개념
+- Coverage: 실행된 line 비율.
+- Mutation: 의도적 bug 주입 후 test pass = bad test.
+- Killed mutant: test 가 잡음.
+- Survived: test 가 못 잡음 → bad test.
+
+## 💻 코드 패턴
+
+### Stryker (JS / TS)
+```bash
+yarn add -D @stryker-mutator/core @stryker-mutator/jest-runner
+```
+
+```js
+// stryker.conf.js
+module.exports = {
+  testRunner: 'jest',
+  coverageAnalysis: 'perTest',
+  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
+  thresholds: { high: 80, low: 60, break: 50 },
+};
+```
+
+```bash
+yarn stryker run
+```
+
+### Output
+```
+Mutation testing report
+=======================
+Files: src/utils/math.ts
+Mutants tested: 25
+- Killed:    18 (72%)
+- Survived:   5 (20%)
+- Timeout:    1
+- No coverage: 1
+- Total time: 45 sec
+
+Mutation score: 72% (warning, threshold 80%)
+```
+
+### Mutator 종류
+```ts
+// Original
+if (x > 5) return 'big';
+
+// Mutants:
+if (x > 5) return 'big';     // ❌ → if (x >= 5)
+if (x > 5) return 'big';     // ❌ → if (x < 5)
+if (x > 5) return 'big';     // ❌ → if (true)
+if (x > 5) return 'big';     // ❌ → return null
+```
+
+→ 매 statement 가 N variant. Test 가 매 variant 잡아야.
+
+### Stryker mutators
+```
+- Arithmetic: + → -, * → /
+- Boolean: true → false
+- Equality: === → !==
+- Comparison: > → >=, < → <=
+- Conditional: if → if (true)
+- Logical: && → ||
+- Update: i++ → i--
+- Block: return X → return null
+- ...
+```
+
+### Survived mutant 의 의미
+```ts
+// src/math.ts
+export function add(a: number, b: number) {
+  return a + b;
+}
+
+// test
+test('add', () => {
+  expect(add(2, 2)).toBe(4);
+});
+
+// Stryker mutates: a + b → a - b
+// Test: 2 - 2 = 0, not 4 → killed ✓
+
+// Mutates: a + b → a * b
+// Test: 2 * 2 = 4 → SURVIVED ❌ (test 가 multiplication 도 통과)
+```
+
+→ 더 다양 input 필요: `expect(add(2, 3)).toBe(5)`.
+
+### Coverage vs mutation score
+```
+Coverage 100% + Mutation 30% = test 가 부드러움.
+- "Code 가 실행" 만 검증
+- "Code 가 정확" 검증 X
+
+→ Mutation score 가 진짜 quality.
+```
+
+### Performance
+```
+Stryker = N x test 시간 (N = mutant 수).
+1000 mutant × 10s test = 10000s (3 hour).
+
+→ Incremental + parallel.
+```
+
+```js
+{
+  incremental: true,    // 변경 file 만
+  concurrency: 4,
+}
+```
+
+### CI integration
+```yaml
+- run: yarn stryker run --incremental
+- run: |
+    if [[ $MUTATION_SCORE -lt 70 ]]; then
+      echo "Mutation score too low"
+      exit 1
+    fi
+```
+
+→ 매 PR (incremental) 검증.
+
+### Threshold
+```
+break: 60   — 미만 = CI fail
+low: 70     — warning
+high: 80    — green
+
+매 PR 기준: 변경 file 의 mutation score.
+```
+
+### Equivalent mutant
+```ts
+// Original
+for (let i = 0; i < n; i++) { ... }
+
+// Mutant
+for (let i = 0; i <= n - 1; i++) { ... }   // 같은 동작
+```
+
+→ Mutant 가 same behavior = test 가 못 잡아도 OK.
+False positive. Manually mark.
+
+### Mutation 의 한계
+```
+- 큰 codebase = 매우 느림.
+- Equivalent mutant.
+- Configuration overhead.
+- Sometimes test 보다 코드 변경 가 더 단순.
+
+→ Critical code 만 mutation test.
+일반 = unit test + coverage 충분.
+```
+
+### When?
+```
+✓ Critical algorithm (math, encryption, finance)
+✓ Library author (모든 use case)
+✓ Test suite quality 검증
+✓ Onboarding (신규 사람 가 test quality 봄)
+
+✗ UI / E2E
+✗ Glue code
+✗ 매일 (cost)
+```
+
+### Subset (file 별)
+```bash
+yarn stryker run --mutate "src/critical/**/*.ts"
+```
+
+→ 핵심만 매주. 전체 = 매월.
+
+### PIT (Java)
+```xml
+<plugin>
+  <groupId>org.pitest</groupId>
+  <artifactId>pitest-maven</artifactId>
+  <configuration>
+    <targetClasses><param>com.example.*</param></targetClasses>
+    <mutationThreshold>80</mutationThreshold>
+  </configuration>
+</plugin>
+```
+
+```bash
+mvn org.pitest:pitest-maven:mutationCoverage
+```
+
+### Mutmut (Python)
+```bash
+pip install mutmut
+mutmut run --paths-to-mutate src/
+mutmut results
+```
+
+### Cosmic Ray (Python alt)
+```bash
+cosmic-ray init config.toml session
+cosmic-ray exec session
+cosmic-ray dump session > result.json
+```
+
+### Mutation testing 의 비판
+```
+"Test 의 quality > 정량 measure".
+-1 character variation 이 의미 가 다양:
+- "잘못 잡음" 가 useless test 의미일 수.
+- "잡음" 가 over-specification 일 수.
+
+→ 100% mutation = brittle test.
+```
+
+→ Sweet spot: 70-80%.
+
+### Practical workflow
+```
+1. Coverage 80%+ 먼저.
+2. Mutation 가 baseline 측정.
+3. Survived mutant 분석 → 매 mutant 가:
+   - 진짜 missing test? → 추가.
+   - Equivalent? → mark + ignore.
+   - Over-test 가능? → review.
+4. Score 가 80%+ 가 목표.
+```
+
+### IDE integration
+```
+- Stryker VS Code extension
+- Survived mutant 가 file 에 highlight
+- 매 mutation 의 specific change visible
+```
+
+### Visual report
+```
+Stryker → HTML report:
+- File 별 mutation score.
+- Survived mutant 의 정확 line.
+- Test 가 호출 한 mutant 추적.
+```
+
+### Combined with property-based test
+```ts
+import { fc } from 'fast-check';
+
+test('add commutative', () => {
+  fc.assert(fc.property(fc.integer(), fc.integer(), (a, b) => {
+    expect(add(a, b)).toBe(add(b, a));
+  }));
+});
+```
+
+→ Property test + mutation = 매우 강.
+
+### LLM 활용
+```
+Survived mutant 가 LLM 에:
+"Test add should kill: a + b → a * b. 새 test case 작성"
+
+→ Test gap 가 LLM 가 자동 채움.
+```
+
+### Cost analysis
+```
+1 시간 = $2 (CI).
+매월 10 hour = $20.
+→ Bug 가 prod 에 1 = $1000s.
+
+→ Critical code 면 가치.
+```
+
+## 🤔 의사결정 기준
+| 상황 | 추천 |
+|---|---|
+| Critical algorithm | Mutation |
+| Library author | Mutation |
+| 일반 app | Coverage 만 |
+| 큰 codebase | Subset / incremental |
+| 정확 검증 | Property + mutation |
+| CI gate | Threshold 70-80% |
+| 작은 PR | Incremental |
+
+## ❌ 안티패턴
+- **Coverage 100% = 좋아 가정**: mutation 가 진실.
+- **모든 file mutation**: 너무 느림.
+- **Equivalent mutant 무시**: false positive.
+- **매일 mutation**: cost 폭발.
+- **Threshold 100%**: brittle.
+- **Survived 안 분석**: 의미 없음.
+- **만 mutation, no coverage**: 기본 안 됨.
+
+## 🤖 LLM 활용 힌트
+- Mutation > coverage 의 quality measure.
+- Stryker (JS), PIT (Java), Mutmut (Python).
+- Critical code 만 — full = 비용.
+- Property + mutation 가 강력 조합.
+
+## 🔗 관련 문서
+- [[Testing_Test_Pyramid]]
+- [[Testing_Property_Based]]
+- [[Testing_Mocking_Boundaries]]