2nd/10_Wiki/Topics/Coding/Testing_Stryker_Mutation.md

---
id: testing-stryker-mutation
title: Mutation Testing — Stryker / PIT / Mutmut
category: Coding
status: draft
source_trust_level: B
verification_status: conceptual
created_at: 2026-05-09
updated_at: 2026-05-09
tags: [testing, mutation, vibe-coding]
tech_stack: { language: "TS / Java / Python", applicable_to: ["Testing"] }
applied_in: []
aliases: [mutation testing, Stryker, PIT, Mutmut, mutation score, killed mutant, survived mutant]
---

# Mutation Testing

> Code coverage 100% ≠ test 가 좋다. **Mutation: 코드 의 작은 변경 → test 가 잡으면 OK**. Stryker (JS/TS), PIT (Java), Mutmut (Python).

## 📖 핵심 개념
- Coverage: 실행된 line 비율.
- Mutation: 의도적 bug 주입 후 test pass = bad test.
- Killed mutant: test 가 잡음.
- Survived: test 가 못 잡음 → bad test.

## 💻 코드 패턴

### Stryker (JS / TS)
```bash
yarn add -D @stryker-mutator/core @stryker-mutator/jest-runner
```

```js
// stryker.conf.js
module.exports = {
  testRunner: 'jest',
  coverageAnalysis: 'perTest',
  mutate: ['src/**/*.ts', '!src/**/*.test.ts'],
  thresholds: { high: 80, low: 60, break: 50 },
};
```

```bash
yarn stryker run
```

### Output
```
Mutation testing report
=======================
Files: src/utils/math.ts
Mutants tested: 25
- Killed:    18 (72%)
- Survived:   5 (20%)
- Timeout:    1
- No coverage: 1
- Total time: 45 sec

Mutation score: 72% (warning, threshold 80%)
```

### Mutator 종류
```ts
// Original
if (x > 5) return 'big';

// Mutants:
if (x > 5) return 'big';     // ❌ → if (x >= 5)
if (x > 5) return 'big';     // ❌ → if (x < 5)
if (x > 5) return 'big';     // ❌ → if (true)
if (x > 5) return 'big';     // ❌ → return null
```

→ 매 statement 가 N variant. Test 가 매 variant 잡아야.

### Stryker mutators
```
- Arithmetic: + → -, * → /
- Boolean: true → false
- Equality: === → !==
- Comparison: > → >=, < → <=
- Conditional: if → if (true)
- Logical: && → ||
- Update: i++ → i--
- Block: return X → return null
- ...
```

### Survived mutant 의 의미
```ts
// src/math.ts
export function add(a: number, b: number) {
  return a + b;
}

// test
test('add', () => {
  expect(add(2, 2)).toBe(4);
});

// Stryker mutates: a + b → a - b
// Test: 2 - 2 = 0, not 4 → killed ✓

// Mutates: a + b → a * b
// Test: 2 * 2 = 4 → SURVIVED ❌ (test 가 multiplication 도 통과)
```

→ 더 다양 input 필요: `expect(add(2, 3)).toBe(5)`.

### Coverage vs mutation score
```
Coverage 100% + Mutation 30% = test 가 부드러움.
- "Code 가 실행" 만 검증
- "Code 가 정확" 검증 X

→ Mutation score 가 진짜 quality.
```

### Performance
```
Stryker = N x test 시간 (N = mutant 수).
1000 mutant × 10s test = 10000s (3 hour).

→ Incremental + parallel.
```

```js
{
  incremental: true,    // 변경 file 만
  concurrency: 4,
}
```

### CI integration
```yaml
- run: yarn stryker run --incremental
- run: |
    if [[ $MUTATION_SCORE -lt 70 ]]; then
      echo "Mutation score too low"
      exit 1
    fi
```

→ 매 PR (incremental) 검증.

### Threshold
```
break: 60   — 미만 = CI fail
low: 70     — warning
high: 80    — green

매 PR 기준: 변경 file 의 mutation score.
```

### Equivalent mutant
```ts
// Original
for (let i = 0; i < n; i++) { ... }

// Mutant
for (let i = 0; i <= n - 1; i++) { ... }   // 같은 동작
```

→ Mutant 가 same behavior = test 가 못 잡아도 OK.
False positive. Manually mark.

### Mutation 의 한계
```
- 큰 codebase = 매우 느림.
- Equivalent mutant.
- Configuration overhead.
- Sometimes test 보다 코드 변경 가 더 단순.

→ Critical code 만 mutation test.
일반 = unit test + coverage 충분.
```

### When?
```
✓ Critical algorithm (math, encryption, finance)
✓ Library author (모든 use case)
✓ Test suite quality 검증
✓ Onboarding (신규 사람 가 test quality 봄)

✗ UI / E2E
✗ Glue code
✗ 매일 (cost)
```

### Subset (file 별)
```bash
yarn stryker run --mutate "src/critical/**/*.ts"
```

→ 핵심만 매주. 전체 = 매월.

### PIT (Java)
```xml
<plugin>
  <groupId>org.pitest</groupId>
  <artifactId>pitest-maven</artifactId>
  <configuration>
    <targetClasses><param>com.example.*</param></targetClasses>
    <mutationThreshold>80</mutationThreshold>
  </configuration>
</plugin>
```

```bash
mvn org.pitest:pitest-maven:mutationCoverage
```

### Mutmut (Python)
```bash
pip install mutmut
mutmut run --paths-to-mutate src/
mutmut results
```

### Cosmic Ray (Python alt)
```bash
cosmic-ray init config.toml session
cosmic-ray exec session
cosmic-ray dump session > result.json
```

### Mutation testing 의 비판
```
"Test 의 quality > 정량 measure".
-1 character variation 이 의미 가 다양:
- "잘못 잡음" 가 useless test 의미일 수.
- "잡음" 가 over-specification 일 수.

→ 100% mutation = brittle test.
```

→ Sweet spot: 70-80%.

### Practical workflow
```
1. Coverage 80%+ 먼저.
2. Mutation 가 baseline 측정.
3. Survived mutant 분석 → 매 mutant 가:
   - 진짜 missing test? → 추가.
   - Equivalent? → mark + ignore.
   - Over-test 가능? → review.
4. Score 가 80%+ 가 목표.
```

### IDE integration
```
- Stryker VS Code extension
- Survived mutant 가 file 에 highlight
- 매 mutation 의 specific change visible
```

### Visual report
```
Stryker → HTML report:
- File 별 mutation score.
- Survived mutant 의 정확 line.
- Test 가 호출 한 mutant 추적.
```

### Combined with property-based test
```ts
import { fc } from 'fast-check';

test('add commutative', () => {
  fc.assert(fc.property(fc.integer(), fc.integer(), (a, b) => {
    expect(add(a, b)).toBe(add(b, a));
  }));
});
```

→ Property test + mutation = 매우 강.

### LLM 활용
```
Survived mutant 가 LLM 에:
"Test add should kill: a + b → a * b. 새 test case 작성"

→ Test gap 가 LLM 가 자동 채움.
```

### Cost analysis
```
1 시간 = $2 (CI).
매월 10 hour = $20.
→ Bug 가 prod 에 1 = $1000s.

→ Critical code 면 가치.
```

## 🤔 의사결정 기준
| 상황 | 추천 |
|---|---|
| Critical algorithm | Mutation |
| Library author | Mutation |
| 일반 app | Coverage 만 |
| 큰 codebase | Subset / incremental |
| 정확 검증 | Property + mutation |
| CI gate | Threshold 70-80% |
| 작은 PR | Incremental |

## ❌ 안티패턴
- **Coverage 100% = 좋아 가정**: mutation 가 진실.
- **모든 file mutation**: 너무 느림.
- **Equivalent mutant 무시**: false positive.
- **매일 mutation**: cost 폭발.
- **Threshold 100%**: brittle.
- **Survived 안 분석**: 의미 없음.
- **만 mutation, no coverage**: 기본 안 됨.

## 🤖 LLM 활용 힌트
- Mutation > coverage 의 quality measure.
- Stryker (JS), PIT (Java), Mutmut (Python).
- Critical code 만 — full = 비용.
- Property + mutation 가 강력 조합.

## 🔗 관련 문서
- [[Testing_Test_Pyramid]]
- [[Testing_Property_Based]]
- [[Testing_Mocking_Boundaries]]