2nd/10_Wiki/Topics/AI_and_ML/Fitness-Landscape.md

---
id: wiki-2026-0508-fitness-landscape
title: Fitness Landscape
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [fitness landscape, ruggedness, NK model, loss landscape, Wright, evolution]
duplicate_of: none
source_trust_level: A
confidence_score: 0.94
verification_status: applied
tags: [evolution, optimization, fitness-landscape, nk-model, loss-landscape, neuroevolution]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: NumPy / DEAP
---

# Fitness Landscape

## 매 한 줄
> **"매 genotype / parameter → 매 fitness / loss 의 mapping"**. Wright 1932. 매 ruggedness, 매 local optima, 매 plateau, 매 valley. 매 modern: 매 NK model, 매 DL loss landscape (flat vs sharp), 매 quality-diversity (MAP-Elites).

## 매 핵심

### 매 dimensions
- **Smooth**: 매 single peak.
- **Rugged**: 매 many local optima.
- **Neutral**: 매 plateau.
- **Holey**: 매 sharp valley.

### 매 model
- **NK model** (Kauffman): 매 N genes, K interactions.
- **HIFF**: 매 hierarchical.
- **Royal Road**.

### 매 ML / DL
- **Loss landscape** (Li 2018): 매 visualize.
- **Flat vs sharp minima** (Hochreiter): 매 generalization 의 affect.
- **Connectivity**: 매 minima 의 path.
- **SGD escape saddle**.

### 매 응용
1. **Evolution**: 매 search.
2. **Drug design**: 매 protein landscape.
3. **DL training**: 매 minimum quality.
4. **Hyperparameter**: 매 tune landscape.

## 💻 패턴

### NK model
```python
import numpy as np
def nk_fitness(genome, K, fitness_table):
    """매 N-bit genome, K-interaction."""
    N = len(genome)
    fitness = 0
    for i in range(N):
        # 매 i 의 K neighbors
        context = tuple(genome[i] for i in [(i + j) % N for j in range(K + 1)])
        fitness += fitness_table[i][context]
    return fitness / N

def init_nk(N, K):
    fitness_table = []
    for _ in range(N):
        fitness_table.append({tuple(np.random.randint(0, 2, K + 1)): np.random.rand() for _ in range(2 ** (K + 1))})
    return fitness_table
```

### Local search (hill climbing)
```python
def hill_climb(fitness_fn, init_genome, max_iter=1000):
    current = init_genome
    current_f = fitness_fn(current)
    for _ in range(max_iter):
        # 매 try flip 1 bit
        i = np.random.randint(len(current))
        neighbor = current.copy()
        neighbor[i] = 1 - neighbor[i]
        new_f = fitness_fn(neighbor)
        if new_f > current_f:
            current, current_f = neighbor, new_f
    return current, current_f
```

### Adaptive walks (count steps)
```python
def adaptive_walk_length(fitness_fn, N, n_starts=100):
    lengths = []
    for _ in range(n_starts):
        g = np.random.randint(0, 2, N)
        f = fitness_fn(g)
        steps = 0
        improved = True
        while improved:
            improved = False
            for i in range(N):
                neighbor = g.copy(); neighbor[i] = 1 - neighbor[i]
                if fitness_fn(neighbor) > f:
                    g, f = neighbor, fitness_fn(neighbor)
                    steps += 1; improved = True; break
        lengths.append(steps)
    return np.mean(lengths)  # 매 longer = smoother
```

### Ruggedness measure (correlation)
```python
def fitness_correlation(fitness_fn, N, distance, n_pairs=1000):
    """매 mutation 1 step → 매 correlation."""
    corrs = []
    for _ in range(n_pairs):
        g1 = np.random.randint(0, 2, N)
        g2 = g1.copy()
        for _ in range(distance):
            i = np.random.randint(N)
            g2[i] = 1 - g2[i]
        corrs.append((fitness_fn(g1), fitness_fn(g2)))
    f1, f2 = zip(*corrs)
    return np.corrcoef(f1, f2)[0, 1]
```

### DL loss landscape visualization
```python
import torch

def loss_landscape_2d(model, loss_fn, X, y, alpha=0.5, beta=0.5):
    """매 random direction 의 의 의 loss surface."""
    theta_orig = [p.data.clone() for p in model.parameters()]
    d1 = [torch.randn_like(p) for p in model.parameters()]
    d2 = [torch.randn_like(p) for p in model.parameters()]
    
    losses = np.zeros((20, 20))
    for i, a in enumerate(np.linspace(-alpha, alpha, 20)):
        for j, b in enumerate(np.linspace(-beta, beta, 20)):
            for p, t, e1, e2 in zip(model.parameters(), theta_orig, d1, d2):
                p.data = t + a * e1 + b * e2
            losses[i, j] = loss_fn(model(X), y).item()
    
    # 매 restore
    for p, t in zip(model.parameters(), theta_orig):
        p.data = t
    return losses
```

### Flat vs sharp
```python
def measure_flatness(model, loss_fn, X, y, eps=0.01):
    """매 small perturbation 의 의 의 loss 변화."""
    base_loss = loss_fn(model(X), y).item()
    perturbed_losses = []
    for _ in range(50):
        # 매 add noise
        for p in model.parameters():
            p.data += eps * torch.randn_like(p)
        perturbed_losses.append(loss_fn(model(X), y).item())
        for p in model.parameters():
            p.data -= eps * torch.randn_like(p)  # 매 wrong but illustrative
    return np.mean(perturbed_losses) - base_loss  # 매 small = flat
```

### Sharpness-Aware Minimization (SAM)
```python
class SAM(torch.optim.Optimizer):
    def __init__(self, params, base_optim, rho=0.05):
        super().__init__(params, dict())
        self.base = base_optim
        self.rho = rho
    
    def first_step(self):
        grad_norm = torch.norm(torch.stack([p.grad.norm() for p in self.param_groups[0]['params'] if p.grad is not None]))
        for p in self.param_groups[0]['params']:
            if p.grad is None: continue
            e_w = self.rho * p.grad / (grad_norm + 1e-12)
            p.data.add_(e_w)
            p.state['e_w'] = e_w
    
    def second_step(self):
        for p in self.param_groups[0]['params']:
            if 'e_w' in p.state:
                p.data.sub_(p.state['e_w'])
        self.base.step()
```

### Mode connectivity
```python
def connect_minima(model_a, model_b, alpha=0.5):
    """매 매 매 minima 의 path 의 loss."""
    interpolated = [(1 - alpha) * pa + alpha * pb 
                    for pa, pb in zip(model_a.parameters(), model_b.parameters())]
    # 매 set + eval
    return eval_loss(interpolated)
```

### Quality-diversity (MAP-Elites)
```python
def map_elites(fitness_fn, behavior_fn, n_iter=10000, behavior_dim=10):
    archive = {}  # 매 behavior cell → (genome, fitness)
    for _ in range(n_iter):
        if archive: parent = random.choice(list(archive.values()))[0]
        else: parent = random_genome()
        child = mutate(parent)
        f = fitness_fn(child)
        b = tuple(int(x * behavior_dim) for x in behavior_fn(child))
        if b not in archive or f > archive[b][1]:
            archive[b] = (child, f)
    return archive
```

### Population diversity (genotype)
```python
def diversity(population):
    """매 average pairwise distance."""
    n = len(population)
    return sum(np.sum(population[i] != population[j]) for i in range(n) for j in range(i+1, n)) / (n*(n-1)/2)
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Smooth landscape | Gradient |
| Rugged | GA + diversity |
| Neutral plateau | Random walk |
| DL training | SGD + SAM |
| Flat min | SAM, weight averaging |
| Quality-diversity | MAP-Elites |

**기본값**: 매 task-specific — 매 DL = SAM/SWA (flat min). 매 evolution = NK + diversity. 매 design = MAP-Elites.

## 🔗 Graph
- 부모: [[Optimization]] · [[Evolution]]
- 변형: [[Loss-Landscape]] · [[NK-Model]]
- 응용: [[Hyperparameters|Hyperparameter-Tuning]]
- Adjacent: [[SAM]]

## 🤖 LLM 활용
**언제**: 매 optimization research. 매 DL training analysis. 매 evolution.
**언제 X**: 매 single-shot deterministic.

## ❌ 안티패턴
- **Assume convex**: 매 rugged 의 ignore.
- **No diversity**: 매 stuck local opt.
- **Sharp min worship**: 매 generalization 의 lose.
- **Visualize 2D for 1B param**: 매 misleading.

## 🧪 검증 / 중복
- Verified (Wright 1932, Kauffman NK, Li 2018 loss landscape, Foret SAM 2021).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-04-26 | EVO-FIT auto |
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — NK / hill / ruggedness / loss / SAM / MAP-Elites code |