2nd/10_Wiki/Topics/AI_and_ML/Fitness-Landscape.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-fitness-landscape	Fitness Landscape	10_Wiki/Topics	verified	self	fitness landscape ruggedness NK model loss landscape Wright evolution	none	A	0.94	applied	evolution optimization fitness-landscape nk-model loss-landscape neuroevolution		2026-05-10	pending	language framework Python NumPy / DEAP

Fitness Landscape

매 한 줄

"매 genotype / parameter → 매 fitness / loss 의 mapping". Wright 1932. 매 ruggedness, 매 local optima, 매 plateau, 매 valley. 매 modern: 매 NK model, 매 DL loss landscape (flat vs sharp), 매 quality-diversity (MAP-Elites).

매 핵심

매 dimensions

• Smooth: 매 single peak.
• Rugged: 매 many local optima.
• Neutral: 매 plateau.
• Holey: 매 sharp valley.

매 model

• NK model (Kauffman): 매 N genes, K interactions.
• HIFF: 매 hierarchical.
• Royal Road.

매 ML / DL

• Loss landscape (Li 2018): 매 visualize.
• Flat vs sharp minima (Hochreiter): 매 generalization 의 affect.
• Connectivity: 매 minima 의 path.
• SGD escape saddle.

매 응용

1. Evolution: 매 search.
2. Drug design: 매 protein landscape.
3. DL training: 매 minimum quality.
4. Hyperparameter: 매 tune landscape.

💻 패턴

NK model

import numpy as np
def nk_fitness(genome, K, fitness_table):
    """매 N-bit genome, K-interaction."""
    N = len(genome)
    fitness = 0
    for i in range(N):
        # 매 i 의 K neighbors
        context = tuple(genome[i] for i in [(i + j) % N for j in range(K + 1)])
        fitness += fitness_table[i][context]
    return fitness / N

def init_nk(N, K):
    fitness_table = []
    for _ in range(N):
        fitness_table.append({tuple(np.random.randint(0, 2, K + 1)): np.random.rand() for _ in range(2 ** (K + 1))})
    return fitness_table

Local search (hill climbing)

def hill_climb(fitness_fn, init_genome, max_iter=1000):
    current = init_genome
    current_f = fitness_fn(current)
    for _ in range(max_iter):
        # 매 try flip 1 bit
        i = np.random.randint(len(current))
        neighbor = current.copy()
        neighbor[i] = 1 - neighbor[i]
        new_f = fitness_fn(neighbor)
        if new_f > current_f:
            current, current_f = neighbor, new_f
    return current, current_f

Adaptive walks (count steps)

def adaptive_walk_length(fitness_fn, N, n_starts=100):
    lengths = []
    for _ in range(n_starts):
        g = np.random.randint(0, 2, N)
        f = fitness_fn(g)
        steps = 0
        improved = True
        while improved:
            improved = False
            for i in range(N):
                neighbor = g.copy(); neighbor[i] = 1 - neighbor[i]
                if fitness_fn(neighbor) > f:
                    g, f = neighbor, fitness_fn(neighbor)
                    steps += 1; improved = True; break
        lengths.append(steps)
    return np.mean(lengths)  # 매 longer = smoother

Ruggedness measure (correlation)

def fitness_correlation(fitness_fn, N, distance, n_pairs=1000):
    """매 mutation 1 step → 매 correlation."""
    corrs = []
    for _ in range(n_pairs):
        g1 = np.random.randint(0, 2, N)
        g2 = g1.copy()
        for _ in range(distance):
            i = np.random.randint(N)
            g2[i] = 1 - g2[i]
        corrs.append((fitness_fn(g1), fitness_fn(g2)))
    f1, f2 = zip(*corrs)
    return np.corrcoef(f1, f2)[0, 1]

DL loss landscape visualization

import torch

def loss_landscape_2d(model, loss_fn, X, y, alpha=0.5, beta=0.5):
    """매 random direction 의 의 의 loss surface."""
    theta_orig = [p.data.clone() for p in model.parameters()]
    d1 = [torch.randn_like(p) for p in model.parameters()]
    d2 = [torch.randn_like(p) for p in model.parameters()]
    
    losses = np.zeros((20, 20))
    for i, a in enumerate(np.linspace(-alpha, alpha, 20)):
        for j, b in enumerate(np.linspace(-beta, beta, 20)):
            for p, t, e1, e2 in zip(model.parameters(), theta_orig, d1, d2):
                p.data = t + a * e1 + b * e2
            losses[i, j] = loss_fn(model(X), y).item()
    
    # 매 restore
    for p, t in zip(model.parameters(), theta_orig):
        p.data = t
    return losses

Flat vs sharp

def measure_flatness(model, loss_fn, X, y, eps=0.01):
    """매 small perturbation 의 의 의 loss 변화."""
    base_loss = loss_fn(model(X), y).item()
    perturbed_losses = []
    for _ in range(50):
        # 매 add noise
        for p in model.parameters():
            p.data += eps * torch.randn_like(p)
        perturbed_losses.append(loss_fn(model(X), y).item())
        for p in model.parameters():
            p.data -= eps * torch.randn_like(p)  # 매 wrong but illustrative
    return np.mean(perturbed_losses) - base_loss  # 매 small = flat

Sharpness-Aware Minimization (SAM)

class SAM(torch.optim.Optimizer):
    def __init__(self, params, base_optim, rho=0.05):
        super().__init__(params, dict())
        self.base = base_optim
        self.rho = rho
    
    def first_step(self):
        grad_norm = torch.norm(torch.stack([p.grad.norm() for p in self.param_groups[0]['params'] if p.grad is not None]))
        for p in self.param_groups[0]['params']:
            if p.grad is None: continue
            e_w = self.rho * p.grad / (grad_norm + 1e-12)
            p.data.add_(e_w)
            p.state['e_w'] = e_w
    
    def second_step(self):
        for p in self.param_groups[0]['params']:
            if 'e_w' in p.state:
                p.data.sub_(p.state['e_w'])
        self.base.step()

Mode connectivity

def connect_minima(model_a, model_b, alpha=0.5):
    """매 매 매 minima 의 path 의 loss."""
    interpolated = [(1 - alpha) * pa + alpha * pb 
                    for pa, pb in zip(model_a.parameters(), model_b.parameters())]
    # 매 set + eval
    return eval_loss(interpolated)

Quality-diversity (MAP-Elites)

def map_elites(fitness_fn, behavior_fn, n_iter=10000, behavior_dim=10):
    archive = {}  # 매 behavior cell → (genome, fitness)
    for _ in range(n_iter):
        if archive: parent = random.choice(list(archive.values()))[0]
        else: parent = random_genome()
        child = mutate(parent)
        f = fitness_fn(child)
        b = tuple(int(x * behavior_dim) for x in behavior_fn(child))
        if b not in archive or f > archive[b][1]:
            archive[b] = (child, f)
    return archive

Population diversity (genotype)

def diversity(population):
    """매 average pairwise distance."""
    n = len(population)
    return sum(np.sum(population[i] != population[j]) for i in range(n) for j in range(i+1, n)) / (n*(n-1)/2)

매 결정 기준

상황	Approach
Smooth landscape	Gradient
Rugged	GA + diversity
Neutral plateau	Random walk
DL training	SGD + SAM
Flat min	SAM, weight averaging
Quality-diversity	MAP-Elites

기본값: 매 task-specific — 매 DL = SAM/SWA (flat min). 매 evolution = NK + diversity. 매 design = MAP-Elites.

🔗 Graph

• 부모: Optimization · Evolution
• 변형: Loss-Landscape · NK-Model
• 응용: Hyperparameters
• Adjacent: SAM

🤖 LLM 활용

언제: 매 optimization research. 매 DL training analysis. 매 evolution. 언제 X: 매 single-shot deterministic.

❌ 안티패턴

• Assume convex: 매 rugged 의 ignore.
• No diversity: 매 stuck local opt.
• Sharp min worship: 매 generalization 의 lose.
• Visualize 2D for 1B param: 매 misleading.

🧪 검증 / 중복

• Verified (Wright 1932, Kauffman NK, Li 2018 loss landscape, Foret SAM 2021).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-04-26	EVO-FIT auto
2026-05-08	Phase 1
2026-05-10	Manual cleanup — NK / hill / ruggedness / loss / SAM / MAP-Elites code