2nd/10_Wiki/Topics/Computer_Science_and_Theory/Simulated-Annealing.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-simulated-annealing	Simulated Annealing	10_Wiki/Topics	verified	self	SA Metropolis-Hastings Optimization	none	A	0.9	applied	optimization metaheuristic combinatorial stochastic		2026-05-10	pending	language framework python scipy/numpy

Simulated Annealing

매 한 줄

"매 hot exploration → cold exploitation.". 1983 Kirkpatrick et al. metallurgy annealing analogy로 도입. Temperature schedule이 acceptance probability 제어 → local optima 탈출. 2026 still production-grade for TSP, scheduling, hyperparam tuning when gradients unavailable.

매 핵심

매 Metropolis 기준

• ΔE ≤ 0 → always accept.
• ΔE > 0 → accept with probability exp(−ΔE / T).
• T → 0: greedy descent. T → ∞: random walk.

매 Cooling Schedules

• Linear: T_k = T_0 − αk (rare — too aggressive).
• Geometric: T_k = T_0 · α^k, α ∈ [0.85, 0.99] (most common).
• Logarithmic: T_k = c / log(k+2) (provable convergence, very slow).
• Adaptive: 매 reheat on stagnation.

매 응용

1. TSP / VRP 등 combinatorial optim.
2. Boltzmann machine training (historical).
3. Hyperparameter search w/ noisy objective.
4. Protein folding, circuit placement.

💻 패턴

Bare-bones SA

import math, random

def simulated_annealing(x0, energy, neighbor,
                         T0=1.0, alpha=0.95, n_iter=10_000):
    x, best = x0, x0
    e = e_best = energy(x0)
    T = T0
    for k in range(n_iter):
        x_new = neighbor(x)
        e_new = energy(x_new)
        de = e_new - e
        if de <= 0 or random.random() < math.exp(-de / T):
            x, e = x_new, e_new
            if e < e_best:
                best, e_best = x, e
        T *= alpha
    return best, e_best

TSP with 2-opt neighbor

def tour_length(tour, dist): 
    return sum(dist[tour[i]][tour[(i+1) % len(tour)]] 
               for i in range(len(tour)))

def two_opt(tour):
    i, j = sorted(random.sample(range(len(tour)), 2))
    return tour[:i] + tour[i:j+1][::-1] + tour[j+1:]

best_tour, _ = simulated_annealing(
    list(range(n)), 
    lambda t: tour_length(t, dist),
    two_opt, T0=10.0, alpha=0.999, n_iter=100_000
)

scipy dual_annealing (production)

from scipy.optimize import dual_annealing

def f(x): return (x[0]-3)**2 + (x[1]+1)**2 + 0.1*np.sin(10*x[0])

result = dual_annealing(f, bounds=[(-10,10), (-10,10)],
                        maxiter=2000, seed=42)
print(result.x, result.fun)

Adaptive reheating

def sa_with_reheat(x0, energy, neighbor, T0, alpha, n_iter, 
                   stall_limit=500):
    x, e_best = x0, energy(x0)
    T, stall = T0, 0
    for k in range(n_iter):
        x_new = neighbor(x); de = energy(x_new) - e_best
        if de <= 0 or random.random() < math.exp(-de/T):
            x = x_new
            if energy(x) < e_best:
                e_best = energy(x); stall = 0
            else: stall += 1
        else: stall += 1
        T *= alpha
        if stall > stall_limit:
            T = T0 * 0.5  # reheat
            stall = 0
    return x, e_best

Parallel tempering (multi-temperature)

def parallel_tempering(x0, energy, neighbor, 
                       Ts=[0.1, 0.5, 1.0, 5.0], n_iter=10_000):
    chains = [list(x0) for _ in Ts]
    energies = [energy(c) for c in chains]
    for k in range(n_iter):
        for i, T in enumerate(Ts):
            x_new = neighbor(chains[i])
            de = energy(x_new) - energies[i]
            if de <= 0 or random.random() < math.exp(-de/T):
                chains[i] = x_new; energies[i] = energy(x_new)
        # Swap adjacent temperatures
        i = random.randint(0, len(Ts)-2)
        de = (1/Ts[i] - 1/Ts[i+1]) * (energies[i+1] - energies[i])
        if random.random() < math.exp(-de):
            chains[i], chains[i+1] = chains[i+1], chains[i]
            energies[i], energies[i+1] = energies[i+1], energies[i]
    return chains[0]

매 결정 기준

상황	Approach
Smooth, differentiable	gradient methods (Adam, L-BFGS)
Combinatorial, no gradient	SA / Tabu / GA
Multi-modal, expensive eval	Bayesian optimization
Massive scale, structure	branch-and-bound + LP relaxation
Real-time low-iter budget	greedy + local search

기본값: SA via scipy.optimize.dual_annealing for continuous; custom 2-opt+SA for combinatorial.

🔗 Graph

• 부모: Optimization
• 응용: Combinatorial-Optimization · Hyperparameters
• Adjacent: Markov-Chain-Monte-Carlo · Evolutionary Biology

🤖 LLM 활용

언제: cooling schedule 추천, neighbor function 설계 brainstorm, convergence diagnosis. 언제 X: 매 high-iter SA 매 LLM 안에서 직접 실행 (use scipy locally).

❌ 안티패턴

• Cooling too fast: α=0.5 → 매 immediate freeze, no exploration.
• Cooling too slow: 매 budget 낭비, gradient method 가 빠름.
• Bad neighbor: 매 너무 큰 jump → random search. 매 너무 작 → local trap.
• No restart: single chain stuck → use parallel tempering / random restart.

🧪 검증 / 중복

• Verified (Kirkpatrick et al. 1983 Science; scipy docs 2025).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — Metropolis 기준, cooling schedules, TSP/parallel tempering 패턴