---
id: wiki-2026-0508-simulated-annealing
title: Simulated Annealing
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [SA, Metropolis-Hastings Optimization]
duplicate_of: none
source_trust_level: A
confidence_score: 0.9
verification_status: applied
tags: [optimization, metaheuristic, combinatorial, stochastic]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: scipy/numpy
---

# Simulated Annealing

## 매 한 줄
> **"매 hot exploration → cold exploitation."**. 1983 Kirkpatrick et al. metallurgy annealing analogy로 도입. Temperature schedule이 acceptance probability 제어 → local optima 탈출. 2026 still production-grade for TSP, scheduling, hyperparam tuning when gradients unavailable.

## 매 핵심

### 매 Metropolis 기준
- ΔE ≤ 0 → always accept.
- ΔE > 0 → accept with probability exp(−ΔE / T).
- T → 0: greedy descent. T → ∞: random walk.

### 매 Cooling Schedules
- **Linear**: T_k = T_0 − αk (rare — too aggressive).
- **Geometric**: T_k = T_0 · α^k, α ∈ [0.85, 0.99] (most common).
- **Logarithmic**: T_k = c / log(k+2) (provable convergence, very slow).
- **Adaptive**: 매 reheat on stagnation.

### 매 응용
1. TSP / VRP 등 combinatorial optim.
2. Boltzmann machine training (historical).
3. Hyperparameter search w/ noisy objective.
4. Protein folding, circuit placement.

## 💻 패턴

### Bare-bones SA
```python
import math, random

def simulated_annealing(x0, energy, neighbor,
                         T0=1.0, alpha=0.95, n_iter=10_000):
    x, best = x0, x0
    e = e_best = energy(x0)
    T = T0
    for k in range(n_iter):
        x_new = neighbor(x)
        e_new = energy(x_new)
        de = e_new - e
        if de <= 0 or random.random() < math.exp(-de / T):
            x, e = x_new, e_new
            if e < e_best:
                best, e_best = x, e
        T *= alpha
    return best, e_best
```

### TSP with 2-opt neighbor
```python
def tour_length(tour, dist): 
    return sum(dist[tour[i]][tour[(i+1) % len(tour)]] 
               for i in range(len(tour)))

def two_opt(tour):
    i, j = sorted(random.sample(range(len(tour)), 2))
    return tour[:i] + tour[i:j+1][::-1] + tour[j+1:]

best_tour, _ = simulated_annealing(
    list(range(n)), 
    lambda t: tour_length(t, dist),
    two_opt, T0=10.0, alpha=0.999, n_iter=100_000
)
```

### scipy dual_annealing (production)
```python
from scipy.optimize import dual_annealing

def f(x): return (x[0]-3)**2 + (x[1]+1)**2 + 0.1*np.sin(10*x[0])

result = dual_annealing(f, bounds=[(-10,10), (-10,10)],
                        maxiter=2000, seed=42)
print(result.x, result.fun)
```

### Adaptive reheating
```python
def sa_with_reheat(x0, energy, neighbor, T0, alpha, n_iter, 
                   stall_limit=500):
    x, e_best = x0, energy(x0)
    T, stall = T0, 0
    for k in range(n_iter):
        x_new = neighbor(x); de = energy(x_new) - e_best
        if de <= 0 or random.random() < math.exp(-de/T):
            x = x_new
            if energy(x) < e_best:
                e_best = energy(x); stall = 0
            else: stall += 1
        else: stall += 1
        T *= alpha
        if stall > stall_limit:
            T = T0 * 0.5  # reheat
            stall = 0
    return x, e_best
```

### Parallel tempering (multi-temperature)
```python
def parallel_tempering(x0, energy, neighbor, 
                       Ts=[0.1, 0.5, 1.0, 5.0], n_iter=10_000):
    chains = [list(x0) for _ in Ts]
    energies = [energy(c) for c in chains]
    for k in range(n_iter):
        for i, T in enumerate(Ts):
            x_new = neighbor(chains[i])
            de = energy(x_new) - energies[i]
            if de <= 0 or random.random() < math.exp(-de/T):
                chains[i] = x_new; energies[i] = energy(x_new)
        # Swap adjacent temperatures
        i = random.randint(0, len(Ts)-2)
        de = (1/Ts[i] - 1/Ts[i+1]) * (energies[i+1] - energies[i])
        if random.random() < math.exp(-de):
            chains[i], chains[i+1] = chains[i+1], chains[i]
            energies[i], energies[i+1] = energies[i+1], energies[i]
    return chains[0]
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Smooth, differentiable | gradient methods (Adam, L-BFGS) |
| Combinatorial, no gradient | SA / Tabu / GA |
| Multi-modal, expensive eval | Bayesian optimization |
| Massive scale, structure | branch-and-bound + LP relaxation |
| Real-time low-iter budget | greedy + local search |

**기본값**: SA via `scipy.optimize.dual_annealing` for continuous; custom 2-opt+SA for combinatorial.

## 🔗 Graph
- 부모: [[Optimization]]
- 응용: [[Combinatorial-Optimization]] · [[Hyperparameters|Hyperparameter-Tuning]]
- Adjacent: [[Markov-Chain-Monte-Carlo]] · [[Evolutionary Biology|Genetic-Algorithms]]

## 🤖 LLM 활용
**언제**: cooling schedule 추천, neighbor function 설계 brainstorm, convergence diagnosis.
**언제 X**: 매 high-iter SA 매 LLM 안에서 직접 실행 (use scipy locally).

## ❌ 안티패턴
- **Cooling too fast**: α=0.5 → 매 immediate freeze, no exploration.
- **Cooling too slow**: 매 budget 낭비, gradient method 가 빠름.
- **Bad neighbor**: 매 너무 큰 jump → random search. 매 너무 작 → local trap.
- **No restart**: single chain stuck → use parallel tempering / random restart.

## 🧪 검증 / 중복
- Verified (Kirkpatrick et al. 1983 *Science*; *scipy* docs 2025).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — Metropolis 기준, cooling schedules, TSP/parallel tempering 패턴 |