---
id: wiki-2026-0508-optimal-control-theory
title: Optimal Control Theory
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [OCT, Dynamic Optimization]
duplicate_of: none
source_trust_level: A
confidence_score: 0.9
verification_status: applied
tags: [control, optimization, dynamic-programming, mpc]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: CasADi/CVXPY
---

# Optimal Control Theory

## 매 한 줄
> **"매 minimize cost over trajectory subject to dynamics"**. 매 OCT는 매 Pontryagin (PMP), Bellman (DP/HJB), Calculus of Variations 의 세 lens 통합, 매 LQR / MPC / iLQR / DDP 가 매 핵심 algorithm, 매 2026 robotics·autonomous driving·aerospace·battery management·RL 의 매 underlying mathematical foundation.

## 매 핵심

### 매 formulation
- min_u ∫₀ᵀ L(x,u,t)dt + Φ(x(T))
- s.t. ẋ = f(x,u,t),  x(0)=x₀,  c(x,u) ≤ 0.

### 매 3 approaches
- **Variational / PMP**: Hamiltonian H = L + λᵀf, costate λ̇ = -∂H/∂x, optimality ∂H/∂u = 0.
- **Dynamic Programming**: V(x,t), Bellman / HJB ∂V/∂t + min_u[L + ∇Vᵀf] = 0.
- **Direct methods**: discretize → NLP (transcription).

### 매 LQR (linear-quadratic, 매 closed form)
- ẋ = Ax + Bu, J = ∫(xᵀQx + uᵀRu)dt.
- u* = -Kx, K = R⁻¹BᵀP.
- P from algebraic Riccati: AᵀP + PA - PBR⁻¹BᵀP + Q = 0.

### 매 MPC (receding horizon)
1. Solve N-step OCP at xₜ.
2. Apply only u₀.
3. Re-solve at xₜ₊₁.
4. Handles constraints + nonlinearity.

### 매 응용
1. Quadrotor / drone trajectory.
2. Autonomous driving (lane keep, overtake).
3. Battery / HVAC / grid optimization.
4. Manipulator motion planning.
5. RL connection (Bellman ↔ Q-learning).

## 💻 패턴

### Discrete LQR (steady-state)
```python
import numpy as np
from scipy.linalg import solve_discrete_are
def dlqr(A, B, Q, R):
    P = solve_discrete_are(A, B, Q, R)
    K = np.linalg.solve(R + B.T @ P @ B, B.T @ P @ A)
    return K, P
K, _ = dlqr(A, B, np.eye(n), 0.1*np.eye(m))
u = -K @ x
```

### iLQR for nonlinear
```python
def ilqr(f, l, lf, x0, u_init, n_iter=50):
    u = u_init.copy(); T = len(u)
    for it in range(n_iter):
        # Forward rollout
        x = [x0]
        for t in range(T): x.append(f(x[-1], u[t]))
        # Backward pass: compute K, k
        Vx = lf_x(x[-1]); Vxx = lf_xx(x[-1])
        Ks = []; ks = []
        for t in reversed(range(T)):
            fx, fu = jacobians(f, x[t], u[t])
            lx, lu, lxx, luu, lux = quadratics(l, x[t], u[t])
            Qx  = lx  + fx.T @ Vx
            Qu  = lu  + fu.T @ Vx
            Qxx = lxx + fx.T @ Vxx @ fx
            Quu = luu + fu.T @ Vxx @ fu
            Qux = lux + fu.T @ Vxx @ fx
            K = -np.linalg.solve(Quu, Qux)
            k = -np.linalg.solve(Quu, Qu)
            Vx  = Qx + K.T @ Quu @ k + K.T @ Qu + Qux.T @ k
            Vxx = Qxx + K.T @ Quu @ K + K.T @ Qux + Qux.T @ K
            Ks.insert(0, K); ks.insert(0, k)
        # Forward update u with line search...
    return u
```

### Nonlinear MPC with CasADi
```python
import casadi as ca
N, dt = 20, 0.1
opti = ca.Opti()
X = opti.variable(nx, N+1); U = opti.variable(nu, N)
x0p = opti.parameter(nx)
opti.subject_to(X[:,0] == x0p)
cost = 0
for k in range(N):
    cost += ca.mtimes([X[:,k].T, Q, X[:,k]]) + ca.mtimes([U[:,k].T, R, U[:,k]])
    opti.subject_to(X[:,k+1] == X[:,k] + dt*dynamics(X[:,k], U[:,k]))
    opti.subject_to(opti.bounded(u_min, U[:,k], u_max))
opti.minimize(cost)
opti.solver('ipopt')
# at runtime:
opti.set_value(x0p, current_state); sol = opti.solve()
u_apply = sol.value(U[:,0])
```

### Direct collocation transcription
```python
# Trapezoidal: x_{k+1} = x_k + (dt/2)*(f(x_k,u_k) + f(x_{k+1},u_{k+1}))
# 매 NLP variables: [x_0..x_N, u_0..u_{N-1}], constraints: dynamics + bounds.
```

### HJB value iteration (small grids)
```python
def value_iter(grid, f, l, dt, gamma=0.99, n_iter=500):
    V = np.zeros(len(grid))
    for _ in range(n_iter):
        V_new = np.copy(V)
        for i, x in enumerate(grid):
            best = np.inf
            for u in U_set:
                x_next = x + dt*f(x, u)
                j = nearest(grid, x_next)
                best = min(best, dt*l(x,u) + gamma*V[j])
            V_new[i] = best
        V = V_new
    return V
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Linear, quadratic cost | LQR (closed form) |
| Mild nonlinear, no constraints | iLQR / DDP |
| Constraints + nonlinearity | Nonlinear MPC (collocation/multiple shooting) |
| Long horizon / stochastic | DP / RL (Q-learning, SAC) |
| Real-time embedded | Explicit MPC / parametric QP |
| Trajectory optimization offline | DIRCOL / SQP |

**기본값**: linear → LQR; nonlinear with constraints → CasADi + IPOPT MPC; 매 long-horizon stochastic → RL.

## 🔗 Graph
- 부모: [[Control-Theory]] · [[Optimization]]
- 응용: [[Robotics]] · [[Autonomous-Driving]]
- Adjacent: [[Reinforcement-Learning]] · [[Dynamic-Programming]] · [[데이터_사이언스_및_ML_엔지니어링|Bellman-Equation]]

## 🤖 LLM 활용
**언제**: cost-function design, MPC weight tuning rationale, Pontryagin/HJB derivation 매 explanation.
**언제 X**: real-time solving (CasADi/acados/HPIPM 매 사용).

## ❌ 안티패턴
- **LQR on nonlinear**: 매 large-deviation regime — 매 iLQR/MPC 매 사용.
- **No terminal cost / set in MPC**: 매 stability lost.
- **Wrong Q/R scaling**: 매 unit-mismatch — 매 normalize states first.
- **Ignoring constraints in CofV**: 매 PMP+constraint 매 KKT-style augmentation 매 필요.
- **Pure RL when MPC works**: 매 model-known + smooth → MPC 매 sample-efficient.

## 🧪 검증 / 중복
- Verified (Bertsekas "DP & OC", Bryson & Ho, Borrelli "Predictive Control").
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — LQR/iLQR/MPC/HJB unified |
