2nd/10_Wiki/Topics/AI_and_ML/Free-Energy-Principle.md

---
id: wiki-2026-0508-free-energy-principle
title: Free Energy Principle (FEP)
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [FEP, Karl Friston, active inference, predictive coding, surprise minimization]
duplicate_of: none
source_trust_level: A
confidence_score: 0.88
verification_status: applied
tags: [neuroscience, philosophy, fep, active-inference, predictive-coding, friston, bayesian]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: NumPy / pymdp
---

# Free Energy Principle (FEP)

## 매 한 줄
> **"매 self-organizing system 의 의 의 surprise (variational free energy) 의 minimize"**. Karl Friston (UCL). 매 perception (predictive coding) + 매 action (active inference) 의 unify. 매 neuroscience theory of everything (controversial). 매 modern: 매 RL / world model 의 connection.

## 매 핵심

### 매 statement
- 매 brain (and body) 의 의 의 statistical model.
- 매 free energy ≈ 매 prediction error.
- 매 minimize via:
  1. **Update belief** (perception).
  2. **Act** (active inference).

### 매 vs RL
- **RL**: 매 reward 의 maximize.
- **FEP**: 매 surprise 의 minimize.
- **Equivalence** (debated): 매 reward = -surprise.

### 매 component
- **Generative model**: 매 internal world model.
- **Prior**: 매 expectation.
- **Likelihood**: 매 sense → state.
- **Posterior**: 매 belief update.
- **Action**: 매 sense 의 expected 의 의 의 의 align.

### 매 응용
1. **Neuroscience**: 매 perception, attention, learning.
2. **Psychiatry**: 매 schizophrenia, autism (precision-weighting).
3. **Active inference RL**.
4. **Robotics** (curiosity, exploration).
5. **Computational psychiatry**.

### 매 critique
- 매 too general 의 의 의 unfalsifiable.
- 매 Lakatos progressive vs degenerate program.
- 매 implementation 의 specific 의 X.

## 💻 패턴

### Predictive coding (perception)
```python
import numpy as np

def predictive_coding_step(prior_mu, prior_var, observation, obs_var):
    """매 Kalman-like update."""
    posterior_var = 1 / (1 / prior_var + 1 / obs_var)
    posterior_mu = posterior_var * (prior_mu / prior_var + observation / obs_var)
    prediction_error = observation - prior_mu
    return posterior_mu, posterior_var, prediction_error
```

### Variational free energy
```python
def variational_free_energy(q_mu, q_var, p_mu, p_var, obs, obs_var):
    """매 F = KL[q||p] - log p(obs|s)."""
    kl = 0.5 * (np.log(p_var / q_var) + (q_var + (q_mu - p_mu)**2) / p_var - 1)
    log_likelihood = -0.5 * (np.log(2 * np.pi * obs_var) + (obs - q_mu)**2 / obs_var)
    return kl - log_likelihood
```

### Active inference (pymdp)
```python
import pymdp
from pymdp.agent import Agent

# 매 simple 1-state, 1-modality agent
A = np.eye(3)  # 매 likelihood (state → obs)
B = np.array([[[0.9, 0.1, 0], [0.1, 0.8, 0.1], [0, 0.1, 0.9]]])  # 매 transition (state, action)
C = np.array([0, 0, 1])  # 매 prior preference (high obs 3)

agent = Agent(A=A, B=B, C=C)
agent.infer_states(observation=[0])
action = agent.sample_action()
```

### Expected free energy (planning)
```python
def expected_free_energy(state_belief, action, A, B, C):
    """매 EFE = epistemic value + pragmatic value."""
    next_state = B[action] @ state_belief
    expected_obs = A @ next_state
    
    # 매 pragmatic (utility)
    pragmatic = (expected_obs * C).sum()
    
    # 매 epistemic (info gain)
    epistemic = -np.sum(expected_obs * np.log(expected_obs + 1e-9))
    
    return -pragmatic + epistemic  # 매 minimize
```

### Predictive coding network (PyTorch)
```python
import torch
import torch.nn as nn

class PCLayer(nn.Module):
    def __init__(self, n):
        super().__init__()
        self.W = nn.Parameter(torch.randn(n, n) * 0.01)
        self.x = nn.Parameter(torch.zeros(n))  # 매 latent state
    
    def forward(self, top_down, bottom_up):
        prediction = self.W @ self.x
        error_top = top_down - prediction
        error_bottom = bottom_up - self.x
        free_energy = (error_top ** 2).sum() + (error_bottom ** 2).sum()
        return free_energy
```

### Curiosity (intrinsic reward = -surprise)
```python
class IntrinsicCuriosity:
    def __init__(self, model):
        self.model = model
    
    def reward(self, state, action, next_state):
        predicted = self.model.predict(state, action)
        prediction_error = (predicted - next_state).abs().mean()
        return prediction_error  # 매 high error = surprise = curiosity
```

### Precision (attention)
```python
def precision_weighted_update(prediction_error, precision):
    """매 attention = 매 high precision 의 weight."""
    return precision * prediction_error
# 매 Friston: 매 schizophrenia = 매 precision 의 wrong
```

### World model (similar to FEP)
```python
class WorldModel(nn.Module):
    """매 Ha & Schmidhuber 2018."""
    def __init__(self, state_dim, action_dim):
        super().__init__()
        self.encoder = Encoder()  # 매 obs → latent
        self.dynamics = nn.LSTM(latent_dim + action_dim, latent_dim)
        self.decoder = Decoder()  # 매 latent → obs
    
    def step(self, obs, action):
        latent = self.encoder(obs)
        next_latent, h = self.dynamics(torch.cat([latent, action]))
        predicted_obs = self.decoder(next_latent)
        return predicted_obs, next_latent
```

### Hierarchical model
```python
class HierarchicalGenerativeModel:
    def __init__(self, levels):
        self.levels = levels  # 매 each = (mu, var, dynamics)
    
    def predict(self):
        # 매 top-down predictions
        for i in range(len(self.levels) - 1, 0, -1):
            self.levels[i-1].mu = self.levels[i].dynamics(self.levels[i].mu)
    
    def update(self, obs):
        # 매 bottom-up errors
        for i in range(len(self.levels)):
            error = obs - self.levels[i].mu if i == 0 else self.levels[i-1].mu - self.levels[i].mu
            self.levels[i].mu += LR * error
```

### Schizophrenia model (precision dysregulation)
```python
def schizophrenia_simulation(true_state, sensory_precision_low):
    """매 low sensory precision → 매 prior 의 dominate → 매 hallucination."""
    sensory_precision = sensory_precision_low  # 매 abnormally low
    prior_strength = 1.0
    posterior = (sensory_precision * sensory_input + prior_strength * prior) / (sensory_precision + prior_strength)
    return posterior  # 매 dominated by prior
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Theoretical neuro | FEP framework |
| Computational model | pymdp |
| RL exploration | Curiosity (FEP-inspired) |
| World model | Schmidhuber-style |
| Psychiatry | Precision-weighting |

**기본값**: 매 FEP 의 high-level theory + 매 specific implementation = pymdp (active inference) or world model (RL).

## 🔗 Graph
- 부모: [[Computational-Neuroscience-RL|Computational-Neuroscience]] · [[Predictive-Processing]]
- 변형: [[Active-Inference]] · [[Predictive-Coding]]
- 응용: [[World-Model]]
- Adjacent: [[Bayesian-Brain]] · [[Default Mode Network (DMN)]] · [[Reinforcement-Learning]]

## 🤖 LLM 활용
**언제**: 매 neuroscience research. 매 active inference RL.
**언제 X**: 매 production engineering.

## ❌ 안티패턴
- **Theory of everything claim**: 매 unfalsifiable.
- **FEP 의 RL 의 isomorphic 의 trust**: 매 detail 의 differ.
- **Skip implementation**: 매 vague.

## 🧪 검증 / 중복
- Verified (Friston papers, pymdp docs, Hohwy Predictive Mind).
- 신뢰도 B+.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-04-20 | Auto-reinforced |
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — FEP + 매 predictive coding / pymdp / EFE / world model code |