---
id: wiki-2026-0508-hebbian-theory
title: Hebbian Theory
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Hebbian Learning, Hebb's Rule, "Cells that fire together wire together"]
duplicate_of: none
source_trust_level: A
confidence_score: 0.85
verification_status: applied
tags: [neuroscience, learning-theory, biological-plasticity, neural-networks, STDP]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: NumPy / Brian2
---

# Hebbian Theory

## 매 한 줄
> **"매 'cells that fire together, wire together' — 매 pre-/post-synaptic activity 의 correlated firing 시 synapse strength 증가"**. 1949년 Donald Hebb "The Organization of Behavior" — 매 modern neuroscience 의 plasticity 의 foundation, STDP / BCM rule / Oja's rule 의 origin. 매 deep learning 의 backprop 과 다른 biological-plausible learning candidate.

## 매 핵심

### 매 Original Hebbian Rule
- **Δw_ij = η · x_i · y_j** (η: learning rate, x_i: pre, y_j: post).
- 매 unstable: weights 매 unbounded growth — 매 normalization 필요.
- Anti-Hebbian: Δw = -η x y — 매 decorrelation.

### 매 Variants
- **Oja's rule** (1982): Δw_ij = η y_j (x_i - y_j w_ij) — 매 weight normalization, 매 PCA 첫번째 component 으로 converges.
- **BCM rule** (Bienenstock-Cooper-Munro 1982): 매 sliding threshold θ — y > θ 시 LTP, y < θ 시 LTD.
- **STDP (Spike-Timing-Dependent Plasticity)**: 매 actual biological — pre-spike 가 post-spike 직전 시 LTP, 직후 시 LTD. 매 millisecond-scale.
- **Triplet STDP**: 매 더 정확한 model.

### 매 Biological evidence
- LTP (Long-Term Potentiation): 매 hippocampal CA1 의 Bliss-Lomo 1973 의 발견.
- LTD (Long-Term Depression): 매 cerebellum 의 motor learning.
- NMDA receptor 의 핵심: 매 coincidence detector.
- 매 calcium dynamics 의 underlying mechanism.

### 매 ML connections
- 매 unsupervised learning 의 model.
- Hopfield network 매 Hebbian-trained associative memory.
- SOM (Self-Organizing Maps) 매 Hebbian-style competitive learning.
- 매 modern: Predictive coding / Free energy principle 의 Hebbian-like rules.

## 💻 패턴

### Basic Hebbian Update
```python
import numpy as np

class HebbianNeuron:
    def __init__(self, n_inputs, lr=0.01):
        self.w = np.random.randn(n_inputs) * 0.01
        self.lr = lr
    
    def forward(self, x):
        return np.dot(self.w, x)
    
    def update(self, x, y):
        # Δw = η x y
        self.w += self.lr * x * y

# Demo: weights 의 unbounded growth (매 problem)
neuron = HebbianNeuron(n_inputs=10)
for _ in range(1000):
    x = np.random.randn(10)
    y = neuron.forward(x)
    neuron.update(x, y)
print(f"Weight norm: {np.linalg.norm(neuron.w):.2f}")  # 매 explodes
```

### Oja's Rule (매 stable + PCA)
```python
class OjasNeuron:
    """매 Δw = η y (x - y w). PCA 첫번째 component 으로 converges."""
    def __init__(self, n_inputs, lr=0.01):
        self.w = np.random.randn(n_inputs)
        self.w /= np.linalg.norm(self.w)
        self.lr = lr
    
    def update(self, x):
        y = np.dot(self.w, x)
        self.w += self.lr * y * (x - y * self.w)

# 매 verify: top eigenvector 으로 converges
np.random.seed(0)
data = np.random.randn(1000, 10) @ np.diag([5, 3, 2, 1, 1, 0.5, 0.5, 0.3, 0.3, 0.1])
neuron = OjasNeuron(10, lr=0.001)
for x in data:
    neuron.update(x)

# 매 compare with NumPy PCA top component
cov = np.cov(data.T)
eigvals, eigvecs = np.linalg.eigh(cov)
top_pc = eigvecs[:, -1]
similarity = abs(np.dot(neuron.w / np.linalg.norm(neuron.w), top_pc))
print(f"Similarity to top PC: {similarity:.4f}")  # ≈ 1.0
```

### BCM Rule (매 sliding threshold)
```python
class BCMNeuron:
    def __init__(self, n_inputs, lr=0.01, tau=100):
        self.w = np.random.randn(n_inputs) * 0.1
        self.lr = lr
        self.theta = 1.0  # sliding threshold
        self.tau = tau    # time constant
        self.y_history = []
    
    def forward(self, x):
        return np.dot(self.w, x)
    
    def update(self, x):
        y = self.forward(x)
        self.w += self.lr * x * y * (y - self.theta)
        # Sliding threshold: θ = E[y²]
        self.y_history.append(y ** 2)
        if len(self.y_history) > self.tau:
            self.y_history.pop(0)
        self.theta = np.mean(self.y_history) if self.y_history else 1.0
```

### STDP (Spike-Timing-Dependent) — Brian2
```python
from brian2 import *

# 매 100 input neurons → 1 output, STDP synapses
N = 100
input_group = PoissonGroup(N, rates=20*Hz)
neurons = NeuronGroup(1, '''
    dv/dt = -v / (10*ms) : 1
''', threshold='v > 1', reset='v = 0')

# 매 STDP synapse model
syn = Synapses(input_group, neurons,
    '''
    w : 1
    dapre/dt = -apre/(20*ms) : 1 (event-driven)
    dapost/dt = -apost/(20*ms) : 1 (event-driven)
    ''',
    on_pre='''
    v_post += w
    apre += 0.01
    w = clip(w + apost, 0, 1)
    ''',
    on_post='''
    apost -= 0.012
    w = clip(w + apre, 0, 1)
    '''
)
syn.connect()
syn.w = 'rand() * 0.5'

run(1000*ms)
hist(syn.w[:], 50)
# 매 bimodal distribution 의 emergence — 매 STDP signature
```

### Hopfield Network (Hebbian-trained associative memory)
```python
class HopfieldNetwork:
    def __init__(self, n):
        self.n = n
        self.W = np.zeros((n, n))
    
    def store(self, patterns):
        """Hebbian: W = Σ p p^T (diagonal=0)."""
        for p in patterns:
            self.W += np.outer(p, p)
        np.fill_diagonal(self.W, 0)
        self.W /= len(patterns)
    
    def recall(self, pattern, max_iter=100):
        s = pattern.copy()
        for _ in range(max_iter):
            s_new = np.sign(self.W @ s)
            if np.all(s_new == s):
                break
            s = s_new
        return s

# Demo: store ±1 patterns, recall from noisy
net = HopfieldNetwork(50)
patterns = [np.random.choice([-1, 1], 50) for _ in range(3)]
net.store(patterns)

noisy = patterns[0].copy()
noisy[:10] = -noisy[:10]  # flip 10 bits
recalled = net.recall(noisy)
print(f"Match: {np.mean(recalled == patterns[0]):.2%}")
```

## 매 결정 기준

| 상황 | Rule |
|---|---|
| Educational / simple demo | Plain Hebb |
| Stable + PCA emergence | Oja's rule |
| Bidirectional plasticity (LTP+LTD) | BCM |
| Biological realism (spiking) | STDP |
| Associative memory | Hopfield (Hebbian) |
| Competitive learning / clustering | SOM (Kohonen) |
| Modern deep learning | Backprop (NOT Hebbian) |
| Biologically-plausible deep learning | Predictive coding / Equilibrium propagation |

**기본값**: 매 conceptual 매 Hebb's rule, 매 ML practice 매 Oja's rule, 매 spiking simulation 매 STDP.

## 🔗 Graph
- 부모: [[Computational-Neuroscience-RL|Computational-Neuroscience]] · [[Synaptic-Plasticity]]
- 응용: [[Hopfield Network]] · [[Associative-Memory]] · [[Predictive-Coding]]
- Adjacent: [[Free-Energy-Principle]] · [[데이터 사이언스 및 ML 엔지니어링|Backpropagation]]

## 🤖 LLM 활용
**언제**: 매 conceptual explanation, 매 history of computational neuroscience, 매 simple rule derivation.
**언제 X**: 매 modern deep learning 의 training — backprop 사용. 매 actual neuroscience research 의 STDP parameter — 매 experimental data 참조.

## ❌ 안티패턴
- **Plain Hebb 의 production 사용**: 매 weights 의 explosion — Oja/BCM/normalization 사용.
- **"Hebbian = backprop" 혼동**: 매 different learning paradigm — backprop 매 supervised, Hebb 매 unsupervised correlation.
- **STDP 의 timing 의 부정확**: 매 millisecond precision 필요 — discrete-time approximation 의 한계.
- **Hopfield 의 capacity overestimate**: 매 ~0.14 N patterns 의 한계 (Hopfield 1982).

## 🧪 검증 / 중복
- Verified (Hebb 1949 "Organization of Behavior", Oja 1982, Bliss-Lomo 1973 LTP, Bi-Poo 1998 STDP).
- 신뢰도 A (foundational neuroscience theory).

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — Hebb/Oja/BCM/STDP rules, Hopfield, biological mechanisms |
