---
id: wiki-2026-0508-hopfield-network
title: Hopfield Network
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Hopfield, associative memory, content-addressable, modern Hopfield, Ramsauer]
duplicate_of: none
source_trust_level: A
confidence_score: 0.92
verification_status: applied
tags: [neural-network, hopfield, associative-memory, energy-based, modern-hopfield]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: PyTorch / NumPy
---

# Hopfield Network

## 매 한 줄
> **"매 recurrent NN 의 의 의 의 associative memory"**. Hopfield 1982. 매 energy 의 의 의 settle. 매 modern Hopfield (Ramsauer 2020) — 매 transformer attention 와 equivalent. 매 응용: 매 pattern restoration, 매 optimization, 매 memory.

## 매 핵심

### 매 classical
- **Binary states** ±1.
- **Symmetric weights** Wij = Wji.
- **Energy**: E = -½ Σ Wij sᵢ sⱼ.
- **Asynchronous update**: 매 si ← sign(Σ Wij sj).
- **Capacity**: 매 ~0.14N patterns.

### 매 modern (Ramsauer 2020)
- **Continuous states**.
- **Exponential storage** (exp).
- **Single-step retrieval**.
- **= attention** (key-value).

### 매 응용
1. **Pattern completion** (denoise).
2. **Combinatorial opt** (TSP).
3. **Modern: attention/memory** in transformers.
4. **Boltzmann-style RBM**.

## 💻 패턴

### Classical Hopfield (NumPy)
```python
import numpy as np

class Hopfield:
    def __init__(self, n):
        self.W = np.zeros((n, n))
    
    def store(self, patterns):
        for p in patterns:
            self.W += np.outer(p, p)
        np.fill_diagonal(self.W, 0)
    
    def retrieve(self, x, max_iter=100):
        for _ in range(max_iter):
            i = np.random.randint(len(x))
            x[i] = np.sign(self.W[i] @ x) or 1
        return x
```

### Modern Hopfield (Ramsauer)
```python
import torch
import torch.nn.functional as F

def modern_hopfield(query, keys, values, beta=1.0):
    """매 = attention with sqrt scaling absorbed."""
    attn = F.softmax(beta * query @ keys.T, dim=-1)
    return attn @ values
```

### Energy
```python
def energy(state, W):
    return -0.5 * state @ W @ state
```

### Pattern denoise demo
```python
patterns = [np.random.choice([-1, 1], 100) for _ in range(5)]
h = Hopfield(100)
h.store(patterns)

noisy = patterns[0].copy()
flip_idx = np.random.choice(100, 20, replace=False)
noisy[flip_idx] *= -1

restored = h.retrieve(noisy)
print(np.array_equal(restored, patterns[0]))
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Associative memory | Modern Hopfield |
| Pattern denoise | Classical or modern |
| Transformer | = attention |
| TSP / opt | Hopfield/Boltzmann |

**기본값**: 매 modern Hopfield = attention 와 동일 → modern transformer 의 default.

## 🔗 Graph
- 변형: [[Modern-Hopfield]]
- 응용: [[Attention-Mechanism]] · [[Associative-Memory]]
- Adjacent: [[Transformer]]

## 🤖 LLM 활용
**언제**: 매 memory + attention concept.
**언제 X**: 매 pure feedforward task.

## ❌ 안티패턴
- **Classical at large scale**: 매 capacity 부족.
- **Asymmetric W**: 매 convergence X.

## 🧪 검증 / 중복
- Verified (Hopfield 1982, Ramsauer 2020).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — Hopfield + modern + energy code |