2nd/10_Wiki/Topics/AI_and_ML/Hopfield Network.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-hopfield-network	Hopfield Network	10_Wiki/Topics	verified	self	Hopfield associative memory content-addressable modern Hopfield Ramsauer	none	A	0.92	applied	neural-network hopfield associative-memory energy-based modern-hopfield		2026-05-10	pending	language framework Python PyTorch / NumPy

Hopfield Network

매 한 줄

"매 recurrent NN 의 의 의 의 associative memory". Hopfield 1982. 매 energy 의 의 의 settle. 매 modern Hopfield (Ramsauer 2020) — 매 transformer attention 와 equivalent. 매 응용: 매 pattern restoration, 매 optimization, 매 memory.

매 핵심

매 classical

• Binary states ±1.
• Symmetric weights Wij = Wji.
• Energy: E = -½ Σ Wij sᵢ sⱼ.
• Asynchronous update: 매 si ← sign(Σ Wij sj).
• Capacity: 매 ~0.14N patterns.

매 modern (Ramsauer 2020)

• Continuous states.
• Exponential storage (exp).
• Single-step retrieval.
• = attention (key-value).

매 응용

1. Pattern completion (denoise).
2. Combinatorial opt (TSP).
3. Modern: attention/memory in transformers.
4. Boltzmann-style RBM.

💻 패턴

Classical Hopfield (NumPy)

import numpy as np

class Hopfield:
    def __init__(self, n):
        self.W = np.zeros((n, n))
    
    def store(self, patterns):
        for p in patterns:
            self.W += np.outer(p, p)
        np.fill_diagonal(self.W, 0)
    
    def retrieve(self, x, max_iter=100):
        for _ in range(max_iter):
            i = np.random.randint(len(x))
            x[i] = np.sign(self.W[i] @ x) or 1
        return x

Modern Hopfield (Ramsauer)

import torch
import torch.nn.functional as F

def modern_hopfield(query, keys, values, beta=1.0):
    """매 = attention with sqrt scaling absorbed."""
    attn = F.softmax(beta * query @ keys.T, dim=-1)
    return attn @ values

Energy

def energy(state, W):
    return -0.5 * state @ W @ state

Pattern denoise demo

patterns = [np.random.choice([-1, 1], 100) for _ in range(5)]
h = Hopfield(100)
h.store(patterns)

noisy = patterns[0].copy()
flip_idx = np.random.choice(100, 20, replace=False)
noisy[flip_idx] *= -1

restored = h.retrieve(noisy)
print(np.array_equal(restored, patterns[0]))

매 결정 기준

상황	Approach
Associative memory	Modern Hopfield
Pattern denoise	Classical or modern
Transformer	= attention
TSP / opt	Hopfield/Boltzmann

기본값: 매 modern Hopfield = attention 와 동일 → modern transformer 의 default.

🔗 Graph

• 변형: Modern-Hopfield
• 응용: Attention Mechanism · Associative-Memory
• Adjacent: Transformer

🤖 LLM 활용

언제: 매 memory + attention concept. 언제 X: 매 pure feedforward task.

❌ 안티패턴

• Classical at large scale: 매 capacity 부족.
• Asymmetric W: 매 convergence X.

🧪 검증 / 중복

• Verified (Hopfield 1982, Ramsauer 2020).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — Hopfield + modern + energy code