2nd/10_Wiki/Topics/General Knowledge/Markov-Random-Fields.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-markov-random-fields	Markov Random Fields	10_Wiki/Topics	verified	self	MRF Undirected Graphical Model Markov Network Gibbs Random Field	none	A	0.9	applied	machine-learning probability graphical-model vision statistics		2026-05-10	pending	language framework Python PyTorch/NetworkX

Markov Random Fields

매 한 줄

"매 undirected graph 의 joint distribution — local Markov property 의 satisfy". 매 Bayes net 의 directed counterpart — 매 conditional independence 의 graph separation 으로 read. Hammersley–Clifford (1971) 의 Gibbs distribution 의 equivalence — 2026 매 image segmentation, CRF, energy-based model (EBM) 의 underlie.

매 핵심

매 정의

• Graph G = (V, E), 매 node 의 random variable X_v.
• Local Markov: X_v \perp X_{V \setminus N[v]} \mid X_{N(v)} — 매 node 의 neighbors 의 condition 시 의 rest 의 independent.
• Hammersley–Clifford: 매 strictly positive joint 의 매 Gibbs form P(x) = \frac{1}{Z} \prod_{C \in \mathcal{C}} \psi_C(x_C) — 매 clique 의 product.
• Partition function: Z = \sum_x \prod_C \psi_C(x_C) — 매 intractable in general.

매 inference

• Exact: tree (sum-product / belief propagation).
• Loopy BP: 매 approximate, 매 often works.
• MCMC: Gibbs sampling — 매 conditional 의 sample 매 cycle.
• Variational: mean-field — 매 factorized q(x) = \prod_v q_v(x_v).
• Graph cut: 매 binary submodular — 매 exact min-cut.

매 응용

1. Image segmentation (foreground/background MRF).
2. Conditional Random Fields (CRF) — sequence labeling.
3. Stereo / depth estimation (smoothness prior).
4. Energy-based generative models (EBMs).
5. Statistical physics (Ising, Potts).

💻 패턴

Ising model — Gibbs sampling

import numpy as np

def ising_gibbs(N=64, beta=0.44, steps=10_000, h=0.0):
    spins = np.random.choice([-1, 1], size=(N, N))
    for _ in range(steps):
        i, j = np.random.randint(N, size=2)
        nb = (spins[(i+1)%N, j] + spins[(i-1)%N, j]
            + spins[i, (j+1)%N] + spins[i, (j-1)%N])
        dE = 2 * spins[i, j] * (beta * nb + h)
        if dE < 0 or np.random.random() < np.exp(-dE):
            spins[i, j] *= -1
    return spins

Loopy belief propagation (binary pairwise)

def loopy_bp(unary, pairwise, edges, iters=20):
    # unary[v]   : log-potential per node, shape (V, K)
    # pairwise   : (E, K, K) log-potentials
    # edges      : list of (u, v)
    msg = {(u, v): np.zeros(unary.shape[1]) for u, v in edges}
    msg.update({(v, u): np.zeros(unary.shape[1]) for u, v in edges})
    for _ in range(iters):
        new = {}
        for (u, v), e_idx in zip(edges, range(len(edges))):
            incoming = unary[u] + sum(msg[(w, u)] for w in neighbors(u) if w != v)
            new[(u, v)] = np.logaddexp.reduce(
                pairwise[e_idx] + incoming[:, None], axis=0)
            new[(u, v)] -= new[(u, v)].max()   # normalize
        msg.update(new)
    beliefs = np.array([
        unary[v] + sum(msg[(u, v)] for u in neighbors(v))
        for v in range(len(unary))])
    return beliefs

Linear-chain CRF (PyTorch — sequence labeling)

import torch, torch.nn as nn

class LinearChainCRF(nn.Module):
    def __init__(self, n_tags):
        super().__init__()
        self.trans = nn.Parameter(torch.randn(n_tags, n_tags))

    def log_partition(self, emissions):  # (T, K)
        T, K = emissions.shape
        alpha = emissions[0]
        for t in range(1, T):
            alpha = torch.logsumexp(
                alpha[:, None] + self.trans + emissions[t][None, :], dim=0)
        return torch.logsumexp(alpha, dim=0)

    def score(self, emissions, tags):
        s = emissions[0, tags[0]]
        for t in range(1, len(tags)):
            s = s + self.trans[tags[t-1], tags[t]] + emissions[t, tags[t]]
        return s

    def nll(self, emissions, tags):
        return self.log_partition(emissions) - self.score(emissions, tags)

Graph cut for binary MRF (submodular)

import maxflow   # PyMaxflow

def binary_mrf_graph_cut(unary_fg, unary_bg, pairwise_w):
    H, W = unary_fg.shape
    g = maxflow.Graph[float]()
    nodes = g.add_grid_nodes((H, W))
    g.add_grid_edges(nodes, pairwise_w)               # smoothness
    g.add_grid_tedges(nodes, unary_fg, unary_bg)      # data term
    g.maxflow()
    return g.get_grid_segments(nodes)                 # bool mask

Mean-field VI

def mean_field(unary, pairwise, iters=10):
    # q(x_v = k) ∝ exp(unary[v,k] + Σ_{u∈N(v)} Σ_l q(x_u=l) * pairwise[v,u,k,l])
    q = torch.softmax(unary, dim=-1)
    for _ in range(iters):
        msg = torch.einsum('uvkl,ul->vk', pairwise, q)
        q = torch.softmax(unary + msg, dim=-1)
    return q

매 결정 기준

상황	Approach
Tree-structured	Sum-product (exact)
Loopy graph, fast	Loopy BP
Loopy graph, accurate	MCMC (Gibbs) — slower
Binary submodular	Graph cut (exact min-cut)
Sequence labeling (NER)	Linear-chain CRF
Image segmentation	Pairwise MRF + α-expansion / DenseCRF
Modern generative	Energy-Based Model (EBM) — score matching

기본값: smallest model 의 first — chain → tree → loopy + BP → MCMC.

🔗 Graph

• 부모: Probabilistic Graphical Models · Probability Theory
• 응용: Image Segmentation · Energy-Based Model
• Adjacent: Bayesian Network

🤖 LLM 활용

언제: clique factorization 의 derive, BP/Gibbs pseudocode, partition-function intractability 의 explain. 언제 X: 매 specific paper algorithm — original 의 의 cross-check.

❌ 안티패턴

• Bayes net 의 mental model 의 reuse: 매 directionality 의 X — separation criterion 매 different.
• Computing Z for large graph: 매 #P-hard — variational / MCMC.
• Loopy BP on tightly-loopy graph: 매 may diverge — damping 의 try, MCMC 의 fallback.
• Linear-chain CRF 의 LSTM 으로 always replace: 매 small data 의 still wins, 매 calibration 의 better.
• Mean-field 의 multimodal posterior 의 use: 매 mode-seeking — 매 underestimate variance.

🧪 검증 / 중복

• Verified (Hammersley & Clifford 1971; Koller & Friedman PGM 2009; Murphy PML 2022).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — MRF basics + BP/CRF/graph-cut 정리