Files

T

Antigravity Agent 504fd5fb42 [G1-Sync] Manual knowledge update

2026-05-10 22:08:15 +09:00

5.3 KiB

Raw Blame History

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

title

Markov Chain Monte Carlo (MCMC)

매 한 줄

"매 MCMC = stationary distribution이 target인 chain 만들기". 정규화 상수 모르고도 posterior 샘플 가능.

매 핵심

매 알고리즘

Metropolis-Hastings: propose q(x'|x), accept α=min(1, π(x')q(x|x')/(π(x)q(x'|x))).
Random walk MH: q = Normal(x, σ²). σ가 acceptance 결정.
Gibbs: 조건부 p(x_i | x_{-i}) 순차 샘플. conjugate에 강함.
Slice sampling: 보조 변수, tuning 적음.
HMC (Hamiltonian): gradient + leapfrog. high-dim 효율.
NUTS: HMC trajectory 자동 결정. Stan/PyMC/NumPyro 기본.
SMC, parallel tempering: multi-modal에 유리.

매 진단

Trace plot: chain 안정성 시각 검사
R̂ (Gelman-Rubin): 다중 chain 수렴, <1.01 권장
ESS (effective sample size): 자기상관 보정 샘플 수
Energy diagnostic (HMC): divergent transitions 0 목표
Posterior predictive check: model fit

매 응용

Bayesian posterior 추정 (intractable normalizer)
Hierarchical models (multilevel regression)
Latent variable models
Bayesian deep learning (BNN, variational alternative)
Phylogenetics, epidemiology

💻 패턴

Metropolis-Hastings (numpy)

import numpy as np
def mh(log_target, x0, n=10000, step=0.5, rng=np.random.default_rng()):
    x = np.array(x0, dtype=float); samples = [x.copy()]
    log_p = log_target(x); accepts = 0
    for _ in range(n):
        x_new = x + rng.normal(scale=step, size=x.shape)
        log_p_new = log_target(x_new)
        if np.log(rng.uniform()) < log_p_new - log_p:
            x, log_p = x_new, log_p_new; accepts += 1
        samples.append(x.copy())
    return np.array(samples), accepts / n      # target ~0.234 (high-d), 0.44 (1d)

Gibbs sampling (bivariate normal)

def gibbs_bvn(rho, n=10000, rng=np.random.default_rng()):
    x = y = 0.0; out = np.empty((n, 2))
    for i in range(n):
        x = rng.normal(rho * y, np.sqrt(1 - rho**2))
        y = rng.normal(rho * x, np.sqrt(1 - rho**2))
        out[i] = (x, y)
    return out

PyMC (NUTS)

import pymc as pm, numpy as np
y = np.random.normal(2, 1, 100)
with pm.Model() as m:
    mu = pm.Normal("mu", 0, 10)
    sigma = pm.HalfNormal("sigma", 1)
    pm.Normal("y", mu, sigma, observed=y)
    idata = pm.sample(2000, tune=1000, chains=4, target_accept=0.9)
print(pm.summary(idata, var_names=["mu", "sigma"]))   # r_hat, ess

NumPyro (JAX, fast)

import jax, numpyro
import numpyro.distributions as dist
from numpyro.infer import MCMC, NUTS

def model(y):
    mu = numpyro.sample("mu", dist.Normal(0, 10))
    sigma = numpyro.sample("sigma", dist.HalfNormal(1))
    numpyro.sample("obs", dist.Normal(mu, sigma), obs=y)

mcmc = MCMC(NUTS(model), num_warmup=1000, num_samples=2000, num_chains=4)
mcmc.run(jax.random.PRNGKey(0), y=y)
mcmc.print_summary()

Diagnostics (ArviZ)

import arviz as az
az.plot_trace(idata)
az.plot_rank(idata)
print(az.rhat(idata).max(), az.ess(idata).min())
az.plot_pair(idata, divergences=True)

효율 팁

# 1) Reparameterize (non-centered): theta = mu + sigma * z, z~N(0,1)
# 2) target_accept 0.9~0.99 if divergences
# 3) 표준화/스케일링 → leapfrog 안정
# 4) Initial values: pm.find_MAP() or jitter+adapt_diag

매 결정 기준

상황	Sampler
Low-dim, custom posterior	MH (간단)
Conjugate hierarchical	Gibbs
Continuous, gradient 가능	NUTS/HMC
Discrete latent	MH within Gibbs, SMC
Multi-modal	Parallel tempering, SMC
대용량 / GPU	NumPyro (JAX), BlackJAX
빠른 prod 근사	VI (대안), Laplace

기본값: continuous → NumPyro/PyMC NUTS. Discrete → Gibbs / SMC.

🔗 Graph

부모: Bayesian-Inference, Sampling-Methods
변형: Metropolis-Hastings, Gibbs-Sampling, Hamiltonian-Monte-Carlo, NUTS
응용: Hierarchical-Models, Bayesian-Regression, Probabilistic-Programming
Adjacent: Variational-Inference, Importance-Sampling, ArviZ, Stan

🤖 LLM 활용

언제: 모델 작성, sampler 선택, divergence 진단 가이드. 언제 X: 복잡 hierarchical model 검증은 도메인 전문가 + posterior predictive.

❌ 안티패턴

Centered hierarchical에 NUTS 그대로 (divergences) → non-centered
Single chain → R̂ 불가, 수렴 진단 X
Burn-in/warmup 무시
Acceptance rate 99% (step 너무 작음) or 1% (너무 큼)
Trace plot 안 보고 mean만 신뢰
VI로 충분한데 MCMC 돌리기 (시간 낭비)

🧪 검증 / 중복

Verified (Gelman BDA3, Neal HMC review, Hoffman NUTS, PyMC/NumPyro docs). 신뢰도 A.
중복: 없음.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — PyMC/NumPyro 패턴, ArviZ diagnostic

5.3 KiB Raw Blame History Unescape Escape