---
id: wiki-2026-0508-posterior-and-prior-probability
title: Posterior and Prior Probability
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Bayesian Inference, Prior, Posterior, Bayes Update]
duplicate_of: none
source_trust_level: A
confidence_score: 0.92
verification_status: applied
tags: [bayesian, statistics, inference, probability]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: pymc/numpyro
---

# Posterior and Prior Probability

## 매 한 줄
> **"매 prior + likelihood = posterior"**. Bayes' theorem 의 사전 belief 의 evidence 의 update 의 posterior 의 derive — 매 modern probabilistic ML / scientific inference 의 backbone (PyMC 5, NumPyro, Stan).

## 매 핵심

### 매 정의
- **Prior P(θ)**: 매 데이터 이전 의 parameter belief.
- **Likelihood P(D|θ)**: 매 parameter 하 의 데이터 의 probability.
- **Posterior P(θ|D) ∝ P(D|θ) · P(θ)**: 매 update 후 의 belief.
- **Marginal P(D) = ∫ P(D|θ)P(θ) dθ**: 매 normalizing constant.

### 매 Prior 종류
- **Informative**: 매 strong domain knowledge (Beta(20,5)).
- **Weakly informative**: 매 mild regularization (Normal(0,10)).
- **Non-informative / Jeffreys**: 매 maximum-entropy invariant prior.
- **Conjugate**: 매 posterior 의 same family (Beta-Binomial, Normal-Normal, Gamma-Poisson).

### 매 응용
1. A/B testing 의 conversion rate posterior.
2. Bayesian neural network weight posterior.
3. Kalman filter (sequential prior→posterior).
4. Spam classification (Naive Bayes).

## 💻 패턴

### Beta-Binomial conjugate update (analytical)
```python
# Prior: Beta(α, β); Data: k successes in n trials
# Posterior: Beta(α+k, β+n-k)
import numpy as np
from scipy.stats import beta

alpha_prior, beta_prior = 2, 2  # weakly informative
k, n = 7, 10                    # observed
alpha_post, beta_post = alpha_prior + k, beta_prior + (n - k)

mean_post = alpha_post / (alpha_post + beta_post)
ci_95 = beta.interval(0.95, alpha_post, beta_post)
print(f"posterior mean={mean_post:.3f}, 95% CI={ci_95}")
```

### PyMC 5 — MCMC posterior
```python
import pymc as pm

with pm.Model() as m:
    theta = pm.Beta("theta", alpha=2, beta=2)
    obs   = pm.Binomial("obs", n=10, p=theta, observed=7)
    idata = pm.sample(2000, tune=1000, target_accept=0.95)

pm.summary(idata, var_names=["theta"])  # mean, hdi_3%, hdi_97%
```

### NumPyro — JAX-accelerated NUTS
```python
import jax, numpyro, numpyro.distributions as dist
from numpyro.infer import MCMC, NUTS

def model(k=None, n=10):
    theta = numpyro.sample("theta", dist.Beta(2, 2))
    numpyro.sample("obs", dist.Binomial(n, theta), obs=k)

mcmc = MCMC(NUTS(model), num_warmup=1000, num_samples=4000)
mcmc.run(jax.random.PRNGKey(0), k=7)
mcmc.print_summary()
```

### Sequential Bayesian update
```python
# Stream of bernoulli observations — posterior becomes next prior
a, b = 1.0, 1.0  # uniform start
for x in stream:
    a += x; b += (1 - x)
# E[θ] = a / (a+b) at any time
```

### Naive Bayes posterior (text classification)
```python
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB

vec = TfidfVectorizer().fit(train_docs)
clf = MultinomialNB(alpha=1.0).fit(vec.transform(train_docs), y_train)
# alpha = Laplace prior strength; predict_proba returns P(class | doc)
proba = clf.predict_proba(vec.transform(test_docs))
```

### Posterior predictive check
```python
with m:
    ppc = pm.sample_posterior_predictive(idata, var_names=["obs"])
# Compare ppc["obs"] distribution with actual data → model adequacy
```

### Empirical Bayes (data-driven prior)
```python
# Estimate prior hyperparameters from grouped data, then update per-group
# e.g. baseball batting averages — shrinkage toward league mean
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| 매 conjugate family fits | Analytical update (closed form) |
| 매 small/medium model | PyMC NUTS or NumPyro NUTS |
| 매 large neural net | VI (variational), Laplace, or SWAG |
| 매 streaming data | Sequential / Kalman-like update |
| 매 strong domain knowledge | Informative prior; document it |
| 매 weak knowledge + caution | Weakly informative (avoid flat improper) |

**기본값**: 매 weakly-informative prior + NUTS sampler — 매 PyMC 5 / NumPyro.

## 🔗 Graph
- 부모: [[Probability Theory]] · [[Statistics]]
- 변형: [[Maximum Likelihood Estimation]] · [[MAP Estimation]]
- 응용: [[Bayesian Networks]] · [[Particle-Filter-Algorithms]] · [[Kalman-Filter-and-State-Tracking]]
- Adjacent: [[Information Theory]] · [[Decision Theory]] · [[Expectation-Maximization]]

## 🤖 LLM 활용
**언제**: 매 small data + domain prior 의 quantified uncertainty 의 필요. 매 sequential / online update.
**언제 X**: 매 huge data + simple point estimate sufficient — 매 frequentist MLE 의 cheaper.

## ❌ 안티패턴
- **Improper flat prior 무지성 사용**: 매 posterior 의 improper 의 risk.
- **Prior cherry-picking ex post**: 매 result 본 후 prior 의 tweak — 매 invalid.
- **Likelihood-prior inconsistency**: 매 prior support 가 likelihood 와 mismatch.
- **No posterior predictive check**: 매 fit 검증 없이 conclude.
- **Overconfident strong prior on tiny data**: 매 prior 가 swamps evidence.

## 🧪 검증 / 중복
- Verified (Gelman *Bayesian Data Analysis* 3e; Kruschke *DBDA* 2e; Bishop *PRML* Ch2).
- Conjugate updates: 매 analytical, 매 verified math.
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — full Bayesian update spec with PyMC 5 / NumPyro patterns |
