---
id: wiki-2026-0508-probability-theory
title: Probability Theory
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Probability, Measure-Theoretic Probability, Stochastics]
duplicate_of: none
source_trust_level: A
confidence_score: 0.93
verification_status: applied
tags: [probability, statistics, measure-theory, foundations]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: numpy/scipy
---

# Probability Theory

## 매 한 줄
> **"매 uncertainty 의 axiomatic calculus"**. Kolmogorov(1933) 의 (Ω, ℱ, P) triple 의 measure-theoretic foundation 의 매 modern ML / inference / quantum 의 모든 reasoning 의 기반.

## 매 핵심

### 매 axioms (Kolmogorov)
1. P(A) ≥ 0 모든 event A.
2. P(Ω) = 1.
3. σ-additivity: 매 disjoint {Aᵢ}, P(∪Aᵢ) = Σ P(Aᵢ).

### 매 핵심 개념
- **Sample space Ω**, **σ-algebra ℱ**, **measure P**.
- **Random variable**: ℱ-measurable X: Ω→ℝ.
- **Distribution / CDF / PDF / PMF**.
- **Independence**: P(A∩B) = P(A)P(B).
- **Conditional**: P(A|B) = P(A∩B)/P(B); Bayes 의 즉시 derive.
- **Expectation E[X] = ∫ X dP**, variance, covariance, MGF.

### 매 limit theorems
- **LLN**: 매 sample mean → E[X].
- **CLT**: 매 sum of iid 의 normalized 의 → 𝓝(0,1).
- **Borel-Cantelli, SLLN, Kolmogorov 0-1 law**.

### 매 응용
1. ML — likelihood, posterior, generalization bounds.
2. Stochastic processes — Brownian motion, Poisson process, MCMC.
3. Quantum mechanics (Born rule probabilities).
4. Cryptography (random oracles, prob. proofs).
5. Information theory (entropy, mutual info).

## 💻 패턴

### NumPy distributions & sampling
```python
import numpy as np
rng = np.random.default_rng(42)
x = rng.normal(0, 1, size=10_000)
print(x.mean(), x.std())            # 매 ~0, ~1
```

### SciPy CDF / PDF / quantile
```python
from scipy import stats
n = stats.norm(loc=0, scale=1)
n.pdf(1.0); n.cdf(1.96); n.ppf(0.975)  # ~1.96
```

### Empirical CLT demo
```python
n, k = 30, 5000
sample_means = rng.exponential(1.0, (k, n)).mean(axis=1)
# Standardize: (mean - 1) * sqrt(n) → 매 ~ 𝓝(0,1)
z = (sample_means - 1.0) * np.sqrt(n)
```

### Conditional expectation via Monte Carlo
```python
# E[Y | X∈A] estimate
mask = (X >= 0.5) & (X <= 0.7)
cond_mean = Y[mask].mean()
```

### Bayes update (discrete)
```python
priors = np.array([0.5, 0.5])      # H1, H2
likes  = np.array([0.9, 0.1])      # P(D|Hi)
post   = priors * likes
post  /= post.sum()
```

### Markov chain stationary distribution
```python
P = np.array([[0.7,0.3],[0.4,0.6]])
vals, vecs = np.linalg.eig(P.T)
pi = np.real(vecs[:, np.isclose(vals,1)].ravel())
pi /= pi.sum()
```

### Tail bound (Chebyshev / Hoeffding)
```python
# P(|X̄ - μ| ≥ ε) ≤ 2 exp(-2 n ε² / (b-a)²)   (Hoeffding, bounded X)
def hoeffding(n, eps, a=0, b=1):
    return 2*np.exp(-2*n*eps**2 / (b-a)**2)
```

### KL divergence
```python
from scipy.special import rel_entr
kl = rel_entr(p, q).sum()           # nats
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| 매 finite sample / discrete | PMF, combinatorial |
| 매 continuous, smooth | PDF + integration / change of variables |
| 매 high-dim posterior | MCMC (NUTS) / VI / SMC |
| 매 streaming / online | sufficient statistics, exp-family |
| 매 worst-case bounds | Markov / Chebyshev / Hoeffding / Bernstein |
| 매 dependence structure | copulas, graphical models |

**기본값**: 매 explicit (Ω, ℱ, P) framing → 매 distribution 의 identify → 매 numpy/scipy 의 simulate.

## 🔗 Graph
- 부모: [[Statistics]] · [[Measure Theory]]
- 변형: [[Probability Theory|Probability-Theory-Foundations]] (canonical-of-this)
- 응용: [[Posterior-and-Prior-Probability]] · [[Entropy in Information Theory|Information Theory]] · [[Statistical-Power]] · [[Sampling-Techniques]]
- Adjacent: [[Decision Theory]] · [[Information-Entropy]] · [[Monte-Carlo-Integration]]

## 🤖 LLM 활용
**언제**: 매 uncertainty quantify / propagate / update 의 필요. 매 generative model 의 design.
**언제 X**: 매 fully deterministic system — 매 logic / algebra 의 sufficient.

## ❌ 안티패턴
- **Frequentist↔Bayesian 의 conflate**: 매 different interpretations of P; 매 results 다를 수 있음.
- **Independence assume 무근거**: 매 correlated data 의 i.i.d. 처리 → underestimated variance.
- **Continuous vs discrete 혼동**: 매 P(X=x)=0 (continuous), 매 density ≠ probability.
- **CLT 의 small n 적용**: 매 heavy tails / dependence — 매 fail.
- **0/0 conditional**: 매 P(B)=0 의 P(A|B) 의 undefined (regular cond. prob 의 use).

## 🧪 검증 / 중복
- Verified (Billingsley *Probability and Measure* 3e; Durrett *Probability* 5e; Kolmogorov 1933).
- 신뢰도 A.
- Note: 매 Probability-Theory-Foundations.md 의 매 redirect 의 here.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — full Kolmogorov axiomatic spec + numerical patterns; canonical for Probability-Theory-Foundations |
