2nd/10_Wiki/Topics/General Knowledge/Variance-Rules.md

---
id: wiki-2026-0508-variance-rules
title: Variance Rules
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Variance Algebra, Var Properties, Bienaymé Identity]
duplicate_of: none
source_trust_level: A
confidence_score: 0.9
verification_status: applied
tags: [statistics, probability, math, identity]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: NumPy
---

# Variance Rules

## 매 한 줄
> **"매 random variable 의 spread 의 algebra — Var(aX + b) = a²Var(X), 매 independence 매 sum 의 add"**. 1853 Bienaymé 의 sum-of-independent identity 부터 매 modern propagation-of-uncertainty, finance VaR, ML loss decomposition 까지 — 매 variance algebra 의 매 day-1 statistics 의 still 매 most-used identity.

## 매 핵심

### 매 core identities
- **Definition**: $\mathrm{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2$.
- **Affine**: $\mathrm{Var}(aX + b) = a^2 \mathrm{Var}(X)$ — 매 constant $b$ 의 drop.
- **Sum**: $\mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\,\mathrm{Cov}(X, Y)$.
- **Independence (Bienaymé)**: $X \perp Y \Rightarrow \mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y)$.
- **Linear comb**: $\mathrm{Var}\!\left(\sum a_i X_i\right) = \sum a_i^2 \mathrm{Var}(X_i) + 2 \sum_{i<j} a_i a_j \mathrm{Cov}(X_i, X_j)$.
- **Law of total variance**: $\mathrm{Var}(Y) = \mathbb{E}[\mathrm{Var}(Y|X)] + \mathrm{Var}(\mathbb{E}[Y|X])$.
- **Sample variance bias correction**: $s^2 = \frac{1}{n-1}\sum (x_i - \bar{x})^2$ — 매 Bessel.

### 매 propagation (delta method)
- **Univariate**: $\mathrm{Var}(g(X)) \approx (g'(\mu))^2 \mathrm{Var}(X)$.
- **Multivariate**: $\mathrm{Var}(g(\mathbf{X})) \approx \nabla g(\mu)^\top \Sigma \nabla g(\mu)$.

### 매 응용
1. Portfolio variance (Markowitz).
2. Error propagation in physics measurement.
3. ML bias-variance decomposition.
4. A/B test sample-size (Welch).
5. Kalman filter — covariance propagation.

## 💻 패턴

### Numerical sample variance — Welford (numerically stable)
```python
def welford_variance(stream):
    n = 0; mean = 0.0; M2 = 0.0
    for x in stream:
        n += 1
        delta = x - mean
        mean += delta / n
        M2 += delta * (x - mean)            # 매 use updated mean
    return mean, M2 / (n - 1) if n > 1 else float('nan')
```

### Linear combination variance
```python
import numpy as np
def linear_combo_var(weights, cov):
    # Var(w^T X) = w^T Σ w
    w = np.asarray(weights); cov = np.asarray(cov)
    return float(w @ cov @ w)
```

### Portfolio variance (Markowitz)
```python
def portfolio_var(weights, returns_matrix):
    cov = np.cov(returns_matrix, rowvar=False, ddof=1)
    return weights @ cov @ weights
```

### Delta-method propagation
```python
import numpy as np

def delta_method(g, grad_g, mu, sigma):
    # mu: vector, sigma: covariance
    g_grad = np.asarray(grad_g(mu))
    return float(g_grad @ sigma @ g_grad)
```

### Law of total variance — verify by simulation
```python
import numpy as np
rng = np.random.default_rng(0)
N = 1_000_000
X = rng.integers(0, 3, size=N)                  # 매 latent class
mu_y = np.array([0.0, 1.0, 5.0])[X]
Y    = rng.normal(mu_y, scale=1.0)
total = Y.var()
inner = np.array([Y[X==k].var() for k in range(3)]).mean()
outer = np.array([Y[X==k].mean() for k in range(3)]).var()
print(total, inner + outer)                     # 매 ≈ equal
```

### Bias-variance decomposition (ML)
```python
def bias_variance(predictions, y_true):
    # predictions: (n_models, n_samples)
    mean_pred = predictions.mean(axis=0)
    bias_sq = ((mean_pred - y_true) ** 2).mean()
    var = predictions.var(axis=0).mean()
    noise_lb = 0.0                              # 매 estimable 의 의 separately
    return bias_sq, var, noise_lb
```

### Welch's t-test variance handling
```python
from scipy import stats
t, p = stats.ttest_ind(a, b, equal_var=False)   # Welch
# 매 unequal variance — Satterthwaite degrees of freedom
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Streaming variance | Welford (numerically stable) |
| Independent sum | Bienaymé — sum the variances |
| Correlated sum | Full covariance — $w^\top \Sigma w$ |
| Nonlinear function $g(X)$ | Delta method (1st-order) — or Monte Carlo |
| Hierarchical / mixture | Law of total variance 의 decompose |
| ML overfitting diagnose | Bias-variance decomposition |
| Sample variance | Bessel correction ($n-1$) |

**기본값**: independence 의 confirm 후 Bienaymé. Doubt — Monte Carlo 의 verify.

## 🔗 Graph
- 부모: [[Probability Theory]]
- 응용: [[Bias-Variance Tradeoff]]

## 🤖 LLM 활용
**언제**: identity recall, derivation hint, code skeleton (Welford, delta).
**언제 X**: 매 specific paper 의 closed-form — derivation 의 cross-check.

## ❌ 안티패턴
- **Bienaymé 의 correlated variable 의 apply**: 매 covariance 의 forget — biased toward zero variance.
- **Two-pass naive variance** ($\sum x_i^2 - (\sum x_i)^2/n$): 매 catastrophic cancellation — Welford 의 use.
- **Sample variance with $n$**: 매 biased — Bessel ($n-1$).
- **Affine 매 $b$ 의 add to variance**: 매 $b$ 의 drop, only $a^2$ matters.
- **Delta method 의 high curvature 의 use**: 매 1st-order — large $\sigma$ 의 의 break, 의 Monte Carlo.

## 🧪 검증 / 중복
- Verified (Bienaymé 1853; Casella & Berger _Statistical Inference_ 2nd ed.; Welford 1962 _Technometrics_).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — variance algebra + Welford + delta method 정리 |