2nd/10_Wiki/Topics/General Knowledge/Variance-Rules.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-variance-rules	Variance Rules	10_Wiki/Topics	verified	self	Variance Algebra Var Properties Bienaymé Identity	none	A	0.9	applied	statistics probability math identity		2026-05-10	pending	language framework Python NumPy

Variance Rules

매 한 줄

"매 random variable 의 spread 의 algebra — Var(aX + b) = a²Var(X), 매 independence 매 sum 의 add". 1853 Bienaymé 의 sum-of-independent identity 부터 매 modern propagation-of-uncertainty, finance VaR, ML loss decomposition 까지 — 매 variance algebra 의 매 day-1 statistics 의 still 매 most-used identity.

매 핵심

매 core identities

• Definition: \mathrm{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2] = \mathbb{E}[X^2] - \mathbb{E}[X]^2.
• Affine: \mathrm{Var}(aX + b) = a^2 \mathrm{Var}(X) — 매 constant b 의 drop.
• Sum: \mathrm{Var}(X + Y) = \mathrm{Var}(X) + \mathrm{Var}(Y) + 2\,\mathrm{Cov}(X, Y).
• Independence (Bienaymé): X \perp Y \Rightarrow \mathrm{Var}(X+Y) = \mathrm{Var}(X) + \mathrm{Var}(Y).
• Linear comb: \mathrm{Var}\!\left(\sum a_i X_i\right) = \sum a_i^2 \mathrm{Var}(X_i) + 2 \sum_{i<j} a_i a_j \mathrm{Cov}(X_i, X_j).
• Law of total variance: \mathrm{Var}(Y) = \mathbb{E}[\mathrm{Var}(Y|X)] + \mathrm{Var}(\mathbb{E}[Y|X]).
• Sample variance bias correction: s^2 = \frac{1}{n-1}\sum (x_i - \bar{x})^2 — 매 Bessel.

매 propagation (delta method)

• Univariate: \mathrm{Var}(g(X)) \approx (g'(\mu))^2 \mathrm{Var}(X).
• Multivariate: \mathrm{Var}(g(\mathbf{X})) \approx \nabla g(\mu)^\top \Sigma \nabla g(\mu).

매 응용

1. Portfolio variance (Markowitz).
2. Error propagation in physics measurement.
3. ML bias-variance decomposition.
4. A/B test sample-size (Welch).
5. Kalman filter — covariance propagation.

💻 패턴

Numerical sample variance — Welford (numerically stable)

def welford_variance(stream):
    n = 0; mean = 0.0; M2 = 0.0
    for x in stream:
        n += 1
        delta = x - mean
        mean += delta / n
        M2 += delta * (x - mean)            # 매 use updated mean
    return mean, M2 / (n - 1) if n > 1 else float('nan')

Linear combination variance

import numpy as np
def linear_combo_var(weights, cov):
    # Var(w^T X) = w^T Σ w
    w = np.asarray(weights); cov = np.asarray(cov)
    return float(w @ cov @ w)

Portfolio variance (Markowitz)

def portfolio_var(weights, returns_matrix):
    cov = np.cov(returns_matrix, rowvar=False, ddof=1)
    return weights @ cov @ weights

Delta-method propagation

import numpy as np

def delta_method(g, grad_g, mu, sigma):
    # mu: vector, sigma: covariance
    g_grad = np.asarray(grad_g(mu))
    return float(g_grad @ sigma @ g_grad)

Law of total variance — verify by simulation

import numpy as np
rng = np.random.default_rng(0)
N = 1_000_000
X = rng.integers(0, 3, size=N)                  # 매 latent class
mu_y = np.array([0.0, 1.0, 5.0])[X]
Y    = rng.normal(mu_y, scale=1.0)
total = Y.var()
inner = np.array([Y[X==k].var() for k in range(3)]).mean()
outer = np.array([Y[X==k].mean() for k in range(3)]).var()
print(total, inner + outer)                     # 매 ≈ equal

Bias-variance decomposition (ML)

def bias_variance(predictions, y_true):
    # predictions: (n_models, n_samples)
    mean_pred = predictions.mean(axis=0)
    bias_sq = ((mean_pred - y_true) ** 2).mean()
    var = predictions.var(axis=0).mean()
    noise_lb = 0.0                              # 매 estimable 의 의 separately
    return bias_sq, var, noise_lb

Welch's t-test variance handling

from scipy import stats
t, p = stats.ttest_ind(a, b, equal_var=False)   # Welch
# 매 unequal variance — Satterthwaite degrees of freedom

매 결정 기준

상황	Approach
Streaming variance	Welford (numerically stable)
Independent sum	Bienaymé — sum the variances
Correlated sum	Full covariance — w^\top \Sigma w
Nonlinear function g(X)	Delta method (1st-order) — or Monte Carlo
Hierarchical / mixture	Law of total variance 의 decompose
ML overfitting diagnose	Bias-variance decomposition
Sample variance	Bessel correction (n-1)

기본값: independence 의 confirm 후 Bienaymé. Doubt — Monte Carlo 의 verify.

🔗 Graph

• 부모: Probability Theory
• 응용: Bias-Variance Tradeoff

🤖 LLM 활용

언제: identity recall, derivation hint, code skeleton (Welford, delta). 언제 X: 매 specific paper 의 closed-form — derivation 의 cross-check.

❌ 안티패턴

• Bienaymé 의 correlated variable 의 apply: 매 covariance 의 forget — biased toward zero variance.
• Two-pass naive variance (\sum x_i^2 - (\sum x_i)^2/n): 매 catastrophic cancellation — Welford 의 use.
• Sample variance with $n$: 매 biased — Bessel (n-1).
• Affine 매 b 의 add to variance: 매 b 의 drop, only a^2 matters.
• Delta method 의 high curvature 의 use: 매 1st-order — large \sigma 의 의 break, 의 Monte Carlo.

🧪 검증 / 중복

• Verified (Bienaymé 1853; Casella & Berger Statistical Inference 2nd ed.; Welford 1962 Technometrics).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — variance algebra + Welford + delta method 정리