2nd/10_Wiki/Topics/Computer_Science_and_Theory/Statistical-Power.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-statistical-power	Statistical Power	10_Wiki/Topics	verified	self	Power-Analysis 1-beta	none	A	0.92	applied	statistics power hypothesis-testing ab-testing sample-size		2026-05-10	pending	language framework python statsmodels-scipy

Statistical Power

매 한 줄

"매 effect 가 있을 때 매 detect 의 확률 = 1 − β". Power 의 true positive rate — sample size, effect size, α, variance 의 함수. 1928 Neyman-Pearson 에서 등장 — 2026 A/B test, clinical trial, 모든 hypothesis test 의 design 의 핵심.

매 핵심

매 4 quantities (lock 3, solve 1)

• n — sample size.
• α — Type I error rate (false positive), 보통 0.05.
• β — Type II error rate (false negative); power = 1 − β, 보통 0.8.
• effect size — Cohen's d (means), Cohen's h (props), r (correlation), f² (regression).

매 effect size 의 convention (Cohen)

• d = 0.2 (small), 0.5 (medium), 0.8 (large) — for two-sample t-test.
• 매 domain-specific minimum detectable effect (MDE) 의 정의 의 우선 — Cohen 의 default 의 X.

매 sample size 공식 (two-sample t-test)

• n_per_group ≈ 2·(z_{1-α/2} + z_{1-β})² · σ² / Δ²
• α=0.05, power=0.8, Δ=0.5σ (medium d) → n ≈ 64/group.

매 응용

1. A/B test design — pre-experiment sample-size calc.
2. Clinical trial — FDA mandate.
3. Replication crisis — underpowered studies → false discovery.
4. Sequential testing — α-spending for early stopping.

💻 패턴

1. statsmodels — t-test power

from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()
n = analysis.solve_power(effect_size=0.5, alpha=0.05, power=0.8, ratio=1.0)
# n ≈ 63.77 per group

2. Proportion (A/B test) sample size

from statsmodels.stats.proportion import proportion_effectsize
from statsmodels.stats.power import NormalIndPower
es = proportion_effectsize(0.10, 0.12)  # baseline 10%, MDE 12%
n = NormalIndPower().solve_power(effect_size=es, alpha=0.05, power=0.8)
# 매 conversion uplift 2pp → ~3800 / arm

3. Power curve (visualize tradeoff)

import numpy as np
import matplotlib.pyplot as plt
ns = np.arange(20, 500, 10)
powers = [analysis.solve_power(effect_size=0.3, nobs1=n, alpha=0.05) for n in ns]
plt.plot(ns, powers); plt.axhline(0.8, color="r")

4. Simulation-based (custom test, complex design)

import numpy as np
def simulate_power(n, effect, sigma, alpha=0.05, B=10000):
    rejects = 0
    for _ in range(B):
        a = np.random.normal(0, sigma, n)
        b = np.random.normal(effect, sigma, n)
        t = (b.mean() - a.mean()) / np.sqrt(2*sigma**2/n)
        if abs(t) > 1.96:
            rejects += 1
    return rejects / B

5. MDE given fixed n (post-hoc planning)

mde = analysis.solve_power(nobs1=1000, alpha=0.05, power=0.8)
# 매 n=1000 에서 detect 가능한 minimum d

6. Bonferroni-corrected power (multiple tests)

k = 10  # 매 10 hypotheses
adjusted_alpha = 0.05 / k
n_corrected = analysis.solve_power(effect_size=0.5, alpha=adjusted_alpha, power=0.8)

7. Sequential / group-sequential (early stopping)

# 매 O'Brien-Fleming spending function 의 사용
# 매 이미 누적 α 를 나눠서 spend → 매 final n 의 약간 증가
from statsmodels.stats.proportion import proportions_ztest  # for fixed test
# Group-sequential: see `gsDesign` in R or `confseq` in Python (2026)

8. Cluster-randomized (intra-class correlation)

# 매 design effect = 1 + (m-1)·ρ
# 매 effective sample = n / DE
icc = 0.05; m = 30  # cluster size
DE = 1 + (m-1)*icc  # ≈ 2.45
n_effective = 1000 / DE

매 결정 기준

상황	Approach
Mean comparison	TTestIndPower (Cohen's d)
Proportion (CTR, conversion)	NormalIndPower
Complex design / non-standard test	Simulation
Multiple testing	Bonferroni / FDR adjusted α
Early-stopping	Group-sequential / α-spending
Cluster trial	Inflate n by design effect

기본값: α=0.05, power=0.8, MDE 의 domain-driven 정의 → statsmodels solve_power.

🔗 Graph

• 부모: Statistics · Probability Theory
• 변형: Sampling-Techniques
• 응용: Regression-Analysis-Foundations · Decision Theory
• Adjacent: Standard-Deviation-and-Variance · Multivariate-Analysis

🤖 LLM 활용

언제: explaining tradeoffs (n vs MDE vs power), interpreting power-analysis result, recommending design. 언제 X: 매 numerical solve — statsmodels / G*Power 의 사용.

❌ 안티패턴

• Post-hoc power on observed effect: 매 always ≈ 50% near significance — 매 meaningless. 매 MDE 의 reporting 의 사용.
• Underpowered + p-hack: small n + multiple comparisons → false-positive 의 폭증.
• Ignoring variance assumption: σ 의 estimate 의 잘못 → n calc 의 wrong.
• Power=0.5 acceptable: 매 0.8 minimum (medical 0.9) — 매 50% 의 coin flip.
• One-sided test for convenience: 매 effect direction 의 진짜 사전 hypothesis 면 OK, 매 fishing 면 X.

🧪 검증 / 중복

• Verified (Cohen 1988 Statistical Power Analysis, statsmodels docs, ICH E9 guideline).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — power formulas, statsmodels patterns, A/B test design.