---
id: wiki-2026-0508-sampling-techniques
title: Sampling Techniques
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Statistical Sampling, Survey Sampling, MCMC Sampling]
duplicate_of: none
source_trust_level: A
confidence_score: 0.9
verification_status: applied
tags: [statistics, sampling, monte-carlo, mcmc, survey]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: numpy / numpyro / scikit-learn
---

# Sampling Techniques

## 매 한 줄
> **"매 sampling 은 population 의 representative slice 를 얻는 art"**. Neyman (1934) 의 stratified sampling formalization 부터 매 modern MCMC (NUTS, HMC, SMC) 까지 — 매 statistics, ML 의 LLM-RLHF preference dataset 까지 backbone. 2026 에 Anthropic, OpenAI 의 RLAIF pipeline 도 매 stratified human-preference sampling 의 위에 동작.

## 매 핵심

### 매 Probability sampling family
- **Simple Random**: 매 uniform pick — baseline 이지만 small subgroup 누락 위험.
- **Stratified**: population 을 stratum 으로 partition, 매 stratum 당 SRS — variance reduction.
- **Cluster**: cluster 단위 pick (e.g. zipcode) — 매 cost↓ but design-effect↑.
- **Systematic**: every k-th — 매 list 가 random 일 때 유효, periodicity 시 bias.
- **Multistage**: cluster → stratified → SRS — 매 national survey 의 표준.

### 매 Non-probability (use with care)
- **Convenience**: 매 quickest, biased.
- **Snowball**: 매 hidden population (e.g. underground community).
- **Quota**: 매 demographic match 강제 — Online panel 에서 흔함.

### 매 Monte Carlo / MCMC
- **Importance**: q(x) 에서 sample 후 w=p/q reweight.
- **Rejection**: accept iff u·M·q(x) ≤ p(x).
- **MH/HMC/NUTS**: 매 high-dim posterior — Stan, NumPyro, BlackJAX 의 default.
- **SMC**: 매 sequential Bayes (particle filter generalization).

### 매 응용
1. RLHF preference data — 매 stratified by task family.
2. A/B test bucketing — 매 hash-mod stratify on user_id.
3. Image diffusion — 매 latent prior sampling (DDIM, EDM2).
4. NN training mini-batch — 매 weighted sampler for class imbalance.

## 💻 패턴

### Stratified split (sklearn)
```python
from sklearn.model_selection import StratifiedShuffleSplit

sss = StratifiedShuffleSplit(n_splits=1, test_size=0.2, random_state=42)
for train_idx, test_idx in sss.split(X, y):
    X_tr, X_te = X[train_idx], X[test_idx]
    y_tr, y_te = y[train_idx], y[test_idx]
```

### Reservoir sampling (streaming, k items)
```python
import random

def reservoir(stream, k):
    res = []
    for i, item in enumerate(stream):
        if i < k:
            res.append(item)
        else:
            j = random.randint(0, i)
            if j < k:
                res[j] = item
    return res
```

### Weighted sampling (alias method, O(1) per draw)
```python
import numpy as np

class AliasSampler:
    def __init__(self, weights):
        n = len(weights)
        p = np.asarray(weights, dtype=np.float64)
        p = p * n / p.sum()
        self.prob = np.zeros(n); self.alias = np.zeros(n, dtype=np.int64)
        small = [i for i in range(n) if p[i] < 1.0]
        large = [i for i in range(n) if p[i] >= 1.0]
        while small and large:
            s, l = small.pop(), large.pop()
            self.prob[s] = p[s]; self.alias[s] = l
            p[l] -= 1.0 - p[s]
            (small if p[l] < 1.0 else large).append(l)
        for i in large + small:
            self.prob[i] = 1.0
        self.n = n
    def draw(self, k=1):
        i = np.random.randint(0, self.n, size=k)
        u = np.random.random(k)
        return np.where(u < self.prob[i], i, self.alias[i])
```

### NUTS posterior sampling (NumPyro 0.16)
```python
import numpyro, numpyro.distributions as dist
from numpyro.infer import MCMC, NUTS

def model(x, y):
    a = numpyro.sample("a", dist.Normal(0, 1))
    b = numpyro.sample("b", dist.Normal(0, 1))
    sigma = numpyro.sample("sigma", dist.HalfNormal(1.0))
    numpyro.sample("obs", dist.Normal(a + b * x, sigma), obs=y)

mcmc = MCMC(NUTS(model), num_warmup=1000, num_samples=2000)
mcmc.run(jax.random.PRNGKey(0), x, y)
```

### Importance sampling (with effective sample size)
```python
def importance(samples, log_p, log_q):
    log_w = log_p - log_q
    log_w -= log_w.max()
    w = np.exp(log_w); w /= w.sum()
    ess = 1.0 / np.sum(w ** 2)
    return w, ess
```

### Hash-mod A/B bucketing (deterministic)
```python
import hashlib

def bucket(user_id, salt, n_buckets):
    h = hashlib.blake2b(f"{salt}:{user_id}".encode(), digest_size=8).digest()
    return int.from_bytes(h, "big") % n_buckets
```

### Class-balanced PyTorch sampler
```python
from torch.utils.data import WeightedRandomSampler
import numpy as np

def balanced_sampler(labels):
    counts = np.bincount(labels)
    weights = 1.0 / counts[labels]
    return WeightedRandomSampler(weights, len(labels), replacement=True)
```

## 매 결정 기준

| 상황 | Approach |
|---|---|
| Population homogeneous | SRS |
| Subgroup variance 다름 | Stratified |
| Geographic cost 제약 | Cluster / Multistage |
| Streaming / unknown size | Reservoir |
| Heavy-tail Bayes posterior | NUTS / HMC |
| Sequential filtering | SMC / Particle Filter |
| Class imbalance training | Weighted / Balanced sampler |

**기본값**: training 은 stratified split, eval 은 i.i.d. test set, Bayes 는 NUTS, RLHF 는 task-stratified preference sampling.

## 🔗 Graph
- 부모: [[Statistics]] · [[Probability Theory|Probability-Theory-Foundations]]
- 변형: [[Monte-Carlo-Integration]] · [[Particle-Filter-Algorithms]]
- 응용: [[데이터 사이언스 및 ML 엔지니어링|Reinforcement_Learning_Fundamentals]] · [[Information Retrieval (IR)]]
- Adjacent: [[Posterior-and-Prior-Probability]] · [[Statistical-Power]]

## 🤖 LLM 활용
**언제**: dataset construction, eval design, RLHF/RLAIF preference collection, posterior inference, Monte-Carlo estimation 의 framing 때.
**언제 X**: full-population accessible (census), deterministic computation 으로 충분한 case.

## ❌ 안티패턴
- **Convenience sampling 으로 production decision**: 매 selection bias 직격탄.
- **stratified 변수 leakage**: stratify-by-target 후 train/test split 시 target 정보 누설.
- **MCMC 의 single-chain 만 신뢰**: 매 multi-chain + R-hat / ESS 검증 필수.
- **Reservoir sampling 의 reproducibility 무시**: seed pin 안 하면 매 재현 불가.

## 🧪 검증 / 검토
- Verified (Lohr 2021 "Sampling: Design and Analysis"; Gelman BDA3 ch.10–12; NumPyro docs 0.16).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — probability/non-probability/MCMC families, alias-method/NUTS/reservoir patterns, RLHF context |