---
id: wiki-2026-0508-principal-component-analysis
title: Principal Component Analysis
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [PCA, Karhunen-Loève Transform, KLT]
duplicate_of: none
source_trust_level: A
confidence_score: 0.93
verification_status: applied
tags: [linear-algebra, dimensionality-reduction, statistics, ml]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: scikit-learn/numpy
---

# Principal Component Analysis

## 매 한 줄
> **"매 max-variance orthogonal directions 의 projection"**. Pearson(1901) 의 origin 의 매 covariance eigendecomposition / SVD 의 reduction — 매 visualization, denoising, compression, 매 ML preprocessing 의 baseline.

## 매 핵심

### 매 수학
- 매 centered data X (n×p), 매 covariance C = XᵀX/(n-1).
- 매 eigendecomposition C = VΛVᵀ — 매 V 의 columns 가 principal axes.
- 매 SVD X = UΣVᵀ — 매 numerically stable path; 매 PC scores = UΣ.
- 매 explained variance ratio = λᵢ / Σλⱼ.

### 매 절차
1. 매 mean-center (and 보통 standardize).
2. 매 SVD or eigendecomp.
3. 매 top-k components 의 select (scree / variance threshold).
4. 매 project: Z = X V_k.

### 매 응용
1. 시각화 (2D/3D scatter).
2. Noise filtering / denoising.
3. Whitening preprocessing.
4. Compression (face images, MNIST baseline).
5. Genomics population structure (PC1 vs PC2).

## 💻 패턴

### scikit-learn full pipeline
```python
from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

pipe = make_pipeline(StandardScaler(), PCA(n_components=0.95, svd_solver="full"))
Z = pipe.fit_transform(X)             # explains >=95% variance
print(pipe[-1].explained_variance_ratio_.cumsum())
```

### NumPy SVD-based PCA (memory-efficient)
```python
import numpy as np
def pca(X, k):
    Xc = X - X.mean(axis=0, keepdims=True)
    U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
    components = Vt[:k]               # (k, p)
    scores     = U[:, :k] * S[:k]     # (n, k)
    explained  = (S**2) / (X.shape[0] - 1)
    ratio      = explained[:k] / explained.sum()
    return scores, components, ratio
```

### Randomized PCA (large p)
```python
from sklearn.decomposition import PCA
pca = PCA(n_components=50, svd_solver="randomized", random_state=0).fit(X)
# 매 O(n p k) cost — 매 huge p 의 effective.
```

### Incremental PCA (out-of-core)
```python
from sklearn.decomposition import IncrementalPCA
ipca = IncrementalPCA(n_components=20, batch_size=2048)
for chunk in batches(X, 2048):
    ipca.partial_fit(chunk)
Z = ipca.transform(X)
```

### Reconstruction & error
```python
X_hat = pca.inverse_transform(pca.transform(X))
recon_err = np.linalg.norm(X - X_hat) / np.linalg.norm(X)
```

### Scree / elbow plot
```python
import matplotlib.pyplot as plt
ev = pca.explained_variance_ratio_
plt.plot(np.arange(1, len(ev)+1), ev.cumsum(), marker="o")
plt.axhline(0.95, ls="--"); plt.xlabel("k"); plt.ylabel("cum. variance")
```

### Whitening for downstream classifier
```python
PCA(n_components=64, whiten=True).fit_transform(X)
# 매 unit-variance, 매 decorrelated → 매 SVM/logistic baseline 향상.
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| 매 small n, p | Full SVD PCA |
| 매 large p, moderate k | Randomized SVD |
| 매 streaming / OOC data | Incremental PCA |
| 매 nonlinear manifold | Kernel PCA, UMAP, t-SNE 의 대신 |
| 매 sparse / count data | TruncatedSVD (no centering) |
| 매 outlier-heavy | Robust PCA (RPCA), or median-centering |

**기본값**: 매 StandardScaler → PCA(n_components=0.95, svd_solver="auto").

## 🔗 Graph
- 부모: [[Linear-Algebra-Foundations]] · [[Singular-Value-Decomposition]] · [[Eigenvalues-and-Eigenvectors]]
- 변형: [[PCA-and-Dimension-Reduction]] · [[Kernel PCA]] · [[Sparse PCA]]
- 응용: [[Multivariate-Analysis]] · [[Signal-Processing-Foundations]] · [[Genomics PCA]]
- Adjacent: [[t-SNE]] · [[UMAP]] · [[Autoencoders]]

## 🤖 LLM 활용
**언제**: 매 dense numeric features 의 linear-correlated 의 visualize / denoise / compress.
**언제 X**: 매 nonlinear manifold (rolled / curved) — 매 UMAP / t-SNE / autoencoder 의 사용. 매 categorical / sparse count — 매 MCA / TruncatedSVD.

## ❌ 안티패턴
- **No mean-centering**: 매 first PC 의 just mean 의 direction 의 됨.
- **Scaling 무시**: 매 unit-mismatched features (mm vs kg) 의 dominate.
- **Top-k 의 magic number**: 매 scree / cum-variance 의 검토 없이 k 의 hardcode.
- **Test set leakage**: 매 fit on full data 후 split — fit 의 train 만.
- **Interpreting PCs as "factors"**: 매 PCA ≠ Factor Analysis (FA).

## 🧪 검증 / 중복
- Verified (Hastie *ESL* 2e §14.5; Bishop *PRML* §12.1; scikit-learn docs 1.5).
- 매 SVD path 의 numerically stable; 매 covariance-eigendecomp 의 ill-conditioned cases 에 worse.
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — SVD/randomized/incremental PCA full spec |