2nd/10_Wiki/Topics/Computer_Science_and_Theory/Principal-Component-Analysis.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-principal-component-analysis	Principal Component Analysis	10_Wiki/Topics	verified	self	PCA Karhunen-Loève Transform KLT	none	A	0.93	applied	linear-algebra dimensionality-reduction statistics ml		2026-05-10	pending	language framework python scikit-learn/numpy

Principal Component Analysis

매 한 줄

"매 max-variance orthogonal directions 의 projection". Pearson(1901) 의 origin 의 매 covariance eigendecomposition / SVD 의 reduction — 매 visualization, denoising, compression, 매 ML preprocessing 의 baseline.

매 핵심

매 수학

• 매 centered data X (n×p), 매 covariance C = XᵀX/(n-1).
• 매 eigendecomposition C = VΛVᵀ — 매 V 의 columns 가 principal axes.
• 매 SVD X = UΣVᵀ — 매 numerically stable path; 매 PC scores = UΣ.
• 매 explained variance ratio = λᵢ / Σλⱼ.

매 절차

1. 매 mean-center (and 보통 standardize).
2. 매 SVD or eigendecomp.
3. 매 top-k components 의 select (scree / variance threshold).
4. 매 project: Z = X V_k.

매 응용

1. 시각화 (2D/3D scatter).
2. Noise filtering / denoising.
3. Whitening preprocessing.
4. Compression (face images, MNIST baseline).
5. Genomics population structure (PC1 vs PC2).

💻 패턴

scikit-learn full pipeline

from sklearn.decomposition import PCA
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline

pipe = make_pipeline(StandardScaler(), PCA(n_components=0.95, svd_solver="full"))
Z = pipe.fit_transform(X)             # explains >=95% variance
print(pipe[-1].explained_variance_ratio_.cumsum())

NumPy SVD-based PCA (memory-efficient)

import numpy as np
def pca(X, k):
    Xc = X - X.mean(axis=0, keepdims=True)
    U, S, Vt = np.linalg.svd(Xc, full_matrices=False)
    components = Vt[:k]               # (k, p)
    scores     = U[:, :k] * S[:k]     # (n, k)
    explained  = (S**2) / (X.shape[0] - 1)
    ratio      = explained[:k] / explained.sum()
    return scores, components, ratio

Randomized PCA (large p)

from sklearn.decomposition import PCA
pca = PCA(n_components=50, svd_solver="randomized", random_state=0).fit(X)
# 매 O(n p k) cost — 매 huge p 의 effective.

Incremental PCA (out-of-core)

from sklearn.decomposition import IncrementalPCA
ipca = IncrementalPCA(n_components=20, batch_size=2048)
for chunk in batches(X, 2048):
    ipca.partial_fit(chunk)
Z = ipca.transform(X)

Reconstruction & error

X_hat = pca.inverse_transform(pca.transform(X))
recon_err = np.linalg.norm(X - X_hat) / np.linalg.norm(X)

Scree / elbow plot

import matplotlib.pyplot as plt
ev = pca.explained_variance_ratio_
plt.plot(np.arange(1, len(ev)+1), ev.cumsum(), marker="o")
plt.axhline(0.95, ls="--"); plt.xlabel("k"); plt.ylabel("cum. variance")

Whitening for downstream classifier

PCA(n_components=64, whiten=True).fit_transform(X)
# 매 unit-variance, 매 decorrelated → 매 SVM/logistic baseline 향상.

매 결정 기준

상황	Approach
매 small n, p	Full SVD PCA
매 large p, moderate k	Randomized SVD
매 streaming / OOC data	Incremental PCA
매 nonlinear manifold	Kernel PCA, UMAP, t-SNE 의 대신
매 sparse / count data	TruncatedSVD (no centering)
매 outlier-heavy	Robust PCA (RPCA), or median-centering

기본값: 매 StandardScaler → PCA(n_components=0.95, svd_solver="auto").

🔗 Graph

• 부모: Linear-Algebra-Foundations · Singular-Value-Decomposition · Eigenvalues-and-Eigenvectors
• 변형: PCA-and-Dimension-Reduction · Kernel PCA · Sparse PCA
• 응용: Multivariate-Analysis · Signal-Processing-Foundations · Genomics PCA
• Adjacent: t-SNE · UMAP · Autoencoders

🤖 LLM 활용

언제: 매 dense numeric features 의 linear-correlated 의 visualize / denoise / compress. 언제 X: 매 nonlinear manifold (rolled / curved) — 매 UMAP / t-SNE / autoencoder 의 사용. 매 categorical / sparse count — 매 MCA / TruncatedSVD.

❌ 안티패턴

• No mean-centering: 매 first PC 의 just mean 의 direction 의 됨.
• Scaling 무시: 매 unit-mismatched features (mm vs kg) 의 dominate.
• Top-k 의 magic number: 매 scree / cum-variance 의 검토 없이 k 의 hardcode.
• Test set leakage: 매 fit on full data 후 split — fit 의 train 만.
• Interpreting PCs as "factors": 매 PCA ≠ Factor Analysis (FA).

🧪 검증 / 중복

• Verified (Hastie ESL 2e §14.5; Bishop PRML §12.1; scikit-learn docs 1.5).
• 매 SVD path 의 numerically stable; 매 covariance-eigendecomp 의 ill-conditioned cases 에 worse.
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — SVD/randomized/incremental PCA full spec