Files

T

Antigravity Agent f8b21af4be Wiki cleanup: error-doc removal, dedup merge, link normalization

10_Wiki/Topics 대규모 정리:
- 오류 캡처/미완성 stub 문서 227개 제거
- 교차폴더 중복 43클러스터 병합 (63파일 → redirect)
- 링크명 정규화: 깨진 링크 수정·redirect 직결·개념 매핑 ~2,400건
- 카테고리 MOC 6개 신규 생성
- Graph 섹션 미해결 related-keyword 링크 10,058건 제거

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-20 23:52:15 +09:00

6.1 KiB

Raw Blame History

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

title

PCA and Dimension Reduction

매 한 줄

"매 high-dim 데이터를 variance 보존하는 lower-dim 부분공간으로 사영". Pearson 1901 / Hotelling 1933 의 PCA 가 시초. 매 2026 의 modern landscape: linear PCA 는 여전히 baseline + interpretation, t-SNE/UMAP 가 visualization 의 default, autoencoder + contrastive 가 representation learning 의 핵심. 매 LLM embedding 의 PCA whitening 도 흔함.

매 핵심

매 PCA 수학

목표: orthogonal directions 중 variance 최대화.
계산: covariance Σ = X^T X / (n-1) → eigendecomposition Σ = V Λ V^T, top-k columns = principal components.
SVD form: X = UΣV^T → top-k V_k 가 components, score = X V_k.
Explained variance ratio: λ_i / Σ λ_j.
Whitening: X V_k Λ_k^{-1/2} → unit variance per dim.

매 family

Linear: PCA, Truncated SVD, Factor Analysis, ICA.
Manifold (preserve local): t-SNE, UMAP, LLE, Isomap.
Neural: Autoencoder, VAE, contrastive (SimCLR), DINO.
Random: Random projection (Johnson-Lindenstrauss).
Sparse: Sparse PCA, NMF.

매 응용

Visualization (t-SNE, UMAP for embeddings/scRNA-seq).
Compression (PCA whitening before downstream task).
Denoising (project, then reconstruct).
Feature engineering pre-classifier.
LLM embedding analysis (cluster interpretation, anisotropy fix).

💻 패턴

sklearn PCA

from sklearn.decomposition import PCA
import numpy as np

X = np.random.randn(1000, 100)
pca = PCA(n_components=10, whiten=True, random_state=42)
X_proj = pca.fit_transform(X)
print(pca.explained_variance_ratio_.cumsum())  # 누적 explained
print(pca.components_.shape)  # (10, 100)

Choose k via cumulative variance

def choose_k(X, threshold=0.95):
    pca = PCA().fit(X)
    cum = pca.explained_variance_ratio_.cumsum()
    return int(np.searchsorted(cum, threshold) + 1)

Truncated SVD (sparse / very large)

from sklearn.decomposition import TruncatedSVD
from scipy.sparse import csr_matrix

X_sparse = csr_matrix(X)  # 매 PCA centers → dense; TruncatedSVD 매 sparse-friendly
svd = TruncatedSVD(n_components=50, n_iter=7, random_state=42)
X_proj = svd.fit_transform(X_sparse)  # 매 LSA 의 핵심

UMAP (manifold, fast)

import umap

reducer = umap.UMAP(
    n_neighbors=15,    # local vs global
    min_dist=0.1,      # cluster separation
    n_components=2,
    metric="cosine",   # 매 LLM embedding
    random_state=42,
)
X_2d = reducer.fit_transform(X)

t-SNE (visualization)

from sklearn.manifold import TSNE

# 매 항상 PCA → t-SNE (속도/안정성)
X_pca = PCA(n_components=50).fit_transform(X)
X_2d = TSNE(n_components=2, perplexity=30, init="pca",
            learning_rate="auto").fit_transform(X_pca)

Autoencoder (PyTorch)

import torch.nn as nn

class AE(nn.Module):
    def __init__(self, d_in, d_latent):
        super().__init__()
        self.enc = nn.Sequential(
            nn.Linear(d_in, 256), nn.GELU(),
            nn.Linear(256, d_latent),
        )
        self.dec = nn.Sequential(
            nn.Linear(d_latent, 256), nn.GELU(),
            nn.Linear(256, d_in),
        )
    def forward(self, x):
        z = self.enc(x); return self.dec(z), z
# loss = MSE(x, x_hat). Nonlinear PCA 의 generalization.

LLM embedding whitening (anisotropy 완화)

def whiten_embeddings(E, k=None):
    """Anisotropy fix: subtract mean, decorrelate."""
    mu = E.mean(axis=0, keepdims=True)
    Ec = E - mu
    U, S, Vt = np.linalg.svd(Ec, full_matrices=False)
    if k is None:
        k = E.shape[1]
    W = (Vt[:k].T) / S[:k]   # whitening matrix
    return Ec @ W, mu, W

Random projection (very high-dim, fast)

from sklearn.random_projection import GaussianRandomProjection

rp = GaussianRandomProjection(n_components="auto", eps=0.1)
X_proj = rp.fit_transform(X)  # 매 Johnson-Lindenstrauss 보존

매 결정 기준

상황	Method
baseline, interpretable	PCA
sparse text-term-matrix	TruncatedSVD (LSA)
visualization 2D/3D	UMAP > t-SNE
nonlinear, learnable, downstream supervised	Autoencoder / SimCLR
n >> d, very high-dim	Random projection
non-negative parts (topics)	NMF
count data	LDA / NMF

기본값: 매 baseline PCA → 매 visualization UMAP → 매 representation learning contrastive.

🔗 Graph

부모: Linear-Algebra-Foundations · Statistics
변형: ICA
응용: Feature Engineering
Adjacent: t-SNE · UMAP · Autoencoder

🤖 LLM 활용

언제: 매 embedding analysis, 매 anisotropy whitening, 매 visualizing high-dim attention/activations. 언제 X: 매 매우 nonlinear task — autoencoder/contrastive 사용. 매 preserving exact distances 필요 — RP 만 보장.

❌ 안티패턴

PCA without scaling: 매 큰-단위 feature 가 dominate. 매 StandardScaler 필수.
t-SNE 결과 해석 으로 cluster 크기/거리 신뢰: 매 t-SNE 매 local-only — 매 global geometry 왜곡.
PCA 사용 후 inverse_transform 으로 outlier 제거 — 매 그러면 다시 fit X 매 outlier 의 영향 그대로.
n_components 선택을 임의 값 (e.g., 항상 2): 매 cumulative variance + downstream metric 기반 선택.
Test set 도 fit_transform: 매 leakage. 매 fit 은 train 만, test 는 transform.

🧪 검증 / 중복

Verified (Bishop PRML Ch 12; ESL Hastie et al.; UMAP McInnes 2018; sklearn docs).
신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — PCA & DR FULL with PCA/UMAP/AE/whitening patterns

6.1 KiB Raw Blame History