---
id: wiki-2026-0508-eigenvalues-and-eigenvectors
title: Eigenvalues and Eigenvectors
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Spectral Decomposition, Eigendecomposition, EVD]
duplicate_of: none
source_trust_level: A
confidence_score: 0.95
verification_status: applied
tags: [linear-algebra, math, ml, pca, spectral]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: NumPy/SciPy
---

# Eigenvalues and Eigenvectors

## 매 한 줄
> **"매 Av = λv — 매 matrix 의 invariant direction 의 stretch factor"**. Cauchy origin (1829), 매 quantum mechanics, PCA, PageRank, GNN, transformer attention spectra 의 omnipresent. 매 ML interpretability 의 2024-26 hot topic (eigenvalue distribution of transformer weights, Hessian spectrum for optimization).

## 매 핵심

### 매 정의
- A **n×n** matrix, A v = λ v with v ≠ 0.
- λ: eigenvalue (scalar). v: eigenvector (direction unchanged).
- Characteristic polynomial: det(A − λI) = 0.
- Spectrum: set of eigenvalues σ(A).

### 매 properties
- Symmetric A: real eigenvalues, orthogonal eigenvectors (spectral theorem).
- Trace = sum of eigenvalues; det = product.
- Rank = number of nonzero eigenvalues (for diagonalizable).
- Condition number κ = |λ_max| / |λ_min| (for SPD).

### 매 응용
1. PCA / dimensionality reduction.
2. PageRank (dominant eigenvector of transition matrix).
3. Spectral clustering / graph Laplacian.
4. Quantum mechanics (energy eigenstates).
5. Stability analysis (Jacobian eigenvalues).
6. Optimizer Hessian (curvature analysis).
7. Transformer attention eigenvalue analysis.

## 💻 패턴

### Compute eigendecomposition
```python
import numpy as np

A = np.array([[4, 1], [2, 3]], dtype=float)
eigvals, eigvecs = np.linalg.eig(A)
print(eigvals)              # [5. 2.]
print(eigvecs)              # columns are eigenvectors

# Symmetric — use eigh (faster, real)
S = (A + A.T) / 2
w, V = np.linalg.eigh(S)
```

### PCA via eigendecomposition
```python
def pca(X, k):
    X_c = X - X.mean(0)
    cov = (X_c.T @ X_c) / (len(X) - 1)
    w, V = np.linalg.eigh(cov)
    idx = np.argsort(w)[::-1][:k]
    return X_c @ V[:, idx], w[idx]
```

### PageRank (power iteration)
```python
def pagerank(P, d=0.85, n_iter=100, tol=1e-8):
    n = P.shape[0]
    v = np.ones(n) / n
    for _ in range(n_iter):
        v_new = d * P.T @ v + (1 - d) / n
        if np.linalg.norm(v_new - v, 1) < tol:
            return v_new
        v = v_new
    return v
```

### Spectral clustering
```python
from scipy.sparse.csgraph import laplacian
from sklearn.cluster import KMeans

def spectral_cluster(W, k):
    L = laplacian(W, normed=True)
    w, V = np.linalg.eigh(L)
    embed = V[:, :k]               # smallest k eigvecs of L
    return KMeans(n_clusters=k).fit_predict(embed)
```

### Power iteration (largest eigenvalue)
```python
def power_iter(A, n_iter=1000, tol=1e-10):
    v = np.random.randn(A.shape[0]); v /= np.linalg.norm(v)
    for _ in range(n_iter):
        Av = A @ v
        lam = v @ Av
        v_new = Av / np.linalg.norm(Av)
        if np.linalg.norm(v_new - v) < tol:
            return lam, v_new
        v = v_new
    return lam, v
```

### Hessian top-k eigenvalues (Lanczos)
```python
from scipy.sparse.linalg import eigsh

# H is LinearOperator approximating Hessian via HVP
top_eigs, _ = eigsh(H, k=20, which='LA')  # Lanczos
```

### Stability of fixed point
```python
def is_stable(jacobian):
    w = np.linalg.eigvals(jacobian)
    return np.all(w.real < 0)  # continuous-time
```

### Transformer weight spectrum
```python
def layer_spectrum(W):
    # Often heavy-tailed in trained transformers (Martin & Mahoney)
    s = np.linalg.svd(W, compute_uv=False)
    return s ** 2  # eigvals of W^T W
```

## 매 결정 기준
| 상황 | Method |
|---|---|
| Small dense matrix | `np.linalg.eig` / `eigh` |
| Large sparse, top-k | Lanczos / Arnoldi (`scipy.sparse.linalg.eigsh`) |
| Symmetric / Hermitian | Always use `eigh` (faster + numerically stable) |
| Just dominant eigenvalue | Power iteration |
| PCA on huge data | Randomized SVD |
| Stability | Real parts of Jacobian eigvals |

**기본값**: `eigh` for symmetric; randomized SVD for high-dim ML.

## 🔗 Graph
- 부모: [[Linear-Algebra-Foundations|Linear-Algebra]]
- 변형: [[SVD]]
- 응용: [[PCA]] · [[Spectral-Clustering]] · [[Kalman-Filter-and-State-Tracking]]
- Adjacent: [[Gimbals-and-Orientation]]

## 🤖 LLM 활용
**언제**: derive eigendecomposition, explain spectral properties, generate PCA/PageRank code, debug numerical issues (e.g. why eig gives complex for "symmetric" matrix).
**언제 X**: large-scale numerical solvers (use ARPACK/Spectra), formal proofs of spectral theorems.

## ❌ 안티패턴
- **`eig` on symmetric matrix**: use `eigh` (real output, 2-3× faster, more stable).
- **Forming `A^T A` for SVD**: numerically poor; use direct SVD.
- **Power iteration on matrix with same-magnitude top eigenvalues**: won't converge; use shifted variants.
- **Ignoring complex eigenvalues**: real matrices can have complex eigvals (rotations).
- **Confusing eigvals with singular values**: only equal for symmetric PSD; for general A, σ_i = √(eigvals(A^T A)).
- **Using PCA on non-centered data**: must subtract mean first.

## 🧪 검증 / 중복
- Verified (Trefethen & Bau "Numerical Linear Algebra", Golub & Van Loan, NumPy/SciPy docs, Martin & Mahoney "Implicit Self-Regularization in Deep Neural Networks" 2018).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — eig/PCA/PageRank/spectral clustering/Lanczos patterns |
