Files

T

koriweb d8a80f6272 chore(wiki): dangling 링크 canonical 정규화 (768파일/1200건)

이름만 다른(표기 변형) [[위키링크]]를 대상 문서의 canonical 제목으로 치환해
끊겼던 1,200개 링크를 연결. 제목/파일명 정규화 일치만 적용하고 별칭 매칭은
과병합 위험으로 제외(애매성 가드). 원본은 _link_reconcile_backup/ 에 백업.
도구: Datacollect/scripts/link_reconcile_apply.mjs

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-06-08 12:24:15 +09:00

4.8 KiB

Raw Blame History

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

title

Linear Algebra Foundations

매 한 줄

"매 ML 의 universal language". Linear Algebra 는 vector space, linear map, eigendecomposition, SVD 의 study — 매 modern ML/DL 의 numerical backbone. 2026 LLM era 에서 attention 은 매 Q@K.T softmax @V — 매 pure linear algebra 의 chain. GPU/TPU 의 design 자체가 매 LA primitive (GEMM) 위에 built.

매 핵심

매 Vector spaces

field \mathbb{F} (보통 \mathbb{R}, \mathbb{C})
closed under linear combinations
basis: minimal spanning set, dim
linear map T: V \to W — represent as matrix in basis

매 Decompositions

LU: A = LU — Gaussian elimination, O(n^3)
QR: A = QR, Q orthogonal — least squares
Eigendecomposition: A = V\Lambda V^{-1}, square only
SVD: A = U\Sigma V^\top — universal, rectangular
Cholesky: A = LL^\top, SPD only, fastest

매 Norms / inner products

\|x\|_2 = \sqrt{x^\top x}, \|x\|_p, \|x\|_\infty
Frobenius: \|A\|_F = \sqrt{\sum a_{ij}^2}
Spectral: \|A\|_2 = \sigma_{\max}(A)
Cauchy-Schwarz: |x^\top y| \le \|x\| \|y\|

매 응용

Attention: \text{softmax}(QK^\top / \sqrt{d}) V.
PCA: SVD of centered X.
Linear regression: normal eq w = (X^\top X)^{-1} X^\top y.
Graph Laplacian: spectral clustering.
Quantum states: complex Hilbert space.

💻 패턴

NumPy basics

import numpy as np
A = np.random.randn(5, 3)
# Solve Ax = b in least-squares
b = np.random.randn(5)
x, *_ = np.linalg.lstsq(A, b, rcond=None)
# rank, condition number, determinant
print(np.linalg.matrix_rank(A), np.linalg.cond(A))

SVD + low-rank approx

U, S, Vt = np.linalg.svd(A, full_matrices=False)
k = 2
A_lr = U[:, :k] @ np.diag(S[:k]) @ Vt[:k]   # rank-k approx
err = np.linalg.norm(A - A_lr, "fro")

Eigendecomposition (symmetric)

S = np.random.randn(4, 4); S = S + S.T
w, V = np.linalg.eigh(S)   # use eigh for symmetric (stable, real)
# reconstruction
assert np.allclose(V @ np.diag(w) @ V.T, S, atol=1e-10)

Cholesky for SPD systems

import scipy.linalg as la
A = np.random.randn(50, 50)
A = A.T @ A + np.eye(50)   # SPD
L = la.cholesky(A, lower=True)
b = np.random.randn(50)
y = la.solve_triangular(L, b, lower=True)
x = la.solve_triangular(L.T, y, lower=False)

JAX matmul + autodiff

import jax, jax.numpy as jnp
@jax.jit
def loss(W, x, y):
    return jnp.mean((x @ W - y) ** 2)
grad_fn = jax.grad(loss)

Power iteration (top eigenvector)

def power_iter(A, n_iter=200, tol=1e-10):
    x = np.random.randn(A.shape[0])
    x /= np.linalg.norm(x)
    for _ in range(n_iter):
        x_new = A @ x
        x_new /= np.linalg.norm(x_new)
        if np.linalg.norm(x_new - x) < tol: break
        x = x_new
    eig = x @ A @ x
    return eig, x

PyTorch attention (linear algebra core)

import torch, math
def attention(Q, K, V, mask=None):
    d = Q.size(-1)
    s = Q @ K.transpose(-2, -1) / math.sqrt(d)
    if mask is not None: s = s.masked_fill(mask == 0, -1e9)
    return torch.softmax(s, dim=-1) @ V

매 결정 기준

Problem	Method
solve Ax = b, square nonsingular	LU (np.linalg.solve)
SPD	Cholesky
least squares	QR or SVD
dimensionality reduction	SVD / PCA
symmetric eigen	eigh
sparse large	scipy.sparse.linalg / iterative

기본값: SVD when in doubt — most stable, universal.

🔗 Graph

변형: SVD · Eigendecomposition
응용: PCA · Attention Mechanism · Linear-Regression

🤖 LLM 활용

언제: 매 ML 의 derivation, debugging matrix shapes, performance reasoning. 언제 X: 매 task 가 combinatorial — graph algorithms 등.

❌ 안티패턴

inv(A) @ b 사용: numerically unstable + slow → use solve.
eig vs eigh 혼동: symmetric 인데 eig 사용 → complex eigenvalues from numerical noise.
Memory layout 무시: row vs column major → 10× slowdown.
Condition number 무시: ill-conditioned matrix → inversion blows up.
Dense for sparse: huge sparse → use scipy.sparse.

🧪 검증 / 중복

Verified (Strang 2016 textbook, Trefethen & Bau 1997, Golub & Van Loan 2013).
신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — vector spaces, decompositions, norms, NumPy/JAX patterns

4.8 KiB Raw Blame History Unescape Escape