Files

T

Antigravity Agent f8b21af4be Wiki cleanup: error-doc removal, dedup merge, link normalization

10_Wiki/Topics 대규모 정리:
- 오류 캡처/미완성 stub 문서 227개 제거
- 교차폴더 중복 43클러스터 병합 (63파일 → redirect)
- 링크명 정규화: 깨진 링크 수정·redirect 직결·개념 매핑 ~2,400건
- 카테고리 MOC 6개 신규 생성
- Graph 섹션 미해결 related-keyword 링크 10,058건 제거

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-20 23:52:15 +09:00

3.5 KiB

Raw Blame History

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

title

Least Squares Methods

매 한 줄

"매 residual의 제곱합을 최소화하라". Gauss(1795)와 Legendre(1805)가 독립 발견; 매 modern ML의 MSE loss·linear regression·Kalman filter의 직접 조상이다. 2026에도 매 LoRA fine-tuning의 closed-form initializer로 살아있다.

매 핵심

매 정식화

OLS: minimize ‖y − Xβ‖²; closed-form β̂ = (XᵀX)⁻¹Xᵀy.
Weighted LS: residual에 weight W; β̂ = (XᵀWX)⁻¹XᵀWy.
Total LS: 매 X에도 noise — SVD 기반.
Regularized: Ridge (L2), Lasso (L1), Elastic Net.

매 수치적 안정성

Normal equation 직접 풀이는 매 condition number² 증폭 → QR/SVD 권장.
매 LAPACK gels 는 QR 사용.

매 응용

Linear regression (econometrics, ML baseline).
Curve fitting (lmfit, scipy.optimize.curve_fit).
Bundle adjustment (SLAM, photogrammetry).
Kalman update step (least-squares projection).

💻 패턴

OLS via NumPy (lstsq)

import numpy as np
beta, residuals, rank, sv = np.linalg.lstsq(X, y, rcond=None)

Ridge regression

def ridge(X, y, lam):
    n_features = X.shape[1]
    return np.linalg.solve(X.T @ X + lam * np.eye(n_features), X.T @ y)

Nonlinear least squares

from scipy.optimize import least_squares
def residual(params, x, y):
    a, b, c = params
    return y - (a * np.exp(-b * x) + c)
result = least_squares(residual, [1, 1, 0], args=(x, y))

QR-based solve (numerically stable)

Q, R = np.linalg.qr(X)
beta = np.linalg.solve(R, Q.T @ y)

Weighted LS

W_sqrt = np.sqrt(weights)
beta = np.linalg.lstsq(X * W_sqrt[:, None], y * W_sqrt, rcond=None)[0]

Recursive LS (online update)

def rls_update(P, beta, x, y, lam=0.99):
    k = P @ x / (lam + x @ P @ x)
    beta = beta + k * (y - x @ beta)
    P = (P - np.outer(k, x @ P)) / lam
    return P, beta

매 결정 기준

상황	Approach
Well-conditioned, small p	Normal equation
Ill-conditioned	QR or SVD
Many features, sparse	Lasso
Heteroscedastic noise	Weighted LS
Streaming data	RLS

기본값: np.linalg.lstsq (uses SVD-based GELSD).

🔗 Graph

부모: Linear-Algebra-Foundations · Optimization
변형: Ridge-Regression · Lasso
응용: Kalman-Filter-and-State-Tracking · Linear-Regression · Bundle-Adjustment

🤖 LLM 활용

언제: Linear/affine model fit, baseline regression, calibration curves. 언제 X: Outliers 다수 (use Huber/RANSAC), heavy non-linearity (use NN/GP).

❌ 안티패턴

Normal equation 남용: 매 ill-conditioned에서 매 silently 망함.
Outlier 무시: L2 loss는 매 outlier 민감 — robust loss 고려.
Multicollinearity 방치: VIF 점검 또는 Ridge.

🧪 검증 / 중복

Verified (Trefethen & Bau "Numerical Linear Algebra").
신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — OLS/Ridge/RLS patterns

3.5 KiB Raw Blame History Unescape Escape