---
id: wiki-2026-0508-inner-product-spaces
title: Inner Product Spaces
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Hilbert Space, Dot Product, Vector Geometry]
duplicate_of: none
source_trust_level: A
confidence_score: 0.95
verification_status: applied
tags: [linear-algebra, geometry, ml-foundations, hilbert]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: numpy
---

# Inner Product Spaces

## 매 한 줄
> **"매 angle 매 length 의 algebraic foundation"**. Inner product $\langle x, y \rangle$ 매 vector space 매 geometry 부여 — 매 norm, distance, orthogonality 의 derived. 매 2026 ML embedding similarity (cosine, dot), kernel methods, Hilbert-space transformer 매 foundation.

## 매 핵심

### 매 axioms
- **Conjugate symmetry**: $\langle x,y \rangle = \overline{\langle y,x \rangle}$.
- **Linearity in 1st arg**: $\langle ax+by, z \rangle = a\langle x,z\rangle + b\langle y,z\rangle$.
- **Positive definite**: $\langle x,x \rangle \ge 0$, $=0 \iff x=0$.

### 매 derived 구조
- **Norm**: $\|x\| = \sqrt{\langle x,x \rangle}$.
- **Cosine**: $\cos \theta = \langle x,y\rangle / (\|x\|\|y\|)$.
- **Cauchy-Schwarz**: $|\langle x,y\rangle| \le \|x\|\|y\|$.
- **Orthogonality**: $\langle x,y\rangle = 0$.
- **Triangle**: $\|x+y\| \le \|x\| + \|y\|$.

### 매 응용
1. 매 ML embedding similarity (BERT, CLIP, GPT).
2. 매 vector DB (FAISS, Pinecone, Qdrant) — cosine/dot index.
3. 매 PCA, SVD — orthogonal basis.
4. 매 kernel methods (RKHS, SVM).
5. 매 Fourier analysis — orthogonal basis functions.

## 💻 패턴

### 1. Dot Product / Cosine
```python
import numpy as np
def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# 매 batched (faster):
def batch_cosine(A, B):
    A_n = A / np.linalg.norm(A, axis=1, keepdims=True)
    B_n = B / np.linalg.norm(B, axis=1, keepdims=True)
    return A_n @ B_n.T
```

### 2. Gram-Schmidt Orthogonalization
```python
def gram_schmidt(V):
    U = []
    for v in V:
        w = v.copy()
        for u in U:
            w -= np.dot(v, u) / np.dot(u, u) * u
        if np.linalg.norm(w) > 1e-10:
            U.append(w)
    return np.array(U)
```

### 3. Projection
```python
def project(v, onto):
    # 매 v 의 onto 위 projection
    return np.dot(v, onto) / np.dot(onto, onto) * onto
```

### 4. FAISS Inner Product Index
```python
import faiss
d = 768  # embedding dim
index = faiss.IndexFlatIP(d)  # inner product
embs_norm = embs / np.linalg.norm(embs, axis=1, keepdims=True)
index.add(embs_norm)
D, I = index.search(query_norm, k=10)  # 매 cosine via normalized IP
```

### 5. Kernel Trick (RBF)
```python
def rbf_kernel(X, Y, gamma=1.0):
    # 매 implicit inner product in infinite-dim RKHS
    sq = np.sum(X**2, 1)[:,None] + np.sum(Y**2, 1)[None,:] - 2*X@Y.T
    return np.exp(-gamma * sq)
```

### 6. QR Decomposition
```python
Q, R = np.linalg.qr(A)
# Q: orthonormal columns, R: upper triangular
# 매 least-squares solve: x = solve(R, Q.T @ b)
```

### 7. Fourier Inner Product (L2)
```python
# 매 <f,g> = ∫ f(t) conj(g(t)) dt
from scipy.integrate import quad
inner = quad(lambda t: f(t) * np.conj(g(t)), -np.pi, np.pi)[0]
```

## 매 결정 기준
| 상황 | Inner Product |
|---|---|
| Normalized embedding | Cosine (= dot of normalized) |
| Raw magnitude matters | Dot product |
| Probability distribution | Bhattacharyya / Hellinger |
| Function space | L2 integral |
| Non-Euclidean / kernel | RBF / polynomial kernel |

**기본값**: 매 cosine similarity 매 ML embedding standard, 매 normalize-once-then-dot 매 fast path.

## 🔗 Graph
- 부모: [[Linear-Algebra-Foundations]] · [[Theoretical-Computer-Science]]
- 변형: [[Hilbert-Space]]
- 응용: [[Similarity-Metrics]] · [[Singular-Value-Decomposition]] · [[PCA-and-Dimension-Reduction]]
- Adjacent: [[Eigenvalues-and-Eigenvectors]] · [[Operator-Theory]]

## 🤖 LLM 활용
**언제**: 매 embedding similarity 설계, 매 vector DB index 선택, 매 orthogonality 직관 설명.
**언제 X**: 매 non-metric (graph distance, edit distance) 영역.

## ❌ 안티패턴
- **Cosine on un-normalized**: 매 magnitude bias — 매 pre-normalize.
- **Gram-Schmidt 의 numerical instability**: 매 modified GS 또는 QR 사용.
- **Kernel 의 wrong gamma**: 매 grid-search / median heuristic.
- **L2 norm 가정 in non-Euclidean**: 매 manifold 의 incorrect.

## 🧪 검증 / 중복
- Verified (Axler, *Linear Algebra Done Right*; Trefethen, *Numerical LA*).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — geometry foundation + FAISS/kernel patterns |
