---
id: wiki-2026-0508-locality-sensitive-hashing
title: Locality Sensitive Hashing
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [LSH, Locality-Sensitive-Hashing, MinHash, SimHash]
duplicate_of: none
source_trust_level: A
confidence_score: 0.93
verification_status: applied
tags: [hashing, ann, similarity-search, embeddings, retrieval]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: python
  framework: datasketch-faiss
---

# Locality Sensitive Hashing

## 매 한 줄
> **"매 가까운 점은 매 같은 bucket에 떨어진다"**. LSH는 매 metric similarity를 hash collision probability로 변환하여 매 sub-linear ANN(approximate nearest neighbor) search를 가능케 한다. 2026 vector DB 시대에 IVF-PQ·HNSW에 밀렸지만, 매 streaming dedup·document near-dup detection에서 매 dominant.

## 매 핵심

### 매 family들
- **MinHash (Jaccard)**: 매 set similarity — shingled documents.
- **SimHash (cosine)**: 매 random hyperplane projection.
- **p-stable LSH (L2)**: 매 random Gaussian projection + bucketing.
- **Cross-polytope LSH**: 매 angular distance, 매 unit sphere 위.

### 매 amplification
- AND-construction: 매 k hashes 모두 일치 → false positive ↓.
- OR-construction: 매 L tables 중 하나라도 collision → recall ↑.
- (k, L) tuning이 매 핵심.

### 매 응용
1. Plagiarism / near-duplicate detection (web, code).
2. Streaming deduplication (logs, training data).
3. Genomic sequence matching.
4. Pre-filter for vector DBs at billion scale.

## 💻 패턴

### MinHash with datasketch
```python
from datasketch import MinHash, MinHashLSH
def shingles(text, k=3):
    return {text[i:i+k] for i in range(len(text)-k+1)}
m = MinHash(num_perm=128)
for s in shingles(doc):
    m.update(s.encode())
lsh = MinHashLSH(threshold=0.7, num_perm=128)
lsh.insert("doc1", m)
matches = lsh.query(m_query)
```

### SimHash (64-bit)
```python
import numpy as np
def simhash(features, bits=64):
    v = np.zeros(bits)
    for f, w in features:
        h = hash(f)
        for i in range(bits):
            v[i] += w if (h >> i) & 1 else -w
    return sum(1 << i for i in range(bits) if v[i] > 0)
```

### Random projection LSH (cosine)
```python
class CosineLSH:
    def __init__(self, dim, n_planes=16):
        self.planes = np.random.randn(n_planes, dim)
    def hash(self, x):
        return tuple((self.planes @ x > 0).astype(int))
```

### p-stable LSH (Euclidean)
```python
def l2_hash(x, a, b, w):
    return int((a @ x + b) // w)
# a ~ N(0, I), b ~ U(0, w)
```

### Banding for MinHash
```python
def bands(sig, b, r):
    return [tuple(sig[i*r:(i+1)*r]) for i in range(b)]
# threshold ~ (1/b)^(1/r)
```

### FAISS LSH index (for L2)
```python
import faiss
index = faiss.IndexLSH(dim, n_bits=256)
index.add(X)
D, I = index.search(query, k=10)
```

## 매 결정 기준
| Metric | LSH family |
|---|---|
| Jaccard | MinHash |
| Cosine | SimHash / random hyperplane |
| Euclidean | p-stable / IndexLSH |
| Hamming | Bit sampling |

**기본값**: HNSW 우선, billion-scale dedup만 LSH.

## 🔗 Graph
- 부모: [[Hash-Functions-and-Maps]] · [[Approximate-Nearest-Neighbor]]
- 변형: [[MinHash]] · [[SimHash]] · [[Cross-Polytope-LSH]]
- 응용: [[Near-Duplicate-Detection]] · [[Vector-Database]] · [[Streaming-Dedup]]
- Adjacent: [[HNSW]] · [[Product-Quantization]] · [[Bloom-Filters in Search]]

## 🤖 LLM 활용
**언제**: Billion-scale dedup, Jaccard-based near-dup detection, streaming pipeline.
**언제 X**: High-recall ANN with dense embeddings (use HNSW/IVF-PQ).

## ❌ 안티패턴
- **단일 hash table**: recall 낮음 — 매 L tables 필수.
- **k 너무 큼**: false neg 폭증.
- **Dense embedding에 MinHash**: 매 잘못된 family 선택.

## 🧪 검증 / 중복
- Verified (Indyk & Motwani 1998; Mining of Massive Datasets ch.3).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — LSH families + datasketch/FAISS examples |