2nd/10_Wiki/Topics/AI_and_ML/Load-Balancing-Strategies.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-load-balancing-strategies	Load Balancing Strategies	10_Wiki/Topics	verified	self	Load Balancer LB Reverse Proxy	none	A	0.9	applied	networking load-balancer nginx envoy haproxy aws-alb		2026-05-10	pending	language framework Config Nginx/Envoy/HAProxy/AWS

Load Balancing Strategies

매 한 줄

"매 LB = traffic 분산 + health check + sticky". 알고리즘은 균등성과 affinity의 trade-off.

매 핵심

매 알고리즘

• Round-robin: 순차. 단순, 동질 서버.
• Weighted RR: 용량 차등.
• Least connections: 활성 연결 최소. long-lived 적합.
• Least response time: 평균 응답 + connection 수.
• IP hash: client IP→server. session affinity.
• Consistent hash: cache/shard. node 추가 시 K/n key만 재배치.
• Random / P2C: power of two choices, least-loaded 선택. 간단·강력.

매 Layer

• L4 (TCP/UDP): 빠름, opaque. NLB, HAProxy TCP mode.
• L7 (HTTP): header/path 기반 라우팅, TLS 종료. ALB, Nginx, Envoy.

매 Health check

• Active: /health 주기적 GET
• Passive: 실제 요청 실패 카운트
• Slow start: 신규 노드에 점진적 traffic
• Outlier detection (Envoy): 연속 5xx → eject

매 Session affinity (sticky)

• Cookie-based (LB가 cookie 발급)
• Source IP (NAT 뒤에 모이면 unbalanced)
• Header-based (custom)

매 응용

1. Web app: ALB + multi-AZ
2. gRPC/HTTP2: Envoy (L7 multiplex)
3. DB read replica: ProxySQL, PgBouncer
4. Cache shard: consistent hash (Redis cluster)
5. Service mesh: Envoy sidecar (Istio, Linkerd)

💻 패턴

Nginx upstream

upstream backend {
    least_conn;                    # algorithm
    server app1:8080 weight=3 max_fails=3 fail_timeout=30s;
    server app2:8080 weight=1;
    server app3:8080 backup;       # only if others down
    keepalive 32;
}
server {
    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_next_upstream error timeout http_502 http_503;
    }
}

HAProxy

backend web
    balance leastconn
    option httpchk GET /health
    server s1 10.0.0.1:8080 check weight 100
    server s2 10.0.0.2:8080 check weight 50
    cookie SRV insert indirect nocache    # sticky

Envoy (L7, gRPC)

clusters:
- name: api
  type: STRICT_DNS
  lb_policy: LEAST_REQUEST
  health_checks:
  - timeout: 1s
    interval: 5s
    http_health_check: { path: "/health" }
  outlier_detection:
    consecutive_5xx: 5
    base_ejection_time: 30s
  load_assignment:
    cluster_name: api
    endpoints:
    - lb_endpoints:
      - endpoint: { address: { socket_address: { address: api1, port_value: 8080 }}}

AWS ALB (Terraform)

resource "aws_lb" "app" {
  name               = "app-alb"
  load_balancer_type = "application"
  subnets            = var.public_subnets
}
resource "aws_lb_target_group" "api" {
  port     = 8080
  protocol = "HTTP"
  health_check { path = "/health"; interval = 15 }
  stickiness { type = "lb_cookie"; enabled = true }
}

Consistent hash (Python sketch)

import bisect, hashlib
class Ring:
    def __init__(self, nodes, vnodes=150):
        self.ring = []
        for n in nodes:
            for i in range(vnodes):
                h = int(hashlib.md5(f"{n}-{i}".encode()).hexdigest(), 16)
                bisect.insort(self.ring, (h, n))
    def get(self, key):
        h = int(hashlib.md5(key.encode()).hexdigest(), 16)
        i = bisect.bisect(self.ring, (h, ""))
        return self.ring[i % len(self.ring)][1]

P2C (power of two choices)

import random
def p2c(servers, load):
    a, b = random.sample(servers, 2)
    return a if load[a] <= load[b] else b

매 결정 기준

상황	LB / 알고리즘
AWS web app	ALB + round_robin
TCP / 빠름	NLB / HAProxy L4
gRPC, mesh	Envoy least_request
Cache shard	Consistent hash
DB connection pool	PgBouncer/ProxySQL
Bare metal HTTP	Nginx least_conn
동질 서버 + short req	RR/Random
이질 서버 / long conn	Weighted least-conn

기본값: HTTP는 ALB or Nginx least_conn. 마이크로서비스는 Envoy.

🔗 Graph

• 부모: Distributed-Systems, Networking
• 변형: Consistent-Hashing, Reverse-Proxy
• 응용: High-Availability, Service-Mesh, CDN
• Adjacent: Health-Check, Auto-Scaling, API-Gateway

🤖 LLM 활용

언제: config 작성, 알고리즘 선택, troubleshoot 가설. 언제 X: production tuning은 metric 기반 검증 필수.

❌ 안티패턴

• IP hash + NAT/CGNAT (모든 client가 한 노드)
• Sticky session 과의존 → 노드 죽으면 세션 다 잃음
• Health check 너무 짧은 interval → flapping
• L7 LB에서 keepalive 비활성화 → 성능↓
• Hash ring vnode 부족 → 불균형
• TLS termination만 보고 backend mTLS 무시

🧪 검증 / 중복

• Verified (Nginx/Envoy/HAProxy docs, Mitzenmacher P2C, AWS ELB docs). 신뢰도 A.
• 중복: 없음.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — Envoy/AWS/consistent-hash 코드 추가