2nd/10_Wiki/Topic_Programming/Coding/DevOps_eBPF_Observability.md

id, title, category, status, source_trust_level, verification_status, created_at, updated_at, tags, tech_stack, applied_in, aliases

id	title	category	status	source_trust_level	verification_status	created_at	updated_at	tags	tech_stack	applied_in	aliases
devops-ebpf-observability	eBPF — Kernel-level Observability / Cilium / Pixie	Coding	draft	B	conceptual	2026-05-09	2026-05-09	devops ebpf observability cilium vibe-coding	language applicable_to C / BPF / Go DevOps		eBPF Cilium Pixie Tetragon Falco kernel observability sidecar-less

eBPF Observability

Kernel 안 sandboxed 코드 실행 → 모든 system call / network packet 관찰. 앱 변경 0 + 거의 zero overhead. Cilium (network), Pixie (auto-instrument), Tetragon (security), Falco (runtime).

📖 핵심 개념

• eBPF 프로그램: 커널에 attach.
• 종류: kprobe / tracepoint / XDP / cgroup hooks.
• BCC / libbpf / aya: 작성 도구.
• Cilium: K8s networking + observability.

💻 코드 패턴

Cilium (K8s networking)

# CNI 로 cilium 설치
helm install cilium cilium/cilium --namespace kube-system --set hubble.enabled=true

# Hubble — flow monitoring
hubble observe --pod prod/api
# 실시간 모든 connection 보임
hubble observe --to-namespace prod --verdict FORWARDED
hubble observe --pod prod/api --type drop

CiliumNetworkPolicy (L7 까지)

apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: api-policy }
spec:
  endpointSelector: { matchLabels: { app: api } }
  ingress:
  - fromEndpoints:
    - matchLabels: { app: web }
    toPorts:
    - ports: [{ port: "8080", protocol: TCP }]
      rules:
        http:
        - method: GET
          path: /api/.*
        - method: POST
          path: /api/orders

→ HTTP method / path 까지 정책. K8s NetworkPolicy 는 L4 만.

Pixie (auto-instrument 모든 service)

px deploy
# → cluster 의 모든 HTTP / DNS / MySQL / Redis call 자동 추적

# 사용자 정의 query (PXL)
df = px.DataFrame('http_events', start_time='-5m')
df.latency_ms = df.latency / 1e6
df = df[df.latency_ms > 1000]
px.display(df)

→ 코드 변경 0. SDK 없음.

Tetragon (security observability)

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata: { name: detect-shell }
spec:
  kprobes:
  - call: sys_execve
    syscall: true
    args:
    - { index: 0; type: string }
    selectors:
    - matchArgs:
      - { index: 0; operator: Equal; values: ["/bin/sh", "/bin/bash"] }

→ 임의 shell 실행 감지 + 알림.

Falco (runtime security)

- rule: Write below /etc
  desc: detect write to /etc
  condition: open_write and fd.name startswith /etc
  output: "File written %fd.name by %proc.cmdline"
  priority: WARNING

bpftrace (즉석 query)

# Read syscall 빈도 by process
bpftrace -e 'tracepoint:syscalls:sys_enter_read { @[comm] = count(); }'

# TCP latency distribution
bpftrace -e 'kprobe:tcp_sendmsg { @start[tid] = nsecs; }
            kretprobe:tcp_sendmsg /@start[tid]/ { @us = hist((nsecs - @start[tid])/1000); delete(@start[tid]); }'

libbpf-go / Aya (Rust) — 자체

// load BPF object
spec, _ := ebpf.LoadCollectionSpec("trace.bpf.o")
coll, _ := ebpf.NewCollection(spec)
defer coll.Close()

// attach
prog := coll.Programs["trace_open"]
link.Tracepoint("syscalls", "sys_enter_openat", prog, nil)

// read events from ringbuf
rd, _ := ringbuf.NewReader(coll.Maps["events"])
for {
    rec, _ := rd.Read()
    // process
}

Cilium service mesh (sidecar-less)

• Sidecar 없이 mesh 기능.
• mTLS / L7 정책 / observability.
• 자원 효율 (Istio 보다).

# 자동 활성
helm upgrade cilium ... --set serviceMesh.enabled=true

Comparison

Sidecar (Istio / Linkerd): 매 pod proxy, 1-3ms overhead.
eBPF (Cilium): 커널 안, 거의 zero overhead.
SDK 기반 (OTel): 코드 변경 필요.

eBPF = sidecar-less + 모든 service 자동.

Kernel 요구사항

eBPF: Linux 4.14+
권장:  5.10+
Cilium: kernel + cgroup v2

⚠️ Mac (M1/M2) 로컬 dev = Lima / Colima + Linux VM.

🤔 의사결정 기준

사용	추천
K8s networking + 정책	Cilium
Auto-observability	Pixie
Security / runtime	Tetragon / Falco
자체 instrumentation	libbpf / Aya
즉석 debugging	bpftrace
Sidecar mesh 싫음	Cilium service mesh

❌ 안티패턴

• Old kernel + eBPF 가정: 5.x 권장. CO-RE 사용.
• eBPF 권한 없음: CAP_BPF / CAP_SYS_ADMIN 필요.
• 모든 syscall trace: 오버헤드. filter.
• 사용자 메모리 dereference: kernel bug. helper functions 사용.
• Production 검증 없이 새 BPF 프로그램: kernel panic 가능 (verifier 가 막지만).
• Pixie 데이터 보안 무시: 모든 HTTP body 가 보임 — PII.

🤖 LLM 활용 힌트

• K8s = Cilium 디폴트 future.
• Auto-observability = Pixie.
• Security = Tetragon.
• Sidecar 자원 부담 → eBPF 가 답.

🔗 관련 문서

• DevOps_Service_Mesh_Deep
• DevOps_Observability_Stack
• Native_Perf_Tracing_Systrace