Antigravity Agent
192 lines
4.9 KiB
Markdown
192 lines
4.9 KiB
Markdown
---
|
|
id: devops-ebpf-observability
|
|
title: eBPF — Kernel-level Observability / Cilium / Pixie
|
|
category: Coding
|
|
status: draft
|
|
source_trust_level: B
|
|
verification_status: conceptual
|
|
created_at: 2026-05-09
|
|
updated_at: 2026-05-09
|
|
tags: [devops, ebpf, observability, cilium, vibe-coding]
|
|
tech_stack: { language: "C / BPF / Go", applicable_to: ["DevOps"] }
|
|
applied_in: []
|
|
aliases: [eBPF, Cilium, Pixie, Tetragon, Falco, kernel observability, sidecar-less]
|
|
---
|
|
|
|
# eBPF Observability
|
|
|
|
> Kernel 안 sandboxed 코드 실행 → 모든 system call / network packet 관찰. **앱 변경 0 + 거의 zero overhead**. Cilium (network), Pixie (auto-instrument), Tetragon (security), Falco (runtime).
|
|
|
|
## 📖 핵심 개념
|
|
- eBPF 프로그램: 커널에 attach.
|
|
- 종류: kprobe / tracepoint / XDP / cgroup hooks.
|
|
- BCC / libbpf / aya: 작성 도구.
|
|
- Cilium: K8s networking + observability.
|
|
|
|
## 💻 코드 패턴
|
|
|
|
### Cilium (K8s networking)
|
|
```bash
|
|
# CNI 로 cilium 설치
|
|
helm install cilium cilium/cilium --namespace kube-system --set hubble.enabled=true
|
|
```
|
|
|
|
```bash
|
|
# Hubble — flow monitoring
|
|
hubble observe --pod prod/api
|
|
# 실시간 모든 connection 보임
|
|
hubble observe --to-namespace prod --verdict FORWARDED
|
|
hubble observe --pod prod/api --type drop
|
|
```
|
|
|
|
### CiliumNetworkPolicy (L7 까지)
|
|
```yaml
|
|
apiVersion: cilium.io/v2
|
|
kind: CiliumNetworkPolicy
|
|
metadata: { name: api-policy }
|
|
spec:
|
|
endpointSelector: { matchLabels: { app: api } }
|
|
ingress:
|
|
- fromEndpoints:
|
|
- matchLabels: { app: web }
|
|
toPorts:
|
|
- ports: [{ port: "8080", protocol: TCP }]
|
|
rules:
|
|
http:
|
|
- method: GET
|
|
path: /api/.*
|
|
- method: POST
|
|
path: /api/orders
|
|
```
|
|
|
|
→ HTTP method / path 까지 정책. K8s NetworkPolicy 는 L4 만.
|
|
|
|
### Pixie (auto-instrument 모든 service)
|
|
```bash
|
|
px deploy
|
|
# → cluster 의 모든 HTTP / DNS / MySQL / Redis call 자동 추적
|
|
```
|
|
|
|
```pxl
|
|
# 사용자 정의 query (PXL)
|
|
df = px.DataFrame('http_events', start_time='-5m')
|
|
df.latency_ms = df.latency / 1e6
|
|
df = df[df.latency_ms > 1000]
|
|
px.display(df)
|
|
```
|
|
|
|
→ 코드 변경 0. SDK 없음.
|
|
|
|
### Tetragon (security observability)
|
|
```yaml
|
|
apiVersion: cilium.io/v1alpha1
|
|
kind: TracingPolicy
|
|
metadata: { name: detect-shell }
|
|
spec:
|
|
kprobes:
|
|
- call: sys_execve
|
|
syscall: true
|
|
args:
|
|
- { index: 0; type: string }
|
|
selectors:
|
|
- matchArgs:
|
|
- { index: 0; operator: Equal; values: ["/bin/sh", "/bin/bash"] }
|
|
```
|
|
|
|
→ 임의 shell 실행 감지 + 알림.
|
|
|
|
### Falco (runtime security)
|
|
```yaml
|
|
- rule: Write below /etc
|
|
desc: detect write to /etc
|
|
condition: open_write and fd.name startswith /etc
|
|
output: "File written %fd.name by %proc.cmdline"
|
|
priority: WARNING
|
|
```
|
|
|
|
### bpftrace (즉석 query)
|
|
```bash
|
|
# Read syscall 빈도 by process
|
|
bpftrace -e 'tracepoint:syscalls:sys_enter_read { @[comm] = count(); }'
|
|
|
|
# TCP latency distribution
|
|
bpftrace -e 'kprobe:tcp_sendmsg { @start[tid] = nsecs; }
|
|
kretprobe:tcp_sendmsg /@start[tid]/ { @us = hist((nsecs - @start[tid])/1000); delete(@start[tid]); }'
|
|
```
|
|
|
|
### libbpf-go / Aya (Rust) — 자체
|
|
```go
|
|
// load BPF object
|
|
spec, _ := ebpf.LoadCollectionSpec("trace.bpf.o")
|
|
coll, _ := ebpf.NewCollection(spec)
|
|
defer coll.Close()
|
|
|
|
// attach
|
|
prog := coll.Programs["trace_open"]
|
|
link.Tracepoint("syscalls", "sys_enter_openat", prog, nil)
|
|
|
|
// read events from ringbuf
|
|
rd, _ := ringbuf.NewReader(coll.Maps["events"])
|
|
for {
|
|
rec, _ := rd.Read()
|
|
// process
|
|
}
|
|
```
|
|
|
|
### Cilium service mesh (sidecar-less)
|
|
- Sidecar 없이 mesh 기능.
|
|
- mTLS / L7 정책 / observability.
|
|
- 자원 효율 (Istio 보다).
|
|
|
|
```yaml
|
|
# 자동 활성
|
|
helm upgrade cilium ... --set serviceMesh.enabled=true
|
|
```
|
|
|
|
### Comparison
|
|
```
|
|
Sidecar (Istio / Linkerd): 매 pod proxy, 1-3ms overhead.
|
|
eBPF (Cilium): 커널 안, 거의 zero overhead.
|
|
SDK 기반 (OTel): 코드 변경 필요.
|
|
|
|
eBPF = sidecar-less + 모든 service 자동.
|
|
```
|
|
|
|
### Kernel 요구사항
|
|
```
|
|
eBPF: Linux 4.14+
|
|
권장: 5.10+
|
|
Cilium: kernel + cgroup v2
|
|
```
|
|
|
|
⚠️ Mac (M1/M2) 로컬 dev = Lima / Colima + Linux VM.
|
|
|
|
## 🤔 의사결정 기준
|
|
| 사용 | 추천 |
|
|
|---|---|
|
|
| K8s networking + 정책 | Cilium |
|
|
| Auto-observability | Pixie |
|
|
| Security / runtime | Tetragon / Falco |
|
|
| 자체 instrumentation | libbpf / Aya |
|
|
| 즉석 debugging | bpftrace |
|
|
| Sidecar mesh 싫음 | Cilium service mesh |
|
|
|
|
## ❌ 안티패턴
|
|
- **Old kernel + eBPF 가정**: 5.x 권장. CO-RE 사용.
|
|
- **eBPF 권한 없음**: CAP_BPF / CAP_SYS_ADMIN 필요.
|
|
- **모든 syscall trace**: 오버헤드. filter.
|
|
- **사용자 메모리 dereference**: kernel bug. helper functions 사용.
|
|
- **Production 검증 없이 새 BPF 프로그램**: kernel panic 가능 (verifier 가 막지만).
|
|
- **Pixie 데이터 보안 무시**: 모든 HTTP body 가 보임 — PII.
|
|
|
|
## 🤖 LLM 활용 힌트
|
|
- K8s = Cilium 디폴트 future.
|
|
- Auto-observability = Pixie.
|
|
- Security = Tetragon.
|
|
- Sidecar 자원 부담 → eBPF 가 답.
|
|
|
|
## 🔗 관련 문서
|
|
- [[DevOps_Service_Mesh_Deep]]
|
|
- [[DevOps_Observability_Stack]]
|
|
- [[Native_Perf_Tracing_Systrace]]
|