---
id: devops-ebpf-observability
title: eBPF — Kernel-level Observability / Cilium / Pixie
category: Coding
status: draft
source_trust_level: B
verification_status: conceptual
created_at: 2026-05-09
updated_at: 2026-05-09
tags: [devops, ebpf, observability, cilium, vibe-coding]
tech_stack: { language: "C / BPF / Go", applicable_to: ["DevOps"] }
applied_in: []
aliases: [eBPF, Cilium, Pixie, Tetragon, Falco, kernel observability, sidecar-less]
---

# eBPF Observability

> Kernel 안 sandboxed 코드 실행 → 모든 system call / network packet 관찰. **앱 변경 0 + 거의 zero overhead**. Cilium (network), Pixie (auto-instrument), Tetragon (security), Falco (runtime).

## 📖 핵심 개념
- eBPF 프로그램: 커널에 attach.
- 종류: kprobe / tracepoint / XDP / cgroup hooks.
- BCC / libbpf / aya: 작성 도구.
- Cilium: K8s networking + observability.

## 💻 코드 패턴

### Cilium (K8s networking)
```bash
# CNI 로 cilium 설치
helm install cilium cilium/cilium --namespace kube-system --set hubble.enabled=true
```

```bash
# Hubble — flow monitoring
hubble observe --pod prod/api
# 실시간 모든 connection 보임
hubble observe --to-namespace prod --verdict FORWARDED
hubble observe --pod prod/api --type drop
```

### CiliumNetworkPolicy (L7 까지)
```yaml
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata: { name: api-policy }
spec:
  endpointSelector: { matchLabels: { app: api } }
  ingress:
  - fromEndpoints:
    - matchLabels: { app: web }
    toPorts:
    - ports: [{ port: "8080", protocol: TCP }]
      rules:
        http:
        - method: GET
          path: /api/.*
        - method: POST
          path: /api/orders
```

→ HTTP method / path 까지 정책. K8s NetworkPolicy 는 L4 만.

### Pixie (auto-instrument 모든 service)
```bash
px deploy
# → cluster 의 모든 HTTP / DNS / MySQL / Redis call 자동 추적
```

```pxl
# 사용자 정의 query (PXL)
df = px.DataFrame('http_events', start_time='-5m')
df.latency_ms = df.latency / 1e6
df = df[df.latency_ms > 1000]
px.display(df)
```

→ 코드 변경 0. SDK 없음.

### Tetragon (security observability)
```yaml
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata: { name: detect-shell }
spec:
  kprobes:
  - call: sys_execve
    syscall: true
    args:
    - { index: 0; type: string }
    selectors:
    - matchArgs:
      - { index: 0; operator: Equal; values: ["/bin/sh", "/bin/bash"] }
```

→ 임의 shell 실행 감지 + 알림.

### Falco (runtime security)
```yaml
- rule: Write below /etc
  desc: detect write to /etc
  condition: open_write and fd.name startswith /etc
  output: "File written %fd.name by %proc.cmdline"
  priority: WARNING
```

### bpftrace (즉석 query)
```bash
# Read syscall 빈도 by process
bpftrace -e 'tracepoint:syscalls:sys_enter_read { @[comm] = count(); }'

# TCP latency distribution
bpftrace -e 'kprobe:tcp_sendmsg { @start[tid] = nsecs; }
            kretprobe:tcp_sendmsg /@start[tid]/ { @us = hist((nsecs - @start[tid])/1000); delete(@start[tid]); }'
```

### libbpf-go / Aya (Rust) — 자체
```go
// load BPF object
spec, _ := ebpf.LoadCollectionSpec("trace.bpf.o")
coll, _ := ebpf.NewCollection(spec)
defer coll.Close()

// attach
prog := coll.Programs["trace_open"]
link.Tracepoint("syscalls", "sys_enter_openat", prog, nil)

// read events from ringbuf
rd, _ := ringbuf.NewReader(coll.Maps["events"])
for {
    rec, _ := rd.Read()
    // process
}
```

### Cilium service mesh (sidecar-less)
- Sidecar 없이 mesh 기능.
- mTLS / L7 정책 / observability.
- 자원 효율 (Istio 보다).

```yaml
# 자동 활성
helm upgrade cilium ... --set serviceMesh.enabled=true
```

### Comparison
```
Sidecar (Istio / Linkerd): 매 pod proxy, 1-3ms overhead.
eBPF (Cilium): 커널 안, 거의 zero overhead.
SDK 기반 (OTel): 코드 변경 필요.

eBPF = sidecar-less + 모든 service 자동.
```

### Kernel 요구사항
```
eBPF: Linux 4.14+
권장:  5.10+
Cilium: kernel + cgroup v2
```

⚠️ Mac (M1/M2) 로컬 dev = Lima / Colima + Linux VM.

## 🤔 의사결정 기준
| 사용 | 추천 |
|---|---|
| K8s networking + 정책 | Cilium |
| Auto-observability | Pixie |
| Security / runtime | Tetragon / Falco |
| 자체 instrumentation | libbpf / Aya |
| 즉석 debugging | bpftrace |
| Sidecar mesh 싫음 | Cilium service mesh |

## ❌ 안티패턴
- **Old kernel + eBPF 가정**: 5.x 권장. CO-RE 사용.
- **eBPF 권한 없음**: CAP_BPF / CAP_SYS_ADMIN 필요.
- **모든 syscall trace**: 오버헤드. filter.
- **사용자 메모리 dereference**: kernel bug. helper functions 사용.
- **Production 검증 없이 새 BPF 프로그램**: kernel panic 가능 (verifier 가 막지만).
- **Pixie 데이터 보안 무시**: 모든 HTTP body 가 보임 — PII.

## 🤖 LLM 활용 힌트
- K8s = Cilium 디폴트 future.
- Auto-observability = Pixie.
- Security = Tetragon.
- Sidecar 자원 부담 → eBPF 가 답.

## 🔗 관련 문서
- [[DevOps_Service_Mesh_Deep]]
- [[DevOps_Observability_Stack]]
- [[Native_Perf_Tracing_Systrace]]