Files

T

Antigravity Agent 9a135bd19d docs(10_Wiki): Topic_Business/General/Graphic/Programming을 Topics/ 하위로 이동

최상위 10_Wiki/Topic_*였던 4개 카테고리 폴더를 10_Wiki/Topics/Topic_* 로 재배치.
콘텐츠 변경 없음(순수 폴더 이동) — Topics/ 하위 나머지 폴더는 이미 지난 커밋에서
전부 정리된 상태(잔존 항목은 에이전트 운영 상태 및 사용자가 보존을 요청한
업데이트0615/무제 3.canvas 뿐).

2026-07-05 00:44:01 +09:00

4.7 KiB

Raw Blame History

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

title

SRE

매 한 줄

"매 reliability 의 feature 의 — 매 first feature 의". SRE (Site Reliability Engineering) 의 Google-originated discipline 의 software engineering 의 ops 의 applying. 핵심: SLOs 의 define, error budgets 의 enforce, toil 의 eliminate, blameless postmortems.

매 핵심

매 SRE 의 핵심 의 concepts

SLI: 매 measurement (e.g., 200-OK rate over 5min).
SLO: 매 target (e.g., 99.9% over 28d rolling).
SLA: 매 customer contract (with $ penalty).
Error budget: 매 100% - SLO. 매 budget 의 burn 시 release freeze.

매 four golden signals (Google)

Latency, Traffic, Errors, Saturation.

매 응용

SLO-driven alerting (multi-window burn rate).
Toil budget (≤50% of SRE time).
Blameless postmortem culture.

💻 패턴

Prometheus SLO recording rules

groups:
  - name: slo.rules
    interval: 30s
    rules:
      - record: api:availability:ratio_rate5m
        expr: |
          sum(rate(http_requests_total{job="api",code!~"5.."}[5m]))
          / sum(rate(http_requests_total{job="api"}[5m]))
      - record: api:availability:ratio_rate1h
        expr: |
          sum(rate(http_requests_total{job="api",code!~"5.."}[1h]))
          / sum(rate(http_requests_total{job="api"}[1h]))

Multi-window multi-burn-rate alert

- alert: ApiErrorBudgetFastBurn
  expr: |
    (1 - api:availability:ratio_rate5m) > (14.4 * 0.001)
    and
    (1 - api:availability:ratio_rate1h) > (14.4 * 0.001)
  for: 2m
  labels: { severity: page }
  annotations:
    summary: "Fast burn — 매 2% budget 의 1h 의 consume 의"

OpenTelemetry instrumentation (Node)

import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

new NodeSDK({
  traceExporter: new OTLPTraceExporter({ url: process.env.OTEL_ENDPOINT }),
  instrumentations: [getNodeAutoInstrumentations()],
}).start();

Runbook automation (Python)

import kubernetes.client as k8s
def remediate(pod_name: str, ns: str):
    api = k8s.CoreV1Api()
    api.delete_namespaced_pod(pod_name, ns)
    notify_slack(f"매 auto-restart {ns}/{pod_name} (high mem)")

Postmortem template

# Incident YYYY-MM-DD: <title>
**Status**: resolved
**Impact**: <users affected, $ lost, duration>
**Severity**: SEV-2

## Timeline (UTC)
- 14:02 alert fired
- 14:05 oncall paged
- 14:18 root cause identified
- 14:31 mitigated

## Root Cause
<technical>

## Action Items
- [ ] (P0) Fix race in checkout-svc — owner: @x
- [ ] (P1) Add SLO alert for queue depth — owner: @y

Toil tracking

type Toil = { repetitive: boolean; manual: boolean; automatable: boolean; ts: Date };
// dashboard: toil hours / total hours per quarter, target ≤50%

매 결정 기준

상황	SLO
user-facing read API	99.9% availability, p99 <300ms
user-facing write API	99.95% availability, p99 <500ms
internal batch	99.5% job completion within window
free-tier feature	99% (lower budget = ship faster)

기본값: 99.9% availability, multi-burn-rate alerts, weekly error-budget review.

🔗 Graph

부모: DevOps · Production Engineering
변형: Platform Engineering · CI/CD Pipeline & IDE Security Integration
응용: Observability · Chaos Engineering
Adjacent: Prometheus · OpenTelemetry

🤖 LLM 활용

언제: postmortem drafting from timeline, log anomaly summarization, runbook generation, oncall question answering. 언제 X: auto-remediation 의 LLM-only — 매 hallucinated kubectl 의 prod 의 destroy.

❌ 안티패턴

No SLO: 매 alert noise — 매 every blip 의 page.
100% uptime goal: 매 unattainable, 매 budget 0 = no innovation.
Blame culture: postmortem 의 finger-pointing — engineers 의 hide incidents.
Toil unbounded: SREs 의 burned out — quit within 12mo.

🧪 검증 / 중복

Verified (Google SRE Book, SRE Workbook, Prometheus docs, Sloth SLO generator).
신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — SLO + burn-rate + OTel patterns

4.7 KiB Raw Blame History

SRE