Antigravity Agent
f8b21af4be
10_Wiki/Topics 대규모 정리: - 오류 캡처/미완성 stub 문서 227개 제거 - 교차폴더 중복 43클러스터 병합 (63파일 → redirect) - 링크명 정규화: 깨진 링크 수정·redirect 직결·개념 매핑 ~2,400건 - 카테고리 MOC 6개 신규 생성 - Graph 섹션 미해결 related-keyword 링크 10,058건 제거 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
211 lines
7.0 KiB
Markdown
211 lines
7.0 KiB
Markdown
---
|
|
id: wiki-2026-0508-로그-logs
|
|
title: 로그 (Logs)
|
|
category: 10_Wiki/Topics
|
|
status: verified
|
|
canonical_id: self
|
|
aliases: [Logs, Logging, Application Logs]
|
|
duplicate_of: none
|
|
source_trust_level: A
|
|
confidence_score: 0.9
|
|
verification_status: applied
|
|
tags: [observability, logging, sre, telemetry, structured-logging]
|
|
raw_sources: []
|
|
last_reinforced: 2026-05-10
|
|
github_commit: pending
|
|
tech_stack:
|
|
language: TypeScript/Go
|
|
framework: OpenTelemetry / Loki / Datadog
|
|
---
|
|
|
|
# 로그 (Logs)
|
|
|
|
## 매 한 줄
|
|
> **"매 log 는 system 의 외부 관측 가능성(observability)의 세 기둥 중 하나로 매 metric/trace 의 sibling 이며 매 high-cardinality narrative event 의 sink"**. 매 2026 의 표준은 structured JSON + OpenTelemetry semantic conventions + 5-level severity + sampling/aggregation pipeline.
|
|
|
|
## 매 핵심
|
|
|
|
### 매 Three pillars of observability
|
|
- **Metrics**: 매 numeric, low-cardinality, aggregable. (Prometheus, OTLP metrics)
|
|
- **Logs**: 매 textual, high-cardinality, event-narrative.
|
|
- **Traces**: 매 request-scoped causal chain across services.
|
|
|
|
### 매 Log levels (RFC 5424 + practice)
|
|
- **FATAL/EMERGENCY**: 매 process death imminent.
|
|
- **ERROR**: 매 user-facing failure or unexpected condition.
|
|
- **WARN**: 매 degraded state, fallback engaged.
|
|
- **INFO**: 매 lifecycle events (start, stop, deploy).
|
|
- **DEBUG**: 매 development troubleshooting.
|
|
- **TRACE**: 매 fine-grained step-level (rarely on in prod).
|
|
|
|
### 매 Structured logging (2026 default)
|
|
- **JSON line format**: 매 1 event = 1 line, machine-parseable.
|
|
- **Required fields**: timestamp (RFC3339), level, service, trace_id, span_id, message.
|
|
- **OTel semantic conventions**: 매 http.method, db.system, exception.type.
|
|
- **PII redaction**: 매 in-stream filter for compliance (GDPR/HIPAA/PCI).
|
|
|
|
### 매 응용
|
|
1. 매 SLO debugging: error logs + trace correlation → root cause.
|
|
2. 매 audit trail: append-only, signed, retention 7y for finance.
|
|
3. 매 anomaly detection: log pattern clustering (Drain3, LogPAI) → unknown unknowns.
|
|
|
|
## 💻 패턴
|
|
|
|
### Structured logger (TypeScript pino)
|
|
```typescript
|
|
import pino from 'pino';
|
|
|
|
export const logger = pino({
|
|
level: process.env.LOG_LEVEL ?? 'info',
|
|
formatters: {
|
|
level: (label) => ({ level: label }),
|
|
bindings: () => ({ service: process.env.SERVICE_NAME, env: process.env.NODE_ENV }),
|
|
},
|
|
timestamp: pino.stdTimeFunctions.isoTime,
|
|
redact: { paths: ['*.password', '*.token', '*.creditCard'], censor: '[REDACTED]' },
|
|
});
|
|
|
|
// usage
|
|
logger.info({ userId, orderId, amountCents }, 'order placed');
|
|
logger.error({ err, requestId }, 'payment failed');
|
|
```
|
|
|
|
### Trace correlation (OpenTelemetry)
|
|
```typescript
|
|
import { trace, context } from '@opentelemetry/api';
|
|
|
|
function logWithTrace(msg: string, fields: object) {
|
|
const span = trace.getSpan(context.active());
|
|
const ctx = span?.spanContext();
|
|
logger.info({
|
|
...fields,
|
|
trace_id: ctx?.traceId,
|
|
span_id: ctx?.spanId,
|
|
}, msg);
|
|
}
|
|
```
|
|
|
|
### Log sampling (high-volume endpoints)
|
|
```typescript
|
|
function sampledLog(level: 'info'|'debug', sampleRate: number, fields: object, msg: string) {
|
|
if (Math.random() < sampleRate) {
|
|
logger[level]({ ...fields, sampled: true, sample_rate: sampleRate }, msg);
|
|
}
|
|
}
|
|
// e.g. health-check endpoint at 1% sampling
|
|
```
|
|
|
|
### PII redaction middleware
|
|
```typescript
|
|
const PII_PATTERNS = [
|
|
/\b\d{3}-\d{2}-\d{4}\b/g, // SSN
|
|
/\b\d{4}-\d{4}-\d{4}-\d{4}\b/g, // credit card
|
|
/\b[\w.+-]+@[\w-]+\.[\w.-]+\b/g, // email
|
|
];
|
|
|
|
function redactPII(s: string): string {
|
|
return PII_PATTERNS.reduce((acc, re) => acc.replace(re, '[REDACTED]'), s);
|
|
}
|
|
```
|
|
|
|
### Append-only audit log
|
|
```typescript
|
|
interface AuditEvent {
|
|
id: string; // ULID for time-ordering
|
|
actor: string;
|
|
action: string;
|
|
resource: string;
|
|
before?: unknown;
|
|
after?: unknown;
|
|
timestamp: string; // ISO8601
|
|
hashChainPrev: string; // tamper-evident chain
|
|
hashChainCurrent: string;
|
|
}
|
|
|
|
function appendAudit(prev: AuditEvent | null, event: Omit<AuditEvent, 'hashChainCurrent'|'hashChainPrev'|'id'>) {
|
|
const id = ulid();
|
|
const prevHash = prev?.hashChainCurrent ?? '0'.repeat(64);
|
|
const payload = JSON.stringify({ id, prevHash, ...event });
|
|
const currentHash = sha256(payload);
|
|
return { id, hashChainPrev: prevHash, hashChainCurrent: currentHash, ...event } as AuditEvent;
|
|
}
|
|
```
|
|
|
|
### Log aggregation pipeline
|
|
```yaml
|
|
# fluent-bit.conf
|
|
[SERVICE]
|
|
Flush 1
|
|
Daemon off
|
|
[INPUT]
|
|
Name tail
|
|
Path /var/log/app/*.log
|
|
Parser json
|
|
[FILTER]
|
|
Name kubernetes
|
|
Match *
|
|
Merge_Log On
|
|
[OUTPUT]
|
|
Name loki
|
|
Match *
|
|
Url https://loki.example.com/loki/api/v1/push
|
|
Labels {job="app", env="prod"}
|
|
```
|
|
|
|
### Log-based alert (Loki LogQL)
|
|
```logql
|
|
sum by (service) (
|
|
rate({env="prod", level="error"} | json | err_type != "ClientError" [5m])
|
|
) > 1
|
|
```
|
|
|
|
### Drain3 log clustering (anomaly detection)
|
|
```python
|
|
from drain3 import TemplateMiner
|
|
miner = TemplateMiner()
|
|
for line in stream:
|
|
res = miner.add_log_message(line)
|
|
# res['change_type'] in ['cluster_created','cluster_template_changed','none']
|
|
if res['change_type'] == 'cluster_created':
|
|
alert(f"new log pattern: {res['template_mined']}")
|
|
```
|
|
|
|
## 매 결정 기준
|
|
| 상황 | Approach |
|
|
|---|---|
|
|
| 모든 service | Structured JSON + trace correlation |
|
|
| 고볼륨 endpoint | Sampling (1-10% INFO/DEBUG) |
|
|
| 금융/헬스케어 | Append-only audit + PII redaction + 7y retention |
|
|
| Distributed system | OTel + centralized Loki/CloudWatch/Datadog |
|
|
| Local dev | Pretty-printed + DEBUG level |
|
|
|
|
**기본값**: 매 JSON + ISO time + level + service + trace_id + message + structured fields.
|
|
|
|
## 🔗 Graph
|
|
- 부모: [[Observability]]
|
|
- 변형: [[Append-only log]] · [[Structured Logging]]
|
|
- 응용: [[스택_트레이스(Stack_trace)]] · [[Logging_Diagnostics]] · [[Architecture Erosion (아키텍처 침식)]]
|
|
- Adjacent: [[OpenTelemetry]] · [[Distributed Tracing]] · [[비기능 요구사항 (Non-functional Requirements)]]
|
|
|
|
## 🤖 LLM 활용
|
|
**언제**: 매 log schema design, redaction rule draft, log-based alert query, log clustering for unknown patterns, postmortem timeline reconstruction.
|
|
**언제 X**: 매 raw log content with PII (privacy), 매 customer-attributable narrative without aggregation.
|
|
|
|
## ❌ 안티패턴
|
|
- **String concat logging**: 매 `"user " + id + " did " + x` → grep-only, no fields.
|
|
- **Log everything DEBUG in prod**: 매 cost 폭증 + signal-to-noise 0.
|
|
- **PII in logs**: 매 password/SSN/credit card untouched → instant breach.
|
|
- **No retention policy**: 매 7-year retention for trivial DEBUG → storage runaway.
|
|
- **Logs as primary metric**: 매 count error logs → use metrics; logs are narrative.
|
|
- **Blocking sync logging**: 매 disk fsync per log → throughput 폭락. Use async batch.
|
|
|
|
## 🧪 검증 / 중복
|
|
- Verified (OpenTelemetry Logs spec v1.31, Google SRE Book Ch.6 "Monitoring Distributed Systems", RFC 5424).
|
|
- 신뢰도 A.
|
|
|
|
## 🕓 Changelog
|
|
| 날짜 | 변경 |
|
|
|---|---|
|
|
| 2026-05-08 | Phase 1 |
|
|
| 2026-05-10 | Manual cleanup — structured logging + OTel + audit + redaction patterns |
|