2nd/10_Wiki/Topics/Coding/Backend_Idempotency_Deep.md

---
id: backend-idempotency-deep
title: Idempotency Deep — keys / windows / storage
category: Coding
status: draft
source_trust_level: B
verification_status: conceptual
created_at: 2026-05-09
updated_at: 2026-05-09
tags: [backend, idempotency, vibe-coding]
tech_stack: { language: "TS", applicable_to: ["Backend"] }
applied_in: []
aliases: [idempotency key, deduplication, exactly-once, retry-safe, request-id]
---

# Idempotency Deep

> Network / retry = 같은 request 가 여러 번. **Idempotency key 가 dedupe**. Stripe / 모든 payment API 의 표준.

## 📖 핵심 개념
- 같은 key + 같은 request = 같은 result (1번 effect).
- Storage 가 매 key 의 result.
- TTL (24-48 hr).
- Concurrency 안전.

## 💻 코드 패턴

### 기본 (단순)
```ts
async function createCharge(req) {
  const key = req.headers['idempotency-key'];
  if (!key) return { error: 'missing key' };
  
  const cached = await db.idempotency.findOne({ key });
  if (cached) return cached.response;
  
  const result = await stripe.charges.create(req.body);
  
  await db.idempotency.insert({ key, response: result, created_at: new Date() });
  return result;
}
```

→ Race condition: 두 request 가 동시 = 2번 호출 가능.

### Concurrency-safe (lock)
```ts
async function createCharge(req) {
  const key = req.headers['idempotency-key'];
  
  // 1. Try insert (atomic)
  try {
    await db.idempotency.insert({
      key, status: 'in_progress', created_at: new Date()
    });
  } catch (e) {
    if (isUniqueViolation(e)) {
      // Existing → return cached or wait
      const cached = await db.idempotency.findOne({ key });
      if (cached.status === 'completed') return cached.response;
      if (cached.status === 'in_progress') {
        // Wait or 409
        return { error: 'in progress' };  // 409 Conflict
      }
    }
    throw e;
  }
  
  // 2. Process
  try {
    const result = await stripe.charges.create(req.body);
    await db.idempotency.update({ key }, { status: 'completed', response: result });
    return result;
  } catch (e) {
    await db.idempotency.update({ key }, { status: 'failed', error: e.message });
    throw e;
  }
}
```

### Postgres unique constraint
```sql
CREATE TABLE idempotency_keys (
    key TEXT PRIMARY KEY,
    status TEXT NOT NULL,
    response JSONB,
    request_hash TEXT,
    created_at TIMESTAMP NOT NULL DEFAULT NOW(),
    expires_at TIMESTAMP NOT NULL
);
```

→ INSERT 가 atomic. Same key 두 번 = unique violation.

### Request hash (validation)
```ts
async function createCharge(req) {
  const key = req.headers['idempotency-key'];
  const reqHash = sha256(JSON.stringify(req.body));
  
  const cached = await db.idempotency.findOne({ key });
  if (cached) {
    if (cached.request_hash !== reqHash) {
      return { error: 'idempotency key reused with different body' };  // 422
    }
    return cached.response;
  }
  
  // ... 처리
  await db.idempotency.insert({ key, request_hash: reqHash, ... });
}
```

→ "Same key 다른 body" = 클라이언트 bug.

### Stripe 식
```ts
// Stripe 가 client 의 패턴
const charge = await stripe.charges.create(
  { amount: 1000, currency: 'usd', source: 'tok_x' },
  { idempotencyKey: 'order_123' }
);
```

→ Stripe 가 24 hr 안 같은 result.

### TTL
```sql
-- 24-48 hr 후 삭제
DELETE FROM idempotency_keys WHERE expires_at < NOW();

-- 또는 partition
CREATE TABLE idempotency_keys_2026_05_09 PARTITION OF idempotency_keys
FOR VALUES FROM ('2026-05-09') TO ('2026-05-10');
```

→ 매일 partition + drop.

### Redis 식
```ts
async function createCharge(req) {
  const key = `idem:${req.headers['idempotency-key']}`;
  
  // SETNX (atomic)
  const set = await redis.set(key, 'pending', 'EX', 86400, 'NX');
  if (set === null) {
    // Already exists
    const cached = await redis.get(key);
    if (cached === 'pending') return { error: 'in progress' };
    return JSON.parse(cached);
  }
  
  const result = await stripe.charges.create(req.body);
  await redis.set(key, JSON.stringify(result), 'EX', 86400);
  return result;
}
```

→ Redis = 빠름. Persistence X (RDB / AOF 가능).

### Locking (advanced)
```ts
import { Redlock } from 'redlock';

async function createCharge(req) {
  const key = req.headers['idempotency-key'];
  
  const lock = await redlock.acquire([`lock:${key}`], 10000);
  try {
    // Check cache → process → save
  } finally {
    await lock.release();
  }
}
```

→ Distributed lock 가 보장.

### HTTP semantics
```
GET, HEAD, PUT, DELETE: idempotent (HTTP spec).
POST: NOT idempotent (default).

→ POST 의 idempotency = key 명시.
PUT 도 idempotency key 가 안전.
```

### Status code
```
200/201: 첫 호출 (success).
200/201: 두 번째 (cached, same response).
422: same key, different body.
409: in progress.
410: expired key.
```

### Async / queue idempotency
```ts
// Outbox pattern
BEGIN;
  INSERT INTO outbox (id, event_type, payload) VALUES ('uuid-1', 'order.placed', ...);
  // duplicate id = unique violation
COMMIT;

// Consumer
async function processEvent(event) {
  if (await db.processed_events.exists(event.id)) return;
  // ...
  await db.processed_events.insert({ id: event.id });
}
```

→ Outbox + processed events = exactly-once.

→ [[Backend_Outbox_Pattern]].

### Kafka (offset 기반)
```
Kafka consumer:
- Offset commit = "처리됨".
- Crash 후 = 옛 offset 부터 다시 → 중복 가능.

→ Idempotent processing (DB upsert + dedup).
```

### Webhook idempotency
```ts
// Webhook receiver
async function handleWebhook(req) {
  const eventId = req.body.id;
  
  if (await db.webhook_events.exists(eventId)) {
    return res.status(200).json({ received: true });
  }
  
  await db.webhook_events.insert({ id: eventId });
  await processWebhook(req.body);
  res.status(200).json({ received: true });
}
```

→ Stripe / GitHub 가 retry — duplicate event 가능.

### Race in receiver
```ts
// 두 webhook 가 동시 도착
// transaction 안에서 insert + process
BEGIN;
  INSERT INTO webhook_events (id) VALUES ($1);  -- unique
  // process
COMMIT;
```

### Multi-step (saga)
```
매 step 의 idempotency.
- reserveItems(itemIds, idempotencyKey): 같은 key = 같은 reservation.
- chargePayment(amount, idempotencyKey): 같은 key = 1번 charge.
- shipOrder(orderId, idempotencyKey): 같은 key = 1번 ship.
```

→ Saga retry 가 안전.

### Generated key
```ts
// Server side
const key = `${userId}:${orderId}:${timestamp}`;

// Client side (UUID)
const key = crypto.randomUUID();
```

→ Client 가 generate (Stripe 식).

### Storage capacity
```
1M req/day × 2 KB / req = 2 GB / day.
24 hr TTL = 2 GB live.

→ 큰 system 가 partition / TTL.
```

### Performance 영향
```
매 request 가 1 DB round-trip (idempotency check).
Cache (Redis) 가 mitigate.
Hot key 가 hot row.

→ Production 가 Redis.
```

### Frontend (UI)
```ts
// Generate key once per "submit"
const idempKey = useRef(crypto.randomUUID());

const handleSubmit = async () => {
  await fetch('/charge', {
    method: 'POST',
    headers: { 'idempotency-key': idempKey.current },
    body: JSON.stringify(data),
  });
};
```

→ Retry on network error 가 안전.

### Failure 시 cleanup
```ts
try {
  // process
} catch (e) {
  // Mark failed (cache 가 안 stale).
  await db.idempotency.delete({ key });
  // 또는
  await db.idempotency.update({ key }, { status: 'failed', expires_at: now + 5min });
  throw e;
}
```

→ Failed 도 some retention (재시도 가 같은 fail).

### LLM call (idempotency)
```ts
// LLM 가 같은 prompt = 같은 답 안 (random).
// Determinism: temperature=0.

const cacheKey = sha256(prompt);
const cached = await cache.get(cacheKey);
if (cached) return cached;

const r = await llm.complete(prompt, { temperature: 0 });
await cache.set(cacheKey, r);
return r;
```

→ LLM cache 도 idempotency 식.

### 함정
```
- TTL 너무 짧음 (1 hr): 늦은 retry 가 missed dedup.
- TTL 너무 길음: storage 폭발.
- Key 가 짧음 (timestamp 만): 충돌.
- Process 후 store (race): 같은 key 가 여러 번.
- Body hash 안: 클라이언트 가 wrong body.
- Failed 가 cache 안 cleanup: 재시도 가 fail.
```

### 표준 (다른 vendor)
```
Stripe: 24 hr.
PayPal: 5 min.
Square: 24 hr.
GitHub API: SHA-based.
Slack: 1 hr.
```

## 🤔 의사결정 기준
| 상황 | 추천 |
|---|---|
| Payment / 외부 API | Idempotency key (Stripe 식) |
| Webhook | Event ID dedup |
| Saga step | 매 step 의 key |
| 빠른 cache | Redis SETNX |
| Persistent | Postgres unique |
| Multi-region | Distributed (Redlock 등) |
| 작은 / 단순 | DB INSERT + unique |

## ❌ 안티패턴
- **No idempotency**: retry = 중복 charge.
- **Process 후 cache**: race.
- **TTL 무한**: storage.
- **Body hash 없음**: wrong body 통과.
- **Failed 가 안 retry**: stuck.
- **Auto-generate key (timestamp)**: collision.
- **Cache only (no persistent)**: restart 후 잃음.

## 🤖 LLM 활용 힌트
- Idempotency = unique key + atomic check.
- Stripe 가 표준 (Idempotency-Key header).
- TTL 24-48 hr.
- Failed 는 짧은 cache.

## 🔗 관련 문서
- [[Backend_Idempotency_Keys]]
- [[Backend_Idempotent_Consumer]]
- [[Backend_Outbox_Pattern]]