--- id: backend-idempotency-deep title: Idempotency Deep — keys / windows / storage category: Coding status: draft source_trust_level: B verification_status: conceptual created_at: 2026-05-09 updated_at: 2026-05-09 tags: [backend, idempotency, vibe-coding] tech_stack: { language: "TS", applicable_to: ["Backend"] } applied_in: [] aliases: [idempotency key, deduplication, exactly-once, retry-safe, request-id] --- # Idempotency Deep > Network / retry = 같은 request 가 여러 번. **Idempotency key 가 dedupe**. Stripe / 모든 payment API 의 표준. ## 📖 핵심 개념 - 같은 key + 같은 request = 같은 result (1번 effect). - Storage 가 매 key 의 result. - TTL (24-48 hr). - Concurrency 안전. ## 💻 코드 패턴 ### 기본 (단순) ```ts async function createCharge(req) { const key = req.headers['idempotency-key']; if (!key) return { error: 'missing key' }; const cached = await db.idempotency.findOne({ key }); if (cached) return cached.response; const result = await stripe.charges.create(req.body); await db.idempotency.insert({ key, response: result, created_at: new Date() }); return result; } ``` → Race condition: 두 request 가 동시 = 2번 호출 가능. ### Concurrency-safe (lock) ```ts async function createCharge(req) { const key = req.headers['idempotency-key']; // 1. Try insert (atomic) try { await db.idempotency.insert({ key, status: 'in_progress', created_at: new Date() }); } catch (e) { if (isUniqueViolation(e)) { // Existing → return cached or wait const cached = await db.idempotency.findOne({ key }); if (cached.status === 'completed') return cached.response; if (cached.status === 'in_progress') { // Wait or 409 return { error: 'in progress' }; // 409 Conflict } } throw e; } // 2. Process try { const result = await stripe.charges.create(req.body); await db.idempotency.update({ key }, { status: 'completed', response: result }); return result; } catch (e) { await db.idempotency.update({ key }, { status: 'failed', error: e.message }); throw e; } } ``` ### Postgres unique constraint ```sql CREATE TABLE idempotency_keys ( key TEXT PRIMARY KEY, status TEXT NOT NULL, response JSONB, request_hash TEXT, created_at TIMESTAMP NOT NULL DEFAULT NOW(), expires_at TIMESTAMP NOT NULL ); ``` → INSERT 가 atomic. Same key 두 번 = unique violation. ### Request hash (validation) ```ts async function createCharge(req) { const key = req.headers['idempotency-key']; const reqHash = sha256(JSON.stringify(req.body)); const cached = await db.idempotency.findOne({ key }); if (cached) { if (cached.request_hash !== reqHash) { return { error: 'idempotency key reused with different body' }; // 422 } return cached.response; } // ... 처리 await db.idempotency.insert({ key, request_hash: reqHash, ... }); } ``` → "Same key 다른 body" = 클라이언트 bug. ### Stripe 식 ```ts // Stripe 가 client 의 패턴 const charge = await stripe.charges.create( { amount: 1000, currency: 'usd', source: 'tok_x' }, { idempotencyKey: 'order_123' } ); ``` → Stripe 가 24 hr 안 같은 result. ### TTL ```sql -- 24-48 hr 후 삭제 DELETE FROM idempotency_keys WHERE expires_at < NOW(); -- 또는 partition CREATE TABLE idempotency_keys_2026_05_09 PARTITION OF idempotency_keys FOR VALUES FROM ('2026-05-09') TO ('2026-05-10'); ``` → 매일 partition + drop. ### Redis 식 ```ts async function createCharge(req) { const key = `idem:${req.headers['idempotency-key']}`; // SETNX (atomic) const set = await redis.set(key, 'pending', 'EX', 86400, 'NX'); if (set === null) { // Already exists const cached = await redis.get(key); if (cached === 'pending') return { error: 'in progress' }; return JSON.parse(cached); } const result = await stripe.charges.create(req.body); await redis.set(key, JSON.stringify(result), 'EX', 86400); return result; } ``` → Redis = 빠름. Persistence X (RDB / AOF 가능). ### Locking (advanced) ```ts import { Redlock } from 'redlock'; async function createCharge(req) { const key = req.headers['idempotency-key']; const lock = await redlock.acquire([`lock:${key}`], 10000); try { // Check cache → process → save } finally { await lock.release(); } } ``` → Distributed lock 가 보장. ### HTTP semantics ``` GET, HEAD, PUT, DELETE: idempotent (HTTP spec). POST: NOT idempotent (default). → POST 의 idempotency = key 명시. PUT 도 idempotency key 가 안전. ``` ### Status code ``` 200/201: 첫 호출 (success). 200/201: 두 번째 (cached, same response). 422: same key, different body. 409: in progress. 410: expired key. ``` ### Async / queue idempotency ```ts // Outbox pattern BEGIN; INSERT INTO outbox (id, event_type, payload) VALUES ('uuid-1', 'order.placed', ...); // duplicate id = unique violation COMMIT; // Consumer async function processEvent(event) { if (await db.processed_events.exists(event.id)) return; // ... await db.processed_events.insert({ id: event.id }); } ``` → Outbox + processed events = exactly-once. → [[Backend_Outbox_Pattern]]. ### Kafka (offset 기반) ``` Kafka consumer: - Offset commit = "처리됨". - Crash 후 = 옛 offset 부터 다시 → 중복 가능. → Idempotent processing (DB upsert + dedup). ``` ### Webhook idempotency ```ts // Webhook receiver async function handleWebhook(req) { const eventId = req.body.id; if (await db.webhook_events.exists(eventId)) { return res.status(200).json({ received: true }); } await db.webhook_events.insert({ id: eventId }); await processWebhook(req.body); res.status(200).json({ received: true }); } ``` → Stripe / GitHub 가 retry — duplicate event 가능. ### Race in receiver ```ts // 두 webhook 가 동시 도착 // transaction 안에서 insert + process BEGIN; INSERT INTO webhook_events (id) VALUES ($1); -- unique // process COMMIT; ``` ### Multi-step (saga) ``` 매 step 의 idempotency. - reserveItems(itemIds, idempotencyKey): 같은 key = 같은 reservation. - chargePayment(amount, idempotencyKey): 같은 key = 1번 charge. - shipOrder(orderId, idempotencyKey): 같은 key = 1번 ship. ``` → Saga retry 가 안전. ### Generated key ```ts // Server side const key = `${userId}:${orderId}:${timestamp}`; // Client side (UUID) const key = crypto.randomUUID(); ``` → Client 가 generate (Stripe 식). ### Storage capacity ``` 1M req/day × 2 KB / req = 2 GB / day. 24 hr TTL = 2 GB live. → 큰 system 가 partition / TTL. ``` ### Performance 영향 ``` 매 request 가 1 DB round-trip (idempotency check). Cache (Redis) 가 mitigate. Hot key 가 hot row. → Production 가 Redis. ``` ### Frontend (UI) ```ts // Generate key once per "submit" const idempKey = useRef(crypto.randomUUID()); const handleSubmit = async () => { await fetch('/charge', { method: 'POST', headers: { 'idempotency-key': idempKey.current }, body: JSON.stringify(data), }); }; ``` → Retry on network error 가 안전. ### Failure 시 cleanup ```ts try { // process } catch (e) { // Mark failed (cache 가 안 stale). await db.idempotency.delete({ key }); // 또는 await db.idempotency.update({ key }, { status: 'failed', expires_at: now + 5min }); throw e; } ``` → Failed 도 some retention (재시도 가 같은 fail). ### LLM call (idempotency) ```ts // LLM 가 같은 prompt = 같은 답 안 (random). // Determinism: temperature=0. const cacheKey = sha256(prompt); const cached = await cache.get(cacheKey); if (cached) return cached; const r = await llm.complete(prompt, { temperature: 0 }); await cache.set(cacheKey, r); return r; ``` → LLM cache 도 idempotency 식. ### 함정 ``` - TTL 너무 짧음 (1 hr): 늦은 retry 가 missed dedup. - TTL 너무 길음: storage 폭발. - Key 가 짧음 (timestamp 만): 충돌. - Process 후 store (race): 같은 key 가 여러 번. - Body hash 안: 클라이언트 가 wrong body. - Failed 가 cache 안 cleanup: 재시도 가 fail. ``` ### 표준 (다른 vendor) ``` Stripe: 24 hr. PayPal: 5 min. Square: 24 hr. GitHub API: SHA-based. Slack: 1 hr. ``` ## 🤔 의사결정 기준 | 상황 | 추천 | |---|---| | Payment / 외부 API | Idempotency key (Stripe 식) | | Webhook | Event ID dedup | | Saga step | 매 step 의 key | | 빠른 cache | Redis SETNX | | Persistent | Postgres unique | | Multi-region | Distributed (Redlock 등) | | 작은 / 단순 | DB INSERT + unique | ## ❌ 안티패턴 - **No idempotency**: retry = 중복 charge. - **Process 후 cache**: race. - **TTL 무한**: storage. - **Body hash 없음**: wrong body 통과. - **Failed 가 안 retry**: stuck. - **Auto-generate key (timestamp)**: collision. - **Cache only (no persistent)**: restart 후 잃음. ## 🤖 LLM 활용 힌트 - Idempotency = unique key + atomic check. - Stripe 가 표준 (Idempotency-Key header). - TTL 24-48 hr. - Failed 는 짧은 cache. ## 🔗 관련 문서 - [[Backend_Idempotency_Keys]] - [[Backend_Idempotent_Consumer]] - [[Backend_Outbox_Pattern]]