refactor(topics): 멀티 에이전트용 지식 재편 — _Common(공통 기본기) + Domain_* 구조

에이전트 8종(대화형/프로그래머 C·S/디자이너/설계자/기획자/QA/PD/PM)에게 [공통 기본 능력 + 롤별 Specialty] 2층으로 지식을 주입하기 위한 재분류. 문서 내용·포맷은 무수정, 폴더 이동만 (6,372개 문서 수 보존 확인). - Topic_Programming → Domain_Programming (내부 구조 보존) - Topic_Graphic → Domain_Design - Topic_Business → Domain_Product - Topic_General → Domain_General - _Common 신설: Math(구 Topic_Math_Specialty), Reasoning(구 General/From_Thinking & Reasoning), Reasoning_Creativity(구 General/From_창의성), Communication(Poetic_Blog_Writing + From_writing) - 타 도메인의 From_* 폴더는 유지 (출처 표기일 뿐, 이미 도메인에 맞게 분류된 문서) - 빈 폴더 정리 (memory/procedures) - 에이전트→폴더 매핑은 workspace의 .astra/agent-knowledge-map.json (9개 에이전트) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 11:05:56 +09:00
parent 6549ead309
commit c24165b8bc
6193 changed files with 1717 additions and 31 deletions
@@ -0,0 +1,208 @@
+---
+id: wiki-2026-0508-queue-management-systems
+title: Queue Management Systems
+category: 10_Wiki/Topics
+status: verified
+canonical_id: self
+aliases: [Message Queues, Job Queues, Task Queues]
+duplicate_of: none
+source_trust_level: A
+confidence_score: 0.95
+verification_status: applied
+tags: [queue, messaging, async, distributed-systems]
+raw_sources: []
+last_reinforced: 2026-05-10
+github_commit: pending
+tech_stack:
+  language: python
+  framework: rabbitmq-sqs-celery
+---
+
+# Queue Management Systems
+
+## 매 한 줄
+> **"매 producer 와 consumer 매 decouple — 매 buffered, durable, retried"**. RabbitMQ / SQS / Redis / Celery / Sidekiq / Temporal — 매 async work 매 backbone. 매 2026 매 Temporal-style durable execution 매 mainstream.
+
+## 매 핵심
+
+### 매 queue types
+- **FIFO point-to-point**: 매 single consumer per message (SQS, RabbitMQ direct).
+- **Pub-sub fanout**: 매 N consumers 모두 받음 (SNS, Redis pub/sub, RabbitMQ fanout).
+- **Priority**: 매 weighted consumption (RabbitMQ priority, Redis sorted set).
+- **Delayed**: 매 ETA-based (SQS DelaySeconds, Sidekiq scheduled).
+- **Dead letter**: 매 failed messages → DLQ.
+
+### 매 delivery guarantees
+- **At-most-once**: ack-on-delivery (Redis pub/sub).
+- **At-least-once**: ack-on-process — 매 default for most prod queues.
+- **Exactly-once**: 매 idempotent consumer + dedup (FIFO SQS, Kafka EOS).
+
+### 매 patterns
+- **Work queue**: 매 N workers, load-balanced.
+- **Competing consumers**: 매 same.
+- **Saga / orchestration**: 매 multi-step workflow (Temporal, Cadence).
+- **Outbox**: 매 transactional message dispatch.
+
+### 매 응용
+1. Background jobs (email, image resize, PDF gen).
+2. Microservice integration (order → fulfillment).
+3. Rate limiting / throttling (queue as buffer).
+4. Workflow orchestration (Temporal).
+5. Batch processing (SQS → Lambda fanout).
+
+## 💻 패턴
+
+### Celery (Python)
+```python
+from celery import Celery
+app = Celery("tasks", broker="redis://localhost:6379/0")
+
+@app.task(bind=True, max_retries=3, retry_backoff=True)
+def send_email(self, to: str):
+    try:
+        smtp.send(to)
+    except SMTPException as e:
+        raise self.retry(exc=e, countdown=2 ** self.request.retries)
+
+# Producer
+send_email.delay("alice@example.com")
+```
+
+### RabbitMQ work queue
+```python
+import pika
+conn = pika.BlockingConnection(pika.ConnectionParameters("localhost"))
+ch = conn.channel()
+ch.queue_declare(queue="tasks", durable=True)
+
+# Publisher
+ch.basic_publish(
+    exchange="", routing_key="tasks", body=b"work",
+    properties=pika.BasicProperties(delivery_mode=2),  # persistent
+)
+
+# Worker
+ch.basic_qos(prefetch_count=1)
+def callback(ch, method, props, body):
+    process(body)
+    ch.basic_ack(delivery_tag=method.delivery_tag)
+ch.basic_consume(queue="tasks", on_message_callback=callback)
+ch.start_consuming()
+```
+
+### AWS SQS (boto3)
+```python
+import boto3
+sqs = boto3.client("sqs")
+url = sqs.get_queue_url(QueueName="jobs")["QueueUrl"]
+
+# Send
+sqs.send_message(QueueUrl=url, MessageBody=json.dumps(payload),
+                 MessageGroupId="orders", MessageDeduplicationId=order_id)  # FIFO
+
+# Receive (long poll)
+resp = sqs.receive_message(QueueUrl=url, WaitTimeSeconds=20, MaxNumberOfMessages=10)
+for msg in resp.get("Messages", []):
+    process(json.loads(msg["Body"]))
+    sqs.delete_message(QueueUrl=url, ReceiptHandle=msg["ReceiptHandle"])
+```
+
+### Redis Streams (consumer groups)
+```python
+import redis
+r = redis.Redis()
+r.xgroup_create("orders", "fulfillment", id="0", mkstream=True)
+
+# Producer
+r.xadd("orders", {"id": "42", "total": "99"})
+
+# Consumer
+while True:
+    msgs = r.xreadgroup("fulfillment", "worker-1", {"orders": ">"}, count=10, block=5000)
+    for stream, entries in msgs or []:
+        for mid, data in entries:
+            process(data)
+            r.xack("orders", "fulfillment", mid)
+```
+
+### Temporal workflow (durable execution)
+```python
+from temporalio import workflow, activity
+from datetime import timedelta
+
+@activity.defn
+async def charge(card: str, amount: int) -> str:
+    return payment_api.charge(card, amount)
+
+@workflow.defn
+class OrderWorkflow:
+    @workflow.run
+    async def run(self, order: dict) -> str:
+        tx = await workflow.execute_activity(
+            charge, order["card"], order["total"],
+            start_to_close_timeout=timedelta(seconds=30),
+            retry_policy=RetryPolicy(maximum_attempts=5),
+        )
+        await workflow.execute_activity(ship, order, ...)
+        return tx
+```
+
+### Outbox pattern (transactional)
+```python
+def place_order(db, order):
+    with db.transaction():
+        db.execute("INSERT INTO orders ...", order)
+        db.execute("INSERT INTO outbox (topic, payload) VALUES (?, ?)",
+                   "orders.created", json.dumps(order))
+    # separate poller relays outbox → broker (at-least-once)
+```
+
+### Dead letter queue handling
+```python
+# RabbitMQ DLX
+ch.queue_declare("tasks", arguments={
+    "x-dead-letter-exchange": "dlx",
+    "x-message-ttl": 60000,
+    "x-max-retries": 3,
+})
+# DLQ consumer logs / alerts / manual replay
+```
+
+## 매 결정 기준
+| 상황 | Choice |
+|---|---|
+| 매 simple background jobs | Celery / Sidekiq / BullMQ |
+| 매 enterprise messaging | RabbitMQ |
+| 매 cloud-managed | SQS / Cloud Tasks |
+| 매 ordered + dedup | FIFO SQS |
+| 매 multi-step workflow | Temporal / Cadence / AWS Step Functions |
+| 매 high throughput log | Kafka (technically not a queue) |
+| 매 in-process | asyncio.Queue / channels |
+
+**기본값**: 매 2026 매 durable workflow → Temporal. 매 simple jobs → BullMQ / Celery.
+
+## 🔗 Graph
+- 부모: [[Async Programming]]
+- 변형: [[Dead Letter Queue]]
+- Adjacent: [[RabbitMQ]] · [[SQS]] · [[Temporal]] · [[Sidekiq]]
+
+## 🤖 LLM 활용
+**언제**: 매 retry policy design, 매 DLQ analysis (cluster failure modes), 매 workflow code scaffold.
+**언제 X**: 매 throughput sizing — load test 직접.
+
+## ❌ 안티패턴
+- **No idempotency**: 매 at-least-once + non-idempotent → duplicate side effects. 매 dedup key 필수.
+- **Infinite retry**: 매 poison message 매 forever — max attempts + DLQ.
+- **Unbounded queue**: 매 producer faster than consumer — OOM. 매 backpressure / drop oldest.
+- **Sync wait for queue result**: 매 anti-async — 매 callback / webhook / polling.
+- **Long-running task in queue with short visibility timeout**: 매 redelivered while still running — race.
+
+## 🧪 검증 / 중복
+- Verified (RabbitMQ docs; AWS SQS docs; Temporal docs 1.20+).
+- 신뢰도 A.
+
+## 🕓 Changelog
+| 날짜 | 변경 |
+|---|---|
+| 2026-05-08 | Phase 1 |
+| 2026-05-10 | Manual cleanup — full queue systems entry with Temporal |