docs(10_Wiki): Topic_Business/General/Graphic/Programming을 Topics/ 하위로 이동
최상위 10_Wiki/Topic_*였던 4개 카테고리 폴더를 10_Wiki/Topics/Topic_* 로 재배치. 콘텐츠 변경 없음(순수 폴더 이동) — Topics/ 하위 나머지 폴더는 이미 지난 커밋에서 전부 정리된 상태(잔존 항목은 에이전트 운영 상태 및 사용자가 보존을 요청한 업데이트0615/무제 3.canvas 뿐).
This commit is contained in:
Antigravity Agent
@@ -0,0 +1,202 @@
|
||||
---
|
||||
id: wiki-2026-0508-broker-topology
|
||||
title: Broker Topology
|
||||
category: 10_Wiki/Topics
|
||||
status: verified
|
||||
canonical_id: self
|
||||
aliases: [Event Broker Topology, Pub-Sub Topology]
|
||||
duplicate_of: none
|
||||
source_trust_level: A
|
||||
confidence_score: 0.9
|
||||
verification_status: applied
|
||||
tags: [event-driven, architecture, kafka, messaging]
|
||||
raw_sources: []
|
||||
last_reinforced: 2026-05-10
|
||||
github_commit: pending
|
||||
tech_stack:
|
||||
language: typescript
|
||||
framework: kafka-nats-rabbitmq
|
||||
---
|
||||
|
||||
# Broker Topology
|
||||
|
||||
## 매 한 줄
|
||||
> **"매 distributed pub-sub mesh — 매 central mediator 없이, broker (queue/topic) 가 routing fabric."**. Broker topology는 Event-Driven Architecture 의 두 종류 중 하나 (vs Mediator topology). 매 single-purpose events 의 fan-out 에 최적. 2026년 Kafka, NATS JetStream, AWS EventBridge, Redpanda, Pulsar 가 dominant brokers.
|
||||
|
||||
## 매 핵심
|
||||
|
||||
### 매 Broker vs Mediator
|
||||
| 측면 | Broker | Mediator |
|
||||
|---|---|---|
|
||||
| Coordination | None — events fan out | Central mediator orchestrates |
|
||||
| Coupling | Loosest | Some (mediator knows steps) |
|
||||
| Use case | Simple fan-out, async notify | Complex multi-step workflow |
|
||||
| Failure recovery | Each consumer handles | Mediator retries/compensates |
|
||||
| Examples | Kafka topics, NATS subjects | Apache Camel, Step Functions |
|
||||
|
||||
### 매 컴포넌트
|
||||
- **Producer**: events 의 publish.
|
||||
- **Broker**: topic/subject/queue store + routing.
|
||||
- **Consumer**: subscribe + react.
|
||||
- **Schema Registry** (Confluent, Apicurio): event contract 의 versioning.
|
||||
- **Dead Letter Queue (DLQ)**: failed messages.
|
||||
|
||||
### 매 broker 종류
|
||||
- **Log-based** (Kafka, Redpanda, Pulsar): replay 가능, partitioned, ordered per partition.
|
||||
- **Queue-based** (RabbitMQ, SQS): consumed once, no replay.
|
||||
- **Subject-based** (NATS): lightweight, JetStream 으로 persistence.
|
||||
- **Cloud-native** (EventBridge, Pub/Sub, EventHubs).
|
||||
|
||||
### 매 응용
|
||||
1. Order events (e-commerce fan-out).
|
||||
2. CDC (Debezium → Kafka → multiple sinks).
|
||||
3. IoT telemetry ingestion.
|
||||
4. Analytics event pipeline.
|
||||
|
||||
## 💻 패턴
|
||||
|
||||
### Kafka producer (TypeScript / KafkaJS)
|
||||
```typescript
|
||||
import { Kafka } from 'kafkajs';
|
||||
|
||||
const kafka = new Kafka({
|
||||
clientId: 'orders',
|
||||
brokers: ['kafka-1:9092', 'kafka-2:9092'],
|
||||
});
|
||||
const producer = kafka.producer();
|
||||
|
||||
await producer.connect();
|
||||
await producer.send({
|
||||
topic: 'order.created',
|
||||
messages: [{
|
||||
key: order.id,
|
||||
value: JSON.stringify(order),
|
||||
headers: { 'content-type': 'application/json' },
|
||||
}],
|
||||
});
|
||||
```
|
||||
|
||||
### Kafka consumer (consumer group fan-out)
|
||||
```typescript
|
||||
const consumer = kafka.consumer({ groupId: 'shipping-svc' });
|
||||
await consumer.connect();
|
||||
await consumer.subscribe({ topic: 'order.created' });
|
||||
|
||||
await consumer.run({
|
||||
eachMessage: async ({ message }) => {
|
||||
const order = JSON.parse(message.value!.toString());
|
||||
await createShipment(order);
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### NATS JetStream
|
||||
```typescript
|
||||
import { connect, JSONCodec } from 'nats';
|
||||
|
||||
const nc = await connect({ servers: 'nats://localhost:4222' });
|
||||
const js = nc.jetstream();
|
||||
const jc = JSONCodec();
|
||||
|
||||
await js.publish('orders.created', jc.encode(order));
|
||||
|
||||
const sub = await js.subscribe('orders.>', {
|
||||
config: { durable_name: 'shipping' },
|
||||
});
|
||||
for await (const m of sub) {
|
||||
const order = jc.decode(m.data);
|
||||
await createShipment(order);
|
||||
m.ack();
|
||||
}
|
||||
```
|
||||
|
||||
### Schema Registry (Avro + Confluent)
|
||||
```typescript
|
||||
import { SchemaRegistry } from '@kafkajs/confluent-schema-registry';
|
||||
|
||||
const registry = new SchemaRegistry({ host: 'http://schema-registry:8081' });
|
||||
const schema = `{
|
||||
"type": "record",
|
||||
"name": "OrderCreated",
|
||||
"fields": [
|
||||
{ "name": "id", "type": "string" },
|
||||
{ "name": "amount", "type": "double" }
|
||||
]
|
||||
}`;
|
||||
|
||||
const { id } = await registry.register({ type: 'AVRO', schema });
|
||||
const encoded = await registry.encode(id, order);
|
||||
await producer.send({ topic: 'order.created', messages: [{ value: encoded }] });
|
||||
```
|
||||
|
||||
### DLQ pattern
|
||||
```typescript
|
||||
await consumer.run({
|
||||
eachMessage: async ({ topic, message }) => {
|
||||
try {
|
||||
await handleOrder(JSON.parse(message.value!.toString()));
|
||||
} catch (err) {
|
||||
await producer.send({
|
||||
topic: `${topic}.dlq`,
|
||||
messages: [{
|
||||
value: message.value,
|
||||
headers: {
|
||||
...message.headers,
|
||||
error: String(err),
|
||||
retryCount: '0',
|
||||
},
|
||||
}],
|
||||
});
|
||||
}
|
||||
},
|
||||
});
|
||||
```
|
||||
|
||||
### Partitioning for ordering
|
||||
```typescript
|
||||
// Same orderId always goes to same partition → ordered per order
|
||||
await producer.send({
|
||||
topic: 'order.events',
|
||||
messages: [{
|
||||
key: order.id, // partitioner uses key hash
|
||||
value: JSON.stringify(event),
|
||||
}],
|
||||
});
|
||||
```
|
||||
|
||||
## 매 결정 기준
|
||||
| 상황 | Approach |
|
||||
|---|---|
|
||||
| High throughput, replay 필요 | Kafka / Redpanda |
|
||||
| Lightweight, low-latency | NATS JetStream |
|
||||
| Cloud serverless | EventBridge, Pub/Sub |
|
||||
| Strong work-queue + ack | RabbitMQ, SQS |
|
||||
| Multi-tenant, geo | Pulsar |
|
||||
|
||||
**기본값**: Kafka (or Redpanda — Kafka API compat) + Schema Registry + Avro/Protobuf for events.
|
||||
|
||||
## 🔗 Graph
|
||||
- 부모: [[Event-Driven Architecture]]
|
||||
- 변형: [[Mediator Topology]] · [[Choreography]]
|
||||
- 응용: [[Kafka]] · [[NATS]]
|
||||
- Adjacent: [[CDC]]
|
||||
|
||||
## 🤖 LLM 활용
|
||||
**언제**: fan-out events, decoupled microservices, CDC pipelines, telemetry ingestion.
|
||||
**언제 X**: synchronous request-reply (use gRPC/HTTP), small monolith (events 가 overhead).
|
||||
|
||||
## ❌ 안티패턴
|
||||
- **No schema registry**: 모든 producer/consumer 가 schema drift — production 깨짐.
|
||||
- **Topic per service**: scalability impossible — topic per event type.
|
||||
- **No DLQ**: poison messages 가 consumer halt — DLQ + alerting.
|
||||
- **Sync over async**: HTTP request-reply 를 broker 위에 — synchronous 면 broker 의 X.
|
||||
|
||||
## 🧪 검증 / 중복
|
||||
- Verified (Richards "Software Architecture Patterns" / Confluent docs / NATS docs 2026).
|
||||
- 신뢰도 A.
|
||||
|
||||
## 🕓 Changelog
|
||||
| 날짜 | 변경 |
|
||||
|---|---|
|
||||
| 2026-05-08 | Phase 1 |
|
||||
| 2026-05-10 | Manual cleanup — Broker vs Mediator + Kafka/NATS/DLQ 패턴 |
|
||||
Reference in New Issue
Block a user