2nd/10_Wiki/Topics/Architecture/Broker Topology.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-broker-topology	Broker Topology	10_Wiki/Topics	verified	self	Event Broker Topology Pub-Sub Topology	none	A	0.9	applied	event-driven architecture kafka messaging		2026-05-10	pending	language framework typescript kafka-nats-rabbitmq

Broker Topology

매 한 줄

"매 distributed pub-sub mesh — 매 central mediator 없이, broker (queue/topic) 가 routing fabric.". Broker topology는 Event-Driven Architecture 의 두 종류 중 하나 (vs Mediator topology). 매 single-purpose events 의 fan-out 에 최적. 2026년 Kafka, NATS JetStream, AWS EventBridge, Redpanda, Pulsar 가 dominant brokers.

매 핵심

매 Broker vs Mediator

측면	Broker	Mediator
Coordination	None — events fan out	Central mediator orchestrates
Coupling	Loosest	Some (mediator knows steps)
Use case	Simple fan-out, async notify	Complex multi-step workflow
Failure recovery	Each consumer handles	Mediator retries/compensates
Examples	Kafka topics, NATS subjects	Apache Camel, Step Functions

매 컴포넌트

• Producer: events 의 publish.
• Broker: topic/subject/queue store + routing.
• Consumer: subscribe + react.
• Schema Registry (Confluent, Apicurio): event contract 의 versioning.
• Dead Letter Queue (DLQ): failed messages.

매 broker 종류

• Log-based (Kafka, Redpanda, Pulsar): replay 가능, partitioned, ordered per partition.
• Queue-based (RabbitMQ, SQS): consumed once, no replay.
• Subject-based (NATS): lightweight, JetStream 으로 persistence.
• Cloud-native (EventBridge, Pub/Sub, EventHubs).

매 응용

1. Order events (e-commerce fan-out).
2. CDC (Debezium → Kafka → multiple sinks).
3. IoT telemetry ingestion.
4. Analytics event pipeline.

💻 패턴

Kafka producer (TypeScript / KafkaJS)

import { Kafka } from 'kafkajs';

const kafka = new Kafka({
  clientId: 'orders',
  brokers: ['kafka-1:9092', 'kafka-2:9092'],
});
const producer = kafka.producer();

await producer.connect();
await producer.send({
  topic: 'order.created',
  messages: [{
    key: order.id,
    value: JSON.stringify(order),
    headers: { 'content-type': 'application/json' },
  }],
});

Kafka consumer (consumer group fan-out)

const consumer = kafka.consumer({ groupId: 'shipping-svc' });
await consumer.connect();
await consumer.subscribe({ topic: 'order.created' });

await consumer.run({
  eachMessage: async ({ message }) => {
    const order = JSON.parse(message.value!.toString());
    await createShipment(order);
  },
});

NATS JetStream

import { connect, JSONCodec } from 'nats';

const nc = await connect({ servers: 'nats://localhost:4222' });
const js = nc.jetstream();
const jc = JSONCodec();

await js.publish('orders.created', jc.encode(order));

const sub = await js.subscribe('orders.>', {
  config: { durable_name: 'shipping' },
});
for await (const m of sub) {
  const order = jc.decode(m.data);
  await createShipment(order);
  m.ack();
}

Schema Registry (Avro + Confluent)

import { SchemaRegistry } from '@kafkajs/confluent-schema-registry';

const registry = new SchemaRegistry({ host: 'http://schema-registry:8081' });
const schema = `{
  "type": "record",
  "name": "OrderCreated",
  "fields": [
    { "name": "id", "type": "string" },
    { "name": "amount", "type": "double" }
  ]
}`;

const { id } = await registry.register({ type: 'AVRO', schema });
const encoded = await registry.encode(id, order);
await producer.send({ topic: 'order.created', messages: [{ value: encoded }] });

DLQ pattern

await consumer.run({
  eachMessage: async ({ topic, message }) => {
    try {
      await handleOrder(JSON.parse(message.value!.toString()));
    } catch (err) {
      await producer.send({
        topic: `${topic}.dlq`,
        messages: [{
          value: message.value,
          headers: {
            ...message.headers,
            error: String(err),
            retryCount: '0',
          },
        }],
      });
    }
  },
});

Partitioning for ordering

// Same orderId always goes to same partition → ordered per order
await producer.send({
  topic: 'order.events',
  messages: [{
    key: order.id, // partitioner uses key hash
    value: JSON.stringify(event),
  }],
});

매 결정 기준

상황	Approach
High throughput, replay 필요	Kafka / Redpanda
Lightweight, low-latency	NATS JetStream
Cloud serverless	EventBridge, Pub/Sub
Strong work-queue + ack	RabbitMQ, SQS
Multi-tenant, geo	Pulsar

기본값: Kafka (or Redpanda — Kafka API compat) + Schema Registry + Avro/Protobuf for events.

🔗 Graph

• 부모: Event-Driven Architecture
• 변형: Mediator Topology · Choreography
• 응용: Kafka · NATS
• Adjacent: CDC

🤖 LLM 활용

언제: fan-out events, decoupled microservices, CDC pipelines, telemetry ingestion. 언제 X: synchronous request-reply (use gRPC/HTTP), small monolith (events 가 overhead).

❌ 안티패턴

• No schema registry: 모든 producer/consumer 가 schema drift — production 깨짐.
• Topic per service: scalability impossible — topic per event type.
• No DLQ: poison messages 가 consumer halt — DLQ + alerting.
• Sync over async: HTTP request-reply 를 broker 위에 — synchronous 면 broker 의 X.

🧪 검증 / 중복

• Verified (Richards "Software Architecture Patterns" / Confluent docs / NATS docs 2026).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — Broker vs Mediator + Kafka/NATS/DLQ 패턴