Files
2nd/10_Wiki/Topics/AI_and_ML/Tokenization Economics.md
T
2026-05-10 22:08:15 +09:00

1.1 KiB

id, title, category, status, canonical_id, duplicate_of, aliases, source_trust_level, confidence_score, verification_status, tags, last_reinforced, github_commit
id title category status canonical_id duplicate_of aliases source_trust_level confidence_score verification_status tags last_reinforced github_commit
wiki-2026-0508-tokenization-economics Tokenization Economics 10_Wiki/Topics duplicate tokenization-subword-processing Tokenization & Subword Processing
A 0.9 redirected
duplicate
tokenization
cost
llm
2026-05-10 pending

Tokenization Economics

이 문서는 Tokenization & Subword Processing 의 중복본입니다. Canonical 문서로 redirect.

핵심 요약 (specialization: cost aspects)

  • Token-based pricing (input/output tokens) → tokenizer choice 가 직접 cost 에 영향.
  • Korean / CJK 의 token 비효율 (English 대비 2-3x tokens per character).
  • Canonical 문서가 tokenizer comparison, prompt caching economics, batch API pricing 을 다룸.

🔗 Graph

🕓 변경 이력

날짜 변경
2026-05-08 Phase 1
2026-05-10 중복 처리 — canonical 문서로 redirect