Antigravity Agent
34 lines
1.1 KiB
Markdown
34 lines
1.1 KiB
Markdown
---
|
|
id: wiki-2026-0508-tokenization-economics
|
|
title: Tokenization Economics
|
|
category: 10_Wiki/Topics
|
|
status: duplicate
|
|
canonical_id: tokenization-subword-processing
|
|
duplicate_of: "[[Tokenization & Subword Processing]]"
|
|
aliases: []
|
|
source_trust_level: A
|
|
confidence_score: 0.9
|
|
verification_status: redirected
|
|
tags: [duplicate, tokenization, cost, llm]
|
|
last_reinforced: 2026-05-10
|
|
github_commit: pending
|
|
---
|
|
|
|
# Tokenization Economics
|
|
|
|
> **이 문서는 [[Tokenization & Subword Processing]] 의 중복본입니다.** Canonical 문서로 redirect.
|
|
|
|
## 핵심 요약 (specialization: cost aspects)
|
|
- Token-based pricing (input/output tokens) → tokenizer choice 가 직접 cost 에 영향.
|
|
- Korean / CJK 의 token 비효율 (English 대비 2-3x tokens per character).
|
|
- Canonical 문서가 tokenizer comparison, prompt caching economics, batch API pricing 을 다룸.
|
|
|
|
## 🔗 Graph
|
|
- 부모: [[Tokenization & Subword Processing]] (canonical)
|
|
|
|
## 🕓 변경 이력
|
|
| 날짜 | 변경 |
|
|
|---|---|
|
|
| 2026-05-08 | Phase 1 |
|
|
| 2026-05-10 | 중복 처리 — canonical 문서로 redirect |
|