--- id: wiki-2026-0508-state-space-model-ssm title: State Space Model (SSM) category: 10_Wiki/Topics status: needs_review canonical_id: self aliases: [P-Reinforce-AUTO-SSMM-001] duplicate_of: none source_trust_level: A confidence_score: 0.97 tags: [auto-reinforced, ssm, mamba, neural-networks, Sequence-Modeling, computational-Efficiency] raw_sources: [] last_reinforced: 2026-04-20 github_commit: pending inferred_by: Claude Opus 4.7 (auto-normalize 2026-05-08) tech_stack: language: unspecified framework: unspecified --- # [[State Space Model (SSM)|State Space Model (SSM)]] ## ๐Ÿ“Œ ํ•œ ์ค„ ํ†ต์ฐฐ (The Karpathy Summary) > "ํŠธ๋žœ์Šคํฌ๋จธ์˜ ๋…์ฃผ๋ฅผ ์œ„ํ˜‘ํ•˜๋Š” ์„ ํ˜•์˜ ๋งˆ๋ฒ•: ๋ฐ์ดํ„ฐ ๊ธธ์ด์— ๋”ฐ๋ผ ์—ฐ์‚ฐ๋Ÿ‰์ด ํญ์ฆํ•˜๋Š” ํ•œ๊ณ„๋ฅผ ๊ทน๋ณตํ•˜๊ณ , ์ž…๋ ฅ ๋ฐ์ดํ„ฐ๋ฅผ ์••์ถ•๋œ '์ƒํƒœ([[State|State]])'๋กœ ๊ด€๋ฆฌํ•˜์—ฌ ๋ฌดํ•œ์— ๊ฐ€๊นŒ์šด ๋ฌธ๋งฅ์„ ๊ฐ€๋ณ๊ฒŒ ์ฒ˜๋ฆฌํ•˜๋Š” ์ƒˆ๋กœ์šด ๋”ฅ๋Ÿฌ๋‹ ์•„ํ‚คํ…์ฒ˜." ## ๐Ÿ“– ๊ตฌ์กฐํ™”๋œ ์ง€์‹ (Synthesized Content) ์ƒํƒœ ๊ณต๊ฐ„ ๋ชจ๋ธ(State Space Model, SSM)์€ ์‹ ํ˜ธ ์ฒ˜๋ฆฌ์™€ ์ œ์–ด ๊ณตํ•™์˜ ๊ณ ์ „์  ์ด๋ก ์„ ํ˜„๋Œ€ ๋”ฅ๋Ÿฌ๋‹์— ์ ‘๋ชฉํ•˜์—ฌ ์‹œํ€€์Šค ๋ฐ์ดํ„ฐ๋ฅผ ํšจ์œจ์ ์œผ๋กœ ์ฒ˜๋ฆฌํ•˜๋Š” ์•„ํ‚คํ…์ฒ˜์ž…๋‹ˆ๋‹ค. 1. **๋™์ž‘ ์›๋ฆฌ (Mamba ๋“ฑ ์ตœ์‹  ๋ชจ๋ธ ๊ธฐ์ค€)**: * **Continuous to Discrete**: ๋ฏธ๋ถ„ ๋ฐฉ์ •์‹์„ ์ด์‚ฐ์ ์ธ ํ˜•ํƒœ๋กœ ๋ณ€ํ™˜ํ•˜์—ฌ ์—ฐ์‚ฐ ์ˆ˜ํ–‰. * **Recurrent Process**: RNN์ฒ˜๋Ÿผ ์ด์ „ ์ •๋ณด๋ฅผ 'State'๋ผ๋Š” ๊ณ ์ •๋œ ํฌ๊ธฐ์˜ ๋ฉ”๋ชจ๋ฆฌ์— ์ €์žฅํ•˜๊ณ  ๋„˜๊น€. * **Parallel [[Processing|Processing]]**: ํ•™์Šต ์‹œ์—๋Š” CNN์ฒ˜๋Ÿผ ๋ณ‘๋ ฌ ์—ฐ์‚ฐ์ด ๊ฐ€๋Šฅํ•˜๊ฒŒ ์ •์‹ํ™”ํ•˜์—ฌ ์ „๋ ฅ ํšจ์œจ ๊ทน๋Œ€ํ™”. 2. **ํŠธ๋žœ์Šคํฌ๋จธ(Attention)์™€์˜ ์ฐจ์ด**: * **Transformer**: ์ž…๋ ฅ์ด ๊ธธ์–ด์งˆ์ˆ˜๋ก ์—ฐ์‚ฐ๋Ÿ‰์ด ์ œ๊ณฑ($O(n^2)$)์œผ๋กœ ๋Š˜์–ด๋‚จ. * **SSM**: ์ž…๋ ฅ ๊ธธ์ด์— ์„ ํ˜•์ ์œผ๋กœ($O(n)$) ๋น„๋ก€ํ•˜์—ฌ ์—ฐ์‚ฐ ์ˆ˜ํ–‰. ๋ฉ”๋ชจ๋ฆฌ ์ ์œ ์œจ์ด ํš๊ธฐ์ ์œผ๋กœ ๋‚ฎ์Œ. 3. **ํ•ต์‹ฌ ์ด์ **: * ๋งค์šฐ ๊ธด ๋ฌธ๋งฅ(Context Window)์„ ๋น„์šฉ ํšจ์œจ์ ์œผ๋กœ ์ฒ˜๋ฆฌ ๊ฐ€๋Šฅ. * ์ถ”๋ก  ์†๋„๊ฐ€ ๋งค์šฐ ๋น ๋ฅด๊ณ  ์ž์› ์ œ์•ฝ์ด ์žˆ๋Š” ๊ธฐ๊ธฐ(Edge Device)์— ์ ํ•ฉ. ## โš ๏ธ ๋ชจ์ˆœ ๋ฐ ์—…๋ฐ์ดํŠธ (Contradictions & Updates) - **๊ณผ๊ฑฐ ๋ฐ์ดํ„ฐ์™€์˜ ์ถฉ๋Œ**: ์ด์ „์—๋Š” ํŠธ๋žœ์Šคํฌ๋จธ๊ฐ€ AI์˜ ์ข…์ฐฉ์ง€๋กœ ์—ฌ๊ฒจ์กŒ์œผ๋‚˜, ์ตœ๊ทผ Mamba์™€ ๊ฐ™์€ SSM ๊ธฐ๋ฐ˜ ๋ชจ๋ธ๋“ค์ด ๋Œ€๊ทœ๋ชจ ์–ธ์–ด ๋ชจ๋ธ๋ง์—์„œ ํŠธ๋žœ์Šคํฌ๋จธ๋ฅผ ๋Šฅ๊ฐ€ํ•˜๋Š” ํšจ์œจ์„ฑ์„ ์ฆ๋ช…ํ•˜๋ฉฐ 'ํƒˆ-ํŠธ๋žœ์Šคํฌ๋จธ' ์ •์ฑ…์˜ ์„ ๋‘ ์ฃผ์ž๋กœ ๋ถ€์ƒํ•จ(RL Update). - **์ •์ฑ… ๋ณ€ํ™”(RL Update)**: ์—๋„ˆ์ง€ ํšจ์œจ์ด ๊ธ€๋กœ๋ฒŒ AI ์—ฐ๊ตฌ์˜ ํ•ต์‹ฌ ์ •์ฑ… ์ง€ํ‘œ๋กœ ๋– ์˜ค๋ฆ„์— ๋”ฐ๋ผ, ์ €์ „๋ ฅ ๊ณ ์„ฑ๋Šฅ์„ ๋ณด์žฅํ•˜๋Š” SSM ์•„ํ‚คํ…์ฒ˜ ์—ฐ๊ตฌ์— ๋Œ€ํ•œ ์ง‘์ค‘ ํˆฌ์ž ๋ฐ ํ•˜๋“œ์›จ์–ด ๊ฐ€์†๊ธฐ(NVIDIA GPU ๋“ฑ) ์„œํฌํŠธ ์ •์ฑ…์ด ๊ฐ•ํ™”๋จ. ## ๐Ÿ”— ์ง€์‹ ์—ฐ๊ฒฐ (Graph) - Foundational Models, [[Complexity Theory|Complexity Theory]], [[Reactive-Programming|Reactive-Programming]], Sequence Modeling, [[memory|memory]] Mechanisms in AI - **Modern Tech/Tools**: Mamba, S4, Hyena Hierarchy, PyTorch Mamba implementation. --- ## ๐Ÿค– LLM ํ™œ์šฉ ํžŒํŠธ (How to Use This Knowledge) **์–ธ์ œ ์ด ์ง€์‹์„ ์“ฐ๋Š”๊ฐ€:** - *(TODO)* **์–ธ์ œ ์“ฐ๋ฉด ์•ˆ ๋˜๋Š”๊ฐ€:** - *(TODO)* ## ๐Ÿงช ๊ฒ€์ฆ ์ƒํƒœ (Validation) - **์ •๋ณด ์ƒํƒœ:** needs_review - **์ถœ์ฒ˜ ์‹ ๋ขฐ๋„:** A - **๊ฒ€ํ†  ์ด์œ :** *(P-Reinforce Phase 1 ์ž๋™ ์ •๊ทœํ™”. ๋ณธ๋ฌธ ๊ฒ€์ฆ ํ•„์š”.)* ## ๐Ÿงฌ ์ค‘๋ณต ๊ฒ€์‚ฌ (Duplicate Check) - **๊ธฐ์กด ์œ ์‚ฌ ๋ฌธ์„œ:** *(TODO: ์ธ๋ฑ์„œ ํด๋Ÿฌ์Šคํ„ฐ ๋ฆฌํฌํŠธ ์ฐธ์กฐ)* - **์ฒ˜๋ฆฌ ๋ฐฉ์‹:** UPDATE (์ž๋™ ์ •๊ทœํ™”) - **์ฒ˜๋ฆฌ ์ด์œ :** Phase 1 ์ •๊ทœํ™” โ€” ์˜› ํ…œํ”Œ๋ฆฟ/๋ˆ„๋ฝ ํ•„๋“œ ๋ณด๊ฐ•. ## ๐Ÿ•“ ๋ณ€๊ฒฝ ์ด๋ ฅ (Changelog) | ๋‚ ์งœ | ๋ณ€๊ฒฝ ๋‚ด์šฉ | ์ฒ˜๋ฆฌ ๋ฐฉ์‹ | ์‹ ๋ขฐ๋„ | |------|-----------|-----------|--------| | 2026-05-08 | P-Reinforce Phase 1 ์ •๊ทœํ™” (frontmatter + ํ—ค๋” ํ‘œ์ค€ํ™”) | UPDATE | A | ## ๐Ÿ’ป ์ฝ”๋“œ ํŒจํ„ด (Code Patterns) **ํŒจํ„ด 1:** *(TODO: ์ด ํ”„๋กœ์ ํŠธ ์ปจ๋ฒค์…˜ ๋ฐ˜์˜ํ•œ ๊ตฌ์กฐ ์Šค์ผˆ๋ ˆํ†ค)* ```text # TODO ``` ## ๐Ÿค” ์˜์‚ฌ๊ฒฐ์ • ๊ธฐ์ค€ (Decision Criteria) **์„ ํƒ A๋ฅผ ์จ์•ผ ํ•  ๋•Œ:** - *(TODO)* **์„ ํƒ B๋ฅผ ์จ์•ผ ํ•  ๋•Œ:** - *(TODO)* **๊ธฐ๋ณธ๊ฐ’:** > *(TODO)* ## โŒ ์•ˆํ‹ฐํŒจํ„ด (Anti-Patterns) - **[์•ˆํ‹ฐํŒจํ„ด]:** *(TODO: ๋ฌด์—‡์„ ํ•˜๋ฉด ์•ˆ ๋˜๋Š”๊ฐ€ + ์ด์œ  + ๋Œ€์‹  ๋ฌด์—‡์„)*