docs(10_Wiki): Topic_Business/General/Graphic/Programming을 Topics/ 하위로 이동
최상위 10_Wiki/Topic_*였던 4개 카테고리 폴더를 10_Wiki/Topics/Topic_* 로 재배치. 콘텐츠 변경 없음(순수 폴더 이동) — Topics/ 하위 나머지 폴더는 이미 지난 커밋에서 전부 정리된 상태(잔존 항목은 에이전트 운영 상태 및 사용자가 보존을 요청한 업데이트0615/무제 3.canvas 뿐).
This commit is contained in:
Antigravity Agent
@@ -0,0 +1,260 @@
|
||||
---
|
||||
id: wiki-2026-0508-foundation-models
|
||||
title: Foundation Models
|
||||
category: 10_Wiki/Topics
|
||||
status: verified
|
||||
canonical_id: self
|
||||
aliases: [foundation model, FM, large pretrained, LLM, VLM, multimodal foundation, GPT, Claude, Gemini, Llama]
|
||||
duplicate_of: none
|
||||
source_trust_level: A
|
||||
confidence_score: 0.98
|
||||
verification_status: applied
|
||||
tags: [ai, foundation-model, llm, vlm, multimodal, scaling-laws, pretraining]
|
||||
raw_sources: []
|
||||
last_reinforced: 2026-05-10
|
||||
github_commit: pending
|
||||
tech_stack:
|
||||
language: Python
|
||||
framework: PyTorch / Transformers / vLLM
|
||||
---
|
||||
|
||||
# Foundation Models
|
||||
|
||||
## 매 한 줄
|
||||
> **"매 large-scale 의 의 의 self-supervised pretrained 의 의 의 다양한 task 의 의 의 adapt"**. Bommasani 2021 Stanford term. 매 LLM (GPT, Claude, Gemini, Llama), VLM (GPT-4V, Claude 3, Gemini), 매 audio (Whisper), 매 protein (ESM, AlphaFold), 매 robot (RT-2, π0).
|
||||
|
||||
## 매 핵심
|
||||
|
||||
### 매 traits
|
||||
- **Scale**: 매 billions of parameter.
|
||||
- **Pretrain**: 매 vast unsupervised data.
|
||||
- **Emergence** (debated).
|
||||
- **Adaptable**: 매 prompt / fine-tune / RAG.
|
||||
- **Foundation 효과**: 매 downstream 의 ↑.
|
||||
|
||||
### 매 modality
|
||||
- **Text**: GPT, Claude, Gemini, Llama, Mistral.
|
||||
- **Vision-Language**: GPT-4V, Claude 3, Gemini, LLaVA.
|
||||
- **Audio**: Whisper, AudioLM.
|
||||
- **Code**: Codex, CodeLlama.
|
||||
- **Protein**: ESM-2, AlphaFold-3.
|
||||
- **Robot**: RT-2, OpenVLA, π0.
|
||||
- **Time-series** (TimeGPT).
|
||||
|
||||
### 매 scaling laws
|
||||
- **Kaplan 2020**: 매 power law (loss vs params/data/compute).
|
||||
- **Chinchilla** (Hoffmann 2022): 매 D ≈ 20·N optimal.
|
||||
- **Modern (2024+)**: 매 over-train (Llama 3 매 15T tokens).
|
||||
|
||||
### 매 modern (2025-2026)
|
||||
- **Frontier**: Claude Opus 4.7, GPT-5, Gemini 2 Ultra.
|
||||
- **Open**: Llama 3.x, Qwen 2.5, Mistral, DeepSeek-V3.
|
||||
- **Multimodal**: Gemini 1.5 1M context, Claude 3.5.
|
||||
- **MoE**: Mixtral, DeepSeek MoE.
|
||||
- **Reasoning**: o1, o3, DeepSeek-R1.
|
||||
|
||||
### 매 응용
|
||||
1. **General assistant**.
|
||||
2. **Code**.
|
||||
3. **Domain expert** (medical, legal).
|
||||
4. **Multimodal analysis**.
|
||||
5. **Agent**.
|
||||
6. **Embedding** (retrieval, clustering).
|
||||
|
||||
## 💻 패턴
|
||||
|
||||
### LLM call (Anthropic)
|
||||
```python
|
||||
from anthropic import Anthropic
|
||||
client = Anthropic()
|
||||
response = client.messages.create(
|
||||
model='claude-opus-4-7',
|
||||
max_tokens=1024,
|
||||
messages=[{'role': 'user', 'content': 'Hi'}],
|
||||
)
|
||||
```
|
||||
|
||||
### Streaming
|
||||
```python
|
||||
with client.messages.stream(
|
||||
model='claude-opus-4-7',
|
||||
max_tokens=1024,
|
||||
messages=[{'role': 'user', 'content': prompt}],
|
||||
) as stream:
|
||||
for text in stream.text_stream:
|
||||
print(text, end='', flush=True)
|
||||
```
|
||||
|
||||
### Tool use (function calling)
|
||||
```python
|
||||
tools = [{
|
||||
'name': 'get_weather',
|
||||
'description': 'Get weather for a location',
|
||||
'input_schema': {'type': 'object', 'properties': {'location': {'type': 'string'}}},
|
||||
}]
|
||||
r = client.messages.create(model='claude-opus-4-7', tools=tools, messages=[...])
|
||||
if r.stop_reason == 'tool_use':
|
||||
tool = next(b for b in r.content if b.type == 'tool_use')
|
||||
result = execute_tool(tool.name, tool.input)
|
||||
```
|
||||
|
||||
### Vision (multimodal)
|
||||
```python
|
||||
import base64
|
||||
img_b64 = base64.b64encode(open('img.jpg', 'rb').read()).decode()
|
||||
client.messages.create(model='claude-opus-4-7', max_tokens=1024, messages=[{
|
||||
'role': 'user',
|
||||
'content': [
|
||||
{'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': img_b64}},
|
||||
{'type': 'text', 'text': 'What do you see?'},
|
||||
],
|
||||
}])
|
||||
```
|
||||
|
||||
### Open-source (Hugging Face)
|
||||
```python
|
||||
from transformers import AutoModelForCausalLM, AutoTokenizer
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained('meta-llama/Llama-3.1-70B-Instruct', torch_dtype='bfloat16', device_map='auto')
|
||||
tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-3.1-70B-Instruct')
|
||||
inputs = tokenizer.apply_chat_template([{'role': 'user', 'content': 'Hi'}], return_tensors='pt')
|
||||
outputs = model.generate(inputs, max_new_tokens=200)
|
||||
```
|
||||
|
||||
### Embedding (text)
|
||||
```python
|
||||
from sentence_transformers import SentenceTransformer
|
||||
m = SentenceTransformer('all-mpnet-base-v2')
|
||||
vecs = m.encode(['hello', 'world'])
|
||||
|
||||
# 매 production
|
||||
from openai import OpenAI
|
||||
client = OpenAI()
|
||||
emb = client.embeddings.create(input='hello', model='text-embedding-3-large').data[0].embedding
|
||||
```
|
||||
|
||||
### CLIP (vision-text)
|
||||
```python
|
||||
from transformers import CLIPModel, CLIPProcessor
|
||||
model = CLIPModel.from_pretrained('openai/clip-vit-large-patch14')
|
||||
processor = CLIPProcessor.from_pretrained('openai/clip-vit-large-patch14')
|
||||
inputs = processor(text=['cat', 'dog'], images=image, return_tensors='pt')
|
||||
outputs = model(**inputs)
|
||||
similarities = outputs.logits_per_image.softmax(dim=-1)
|
||||
```
|
||||
|
||||
### Foundation model for robotics (OpenVLA)
|
||||
```python
|
||||
# 매 example concept
|
||||
from transformers import AutoProcessor, AutoModelForVision2Seq
|
||||
processor = AutoProcessor.from_pretrained('openvla/openvla-7b')
|
||||
model = AutoModelForVision2Seq.from_pretrained('openvla/openvla-7b', torch_dtype=torch.bfloat16)
|
||||
inputs = processor('In: What action?\nOut:', image, return_tensors='pt')
|
||||
action = model.predict_action(**inputs, unnorm_key='bridge_orig')
|
||||
```
|
||||
|
||||
### vLLM serving
|
||||
```python
|
||||
from vllm import LLM, SamplingParams
|
||||
llm = LLM(model='meta-llama/Llama-3.1-8B-Instruct', tensor_parallel_size=2)
|
||||
outputs = llm.generate(prompts, SamplingParams(max_tokens=100, temperature=0.7))
|
||||
```
|
||||
|
||||
### Fine-tune (LoRA)
|
||||
```python
|
||||
from peft import LoraConfig, get_peft_model
|
||||
config = LoraConfig(r=16, lora_alpha=32, target_modules=['q_proj', 'v_proj'])
|
||||
model = get_peft_model(model, config)
|
||||
# 매 train on task data
|
||||
```
|
||||
|
||||
### Adapter via prompt
|
||||
```python
|
||||
def domain_assistant(question, system_prompt):
|
||||
return client.messages.create(
|
||||
model='claude-opus-4-7',
|
||||
max_tokens=1024,
|
||||
system=system_prompt,
|
||||
messages=[{'role': 'user', 'content': question}],
|
||||
)
|
||||
|
||||
medical_system = 'You are a medical expert. Always recommend consulting a physician.'
|
||||
```
|
||||
|
||||
### Caching (prompt cache)
|
||||
```python
|
||||
# 매 Anthropic prompt caching
|
||||
client.messages.create(
|
||||
model='claude-opus-4-7',
|
||||
max_tokens=1024,
|
||||
system=[
|
||||
{'type': 'text', 'text': 'You are an expert.', 'cache_control': {'type': 'ephemeral'}},
|
||||
{'type': 'text', 'text': long_context, 'cache_control': {'type': 'ephemeral'}},
|
||||
],
|
||||
messages=[{'role': 'user', 'content': question}],
|
||||
)
|
||||
```
|
||||
|
||||
### Agent loop
|
||||
```python
|
||||
def agent(goal, tools, max_steps=10):
|
||||
history = [{'role': 'user', 'content': goal}]
|
||||
for _ in range(max_steps):
|
||||
r = client.messages.create(model='claude-opus-4-7', tools=tools, messages=history)
|
||||
if r.stop_reason == 'end_turn': return r
|
||||
if r.stop_reason == 'tool_use':
|
||||
tool_block = next(b for b in r.content if b.type == 'tool_use')
|
||||
result = execute(tool_block.name, tool_block.input)
|
||||
history.extend([{'role': 'assistant', 'content': r.content}, {'role': 'user', 'content': [{'type': 'tool_result', 'tool_use_id': tool_block.id, 'content': result}]}])
|
||||
```
|
||||
|
||||
### Evaluate (LLM judge)
|
||||
```python
|
||||
def llm_judge(response, criteria):
|
||||
judge_prompt = f"""Rate this response on {criteria}.
|
||||
Response: {response}
|
||||
Output JSON: {{"score": 0-10, "rationale": "..."}}"""
|
||||
return json.loads(client.messages.create(model='claude-opus-4-7', max_tokens=200, messages=[{'role': 'user', 'content': judge_prompt}]).content[0].text)
|
||||
```
|
||||
|
||||
## 매 결정 기준
|
||||
| 상황 | Model |
|
||||
|---|---|
|
||||
| Top quality | Claude Opus 4.7 / GPT-5 |
|
||||
| Cost-aware | Claude Sonnet / GPT-4o-mini |
|
||||
| Open-source | Llama 3.x / Qwen 2.5 |
|
||||
| Code | Claude / DeepSeek-Coder |
|
||||
| Vision | Claude 3.5 / Gemini |
|
||||
| Embedding | text-embedding-3-large / mpnet |
|
||||
| On-device | Llama 3.2 1B/3B / Phi-3 |
|
||||
| Reasoning | o1 / DeepSeek-R1 |
|
||||
|
||||
**기본값**: 매 Frontier API for quality + 매 OSS for cost / control + 매 multimodal where needed + 매 RAG + 매 prompt caching.
|
||||
|
||||
## 🔗 Graph
|
||||
- 부모: [[AI]] · [[Deep Learning]]
|
||||
- 변형: [[Transformer_Architecture_and_LLM_Foundations|LLM]] · [[VLM]] · [[Multimodal-LLM]] · [[Embodied-AI]]
|
||||
- 응용: [[Fine-tuning]] · [[RAG]] · [[Prompt_Engineering|Prompt-Engineering]] · [[Agent]]
|
||||
- Adjacent: [[Scaling-Laws]] · [[Mixture-of-Experts]] · [[Flash Attention]] · [[Edge-AI-and-Computing]]
|
||||
|
||||
## 🤖 LLM 활용
|
||||
**언제**: 매 modern AI 의 default. 매 NLP, multimodal, agent, code.
|
||||
**언제 X**: 매 strict latency / cost / privacy → smaller / on-device.
|
||||
|
||||
## ❌ 안티패턴
|
||||
- **Largest model always**: 매 cost.
|
||||
- **Fine-tune for facts**: 매 RAG 의 better.
|
||||
- **No eval**: 매 quality 의 invisible.
|
||||
- **Single API lock-in**: 매 fallback 의 X.
|
||||
- **No prompt cache**: 매 cost ↑.
|
||||
|
||||
## 🧪 검증 / 중복
|
||||
- Verified (Bommasani 2021, Kaplan 2020, Hoffmann 2022, Anthropic / OpenAI / Google docs).
|
||||
- 신뢰도 A.
|
||||
|
||||
## 🕓 Changelog
|
||||
| 날짜 | 변경 |
|
||||
|---|---|
|
||||
| 2026-04-20 | Auto-reinforced |
|
||||
| 2026-05-08 | Phase 1 |
|
||||
| 2026-05-10 | Manual cleanup — modalities + 매 Anthropic / HF / vLLM / agent / cache code |
|
||||
Reference in New Issue
Block a user