docs(10_Wiki): 위키 전체 재구성 — Topic_* 폴더를 4개 카테고리로 통합 + 대규모 중복 제거

Topic_Agent/Topic_Blog/Topics/Topics_Biz/Topics_Meeting/Topics_Rag의 마크다운 지식 문서를 Topic_General/Topic_Programming/Topic_Graphic/Topic_Business 4개 카테고리로 재분류. - 중복 제거: frontmatter의 status:duplicate/merged + duplicate_of/redirect_to 필드로 자기 자신을 중복으로 선언한 리다이렉트 stub 1032개 제거, 완전 동일 내용 파일 472개 제거, 동일 파일명·다른 내용 충돌 시 더 큰(완전한) 버전만 유지(162개 제거) — 총 1639개 중복 제거. - 분류: 폴더 단위로 명확한 항목(AI_and_ML/Coding/Architecture 등 → Programming, Comfyui/Visual_Effects → Graphic, Topics_Biz/Topics_Meeting/사업 등 → Business, Poetic_Blog_Writing/창의성/Game_Design 등 → General)은 폴더 우선순위로, 나머지 혼재 폴더(Topic_Agent/Topic_Blog/Topics 루트/Thinking & Reasoning/Other/UI_UX_Assets)는 title/tags 키워드 스코어링으로 파일 단위 분류(불명확한 경우 General로 폴백). 원본 폴더명은 "From_*" 서브폴더로 보존해 추적 가능성 유지. - 최종 배치: Programming 2784 / General 1608 / Graphic 285 / Business 249 = 4926개 문서. - 에이전트 운영 상태(.astra/.agent/.obsidian/sessions/memory/_company/docs/lessons/_shared/src)는 지식 콘텐츠가 아니므로 재분류 대상에서 제외하고 원위치 유지. - Topics/Topic_email(상위 보호 폴더 Topic_email과 파일명 100% 중복) 삭제 — 보호 폴더 자체는 미변경. - 완전히 비게 된 Topic_Agent/Topic_Blog/Topics_Biz/Topics_Rag 폴더 제거.
2026-07-05 00:33:48 +09:00
parent 1cfd3bbb56
commit 9148c358d0
6455 changed files with 1 additions and 86875 deletions
@@ -0,0 +1,245 @@
+---
+id: wiki-2026-0508-pytorch-lightning
+title: PyTorch Lightning
+category: 10_Wiki/Topics
+status: verified
+canonical_id: self
+aliases: [Lightning, pl, lightning.pytorch]
+duplicate_of: none
+source_trust_level: A
+confidence_score: 0.9
+verification_status: applied
+tags: [pytorch, training, framework, distributed]
+raw_sources: []
+last_reinforced: 2026-05-10
+github_commit: pending
+tech_stack:
+  language: python
+  framework: pytorch-lightning
+---
+
+# PyTorch Lightning
+
+## 매 한 줄
+> **"매 PyTorch boilerplate 의 elimination — research-style structured trainer"**. LightningModule (model + optim + step) + Trainer (loop + distributed + logging) 의 separation. 2026 현재 매 still strong for research / classical DL, 매 LLM-era 의 HuggingFace Trainer / Accelerate / TRL 의 dominate.
+
+## 매 핵심
+
+### 매 LightningModule lifecycle
+- `__init__`: model + hparams.
+- `forward(x)`: inference.
+- `training_step(batch, idx) -> loss`: per-batch train.
+- `validation_step` / `test_step`: eval.
+- `configure_optimizers() -> optim | (optim, sched)`: opt + scheduler.
+- `on_*_epoch_end` hooks for aggregation.
+
+### 매 Trainer features
+- Multi-GPU (DDP, FSDP), TPU, MPS automatic.
+- Mixed precision (`precision="bf16-mixed"`).
+- Gradient accumulation, clipping built-in.
+- Callbacks (EarlyStopping, ModelCheckpoint, LR monitor).
+- Loggers (TensorBoard, WandB, MLflow, CSV).
+- `fast_dev_run`, `overfit_batches`, `limit_*_batches` for debug.
+
+### 매 vs alternatives (2026)
+| Framework | Best for |
+|---|---|
+| Lightning | research, classical CV/NLP, structured projects |
+| HF Trainer | HF-ecosystem (transformers + datasets), LLM SFT |
+| HF Accelerate | minimal wrapper, retain raw PyTorch loop |
+| TRL | RLHF / DPO / GRPO, LLM post-training |
+| MosaicML Composer | streaming, throughput-optimized |
+| raw PyTorch | full control, simple scripts |
+
+### 매 응용
+1. CV training (image classification, segmentation, detection).
+2. Tabular DL (TabNet, FT-Transformer).
+3. Audio / speech (W2V2 finetune).
+4. Mid-size LLM finetune (when not using HF Trainer).
+5. Self-supervised pretraining (SimCLR, MAE).
+
+## 💻 패턴
+
+### Minimal LightningModule
+```python
+import lightning as L
+import torch
+import torch.nn as nn
+from torch.utils.data import DataLoader
+
+class LitClassifier(L.LightningModule):
+    def __init__(self, lr=1e-3):
+        super().__init__()
+        self.save_hyperparameters()
+        self.net = nn.Sequential(
+            nn.Flatten(), nn.Linear(28*28, 256), nn.ReLU(), nn.Linear(256, 10),
+        )
+        self.loss = nn.CrossEntropyLoss()
+
+    def forward(self, x):
+        return self.net(x)
+
+    def training_step(self, batch, idx):
+        x, y = batch
+        logits = self(x)
+        loss = self.loss(logits, y)
+        self.log("train_loss", loss, prog_bar=True)
+        return loss
+
+    def validation_step(self, batch, idx):
+        x, y = batch
+        logits = self(x)
+        acc = (logits.argmax(-1) == y).float().mean()
+        self.log("val_acc", acc, prog_bar=True)
+
+    def configure_optimizers(self):
+        opt = torch.optim.AdamW(self.parameters(), lr=self.hparams.lr)
+        sched = torch.optim.lr_scheduler.CosineAnnealingLR(opt, T_max=10)
+        return [opt], [sched]
+```
+
+### Trainer with callbacks
+```python
+from lightning.pytorch.callbacks import EarlyStopping, ModelCheckpoint, LearningRateMonitor
+from lightning.pytorch.loggers import WandbLogger
+
+trainer = L.Trainer(
+    max_epochs=20,
+    accelerator="auto",        # cuda / mps / cpu
+    devices="auto",
+    precision="bf16-mixed",
+    accumulate_grad_batches=4,
+    gradient_clip_val=1.0,
+    callbacks=[
+        EarlyStopping(monitor="val_acc", mode="max", patience=3),
+        ModelCheckpoint(monitor="val_acc", mode="max", save_top_k=2),
+        LearningRateMonitor(),
+    ],
+    logger=WandbLogger(project="lit-mnist"),
+)
+trainer.fit(LitClassifier(), train_dl, val_dl)
+```
+
+### Multi-GPU DDP
+```python
+trainer = L.Trainer(
+    accelerator="gpu",
+    devices=4,
+    strategy="ddp",            # or "fsdp" for >7B params
+    precision="bf16-mixed",
+    sync_batchnorm=True,
+)
+# 매 launch with `python train.py` — Lightning 의 spawn workers
+```
+
+### FSDP for large model
+```python
+from lightning.pytorch.strategies import FSDPStrategy
+from torch.distributed.fsdp.wrap import transformer_auto_wrap_policy
+from functools import partial
+
+policy = partial(transformer_auto_wrap_policy, transformer_layer_cls={MyTransformerBlock})
+
+trainer = L.Trainer(
+    devices=8,
+    strategy=FSDPStrategy(auto_wrap_policy=policy, cpu_offload=False),
+    precision="bf16-mixed",
+)
+```
+
+### LightningDataModule
+```python
+class MNISTDataModule(L.LightningDataModule):
+    def __init__(self, batch_size=64):
+        super().__init__()
+        self.bs = batch_size
+
+    def prepare_data(self):
+        from torchvision.datasets import MNIST
+        MNIST(".", train=True, download=True)
+
+    def setup(self, stage=None):
+        from torchvision.datasets import MNIST
+        from torchvision import transforms
+        t = transforms.ToTensor()
+        self.train = MNIST(".", train=True, transform=t)
+        self.val = MNIST(".", train=False, transform=t)
+
+    def train_dataloader(self):
+        return DataLoader(self.train, batch_size=self.bs, num_workers=4, shuffle=True)
+
+    def val_dataloader(self):
+        return DataLoader(self.val, batch_size=self.bs, num_workers=4)
+```
+
+### LightningCLI (config-driven)
+```python
+# train.py
+from lightning.pytorch.cli import LightningCLI
+
+def main():
+    LightningCLI(LitClassifier, MNISTDataModule)
+
+if __name__ == "__main__":
+    main()
+# python train.py fit --config config.yaml --trainer.max_epochs=30
+```
+
+### Resume from checkpoint
+```python
+trainer.fit(model, datamodule, ckpt_path="lightning_logs/version_3/checkpoints/last.ckpt")
+# or load model standalone
+model = LitClassifier.load_from_checkpoint("path.ckpt")
+```
+
+### Manual optimization (GAN, RL)
+```python
+class LitGAN(L.LightningModule):
+    def __init__(self):
+        super().__init__()
+        self.automatic_optimization = False
+        ...
+
+    def training_step(self, batch, idx):
+        opt_g, opt_d = self.optimizers()
+        # discriminator step
+        opt_d.zero_grad(); d_loss = ...; self.manual_backward(d_loss); opt_d.step()
+        # generator step
+        opt_g.zero_grad(); g_loss = ...; self.manual_backward(g_loss); opt_g.step()
+```
+
+## 매 결정 기준
+| 상황 | Approach |
+|---|---|
+| Research, multi-experiment, structured | Lightning |
+| HF transformers SFT | HF Trainer (closer to ecosystem) |
+| Custom training loop, retain control | Accelerate |
+| LLM RLHF / DPO / GRPO | TRL |
+| Single-GPU script <100 lines | raw PyTorch |
+| Need callbacks + DDP fast | Lightning |
+
+**기본값**: 매 research / non-HF training 의 Lightning + bf16-mixed + DDP. 매 HF transformers job 의 HF Trainer. 매 LLM post-training 의 TRL.
+
+## 🔗 Graph
+- 응용: [[Distributed-Training]]
+
+## 🤖 LLM 활용
+**언제**: scaffold LightningModule from arch description, generate callback config, debug DDP issues.
+**언제 X**: deep performance tuning (FSDP wrap policy, custom strategy) — 매 verify with profiler, 매 LLM 의 outdated API common.
+
+## ❌ 안티패턴
+- **`.cuda()` inside LightningModule**: Lightning manages device — use `self.device` or just rely on Trainer.
+- **Manual DDP setup**: Lightning handles, don't double-wrap.
+- **Logging in DDP without `sync_dist=True`**: rank-0 only logs, miss aggregation.
+- **`automatic_optimization=True` for GAN**: silent wrong loss flow — manual mode.
+- **Pinning to old Lightning 1.x**: 매 2.x API change (lightning.pytorch namespace), 매 2026 의 2.x+ standard.
+
+## 🧪 검증 / 중복
+- Verified (lightning.ai docs 2026, Lightning 2.x release notes, Falcon 2019 origin paper, Lightning Studios).
+- 신뢰도 A.
+
+## 🕓 Changelog
+| 날짜 | 변경 |
+|---|---|
+| 2026-05-08 | Phase 1 |
+| 2026-05-10 | Manual cleanup — LightningModule + Trainer + DDP/FSDP patterns, 2026 alt comparison |