2nd/10_Wiki/Topics/CNN.md

---
id: wiki-2026-0508-cnn
title: CNN (Convolutional Neural Network)
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [ConvNet, Convolutional Network]
duplicate_of: none
source_trust_level: A
confidence_score: 0.95
verification_status: applied
tags: [deep-learning, computer-vision, cnn, neural-network]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python
  framework: PyTorch 2.5 / JAX
---

# CNN (Convolutional Neural Network)

## 매 한 줄
> **"매 CNN 의 핵심: spatial locality + parameter sharing + translation equivariance"**. 매 1989 LeCun LeNet 으로 시작, 매 2012 AlexNet 의 ImageNet breakthrough 가 deep-learning era 의 trigger. 매 2026 현재 ViT 의 주류 진입 unauthenticated, ConvNeXt-V2 / EfficientNet-V2 / RegNet 같은 modern CNN 의 efficiency 의 강점, 매 mobile / edge 의 dominant.

## 매 핵심

### 매 architectural primitive
- **Conv2d**: 매 sliding kernel — 매 (in_ch, out_ch, kH, kW) parameters.
- **Pooling**: max/avg — 매 spatial downsampling.
- **BatchNorm / GroupNorm**: 매 internal covariate shift mitigation.
- **Residual connection (ResNet)**: 매 identity skip — 매 vanishing gradient solved.
- **Depthwise-separable conv (MobileNet)**: 매 efficient — 매 9× FLOPs 감소.

### 매 inductive biases
- **Locality**: 매 nearby pixels correlated.
- **Translation equivariance**: 매 object 의 위치 shift 도 같은 feature.
- **Hierarchy**: 매 edge → texture → part → object.

### 매 응용
1. Image classification (ResNet, ConvNeXt, EfficientNet).
2. Object detection (YOLO v11, RT-DETR backbone).
3. Segmentation (U-Net, DeepLab v3+).
4. Audio spectrograms, time-series, medical imaging.

## 💻 패턴

### Basic CNN block (PyTorch)
```python
import torch.nn as nn

class ConvBlock(nn.Module):
    def __init__(self, in_c, out_c, k=3, s=1):
        super().__init__()
        self.conv = nn.Conv2d(in_c, out_c, k, s, padding=k//2, bias=False)
        self.bn = nn.BatchNorm2d(out_c)
        self.act = nn.GELU()
    def forward(self, x):
        return self.act(self.bn(self.conv(x)))
```

### Residual block (ResNet-style)
```python
class ResBlock(nn.Module):
    def __init__(self, c):
        super().__init__()
        self.b1 = ConvBlock(c, c)
        self.b2 = ConvBlock(c, c)
    def forward(self, x):
        return x + self.b2(self.b1(x))
```

### Depthwise-separable (MobileNet)
```python
class DWSep(nn.Module):
    def __init__(self, in_c, out_c, s=1):
        super().__init__()
        self.dw = nn.Conv2d(in_c, in_c, 3, s, 1, groups=in_c, bias=False)
        self.pw = nn.Conv2d(in_c, out_c, 1, 1, 0, bias=False)
        self.bn = nn.BatchNorm2d(out_c)
        self.act = nn.GELU()
    def forward(self, x):
        return self.act(self.bn(self.pw(self.dw(x))))
```

### ConvNeXt block (2026 modern CNN)
```python
class ConvNeXtBlock(nn.Module):
    def __init__(self, dim):
        super().__init__()
        self.dwconv = nn.Conv2d(dim, dim, 7, padding=3, groups=dim)
        self.norm = nn.LayerNorm(dim)
        self.pw1 = nn.Linear(dim, 4 * dim)
        self.act = nn.GELU()
        self.pw2 = nn.Linear(4 * dim, dim)
    def forward(self, x):
        i = x
        x = self.dwconv(x).permute(0, 2, 3, 1)  # NCHW -> NHWC
        x = self.pw2(self.act(self.pw1(self.norm(x))))
        return i + x.permute(0, 3, 1, 2)
```

### Training loop with mixed precision
```python
import torch
from torch.cuda.amp import autocast, GradScaler

scaler = GradScaler()
opt = torch.optim.AdamW(model.parameters(), lr=3e-4, weight_decay=0.05)
for x, y in loader:
    opt.zero_grad()
    with autocast():
        loss = nn.functional.cross_entropy(model(x.cuda()), y.cuda())
    scaler.scale(loss).backward()
    scaler.step(opt)
    scaler.update()
```

### Inference with TorchScript / compile
```python
model.eval()
model = torch.compile(model, mode="reduce-overhead")  # PyTorch 2.5+
with torch.no_grad():
    out = model(x)
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Small data (<10k images) | Pretrained ResNet-50 + finetune |
| Mobile / edge | MobileNetV4 / EfficientNet-Lite |
| SOTA on ImageNet | ConvNeXt-V2 or hybrid (CNN+ViT) |
| Real-time detection | YOLOv11 (CSPDarknet backbone) |
| Medical seg | U-Net++ or nnU-Net |

**기본값**: 매 timm 의 pretrained ConvNeXt-Tiny — 매 81%+ ImageNet, 매 28M params.

## 🔗 Graph
- 부모: [[Deep Learning]] · [[Neural Networks]]
- 변형: [[ResNet]] · [[EfficientNet]]
- 응용: [[Computer Vision]] · [[Object Detection]] · [[Image Segmentation]]
- Adjacent: [[Transformer_Architecture_and_LLM_Foundations|Attention Mechanisms]]

## 🤖 LLM 활용
**언제**: 매 architecture sketch 의 generation, 매 training-loop boilerplate, 매 hyperparameter starting points, 매 debugging shape mismatches.
**언제 X**: 매 SOTA tuning / benchmark 의 LLM 의존 X — 매 paper + timm 의 reference.

## ❌ 안티패턴
- **Vanilla VGG-style 의 2026 사용**: 매 outdated — 매 ResNet/ConvNeXt 의 사용.
- **No data augmentation**: 매 immediate overfit on small data.
- **BatchNorm with batch size 1**: 매 statistic 무의미 — 매 GroupNorm 사용.
- **Conv 후 immediate ReLU + BN order 의 inconsistent**: 매 BN→Act 의 standard.
- **No mixed precision on modern GPU**: 매 free 2× speedup 의 손실.

## 🧪 검증 / 중복
- Verified (LeCun 1989, He et al. 2015 ResNet, Liu et al. 2022 ConvNeXt, 2024 ConvNeXt-V2).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — CNN fundamentals + ConvNeXt modern patterns |