2nd/10_Wiki/Topics/AI_and_ML/ResNet-Architectures.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-resnet-architectures	ResNet Architectures	10_Wiki/Topics	verified	self	ResNet Residual Networks Skip Connection	none	A	0.9	applied	resnet cnn skip-connection deep-learning vision		2026-05-10	pending	language framework python pytorch

ResNet Architectures

매 한 줄

"매 identity shortcut 의 deep network 의 가능". ResNet (He et al. 2015) 의 skip connection 으로 152-layer training 의 enable, ImageNet 우승. 2026 의 ConvNeXt-V2/Hiera 의 ResNet idea 의 Vision Transformer 의 hybrid.

매 핵심

매 핵심 idea

• Residual block: y = F(x) + x — 매 identity 의 default.
• Vanishing gradient 의 mitigate: gradient 의 skip 의 통해 직접 flow.
• Deeper = better (until 1000+ where diminishing).
• Bottleneck (1x1 → 3x3 → 1x1): ResNet-50+ 의 efficiency.

매 Variants

• ResNet-18 / 34: basic block (no bottleneck).
• ResNet-50 / 101 / 152: bottleneck — 매 default backbone.
• Wide ResNet: wider, shallower.
• ResNeXt: grouped conv (cardinality).
• DenseNet: 매 모든 prev layer 의 concat (vs sum).
• ConvNeXt (2022) / V2 (2023): ResNet 의 ViT-style modernize — depthwise conv, LayerNorm, GELU, 매 ImageNet SOTA 의 ViT 의 match.
• Hiera (Meta, 2023): hierarchical ViT, ResNet 의 spirit.

매 응용

1. Image classification backbone (ImageNet, medical imaging).
2. Object detection (Faster R-CNN, RetinaNet 의 ResNet backbone).
3. Segmentation (U-Net + ResNet encoder).
4. Feature extraction for downstream (CLIP image encoder origin).
5. Diffusion model U-Net (residual 의 everywhere).

💻 패턴

Residual block (PyTorch)

import torch.nn as nn

class BasicBlock(nn.Module):
    expansion = 1
    def __init__(self, in_c, out_c, stride=1):
        super().__init__()
        self.conv1 = nn.Conv2d(in_c, out_c, 3, stride, 1, bias=False)
        self.bn1   = nn.BatchNorm2d(out_c)
        self.conv2 = nn.Conv2d(out_c, out_c, 3, 1, 1, bias=False)
        self.bn2   = nn.BatchNorm2d(out_c)
        self.shortcut = nn.Identity()
        if stride != 1 or in_c != out_c:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_c, out_c, 1, stride, bias=False),
                nn.BatchNorm2d(out_c),
            )
    def forward(self, x):
        out = torch.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        out = out + self.shortcut(x)        # 매 핵심
        return torch.relu(out)

Bottleneck block (ResNet-50+)

class Bottleneck(nn.Module):
    expansion = 4
    def __init__(self, in_c, mid_c, stride=1):
        super().__init__()
        out_c = mid_c * self.expansion
        self.conv1 = nn.Conv2d(in_c, mid_c, 1, bias=False)
        self.bn1   = nn.BatchNorm2d(mid_c)
        self.conv2 = nn.Conv2d(mid_c, mid_c, 3, stride, 1, bias=False)
        self.bn2   = nn.BatchNorm2d(mid_c)
        self.conv3 = nn.Conv2d(mid_c, out_c, 1, bias=False)
        self.bn3   = nn.BatchNorm2d(out_c)
        self.shortcut = nn.Identity()
        if stride != 1 or in_c != out_c:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_c, out_c, 1, stride, bias=False),
                nn.BatchNorm2d(out_c))
    def forward(self, x):
        out = torch.relu(self.bn1(self.conv1(x)))
        out = torch.relu(self.bn2(self.conv2(out)))
        out = self.bn3(self.conv3(out))
        return torch.relu(out + self.shortcut(x))

Pretrained ResNet-50 (torchvision)

import torch
from torchvision.models import resnet50, ResNet50_Weights

weights = ResNet50_Weights.IMAGENET1K_V2
model = resnet50(weights=weights).eval().cuda()
preprocess = weights.transforms()

img = preprocess(load_image("cat.jpg")).unsqueeze(0).cuda()
with torch.no_grad():
    logits = model(img)
print(weights.meta["categories"][logits.argmax(1).item()])

ConvNeXt (2026 modern alt)

from torchvision.models import convnext_base, ConvNeXt_Base_Weights
model = convnext_base(weights=ConvNeXt_Base_Weights.IMAGENET1K_V1)
# 매 ResNet 의 spirit, ViT-grade accuracy

Fine-tune for custom task

model = resnet50(weights=ResNet50_Weights.IMAGENET1K_V2)
for p in model.parameters(): p.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, num_classes)  # train only head

매 결정 기준

상황	Approach
Quick CV baseline	ResNet-50 pretrained
SOTA accuracy	ConvNeXt-V2 / ViT-L / Hiera
Edge / mobile	MobileNetV3 / EfficientNet-Lite
Detection backbone	ResNet-50 + FPN, ConvNeXt for SOTA
Diffusion U-Net	Residual blocks (ResNet-style)

기본값: ResNet-50 의 baseline, ConvNeXt-Base 의 production target.

🔗 Graph

• 부모: CNN · Deep-Learning
• 변형: ConvNeXt · ResNeXt · Wide-ResNet · DenseNet
• 응용: Image-Classification · Object-Detection · Diffusion-Models
• Adjacent: Skip-Connection · Vision-Transformer · Batch-Normalization

🤖 LLM 활용

언제: ResNet implementation 의 explain, paper summarization (He 2015), debugging gradient flow. 언제 X: actual training (use PyTorch + GPU), benchmark numbers (verify on Papers with Code).

❌ 안티패턴

• No skip connection 매 deep: 매 50+ layers 의 vanishing gradient.
• BatchNorm 의 small batch: <16 의 broken — GroupNorm/LayerNorm 의 use.
• Train from scratch 매 small data: 매 pretrain 의 always.
• Skip connection 의 add 의 다른 shape: 매 1x1 conv projection 의 needed.
• ResNet-152 매 mobile: 60M params — MobileNet/EfficientNet 의 use.

🧪 검증 / 중복

• Verified (He et al. 2015 ResNet, Liu et al. 2022 ConvNeXt, torchvision docs).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — ResNet 매 ConvNeXt revival 의 connect