docs(10_Wiki): 위키 전체 재구성 — Topic_* 폴더를 4개 카테고리로 통합 + 대규모 중복 제거
Topic_Agent/Topic_Blog/Topics/Topics_Biz/Topics_Meeting/Topics_Rag의 마크다운 지식 문서를 Topic_General/Topic_Programming/Topic_Graphic/Topic_Business 4개 카테고리로 재분류. - 중복 제거: frontmatter의 status:duplicate/merged + duplicate_of/redirect_to 필드로 자기 자신을 중복으로 선언한 리다이렉트 stub 1032개 제거, 완전 동일 내용 파일 472개 제거, 동일 파일명·다른 내용 충돌 시 더 큰(완전한) 버전만 유지(162개 제거) — 총 1639개 중복 제거. - 분류: 폴더 단위로 명확한 항목(AI_and_ML/Coding/Architecture 등 → Programming, Comfyui/Visual_Effects → Graphic, Topics_Biz/Topics_Meeting/사업 등 → Business, Poetic_Blog_Writing/창의성/Game_Design 등 → General)은 폴더 우선순위로, 나머지 혼재 폴더(Topic_Agent/Topic_Blog/Topics 루트/Thinking & Reasoning/Other/UI_UX_Assets)는 title/tags 키워드 스코어링으로 파일 단위 분류(불명확한 경우 General로 폴백). 원본 폴더명은 "From_*" 서브폴더로 보존해 추적 가능성 유지. - 최종 배치: Programming 2784 / General 1608 / Graphic 285 / Business 249 = 4926개 문서. - 에이전트 운영 상태(.astra/.agent/.obsidian/sessions/memory/_company/docs/lessons/_shared/src)는 지식 콘텐츠가 아니므로 재분류 대상에서 제외하고 원위치 유지. - Topics/Topic_email(상위 보호 폴더 Topic_email과 파일명 100% 중복) 삭제 — 보호 폴더 자체는 미변경. - 완전히 비게 된 Topic_Agent/Topic_Blog/Topics_Biz/Topics_Rag 폴더 제거.
This commit is contained in:
Antigravity Agent
@@ -0,0 +1,290 @@
|
||||
---
|
||||
id: wiki-2026-0508-cv-synthesis
|
||||
title: Computer Vision Synthesis (Synthetic Data)
|
||||
category: 10_Wiki/Topics
|
||||
status: verified
|
||||
canonical_id: self
|
||||
aliases: [synthetic data, sim2real, domain adaptation, NVIDIA Omniverse, Unity Perception, model collapse]
|
||||
duplicate_of: none
|
||||
source_trust_level: A
|
||||
confidence_score: 0.9
|
||||
verification_status: applied
|
||||
tags: [computer-vision, synthetic-data, sim2real, domain-adaptation, omniverse, unity, autonomous-driving, robotics]
|
||||
raw_sources: []
|
||||
last_reinforced: 2026-05-10
|
||||
github_commit: pending
|
||||
tech_stack:
|
||||
language: Python
|
||||
framework: NVIDIA Omniverse / Unity Perception / Blender / Diffusers
|
||||
---
|
||||
|
||||
# CV Synthesis (Synthetic Data)
|
||||
|
||||
## 📌 한 줄 통찰
|
||||
> **"매 가상 의 image 의 학습 data"**. 매 perfect ground truth + 매 rare event + 매 privacy. 매 sim-to-real domain gap 의 핵심 challenge. 매 modern: 매 generative AI (Stable Diffusion) + 매 photoreal sim (Omniverse, Gaussian Splatting). 매 model collapse 의 risk.
|
||||
|
||||
## 📖 핵심
|
||||
|
||||
### 매 motivation
|
||||
1. **Perfect ground truth**: 매 pixel mask, 매 3D position, 매 depth, 매 segmentation 의 free.
|
||||
2. **Rare event**: 매 accident, 매 edge case 의 endless.
|
||||
3. **Privacy**: 매 face / plate 의 X.
|
||||
4. **Cost**: 매 real annotation $$$ 의 X.
|
||||
5. **Coverage**: 매 lighting / weather / pose 의 systematic.
|
||||
|
||||
### 매 source
|
||||
- **Game engine** (Unity, Unreal): 매 photoreal.
|
||||
- **Synthetic 3D pipeline** (Blender, Houdini).
|
||||
- **NVIDIA Omniverse**: 매 industrial digital twin.
|
||||
- **Diffusion model**: 매 prompt-based.
|
||||
- **GAN**: 매 specific domain (face, etc).
|
||||
- **Procedural generation**: 매 controllable.
|
||||
|
||||
### 매 응용
|
||||
1. **Autonomous driving**: 매 CARLA, 매 Waymo Carcraft.
|
||||
2. **Robotics**: 매 Isaac Sim, 매 sim2real.
|
||||
3. **Surveillance**: 매 person re-id.
|
||||
4. **Medical**: 매 augment rare condition.
|
||||
5. **Aerial**: 매 drone training.
|
||||
6. **Manufacturing**: 매 defect detection.
|
||||
7. **Retail**: 매 product variation.
|
||||
|
||||
### Sim-to-Real domain gap
|
||||
- **Visual gap**: 매 lighting / texture / shadow.
|
||||
- **Distribution gap**: 매 prevalence.
|
||||
- **Causal gap**: 매 dynamics / physics.
|
||||
|
||||
### 매 mitigation
|
||||
- **Domain randomization**: 매 random texture / lighting.
|
||||
- **Domain adaptation**: 매 GAN / CycleGAN.
|
||||
- **Photorealism push**: 매 ray tracing, Gaussian splatting.
|
||||
- **Hybrid training**: 매 real + synthetic.
|
||||
- **Self-supervised**: 매 unlabeled real.
|
||||
|
||||
### 매 Model Collapse (modern concern)
|
||||
- 매 synthetic data 만 의 train → 매 real distribution drift.
|
||||
- 매 generation 의 amplify own bias.
|
||||
- 매 mitigation: 매 fresh real 의 mix.
|
||||
- → Shumailov et al. 2024.
|
||||
|
||||
### 매 platform
|
||||
- **NVIDIA Omniverse / Replicator**: 매 enterprise.
|
||||
- **Unity Perception SDK**: 매 game-engine.
|
||||
- **Unreal MetaHuman**: 매 photoreal humans.
|
||||
- **Mitsuba 3**: 매 differentiable rendering.
|
||||
- **Kubric** (Google): 매 video synthesis.
|
||||
- **Habitat / iGibson**: 매 robot indoor.
|
||||
- **CARLA / AirSim**: 매 driving / drone.
|
||||
|
||||
### 매 generative augmentation
|
||||
- **Stable Diffusion**: 매 prompt-driven.
|
||||
- **ControlNet**: 매 layout-controlled.
|
||||
- **DreamBooth + LoRA**: 매 specific class.
|
||||
- **Inpainting**: 매 selective augment.
|
||||
|
||||
## 💻 패턴
|
||||
|
||||
### Unity Perception (label config)
|
||||
```csharp
|
||||
// 매 Unity Perception SDK
|
||||
using UnityEngine.Perception.GroundTruth;
|
||||
|
||||
[CreateAssetMenu]
|
||||
public class ProductLabelConfig : IdLabelConfig {}
|
||||
|
||||
// 매 Perception camera 의 attach
|
||||
// 매 RGB + bounding box + segmentation + depth 의 auto.
|
||||
```
|
||||
|
||||
### Domain randomization (Python)
|
||||
```python
|
||||
import random
|
||||
import bpy # 매 Blender API
|
||||
|
||||
def randomize_scene():
|
||||
# 매 light
|
||||
light = bpy.data.objects['Light']
|
||||
light.data.energy = random.uniform(500, 2000)
|
||||
light.location = (random.uniform(-5, 5), random.uniform(-5, 5), random.uniform(3, 10))
|
||||
|
||||
# 매 camera angle
|
||||
cam = bpy.data.objects['Camera']
|
||||
cam.rotation_euler = (random.uniform(0, 0.5), random.uniform(0, 0.5), random.uniform(0, 6.28))
|
||||
|
||||
# 매 background HDR
|
||||
world = bpy.data.worlds['World']
|
||||
hdri = random.choice(['hdri/sunset.exr', 'hdri/cloudy.exr', 'hdri/night.exr'])
|
||||
world.node_tree.nodes['Environment Texture'].image = bpy.data.images.load(hdri)
|
||||
|
||||
# 매 material color
|
||||
for obj in bpy.context.scene.objects:
|
||||
if obj.data and obj.data.materials:
|
||||
obj.data.materials[0].diffuse_color = (
|
||||
random.random(), random.random(), random.random(), 1
|
||||
)
|
||||
|
||||
# 매 1000 frame 의 render
|
||||
for i in range(1000):
|
||||
randomize_scene()
|
||||
bpy.context.scene.render.filepath = f'./synthetic/img_{i:04d}.png'
|
||||
bpy.ops.render.render(write_still=True)
|
||||
```
|
||||
|
||||
### NVIDIA Omniverse Replicator (Python)
|
||||
```python
|
||||
import omni.replicator.core as rep
|
||||
|
||||
with rep.new_layer():
|
||||
camera = rep.create.camera(position=(0, 0, 1000))
|
||||
light = rep.create.light(rotation=(-90, 0, 0), light_type='distant')
|
||||
|
||||
# 매 distractor objects
|
||||
distractors = rep.create.cube(count=20)
|
||||
|
||||
# 매 target object
|
||||
target = rep.create.sphere()
|
||||
|
||||
with rep.trigger.on_frame(num_frames=1000):
|
||||
with target:
|
||||
rep.modify.pose(
|
||||
position=rep.distribution.uniform((-200, -200, 0), (200, 200, 200)),
|
||||
rotation=rep.distribution.uniform((0, 0, 0), (360, 360, 360)),
|
||||
)
|
||||
with distractors:
|
||||
rep.randomizer.scatter_2d()
|
||||
with light:
|
||||
rep.modify.attribute('intensity', rep.distribution.uniform(500, 5000))
|
||||
|
||||
# 매 writer (RGB + bbox + mask)
|
||||
writer = rep.WriterRegistry.get('BasicWriter')
|
||||
writer.initialize(output_dir='./synthetic/', rgb=True, bbox=True, mask=True)
|
||||
writer.attach([rep.create.render_product(camera, (1024, 1024))])
|
||||
```
|
||||
|
||||
### CARLA driving sim
|
||||
```python
|
||||
import carla
|
||||
|
||||
client = carla.Client('localhost', 2000)
|
||||
world = client.get_world()
|
||||
|
||||
# 매 spawn vehicle + camera
|
||||
blueprint_library = world.get_blueprint_library()
|
||||
vehicle_bp = blueprint_library.filter('vehicle.tesla.model3')[0]
|
||||
spawn_point = world.get_map().get_spawn_points()[0]
|
||||
vehicle = world.spawn_actor(vehicle_bp, spawn_point)
|
||||
|
||||
camera_bp = blueprint_library.find('sensor.camera.rgb')
|
||||
camera_bp.set_attribute('image_size_x', '800')
|
||||
camera = world.spawn_actor(camera_bp, carla.Transform(carla.Location(x=2.5, z=1.0)), attach_to=vehicle)
|
||||
|
||||
# 매 listen
|
||||
camera.listen(lambda image: image.save_to_disk(f'./out/{image.frame:08d}.png'))
|
||||
|
||||
# 매 weather randomization
|
||||
weather = carla.WeatherParameters(cloudiness=80, precipitation=30, sun_altitude_angle=45)
|
||||
world.set_weather(weather)
|
||||
```
|
||||
|
||||
### Diffusion-based augmentation
|
||||
```python
|
||||
from diffusers import StableDiffusionInpaintPipeline
|
||||
import torch
|
||||
|
||||
pipe = StableDiffusionInpaintPipeline.from_pretrained(
|
||||
'runwayml/stable-diffusion-inpainting', torch_dtype=torch.float16,
|
||||
).to('cuda')
|
||||
|
||||
# 매 existing image + 매 mask → 매 inpaint with new variation
|
||||
def augment_with_inpaint(image, mask, prompts):
|
||||
augmented = []
|
||||
for p in prompts:
|
||||
result = pipe(
|
||||
prompt=p,
|
||||
image=image,
|
||||
mask_image=mask,
|
||||
num_inference_steps=30,
|
||||
guidance_scale=7.5,
|
||||
).images[0]
|
||||
augmented.append(result)
|
||||
return augmented
|
||||
|
||||
# 매 e.g., medical scan 의 condition variation
|
||||
prompts = ['a benign tumor', 'a malignant tumor stage 1', '...']
|
||||
```
|
||||
|
||||
### Sim2Real domain adaptation (CycleGAN)
|
||||
```python
|
||||
# 매 sim → real domain
|
||||
from torch_cyclegan import CycleGAN
|
||||
|
||||
cyclegan = CycleGAN(
|
||||
generator_S2R=Generator(),
|
||||
generator_R2S=Generator(),
|
||||
discriminator_R=Discriminator(),
|
||||
discriminator_S=Discriminator(),
|
||||
)
|
||||
|
||||
# 매 sim image 의 real-style transfer
|
||||
real_styled = cyclegan.S2R(sim_image)
|
||||
# 매 train downstream model on (sim_label, real_styled).
|
||||
```
|
||||
|
||||
### Mix ratio (model collapse mitigation)
|
||||
```python
|
||||
def adaptive_mix(epoch, real_data, synthetic_data):
|
||||
"""매 early: synthetic 의 lots, late: real 의 emphasize."""
|
||||
real_ratio = min(0.7, 0.2 + epoch * 0.05)
|
||||
n_real = int(BATCH_SIZE * real_ratio)
|
||||
n_synth = BATCH_SIZE - n_real
|
||||
|
||||
return DataLoader(
|
||||
ConcatDataset([
|
||||
Subset(real_data, random.sample(range(len(real_data)), n_real)),
|
||||
Subset(synthetic_data, random.sample(range(len(synthetic_data)), n_synth)),
|
||||
]),
|
||||
batch_size=BATCH_SIZE,
|
||||
)
|
||||
```
|
||||
|
||||
## 🤔 결정 기준
|
||||
| 상황 | Tool |
|
||||
|---|---|
|
||||
| Driving | CARLA / Waymo Carcraft |
|
||||
| Robot indoor | Habitat / iGibson |
|
||||
| Industrial | NVIDIA Omniverse |
|
||||
| Bbox / segmentation | Unity Perception |
|
||||
| Photoreal humans | MetaHuman + Unreal |
|
||||
| Augmentation | Stable Diffusion + ControlNet |
|
||||
| Domain gap | CycleGAN / domain randomization |
|
||||
| Tabletop | Blender + scripted |
|
||||
|
||||
**기본값**: 매 photoreal + 매 domain randomization + 매 mix with real.
|
||||
|
||||
## 🔗 Graph
|
||||
- 부모: [[Computer Vision|Computer-Vision]] · [[Synthetic-Data]] · [[Simulation]]
|
||||
- 변형: [[Sim2Real]] · [[Domain-Adaptation]] · [[CycleGAN]]
|
||||
- 응용: [[Autonomous Vehicles]] · [[Robotics]] · [[Unity-Perception]]
|
||||
- Adjacent: [[Diffusion-Models]] · [[ControlNet]] · [[Model-Collapse]] · [[Algorithmic-Biology]]
|
||||
|
||||
## 🤖 LLM 활용
|
||||
**언제**: 매 data scarcity. 매 rare event. 매 privacy-sensitive. 매 cost reduction. 매 systematic coverage.
|
||||
**언제 X**: 매 real data abundant + cheap. 매 high domain-gap not addressed.
|
||||
|
||||
## ❌ 안티패턴
|
||||
- **Synthetic 만 의 train**: 매 model collapse + sim2real gap.
|
||||
- **Single environment**: 매 over-fit.
|
||||
- **No domain randomization**: 매 unrealistic 학습.
|
||||
- **Generative 의 cycle (synth → train → gen → train)**: 매 collapse.
|
||||
- **No real validation**: 매 fake metric.
|
||||
|
||||
## 🧪 검증 / 중복
|
||||
- Verified (Tobin domain randomization 2017, Tremblay 2018, NVIDIA Omniverse).
|
||||
- 신뢰도 A.
|
||||
- Related: [[Autonomous Vehicles]] · [[Diffusion-Models]] · [[Robotics]] · [[Algorithmic-Biology]] · [[Model-Collapse]].
|
||||
|
||||
## 🕓 Changelog
|
||||
| 날짜 | 변경 |
|
||||
|---|---|
|
||||
| 2026-05-08 | Phase 1 |
|
||||
| 2026-05-10 | Manual cleanup — sim2real + domain randomization + 매 Unity / Omniverse / CARLA / SD / CycleGAN code |
|
||||
Reference in New Issue
Block a user