2nd/10_Wiki/Topics/AI_and_ML/Robotics.md

---
id: wiki-2026-0508-robotics
title: Robotics
category: 10_Wiki/Topics
status: verified
canonical_id: self
aliases: [Robotic Systems, Robot Engineering, Robotics Foundations]
duplicate_of: none
source_trust_level: A
confidence_score: 0.95
verification_status: applied
tags: [robotics, perception, planning, control, foundation-models, embodied-AI]
raw_sources: []
last_reinforced: 2026-05-10
github_commit: pending
tech_stack:
  language: Python/C++
  framework: ROS2/Isaac/MuJoCo/PyTorch
---

# Robotics

## 매 한 줄
> **"매 sense → think → act 의 closed loop 의 physical world 의 agent."**. Robotics 의 perception (vision/lidar/IMU), planning (motion + task), control (PID → MPC → RL) 의 stack 의, 매 2024-2026 inflection 의 vision-language-action (VLA) foundation models — Google RT-2, Physical Intelligence π0, Figure 02 — 의 generalist embodied AI 의 era 의 driving.

## 매 핵심

### 매 classical stack
1. **Perception**: SLAM (ORB-SLAM3, Kimera), object detection, pose estimation.
2. **State estimation**: Kalman filter, factor graphs (GTSAM).
3. **Planning**:
   - Task: PDDL, behavior trees, LLM planners.
   - Motion: RRT*, trajectory optimization (TrajOpt, CHOMP).
4. **Control**: PID, LQR, MPC, impedance control.
5. **Hardware abstraction**: ROS2, drivers, real-time scheduling.

### 매 modern (2026) shift
- **VLA models**: π0, RT-2, OpenVLA — image+text→action tokens.
- **Diffusion policies**: Chi et al. 2023 — multimodal action distributions.
- **Sim2Real**: Isaac Lab, MuJoCo MJX — massive parallel simulation + DR.
- **Humanoids**: Figure 02, Tesla Optimus, Apptronik Apollo, 1X Neo — commercial pilots.
- **Whole-body MPC**: real-time on humanoid (Boston Dynamics Atlas).

### 매 응용
1. Industrial automation (pick-and-place, welding).
2. Autonomous mobile robots (Amazon Proteus, warehouse AGVs).
3. Surgical robotics (da Vinci, Intuitive Ion).
4. Humanoid general purpose (Figure, Optimus, BMW pilot 2025).
5. Self-driving (Waymo, Tesla FSD).

## 💻 패턴

### ROS2 Node Skeleton
```python
import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from geometry_msgs.msg import Twist

class Controller(Node):
    def __init__(self):
        super().__init__("controller")
        self.sub = self.create_subscription(Image, "/cam", self.cb_img, 10)
        self.pub = self.create_publisher(Twist, "/cmd_vel", 10)

    def cb_img(self, msg):
        cmd = Twist()
        cmd.linear.x = 0.5
        self.pub.publish(cmd)

def main(): rclpy.init(); rclpy.spin(Controller()); rclpy.shutdown()
```

### Extended Kalman Filter (state estimation)
```python
import numpy as np

class EKF:
    def __init__(self, x0, P0, Q, R):
        self.x, self.P, self.Q, self.R = x0, P0, Q, R

    def predict(self, f, F):
        self.x = f(self.x)
        self.P = F @ self.P @ F.T + self.Q

    def update(self, z, h, H):
        y = z - h(self.x)
        S = H @ self.P @ H.T + self.R
        K = self.P @ H.T @ np.linalg.inv(S)
        self.x = self.x + K @ y
        self.P = (np.eye(len(self.x)) - K @ H) @ self.P
```

### MPC (acados / cvxpy)
```python
import cvxpy as cp
import numpy as np

def mpc_step(x0, A, B, Q, R, N=10, u_max=1.0):
    nx, nu = A.shape[0], B.shape[1]
    x = cp.Variable((nx, N+1))
    u = cp.Variable((nu, N))
    cost = sum(cp.quad_form(x[:,k], Q) + cp.quad_form(u[:,k], R) for k in range(N))
    cons = [x[:,0] == x0]
    for k in range(N):
        cons += [x[:,k+1] == A @ x[:,k] + B @ u[:,k],
                 cp.norm_inf(u[:,k]) <= u_max]
    cp.Problem(cp.Minimize(cost), cons).solve()
    return u.value[:,0]
```

### RRT* Motion Planning
```python
def rrt_star(start, goal, obstacles, max_iter=2000, step=0.5, radius=2.0):
    tree = {tuple(start): {"parent": None, "cost": 0.0}}
    for _ in range(max_iter):
        rnd = sample_free(obstacles)
        nearest = min(tree, key=lambda n: dist(n, rnd))
        new = steer(nearest, rnd, step)
        if collision_free(nearest, new, obstacles):
            neighbors = [n for n in tree if dist(n, new) < radius]
            best = min(neighbors, key=lambda n: tree[n]["cost"] + dist(n, new))
            tree[tuple(new)] = {"parent": best, "cost": tree[best]["cost"] + dist(best, new)}
            if dist(new, goal) < step: return reconstruct_path(tree, tuple(new))
    return None
```

### Diffusion Policy (modern)
```python
import torch
import torch.nn as nn

class DiffusionPolicy(nn.Module):
    def __init__(self, obs_dim, action_dim, T=100):
        super().__init__()
        self.net = nn.Sequential(nn.Linear(obs_dim+action_dim+1, 256),
                                 nn.SiLU(),
                                 nn.Linear(256, action_dim))
        self.T = T

    def forward(self, obs, a_t, t):
        return self.net(torch.cat([obs, a_t, t.float().unsqueeze(-1)/self.T], -1))

    @torch.no_grad()
    def sample(self, obs, scheduler):
        a = torch.randn(obs.shape[0], self.action_dim, device=obs.device)
        for t in reversed(range(self.T)):
            eps = self.forward(obs, a, torch.full((obs.shape[0],), t, device=obs.device))
            a = scheduler.step(eps, t, a)
        return a
```

### VLA Inference (π0-style)
```python
def vla_act(model, image, instruction: str):
    """매 VLA: image + text → action chunk."""
    inputs = model.processor(image=image, text=instruction, return_tensors="pt")
    out = model.generate(**inputs, max_new_tokens=64)
    action_chunk = model.detokenize_actions(out)   # e.g. (8, 7) — 8 timesteps × 7 DoF
    return action_chunk
```

### Sim2Real Domain Randomization
```python
def randomize_env(env):
    env.set_friction(np.random.uniform(0.5, 1.5))
    env.set_mass_scale(np.random.uniform(0.8, 1.2))
    env.set_lighting(np.random.uniform(0.3, 1.0))
    env.add_observation_noise(scale=np.random.uniform(0.0, 0.05))
```

## 매 결정 기준
| 상황 | Approach |
|---|---|
| Industrial repetitive | Classical: vision + IK + PID |
| Mobile navigation | SLAM + planning (Nav2) + MPC |
| Manipulation, varied tasks | Diffusion policy or VLA |
| Generalist humanoid | VLA (π0/RT-2) + whole-body MPC |
| Safety-critical (surgery, AV) | Verified controllers + redundant perception |

**기본값**: Classical stack for known tasks; VLA + diffusion policy 의 manipulation/general; Sim2Real with massive DR for new skills.

## 🔗 Graph
- 부모: [[Embodied AI]] · [[Control Theory]] · [[Computer Vision]]
- 변형: [[Surgical-Robotics]]
- 응용: [[Self-Driving]]
- Adjacent: [[Reinforcement Learning]] · [[SLAM]] · [[MPC]]

## 🤖 LLM 활용
**언제**: high-level task planning (LLM-as-planner), code generation for robot skills, error diagnosis, sim asset generation.
**언제 X**: 매 closed-loop control 의 X — latency/safety 의 LLM 의 unsuitable; small specialized policy 의 fast inference 의 use.

## ❌ 안티패턴
- **Open-loop trust**: sensor noise 의 underestimate — 매 always close the loop.
- **Sim-only training**: no DR / no real fine-tune → fails reality.
- **VLA for everything**: 매 simple reach-and-place 의 IK 의 better — VLA 의 long-tail 의 reserve.
- **No safety layer**: ML policy 의 unbounded — torque limits + collision shield 필요.
- **Real-time on Python only**: critical loop 의 C++/Rust + ROS2 real-time scheduler.

## 🧪 검증 / 중복
- Verified (Siciliano "Robotics: Modelling, Planning and Control"; π0/RT-2 papers; Boston Dynamics Atlas reports 2024-25).
- 신뢰도 A.

## 🕓 Changelog
| 날짜 | 변경 |
|---|---|
| 2026-05-08 | Phase 1 |
| 2026-05-10 | Manual cleanup — robotics canonical: classical + VLA + diffusion policy + Sim2Real |