2nd/10_Wiki/Topics/AI_and_ML/Robotics.md

id, title, category, status, canonical_id, aliases, duplicate_of, source_trust_level, confidence_score, verification_status, tags, raw_sources, last_reinforced, github_commit, tech_stack

id	title	category	status	canonical_id	aliases	duplicate_of	source_trust_level	confidence_score	verification_status	tags	raw_sources	last_reinforced	github_commit	tech_stack
wiki-2026-0508-robotics	Robotics	10_Wiki/Topics	verified	self	Robotic Systems Robot Engineering Robotics Foundations	none	A	0.95	applied	robotics perception planning control foundation-models embodied-AI		2026-05-10	pending	language framework Python/C++ ROS2/Isaac/MuJoCo/PyTorch

Robotics

매 한 줄

"매 sense → think → act 의 closed loop 의 physical world 의 agent.". Robotics 의 perception (vision/lidar/IMU), planning (motion + task), control (PID → MPC → RL) 의 stack 의, 매 2024-2026 inflection 의 vision-language-action (VLA) foundation models — Google RT-2, Physical Intelligence π0, Figure 02 — 의 generalist embodied AI 의 era 의 driving.

매 핵심

매 classical stack

1. Perception: SLAM (ORB-SLAM3, Kimera), object detection, pose estimation.
2. State estimation: Kalman filter, factor graphs (GTSAM).
3. Planning:
   • Task: PDDL, behavior trees, LLM planners.
   • Motion: RRT*, trajectory optimization (TrajOpt, CHOMP).
4. Control: PID, LQR, MPC, impedance control.
5. Hardware abstraction: ROS2, drivers, real-time scheduling.

매 modern (2026) shift

• VLA models: π0, RT-2, OpenVLA — image+text→action tokens.
• Diffusion policies: Chi et al. 2023 — multimodal action distributions.
• Sim2Real: Isaac Lab, MuJoCo MJX — massive parallel simulation + DR.
• Humanoids: Figure 02, Tesla Optimus, Apptronik Apollo, 1X Neo — commercial pilots.
• Whole-body MPC: real-time on humanoid (Boston Dynamics Atlas).

매 응용

1. Industrial automation (pick-and-place, welding).
2. Autonomous mobile robots (Amazon Proteus, warehouse AGVs).
3. Surgical robotics (da Vinci, Intuitive Ion).
4. Humanoid general purpose (Figure, Optimus, BMW pilot 2025).
5. Self-driving (Waymo, Tesla FSD).

💻 패턴

ROS2 Node Skeleton

import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from geometry_msgs.msg import Twist

class Controller(Node):
    def __init__(self):
        super().__init__("controller")
        self.sub = self.create_subscription(Image, "/cam", self.cb_img, 10)
        self.pub = self.create_publisher(Twist, "/cmd_vel", 10)

    def cb_img(self, msg):
        cmd = Twist()
        cmd.linear.x = 0.5
        self.pub.publish(cmd)

def main(): rclpy.init(); rclpy.spin(Controller()); rclpy.shutdown()

Extended Kalman Filter (state estimation)

import numpy as np

class EKF:
    def __init__(self, x0, P0, Q, R):
        self.x, self.P, self.Q, self.R = x0, P0, Q, R

    def predict(self, f, F):
        self.x = f(self.x)
        self.P = F @ self.P @ F.T + self.Q

    def update(self, z, h, H):
        y = z - h(self.x)
        S = H @ self.P @ H.T + self.R
        K = self.P @ H.T @ np.linalg.inv(S)
        self.x = self.x + K @ y
        self.P = (np.eye(len(self.x)) - K @ H) @ self.P

MPC (acados / cvxpy)

import cvxpy as cp
import numpy as np

def mpc_step(x0, A, B, Q, R, N=10, u_max=1.0):
    nx, nu = A.shape[0], B.shape[1]
    x = cp.Variable((nx, N+1))
    u = cp.Variable((nu, N))
    cost = sum(cp.quad_form(x[:,k], Q) + cp.quad_form(u[:,k], R) for k in range(N))
    cons = [x[:,0] == x0]
    for k in range(N):
        cons += [x[:,k+1] == A @ x[:,k] + B @ u[:,k],
                 cp.norm_inf(u[:,k]) <= u_max]
    cp.Problem(cp.Minimize(cost), cons).solve()
    return u.value[:,0]

RRT* Motion Planning

def rrt_star(start, goal, obstacles, max_iter=2000, step=0.5, radius=2.0):
    tree = {tuple(start): {"parent": None, "cost": 0.0}}
    for _ in range(max_iter):
        rnd = sample_free(obstacles)
        nearest = min(tree, key=lambda n: dist(n, rnd))
        new = steer(nearest, rnd, step)
        if collision_free(nearest, new, obstacles):
            neighbors = [n for n in tree if dist(n, new) < radius]
            best = min(neighbors, key=lambda n: tree[n]["cost"] + dist(n, new))
            tree[tuple(new)] = {"parent": best, "cost": tree[best]["cost"] + dist(best, new)}
            if dist(new, goal) < step: return reconstruct_path(tree, tuple(new))
    return None

Diffusion Policy (modern)

import torch
import torch.nn as nn

class DiffusionPolicy(nn.Module):
    def __init__(self, obs_dim, action_dim, T=100):
        super().__init__()
        self.net = nn.Sequential(nn.Linear(obs_dim+action_dim+1, 256),
                                 nn.SiLU(),
                                 nn.Linear(256, action_dim))
        self.T = T

    def forward(self, obs, a_t, t):
        return self.net(torch.cat([obs, a_t, t.float().unsqueeze(-1)/self.T], -1))

    @torch.no_grad()
    def sample(self, obs, scheduler):
        a = torch.randn(obs.shape[0], self.action_dim, device=obs.device)
        for t in reversed(range(self.T)):
            eps = self.forward(obs, a, torch.full((obs.shape[0],), t, device=obs.device))
            a = scheduler.step(eps, t, a)
        return a

VLA Inference (π0-style)

def vla_act(model, image, instruction: str):
    """매 VLA: image + text → action chunk."""
    inputs = model.processor(image=image, text=instruction, return_tensors="pt")
    out = model.generate(**inputs, max_new_tokens=64)
    action_chunk = model.detokenize_actions(out)   # e.g. (8, 7) — 8 timesteps × 7 DoF
    return action_chunk

Sim2Real Domain Randomization

def randomize_env(env):
    env.set_friction(np.random.uniform(0.5, 1.5))
    env.set_mass_scale(np.random.uniform(0.8, 1.2))
    env.set_lighting(np.random.uniform(0.3, 1.0))
    env.add_observation_noise(scale=np.random.uniform(0.0, 0.05))

매 결정 기준

상황	Approach
Industrial repetitive	Classical: vision + IK + PID
Mobile navigation	SLAM + planning (Nav2) + MPC
Manipulation, varied tasks	Diffusion policy or VLA
Generalist humanoid	VLA (π0/RT-2) + whole-body MPC
Safety-critical (surgery, AV)	Verified controllers + redundant perception

기본값: Classical stack for known tasks; VLA + diffusion policy 의 manipulation/general; Sim2Real with massive DR for new skills.

🔗 Graph

• 부모: Embodied AI · Control Theory · Computer Vision
• 변형: Surgical-Robotics
• 응용: Self-Driving
• Adjacent: Reinforcement Learning · SLAM · MPC

🤖 LLM 활용

언제: high-level task planning (LLM-as-planner), code generation for robot skills, error diagnosis, sim asset generation. 언제 X: 매 closed-loop control 의 X — latency/safety 의 LLM 의 unsuitable; small specialized policy 의 fast inference 의 use.

❌ 안티패턴

• Open-loop trust: sensor noise 의 underestimate — 매 always close the loop.
• Sim-only training: no DR / no real fine-tune → fails reality.
• VLA for everything: 매 simple reach-and-place 의 IK 의 better — VLA 의 long-tail 의 reserve.
• No safety layer: ML policy 의 unbounded — torque limits + collision shield 필요.
• Real-time on Python only: critical loop 의 C++/Rust + ROS2 real-time scheduler.

🧪 검증 / 중복

• Verified (Siciliano "Robotics: Modelling, Planning and Control"; π0/RT-2 papers; Boston Dynamics Atlas reports 2024-25).
• 신뢰도 A.

🕓 Changelog

날짜	변경
2026-05-08	Phase 1
2026-05-10	Manual cleanup — robotics canonical: classical + VLA + diffusion policy + Sim2Real