Full-Stack Physical AI
From perception to action — vision-language and vision-language-action models, on real robots, with a deterministic safety boundary.
Perception
Policy (VLA)
Control
Safety monitor
A four-stage pipeline: perception, then the policy (a vision-language-action model), then control, then an independent safety monitor.
A robot that works in a demo isn't one you can deploy. We take perception-to-action systems from feasibility to production — choosing the right model (VLM vs VLA), validating in simulation, and constraining every action with an independent safety boundary so a model's mistake can't become an unsafe motion.
The sensor-to-action pipeline (perception → policy → control), the VLM-vs-VLA split (a VLM perceives and reasons; a VLA outputs actions), a training and evaluation loop with simulation and sim-to-real, and a deterministic safety boundary — the policy proposes, an independent monitor constrains.
ROS 2; vision-language-action policies (π-family, SmolVLA, NVIDIA Isaac GR00T) and VLMs; LeRobot for data, training and evaluation; teleoperation for demonstrations; simulation (Isaac Sim/Lab) and sim-to-real; fine-tuning (LoRA/QLoRA) and closed-loop evaluation; runtime safety monitor, operating envelope and emergency stop.
Learned models propose; a deterministic boundary disposes. Every motor command passes an independent safety monitor before it reaches hardware.
Flow from sensors through perception, VLM and VLA, planner and controller, to an independent safety monitor that gates the actuators, with feedback to the sensors.
Deterministic safety boundary
Only the safety monitor's ACCEPT lets a command reach the actuators; REJECT overrides to a safe state.
Closed loop: actuator and environment state feed back to the sensors.
Two model classes with different jobs. Only the VLA's output reaches motors — so only the VLA's output must pass the deterministic safety boundary.
| Dimension | VLM — Vision-Language Model | VLA — Vision-Language-Action |
|---|---|---|
| Output | Text / structured language | Robot actions / action chunks |
| Question it answers | "What am I looking at?" | "What should the joints do now?" |
| Role | High-level perception, reasoning, grounding | Closes the loop on hardware |
| Typical rate | ~1 Hz / on-demand | ~10–50 Hz |
| Trained on | Image-text pairs | Teleoperation demos (+ Open X-Embodiment) |
| Failure mode | Hallucinated plan | Unsafe motion — mandates a safety monitor |
68.71 GB
1.08 → 0.13 (300 steps)
0% → 96%
12 / 12
2.3× – 14.3×