From perception to action — vision-language and vision-language-action models, on real robots, with a deterministic safety boundary.
Hyperion combines product leadership with scoped model engineering and cybersecurity work. A technical mission defines the configuration, adaptation, integration and handover to be delivered; specialist testing and independent qualification are agreed explicitly.
Perception
Policy (VLA)
Control
Safety monitor
From sensors to motion, behind an independent safety monitor.
A four-stage pipeline: perception, then the policy (a vision-language-action model), then control, then an independent safety monitor.
A robot that works in a demo is not one you can deploy. The consequential choices — VLM or VLA, what a simulation is allowed to prove, where the deterministic safety boundary sits — are product decisions before they are engineering ones. Hyperion owns them, names the evidence that closes each one, and coordinates the specialists who implement the result.
The sensor-to-action pipeline (perception → policy → control), the VLM-vs-VLA split (a VLM perceives and reasons; a VLA outputs actions), a training and evaluation loop with simulation and sim-to-real, and a deterministic safety boundary — the policy proposes, an independent monitor constrains.
ROS 2; vision-language-action policies (π-family, SmolVLA, NVIDIA Isaac GR00T) and VLMs; LeRobot for data, training and evaluation; teleoperation for demonstrations; simulation (Isaac Sim/Lab) and sim-to-real; fine-tuning (LoRA/QLoRA) and closed-loop evaluation; runtime safety monitor, operating envelope and emergency stop.
Perception to action, with a boundary that fails safe
Learned models propose; a deterministic boundary disposes. Every motor command passes an independent safety monitor before it reaches hardware.
Input
Deterministic
Learned
Safety
Output
Flow from sensors through perception, VLM and VLA, planner and controller, to an independent safety monitor that gates the actuators, with feedback to the sensors.
LearnedVLMGrounds the goal — "what am I looking at?"
LearnedVLAGenerates action chunks on hardware
DeterministicPlanner & ControlTrajectory and ~1 kHz control
SafetySafety monitorIndependent, out-of-band envelope and e-stop
OutputActuatorsMotion in the physical world
Deterministic safety boundary
Only the safety monitor's ACCEPT lets a command reach the actuators; REJECT overrides to a safe state.
⤿Closed loop: actuator and environment state feed back to the sensors.
Illustrative control-loop budgets: reflex/safety ~1 kHz · perception ~10–40 ms · policy ~20–100 ms · VLM ~1 Hz.
VLM reasons. VLA acts.
Two model classes with different jobs. The VLA's output drives motors directly, so it must pass a deterministic safety boundary. A VLM's goal, plan or tool call that can influence motion needs its own authority limits too, and neither check is the whole safety argument.
VLM grounds the goal the VLA executes. Model examples are illustrative of open classes (e.g. Pixtral-class VLMs; π0 / SmolVLA-class VLAs).
Dimension
VLM — Vision-Language Model
VLA — Vision-Language-Action
Output
Text / structured language
Robot actions / action chunks
Question it answers
"What am I looking at?"
"What should the joints do now?"
Role
High-level perception, reasoning, grounding
Closes the loop on hardware
Typical rate
~1 Hz / on-demand
~10–50 Hz
Trained on
Image-text pairs
Teleoperation demos (+ Open X-Embodiment)
Failure mode
Hallucinated plan
Unsafe motion — mandates a safety monitor
What we measured
Measured
68.71 GB
π0.5 full fine-tune — peak memory
Measured
1.08 → 0.13 (300 steps)
Real fine-tune (LIBERO, language-conditioned) — loss
Fine-tuned policy: 48/50 episodes (96%). Ten LIBERO-Spatial tasks seen in training, new simulator starting positions, official LeRobot evaluator. One training seed, a separate 1,000-step fine-tune. This is a pilot result on its own; it is not presented as a gain over a baseline.