Skip to content

Capability · applied inside an engagement

Robotics, VLMs & VLAs

From perception to action: select and qualify models, control interfaces and protective functions for the actual robot and operating context.

Frame a robotics product decision

Hyperion combines product leadership with scoped model engineering and cybersecurity work. A technical mission defines the configuration, adaptation, integration and handover to be delivered; specialist testing and independent qualification are agreed explicitly.

A proposal is not authority to move
  1. Observe

    Sensors, state and uncertainty

  2. Propose

    Learned policy with bounded authority

  3. Constrain locally

    Controller limits and independent protective functions

  4. Act and observe

    Verify physical outcome before continuing

Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.

Architecture view

  1. Observe

  2. Propose

  3. Constrain locally

  4. Act and observe

Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.

Observe → Propose → Constrain locally → Act and observe

A robot that works in a demo is not one you can deploy. From perception to action: select and qualify models, control interfaces and protective functions for the actual robot and operating context. Hyperion leads the product decisions and acceptance criteria. A scoped Mission may include named implementation work by Hyperion. The proposal allocates the remaining engineering, specialist assurance and operating responsibilities.

Separate sensing, learned proposals, control and protective functions. Define each interface, its owner and its qualification evidence. Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.

ROS 2; vision-language-action policies (π-family, SmolVLA, NVIDIA Isaac GR00T) and VLMs; LeRobot for data, training and evaluation; teleoperation for demonstrations; simulation (Isaac Sim/Lab) and sim-to-real; fine-tuning (LoRA/QLoRA) and closed-loop evaluation; runtime safety monitor, operating envelope and emergency stop.

Separate learned proposals from control and protection

VLMs and VLAs serve distinct roles; the architecture depends on the task and controller. A model proposal needs an authorised interface, defined control limits and robot-specific safety evidence before physical use.

  • Input

    Sensors

    Cameras, depth, force-torque, proprioception

  • Input

    Perception

    Fusion, SLAM, 6-DoF pose

  • Learned

    VLM

    Grounds the goal — "what am I looking at?"

  • Learned

    VLA

    Action representations matched to the robot and controller

  • Configured control

    Planner & Control

    Trajectory and control budgets qualified on the target

  • Safety

    Safety monitor

    Independent protective functions; stopping and recovery tested in context

  • Output

    Actuators

    Motion in the physical world

Separate learned proposals from control and protection

Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.

Closed loop: actuator and environment state feed back to the sensors.

Measure the exact model, runtime, hardware and control integration

VLMs and VLAs serve distinct roles; the architecture depends on the task and controller. A model proposal needs an authorised interface, defined control limits and robot-specific safety evidence before physical use. Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.

VLM and VLA: different output contracts

Compare the output contract, embodiment fit and integration evidence. Neither model class, by itself, establishes readiness for physical deployment.

Control, stopping and recovery require robot-specific qualification. Cybersecurity is a separate assurance scope.
DimensionVLM — Vision-Language ModelVLA — Vision-Language-Action
OutputText / structured languageRobot actions / action chunks
Question it answers"What am I looking at?""What should the joints do now?"
RoleHigh-level perception, reasoning, groundingAction representations matched to the robot and controller
Timing validationMeasure the exact model, runtime, hardware and control integrationMeasure the exact model, runtime, hardware and control integration
Training evidenceVerify the release: modalities, data rights and task coverageVerify the release: demonstrations, embodiments and action representation
Failure modeHallucinated planIncorrect or unsafe action proposals; test limits, refusal and recovery

What we measured

  • Measured

    68.71 GB

    π0.5 full fine-tune — peak memory
  • Measured

    1.08 → 0.13 (300 steps)

    Real fine-tune (LIBERO, language-conditioned) — loss
  • Simulation

    48/50

    Simulation success (LIBERO-Spatial), fine-tuned policy

    Fine-tuned policy: 48/50 episodes (96%). Ten LIBERO-Spatial tasks seen in training, new simulator starting positions, official LeRobot evaluator. One training seed, a separate 1,000-step fine-tune. This is a pilot result on its own; it is not presented as a gain over a baseline.

  • On-robot, dry-run

    perceive 10/12 · understand 12/12 · interpret 10/12 · end-to-end 10/12

    On-robot cognitive benchmark (12 scenes) — perceive · understand · interpret
  • Measured

    2.3× – 14.3×

    bf16 throughput vs FP32 (ACT)

Not yet measured

  • On-robot task success rate — the arm loop is dry-run (no physical motion yet).
  • NPU (XDNA2) execution — the INT8 toolchain validates, but no DPU subgraph compiled; iGPU/CPU fallback only.
  • Convergence (the fine-tune stops at 300 steps), energy, and multi-task generalisation.

Discuss embodied-AI work