1. Export one perception model to ONNX and establish an FP16 latency-and-accuracy baseline. 2. Record a representative sensor dataset, generate an INT8 calibration cache, and compare p99 latency. 3. Package the inference service with health monitoring and test its SENSE → REASON → ACT behavior under thermal and network faults.

How to Set Up NVIDIA JetPack (Jetson): A Practical Guide

TL;DR

JetPack 7.2 pairs Jetson Linux 39.2 GA with Ubuntu 24.04, CUDA 13.2.1, cuDNN 9.20.0, and TensorRT 10.16.2. JetPack Downloads

Use the unified bootable ISO or SDK Manager to flash supported Jetson Orin and Thor hardware. ISO Installation Guide

Start optimization with FP16, introduce calibrated INT8 when necessary, and reserve FP4 for compatible hardware and explicitly quantized models.

JetPack has no paid software tier; hardware is separate, with the Orin Nano Super Developer Kit listed from $249. JetPack EULA Jetson Purchase Page

Flash JetPack and Complete First Boot

JetPack 7.2 supports the Jetson Orin family, AGX Thor Developer Kit, T5000, and T4000. Xavier, TX, and original Nano devices require an older JetPack branch. JetPack Downloads JetPack Archive

Option A: Unified ISO

Download the JetPack 7.2 ISO, write it to a USB drive with Etcher, boot the Jetson from USB, and follow the installer. NVIDIA recommends a monitor-based installation; headless installation requires additional manual steps. ISO Installation Guide

JetPack 7.2 does not provide a separate Orin Nano SD-card image. JetPack Downloads

On Orin hardware with old QSPI firmware, accept the installer’s capsule update. Skipping it can leave the firmware incompatible with the new image. Jetson Linux 39.2 Release Notes

Option B: SDK Manager

On Ubuntu 22.04 or 24.04, install SDK Manager, connect the recovery USB port, enter Force Recovery Mode, and select:

Product: Jetson
Hardware: Your Orin or Thor target
SDK Version: JetPack 7.2
Flash method: Direct Flash

These Ubuntu versions are validated for Jetson Linux 39.2. Windows 10/11 direct flashing is also documented for AGX Orin and AGX Thor through WSL2, USBIPD, ADB, and the NVIDIA APX driver. macOS is not a native SDK Manager host. Direct-Flash Guide SDK Manager Requirements

After Ubuntu’s first-boot setup, verify the BSP:

cat /etc/nv_tegra_release
uname -r

Expected:

# R39 (release), REVISION: 2.0, ...
6.8.x-tegra

For an already-flashed r39.2 device, install the complete SDK:

sudo apt update
sudo apt dist-upgrade -y
sudo reboot

sudo apt install -y nvidia-jetpack

An older BSP must be upgraded by flashing or ISO installation before using the r39.2 packages. JetPack Setup

Verify CUDA, cuDNN, and TensorRT

JetPack 7.2 includes CUDA 13.2.1, cuDNN 9.20.0, TensorRT 10.16.2, VPI 4.1.3, DeepStream 8.0, Holoscan 3.9.0, and NVIDIA Container Toolkit 1.19. JetPack Downloads

Check the installed stack:

nvcc --version

dpkg-query -W \
  'cuda-*' \
  'libcudnn*' \
  'libnvinfer*' 2>/dev/null |
  sort

trtexec --version

Expected version families:

CUDA compilation tools, release 13.2
libcudnn... 9.20...
libnvinfer... 10.16...
TensorRT version 10.16...

If nvcc is missing but CUDA packages are installed, add it to the shell path:

echo 'export PATH=/usr/local/cuda/bin:$PATH' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
source ~/.bashrc

Previously installed CUDA or SDK packages can conflict with SDK Manager components. Inspect package versions before removing anything. SDK Manager Release Notes

Optimize a Model with TensorRT

Assume you have exported a fixed-batch ONNX model as model.onnx.

First, validate it:

trtexec \
  --onnx=model.onnx \
  --shapes=input:1x3x640x640 \
  --dryRun

Build an FP16 engine:

trtexec \
  --onnx=model.onnx \
  --saveEngine=model-fp16.plan \
  --shapes=input:1x3x640x640 \
  --fp16 \
  --memPoolSize=workspace:2048

Build INT8 only after generating calibration.cache from representative robot-camera data:

trtexec \
  --onnx=model.onnx \
  --saveEngine=model-int8.plan \
  --shapes=input:1x3x640x640 \
  --int8 \
  --calib=calibration.cache \
  --fp16 \
  --memPoolSize=workspace:2048

For FP4, quantize the network into an FP4 Q/DQ ONNX graph using a compatible quantization workflow, then let TensorRT preserve its explicit data types:

trtexec \
  --onnx=model-nvfp4-qdq.onnx \
  --saveEngine=model-fp4.plan \
  --shapes=input:1x3x640x640 \
  --stronglyTyped \
  --memPoolSize=workspace:2048

Do not assume FP4 is a drop-in flag for every Jetson. Confirm hardware support, operator coverage, and task accuracy. In practice: establish an FP16 baseline, test calibrated INT8, and attempt FP4 only when memory or throughput justifies the extra validation.

A common build error is an unsupported ONNX operator. Run with --verbose, simplify or replace the operator during export, and rebuild:

trtexec --onnx=model.onnx --verbose 2>&1 | tee build.log

Measure Real On-Device Latency

Benchmark the saved engine under the same shape and power mode used by the robot:

trtexec \
  --loadEngine=model-fp16.plan \
  --shapes=input:1x3x640x640 \
  --warmUp=2000 \
  --duration=30 \
  --iterations=1000 \
  --useCudaGraph \
  --percentile=90,95,99

Expected summary:

Throughput: ... qps
Latency: min = ... ms, max = ... ms, mean = ... ms
Enqueue Time: min = ... ms, mean = ... ms
GPU Compute Time: min = ... ms, mean = ... ms

Run telemetry in a second terminal:

sudo tegrastats --interval 1000

trtexec measures the engine, not the complete SENSE-to-ACT path. Also timestamp camera capture, preprocessing, inference, postprocessing, middleware publication, and actuator command delivery:

latency_e2e =
  t_act_command - t_camera_capture

Avoid oversized CUDA allocations during testing: Jetson Linux 39.2 documents cases where they can reboot the device. Jetson Linux 39.2 Release Notes

Configure Power and Thermals

List the available board-specific power profiles before selecting one:

sudo nvpmodel -q --verbose
sudo nvpmodel -m 0
sudo jetson_clocks
sudo tegrastats --interval 1000

Mode IDs vary by device and image, so inspect nvpmodel -q --verbose instead of copying an ID blindly.

Monitor these signals during a sustained workload:

CPU/GPU utilization
RAM and swap use
GPU/CPU temperatures
Power-rail readings
Clock throttling

Use active cooling, unrestricted airflow, and a production-representative enclosure. Benchmark for several minutes: a fast ten-second result can hide thermal throttling.

An ISO upgrade on Orin Nano preserves the old power profile and does not automatically enable Super mode; NVIDIA recommends flashing through a Linux host or SDK Manager when that transition is required. Jetson Linux 39.2 Release Notes

Deploy a VLA or Perception Model

A simple native deployment layout is:

/opt/robot-inference/
├── app.py
├── model-fp16.plan
├── labels.json
└── requirements.txt

Create an isolated environment:

sudo mkdir -p /opt/robot-inference
sudo chown "$USER":"$USER" /opt/robot-inference

python3 -m venv --system-site-packages /opt/robot-inference/.venv
/opt/robot-inference/.venv/bin/pip install \
  numpy opencv-python-headless

Use --system-site-packages so the application can access JetPack-provided Python bindings without replacing NVIDIA packages.

Add /etc/systemd/system/robot-inference.service:

[Unit]
Description=Jetson robot inference
After=network-online.target

[Service]
Type=simple
User=robot
WorkingDirectory=/opt/robot-inference
ExecStart=/opt/robot-inference/.venv/bin/python app.py \
  --engine model-fp16.plan \
  --camera /dev/video0
Restart=on-failure
RestartSec=2

[Install]
WantedBy=multi-user.target

Enable it:

sudo systemctl daemon-reload
sudo systemctl enable --now robot-inference
journalctl -u robot-inference -f

In Physical AI Stack terms, JetPack supplies the local COMPUTE layer, accelerates SENSE, and can host REASON workloads. Keep ACT safety checks outside a probabilistic VLA policy.

On-Device or Server-Side?

Keep inference on-device when control-loop latency, intermittent connectivity, privacy, or safe degraded operation dominates. Use server-side inference when the model exceeds local memory, requires frequent heavyweight updates, or benefits more from datacenter batching than immediate response.

A useful hybrid split is:

on_device:
  - camera preprocessing
  - obstacle detection
  - safety policy
  - short-horizon control

server:
  - large VLA planning
  - fleet analytics
  - model training
  - non-urgent scene reasoning

Brief Alternatives

Raspberry Pi 5 + AI HAT+ 2: a lower-cost modular path; the $200 accelerator provides 40 INT4 TOPS and 8GB dedicated RAM, but JetPack provides a more integrated CUDA, TensorRT, multimedia, robotics, and deployment stack. AI HAT+ 2 JetPack Overview
Intel OpenVINO: more portable across Intel CPUs, GPUs, NPUs, Windows, Linux, and macOS, but it is an inference toolkit rather than a complete embedded BSP comparable to JetPack. OpenVINO System Requirements JetPack Overview

Practical Tips

Pin the BSP, engine, model checksum, power mode, and input shape in every benchmark report.
Rebuild TensorRT engines on the target software and hardware stack instead of treating plan files as portable artifacts.
Measure p95/p99 end-to-end latency, thermals, accuracy, and recovery behavior—not only average GPU time.
Validate memory pressure with the full camera, middleware, logging, and control workload running.

What's Next?

Export one perception model to ONNX and establish an FP16 latency-and-accuracy baseline.
Record a representative sensor dataset, generate an INT8 calibration cache, and compare p99 latency.
Package the inference service with health monitoring and test its SENSE → REASON → ACT behavior under thermal and network faults.

Hyperion Consulting helps robotics teams turn this Jetson baseline into production-ready Physical AI systems through practical AI tools consulting at hyperion-consulting.io.