Most conversations about local LLM inference start with a GPU spec sheet and end with a VRAM wall. The interesting thing about AMD's Strix Halo class — the Ryzen AI MAX+ APUs — is that the wall moves: CPU, GPU (gfx1151) and the XDNA2 NPU share up to 128GB of unified memory, so the question stops being "what fits in 24GB of VRAM?" and becomes "what does the whole box need to do?"
The Hyperion lab workstation is exactly this class of machine. These are field notes from running open-weight models on it — measured numbers, the setup that works, and an honest account of the rough edges. For the deepest community resource on this hardware, see Donato Capitella's Strix Halo Toolboxes — containerized environments for llama.cpp, vLLM and fine-tuning workflows that saved us real setup time.
Why this hardware class matters for the industrial edge
Industrial edge AI has lived with an awkward gap. On one side: thin NPU boxes that run a quantised vision model and nothing else. On the other: rack servers that need power, cooling and a cabinet redesign before the first inference runs. The middle — one low-power box that runs a capable language model and the vision stack and the application — was mostly empty.
A 128GB unified-memory APU fills exactly that gap. A 30B-parameter open-weight model, a RAG index over your machine manuals, the OPC-UA bridge and the operator UI can share one machine that draws workstation power on a DIN-rail-adjacent footprint.
Measured: three models, one machine, no tuning
Methodology, stated plainly: ollama run --verbose with a fixed ~30-token prompt, models as shipped by the Ollama library (default quantisations), single runs on a developer workstation under normal daily load. No kernel tuning, no allocator flags, no cherry-picking. These are floor numbers, not ceiling numbers — the toolboxes linked above document what tuned llama.cpp setups achieve.
| Model | Params | Load time | Prompt eval | Generation |
|---|---|---|---|---|
| Qwen 2.5-VL | 7B (vision-language) | 6.9s | 101.0 t/s | 16.6 t/s |
| GPT-OSS | 20B | 44.9s | 58.1 t/s | 22.8 t/s |
| Qwen 3 (MoE) | 30B (A3B active) | 30.7s | 32.1 t/s | 19.3 t/s |
Two honest observations from that table.
The counter-intuitive one: the 20B and 30B models generate faster than the dense 7B vision-language model. The 30B is a mixture-of-experts with ~3B active parameters per token, and the VLM carries vision-tower overhead — a useful reminder that parameter count alone predicts very little. Architecture dominates.
The practical one: ~20 tokens/second is comfortably past the threshold where an operator-facing assistant feels responsive, and the machine still had headroom — during these runs the same box was running a development environment, containers and an embedding service. That co-tenancy is the point of unified memory.
What actually works
- Ollama as the front door. For getting models running with zero ceremony, it works on this hardware today. The ROCm/Vulkan picture underneath is evolving fast; the toolboxes project tracks the current best backends per workload.
- Vision-language locally. Qwen 2.5-VL at 7B answers questions about photographs of equipment — the industrial use case is inspection triage and maintenance documentation, and it runs entirely on the box.
- Big-model headroom. 13GB and 18GB models load without a second thought. On a 24GB discrete GPU, the 30B MoE would already be a negotiation.
Honest limits
- Ecosystem maturity. CUDA-first tooling still assumes NVIDIA. ROCm support for gfx1151 is moving quickly but you will read GitHub issues. Budget integration time accordingly.
- Load times. Tens of seconds to first token on big models matters if your edge box power-cycles; keep models resident.
- Sustained load. A workstation APU under continuous batch inference is a thermal design question — measure before committing a fanless enclosure.
- This is not a fleet story. One capable box is a pilot enabler. Production still means signed updates, observability, recovery and a safety boundary that does not depend on the model — the pilot-to-production ladder exists precisely because the hardware is the easy third of the problem.
Where this fits a Physical AI stack
Our stack preference is unchanged: Mistral and open-weight models for sovereignty and edge deployability, deterministic safety around every learned component, and edge-first architectures that keep working when the backhaul does not — the pattern class we prove in Auralink's simulated operations. Hardware like Strix Halo makes that architecture cheaper to pilot: the box that runs your model in the lab can be the box that runs it beside the line.
If you are weighing edge hardware for an industrial AI initiative, the hardware selector maps constraints to platform classes — and a fit call is thirty minutes with the person who ran these benchmarks.
