A large unified-memory workstation can make models fit. That is useful, but it is not the same as proving that a model, runtime and device are fit for an industrial product. This field note separates those two questions.
Evidence status — source-reviewed 2 August 2026. Hyperion's lab host was re-inventoried for this revision. The operating system reports an AMD Ryzen AI Max+ PRO 395 with 16 CPU cores / 32 threads, a Radeon 8060S-class Strix Halo display controller and about 125 GiB of usable system memory. Those observations establish the test host. They do not establish model throughput, accuracy, thermal stability or production readiness. Earlier single-run token-rate figures have been withdrawn until the model files, runtime commit, command line, raw output and machine state can be published together.
That evidence boundary is deliberate: a benchmark without its full run record is marketing, not engineering evidence.
What the platform specification establishes
AMD lists the Ryzen AI Max+ PRO 395 with 16 Zen 5 CPU cores, 40 RDNA 3.5 graphics compute units, support for up to 128 GB of LPDDR5x-8000 memory and a configurable 45–120 W TDP range. AMD's Ryzen AI Halo developer platform specifies a 128 GB configuration and 256 GB/s memory bandwidth. AMD's Strix Halo optimisation guide explains that the GPU uses GPU virtual memory rather than a separate discrete-VRAM pool.
Those facts change the capacity question. A team can evaluate a wider set of quantised open-weight models without first buying a discrete accelerator with an equally large VRAM pool. The shared pool can also accommodate the model, retrieval index and application processes on one development host.
They do not establish five things that matter in production:
- the task quality of any particular model;
- latency at the required prompt, context and concurrency distribution;
- sustained performance inside the intended enclosure and ambient range;
- runtime stability on the exact operating-system, kernel and driver combination;
- lifecycle properties such as signed updates, rollback, observability and field support.
Memory capacity opens an option. Only a workload-specific evaluation earns a product decision.
Start with the product envelope, not a model leaderboard
Write one page before downloading a model. It should state the decision the system supports, who acts on the output, the maximum tolerable response time, the offline period, the data boundary, the failure response and the expected device lifetime. For an operator assistant, the envelope might include multilingual maintenance questions, retrieval from approved manuals, citations in every answer and an abstention path. For a perception workload, it will instead specify sensor conditions, class-level error costs, cycle time and the deterministic safe state.
This prevents a common category error: selecting the largest model that fits, then trying to invent a product around it. The smallest configuration that clears the acceptance envelope is usually the stronger fleet candidate because it preserves thermal, memory and update headroom.
The six-part evaluation record
1. Pin the machine
Record the exact product SKU, firmware, memory configuration, power mode, cooling configuration, operating-system image, kernel, driver and free memory. lscpu, free -h, lspci -nn, runtime device discovery and temperature/power telemetry belong in the run bundle. A product name alone is not enough: OEM cooling and power policy can change sustained behaviour.
2. Pin the model artefact
Record the model repository and revision, licence, weight-file hash, quantisation format, context limit and chat template. Parameter count is not a workload metric. Dense, mixture-of-experts and multimodal models exercise memory and compute differently, while quantisations with similar file sizes can produce different task results.
3. Pin the runtime
Record the runtime version or Git commit, backend, build flags, driver versions and every command-line option. llama.cpp supports multiple backends and exposes llama-bench; results from different commits or backends should not be merged into one comparison. AMD also publishes Ryzen and Radeon limitations, which should be checked against the chosen software release rather than assumed away.
Choosing Ollama, LM Studio or llama.cpp on Ubuntu 24.04
The runtime is part of the evidence, not a cosmetic interface. On Strix Halo, three practical starting points serve different evaluation jobs:
| Runtime | Best fit for the first evaluation | What the run record must retain |
|---|---|---|
| Ollama | The quickest service and local-API path for repeatable application experiments. Ollama publishes a separate AMD ROCm package for Linux, and its current hardware register explicitly lists Ryzen AI Max+ 395 / gfx1151. | Ollama version, detected backend and device, model digest, context, concurrency, environment overrides and service logs. |
| LM Studio | Interactive model comparison and local-server experiments when a desktop workflow helps product and domain reviewers inspect outputs together. | Application and runtime version, model file and hash, selected backend, context and sampling settings, plus exported prompts and outputs. LM Studio supports Linux x64 through an AppImage but says Ubuntu versions newer than 22 are not well tested, so Ubuntu 24.04 is itself part of the validation target. |
| llama.cpp | Maximum control over build flags, backend choice, device enumeration and raw benchmark output. Its official build guide documents Vulkan builds on Ubuntu and runtime device selection. | Git commit, compiler and build flags, Vulkan or HIP/ROCm backend, device list, exact command, model hash, raw output and rejected-run reasons. |
Do not compare the three by copying a single tokens-per-second number from each UI. Use the same model revision, quantisation, prompt distribution, context, concurrency and machine state. First prove that the intended backend and gfx1151 device are actually selected; then compare task quality, latency distribution, memory pressure, sustained behaviour and operational fit. Ollama and LM Studio can be excellent exploration tools, while llama.cpp exposes more of the measurement surface. None removes the need for a pinned product-evaluation record.
4. Measure the whole request distribution
Separate model load, prompt processing and token generation. Test the prompt and context-length distribution the product will actually receive, including cold start, warm start and simultaneous users. Report median and tail latency, not only a best run. Keep the raw machine-readable output and rejected runs with their rejection reason.
5. Measure quality and failure behaviour
Use a versioned, task-specific evaluation set with clear provenance. Score factuality, citation correctness, abstention, tool-use boundaries and any safety-relevant behaviours that apply. Review failures by category. A faster model that sends an operator to the wrong procedure has not won the evaluation. NIST's AI RMF Measure function calls for documented, repeatable evaluation in conditions similar to deployment and continued production monitoring.
6. Measure sustained system behaviour
Run the intended workload long enough to reach thermal steady state, with the target enclosure, ambient and co-resident services. Capture clocks, temperature, power, memory pressure, queue depth and failures. Repeat after reboots and updates. AMD's platform documentation helps configure the host; it does not replace this product-specific soak test.
A decision table for this hardware class
| Question | Evidence required before a decision |
|---|---|
| Does the candidate fit? | Weight, KV-cache, retrieval index and application memory measured at the maximum supported context and concurrency |
| Is it responsive enough? | Cold/warm and tail latency across the real request distribution |
| Is it good enough? | Versioned task evaluation with error analysis, citations and abstentions |
| Is it stable enough? | Sustained run in the target thermal/power envelope, plus reboot and recovery tests |
| Is it supportable? | Pinned software bill of materials, update/rollback path and vulnerability response owner |
| Is it economical at fleet scale? | Device, integration, energy, operations, replacement and support cost over the product lifetime |
A yes in the first row cannot compensate for a no in any later row.
Where a 128GB APU is a strong option
This class is particularly interesting for development and bounded edge deployments where local data handling, intermittent connectivity or model capacity matters; where a compact workstation can host both inference and supporting services; and where the team can own the Linux/driver integration. It can also be a useful reference machine for model compression and retrieval experiments before a smaller target is chosen.
It is a weaker default when the product requires a certified hardware path the platform does not provide, a sealed thermal envelope that has not been validated, a CUDA-only dependency, deterministic hard-real-time control, or field support that the organisation is not prepared to own. A learned component should not be the sole safety mechanism merely because it runs locally.
The publication rule for future Hyperion benchmarks
A future performance table on this page must ship with: the host inventory; model and weight hashes; runtime commit and build flags; exact commands; raw JSON/CSV output; warm-up and repetition policy; power/thermal state; task-evaluation version; and the date of the run. If one of those is missing, the number will be labelled exploratory or omitted.
That standard is more useful than a dramatic tokens-per-second figure. It lets another team reproduce the result, tells a product leader what remains unproven, and makes later updates comparable instead of merely newer.
For the broader decision, use the open-weight model capability and Physical AI Product System: hardware selection is one architecture decision inside a full product lifecycle, not a shortcut around it.
