Ollama and Strix Halo: Local AI Setup and Evaluation

Set up a local Ollama evaluation

1. Install Ollama from its official download for your operating system. Select a local model whose licence, task capability and memory needs fit the evaluation. Downloading weights and using a cloud model are different operations. Ollama quickstart.
2. For a local-only requirement, set OLLAMA_NO_CLOUD=1 in the environment of the actual server process and restart it. On a service-managed Linux installation, changing a terminal variable alone does not reconfigure the running service. Verify the effective setting in its logs.
3. Keep the evaluation on the default loopback address, 127.0.0.1:11434. A production application needs its own access controls, data policy and operating owner before sharing the service. Configuration and network behaviour.
4. Record the installed version and model inventory. Run a harmless representative prompt, then inspect the active model with ollama ps. Save the processor allocation with the test result; a successful response alone does not establish GPU acceleration.

ollama --version
ollama list
ollama ps

These are inspection commands for an installed runtime. This guide does not report a hardware test or a measured throughput result.

Check the Strix Halo configuration before buying or tuning

AMD lists Radeon 8060S graphics and up to 128 GB of system memory for the Ryzen AI Max+ 395. Check the exact computer configuration: installed RAM, firmware and memory available to the runtime are separate facts. The product specification does not establish how fast your model will answer. AMD specifications.

Record the OS release, kernel, graphics driver, runtime version, model revision, quantization, context size and power mode. On Ubuntu, check the compatibility of that complete combination. Avoid carrying settings from an old Ubuntu 24 guide into a different kernel or runtime without a new test.

Ollama documents Vulkan support on Linux and Windows when the backend is installed. Confirm the detected GPU and the processor allocation in the running application. Follow the current support matrix for your installed release; a supported AMD product name alone is not a verified backend configuration. Ollama hardware support.

For a controlled llama.cpp comparison, pin the source revision and model file. Its upstream build guide documents Vulkan prerequisites and -DGGML_VULKAN=ON. Record the detected device and keep model, context and workload identical across runs. llama.cpp build instructions.

Leave room for the operating system, model state and concurrent requests. If loading fails, reduce the test size and inspect logs before changing drivers or memory allocations. Record CPU fallback and failures instead of discarding them from the comparison.

Choose the runtime for the work

Ollama: a local service and repeatable application interface

Useful when the evaluation includes an application calling a model. Record both the server configuration and the calling application’s behaviour.

LM Studio: an interactive desktop evaluation

Useful for reviewing model behaviour with a person at the workstation. Check its current OS requirements; the Linux documentation specifically qualifies testing on newer Ubuntu releases. LM Studio requirements.

llama.cpp: direct control of a recorded build

Useful when backend settings and reproducibility are part of the question. That control also creates responsibility for maintaining the build and its operating configuration.

These are evaluation choices, not a speed ranking. Compare the same task and record differences in model files, prompt formatting and runtime defaults.