Local AI Hardware Just Got Way More Expensive

Summary

The video reassesses local-AI hardware prices and performance as of October 6, 2026. The presenter says many older buying guides no longer reflect current prices, attributing broad increases to shortages and rising prices for high-capacity memory and GPUs.

Prices and benchmark results can change quickly. Comparisons in the video come from different systems and test conditions.

How Performance Is Measured

• Memory capacity determines which models can fit. Memory bandwidth is a major limit on decoding speed: a machine repeatedly reads a model’s active weights while generating tokens. A rough performance limit can be estimated by dividing bandwidth by model-weight size.
• Decoding is the generation of the response. Prefill is the processing of the prompt before the response begins. A machine can perform well at one and relatively poorly at the other.
• Dense 70B models are slow on all the compared 128GB systems. The presenter says these machines are more compelling for mixture-of-experts models, which activate only some of their parameters for each token.
• Benchmark results are not directly interchangeable. The model file, quantization, engine, context length, drivers, and test date all affect performance.

Hardware and Performance Comparisons

AMD Strix Halo Mini PCs

The Bosgame M5 with 128GB was listed at about $3,999, and the Minisforum MS-01 Max at $3,799. Both use the Ryzen AI Max Plus 395 chip.

Community tests show that software updates have improved performance. One cited GPT-OSS 120B result rose from about 34 to 56 tokens/s across different tests. The main drawback is slower prefill compared with the Spark.

In Windows, graphics memory is capped at about 96GB, with system memory available separately. Linux configuration can raise the usable graphics-memory allocation, though reports vary. The video also mentions a 192GB Strix Halo tier, but its bandwidth is only modestly higher.

Mac Studio Max

The video cites a 128GB configuration priced at roughly $5,199, with 614GB/s of memory bandwidth. A reported GPT-OSS 120B result reached 87.9 tokens/s at 4K context, falling to 64.5 tokens/s at 32K.

That was the fastest generation result the presenter found for the model on a 128GB device, though the test used a different model file and quantization from some comparisons. Limitations include the lack of CUDA and slower prefill than a high-end Nvidia workstation GPU. The video’s chip-generation labels are inconsistent.

Nvidia DGX Spark

The DGX Spark’s price rose from $3,999 at launch to $6,950. It was out of stock at Nvidia’s store on the date checked.

Its 273GB/s bandwidth makes decoding performance similar to Strix Halo, but cited tests show much faster prompt processing—particularly relevant for agents that repeatedly handle long contexts. Advantages include CUDA tools and support for Nvidia inference stacks; up to four GB10 systems can be connected.

The video also mentions reports of power, firmware, and reliability problems. Buyers should check that their software supports the system’s MediaTek ARM CPU and DGX OS.

Four AMD MI50 Cards

The old guide price of about $600 for four cards is no longer realistic. The presenter found individual 32GB cards listed for roughly $588–$930. Four cards, plus a suitable host, power supply, risers, cooling, and chassis, were estimated to cost $3,800–$5,600.

This setup is technically demanding, draws substantial power, and depends on community-maintained software because ROCm support for the older GPU architecture is limited. A cited estimate of about 20 tokens/s on GPT-OSS 120B is described as approximate.

Multiple GPUs Generally

More cards can increase total model capacity, but simply splitting layers across cards does not necessarily improve single-user decoding speed. Tensor parallelism can run parts of layers simultaneously, but results depend on the software, connections, and workload.

Multi-GPU systems may make more sense for serving multiple users than for one person chatting.

Used RTX 3090s

Three cards provide 72GB of VRAM and can run some 70B models. A cited rented-system test achieved about 73 tokens/s at shorter context and 41 tokens/s at longer context.

However, the presenter says current card prices put a three-card build at around $4,248 before the host system, substantially more than earlier guide estimates.

Cost and Buying Guidance

The presenter compares purchase price with decoding speed and finds the Mac Studio Max and a well-tuned Strix Halo system broadly similar in cost per token/s, while Strix Halo has a lower upfront price. At its new price, the Spark costs considerably more and is mainly justified if CUDA or faster prefill is important.

For people who need local inference only occasionally, the video suggests comparing purchase costs with cloud GPU rental. It cites an RTX Pro 6000 rental at $2.09 per hour and estimates how many rental hours correspond to buying each local system, before accounting for electricity or resale value.

Practical Recommendations

• Choose Strix Halo for a relatively quiet, lower-cost local system if Linux configuration and slower prefill are acceptable.
• Consider a Mac Studio Max for fast generation at this memory capacity if macOS and the lack of CUDA are acceptable.
• Consider the DGX Spark when CUDA or fast prompt processing is essential, while accounting for its price and reported reliability concerns.
• Build a multi-GPU system mainly if you already own much of the host hardware or intend to serve several users.

Outlook

The video concludes that memory pricing may remain difficult. It cites a Micron executive’s warning, reported by The Register, that demand could outstrip supply into 2027–2028. The presenter stresses that future price or software changes could alter the comparisons.

Guides, Reviews, and Analyses Referenced

The video reassesses older local-AI assembly guides recommending four MI50s, Framework desktops, or DGX Spark against current listings.

Its comparisons draw on a Strix Halo community guide and wiki, Hardware Corner and llama.cpp benchmark results, and Reddit posts about DGX Spark availability, pricing, and hardware reports. It also references analysis and tests from The Register, ServeTheHome, Puget Systems, and community benchmark repositories.

Main Speaker and Sources

• Main speaker: The narrator from Local Explained.
• Sources cited: Hardware store pages, eBay listings, community guides and benchmark repositories, Hardware Corner, llama.cpp maintainers, Reddit users, AMD ROCm documentation, The Register, ServeTheHome, and Puget Systems.