128GB Unified Memory Makes Local LLMs a Commodity, but the Real Fork in the Road Is Software
For developers sizing up local inference, the practical takeaway is blunt: if you want long-context 100B models on the desktop, 128GB unified memory is the only threshold, and the bandwidth race is already over. The next battleground is software that decides what runs locally, what goes to the cloud, and how agents route between them.
The wall for running large models locally is VRAM: a 24GB GPU cannot fit a 70B model once you add the KV cache for long context. The form factor that actually crosses that threshold is a mini PC with 128GB of unified memory, where CPU and GPU share one memory pool. Tests show that a 100B-class model can consume 80-100GB, which is impossible for a 24GB card but routine for a 128GB machine.
Memory capacity also dictates context length. Ollama defaults to 4k context under 24GB, and only opens 256k above 48GB. On the model side, Qwen3.6's 35B MoE runs at 20 tokens per second on a 64GB laptop's integrated GPU, though it tends to lose its way on long multi-step tasks. Measured alongside each other in November, an NVIDIA DGX Spark and an AMD-platform mini PC produced nearly identical generation speeds (273 vs 256 GB/s bandwidth, a "rounding error"); the difference is capacity, not velocity. The AMD machine costs about half as much and runs the full x86 software stack, while the Spark offers faster inference peaks and CUDA.
Since October, system makers worldwide have converged on this 128GB unified memory form factor, and Microsoft added AMD's Ryzen AI Max to its local model support list. Hardware is becoming an entry ticket. The real differentiation now sits in software: Codigger proposes three layers — local inference for high-frequency tasks, cloud workstations for peak compute, and agent orchestration that automatically routes tasks between the two, so users never shuffle files by hand.
The 273 vs 256 GB/s bandwidth difference being called a 'rounding error' suggests that memory bandwidth in this class of unified memory devices has effectively plateaued for current model workloads.
The real constraint for local LLMs is no longer raw compute speed but memory capacity and the software that manages context and task routing.
Microsoft's formal support for AMD's Ryzen AI Max signals that the 128GB unified memory mini PC is becoming a mainstream target for local model deployment, not just a niche enthusiast toy.
Codigger's three-layer approach (local, cloud, agent) indicates that the winning software will treat the data-center and the desktop as two ends of the same pipeline, rather than separate silos.
The report that Qwen3.6's 35B MoE gets lost on long tasks suggests that memory capacity alone doesn't guarantee reliable multi-step reasoning; stability is a separate quality that depends on model architecture and context handling.