RTX 5090 vs. 128GB Unified Memory: What Local AI Actually Needs
The cost of ignoring memory capacity is a $3,000 GPU that can't load the models you want. Unified-memory mini PCs become competitive for MoE models, while the 5090's bandwidth advantage only applies when the entire model fits in VRAM.
The article compares a discrete GPU (RTX 5090 with 32GB VRAM, 1792GB/s bandwidth, 104.8 TFLOPS) against a unified-memory mini PC (AMD Ryzen AI Max+ 395 with 128GB shared memory, 256GB/s bandwidth, 14.8 TFLOPS). Raw FP32 compute favors the GPU 7:1, but for LLM inference the real constraints are memory capacity and bandwidth. A 70B model with 4-bit weights needs roughly 35GB, which doesn't fit in the 5090's VRAM. The mini PC's 128GB unified memory handles it by letting CPU and GPU share the same pool, but its 256GB/s bandwidth means reading all weights for each generated token yields only about 6 tokens/s for a 40GB model.
Mixture-of-Experts (MoE) models change the picture. Qwen3-30B-A3B activates only 3.3B of its 30.5B parameters per token, so on the AMD machine it can read the needed 1.65GB about 150 times per second. Similarly, DeepSeek V4.1 Flash (version 1017) has 763B total parameters but activates ~16B per token, requiring 8GB reads; the mini PC gets 32 reads/s, while the RTX 5090's 1792GB/s bandwidth would give 224 reads/s, but the full model won't fit in its VRAM. The mini PC also processes prompts slowly, e.g., about 95 input tokens/s for a 70B model, making long contexts take seconds to minutes. The conclusion: neither hardware is ideal; users can run only medium-sized MoE models with limited prompt lengths.
The article's arithmetic shows that for LLM inference, raw FP32 compute is a distraction; memory capacity and bandwidth do the heavy lifting, which is why a 7x slower mini PC can be a better local AI buy.
MoE architectures flip the bottleneck: as long as the active parameters fit in memory, bandwidth becomes the speed limit, explaining why DeepSeek V4.1 Flash is popular among local-model enthusiasts despite its massive total parameter count.
The prompt-processing bottleneck is the often-overlooked cost of low compute; even when token generation is acceptable, a slow chip turns a long document into minutes of waiting, a practical constraint for real workloads.