Inkling 2 Mini Loops Its Middle Layers 3x for Deep Reasoning in a 20GB Laptop Memory
This brings high-performance agentic coding to commodity laptops, removing the cloud API dependency and opening long-running local agents for a wider developer base, while exposing a new trade-off: looped depth boosts reasoning tokens but not factual recall or strict format compliance.
Thinking Machines Lab's open-source Inkling 2 Mini takes a different route to deep reasoning: instead of stacking more layers, it reuses a 12-layer block three times per token. The 26.4B-parameter MoE model stores 24 physical layers, but each token passes through 48 logical layers because the middle two blocks (layers 7-18) loop three times. An Attention Residuals mechanism with a learned gate controls cross-round information flow, and the model activates 5.4B parameters per token when accounting for loops. The result is a ~20GB memory footprint that runs offline on 32GB MacBooks or Mac Minis through Ollama, without cloud APIs.
Training uses low-bit precision throughout: first 20 layers in native 4-bit NVFP4, last 4 in 8-bit MXFP8, avoiding post-training quantization loss. The team also pruned experts from 384 to 128 per layer mid-training, then distilled 1T tokens to recover accuracy, followed by SFT and 2.2T tokens of RL. Independent testing by Stanford's Hazy Research Lab measured 0.703 Intelligence per Watt on a DGX Spark, beating several larger rivals. Official self-reported benchmarks show strong agentic coding performance (e.g., 48.2% on DeepSWE v1.1, 79.2% on SWEBench Verified bash-only), though independent verification is still thin.
The trade-offs are clear: looped depth doesn't expand factual knowledge (closed-book QA scores barely improve), and multi-turn instruction following lags smaller dense models. The model excels at tool-using agentic tasks and consumes far fewer reasoning tokens (chain-of-thought length halves from 3703 to 1661 tokens with 5 loops). For developers with 32GB machines who want a local, offline coding agent, this is a rare sweet spot, but the team warns that internal silent reasoning becomes harder to monitor as loops scale.
The loop-then-prune strategy effectively trades training compute for inference compute: you train a small model once, then gain extra depth at runtime by paying more activation per token (5.4B vs 3.4B). This inverts the usual model scaling trade-off and could shift how local AI is optimized.
The dramatic drop in chain-of-thought tokens with more loops suggests the model internalizes reasoning, which could make agentic behavior harder to audit or steer, especially at larger scales—a problem the team flags as open.
Self-reported benchmarks and a single third-party measurement leave the real-world advantage over Qwen and Gemma models uncertain; developers should re-run their own workloads before switching.
The loop approach fits local deployment better than simply stacking more layers. Reusing the middle layers lets 24 layers achieve 48 layers of depth, and combined with native 4-bit training, the size is squeezed to around 20GB, which is more convincing than post-hoc quantization. However, multichallenge is only 50.0% and complexconstraints 33.4%, so following complex constraints remains a weak spot. Before running long-context tasks locally, validate on your own codebase first, and don't just trust the official self-reported scores.