Rec Builds a Model-Specific Inference Engine to Run Flint Coding Agents 1.4–2.6x Faster Than llama.cpp on Mac
The claimed speedup excludes prefill, so end-to-end coding-agent latency may not show anywhere near 1.4–2.6x; the figure is a benchmark to reproduce on your own hardware. The model-specific engine also raises the practical question of whether vertical optimization for one model is worth the lock-in and the 48 GB hardware floor.
rec is Recursive's dedicated inference engine for Flint, its own coding model. Instead of another general-purpose runtime, it optimizes for one model, one hardware family, and one workload: single-session local coding on high-end Apple Silicon. `rec launch pi` or `rec launch opencode` downloads the model, starts the inference service, and opens a harness; `rec serve` exposes the model alone, and both OpenAI- and Anthropic-compatible local endpoints are available with a token.
The headline claim is 1.4–2.6x throughput over llama.cpp on an M4 Pro 48 GB, at 4-bit weights, 1,024 output tokens, five runs per context length, and decode-only measurements that exclude prefill. No technical explanation of the underlying optimization or the llama.cpp build used has been published. rec is also very early, with 6 commits and 151 stars at the time of writing, and it only supports M4 Pro, M5 Pro, and M5 Max Macs with at least 48 GB unified memory.
The practical takeaway is a reproducible benchmark to run yourself, not a settled speed record.
A speedup measured only on decode after prefill can overstate gains for coding agents, where long-context prefill is often the dominant cost.
Specializing an inference stack to one model and one Mac lineup bets vertical optimization against ecosystem generality; with no engine details published, there is nothing yet to verify beyond rerunning the benchmark.
The 48 GB memory floor makes rec a niche tool today, and it sets a high baseline for local agentic models that vendors expect developers to buy into.