跪拜 Guibai
← Back to the summary

Rec Builds a Model-Specific Inference Engine to Run Flint Coding Agents 1.4–2.6x Faster Than llama.cpp on Mac

One Open Source Project a Day (No. 246): rec — Recursive's self-developed local inference engine that runs the Flint coding model on Mac, claiming 1.4–2.6x faster than llama.cpp Author: 冬奇Lab Tags: Artificial Intelligence, Open Source, LLM

"Our lightning-fast inference engine."

This is article No. 246 in the "One Open Source Project a Day" series. Today's project is rec (recursive-oss/rec).

For running models locally, everyone already has mature options like llama.cpp, Ollama, and LM Studio. rec takes another path: no generality, only one model and one category of scenario. It is the inference engine that Recursive built specifically for its own Flint language model, with the goal of letting coding agents run fast enough locally on a Mac.

To be clear up front: this is a very new, very small project (at the time of writing: 151 Stars, 6 commits), so judgment on it should be restrained. This article focuses on how to use it, where its boundaries are, and which claims are only the project's own statements for now.

What you will learn

Prerequisites

Project Background

Project Introduction

rec is a self-developed inference engine for running Recursive's language model Flint locally, aimed at single-session scenarios. Flint is positioned in the documentation as "fast, local agentic coding"; it can run locally or in the cloud, and then be connected to coding tools (harnesses).

rec's responsibilities can be split into three parts:

  1. Automatically download the model and perform efficient inference on Mac
  2. Automatically configure and start a coding harness (pi or OpenCode)
  3. Expose a local OpenAI / Anthropic-compatible API so you can connect your own client

Author / Team

Project Data

Main Features

Core Role

Run Flint locally on Apple Silicon Macs at a speed beyond general-purpose runtimes, and connect it to coding agents.

Usage Scenarios

  1. Local coding agent

    • Use rec launch pi to start the model and harness with one command; code stays on the machine.

  2. Provide a local model to your own client

    • Connect any compatible client through the OpenAI-compatible endpoint /v1/chat/completions.

  3. Provide a local model to Anthropic-protocol tools

    • Through the /v1/messages endpoint, point ANTHROPIC_BASE_URL at the local service.

  4. Compare local inference engine throughput

    • Compare single-session decoding speed with llama.cpp on the same machine.

Quick Start

Hardware and Environment Requirements

Installation

curl -fsSL https://raw.githubusercontent.com/recursive-oss/rec/main/install.sh | bash

rec installs to ~/.local/bin and is added to PATH for new terminals; other data lives in ~/.local/share/rec.

Starting

rec launch pi        # officially recommended
rec launch opencode

The first run downloads the model and the harness. After you close the terminal, the service keeps running; use rec stop to stop it. rec's harness is isolated from ones you install on the system, and you can run commands in its environment with rec exec:

rec exec pi install npm:PACKAGE

Official note: rec is optimized for single-session use. To run multiple agents in parallel, refer to the other run modes in the official documentation.

Using Your Own Harness

Start the service first (keep the terminal open; after flint is ready. appears, connect from another terminal):

rec serve

To see which environment variables and command-line arguments rec launch uses, print them without executing:

rec launch --args pi
rec launch --args opencode

Replace the harness path with your own. If you want to keep your own settings and sessions, remove the variables that point to rec's settings and sessions.

Call the OpenAI-compatible API directly:

curl -fsS http://127.0.0.1:11500/v1/chat/completions \
  -H "Authorization: Bearer $(rec auth token)" \
  -H 'Content-Type: application/json' \
  -d '{"model":"flint","messages":[{"role":"user","content":"Say hello."}],"max_tokens":128}'

Configuration for other clients:

Core Features

  1. Inference engine dedicated to Flint

    • It does not aim to support all models, gaining room for targeted optimization.

  2. One-command local coding environment

    • rec launch automatically downloads the model, configures it, and starts pi or OpenCode.

  3. Two compatible APIs

    • OpenAI and Anthropic protocols, so existing toolchains can connect locally.

  4. Isolated harness

    • The rec-managed harness does not interfere with system-installed ones.

  5. Local authentication

    • The service requires a key generated by rec auth token, rather than exposing a bare local port.

  6. Complete CLI management

    • Start/stop, status, model download, environment self-check, update, and uninstall.

Command List

Command Purpose
`rec launch pi opencode`
`rec exec pi opencode ARGS`
rec serve Start the model service in the current terminal
rec status Check whether the service is running
rec stop Stop the service
rec auth token Print the local API key
rec pull Only download the model
rec doctor Check hardware and installation
rec update Update to the latest version
rec uninstall Remove rec, the model, and settings

For options of each command, use rec COMMAND --help.

Project Advantages

The project's self-reported performance claim: on M4 Pro, compared with llama.cpp, throughput is 1.4–2.6 times, covering context lengths from 1K to 252K.

The test conditions are as follows:

Item Condition
Device Apple M4 Pro (48 GB)
Weight precision 4-bit
Output length 1,024 tokens
Repetition five runs per context length
Measurement scope excludes prefill

Qualitative comparison with other options (no formal comparison is given on the project page; the table below is based on rec's positioning):

Dimension rec llama.cpp / Ollama / LM Studio
Supported models dedicated to Flint general, covers many models
Hardware Apple Silicon only (M4 Pro / M5 Pro / M5 Max), 48 GB minimum much wider range
Concurrency optimized for single session depends on the runtime
Ready-to-use coding environment built-in one-click pi / OpenCode needs its own configuration
Maturity very new, 6 commits battle-tested

When is it worth trying?

Detailed Project Analysis

Design Trade-off: Dedicated Rather Than General

The most discussable part of rec is its trade-off. General inference runtimes must support a large number of model architectures, quantization formats, and backends; the cost is that it is hard to be extreme for one model. rec does the opposite: it serves only Flint, only Apple Silicon, and only optimizes single-session decoding.

The benefits and costs of this trade-off are clear:

Benefit Cost
Only supports Flint can optimize around one model architecture cannot use it for another model
Only supports high-end Mac can assume unified memory bandwidth and capacity high hardware threshold (48 GB minimum)
Only optimizes single session best decoding latency multi-agent parallelism is not its home turf

Project Structure

The root directory shows directories such as cli, engine, packaging, tests, as well as config files like go.mod, .golangci.yml, and ruff.toml. This suggests at least a division between CLI and engine, involving both Go and Python toolchains. The language proportions and the specific engine implementation are not given on the project page; this is only an inference from file names.

What optimizations the engine performs internally and why it is faster than llama.cpp are not explained on the project page or the docs site, so this article will not speculate. If you want to know the principle, you need to read the engine directory source code or wait for official technical documentation.

How to View the "1.4–2.6x" Claim

This is the project's own test result. Several points to note:

Therefore, the safer way is to treat it as a lead worth re-testing yourself, not as a conclusion.

Limitations and Points to Note

Project Address and Resources

Official Resources

Related Resources

Summary and Outlook

Key Points Review

  1. Dedicated engine: rec optimizes only for Flint, only Apple Silicon, and only single session; it trades generality for speed.
  2. Ready out of the box: one-command installation, and rec launch pi gives you a local coding agent environment.
  3. Friendly interfaces: it provides local OpenAI- and Anthropic-compatible APIs with local authentication.
  4. Treat the performance claim with caution: the 1.4–2.6x figure comes from the project's own statement, covers decode only and a single machine, and is worth re-testing yourself.

Who It Is For

Learning Suggestions

First run rec doctor to confirm the machine meets the requirements, then run rec launch pi to experience the end-to-end flow. To verify the performance claim, re-test on the same machine under the stated conditions (4-bit, 1,024 output tokens, five runs per context length), and record prefill time separately.

Explore PrimeSkills — a selection of AI agents and skill tools, each verified through real workflows. No hype, only tools that actually work.

Visit my personal homepage for more insights and interesting products.