1.3 Billion Free Tokens Every Month: 14 AI Giants' APIs at Your Fingertips, Including Gemini

Hello everyone, I'm Brother Er.

After spending the better part of a year using AI to write code and run Agents, my biggest takeaway is that model capabilities are indeed getting stronger and stronger.

But it's also really expensive. Claude Code + Codex costs me over $400 a month.

For ordinary people, this is a hefty expense. Heaven knows, I'm reluctant to pay it too, but I truly can't live without it. 😄

Luckily, brothers, there's always someone who finds a way. Last week I stumbled upon a project on GitHub, and the first line of its README read: "1.3 billion free tokens per month."

https://github.com/tashfeenahmed/freellmapi

It's already at 6.2k stars.

01. What is FreeLLMAPI?

Let's first clarify what this thing does.

FreeLLMAPI is a self-hosted API gateway that aggregates the free quotas from 14 AI vendors — including Google Gemini, Groq, Mistral, Cerebras, SambaNova, OpenRouter, GitHub Models, Cloudflare Workers AI, Cohere, and others — into a single endpoint, exposing a standard OpenAI-compatible interface.

You just register for each platform's free API key (no credit card required), configure them in FreeLLMAPI's backend, and it generates a unified Bearer Token for you. After that, all requests go to http://localhost:3001/v1/chat/completions, and the router automatically selects the best available model at the time.

Other platforms include Cohere, Z.ai (Zhipu), HuggingFace, NVIDIA NIM, and more.

02. Prerequisites

Node.js 20 or higher

On macOS, install directly with Homebrew:

brew install node@22

On Windows, it's recommended to use WSL2 or download the installer from the Node.js official website. After installation, verify:

node -v

Git

You need Git to clone the repository. macOS comes with it pre-installed. Windows users can install Git for Windows.

git --version

Register for Free API Keys from Each Platform

This step is the most time-consuming part of the whole process, but you only need to do it once.

I suggest starting by registering for three: Groq, Mistral, and OpenRouter.

Once these three are configured, they're basically enough for daily use.

The registration process is similar for all: go to the official website, create an account, find the API Keys page in the backend, create a new key, and copy it.

https://openrouter.ai/workspaces/default/keys

GitHub Models is a bit different. You need to create a Personal Access Token with Models permission in GitHub's Settings → Developer settings → Personal access tokens.

Save all the created keys in a text file for later configuration.

03. Cloning and Starting

Once the environment is ready, start the actual installation.

Clone the Repository

git clone https://github.com/tashfeenahmed/freellmapi.git
cd freellmapi

Install Dependencies

npm install

This step installs both frontend and backend dependencies.

Generate an Encryption Key

FreeLLMAPI uses AES-256-GCM to encrypt and store your API keys, so it needs an encryption key.

cp .env.example .env
echo "ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")" >> .env

This command generates a 64-character hexadecimal key and writes it to the .env file.

Start the Development Server

npm run dev

After starting, you'll see two addresses:

• Frontend Dashboard: http://localhost:5173
• Backend API: http://localhost:3001

Open http://localhost:5173 in your browser. You should see the FreeLLMAPI management dashboard, which means the installation was successful.

For production deployment, use npm run build && node server/dist/index.js; both frontend and backend will run on port 3001.

04. Configuring Providers and API Keys

Once the Dashboard is open, the first thing to do is configure the API keys for each platform.

On the left side of the Dashboard, find the Provider management page, click Add Provider, select the platform (e.g., Groq), and paste your API key.

You can use FreeLLMAPI to check the health status of this key. Green means available, red means invalid or rate-limited.

After configuring each Provider, FreeLLMAPI automatically registers the models supported by that platform into the routing table.

You don't need to manually specify which model to use; the router automatically selects based on availability.

After configuring all Providers, go to the API Key page in the Dashboard. You'll see the system-generated unified API key in the format freellmapi-xxxx. Copy this key; all clients will use it for authentication from now on.

05. Quick Verification

After configuration, test it with curl first to ensure end-to-end connectivity.

curl http://localhost:3001/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer freellmapi-YourKey" \
  -d '{
    "model": "auto",
    "messages": [{"role": "user", "content": "Introduce yourself in one sentence"}]
  }'

Note that the model field is set to auto, letting the router automatically select the best available model. The response header includes an x-routed-via field, telling you which Provider actually handled this request.

If a normal JSON response is returned, it means the entire flow from client to router to Provider is working.

06. Integrating with PaiCLI

PaiCLI is an open-source Agent CLI tool we built, similar to Claude Code.

Under the hood, it uses the Template Method pattern to manage LLM Providers, extracting common logic like HTTP requests, streaming parsing, Tool Calling, and Token counting into a base class. Each specific Provider just needs to inherit the base class and fill in the API address and key.

Currently, PaiCLI has four built-in Providers: GLM, DeepSeek, StepFun, and Kimi.

Configuration Method

Open PaiCLI.

/config provider freellmapi --base-url http://127.0.0.1:5173/v1 --api-key <FreeLLMAPI unified API key> --model auto 

After configuration, use /model freellmapi to switch to FreeLLMAPI, and you're ready to go.

Actual Test Results

After switching, you can try a few prompts directly in PaiCLI to verify that FreeLLMAPI is working correctly.

Case 1: Code Explanation

Read the code in the src/main/java/com/paicli/llm directory, organize the integration methods and core differences of all Providers, and output a comparison table.

This task triggers PaiCLI's file reading and code search tools. The model needs to understand Java code structure. You can intuitively feel how well the model routed by FreeLLMAPI performs in code understanding.

Case 2: Batch File Processing

Scan all TODO and FIXME comments in the current project, organize them into a list sorted by priority, and output to todo-report.md.

This task heavily invokes the grep tool. Over multiple rounds of conversation, you can observe whether FreeLLMAPI's Sticky Session mechanism is working — consecutive requests within 30 minutes are routed to the same model, maintaining context coherence.

How to Write It on Your Resume

If you've done integration development of FreeLLMAPI based on PaiCLI, you can package it on your resume like this:

Project Name: PaiCLI — Open Source Agent CLI Framework

Project Description: A terminal AI assistant similar to Claude Code, supporting ReAct loops, Plan-and-Execute, and multi-Agent collaboration

Tech Stack: Java, OpenAI Compatible API, Template Method Pattern, Streaming SSE Parsing, SQLite

Core Responsibilities:

1. Implemented the LLM Provider abstraction layer based on the Template Method pattern. Adding a new Provider requires only about 30 lines of code, supporting rapid integration of 6 model vendors including GLM, DeepSeek, StepFun, and Kimi.
2. Integrated the FreeLLMAPI gateway, aggregating 14 free APIs through the OpenAI-compatible protocol.
3. Encapsulated the OpenAI-compatible protocol base class, unifying streaming response parsing, Tool Calling parameter conversion, and Token counting, covering the differences of 4 mainstream API protocols.

07. Routing Mechanism and Rate Limiting Strategy

FreeLLMAPI's router is the most interesting part of the whole project. I spent an evening reading its source code and found the design to be much more refined than I expected.

Dynamic Penalty Routing

The router doesn't simply sort by a fixed priority.

It maintains a dynamic penalty mechanism — each model has a base priority, but the actual sorting uses "base priority + penalty score".

How is the penalty score derived?

Each time a model returns a 429 (rate limited), the penalty score increases by 3, with a maximum of 10.

It automatically decays by 1 every 2 minutes, and also decreases by 1 for each successful request. This means that after a model is rate-limited, it sinks down in the priority queue, allowing other models to take over. Once the cooldown period is over and the penalty score decays to 0, it returns to its original position.

The brilliance of this design over static priority is its adaptability. You don't need to manually adjust the order of Providers; the router automatically finds the optimal solution based on real-time rate limiting conditions.

Sliding Window Rate Limiting

The rate limit check uses a sliding window algorithm, not a fixed window.

What's the difference?

A fixed window is "100 times per minute," resetting the counter at the end of the minute. A sliding window is "100 times in the past 60 seconds," sliding every second.

FreeLLMAPI's implementation writes to both memory and SQLite.

Each request appends an entry to an in-memory timestamp array and also writes a record to the rate_limit_usage table in SQLite. When checking the limit, it looks back 60 seconds (RPM) or 24 hours (RPD) from the current time and counts how many request records exist.

Why dual writes?

Memory is fast, with query response times in microseconds. SQLite is slower, but data persists after a restart.

If the FreeLLMAPI process crashes and restarts, all in-memory counts are lost, but the records in SQLite remain, preventing the limit from being exceeded after a restart.

There are four check dimensions: RPM (Requests Per Minute), RPD (Requests Per Day), TPM (Tokens Per Minute), and TPD (Tokens Per Day). If any dimension hits the limit, that Provider is skipped.

Escalating Cooldowns

Even more interesting is the escalating cooldown strategy. If the same model triggers a 429 for the first time within 24 hours, it cools down for 2 minutes. The second time: 10 minutes. The third time: 1 hour. The fourth time: a full 24-hour cooldown.

The logic behind this design: if a model is repeatedly rate-limited, it likely means its free quota is nearly exhausted. Instead of wasting time trying every few seconds, it's better to let it cool down for a whole day and try again after the quota refreshes.

The cooldown status is stored in the rate_limit_cooldowns table in SQLite. The router checks the cooldown time before selecting each model; any model whose cooldown hasn't expired is skipped directly.

Sticky Session

There's a problem in multi-turn conversation scenarios: if the first turn routes to Groq's Llama, but the second turn suddenly switches to Google's Gemini, the context passed earlier might be in an incompatible format for Gemini, or the model's understanding of the context might differ, leading to a drop in response quality.

FreeLLMAPI's solution is Sticky Session.

It uses the SHA1 hash of the first user message as the Session Key, along with a single-turn/multi-turn flag.

After the first successful routing, the mapping between the Session Key and the model ID is stored in memory, valid for 30 minutes.

When a request from the same conversation comes in, the router first checks the Sticky mapping. If found, it preferentially uses the previous model.

Note: it's "preferential," not "mandatory." If the previous model has been rate-limited, the router skips it and continues down the normal fallback chain. It only reuses the previous model if it's still available.

Also, Sticky Session only applies to multi-turn conversations. If the message list doesn't contain any historical messages with the assistant role, it's considered a single-turn Q&A, and model binding is not needed.

Fallback and Retry

The entire request chain can retry up to 20 times.

On each retry, the combination of the failed model and key from the previous attempt is added to a skip set, and the router bypasses it when selecting the next one. Combined with the cooldown mechanism and dynamic penalties, 20 retries are generally enough to cover all available Providers.

If all 20 retries fail, a 429 error is returned, with the error message including the reason for the last failure.

Which errors trigger a retry?

429 rate limiting, timeouts, connection refused, 503 unavailable, and 500 server errors all trigger retries.

401 authentication failures do not trigger retries, because switching models won't solve an invalid key problem.