New API Source Code Analysis: Architecture and Implementation of a Production-Grade AI Gateway

This article is based on a full reading of the October 2026 source code of QuantumNous/new-api (AGPLv3, forked from one-api). All code references are annotated with file and location so you can cross-check against the repository. Line numbers may shift in later versions; search by function name.

If you take a self-hosted AI gateway apart, the problems it must solve come down to three: who to send requests to (scheduling), how to speak so every model understands (protocol), and how to charge correctly (billing).

Most articles about New API cover deployment and channel configuration. This one doesn't. I read through the entire repository and picked out seven mechanisms worth learning, laid out in order of "one request from arrival to settlement." Once you understand the abilities inverted index, the RelayKit star-topology conversion, and the pre-deduct-settle-refund ledger, you understand most of the project's engineering decisions.

First, four numbers to build intuition:

• 47: number of channel adapter implementations, covering 40+ vendors including OpenAI / Claude / Gemini / Azure / Bedrock / Vertex;
• 4: natively interconverted protocols: OpenAI Chat, OpenAI Responses, Anthropic Messages, Gemini;
• 500,000: internal billing unit conversion (500K quota = $1). Every ledger numerator and denominator uses it;
• 60s: config sync cycle between nodes in multi-node deployments. Change a config once and the whole cluster picks it up within a minute without a restart.

I. Overall Architecture: Three-Stage Layering + A Storage Middleware Band

The whole system is a typical "management plane + data plane" gateway: the management plane (/api) handles console operations, and the data plane (/v1, /v1beta, /v1/tasks) relays traffic for AI clients.

Three engineering decisions worth calling out:

1. Single-binary delivery. The front-end React build output is compiled directly into the Go executable via go:embed — docker run starts it, and the Electron desktop version just spawns the same binary.
2. Protocol conversion and token counting are split into standalone modules. The relaykit/ and tokenkit/ Go modules are explicitly forbidden from depending on the host and can be built and reused independently. relaykit only depends on four libraries at runtime: uuid / lo / gjson / sjson.
3. Storage is split into four roles, not one database for everything: in-process cache handles routing and pricing (rebuilt in full from the primary DB every 60 seconds), the primary database handles business tables, the log database can be deployed independently (heavy users run ClickHouse, partitioned by month with TTL), and Redis handles the cross-node ledger and broadcast. The billing section expands on the last one.

II. The Complete Journey of One Request

This is the main thread of the article. One POST /v1/chat/completions, from arrival to settlement, seven steps:

01 TokenAuth   multi-protocol credential normalization → Redis-first token lookup → IP whitelist → inject user context
02 Rate limit  total token bucket + success sliding window (failures don't consume success quota)
03 Distribute  parse model → abilities index selects channel → take the key for this round
04 Pre-deduct   tokenkit estimates input locally → assemble pricing → Redis Lua atomic deduction
05 Retry loop  on failure switch channels and drop priority tier → BodyStorage replays the request body
06 Adapter relay  ConvertRequest → DoApiRequest sends upstream
07 Streaming settlement  SSE frames relayed to client → settle against real upstream usage

One design worth remembering across the whole pipeline: the request body has exactly one copy through its entire lifetime. common/body_storage.go implements a Seek-able storage (small requests stay in memory, large requests spill to disk). Middleware model parsing, adapter protocol conversion, pass-through mode, and the "replay" of every retry round all read from the same storage; outbound requests also get ContentLength and GetBody filled in so net/http's transport-layer retries can replay fully — eliminating "retry sent an empty body" at the root.

Keep this seven-step diagram; every mechanism below expands one of its steps.

III. Channel Scheduling: The Abilities Inverted Index

This is the design I most recommend reading closely in the whole project.

3.1 Why "Inverted"

The natural form of the channels table is a forward index from subject to attribute:

id=1  name="OpenAI-A"  models="gpt-4o,gpt-4o-mini,o3"  group="default,vip"
id=2  name="Claude-B"  models="claude-3.5,claude-4"    group="default"

The model list is an attribute of the channel — like a book's table of contents, content ordered by chapter. But the gateway's question is always the reverse: "Who can serve gpt-4o?" Answering with a forward index requires pulling the whole table, parsing each comma-separated models string, and filtering: O(channels × models), and LIKE queries on a TEXT field can't use an index.

New API pre-reverses all channel attributes — making the query key the primary key — and materializes an abilities table:

// model/ability.go
type Ability struct {
    Group     string `gorm:"type:varchar(64);primaryKey;autoIncrement:false"`
    Model     string `gorm:"type:varchar(255);primaryKey;autoIncrement:false"`
    ChannelId int    `gorm:"primaryKey;autoIncrement:false;index"`
    Enabled   bool
    Priority  *int64 `gorm:"bigint;default:0;index"`
    Weight    uint   `gorm:"default:0;index"`
}

The composite primary key (group, model, channel_id) compresses the triangular relationship of "which channel can serve which model in which group" into one index query:

SELECT * FROM abilities WHERE `group`=? AND model=? AND enabled=true;

This is the idea of an inverted index — the same thing as "word → pages containing it" in Google Search or "term → doc_ids" in Elasticsearch: query key as primary index, materialize many-to-many relationships, pre-build indexes for high-frequency reverse queries. It pays off because this is write-rarely, read-often data: channel config changes once every few months, and tens of requests per second query it. The function that builds the inverted index is AddAbilities() in the same file (reads the Models × Group Cartesian product and inserts in chunks); channel changes maintain it with a delete-then-insert in a transaction.

3.2 Selection Algorithm: Three Mechanisms

On top of the inverted index is GetRandomSatisfiedChannel in model/channel_cache.go (with Redis enabled, an in-memory cache is implicitly on — the full set of channels built into a group2model2channels two-level map):

1. Priority tiers, retry is degradation. This is the cleverest piece of code in the project — retry count is reused directly as the priority tier subscript:

// model/ability.go · getPriority
// The Nth retry automatically uses the Nth priority tier
priorities, _ := getPriority(group, model)   // dedup, descending
targetPriority := priorities[retry]

The first round lands on the highest tier, the first retry automatically drops to the next, and beyond the tier count it stays at the lowest. The auto group is smarter: within each group, all tiers are exhausted before switching to the next group — a small state machine of "group subscript + in-group tier" implements the full failure-degradation chain of "same-tier retry → drop tier → cross-group" without introducing any extra state object.

1. Smooth weighted random. All-zero weights mean equal probability; when average weight is under 10, weights are scaled 100x before randomizing — preventing small-weight channels from being starved out by integer rounding, keeping traffic proportional to config.

2. Session affinity + multiple keys. The same session prefers reusing the last successful channel (keeping context stable); one channel can hold multiple upstream keys, rotated in turn with a persisted cursor — cache rebuilds don't restart from scratch; a single key being disabled is written to a status list, and only when all keys fail is the whole channel disabled.

IV. Failure Retry Closed Loop: Decision and Side-Effect Separation

The retry main loop lives in Relay() in controller/relay.go, and getChannel() sits inside the loop body — so the real meaning of "retry" in this project is re-schedule with a different channel, not resend on the original route:

for ; retryParam.GetRetry() <= common.RetryTimes; retryParam.IncreaseRetry() {
    channel, err := getChannel(c, info, retryParam)      // ① re-select channel each round
    bodyStorage, _ := common.GetBodyStorage(c)           // ② replay request body
    c.Request.Body = io.NopCloser(bodyStorage)
    newAPIError = relayHandler(c, info)                  // ③ relay
    if newAPIError == nil { return }                     //    success exits
    decision := service.DecideRelayRetry(...)            // ④ decide
    service.RecordPolicyFailure(...)                    // ⑤ policy events to log
    processChannelError(...)                             // ⑥ side effects
    if decision.Action != "retry" { break }
}

Scheduling and retry are interlocked, at three points:

• Each loop round is a dispatch — the retry loop is the driver, the scheduler is the per-round decision function;
• One shared number: retry on the retry side is "which attempt" (time dimension), on the scheduling side it's "which priority tier" (space dimension); one parameter passing completes "retry is degradation";
• Failure information flows back to change the scheduling input: each round's side effect disables the channel and writes abilities.enabled=false and removes it from the memory cache — the next round (and all concurrent request scheduling) simply doesn't see this channel. The reverse also holds: when a pinned channel (sk-xxx-{id}) or strict session affinity fails, DecideRelayRetry returns stop directly — it doesn't even qualify for a retry.

DecideRelayRetry (service/relay_error.go:20) is worth reading in full — in just 99 lines, 9 return branches arranged by priority: strict session → pinned channel → channel-level error → non-retryable marker → attempts exhausted → 2xx excluded → unknown status code → retry-code hit → fallback no-retry.

The most valuable lesson is "decision and side-effects separated": each error object NewAPIError carries its own meta tags (IsChannelError / IsSkipRetryError, relaykit/types/error.go:371/378); the policy function only reads tags and doesn't make judgments; errors say everything they need to say where they occur. The default retry code range is 100-199, 300-399, 401-407, 409-499, 500-503, 505-523, 525-599 — 429 and most 5xx are retryable (with a channel switch), 400, 408, 504, 524 are explicitly not retried (400 is the client's fault; switching doesn't help; timeout-type errors mean the link is slow, so switching channels immediately beats waiting on the original route). Every stop/retry decision reason is fully written into the request log as a "policy event" — when a problem happens you can answer "why did it not retry back then", which is traceability.

Side effects are async: on a 401 hit with an auto-disable code, gopool.Go asynchronously disables the channel (without blocking the request thread), and Redis Pub/Sub broadcasts to all nodes to kill WebSocket long connections on that channel.

V. Adapter Pattern: 47 Adaptors, One Contract

47 channel adapters share one Adaptor contract (relay/channel/adapter.go), dispatched by GetAdaptor(apiType) switch-case at relay/relay_adaptor.go:50. The design tradeoff is smart: format conversion (ConvertOpenAIRequest / ConvertClaudeRequest / ConvertGeminiRequest etc.) is made into composable pure functions, and the real network I/O is delegated to three shared implementations — DoApiRequest (L315) / DoFormRequest (L345) / DoWssRequest (L381) in relay/channel/api_request.go.

The effect: to integrate a new channel, you only care about three things — how the URL is assembled, how the auth header is set, and what the request body looks like. Comparing SetupRequestHeader in relay/channel/openai/adaptor.go:192 and relay/channel/claude/adaptor.go:116, apart from the auth headers (Authorization: Bearer vs x-api-key + anthropic-version) and URL assembly, the structure is identical — the difference is compressed into a few pure functions, everything else is drowned in shared implementation reuse. claude/adaptor.go is only 182 lines, a living textbook of the "minimal complete adapter".

VI. Protocol Conversion: OpenAI Chat Is the Star Hub

Four protocols convert to each other, and the registry has 12 converters — but not a 12-line mesh of pairwise direct connections; rather a star topology centered on OpenAI Chat. Only 6 actually write conversion logic (Claude↔Chat, Gemini↔Chat, Chat↔Responses, one pair each); the other 6 are compositions. Look at Claude→Gemini's registration in relaykit/relayconvert/text_converter_registry.go:

ID:      requestConverterClaudeToGemini,
Quality: TextConverterQualityDiscouraged,
Req: TextRequestSide{
    StepConverters: []string{
        ConverterClaudeMessagesToOpenAIChat,   // hop 1: Claude → Chat
        ConverterOpenAIChatToGeminiContent,    // hop 2: Chat → Gemini
    },
},

It has no conversion logic of its own; it just chains two existing direct converters. This is the essence of the star: n protocols all converting to each other needs O(n²) converters; with Chat as hub it's O(n), and remaining paths are covered by composition — adding one protocol means writing 2 (that protocol ↔ Chat), and existing paths reuse automatically.

Why OpenAI Chat as the hub? It's the de facto standard of the client ecosystem (most SDKs speak it), and it's the most expressive intermediate format (tool calls, multimodality, streaming deltas all map), and Chat↔Responses are in the same OpenAI family, that pair is labeled good quality (the rest fair). The cost is that two hops may drop fields (every conversion is a lossy translation), which is why the pure two-hop Claude↔Gemini path is marked discouraged.

Implementation is four layers (relaykit/relayconvert/, ~2600 lines):

1. Registry and routing: converters register into a routing table by (From, To), with hard defensive validation at registration (Convert and StepConverters are mutually exclusive, step chains must connect end-to-end) — a wrong conversion matrix is blocked by panic at startup;
2. Request-side execution chain (executeRequestSteps): pull tools out first, convert hop by hop, put tools back. Tool calls are where the four protocols differ most; first they're normalized into one "tool collection", conversion doesn't touch it, and finally they're re-encoded for the target protocol — avoiding tool-difference handling at every hop;
3. Streaming response-side state machine (ResponseStreamState): multi-hop streaming isn't "convert line by line". Each hop holds independent state, one upstream event may be amplified or compressed into multiple, and trailing end events flow through subsequent steps;
4. Host glue layer (convmeta.Meta): converters themselves have no network and no billing; all session context (upstream model name, user settings, inference parameter state) passes through this interface, implemented by the host's RelayInfo.

All unmappable fields generate structured diagnostics (code/path/severity) written into the request log; the loss policy ToolLossPolicy has three tiers: allow (default), safe (request-stage error-level loss errors out), strict (any loss is rejected). For business-sensitive scenarios, turn on strict — prefer failure over silent loss.

VII. SSE Streaming Engine: Three Coroutines + One Main Loop

Streaming relay is where a gateway most easily breaks. The whole engine lives in relay/helper/stream_scanner.go (316 lines), and all channels share one entry point, StreamScannerHandler (L83) — each channel only passes in one closure (translating its own upstream format to the client format), and the engine uniformly handles concurrency, timeouts, keep-alive, and disconnects:

Several key defensive designs:

• Disconnect means stop losses: the main loop listens for client disconnect, and cleanup() immediately closes the upstream resp.Body — don't keep burning money and tokens for a request that's already been abandoned. This is the most important stop-loss design in a long-connection + billing scenario;
• Per-frame write timeout: DataHandler calls ExtendWriteDeadline before every write back, giving a 30-second write timeout — a slow client can't stall the coroutine;
• Write lock is request-scoped: writeMutex is a function-local variable, only preventing ping and dataHandler from crossing writes on the same ResponseWriter; there's no shared lock across requests;
• Careful coroutine reclamation: cleanup uses sync.Once for idempotency, and wg.Wait() only returns the gin.Context to the connection pool after all three coroutines exit — preventing coroutine references to a reused Context from producing cross-request writes;
• Full panic coverage: all three coroutines carry recover and write the reason into the unified StreamStatus (9 kinds of EndReason) for billing and log review.

Also an easy-to-miss boundary: the engine entry sits after the httpResp.StatusCode != 200 check at compatible_handler.go:151 — when upstream rejects (429/500), the client has received no bytes yet, and constructing an error back into the retry loop to switch channels costs nothing. But once 200 enters the engine and SSE response headers are written to the client, the HTTP response is already committed; a mid-stream drop can never be retried — only the EndReason gets recorded and billing runs on data received so far. This watershed cleanly separates the "retry" world from the "streaming" world.

VIII. Billing System: Pre-deduct — Settle — Refund

Billing is a full lifecycle. The code triangle is: relay/helper/price.go (pre-deduct estimate), service/text_quota.go (actual calculation), service/billing.go (delta settlement).

8.1 Pre-deduct: A Deposit

Before a request enters the retry loop, PrepareRequestBilling (relay/request_billing.go:25) does four things: sensitive-word check → tokenkit locally estimates input tokens (tiktoken exact counting for the OpenAI family, weighted estimation across 10 text feature classes for others) → assemble pricing → pre-deduct quota:

Pre-deduct = estimated input tokens × pre-deduct multiplier × ModelRatio × GroupRatio      // pay-per-token
Pre-deduct = ModelPrice × QuotaPerUnit × GroupRatio                                       // pay-per-call

Note pre-deduct only counts input (output is unknown); this is a deliberate conservative estimate — it's a deposit, not the final bill. Three special cases: users whose balance exceeds a "trust threshold" (default $10) skip pre-deduct entirely (a ledger-pressure optimization at high concurrency; settled in one pass at the end); free models skip the whole ledger flow; tiered expression pricing freezes a billing snapshot at pre-deduct (expression hash, hit tier, group multiplier) and retried channels reuse that same snapshot, so one request charges under exactly one rule set.

The physical implementation of pre-deduct is a Redis Lua script (model/quota_reserve.go:19) — "check balance" and "deduct" are atomic in one script:

if HGET key 'Quota' < amount then return 0 end   -- insufficient → explicit reject
HINCRBY key 'Quota' -amount                      -- atomic deduct

8.2 Settlement: Full Recalculation + Delta Calibration

After the response completes, calculateTextQuotaSummary (text_quota.go:227) recalculates the whole order against the real usage returned by upstream (using shopspring/decimal throughout to prevent float drift):

Actual cost = (input detail + output × CompletionRatio) × ModelRatio × GroupRatio
Input detail = base tokens
             + cache reads × CacheRatio           // gpt-4o=0.5, claude=0.1
             + cache writes (5m/1h) × tiered multiplier  // Claude long-lived cache is more expensive
             + image tokens × ImageRatio
             + Gemini audio separate input price

One semantic trap that must be handled: Claude and OpenAI have different usage semantics — OpenAI's prompt_tokens includes the cached portion (must be subtracted and priced separately at the cache rate), Claude's input_tokens doesn't include cache (must not be subtracted). The two branches at IsClaudeUsageSemantic (L305/L314) exist exactly for this; get them wrong and cache money gets double-counted. OpenAI's cache-write also reports an unadjusted prefix count that makes the base amount negative — L344 has a defensive clamp to 0.

Then delta settlement (billing.go:52): delta = actual cost − pre-deduct, positive charges the difference, negative refunds, zero does nothing. Note that at settlement the input portion is also recalculated — it's not "keep the pre-deducted input fee as-is, only add the output fee"; the whole order is one atomic account, and pre-deduct is merely a temporary deposit. Hotel analogy: pre-deduct is the front-desk pre-authorization, settlement is the final bill at checkout computed on actual consumption (room + minibar + late checkout), the pre-authorization settles the difference.

Settlement's two-phase commit (BillingSession.Settle) guards against double-refunds: funds are committed and fundingSettled is set; after that, token-adjustment failures only log, and the settled flag makes any subsequent Refund short-circuit — a session whose funds are settled can never be refunded again. The Refund on the all-failure path is idempotent (flag) + async (coroutine pool) + non-retryable (wallet refunds are non-idempotent; better to miss and leave a visible log than to double-refund).

8.3 Redis Is Not a Cache, It's the Authoritative Ledger

This is the layer most worth unpacking in the whole billing system. Look at a typical function, DecreaseUserQuota (model/user.go:1387):

func DecreaseUserQuota(id, quota int, db bool) error {
    gopool.Go(func() { cacheDecrUserQuota(id, quota) })   // ① Redis takes effect immediately
    if !db && common.BatchUpdateEnabled {
        addNewRecord(BatchUpdateTypeUserQuota, id, -quota) // ② DB catches up in 5s via aggregation
        return nil
    }
    return decreaseUserQuota(id, quota)
}

Pre-deduct and settlement always touch Redis first (millisecond-level); the database is caught up by the BatchUpdater 5 seconds later. In that 5-second window, balance checks and atomic pre-deducts all defer to Redis — two concurrent requests won't double-deduct because of DB latency. When Redis is unavailable, it degrades to a conditional DB update (UPDATE ... WHERE quota >= ?, affected-row count is success/failure) keeping no-race behavior.

Supporting details all orbit "the ledger must not be wrong":

• Recharges must also update the cache (syncCreditUserQuotaCache): if credit bypasses the cache, newly added quota is "invisible" until the cache expires and gets misjudged as insufficient balance — debits and credits must go through the same ledger;
• fence change barrier (token_cache.go:28): before writing a token change to the DB, SET fence EX 10s then delete the cache — a reader holding a pre-change snapshot can't write the old value back immediately after the cache is cleared; 10 seconds covers the "DB write + reader read-modify-write" window;
• auth version floor: restrictive changes like credential rotation use a monotonic version number as a stronger fence — old session snapshots can never re-authorize. This is a security property a time-based fence can't provide;
• live hash only refreshes TTL, never overwrites fields: no DB snapshot may overwrite the hot hash's RemainQuota — pre-deduct changes Redis first, so a DB snapshot is always older; overwriting once loses one deduction.

8.4 What Happens When There Isn't Enough to Deduct: Three Defense Lines + One Philosophy

When settlement's delta exceeds the balance, WalletFunding.Settle has no failure branch — deduct unconditionally, balance goes negative. This isn't an oversight: every path that could cause a deficit is either blocked in advance (entry check 403, pre-deduct atomic reject, mid-flight supplemental deduction for image tasks) or deliberately allowed by design — the code comment in reserveFunding says it plainly:

Deduct fully and unconditionally; the portion beyond the balance is recorded as debt (balance may go negative) without interrupting the request, keeping the pre-deduct amount recorded in the log always reconciled with the actual change in user balance.

The main sources are the trust bypass (orders that skipped pre-deduct because balance was sufficient but actually overspent) and mid-stream responses that can't be undone (you can't send the user half an answer and insert a "insufficient balance" line). After the debt: the balance displays negative, the next request gets a 403 at the door, and audit logs are traceable. The debt ceiling is naturally capped by "maximum cost of a single request" — this tradeoff preserves three things: request-path throughput (no blocking), ledger reconciliation consistency (never double-deduct or miss records), and user experience (started responses aren't interrupted). Better to take one unpaid order than to block every request on pre-deduction.

IX. Auth and Access Control: Two Planes + Two Counters

The data plane (/v1/*) runs TokenAuth (middleware/auth.go:528). Step one is multi-protocol credential normalization: Claude SDK's x-api-key, Gemini SDK's ?key=, Realtime WebSocket's Sec-WebSocket-Protocol: openai-insecure-api-key.{key}, and MJ proxy's mj-api-secret are all recognized and rewritten into a standard Bearer — one gateway-issued Key speaks all four protocol SDKs. Then: token lookup (Redis cache key is the token's HMAC; plaintext keys don't touch Redis) → IP CIDR whitelist → user cache → two-level group decision (the token's group must be within the user's available group set, preventing privilege escalation to advanced groups) → inject model whitelist, auto group, pinned-channel constraints, and other context. The sk-xxx-{channelId} pinned-channel suffix is admin-only.

The control plane (/api) has three credential layers: PAT (only a SHA-256 fingerprint is stored in the DB; route-level scope declaration table is fail-closed — undeclared routes are rejected), sessions (short-lived JWT + refresh Cookie + auth_version bump invalidates old sessions). Fine-grained permissions use Casbin, but the model is deliberately minimal — pure ACL with no role-inheritance section; roles are encoded into the subject string, and the decision chain is: root short-circuit → unregistered permission deny → user-level explicit override (explicit deny wins) → role baseline fallback. Permission resources are declared by a code registry; multi-node convergence comes from StartPolicySync periodically reloading policy.

Rate limiting sits after auth and before channel selection. Model-request rate limiting maintains two independent counters: total limit uses a token bucket (allows bursts, smooths rate), success limit uses a List-based approximate sliding window — and only successful responses count; failures don't consume the success quota (429/timeouts don't consume upstream success quota anyway; counting them would be fighting yourself). In Redis mode all nodes share counters; in memory mode each node counts independently, and deployments must evaluate the total by topology.

Why rate limiting? Of five layers of reasons, the two hardest: upstream quota protection (preventing one user from burning the upstream resources you paid for, turning a "shared pool" into "per-head allocation") and cost runaway protection (LLMs bill per token with no output cap; a buggy infinite loop script can burn a month's budget in a minute — rate limiting is the first gate beyond the bill).

X. Three Engineering Philosophies

Finally, three engineering lessons from the source you can take back to your own projects:

1. Single-copy replayability. BodyStorage's one Seek-able store runs through auth, conversion, pass-through, and every retry round; outbound requests get ContentLength and GetBody filled in. Direct benefits: zero-copy retries, absolute semantic consistency (the retried request body is byte-identical to the first). Often "build the data once, read-only everywhere" is cheaper and safer than deep-copying everywhere.

2. Retry is degradation. Retry count is reused as the priority tier subscript; one parameter passing completes the full "same-tier retry → drop tier → cross-group" failure-degradation chain. Not all state needs a new object — when you semantically connect two quantities that already exist (attempt count, tier index), complexity disappears.

3. Strong/weak consistency split. Config-type data (options, channel routing) goes eventually-consistent (60s full rebuild, no restart); ledger-type data (balances) goes strongly-consistent (Redis Lua atomic ops, DB catches up async). Split consistency level by data nature instead of bolting on distributed transactions for everything — with Redis authoritative in the window, you get both throughput and correctness.

Conclusion

New API's essence is splitting the distributed problem of "many upstreams, many protocols, many formats" into three independently evolvable main lines — channel scheduling, protocol conversion, billing ledger — then threading them into a highly available data plane with a replayable request body and an auditable retry policy.

If you're short on time and only read three pieces of code, in this order: getPriority in model/ability.go + RetryParam in service/channel_select.go (understanding "retry is degradation"); the whole 316 lines of relay/helper/stream_scanner.go (understanding every streaming pitfall); the Lua script in model/quota_reserve.go + Settle/Refund in service/billing_session.go (understanding where ledger correctness comes from).