AI Gateway: Turning 'Switch to DeepSeek' into a Config Change

Teams juggling multiple providers and dozens of apps already pay for mismatched error handling and invisible cost attribution. A gateway compresses that governance into a single configurable layer, making provider swaps and token-level cost controls a config edit rather than a rewrite.

Direct API calls break down as soon as a system grows past one model and one caller. Open different providers and you get `tool_calls` vs. `tool_use` vs. `functionCall`, different streaming shapes, and different retry semantics. Three adapters plus scattered `if provider ==` checks later, the request path is a pile of glue code, and the monthly bill can't be attributed to any business line.

AI Gateway pulls all of that into a single OpenAI-compatible endpoint. It normalizes protocols, routes requests by fixed rules, cost tiers, or fallback policies, maps virtual keys to real provider keys, and meters token consumption and cost. LiteLLM serves as the concrete example: install the proxy, define model routes in YAML, and point your OpenAI client at `http://localhost:4000`; switching providers becomes a config edit.

The gateway's own tradeoffs are real. It adds a failure point, can silently drop provider-specific parameters, and self-hosted versions carry patching duties (LiteLLM alone has logged over 25 CVEs, per Thoughtworks). Teams that only run one model don't need it. But for multi-model, multi-provider setups with cost and audit requirements, the gateway shifts 'calling the model' from a code chore into a governed infrastructure concern.