Adapter Patterns and Structured Output Are the Plumbing That Make LLM Agents Reliable
Without an adapter layer, every model switch becomes a rewrite. Without structured output, every downstream consumer becomes a fragile text parser. Together they turn an LLM from a text generator into a programmable, swappable, type-safe component that can sit inside a production pipeline.
Switching LLM providers without rewriting application code requires an adapter layer that presents a uniform interface. LangChain's `BaseChatModel` does this for OpenAI, Anthropic, Ollama, and others, giving every model the same `invoke`, `stream`, and `bindTools` methods. A factory function keyed on a runtime `provider` config field turns model selection into a single switch, and a fallback chain—cloud primary to cloud secondary to local Ollama—keeps the system running when a vendor goes down. Cost-aware routing further selects cheaper, smaller models for simple tasks like classification or translation, reserving expensive frontier models for reasoning and planning.
Structured output solves the downstream parsing problem that plagues LLM integrations. Instead of scraping names and ages from unpredictable free text with brittle regex, a Zod schema passed to `withStructuredOutput` guarantees a valid, typed object. The mechanism works by converting the schema into vendor-specific tool-calling formats under the hood, so the model effectively calls a virtual function whose parameters are the structured data. For large schemas, streaming partial JSON through a `JsonOutputParser` reduces perceived latency, though for small objects a single `invoke` call is faster.
Production use demands error boundaries. Models still produce invalid JSON, wrong types, extra fields, or missing required fields. A retry wrapper that catches parse and Zod validation failures, re-prompts, and ultimately returns `null` for caller-side fallback logic is the minimum viable pattern. On critical paths like tool parameter parsing, structured output with Zod validation is effectively mandatory.
The adapter pattern is not novel, but its application to LLMs exposes a hard truth: vendor API divergence is the real lock-in, not model quality. Standardizing on `BaseChatModel` makes the model a commodity, which is exactly what cloud providers don't want.
Cost-aware routing is under-adopted. Most teams default to a single frontier model for every request, leaving 50-80% cost savings on the table for simple tasks that a Haiku-class model handles just as well.
Structured output is often pitched as a convenience feature, but in Agent architectures it is a correctness requirement. A tool call with a malformed parameter doesn't just fail—it can corrupt state or trigger side effects.
The fallback chain design reveals a pragmatic truth about LLM reliability: the final tier should be a local model not because it's good, but because it's yours. A slow, mediocre answer beats no answer at all.