Adapter Patterns and Structured Output Are the Plumbing That Make LLM Agents Reliable

1. LLM Adapter Provider

This is an operation to decouple upper-layer business code from specific LLM vendors, aiming to support "switching models without changing business code."

1.1 Why do we need an adapter to access LLMs?

There are dozens of LLM vendors on the market today—OpenAI, Anthropic, Google, Alibaba Tongyi, ByteDance Doubao, local Ollama... Each vendor's API protocol, authentication method, and parameter naming are not exactly the same. If your business code directly fetches a specific vendor's API, once you want to switch models (e.g., from GPT-4 to Claude), you have to change all call sites.

The Adapter Pattern solves this problem: define a unified interface, each vendor implements its own adapter, and business code only programs against the interface.

Your Business Code (only knows the BaseChatModel interface)
        │
        ├─ ChatOpenAI (adapts OpenAI API)
        ├─ ChatAnthropic (adapts Anthropic API)
        ├─ ChatOllama (adapts local inference)
        └─ ChatGoogleGenerativeAI (adapts Gemini)

Switching models = changing one line of instantiation code, with zero changes to business logic. This is also why "using Claude or local Ollama" can be made a runtime configuration item (e.g., provider: 'anthropic' | 'openai' | 'ollama'), rather than hardcoded.

1.2 Capabilities of the Adapter Provider

LangChain abstracts this set of adapters into the BaseChatModel base class (in @langchain/core/language_models/chat_models). All vendor adapters inherit from it, providing consistent capabilities:

Capability	Method	Description
Non-streaming call	invoke(messages)	Blocking return of a complete AIMessage
Streaming call	stream(messages)	Returns AsyncIterable<AIMessageChunk>
Tool calling	bindTools(tools)	Lets the model know which tools are available, returns the bound model
Structured output	withStructuredOutput(schema)	Forces the model to return according to a Schema (see Section 2)
Token usage	response.usage_metadata	Returns input/output/cache usage after the call
Batch calling	batch(messagesList)	Processes multiple message groups concurrently

The instantiation style for all adapters is highly consistent:

import { ChatAnthropic } from '@langchain/anthropic';
import { ChatOpenAI } from '@langchain/openai';
import { ChatOllama } from '@langchain/ollama';

// Same constructor parameter style, just change the class name
const models = {
  anthropic: new ChatAnthropic({
    model: 'claude-sonnet-4-20250514',
    temperature: 0.7,
    maxTokens: 4096,
  }),

  openai: new ChatOpenAI({
    model: 'gpt-4o',
    temperature: 0.7,
    maxTokens: 4096,
  }),

  // Local Ollama: zero privacy leakage, offline capable
  local: new ChatOllama({
    model: 'qwen2.5:14b',
    temperature: 0.7,
  }),
};

// Business code: doesn't know or care about the underlying vendor
async function ask(model: any, question: string) {
  const response = await model.invoke([
    { role: 'user', content: question },
  ]);
  return response.content;
}

// Same business logic, seamlessly switching the underlying model
await ask(models.anthropic, 'Hello'); // Using Claude
await ask(models.openai, 'Hello');    // Using GPT-4o
await ask(models.local, 'Hello');     // Using local model

A common encapsulation pattern: write a createModel(config) factory function that dynamically selects an adapter based on the config.provider field, converging the "which provider to choose" decision into a single source of truth. Adding support for a new vendor only requires adding one case branch here.

1.3 Model Routing / Intelligent Model Selection

Not all tasks require large models like GPT-4o / Claude Sonnet—automatically selecting models based on task complexity can save 50%+ in costs:

import { ChatAnthropic } from '@langchain/anthropic';
import { ChatOpenAI } from '@langchain/openai';

// Task classifier: determines task complexity
function selectModelByComplexity(task: string) {
  // Simple tasks (classification, extraction, translation) → use small model
  const simplePatterns = [/Translate.*to/, /Classify.*:/, /Extract.*keywords/];
  if (simplePatterns.some(p => p.test(task))) {
    return new ChatAnthropic({
      model: 'claude-haiku-4-5', // Small model, 10x cheaper
    });
  }
  // Complex tasks (reasoning, creation, planning) → use large model
  return new ChatAnthropic({
    model: 'claude-sonnet-4-20250514',
  });
}

async function ask(task: string, input: string) {
  const model = selectModelByComplexity(task);
  return await model.invoke([{ role: 'user', content: input }]);
}

More advanced: Cost + Latency-based routing:

// Model catalog (cost, latency, capability tags)
const MODEL_CATALOG = {
  'haiku': { cost: 0.25, latency: 0.5, capability: ['classification', 'extraction'] },
  'sonnet': { cost: 3, latency: 2, capability: ['reasoning', 'coding', 'writing'] },
  'opus': { cost: 15, latency: 5, capability: ['complex-reasoning', 'research'] },
};

function selectModel(requiredCapability: string, maxCost = 5) {
  const candidates = Object.entries(MODEL_CATALOG)
    .filter(([_, m]) => m.capability.includes(requiredCapability) && m.cost <= maxCost);

  // Choose the cheapest from the candidates (while meeting capability requirements)
  candidates.sort((a, b) => a[1].cost - b[1].cost);
  return candidates[0][0];
}

1.4 Fallback / Disaster Recovery

Automatically switch to a backup Provider when the primary fails—ensuring service availability:

import { ChatAnthropic } from '@langchain/anthropic';
import { ChatOpenAI } from '@langchain/openai';
import { ChatOllama } from '@langchain/ollama';

const primary = new ChatAnthropic({ model: 'claude-sonnet-4-20250514' });
const fallback1 = new ChatOpenAI({ model: 'gpt-4o-mini' });
const fallback2 = new ChatOllama({ model: 'qwen2.5:14b' }); // Local fallback, never goes down

// ① Simple version: chained fallback
async function invokeWithFallback(input: any) {
  for (const model of [primary, fallback1, fallback2]) {
    try {
      return await model.invoke(input);
    } catch (err) {
      console.warn(`Provider ${model.constructor.name} failed:`, err.message);
      // Continue to try the next one
    }
  }
  throw new Error('All Providers are down');
}

// ② Advanced version: with timeout + automatic retry
async function invokeWithResilience(input: any) {
  const providers = [
    { name: 'anthropic', model: primary, timeout: 30_000 },
    { name: 'openai', model: fallback1, timeout: 20_000 },
    { name: 'ollama', model: fallback2, timeout: 60_000 }, // Local is a bit slower
  ];

  for (const { name, model, timeout } of providers) {
    try {
      const controller = new AbortController();
      const timer = setTimeout(() => controller.abort(), timeout);

      const result = await model.invoke(input, { signal: controller.signal });
      clearTimeout(timer);
      return { provider: name, result };
    } catch (err) {
      console.warn(`[${name}] Failed: ${err.message}, switching to next`);
    }
  }
  throw new Error('All Providers are unavailable');
}

Cost considerations for Fallback: Fallbacks typically use cheaper/more stable alternatives (like local Ollama), not more expensive ones. Because when a fallback is triggered, it's already a degraded scenario—it just needs to work, not pursue quality. A typical three-tier fallback design is "Cloud Anthropic → Cloud OpenAI → Local Ollama".

---

2. Structured Output

Make the model return a verifiable object according to a Schema, instead of you guessing and extracting from free text.

2.1 Reasons for Needing Structured Returns

LLMs output free text by default—you ask "give me the user's name and age," and it might answer "Zhang San, 25 years old", "Name: Zhang San, Age: 25", "Zhang San (25)"... the format is completely non-deterministic.

If you need to feed this to a downstream program (store in a database, call another API, render a UI form), you have to write fragile regex/string parsing that breaks with any slight format change. This is the most common source of bugs in Agent systems.

The Structured Output mechanism lets you declare a Schema (using Zod / JSON Schema), the model guarantees to return data according to this structure, and you can directly JSON.parse it and access fields in a type-safe manner.

Method	Output Example	Parsing Difficulty
Free text	"Zhang San is 25 years old this year"	😩 Requires regex, fragile
Manual prompt for JSON	``` ```json\n{"name":"Zhang San","age":25}\n``` ```	😟 Model might wrap in markdown, needs cleaning
Structured Output	{"name":"Zhang San","age":25}	✅ Guaranteed valid JSON conforming to Schema

2.2 Standardizing Model Responses with Structured Returns

LangChain implements this via the withStructuredOutput(schema) method. Pass in a Zod schema, and it returns a new model instance whose invoke / stream methods directly return objects conforming to the schema (instead of AIMessage strings).

import { ChatAnthropic } from '@langchain/anthropic';
import { z } from 'zod';

// 1. Use Zod to define the output structure you want
const UserSchema = z.object({
  name: z.string().describe('The user\'s name'),
  age: z.number().int().min(0).max(150).describe('The user\'s age'),
  hobbies: z.array(z.string()).describe('The user\'s list of hobbies'),
});

// 2. Bind structured output
const model = new ChatAnthropic({ model: 'claude-sonnet-4-20250514' });
const structuredModel = model.withStructuredOutput(UserSchema);

// 3. Invoke: the return value is directly an object conforming to the Schema, no parsing needed
const result = await structuredModel.invoke([
  { role: 'user', content: 'My name is Zhang San, I am 25 years old, I like programming, hiking, and watching movies.' },
]);

// TypeScript knows the type of result is { name: string; age: number; hobbies: string[] }
console.log(result.name);     // "Zhang San"
console.log(result.age);      // 25
console.log(result.hobbies);  // ["programming", "hiking", "watching movies"]

Underlying principle: withStructuredOutput actually converts the schema into a vendor-specific "tool calling" format (e.g., OpenAI's function calling, Anthropic's tool_use), making the model "call" a virtual tool whose parameters are the structured data. Implementation details differ across vendors, but LangChain abstracts these differences away.

💡 .describe() is very important—it gets translated into the schema description, telling the model what each field should contain. The clearer the description, the more accurate the model's output.

2.3 Streaming Structured Output

stream returns string chunks by default, but if your schema is a large object (deeply nested, many fields), you can stream partial JSON—parsing as it generates, showing progress to the user in real-time:

import { ChatAnthropic } from '@langchain/anthropic';
import { z } from 'zod';
import { JsonOutputParser } from '@langchain/core/output_parsers';

// Define schema (assuming 5 fields)
const ArticleSchema = z.object({
  title: z.string(),
  outline: z.array(z.string()),
  sections: z.array(z.object({
    heading: z.string(),
    content: z.string(),
  })),
  summary: z.string(),
  keywords: z.array(z.string()),
});

const model = new ChatAnthropic({ model: 'claude-sonnet-4-20250514' });
const parser = new JsonOutputParser();

// Use stream + parser to parse as it generates
const stream = await model.stream([
  { role: 'user', content: 'Write an article about RAG' },
]);

let partial: any = {};
for await (const chunk of stream) {
  // JsonOutputParser incrementally merges JSON fragments
  partial = parser.parsePartialJson(chunk.content as string);
  console.log(`Generated so far: ${Object.keys(partial).join(', ')}`);
  // Display completed fields in real-time
}

Trade-offs of streaming structured output: For small schemas (3-5 fields), invoke is actually faster—the LLM generates the complete JSON in one go. For large schemas (10+ fields, deeply nested), streaming can significantly reduce time-to-first-token.

2.4 Error Boundary Handling

Structured output is not 100% reliable. Common failure scenarios:

Failure Scenario	Phenomenon	Handling Method
Model generates invalid JSON	JSON.parse throws an exception	try-catch + retry
Field type mismatch (age is string "25")	Zod validation fails	Zod auto-coerces some types, or explicitly use .or(z.string().transform(Number)) in the schema
Model adds extra fields	Extra {"name":"Zhang San","extra":"foo"}	Zod's default .strict() will reject; non-strict ignores extra fields
Model omits a required field	Missing age	Zod throws an error, catch it to set a default value for the field or retry

Robust encapsulation pattern:

async function safeStructuredInvoke<T>(
  model: any,
  schema: z.ZodSchema<T>,
  input: any,
  retries = 2,
): Promise<T | null> {
  const structured = model.withStructuredOutput(schema);
  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      const raw = await structured.invoke(input);
      // Zod secondary validation to ensure type safety
      return schema.parse(raw);
    } catch (err) {
      if (attempt === retries) {
        console.error(`Structured output failed (still failed after ${retries} retries):`, err);
        return null; // Degrade: return null, let the caller decide fallback logic
      }
      // Emphasize format requirements in the prompt during retries
      console.warn(`Structured output attempt ${attempt + 1} failed, retrying...`);
    }
  }
  return null;
}

// Usage
const UserSchema = z.object({
  name: z.string(),
  age: z.number(),
});
const user = await safeStructuredInvoke(model, UserSchema, [
  { role: 'user', content: 'My name is Li Si, 30 years old' },
]);
if (user) {
  console.log(user.name, user.age); // Type-safe
} else {
  // Degradation handling: prompt the user or fall back to free text path
}

Practical advice: Critical paths (like tool parameter parsing) must use structured output + retries; non-critical paths (like generating copy) can use free text + fault-tolerant parsing. In scenarios where an Agent calls tools, parameter parsing using withStructuredOutput + Zod validation is almost standard—ensuring parameter types are correct when the Agent calls a tool, so the tool implementation side doesn't need to do its own parameter validation.