跪拜 Guibai
← All articles
LangChain

AI App Stability Hinges on Data Boundaries, Not Model Calls

By BreezeJiang ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Most AI integration bugs are not model problems; they are data-boundary problems. Recognizing that a network chunk is not a JSON message, or that a text split must respect sentence endings, prevents the silent failures that make AI features feel flaky in production.

Summary

Three common AI application patterns — retrieval-augmented generation, streaming token display, and automated meeting minutes — share a single failure mode: irregular raw input that crosses processing boundaries. Web content must be split into semantically intact chunks before vector storage, or retrieval returns noise. Server-sent event streams arrive in network packets that do not align with JSON messages, so a buffer must hold incomplete lines across reads. Meeting transcripts need explicit rules for fact extraction and field population, because models will hallucinate missing owners or deadlines if allowed to guess.

The article walks through concrete JavaScript examples for each pipeline. A chunk size of 400 with Chinese sentence separators and 100-character overlap keeps retrieval context coherent. A buffer that splits on newlines and saves the trailing fragment prevents JSON parse failures from half-delivered SSE messages. A meeting-minutes Skill that forbids the model from inventing action-item fields produces reliable structured output from noisy transcripts.

Two practical gotchas surface: calling `new` on LangChain's `MemoryVectorStore.fromDocuments` throws a constructor error because it is a static factory method, and embedding API keys in a Vue frontend via `VITE_` variables exposes them in browser bundles. A four-layer debugging framework — input, boundary, transformation, external — keeps troubleshooting from wasting time on the wrong layer.

Takeaways
Chunk size, overlap, and language-appropriate separators determine whether RAG retrieves coherent context or fragmented noise.
One `reader.read()` call does not equal one SSE message or one complete JSON object; a buffer must accumulate partial lines across reads.
A meeting-minutes Skill should explicitly forbid the model from guessing missing owners, deadlines, or deliverables.
`MemoryVectorStore.fromDocuments` is a static factory method, not a constructor; calling `new` on it throws an error.
API keys passed via `VITE_` environment variables end up in browser bundles and are visible to anyone inspecting network requests.
A four-layer debugging order — input availability, boundary parsing, data transformation, external dependencies — prevents misdirected troubleshooting.
Conclusions

Streaming AI output is often taught as a frontend animation problem, but the real engineering is in reassembling byte streams across arbitrary network chunk boundaries.

The same boundary-discipline principle that fixes RAG chunking also fixes SSE parsing and structured output generation, suggesting a unified mental model for AI application reliability.

Explicitly forbidding a model from filling missing fields is more reliable than prompting it to be careful, because the failure mode shifts from probabilistic hallucination to deterministic emptiness.

Concepts & terms
SSE (Server-Sent Events)
A protocol where a server pushes a stream of text lines prefixed with `data: ` to a client over a single HTTP connection, commonly used by LLM APIs to deliver tokens as they are generated.
Half-packet and sticky packet
Network-level phenomena where a single application-layer message arrives split across multiple TCP reads (half-packet), or multiple messages arrive bundled in one read (sticky packet), requiring a buffer to reassemble logical message boundaries.
RAG chunk overlap
A text-splitting technique where adjacent chunks share a portion of text, preventing semantic meaning from being severed at an arbitrary cut point and improving retrieval relevance.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗