LLM · DeepSeek · Artificial Intelligence

The Message Protocol That Runs a Local LLM Inside a Browser Worker

By 烬羽 · Aug 12, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Running an LLM locally in the browser demands a messaging architecture that handles streaming, cancellation, and state synchronization across threads. Getting the protocol right determines whether the UI stays responsive and whether multi-turn conversations run at acceptable speed.

Summary

A full inference engine for DeepSeek-R1-Distill-Qwen-1.5B executes entirely in a browser Web Worker, communicating with the main thread through a 13-message protocol. The design routes every lifecycle event—WebGPU detection, model download progress, token-by-token streaming, interruption, and context reset—through typed postMessage calls. A dual-callback TextStreamer separates raw token-ID processing for performance stats and state detection from decoded text delivery for UI rendering. KV-cache reuse cuts multi-turn conversation latency by avoiding repeated attention computation, while an InterruptableStoppingCriteria flag lets users halt generation mid-stream without losing already-displayed tokens. A two-state machine tracks DeepSeek's thinking/answering phases by matching exact token IDs, giving the UI a reliable signal to style reasoning output differently from final answers.

Takeaways

— Model inference must run in a Web Worker; calling model.generate() on the main thread blocks DOM rendering and can trigger a browser 'page unresponsive' dialog.

— Worker instances should be stored in useRef, not useState, to avoid creating new threads on every render and leaking memory.

— The message protocol grew from 5 to 13 types organically—each new message was added when the main thread lacked visibility into a Worker state.

— apply_chat_template preserves the exact training format of DeepSeek-R1; manually concatenating prompt strings degrades output quality.

— TextStreamer provides two callbacks: token_callback receives raw token IDs for zero-cost TPS calculation and state detection, while callback_function receives decoded text for UI updates.

— KV-cache reduces multi-turn attention computation from O(N²) to O(N) by reusing historical Key/Value matrices, making continued conversations dramatically faster than fresh ones.

— InterruptableStoppingCriteria implements cooperative interruption by checking a flag before each token generation step; already-generated tokens are preserved and displayed.

— A two-state machine (thinking/answering) detects DeepSeek's response token ID to switch modes, avoiding fragile regex parsing on decoded text.

— React's useTransition cannot replace a Web Worker because it only deprioritizes state updates—synchronous blocking operations still freeze the main thread.

Conclusions

Evolving a message protocol from 5 to 13 types by adding messages whenever the main thread lacked state visibility is a practical counterpoint to upfront UML design—the protocol emerged from runtime gaps, not a whiteboard.

Using token IDs instead of decoded text for state detection (thinking vs. answering) eliminates an entire class of parsing bugs; a token ID match is exact and costs nothing, while regex on strings is fragile and adds overhead.

The cooperative interruption pattern—checking a flag rather than killing a thread—is the only viable approach inside a Worker, and it naturally preserves partial results that have already been pushed to the UI.

Storing the Worker reference in useRef rather than useState is a concrete React pattern that prevents memory leaks from abandoned Worker threads, yet many tutorials still get this wrong.

Concepts & terms

KV-Cache

A cache that stores the Key and Value matrices computed during Transformer attention for all previous tokens. Without it, each new token requires recomputing attention over the entire sequence (O(N²) cost). With it, only the new token's attention is computed (O(N) cost), dramatically speeding up multi-turn conversations.

TextStreamer

A Transformers.js utility that wraps model.generate() to emit tokens one at a time through callbacks. It provides two hooks: a token callback that receives raw token IDs before decoding, and a text callback that receives the accumulated decoded string, enabling both performance monitoring and real-time UI updates.

InterruptableStoppingCriteria

A cooperative interruption mechanism for model generation. Rather than forcefully killing a thread (impossible in a Worker), it sets an internal flag that the generation loop checks before producing each new token. Already-generated tokens remain available to the UI.

apply_chat_template

A tokenizer method that formats a messages array into the exact prompt structure the model was trained on, including special tokens like <|im_start|> and <|im_end|>. Manual string concatenation produces format mismatches that degrade output quality.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗