Frontend · React.js · TypeScript

Streaming DeepSeek-R1 in the Browser: The Worker-to-UI Pipeline, Step by Step

By dzhd · Aug 12, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Running an LLM entirely in the browser sidesteps server costs, latency, and privacy concerns, but the main-thread/Worker coordination is where most prototypes break. This walkthrough surfaces the exact guards, singleton patterns, and streaming callbacks that prevent UI freezes, infinite loops, and memory leaks.

Summary

The third installment of a browser-based LLM series dissects the full communication pipeline between a React frontend and a Web Worker running DeepSeek-R1 via Transformers.js and WebGPU. A state machine in App.tsx drives three UI views—welcome, loading, and chat—while a singleton pattern ensures the tokenizer and 4-bit quantized model download only once. Two useEffect guards prevent the infinite generation loop that would otherwise fire on every streaming update.

Inside the Worker, message routing dispatches check, load, generate, interrupt, and reset commands. The generate path applies a chat template to convert OpenAI-format messages into model-specific tokens, then streams output through a dual-callback TextStreamer: a token-level callback tracks speed and detects the transition from thinking to answering, while a text-level callback pushes decoded fragments back to the UI. KV Cache is saved between turns to cut attention computation from O(n²) to O(n) and cleared on reset to free GPU memory.

An InterruptableStoppingCriteria singleton, shared between the message handler and the generation loop, lets a user click stop and immediately halt token production. The complete sequence from pressing Enter to the final decoded output is traced end-to-end.

Takeaways

— Web Workers keep LLM inference off the main thread so scrolling and clicks stay responsive during generation.

— A single useEffect with two guards—no user messages present, and last message already from the assistant—prevents duplicate generate calls during streaming.

— The `??=` operator in a static async getInstance method downloads the tokenizer and model exactly once, no matter how many times it is called.

— `apply_chat_template` with `add_generation_prompt: true` converts OpenAI-format messages into model-specific token sequences and appends the assistant prompt marker.

— Encoding the string " thinking response" without special tokens yields the start and end token IDs used to split the stream into thinking and answering phases.

— A token-level callback handles timing and TPS calculation on every token, while a separate text-level callback batches decoded fragments for UI updates, keeping postMessage frequency manageable.

— KV Cache reduces per-step attention computation from O(n²) to O(n) and must be set to null on reset to isolate conversations and free WebGPU memory.

— InterruptableStoppingCriteria must be a module-level singleton so the interrupt message handler and the generate loop share the same boolean flag.

Conclusions

Browsing the full sequence diagram reveals that the entire streaming UX hinges on a single `useEffect` dependency array and two early returns—remove either guard and the app either never generates or loops infinitely.

The dual-callback TextStreamer design is a pragmatic split: raw token statistics stay cheap and synchronous, while decoded text pushes are batched to avoid flooding the main thread with postMessage calls.

Storing the Worker instance in a useRef rather than useState is a small decision with outsized impact; putting it in state would trigger re-renders on every message, degrading UI performance during streaming.

Using `??=` for model singleton initialization is elegant but brittle—any code path that resets the static properties would trigger a re-download with no recovery path visible in this architecture.

Concepts & terms

KV Cache

A cache that stores the Key and Value tensors from previous attention steps during autoregressive generation. Without it, each new token would recompute attention over all prior tokens (O(n²) cost); with it, only the new token's KV pair is computed (O(n) cost). Must be cleared between independent conversations to prevent cross-talk and free memory.

Greedy Decoding

A text generation strategy that always selects the token with the highest probability at each step, producing deterministic output. Contrasts with sampling methods (temperature, top-p, top-k) that introduce randomness. Preferred for tasks like math reasoning where consistency matters more than creativity.

Chat Template

A model-specific formatting rule (stored in tokenizer_config.json) that converts a generic messages array like [{role: 'user', content: '...'}] into the exact special-token structure the model was trained on—e.g., <|im_start|>user\n...<|im_end|> for Qwen models. `apply_chat_template` automates this so developers never hand-craft model-specific prompt strings.

InterruptableStoppingCriteria

A Transformers.js class that extends StoppingCriteria with a boolean `interrupted` flag. The generate loop calls `_call()` on each step; when the flag is true, the loop terminates. Must be a singleton shared between the interrupt message handler and the generate call so both reference the same flag.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗