Streaming DeepSeek-R1 in the Browser: The Worker-to-UI Pipeline, Step by Step

Running DeepSeek-R1 in the Browser from Scratch: A Full-Stack Practical Guide with WebGPU + Transformers.js (Part 3)

This is the third article in the series, focusing on the complete communication link between the main thread (React UI) and the Worker thread (LLM inference). From the moment the user presses Enter to the model spitting out the first token, to streaming output, separating thinking from answering, and interrupt/reset—every line of code is explained thoroughly.

1. Recap and Positioning of This Article

In the previous two articles, we covered:

(1) Environment Setup: WebGPU browser compatibility detection, Transformers.js's ONNX Runtime Web backend, Vite engineering.
(2) Model Loading: The from_pretrained process for AutoTokenizer and AutoModelForCausalLM, singleton pattern management, and progress callbacks.

This article (Part 3) focuses on the core interaction between two files—App.tsx (main thread React UI) and worker.js (LLM inference in a Web Worker)—connecting the entire link.

2. Overall Architecture: Two Threads, One Protocol

┌──────────────────────────────────────────────────────────────────┐
│                        Browser Tab                               │
│                                                                   │
│  ┌─────────────────────────┐    postMessage    ┌────────────────┐ │
│  │     Main Thread (React)  │ ←──────────────→ │  Worker Thread  │ │
│  │                         │                   │                │ │
│  │  App.tsx                │   {type, data}    │  worker.js     │ │
│  │  - UI Rendering         │                   │  - LLM Inference│ │
│  │  - User Interaction     │                   │  - GPU Compute  │ │
│  │  - State Management     │                   │  - No DOM Access│ │
│  └─────────────────────────┘                   └────────────────┘ │
└──────────────────────────────────────────────────────────────────┘

Why use a Web Worker? Because LLM inference is a computationally intensive task. If placed on the main thread, the UI would completely freeze. The Worker runs in an independent thread, so scrolling, clicking, and animations on the main thread are unaffected.

Both ends communicate via postMessage. The message format follows a state machine protocol:

Direction	type	Meaning	Carried Data
Main→Worker	`check`	Check WebGPU availability	—
Main→Worker	`load`	Start downloading model	—
Main→Worker	`generate`	Start LLM generation	`data: messages[]`
Main→Worker	`interrupt`	User clicks stop	—
Main→Worker	`reset`	Reset conversation	—
Worker→Main	`status` series	Loading progress/Generation update/Completion/Error	See below

3. Main Thread Side: App.tsx State Machine

3.1 Core State Overview

const [status, setStatus] = useState(null);
// null → "loading" → "ready"  Three states drive the entire UI

const [messages, setMessages] = useState([]);
// Conversation history [{role, content}, ...]  Follows OpenAI message format

const [input, setInput] = useState("");
// Controlled input box text

const [isRunning, setIsRunning] = useState(false);
// Whether generation is in progress (controls button toggle and input disabling)

const worker = useRef(null);
// Worker instance stored in ref, does not trigger re-renders

status is the global state machine, driving three views:

status = null      →  Welcome page (Logo + Load model button)
status = "loading" →  Progress bar page (shows model file download progress)
status = "ready"   →  Chat page (message list + input box)

3.2 Birth of the Worker: useRef + useEffect

useEffect(() => {
    if (!worker.current) {
        worker.current = new Worker(
            new URL("./worker.js", import.meta.url),
            { type: "module" }
        );
        worker.current.postMessage({ type: "check" });
    }

    const onMessageReceived = (e) => { /* Handle messages from Worker */ };

    worker.current.addEventListener("message", onMessageReceived);
    return () => worker.current.removeEventListener("message", onMessageReceived);
}, []);  // ← Empty dependency array, executes only once on component mount

Key design points:

useRef instead of useState: The Worker instance doesn't need to drive rendering; storing it in a ref avoids unnecessary re-renders.
type: "module" : Allows the Worker to support ES Modules (import/export), compatible with the Vite build system.
[] dependency: The entire effect runs only once, the Worker is created only once, and is never destroyed and recreated.
Cleanup function: Removes the event listener when the component unmounts to prevent memory leaks.

3.3 Message Receiving: From Worker to UI

const onMessageReceived = (e) => {
    switch (e.data.status) {
        case "loading":   // Model loading, update status text
        case "initiate":  // A file starts downloading, initialize progress bar
        case "progress":  // A file downloading, update progress percentage
        case "done":      // A file download complete, remove progress bar
        case "ready":     // All ready, enter chat interface
        case "start":     // Generation started (first streaming signal)
        case "update":    // Streaming text arrived, append to conversation
        case "complete":  // Generation finished
        case "error":     // An error occurred
    }
};

loading → initiate → progress → done → ready is a chain. Why use functional updates (prev) => for progress and done?

// ❌ Direct reference might read stale value
setProgressItems([...progressItems, e.data]);

// ✅ Functional ensures based on latest snapshot
setProgressItems((prev) => [...prev, e.data]);

Because multiple model files download concurrently, progress callbacks trigger extremely fast. React's batching updates could cause some progress updates to be lost. The functional approach ensures each append is based on the latest state.

4. From Enter to Generation: The Double Safety Trigger

4.1 onEnter: User Presses Enter

function onEnter(message) {
    setMessages((prev) => [...prev, { role: "user", content: message }]);
    setInput("");          // Clear input box
    setIsRunning(true);    // Lock UI, send button becomes stop button
}

Four things happen in one go: ① Immediately append a user bubble to the chat list (instant UI feedback), ② Clear the input box, ③ Lock the button to prevent duplicate sends, ④ Wait for useEffect to trigger the next step.

4.2 useEffect Double Guard: Precisely Triggering generate

useEffect(() => {
    // Guard ①: Don't trigger if there are no user messages
    if (messages.filter((x) => x.role === "user").length === 0) {
        return;
    }
    // Guard ②: Don't trigger if the last message is already from the AI
    if (messages.at(-1).role === "assistant") {
        return;
    }
    // Both guards passed → Last message is from user → Trigger generation
    worker.current.postMessage({ type: "generate", data: messages });
}, [messages]);

What do the two guards prevent?

Guard	Intercepted Scenario	Typical Moment
`filter(user).length === 0`	No user messages	Page just loaded, after reset
`.at(-1).role === "assistant"`	Last message is AI message	Every `setMessages` during streaming output, after generation completes

Guard ② is the most critical. During LLM streaming output, every time a text fragment is received, the main thread calls setMessages to update the assistant message content. messages is an array reference, and each setMessages creates a new array → [messages] dependency detects a change → effect re-executes. Without Guard ②, every streaming update would send another generate, immediately causing an infinite loop.

5. Worker Side: generate() Full Link Breakdown

5.1 Message Routing

self.addEventListener("message", async (e) => {
    const { type, data } = e.data;
    switch (type) {
        case "check":     check();    break;
        case "load":      load();     break;
        case "generate":  stopping_criteria.reset();
                          generate(data);  break;
        case "interrupt": stopping_criteria.interrupt();  break;
        case "reset":     past_key_values_cache = null;
                          stopping_criteria.reset();      break;
    }
});

The Worker is essentially a single-threaded event loop. Upon receiving a message from the main thread, it dispatches to different handler functions based on type. Note that reset() is called before generate—clearing any interrupt flag that might be left over from the previous round.

5.2 Step 1: Get tokenizer and model

const [tokenizer, model] = await TextGenerationPipeline.getInstance();

The singleton pattern behind it:

class TextGenerationPipeline {
    static model_id = "onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX";

    static async getInstance(progress_callback = null) {
        this.tokenizer ??= AutoTokenizer.from_pretrained(this.model_id, {
            progress_callback,
        });
        this.model ??= AutoModelForCausalLM.from_pretrained(this.model_id, {
            dtype: "q4f16",     // 4-bit quantization, saves memory
            device: "webgpu",   // Runs on GPU
            progress_callback,
        });
        return Promise.all([this.tokenizer, this.model]);
    }
}

The ??= operator ensures no matter how many times it's called, it only downloads once. On the first call, this.tokenizer and this.model are both undefined (nullish), triggering the download. Subsequent calls directly return the instances already stored on the static properties.

dtype: "q4f16" is ONNX Runtime's 4-bit quantization format, compressing FP16 model weights to 4-bit, reducing memory usage to 1/4 of the original. This is how a 1.5B parameter model can run within the limited memory of browser WebGPU.

5.3 Step 2: Chat Template to token IDs

const inputs = tokenizer.apply_chat_template(messages, {
    add_generation_prompt: true,
    return_dict: true,
});

This step converts a JSON message array like this:

[{ "role": "user", "content": "What is 1+1?" }]

Into the special token format text used during model training:

<|im_start|>user
What is 1+1?<|im_end|>
<|im_start|>assistant

And then tokenizes it into a numerical array:

inputs = {
    input_ids: [151644, 8948, 198, 16, 18, 16, 41118, 17564, 16199, 151645, 198, 151644, 77091, 198],
    attention_mask: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
}

Why not use tokenizer("raw text") to manually concatenate? Because each model has different special tokens—Llama uses <|begin_of_text|>, Qwen uses <|im_start|>, ChatML uses another set. apply_chat_template reads the correct template from the model's tokenizer_config.json, so it never goes wrong.

Roles of the three parameters:

Parameter	Role
`add_generation_prompt: true`	Appends `<
`return_dict: true`	Returns a `{input_ids, attention_mask}` object, directly feedable to `generate()`
Without `add_generation_prompt`	Suitable for training scenarios, where a complete assistant response already exists

5.4 Step 3: Extract Thinking Tag Token IDs

const [START_THINKING_TOKEN_ID, END_THINKING_TOKEN_ID] = tokenizer.encode(
    " thinking response",
    { add_special_tokens: false },
);

DeepSeek-R1's inference mode is a thinking→answering two-stage process:

 thinking
What is 1+1? This is a simple addition problem...       ← Thinking process (model talking to itself)
The answer is 2.
 response

1+1 equals 2.                                 ← Final answer

To distinguish the two stages in real-time during streaming generation, we need to know the token IDs corresponding to <> and </>:

// Encoding result illustration:
// "<", "think", ">", "<", "/", "think", ">"
// [27,  17845,  29,  27, 1526, 17845,  29]
//  ↑                              ↑
//  Destructure takes the first    Destructure takes the last
//  START = 27                     END = 29

add_special_tokens: false is key—no BOS/EOS token needed, just the pure tag token IDs.

5.5 Step 4: State Tracking Variables + Dual Callbacks

let state = "thinking";    // 'thinking' | 'answering'
let startTime;             // Performance timing start point
let numTokens = 0;         // Total tokens generated
let tps;                   // Tokens per second

const token_callback_function = (tokens) => {
    startTime ??= performance.now();              // Start timing on first token
    if (numTokens++ > 0) {                        // Calculate TPS from the 2nd token onwards
        tps = (numTokens / (performance.now() - startTime)) * 1000;
    }
    if (tokens[0] == END_THINKING_TOKEN_ID) {     // Detected the last character of </>
        state = "answering";
    }
};

const callback_function = (output) => {
    self.postMessage({
        status: "update",
        output,        // Decoded human-readable text fragment
        tps,           // Real-time speed
        numTokens,     // Count
        state,         // Current stage
    });
};

Division of labor between the two callbacks:

Each token ID generated
  │
  ├─→ token_callback_function
  │     ├─ Record timestamp (only once)
  │     ├─ numTokens++ and calculate TPS
  │     └─ Detect if entering "answering" stage
  │
  └─→ TextStreamer internal accumulation → Decoding
        └─→ callback_function
              └─ postMessage("update") push to main thread

token_callback is token-granularity (triggers on every one), responsible for background statistics. callback_function is text-granularity (triggers after accumulating a segment), responsible for foreground pushing. This separated design keeps the frequency of postMessage received by the main thread controllable, avoiding sending a message for every single token.

5.6 Step 5: Assemble TextStreamer

const streamer = new TextStreamer(tokenizer, {
    skip_prompt: true,          // Skip input part, only output newly generated
    skip_special_tokens: true,  // Filter out control tokens like <|im_end|>
    callback_function,          // Text-level callback
    token_callback_function,    // Token-level callback
});

TextStreamer is a built-in streaming decoder in Transformers.js, architecturally a callback-driven pipeline:

Token ID → [token_callback statistics] → [internal buffer] → [tokenizer.decode] → [callback_function output]

skip_prompt: true ensures the user doesn't see the entire conversation template being "typed out" again, only the new content. skip_special_tokens: true filters out special control tokens like <|im_start|>, <|im_end|>.

5.7 Step 6: Call model.generate()

self.postMessage({ status: "start" });

const { past_key_values, sequences } = await model.generate({
    ...inputs,                    // input_ids + attention_mask
    do_sample: false,             // Greedy decoding (deterministic output)
    max_new_tokens: 2048,         // Max 2048 new tokens generated
    streamer,                     // Streaming callback
    stopping_criteria,            // Interruptable stop condition
    return_dict_in_generate: true,// Return KV Cache
});

Detailed explanation of parameters:

...inputs: Spreads the {input_ids, attention_mask} returned by apply_chat_template.

do_sample: false: Greedy Decoding. At each step, directly selects the token with the highest probability, without random sampling. For mathematical reasoning tasks (DeepSeek-R1's strength), greedy decoding guarantees consistency and determinism in answers. For creative writing, sampling should be enabled (do_sample: true + temperature + top_p).

max_new_tokens: 2048: Only counts newly generated tokens, excluding the input prompt. Automatically stops when the limit is reached, acting like a safety valve.

streamer: Attaches the TextStreamer instance created above. Without it, model.generate() would return everything at once after full generation, resulting in a user experience of "wait 10 seconds, then suddenly the full text appears."

stopping_criteria: Passes in the InterruptableStoppingCriteria instance. The internal generation loop of model.generate() calls the _call() method at each step, checking the this.interrupted flag. When the user clicks the stop button, the flag becomes true, and the loop terminates immediately.

return_dict_in_generate: true: In addition to sequences (the generated token sequence), also returns past_key_values (KV Cache), stored for reuse in the next round.

5.8 Step 7: Save KV Cache + Final Decoding

past_key_values_cache = past_key_values;

const decoded = tokenizer.batch_decode(sequences, {
    skip_special_tokens: true,
});

self.postMessage({
    status: "complete",
    output: decoded,
});

past_key_values is saved to a module-level variable, passed in for the next generate call, saving redundant computation. The principle of KV Cache is explained separately below.

batch_decode converts the complete token ID sequence back to readable text in one go, skipping special tokens. This is the final complete result, complementing the streaming update:

Message	Timing	Content
`update`	During generation (N times)	Text fragments, appended piece by piece
`complete`	End of generation (1 time)	Complete text after decoding all tokens

6. KV Cache: Attention Acceleration via Space-Time Tradeoff

6.1 Background Problem

Every time a Transformer generates a new token, that token must "attend" to all previous tokens. Without caching:

Step 1: Compute K₁V₁
Step 2: Recompute K₁V₁ + Compute K₂V₂        ← Recomputing!
Step 3: Recompute K₁V₁K₂V₂ + Compute K₃V₃    ← Recomputing again!
...
Step N: Recompute KV of first N-1 pairs + Compute KₙVₙ ← O(n²)!

6.2 KV Cache Solution

The K and V of previous tokens do not change—just store them:

Step 1: Compute K₁V₁ → Store in Cache: [K₁V₁]
Step 2: Only compute K₂V₂ → Append Cache: [K₁V₁|K₂V₂]
Step 3: Only compute K₃V₃ → Append Cache: [K₁V₁|K₂V₂|K₃V₃]
...
Each step only computes 1 KV pair, O(n)

6.3 Manifestation in Code

// Module top-level declaration
let past_key_values_cache = null;

// Saved in generate() (Step 6)
const { past_key_values } = await model.generate({ ... });
past_key_values_cache = past_key_values;

// Cleared in reset
case "reset":
    past_key_values_cache = null;
    stopping_criteria.reset();
    break;

Why set to null on reset?

Isolate conversations: Old KV Cache stores attention information from the previous conversation. Not clearing it would cause the new conversation to "cross-talk".
Free memory: For WebGPU (running in the browser), memory is extremely precious. The KV Cache for a 1.5B parameter model generating 2048 tokens can occupy 350MB+.
State reset: A non-null past_key_values would make the model mistakenly think this is "continuing generation" rather than "starting from scratch".

7. Interrupt Mechanism: InterruptableStoppingCriteria

// Created once at module top level
const stopping_criteria = new InterruptableStoppingCriteria();

// User clicks stop button
case "interrupt":
    stopping_criteria.interrupt();  // this.interrupted = true
    break;

Source code (from Transformers.js):

class InterruptableStoppingCriteria extends StoppingCriteria {
    constructor() { super(); this.interrupted = false; }
    interrupt()   { this.interrupted = true; }
    reset()       { this.interrupted = false; }
    _call(input_ids, scores) {
        return new Array(input_ids.length).fill(this.interrupted);
    }
}

Mechanism:

model.generate() internal loop
  │
  ├─ Generate token
  ├─ _call() → Check interrupted flag
  │     ├─ false → Continue loop
  │     └─ true  → Terminate immediately    ← interrupt() triggered
  │
  └─ ...repeat

Why placed at module top level? Because the internal loop of model.generate() and the interrupt message handler must share the same object reference. If placed inside a function, each call to generate() would create a new instance, and interrupt() would modify the interrupted field of a different object.

8. Complete Sequence: From Enter to Completion

Stringing the entire link together:

User presses Enter
    │
    ├── onEnter("What is 1+1?")
    │     ├── setMessages(prev => [...prev, {role:"user", content:"What is 1+1?"}])
    │     ├── setInput("")
    │     └── setIsRunning(true)
    │
    ├── React re-renders
    │     └── useEffect([messages]) triggers
    │           ├── Guard ①: filter(user).length > 0 ✅
    │           ├── Guard ②: at(-1).role !== "assistant" ✅
    │           └── worker.current.postMessage({type:"generate", data:messages})
    │
    ├── ──────────── Cross-thread boundary ────────────
    │
    ├── Worker: case "generate"
    │     ├── stopping_criteria.reset()      // Release the brake first
    │     └── generate(messages)
    │           │
    │           ├── ① getInstance() → tokenizer + model
    │           ├── ② apply_chat_template(messages) → input_ids + attention_mask
    │           ├── ③ encode(" thinking response") → START/END ID
    │           ├── ④ Initialize state/tps/numTokens
    │           ├── ⑤ new TextStreamer(tokenizer, {...})
    │           ├── ⑥ postMessage({status:"start"})
    │           ├── ⑦ await model.generate({...})
    │           │      │
    │           │      ├── [Token loop starts]
    │           │      │    ├── token_callback: timing/counting/state detection
    │           │      │    └── streamer → callback_function → postMessage("update")
    │           │      │         ↓ To main thread → setMessages updates UI
    │           │      └── [Generation ends]
    │           │
    │           ├── ⑧ past_key_values_cache = past_key_values
    │           └── ⑨ batch_decode → postMessage("complete")
    │
    ├── ──────────── Cross-thread boundary ────────────
    │
    └── Main thread handles complete
          ├── setMessages appends final content
          ├── setIsRunning(false) → Stop button reverts to send button
          └── UI becomes available again

9. Summary

This article covered the complete interaction link of App.tsx + worker.js, with the core points being:

Dual-thread architecture: Main thread manages UI, Worker manages inference, communicating via postMessage.
Singleton pattern: The ??= operator ensures tokenizer and model are downloaded/initialized only once.
Chat Template: apply_chat_template automatically handles different models' special token formats; add_generation_prompt tells the model "it's your turn to speak".
Dual-callback streaming output: token_callback does background statistics (timing/counting/stage detection), callback_function does foreground pushing (text→main thread→UI).
Two useEffect guards: Prevent triggering generation on empty conversations and prevent duplicate triggers during streaming updates.
KV Cache: Space-time tradeoff, O(n²)→O(n), must be cleared on reset to prevent memory leaks.
InterruptableStoppingCriteria: Module-level singleton, interrupt() and model.generate() share the same boolean flag to achieve emergency stop.