Loading a 1.5B-Parameter LLM in the Browser with WebGPU and Transformers.js
Running a 1.5B-parameter reasoning model in a browser with no backend server eliminates the Python/CUDA dependency that gatekeeps local LLM experimentation. The Worker-thread pattern and progress-callback pitfall documented here are directly reusable for any browser-side ML project.
The dpsk-webgpu project demonstrates how to load the ONNX-converted DeepSeek-R1-Distill-Qwen-1.5B model directly in a browser tab. It uses React 19, TypeScript, and Tailwind CSS for the UI, with transformers.js as the sole dependency. Model weights are cached in the browser's Cache Storage, so subsequent loads skip the download entirely.
A Web Worker handles all heavy lifting—WebGPU detection, model downloading, and eventually inference—while the main thread manages only UI state. The two communicate through a simple postMessage protocol with status codes like 'webgpu-check', 'download', and 'ready'. A progress bar driven by transformers.js's progress_callback gives real-time download feedback, with a specific warning: only messages with status 'progress' carry a numeric percentage; the 'download' status is just a start notification and will break the bar if used.
The project's generate function is currently a stub that echoes input. The infrastructure—Worker channel, state management, GPU detection, and progress reporting—is complete and ready for real inference via tokenizer.encode and model.generate.
Browser-side LLM loading is now practical enough that the infrastructure—Worker channels, progress reporting, GPU detection—can be treated as a solved pattern and reused across projects.
The progress_callback pitfall with transformers.js is a sharp edge that will silently break any UI relying on download progress; the library's status naming ('download' vs 'progress') is misleading enough to warrant a warning in its own documentation.
Using a static singleton pattern for the pipeline inside a Worker ensures the model and tokenizer are loaded exactly once, which matters when the Worker stays alive for the page session.