Frontend · Backend · JavaScript

ONNX Runtime Web Puts YOLO, Whisper, and BERT Directly in the Browser

By 半刻维度 · Aug 27, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Client-side inference eliminates server costs, latency, and privacy risks for a wide class of AI features. With onnxruntime-web as the shared runtime under Transformers.js, MediaPipe, and face-api.js, a single dependency unlocks vision, speech, and text models that run offline in any browser.

Summary

onnxruntime-web compiles ONNX models to WebAssembly via Emscripten, turning any modern browser into an inference engine. A single import from a CDN and a few lines of JavaScript load a model, create a tensor, and run inference — no server round-trips, no Python environment. The library carries 23,000 GitHub stars and ships with official demos for YOLOv5 object detection, Whisper speech recognition, and Real-ESRGAN image upscaling.

The ONNX model zoo supplies pre-exported weights for MobileNet, ResNet, SSD, DeepLabV3, BERT, GPT-2, and more, contributed by Microsoft, Tencent, and Megvii. Because ONNX is framework-agnostic, models trained in PyTorch or TensorFlow drop into the same runtime without conversion friction.

Major browser-AI projects already build on this foundation. HuggingFace's Transformers.js runs BERT and GPT pipelines through onnxruntime-web. Google's MediaPipe Web uses an ONNX inference layer for face, pose, and gesture recognition. face-api.js, at 53,000 stars, relies on the same ONNX runtime underneath. The ecosystem means a developer can pick up a pre-trained model and have it running client-side in minutes.

Takeaways

— onnxruntime-web loads ONNX models in the browser, Node.js, or Bun with zero backend and zero Python.

— A YOLOv5 object-detection session requires only an import from jsDelivr, a tensor from pixel data, and a single `session.run()` call.

— The ONNX model zoo includes MobileNet (14 MB), SqueezeNet (5 MB), ResNet, YOLOv5, SSD, DeepLabV3, Whisper, BERT, and GPT-2, all pre-exported and ready to use.

— Under the hood, ONNX Runtime Web compiles to WebAssembly via Emscripten, making the browser the inference engine.

— Transformers.js from HuggingFace, Google's MediaPipe Web, and face-api.js all use onnxruntime-web as their inference layer.

— Real-ESRGAN runs 4× image super-resolution entirely in the browser through an ONNX model.

— Whisper Web, an official Microsoft project, demonstrates speech-to-text with a tiny ONNX model loaded client-side.

Conclusions

onnxruntime-web has quietly become the de facto inference substrate for browser AI, not because it was marketed heavily, but because it solves the one problem every framework hits: running models without a server.

The ONNX model zoo turns model selection into a shopping exercise — pick a task, grab the ONNX file, and wire it up. This commoditizes what used to require framework-specific serving infrastructure.

With Transformers.js, MediaPipe, and face-api.js all converging on the same runtime, the browser AI stack is consolidating around a single WASM-based execution layer, which simplifies bundling and reduces duplicate model downloads across libraries.

Concepts & terms

ONNX (Open Neural Network Exchange)

An open format for representing machine learning models. Models trained in PyTorch, TensorFlow, or other frameworks can be exported to ONNX and then run with any ONNX-compatible runtime, decoupling training from deployment.

onnxruntime-web

Microsoft's JavaScript library that loads ONNX models and executes them in the browser or in Node.js/Bun. It compiles the native ONNX Runtime to WebAssembly using Emscripten, enabling GPU-accelerated inference via WebGL or WebGPU.

WebAssembly (WASM) + Emscripten

Emscripten compiles C/C++ code (here, the ONNX Runtime engine) into WebAssembly, a low-level binary format that browsers can execute at near-native speed. This is what lets a full ML inference engine run without a server.

From the discussion

The discussion is thin, with only one substantive contribution. That comment flags a silent WebGPU-to-WASM fallback in onnxruntime-web when executionProviders isn't explicitly set, which misleads performance debugging because the backend still claims webgpu. The other two remarks are drive-by questions about use cases and a non-sequitur about Deno/Bun runtimes.

— Without an explicit executionProviders:['webgpu'] setting, onnxruntime-web can silently fall back to WASM mid-call while the backend continues to report webgpu, corrupting performance investigations.

Featured comments

szp2005

Filling in a gap: ORT Web also has a WebGPU EP. If you don't explicitly write executionProviders:['webgpu'] when creating, it can silently fall back to WASM within the same call, while the backend still reports webgpu, completely throwing off performance debugging.

See top comments, translated →

Source: juejin.cn ↗ Google Translate ↗ Backup ↗