Baidu's 6 MB OCR Model Runs Entirely in the Browser with No Backend

40 years old, still doing development, just grinding a bit more, sigh

I wanted to add an OCR feature to an internal system where users upload screenshots and the text is automatically recognized. Then I discovered Baidu's PaddleOCR.

After all, a dedicated OCR model has a better recognition rate than a large model.

The traditional approach: set up a Python service on the backend, install PaddleOCR, deploy Flask. Not only is it troublesome, but you also have to consider concurrency and resource usage.

I just thought: can the browser do this job itself?

This is my demo, the complete code is at the end.

1. What is onnxruntime-web

ONNX Runtime is an open-source inference engine by Microsoft, and onnxruntime-web is its browser version.

Under the hood, it uses WebGL or WebGPU, so your graphics card is working for AI inside the browser.

Integration is one line:

If the project uses npm:

npm install onnxruntime-web

Same with bun:

bun add onnxruntime-web

After installation, create src/index.ts as the entry point:

import * as ort from 'onnxruntime-web';
ort.env.wasm.wasmPaths = '/ort/';
(window as any).ort = ort;

Use a build tool to package it into the static directory:

# bun
bun build ./src/index.ts --outfile=./static/ort.js --target=browser

# Or use webpack / vite, both work

Reference the built artifact in the HTML:

<script src="ort.js"></script>

Load the model, prepare the input, run inference, get the result — four steps, no server-side, all in the browser.

2. Download the Model

Download the ONNX model for PP-OCRv6 using ModelScope, which is fast in China:

Search for PP-OCRv6_tiny on ModelScope.

Then download the corresponding onnx files.

Explain what these two files do:

PP-OCRv6_tiny_det_onnx: Text Detection Model. Core function: Precisely locate and mark the positions of all text regions in an image, usually outlined with bounding boxes.

Main features: This model belongs to the tiny series, designed specifically for edge and IoT scenarios, pursuing extreme lightness and high speed. The model has only 430,000 parameters (0.43M) and a size of about 1.9 MB.

Application scenarios: Excels at handling text in various complex scenarios such as handwriting, printing, rotation, curves, and artistic fonts.

PP-OCRv6_tiny_rec_onnx: Text Recognition Model. Core function: Receives the text region images provided by the detection model and converts the visual information within them into electronic text that can be edited and searched by a computer.

Main features: Also part of the tiny series, it is the lightest recognition model in PP-OCRv6. The model has 1.1 million parameters (1.1M), a size of about 4.4 MB, and supports recognition in 49 languages.

Application scenarios: Efficiently recognizes Simplified Chinese, Traditional Chinese, English, Japanese, etc., and can handle complex texts like handwriting, vertical text, pinyin, and rare characters.

static/
  models/
    PP-OCRv6_det_tiny.onnx
    PP-OCRv6_rec_tiny.onnx

3. OCR Pipeline

OCR is not a single model; it's a series connection of a text detection model and a text recognition model.

I chose PP-OCRv6, Baidu's latest open-source OCR model, which comes in three sizes: tiny, small, and medium.

Upload Image
  → Resize to within 960 (multiple of 32)
  → Detection model inference, get probability map
  → Post-processing: Binarization → Connected Components → Text Boxes
  → Crop each text box
  → Resize to 48xW
  → Recognition model inference
  → CTC Decoding → Get text

The tiny version's detection model is 1.74 MB, and the recognition model is 4.28 MB, totaling just over 6 MB, which loads very quickly in the browser.

4. Image Preprocessing

The detection model requires input of a specific size: the longest side must not exceed 960, and it must be a multiple of 32.

Why 32? Because the model has downsampling layers with a stride of 32 internally; an input size that is not a multiple of 32 will cause output truncation.

Preprocessing involves three things:

The parameters come from the official model configuration, not chosen arbitrarily.

5. Text Detection: DBNet Post-processing

The detection model outputs a probability map, where each pixel represents the probability that "this is text."

Post-processing is divided into three steps:

Binarization: Pixels with a value greater than 0.2 are marked as text, otherwise marked as background.

Connected-component labeling: Use BFS to connect adjacent white points into a single area. Each connected component is a text region.

Unclip expansion: The boundaries output by the model are usually slightly smaller than the actual text. Expand outward by the ratio of area × coefficient / perimeter, with a coefficient of 1.4.

Filter out regions that are too small (short side less than 3 pixels) and regions with too low confidence.

Sort by y-coordinate to ensure reading order.

6. Text Recognition: CRNN + CTC Decoding

Each text box is cropped out and scaled to a height of 48 pixels, with the width kept proportional.

The recognition model outputs [1, T, C], where T is the number of time steps and C is the number of character classes (6906).

Use CTC greedy decoding to turn probabilities into text:

At each time step, take the index with the highest probability, remove blanks (index 0), and merge consecutive duplicate indices. Mapping these to the character set yields the text.

For example: output [15, 15, 15, 0, 0, 23, 23, 0, 5] → merge and deduplicate → [15, 23, 5] → map to character set → "你好".

7. Pitfall: Garbled Recognition Output

When I first ran the test, the results were like this:

#1 沼桷轻哔茸藩舅爪锵喇
#2 ，辜唤
#3 沼桷轻哔暖敬茸藩舅爪锵喇梓备字蹿心航瞠郊

All garbled.

The problem was the character set. The ONNX model outputs 6906 dimensions, but the dictionary I had downloaded earlier only had 6622 characters.

If the index calculated by argmax was greater than 6622, it couldn't be mapped to a character, all becoming ``.

Solution: Extract the character set directly from the ONNX model metadata.

PP-OCRv6's ONNX model has the character list embedded internally. I wrote a Python script to parse the protobuf metadata, using varint decoding to read field lengths, and extracted 6904 characters.

Adding index 0 (blank) and the last one (space) exactly matches the model's 6906-dimensional output.

Save it as a JSON file, and concatenate it into a character array when the frontend loads.

8. Pitfall: softmax Produces NaN

After adding confidence calculation, a new problem appeared:

#1  conf NaN%  undefined字

The reason was that the model output contained NaN values, and Math.exp(NaN) propagated and spread throughout the entire row's softmax.

Added protection: skip values where !isFinite(v) during argmax; skip values where diff < -50 and !isFinite(diff) during softmax. If the entire row is unusable, skip that time step. Confidence is safeguarded with Math.max(0.001, Math.min(p, 0.999)).

9. Pitfall: softmax Threshold Filters Out All Results

After adding confidence filtering, all results disappeared:

⚠ No text detected

The reason is that with softmax over 6906 dimensions, the probability for a single character is very small, with the highest being around 0.05. The 0.3 threshold I set filtered out all results.

Changed it to 0 (no filtering), and will adjust after seeing the actual distribution.

10. Building the Complete Application

String together detection and recognition, add a UI, and it's a complete application.

Page layout: a drag-and-drop upload area plus a Canvas on the left, and model status, progress bar, and recognition results displayed on the right.

Recognition results are displayed by line, each with a sequence number, confidence percentage, and recognized text. High confidence is green, medium is yellow, and low is not displayed.

On the canvas, colored rectangular boxes mark the position of each text box, with the text and confidence displayed above.

11. Pitfall: The small Model Has a Completely Different Architecture

I wanted to try the larger small model. The results showed:

| Comparison Item | tiny | small

The small model changed the detection head architecture; it's no longer DBNet, but outputs a binary map and then uses a contour method to find text areas. The frontend post-processing code needs to be rewritten.

tiny is sufficient for now, small will be studied another day.

11. Complete Project Structure

demo/2/
├── package.json
├── server.ts              # Bun static server
└── static/
    ├── index.html         # OCR application
    ├── ppocr_keys_v6_tiny.json  # 6904 character set
    ├── ppocr_keys_v6_tiny.txt   # Character set (text format)
    └── models/
        ├── PP-OCRv6_det_tiny.onnx   # Detection model 1.74MB
        └── PP-OCRv6_rec_tiny.onnx   # Recognition model 4.28MB

Startup:

cd demo/2
bun install
bun run dev

Open http://localhost:3001 to use it.

13. Complete Code

WeChat Official Account: 半刻纬度, send the keyword "ocr" to get the complete code download link.