OCR Comparison: Is Local Multimodal Feasible?

0. Conclusion First

Local multimodal is not only feasible, its recognition quality already exceeds traditional OCR — but it is not suited for bulk throughput. Measured results:

• qwen3.5-9b (LM Studio local): CER 0.5%, best quality, can fill in missing characters;
• Apple Vision (system built-in): CER 3.2%, good enough quality, 15x faster, does not occupy a model slot;
• gemma-4-e4b (local VLM): CER 40.1%, unusable for Chinese.

Recommended architecture: Vision does the full-volume primary pass (runs through 12,000 pages overnight), qwen-VLM does sampled quality checks, difficult-page escalation, and an auditable second proofreading pass with controlled character changes; gemma is eliminated.

1. Test Design

• Samples: 3 pages from The Cambridge History of China, Volume 1: The Ch'in and Han Empires — p121 (contains ancient source quotes from the Guo Qin Lun and footnotes), p401 (Cao Pi's usurpation narrative), p701 (Later Han official titles); rendered at 300 DPI.
• Baseline ground truth: transcribed page by page from images by ark-code-latest, with two difficult character forms ("千乘" and "刺吏") cropped at 400 DPI and magnified to confirm.
• Metric: Character Error Rate (CER) = edit distance / ground-truth character count; before calculation, unify bracket variants, full-width punctuation, and whitespace.
• An additional canary page: script-generated page containing text no publication could ever hold (random ID QX-90427, year 3057, "鹩哥衔来半块半块鼍龙鼓，麈尾敲落鼻涕", hidden code 74103/龘-42), specifically testing whether the model truly sees the image or merely recites from memory.

2. Overall Results

Engine	Average CER across 3 pages	Healthy per-page time	Projected for 11,936 pages	Occupies LM Studio slot	Error nature
qwen3.5-9b (local VLM)	0.5%	78–97 s (reasoning off)	~12 days on one slot, ~6 days on two slots but starves m5/automation	Yes	Punctuation and typographic normalization + very few semantic "corrections"
Apple Vision	3.2%	12 s	~40 h serial, ~one night with 3–4 parallel processes	No, uses Neural Engine, runs in parallel with everything else	Visually similar character substitutions, never fabricates
gemma-4-e4b (local VLM)	40.1%	112 s	15.5 days on one slot	Yes	Multilingual garbling, nonsense in proper names and digits, self-truncation
ark-code-latest	0 (baseline)	—	Not used for external scanned pages	—	—

Per-page CER:

Page	Vision	qwen	gemma
p121 (Guo Qin Lun quotes)	3.0%	0.3%	36.7%
p401 (Cao Pi replaces Han)	4.5%	0.9% (all 7 errors are punctuation variants)	44.7%
p701 (Later Han official titles)	2.0%	0.3%	38.5%

3. A Key Twist: Did qwen Actually See the Image?

A misleading phenomenon appeared midway: qwen's output for p121 was near-perfect, yet the API returned prompt_tokens=58 — image tokens seemingly never entered the model, so the first reaction was "it's silently reciting the Guo Qin Lun from training data." A first call on p401 also returned 0 characters, looking like a blind-eye fabrication failure.

The canary page settled it: the text was generated 10 minutes before the test, and qwen reproduced every character — 「QX-90427」, 「公元3057年」, 「鹩哥衔来半块鼍龙鼓，麈尾敲落鼻涕」, 「七四一零三」, 「74103」. The random ID and absurd content have only one source: the pixels.

Conclusion:

1. qwen3.5-9b in LM Studio does receive and understand images; prompt_tokens=58 is a metering quirk of that server (image tokens are not counted in that field), so you cannot use it to judge whether a model is multimodal. The only way to judge whether it "saw" the image is the canary.
2. The 0-character response on p401 was caused by long reasoning (1467 reasoning tokens) exhausting the budget under slot contention, unrelated to vision; adding reasoning_effort=none + max_tokens produced the full text in 78 s. This matches the project's prior lesson: batch JSON/transcription tasks must disable reasoning, cap tokens, and validate non-empty output.

4. Error Profiles of the Three Engines

qwen3.5-9b: 0.5%, but every error is "deliberate"

All differences total 11, and none is a common visually-similar misread:

• Punctuation/typographic normalization: em-dash —→-, middle dot ·→・, footnote title brackets 〈〉→《》 — friendlier to readers, but tampering with a verbatim archive;
• Semantic-level "correction": the scanned page printed "太守或刺吏下面供职" (common editions use "刺史"), and qwen output "刺史"; the same page's "在郡里当下级官吏" was changed to "在那里";
• Missing-character completion: on the canary page, the font lacked one glyph, leaving a blank box where "齑" should stand, and qwen inferred the correct 「齑粉」 from context (Vision honestly left it blank or misread it as 「前」); it also misread 「荜拨」 as 「荪拨」.

For reading and event extraction, these traits are almost entirely advantages; for archival storage, they mean the model silently takes over at 0.3% of positions, while the foundation of an archive demands verbatim correspondence between quote and source image.

Apple Vision: 3.2%, errors are "honest"

• Systematic misreading of 「的」: 的→韵/約/钓/哟 (up to 5 occurrences on a single page);
• Visually similar proper-name substitutions: 秦→泰/素, 郡→都, 迫→追, 千乘→干乘, 梯→样;
• Footnote square brackets and digits stick together ([105]→[1053), occasional missing 「的」;
• Headers, page numbers, footnotes, and line breaks all preserved; never adds characters not present in the image; leaves blanks where glyphs are missing. On the canary page, QX-90427, 3057, 七四一零三, 74103 were all read correctly.

It is a "recognizer" not a "generator": it would rather misread one character than invent a sentence. Error patterns are high-frequency, enumerable, and all fall within the range that bge-m3 semantic retrieval is insensitive to and downstream qwen extraction can correct from context.

gemma-4-e4b: 40%, eliminated

• Chinese text mixed with Thai (ไปด้วย), Arabic, Russian (войска), French (urètre), and LaTeX (+|\text{);
• 曹丕→曹王 (consistent across three pages), 武荣 dies ~168 AD→163 AD, Chapter 5 汉代的灭亡→次代的灭亡;
• Fluently rewrites historical facts, output only about 40% the length of the source image (self-truncation).
• Yet on the canary page it correctly read QX-90427/3057/74103 — the visual pathway works, the problem is the weak Chinese language model. This model is unsuitable for Chinese OCR.

5. Feasibility Judgment and Deployment Plan

1. Local multimodal at the 9B scale has passed the "usable" bar (0.5% CER, one-sixth the error rate of a traditional OCR basline), the real constraint is not quality but throughput and contention: 80–100 s per page, and it must exclusively hold an LM Studio slot, which is already occupied by the m5 24-histories extraction and automation review; a full book takes ~12 days on one slot.
2. Run full-volume OCR with Apple Vision: 12 s/page, Neural Engine runs independently of LM Studio, 3–4 parallel processes finish 12,000 pages in about one night; 3.2% CER is harmless to the downstream closed loop — quote validation is a self-consistency comparison between "extraction result ↔ same OCR text", and bge-m3 is immune to single-character substitutions.
3. Use qwen-VLM only in three high-value positions (not bulk):
   • Sampled quality checks (double-run 10–20 pages per book, monitor Vision CER drift);
   • Difficult-page escalation (pages where Vision outputs an abnormally low CJK character count or complex layout);
   • A pure-text second proofreading of Vision output (output a diff, allow only minimal changes, lock proper names and quotes, spot-check manually);
   • Always disable reasoning, cap max_tokens, retry on empty output.
4. Raw scanned pages never go to the cloud; ark-code-latest is used only for sampled baselines.
5. gemma-4-e4b is not used in the Chinese pipeline.

6. Implementation Steps (Vision Channel + qwen Sampling)

1. chronicle/import_cambridge.py: PyMuPDF renders per page (200 DPI) → multi-process calls the compiled vision_ocr → aggregates into page sections by table of contents/chapter (modeled on import_hist: title=book name+chapter, URL uses a stable pseudo-link cambridge://剑桥中国秦汉史#p0121), adds source_type='cambridge'; OCR original text lands per-page in local JSON, resumable.
2. Run The Cambridge History of China first (1,039 pages, ~1–3 hours in parallel), sample 10 pages for double-run with qwen-VLM.
3. Ingestion uses existing insert_page() with jieba + bge-m3 dual-write, event extraction reuses m5 prompt, review reuses automation — zero new mechanisms end-to-end.
4. Front-end SRC_KIND_NAME adds cambridge: 剑桥中国史, event-detail provenance loop automatically works.

---

Appendix: test artifacts in /tmp/chronicle-src/camb_ocr/ (sample page images, canary source, four raw outputs, baseline text, cer.py); Vision tool source chronicle/vision_ocr.swift (compiled, reusable directly).