Frontend

A 30-File Medical Record Mini-Program Hits Every AI Bottleneck: Timeouts, Cold Starts, and a 1x1 PNG Fix

By 小小善后师 · Oct 30, 2026

Read original on juejin.cn ↗ Google Translate ↗ Alt translation

AI features inside constrained clients (mini-programs, mobile, edge) hit hard platform limits; this write-up shows concrete workarounds: async job polling, a provider abstraction to avoid vendor lock-in, and a 1x1 PNG preheat that shaves first inference from 50s to 6s. The lessons apply directly to any small service where a local or remote model must respond under a client-imposed timeout.

Summary

A developer building a small WeChat mini-program for archiving medical records (photos of lab reports, discharge summaries, vaccine booklets) found that the hard part was never the AI model itself. Local Qwen2.5-VL 3B inference on CPU takes 40-60 seconds, while the mini-program platform caps `wx.request` at 60 seconds. The solution is an async job queue with polling: the server returns a `jobId` immediately, the client polls every 1.2 seconds, and the server controls the polling interval, exposing progress to the UI so a 40-second wait feels like a process, not a hang.

The project uses a provider abstraction controlled by an environment variable (`VISION_PROVIDER`) with three backends: local Ollama, Tencent Cloud OCR, and any OpenAI-compatible API. All return the same `{ result, error }` shape, and a health endpoint tells the frontend what model is installed, whether it's ready, and the expected latency. For Tencent Cloud, the author handwrites the TC3-HMAC-SHA256 signature with Node's built-in `crypto` to keep zero npm dependencies. A text-only preheat fails to load the vision tower, so the author sends a 1x1 pixel PNG at startup, cutting first inference from 50 seconds to 6. Error messages get translated into human-readable next steps.

The post closes with three transferable rules: always make long tasks async, abstract AI providers behind a unified interface, and preheat vision models with an image. It also argues that 80% of AI app engineering is traditional backend work, that zero dependencies is a productivity win for small tools, and that 3B-class local models are now viable for structured extraction on CPU.

Takeaways

— WeChat mini-program `wx.request` has a 60-second hard timeout, and local CPU inference on Qwen2.5-VL 3B takes 40-60s, so synchronous calls always fail.

— Async task pattern: server returns HTTP 202 with a `jobId` immediately; client polls every 1.2 seconds; the server sends `pollAfterMs` to control polling rhythm.

— In-memory Map works as a task queue for personal projects; set cap 50 and auto-evict after 5 minutes.

— Provider abstraction: one `recognize()` function behind a `VISION_PROVIDER` environment variable, with three backends (ollama, tencent, openai), all returning the same `{ result, error }` shape.

— A health endpoint exposing `visionProvider`, model `ready` status, installed models, and expected seconds lets the UI show distinct states (unconfigured, model missing, normal).

— Tencent Cloud TC3-HMAC-SHA256 signing can be handwritten in ~20 lines using Node's built-in `crypto`, avoiding the SDK entirely.

— Date extraction from OCR text must require a 4-digit year (e.g., `20\d{2}`) to avoid mistaking "1日3次" or "5ml" for a date.

— Text-only preheat does not load the vision tower; sending a 1x1 PNG (base64 ~100 bytes) loads the full vision pipeline.

— Cold start dropped from 50s to 6s with the 1x1 PNG preheat at service startup, plus a backoff retry sequence covering about 4 minutes.

— Error messages should include a concrete next step, e.g., 'run `ollama pull qwen2.5vl:3b`' instead of a bare HTTP 500.

— Long tasks exceeding client timeout (60s mini program, 300s default fetch) should use job-id + polling or SSE/WebSocket.

— Qwen2.5-VL 3B quantized (3.2GB) can run structured extraction on pure CPU, making local, privacy-first AI viable.

Conclusions

The post's claim that 80% of AI app engineering is outside AI is well supported by its own detail: timeouts, queues, signature handling, preheat strategies, and error translation get far more space than prompt work.

A provider abstraction controlled by a single environment variable, rather than an SDK, is a cheap hedge against model vendor churn; small projects gain more from that than from any SDK's convenience.

A 1x1 PNG is a clever, zero-cost workaround for a real multimodal model quirk: text-only warmup leaves the vision encoder cold, so any image-loading trigger suffices.

The "zero npm dependencies" stance flips the usual bias: for a small tool, managing dependencies is overhead, and built-in modules can handle routing, signing, and parsing fine.

Hand-rolling TC3 signing is a nice zero-dependency showcase, but it only pays off when you already avoid SDKs everywhere; the real architectural takeaway is the uniform return contract, not the signature code.

Concepts & terms

vision tower

The vision encoder component of a multimodal LLM (like Qwen2.5-VL) that processes image inputs separately from the language model. A text-only request loads only the language weights, leaving the vision encoder cold, which causes a long delay when the first real image arrives.

TC3-HMAC-SHA256

Tencent Cloud's request signing algorithm (similar to AWS SigV4). It hashes the canonical request and signs it with a series of HMAC-SHA256 keys derived from the secret key, date, and service. It can be implemented in ~20 lines with Node's crypto module.

async job + polling

A pattern for long-running operations in constrained clients: the server returns a job ID immediately (HTTP 202), runs the actual work in the background, and the client periodically polls a status endpoint until the job completes or fails.

cold start (model inference)

The delay caused by loading a model's weights into memory on its first request. For local CPU models this can be tens of seconds and is often worsened when only part of the model (e.g., language but not vision) gets loaded.

Source: juejin.cn ↗ Google Translate ↗ Backup ↗