Three Chinese Multimodal Models Go Head-to-Head on Real Production Tasks
Multimodal models are crossing from demo to production, and the bottleneck is no longer accuracy — it's latency and cost per call. These results show that on structured extraction tasks, the cheapest model can be the fastest without sacrificing quality, which changes the default choice for high-volume agent and API workloads.
Three domestic Chinese multimodal models — Step 3.7 Flash, Qwen3.6-flash, and MiniMax M3 — were benchmarked on the same production tasks with identical prompts and parameters. The first task asked each model to reconstruct a 10-step WeChat Mini Program login flow from a single diagram. The second required extracting 12 structured JSON fields from an electronic invoice.
All three models produced correct outputs with no errors across both scenarios. Quality was indistinguishable. The differences emerged in speed and cost: Step 3.7 Flash was consistently faster and cheaper, with the lowest token consumption and API latency in every test. Invoice extraction cost less than one Chinese cent per call on Step 3.7 Flash.
The evaluation framework itself is worth noting. Rather than relying on public leaderboard scores, the tests measured three production-critical dimensions — one-shot output quality, end-to-end response time, and total cost including any manual follow-up. This mirrors the criteria any team would use when deciding whether a model is safe to wire into an agent loop or a customer-facing API.
Public benchmark scores are a poor proxy for production readiness. All three models scored well on leaderboards, but the real differentiator was cost and speed, not accuracy.
The evaluation criteria — one-shot quality, latency, and total cost including human intervention — are a practical template for any team vetting a model for agent or API use.
Multimodal models have reached a point where structured extraction from invoices and diagrams is a solved problem; the remaining competition is on price and throughput, not capability.