Five AI Models Run the Same Agent Task: What Actually Ships vs. What Just Looks Good
Model selection for agent workflows is still a guessing game when all you have are benchmark scores and marketing pages. This comparison surfaces the real trade-offs — cost, tool-calling persistence, data completeness, and visual quality — that determine whether a model actually finishes the job or just generates plausible-looking code.
Five models were given an identical prompt to build a full AI tool navigation site: search for 20 tools, organize structured data, generate a single-file HTML page with filtering and search, then self-check and fix issues. Every model hit a 100% task completion rate across multiple runs, but the path to that result varied significantly. MiniMax-M3 ran the most stable end-to-end process at roughly ¥1.33 per run, while DeepSeek-V4-flash delivered a working page for just ¥0.20, though it skipped some detail checks and needed occasional human nudging.
Step-3.7-flash stood out for data completeness and proactive tool calling — it searched, read, organized, generated, and checked without being prompted — making it the closest to a production-grade agent in the lineup. Gemini 3.5 Flash produced the most polished, visually refined page but collected less data and cost ¥9 per run, roughly 45 times more than DeepSeek-V4-flash. GLM5.2 landed in the middle: balanced capability at ¥3.66, but its tool-calling initiative lagged behind Step-3.7-flash.
The takeaway is not which model wins, but that the choice depends entirely on the job. A quick demo needs speed and low cost; a production agent pipeline needs thoroughness and autonomous error recovery; a client-facing showcase needs visual design sense. Benchmarks that only measure single-turn accuracy miss these trade-offs.
Benchmark scores and single-turn accuracy tell you almost nothing about how a model behaves in a multi-step agent loop. The models that looked best on paper were not always the ones that ran the most complete process.
Tool-calling proactiveness is a separate axis from code-generation quality. Step-3.7-flash searched and verified data autonomously; DeepSeek-V4-flash generated code fast but skipped verification steps unless prompted.
Visual design quality and data thoroughness appear to trade off in current models. Gemini 3.5 Flash made prettier pages but collected less data; Step-3.7-flash gathered more complete data but defaulted to a generic dark theme.
Cost-per-task varies by a factor of 45× across these models for the same prompt, which makes model choice an economic decision as much as a technical one when running agent workflows at scale.
Every model completed the task, which suggests the floor for agent capability is rising fast — but the ceiling on thoroughness, autonomy, and polish still separates the contenders.