DeepSeek's First Vision Model Gets a Real Multimodal Stress Test
DeepSeek's first vision model is fast, cheap, and already available through its harness tooling, but the over-inference pattern means any developer piping its output into a downstream agent or user-facing feature needs a verification layer. The model will confidently tell you where the elevator is, even when the photo shows no elevator at all.
DeepSeek's experimental vision model, V4-Flash-Vision-Exp, was put through four multimodal reasoning challenges that went far beyond simple image captioning: a detective's desk puzzle, a dense supermarket shelf, a train station navigation table, and a parking lot spatial-relations task. The model handled OCR, object counting, and structured data extraction reliably, and it even spotted contradictions across multiple clues in the detective scenario. But a pattern emerged across every test: when the image lacked explicit real-world detail, the model confidently fabricated it, turning a supermarket receipt into proof of a person's location or inventing elevator directions from a map that showed none.
The parking-lot challenge, which required strict rule-based spatial reasoning, ran for 48 steps and over ten minutes in DeepSeek Harness's minimalist mode, yet produced the most stable result of the four. The station navigation task, by contrast, triggered the most aggressive hallucination, with the model generating turn-by-turn walking directions and accessibility advice from a static information board. The total API cost for the entire battery of tests came to ¥2.50, making the experiment cheap enough to run repeatedly while the model is still in its experimental phase.
Most published 'tests' of new vision models stop at single-image captioning, which exercises perhaps a tenth of a model's actual multimodal reasoning surface. Structured, multi-question challenges reveal failure modes that simple recognition never touches.
The model's over-inference is not random noise; it follows a consistent pattern of completing partial real-world cues into full narratives. A receipt implies a person was present; a map implies navigation infrastructure. This is the same class of hallucination that makes LLMs dangerous in high-stakes domains, now extended into the visual modality.
Spatial reasoning with explicit, rule-bound constraints (parking spaces, EV designations, exit paths) produced better results than open-ended real-world inference (train stations, detective clues). The model performs better when the answer space is closed and the rules are visible in the image itself.
Running a vision model through an agentic harness that takes 48 internal steps for a single parking-lot query suggests the model is doing heavy internal chain-of-thought work. The cost stayed at ¥2.50, which implies the per-step pricing is low enough to make this viable for batch processing, but the latency makes it unsuitable for real-time use.