DeepSeek V4 in the Elevator Shaft: A Maintenance Agent That Can't Freelance on Safety
The pattern of read-only tools, human approval gates, and confidence circuit breakers is a reusable template for any LLM agent that touches physical equipment. It shows how to run a heavy reasoning model behind a cheap fast model and still keep safety guarantees, which is exactly what industrial, medical, or robotics applications need.
An elevator maintenance Agent built for production uses two models: a 7B lightweight model for routine lookups, form filling, and inventory checks, and DeepSeek-V4-FPro for complex root-cause analysis. A router splits traffic, keeping 90% of interactions under 500ms. A dedicated vision model reads mainboard indicator lights, and offline speech recognition (Sherpa-ONNX) works in basement machine rooms with no signal.
The system wraps every write action behind a safety gateway: the Agent can only emit a JSON draft, which the gateway validates against the worker's certificate and an OTP confirmation. A reranker blocks generation when retrieved chunks fall below a 0.65 relevance threshold, falling back to a hardcoded reply. Milvus hybrid retrieval with BM25 and metadata filters handles fault codes that look identical in embedding space.
For scanned manuals, MinerU rebuilds cross-page tables and splits content atomically by fault code. For a recurring E12 fault, parallel tool calling pulls IoT metrics, long-term maintenance history, and manual rules to conclude the real cause was a broken machine-room AC and a dusty fan, not the inverter. The rollout plan runs in three phases, from voice-only MVP to IoT-linked diagnosis to data-driven preventive maintenance.
The dual-model router is a cost/latency architecture that generalizes: keep a cheap model for the 90% of calls and spend on heavy reasoning only when the task actually needs it. The elevator domain just makes the trade-off visible in seconds and dollars.
A safety gateway that forces human approval on every write, plus a confidence threshold that refuses to guess, is a template for any agent operating on physical infrastructure. The elevator is an extreme case, but the principle applies to robots, HVAC, or any device with real consequences.
Embedding models' failure on alphanumeric fault codes is a blunt reminder: semantic search alone isn't enough for precision codes. The fix here—exact-match metadata filters and BM25—is a practical answer that many RAG systems outside this domain ignore.
The confidence circuit-breaker threshold of 0.65 earns its credibility from a PR curve over historical work orders, with >98% accuracy as the floor, and the thread accepts that as far more rigorous than a guess. But the offline evaluation basis raises a real concern: new equipment and shifting work-order distributions will drag the line out of calibration unless online feedback re-checks or periodic PR reruns happen, and brand-specific manual quality suggests per-brand thresholds may be needed. Meanwhile, the no-signal shaft scenario makes the case that on-device inference delivering stable offline results is the core value of field agents.
What struck me most in this piece is the E01 and E02 example. Vector retrieval looks semantically close, but physically one is a door-lock fault and the other is an inverter that blew up. Doing a hard filter by brand plus control cabinet first, then using BM25 to hammer on the fault code, that ordering is right. On the safety gateway side, the agent only has read-only tools, write operations only produce a JSON draft plus manual OTP confirmation, which really locks down the destructive power of hallucination. One curious question: how was the confidence circuit-breaker threshold of 0.65 calibrated? Manual quality varies a lot across brands—should the threshold be tuned per brand?
It was pinned down by building an evaluation set from historical work orders and running a PR curve. In industrial troubleshooting, the cost of 'random guidance' is far greater than 'handing off to a human.' We set a baseline of >98% accuracy and pulled the Rerank score distribution, and found that below 0.65 cross-model forced explanations start to appear, so we hard-locked the circuit-breaker line there.
I accept the 0.65 circuit-breaker line. Using historical work orders to run a PR curve and having >98% accuracy as a backstop is far more solid than picking a threshold out of thin air. But 0.65 was still calibrated on an offline evaluation set. As soon as new models come in and the work-order distribution drifts, that line will likely need to move; if you only set the threshold and don't do online feedback review, it will slowly become distorted over time. Do you currently rerun the PR curve periodically, or do you just rely on manual handoff as a backstop?
A shaft with no signal is exactly where on-device inference truly shines. For field agents, the most valuable thing isn't intelligence—being able to give stable results even when disconnected is what's really impressive.