跪拜 Guibai
← All articles
Frontend · Agent

DeepSeek V4 in the Elevator Shaft: A Maintenance Agent That Can't Freelance on Safety

By 去伪存真 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

The pattern of read-only tools, human approval gates, and confidence circuit breakers is a reusable template for any LLM agent that touches physical equipment. It shows how to run a heavy reasoning model behind a cheap fast model and still keep safety guarantees, which is exactly what industrial, medical, or robotics applications need.

Summary

An elevator maintenance Agent built for production uses two models: a 7B lightweight model for routine lookups, form filling, and inventory checks, and DeepSeek-V4-FPro for complex root-cause analysis. A router splits traffic, keeping 90% of interactions under 500ms. A dedicated vision model reads mainboard indicator lights, and offline speech recognition (Sherpa-ONNX) works in basement machine rooms with no signal.

The system wraps every write action behind a safety gateway: the Agent can only emit a JSON draft, which the gateway validates against the worker's certificate and an OTP confirmation. A reranker blocks generation when retrieved chunks fall below a 0.65 relevance threshold, falling back to a hardcoded reply. Milvus hybrid retrieval with BM25 and metadata filters handles fault codes that look identical in embedding space.

For scanned manuals, MinerU rebuilds cross-page tables and splits content atomically by fault code. For a recurring E12 fault, parallel tool calling pulls IoT metrics, long-term maintenance history, and manual rules to conclude the real cause was a broken machine-room AC and a dusty fan, not the inverter. The rollout plan runs in three phases, from voice-only MVP to IoT-linked diagnosis to data-driven preventive maintenance.

Takeaways
— All IoT tools in the Agent are read-only; write requests generate a JSON draft that requires a human slide-to-confirm and a valid certificate before reaching the IoT platform.
— A reranker blocks LLM generation when the top retrieved chunk scores below 0.65, returning a hardcoded fallback that tells the technician to consult paper drawings.
— Milvus hybrid retrieval uses Dense + BM25 with metadata filtering on brand and controller, because pure embeddings treat E01 and E02 as identical.
— Routing 90% of interactions to a 7B model keeps responses under 500ms and cuts compute waste by 80%, reserving DeepSeek-V4-FPro for complex cases.
— Prefix caching pins the safety rules and static record summary at the start of the prompt, cutting time-to-first-token by over 60% and token costs by half.
— MinerU rebuilds scanned maintenance manuals, merging cross-page tables, and chunks by individual fault-code object to keep codes, triggers, and steps together.
— For the recurring E12 fault, parallel tool calling pulls TDengine metrics, long-term maintenance history, and the manual's de-rating rule to identify a broken machine-room AC and dust-clogged fan as the root cause.
Conclusions

The dual-model router is a cost/latency architecture that generalizes: keep a cheap model for the 90% of calls and spend on heavy reasoning only when the task actually needs it. The elevator domain just makes the trade-off visible in seconds and dollars.

A safety gateway that forces human approval on every write, plus a confidence threshold that refuses to guess, is a template for any agent operating on physical infrastructure. The elevator is an extreme case, but the principle applies to robots, HVAC, or any device with real consequences.

Embedding models' failure on alphanumeric fault codes is a blunt reminder: semantic search alone isn't enough for precision codes. The fix here—exact-match metadata filters and BM25—is a practical answer that many RAG systems outside this domain ignore.

Concepts & terms
LangGraph
An orchestration framework for building stateful, graph-based agent flows. Here it hard-codes deterministic state nodes for regulated processes like trapped-person alarms, preventing the LLM from improvising around compliance steps.
ReAct loop
A pattern where an LLM reasons and acts in a cycle, choosing tools and interpreting results. The article deliberately avoids free ReAct loops for safety-critical steps, reserving looped reasoning only for non-standard troubleshooting.
Prefix Caching (KV Cache)
Reusing the key-value cache of the attention computation for a fixed prompt prefix across multiple turns. Pinning safety rules and static summaries at the prompt start lets multi-turn conversations cut time-to-first-token and token cost.
Reranker
A model that re-scores retrieved chunks to improve precision. Here it gates generation with a relevance threshold, blocking the LLM from producing an answer when retrieval confidence is too low.
From the discussion

The confidence circuit-breaker threshold of 0.65 earns its credibility from a PR curve over historical work orders, with >98% accuracy as the floor, and the thread accepts that as far more rigorous than a guess. But the offline evaluation basis raises a real concern: new equipment and shifting work-order distributions will drag the line out of calibration unless online feedback re-checks or periodic PR reruns happen, and brand-specific manual quality suggests per-brand thresholds may be needed. Meanwhile, the no-signal shaft scenario makes the case that on-device inference delivering stable offline results is the core value of field agents.

— The 0.65 threshold was calibrated from historical work orders using a PR curve with a >98% accuracy backstop, and below it cross-model forced explanations appear.
— Offline calibration alone risks drift as new models arrive and work-order distributions shift; without online feedback review or periodic PR reruns, the threshold will slowly become invalid.
— Manual quality varies across brands, so the threshold may need to be tuned per brand rather than held constant.
— The safety gateway design—read-only tools, JSON drafts for writes, and manual OTP confirmation—effectively contains the destructive potential of hallucinations.
— On-device inference is the real differentiator for field agents in no-signal environments: stable results under disconnection matter more than raw intelligence.
Featured comments
组合搭配函数

What struck me most in this piece is the E01 and E02 example. Vector retrieval looks semantically close, but physically one is a door-lock fault and the other is an inverter that blew up. Doing a hard filter by brand plus control cabinet first, then using BM25 to hammer on the fault code, that ordering is right. On the safety gateway side, the agent only has read-only tools, write operations only produce a JSON draft plus manual OTP confirmation, which really locks down the destructive power of hallucination. One curious question: how was the confidence circuit-breaker threshold of 0.65 calibrated? Manual quality varies a lot across brands—should the threshold be tuned per brand?

去伪存真

It was pinned down by building an evaluation set from historical work orders and running a PR curve. In industrial troubleshooting, the cost of 'random guidance' is far greater than 'handing off to a human.' We set a baseline of >98% accuracy and pulled the Rerank score distribution, and found that below 0.65 cross-model forced explanations start to appear, so we hard-locked the circuit-breaker line there.

组合搭配函数  → 去伪存真

I accept the 0.65 circuit-breaker line. Using historical work orders to run a PR curve and having >98% accuracy as a backstop is far more solid than picking a threshold out of thin air. But 0.65 was still calibrated on an offline evaluation set. As soon as new models come in and the work-order distribution drifts, that line will likely need to move; if you only set the threshold and don't do online feedback review, it will slowly become distorted over time. Do you currently rerun the PR curve periodically, or do you just rely on manual handoff as a backstop?

明略科技

A shaft with no signal is exactly where on-device inference truly shines. For field agents, the most valuable thing isn't intelligence—being able to give stable results even when disconnected is what's really impressive.

See top comments, translated →
Source: juejin.cn ↗ Google Translate ↗ Backup ↗