Three Rule-Based Guardrails Keep AI Interview Reports Leak-Free and Self-Consistent
Simulated Interview Agent Development Practice 08: The Final Report Must Pass Three Guardrails—Anti-Leak, Anti-Exaggeration, Anti-Contradictory Advice—Before It's Safe to Send to Candidates
Author: 威哥爱编程
After finishing Part 7, the system could finally "give an account": it gives a scored comment after each question, and aggregates a final review at the end of the session. I eagerly posted the final review to the group, intending to use it as a deliverable, but someone stopped me with a single question: "Would you really send this report directly to a candidate?"
I stopped and thought: no, I wouldn't. Three problems stood out: First, the report could leak answers. The model judges "against the reference answer," and if it casually copies the reference answer into the comment, the candidate will just memorize it next time, and the question bank is ruined. Second, the score could be exaggerated. The model sometimes says "excellent" in words but gives a score of only 50—the level and score don't match, and the report contradicts itself. Third, the advice could be empty or self-contradictory. The overall review says "strongly recommended" yet one question collapsed to below passing, leaving the candidate baffled.
This article adds the final guardrail to the system: render the overall review into a report that can be sent to a person, and before sending it, pass it through three barriers—prevent leaks, prevent exaggeration, prevent wrong advice. The core remains the same maxim: these three barriers are all deterministic rules; no model is called. They are cheap, testable, and cannot be led astray by the model's "creativity."
1. First, a primer: why reports must have guardrails
1.1 Reports are for candidates; reference answers must never leak
Earlier articles repeatedly emphasized that "the model only judges against the reference answer and does not introduce its own standards." This has a side effect: the reference answer appears verbatim in the prompt visible to the model. The model might casually copy it into the comment. For example, if a candidate is asked "How is Redis AOF rewrite triggered?" the model's comment might say, "The correct answer is to first trigger the auto-aof-rewrite-percentage threshold..."—this phrase gives away the key terms of the reference answer. That's fine for internal review, but once the report is sent to a candidate, "the standard answer to this question" is leaked—next time they will just memorize it, and the evaluation becomes meaningless. So leak prevention is the top priority: the report may only contain "public fields" such as score, level, strengths, weaknesses, advice, and summary; not a single word of the reference answer may appear.
1.2 The model's level and recommendation can contradict the score
The model is not an arithmetic machine. If you ask it to output both "score" and "level" at the same time, it might give a score of 50 but a level of "excellent"; or a total score of 72 but a recommendation of "strongly recommended." These inconsistencies appear frequently in real calls because the model is essentially "guessing" a threshold it doesn't remember precisely. The only solution: level and recommendation are always determined by a code lookup; the model only provides numbers. A score of 50 → lookup gives "needs improvement"; whatever level the model writes doesn't count.
1.3 Empty advice and contradictory overall reviews break the review
The value of a review lies in "what to practice next." If a question's advice is empty, or the overall review says "recommended" but one question crashed to 40, the candidate doesn't know which sentence to trust. These are data integrity problems. Relying on "a human eyeballing it" will eventually let something slip. Rules must automatically check and catch contradictions before they are sent.
Together, the three barriers place a model-independent, offline-testable filter layer between "model output" and "a report that can be sent to a person."
1.4 Why none of the three barriers uses the model
You might ask: if the model can judge and comment, why not just let it "self-check whether the report is compliant"? No, for three reasons. First, these checks are rule-based, not semantic. Whether a level matches a score, whether advice is empty, whether a leak word appears—these are deterministic string comparisons and lookups. Using a model would be slower, more expensive, and less stable. Second, guardrails must be testable. They are the system's last line of defense and must be able to run fully green offline, at zero cost, without an API key—a model call cannot do that. Third, letting the model check itself is like letting a student grade their own exam. If it writes "excellent" and then judges whether "excellent" is reasonable, it inherently lacks independence. So the guardrails are all pure functions—this follows the same maxim from earlier articles: "for cheap, deterministic things, use arithmetic; for language processing, use the model."
2. Engineering structure: two layers—report rendering and guardrails
08-final-report-and-guardrails/
├── src/
│ ├── settings.py # Central config (reuses earlier articles: SiliconFlow + DeepSeek-V3)
│ ├── llm.py # Model factory (reuses earlier: real key / offline script model / error)
│ ├── rubric.py # Pydantic models for review and overall evaluation + thresholds (reuses Part 7)
│ ├── guardrails.py # Three guardrails: leak prevention / exaggeration / wrong advice (new in this part)
│ └── report.py # Report rendering: text / Markdown / optional intro (new in this part)
├── tests/ # 26 test cases, fully offline
└── demo.py
rubric.py contains the QuestionReview, Evaluation, and threshold functions level_of, recommendation_of already defined in Part 7. This article reuses them directly—the guardrails need to "align scores and levels," and they rely on the same thresholds. The two new modules have clear responsibilities: guardrails.py checks "is it correct before sending," while report.py controls "what it looks like when sent."
3. The three guardrails: hard-code the rules
guardrails.py is the heart of this article. All three barriers are pure functions; they take a review/overall evaluation and return a corrected result or a warning list.
3.1 Leak prevention: detect_leak scans for forbidden tokens
# Words that must never appear in the report (question-bank / reference-answer related)
LEAK_TOKENS = ("参考答案要点", "参考答案", "正确答案", "标准答案", "题库原文")
def detect_leak(text: str, extra_tokens=()) -> list[str]:
"""Scan text for leak words, return the list of hits (empty = safe)."""
tokens = LEAK_TOKENS + tuple(extra_tokens)
return [t for t in tokens if t in text]
This is a fallback scan. The real first wall is in the rendering layer (next section): the report-rendering function never outputs the reference-answer field at all, so the answer never enters. detect_leak is a "final sweep before sending"—just in case some part accidentally splices the answer into the report, this catches it and stops it.
3.2 Exaggeration prevention: level and recommendation must align with score
def reconcile_level(score: int, level: str) -> str:
correct = level_of(score)
return correct if level != correct else level
def enforce_verdict_ceiling(review, verdict_quality) -> QuestionReview:
"""For wrong/off-topic questions, the score must not exceed the needs-improvement line (54)."""
if verdict_quality in ("wrong", "offtopic") and review.score > 54:
return review.model_copy(update={"score": 54, "level": level_of(54)})
return review
reconcile_level exposes drift like "score 50 but says excellent": it uses the score as the source of truth, looks up the table, and forces the level back to "needs improvement." More aggressive is enforce_verdict_ceiling—it directly connects to the judgment result from Part 6: if a question was already judged wrong (factual error) or offtopic (off-topic answer) in Part 6, its score must not exceed 54 (the needs-improvement line). This cuts off "wrong but high-scoring" contradictions at the source.
There's an easily overlooked chain reaction here: once any question is capped, the total must be recalculated. The guardrail doesn't just cap a single question and stop; it must re-average all scores after the cap and re-look-up the level and recommendation—otherwise you get "question 3 dropped from 55 to 54, but the total still shows the old 72," a numerically inconsistent result. This is also why apply_guardrails returns corrected per-question reviews, not just the overall evaluation.
3.3 Wrong-advice prevention: complete fields + consistent overall review
def validate_reviews(reviews) -> list[str]:
problems = []
for idx, r in enumerate(reviews, start=1):
if not (r.advice or "").strip():
problems.append(f"第{idx}题建议为空")
if not r.strengths:
problems.append(f"第{idx}题亮点为空")
if not r.weaknesses:
problems.append(f"第{idx}题短板为空")
return problems
def cross_consistency_guard(evaluation, reviews) -> list[str]:
warnings = []
for idx, r in enumerate(reviews, start=1):
if evaluation.recommendation in ("推荐", "强烈推荐") and r.score < 50:
warnings.append(f"第{idx}题得分{r.score}偏低,与总评'{evaluation.recommendation}'矛盾")
return warnings
The two functions each handle one side: validate_reviews checks "are the fields for each question complete," and cross_consistency_guard checks "does the overall conclusion contradict any per-question score." If the overall says "recommended" but one question scored below 50, it raises a warning, exposing the contradiction before it's sent.
3.4 One-entry point: apply_guardrails returns three things
Rather than making the caller assemble the three barriers one by one, I wrapped them into a single entry point:
def apply_guardrails(evaluation, reviews, verdicts=None):
"""Returns (corrected overall eval, corrected per-question reviews, warning list)."""
fixed_reviews = []
for r, v in zip(reviews, verdicts or [None] * len(reviews)):
quality = v if isinstance(v, str) else getattr(v, "quality", None)
r = enforce_verdict_ceiling(r, quality)
r = r.model_copy(update={"level": reconcile_level(r.score, r.level)})
fixed_reviews.append(r)
overall = round(sum(r.score for r in fixed_reviews) / len(fixed_reviews))
fixed_eval = evaluation.model_copy(update={
"overall_score": overall,
"level": reconcile_level(overall, evaluation.level),
"recommendation": reconcile_recommendation(overall, evaluation.recommendation),
})
...
return fixed_eval, fixed_reviews, warnings
Notice it returns not two values but three: the corrected overall evaluation, the corrected per-question reviews, and the warning list. Why also return corrected per-question reviews? Because if any question is capped (say 55 → 54), the total is recalculated, but the per-question score shown in the report must also change—otherwise the report says "Question 3: 55 points" while the header says "Total: 71," which is self-contradictory. I missed this at first; the first version only returned the overall evaluation, and the report still used the original reviews, producing demo output where the total and per-question scores didn't match. Switching back to the three-part return closed the loop.
4. Report rendering: what a sendable report looks like
report.py turns Evaluation + per-question reviews into two reports. The core discipline is a single rule: the rendering layer only outputs public fields; it never carries any reference answer—this is the first wall against leaks, more thorough than scanning because the answer never enters.
def render_text_report(evaluation, reviews) -> str:
"""Plain-text report, suitable for pasting into IM/email to a candidate."""
lines = ["=" * 30, " 模拟面试 · 总评报告", "=" * 30]
lines.append(f"总分:{evaluation.overall_score}({evaluation.level})")
lines.append(f"推荐结论:{evaluation.recommendation}")
lines.append("【综合亮点】")
for s in evaluation.strengths:
lines.append(f" + {s}")
...
for idx, r in enumerate(reviews, start=1):
lines.append(f" 第{idx}题 得分 {r.score}({r.level})")
if r.strengths:
lines.append(f" 亮点:{';'.join(r.strengths)}")
if r.weaknesses:
lines.append(f" 短板:{';'.join(r.weaknesses)}")
lines.append(f" 建议:{r.advice}")
...
return "\n".join(lines)
render_markdown_report has the same structure but uses Markdown headings and lists, making it suitable for storage, rendering into a web page, or later rich-text display on the HarmonyOS client. Both reports contain only "score / level / recommendation / strengths / weaknesses / advice / summary"; the reference_answer field is never referenced from start to finish—there's nothing to leak even if you wanted to.
There is also an optional intro paragraph, polish_intro: it asks the model to write an encouraging but honest sentence (no more than 40 characters, no reference answers). This continues the principle of "send language processing to the model"—the numbers are already computed by arithmetic; only the "human words" are left to the model. Of course, even if the model "gets creative" in the intro, detect_leak will sweep it one last time before sending as a fallback.
5. Run it to see the real effect
python demo.py runs fully offline and demonstrates the complete chain of "three guardrails + report rendering + leak scan":
① After guardrail correction: total score 71 (good) / recommended
No warnings; ready to send
② Plain-text report:
==============================
模拟面试 · 总评报告
==============================
总分:71(良好)
推荐结论:推荐
【综合亮点】
+ 基础扎实
+ 讲清取舍
+ 概念清楚
【综合短板】
- 生产经验少
- 深度不够
【逐题复盘】
第1题 得分 88(优秀)
亮点:覆盖 RDB/AOF 两种;讲清文件大小与恢复速度的取舍
短板:没提 fsync 三档
建议:把 AOF 的 fsync 三档区别讲清楚
第2题 得分 72(良好)
亮点:概念清楚
短板:深度不够
建议:补充持久化与主从同步的关系
第3题 得分 54(待提升)
亮点:答了要点
短板:有事实错误;表述乱
建议:先纠正 RDB 触发时机的说法
【总结】
整体良好,建议进一步补生产深度。
==============================
④ Pre-send leak scan:
Text report: safe (no leak words)
Markdown: safe (no leak words)
Notice a few details: Question 3 was originally 55; because Part 6 judged it wrong, the guardrail capped it to 54 and changed its level from "pass" to "needs improvement." The total was recalculated from 72 to 71 ((88+72+54)/3). This report is fully internally consistent—total, level, recommendation, and per-question scores all match, and there is not a single word from the reference answer. This is what a "sendable" report should look like.
The demo above is offline. When connected to SiliconFlow's DeepSeek-V3, the comment and overall review content are generated live by the model, but the guardrail and rendering layers don't need a single line changed—apply_guardrails and render_*_report consume the same Evaluation / QuestionReview structures; the model simply writes the text in a more human tone. The config still goes through .env (LLM_API_KEY=sk-your-key, LLM_BASE_URL=https://api.siliconflow.cn/v1, LLM_MODEL=deepseek-ai/DeepSeek-V3), and keys never enter the code repository. So the "capping, recalculation, deduplication, leak scan" you verified offline on your machine takes effect when you go live, with no behavioral difference when you switch to the real model.
6. Test coverage: 26 cases nail down the three barriers
tests/test_report.py Report structure, field completeness, never contains reference answer, intro
tests/test_guardrails.py Three barriers: level/recommendation alignment, capping, leak scan, empty fields, consistency
tests/test_demo.py End-to-end: guardrails → rendering → leak scan, report internally consistent
─────────────────────────
26 passed in 1.84s
Coverage includes: both text and Markdown reports contain total, level, recommendation, and per-question scores and advice; neither report contains forbidden words like "参考答案" (the core leak-prevention assertion); reconcile_level four-tier boundaries; reconcile_recommendation four-tier boundaries; enforce_verdict_ceiling caps wrong/offtopic to 54, leaves correct/partial/None untouched, and doesn't further lower already-low scores; detect_leak catches/doesn't catch hits, including custom words; validate_reviews catches empty advice, empty strengths; cross_consistency_guard warns on collapsed questions; apply_guardrails recalculates the total after capping, returns corrected per-question reviews, and corrects the recommendation. All offline, no key, no network.
The most notable part is the assertion that the report never contains a reference answer. Other tests verify "does the function work correctly"; this one verifies "did the safety bottom line hold." It directly asserts that the output of render_text_report and render_markdown_report contains none of 参考答案, 正确答案, 标准答案, or 题库原文. If anyone ever changes the rendering function and accidentally adds the answer field, the test immediately turns red. The value of guardrails is half in stopping bad data, and the other half in "making people afraid to casually remove this shield."
7. Pitfall list
- First version only returned the overall evaluation, not the corrected per-question reviews: after a question was capped, the total changed, but the report still rendered the original
reviews, resulting in "total 71" while showing "Question 3: 55." Switching to the three-part return (overall + per-question + warnings) made the report consistent. - Relying only on the prompt to tell the model not to write answers: prompt constraints are soft; the model still occasionally copies the reference answer into comments. Changing to a dual guarantee of "rendering layer doesn't include it + scan before sending" means the answer physically cannot enter the report.
- Letting the model decide level and recommendation: the model's level and score don't match. Now level and recommendation always come from threshold functions; the model only provides numbers, giving a single source of truth.
- Setting the cap threshold to 60 out of assumption: initially wanted to cap wrong questions at 60, but 60 is still the "pass" tier, looking like "okay." Changed to 54 (below the needs-improvement line) so "wrong answer" strictly maps to "needs improvement."
- Mixing report rendering and guardrails in one function: at first it rendered and checked at the same time, tangled logic and hard to test. Split into
guardrails.py(correctness) andreport.py(appearance), each independently testable. detect_leakonly scanned the report body, not the intro: initially the leak scan only ran on the report text, missing thepolish_intromodel-polished opening. Later, the intro was also included in the "pre-send scan" scope—any text the model freely generates must pass adetect_leakcheck; no exceptions.
8. Wrap-up for this article
Now the backend loop is completely closed: from Part 5 (connecting the question bank), Part 6 (follow-up judgment), Part 7 (structured comments), to this article (passing guardrails and rendering a sendable final report). In the report, scores are computed by arithmetic, levels and recommendations are looked up from a table, strengths and weaknesses are deduplicated and distilled, and leaks are blocked—every step can be re-checked and tested; it is not a single "creative" output from the model.
At this point, all 8 backend parts are complete. Next is to get it "on the phone": Part 9 begins the HarmonyOS (ArkTS) client, first building the chat interface so the interviewer's questions stream onto the screen, and the candidate talks to it directly in their phone. The JSON fields from the backend are the exact data source the HarmonyOS UI will bind to.
Final sentence: Whether a report can be sent to a person doesn't depend on how pretty it looks, but on whether it dares to be questioned—scores are re-checkable, answers don't leak, and conclusions don't contradict themselves. Only then is this review genuinely presentable.
Top 1 from juejin.cn, machine-translated. The original thread is authoritative.
Thumbs up