跪拜 Guibai
← All articles
Agent · Python · HarmonyOS

Three Rule-Based Guardrails Keep AI Interview Reports Leak-Free and Self-Consistent

By 威哥爱编程 ·
Read original on juejin.cn ↗ Google Translate ↗ Alt translation

Any team shipping AI-generated scores or feedback to end users needs a deterministic validation layer between the model and the published report. Without it, a model's tendency to copy reference text or mismatch scores with adjectives will leak answer keys and produce self-contradictory reports that undermine trust in automated interviews.

Summary

An AI interview feedback system now puts its final report through three guardrails, all pure functions with no model calls, before a candidate ever sees it. The first scans for forbidden phrases like "reference answer" and "correct answer" to catch leaks, and the rendering layer never even includes the reference field. The second reconciles the model's stated level and recommendation against score thresholds, and caps any question judged wrong or off-topic at 54 points. The third checks that every question has non-empty advice and that the overall recommendation doesn't contradict a per-question score below 50.

Because any per-question score adjustment changes the total, the guardrail entry point returns corrected per-question reviews, a corrected overall evaluation, and a warning list. The report renderer then produces text and Markdown versions from the same corrected data. The whole pipeline runs offline with 26 tests, including an assertion that no forbidden token appears in the output, which turns leak prevention into a regression test.

Takeaways
— Three guardrails are added: forbidden-token leak scan, score/level reconciliation, and consistency checks for empty fields and contradictions.
— The leak scan uses a small list of forbidden tokens such as '参考答案' (reference answer) and '正确答案' (correct answer); the rendering layer never outputs the reference_answer field at all.
— The `reconcile_level` function forces a level string to match the threshold lookup for a given score; the model's level output is ignored.
— The `enforce_verdict_ceiling` caps scores at 54 for questions judged wrong or off-topic, so 'wrong answer' always maps to 'needs improvement'.
— Any score cap triggers a recalculation of the overall average, and `apply_guardrails` returns corrected per-question reviews alongside the corrected evaluation.
— `cross_consistency_guard` warns when the overall recommendation is '推荐' or '强烈推荐' but a question scores below 50.
— The report renderer produces pure-text and Markdown versions, both containing only public fields: score, level, recommendation, strengths, weaknesses, advice, and summary.
— 26 offline tests cover guardrail boundaries, leak detection, and the end-to-end pipeline; no API key or network is needed.
Conclusions

The guardrails are deliberately pure functions, not model calls, because a model judging its own output lacks the independence needed for a final check; the source explicitly compares it to a student grading their own exam.

Leak prevention is structural: the rendering layer never pulls the reference_answer field, so the answer cannot appear regardless of prompt drift. The scan is just a last-ditch safety net.

The 54-point cap creates a hard visual signal that 'wrong' means 'needs improvement', which avoids the awkward situation where a failed answer still looks acceptable next to a 'good' overall level.

The article's pitfall list shows that the first version only returned the overall evaluation, causing mismatches between the recalculated total and per-question scores. Returning the corrected per-question reviews closes that loop.

Concepts & terms
forbidden-token scan
A string-matching check that looks for a fixed list of phrases (e.g. '参考答案' and '正确答案') that must never appear in an outgoing report, used to catch accidental leaks of reference answers.
level reconciliation
A lookup that maps a numeric score to a fixed level (e.g., >=85 Excellent, 70-84 Good, 55-69 Needs Work, <55 Needs Improvement) so that the model's free-text level string is overridden by the computed one.
verdict ceiling
A rule that caps a question's score at the 'needs improvement' threshold (54) when the question was previously judged wrong or off-topic, ensuring the score visually matches the verdict.
Source: juejin.cn ↗ Google Translate ↗ Backup ↗