LLM-Returned JSON Breaks Three Times Out of Ten: Stop Blaming the Model — A 60-Line Fault-Tolerant Parser Catches It First (With Runnable Code)

Conclusion: When an Agent is asked to call tools, the JSON the model gives is often malformed. That's normal; the first reaction shouldn't be to switch models, but to add a fault-tolerant parsing layer.

I originally hit this pitfall too. While an Agent was running, it suddenly crashed. Checking the logs, json.loads threw an exception. I stared for a while thinking the model was broken; later I printed the raw output and saw the "JSON" the model gave looked like this:

```json
{"city": "北京", "temp": 25}
```

It was wrapped in markdown code fences. You could say it's wrong, but the JSON inside is correct; you could say it's right, but json.loads gives you a JSONDecodeError. This "content is right, outer layer errors" situation covers the vast majority of bad JSON I've encountered.

1. The common bad JSON the model spits out

I categorized the pitfalls I stepped on; there are basically six types:

1. Markdown fences: ```json ... ``` wraps it; the model thinks it's "pretty".
2. Trailing commas: {"a": 1,}—JavaScript style, JSON doesn't accept.
3. Single quotes: {'a': 1}—Python literal habits.
4. Python literals: True / False / None—not converted to true / false / null.
5. Unquoted keys: {city: "北京"}—omitted quotation marks.
6. Text before and after: 查询结果如下：{...} 以上供参考。—the model wants to chat a bit more.

For these six, none is the model 'can't'—all are it being 'casual'. It read a bunch of code with fences and comments, so it writes with that flavor. If you insist on using standard json.loads to pin it down, it's like using classical Chinese grammar to parse colloquial speech—it might work, but it fails every day.

2. Code: try standard parsing first, repair only if it crashes

The approach is straightforward: don't jump straight to regex combat.

• First run standard json.loads. If it passes, don't touch it—zero overhead, and avoids mis-harming already conforming output.
• Only if it crashes, enter the repair flow, fixing from outer to inner: strip fences → extract {...} → convert literals → drop trailing commas → quote keys → convert single quotes to double quotes.
• After repair, run json.loads again. If it still crashes, then it's really not a format issue; throw it.

Below is the complete code, pure standard library, only json and re, no network access, run directly:

"""LLM 吐的 JSON 总是坏的：一个容错解析器。

纯标准库，无第三方依赖。用法：python3 tolerant_json.py
"""

import json
import re


def _strip_fences(s: str) -> str:
    """剥掉 markdown 代码围栏（三个反引号包着的 json 块）。

    源码里故意不直接写连续三个反引号，免得这段代码嵌进 markdown
    代码块时把围栏提前闭合——用单反引号拼出 fence。
    """
    fence = "`" * 3
    m = re.search(fence + r"(?:json)?\s*(.*?)" + fence, s, re.S)
    return m.group(1).strip() if m else s.strip()


def _extract_json(s: str) -> str:
    """从夹着说明文字的输出里，抠出最外层的 {...} 或 [...]。"""
    for open_ch, close_ch in (("{", "}"), ("[", "]")):
        start, end = s.find(open_ch), s.rfind(close_ch)
        if start != -1 and end != -1 and end > start:
            return s[start:end + 1]
    return s


def _fix_literals(s: str) -> str:
    """Python 字面量转 JSON：True/False/None → true/false/null。"""
    s = re.sub(r"\bTrue\b", "true", s)
    s = re.sub(r"\bFalse\b", "false", s)
    s = re.sub(r"\bNone\b", "null", s)
    return s


def _drop_trailing_commas(s: str) -> str:
    """去掉 ] 或 } 前多余的逗号：{"a":1,} → {"a":1}。"""
    return re.sub(r",\s*([\]}])", r"\1", s)


def _quote_keys(s: str) -> str:
    """给没加引号的键补上双引号：{name: 1} → {"name": 1}。"""
    return re.sub(r"([{,]\s*)([A-Za-z_]\w*)\s*:", r'\1"\2":', s)


def _single_to_double_quotes(s: str) -> str:
    """把双引号外的单引号换成双引号（简单场景：值里不含撇号）。"""
    out, in_dq = [], False
    for ch in s:
        if ch == '"':
            in_dq = not in_dq
            out.append(ch)
        elif ch == "'" and not in_dq:
            out.append('"')
        else:
            out.append(ch)
    return "".join(out)


def repair(s: str) -> str:
    """按从外到内的顺序修：围栏 → 抠 JSON → 字面量 → 逗号 → 键 → 引号。"""
    s = _strip_fences(s)
    s = _extract_json(s)
    s = _fix_literals(s)
    s = _drop_trailing_commas(s)
    s = _quote_keys(s)
    s = _single_to_double_quotes(s)
    return s


def loads(s: str):
    """容错解析：先走标准 json.loads，崩了就修复后再试。

    返回 (解析结果, 是否动过修复)。修不好 → 抛 json.JSONDecodeError。
    """
    try:
        return json.loads(s), False
    except (json.JSONDecodeError, ValueError):
        return json.loads(repair(s)), True

I ran it locally (Python 3.12, no dependencies), inputted the six bad JSONs above, and the actual output:

==============================================================
容错 JSON 解析器 · 6 个坏 JSON 实测
==============================================================

[markdown 围栏]
  原始 : ```json
{"city": "北京", "temp": 25}
```
  ✅ 修复后解析
  结果 : {'city': '北京', 'temp': 25}

[尾部多余逗号]
  原始 : {"city": "北京", "temp": 25,}
  ✅ 修复后解析
  结果 : {'city': '北京', 'temp': 25}

[单引号]
  原始 : {'city': '北京', 'temp': 25}
  ✅ 修复后解析
  结果 : {'city': '北京', 'temp': 25}

[Python 字面量]
  原始 : {"city": "北京", "is_rain": False, "note": None}
  ✅ 修复后解析
  结果 : {'city': '北京', 'is_rain': False, 'note': None}

[键没加引号]
  原始 : {city: "北京", temp: 25}
  ✅ 修复后解析
  结果 : {'city': '北京', 'temp': 25}

[前后夹说明]
  原始 : 查询结果如下：{"city": "北京", "temp": 25} 以上供参考。
  ✅ 修复后解析
  结果 : {'city': '北京', 'temp': 25}

==============================================================
小结：6/6 个坏 JSON 都解析成功。
  → 先试标准解析（零开销），崩了再修，别一上来就正则硬扛。
  → 教学版只兜常见坑；生产里单引号带撇号、深层嵌套这些，
    上 json5 / json-repair 这类专业库更稳。
==============================================================

All 6 passed.

3. Rules and steps

The step of trying standard parsing first cannot be skipped. Some people, to save effort, run all outputs through regex repair from the start. Don't. Properly formatted output could be mis-handled by your regexes; for example, if a value legitimately contains the word True, you replace it with true and change its semantics. Run json.loads once; if it passes, don't touch it—it's both more efficient and safer.

The single-quote step is the easiest to get wrong; I provided detailed comments. My _single_to_double_quotes scans character by character, maintaining a "are we inside double quotes" state, and treats any single quote outside double quotes as a double quote. This works fine for simple cases like {'a': 1}, but if a value itself contains an apostrophe—like a single-quoted string with an apostrophe inside {'note': 'it's fine'}—it will mis-handle it. So I wrote clearly in the code comments: The teaching version only catches common pitfalls; for production with apostrophes in single-quoted values, deep nesting, etc., use professional libraries like json5 or json-repair. You can't ignore that.

Where should this layer go? Between "model output" and "tool input". When the Agent gets model output, it first passes through this fault-tolerant parsing, then goes to the tool. That way tools always receive a clean dict, and each tool doesn't have to defend against JSON problems on its own. One layer catches it; don't scatter defenses everywhere.

Don't treat fault tolerance as a silver bullet. Fault tolerance catches "format problems", not "content errors". If the model gives you {"city": "北京"} with clean formatting but it's answering off-topic or inventing a non-existent city, this layer can't help; that's another line of defense (parameter validation, hallucination detection). You can't expect a parser to solve all problems.

4. What I'm doing on AI reliability

The 60 lines above are a small slice I extracted from AIHCC, an AI reliability tool. I rewrote it as a teaching version: same idea, but engineering details removed. The version running in production also includes schema validation; after parsing, the dict is checked against the tool's input parameter definitions—whether fields are all present and types are correct. That's the next line of checks.

If you're also connecting Agents to production (letting them call tools, touch data), you'll eventually hit this output parsing wall. The above code catches the common pitfalls first, better than digging through logs every time something crashes.

---

Code note: Example is based on Python 3.10+, pure standard library (json / re), no third-party dependencies, no network access; output in the text is from local tests (Python 3.12.2). Fault-tolerant parsing catches "dirty formatting" (fences/commas/quotes/literals), not "content errors" (off-topic, fabricated values); the latter requires separate parameter validation and hallucination detection. Single-quote handling is for simple scenarios; for values containing apostrophes or deep nesting, use json5 / json-repair professional libraries.

#Agent #大模型 #Python #工程实践 #可靠性