Stop Blaming the Model: A 60-Line Parser Fixes the JSON LLMs Routinely Break
Agent-tool integrations routinely crash on trivial format errors; a 60-line parser removes that class of failures before you start blaming the model. It's a stopgap, though—production systems should still pair it with schema validation or a mature library like json-repair.
LLM-generated JSON often breaks standard json.loads when an agent calls a tool. Six recurring habits explain the failures: wrapping output in markdown code fences, trailing commas, single quotes, Python literals like True/None instead of true/null, unquoted keys, and explanatory text around the JSON. None of these are capability gaps; they're style artifacts from training on mixed code.
The fix is a 60-line parser that tries plain json.loads first and, on failure, runs a six-step repair: strip fences, extract the outermost braces or brackets, convert literals, drop trailing commas, quote keys, and convert single quotes to double quotes. Tested against all six malformed inputs on Python 3.12.2, it repaired every one. The layer belongs between the model output and the tool input, so tools receive clean dicts without each tool defending against format issues.
The parser handles format errors only; it can't catch hallucinated or off-topic values. Those need separate parameter validation and hallucination detection. The production version adds schema validation. For edge cases like apostrophes inside single-quoted values or deeply nested structures, the guidance is to switch to json5 or json-repair rather than extending the regex.
The article reframes LLM JSON failures as style habits rather than capability gaps, which pushes debugging effort toward a translation layer instead of model swapping.
The repair pipeline's fixed order matters: later steps assume earlier fixes, making it a deliberate micro-workflow rather than a bag of regexes.
The warning about single-quote handling—that apostrophes inside values will corrupt data—shows the boundary of regex-based repair; the honest advice to switch to a library for production is the right call.
Two malformed shapes your version will miss. First: when the top level is an array, _extract_json picks the wrong bracket — the result is: [{"a":1},{"b":2}] it hits the { branch first, extracts and turns it into {"a":1},{"b":2}, then reports Extra data. Your six cases are all objects, so you never hit that. Second, more troublesome: _quote_keys's regex doesn't track quote state; {"msg": "a, b: c"} after fixing becomes {"msg": "a, "b": c"}; \bTrue\b also lowercases "answer": "True or False?". Normally it doesn't fire because the first loads passes; but once a trailing comma is also present, it gets corrupted along with it, parsing succeeds but the semantics change. Also truncation. When output is cut by max_tokens, after repair it reports char 28 while the original length is 39 — that coordinate is from the repaired text. Following the error position back into the log makes it look like the model wrote incorrectly at the same spot, when in fact the whole chunk is missing. So on my side I record the original length and whether it hit max_tokens, not the parser's line/column numbers.
Thanks for the addition, bro. I'll debug it in a bit and update it in my code; I won't add it to the article. [thumbs up][fist][fist]