A Missed Parse Is a License to Fabricate

Posted by on · Topic: LLM Agents

LLM Agents web

TL;DR

Agents don’t fabricate because the model is bad. They fabricate because something in YOUR pipeline silently failed — a dropped system prompt, a parser that missed a tool call, a truncation with no label — and the model did what language models do: completed the gap with plausibility. Every lossy step between the model and reality must either succeed loudly or fail loudly. Silent is where the lies live.

Eddie is the embedded agent in my life-dashboard — a ReAct loop over a 172-operation personal-telemetry API, running gpt-oss-120b on Cerebras at ~300ms a round trip. This session I caught him fabricating twice, ran an external anti-confabulation audit, and came away with a conclusion I now consider load-bearing for any agent system: fabrication is almost always a pipeline bug wearing a model-behavior costume.

Case 1: The system prompt that rode only the first step

The ReAct loop sent the system prompt on step 0, then dropped it for subsequent steps to save tokens. gpt-oss, mid-conversation, forgot the protocol entirely — leaked its reasoning channel into the chat, and then simulated its own tool results. It produced five plausible audiomarks, complete with fake hex event IDs and oddly marketing-voiced passage text, that were provably never in the database. It wasn’t calling the tool and lying about the result; it was roleplaying the entire tool round-trip.

Fix: the system prompt rides EVERY step of the loop, plus an answer sanitizer that strips reasoning-channel leakage. With reasoning models on OpenAI-compatible APIs, also note: content can come back None when max_tokens dies mid-reasoning, and how the reasoning channel is separated varies by provider. Handle both or inherit someone else’s serialization bug as a “hallucination.”

Case 2: The regex parser that couldn’t count braces

The tool-call parser extracted JSON from model output with a regex that handled one level of nesting. The first live test of the new api_get tool emitted a depth-3 call:

{"tool": "api_get", "args": {"params": {}}}

The regex missed it. The un-executed call leaked into the answer text, and the model — having announced it would check the cached-books endpoint and received nothing — invented the result: “42 cached books.” The real number was 4.

This is the sharpest version of the principle: a missed parse is a license to fabricate. The model committed to a tool call; the harness silently didn’t execute it; the model filled in what the result “should” have been. The fix was a string-aware brace-walking JSON extractor — walk characters, track depth, respect string literals and escapes — replacing both the parser and the answer sanitizer. Regex JSON extraction for LLM tool calls is a fabrication factory with a delay timer. Brace-walking is maybe thirty lines. Write the thirty lines.

Case 3: Truncation without a label

An external anti-confabulation audit found the through-line behind the smaller weirdnesses: prose reached the model lossy and unlabeled. Fields silently truncated at 300 chars. JSON null slipping into string slots (a live None[:300] TypeError class, when it wasn’t crashing, was producing garbage context). The model, handed a sentence that just stops, does the only thing it knows: completes it. Structured fields — counts, IDs, timestamps — were always honest. Free prose was where every fabrication lived.

Fixes, all boring, all effective:

  • Every truncation appends an explicit … [TRUNCATED] sentinel, plus a system-prompt rule forbidding completion past the marker.
  • Caps raised (300→700 per field, 4000→8000 per result) — most truncation simply shouldn’t happen.
  • _safe_payload coerces JSON null before any slicing.
  • consult_memory (the Honcho dialectic tool) prefixes “About Wei:” server-side, so the memory is consulted about the user and never speaks as him.

One correction for the audit-believers: the audit’s smoking-gun story — synthetic DB fixtures polluting answers — was disproven against the actual database. Verify even sharp audits. They confabulate too.

The boundary that makes self-discovery safe

The same session added agent self-discovery: api_docs searches the app’s own openapi.json (~170 endpoints), and api_get executes GETs against an allowlist that excludes auth-ish and binary routes. The principle that fell out:

Knowledge can scale automatically; agency only by hand.

Eddie’s read surface now grows for free with every endpoint I ship. His write surface stays at five hand-curated tools (kanban add/move/delete, write journal, create audiomark). This is the middle path between two bad defaults — a hand-maintained tool list that rots, and tool-per-endpoint bloat that hands the model a loaded weapon per route. Reads are cheap to grant and self-documenting via OpenAPI. Writes are where a fabricating agent becomes a destructive one, so writes get a human in the loop per tool, forever.

Takeaways

  • System prompt on every step of a ReAct loop. Token savings from dropping it are paid back in protocol amnesia.
  • Never regex-extract tool-call JSON. String-aware brace-walking, depth-N, ~30 lines.
  • Label every cut: truncation sentinels plus a prompt rule. Models complete dangling clauses — that’s the whole job description.
  • Coerce nulls before slicing. A None[:300] is either a crash or a silent context hole; both end in fiction.
  • Structured fields don’t get fabricated; unlabeled prose does. Push facts into structure where you can.
  • OpenAPI self-discovery for reads, hand-curated tools for writes.
  • Audit the audit. Mine was right about the mechanism and wrong about the smoking gun.

The uncomfortable part is that none of these fixes made the model smarter. They made the pipeline honest, and the fabrication stopped. If your agent is lying to you, check what you’re feeding it before you check what it’s made of.