Getting Reliable Structured Outputs from LLMs: Validation, Retries and Constrained Decoding

The Problem of Semantic Hallucination in Structured Data
The industry has reached a consensus that structured output is a standard requirement for modern AI applications. With OpenAI’s Responses API utilizing JSON schema or Pydantic models, and platforms like vLLM implementing xgrammar for local deployment, developers can now enforce rigid syntactic structures. However, a prevailing misconception is that successfully parsing a JSON string is equivalent to achieving data accuracy.
This issue is best illustrated by the processing of financial invoices. Consider a standard invoice for Northwind Traders totaling EUR 1,250.00. An LLM tasked with extracting the invoice ID, currency, total amount in cents, and a textual quote might produce the following JSON object: "invoice_id": "2026-0417", "currency": "EUR", "total_cents": 12500, "total_quote": "Total due: EUR 1,250.00".
From a technical perspective, this output is perfect: it is valid JSON, adheres to the defined schema, contains the correct types, and includes a verbatim quote from the source document. However, it is fundamentally wrong. The model interpreted "1,250.00" as 125.00 EUR (12,500 cents), failing to account for the standard decimal notation used in the source. Because 12,500 is a valid integer, no schema validator will flag this as an error. This "semantic hallucination" demonstrates that while syntax can be constrained, truth must be verified against the source text by the application layer.
A Three-Layer Architecture for Reliability
To solve for accuracy, development teams are increasingly adopting a three-tiered defense strategy: strict validation, controlled retries, and constrained decoding.
- Validation: This layer involves checking the returned object against a predefined contract—typically managed via Python’s Pydantic library—and subsequently comparing the data points against the original source document.
- Controlled Retries: Rather than an infinite loop of regeneration, retries are restricted to a maximum of three attempts and are only triggered when a specific, fixable error occurs.
- Constrained Decoding: By masking invalid tokens during the generation process, the model is physically prevented from emitting tokens that would violate the defined schema.
Defining the Contract: The First Gate
Before a model is even invoked, the application must define a rigid data contract. By utilizing Pydantic in "strict mode" with extra="forbid", developers can effectively shut down the model’s tendency to coerce types or invent fields.
For instance, if an invoice_id field expects a string and the model provides an integer, strict mode will trigger an int_type error rather than attempting to cast the value. Similarly, extra="forbid" ensures that any unexpected data—such as an unrequested "vendor" field—results in an extra_forbidden error, forcing the model to adhere strictly to the schema. This creates a high-fidelity gate that filters out structural noise before the more complex semantic validation begins.
Semantic Verification and the Acceptance Pipeline
Once the object passes the structural gate, the application must perform a secondary check against the evidence. The proposed accept() function represents a robust approach to this process. It categorizes outcomes into four distinct states: accepted, needs_review, rejected, or shape_error.

In this workflow, shape_error is reserved for parsing or schema-mismatch issues. rejected is the terminal state for logical errors, such as the total amount failing to match the quoted source text. needs_review acts as a safety valve for ambiguous data, where the model correctly identifies that a field is missing or unstated, allowing human intervention rather than forcing an automated guess. By treating "missing information" as a valid state rather than an error, the pipeline maintains integrity and avoids the risk of data fabrication.
Intelligent Retry Policies
The implementation of a retry loop must be deliberate. A common pitfall is the implementation of an infinite retry loop, which can lead to runaway costs and latency. A disciplined approach limits the "attempt budget" to three, acknowledging that if a model cannot produce a correct, schema-compliant output in three iterations, the prompt or the source document itself is likely the limiting factor.
Crucially, retries should only be triggered for shape_errors. If the model provides a well-formed JSON object that contains factually incorrect data, the system must immediately move to a rejected status. Repeatedly querying the model for a value that it has already incorrectly extracted from a static source document is a futile exercise that wastes tokens and increases latency.
The Role of Constrained Decoding
The most advanced layer of this architecture is constrained decoding. By restricting the model’s token output space at the decoder level, developers can prevent structural errors before they happen. Technologies like OpenAI’s client.responses.parse method allow developers to provide a Pydantic model directly to the API. This forces the model to generate tokens that adhere to the structure, eliminating the need to parse code fences or handle malformed JSON.
However, even with constrained decoding, semantic validation remains non-negotiable. While the decoder can ensure total_cents is an integer, it cannot determine if that integer correctly reflects the source text’s currency format. Consequently, even the most sophisticated constrained models must still be passed through the accept() validation layer.
Broader Implications for Enterprise AI
The shift toward these rigorous validation patterns marks a turning point in the adoption of generative AI. As businesses transition from using LLMs for creative tasks to utilizing them for high-stakes, automated data processing, the requirement for auditability and correctness becomes paramount.
Data from recent industry benchmarks suggest that implementing these validation layers can reduce the error rate in automated extraction tasks by up to 85%. Furthermore, by explicitly defining an error-handling policy—specifically, distinguishing between structural failures, logic errors, and missing information—organizations can build systems that fail gracefully.
In conclusion, the path to reliable structured outputs lies not in prompt engineering alone, but in the synthesis of strict data contracts, controlled retry logic, and programmatic verification against source evidence. By treating LLM outputs as untrusted data until they have cleared these three layers, developers can effectively bridge the gap between the probabilistic nature of generative models and the deterministic requirements of enterprise data pipelines. The future of AI integration in the enterprise will be defined not by the model’s ability to "think," but by the developer’s ability to constrain, validate, and verify that thought process against the ground truth of the real world.







