Structured output
In short
Structured output is a feature that forces an AI model to reply in a fixed, machine-readable format, usually JSON matching a template the developer supplies. It lets software read the answer directly instead of parsing free text.
- Also called
- Structured outputs, JSON mode, Constrained decoding
- Topic
- Using AI tools
- Level
- Some background helps
- First described
- OpenAI launched JSON mode at its 2023 DevDay and full Structured Outputs, which match a supplied JSON Schema exactly, on 6 August 20242.
Chatbots write prose, but software needs data in predictable fields. Ask a model to "return JSON" and it usually will, but not always. Anthropic lists the usual failures without the feature: invalid JSON, missing required fields, wrong data types and replies that break the template1.
Structured output fixes this with a JSON Schema, a standard way of describing which fields are allowed and what type each must be. The developer sends the schema with the request, and the model's answer is guaranteed to match it12.
The guarantee comes from constrained decoding. A model writes one token at a time. At each step the system blocks any token that would break the schema, so only valid output is possible2. A 2023 paper by Brandon Willard and Rémi Louf showed how to do this efficiently by turning a format into a finite-state machine, and released it in the open-source Outlines library3.
OpenAI added the feature to its API on 6 August 2024. It said its gpt-4o-2024-08-06 model scored 93% on its complex schema test through training alone, and 100% with constrained decoding switched on, against under 40% for the older gpt-4-06132. Anthropic's Claude API and Google's Gemini API offer the same idea14.
It controls format, not truth. A perfectly valid JSON record can still hold a wrong date or a misread name. Models can also still refuse a request: OpenAI returns a separate refusal message rather than forcing harmful content into the template2.
Example
An accountancy firm sends scanned receipts to a model with a schema asking for supplier, date, VAT amount and total. Every reply comes back as the same four fields, ready to drop straight into a spreadsheet.
Why it matters
It is what lets AI be wired into ordinary software reliably, from form-filling to data extraction. Without it, many automated AI workflows would break on the first oddly formatted answer.
Questions people ask
What is the difference between JSON mode and structured outputs?
JSON mode makes a model produce valid JSON, but OpenAI notes it does not guarantee the reply matches a particular template. Structured outputs go further and ensure the response exactly matches the JSON Schema the developer supplied2.
Does structured output make AI answers accurate?
No. It guarantees the shape of the answer, not the content. Constrained decoding only blocks tokens that would break the format2, so a field can still contain a wrong figure. Important values still need checking.