01
what the provider actually guarantees
If you extract invoices, tickets or contracts with an LLM and write the result to a database, you've probably turned on structured outputs and stopped seeing parse errors. That part works. The trouble is that a response which matches your schema can still carry a wrong total, an invented SKU or a confident enum value that nobody would have picked, and none of that shows up as an error anywhere.
It helps to be exact about what each mode promises. The older JSON mode (response_format: {"type": "json_object"} in OpenAI's Chat Completions, text.format in the Responses API) only promises syntactically valid JSON. OpenAI's own docs say it does not guarantee the output matches any particular schema. Strict structured outputs are a different thing: OpenAI shipped them on 6 August 2024, both as response_format with type: "json_schema" and strict: true and as strict: true on function definitions. The model's decoding is constrained to the schema, so required keys are present, types match and enum fields contain one of your values.
Anthropic took longer. Until November 2025 the usual trick was to define a tool with an input schema and force the model to call it, which shapes the output well but doesn't constrain decoding. Structured outputs arrived as a public beta on 14 November 2025 (header structured-outputs-2025-11-13, Sonnet 4.5 and Opus 4.1 at first) and came out of beta on 29 January 2026, when the parameter moved from output_format to output_config.format. Strict tool use ("strict": true on a tool definition) is the same mechanism applied to tool inputs.
The two providers don't accept the same JSON Schema subset, and this matters when you generate the schema from your models. OpenAI's strict mode requires every property to be listed in required (optional fields become a union with null), requires additionalProperties: false on every object, and supports pattern, a fixed list of format values, minimum, maximum, multipleOf, and minItems/maxItems. It rejects allOf, not and the if/then/else family. Anthropic's current docs list minimum, maximum, multipleOf, minLength and maxLength as unsupported, and minItems only for the values 0 and 1. A raw request that uses them gets a 400, while the Python and TypeScript SDK helpers strip those keywords, append them to the field description and, for the helpers that validate, check the response against the original schema on your side. So on Anthropic a "minimum": 0 is a hint to the model plus a client-side check, not a decoding guarantee.
Neither provider can express a rule that spans two fields. "Line items add up to the subtotal" and "the due date comes after the invoice date" are outside JSON Schema entirely, and they're exactly the rules that keep financial data coherent.
02
the failures a valid schema lets through
The classic case is an enum that's always valid and sometimes wrong. Give a field payment_terms the values net_14, net_30, net_60 and immediate, and constrained decoding guarantees one of the four. On a blurry scan with no visible terms, the model still has to pick one, because the grammar gives it no other option. You get a plausible value, a passing schema and a wrong record in the accounting system.
The fix is cheap: add an explicit unknown (or not_stated) value to every enum where "none of these" is a real answer, and route those records to review instead of treating them as data. The same goes for nullable fields. If the document might not contain a due date, make the field date | null and tell the model in the field description that null is the right answer when the date isn't printed. A model forced to fill a required string will fill it.
Free-text fields have no such guard. An extracted SKU that matches your pattern regex is indistinguishable from a real one at the schema layer, even when the model made it up because the original was smudged. The only defence is checking it against something you know, such as your product table or the vendor's catalogue.
Arithmetic is the last big category. Constrained decoding works token by token, so it can force total to be a number but has no way to make it equal the sum of line items the model wrote a few hundred tokens earlier. For anything financial we'd treat a reconciliation check as mandatory.
03
check the stop reason before you parse
A constrained response can still be cut off. If generation hits the output token limit, you get JSON that stops mid-string and doesn't parse, not a shorter object that happens to validate. OpenAI's Responses API marks this with status: "incomplete" and incomplete_details.reason: "max_output_tokens" (Chat Completions uses finish_reason: "length"), and Anthropic returns stop_reason: "max_tokens" with a note in the docs that the output may not match the schema.
Refusals are the other exception. Both providers let the model refuse even with a schema, and a refusal doesn't follow your schema. OpenAI returns a separate refusal field, and Anthropic returns stop_reason: "refusal" with a 200 status. Anthropic's docs also mention that asking for step-by-step reasoning inside a schema property can trigger a refusal, so ask for a short explanation instead.
So the first thing your handler does is look at why generation stopped, and only then parse. A truncated response is not worth retrying with the same token budget and the same document, because it will usually truncate again. Raise the limit, split the document, or send it to the dead-letter table with the raw text.
04
a contract layer in pydantic
Validation belongs in your code, close to the database write, as a model written to reject things rather than describe them. This contract is the whole invoice, with money as Decimal, blank strings rejected after stripping, unknown keys forbidden and the cross-field rules in one validator:
A few details are deliberate. model_validate_json parses JSON numbers straight into Decimal, so 1234.5 arrives exact and the equality checks need no float tolerance. decimal_places=2 turns a VAT of 777.745 into a validation error instead of silently rounding it. A plain Field(min_length=1) accepts " ", which is why NonBlank strips first. And due_date: date | None has no default on purpose, so the field is required but nullable, which is the shape OpenAI's strict mode wants anyway. The code runs on current Pydantic 2 (2.14 at the time of writing), and the date | None syntax needs Python 3.10 or newer.
Don't send this model's generated schema to the provider unchanged. Invoice.model_json_schema() renders every Decimal as anyOf number or string and drops max_digits and decimal_places, and it emits minLength, which Anthropic rejects in a raw request. Treat the provider schema as a looser projection of the contract that steers the model, and the Pydantic model as the thing that decides what gets stored. The contract is also the most honest documentation of what your pipeline assumes, so version it (invoice.v3) and keep the old version around while a prompt change rolls out.
05
retries and a dead-letter table
When validation fails, one retry with the errors fed back is worth it, because the model often fixes a specific, named mistake. Format the errors from ValidationError.errors() rather than dumping str(e), and keep the budget small:
call_model is your provider wrapper, which maps OpenAI's and Anthropic's stop signals onto the three normalised values. A total that doesn't reconcile comes back to the model as - (root): Value error, subtotal + vat != total, which is specific enough to act on.
Whatever still fails goes to a dead-letter table, not to an exception that kills a batch of five hundred documents and not to a log line nobody reads. The pipeline carries on, nothing is silently dropped and nothing is silently accepted:
The columns are the ones you need when you go looking for patterns later: which contract, model and prompt version produced the failure, why generation stopped, and the exact errors. Records with payment_terms = 'unknown' can go to the same review queue even though they validated, because they're the model telling you it didn't know.
06
watching the distribution, not just the failures
Validation catches records that break a rule. It can't catch a model that has started picking net_30 for documents that used to come out as net_14, because each record is still valid on its own. That kind of drift only shows up in aggregate, usually right after a model upgrade or a prompt edit, so store the model and prompt version on every extracted row and look at the value distribution per version:
If net_30 jumps from 20 to 60 percent in the week p7 replaced p6 and your customers didn't change, the prompt changed the answers. The same query on the unknown share tells you whether a change made the model more or less willing to admit it can't read a field, and a falling unknown rate isn't automatically good news. Run it per field that drives money or routing, alert on large week-over-week shifts, and pair it with a small random sample of records that a person checks against the source document. Random sampling is what finds the systematic mistake that every individual check passes, the one where a reviewer looks at a handful of records and notices they're all wrong in the same way.
