Deterministic extraction vs generation when the output is a record

The difference is in what the engine is permitted to say

Two systems can be handed the same document, asked the same question, and differ in a way that has nothing to do with how capable either one is. The difference is in the output alphabet. A span-copying extractor draws its answer from a closed set — the substrings of the document in front of it, plus a fixed list of labels. A generative model draws its answer from its entire vocabulary, in any order. Nearly everything downstream — the audit story, the failure modes, the recall gap — falls out of that one structural fact.

The distinction is old and well-studied under the names extractive and abstractive. The pointer-generator architecture made it unusually explicit: the model carries a scalar that decides, at each step, whether to copy a token from the source or generate one from the vocabulary (See, Liu and Manning, "Get To The Point: Summarization with Pointer-Generator Networks," Proceedings of ACL 2017; the copy mechanism itself descends from Vinyals, Fortunato and Jaitly, "Pointer Networks," NeurIPS 2015). Set that switch to copy-only and you have given up paraphrase, normalization and synthesis in exchange for one property: every token you emit is a token that was already there.

That property is the whole argument. For a copy-constrained engine, the question did this text appear in the source? is answered by string containment against a stored offset — no second model, no human re-reading the document. Verification uses a different mechanism than production did. For a generative system the same question requires a fresh judgment, human or model, and that judgment is itself fallible and itself unmeasured until someone measures it.

The guarantee is narrower than the marketing version of it

What the copy constraint rules out is a specific failure class. The hallucination literature splits it usefully: intrinsic errors contradict the source, extrinsic ones cannot be verified from the source at all (Maynez, Narayan, Bohnet and McDonald, "On Faithfulness and Factuality in Abstractive Summarization," ACL 2020; the taxonomy is carried forward in Ji et al., "Survey of Hallucination in Natural Language Generation," ACM Computing Surveys 55(12), 2023, arXiv:2202.03629). NIST's Generative AI Profile lists the same behavior as confabulation among the risks specific to generative systems (NIST AI 600-1, July 2024). A copy-only engine cannot commit the extrinsic error. There is no path in the architecture for content with no antecedent in the source to reach the output.

It can still be wrong, and the ways it goes wrong are unglamorous. It can quote a negated finding as if it were affirmed — the reason clinical NLP has had dedicated negation logic since Chapman et al.'s NegEx algorithm (Journal of Biomedical Informatics 34(5), 2001). It can attach a correctly-copied value to the wrong entity. It can lift a span out of a conditional or hypothetical clause where the surrounding scope reversed its meaning, or out of a history section where the tense did. Every one of those produces a perfectly traceable citation to a real span that supports a false conclusion.

Six questions that separate the two architectures

What a reviewer asks	Engine limited to spans it located	Engine that can generate text
Can the output contain a statement absent from the source?	No — the output alphabet does not include it.	Yes. That capability is the point of generation, not a defect in it.
How is one output line verified?	String containment against stored offsets. No model in the loop.	A second judgment — human or model — which is itself fallible and needs its own evaluation.
Same input twice, same output?	Yes, given a fixed rule set and index.	Not guaranteed. On our own benchmark, three trials returned a different item set on 11 of 60 documents.
Can it normalize, paraphrase, or convert units?	No. It returns the words as written.	Yes, and this is frequently the reason to choose it.
Can it state a relationship no single span expresses?	No. Synthesis across records is outside its output space.	Yes.
What does failure look like?	A silent miss, or a correct quotation of the wrong span.	A miss, a wrong span, or a fluent statement with no source behind it.

The recall price, with our own number in it

The trade is not free and we have measured what it cost us. On 60 discharge documents carrying 516 hand-labeled gold items, a current frontier model out-recalled our deterministic extractor 97.9% to 68.2%. On medication changes — the category that matters most — ours managed 56.6%. That is a large loss and we publish it as one: the full benchmark, including the number that goes against us.

The mechanism of that loss is structural rather than incidental. A rule-governed extractor fires only when the phrasing in front of it is one the rule set recognizes. An author who states the same fact in a construction nobody anticipated produces a miss, and the miss is silent — from the engine's point of view, nothing matched. A generative model reads around phrasing. That is not mysterious; it is generalizing over surface form, which is exactly what copy-only rules cannot do. Every increment of recall a rule set gains is paid for in rule authoring against real documents, and the curve flattens.

The counterweight matters too, and it cuts against the easy story. On that benchmark the frontier model was disciplined: every source span it cited was actually present, it preserved 39 of 39 numeric thresholds on every trial, and across 180 document-runs it introduced exactly one number the source did not contain. If your mental model of a 2026 frontier model is a confident fabricator, it needs updating. What it did do was talk more — 657 emitted items against 516 gold, at a 6.1% wrong-item rate versus our 2.6%. Part of its recall lead was better reading and part of it was volume, and you cannot tell those apart from a recall figure alone.

The thing a span engine must never be allowed to say

There is a second, sharper limit that gets missed. An engine that only re-presents what it located cannot assert what a record does not contain. "No restriction was documented" is not an extraction — it is a claim about absence, and absence is only knowable under a closed-world assumption over a defined corpus. The engine has no access to that assumption. Its silence on a topic is indistinguishable from a miss, and we already established the misses are silent.

This is a governance boundary, not a technical curiosity. If a downstream consumer reads an empty field as "the source says no," a 68.2% recall rate has quietly become a 31.8% rate of false negative assertions in a record that someone signs. When absence genuinely has to be reported, it needs to be a separate, explicitly scoped piece of logic with its own corpus boundary and its own evaluation — never an inference drawn from the extractor's output being empty. We write that boundary into the interface: fields are found or not found, and "not found" is never rendered as "none."

Constrained decoding guarantees form, not provenance

Schema-constrained generation is the popular middle path and it is genuinely useful, so it is worth being precise about what it does. Grammar- and schema-guided decoding masks the token distribution at each step so that only continuations consistent with a formal grammar remain reachable (Willard and Louf, "Efficient Guided Generation for Large Language Models," arXiv:2307.09702, 2023; the same technique appears as grammar files in local runtimes and as JSON-Schema-constrained modes in current hosted APIs). The output parses. Enum values are legal. Required fields are present.

None of that says the value came from the document. A schema-valid record containing an invented date is still an invented date, and it is arguably more dangerous than a malformed one, because malformed output stops at the parser while well-formed output flows straight into a database. We treat schema validation as a plumbing check and never as an evidence check, and we say so in writing when we hand over an evaluation.

Reproducibility deserves the same care. Temperature zero is not the same thing as bit-reproducible: published engineering analysis attributes run-to-run variation under greedy decoding to batch-size-dependent reduction order in GPU kernels rather than to sampling, since floating-point addition is not associative (Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference,” September 2025). You do not have to accept any particular explanation to act on it: run the same input twice, on the serving configuration you will actually deploy, before you believe a benchmark.

When generation is the correct choice

We are not arguing against generation, and a firm that only ever recommends the thing it happens to build is not giving advice. The deciding question is not which architecture is better. It is what the output is consumed as — a draft that a human will revise, or a record that gets filed, signed, or acted on.

Drafting under human review. If every line is read before it goes anywhere, generation's recall advantage is nearly free and its fluency is a real benefit. The reviewer is the provenance layer.

Coverage sweeps where a miss is expensive and over-production is cheap. If a human reads everything anyway and the cost of missing an item dwarfs the cost of checking a wrong one, tune toward recall and accept a higher wrong-item rate. That is the correct engineering answer for that job.

Normalization, coding and unit conversion. Mapping "40 mg twice daily" into a structured dose, or a free-text description into a code system, is by definition not a span-copying task. The target value does not appear in the source. Refusing generation here means refusing the task.

Exploration over an unbounded phrasing space. When you do not yet know what the documents say, rule authoring has nothing to aim at. Generation is the right instrument for finding out what categories even exist, before anyone writes a gold standard.

The composition we actually ship

These are not mutually exclusive, and the useful architecture is usually both. Generation proposes; extraction admits. A generative pass produces candidate claims over the corpus; every candidate must then carry a span with offsets into a named source record before it is permitted into the output. A candidate with no span is surfaced as an unsupported suggestion or dropped — never silently rewritten into the record. That recovers part of generation's recall without moving the provenance boundary, because the boundary is enforced by a mechanism the generator does not control.

It is not free either, and the honest accounting has three entries. Two passes cost latency and compute. The span check has to be evaluated on its own, because it will happily pass a span that is present but does not actually support the claim attached to it. And the composition adds a component whose failure is a false admission, which is the failure mode with the worst blast radius, so it is the one that needs the most test data.

Where this pays for itself is at the accreditation boundary. Systems entering a Department of Defense authorization inherit the Impact Level structure defined in the DoD Cloud Computing Security Requirements Guide published by DISA, and the audit expectations underneath are ordinary NIST SP 800-53 Rev. 5 control language — AU-3 on the content of audit records, AU-10 on non-repudiation. An output line that carries a document identifier and a byte offset satisfies that reading almost mechanically. An output line that carries a model version and a prompt does not, because a reviewer cannot reconstruct the claim from it. And where the output is a federal record in the statutory sense (44 U.S.C. § 3301), reconstructability is not a nice-to-have — it is the thing the record is for.

What we hold ourselves to, and what we refuse

The standard we work to on extraction-for-records engagements:

• Every emitted statement carries a source identifier and offsets — checkable by a party who does not have our engine.
• "Not found" is never rendered as "none" — absence claims come from separately scoped logic with a defined corpus boundary, or they do not appear.
• Recall and false-extraction rate reported together, per category, on a split labeled before the engine ran against it.
• Repeat runs on identical input, on the serving configuration that will actually be deployed.
• Selection error measured separately from provenance — negation, scope, and entity attribution get their own counts.
• The match rule published in full, including how partial matches are scored.

The refusals matter more, and they are the reason this page exists. We will not put a free-generation path on the last mile of a record that is filed, signed, or acted on with no human between it and the consequence — not because generation is careless, but because a record whose content cannot be reconstructed is not a record. We will not describe determinism as accuracy, or call a span-grounded engine "hallucination-free" without stating in the same sentence that it can cite the wrong span. We will not sell a rule-governed extractor into a problem whose phrasing space is genuinely open; that is a recall commitment we would fail, and we would rather say so before the contract than after. We do not do human-subjects research and we are not a clinical or regulatory validation laboratory — a software measurement is not a safety determination, and we always say which one we are making. And we will decline an engagement that wants the provenance guarantee and generation's recall on the same output path unless the customer will state which one governs when they conflict. They will conflict. Deciding it in advance is cheaper than discovering it in an audit.

Common objections

Frequently asked questions