What a span actually is
A citation in a research paper points at a document. A span points at a location inside a document — a record identifier, a start offset, an end offset, and the exact characters between them. “This came from the 2024 filing” is an assertion about a document, and the only way to check it is for a person to go read the filing. “This came from characters 41,207 through 41,349 of record F-2024-0881, whose contents hashed to the value we recorded at read time” is a claim a machine can check in microseconds without asking anyone's opinion — including the model's.
We build small models that read through a body of records and produce a written conclusion. One design constraint shapes everything else in that system: every sentence in the conclusion carries a pointer to the exact span of the exact record it came from, and a sentence that cannot carry one does not get written. This article is about what that constraint buys, what it costs, and where it stops working.
Why “add the citations at the end” is a different product
The common architecture goes the other way around. Generate the paragraph first, then run a second pass — a retrieval query, a natural-language-inference model, or another language model — to find a passage that supports each sentence. In the interface, the result is indistinguishable from provenance. Underneath it is a different object: the second pass finds support for a sentence that already exists, and it cannot make the sentence have come from there.
That distinction is not theoretical, and it has been measured on shipped systems. Liu, Zhang and Liang audited four deployed generative search engines and reported that, on average, 51.5% of generated sentences were fully supported by their citations, and 74.5% of citations supported the sentence they were attached to (Evaluating Verifiability in Generative Search Engines, Findings of EMNLP 2023, arXiv:2304.09848). Every one of those systems displayed citations. Displaying a citation and having the property are separate facts.
The alternative is to change the order of operations. Slobodkin, Hirsch, Cattan, Schuster and Dagan call it attribute first, then generate (ACL 2024, arXiv:2403.17104): select the source segments, then condition generation on exactly those segments, so the selected segments are the attribution. Attribution stops being a repair step and becomes a consequence of how the text was produced. That is the family our systems live in.
The evaluation vocabulary is settled enough to borrow. Rashkin and colleagues' AIS framework — “Attributable to Identified Sources” — defines the property carefully: a standalone interpretation of the statement must be supported by the identified source (Computational Linguistics 49:4, 2023). Read it before designing an attribution evaluation. It forces you to decide what “standing alone” means, which is where most attribution metrics quietly cheat.
The engineering is mostly about not moving the text
Spans are offsets, and offsets are meaningless unless the text they index is frozen. The failure mode is boring, silent, and the one that actually happens: a document is extracted, spans are recorded, and then something downstream normalizes the text. Unicode is re-normalized from NFC to NFKC. Whitespace is collapsed. A hyphenated line break is rejoined. A running header is stripped. Every offset recorded before that point now indexes different characters. Nothing throws. The citations still render. They point at the wrong words.
So most of the discipline is negative — things the pipeline is not allowed to do:
- Canonicalize once, then store it — the extraction output is the record of truth, not the source file. Never re-derive it at read time; two runs of an extractor will not always agree.
- Hash the canonical text, carry the hash on the span — a span whose record hash no longer matches is a broken pointer, and it should fail loudly rather than render politely.
- Keep formatters away from cited text — rendering, wrapping and highlighting operate on a copy; the indexed string is never the string a formatter touched.
- Verify by re-slicing, not by inspection — cut the record at the cited offsets and compare byte-for-byte against what the output claims. This is a unit test, not a model call.
- Treat document reading order as a hazard — two-column layouts, footnotes and tables interleave in extraction. Whatever the extractor decided, that decision has to be stored alongside the offsets or the offsets do not survive a version bump.
- Log the corpus snapshot — a span is scoped to a version of a record set. Without the snapshot identifier, a citation that was true in March is unfalsifiable in July.
Verification is the step people underestimate. It costs almost nothing — a substring comparison — so it runs on every output in production rather than on a sample, and it is the only check in the stack that does not depend on the thing being checked.
One line drives the whole design
A sentence that cannot carry a verifiable span is not emitted. Not flagged, not scored, not hedged — not emitted. Everything below that reads like a cost is a direct consequence of that single rule.
What it costs in recall
The constraint deletes a large class of statements that are true. If a conclusion requires arithmetic across records, a multi-hop inference, a comparison over time, or a negative finding, there is no single span that says it. The corpus implies it. Implication has no offsets.
There are three honest ways to handle that, and each costs something real.
Don't say it. Highest precision, lowest usefulness. Perfectly defensible and frequently useless — a system that will only quote is a search engine with better manners.
Type it differently. A derived statement carries an operation identifier and the list of input spans instead of a single span, so a reader can tell a quoted claim from a computed one. This is what we usually build. The cost is that every operation has to be defined, implemented and tested individually, and the interface has to communicate two kinds of statement without confusing the reader.
Emit it unsourced with a marker. We do not do this. In every deployment we have watched, the marker stops being read, and then the unsourced sentences are the ones that get quoted upward.
Negation deserves its own note. “The records contain no evidence of X” cannot anchor to a span, because absence has no location. It can only anchor to a query, a corpus snapshot identifier and a timestamp. That is a genuinely weaker form of provenance and it should be labeled as one, not dressed in the same footnote styling as a quotation.
And the headline cost: a verbatim-grounded system will lose recall benchmarks to an unconstrained generator, sometimes badly. We published our own measurement of exactly that — an extractor we shipped scored 68.2% recall against a frontier model's 97.9% on the same corpus and the same scorer (the full benchmark, including the number that goes against us). If a vendor tells you grounding is free, they have not measured it.
Three ways to attach a citation, and what each one survives
| Approach | What the pointer resolves to | What breaks it |
|---|---|---|
| Document-level citation | A record identifier. A reader has to find the supporting text themselves. | Nothing mechanical — which is the problem. It cannot be falsified without a human reading the whole record. |
| Post-hoc passage matching | A retrieved passage that scores as similar to a sentence already written. | Near-miss matches. A softened threshold, a dropped qualifier or two fused records still return a plausible passage. |
| Span-level, attribute-first | Character offsets into a hashed, frozen canonical text, produced before the sentence was written. | Any re-normalization of the indexed text, an unrecorded extractor change, or a corpus snapshot that was not pinned. |
Why this matters at an accreditation boundary
The Defense Information Systems Agency defines cloud impact levels in the DoD Cloud Computing Security Requirements Guide, published on DoD Cyber Exchange. IL5 covers higher-sensitivity controlled unclassified information, mission-critical information and national security systems; IL6 is reserved for information classified up to SECRET and runs in a dedicated enclave. For a model that has to drop into one of those environments, the practical consequence is one rule: assume no outbound call.
That has a specific implication for provenance design. If verification is a call to a hosted model or an external retrieval service, verification does not exist inside the enclave. Span verification survives the boundary because it is a string comparison against records that are already there — no network, no external service, no second model, no vendor dependency at the moment of checking. That is not an incidental property. It is a large part of why we build this way rather than the easier way.
The same argument applies to logging. NIST SP 800-53 Rev. 5 does contain a control named Provenance — SR-4, in the supply chain risk management family — but read it before citing it: it is about documenting and monitoring the chain of custody of systems, components and associated data, not about which characters produced which sentence. The controls that bear on generated text sit in the AU family — AU-3 on the content of audit records, AU-10 on non-repudiation — and behind them is the assessor's real question: whether the record you retain is sufficient to reconstruct what happened. A span log answers that for generated text in a way a confidence score never will. For pipeline lineage, the W3C PROV Ontology (W3C Recommendation, 30 April 2013) supplies a standard vocabulary of entity, activity and agent. PROV tells you which process produced an artifact; spans tell you which characters produced a sentence. Defensible systems usually need both.
The standard the receiving organization already has
For readers coming from an analytic background, none of this is novel — it is the machine version of a discipline their organization already runs on. Intelligence Community Directive 206, Sourcing Requirements for Disseminated Analytic Products, requires disseminated analytic products to carry structured source references identifying the originator, the date, the classification and the factors affecting source quality. ICD 203, Analytic Standards, requires products to properly describe the quality and credibility of underlying sources. Both are Office of the Director of National Intelligence directives and both are posted publicly on dni.gov.
Those directives govern human analytic products, not our pipelines, and we will not claim compliance with a directive that does not apply to us. What they do is set the expectation already in the reader's head. An analyst trained under ICD 206 looks at a generated paragraph and asks which endnote belongs to which clause. A system that answers at clause granularity meets a bar that reader already carries; one that answers at document granularity does not.
Common questions on where this breaks
Does span-level provenance stop hallucination?
No, and anyone claiming it does is selling. It stops unsourced assertion, which is a different failure. A model can select a genuine span and still characterize it wrongly in the sentence it writes. Span verification proves the source exists, was read, and says what the output claims it says verbatim — it does not prove that a paraphrase is faithful. Faithfulness is a separate evaluation with separate labels and a separate cost, and we budget for it separately.
What about OCR'd, scanned or handwritten records?
Spans are only as good as the character stream underneath them. If OCR invents a character, the span points at the invented character and verifies cleanly, because verification checks consistency with the extraction, not with the paper. The honest control is to carry extraction confidence alongside the span and treat low-confidence regions as unquotable — a narrower system, and a truthful one. See our note on OCR pipelines for legacy federal documents for where those confidence signals come from.
Does it work on tables and figures?
Partially. Cell-level anchoring works when the extractor produces a stable cell addressing scheme that survives re-extraction. Figures generally do not work, and we say so rather than anchoring a claim to a figure caption and implying we read the chart. A system that quietly cites the caption when it means the plot is producing a citation that is technically valid and substantively false.
Can this be retrofitted onto an existing system?
Sometimes. The answer depends entirely on whether the generation step ever had the source segments in hand. If it did, retrofitting is plumbing and it is worth doing. If the text was generated from a compressed context or from a fine-tuned model's parameters, there is nothing to point at, and the only thing a retrofit can add is a search pass. We will build that pass if a customer wants it — labeled as a plausibility check, not as provenance.
What we will not do
We will not bolt a citation-finding pass onto a generator that never carried spans and call the result provenance. If the work is useful anyway, we will build it and label it what it is: a plausibility check on text that was produced unsourced. The label is the deliverable.
We will not claim an authorization we do not hold or inherit. “IL5-ready” from a vendor with no sponsoring authorizing official and no assessment is a marketing phrase, not a status. The accreditation belongs to the organization that owns the enclave. Our job is to hand them an artifact that does not fight the boundary — no outbound calls in the verification path, no hosted dependency at inference time, and a build their assessors can read.
We will not build a domain ontology without the customer's practitioners in the room. The predictable failure is a schema that models what we understood in three weeks instead of the distinctions the analysts actually make, and it fails late — after the extraction rules are written against it.
We will not report recall without reporting the false-extraction rate beside it. Recall alone is not a measurement; it is a selection. The same logic applies to retrieval, which is why we evaluate retrieval quality separately from generation quality rather than reporting one number for the pipeline.
Bottom line
Span-level provenance is not a feature you add to a language system. It is an ordering constraint on how text gets produced, and it propagates into the extraction layer, the storage format, the evaluation design and the deployment posture. It costs recall and interface complexity, and it rules out architectures that would otherwise be cheaper. What it buys is output that can be checked mechanically, inside a closed environment, by someone who does not trust the vendor — which is the one property that cannot be added later.
Frequently asked questions
A design in which every generated sentence carries a pointer to a specific character range inside a specific source record — record identifier, start offset, end offset, and a hash of the record's canonical text at read time. It is stronger than a document-level citation because the pointer can be verified mechanically by re-slicing the record and comparing the result.
Most retrieval-augmented systems cite the chunk that was retrieved, or attach citations in a pass that runs after the text is written. Both point at plausible support rather than actual origin. Span-level provenance requires the source segments to be selected first and the generation to be conditioned on exactly those segments, so the pointer records where the sentence came from rather than where support was found.
It reduces recall — the system cannot state things the corpus implies but does not say. Precision generally improves, because unsourced assertions never make it out. Whether that trade is right depends on whether the output is read as a draft a human will edit or as an instruction someone will act on.
Because you cannot assume an outbound network call. Any verification step that depends on a hosted model or an external retrieval service stops existing when the system is deployed into a closed environment. Span verification is a local string comparison against records already inside the boundary, so it works the same on both sides of it.
Re-normalization of the indexed text is the most common cause — Unicode form changes, whitespace collapsing, de-hyphenation, header stripping. Others include an unversioned extractor change and an unpinned corpus snapshot. All of them fail silently, which is why the record hash and the snapshot identifier travel with the span and why verification runs on every output rather than on a sample.