Regulatory submissions and the traceability burden

Ask anyone who has assembled a regulatory submission where the time goes and the answer is not composition. It is reconciliation — confirming that the number in the narrative matches the table, that the table matches the dataset, and that the dataset matches what was actually collected.

Why authoring automation targets the wrong bottleneck

Tools that draft sections from templates address a real irritation and a small fraction of the effort. The submission is not slow because writing is slow. It is slow because every assertion has to be defensible under inspection, and establishing that is manual, repetitive, and unforgiving.

Adding a generative drafting layer on top of an unreconciled evidence base makes this worse. It produces polished text faster than the traceability can be established, which shifts more work to the reconciliation stage rather than less.

Faster drafting against an unverified source is not throughput. It is deferred rework.

The traceability chain, stated plainly

Every clinical or technical assertion in a submission should be walkable backward without ambiguity.

• Assertion in the narrative, carrying the identifier of the table or figure it summarizes
• Table or figure, carrying the analysis dataset and the program version that produced it
• Analysis dataset, carrying its derivation from the collected data
• Collected data, carrying its source record
• Every link timestamped and versioned, so a regenerated output is distinguishable from the original

Where this chain exists as data rather than as a convention people follow, most of the reconciliation labor disappears — not because anyone writes faster, but because the checking becomes a query.

Where a language model belongs in this

Two places, both narrow and both valuable.

First, generating the narrative from a structured result object, under one hard constraint: any figure in the output that does not resolve to a field in the object is removed. This produces text that is correct by construction rather than correct on review.

Second, the reverse direction — reading an existing document and proposing the assertion-to-source links, which a human then confirms. That is a retrieval and alignment task, it is tedious, and it is exactly the kind of work that benefits from a machine proposing candidates for human adjudication.

The regeneration property

A well-built submission pipeline has a property that is worth naming: when an underlying dataset is corrected, every downstream table, figure, and narrative statement can be identified and regenerated, and the difference from the prior version can be produced as a list.

Teams without this property handle a late data correction as a document-wide manual sweep under deadline. Teams with it handle it as a rebuild plus a diff review. The difference is measured in weeks, and it recurs on every submission.

Where the weeks actually go

Teams estimating a submission consistently allocate to authoring and are consistently surprised by reconciliation. Naming the specific activities makes the estimate more honest.

Reading the "automatable" column explains why authoring tools underdeliver. They target the one row that was never the bottleneck, and they can make the first two rows harder by generating text faster than it can be reconciled.

The link is the deliverable

The traceability chain described earlier is often treated as documentation of the work. It is more useful to treat it as the primary artifact and the documents as renderings of it.

Under that framing, a narrative statement is not text that happens to mention a number. It is a template bound to a specific cell of a specific table produced by a specific program version from a specific dataset version. Rendering produces the text. Changing the dataset and re-rendering produces updated text, and a diff of what changed.

Teams that adopt this find that the discipline pays off before any automation does, because the act of making links explicit surfaces the places where nobody could say where a number came from. Those places existed before; they were simply invisible until someone tried to write the link down.

Constrained generation, precisely specified

"The model may only use the result object" is the right rule and needs an implementation, because a model given a result object and asked to write will still occasionally produce a number that is not in it — from context, from convention, from a plausible rounding.

• Input restriction. The drafting call receives the structured result object and section template, never source documents or prior submissions
• Numeric extraction and matching. Every number in the output is extracted and matched against a field, including rounded and reformatted forms
• Entity and date matching. Same treatment for study identifiers, arm names, timepoints, and units
• Rejection, not flagging. Unmatched content fails the generation and triggers a retry, rather than passing to review with a warning
• Deterministic settings and retention. The generation is reproducible, and the object plus configuration is retained with the output

The fourth item is the one that gets softened in practice and should not be. A warning that reviewers can dismiss becomes a warning reviewers dismiss, particularly under deadline. A hard failure keeps the property that the text is correct by construction.

The reverse direction: retrofitting traceability

Most organizations are not starting clean. They have submissions, documents, and no link layer, and building one prospectively does nothing for what already exists.

Retrofitting is a genuinely good machine task. The system takes an existing document, identifies each assertion containing a figure, searches the available tables and datasets for candidate sources, and proposes a link with a confidence. A human confirms or corrects.

This is tedious, mechanical, high-volume work of exactly the kind that benefits from a machine proposing and a human adjudicating. It also produces something valuable independent of any future automation: a map of which assertions in the existing corpus cannot be traced to a source at all. That list is usually shorter than feared and more interesting than expected.

What regeneration is worth

The property described earlier — a corrected dataset propagating automatically to every downstream artifact with a reviewable diff — deserves a concrete comparison, because it is the largest single return in this whole area.

Without it, a late correction means locating every affected table and figure by hand, regenerating them, finding every narrative statement that references them, editing each, and re-checking cross-document consistency. Under deadline, with several people, and with the error risk that always accompanies manual sweeps.

With it, the correction means updating the dataset, rebuilding, and reviewing a diff that lists exactly what changed. The scientific review of whether the changes are correct still happens and should. The mechanical hunt does not.

The difference recurs on every submission and on every amendment, which is what turns a one-time infrastructure investment into a compounding one.

Validation is the constraint people forget to plan for

Systems that produce or transform submission content generally sit inside a validated environment, and the validation burden shapes what is worth building.

A deterministic transformation — a table generated from a dataset by a versioned program — validates in a familiar way. A generative component does not, because the same input can produce different output and the standard qualification approach assumes it will not.

The practical resolution is to constrain the generative component until it behaves deterministically enough to qualify: fixed configuration, fixed template, a structured result object as the only input, and a post-generation check that rejects anything unmatched. What is being validated then is the checking, which is deterministic, rather than the generation, which is not.

Teams that plan for this from the start build the check first. Teams that add it afterward frequently discover their generation step cannot be qualified in its current form and has to be rebuilt under the constraint anyway.

Change control on the link layer

Once the traceability chain is data, it becomes something that can be modified — and a chain that can be silently edited is worth less than no chain, because it looks authoritative.

• Links are versioned and additive; corrections supersede rather than overwrite
• Every change carries an actor, a timestamp, and a reason
• Regeneration is recorded as an event, distinguishable from an original
• The chain for a submitted document is frozen at submission and retained
• Access to modify links is narrower than access to read them

Team structure that makes this work

The chain crosses functions that do not usually share a system — data management owns the datasets, statistics owns the analysis programs, medical writing owns the narrative, regulatory owns the submission. Each has its own tooling and its own definition of done.

The link layer only works if one function owns it end to end, and in practice the choice matters less than the fact of choosing. Programs where the chain is everyone's responsibility produce a chain with gaps at every handoff, and the gaps are exactly where the reconciliation effort was supposed to disappear.

Frequently asked questions