The contract data requirements list on your program was written for software. It asks for a software design document, a version description document, a test report, and user documentation, each on a form that has been in service for decades and each with a distribution statement and a data-rights assertion attached. Then a machine learning workstream lands inside that structure, and the mapping stops working. A model is not a build. Its behavior comes from data your team may not own, its correctness is a distribution rather than a pass or fail, and the thing the government most needs to receive is not the source file but the ability to produce the model again. This is written for the person who has to define those deliverables before the subcontract is signed, and who will be the one explaining at the review why what arrived does not match what was asked for.
We build and deliver machine learning and data systems inside federal programs, usually as the specialist under a prime. The pattern we see repeatedly is not that a sub cannot produce documentation. It is that nobody decided what an AI deliverable is until the month it was due, and the resulting package is a set of software artifacts with a model mentioned in them. That package passes a superficial review and fails the first time somebody outside the original team tries to use it.
Why software CDRLs miss on a model
Take the four standard items and ask what each one would contain for a trained model.
A software design document describes structure: modules, interfaces, control flow. A model has an architecture, and describing it takes about a page. What determines whether the model works is nowhere in that page. It is in the label definitions, the sampling policy, the class balance, the split strategy, the feature transformations and the twenty decisions that were made when the training data was assembled. A design document that follows the software template will describe the training script and omit all of it.
A version description document lists what is in the build and how it was made. For a model, the build inputs include a dataset snapshot that may be many gigabytes, a random seed, a library version stack where a minor bump changes numeric results, and hardware that affects reproducibility. A version description that names a git commit and a container tag is closer than most, and still incomplete unless the dataset has an identity of its own.
A test report records pass and fail against requirements. Model evaluation produces distributions. The honest answer to "does it meet the requirement" is a point estimate with an interval around it, computed on a specific held-out set, valid for inputs that resemble that set. A test report that prints a single accuracy number and a green checkmark is not wrong so much as it is unusable, because it gives the receiving organization no way to tell whether next quarter's number moved because the model changed or because the world did.
User documentation tells an operator how to run the system. For a model it has to also tell an operator when not to trust it: the input conditions where performance degrades, the populations the training data underrepresents, the failure modes that look like confident answers. That content does not exist in a software documentation template because software does not have that behavior.
What determines whether a delivered model is usable a year later
Editorial weighting, illustrative rather than measured. The last row is deliberately low: the weights are the easiest item to deliver and the least useful on their own.
The six items that make a complete AI deliverable
Written as data items, these fit the structures your program already uses. Each can be mapped onto an existing data item description or written as a program-unique one, and each has a natural inspection and acceptance criterion.
1. The model card
One document per model version, and it is short. Intended use and, explicitly, out-of-scope use. The task definition in mission terms, not machine learning terms: what decision this output feeds, who acts on it, what happens when it is wrong in each direction. Architecture and size. Training data summary with provenance and time coverage. Evaluation results by segment. Known limitations. Ethical and legal considerations relevant to the mission. Contact and version identity.
The segment breakdown is the part that earns the document. An overall number hides the cases that matter. If the model performs well on the common condition and poorly on the rare one, and the rare one is what the operator uses it for, an aggregate metric tells the program office exactly the wrong thing. Insist that evaluation be reported by the segments the mission cares about, defined jointly with the government's subject matter experts before training starts.
2. The data description
Separate from the model card, because it has a different audience and a different lifetime. It covers the training, validation and test sets: source systems, extraction dates and queries, record counts, field definitions, label definitions and the labeling procedure, inter-annotator agreement where humans labeled, the split strategy and why it was chosen, exclusions and their reasons, class balance, and known quality problems left in deliberately.
Two things in that list carry most of the weight. The label definition is the requirement in disguise: a model trained on labels that mean something slightly different from what the operator means will be accurate against its own data and wrong in use. And the split strategy determines whether the evaluation number means anything at all. A random split on records that come in groups, cases from the same site, images from the same sensor, documents from the same case file, leaks information across the split and produces a number that will not survive contact with new data. Ask which split was used and why. If the answer is "random" and the data has group structure, the evaluation is optimistic and everyone will find out later.
3. The evaluation report
Written against the acceptance criteria in the statement of work, not against whatever metric was convenient. Primary metric with a confidence interval and the method used to compute it. Secondary metrics. Performance by segment. A baseline for comparison, which should be the current process, not a null model, so the program office can answer the question it actually has: is this better than what we do now, and by how much. Failure analysis with categorized examples. Stability checks under the shifts expected in the field. Where relevant, checks for differential performance across groups the program has a reason to care about.
The report should be generated by a script that anyone can run against the delivered evaluation package. If it was assembled by hand from notebook output, it will not be reproducible and it will not be repeatable at the next release.
4. The deployment package
Container images or equivalent, the inference service with its interface specification, configuration and its schema, dependency manifests pinned to exact versions, infrastructure as code for the target environment, and the security artifacts the accreditation path needs: software bills of material, vulnerability scan output, and the control implementation statements that map to the system's baseline. If the destination is a NIST SP 800-53 baseline through an authorization process, the model's package should carry its share of that evidence rather than leaving it to the integrator to reconstruct.
5. The monitoring plan
What is measured in production and at what cadence: input distribution drift, output distribution drift, latency and error rates, and where ground truth is eventually available, live performance against it. Thresholds that trigger investigation and thresholds that trigger rollback. The retraining decision rule. Who owns the alert and what they do with it. This is a deliverable, not an operations task to be figured out after transition, because the thresholds come from the evaluation work and only the team that did the evaluation knows where to set them.
6. Source, configuration and the pipeline
The training pipeline as code, the evaluation suite, the feature transformation code shared between training and serving, the experiment configuration, and the environment definitions. Delivered so that a competent engineer on the receiving team can retrain from the described data and reproduce the evaluation within a stated tolerance. That tolerance should be in the statement of work: exact bit reproduction is expensive and sometimes impossible on different hardware, and a stated tolerance is honest and testable.
Data rights, asserted item by item
The rights question on an AI deliverable is more granular than on a software one, and the granularity is where programs get hurt. The trained weights, the training code, the evaluation suite, the feature transformation library, the data itself and any pre-existing tooling the sub brought with them can each carry a different assertion, and the government's rights in each follow from who funded that item's development.
Under the defense data rights framework, rights in technical data and computer software generally follow the source of funding for the development: items developed exclusively at government expense come with unlimited rights, items developed exclusively at private expense are delivered with restricted or limited rights depending on category, and mixed funding yields government purpose rights that convert after a stated period. The mechanism that matters procedurally is assertion: restrictions must be identified and asserted before award, in the format the solicitation requires, and an item delivered without an assertion is generally delivered with the broader rights.
That has a direct consequence for how you write the subcontract. Ask the sub for its assertions list at proposal time and read it as a program document rather than a legal formality. A sub that arrives with a pre-existing evaluation framework or feature library will want to keep it, and that is reasonable; what is not reasonable is discovering the restriction in the delivery package. The workable arrangement names the background items, excludes them from assignment, and takes a license broad enough that a future maintainer or a follow-on competitor is not blocked. If the program intends to recompete sustainment, the license has to reach the successor, and that is a sentence to write at subcontract award rather than at closeout.
One more item deserves explicit treatment: the data. Training data assembled under the contract, labels created under the contract, and synthetic data generated under the contract are each things somebody paid for, and each should have a stated disposition. Whether the sub may retain a copy, whether it may be used to train anything else, and what is destroyed at the end all belong in writing before the first extract moves.
| Deliverable item | What the software template asks for | What the AI version must contain | How it is inspected |
|---|---|---|---|
| Design description | Modules, interfaces, control flow | Model card plus data description: task in mission terms, label definitions, split strategy, segment results, limitations | Government subject matter expert reads the label definitions and confirms they match operational meaning |
| Version description | Build contents, commit, release notes | Dataset identity and hash, code commit, pinned dependency stack, seeds, hardware notes, reproduction tolerance | Receiving team reruns training on a clean environment and lands inside the stated tolerance |
| Test report | Pass or fail against requirements | Metric with interval, baseline comparison, segment breakdown, failure analysis, shift and fairness checks | Evaluation script is run by the government on the held-out set and reproduces the report |
| User documentation | How to operate the system | Operating guidance plus when not to trust the output, degradation conditions, escalation path | An operator who did not build it completes a scripted task set unaided |
| Software product | Source and build instructions | Training pipeline, evaluation suite, shared transformation code, deployment package, infrastructure as code | Clean checkout builds and deploys into the target environment without the sub present |
| Sustainment item | Often absent | Monitoring plan with thresholds, retraining decision rule, alert ownership | Thresholds are exercised against historical data and produce the expected alerts |
Produced as a by-product, not as a scramble
Everything above sounds like a documentation burden bolted onto engineering work. It is not, if the work is instrumented from the first week. Almost every item on the list is a rendering of something the team needs anyway to do the job well.
The model card is generated from the training run's metadata and the evaluation output. If every training run writes its configuration, dataset identity, metrics and segment breakdown into a registry entry, the card is a template applied to that entry. Written by hand at the end, it takes a week and contains errors nobody catches; generated from the record, it takes an afternoon and is correct because it came from the run.
The data description comes from the pipeline. A data assembly step that emits record counts, field statistics, class balance, exclusion counts and the query text as an artifact of each run produces the document as a side effect. Reconstructed from memory six months later, it is approximately right, and approximately right is the failure mode that makes a delivery unusable.
The evaluation report is the evaluation suite with a formatter on it. This is the single highest-value piece of infrastructure on an AI workstream, because it gets used every week for internal decisions and once per release for the deliverable. A team that builds it early is faster all the way through and produces the CDRL for free. A team that treats evaluation as a phase at the end has neither.
The deployment package and infrastructure as code exist because the system had to run somewhere in development. If development happened in the target environment, or in an environment built by the same code, the package is already the deliverable. If development happened on a presenter's machine or a personal cloud account, it does not exist and it will be created under deadline pressure, which is where accreditation schedules go to die.
This is why we ask, in the first week of a subcontract, for the registry, the evaluation suite and the environment definition to be treated as work-package deliverables in their own right rather than as internal tooling. It changes the cost profile: a little more effort in month one, and the documentation months disappear.
Which delivery items are cheapest to produce when the work is instrumented early
Editorial weighting, illustrative rather than measured. The last row is deliberately low: no amount of tooling decides who owns what.
How we work inside your program
Precision Federal is a small business engineering firm. We build machine learning systems, data platforms, cloud infrastructure and full-stack applications, and we deliver them into production inside federal agencies as a specialist subcontractor, teaming partner or nontraditional partner on an other transaction. On a program like the one described here, we take a defined AI workstream and its data items, and we own them end to end.
What arrives in the first weeks is concrete. In week one, a written data item plan that maps each AI deliverable to the CDRL structure your program already uses, with the acceptance criterion for each stated as something testable. By the end of the first month, the evaluation suite running against a real held-out set, the experiment registry recording every run, and a measured baseline for whatever the current process is, so the program office has a comparison point before any model is trained. From there the deliverables accrete as the work proceeds; nothing is written from memory at the end.
The prime keeps everything that matters. The customer relationship is yours and stays yours. The code is written into your repositories under your configuration management from the first commit. The data never leaves the environment your security agreement permits. Our background tooling is named at subcontract award, excluded from assignment, and licensed to you broadly enough that a follow-on team is not blocked. We are content to be visible as a named subcontractor with an evaluated scope, or to work inside your delivery organization without a public role, whichever fits your posture with the customer.
Pricing takes one of two shapes. Firm fixed price against milestones that are the deliverables themselves, which suits a bounded workstream with a clear acceptance criterion. Or a committed team at a stated allocation for a defined period, which suits a program where scope moves with the customer. On an earned value program, we structure work packages so each closes on an objective evaluation result rather than a percentage estimate, which keeps your program control team out of an argument every month.
The first step is one email with a one-page brief: the program, the AI workstream, the data items as currently written, the target environment, the schedule that matters and the contract instrument. We read it and return a scoped, priced statement of work with the acceptance criteria written as tests. That takes days, not weeks, and costs nothing.
Five failures worth designing against
The evaluation set was seen during development. Somebody tuned against the test set, or the split leaked through group structure, and the delivered number is optimistic. The defense is a held-out set that is locked before work starts, controlled by the government or the prime, and used only for acceptance.
The label definition drifted from the operational meaning. The model is accurate against its labels and useless in the field. The defense is a written label definition, reviewed by the government's subject matter expert, before annotation begins rather than after.
The data cannot be reassembled. The extract came from a system that has since changed, no snapshot was kept, and the model cannot be retrained. The defense is dataset identity: a hash, a stored snapshot or a versioned extract with the query recorded, treated as a configuration item.
Rights were not asserted, or were asserted too broadly. Either the government finds a restriction it did not price for, or the sub finds it has given away a framework it uses everywhere. The defense is an assertions list read at proposal time by someone who understands both the program's sustainment plan and the sub's tooling.
Monitoring was left for later. The system deploys, performance decays quietly, and nobody notices until an operator complains. The defense is treating the monitoring plan as a contract data item with thresholds derived from the evaluation work, delivered with the model.
Bottom line
An AI deliverable is complete when someone who was not on the team can retrain the model, reproduce the evaluation, deploy the result and know when to stop trusting it. That is six items: the model card, the data description, the evaluation report, the deployment package, the monitoring plan, and the source and pipeline under correctly asserted rights. Write them into the subcontract as data items with testable acceptance criteria, ask for them incrementally rather than at the end, and require the evaluation suite and experiment registry in the first month so the documents are generated rather than composed. A workstream instrumented that way costs slightly more in month one and removes the closeout scramble entirely.
Frequently asked questions
Six items. A model card covering intended and out-of-scope use, architecture, training data summary, evaluation by segment and known limitations. A data description covering sources, label definitions, split strategy, counts and quality issues. An evaluation report written against the statement of work acceptance criteria with intervals, a baseline comparison and failure analysis. A deployment package with pinned dependencies, infrastructure as code and security artifacts. A monitoring plan with thresholds and a retraining rule. And the training pipeline, evaluation suite and configuration, each with its data rights asserted.
Partly, with tailoring. The version description, test report and software product items can carry AI content if the tailoring block specifies dataset identity, reproduction tolerance, metric intervals and segment breakdowns. What no software item covers is the model card, the data description and the monitoring plan, so those are usually written as program-unique data items. Deciding this before award is far cheaper than negotiating it at the first delivery review.
It depends on the item and the funding source, which is why the assertions list matters. Rights in technical data and software generally follow who paid for the development of each item, with unlimited rights where development was exclusively at government expense, restricted or limited rights where it was exclusively at private expense, and government purpose rights for mixed funding. Restrictions must be asserted in the required format before award. Trained weights, training code, evaluation suite, shared transformation libraries and the data itself can each land differently, so list them separately.
Run it. Take the delivered package to a clean environment, execute the evaluation script against the locked held-out set, and check that the numbers match the report within the stated tolerance. Then retrain from the delivered pipeline and confirm the result lands inside the reproduction tolerance in the statement of work. Then have someone who did not build it deploy the package. Reading documents catches formatting problems; running the package catches the problems that matter.
Incrementally, tied to the evaluation milestones rather than clustered before final delivery. A first version of the model card, data description and evaluation report at the first measured baseline, updated at each release. That way the program office sees the shape of the documentation early enough to correct it, the subcontractor produces it from working infrastructure rather than from memory, and the closeout package is an assembly of accepted items rather than a month of writing under deadline.