A demonstration is not evidence. It is a performance, and the program office has watched several. Somebody ran a model on inputs the team chose, the outputs looked right, and everyone in the room learned nothing about whether the system works on the data it will actually see next year. When an independent test organization gets involved, or when the authorizing official asks what happens when the model is wrong, the demonstration is worth nothing and the program discovers it needs evidence it never built the capacity to produce. This is written for the chief engineer or test lead who has to decide what "evaluated" means on an AI delivery before the schedule makes that decision for them.
We build machine learning systems into production inside federal agencies, usually as the specialist subcontractor on a prime's program, and evaluation infrastructure is the first thing we stand up. Not because it is virtuous but because it is the cheapest way to go fast. A team that can measure the effect of a change in twenty minutes tries ten things a week. A team that cannot tries one and argues about it.
What a serious reviewer is actually asking
Strip the vocabulary away and a test organization is asking four questions, in order.
Was the model measured on data it had never seen? Everything else depends on this. If the held-out set was touched during development, whether by direct tuning or by the slower route of a hundred design decisions made while watching that number, the reported performance is optimistic by an unknown amount and no amount of statistical care recovers it.
Does the metric correspond to the mission consequence? Accuracy on a problem where one error type costs a hundred times more than the other is a number that answers nobody's question. The metric has to be chosen against what happens when the system is wrong in each direction.
How much of the reported number is noise? A model measured on four hundred examples has a confidence interval wide enough to cover most of the claims people make about it. Two systems whose intervals overlap are not distinguishable on that evidence, and saying so in the report is a mark of a team that can be trusted with the next one.
Can somebody else reproduce it? If the evaluation cannot be run by the government on the government's machine and land on the same numbers, the evidence is an assertion with a chart attached.
Every element below exists to answer one of those four. Anything that does not answer one of them is not evidence; it is presentation.
What a program office and its test organization weigh when accepting AI performance evidence
Editorial weighting, illustrative rather than measured. The last row is deliberately low: a demonstration is a communication event, not evidence.
The held-out set is a program asset, not a team convenience
Treat the evaluation data the way a program treats a calibration standard. It is defined once, held by the program office or the prime's test organization rather than the developing team, and used only for acceptance measurements at defined points.
Three properties decide whether it does its job.
It is drawn to match deployment conditions, not training convenience. If the system will run on data collected next year, from sites not in the training set, under conditions that shift seasonally, the evaluation set should reflect that. A temporal split, where the evaluation data comes strictly after the training data in time, is usually closer to the truth than a random split and usually produces a lower number. The lower number is the real one.
It respects group structure. This is the most common technical error we find in delivered evaluations, and it is almost always invisible in the report. Records that arrive in groups, images from one sensor, documents from one case, records from one facility, must not be split across training and evaluation. If they are, the model learns the group and the evaluation measures memorization. Ask the question directly: what is the unit of independence in this data, and was the split drawn on it. A team that has not thought about it will not have a ready answer.
It is large enough to say something. Compute the interval before the work starts, not after. If the acceptance criterion is a five point improvement and the evaluation set only supports a ten point interval, the criterion cannot be demonstrated at any performance level and the statement of work needs fixing before anyone writes code. That check takes an hour and prevents a category of acceptance dispute.
A second, smaller set held entirely in reserve, used once at final acceptance and never shown to anyone, is worth the cost on programs where the stakes are high. It is the only defense against the slow leakage that happens when a team optimizes against the same evaluation set for a year.
Metrics chosen against consequence
The metric conversation should happen with the government's subject matter experts and should start from the operational decision rather than from the model. Ask what the output is used for, who acts on it, and what the two kinds of error cost.
Where a missed detection is expensive and a false alarm is merely annoying, the operating point should favor recall and the report should state performance at a fixed false alarm rate the operator can live with. Where the output triggers costly action, precision at the operating point matters more. Where the model ranks a queue that humans work top-down, the honest metric is performance in the region humans actually reach, not average performance across the whole ranking. Where the system produces a score used for a threshold decision, calibration matters: a score of 0.8 should mean the event happens about eighty percent of the time, and a model that discriminates well but is badly calibrated will produce confident nonsense at the threshold.
Report the operating point explicitly, and report the curve behind it, so the program office can move the threshold later without a new study. Fixing the threshold in the deliverable and hiding the curve is a way of making every future policy question expensive.
Intervals, baselines and the comparison that matters
Every reported number gets an interval, and the report states how it was computed. Bootstrap resampling over the evaluation set is the usual method and is easy to implement; the important part is that the method is stated and the code is delivered, so a reviewer can check it.
Every reported number also gets a comparison. The comparison the program office wants is almost never against a published academic result. It is against the current process: the existing rule set, the existing tool, or the human doing the task today. Measuring the current process is work, and it is work most teams skip, and skipping it is why so many AI deliveries cannot answer the only question the program office has. Fund it as its own early work package. It is also the number that survives leadership changes, because it is the one that translates into mission terms.
When two candidate systems are compared, compare them on the same evaluation set with paired statistics rather than by looking at two point estimates. Two models that differ by two points on a small set are usually not different at all, and a team that says so has earned the reviewer's confidence for everything else in the report.
Failure analysis, stability, and differential performance
Aggregate numbers tell a reviewer whether to keep reading. The next three sections tell them whether to sign.
Failure analysis. Take a sample of errors, categorize them by cause, count each category, and show examples. Typical categories: the label was wrong, the input was degraded, the case is genuinely ambiguous, the condition is underrepresented in training, or the model made a systematic mistake with a nameable pattern. The distribution across those categories tells the program what to do next, which is what a test report is for. It also tells the reviewer whether the team looked at their own errors, and reviewers can tell.
Stability under shift. Measure performance under the conditions the field will produce: degraded inputs, a new site or sensor, a time period after the training window, formats slightly out of specification, and the volume conditions the system will actually see. Report where performance falls off, because the operator needs to know the edge of the envelope and the accreditation package needs it in writing.
Differential performance. Where the program has a reason to care about performance across groups, whether for legal, policy or plain mission reasons, measure it and report it by segment with intervals. Do this early, because a differential found at acceptance is a program problem and a differential found in month three is a data collection task.
The reproducible package
The last requirement subsumes the others. The evaluation must be delivered as a package a competent engineer outside the team can run.
That means the evaluation code with pinned dependencies, an environment definition or container, a pointer to the held-out data with a hash so the reviewer knows they have the right version, the model artifact with its own identity, a single command that produces the full report, and the report itself generated by that command rather than assembled in a document. If the report was hand-built from notebook output, it is not reproducible, and it will not be regenerable at the next release when it matters most.
Add a stated tolerance. Exact bit-level reproduction across different hardware is expensive and sometimes impossible, so the statement of work should name the tolerance within which reproduction counts as successful. That is honest, testable, and removes an argument.
| Evidence element | The demonstration version | The version a test organization accepts | How it is checked |
|---|---|---|---|
| Test data | Examples chosen by the team | Held-out set locked before development, split on the unit of independence | Split logic reviewed; group leakage checked directly |
| Metric | Accuracy | Metric tied to error cost, reported at a stated operating point with the curve | Subject matter expert confirms it matches the operational decision |
| Uncertainty | A single number | Point estimate with interval and the method stated | Reviewer recomputes the interval from the delivered code |
| Comparison | Against a published result | Against the measured current process, paired on the same data | Baseline measurement is its own delivered artifact |
| Failure behavior | Not discussed | Categorized error analysis with counts, plus stability under expected shift | Categories and counts reproduce from the delivered analysis code |
| Reproducibility | Slides | One command, pinned environment, hashed data, generated report | Government runs it and lands inside the stated tolerance |
Return on building evaluation infrastructure once, at the start of a program
Editorial weighting, illustrative rather than measured. The last row is deliberately low: built late, it produces a report and none of the other returns.
Built once, reused on every release
The reason this is affordable is that the infrastructure is one system used many ways. A single evaluation suite serves internal iteration, milestone evidence, acceptance testing, regression checking on each release, drift detection in production, and the comparison gate in the retraining pipeline. Built at the start, it pays for itself several times inside the first program.
The design that gets that reuse has a few properties. Data loading is separated from metric computation so a new dataset does not require new metric code. Metrics are declared in configuration rather than written per experiment. Every run writes a record with the model identity, the data identity, the configuration, the metrics, the intervals and the segment breakdowns into a registry. The report generator reads that record, so the acceptance report, the monthly status and the internal dashboard are three renderings of one source. And it runs in the target environment on the government's infrastructure, not on developer machines, so reproducibility is proven continuously rather than discovered at delivery.
How we work inside your program
Precision Federal is a small business engineering firm. We build machine learning systems, data platforms, cloud infrastructure and full-stack applications and deliver them into production inside federal agencies, working as a specialist subcontractor, teaming partner or nontraditional partner on an other transaction. On an AI delivery, we usually own the evaluation infrastructure as well as the modeling, because separating them is how programs end up with numbers nobody can defend.
The first weeks are concrete. Week one: a written evaluation plan naming the metric and why it matches the mission decision, the segment definitions, the split strategy and its justification, the sample size check against the acceptance criterion, and the reproducibility tolerance we will meet. Reviewed with your chief engineer and the government's technical lead before any modeling starts. First month: the evaluation suite running in your environment against the locked held-out set, the experiment registry recording every run, and a measured baseline for the current process, so every later claim has something to be compared against.
The prime keeps everything. The customer relationship is yours. The evaluation code and the registry live in your repositories under your configuration management from the first commit, which means your team can run the evaluation without us on any day of the program. Data stays inside the environment your security agreement permits. Our background tooling is named at award, excluded from assignment, and licensed to you and to any follow-on team.
Pricing takes one of two shapes. Firm fixed price against milestones that are the measured results themselves, or a committed team at a stated allocation. On a program reporting earned value, work packages close on an objective evaluation result rather than a percentage estimate.
The first step is one email with a one-page brief: the program, the AI capability, the acceptance criteria as currently written, the test organization involved, the target environment and the contract instrument. We read it and return a scoped, priced statement of work with the evaluation plan and acceptance criteria written as tests. Days, not weeks, and no cost.
Five ways evaluation evidence fails review
The split leaked. Records from the same group appear on both sides, the number is inflated, and it collapses in the field. Ask what the unit of independence is and how the split was drawn.
The evaluation set was used for development. Tuned against directly, or drifted into through a year of decisions made while watching it. Lock it, hold it outside the developing team, and keep a reserve set for final acceptance.
There is no baseline. The report says the model achieves a number and nobody can say whether that beats what the agency does today. Measure the current process as an early, separately funded work package.
No interval, so no comparison is possible. Point estimates from a small set are reported as facts, and two indistinguishable systems get ranked. Compute intervals, state the method, deliver the code.
The report cannot be regenerated. It was built by hand from notebook output, so the next release requires the whole effort again and the numbers quietly stop being comparable. Generate the report from the suite, every time.
Bottom line
Evidence a program office can accept has six parts: a held-out set locked before development and split on the real unit of independence, a metric chosen against the cost of each error type and reported at a stated operating point, confidence intervals with the method named, a comparison against the measured current process, categorized failure analysis with stability and differential checks, and a package the government can run to reproduce all of it within a stated tolerance. Build the suite that produces those in the first month rather than the last, because the same system that satisfies the test organization is what lets the engineering team move quickly for the rest of the program. A demonstration persuades a room once. A reproducible evaluation is the thing that still works at the next release, the next data refresh, and the recompete.
Frequently asked questions
Six things. A held-out evaluation set locked before development and split on the correct unit of independence. A metric tied to the operational cost of each error type, reported at a stated operating point with the curve behind it. Confidence intervals with the computation method named. A comparison against the measured current process rather than a published result. Categorized failure analysis plus stability under expected shift and, where relevant, performance by group. And a reproducible package the government can run to regenerate the report within a stated tolerance.
Because the inputs were chosen by the people who built the system, and nothing about the demonstration bounds performance on data the system has not seen. A demonstration cannot show whether the split leaked, whether the metric matches the mission consequence, how much of the result is sampling noise, or how the system behaves at the edge of its envelope. It is a useful communication event for stakeholders and it is not a substitute for a measured, reproducible evaluation.
Large enough that the confidence interval is narrower than the difference the acceptance criterion asks you to demonstrate. Work that backwards before the statement of work is signed: if the criterion requires showing a five point improvement and the available evaluation data supports only a ten point interval, the criterion cannot be met at any performance level and the requirement needs changing. The check takes about an hour and prevents a whole category of acceptance dispute later.
Leakage is information from the evaluation set influencing the model. The common form is group leakage: records that come in natural groups, images from one sensor, documents from one case file, records from one facility, get split randomly so related records land on both sides and the model measures memorization. Detect it by asking what the unit of independence is, checking the split was drawn on that unit, and comparing a random split against a grouped or temporal split. A large gap between the two is the signature.
The developing team builds it and the program owns it. In practice that means the evaluation code, the metric definitions and the registry live in the program's repositories under the program's configuration management from the first commit, while the held-out data is held by the program office or the prime's test organization rather than by the modeling team. The result is that anyone on the program can measure any candidate model on any day, which is what makes both acceptance and sustainment work.