The model-evaluation evidence your enterprise customer will demand

The slide that used to close the deal

Two years ago a model shipped on a slide. One chart, a held-out accuracy figure, a comparison against whatever baseline made the number look best, and the deal moved to pricing. That still works with a line-of-business sponsor. It stopped working the moment the deal reached the buyer's second-line risk function, and in most large organizations every AI purchase now reaches it. The sponsor loves the product. The model risk reviewer asks one question the slide cannot answer, and the deal sits for a quarter.

The question is usually some version of: how would we know if this stopped working. Not whether the model is good. Whether the buyer, using their own people and their own data, could detect degradation before a customer or a regulator did. A benchmark number says nothing about that. It is a single measurement, taken once, by the party with the strongest incentive for it to be high, on data the buyer has never seen.

What follows is the evidence set that answers the question. It is not a compliance checklist, and none of it is exotic. It is the documentation that a bank's model risk group, a hospital system's clinical informatics committee, or a prime contractor's supply-chain security office will ask for, expressed as artifacts an engineering team can actually produce.

Who is actually reading your evidence

Three reviewers, three different questions, and a package aimed at only one of them stalls in front of the other two.

The model risk reviewer comes out of a quantitative validation tradition. In financial services that tradition has a written source: SR 11-7, the Federal Reserve and OCC guidance on model risk management from 2011, which predates the current AI wave by more than a decade and governs it anyway. SR 11-7 asks for three things on every model. Conceptually sound development with documented design choices. Ongoing monitoring with defined thresholds. Outcomes analysis by a party independent of the developers. If your buyer is a bank, an insurer, or a ratings firm, those three words show up verbatim in the questionnaire, and a vendor who has never read the guidance answers all three badly.

The security reviewer is asking a different question entirely: what can an attacker make this system do. Their reference points are the OWASP Top 10 for LLM Applications and MITRE ATLAS, the adversarial-threat knowledge base modeled on ATT&CK. They want to know whether prompt injection was tested, whether the retrieval layer can be poisoned, whether model outputs flow into anything with side effects, and what happens when they do.

The procurement and compliance reviewer wants the paper trail to map onto a framework their auditor recognizes. That means NIST AI RMF, increasingly ISO/IEC 42001 for an AI management system, and where the deployment touches federal data, the control families in NIST SP 800-53 or the CUI requirements in 800-171. They are not evaluating your model. They are evaluating whether your artifacts can be filed.

Reproducibility is the whole game

Every item below matters less than this one. If the buyer's own engineer can rerun your evaluation on their machine, against their data, and land within a stated tolerance of your published number, most of the remaining questions answer themselves. If they cannot, nothing else in the package carries weight, because the reviewer has no way to distinguish a real result from a favorable one.

Reproducible means specific things. The evaluation harness is code the buyer receives, not a notebook you ran once. The eval set is addressed by a content hash, so both parties know they scored the same items. Random seeds are fixed and recorded. The model artifact is pinned by digest rather than by a tag like latest. The decision threshold is written down as a number. Every dependency is locked. Miss any one of these and reruns diverge by a few points, which is exactly the size of difference an argument forms around.

The practical version we hand buyers is a single command that takes a data directory and a model reference and writes a scored report. It runs on a laptop with no network access. When a reviewer runs it and gets our number back, the meeting is over in ten minutes. That outcome is worth more than any amount of narrative about methodology.

The eval set is an instrument, and whoever holds it decides the verdict

Vendor-supplied test data measures the vendor's skill at choosing a test. Buyers have learned this. Sophisticated ones now insist on assembling the acceptance set themselves, drawn from their own operational records and sampled the way real work arrives rather than the way clean data arrives. Federal buyers have explicit policy behind them here: OMB Memorandum M-25-22 tells agencies to use their own validation and testing data for independent evaluation, says that data should not be accessible to the vendor, and requires vendors to provide the access and time needed to complete the evaluation. Commercial buyers with no memo to cite write the same terms into a master services agreement, and more of them do every year.

Get ahead of it. Offer the buyer-held set before they ask for it. Propose a written sampling rule, a freeze date after which labels are final, and a limit on formal attempts. Two attempts against the primary set, with a third scored against a reserve partition sealed at the same time and never opened until needed. A vendor proposing that structure signals something no benchmark can: that the number will hold up when someone else takes the measurement.

Sample size is the part almost everyone skips, and it is arithmetic rather than judgment. A demo of 27 correct out of 30 is consistent with a true rate near 74%. If the acceptance criterion is "better than the incumbent process," the set has to be sized for a comparison, which is far more expensive: separating a true 90% from a true 85% at conventional confidence and power takes roughly 690 items per arm, and separating 90% from 88% takes around 3,800.

Averages hide the failure that gets you fired

A single aggregate number is the wrong shape for a criterion, because the failures that end deployments are concentrated. The document classifier is 94% overall and 61% on the scanned faxes from one region. The clinical model performs well and degrades on the payer mix that dominates one hospital system. The average never moved.

Slice the evaluation along every dimension the buyer's business actually cares about, and report each slice with its own count. Report the slices where you are weak. A vendor who volunteers a bad slice with a mitigation plan reads as competent; a vendor whose slices all pass reads as someone who chose the slices. Reviewers who have seen a hundred of these packages notice which one they are holding.

Set floors per slice, not just on the average, and a minimum count below which a slice reports as underpowered rather than scored. A slice of 14 items is not a result. It is noise with a decimal point.

Adversarial results, or the security review will supply its own

If your system takes text from anywhere a user or a document can reach, the security reviewer will test prompt injection whether or not you did. Better that your package already contains the results. The structure that lands well: a threat model naming what an attacker would want from this system, a set of attack classes drawn from the OWASP LLM Top 10 and MITRE ATLAS, the tests run against each, the pass rate, and the mitigations for what failed.

The classes worth covering for a retrieval-backed or tool-using system are direct injection in user input, indirect injection through retrieved documents, tool and function-call abuse where model output triggers something with side effects, sensitive-information disclosure from context or training data, and denial of service through unbounded generation or retrieval. Each needs a concrete test corpus, not a paragraph asserting the risk was considered.

The honest finding is more persuasive than the clean one. Publishing that indirect injection through retrieved content succeeded 8% of the time before mitigation and 0.4% after, with the residual failures characterized, tells the reviewer you did real work. A report with no failures tells them you did not look hard.

Monitoring is the artifact that proves the thing is operable

SR 11-7's ongoing-monitoring requirement and the Manage function of the NIST AI RMF converge on the same demand: what will you watch, at what threshold, and what happens when the threshold trips. Buyers ask for this because a model that was accurate at signature and unwatched afterward is a liability with a start date.

A monitoring plan that satisfies a reviewer names four things. The input-distribution signals you track, with the statistic and the alert threshold. The output-distribution signals, including the rate of abstentions and escalations. The delayed ground-truth loop, meaning how labels eventually arrive and how long that takes. The response procedure, meaning who is paged, what they can do in the first ten minutes, and how a rollback executes. Rollback is the only mitigation available before anyone understands the cause, so it needs to be a named, tested procedure rather than an intention.

Independent review, and what it actually means

Independence in SR 11-7 means the validator is not the developer and does not report to the developer. In practice buyers accept several arrangements. A separate internal team with its own reporting line. A third-party engineering firm that reruns the evaluation and writes its own findings. A buyer-side team given the harness, the data specification, and enough access to run it themselves.

The third is usually cheapest and most convincing, and it is where a vendor should push. Handing the buyer the instrument and the time to use it costs a few engineer-days and removes the objection entirely. What does not work is a letter from the vendor's own leadership attesting that the evaluation was thorough. That is not independence, and the reviewer will say so.

What the package contains

Concretely, this is the set of documents. It is short, and every item exists to answer a question a specific reviewer will ask.

Artifact	The question it answers	Who asks
Model card	What is this system, what is it for, and what is explicitly out of scope	All three reviewers
Data statement	Where training data came from, how it was collected and labeled, what it is missing	Risk, compliance
Evaluation protocol	How the set was assembled, frozen, sized, and who holds it	Model risk
Results with slices	Performance overall and on every subgroup that matters, with counts and intervals	Model risk, business sponsor
Reproduction kit	Harness code, pinned artifact digest, locked dependencies, seeds, one command	Model risk, security
Adversarial report	Threat model, attack classes tested, pass rates, mitigations, residual risk	Security
Monitoring plan	Signals, thresholds, ground-truth latency, paging, rollback procedure	Risk, operations
Framework crosswalk	Which artifact satisfies which NIST AI RMF subcategory or ISO 42001 clause	Compliance, procurement

The crosswalk is the item teams most often skip and the one that most reliably shortens a review. It is a two-column table mapping your artifacts to the framework the buyer's auditor uses, and it takes a day. Without it the compliance reviewer does the mapping, which means they do not, and the package goes back with questions.

What it costs and how long it takes

For a single production model with a defined use case, building this package from a standing start is roughly four to eight weeks of focused work by two engineers, and the largest line item is not writing. It is building the reproduction kit and the slice evaluation, because that work usually surfaces genuine problems: a preprocessing step that differs between training and serving, a threshold living in a config map that nobody versioned, an eval set with overlap into training data.

Expect the evaluation work itself to consume 60% of the effort, the adversarial testing 20%, and the documents and crosswalk the remaining 20%. Retrofitting onto a system that was never built with a pinned artifact and a frozen eval set costs meaningfully more, because reconstruction comes first. Budget an additional two to four weeks when the version history is thin.

The return is measured in sales cycle. A review that would have taken a quarter of back-and-forth typically closes in two or three weeks when the reproduction kit is in the buyer's hands on day one. For a deal of any size, that is the cheapest engineering work in the program.

The failure mode to avoid

The common failure is treating this as a documentation exercise, assigning it to whoever has capacity, and producing a package that is internally inconsistent. The model card cites a metric the results section does not contain. The monitoring plan names a threshold on a signal nobody instruments. The crosswalk maps to a subcategory the artifact does not address. Any single inconsistency costs credibility across the whole package, because the reviewer's job is precisely to find them, and finding one changes how they read the rest.

The fix is that the same team writes the documents and runs the evaluations, in that order, and every number in the prose is generated by the harness rather than typed. When the results table is produced by the same command that produces the score, drift between document and system stops being possible.

Bottom line

Enterprise buyers stopped grading AI on the demo. They grade on whether the claims survive contact with their own people and their own data, and they have adopted a shared vocabulary for asking, drawn from SR 11-7, the NIST AI RMF, ISO 42001, the OWASP LLM Top 10 and MITRE ATLAS. The vendors clearing those reviews are not the ones with the highest benchmark. They are the ones who hand over a harness, a hash, and a slice table, and let the buyer take the measurement themselves.

Frequently asked questions