An examiner asks for the model inventory. Someone forwards a spreadsheet that was last edited eleven months ago, and three of the systems on it were retired, and two systems now making decisions about customers are not on it at all. That gap is the audit finding. It is not a documentation problem and it will not be fixed by writing a better policy, because the inventory was never connected to anything that knows when a model is deployed. The evidence a regulator will accept is produced by the engineering, not assembled after the fact by a team of people reading Confluence pages.
This is written for the person who has to hand that evidence over. Compliance officer, general counsel, head of internal audit, chief risk officer. The problem is not that your firm lacks controls. It is that the controls live in policy documents and the systems live in production, and nothing joins them, so every examination becomes a six-week archaeology project staffed by the people you can least afford to divert.
What an audit actually asks for
Strip away the framework names and every AI audit, whether it comes from a banking supervisor, a European regulator, an enterprise customer's third-party risk team, or your own internal audit function, converges on the same seven questions. They are worth stating plainly because they define what you need to build.
What AI systems do you have. Not what your policy says should be registered. What is running, what version, on what infrastructure, taking what inputs, producing what outputs, consumed by what downstream process. This is an inventory question and almost every firm fails it first.
What is each one for, and how much does it matter. Purpose stated in business terms, and a risk classification that follows a written rule rather than an opinion. A model that ranks marketing emails and a model that declines a credit application are not the same object and should not carry the same controls.
Where did the data come from. Lineage from source system to training set to feature to prediction, with the transformations named. Including the rights question: what license or contract permits this data to be used this way, and does that permission cover model training.
How was it built and what does it do. Model documentation that a reviewer who did not build it can follow. Assumptions, limitations, known failure conditions, the alternatives considered and why this one was chosen.
What testing did you do, and can you show the results. Performance on a held-out set, yes, but also the tests a regulator specifically asks about: performance across demographic groups where the law reaches, behavior under distribution shift, sensitivity to input perturbation, and for generative systems, behavior on adversarial and edge-case prompts.
What happens after deployment. Monitoring that runs on a schedule, thresholds that trigger something, and the record of what triggered and what was done about it.
Who is accountable and what can they do. Named human oversight, with a described intervention path, and an incident process that has been exercised rather than merely written.
Where AI audit evidence most often falls apart
Editorial weighting, illustrative rather than measured. The last row is low because documentation is the one artifact firms reliably produce, even when nothing behind it is verifiable.
The inventory is a system, not a spreadsheet
Every AI governance program that works has the same foundation, and it is unglamorous: a registry that cannot be bypassed, because deployment goes through it.
The engineering shape is straightforward. A model registry holds an entry for each trained artifact: a content hash of the model weights, the training run that produced it, the dataset version, the code commit, the evaluation results attached to that specific artifact, and an approval state. The deployment pipeline refuses to promote anything to production that does not have a registry entry in an approved state. Now the inventory is not a document someone maintains. It is a query against the registry joined to what the orchestrator says is running, and it is correct by construction because there is no other path to production.
Two details decide whether this survives contact with a real firm. First, the registry has to cover the systems that did not come from your data science team: a vendor scoring API, a foundation model called through a hosted endpoint, a feature inside a SaaS product your business bought. Those are AI systems making decisions in your name and an examiner will treat them that way. The registry entry for a third-party system holds the contract, the vendor's own documentation, the tests you ran against it, and the monitoring you put in front of it. Second, the registry has to record retirement. A model that stopped running last quarter still has to be explainable, because the decisions it made are still on the books and someone can still complain about one.
Lineage, and the question about rights
Data lineage is where audits stall longest, because the honest answer at most firms is that a feature was computed by a job that reads a table that was populated by a pipeline whose author left. Reconstructing that chain by hand takes weeks and produces a diagram nobody trusts.
The fix is instrumentation at the point the data moves. Every transformation job emits a lineage event naming its inputs by dataset and version, its outputs, and the code that ran. Collected centrally, those events form a graph, and the audit question becomes traversal: given this prediction, what feature values fed it, what tables produced those features, what source systems fed those tables, and what was the state of each on the date the prediction was made. Modern orchestration and warehouse tooling emits most of this if you turn it on and standardize the event format. The work is less about building a lineage system than about making every job in the estate report into one.
Attached to lineage is a question that has moved from legal footnote to first-order audit item: what permits this data to be used for training. Customer data governed by a privacy notice that describes one purpose. Licensed data whose contract may or may not contemplate model training. Content scraped or purchased whose provenance is thin. Each dataset in the registry carries a rights record: the source, the instrument that grants use, the permitted purposes, and any restriction on derived works. When a regulator or a customer's counsel asks whether your model was trained on data you were entitled to use, the answer is a lookup and not a research project.
Testing evidence that means something
Most firms can produce an accuracy number. Few can produce testing evidence that answers the question actually being asked, which is not "is it accurate" but "under what conditions does it fail, and did you look."
Build the test suite as code that runs in the pipeline, not as an analysis someone performed once. Concretely, for a decisioning model, the suite runs against a frozen evaluation set and produces:
- Performance overall and by segment. The segments include whatever the applicable law reaches for your use case, and also the business segments where a quiet failure would be expensive. Report the metric, the sample size in each cell, and a confidence interval, because a difference computed on forty cases is not a finding and should not be presented as one.
- Stability under shift. Score the model on data from a period it was not trained on and report the change. This is the single most informative cheap test, and it is the one that catches the model that memorized a market regime.
- Sensitivity. Perturb inputs within plausible ranges and report how much the output moves. A decision that flips on a rounding difference in one input is a defect regardless of aggregate accuracy.
- Behavior at the boundary. Missing values, out-of-range values, categories not seen in training, and for language systems, prompts designed to extract training data or bypass instructions.
- Comparison to the incumbent. Whatever the model replaces, including a simple rule or a human, scored on the same set. A reviewer wants to know the counterfactual.
The output of the suite is a signed artifact stored against the model version in the registry. That artifact is what goes into the audit package. Because it was generated by a pipeline, rerunning it on a new version costs nothing, which is the whole point: evidence that is expensive to produce gets produced once and goes stale.
Monitoring that produces a record
A monitoring dashboard is not evidence. Evidence is a threshold, an evaluation of that threshold on a schedule, a record of every evaluation, and a documented response to every breach.
Four families of monitor cover most of what matters. Input distribution monitors compare the live feature distribution against the training distribution and alert on drift. Output distribution monitors watch the score distribution and the approval or action rate, which catches problems faster than input monitors when a change is upstream. Performance monitors compute the real metric once outcomes arrive, which for credit or claims may be months later, so the design has to handle delayed labels rather than pretending they are immediate. And operational monitors watch latency, error rate, and fallback frequency, because a model that is timing out and silently falling back to a default is producing decisions nobody reviewed.
Each monitor has an owner, a threshold with a written rationale, and a runbook. Every breach opens a ticket in the system your firm already uses for incidents, and the ticket is the audit trail. When an examiner asks what you did about the drift in March, the answer is a ticket with a timestamp, an assignee, an analysis, and a resolution, not a recollection.
Human oversight, described honestly
Regulators have become skilled at spotting oversight that exists only on paper. The test they apply is whether the human can actually change the outcome, and whether the record shows them doing it.
That means the workflow has to give the reviewer three things: the decision, the information needed to evaluate it, and a control that overrides it. If a reviewer sees a score and an approve button and nothing else, the oversight is nominal and an audit will say so. If the reviewer sees the score, the drivers behind it, the comparable cases, the policy that applies, and can record a different decision with a reason code, the oversight is real and the override log becomes one of the more useful datasets in the firm, because a pattern of overrides in one segment is a defect report from your own staff.
Write down the override rate you expect. If it turns out to be near zero, either the model is very good or the reviewers are rubber-stamping, and the audit will ask which. If it is very high, the model is not adding value and the process is theater. Both cases are worth knowing before an examiner tells you.
What the audit package looks like
When the pieces above exist, the package assembles itself. For each system in scope, generated from the registry rather than written by hand:
| Audit question | Assembled by hand | Produced by the engineering |
|---|---|---|
| What systems are in scope | A survey of teams, months stale, incomplete by construction | A registry query joined to the deployment state |
| Where the data came from | Interviews and reconstructed diagrams | A lineage graph traversal with dataset versions |
| What testing was done | Notebooks and screenshots from the build phase | Signed test artifacts tied to the deployed version |
| How it is monitored | Dashboard screenshots taken during the exam | Threshold definitions plus the full evaluation history |
| Who oversees it | A policy naming a role | An override log with reasons and named reviewers |
| What went wrong and when | Memory, and the incidents people remember | The incident record joined to the model version |
The difference between the two columns is not thoroughness. It is cost and truth. The left column costs six weeks of senior time per examination and produces an account that is partly reconstructed. The right column costs a build and then costs nearly nothing per examination, and it is accurate because it was generated from the systems themselves.
What most changes an examiner's impression, in order
Editorial weighting, illustrative rather than measured. The last row is deliberately low: a policy nothing enforces is the finding, not the defense.
The classification rule, written down once
Risk classification is where governance programs generate the most argument and the least value, because the classification is usually an opinion recorded in a field. Replace the opinion with a rule that a person who has never met your business can apply and get the same answer twice.
A workable rule has three inputs. What the system decides or influences, expressed as the effect on the person or the counterparty: does it determine access to a service, price something, allocate a scarce resource, flag someone for review, or merely rank internal content. How autonomous it is: does a human evaluate every output before it takes effect, does a human evaluate a sample, or does the output act on its own. And what the blast radius is: how many decisions per period, and how reversible each one is once made.
Those three inputs produce a tier, and the tier determines the controls. A high tier requires independent validation before deployment, segment testing with stated minimum sample sizes, monitoring at a defined cadence, and named oversight with an override path. A low tier requires registration, basic testing and drift monitoring, and nothing more. Writing this down converts an endless committee conversation into a lookup, and it also means the answer to an examiner's question about why a system is tiered where it is has a citation instead of a memory.
One caution learned the hard way. The tier attaches to the use, not to the model. The same forecasting model can be low tier when it informs a planning meeting and high tier when its output is wired into an automatic action, and a firm that classifies the artifact rather than the deployment will eventually find a high-tier use running under low-tier controls because someone reused a model that was already approved.
Sequencing, when you cannot do it all at once
No firm builds all of this in one program, and attempting it is how these efforts die. The order that works has each phase producing evidence a regulator would accept on its own.
Phase one, the registry and the gate. Stand up the registry, backfill the systems you know about, and put the deployment gate in place so the inventory stops decaying. Include third-party AI. This phase alone answers the first two audit questions and it is the one that most changes an examiner's impression, because a firm that can produce a current inventory in an hour is presumed to have its house in order.
Phase two, testing as a pipeline. Convert the evaluation work your teams already do into a suite that runs on every candidate version and writes signed artifacts. Start with the highest-risk system and use it as the template.
Phase three, monitoring and incidents. Wire the four monitor families, set thresholds with written rationales, and route breaches into the incident system. This is where delayed labels and fallback behavior get designed properly.
Phase four, lineage. Instrument the pipelines and stand up the graph. This is last not because it matters least but because it touches the most surface area, and doing it after the registry exists gives you a place to attach the results.
Phase five, the rights record and the generated package. Attach source and permission records to datasets, and build the generator that turns a registry entry into an audit package on demand.
How we work inside your organization
Precision Federal builds and deploys these systems. We are engineers, not an advisory practice, and what we hand over runs in your environment under your control.
The first engagement is scoped small and produces something real. In the first two weeks we map what is actually deployed, which is usually more than the current inventory shows, and we stand up the registry with the deployment gate in one environment. By week four the gate is live in production for at least one path, and the inventory query returns something you would be willing to show an examiner. From there each increment adds one of the phases above, and each has a written acceptance test so nobody argues later about whether it was delivered.
You keep everything. The code is yours, in your repositories, under a written assignment. The infrastructure runs in your cloud accounts and your identity system. The data never leaves your boundary unless you decide it should. There is no proprietary platform you have to keep licensing from us to keep the system working, and there is no component that stops functioning if you stop paying us. We build so that your own engineers can carry it, and part of the delivery is the handover: documentation written for the people who will run it, and time spent working alongside them.
Pricing takes one of two shapes. Fixed-price milestones when the scope is clear, with each milestone tied to an acceptance test you have approved in advance, so you know the cost before the work starts and you pay for a result rather than for effort. Or a committed team at a fixed monthly rate when the work is genuinely exploratory and milestones would be fiction. Most governance programs start as the first and move to the second only if the firm decides it wants sustained engineering capacity.
We have delivered systems inside U.S. federal agencies, where the evidence bar is high and the review is adversarial: authorization to operate, continuous monitoring, controlled unclassified information handling, accessibility conformance. That work shapes how we build for regulated commercial firms, because the discipline is the same. Produce the evidence as a byproduct of the system, not as a project that runs after it.
The first step is one email with a one-page brief: what systems are in scope, who is asking for the evidence and by when, and what you already have. We come back with a scoped, priced statement of work.
Bottom line
An AI audit is expensive when evidence has to be assembled by people and cheap when it is produced by the systems. The difference is a handful of engineering decisions made once: a registry that deployment cannot bypass, tests that run in the pipeline and write signed artifacts, monitors with thresholds and a ticket trail, lineage emitted by the jobs that move data, and oversight built into the workflow rather than named in a policy. Firms that make those decisions answer an examination in days with material they did not have to write. Firms that do not will keep paying six weeks of senior time per exam, and will keep discovering that the inventory was wrong at the worst possible moment.
Frequently asked questions
Seven things, whatever the framework is called. A current inventory of AI systems including third-party ones. A stated purpose and a risk classification for each. Data lineage from source to prediction, plus the rights that permit the data to be used. Model documentation a reviewer who did not build it can follow. Testing evidence covering performance by segment, stability under shift, sensitivity, and boundary behavior. Post-deployment monitoring with thresholds and a record of breaches. And named human oversight with a real intervention path and an override log.
By making it a byproduct of deployment rather than a document. A model registry holds an entry per trained artifact with the weights hash, training run, dataset version, code commit, evaluation results, and approval state. The deployment pipeline refuses to promote anything without an approved entry. The inventory then becomes a query joined to what is actually running, and it is correct because there is no other route to production. Include vendor APIs and hosted foundation models, and record retirement as well as deployment.
Not just an accuracy figure. Performance overall and by the segments the applicable law reaches, reported with sample sizes and confidence intervals so small-cell differences are not presented as findings. Stability when scored on a period outside the training window. Sensitivity to plausible input perturbation. Behavior on missing, out-of-range and unseen values, and for language systems on adversarial prompts. And a comparison against whatever the model replaced, including a simple rule or a human, scored on the same data.
Generally not on its own. A control needs a threshold, an evaluation of that threshold on a schedule, a record of every evaluation, and a documented response to each breach. Cover input distribution, output distribution and action rate, real performance once outcomes arrive with a design that handles delayed labels, and operational health including fallback frequency. Route every breach into the incident system your firm already uses, so the audit trail is a ticket with a timestamp and a resolution rather than a recollection.
Phased, with each phase producing evidence that stands alone. The registry and the deployment gate come first and can be live in one environment within weeks, which answers the inventory and classification questions. Testing as a pipeline follows, starting with the highest-risk system as a template. Then monitoring and incident routing. Lineage instrumentation comes later because it touches the most surface area and benefits from having a registry to attach results to. Rights records and the generated audit package close it out.