AI for financial controls and audit: anomaly detection that auditors accept

The question the model is actually being graded on

Most anomaly-detection projects inside a CFO organization are scoped around one question: can we find the bad payments? That is the wrong question to build around, because it is not the question asked at the end. The financial statement auditor, the IG's audit shop, and the A-123 assessment team all ask a version of the same thing: show us the population you scored, show us how you selected what you examined, and show us we get the same answer running it again. A ranked list of scores does not answer that. A defined population plus a documented, reproducible selection does.

The two goals pull the design in opposite directions. Detection performance rewards ensembles, frequent retraining, and features nobody can articulate. Audit acceptance rewards frozen snapshots, versioned scoring runs, and features an accountant can name. Our engineers build for both, and when they conflict the audit side wins. A model that is two points better at ranking and cannot be re-performed is worth less than a model that is two points worse and can.

The fix is a decision made before the first model is trained: the deliverable is not a score column. It is a population, a selection method, a sample, a set of workpapers, and a frozen model artifact retrievable for as long as the records schedule requires.

Improper payments: the statutory frame you are building inside

Federal payment integrity work runs on the Payment Integrity Information Act of 2019 (Pub. L. 116-117), codified at 31 U.S.C. §§ 3351 to 3358, with guidance in OMB Circular A-123, Appendix C. The statute sets the trigger that drives the reporting burden: a program is susceptible to significant improper payments when improper plus unknown payments exceed both $10 million and 1.5 percent of program outlays, or exceed $100 million regardless of rate. Cross that line and the program owes an annual estimate built on a statistically valid sampling and estimation plan, a corrective action plan, and reduction targets.

Read the trigger closely, because it holds the most useful lever a model can pull. Unknown payments count against the program exactly as confirmed improper payments do. A payment the agency cannot classify, because documentation is missing, unreadable, or never linked to the transaction, lands in the same bucket as one that was genuinely wrong. A model that resolves unknowns by locating and linking the supporting record moves the reported rate without recovering a dollar. In year one that is usually the fastest win available, and most vendors skip it because it looks like plumbing.

Recovery has its own hook. PIIA requires a recovery audit for each program with at least $1 million in annual payments where one would be cost-effective, so detection quality translates into budget authority: a higher hit rate on the reviewed population moves marginal programs across that line. Treasury's Do Not Pay service, under 31 U.S.C. 3354, already screens the SAM exclusions file, the Death Master File, and the HHS OIG List of Excluded Individuals/Entities. Build downstream of those matches and take what deterministic matching cannot: the same entity under a different name, the address on twenty unrelated claims, the invoice paid twice through two intake channels.

Journal-entry review is a search problem, not a scoring problem

Journal-entry testing is required work, not optional analytics. Under AU-C section 240 in the AICPA standards and PCAOB AS 2401 for issuer audits, the auditor must test journal entries and other adjustments for evidence of material misstatement due to fraud. The standards even list what to look for: entries to seldom-used accounts, entries recorded post-closing, minimal descriptions, round-dollar amounts, entries by people who do not normally post them, and amounts placed just below an approval threshold.

That list is why journal-entry work should start with rules and only then reach for a model. Every characteristic on it is computable from the general ledger and the approval workflow. Our engineers implement them as named, versioned tests with an owner and a rationale, because a named test is a control activity someone can evaluate. The model then does what rules cannot: rank the residual population by how unusual an entry looks against the account's posting history, the preparer's history, and the period's seasonal shape.

The population also has a property that makes this tractable. A general ledger is finite, enumerable, and closed. You can score every entry for a period, know that you did, and prove the count reconciles to the trial balance. Control totals on the scored population are the first thing an experienced auditor checks.

Approach	What it finds	What the auditor can re-perform
Deterministic rules	Threshold splitting, weekend posting, duplicate invoice numbers, blocked-vendor payments.	Everything. Re-running the rule on the same snapshot returns the same rows.
Statistical outlier tests	Amounts, ratios, and frequencies far from the account's own distribution; digit tests where the population qualifies.	Everything, if parameters and the reference distribution are stored with the run.
Supervised classifier	Patterns resembling past confirmed findings, learned from resolved cases.	The prediction, given a pinned artifact and pinned inputs. Training data needs its own lineage record.
Unsupervised and embedding methods	Entity clusters, near-duplicate free text, relationship anomalies across vendor and payment graphs.	Only the ranking, and only with a fixed seed and frozen index. Leads, never conclusions.
Rules first, model-ranked	Rule hits ordered by likelihood, so limited review capacity reaches the strongest items first.	Both layers, separately. What we recommend for anything headed into a workpaper.

Vendor risk screening is mostly record linkage

Vendor risk gets described as screening, and screening is the easy half. FAR subpart 9.4 governs debarment and suspension, FAR 9.405 requires the contracting officer to check SAM exclusions before award, and FAR 52.209-5 puts the certification obligation on the offeror. Section 889 of the FY2019 NDAA (Pub. L. 115-232) adds the covered telecommunications prohibition through FAR 52.204-24, -25, and -26. Checking a list against a unique entity identifier is solved. If that is all a tool does, it is a lookup with a dashboard on it.

The hard half is the resolution problem underneath. Exclusion lists identify entities; payment systems contain strings. One firm appears in a vendor master as four records with different suffixes, two addresses, and a transposed character. An excluded principal reappears as an officer of a newly formed entity with a clean record. A remit-to bank account changes three days before a large invoice posts, on an email nobody verified out of band. Those are entity resolution and change-detection failures, and that is where the money goes.

So our team treats vendor risk as a data quality program with a screening layer on top. The measurable outputs are duplicate records collapsed, remit-to changes confirmed through an independent channel, related-party links surfaced, and a match-confidence threshold that someone owns. That last item matters more than it sounds: a fuzzy match without a stated threshold and a stated false-match rate is an assertion, and an assertion is not evidence.

Why the auditor needs a sample rather than a score

Audit sampling has its own standard, AU-C section 530, and its own logic. The auditor draws a sample to say something about a population, so the population must be defined and complete, the selection must give every item a knowable chance of selection, and the work must be documented well enough to re-perform. Monetary unit sampling, the workhorse for substantive testing of disbursements, assigns selection probability proportional to dollar value so the conclusion can be projected back in dollars.

A machine learning score breaks that chain. Confine review to the top of a ranked list and the examined items are no longer a probability sample of anything, so no finding can be projected. That does not make the model useless; it makes it a different instrument that has to be labeled as one. The clean structure is two tracks from one frozen population: a risk-directed track where the model steers limited review capacity toward the highest-value items, and a separate statistical sample drawn by a documented method for projection and for the payment integrity estimate. The first track produces findings. The second supports a rate.

Both tracks rest on one technical requirement that most pipelines miss. The population must be an immutable snapshot with a timestamp, a row count, and control totals that tie to the ledger. Scores must be written once and kept, never recomputed on demand, because a model retrained since the sample was drawn will not reproduce the ranking. Selection must use a recorded pseudorandom seed. Do that and re-performance is a matter of pulling the snapshot and the artifact. Skip it and the honest answer to "show me how you picked these" is no.

The workpaper package a scoring run has to produce

• Snapshot identifier, extraction timestamp, row count, and control totals that tie to the ledger
• Model artifact hash, training data version, feature definitions in accounting language, freeze date
• Full score set retained for every item in the population, not only the items reviewed
• Selection rule stated in advance: threshold, stratification, sample size basis, seed
• Disposition for every selected item, with reviewer, date, evidence, and conclusion
• Change log for every model, rule, and threshold revision, with approver and effective date

The false-positive cost, with the arithmetic

False positives get treated as a nuisance. In a financial controls program they are the dominant cost, and the arithmetic changes what a buyer should ask for. Take a disbursement program processing four million payment lines a quarter. A model tuned to a 0.5 percent alert rate produces 20,000 alerts. At twenty minutes of review per alert and a fully loaded reviewer cost of $60 an hour, that is 6,667 review hours and about $400,000 a quarter, or $1.6 million a year, in review labor alone. If precision on that alert set is 6 percent, roughly 1,200 alerts are real and 18,800 are not, so about $1.5 million of the annual review budget goes to examining payments that were fine.

Then the capacity constraint, which is what actually breaks programs. Eight analysts at 450 productive review hours a quarter have 3,600 hours, enough for about 10,800 alerts. The model produced 20,000. Half the queue is never opened, and it is the tail that goes unexamined, which quietly changes effective recall to a number nobody has measured. Raising precision from 6 percent to 18 percent at constant recall is worth three times more here than raising recall, because it converts unreviewed alerts into reviewed ones without hiring.

A third cost sits outside the review team's budget. Holding a payment on a false alert has a price under the Prompt Payment Act (31 U.S.C. §§ 3901-3907, implemented at 5 CFR part 1315), which generally makes an invoice due 30 days after receipt of a proper invoice and obligates interest when the agency pays late. Interest accrues automatically, without the vendor asking. Set target alert volume against measured review capacity and payment terms, never against a performance curve alone.

Where the model sits in the Green Book structure

Federal internal control runs on GAO's Standards for Internal Control in the Federal Government, GAO-14-704G, the Green Book, applied through OMB Circular A-123 to meet agency obligations under the Federal Managers' Financial Integrity Act (31 U.S.C. 3512). Its five components and seventeen principles are the vocabulary the assessment team uses, so design the model to land against specific principles rather than describing it as a capability.

Four principles carry most of the weight. Principle 10, design control activities, is where the model is characterized as a detective control with a stated objective and a stated risk. Principle 11, design activities for the information system, covers data lineage, access control, and change management around the scoring pipeline. Principle 13, use quality information, sinks more models than any other, because it requires the information used to be appropriate, current, complete, and accurate. A model fed from a vendor master with a 12 percent duplicate rate fails Principle 13 whatever the algorithm. Principle 17, evaluate issues and remediate deficiencies, is where dispositions and the remediation loop belong.

Fraud risk has a parallel structure in GAO's A Framework for Managing Fraud Risks in Federal Programs, GAO-15-593SP, and Green Book Principle 8 requires an entity to assess fraud risk directly. So the model should trace to a specific risk in the program's fraud risk assessment. When an IG asks why the model exists, "it detects anomalies" is a weak answer; "it addresses the third and fifth risks in the documented fraud risk assessment, and here is the measured coverage of each" is a strong one.

Keep the model advisory, deliberately

Federal agency AI use is governed by OMB M-25-21, which sets minimum risk management practices for high-impact AI, and by M-25-22 for AI acquisition. A model that ranks transactions for human review generally sits outside the heaviest category. A model that automatically holds, denies, or recoups a payment does not, and it raises due process questions unrelated to model quality.

Make that choice early. A human decision-maker preserves the audit trail examiners want, keeps the governance burden proportionate, and produces the labeled dispositions that make the next version better. Two habits belong with it. Never let a model auto-close low-score alerts, because the systematic error hiding under a threshold is always found from outside. Never auto-tune a threshold to hit an alert-volume target, because that moves a control's sensitivity without an approval record. NIST's AI Risk Management Framework supplies the documentation spine for the rest.

Common questions we get from CFO and IG shops

Bottom line

Detection quality is necessary and it is not sufficient. Review capacity governs value; re-performability governs survival. Both are engineering choices made long before a model is trained. Freeze the population. Keep every score. Record the seed. State the threshold and who owns it. Measure precision against what reviewers actually found, and set alert volume against the hours you actually have. A program built that way improves every period, because the dispositions accumulate into training data. A program built around a score alone stalls at the first audit and gets quietly retired.

Frequently asked questions