The board has asked what the bank is doing about AI. The answer that survives the next two years is not a strategy document and not a vendor pilot. It is one system, in production, that a regulator has seen, that runs on data the bank already owns, and that produces a number the chief financial officer will put in a deck. Everything else follows from that first build, because the first build is where the bank learns whether it can govern, deploy and maintain a model at all. Choose it badly and the program stalls in a committee. Choose it well and the second and third systems cost a third as much, because the hard parts were paid for once.
This is written for the person who has to pick. A regional bank does not have the option a money-center bank has, which is to run six programs and see which survives. One program has to work. The selection criteria matter more than the technology, and they are knowable before anyone writes code.
The three tests a first build has to pass
Every candidate we see at a bank fails or passes on the same three questions, and they are worth stating plainly before the candidate list.
Does the bank already own the data, in a system it controls? If the answer requires a new vendor feed, a data-sharing agreement or a core provider's roadmap, the project's start date is not a project decision. It is somebody else's release schedule. The strongest first candidates run on documents, transactions and servicing records the bank has held for years.
Is there a number that moves, and does finance agree in advance what it is? Handle time, cycle time, loss rate, false positive rate, cost per file, staff hours redeployed. The measurement has to be defined before the build, on a baseline someone has already computed, or the result will be argued about instead of banked.
Has an examiner seen this pattern before? A model that scores a document, ranks an alert or drafts a memo a human approves sits inside supervisory expectations that exist and are written down. A model that makes a credit decision without a person in the loop, or that speaks directly to a customer about their account, is a harder first conversation. Save it for the third system, when the governance record is real.
How first-build candidates rank against the three tests
Editorial weighting, illustrative rather than measured. The last row is deliberately low: it is a good third system and a poor first one.
Why documents win the first slot
Document and onboarding automation wins for an unglamorous reason. The work is already being done by people, at a known cost, on files the bank owns, with an output a human already checks. That means a baseline exists, an approval step exists, and the failure mode is a person doing what they do today. There is no cliff.
The concrete shape at a regional bank looks like this. A commercial loan file arrives as forty to two hundred pages of mixed material: tax returns, financial statements, entity documents, insurance certificates, appraisals, purchase agreements, driver's licenses, bank statements from another institution. Today a processor opens each one, decides what it is, keys the fields that matter into the origination system, and chases what is missing. The cycle time is dominated not by keying but by waiting, because nobody discovers a missing insurance certificate until a human reaches page ninety.
The system that replaces that has four parts, and the middle two are where the engineering actually lives.
Classification. Split the packet into documents and label each one. This is where naive builds go wrong: they classify at the file level, then discover that a single scanned PDF holds nine documents and a fax cover sheet. The correct unit is the page, with a boundary model that decides where one document ends and the next begins, and a document-level label assembled from page votes. Confidence per boundary matters more than confidence per label, because a wrong boundary corrupts everything downstream while a wrong label is caught by the extractor failing to find its fields.
Extraction. Pull the fields. A tax return has a stable schema and should be extracted with a template-and-anchor approach, checked by arithmetic that has to balance. A financial statement prepared by a small accounting firm has no stable schema at all and needs a language model reading the page with the schema in the prompt and the page image alongside the text layer. Mixing the two approaches inside one pipeline, chosen by document type, is what separates a system that works on the messy sixty percent from a demo that works on the clean forty.
Validation. Every extracted field carries a confidence and a provenance: which page, which bounding box, which method. Rules run across fields, not within them. Does the entity name on the operating agreement match the borrower on the application. Do the statement totals foot. Is the insurance certificate expiration after the expected closing date. Is the appraisal dated inside the policy window. These cross-field checks catch more real problems than any single-field confidence threshold, and they are the part a credit officer will actually trust, because they are the checks that officer already performs.
Review. The processor sees the document image on the left, the extracted fields on the right, every field clickable to the exact place on the page it came from, and only the fields below threshold or failing a rule highlighted for attention. Nobody re-reads a clean tax return. The measurement that matters here is not extraction accuracy in the abstract. It is the fraction of files that pass with zero human corrections, and the median seconds spent on the files that do not.
The candidates, compared honestly
The table below is how we advise a bank to rank the field. Note that fraud detection, which most boards name first, sits in the middle rather than at the top. It is a strong system and usually the second or third build. As a first build it has two problems: the baseline it must beat is an existing rule engine that is already tuned, and the labels it must learn from are sparse and delayed, because confirmed fraud arrives weeks after the transaction.
| Candidate | Data the bank already owns | What finance can measure | Governance weight |
|---|---|---|---|
| Document and onboarding automation | Loan files, deposit account packets, imaged archives going back years | Cycle time to decision, processor hours per file, exception rate | Moderate. A person approves every file; the model is a preparer |
| Servicing and collections triage | Servicing records, payment history, contact logs, hardship notes | Contacts per resolution, roll rate by bucket, agent handle time | Higher. Consumer treatment and fair-lending review apply to prioritization |
| Credit memo preparation | Spread financials, prior memos, policy documents, covenant records | Analyst hours per memo, time from complete file to committee | Moderate. The analyst signs the memo and owns every number in it |
| Regulatory and management reporting | General ledger, data marts, prior filings, reconciliations | Close-cycle days, manual adjustments, restatements avoided | Moderate to high. Reporting accuracy has its own control regime |
| Fraud and anomaly detection | Transaction streams, device and channel data, confirmed-loss history | False positive rate at fixed capture, analyst alerts per confirmed case | High. A scoring model with consumer impact draws full model governance |
Model risk management from the first week, not the last
The single most expensive mistake a bank makes with a first AI build is treating governance as a review at the end. It is not a review. It is a set of artifacts that are cheap to produce while the work happens and nearly impossible to reconstruct afterward. Federal supervisory guidance on model risk management, issued jointly by the banking agencies and generally known by its Federal Reserve designation SR 11-7, sets the frame most examiners use: a model is developed with documented purpose and design, it is validated by someone independent of its developers, and it is governed by policies with named owners and ongoing monitoring. The document is publicly available and worth reading in full before the first design meeting.
What that means in engineering terms, on a document automation build, is specific.
- A written statement of purpose and limits before the first model runs. What the system decides, what it does not decide, what a person must approve, and what happens when it abstains. One page. It becomes the first section of the model documentation and it stops scope drift better than any change-control process.
- Versioned everything, from the beginning. Model version, prompt version, schema version, rule-set version, and the code that assembled them, all tagged and reproducible. When a validator asks in month nine why a file processed in month three came out the way it did, the answer has to be a command, not an archaeology project.
- A held-out evaluation set the developers never tune against. Built by the business, not by the engineers. Several hundred real files, labelled by the people who do the work today, covering the ugly cases on purpose: the scanned fax, the handwritten amendment, the statement in a format nobody has seen since 2011.
- Performance measured by segment, not in aggregate. By document type, by source, by scan quality, by borrower size. An aggregate number hides the segment where the system fails, and the segment where it fails is what a validator will find.
- Monitoring in production with alert thresholds set on day one. Extraction confidence distributions, abstention rate, human correction rate, and cross-field rule failure rate, tracked weekly with a stated level at which the model is pulled back to full review. Drift in this domain is real: a vendor changes a form layout and a field quietly stops parsing.
- An independent validation package assembled as you go. Data lineage, development record, testing evidence, limitations, and the compensating controls. Written during the build, not requested after it.
- Third-party and vendor risk documented where a model is not built in-house. If a hosted language model is in the pipeline, the bank needs to know where the data goes, what retention applies, whether anything is used for training, and what happens when the provider changes the model behind the same version label. That last point catches banks repeatedly.
Two of those deserve emphasis because they are where we see the most rework. Version pinning on a hosted model is not optional; a silently updated model is an unversioned model and a validator will treat it as such. And the held-out set must be built by the business, because an evaluation set built by engineers reflects what engineers found interesting, which is never the same as what goes wrong.
What the architecture looks like when it is right
A bank does not need a data lake to start, and the programs that begin by building one usually spend their first year on plumbing and their political capital with it. The first build needs four things, and they can be stood up in weeks.
A landing zone inside the bank's own cloud tenancy or data center. Documents and extracted data stay inside the bank's boundary, encrypted, with access logged. Nothing leaves except what has to, and what has to is a matter of written policy rather than an engineering convenience.
A processing pipeline that is idempotent and replayable. Every file gets an identifier. Every stage writes its output keyed to that identifier and the version of the code that produced it. Reprocessing a month of files under a new model version is one command and it does not touch the originals. This is the property that makes validation, backtesting and incident response tractable, and it is nearly free if designed in and expensive to add later.
A queue and a review application that the operations team actually likes. This is the part outside firms underinvest in and the part that decides adoption. Keyboard-driven. Field-to-page linking. Bulk approve for clean files. A one-key path to flag a bad extraction that feeds directly into the evaluation set. If a processor has to click four times to correct a field, they will stop using the tool and quietly go back to the old screen.
Write-back into the system of record, with a reconciliation. The extracted data has to land in the origination or servicing platform through a supported interface, and a daily reconciliation has to prove that what the pipeline said it wrote is what the platform holds. Silent write failures are the most common production defect in this class of system and the easiest to catch.
Where the effort actually goes on a first document automation build
Editorial weighting, illustrative rather than measured. The last row is deliberately low: model choice is the least of the work.
Failure modes we plan around
The pilot runs on clean data and dies on real data. A team assembles fifty tidy files, hits high accuracy, and presents. Then the system meets the archive. The fix is procedural: the evaluation set is drawn at random from real intake before anything is built, and the ugly cases are counted rather than set aside.
The core system integration is discovered late. The bank's origination or servicing platform is licensed, and what it will accept through an interface is a contractual and technical question with a lead time. Ask it in week one. It has changed the sequencing of more than one program we have run.
Nobody owns the model after launch. The build team leaves, monitoring alerts go to a distribution list, and six months later the abstention rate has doubled and nobody noticed. Name an owner inside the bank before the first line of code, and make the monitoring dashboard something that person looks at because it also tells them something they want to know about their operation.
The benefit is real but unbookable. Twenty minutes saved per file across eleven processors is not a headcount reduction and finance will not credit it. Decide in advance what the saved capacity is for: more files at the same staffing, a service-level commitment the bank can market, or a redeployment that is written down. A benefit with no destination is a benefit that gets argued away.
Governance is treated as an obstacle rather than a design input. The teams that resent the validation process build twice. The teams that write the model documentation as they go find that it improves the system, because stating plainly what a model may and may not decide surfaces the design questions nobody had answered.
An illustrative worked example, so the arithmetic is visible
Numbers below are illustrative and structural rather than a claim about any institution. Use the shape, put the bank's own figures in it, and have finance agree the baseline before the build.
Take a bank processing 900 commercial loan packets a year, each consuming an average of 3.5 processor hours across intake, classification, keying and exception chasing. That is 3,150 hours. Suppose the system passes 55 percent of files with no human correction and cuts the remainder to 1.4 hours each. The new total is roughly 567 hours, against 3,150. The value of the recovered 2,583 hours depends entirely on what the bank does with them, which is why the destination question above matters more than the accuracy question.
The second-order effect is usually larger and always harder to book. If missing-document detection moves from page ninety of a manual review to the first ten minutes of intake, the chase starts days earlier and the cycle time to a credit decision compresses by more than the processing hours suggest. That compression is what a commercial lender will talk about with a borrower, and it is the reason this build tends to have an internal advocate the fraud build does not.
How we work inside a bank
Precision Federal builds and deploys production systems, including inside U.S. federal agencies where the security and accessibility bar is set by the government and audited. That work shapes how we run a bank engagement: documentation as a deliverable, controls designed in rather than bolted on, and a system that has to pass someone else's review rather than our own.
The shape of a first engagement is consistent. In the first two weeks our engineers work on site or with the bank's data in the bank's environment, and produce four things: a written technical scope with measurable acceptance criteria, a real evaluation set drawn from live intake, an architecture that names every system it touches, and a governance plan mapped to the bank's model risk policy. That package is useful even if the bank stops there, which is the point of structuring it that way.
The build then runs in increments against real data in the bank's environment, never in a side account with a copy of production. Security review, access control and accessibility work happen inside the increments rather than after them. We deliver the source, the infrastructure as code, the pipelines, the evaluation suite and the model documentation, and the handover is a rehearsal: the bank's team deploys the system while our engineers watch, before the last invoice.
The bank keeps everything. The code, the models, the extracted data, the evaluation sets, the documentation and the customer relationship. Our pre-existing tooling is named and carved out, and the bank takes a perpetual license to use it inside the delivered system so no future maintainer is blocked. Pricing takes one of two shapes: fixed-price milestones tied to written acceptance criteria, or a committed team for a defined period when the roadmap is longer than the scope. We will tell you which one fits before you ask.
The first step is one email with a one-page brief: what the bank wants the system to do, which platform it must live inside, what data exists and who can grant access to it, the date that matters, and who approves a scope change. We return a scoped, priced statement of work.
Bottom line
Pick the first build on data ownership, a measurable number and supervisory familiarity, in that order, and document and onboarding automation usually wins. Build governance artifacts while the work happens rather than after it, because they cannot be reconstructed. Design the pipeline to be replayable and the review tool to be liked, since those two properties decide whether the system survives its first year. Name the owner and the destination for the recovered capacity before writing code. Do that once and the bank has more than a system: it has a proven path, and the second and third builds are ordinary projects instead of arguments.
Frequently asked questions
Document and onboarding automation, in most cases. It runs on files the bank already owns, it replaces work being done today at a known cost, a person still approves every file, and the pattern is familiar to examiners. Fraud detection is a strong second or third build, but as a first build it has to beat an already-tuned rule engine using labels that arrive weeks late. Customer-facing assistants with account authority should wait until the bank has a real governance record.
Federal supervisory guidance on model risk management, generally known by its Federal Reserve designation SR 11-7, frames what most examiners expect: documented purpose and design, validation independent of the developers, named owners, and ongoing monitoring. In practice that means a written statement of what the model decides and does not decide, versioned models and prompts, a held-out evaluation set the developers never tune against, performance measured by segment, and production monitoring with thresholds set before launch. Produce these during the build; they cannot be reconstructed afterward.
Plan on two weeks for scope, evaluation set and governance plan, then increments of six to ten weeks each against real data in the bank's own environment. The variable that moves the schedule most is not modelling. It is access: identity provisioning, a data extract, and what the origination or servicing platform will accept through a supported interface. Ask the integration question in week one, because it has a lead time set by the platform vendor rather than by the project.
The first system is usually better with a partner and the fifth is usually better in-house. The first build carries one-time costs that a partner has already paid: the pipeline architecture, the evaluation method, the review tooling, the validation package format. A partner should deliver those as artifacts the bank keeps, along with the source, the infrastructure as code and the documentation, and should structure the handover as a rehearsal in which the bank's own team deploys the system.
Agree the baseline with finance before the build: hours per file today, cycle time to decision today, exception rate today. Then measure the fraction of files passing with zero human corrections, the median time on the files that need attention, and the change in cycle time. Decide in advance what the recovered capacity is for, whether more volume at the same staffing, a service-level commitment, or a written redeployment. Saved hours with no stated destination do not get credited.