Document processing at volume

Volume changes the problem, not just the size of it

Fifty documents a day is a person with a keyboard, and no software will beat her on cost or judgment. Five hundred a day is a queue with a backlog that grows on Mondays. Five thousand a day is a system, and at that point the design decisions that matter are almost entirely about what happens to the documents the system cannot handle. Most projects in this space are scoped as if the interesting part were reading the document. It is not. The interesting part is deciding, per document, whether a human needs to look, and making that human fast when they do.

Before anything else, get the two numbers that decide whether this is a project at all. How many documents per day, and what does one mistake cost. A wrong quantity on a purchase order costs a phone call. A wrong dosage read off a lab report costs something else entirely. A wrong remittance allocation costs a month of someone chasing a balance that was never actually open. The cost of a mistake sets your confidence thresholds, your review rate and, honestly, whether automation is appropriate at all.

What actually arrives is not what was in the sample folder

Every project starts with a sample set, and every sample set is unrepresentative in the same direction. Someone picks two hundred documents that are clean, from the top five senders, in one format. Real intake looks nothing like it.

Real intake contains scans made on a machine last serviced in 2016, with a diagonal shadow down the right side. Photographs taken on a phone in a warehouse, at an angle, with a thumb in the corner. A single forty-page PDF containing six separate documents that a mailroom scanned in one pass. Emails where the actual data is in the body and the attachment is a signature graphic. Documents in two languages. Handwriting in the margin that changes the meaning of a printed field. Duplicates, because the sender emailed and also faxed. And roughly one percent of things that are not the document type at all — a business card, a screenshot, someone's shipping label.

Build the sample set from a random slice of a real week, not a curated folder. Three hundred documents pulled at random will contain the ugliness in roughly the proportion you will actually face, and it will change your estimates. It usually changes them by a lot.

Splitting and classification are harder than extraction, and get no attention

Extraction is the part everyone plans for. The two steps in front of it are where projects lose their schedule.

Splitting. That forty-page PDF has to become six documents with correct boundaries. Page one of the third document is the only reliable signal that the second one ended. Get it wrong and you have merged an invoice with the packing slip behind it, and every field you extract afterward is from the wrong page. Header patterns, page-number resets, blank-page separators and layout change detection all help; none is sufficient alone, and a review step for low-confidence splits is not optional.

Classification. Which of your document types is this? Invoice, credit memo, statement, remittance advice, packing slip. These look similar and mean opposite things. Misclassifying a credit memo as an invoice produces a payable that should have been a receivable, and it will be caught eventually, by an argument.

Budget real time for both. In our experience they routinely take a third of the pipeline effort in a project where the plan allocated ten percent.

Field accuracy is the wrong number

Here is the arithmetic that reframes most of these projects. Take an invoice with twelve fields you need: vendor, invoice number, date, due date, purchase order, subtotal, tax, freight, total, terms, remit-to, and currency. Suppose the extractor is right on ninety-seven percent of fields. That sounds excellent.

If errors were independent, the chance all twelve are right on one document is 0.9712, about sixty-nine percent. Roughly three in ten documents would need a human. Errors are not independent in practice — a bad scan tends to break several fields at once, which actually helps, because the failures cluster into fewer documents. But the direction of the point holds and it is the thing to internalize: per-field accuracy is not the business metric.

The business metric is straight-through rate: the share of documents that go from intake to posted with no human touch. That is the number that maps to headcount, to backlog, to close timing. Ask any vendor for it, on your documents, and treat a quoted field accuracy as a non-answer.

Document population	Realistic first-year straight-through rate	What limits it
Top-20 senders, stable layouts	Often 80–90%	Occasional layout changes, scan quality
Broad vendor population, invoices	Commonly 40–70%	Layout variety, line-item tables, missing purchase orders
Mixed intake with splitting	Lower, until splitting is solved	Boundary errors poison everything downstream
Handwriting present in key fields	Much lower; sometimes not worth it	Legibility, and the cost of being wrong
Regulated or clinical content	Set by policy, not by the model	Review requirements you should not argue with

Those are ranges we consider honest to plan against, not guarantees. The spread within each row is driven mostly by how much validation you can do against systems you already own, which is the next section and the most underrated lever in the whole discipline.

Validation beats extraction

The cheapest accuracy in document processing does not come from a better model. It comes from checking the extracted values against things you already know.

Line items should sum to the subtotal. Subtotal plus tax plus freight should equal the total. The purchase order number should exist in your system, and the vendor on the document should match the vendor on that purchase order. The date should be within a plausible window — not three years ago, not next March. Quantities should reconcile against what receiving logged. The remit-to bank details should match what is on file for that vendor, and if they do not, that is not an extraction problem, that is a fraud check and it should stop the document cold.

Each of these is a few lines of logic. Together they catch a large share of extraction errors, they catch document-level problems the model cannot see, and they turn a low-confidence guess into either a confirmed value or a flagged exception. A pipeline with good validation and a mediocre extractor beats an excellent extractor with none, and it costs less to build.

The exception queue is the product

Most teams build the extractor for ten weeks and the review screen in the last three days. Invert that. If your straight-through rate is sixty percent at five thousand documents a day, two thousand documents a day flow through the review screen. That screen is where your staff spends their working life, and its design decides your actual cost per document more than the model does.

What a good one looks like: the document image on the left, the extracted fields on the right, and the field currently focused highlighted on the image so the eye lands in the right place. Entirely keyboard-driven — tab to move, type to correct, enter to accept, one key to reject the document to a supervisor. No mouse required for the common path. Fields sorted so the most-often-wrong ones come first. The reason it was flagged shown plainly at the top: “total does not match line items” is a different job from “low confidence on vendor name.”

A reviewer working a well-built screen on a familiar document type sustains something like two hundred to five hundred documents a day. On a badly built one — PDF in one window, data entry form in another, no highlighting — the same person does a fraction of that, and this is where the promised savings quietly disappear. The difference between those two numbers is the entire business case, and it is a user-interface problem, not a machine learning one.

Confidence, thresholds and the cost of being wrong

Route by confidence, and set the threshold from the cost of a mistake rather than from a round number. Three lanes work well: auto-post, review, and reject to a supervisor. A field whose error costs a phone call can auto-post at a lower threshold than a field whose error moves money to the wrong bank account.

One caution about confidence scores: a raw model score is not a probability of correctness unless somebody calibrated it. Before trusting a threshold, take a few hundred documents, bucket the predictions by reported confidence, and measure how often each bucket was actually right. If the bucket that reports ninety percent is right seventy percent of the time, your threshold means nothing and your auto-post lane is leaking errors into the ledger.

Where a language model helps, and where it should not be near the data

Modern language models changed one specific thing in this field, and it is worth being precise about what. They are very good at semi-structured variety — nine hundred invoice layouts that a template system would need nine hundred templates for. They read messy phrasing well. They handle a field that is sometimes labeled “Net Due” and sometimes “Amount Payable” and sometimes not labeled at all.

What they should not do is produce values from their own knowledge. If the model is asked for the vendor's address and the document does not clearly show it, the wrong behavior is a plausible address, and a plausible address is much worse than a blank, because nothing downstream will flag it. The design that prevents this is the one described above: every value carries the region of the page it came from, and a value without one is refused. Combine that with your validation rules and you have a system whose errors are visible instead of silent.

For genuinely fixed forms — the same layout every time, from a system that generates them — a deterministic template is still faster, cheaper and more predictable than a model. Use the expensive tool where the variety is, not everywhere.

Corrections have to flow back

Every correction a reviewer makes is a labeled example, keyed to a document, with the original prediction beside it. That is training data and evaluation data of a quality you cannot buy, generated for free by work you are already paying for. Most pipelines throw it away.

Capture it from day one: the document, the model's output, the corrected output, the reviewer, the timestamp. It gives you three things. A held-out evaluation set that reflects your actual documents. A ranked list of which fields and which senders cause the most rework, which is where the next improvement should go. And an early warning when a sender changes their layout, because corrections for that sender spike before anybody complains.

Do the cost arithmetic before the design

Page-level costs are small and volume makes them real. Optical character recognition and parsing typically runs from a fraction of a cent to a few cents per page depending on the service and whether you self-host. A language model pass over a dense page is usually somewhere between a cent and a couple of dozen cents, driven by how many tokens the page becomes and whether you send the image.

At five thousand pages a day that spread is the difference between a rounding error and a five-figure annual line item, and it is decided by design choices: whether every page goes to the expensive path or only the ones a cheap classifier could not handle, whether you re-run the whole document when one field is uncertain, whether you cache results for the duplicates that will absolutely arrive. Do this arithmetic on a napkin before the architecture is set, because retrofitting a cheap path is much harder than designing one.

And plan retention. Documents of this kind almost always contain personal or commercially sensitive data. Decide what is stored, where, for how long, and who can see it, before the first document lands. Retrofitting retention onto a pipeline that has been running for a year is an unpleasant and expensive quarter.

What it costs and how long it takes

A production pipeline for one document type — intake, splitting, classification, extraction, validation, an exception queue people can work at speed, and posting into your system of record — is typically ten to eighteen weeks and a build cost from the mid-five figures into the low six, depending mostly on how hard the target system is to write into and how bad the intake is. The second document type is faster but not free; expect somewhere around half, because splitting, review tooling and posting mostly carry over while validation rules do not.

Then there is the part that never appears in a proposal: someone owns the exception queue as a job. Not the software — the queue. They watch the straight-through rate weekly, notice when a big sender's layout changed, and decide when a recurring exception deserves a new validation rule. Pipelines do not fail loudly. They drift, the review load creeps up, and eighteen months later a business that automated its way from four clerks to one is quietly back to three.

When you should not build this

Under roughly a hundred documents a day, from a handful of senders, in stable formats, a person with a good template and a decent screen is cheaper than any system, and she will handle the weird ones without a meeting. If the documents are all genuinely different from each other — one-off contracts, correspondence — there is no repeating structure to learn and the automation has nothing to grip. And if a mistake carries clinical, legal or safety consequence, the answer may be that every document gets reviewed regardless, in which case the value of the project is making review faster, which is a real and worthwhile project, but a different one with different arithmetic.

The mistakes we get called in to fix

• A pilot on two hundred clean documents from one sender, which proved nothing about production
• Field accuracy quoted as the result, with straight-through rate never measured
• No splitting step, so multi-document scans silently corrupt every field
• Validation rules skipped in favor of tuning the model harder
• A review screen built in the last week, with no highlighting and no keyboard path
• Uncalibrated confidence driving an auto-post threshold nobody tested
• Extracted values with no page location, so invented data is indistinguishable from read data
• Corrections discarded, throwing away the best training and monitoring data you will ever have

Before you approve the build

• Volume per day and cost per mistake are both written down
• The sample set is a random week, not a curated folder
• Splitting and classification have their own schedule line
• Straight-through rate is the acceptance metric, measured on your documents
• Every field has at least one validation rule against a system you own
• Every extracted value carries the page region it came from
• Confidence is calibrated against measured accuracy before thresholds are set
• The reviewer screen is designed first and timed with a real reviewer
• Corrections are captured with the original prediction beside them
• Retention and access are decided before the first document lands

Bottom line

Document automation pays when it changes how many documents a human has to touch, and everything that decides that number sits outside the extraction model. Get a real sample. Solve splitting and classification properly. Validate against what you already know. Design the exception queue as the primary interface, because at a realistic straight-through rate it is where most of the labor still is. Keep every value tied to a place on the page. Then measure the one number that maps to money, and keep measuring it, because it drifts.

Frequently asked questions