Three different problems wearing the same file extension
PDF is a page-description format. It says where to put ink. It says nothing about what is a column, what is a total, or which of the four dates on the page is the invoice date. That is the whole difficulty in one sentence, and it is why a project that sounds like "read the number off the page" turns into a quarter of work. Before anything else, sort your incoming documents into three piles, because they are three different engineering problems and mixing them is how estimates go wrong.
Born-digital. Generated by somebody's accounting or ordering system and never printed. There is a real text layer with exact character positions. These are the good ones and they are usually 40 to 70 percent of a supplier-document flow, though the share varies enormously by industry — higher in software and professional services, much lower in construction, freight and healthcare.
Scanned images. Paper that went through a scanner or a multifunction printer, often at 200 dots per inch because that was the default in 2011 and nobody changed it. No text layer. Skew, speckle, staple shadows, a slightly grey background, and the occasional page scanned upside down.
The hybrid, which causes the most trouble. A scan that already carries a text layer added by some earlier tool, of completely unknown quality. Your parser reads it happily, gets confident text, and the text is wrong — a 5 read as an S, a decimal point dropped, two columns interleaved into one line. Born-digital failures are loud. Bad-OCR-layer failures are quiet, and quiet failures are the expensive kind. Detect this pile deliberately: compare the embedded text against a fresh recognition pass on a sample, and if they disagree badly, ignore the embedded layer for that source.
You are probably here because
- Two or three people key documents into a system all day and the volume keeps growing
- You bought a tool that works on your top five suppliers and nothing else
- A template changed and nobody noticed for six weeks
- Someone quoted a 99 percent accuracy figure and you cannot work out what it was 99 percent of
The last one is the important one. Field accuracy and document accuracy are different by a wide margin, and the gap between them is the entire economics of the project.
The number that matters is straight-through rate
Field-level accuracy is what vendors quote and it is close to useless on its own. A document with twelve fields, each read correctly 97 percent of the time and each failing independently, is fully correct about 69 percent of the time. Errors are not independent in practice, which helps, but the direction is right: per-field figures always flatter.
The number to hold everyone to is straight-through rate — the percentage of documents that need no human touch at all, because every field passed both extraction and your validation rules. That is what determines headcount, and it is the only figure that maps to money. On a mixed supplier flow with header fields plus line items, a well-built system typically lands somewhere between 55 and 80 percent straight-through in its first production quarter, and climbs with tuning against the specific senders that fail.
The second number is correction time. If a document that fails takes a person forty seconds to fix in a good review screen, a 65 percent straight-through rate on 3,000 documents a month is a genuine saving. If it takes four minutes because they have to open the source PDF separately, find the page, and retype the whole thing, the same 65 percent saves almost nothing. Correction time is a design decision, not a fact about the documents, and it is where a lot of projects quietly fail.
Header fields are easy. Line items are the project.
Invoice number, date, total, supplier identity: these have strong local cues, they appear once, and they are usually printed in a distinct position. Expect high numbers on born-digital documents and merely good numbers on decent scans.
Line items are a different discipline. Real tables have merged cells, no ruling lines, wrapped descriptions running to three visual rows that are one logical row, sub-totals that look exactly like items, freight and surcharge rows that are not items at all, and a table that continues onto page four with a repeated header and a "carried forward" line. Row-level accuracy on unruled multi-page tables of that kind commonly runs in the low-to-mid eighties before validation and can be pushed higher with source-specific tuning — but never assume the header figure applies to the body of the document.
| What you are pulling | Born-digital | Clean scan at 300 dpi | Poor scan or phone photo |
|---|---|---|---|
| Single header fields number, date, total, currency | Very high; failures are layout surprises, not reading errors | High; digits and dates are the usual losses | Mixed. Expect to review most of them |
| Party identity which supplier, which site | High if you match against your own master data, not the printed name | Same — matching beats reading | Same, and often the only field that survives |
| Ruled table rows | High | Good | Poor. Ruling lines break under compression |
| Unruled or wrapped table rows | Good, with per-source tuning | Moderate | Route to a person |
| Handwriting quantities pencilled on a delivery note | n/a | Do not plan on it | Do not plan on it |
| Stamps, ticks, signatures | Detectable as present or absent | Detectable as present or absent | Detectable, unreliable |
Business validation beats confidence scores
Model confidence tells you how sure the model is, which is not the same as whether the answer is right, and on documents the two come apart in a specific and annoying way: a clean-looking mis-read is confident. The check that catches those is arithmetic and reference data, not probability.
Do the sums. Line items plus tax plus freight should equal the stated total; if they do not, something is wrong somewhere on the page and you do not need to know where to route it for review. Check the supplier against your own master list rather than trusting the printed name. Check the order number exists and is open. Check the quantity against what the warehouse actually received. Check the date is inside a sensible window — an invoice dated three years ago is either a duplicate, a re-send, or a mis-read year.
These checks are cheap, they are explainable to an accounts payable clerk in one sentence, and they catch a class of error that no confidence threshold will. They also give you something a score cannot: a reason. "Line items sum to 4,120.50 but the total says 4,210.50" tells the reviewer exactly where to look, and it turns a forty-second correction into a five-second one.
Page preparation is worth more than the choice of recognition engine
Teams spend weeks comparing recognition engines and an afternoon on image handling, and the returns are the other way round. Deskew, correct rotation, normalise contrast, drop the scanner's grey background, and get to 300 dpi or better before recognition rather than after. On genuinely poor scans this moves accuracy more than any engine swap we have measured. If your suppliers are sending 200 dpi faxes, the cheapest improvement available to you is an email asking them to change one setting.
The supplier tail, and the day a template changes
Document flows follow a steep distribution. Across the supplier bases we have looked at, the top twenty senders typically account for 70 to 85 percent of documents, and then there is a tail of hundreds of parties who send you two documents a year each. This shape decides your strategy.
For the head, per-source handling pays. You know exactly what their layout looks like, you can tune to it, and you can validate hard. For the tail, per-source work is a losing game — you will never recoup the setup on two documents. Use a general layout-agnostic approach for the tail, accept a lower straight-through rate there, and size the review queue accordingly. Trying to reach the same rate on a two-document-a-year sender is a common and expensive misallocation.
Then plan for the template change, because it is not an exception. Across a base of a few hundred suppliers, layouts change constantly — a new accounting system, a rebrand, an added tax line. The failure mode that hurts is silent: the supplier moves the invoice number two centimetres, your extractor starts reading the customer reference instead, and it is a valid-looking number so nothing complains. Guard against it with a per-source drift monitor: track straight-through rate, validation-failure rate and field-length distributions by sender, weekly, and alert on a break. A supplier whose pass rate drops from 94 to 40 in a week has changed something.
Where a document extraction project actually spends its time
Indicative allocation across projects of this shape. The extraction step is rarely the largest line and never the riskiest.
The review screen is the product
Whatever the model does, some documents come to a person. The screen those people use determines whether the project pays back, and it is usually treated as an afterthought.
What a good one does: shows the page image and the extracted fields side by side, highlights the exact region each value came from, puts the cursor in the first failing field, states the validation rule that failed in plain words, and lets the whole thing be corrected and confirmed on the keyboard without touching a mouse. Every correction is stored with the page region, so it becomes training data and a per-source diagnostic at the same time.
What a bad one does: shows a form. The reviewer opens the PDF in another window, scrolls to page three, reads the number, comes back, and types it. That is the four-minute correction, and at 1,000 failing documents a month it is more than a full-time role you did not budget for.
Send us fifty real documents and we will tell you the honest ceiling.
Fifty representative documents with commercially sensitive values redacted, plus the list of fields you need and your monthly volume, to contact@precisionfederal.com. You get back a written note: the three-pile split we see, which fields will be reliable, which will not, and a realistic straight-through range. One business day. No charge, no meeting.
contact@precisionfederal.comKeep the page. Forever.
Store the source document, the page image, and the coordinates of every extracted value, and keep them for as long as you keep the transaction. This is not a nicety. When a supplier disputes an invoice eighteen months later, or an auditor asks where a figure came from, the answer needs to be a highlighted region on the original page, not a database row. It also means a future improvement to your extraction can be re-run over history without asking anyone to re-send anything.
Two related habits. Never overwrite an extracted value in place — keep the original reading beside the corrected one, with who changed it and when, because that pair is how you tell a model problem from a document problem. And keep a hash of the source file, because duplicate submissions are constant and the cheapest duplicate check in existence is a byte-identical match.
When you should not build this
Low volume. Under roughly 200 to 300 documents a month with a handful of fields each, a person doing it in an hour a day is hard to beat, and the build plus the ongoing maintenance will not pay back for years. Say the number out loud before you start.
The sender could give you data instead. A meaningful share of document extraction projects exist because nobody asked. Your three largest suppliers can very likely send a structured file, or a portal export, and getting one of them to do so removes a chunk of the problem permanently rather than approximating it forever. This conversation is worth having before an engineering budget is approved, not after.
The document is not the bottleneck. If invoices are keyed within a day and then wait a week for approval, faster keying does nothing to the cycle time anyone complains about.
The stakes make review mandatory anyway. If every extracted value must be checked by a person for legal or safety reasons regardless of confidence, extraction is still useful — as pre-fill that makes checking fast — but you should scope and price it as a speed-up, not as automation, and set expectations accordingly.
The failures we get called in to fix
- A per-field accuracy target in a contract, with no definition of what counts as a document
- Trusting an embedded text layer that was produced by an unknown tool on a bad scan
- Confidence thresholds with no arithmetic checks, so confident mis-reads sail through
- Per-supplier templates for the whole tail, burning weeks on senders with two documents a year
- No drift monitoring, so a template change is found by a customer or an auditor
- Line items promised at header-field accuracy in a scoping document nobody re-read
- A review screen with no page image, turning every correction into a four-minute job
- Corrections overwriting the original value, destroying the only signal about what is going wrong
A realistic path to production
Document extraction: first production version
Step two is the one that gets cut and should not. Keying 150 documents by hand, twice, and comparing the two passes gives you two things nothing else will: a test set that is genuinely independent of the system, and a measurement of how often two careful people disagree about what is on the page. On messy documents that disagreement is real — a surprising share of it turns out to be genuine ambiguity in the document rather than carelessness — and knowing the size of it stops the whole project chasing a target that was never achievable.
Before you sign anything
- The accuracy target is stated per document, not per field
- Line-item and header-field targets are stated separately
- The test set was keyed by hand and is not visible to whoever tunes the system
- Validation rules are written in business terms and shown to the people who will review failures
- Correction time is measured, with a target, on the actual screen
- Source page, region coordinates and original reading are retained for the life of the record
- Per-sender drift monitoring exists and someone owns the alert
- Duplicate detection covers both identical files and re-sent documents
- The tail has its own expected rate and its own staffing assumption
- Somebody has asked the top suppliers whether they can just send data
Bottom line
Extracting data from documents nobody standardized is a solvable problem, and it is solved mostly outside the extraction step: in sorting your documents honestly, in validation rules that use your own business facts, in a review screen that makes a correction take seconds, and in monitoring that catches a template change in a week rather than a quarter. Hold everyone to straight-through rate per document. Measure correction time. Keep the page. And check the volume first — if it is small, the person doing it now may genuinely be the right answer, and finding that out early is worth more than a pilot.
Frequently asked questions
Header fields on born-digital documents read very reliably; line items on unruled multi-page tables are considerably harder, and poor scans harder again. The figure worth agreeing is straight-through rate — documents needing no human touch — which commonly lands between 55 and 80 percent in a first production quarter on a mixed sender base, then improves as the largest senders are tuned.
Use both, split by volume. Per-source handling earns its keep on the twenty or so senders who make up most of your documents, because you can validate hard and tune precisely. For the long tail of parties who send two documents a year, template work never pays back — use a layout-agnostic approach, accept a lower rate, and staff the review queue for it.
Monitor by sender, weekly: straight-through rate, validation-failure rate and the distribution of field lengths and formats. A layout change usually shows as a sharp drop for one sender while everything else is stable. Without per-sender monitoring the change is silent, because a value read from the wrong place still looks like a valid value.
Yes, and the plan should say so. Some share of documents will always fail validation, handwriting and poor photographs will not be reliable, and disputes need someone who can look at the original page. The realistic outcome is a smaller team doing more interesting work on exceptions, not an empty desk.
As a rough threshold, a few hundred documents a month with several fields each starts to make a case, and a few thousand makes a strong one. Below that, look hard at the alternatives first: asking your largest senders for a structured file, or a short intake form, removes part of the problem permanently instead of approximating it every month.