Two questions, and most buyers only ask the first
The first question is whether the model is good enough at the task. Vendors will answer that one all day, because the answer is usually yes for a curated demonstration. The second question is what it costs to find out when it is wrong, and that one decides whether the project makes money. A language model that is right ninety-two percent of the time is excellent for drafting an email and unusable for producing a payment amount, and the difference has nothing to do with the model. It is entirely about who has to check, how long checking takes, and what happens to the eight percent that slips through.
This article is about the second question. It is written for someone deciding where to spend a budget rather than which model to call, because in our experience the model choice moves the outcome far less than the task choice does.
You are probably here because
- A pilot came in under budget and the production version quoted at ten times the pilot
- The model bill is trivial but the team says the feature is not paying for itself
- Someone proposed a language model for a job a database query would do exactly
- Two vendors quoted the same workload and one is a hundred times cheaper per document
All four have the same root: the cost that matters is not per token, it is per checked output, and nothing in a token price tells you what that will be.
The price is not the cost
Start with the number everybody quotes. As of the middle of 2026, frontier-tier model pricing runs roughly one to fifteen dollars per million input tokens and something like five to seventy-five dollars per million output tokens depending on the tier and the vendor. Smaller models sit one to two orders of magnitude below that. These numbers have fallen consistently for three years and will have moved by the time you read this, so treat them as a shape rather than a quote.
Now put a document through it. A ten-page report is roughly five to eight thousand tokens. Extracting a dozen fields from it might cost two or three cents on a mid-tier model and a fraction of a cent on a small one. At a hundred thousand documents a year, the model bill is a few thousand dollars. That is a rounding error in a business case, and it is why "the API is cheap" is technically true and strategically useless.
Then price the check. If a person spends three minutes verifying that document at a fully loaded forty-five dollars an hour, the review costs about $2.25 — roughly a hundred times the model call. Multiply by the same hundred thousand documents and review is $225,000 a year. The model is noise. The review is the project.
This reframes the whole engineering problem. A team optimizing token spend is optimizing the small number. A team designing so that eighty percent of outputs are safe to accept without a human look is optimizing the large one. The second team wins by an enormous margin, and the techniques that get them there — confidence thresholds, routing the uncertain cases to a person, validating against a source of truth, refusing rather than guessing — cost more engineering and almost no inference.
| Cost line | Typical share of the first year | What moves it |
|---|---|---|
| Model inference | Usually under 10 percent, often under 3 | Model tier, prompt length, retries. The easiest lever and the least valuable |
| Build | The majority in year one | How many exception paths the process actually has |
| Human review | The majority in year two onward | What fraction of outputs a person must look at. This is the design decision |
| Evaluation and monitoring | 10 to 20 percent, and permanent | How often the model or the upstream data changes |
| Rework when something moves | Lumpy, once or twice a year | Vendor deprecations, prompt changes, a new document format |
The failure mode you are actually buying
Conventional software fails loudly. A field is missing, a service is down, an exception is thrown, and something turns red. A language model fails quietly and fluently: it produces a well-formed, plausible, confident answer that is wrong. Nothing turns red. There is no stack trace. The output looks exactly like the correct output.
That single property explains most of what follows. It means the cost of a mistake is not just the mistake but the search for it. It means tasks with an existing downstream control are cheap to automate and tasks without one are expensive. And it means the question "how accurate is it?" is less useful than "what does it do when it does not know?" A system that declines on the hard fifteen percent and is reliable on the rest is worth far more than one that is a little more accurate overall and never admits doubt, because the first one lets you skip review on most of the volume and the second one does not.
Where they clearly earn their money
Turning unstructured material into structured fields. Contracts, invoices, inspection reports, clinical notes, correspondence. The alternative is people reading, the volume exceeds the people, and each field can be checked against the source text. This is the most reliable win available today and it is not close.
Triage and routing. Deciding which queue a ticket, claim or application belongs in. Errors are visible within hours, correction is cheap, and even a modest improvement over round-robin assignment pays back quickly because it shortens every downstream cycle.
Search across your own material. Where people currently ask a colleague, dig through a shared drive, or give up. The value is in the giving-up cases, and it depends entirely on the answer citing its source so the reader can verify in five seconds rather than trusting.
First drafts a human will edit. Responses, summaries, descriptions, specifications. Editing is genuinely faster than writing from nothing, the human is the control, and the failure mode is visible to the person who owns the output. The one condition is that somebody actually edits rather than forwards.
Translation between formats and dialects. Free text to a schema, one vocabulary to another, an old code set to a new one, a specification to test cases. These are tasks where the correct answer is constrained enough to validate programmatically, which makes the review cheap.
Where they clearly do not
Arithmetic and anything that must reconcile. A model can find the numbers; it should not add them. Extraction plus ordinary code is right every time at no marginal cost, and it is auditable. Any design where a model produces a total is a design with a defect in it.
Queries over structured data. If the answer lives in a database, the answer is a query. A model may help write the query, and a person should be able to see it. Asking a language model to reason its way to a number that SQL would return exactly is paying more for a worse answer.
High-volume classification where you already have labels. If you have fifty thousand historical examples of a two-way decision, a small purpose-trained classifier will typically run at a fraction of a cent, respond in milliseconds, hold still between releases, and often beat a general model on your specific distribution. This is one of the most common overspends we see.
Anything requiring a guarantee. Regulatory determinations, safety interlocks, payment authorizations, anything where "usually correct" is a legal problem rather than an operational one. A model can prepare the input to those decisions. It should not be the decision.
Small volumes. A task performed thirty times a year cannot repay the evaluation harness, the monitoring, and the attention it will need. That is true no matter how well the model does it.
Relative cost per unit of work — order-of-magnitude, not a benchmark
Our rough ordering for a document-processing workload at scale, including review labour. The gaps are the point; the exact values will differ for your data.
The middle, which is most real work
Between the clear yes and the clear no sits the majority of proposals, and they resolve on one number: what share of outputs can be accepted without a human looking at them. Call it the auto-accept rate. At zero, you have bought a slightly faster typist and the economics rarely work. At eighty percent you have removed four fifths of the labour and almost anything pays back.
The honest way to plan is to assume the first version has a low auto-accept rate and that raising it is the work. What raises it is not a better model. It is knowing which outputs to trust: a validation rule that checks the extracted date against the document, a cross-check between two fields that must agree, a rule that anything below a calibrated confidence goes to a person, a refusal behaviour when the source does not contain the answer. Each of those is engineering, each is cheap to run, and together they are what turns a demonstration into a system with a defensible unit cost.
It also means you should ask a vendor for the auto-accept rate rather than the accuracy. Accuracy on a curated set is a marketing number. The share of production volume that ships without a human touch, at your required error rate, is the number that appears in your budget.
Retries, evaluation runs and long prompts are the three quiet multipliers
A pilot's model bill is usually measured on a single clean pass. Production adds retries on failure, a second call to check the first, an evaluation suite that reruns hundreds of examples on every change, and prompts that grow as edge cases are handled. Three to five times the naive estimate is normal, and it is still usually a small number. Budget for it, but do not let it drive the design — a decision that saves tokens and adds review time is a loss.
Send the workload and we will price the honest version.
Email the task, the annual volume and who checks the output today to contact@precisionfederal.com. You get back a written estimate of cost per checked output for a language-model approach and for the cheapest alternative we can see, including the one where the answer is not to use a model. One business day. No charge, no meeting, no deck.
contact@precisionfederal.comBig model, small model, and when it matters
The instinct is to use the strongest available model everywhere, and for a first version that instinct is correct. It removes one variable while you find out whether the task is tractable at all, and at pilot volumes the cost difference is measured in tens of dollars.
It stops being correct at volume, and the transition is usually well signposted. Once the task is stable, well specified, and running at scale, the same work often holds up on a cheaper model, sometimes with a small amount of tuning on your own examples. The saving is real, but so is the cost: a second model to evaluate, a second thing that changes when a vendor updates it, and a migration that has to be proven with a golden set. Do it when the annual inference bill is large enough to fund that work, and not before. Below roughly the cost of a couple of engineering weeks a year, it is a distraction from the review cost that is actually dominating.
The three costs nobody puts in the first budget
Evaluation. A set of real examples with known correct answers, run on every change. Building the first one takes one to three weeks of somebody who knows the domain, and it is the highest-value artifact in the project. Without it you cannot tell an improvement from a regression, which means you cannot safely change anything, which means the system freezes.
Review capacity. Someone has to check the uncertain cases, and that person needs a queue, a tool, and time in their week. This is an operational commitment, not a project cost, and it is the line most often discovered after go-live.
The re-do. Model vendors deprecate versions, prompts drift as exceptions accumulate, and upstream document formats change without warning. Assume one or two events a year that require re-validating the system against the golden set. Budget a week or two of engineering for each. Teams that plan for this treat it as maintenance; teams that do not treat it as a crisis.
How to price a pilot honestly
Pick a real slice of volume, not a curated one. Run it end to end. Then report three numbers: cost per output including review, the auto-accept rate at your required error level, and the time it took a person to check a sample. Those three, on your own data, tell you what production costs. Nothing else does, and a pilot that does not produce them has not answered the question it was funded to answer.
The most useful thing a pilot can discover is that the answer is no. That result costs weeks instead of quarters, and it is far more valuable than a successful demonstration that quietly assumed a review rate nobody will staff.
The mistakes we are called in to fix
- A business case built on token cost, with the review labour missing entirely
- A model computing totals that a formula would compute exactly and for free
- A general model doing a two-way classification with fifty thousand labelled examples sitting unused
- Retrieval bolted onto structured data where a query returns the exact answer
- Accuracy reported on curated examples, with no measurement of the auto-accept rate
- No abstention behaviour, so the system answers confidently on inputs that contain no answer
- Token optimisation that increased review time, trading the small cost for the large one
- No evaluation set, so nobody can approve a prompt change and the system stops improving
A three-week way to find out
Unit-Economics Pilot
Before you sign
- Cost per checked output is calculated, not cost per token
- The auto-accept rate is stated at your required error level
- Review capacity is named as an operating commitment with a person attached
- The system has a defined behaviour when the answer is not present
- Any total or computation is produced by code, not by the model
- The cheapest non-model alternative was priced and rejected for a reason
- An evaluation set of real examples exists and belongs to you
- The plan says what happens when the vendor deprecates a model version
- Year-two cost is in the case, with review as the dominant line
Bottom line
Language models are cheap to call and expensive to trust, and the whole economics of a project follows from that. They earn their keep where the input is messy, the volume is high, the alternative is a person reading, and a mistake is caught by something that already exists. They lose to a query, a rule or a small trained model wherever the problem is well specified, and they should never be the last word on a number that has to reconcile. When you compare options, compare cost per checked output on your own data. That single number reorders most vendor shortlists, and it usually reorders them in favour of a smaller, duller, more reliable design than anyone brought to the first meeting.
Frequently asked questions
Almost always because every output is being reviewed by a person. At any real volume, review labour dominates the cost by one to two orders of magnitude over inference. The fix is not a cheaper model; it is a design that can safely skip review on the majority of cases, which means validation rules, calibrated confidence, routing the uncertain cases to a human, and a system that declines rather than guesses.
Rarely first. Fine-tuning helps most with format, style and a narrow well-defined classification where you have thousands of labelled examples. It does not add knowledge reliably and it does not stop a model producing confident wrong answers. Try retrieval, better instructions and validation first; consider tuning when the task is stable, the volume is high enough to fund the ongoing evaluation, and the goal is cost or latency rather than knowledge.
Give both the same slice of your real data, including the difficult cases, and ask for three numbers: cost per output including any review they assume, the share of outputs that ship without human involvement at your error tolerance, and what the system does when the answer is not in the source. Price alone will mislead you, because the cheap quote frequently assumes a review rate the buyer has to staff.
Only once inference is a meaningful share of the total and the task is stable. The switch costs evaluation work, a migration, and a second thing to maintain, so it should be funded by savings larger than a couple of engineering weeks a year. Until then, effort spent reducing how much output needs checking returns far more than effort spent reducing the cost per call.
Whether a wrong answer is caught by something that already exists. A task sitting in front of an existing approval step, reconciliation, or downstream check is cheap to automate because the control is already funded. A task where a mistake surfaces months later through a customer or an auditor is expensive, because you have to build and staff the control yourself before any of the savings are real.