The decision you are actually making
A company that says "we need to build an AI capability" is usually deciding two things at once and only noticing one of them. The visible decision is whether to hire or to buy. The invisible decision, the one that determines whether anything survives eighteen months, is who inside the company will own a system after the excitement wears off. Every failed in-house effort we have looked at failed at the second question, not the first. The model was fine. The pipeline ran. Then the one person who understood it changed teams, and the thing quietly stopped being maintained.
Owning a capability means something narrower and harder than owning a model. It means your staff can change a prompt, retrain on new data, read the evaluation dashboard, explain a bad output to a regulator or a customer, and deploy a fix without calling a vendor. That is a small set of skills. It is also a set most companies never explicitly staff for, because the hiring conversation starts at "machine learning engineer" instead of at "who answers the phone when the extraction is wrong."
The good news is that the work decomposes. There is a first hire, a short list of platform choices that are painful to undo, an evaluation habit that has to be installed before the first production release, and a governance layer that federal and regulated customers will demand in writing. Get those four in the right order and an in-house capability is a two-quarter project. Get them out of order and it becomes a permanent consulting line item.
What determines whether an in-house AI capability survives year one
Editorial weighting from public sources and practitioner reading. Illustrative, not a measured statistic.
The first hire
The instinct is to hire a research scientist. Resist it. A research hire with no data access spends the first six months writing notebooks against sample files and the next six months looking for a new job. The first hire should be the person who can get the data moving and keep a service running: a senior data or platform engineer who has shipped something into production and is comfortable with model APIs, containers, and identity.
Concretely, the first person should be able to stand up an ingestion path from your systems of record, write the transformations, put a service behind an authenticated endpoint, wire logging and metrics, and hold a straight conversation with your security team about where records are stored. Modeling skill matters less at this stage than most hiring managers assume. In the majority of first-year use cases the model is a call to a hosted frontier model or a small open-weights model on your own hardware, and the differentiating work is retrieval quality, data hygiene, and the interface a human uses to accept or reject an output.
The second hire is the one who thinks in metrics. This is the person who builds the labeled evaluation set, argues about what counts as correct, tracks per-slice performance, and refuses to ship on a demo. Some companies find this person inside the business rather than inside engineering. A claims examiner, a contracts specialist, or a lab supervisor who is good with a spreadsheet and stubborn about definitions is frequently a better second seat than another engineer.
The third seat is the domain owner, and this seat should be internal by definition. It is the person whose workflow the system touches. They set the acceptance bar, they decide what happens on a low-confidence output, and they own the escalation path. If nobody is willing to take this seat, the project is not ready and no amount of engineering will make it ready.
Platform decisions that are expensive to reverse
Most technical choices in an AI system are cheap to revisit. You can swap a model in an afternoon. You can rewrite a prompt in ten minutes. A short list of decisions is different, because reversing them means re-processing history, re-negotiating access, or re-doing an accreditation. Those are the ones to slow down on.
| Decision | Cost to reverse later | What to decide up front |
|---|---|---|
| Where the data physically sits | High. Moving a workload from commercial cloud into a GovCloud or on-premises boundary means re-doing network design, key management, and any authorization already granted. | Pick the strictest boundary any near-term customer will require, then run everything there. It is cheaper than a migration. |
| Identity and access model | High. Retrofitting attribute-based access onto a system built with a shared service account touches every query path. | Federate to your existing identity provider on day one. Carry the requesting user's identity all the way to the data layer. |
| Log and audit schema | High. You cannot reconstruct which model version produced last quarter's output if you never wrote it down. | Log input hash, model and prompt version, retrieved sources, output, confidence, reviewer action, and timestamp on every call. |
| Document chunking and index contract | Medium to high. Changing chunking means re-embedding the corpus and invalidating every stored citation offset. | Store the raw source plus stable offsets, so a re-index is a batch job rather than a re-ingest. |
| Model access abstraction | Low if a gateway exists. High if provider SDK calls are scattered through the codebase. | One internal interface for all model calls. Providers change; your call sites should not. |
| Evaluation and label store | Medium. Labels created without provenance cannot be trusted or reused. | Treat labels as a first-class dataset with versioning, annotator identity, and disagreement records. |
The boundary decision deserves extra attention if any part of your future revenue is federal. Federal customers care about where data rests. A Department of Defense workload handling controlled unclassified information sits under DFARS 252.204-7012, which points at the NIST SP 800-171 control set and carries a 72-hour cyber incident reporting obligation. Sensitive DoD workloads are placed at Impact Level 4 or 5 under the Cloud Computing Security Requirements Guide, and civilian agencies expect FedRAMP authorization for the hosting layer. None of that is impossible. All of it is much easier if the first line of code was written inside the right boundary.
Why the data access step is where projects die
The single most common stall we see has nothing to do with modeling. It is that six weeks into the effort, the engineering team still cannot read the production records they were hired to work on. The data lives in a system owned by another department. The owner of that system has no incentive to grant access and a long list of reasons not to. Legal has a question about the third-party contract that governs the records. Privacy wants a de-identification plan.
Handle this before the first hire starts, not after. Write down each source system, its owner, the legal instrument that governs it, and the specific access mechanism you will use. If the records are protected health information, decide early whether you will work under 45 CFR 164.514(b) Safe Harbor de-identification, expert determination, or a limited data set with a data use agreement. If they are student records, the FERPA rules at 34 CFR Part 99 govern the disclosure. If they are federal-furnished, the contract clause that delivered them controls what you may do with them, and the rights are not always what people assume. Our note on why AI projects fail at the data access step goes through the failure pattern in more depth.
The practical move is to make data access a named deliverable with a date and an owner, ahead of any model work. A team that can query real records in week two will outperform a better-credentialed team that gets access in month five, every time.
Evaluation discipline is the capability
Here is the uncomfortable truth about AI systems: the model is a commodity and the evaluation rig is the asset. Anyone can call a frontier model. Very few organizations can say, with evidence, how often their system is wrong, in which situations, and by how much. That evidence is what lets you ship, what lets you defend the system in an audit, and what lets you swap in a cheaper model next year without fear.
Build the evaluation set before you build the system. Two hundred to five hundred real examples, labeled by people who actually do the work, is enough to start and is dramatically more useful than ten thousand synthetic ones. Hold out a portion that nobody is allowed to look at during development. Measure disagreement between your own labelers first, because a task where two experts agree only 70 percent of the time cannot be scored against a single gold answer.
Then measure per slice, never in aggregate. A 92 percent overall accuracy that hides 61 percent on the document type representing your largest customer is worse than useless, because it manufactures confidence. Slice by source system, document type, record age, region, and any protected attribute that appears in your regulatory posture.
- False extraction rate. How often the system produces a confident value that is not in the source. The number that matters most for records work, covered in this piece.
- Refusal and abstention rate. How often the system declines. A system that never abstains is not calibrated; a system that abstains constantly is not useful.
- Retrieval hit rate at k. Whether the right source document reaches the model at all. Most "hallucination" is a retrieval failure wearing a costume.
- Reviewer override rate. How often a human changes the output. The cheapest production signal you will ever collect, and it arrives free.
- Per-slice worst case. The lowest-performing slice, tracked as its own release-gating metric.
- Latency and cost per task. The two numbers that decide whether the workflow survives contact with volume.
Wire a regression suite into your deployment path so that no prompt change, model upgrade, or index rebuild reaches production without the numbers being re-run and compared. This is ordinary software engineering applied to a probabilistic component, and it is the difference between a capability and a demo.
The governance layer, and why it is lighter than you fear
Companies delay AI work because they imagine a governance burden that does not exist yet at their scale. The workable version is short. Adopt the NIST AI Risk Management Framework 1.0 structure of GOVERN, MAP, MEASURE, MANAGE as your outline, write a one-page model card per system, keep an inventory, and define which use cases require a human in the loop. That is a defensible program for most private-sector companies.
If you sell to federal agencies, know what your customer is operating under. OMB Memorandum M-25-21, issued April 3, 2025, sets the framework for federal agency use of AI, including designated Chief AI Officers, agency AI use case inventories, and minimum risk management practices for high-impact AI. Your agency buyer will be asking you for the artifacts that let them satisfy those practices: documentation of training data provenance, testing results, monitoring plans, and an explanation of the human oversight design. Producing those from an existing evaluation suite is a one-week task. Producing them from nothing, after the fact, is a quarter.
Two more items belong on the list. If the interface will be used by federal employees or the public, Section 508 applies through 29 U.S.C. 794d and the Revised 508 Standards at 36 CFR Part 1194, which incorporate WCAG 2.0 Level AA. And if any part of the work touches technical data controlled under the International Traffic in Arms Regulations, staffing rules follow the ITAR definition of a U.S. person at 22 CFR 120.62, which covers citizens and lawful permanent residents alike. Plan the team composition against the actual rule, not against folklore.
Build then hand off: how the outside partner should work
There is a version of outside help that leaves you dependent and a version that leaves you capable. The difference is designed in at the start, in the contract and the staffing plan, and it is visible in whether your people are writing code in week three or watching a demo in month four.
Build-and-hand-off engagement shape
The handoff gate in step six is the only honest test. Not a knowledge transfer session. Not a slide deck. Your engineer changes the extraction schema, runs the evaluation suite, and deploys, while the outside team watches and says nothing. If that works, the capability is yours. If it does not, the remaining gaps are now specific and can be closed in days.
Staffing the engagement matters as much as the plan. Our engineers work in your repository, in your ticket system, in your review process, with your people on the pull requests. Precision Federal is led by a former professor in technology who ranks in the Kaggle Top 200 of more than 200,000 competitors and holds seven cloud certifications, and the firm carries twenty years of building production systems for federal agencies across five consulting firms, three of them federal. We bring a standing bench of named engineers, licensed professional engineers, and domain specialists across defense, health, energy, transportation, and public-sector data, and we size the team to the problem instead of to a bench utilization target.
What a real handoff contains
- The repository, in your organization, with full history and no vendor-only branches
- Infrastructure as code that rebuilds the whole environment from empty
- The evaluation set, the label store, and the regression suite wired into your pipeline
- A model card and a data flow diagram per deployed system
- Runbooks for the five failures that will actually happen, written by whoever hit them first
- Named ownership of every credential, key, and cloud account, with the vendor removed
- Written data rights that match your contract posture, including any federal flow-downs
That last line is worth a sentence. Rights in software and data are governed by clause, not by intuition. Commercial work usually turns on what your master services agreement says about work made for hire and background intellectual property. Federal work turns on FAR Part 27 and clauses such as FAR 52.227-14 for data, DFARS 252.227-7014 for noncommercial computer software, and DFARS 252.227-7018 with its protection period for anything developed under a Small Business Innovation Research award. Decide which regime applies before the first commit, because the answer changes what you may reuse across customers.
The cost math, honestly
A senior machine learning or data engineer in a major United States market runs roughly $190,000 to $260,000 in total compensation. Loaded with payroll taxes, benefits, equipment, and software, the real annual cost per seat lands nearer $250,000 to $340,000. A three-person pod is therefore $750,000 to $1,000,000 a year before a single GPU hour or a single dollar of model spend. Recruiting takes three to six months per seat, and the first hire will not be productive until data access exists.
Set that against a scoped build-and-hand-off engagement that produces a working system, an evaluation suite, and two trained internal owners inside four months. The engagement is usually a fraction of the first-year payroll and it removes the recruiting lag entirely. The right sequence for most companies is to buy the build, hire into the running system, and let the outside team fade out on a schedule you set in the contract. Hiring into something that already works is much easier than hiring to create it, and the candidates are better because the job is real.
When not to build in-house at all
Three situations argue against it. First, if the workflow is genuinely commodity and a mature product covers it, buy the product and spend your engineering on something proprietary. Second, if the volume is low and stays low, a capability with a $1,000,000 annual carrying cost cannot be justified by a task performed forty times a month. Third, if no internal person will accept ownership, do not proceed. Build a service relationship instead and be honest that it is one.
Everything else is buildable. The question is only sequencing, and the sequencing is the same nearly every time: owner, access, evaluation set, boundary, build, measure, hand off.
Common questions from companies making this call
Can we start with a pilot and decide later?
Yes, if the pilot is built on the platform you would keep. A pilot on a throwaway stack teaches you almost nothing about the real cost and produces no reusable asset. Run the pilot inside the boundary, with the logging schema and the evaluation suite you intend to keep, and the pilot becomes step one of the build instead of a sunk cost.
Do we need our own GPUs?
Usually not in year one. Hosted inference covers most first workloads, and open-weights models in the 7B to 70B range run acceptably on modest rented capacity when a boundary requires self-hosting. Buy hardware when your token volume, latency floor, or data boundary makes the arithmetic obvious, not before.
How do we keep the capability from decaying after handoff?
Two mechanisms. Put the regression suite in the deployment path so decay is loud rather than silent, and keep a small retained review arrangement with the build team for the first two quarters. A few hours a month of code review from the people who built it is inexpensive insurance against drift.
What if our security team says no?
Bring them into week one rather than week ten. Most security objections to AI systems are about data egress, logging of sensitive content, and unclear identity propagation. All three are design decisions, and all three are cheap to satisfy at the start and expensive to retrofit.
Frequently asked questions
A senior data or platform engineer who has shipped a production service, not a research scientist. The first six months are ingestion, identity, logging, and interfaces. Modeling depth matters more at the second or third seat, once real data is flowing and an evaluation set exists.
Where the data physically rests, how identity propagates to the data layer, and what you write into the audit log. Each of those is expensive to change after history accumulates. Model choice, prompts, and even the vector index are comparatively cheap to revisit.
Two hundred to five hundred real, expert-labeled examples is enough to gate a first release, provided it is sliced by document type, source system, and any attribute your regulator cares about. Measure labeler agreement first; a task where experts disagree cannot be scored against one gold answer.
Your repository, infrastructure as code, the evaluation and label store, model cards, runbooks, and full credential ownership with the vendor removed. The gate is behavioral: your engineer ships a change end to end while the partner watches without helping.
A three-person pod is roughly $750,000 to $1,000,000 a year fully loaded, plus three to six months of recruiting per seat. A scoped build-and-hand-off engagement usually costs a fraction of the first year and removes the recruiting lag, after which you hire into a system that already runs.