A model your sustainment team cannot retrain is a liability with a scheduled failure date. It works on the day it is delivered, it degrades quietly for a year, and then a data source changes or a policy shifts and the only people who know how to fix it have moved on to another contract. The program then has a choice between paying the original developer to come back on a sole-source justification nobody wants to write, or rebuilding from scratch what was already paid for once. This is written for the program manager who is planning transition on an AI workstream, and who would rather decide now what the handover contains than discover in month twenty-two that it contains a slide deck.
We build machine learning and data systems into production inside federal programs, usually as the specialist subcontractor under a prime, and transition is the part of the job we think about first. Not because it is a courtesy, but because a workstream designed to be handed over is also a workstream that runs better while it is live. The disciplines are the same: reproducible builds, written decisions, an evaluation you can run on demand, and no step that exists only in one person's head.
What makes an AI system harder to hand over than software
A conventional software handover works because the artifact is the answer. Source code plus a build pipeline plus a deployment definition gets a competent team to a running system, and from there they can read the code to learn the behavior. Machine learning breaks that chain in three places.
First, the source does not contain the behavior. The training script is a hundred lines; the behavior lives in the data and in the decisions made about the data. A team handed the script and the weights can run the model and cannot change it in any controlled way, because they do not know what the labels mean, why records were excluded, how the split was drawn, or which of forty hyperparameter choices were deliberate.
Second, correctness is not a test suite. Software has a passing state. A model has a performance distribution on a particular set of inputs, and the receiving team has no way to know whether a change they make is an improvement unless they inherit the same measurement the original team used, running the same way on the same held-out data. Without that, the first maintenance change is a coin flip, and the second is a rollback.
Third, the system decays without anyone touching it. Inputs shift, upstream systems change their formats, the population being scored changes, and performance falls while all tests pass and no alert fires. Sustainment on a model is therefore an active job with its own instrumentation, not a ticket queue.
Put together, this means the transition package for an AI workstream has to convey the ability to retrain, the ability to measure, and the ability to notice. Those three capabilities, not the weights file, are the deliverable.
What decides whether a sustainment team can actually maintain a delivered model
Editorial weighting, illustrative rather than measured. The last row is deliberately low: documents are the weakest predictor of a successful handover.
The transition package that works
Six components. Each is testable, which is the point: transition acceptance should be a demonstration, not a review of paperwork.
Runbooks written from the receiving team's position
Not a description of the system. A set of procedures for the things that will actually happen: deploy a new model version, roll back to the previous one, retrain on refreshed data, respond to each monitoring alert, rotate credentials, restore from backup, and onboard a new engineer to the workstream. Each written as steps a person on the sustainment team can execute without asking anyone, and each validated by having someone on that team execute it while the developing team stays quiet.
The validation is what makes a runbook real. A procedure that has never been run by its intended reader is a hypothesis. We schedule runbook execution as part of the work, not as a review of the document.
The retraining pipeline as a first-class system
Retraining should be one command or one pipeline trigger, and it should carry the whole chain: pull or receive the refreshed data, validate it against the expected schema and distribution, assemble the training and evaluation sets by the same split logic used originally, train, evaluate, compare against the incumbent model, and either promote or stop with a reason. If any of those steps is a person running a notebook, retraining will not happen after the sub leaves.
The comparison step deserves emphasis. A retraining pipeline that produces a model but does not tell you whether it is better than the one in production has moved the risk rather than removed it. The pipeline should refuse to promote a candidate that does not beat the incumbent on the primary metric and hold its own on the segment breakdowns.
The evaluation suite, owned by the program
The suite is the sustainment team's instrument. It contains the held-out data or the pointer to it, the metric definitions, the segment definitions, the baseline comparison, and the report generator. It lives in the program's repository under the program's configuration management from the first month, not delivered at the end. Anyone on the receiving team can run it against any candidate model and get a report that looks exactly like the acceptance report.
When the sustainment team owns the evaluation suite, every later question has an answer. Did the vendor's new version help. Did the data refresh hurt anything. Is the drift alert real or noise. Without it, each of those becomes a study.
Monitoring with named owners and rehearsed responses
Input drift, output drift, latency, error rate, and where ground truth arrives eventually, live accuracy against it. Each metric has a threshold, each threshold has an alert, each alert routes to a named role on the sustainment team, and each has a runbook entry describing what to do. Before handover, fire each alert deliberately, using historical data or an injected condition, and watch the receiving team respond. An alert nobody has ever seen is an alert that will be muted the first month it fires.
The decision record
The single highest-value document and the one most often missing. A short log of the choices that shaped the system and the reasons behind them: why these labels, why this split, why this architecture over the two alternatives tried, why this feature was dropped, what was tried and did not work, what the known weaknesses are and what would fix them. Ten to thirty entries, a paragraph each.
Its value is negative-cost avoidance. Without it, the sustainment team's first improvement idea is usually something the original team already tried and rejected for a reason nobody recorded, and the program pays twice for the same discovery. Written as the work happens, it costs nothing.
Paired operation for a defined period
The receiving team runs the system while the developing team is available but not acting. Long enough to cover a real retraining cycle and at least one unplanned event. This is where the gaps show, and the gaps always show, because no document survives contact with an operator who has a different mental model.
Writing it into the subcontract from the start
Every part of that package is cheap when planned and expensive when retrofitted, and the difference is decided at subcontract award. Five provisions carry most of the weight.
Transition is a priced line, not an obligation. Give it its own contract line item with its own funding and its own milestone. Written as an unfunded obligation at the end of a fixed-price scope, transition competes with the sub's margin, and the incentives are wrong for both parties. Priced separately, it is work like any other and both sides plan for it.
Acceptance is a demonstration. Write the transition acceptance criterion as: the receiving team, without assistance, retrains the model from refreshed data, runs the evaluation suite, deploys the result to the staging environment, and responds correctly to a triggered alert. That criterion cannot be met by a document, which is exactly why it works.
Deliverables accrue during performance. Require the evaluation suite, the runbooks and the decision record as incremental deliverables tied to development milestones, not as a closeout package. A monthly rendering of the decision record is a paragraph or two; a retrospective one is a week of interviews and half of it will be wrong.
A support tail, priced in advance. A defined block of hours or a stated monthly allocation for six to twelve months after transition, at a rate agreed at award. This is the provision that most reduces program risk, because the alternative when something breaks in month four is an unpriced negotiation under pressure. Cap it, define response expectations, and let it expire unused if all goes well.
Rights and background tooling settled up front. The sub's pre-existing frameworks are named at award, excluded from assignment, and licensed broadly enough that the sustainment team and any follow-on contractor can use, modify and maintain the delivered system without returning to the sub for permission. If sustainment will be recompeted, the license has to reach the successor. This is a one-paragraph conversation at award and a program-stopping one at closeout.
| Transition element | The weak version | The version that holds | Acceptance test |
|---|---|---|---|
| Retraining | Training script plus instructions | Pipeline with data validation, split logic, incumbent comparison and a promotion gate | Sustainment engineer retrains unaided and the pipeline blocks a worse candidate |
| Measurement | Final evaluation report | Evaluation suite in the program repository with locked held-out data and a report generator | Receiving team reproduces the accepted numbers within tolerance |
| Operations | System description document | Task-based runbooks, each executed by the receiving team before handover | Every runbook has a dated execution record signed by a receiving engineer |
| Monitoring | Dashboard link | Thresholds, alerts, named owners, response procedures, rehearsed | Each alert is fired deliberately and handled correctly |
| Knowledge | Exit interview and a slide deck | Decision record written during performance, plus paired operation | Receiving team explains three design choices and why alternatives were rejected |
| Continuity | Goodwill | Priced support tail with response expectations and an expiry | Rate and terms are in the subcontract at award, not negotiated later |
A schedule that makes transition boring
The shape below is what we plan against on a workstream of roughly a year. Adjust the durations; keep the sequence, because each step depends on the one before it having actually happened.
From the first month, the evaluation suite and the experiment registry live in the program's repositories, and the decision record gets its first entries. This costs a few days and it is the whole foundation, because every later transition artifact is a rendering of something recorded here.
At roughly the midpoint, the first end-to-end retraining rehearsal, run by the developing team but watched by whoever will inherit it, with the runbook drafted from what actually happened rather than from what was supposed to happen. Gaps found here are cheap.
Two to three months before transition, the receiving team's engineers get accounts, access and their first real task on the workstream. Not observation: a defect fix, a small feature, a data refresh. Access provisioning inside a federal environment takes longer than anyone plans for, and discovering that during the handover window is how transitions slip.
In the final six to eight weeks, paired operation. The receiving team executes; the developing team is reachable and does not touch the keyboard. Each runbook gets executed and dated. Each alert gets fired. One full retraining cycle completes with the receiving team driving.
At handover, the demonstration acceptance runs, and the support tail begins. If the paired period was real, this day is uneventful, which is the outcome to design for.
Where the cost of a transition artifact lands when it is deferred to closeout
Editorial weighting, illustrative rather than measured. The last row is deliberately low: packaging is the one item that retrofits cheaply.
How an engineering partner works inside your organization
Precision Federal is a small business engineering firm. We build machine learning systems, data platforms, cloud infrastructure and full-stack applications and deliver them into production inside federal agencies, working as a specialist subcontractor, teaming partner, protégé or nontraditional partner on an other transaction. On a program where sustainment will pass to your team or to a follow-on contractor, we plan the handover into the work from the start rather than treating it as a closeout task.
What we deliver in the first weeks is concrete. Week one: a written transition plan naming every artifact, where it will live in your repositories, and the demonstration that will accept it. First month: the evaluation suite running in your environment against a real held-out set, the experiment registry recording every run, the decision record open, and a measured baseline for the current process so there is a comparison point before any model exists. Everything after that accretes toward transition instead of away from it.
The prime keeps everything. The customer relationship is yours. Code is written into your repositories under your configuration management from the first commit, not delivered as a drop at the end. Data stays inside the environment your security agreement permits. Our background tooling is named at award, excluded from assignment, and licensed to you and to any successor. We work as a named subcontractor with an evaluated scope where that helps your proposal, or inside your delivery organization without a public role, whichever fits your posture.
Pricing takes one of two shapes. Firm fixed price against milestones that are the deliverables themselves, with transition as its own funded milestone accepted by demonstration. Or a committed team at a stated allocation, with the transition block and the support tail defined and priced at award. On an earned value program, work packages close on objective evaluation results rather than percentage estimates.
The first step is one email with a one-page brief: the program, the workstream, who will sustain it, the target environment, the schedule and the contract instrument. We read it and return a scoped, priced statement of work with the transition criteria written as tests. Days, not weeks, and no cost.
Five ways transitions fail
Transition is scheduled after the money runs out. The last two months are unfunded, the sub's engineers have been moved to the next program, and the handover becomes a document drop. Funding transition as its own line item at award prevents this entirely.
The receiving team is named but never staffed. Sustainment engineers are assigned on paper and are fully committed elsewhere until the week of handover. The fix is to give them a real task on the workstream months early, which also surfaces access problems while there is time.
The system was built somewhere it cannot be maintained. Development happened on infrastructure the sustainment team does not have, with tools they do not license, in an environment nobody can rebuild. Build in the destination environment from the first sprint, or against a stated path to it.
The evaluation set was never locked or never shared. The receiving team cannot reproduce the accepted numbers, so they cannot tell whether anything they do makes the system better or worse. Lock the held-out set early, hold it on the program side, and require the evaluation suite as an incremental deliverable.
Everything is documented and nothing is rehearsed. The package is complete, the review passes, and three months later a data refresh fails because a step in the runbook assumed a permission the sustainment team does not have. Rehearsal is the only cure, and it has to happen while the original team is still under contract.
Bottom line
Transition succeeds when the receiving team has already done the work once with the original team watching, and it fails when a document was substituted for that. The package is six things: task-based runbooks that have been executed, a retraining pipeline with a promotion gate, the evaluation suite in the program's repository, monitoring with named owners and rehearsed responses, a decision record written during performance, and a paired operating period. Write transition into the subcontract as a funded line item accepted by demonstration, require the artifacts incrementally, price a support tail at award, and settle background rights before the first commit. Done that way, handover day is uneventful and the program keeps a system it can still improve in year three.
Frequently asked questions
Six things. Task-based runbooks covering deployment, rollback, retraining, alert response and onboarding, each executed by the receiving team before handover. A retraining pipeline that validates data, trains, evaluates, compares against the incumbent model and gates promotion. The evaluation suite with locked held-out data, living in the program's repository. Monitoring with thresholds, named alert owners and rehearsed responses. A decision record explaining the choices that shaped the system and what was tried and rejected. And a paired operating period long enough to cover a full retraining cycle.
Long enough to cover at least one complete retraining cycle and one unplanned event, which usually means six to eight weeks on a program with monthly or quarterly refresh. The shape matters more than the length: the receiving team executes and the developing team is reachable but does not touch the system. If the developing team is still doing the work in week five, the period has not started yet and the schedule should say so.
As a separately funded contract line item with acceptance by demonstration rather than by document review. The criterion should require the receiving team, unassisted, to retrain from refreshed data, run the evaluation suite, deploy to staging and handle a triggered alert. Pair that with incremental delivery of the runbooks, evaluation suite and decision record during performance, and a support tail with rates and response expectations agreed at award rather than negotiated after something breaks.
Whatever the subcontract says, item by item, which is why it should be settled at award rather than at closeout. Trained weights, training code, the evaluation suite, shared transformation libraries, the data and any pre-existing tooling the sub brought can each land differently depending on who funded their development. The workable arrangement names the background items, excludes them from assignment, and grants a license broad enough that the sustainment team and any follow-on contractor can use, modify and maintain the system without returning for permission.
Silent decay. Inputs shift, upstream systems change, the scored population changes, and performance falls while every test passes and no error appears in a log. The defense is monitoring designed with the model rather than after it, with thresholds derived from the evaluation work, alerts routed to a named role, response procedures written as runbook entries, and each alert fired deliberately before handover so the sustainment team has seen it work.