Rescuing a stalled AI project: triage, restart, and when to stop

What a stall actually looks like from the outside

The pattern repeats across agencies, primes, and commercial buyers. Nine months in, the status reports are still green. The demo works on a laptop. The slides show accuracy numbers that sound impressive. And nobody can say what the system will do for the office that is paying for it, or when. Money is burning at full rate on work that has quietly stopped moving toward any decision anyone will make. Our team gets the call around this point, usually from the person who has to brief it upward next month.

The first thing worth saying: this is not a rare failure. It is the normal outcome of a project that was scoped as "apply AI to X" rather than as "change this specific decision, measured this specific way, by this date." The technology is usually the healthiest part of the effort. The wiring around it is what came loose.

Our triage asks four questions, in a fixed order, because each one makes the next one answerable. Is the data actually available? Is the metric the right one? Is the baseline known? Is the deliverable defined? A project that fails the first question cannot be saved by better modeling. A project that fails the fourth cannot be finished at all, because there is nothing to finish.

Question one: is the data actually available

"Available" has a hard definition. The data exists, in a form someone can query, in an environment the engineering team can legally reach, with a named person who has already approved that reach. Anything short of all four is a promise, not a data set. Most stalls we see live here. A kickoff deck listed six sources. Two arrived. Two sit behind a privacy review nobody scheduled. Two turned out to be screenshots of a dashboard whose underlying tables were retired.

The approval chain is the part that gets underestimated. For federal health data, de-identification under 45 CFR 164.514(b) is either the safe-harbor path or an expert determination, and each takes calendar time and a signature. For a new use of records already covered by a Privacy Act system of records, the notice under 5 U.S.C. 552a(e)(4) may need to be amended before the data can be used the way the project assumes. For anything marked CUI, the receiving environment has to satisfy NIST SP 800-171, and for defense contracts the CMMC program rule at 32 CFR Part 170 sets the assessment level that the contract flows down. None of it is exotic, all of it is measured in weeks, and none of it appears on a modeling Gantt chart.

Our first working session on a rescue is a data-access audit with dates and names attached. For each source: who owns it, who signs, what the transfer mechanism is, what the environment is, and what the realistic date is. Where a source is more than sixty days out, we design around it, using a bridge data set that carries the same schema and the same failure regimes so the engineering keeps moving while the paperwork clears. A project that waits idle on an approval is burning budget to produce nothing.

Question two: is the metric the right one

A wrong metric is more dangerous than a bad model, because it hides the failure behind a number that keeps going up. The classic version: a fraud or safety detector trained on a class that appears in 0.4% of records, reported by overall accuracy. Predicting "no" for every record scores 99.6%. The model looks superb and finds nothing.

The metric has to be the shape of the decision. If a human reviewer will work a queue of 200 flagged items a day, the metric is precision at the top 200, not area under the curve. If a missed case costs three orders of magnitude more than a false alarm, the metric is recall at a fixed, budgeted false-alarm rate, and the threshold belongs in the requirement. If the output is a number a program office will plan against, calibration matters more than ranking, which means expected calibration error and a reliability curve, not a single headline score.

We also check where the number was computed. Three failures account for most inflated results: the test set was resampled to balance classes, so the reported precision has no relationship to the live base rate; a feature encodes the answer, such as a case-closure code that only exists after the outcome; or the split was random across time, letting the model see the future. Rebuilding the evaluation on an honest, time-ordered, unbalanced holdout is a two-day job that often changes the entire read on the project.

Question three: is the baseline known

Almost every stalled project we open has no measured baseline. Nobody wrote down what the current process achieves, what it costs, or how long it takes, before the AI work started. Without that, there is no way to prove the system helped, and no way to know when it is good enough to stop building.

The baseline is not a research benchmark. It is today's process, measured on the same data slice, over the same period, with the same metric. If claims are adjudicated by hand at 82% agreement with a senior reviewer and 14 minutes per case, that pair of numbers is the bar. If a rules engine already catches 61% of the events, then a model at 68% is a modest improvement that has to justify a hosting footprint, an authorization package, and a maintenance obligation. Programs are canceled every year over models that beat a research benchmark and lost to the incumbent spreadsheet.

Reconstructing a baseline late is usually possible. Historical case files, audit logs, ticket timestamps, and prior-year reports carry enough signal to establish it, and our engineers treat that reconstruction as a deliverable in its own right because it is the thing that makes every later claim defensible. When the baseline genuinely cannot be reconstructed, that finding by itself changes the plan: the first phase becomes measurement, and the modeling waits.

Question four: is the deliverable defined

"A model" is not a deliverable. A deliverable is an artifact a named person accepts against written criteria. Weights in a repository satisfy nobody. What gets accepted looks more like this: a container that runs inside the customer's accredited enclave at a stated impact level, an inference endpoint holding a 400-millisecond 95th-percentile latency under a stated load, an evaluation suite that reproduces the reported numbers from raw data with one command, model documentation sufficient for the reviewing authority, and a written procedure for the day the model is wrong.

Acceptance is a contract question as much as an engineering one. Under FAR Part 46, the government inspects and accepts against the criteria in the contract, and nonconforming work is handled under FAR 46.407. If the statement of work says the contractor will "develop machine learning capabilities," there is nothing to inspect and no moment at which anyone can say the work is done. That ambiguity is comfortable early and fatal late. Rewriting it costs a bilateral modification under the Changes clause at FAR 52.243-1 for a fixed-price contract or FAR 52.243-2 for cost reimbursement, and the change order should be definitized rather than left open, per FAR 43.204(b).

• The statement of work and every modification to it
• The last three status reports, unedited
• The current evaluation numbers and the script that produced them
• A list of data sources with owner, approval status, and delivery date
• The environment the system must eventually run in, and its authorization status
• Burn to date against obligated funds, and the period of performance end date
• The name of the person who will accept the final deliverable

What a rescue engagement examines first

We keep the diagnostic short and fixed, because a long one is just more drift. Two weeks, a written memo, a decision the customer can act on. The single most informative step is the reproduction: take the current reported result and rebuild it end to end from raw data. Roughly half the time it does not reproduce, and the gap between the reported number and the reproduced number is the truest measure of how far the project actually is.

Two contract facts shape what the memo can recommend. On a cost-reimbursement contract, the Limitation of Cost clause at FAR 52.232-20 requires the contractor to give written notice when costs over the next sixty days will push spending past 75% of the estimated cost, and the incrementally funded version at FAR 52.232-22 works the same way against allotted funds. That notice is a decision point most programs waste. Separately, the Stop-Work Order clause at FAR 52.242-15 lets a contracting officer pause performance for up to 90 days, which is the right instrument when a legal or authorization blocker has to be cleared before more money is spent on engineering. On the grant and cooperative-agreement side, 2 CFR 200.308 requires prior approval for a change in scope or in the key person, and 2 CFR 200.340 governs termination. Knowing which lever exists changes the recommendation.

When the right answer is to stop

Telling a customer to stop is the most valuable thing a rescue can produce, and the hardest thing for an incumbent vendor to say. We say it when the evidence supports it. Stopping cleanly at month ten costs far less than a managed decline through month twenty-four, and it preserves the relationship, the reusable assets, and the credibility of the person who sponsored the work.

Signal in the triage	What it means	Our call
Primary data promised at kickoff, still undelivered after 90 days, no named approver	There is no path to a working system, only a hope	Stop the build; convert to a short data-access engagement with dated gates
The metric improves but the decision does not change	Even a perfect model produces no value at the point of use	Stop the modeling; redesign around the decision, or close out
No baseline exists and none can be reconstructed	Success is unprovable, so acceptance will never happen	Restructure: measurement phase first, modeling after
No named person will accept the deliverable	The work has no landing point inside the organization	Restructure: write acceptance criteria into a modification, or close out
Funds past 75% with under a third of scope demonstrated	The remaining budget cannot reach the stated end state	De-scope to one provable outcome, or stop and bank the assets
Authorization, data-rights, or privacy blocker unresolved	Engineering cannot legally reach production from here	Stop-work while the paperwork path is run, then restart

Stopping well is its own piece of engineering. The shutdown package should leave the customer with the reconstructed baseline, the cleaned data pipeline, the evaluation suite, the documented failure analysis, and a plain statement of what would have to be true for the effort to be worth restarting. That package is often worth more than the model would have been: the next attempt starts from evidence, not assumptions.

What restart looks like when the answer is continue

When the four questions come back answerable, the restart is deliberately narrow. One decision, one metric with a threshold written into the scope, one data path with named approvers and dates, one accepting official, and a first demonstrable result inside six to eight weeks. We keep the original end state in view and refuse to chase it in one leap, which is what stalled the project the first time.

Data rights get settled at restart, not at closeout. On defense work, noncommercial technical data falls under DFARS 252.227-7013 and noncommercial computer software under DFARS 252.227-7014, while work developed under an SBIR or STTR award carries SBIR data rights with a protection period of 20 years from the date of award under the SBIR/STTR Policy Directive. Which clause governs the model weights, the training pipeline, and the evaluation suite has a real answer, and settling it early removes an argument that surfaces later at the worst moment.

The last piece is cadence. Weekly written status against dated milestones, with a red-amber-green that is allowed to be red. A status report that has been green for nine consecutive months is not a sign of health. It is a sign that nobody has been asked a question they could fail.

Who we are on a rescue

Precision Federal is an SBIR and STTR shop that builds production AI, ML, data, and cloud systems, and we take rescue work as prime or as a subcontractor to the firm currently holding the contract. Our practice is led by a former professor in technology who ranks in the top 200 of more than 200,000 on Kaggle, holds seven cloud certifications, and has spent twenty years building production systems for federal agencies across five consulting firms, three of them federal. Behind that sits a standing bench of named engineers, licensed professional engineers, and domain specialists across defense, health, energy, transportation, and public-sector data, so the person reading a clinical pipeline is not the same person reading a grid-telemetry pipeline. We are SAM.gov active, CAGE 1AYQ0, and JCP / DD-2345 certified.

We work well alongside an incumbent. Most rescues do not need anyone replaced. They need someone with no stake in the last nine months to reproduce the numbers, name the blocker, and put dates on the fix. When the finding is that the incumbent's engineering is sound and the scope was wrong, we say that plainly, and we say it in writing.

The ask

If a project of yours is late and you are not sure why, send two documents to [email protected]: the statement of work and the latest status report. Within two business days you will get a written read back covering what looks like the binding blocker, which of the four questions the project currently fails, and whether we would recommend continuing, restructuring, or stopping. If a full triage makes sense, that read includes a fixed-price two-week scope. If it does not, we will tell you that too, and there is no charge for the read.

Frequently asked questions