The systems that hold everything
Every police, fire, and EMS agency runs on a small number of data systems, and every public safety AI project either respects them or dies during integration. Computer-aided dispatch holds the call: time stamps, location, the call type as the dispatcher coded it, units assigned, and every status change from dispatched to cleared. The records management system holds the legal record: incidents, arrests, property, cases, and the officer narrative. EMS agencies file a patient care report into a NEMSIS-conformant state repository. Fire agencies filed to NFIRS for four decades and are moving to NERIS, the replacement the U.S. Fire Administration stood up.
None of these were built to be read by a model. CAD is a transaction log tuned so a dispatcher gets the next screen in under a second. RMS is a legal filing cabinet whose schema was shaped by state statute and whatever a records supervisor decided in 2014. They come from different vendors, are joined by a nightly export, and disagree about the same incident more often than anyone in the building says out loud. So the first honest week of an engagement measures how often the CAD event number and the RMS case number reconcile. That number predicts everything built after it.
Public safety AI: readiness by workflow
Editorial weighting of payback against accountability risk. Illustrative, not a measured statistic.
CAD is a record of dispatch, not a record of crime
This is the mistake that ends most law enforcement analytics programs. The call type in CAD is the caller's account, filtered through a call-taker who had eleven seconds to pick a code from a dropdown. It is often wrong on arrival, gets reclassified later, and the reclassification rarely propagates back. Officer-initiated events are worse: a self-initiated stop appears in CAD because an officer was there, and officers are there because of last year's deployment plan.
Train a model on that and it learns the deployment plan. Chicago's Strategic Subject List is the best-documented case: a 2016 RAND evaluation found no measurable effect on victimization, and the city's Office of Inspector General published an advisory in January 2020 on the data quality and governance problems underneath it. Los Angeles ended its PredPol contract in April 2020.
A second gap sits above that one. The Bureau of Justice Statistics' National Crime Victimization Survey has consistently found that fewer than half of violent victimizations are reported to police at all. Reported crime and crime are different quantities, and a system that treats them as one will under-serve the neighborhoods that call least. Say so in the design document, or a defense expert will say it for you at trial.
NIBRS is a data quality program wearing a reporting requirement's clothes
The FBI retired the Summary Reporting System on January 1, 2021, making the National Incident-Based Reporting System the national standard for crime data. NIBRS is incident-based: up to ten offenses per incident with no hierarchy rule, across 52 Group A offenses in 24 categories plus ten Group B categories, in seven segment types covering administrative, offense, property, victim, offender, and arrestee data. Submissions move through a state UCR program in a NIEM-conformant format, and the technical specification defines three families of validation: structural checks, data element edits, and relationship edits between segments. Participation dropped sharply in the 2021 data year when large agencies could not certify in time, and the burden landed on records units that still spend hours clearing rejection files.
This is the least glamorous and highest-return AI application in the building. A model that reads the narrative and the structured fields together, and flags the relationship edit that will bounce a submission before it is sent, gives a records clerk back a week a month. The failure mode is a false alarm rather than a wrongful arrest, and the acceptance test writes itself.
| System | Authoritative for | What it cannot tell you |
|---|---|---|
| CAD | Times, locations, units, status changes, initial call coding | What the incident was; whether a report was ever written |
| RMS | Final classification, arrests, property, cases, narratives | Response performance; calls that never became reports |
| NIBRS extract | Standardized offense, victim, offender, arrestee structure | Unreported crime; anything below the threshold |
| NEMSIS ePCR | Patient assessment, interventions, transport, outcome fields | Encounters handled off the record; hospital outcome |
| NFIRS / NERIS | Fire incident type, casualties, property loss, resources | Cause certainty; anything the officer did not code |
Call triage: what a model may touch, and what it may not
Emergency medical dispatch runs on protocol. Most centers use a structured card system with determinant codes that assign acuity, and the protocol is defensible precisely because it is deterministic and documented. Replacing it with a model is a bad idea and, in most states, an unsellable one. The opportunity sits alongside it: flagging cardiac-arrest cues in the audio for the call-taker to verify, identifying the caller's language and routing to interpretation, spotting duplicate calls so three units are not sent, surfacing premise hazard history, and pre-filling fields the call-taker would otherwise type while talking. Every one is an assist a human confirms. None silently changes an acuity level.
The clock is the constraint that kills naive designs. NFPA 1710 measures fire and EMS response in seconds at the 90th percentile across alarm processing, turnout, and travel, with 240 seconds of travel for the first-arriving engine as the benchmark. A suggestion that lands after the dispatcher already sent the unit is worse than no suggestion, because it becomes something the agency has to explain later. Any model in the call path needs a hard latency budget in the requirement. Two shifts widen what is possible: NG911 built on the NENA i3 standard puts text, images, and location into the call, and alternative response, from the 988 Suicide and Crisis Lifeline to community responder programs, means dispatch now routes to more than three unit types.
Resource allocation is an argument about the objective function
Demand forecasting is solved in the technical sense. Call volume by hour of week and small geography is one of the more predictable time series in government, and a sound forecast plus AVL-driven closest-unit recommendation plus a move-up model improves coverage in almost any system that lacks one. The hard part is the objective. Minimizing mean response time, minimizing the 90th percentile, and minimizing the worst-served census tract produce three different deployment plans, and the differences are largest in the neighborhoods where the choice is politically visible. Whoever writes the objective function is making policy, so build the optimizer with the objective as a configurable, documented choice, produce the map for each option, and let the chief and the council pick. Crew fatigue and the station, contract, and mutual aid terms that make some plans illegal to execute belong inside the model, not in a footnote.
Transcription and report drafting
Officers spend a large share of a shift writing. Transcription and drafting has the clearest payback in public safety and the most legal exposure, which is why it became a legislated question in under two years. Axon released Draft One in 2024, generating draft narratives from body-worn camera audio. Prosecutors in King County, Washington directed law enforcement partners in 2024 not to submit AI-generated reports in their cases. Utah enacted S.B. 180 in 2025, requiring a disclosure on reports drafted with generative AI, and California followed the same year with draft retention and officer certification requirements. Any agency buying here should assume its own legislature is drafting something similar.
The legal mechanics explain why. A police narrative is the officer's account, and the officer will be cross-examined on it. Federal Rule of Evidence 602 requires personal knowledge and Rule 901 requires authentication. Under Brady v. Maryland, 373 U.S. 83 (1963) and the cases following it, material favorable to the defense has to be turned over, which now plausibly includes the difference between the first machine draft and the signed report. An agency that cannot produce that difference two years later has made a discovery problem for itself.
The design that survives is narrower than what most vendors demonstrate. Draft from the officer's own dictated account, not from recorded statements of witnesses and suspects. Store the first draft, the final, and the diff as linked records. Force field-by-field review rather than a single approve button. Carry the disclosure on the document. And measure word error rate on the agency's real audio, because radio traffic, cross-talk, and accented speech move accuracy far more than model choice does. Title VI, Executive Order 13166, and the Justice Department's language access guidance at 67 FR 41455 govern how a federally funded agency serves limited-English-proficiency speakers, and unmeasured accuracy across languages is a compliance finding waiting to happen.
What auditability means in technical terms
"Auditable" appears in nearly every public safety AI solicitation and is almost never defined. Here is what it has to mean when a records request, an inspector general, or a subpoena arrives.
Six mechanics of a defensible audit trail
Retention is where this gets expensive and needs honest pricing at the start. Public safety records live under state schedules measured in years, and an open homicide case has no end date. If the model that generated a lead is deleted because the vendor deprecated it, the agency loses the ability to explain a decision still in litigation. Tie model and artifact retention to the records schedule and legal hold, not the vendor's release cadence. And note that audit logs are themselves records: they fall under most state public records acts and they are discoverable.
Bias review that survives a defense attorney
Bias dominates these procurements, and most vendor answers to it are a paragraph of intent. A review that holds up is a measurement plan written before deployment naming five things: which subgroups get measured, which metric is used, what sample detects a difference that matters, how often it runs, and who can suspend the system when a threshold is crossed. Without the last one it is not governance.
The metric choice is substantive. False positive rate parity, calibration within groups, and equal selection rates are mutually incompatible in most real data, so picking one is a value judgment that belongs in the open with counsel in the room. NIST's face recognition study NIST IR 8280 (2019) showed why: measured demographic differentials in false positive rates varied by orders of magnitude across algorithms. The 2024 settlement between the City of Detroit and the ACLU following a wrongful arrest is the operational lesson, establishing that a facial recognition match alone cannot supply probable cause. The NIST AI Risk Management Framework 1.0 gives agencies vocabulary reviewers recognize, and state law is arriving on top of it, including Colorado's 2024 statute on consequential automated decisions taking effect in 2026.
The security floor: CJIS and 28 CFR Part 23
Anything touching criminal justice information falls under the FBI CJIS Security Policy, whose recent revisions restructured it around NIST SP 800-53 control families. The practical vendor requirements: multi-factor authentication, FIPS-validated encryption in transit and at rest, fingerprint-based screening for anyone with access, a signed CJIS Security Addendum, access limited to screened personnel inside the United States, and audit logging that supports the periodic audits state CJIS systems agencies and the FBI conduct. FedRAMP authorization of the underlying cloud is a separate question from CJIS compliance, and buyers routinely conflate the two.
Criminal intelligence carries a second regime. 28 CFR Part 23 governs criminal intelligence systems operating with federal funding: entry requires reasonable suspicion, information must be reviewed and purged on a five-year cycle, and dissemination has to be logged and limited to those with a need and a right to know. Here is the trap. A model that infers an association between two people and writes it back into the system has created a new intelligence record on a machine's suspicion rather than an officer's. Mark the inference, source it, and subject it to the same review and purge rules as everything else.
Put explainability in the statement of work
All of this is achievable. What stops it is that it gets discussed in the demo and never written into the contract, so it becomes a change order when the agency has the weakest bargaining position. These are the clauses our team writes into public safety scopes.
- The explanation artifact is defined per decision type, with a named audience. A records clerk, a supervisor, and a prosecutor need different documents.
- Decision replay is an acceptance test. The agency picks ten past decisions at random and the vendor reproduces each from stored artifacts.
- The log schema is agency property and exportable in bulk, documented and testable without vendor assistance.
- The bias measurement plan is an attachment with thresholds and a suspension trigger: metric, subgroups, cadence, and the role that can pull the system.
- Model and version changes require notice and a re-baseline. No silent upgrades to a system whose outputs sit in open cases.
- Retention follows the records schedule and legal hold, not the product roadmap, and is priced accordingly.
- Transition-out obligations are specific. Data, logs, and model artifacts delivered in a documented format on termination, at a price fixed at award.
Federal buyers have a template for much of this. OMB's 2025 memoranda on federal AI use and AI acquisition set minimum risk management practices for high-impact uses and push agencies toward data rights and lock-in protections. They bind federal agencies rather than cities and counties, but state and local counsel are already borrowing the language. Our engineers scope this work in the same order every time: a data reconciliation study first, then one workflow end to end whose failure mode is recoverable, then the audit and evaluation layer around it. What we demonstrate at the end is the audit trail.
Common objections we hear
Our vendor says the model is proprietary and cannot be explained.
Model internals and decision records are different things. No agency needs the weights. Every agency needs the input, the version, the output, the threshold, and the human action stored in its own systems. A vendor that cannot provide the second set is declining to be accountable, and should be scored that way.
Is this not really a records management problem rather than an AI problem?
Often, yes, and saying so is part of the job. Much of what agencies want from AI is delivered by fixing the CAD to RMS join, standardizing call type coding, and cleaning the NIBRS pipeline. Those fixes make the model work possible, and doing them first is cheaper than modeling around the mess.
Frequently asked questions
Yes, and it is one of the safest places to start. Models that read narratives together with structured fields catch structural, data element, and relationship errors before submission and flag likely misclassification for a supervisor. The acceptance test: run it against the agency's own rejected submissions and count what it would have caught.
It depends on the state and the prosecutor. Utah enacted a disclosure requirement in 2025 and California adopted retention and certification requirements the same year, and some prosecuting offices have directed agencies not to submit AI-drafted narratives at all. Get the prosecutor's position in writing first, and require draft retention and a draft-to-final diff either way.
Name the decision types and the audience for each explanation, require a decision-replay demonstration during acceptance, require the log schema as agency-owned property with bulk export, and attach a bias measurement plan with thresholds and a named authority to suspend. Those items turn a vague adjective into testable requirements.
