The question your inventory cannot answer
A customer's security team sends a questionnaire. One line reads: list every third-party model that processed our data in the last twelve months, with dates. Somebody opens the AI inventory. Thirty-one rows, each with an owner, a purpose, a tier, and a model family, all accurate the day they were written. It does not say that one service switched providers in March when the old endpoint was deprecated, that a fallback path routes four percent of traffic to a second vendor when the primary times out, or that an engineer swapped in a larger variant during a June quality push and never changed it back. The inventory is not wrong. It answers a different question than the one being asked.
The inventory records intent: a governance artifact, filled in by a person, updated when somebody remembers. The questionnaire asks about execution. The two diverge the moment the first system ships and the gap widens every quarter, because the inventory is maintained at human cadence and code changes at deploy cadence. By month eighteen it describes a system that no longer exists.
The fix is not a better inventory, because discipline does not survive a release schedule. It is a second record no human writes: an append-only log with one row per model call, produced by the code that makes the call, at the moment it makes it. We call it the dispatch ledger. Every claim your company makes about its AI use should come from that table by query, not from anyone's memory.
Questions an AI inventory answers vs. questions a ledger answers
Our ranking from inventory-and-audit work, not a survey. The shape is the point: inventories are strong on intent and near-blind on execution.
Four ways the record and the reality separate
The divergence is not one failure. It is four, and they have different fixes, which is why a single policy statement never closes the gap.
Silent provider substitution. Providers deprecate endpoints on their own schedule, and SDKs frequently resolve an alias rather than a version string, so a library quietly answers with a newer model. The call succeeds, the output looks fine, nothing in the inventory changes. The first sign is usually a bill or a quality complaint.
Fallback paths. Almost every production system calling a hosted model has retry logic, and many have a second provider behind it for availability. That path is real data processing under a real contract, and it is almost never in the inventory, because an engineer added it to solve an uptime problem, not to answer a governance form.
Configuration drift below the model name. Temperature, system prompt, retrieval index, tool definitions, truncation rules. Every one changes what the system outputs and none appear in an inventory row that names a model family. A prompt edit is a model change, and teams routinely fail to treat it as one.
Calls from places nobody thought to inventory. A nightly report generator. A test harness pointed at production credentials. A build step that summarizes commits. An analytics notebook. Real calls with real data, invisible to a governance process that assumes AI arrives through product features.
One chokepoint, or the ledger is fiction
The pattern has one structural precondition: exactly one code path in the company may reach a model provider, and it writes the ledger row. If application code can construct its own client, the ledger records a subset of reality and reports a number that is confidently wrong. A partial ledger is more dangerous than none, because people trust it.
The chokepoint is a small internal library or a gateway service. The library is cheaper; the gateway is enforceable at the network layer and works across languages, which matters when a Python data team and a Java platform team both call models. Most companies past a few hundred engineers end up with the gateway. Under that, a shared client with a network rule behind it is enough.
Enforcement has to be mechanical. A policy that says "always use the shared client" produces a ledger that is missing exactly the calls you most needed to see. Three mechanisms hold in practice. Egress control: provider domains are reachable only from the gateway's network path, so a direct call fails rather than silently succeeding. Credential scoping: provider keys exist only in the gateway's secret store, and application code holds a gateway token instead. A build check: a lint rule that fails the build when application code imports a provider SDK. Credential scoping does most of the work, because a call with no key goes nowhere.
Build the guard, then prove it fires. Write a test that attempts a direct provider call from an ordinary service and assert it fails. A control nobody has tried to break is a control nobody knows works, and this class of guard fails in the shape of a pass: it keeps returning green while a proxy exemption added during a debugging session quietly excuses the one service that mattered.
The schema, on one screen
We have built this several times and it collapses to one wide append-only table and two small ones. The columns that carry weight, none that do not.
dispatch -- APPEND ONLY, one row per model call
id, occurred_at
caller_service, caller_env, caller_commit
actor_type, actor_id -- user | service | job (id hashed if a person)
purpose_code -- FK to the inventory row
provider, model_id_requested
model_id_returned -- what the provider says it served
params_hash, prompt_template_id, prompt_template_version
retrieval_index_id, tool_set_hash
data_classes -- public | internal | confidential | pii | phi | cui
tenant_id, region
tokens_in, tokens_out, latency_ms, cost_micros
outcome -- ok | error | filtered | timeout
fallback_of -- id of the attempt this one replaced
trace_id
prompt_template -- APPEND ONLY
id, version, sha256, body_uri, created_at, created_by
purpose -- THE INVENTORY, joined not duplicated
purpose_code, system_name, owner, tier, approved_providers[]
approved_data_classes[], review_dueThree columns do disproportionate work. model_id_returned is the one most implementations omit and the one that catches silent substitution, because it records what the provider says it served rather than what you asked for. Compare the two nightly and every deprecation-driven swap becomes a count. fallback_of makes retry chains legible, so a question about second-vendor exposure becomes a query instead of a code review. purpose_code joins to the inventory and turns two disconnected artifacts into one system: the ledger records execution, the inventory records approval, and the interesting reports are the disagreements.
What you do not log, and why that is not a compromise
The ledger does not store prompt or completion text by default. That decision is what makes the pattern deployable in a regulated company, and it costs almost nothing on the governance questions.
Content in a governance log creates a second copy of your most sensitive data in a system built for retention rather than protection, and it inherits every obligation attached to the original. If the input held protected health information, the ledger is now a HIPAA system. If it held personal data from EU subjects, the ledger is in scope for erasure requests, and honoring one against an append-only table is a genuinely hard engineering problem. If it held controlled unclassified information, the ledger sits inside the NIST SP 800-171 boundary, with everything that implies for access control and audit.
Store the template id and a hash of the resolved parameters instead. Between them you can reconstruct what class of call was made, prove which prompt version was in force, and detect when either changed, without holding the payload. Where content genuinely is needed, such as an incident review or an evaluation set, put it in a separate store with its own retention clock, its own access control, and a foreign key back to the dispatch row. The ledger stays queryable because it holds no content; the content store stays narrow because almost nobody needs it.
One caution on identifiers. actor_id for a human should be a salted hash with the salt held separately, so routine queries cannot reidentify a user and an authorized investigation still can. Same for tenant_id where contracts constrain who may see customer identity in operational data.
Where the ledger pays for itself · our ordering by value returned per week of build
Editorial ordering from build and audit work. Illustrative, not a measured statistic.
The five reports that justify the build
A table nobody queries is storage cost. Five standing reports turn it into a control, each answering a question somebody outside engineering already asks.
Requested versus returned. Rows where model_id_requested and model_id_returned disagree, grouped by service, daily. This finds silent substitution the week it starts rather than the quarter it reaches a bill.
Off-inventory dispatch. Rows whose provider is not in the joined purpose row's approved_providers, or whose data_classes exceed approved_data_classes. This closes the gap between the two artifacts, and its first run at a company of any size returns something surprising.
Fallback exposure. Volume through each secondary provider by month, with data classes. This answers the vendor question on nearly every enterprise security review, and a number rather than an assurance changes that conversation.
Prompt version in force. Distinct prompt_template_version per purpose over time. A prompt change is a behavior change, and this is the only place most companies could see one after the fact.
Unreviewed volume. Dispatch volume grouped by purpose rows whose review_due has passed. It turns a governance calendar into a weighted queue, so the reviews that happen first sit behind the most traffic rather than the top of an alphabetical list.
How this reads against the frameworks
None of the major frameworks name a dispatch ledger. All ask for something hard to produce without one, which is why the pattern keeps arriving independently at companies that have been through a serious audit.
The NIST AI Risk Management Framework's Map function presumes you can state each system's context, and Measure presumes you can characterize behavior over time. Both get much harder when the actual configuration is knowable only by reading code. ISO/IEC 42001, published in December 2023, requires documented operational control and records of AI system operation. An auditor asking for evidence that a control operated across the period, not on the day of the sample, is asking for a ledger whether or not they use the word.
In financial services, the Federal Reserve and OCC's SR 11-7 guidance on model risk management has required a comprehensive model inventory since 2011, and examiners test it by sampling production activity and checking whether the inventory accounts for it. That test is the off-inventory report, run by somebody else. FedRAMP and the NIST SP 800-53 AU control family take the same shape: record the event, protect the record, review it, reconstruct who did what when. A dispatch row is an AU-3 content-of-audit-record answer for a call whose consequence lives in the model rather than in the application code.
The security frameworks add their own reason. The OWASP Top 10 for LLM Applications puts prompt injection and supply-chain risk near the top, and MITRE ATLAS catalogs adversarial techniques against deployed models. Detecting either after the fact requires knowing what ran. A company that cannot reconstruct its own model calls cannot investigate an incident involving them, which turns a contained problem into a disclosure with no defined scope.
What it costs to build
For a company with a handful of services calling hosted models, the honest range is four to eight weeks of one senior engineer, and the variance sits almost entirely in the chokepoint rather than the table.
The gateway, with the ledger write in it, is one to two weeks. Schema and storage are days: an append-only table with a time-based partition, which any columnar warehouse handles without tuning. Retrofitting call sites is where the weeks go, scaling with how many teams wrote their own client. Ten services behind a common HTTP wrapper is a week. Ten services with ten integration styles, one in a language nobody maintains, is a month. Enforcement, meaning egress rules and credential rotation into the gateway's store, is one to two weeks and touches teams outside engineering, which usually makes it the long pole in calendar time even though the effort is small.
Running cost is minor. A dispatch row is a few hundred bytes, so ten million calls a month is a few gigabytes a year. At volumes where that matters, sample the low-tier purposes and keep every row for the tiers where a person is affected.
The comparison is not against zero. It is against a multi-week archaeology exercise the next time a customer, an auditor, or an incident asks, repeated every time it is asked, producing an answer nobody can fully stand behind.
Sequencing it so it survives
The failure mode of this build is doing the schema first. A perfect table with three of eleven services reporting into it is the partial ledger, and it will be quoted as though it were complete. Order the work so completeness comes before richness.
Establish the chokepoint first and route one high-volume service through it with a minimal row: timestamp, caller, provider, model requested, model returned, outcome. Prove it by trying to bypass it. Then route the remaining services, refusing new columns until the last one is in, because a narrow complete ledger beats a wide partial one. Add the requested-versus-returned comparison and the off-inventory report next; those carry most of the early value. Add prompt template versioning once templates live in a registry rather than string literals. Add cost attribution last, when finance asks, which they will within a quarter.
One organizational rule holds it together. The gateway team owns the ledger, the governance team owns the inventory, and the disagreement report goes to both. When one team owns both artifacts, the disagreement quietly stops being reported.
Bottom line
An AI inventory is necessary and not sufficient. It records what people meant to run, updated at the speed people update documents, while code changes at the speed of deploys. Build one narrow chokepoint every model call passes through, have it write an append-only row with what was asked for and what actually answered, join that to the inventory, and run five reports off the join. Audit answers become queries, silent substitution surfaces in days, incident scope takes minutes, cost attribution arrives as a side effect. The build is weeks, not quarters, and it replaces an investigation you would otherwise repeat every time somebody asks.
Frequently asked questions
Application tracing records that a call happened and how long it took. A dispatch ledger records the governance-relevant facts: which model version answered, which prompt version was in force, what class of data went in, whether it was a fallback. Traces are sampled and short-lived by design; the ledger is complete and retained. Many teams write the row from the same instrumentation point, but the retention and the schema are different products.
No, and by default you should not. Store a prompt template id and version plus a hash of resolved parameters. That proves which configuration was in force and detects a change. If content capture is needed for incident review or evaluation, put it in a separate store with its own retention and access control, keyed back to the dispatch row.
Then the ledger is incomplete and will be quoted as though it were complete, which is worse than not having it. Enforcement is mechanical: provider credentials live only in the gateway's secret store, provider domains are reachable only from the gateway's network path, and a build check fails on a direct SDK import. Write a test that attempts a bypass and assert it fails.
No single artifact satisfies a management-system standard. The ledger supplies the operational records those standards assume exist: evidence that a control operated across a period rather than on a sampled date, and a production activity record an examiner can test the inventory against. The policy, tiering, review cadence, and accountable owners are still yours to build.
Long enough to cover the longest question you will be asked. Security questionnaires commonly look back twelve months and audit cycles run annually. Two years covers most cases, and because the rows hold no content, retention is a storage question rather than a privacy question.