Your data warehouse is a compliance artifact now

The moment the warehouse changes jobs

There is usually a specific week when it happens. A federal prime sends a subcontract with DFARS 252.204-7012 attached and asks where the technical data lands. A bank's model risk group asks, under SR 11-7, for the derivation of an input feature your ratings product ships. A hospital customer's privacy office asks who queried a table last March. A large buyer sends a 240-question security questionnaire and about forty of the questions are about the analytics platform. Nothing in the warehouse broke. Its job changed.

Analytics platforms are optimized for a set of properties that have nothing to do with what a reviewer wants. Fast reads, wide permissions so analysts are not blocked, generous retention because storage is cheap, and a culture of copying a table to answer a question quickly. Every one of those is a virtue when the goal is answering business questions and a finding when the goal is demonstrating control. The gap is not usually a security gap. It is an evidence gap.

The distinction matters because it sizes the project. Poor access control is a rebuild. Reasonable access control with no way to prove it for a past period is an instrumentation project measured in weeks. Most companies we look at are in the second category and assume they are in the first, so they overbuy a governance platform or delay until a deal is at risk.

Four regimes, one platform, different questions

The frameworks that arrive at a commercial data platform are not interchangeable, and treating them as one undifferentiated compliance blob is how budgets get wasted. Each asks a distinct question and each has a different cheapest answer.

NIST SP 800-171, arriving through DFARS flow-down and enforced through CMMC assessment, asks a boundary question: where does controlled unclassified information live, and is that place separated and monitored. Of its 110 requirements in fourteen families, the ones that bind a warehouse are access control, audit and accountability, media protection, and system and communications protection. FedRAMP and NIST SP 800-53 ask the same question with a far larger control set, and apply when you sell a service to an agency rather than handle their data as a subcontractor.

SR 11-7, the Federal Reserve and OCC model risk guidance, asks a derivation question: can you show the data a model consumed, the transformations applied, and that the inputs were checked for quality and fitness for purpose. It applies through your customer even when it does not apply to you, which is why a fifty-person data vendor ends up answering it. ISO 42001 and the NIST AI Risk Management Framework ask a governance question: is there a management system around the AI work, with defined roles, documented decisions and evidence they were followed. None of the four asks whether your dashboards are fast.

Classification is the whole project in disguise

Nobody wants to start here because it feels like paperwork. It is the load-bearing step, because every downstream control depends on knowing which tables are in scope. A boundary drawn around the whole warehouse means every control applies to every table, which is expensive and usually fails. A boundary drawn around forty tables is defensible and affordable, provided you can show those forty are the right forty.

The mechanism that works is column-level tagging enforced at write time rather than a spreadsheet inventory maintained by hand. Every modern platform has the primitive: Snowflake object tags with masking policies attached, BigQuery policy tags backed by Data Catalog taxonomies, Databricks Unity Catalog tags with column masks, and plain column comments plus a check in the transformation layer if you are on Postgres. The rule to enforce in CI is that a new column in a governed schema fails the build until it carries a classification tag. That single check is the difference between an inventory that is accurate on the day it is written and one that stays accurate.

Then find the copies, because copies are where the boundary leaks. A governed table gets copied into a personal scratch schema, extracted to a CSV for a partner, cached in a BI tool's own storage, or pulled into a notebook that writes to object storage outside the governed prefix. We have never run a boundary exercise that found none of these. Query history is the tool: read six months of it, find every statement that selected from a governed table and wrote somewhere else, and you have the real map.

Access control that survives a question about last March

Most warehouses fail here in a specific and fixable way. Permissions were granted to individuals, over years, by people who have since left, in response to urgent requests that were correct at the time. The current state is knowable with a query. The state on an arbitrary past date is not, and that is what the reviewer asks for.

Three changes fix it. Grant to roles, never to people, so the membership list is the only thing that changes and the grant graph stays small enough to reason about. Provision role membership from your identity provider through SCIM, so a departure in the HR system removes warehouse access without anyone remembering to do it. And snapshot the effective permission set on a schedule, daily is plenty, into a table you keep. That last one costs an afternoon and answers the question that otherwise costs a week, because grant tables in every warehouse show current state only.

Then add the review cycle, which 800-171 and every customer questionnaire ask about by name. Quarterly, generate who can read what, route it to each governed schema's owner, and record the approval with a date and a name. The failure mode is a rubber stamp, because nobody reads four hundred rows of grants. Present the delta instead: what changed since last quarter, and which accounts have not run a query in ninety days. A reviewer who sees fifteen rows actually reads them.

Audit logs are not the same as a log of who saw the data

Every warehouse writes query history, and teams assume that covers the audit requirement. It usually does not, for two reasons that are both about retention and one that is about content.

Retention first. Snowflake's ACCOUNT_USAGE query history holds a year but the faster INFORMATION_SCHEMA views hold seven days. BigQuery's Cloud Audit Logs default to a retention period measured in months in the _Default bucket unless you configure otherwise. Databricks system tables have their own retention. Meanwhile the contract you signed or the framework you are under likely wants three years, and 800-171's audit family expects you to have defined the period deliberately rather than inherited it. The fix is a scheduled export to object storage with lifecycle rules and versioning enabled, which is a day of work and is the single highest-value thing on this list relative to its cost.

Content second. Query text proves a statement ran. It does not say which rows came back, and a privacy reviewer wants to know whose records were exposed. For high-sensitivity tables the practical answer is a view layer requiring a purpose parameter logged alongside the query, so the record shows not just that an analyst read the patient table but why. That is only heavy if you apply it everywhere. Apply it to the four tables that matter.

Reproducing a number from eighteen months ago

This is the question that separates platforms that pass review from platforms that do not, and it comes in the same form every time. Here is a figure we published in Q1 of last year. Reproduce it. Not approximately. Exactly.

Three things have to be true simultaneously. The input data as it stood at that moment must be recoverable, which means time travel with a retention window long enough to matter, or immutable snapshots, or a table format like Iceberg or Delta whose history you have not vacuumed away. The transformation code must be identifiable by version, which means the run recorded a git commit rather than merely a timestamp. And the run parameters must be recorded, because a job that filtered on now() instead of an explicit interval cannot be reproduced even with perfect code and perfect data.

Note the cost trap in the first one. Snowflake time travel is one day on standard edition and configurable to ninety on enterprise, and the storage bill for ninety days of a large mutable table surprises people. Iceberg and Delta snapshot retention has the same shape. The right move is per-table: long retention on the twenty tables that feed reported figures, short retention on everything else. A blanket ninety-day setting across a petabyte is how a governance project gets cancelled by finance. We have covered the mechanics of proving a derivation in more depth in data lineage that satisfies an auditor.

What is asked for	What has to exist	Typical effort	What it costs to skip
Which tables are in scope	Column-level tags enforced at write time, plus a copy map from query history	3 to 6 weeks	Every control applies to everything; scope inflates three to five times
Who could read this in March	Role-based grants, SCIM provisioning, daily permission snapshots	4 to 8 weeks	A week of manual reconstruction per question, and it is still an estimate
Who did read it	Audit log export to immutable storage, retention set deliberately, queryable	1 to 2 weeks	The window closes at 7 to 365 days and the evidence is simply gone
Reproduce this figure	Snapshot retention on reported tables, code version and parameters per run	4 to 10 weeks	The answer becomes "close enough," which is not an answer
Prove deletion happened	Executed retention jobs with a per-run record of what was removed	2 to 4 weeks	A written policy with no execution evidence, which reads worse than none

The retention policy nobody runs

Almost every company has a written retention schedule. Far fewer have a job that enforces it, and the gap is worse than having no policy at all, because the policy is a documented commitment and the absence of deletion evidence is a documented failure to meet it. A reviewer who finds a policy saying ninety days and a table holding four years has found something specific.

Make deletion a pipeline like any other. It runs on a schedule, it is parameterized, it logs what it removed and how many rows, and its output is a record you can hand over. Then handle the three places deletion quietly does not reach: soft deletes that only set a flag, time travel and snapshot history that still holds the deleted rows for the retention window, and backups. The backup one is genuinely hard and the honest engineering answer is usually crypto-shredding, where a per-tenant or per-subject key is destroyed so the ciphertext in backups becomes unrecoverable. State the approach explicitly rather than letting a reviewer discover the gap.

What this costs and how long it takes

For a commercial platform of ordinary size, a few hundred tables and thirty to a hundred people with some form of access, the sequence takes four to seven months of concentrated work and lands somewhere between $180,000 and $450,000 in engineering time. The spread is driven almost entirely by two things: how much of the data was copied out of the platform into places nobody tracks, and whether the transformation layer is version-controlled or is a collection of stored procedures and scheduled queries edited in a console.

The second is the expensive variable. A team on dbt or a similar framework with everything in git already holds half the reproducibility answer and did not build it for compliance. A team whose logic lives in a console has to get it into version control first, a real project that pays for itself independently of any framework. Tooling is the smaller line: governance platforms quote $60,000 to $200,000 a year and earn it once classification is done, close to nothing before, because they inventory whatever you point them at without deciding what matters. Buy the platform after you know your boundary, not to discover it.

The sequence that does not waste money

Order matters here more than in most engineering work, because each step makes the next one cheaper. Classify first, because scope determines everything downstream. Fix access second, because it is the most commonly asked question and the permission snapshot starts accumulating history the day you turn it on, and history cannot be created retroactively. Export audit logs third, for the same reason and at a fraction of the cost. Then lineage and reproducibility, which is the largest single piece. Then retention execution. Then the documents, written from what the system actually does.

The common failure is running that list backwards. A consultant writes the policies, the policies describe controls the platform does not implement, and the organization now has documentation that a reviewer can use as a checklist of things to find missing. Write the document last, describing the system as built. That is slower to start and much faster to finish.

What good looks like from the other side of the table

The companies that pass these reviews cleanly are not the ones with the most controls. They are the ones that can answer a question in an hour instead of a week, and that state their gaps before anyone finds them. A firm that says column-level lineage covers 84 percent of the paths feeding reported figures, the remaining 16 percent is these four named systems, and here is the handling for each, is far more credible than one presenting a clean picture with silent holes in it. The first invites a specific conversation. The second collapses the moment somebody finds a hole, and takes the rest of the work down with it.

None of this means the warehouse was built wrong. It was built correctly for the job it had, and the job changed. Done in the right order these are also the changes that make the platform easier to operate: fewer standing permissions, deletions that run, numbers you can rebuild. The compliance requirement is only the thing that finally funds them.

Bottom line

Your analytics platform is now evidence, and evidence has properties that dashboards do not. Know which tables are in scope and enforce that at write time. Grant to roles and snapshot the permission set daily, because past state cannot be reconstructed later. Export audit logs somewhere immutable before the default retention window closes. Make reported figures reproducible by pinning data, code version and parameters together. Run deletion as a logged job. Do those five and the questionnaire, the model risk review and the flow-down clause all get answered out of the same instrumentation, which is the only reason the work is affordable at all.

Frequently asked questions