How we handle data we should not see

The first question is not when the data arrives

Programs stall in the same place over and over. The model was never the hard part. The data sits inside a system of record someone else owns, governed by a clause nobody on the technical side has read, and the schedule assumes engineering begins the week after kickoff. Two months later the kickoff deck is still accurate and no software exists. We treat that as the normal starting condition, because on federal and regulated commercial work it is. So the first question we ask a new partner is not when the data will land. It is what regime governs the data, and who personally owns the approval.

Those two answers do more scheduling work than any Gantt chart. The regime tells us which techniques are legal and which are theater. The named owner tells us how long the approval takes and what will convince that person to say yes. From there, every task in the program sorts into one of three buckets: work that starts immediately with no data at all, work that runs against a synthetic or surrogate stand-in, and work that genuinely cannot begin until real records are reachable. On most programs the third bucket is far smaller than anyone assumed at kickoff.

Four different regimes, four different answers

"Data we should not see" is a single phrase covering at least four legal regimes that behave nothing alike. Confusing them is where good teams get into trouble, because a control that satisfies one is irrelevant to another.

Controlled Unclassified Information. Established by Executive Order 13556 and implemented at 32 CFR Part 2002, with roughly 125 categories in the NARA CUI Registry across groupings such as Privacy, Procurement and Acquisition, Critical Infrastructure and Export Control. CUI is a marking regime, not a classification regime. On DoD contracts, DFARS 252.204-7012 pulls in the NIST SP 800-171 control set, requires reporting of a cyber incident to DIBNet within 72 hours, flows down to subcontractors, and requires any cloud service holding the data to meet the FedRAMP Moderate baseline or its equivalent.

Protected health and personal information. The HIPAA Privacy and Security Rules for covered entities and business associates; the Privacy Act of 1974 where a federal system of records is involved, with FAR 52.224-1, 52.224-2 and 52.224-3 appearing in the contract. This is the regime where de-identification is a real, defined technical path with two named methods.

Export-controlled technical data. ITAR under 22 CFR Parts 120 through 130, or the EAR at 15 CFR Parts 730 through 774. The distinctive feature is the deemed-export rule: a foreign person reading a controlled drawing on a screen is a release, whether or not a file ever moved. Precision Federal is certified under the Joint Certification Program with an approved DD Form 2345, CAGE 1AYQ0, which is what lets us receive export-controlled technical documents from DoD and DND repositories directly.

Classified national security information. Governed by the contract's DD Form 254 and NISPOM. This is not a de-identification problem in any form. The work happens in an accredited facility on an accredited system or it does not happen, and the honest engineering answer is to structure the unclassified portion so it carries real weight.

Several regimes sit alongside these and follow the same design logic: CJIS Security Policy for criminal justice information, FERPA for student records, IRS Publication 1075 for federal tax information, and Critical Energy/Electric Infrastructure Information at 18 CFR 388.113 for grid data. Proprietary third-party data under a commercial NDA often has the tightest practical constraints of all, because the remedy is a lawsuit rather than a corrective action plan.

Minimum necessary is an engineering decision

HIPAA writes minimum necessary as a rule at 45 CFR 164.502(b) and 164.514(d), and CUI doctrine says the same thing in different words: lawful government purpose, no more. In practice it is not a legal decision at all. It is made by whoever drafts the data request, which means it is made by an engineer.

A request that reads "we need access to the claims database" will be reviewed for months and may be denied, because the reviewer cannot bound the risk. A request that reads "eleven named columns, at encounter grain, for fiscal years 2023 through 2025, with dates shifted by a stable per-patient offset, delivered to a government-owned bucket, deleted at period of performance end" often clears in two weeks. Same program, same model, different paperwork. The second version can be approved narrowly; the first can only be approved broadly, and nobody wants to sign that.

We write the field-level request ourselves and justify each field in one sentence against a modeling purpose. Fields that cannot survive that sentence get cut before the request is sent. We also sequence the ask so the highest-sensitivity elements come last: free-text clinical notes, precise coordinates, raw imagery with intact EXIF, direct identifiers, and anything joining to a person by name. Very often the model reaches acceptable performance on the structured core, and the sensitive request never has to be made.

De-identification the way a reviewer will actually check it

HIPAA gives two methods and they lead to very different systems. Safe Harbor, at 45 CFR 164.514(b)(2), removes eighteen classes of identifier: names, all geographic subdivisions smaller than a state except the first three ZIP digits where that unit holds more than 20,000 people, all date elements finer than year, ages over 89, device identifiers, biometrics, full-face images, and the rest. It is mechanical and easy to defend. It also destroys most temporal models. A 30-day readmission model that cannot see admission dates at day resolution is not a readmission model any longer.

Expert Determination, at 45 CFR 164.514(b)(1), is the path we design toward when time or geography carries the signal. A qualified person applies accepted statistical principles, documents the methods and the analysis, and concludes that the risk of re-identification is very small. It costs more and preserves the variables the model runs on. The middle option most teams forget is the limited data set at 45 CFR 164.514(e), which keeps dates and city, state and ZIP under a data use agreement without meeting the full de-identification bar.

Re-identification risk is measurable, so we measure it rather than assert it. Latanya Sweeney's work established that five-digit ZIP, gender and date of birth uniquely identify a large majority of the US population. Narayanan and Shmatikov re-identified Netflix Prize users by matching against public IMDb reviews. We compute k-anonymity across the quasi-identifier set, check l-diversity on the sensitive attribute, and where a release is genuinely public we write down a differential privacy budget with the epsilon stated in the deliverable. NIST SP 800-188, De-Identifying Government Data Sets, is the document we hand a reviewer who asks how the choices were made, because it lets the conversation be about a published standard instead of about our judgment.

What synthetic and surrogate data can honestly do

Synthetic data has been oversold, which is unfortunate, because it does three specific jobs extremely well and those three jobs are usually the ones blocking the schedule.

• Schema-faithful synthetic. Records generated from the real DDL with plausible distributions, working referential integrity, and the null patterns, code-set drift and malformed rows the production feed actually contains. This exercises ingestion, joins, error handling, the interface and load behavior.
• Public surrogate corpora. A real open dataset with the same shape as the restricted one. CMS synthetic public use files, MIMIC-IV under its credentialed terms, NHTSA FARS crash records, EIA-861 utility filings, NOAA and USGS feeds, state open-data portals. A model developed here transfers its architecture and its evaluation suite, not its weights.
• Rare-event and adversarial injection. Deliberately generating the tail the real corpus lacks: the fraud pattern seen four times, the sensor failure mode, the out-of-distribution input. Useful for robustness testing even after real data arrives.
• Interface and acceptance rehearsal. Putting a working screen in front of the actual end users months earlier than access would allow, so the requirements argument happens before the model is built rather than after.

What synthetic data never does is validate an accuracy claim. A number computed on generated records is a fact about the generator, not about the world. Every deliverable we produce labels synthetic-derived figures as synthetic, in the caption, on the chart, in the table footnote. Federal reviewers and technical diligence teams punish a laundered synthetic number far harder than they punish a missing one, and they are right to.

The schema is the contract

Before any record moves, we ask the data owner for four artifacts that contain no sensitive content whatsoever and can usually be released in days rather than months.

• Structure: the DDL, the API contract, the file layout, or the message spec.
• A data dictionary with code sets, allowed values, units and the meaning of each flag.
• Column profiles: row counts, null rates, cardinality, min and max, distribution shape. Aggregates only, no rows.
• Ten to twenty fabricated records the owner writes or approves, including a few deliberately malformed cases.

From those four artifacts we build the ingestion layer, the validation layer, transformations, feature computation, the evaluation suite, the logging and the interface. Then we write contract tests that assert the schema itself: column presence, type, nullability, range, code-set membership, referential integrity and distribution drift against the profile. Those tests execute the first time real data lands. When access clears, day one is a validation run, not a debugging run, and the difference is typically four to six weeks of schedule.

The most common damage we repair on a stalled program is the inverse of this. A team received one sample extract early, wrote everything against it, and discovered at integration that the production feed used different code sets, a different null convention and a different grain. Building against the declared schema instead of a convenient sample avoids that entirely, and it has the side benefit that the schema questions surface while the data owner is still in the room.

What proceeds at each access tier

Access tier	What we build immediately	What genuinely waits
Nothing yet, schema only	Ingestion, validation, contract tests, transformation logic, interface, deployment infrastructure	Any performance claim
Synthetic from real schema	End-to-end pipeline runs, load and failure testing, user acceptance rehearsal, feature engineering	Accuracy, calibration, subgroup fairness
Public surrogate corpus	Model architecture selection, hyperparameter ranges, evaluation suite, baseline comparisons	Transferable weights, domain-specific thresholds
De-identified extract	Nearly the full model build, including honest held-out evaluation	Anything needing dates or geography stripped by Safe Harbor
Limited data set under a DUA	Temporal and geospatial modeling, cohort logic, drift monitoring	Person-level linkage to outside sources
Production access in the owner's enclave	Final training, integration, acceptance testing, transition to operations	Nothing, and this is also where retention rules bite hardest

Sending the code to the data

Sometimes the correct answer is that the data never moves at all. That is not a fallback. It is frequently the fastest route to approval, because the data owner is being asked to permit a computation rather than to authorize a transfer, and those are different signatures with very different review paths.

Four patterns cover nearly everything we do. First, we run inside the owner's environment: their AWS GovCloud or Azure Government subscription, their DoD Impact Level 4 or Impact Level 5 boundary under the Cloud Computing SRG, their hospital VPC, their on-premises GPU host. We deliver containers, infrastructure as code, a runbook and a trained engineer, and the records never cross our boundary. Second, federated training, where the model travels to each site and only parameter updates return, which fits multi-site health and multi-utility work well. Third, an enclave with a controlled export path, where every artifact leaving the boundary is reviewed by two named people against a written export checklist. Fourth, blind delivery, where we ship a signed artifact plus a validation script and the customer runs it against data we never touch.

For export-controlled technical data, the encryption carve-out matters more than most teams realize. Under 22 CFR 120.54(a)(5), technical data secured end to end with FIPS 140-2 or 140-3 validated cryptography, with keys not held by a foreign person and no storage in a country listed in 22 CFR 126.1, is not an export when it transits. The EAR carries a parallel provision at 15 CFR 734.18. Those two citations are why a properly built cloud architecture can hold ITAR technical data at all, and they are worth having in hand before an architecture review rather than after.

Read the clauses before drawing the architecture

The architecture is decided by the contract more than by the technology. On DoD work, DFARS 252.204-7012 sets the safeguarding and incident-reporting floor, while DFARS 252.204-7019 and 252.204-7020 govern the NIST SP 800-171 self-assessment score posted to SPRS and the government's right to conduct its own assessment. The CMMC program rule at 32 CFR Part 170 took effect in December 2024, and the acquisition-side rule that places CMMC requirements into solicitations followed, so the level required now appears in the solicitation itself and should be read there rather than assumed. NIST SP 800-171 Revision 3 published in 2024; many active contracts still point at Revision 2, which is one more reason to read the clause rather than the calendar.

On SBIR and STTR work there is a further wrinkle worth knowing. Data developed under an SBIR award carries SBIR data rights under DFARS 252.227-7018, with a protection period running twenty years from award, and government-furnished data arrives with its own separate terms. Marking correctly at delivery is what preserves those rights. Marking incorrectly, or not at all, is how firms hand over commercial value they did not intend to hand over.

The exit is part of the design

Every access agreement we sign has a defined end, and we build the end at the same time we build the beginning. Retention periods go into the infrastructure as lifecycle rules rather than into a calendar reminder. Derived artifacts get tracked, because a feature store, a model checkpoint and a log file can all carry protected content forward past the deletion of the source table. At close-out we produce a written destruction record naming what was held, where, for how long, and what was deleted, and the customer keeps a copy.

That document does more work than its length suggests. It is the artifact a compliance officer needs when the program is audited two years later, and it is the reason the next data request from the same owner moves faster than the first one did. Access is a relationship. Teams that treat it as a hurdle get slower over time. Teams that treat the owner as a stakeholder with real accountability get more access, sooner, on every following program.

Common questions on where the line sits

Frequently asked questions