Missingness is a signal, and it points the wrong way
If you build anything on private company data — a credit model, a lead score, a supplier risk view, a market map — you have already met the central problem, probably without naming it. Rows are missing, and fields inside present rows are missing, and neither kind of gap is random. Disclosure obligations, filing triggers and third-party observability all scale with size, formality and funding. So the companies with complete records are systematically different from the companies without them, in exactly the dimension most models care about.
The consequence is quiet and severe. Train on the rows that have revenue and you have trained on the larger, better-documented, more formal subpopulation. Validate on the same rows and the model looks fine. Deploy it against the full universe and performance falls, unevenly, in the segment nobody measured. Nothing in the pipeline breaks. The dashboards stay green.
This is a solvable problem, but not by buying a more complete file. It is solved by measuring what you have, against the universe you actually serve, and being explicit about what the model is allowed to say when the inputs are absent.
You are probably here because
- A model that tested well is underperforming on small accounts and nobody can say why
- You are comparing two vendors and both quote a coverage number you cannot reconcile
- Someone asked what fraction of your universe you can actually score, and the honest answer was a shrug
- You are about to renew a firmographics feed and want a defensible reason either way
The three-axes section names the thing you are measuring. The truth-panel section is the method, and it is a week of work rather than a project.
Where the data comes from decides where the holes are
Private company records are assembled from a handful of source types, and each has a characteristic blind spot. Knowing which sources sit under a file tells you where its gaps will be before you measure anything.
Registries. Company registration records from state and national authorities. Excellent for existence, legal name, formation date and registered agent. Weak on anything operational, and the disclosure requirements differ by jurisdiction, which means a multi-country file has structurally different depth by country.
Event-triggered filings. Secured lending filings, litigation dockets, permits, trademarks, import and export records. These only exist when something happened, so a company that has never borrowed, never been sued and never imported is invisible in all of them — which is not the same as being small.
Observed digital activity. Websites, job postings, app listings, review sites, technology detection. Fast, current, and biased toward companies whose business model has a public digital surface. A distributor with three customers and no website exists and will never appear here.
Self-reported profiles. Directory entries, professional network pages, submissions to the vendor. Present when someone had a marketing reason to fill in a form, absent otherwise, and rarely revised downward.
Vendor estimates. Modeled revenue and headcount, usually from a regression on whatever is observable. Frequently reasonable and frequently indistinguishable from reported values in the delivered file, which is the same flattening problem that afflicts every derived dataset.
Completeness has three axes and they are usually conflated
When someone says a file is 80 percent complete, ask which of these three they mean, because the three have different fixes and different consequences.
| Axis | The question it answers | How it fails silently |
|---|---|---|
| Entity coverage | Is the company in the file at all, exactly once? | A company appears three times under variant names, so coverage looks high and joins look low |
| Attribute coverage | Does the row carry the fields you need? | The row exists, revenue is null, and downstream code treats null as zero |
| Temporal coverage | How recently was this row actually touched by evidence? | A record is 100 percent populated and four years stale, which is worse than blank |
| Origin | Is the value reported, observed or modeled? | An estimate is consumed as a fact, and its error is attributed to the company |
| Universe fit | Complete relative to which population? | The vendor's denominator includes entities you would never score, inflating every ratio |
The last row is the one that quietly ruins vendor comparisons. A headline entity count is a marketing number and it is not wrong, it is just measured against a universe that includes dormant registrations, dissolved shells, sole proprietorships and duplicate filings. Your universe is different, smaller and knowable. Every completeness figure in your evaluation should be computed against it.
Build a truth panel. It is a week, not a project
There is no substitute for hand verification, and the good news is that you need much less of it than people expect. The method is a stratified sample with human adjudication, and it produces numbers you can defend in a renewal, a model risk review or a board deck.
Define the universe first, in writing. Not “US businesses” but something enumerable: active entities in these NAICS codes, in these states, above this size threshold, that we would consider a prospect or a counterparty. If you cannot write it down, that is the first finding, and it usually explains the disagreement between two teams about whether a file is good.
Stratify by the thing that drives missingness. Size band almost always. Then jurisdiction, industry, and age of the entity. Uniform random sampling will fill your panel with the segment you already cover well and tell you nothing about the segment that is failing.
Sample a few hundred, not a few thousand. Somewhere in the low hundreds per stratum gets you a confidence interval tight enough to make a decision, and it is small enough that a person can genuinely verify each one. Report the interval alongside the point estimate, always. A coverage figure without an interval invites a false comparison between two vendors that are statistically indistinguishable.
Verify by hand, against a source that is not the vendor. The registry, the company's own site, a filing, a phone call. Record what you checked and when. This artifact outlives the evaluation — it becomes your regression panel for every future vendor and every future model.
Report per stratum, never in aggregate only. The aggregate is dominated by whichever stratum has the most rows, and it hides the failure you were looking for. The useful output is a small table: stratum, entity coverage, attribute coverage for each field that matters, staleness distribution, and the interval on each.
Where a hand-verified hour buys the most — our read
Our judgment of where manual verification changes a decision. The bottom row is where most evaluations spend their time because it is the easiest to check.
Stale is a kind of incomplete, and it is invisible
A populated field with no date behind it is an assertion about the past presented as an assertion about now. Companies close, move, get acquired, change name, and shed half their headcount, and a file only learns about it when something triggers a refresh.
So ask a question most evaluations skip: what causes a record to be updated, and how often does that happen for a company like the ones in my universe? A file refreshed by registry sweeps updates on registry events. A file refreshed by web crawling updates when the website changes. Neither notices a quiet business that laid off two-thirds of its staff.
Then measure it. Take your panel, look at the last-observed date on each record, and plot the distribution by stratum. The mean is not the number you want — the tail is. If a tenth of your smallest-segment records have not been evidenced in two years, that is a real operational fact and it will not appear in any vendor material.
If the file has no per-record date at all, treat every record as of unknown age, and say so in whatever you publish. A vendor that cannot tell you when a row was last confirmed has told you something important about the row.
Imputation: legitimate, dangerous, and mostly a labeling problem
Filling a gap is not automatically wrong. Doing it silently is. A few rules keep it honest.
Never impute the target. If you are predicting revenue, imputed revenue in the training set teaches the model to reproduce the imputer. It will look like accuracy.
Keep a missingness indicator. A binary flag for “this was absent” is frequently one of the more predictive features in the model, because absence carries information about formality and size. It also lets you decompose performance later.
Report performance twice. Once on the complete-case subset, once on the full universe with imputation. Two numbers, always presented together. If they diverge sharply, you have learned that your model is really a model of well-documented companies.
Set an abstention rule. Decide in advance how many key inputs may be imputed before the system declines to produce a score. Returning “insufficient data” is a legitimate and often preferable output, and it is far easier to defend to a customer or a regulator than a confident number derived from three defaults and a sector median.
What a second vendor actually buys
The instinct when coverage is thin is to buy another file. Sometimes that is right. The way to find out is an overlap analysis on your own universe, before the contract, and the answer is frequently smaller than expected because vendors often draw on the same underlying registries and panels.
Three numbers decide it. Marginal entity lift: how many entities in your universe does vendor B have that vendor A does not, in the strata you care about. Marginal attribute lift: for entities both have, how often does B fill a field A leaves blank. Disagreement rate and who wins: where both have a value and they differ, adjudicate a sample by hand. That last one is the only way to convert agreement into evidence, because two vendors sharing a source will agree while both being wrong.
Run all three on a trial file against your real universe, not against a sample the vendor prepared. Ask for the trial file with your entity list, so the comparison is on your ground. A vendor confident in their coverage will do this, and one who will not has answered a different question for you.
Where you do not need help
If your universe is a few thousand named accounts, do not buy a firmographics file to fill it. A person with a browser and a week will produce a better record than any purchased file, because the ground truth for a named account is a public website, a registry lookup and sometimes a phone call. The economics of purchased data favor scale, and at small scale the economics favor a human.
Likewise, if you already have a truth panel and your coverage is adequate in the strata that drive your decisions, an incremental vendor is unlikely to pay for itself. The number that decides it is marginal lift in the segments that matter, and that number is often disappointing in a way nobody wants to be the one to find out.
The mistakes we are called in to fix
- Nulls coerced to zero at load time, so absent revenue becomes a company with no revenue
- Models trained on complete cases and deployed on everything, with no complete-case comparison ever reported
- Coverage quoted from the vendor's denominator rather than measured against the buyer's universe
- Duplicate entities inflating coverage and deflating join rates at the same time
- No last-observed date on any record, so staleness cannot be measured even in principle
- Estimated and reported values in one column, with the estimate's error charged to the company
- Uniform random sampling for the evaluation panel, which oversamples the segment already working
- Two vendors agreeing treated as validation when both are reading the same registry
Before you sign or renew
- Your universe is written down and enumerable, and every ratio is computed against it
- A stratified truth panel exists, hand-verified, with per-stratum results and intervals
- Entity, attribute and temporal coverage are reported separately
- Every record carries a last-observed date, and you have seen its distribution, not its mean
- Reported and modeled values are distinguishable in the delivered file
- Duplicates are measured on your key, before coverage is claimed
- Missingness indicators survive into the model, and the target is never imputed
- An abstention rule exists and returns “insufficient data” rather than a confident default
- Marginal lift over what you already have is measured before a second file is bought
Bottom line
Private company data is incomplete for structural reasons that will not be fixed by a better vendor, and the gaps correlate with size and formality, which is usually the axis your decisions run along. The work is to measure your file against your own universe, keep the distinction between absent, observed and modeled all the way through to the output, and let the system say “I do not know” when it does not. A team that can state its coverage per segment, with an interval, is in a much stronger position than one holding a larger file and a vendor's percentage.
Frequently asked questions
Smaller than most teams assume. A few hundred records per stratum is usually enough to separate a good file from a poor one with a confidence interval you can put in front of a reviewer. What matters far more than size is stratification — the sample must be built around whatever drives missingness in your universe, which is nearly always company size, then jurisdiction and industry.
It is true and it is not useful on its own, because the denominator includes entities you would never score: dormant registrations, dissolved shells, sole proprietorships and duplicates. Ask instead for coverage of your entity list. Send them the list, take the match file back, and compute the ratio yourself against your own universe definition.
For features, often yes, provided you keep a missingness indicator and report performance separately on the complete-case subset. For the target variable, no — imputed targets teach a model to reproduce the imputer, and the resulting accuracy is an artifact. Whatever you do, decide in advance how many imputed inputs are too many, and let the system abstain past that point.
No, and the reason is worth internalizing: commercial files frequently share upstream sources, so agreement can simply mean both read the same registry entry. Agreement is evidence only when the sources are genuinely independent, and you usually cannot tell from the outside. Adjudicate a sample of disagreements by hand instead; that tells you which file is right and by how much.
Treat it as of unknown age and say so wherever the value is used. Ask the vendor what event triggers a refresh for a company in your segment, because that answer predicts staleness better than any published update cadence. If neither a per-record date nor a refresh trigger can be described, price the file as though the tail of it is several years old, because part of it will be.