The number is a chain, and the column is one link
Ask where a Scope 1 emissions figure came from and you will usually get a single answer: the vendor. Ask one layer deeper and the answer splits. Some of those figures were disclosed by the company in a sustainability report. Some were estimated by the vendor from a model. Some are an industry average scaled to the company's revenue. Some belong to a parent entity and were pushed down to a subsidiary that has never reported anything. All four arrive in a column called scope1_tco2e, as a float, with no flag.
This is not a scandal and nobody is hiding it. Vendors document their methodologies, often at length, and estimation is a legitimate response to a world where most entities do not publish the underlying data. The problem is one of representation. A dataset that collapses four kinds of claim into one field forces every downstream consumer — your model, your report, your customer — to treat a measured value and a category average as the same kind of thing. They are not. They have different error, different revision behavior and different defensibility when somebody asks.
Everything in this article follows from that one idea: the fix is not better numbers. It is keeping the provenance attached to the number you already have.
You are probably here because
- A customer asked how one of your published scores was derived and the answer took three days to assemble
- Your vendor restated two years of history and your back-tested product changed underneath you
- A large customer is passing a supplier questionnaire down to you and you have to answer it with something defensible
- You are deciding whether to buy a second ESG feed, and cannot tell whether it would add anything
The four-labels section is the map. The point-in-time section is the one that saves a backtest. If you publish figures to customers, read the provenance-record section twice.
Four labels, and they are not interchangeable
Before anything else, split the field. Every value in an ESG dataset is one of four things, and a single extra column carrying which one it is will do more for your product than a year of methodology reading.
Reported. The company published this number, in a document, for a stated period, under a stated boundary. It is the strongest kind of value and still not a measurement in the physics sense — a reported Scope 1 figure is itself the output of an internal calculation with its own assumptions about emission factors and consolidation.
Estimated. The vendor produced the number with a model that used entity-specific inputs. A regression on sector, revenue, asset base and geography is an estimate. It can be good. It is not a disclosure, and its error is a property of the model, not of the company.
Modeled from an average. The vendor applied a sector-level intensity factor to a financial line item. This is arithmetic on someone else's average. Two companies in the same sector with the same revenue and wildly different operations will receive nearly identical values, which is exactly the case where an analyst most needs a difference.
Inherited. The value belongs to a parent, an affiliate or a peer group, and has been assigned to this entity because nothing entity-specific exists. Inheritance is often reasonable and is almost never visible in the delivered file.
| Label | What it actually asserts | What to store beside it |
|---|---|---|
| Reported | A named document says this, for a named period, under a named boundary | Document, page or section, reporting period, consolidation basis, assurance status |
| Estimated | A model produced it from inputs specific to this entity | Model identifier and version, the inputs used, an uncertainty band if the vendor publishes one |
| Modeled from average | A sector factor was multiplied by a financial quantity | The factor, its source and vintage, the financial line and its currency and period |
| Inherited | Another entity's value was assigned to this one | The source entity, the relationship, and the rule that permitted the assignment |
| Absent | Nothing is known | Null, distinguishable from zero — and it must never be silently imputed on the way in |
If your vendor publishes a flag for this, use it and validate it. If they do not, ask — several will provide one on request because the flag exists internally. If they will not, treat the entire field as estimated and say so where the number is used. That single sentence in a footnote has settled more customer questions than any amount of methodology prose.
Scope 3 is the honest illustration
Value-chain emissions are where the gap between the shape of the number and the shape of the claim is widest, and the pattern repeats everywhere else in the category.
A large share of reported Scope 3 comes from spend-based estimation: take what a company spent in a purchasing category, multiply by an emissions factor per unit of currency, sum. The arithmetic is sound; the interpretation is where it goes wrong. A spend-based estimate describes a category average, so it moves with prices and procurement mix rather than with anything a supplier does. Buy the same physical goods cheaper and the estimate falls. Switch to a genuinely cleaner supplier at the same price and it does not move at all.
The practical consequence for anyone building analytics on this: Scope 3 series are not comparable across companies with different procurement structures, and year-over-year changes in them are frequently price effects. If your product ranks companies, rank within a peer group and disclose the method. If you are a supplier being asked for data, an activity-based figure — actual fuel, actual kilowatt-hours, actual tonne-kilometres — is worth more to your customer than a polished spend-based one, and it is easier to defend because the underlying records already exist in your systems.
Restatements are routine, and they break anything time-indexed
Sustainability data is revised backward more often than financial data, and for good reasons: a company recalculates a base year after an acquisition, a factor library is updated, a methodology change propagates through a vendor's whole history. The revision is legitimate. What it does to your system is not.
If your database stores one value per entity per year and overwrites on each delivery, then every backtest you have ever run is contaminated. You are testing a strategy or a model against numbers that did not exist on the date you claim to have used them. This is the same point-in-time discipline that market data practitioners settled decades ago, and it has not fully arrived in this category.
The fix is bitemporal storage, and it is less work than it sounds. Keep two dates on every row: the period the value describes, and the date you received it. Never update in place; insert a new row and let the old one stand. Then a query for “what did we believe on 3 March” is a filter rather than an archaeology project. The storage cost is trivial next to the cost of one wrong answer to a customer asking why last quarter's report no longer reproduces.
Two things follow from having the history. You can measure how much your vendor restates, per field, per year, which is a hard number to put in a renewal conversation. And you can show a customer the exact vintage behind a published figure.
Assurance covers less than the word suggests
Assurance on sustainability information is generally offered at two levels. The lower one, commonly called limited assurance, is expressed as a negative conclusion — nothing came to the practitioner's attention suggesting the subject matter is materially misstated. The higher one, reasonable assurance, is a positive opinion and requires substantially more work. Financial statement audits sit at the higher level; a great deal of sustainability reporting sits at the lower one.
Three details matter more than the level itself, and all three live in the assurance report rather than in the vendor's field. Scope: which metrics were covered, frequently a subset — Scope 1 and 2 assured, Scope 3 not, or only certain categories. Boundary: which entities and sites were included. Criteria: the framework the preparer applied, since a figure is only ever assured against stated criteria.
So an assurance flag stored as a boolean on the entity is close to meaningless. Store it at the metric level, with the level, period, practitioner and scope note — and where you cannot get that granularity, store nothing rather than a boolean that will be read as more than it is.
The rules are moving, and that is an architecture fact
Several regimes now bear on this data at once: the European corporate sustainability reporting regime and its associated standards, the ISSB standards and their varying jurisdictional adoption, California's climate disclosure statutes, and the US Securities and Exchange Commission's climate disclosure rulemaking. Effective dates, scope thresholds and phase-in schedules across these have been amended, delayed and challenged in court, in some cases repeatedly, and any specific date printed in an article ages badly.
So rather than assert what applies to you, here is what to verify. Which regimes capture your entity by size, listing status and jurisdiction — a counsel question, and the answer often includes regimes you do not file under but whose reporters will ask you for data. The current effective date and phase-in wave for each, at the issuing body, this quarter. Which metrics require assurance and at what level, since that ratchets over time. And whether your customers' obligations flow down to you contractually, which is how most suppliers first encounter this.
The architecture consequence is the durable part, and it does not depend on which way the rules land. Build for multiple simultaneous frameworks over one underlying set of facts. Store activity data at the finest grain you actually have — fuel by site by month, electricity by meter, refrigerant by charge event — and compute framework-specific presentations on top. Teams that stored the presentation instead of the activity data have rebuilt from scratch every time a definition changed. Teams that stored the activity data have written a new view.
How defensible is a field, by origin — our read
Our judgment of how each origin holds up when a customer, an assurance provider or an acquirer asks for the derivation. Not a survey, and not a comment on any vendor.
Coverage is a function, not a percentage
Every vendor publishes a coverage figure. It is nearly always true and nearly always unhelpful, because coverage is not uniform: high for large listed companies in jurisdictions with disclosure requirements, and falling off steeply everywhere else — private companies, smaller listings, emerging markets, and the long tail of subsidiaries that make up an actual supply chain.
Measure it against your universe instead. Take the entity list you actually care about, join it, and report coverage by segment: by size band, by region, by listing status, by sector, and separately for reported values versus any value at all. That last split is the one that changes decisions. A field can be 90 percent populated and 15 percent reported, and those two numbers describe completely different products.
Do the same for your own supply chain if you are the one being asked. Most suppliers find that a small number of categories account for most of the footprint and that the data exists for those, which turns an impossible-looking questionnaire into a short list of real work.
Entity resolution decides more than the methodology does
Before a value can be right, it has to be attached to the right thing, and this category makes that unusually hard. A reporting entity is a consolidation boundary, not a legal entity. The company that publishes the report may not be the company you have in your system, the ownership share may be partial, and operational control and equity share give different answers for the same asset. Subsidiaries change hands mid-year. Joint ventures are counted by one partner, both, or neither.
Two practices avoid most of the damage. Store the identifier the source actually used, unmodified, beside your internal key — legal entity identifier, registration number, ticker with its exchange, whatever came in. And make consolidation basis a field, because a value computed on operational control and one computed on equity share are not comparable, and someone will eventually put them in the same chart.
The provenance record worth attaching to every number
All of the above collapses into one design decision: a value never travels alone. Attach a small record to it, generated automatically, and carry it through to whatever you publish. Ours is deliberately short, because a provenance schema nobody can populate is worse than none.
- Origin — reported, estimated, modeled from average, or inherited
- Source — the vendor and file, or the document and section, with a retrievable pointer
- Period — what span the value describes, and the fiscal basis
- As-of — when you received it, so point-in-time queries are possible
- Method version — which methodology or model version produced it
- Boundary — consolidation basis, and which entities are inside it
- Assurance — level, scope and practitioner, at the metric level or not at all
- Units — explicit, including the currency and year for anything monetary
- Revision — whether this supersedes an earlier value, and the earlier value's key
Nine fields, generated by the loader rather than typed by an analyst. The test is simple: pick a published figure at random and reconstruct it end to end without asking a person. If that takes more than a few minutes, the record is not being populated where the data enters.
The mistakes we are called in to fix
- One column for four kinds of claim, with no origin flag, feeding a model that treats them identically
- Overwriting on each delivery, which quietly invalidates every backtest and every reproduction of an old report
- Nulls imputed at load time, so absence becomes indistinguishable from a measured zero one layer down
- A boolean assurance flag on the entity where the real answer is per-metric and scoped
- Storing the framework presentation instead of the underlying activity data, so every definition change is a rebuild
- Comparing across consolidation bases because the basis was never a field
- Reporting the vendor's coverage percentage rather than coverage of your own universe
- Year-over-year Scope 3 movement presented as performance when the estimate is spend-based and prices moved
Where you do not need help
If you are a single-site operator answering a customer questionnaire, this is a spreadsheet job and it should stay one. Pull twelve months of utility bills and fuel purchases, apply published factors, write down which factor set and which year, and keep the bills. That is a defensible submission, and buying software for it would waste money. The same holds if you consume one vendor feed, publish nothing derived from it and have no history to protect.
The work becomes real when you publish figures other people rely on, when you have a back history that must reproduce, or when several frameworks and several vendors have to reconcile to one set of facts. That is a data engineering problem with a sustainability subject, and it is worth staffing as one.
Before you publish a derived figure
- Every value carries an origin label, and the label came from the loader rather than an assumption
- Nothing is overwritten; revisions insert, and point-in-time queries work
- Coverage is stated against your universe, segmented, with reported split out from populated
- Units, currency and currency year are explicit on every monetary input
- Consolidation basis is a field, and nothing crosses bases in one comparison
- Assurance is recorded per metric, with level and scope, or not recorded at all
- Activity data is stored at its finest available grain, with framework views computed on top
- Spend-based estimates are labeled as such wherever a change over time is shown
- One figure can be reconstructed end to end in minutes, by someone who did not build it
Bottom line
The numbers in this category are mostly derived, derivation is legitimate, and the harm comes from losing the derivation on the way into your database. Keep the origin, the as-of date and the boundary, and store activity data rather than presentations. Do that and the hard questions — from a customer, an assurance provider, a regulator, an acquirer — become lookups instead of investigations. Skip it and you answer them anyway, later, in front of a larger audience.
Frequently asked questions
It varies enormously by metric, company size and jurisdiction, which is why a single figure is not worth quoting. Reported values concentrate in large listed companies in jurisdictions with disclosure requirements and fall away sharply elsewhere. Measure it against your own entity universe, split by size band and region, and report populated and reported as two separate numbers.
First, determine whether you still have the old values. If you overwrote, any published figure derived from the old vintage cannot be reproduced, and that is worth telling affected customers before they find out. Then move to bitemporal storage so it cannot happen again: two dates per row, insert rather than update. Then measure restatement frequency per field and bring that number to the renewal.
Sometimes, but agreement between two vendors is not evidence of accuracy, because they frequently draw on the same underlying disclosures. Run an overlap test on a sample first: how often they agree on reported values, how often on estimated ones, and whether the second adds entity coverage in the segment you care about. Disagreement concentrated in estimated fields tells you the size of model uncertainty, which is useful but is not a second opinion.
Start by finding out which categories dominate your footprint, which is usually a small number, and get activity-based data for those — actual quantities of fuel, electricity, materials or freight. An activity-based figure for the few categories that matter is more defensible and more useful to your customer than a spend-based estimate covering everything. Write down the factor set and vintage you used and keep the source records.
It means a practitioner performed procedures against stated criteria, over a stated scope, at a stated level. Limited assurance is a negative conclusion and involves substantially less work than the reasonable assurance used for financial statements. Read the report for which metrics and entities were covered — partial scope is common — and store that scope rather than a true or false on the company.