Start with the data that already exists
Commercial real estate holds more data than almost anyone realizes and less usable data than almost anyone assumes. A mid-size portfolio already contains executed leases and their amendments, monthly rent rolls, general-ledger exports, work-order histories, utility invoices, interval meter reads, building automation points, appraisals, and a decade of broker spreadsheets. All of it exists. Very little of it agrees with itself. That gap is where an AI budget either earns its keep or quietly disappears, and the deciding factor is almost never which model you picked.
The scale of the asset base is public. The Energy Information Administration's 2018 Commercial Buildings Energy Consumption Survey counted 5.9 million commercial buildings in the United States totaling 97 billion square feet. County assessors maintain parcel records across more than 3,100 counties and county equivalents, covering roughly 150 million parcels nationally. The EPA's ENERGY STAR Portfolio Manager holds benchmarking records for a large share of that floor area, because dozens of city and state ordinances require annual submission through it. None of it is proprietary.
Inside a single owner or operator, the same building typically lives in five systems at once. It has a property code in the accounting platform, a lease record in the administration system, an assessor parcel number in the county file, a utility account number at each meter, and a market identifier in a licensed comparables feed. Those five identifiers were assigned by five different parties who never coordinated. Reconciling them is the first real engineering task, and it is the one most vendors skip in a demo.
Data readiness by property workstream
Editorial weighting from public sources and practitioner reading, illustrative rather than a measured statistic.
Lease abstraction is a schema problem before it is a reading problem
Lease abstraction is the most common first request, and for good reason. Accounting Standards Codification Topic 842 put operating leases on the balance sheet, effective for public business entities in fiscal years beginning after 15 December 2018 and for private companies after 15 December 2021. That standard cannot be satisfied without structured lease terms: commencement date, term, renewal options, escalations, and the discount rate applied. IFRS 16 imposes the same demand outside the United States. Every organization that closes a book has already been forced to abstract its leases at least once.
A typical institutional office or industrial lease runs 40 to 120 pages before amendments, and a working abstract carries 80 to 250 fields. The hard part is not that the language is difficult. It is that the target schema encodes decisions the business has never written down. Lease commencement, rent commencement, possession, and delivery are four different dates that a careless schema collapses into one. Triple net, modified gross, and full service gross recoveries produce different expense pass-throughs from identical dollar figures. Base year stops, expense caps, and gross-up provisions change the recoverable amount without changing any number printed on the page.
Our engineers treat the schema as the deliverable and the extraction as the implementation detail. Fields get a type, a unit, a controlled vocabulary where one exists, a provenance pointer back to the page and clause, and a defined behavior when the document is silent. Silence matters most. A model that infers a missing renewal notice period produces a plausible number no one can trace and no one will catch until the deadline passes. We would rather ship a blank with a citation than a guess without one.
Portfolio rollup is where the value concentrates
One abstracted lease saves an analyst an afternoon. Four hundred abstracted leases, normalized against a rent roll and rolled into one queryable model, answers questions the organization currently cannot answer at all. What is our weighted average lease term by market. Which recovery structures leave us exposed if operating expenses rise nine percent. Those are portfolio questions, and they are worth an order of magnitude more than document-level convenience.
The obstacle is identity. Square footage is the clearest example. ANSI/BOMA Z65.1-2017 defines rentable and usable area for office buildings, and earlier BOMA revisions plus the International Property Measurement Standards define them differently. Two internal systems can report different areas for the same floor, both correctly, because they applied different standards in different years. A rollup that sums those numbers without reconciling the measurement basis produces a portfolio total that is wrong in a way no validation rule will flag.
The fix is unglamorous and it works. Build a property master with one durable internal identifier per asset. Map every external key to it explicitly, with the mapping stored as data rather than buried in a transformation script. Record the measurement standard alongside every area figure. Then every rollup is reproducible, and every disagreement between two systems becomes a visible, assignable discrepancy instead of a silent averaging error.
Energy performance, where the meter is the ground truth
Energy analytics is the one property workstream with an external referee. The utility meter does not care what the model predicted, which makes it the best place to start when a buyer wants proof.
Regulation supplies the deadline. New York City Local Law 97 of 2019 sets carbon intensity limits on most buildings over 25,000 square feet, with the first compliance period running 2024 through 2029 and a penalty of $268 per metric ton of CO2 equivalent over the limit. Boston's BERDO 2.0 sets an alternative compliance payment of $234 per metric ton. Washington State's Clean Buildings Act sets energy use intensity targets for covered commercial buildings with penalties reaching $5,000 plus one dollar per square foot per year of noncompliance. Federal facilities operate under 42 U.S.C. 8253(f), which requires covered facilities to undergo energy and water evaluations on a four-year cycle and to benchmark through Portfolio Manager.
The data volume is smaller than people expect. Interval meter data at 15-minute resolution produces 96 readings per meter per day, or 35,040 per year. A 200-building portfolio with six metered commodities per building generates about 42 million readings annually. That is a few gigabytes. Storage is not the problem and compute is not the problem. Gap-filling is the problem, along with meter changes, billing periods that do not align to calendar months, submeters that double-count, and the vacancy and weather normalization required before any two months can be compared honestly. Degree-day normalization against a public weather station is a half-day of work that changes the answer more than any model selection will.
Volumes, latency, and cost stated plainly
Buyers get quoted architectures instead of numbers. Here are the numbers that shape a design.
| Workstream | Typical volume | Latency the business needs | Where cost concentrates |
|---|---|---|---|
| Lease abstraction | 400 to 2,000 documents, 40 to 120 pages each | Overnight batch; nothing here is real time | Human review of low-confidence fields |
| Rent roll normalization | One export per property per month, 50 to 5,000 rows | Same day, monthly cadence | Identity mapping and unit reconciliation |
| Interval meter analytics | 35,040 reads per meter per year; tens of millions per portfolio | Daily, hourly for anomaly alerts | Gap-filling and normalization logic |
| Building automation telemetry | 5,000 to 50,000 points per building at 1 to 15 minutes | Minutes for alarms, daily for trends | Point mapping and network segmentation |
| Underwriting screens | Hundreds of deals per quarter | Seconds, inside the analyst's tool | Data licensing, not inference |
| Compliance reporting | One filing per building per year | Annual, with an immovable deadline | Reconciliation to the meter of record |
Inference cost is rarely the binding constraint. A 60-page lease runs roughly 40,000 to 60,000 input tokens. At published list prices for frontier models in the range of a few dollars to fifteen dollars per million input tokens, a full-document pass costs cents to about a dollar. Outsourced human abstraction is commonly quoted in the $50 to $250 per lease range depending on complexity. The economics are not close. What is expensive is the review loop, the schema work, and the integration into the system of record, which is why an honest quote allocates most of its hours there.
The rules that shape the architecture
Commercial property work touches several bodies of law that constrain design rather than merely requiring a disclaimer.
Tenant screening and credit decisions. Where a portfolio includes residential or mixed-use assets, screening reports are consumer reports under the Fair Credit Reporting Act, 15 U.S.C. 1681, and denials trigger adverse action duties under 15 U.S.C. 1681m. Where credit is extended, Regulation B at 12 CFR 1002.9 requires a statement of specific reasons for adverse action. The Consumer Financial Protection Bureau's 2022 circular on this point made the operative rule explicit: a model whose reasons cannot be stated does not satisfy the requirement simply because it is complex. Architecturally, that means reason codes are a first-class output, not a reporting afterthought.
Fair housing exposure. The Fair Housing Act, 42 U.S.C. 3601 and following, reaches practices with discriminatory effects, and HUD's discriminatory effects standard at 24 CFR 100.500 supplies the framework. Any scoring or targeting system that touches residential occupancy needs disparate impact testing on held-out data before deployment and on live decisions afterward, with the test results retained.
Algorithmic pricing. In August 2024 the Department of Justice filed a Sherman Act complaint against RealPage over revenue management software alleged to set rents using competitors' nonpublic data, and San Francisco and several other cities have since restricted algorithmic rent-setting. The engineering consequence is direct. A pricing model trained on a client's own transactions and on public listings sits in a different legal position than one trained on a pooled set of competitors' confidential terms. Provenance of every training input has to be recorded and defensible.
Valuation. Appraisal work follows the Uniform Standards of Professional Appraisal Practice. The interagency automated valuation model rule implementing 12 U.S.C. 3354 took effect on 1 October 2025, and its quality control requirements attach to covered transactions secured by a consumer's principal dwelling rather than to commercial assets. Knowing which side of that boundary a product sits on is a design input, and a valuation aid built for an internal analyst is a different regulated object than one that feeds a mortgage origination.
Building controls are operational technology, not IT
Reading a building automation system is where a property analytics project most often stalls, and the reason is security rather than protocol. Controllers speak BACnet/IP, Modbus, and vendor-specific variants, and the integration itself is well-trodden. The concern is that the same network runs chillers, air handlers, and access control. NIST Special Publication 800-82 Revision 3, published in September 2023, is the reference for securing that class of system, and it exists because the failure mode is physical. The 2013 Target breach began with credentials belonging to an HVAC service contractor.
The pattern our team recommends is one-way. Telemetry flows out of the control network through a historian or a data diode arrangement into an analytics environment. No analytics component holds write access to a controller, ever, and any sequence change is a documented action by the controls contractor. Analytics recommends; the building operator and the controls integrator act. That boundary keeps an optimization experiment from becoming a comfort complaint or a frozen coil.
Point naming is the other tax. The same sensor may be labeled six ways across six buildings, and semantic tagging is the fix. Project Haystack, Brick Schema, and ASHRAE Standard 223P are the three vocabularies in play, and none has fully won. Pick one, map to it explicitly, and hold the mapping as reviewable data. ASHRAE Guideline 36 gives standardized high-performance sequences of operation and is the right yardstick for judging whether a system is running as designed before anyone proposes to improve on it.
Why data quality decides the outcome
Across property workstreams, the difference between a system that holds up and one that gets abandoned is almost never model selection. Modern extraction models read lease language well. What they cannot do is resolve which of your systems is authoritative when they disagree, and that is the question the business actually has.
Silent unit errors. Rent stated per square foot per year versus per month, areas in rentable versus usable terms, energy in kBtu versus therms versus ccf. Each of these is a factor of twelve, a load factor, or a conversion constant away from a plausible wrong answer that passes every range check.
Amendment ordering. The governing term is the one in the most recent amendment, and amendments arrive out of order, sometimes scanned, sometimes unsigned drafts filed alongside executed copies. Document classification before extraction is worth more than any accuracy tuning after it.
Effective dating. A rent roll is a snapshot. A lease is a contract with a timeline. Comparing them without an as-of date produces variances that are not errors, and chasing phantoms burns the credibility of the whole effort in the first month.
Abstention behavior. The metric that predicts success is not average accuracy. It is the rate at which the system produces a confident wrong answer, and the rate at which it correctly declines. A system that abstains on eight percent of fields and is right on the rest is worth far more than one that answers everything at ninety-four percent, because the first one tells a reviewer where to look.
The federal version of the same problem
Federal real property is the largest single portfolio in the country and it has the same data conditions. The General Services Administration manages roughly 360 million rentable square feet across more than 8,000 owned and leased assets. The Federal Real Property Profile, established under Executive Order 13327, is the government-wide inventory, and the Federal Assets Sale and Transfer Act of 2016, Public Law 114-287, created the Public Buildings Reform Board to act on disposal candidates. The Government Accountability Office has carried federal real property management on its High-Risk List since 2003, and inventory data quality is a recurring theme in that reporting.
For an agency or a federal landlord, the work is the same with an added accreditation path. Data on federal tenants and facilities is often Controlled Unclassified Information, which brings NIST SP 800-171 expectations, and a hosted analytics environment brings a FedRAMP conversation. Our team builds those boundaries in at the start, because the security plan determines whether a working system is allowed to run.
Scoping a first engagement that can fail cheaply
The best protection a buyer has is a first engagement small enough that a negative result is affordable and specific enough that the result is unambiguous. Six weeks, fixed price, one asset class, one named decision.
A six-week first engagement
Three conditions make that structure honest. The gold set is built by the client's people, not by the vendor, so the answer key is not authored by the party being graded. The threshold is written down before results exist, per field and not in aggregate, because a portfolio average hides the fields that matter. And the sample is drawn from production at random, including the 1998 scan with the coffee ring, because a curated sample measures the curator.
Expect a pilot of this scope in the low tens of thousands of dollars, and the production build that follows at several times that, with most of the increase in integration, review workflow, monitoring, and the documentation that lets a new analyst trust the output. Any proposal where the pilot is the expensive part and production is a rounding error has the shape of the work backwards.
How to tell whether it worked
Measure against the decision, not against the demo. If the stated purpose was to close the books faster, the test is days to close. If it was to catch recovery underbilling, the test is dollars recovered against a baseline period. If it was to hit a benchmarking deadline, the test is whether the filing was submitted on time with numbers that reconcile to the utility invoices. Each is checkable by someone who does not work on the project, which is the property a good acceptance criterion has.
Then measure the maintenance load. A property data system that requires an engineer every time a lease template changes or a meter is swapped is a system with a hidden subscription cost. Ask, before signing, who fixes a field that starts failing in month nine, how a schema change is versioned, and what happens to historical extractions when the schema moves. The answers to those three questions separate a delivered system from a delivered demonstration.
Common questions on scope and constraint
Do we need a large language model for this at all?
For lease and contract language, yes, and it earns its place. For rent roll normalization, meter analytics, and portfolio rollup, usually no. Those are deterministic transformations with validation rules, and a model in that path adds variance without adding capability. The right architecture uses generation where the input is unstructured prose and plain code everywhere else.
Can this run without our data leaving our environment?
Yes. Document extraction runs inside a client tenancy or on a private endpoint in any major cloud, and open-weight models handle a meaningful share of the workload when policy forbids external calls. The cost and accuracy trade is real and should be measured on the client's own documents rather than assumed in either direction.
Our data is a mess. Should we clean it first?
No. Cleaning without a target schema produces a tidier version of the same ambiguity. Define the decision, derive the schema from it, and let the schema tell you which fields are worth cleaning. Most portfolios find that a quarter of the fields carry nearly all the decision weight.
What if the pilot fails the threshold?
Then it fails, the result is documented with per-field numbers, and the organization has bought real information at a known price. A failed pilot that identifies which document classes and which fields are unreliable is a useful asset. A pilot that cannot fail was never a measurement.
Frequently asked questions
Lease abstraction feeding a portfolio model, recovery and reimbursement audit against executed terms, and energy performance analytics tied to a benchmarking or carbon compliance deadline. All three have a measurable dollar outcome and an external check on the answer, which is what makes them defensible purchases.
Accuracy targets belong per field, not per document. Critical dates and rent figures should be held to a very high bar with citation to the source clause, while descriptive fields tolerate more error. The measure that matters most is how often the system produces a confident wrong value versus correctly declining to answer.
For residential and mixed-use exposure, the Fair Housing Act with HUD's discriminatory effects standard at 24 CFR 100.500, the Fair Credit Reporting Act at 15 U.S.C. 1681, and Regulation B adverse action requirements at 12 CFR 1002.9. For pricing, antitrust exposure around pooled competitor data. For valuation touching consumer dwellings, the automated valuation model rule at 12 U.S.C. 3354. For building control networks, NIST SP 800-82 Revision 3.
Less than most buyers assume. Fifteen-minute interval data produces 35,040 readings per meter per year, so a 200-building portfolio with six commodities per building lands near 42 million rows annually, a few gigabytes. The engineering effort goes into gap-filling, meter changes, billing period alignment, and weather and vacancy normalization.
Fixed price, roughly six weeks, one asset class, one named decision, a client-built gold set held out from development, per-field thresholds written before any result exists, and a documented go or no-go at the end. The goal is a first project that can fail for a price the organization already agreed to pay.