Risk adjustment data quality

The gap, not the score

There is one report a risk adjustment program can run that tells it more than any dashboard: take every diagnosis your organization believes it submitted for a payment year, take every diagnosis the payer's return files say were accepted as risk adjustment eligible, and diff them. Two piles fall out. Diagnoses you sent that never landed, which is money you did the clinical work for and did not get. And diagnoses that landed which you cannot trace back to a specific encounter, a specific rendering provider and a specific line in a specific medical record, which is the pile that costs you far more than the first one if anybody ever asks.

Most programs cannot run that diff. Not because it is technically hard — it is a join — but because the two sides of it live in different systems owned by different teams, the identifiers do not line up cleanly, and vendor-submitted records arrive without the provenance needed to attribute them. So the organization measures what it can measure, which is the aggregate risk score, and the score goes up, and nobody knows which part of the increase is documentation improvement and which part is a defect waiting to be found.

Everything in this article is downstream of that one idea. Risk adjustment is not a coding problem with a data component. It is a records problem with a coding component, and the records problem is the one that goes wrong quietly.

A diagnosis has a source, and the source decides everything

Every condition that contributes to a risk score arrived by one of a small number of routes, and the routes have very different audit strength. Any pipeline that stores the condition without storing the route has thrown away the only thing that matters later.

The encounter itself. A clinician saw the patient, documented an assessment, and the diagnosis came through on the encounter record. This is the strongest source and it is where a well-run program gets most of its capture.

Retrospective chart review. A coder read the record after the fact and found a condition that was documented but not submitted. Legitimate, common, and the source that draws the most scrutiny — because a review program that only ever adds, and never removes anything it finds unsupported, is a program whose one-directional design is itself the finding. That theory has featured in federal enforcement against risk-bearing organizations. The engineering answer is simple and uncomfortable: the same review must be able to emit a delete, and your pipeline must be able to carry it.

Prospective assessments. A visit arranged largely to document conditions. Whether these count, and under what conditions, has moved more than once. Read the current program rules for your line of business rather than assuming last year's answer holds.

Supplemental feeds. Lab results, pharmacy data, and vendor files that suggest a condition. These are leads for a clinician, not diagnoses. A pipeline that lets a lab value become a submitted condition without a clinician documenting it has built a compliance problem with good intentions.

Store the route on the record. Store the encounter identifier, the rendering provider, the date of service, the document the coder read, the page or section, and the coder's identity. It costs a handful of columns and it is the difference between answering an audit in an afternoon and answering it in a quarter.

Reconciliation is the discipline, and almost nobody does it at the right grain

Payer programs return files that tell you what happened to what you sent. In Medicare Advantage the shape is familiar: files that tell you whether each encounter was accepted or rejected and why, a file that tells you which diagnoses on accepted encounters were deemed risk adjustment eligible, a monthly membership file carrying the risk score used for payment, and a model output file with the condition categories behind that score. Commercial and Medicaid programs have their own equivalents. The names differ. The structure does not.

Three reconciliations are worth building, and they are usually built in the wrong order.

Submission to acceptance. Did each encounter you sent get accepted? Rejections are frequently boring — a provider identifier that is not on file, a member identifier that does not match on the date of service, a required field the clearinghouse dropped. Boring and systematic, which is what makes it worth automating: a single upstream defect can silently reject thousands of encounters, and if nobody watches the return files, it goes unnoticed until the payment year closes.

Acceptance to eligibility. An accepted encounter does not mean an accepted diagnosis. Diagnoses drop out for reasons that have nothing to do with clinical truth: the provider type on the encounter is not an acceptable source for risk adjustment, the service date falls outside the collection window, the code is not one that maps to a condition category in the model version in force. This is the reconciliation with the most recoverable money in it and the one most often skipped.

Eligibility to payment. Do the condition categories in the payer's model output match the ones you computed? A mismatch usually means a model version difference, a hierarchy you applied differently, or an enrollment segment you got wrong. Resolve it before you build a forecast on your own numbers.

Do all three at the diagnosis level, keyed on member, date of service and code. Aggregate reconciliation — "we submitted 412,000 encounters and 408,000 were accepted, so we are at 99 percent" — hides exactly the systematic defects you are looking for, because a defect that costs you one whole provider group is a rounding error at the aggregate and a serious problem at the grain.

Adds are easy. Deletes are the engineering problem.

This is the paragraph to take away. Every risk adjustment pipeline we have seen is built to add. Very few are built to remove, and the ones that are not have a structural compliance exposure that no amount of coder training fixes.

A delete is genuinely harder. To remove a diagnosis you already submitted, you have to know exactly what you submitted, on which encounter, in which file, on which date, under which submitter identifier. If a vendor made the submission on your behalf, you need their record of it, in a form you can reconcile to your own. Then you have to emit a correctly formed deletion record, confirm it was accepted, and confirm the condition actually came off the payer's model output in the next cycle. Four systems, two organizations, and a confirmation loop that runs on a monthly cadence.

Programs skip it. The reasons are always practical rather than malicious: the vendor contract is written around adds, the retention schedule does not keep submission artifacts long enough, nobody owns the delete queue. The result is a program that structurally cannot correct itself, and that is a finding waiting to be written.

Build for it from the start. Keep the exact submitted artifact, not a summary of it. Give every submitted diagnosis a stable internal identifier that survives into the return file join. Make "unsupported on review" a first-class outcome in your coder tooling with the same workflow weight as "supported, add." Then report the delete count monthly beside the add count, to the same audience. A program that reports both numbers to its board is a program with a very different conversation available when an auditor arrives.

Defect	How it shows up	How to detect it
Rejected encounters nobody read	Capture looks low for one group or one site with no clinical explanation	Automated parse of return files, grouped by reject reason and by submitter
Accepted encounter, ineligible diagnosis	Your capture number and the payer's disagree by a stable percentage	Diagnosis-level join of submission to the eligibility return file
Chart review adds with no linked encounter	Conditions that cannot be traced to a date of service	Query for submitted conditions lacking an encounter key
Carry-forward without documentation	A chronic condition reappears every year, and the current year's record does not address it	Compare submitted conditions to conditions addressed in this year's notes
Provider type not an acceptable source	A whole class of encounters produces zero eligible diagnoses	Eligibility rate grouped by provider type and place of service
Duplicate submission through two channels	Direct submissions and vendor submissions overlap	Deduplicate on member, date of service, provider and code before submitting
No delete path	The unsupported queue has entries older than the submission deadline	Age the unsupported queue; if it only grows, there is no path

What changed the arithmetic, and why the old assumption is expensive

For years, organizations sized their risk adjustment exposure on the assumption that an audit finding applied to the sampled charts and stopped there. Regulatory policy has moved toward extrapolating findings from a sample to the wider population, and toward removing adjustments that previously softened the comparison. The direction of travel is settled even where the specifics are still being litigated and refined.

The engineering consequence is straightforward. When findings do not extrapolate, a defect rate of a few percent is a nuisance. When they do, that same rate is a multiplier applied to a population, and the value of being able to demonstrate a defect rate — measured by you, on your own sample, before anyone asks — rises sharply.

So run your own validation. Pull a random sample of submitted conditions each quarter, blind a coder to the original decision, and have them adjudicate from the record alone. Report the agreement rate. This is not an exotic capability; it is the same second-pass audit design used in any regulated review process, and it produces the one number that lets you answer an external question with evidence instead of adjectives.

The documentation is the asset. The code is a pointer to it.

A condition category is a claim about a person. What supports the claim is a clinician's note saying the condition was assessed and what was done about it. The widely used shorthand for that — that the record should show the condition being monitored, evaluated, assessed or treated — is an auditor convention rather than a rule with that name in any regulation, and it is a good convention because it describes what a reviewer actually looks for when they open the chart.

Which means the highest-return intervention in most programs is not analytics at all. It is making it easier for a clinician to document what they already did. A problem list that carries forward without being touched is worth nothing and looks bad. An assessment and plan that names the condition and says what is being done about it is worth the full value and survives review.

That is a workflow and template problem inside the record system, and it is frequently solvable without a model, without a vendor, and without a data platform. If your capture gap is concentrated in a few conditions and a few clinicians, the answer is probably a template change and a conversation, and you should do that before you buy anything.

Where models help, and the one place they are dangerous

There is real and safe work for machine learning here, and there is one application that we will not build.

Safe and useful: finding records worth a human's time. Chart review is expensive because a coder reads a lot of records that contain nothing. A model that ranks records by the probability that a supported, unsubmitted condition is present makes the same coder budget cover more ground. The output is a queue, the human makes every decision, and the measurable result is the coder's yield per hour.

Safe and useful: surfacing the evidence. Given a record, highlight the passages that mention a condition and the passages that show it being addressed. The coder still reads and still decides, but the reading is faster and the citation for the decision is captured automatically, which is exactly the provenance the audit needs.

Safe and useful: finding what should come off. Point the same machinery at conditions you already submitted and rank them by weakness of support. This is the queue almost nobody builds, and it is the one that changes an auditor's view of your program.

The dangerous one: prompting a clinician to add a diagnosis at the point of care with the payment consequence attached. The failure mode is not subtle. A prompt that appears because a condition is valuable, rather than because the record suggests it, produces documentation that follows the prompt, and the resulting pattern is visible in the data to anyone who looks. If a decision support tool is going to suggest a condition, the suggestion must be driven by clinical evidence in the record, must be as easy to dismiss as to accept, must record the dismissal, and must never display what the condition is worth. That last constraint is unpopular and it is the one that keeps the program clean.

What the data will actually look like

Six things show up in nearly every engagement, and finding them in week one is cheaper than finding them in month six.

Member identity is not stable. Members change plans, change identifiers, are enrolled twice, or appear under a different identifier in the vendor's file than in yours. Every reconciliation you build depends on resolving that, and the resolution rule needs to be written down once and used everywhere.

Dates mean different things. Date of service, date of submission, date of acceptance, date the chart was reviewed. Programs routinely join on the wrong one and then explain the resulting gap as a clinical trend.

The vendor's numbers are the vendor's numbers. A capture rate computed by the organization being paid for capture is not a control. Recompute it yourself from the raw return files. If you cannot get the raw return files, that is itself the finding, and it is a contract term for the next renewal.

Model versions blend. Payment years frequently mix versions, and a comparison across years that ignores the blend produces a trend that is entirely artificial. Confirm which version and which blend applies to the year you are analyzing before you show anyone a chart.

Encounters arrive late. Any month's numbers keep moving for months afterward. Report every metric with a data-as-of date and a completion curve, or you will spend your quarterly meetings explaining why last quarter changed.

Retention schedules delete your evidence. Submission artifacts, return files and coder decisions frequently live in systems with retention windows shorter than the audit lookback. Check yours before you need it.

What we see go wrong

• Measuring the risk score instead of the submitted-to-accepted gap — the score can rise while the defect rate rises with it
• Storing the condition without the encounter, provider, document and coder — unrecoverable later, and cheap to store now
• A review program with no delete path, which is a design decision even when nobody decided it
• Taking the vendor's capture rate as the measurement rather than as a claim to be verified
• Aggregate reconciliation that hides a defect concentrated in one group, one site or one provider type
• Point-of-care prompts that show the payment value of a condition to the person documenting it
• No internal validation sample, so the first estimate of the defect rate is somebody else's
• Comparing payment years across model versions without accounting for the blend

What good looks like

• Every submitted condition carries member, encounter, provider, date, document, page and coder
• Return files are parsed automatically and reconciled at the diagnosis grain
• Rejections are a work queue with an owner, not a file in a folder
• The delete path works end to end and has been exercised in production
• Adds and deletes are reported to the same audience, monthly
• A blinded second coder validates a random sample each quarter, and the agreement rate is published internally
• Vendor capture claims are recomputed from raw return files
• Every metric carries a data-as-of date and a completion curve
• Clinical decision support never shows the payment value of a condition
• Submission artifacts are retained at least as long as the audit lookback

Bottom line

Risk adjustment goes wrong in the plumbing far more often than in the coding. The single highest-value thing most programs can build is a diagnosis-level reconciliation between what they submitted and what came back, with provenance on every record and a working path to remove something that should not be there. That is unglamorous engineering — joins, identifiers, return-file parsers, retention — and it is what turns an audit from a research project into a query. If your program can already do all of it, you do not need us; you need a template change and a conversation with three clinicians. If it cannot, start with the reconciliation, because everything else is easier once the two sides of the ledger agree.

Frequently asked questions