Four analyses, four different questions
A program office that asks for "the economics" of a health intervention will get back one of four analyses, and which one depends more on the analyst's habits than on the decision. Cost of illness tells you how big a problem is. Cost-effectiveness tells you what a unit of health gained costs. Budget impact tells you whether you can afford it. Return on investment tells you whether it pays back in money. Each is a legitimate tool, and each gives a misleading answer when it is used for one of the other three questions.
The first job of anyone commissioning health economics is to name the decision. Are you choosing between programs, defending an appropriation, setting a price, deciding whether to cover something, or arguing that a problem deserves attention at all? The answer picks the analysis. The rest of this primer explains each one, the settings that change its answer, and how to read the results without being misled.
The four analyses side by side
| Analysis | Question it answers | Main output | Typical audience |
|---|---|---|---|
| Cost of illness | How much does this condition cost society or a payer today? | Total annual cost, split into direct medical, direct non-medical and productivity costs | Advocates, priority setting, the opening paragraph of a budget justification |
| Cost-effectiveness | What does each unit of health gained cost, compared with the alternative? | Incremental cost-effectiveness ratio, usually cost per QALY gained or per DALY averted | Coverage decisions, guideline bodies, program design, choosing between options |
| Budget impact | What happens to our budget each year if we adopt this? | Year-by-year change in spending for a defined payer or program | Budget officers, payers, appropriations staff |
| Return on investment | Does this return more money than it costs, and when? | ROI ratio, net savings, payback period | Leadership, employers, appropriators asking for savings |
A fifth analysis, cost-benefit, puts a dollar value on health outcomes themselves and compares total benefits with total costs. It is common in environmental and transportation regulation, less common in health program decisions, and it raises the hardest valuation questions of all. When a program office says ROI, it sometimes means cost-benefit, and it is worth asking which.
Cost of illness tells you size, not choice
A cost-of-illness study adds up what a condition costs: treatment, care, travel, lost work and sometimes caregiver time. It is useful for establishing that a problem is large. It cannot tell you whether any particular intervention is worth funding, because it says nothing about what an intervention would change.
The common misuse is arithmetic like this: the disease costs a large sum each year, the program will reduce cases by a fifth, so the program saves a fifth of that sum. That chain assumes the cost per case is the same for prevented cases as for the average case, that the reduction applies to every cost category, and that savings arrive in the same year as spending. None of those are usually true. How cost-of-illness numbers are constructed, and why they break, is covered in our piece on cost of illness and other measurement problems.
QALYs and DALYs in plain terms
Cost-effectiveness needs a common unit of health so that programs with different outcomes can be compared. The two common units weight time by quality of health.
A quality-adjusted life year (QALY) counts years of life, each multiplied by a weight between 0 (equivalent to death) and 1 (full health). Two years lived at a weight of 0.5 count as one QALY. Programs are judged by the QALYs they gain. Weights come from population surveys using standardized instruments.
A disability-adjusted life year (DALY) counts health lost rather than health gained. It adds years of life lost to early death and years lived with disability, weighted by severity. Programs are judged by DALYs averted. DALYs are standard in global health and burden-of-disease work. QALYs are more common in United States and European coverage decisions.
Know whether your program's governing statute restricts their use. The Affordable Care Act, for example, bars the Patient-Centered Outcomes Research Institute from using a dollars-per-QALY threshold. Where QALYs are restricted, analysts report alternatives such as cost per life year gained, cost per case averted or an equal-value life year measure, and the program office should know in advance which one it can use.
Perspective decides which costs count
Perspective is whose costs and benefits the analysis includes. It is the setting most likely to change the answer, and the one most often left implicit.
Societal perspective counts everything: health care costs regardless of who pays, patient time and travel, out-of-pocket spending, caregiver time, productivity and effects on other sectors such as education or criminal justice. Health care sector or payer perspective counts only medical costs, sometimes only those borne by one payer. Government or program perspective counts what a specific budget pays and saves, which can exclude savings that land in another agency's budget.
A home visiting program can look expensive from a health payer's perspective and good value from a societal one, because much of its benefit appears in schools and labor markets. Neither is wrong. They answer different questions. The Second Panel on Cost-Effectiveness in Health and Medicine recommended reporting both a health care sector and a societal reference case, with an inventory listing which effects each includes. Ask for both, and ask for the inventory.
Discounting and the time horizon
Costs and health effects that occur in the future are discounted, meaning reduced to present value, because people and governments prefer benefits sooner and money spent now could be invested. United States analyses following the Second Panel commonly use 3 percent a year for both costs and health effects. England's NICE uses 3.5 percent. At 3 percent, a $1,000 saving twenty years from now is worth about $554 today.
Discounting hits prevention hardest. A vaccination or screening program spends money now and gains health years or decades later, so the higher the rate, the worse prevention looks. That is not a flaw to be engineered away. It is a real feature of the comparison. But it means the discount rate should be fixed before results are seen, and results at 0 and 5 percent should be reported as sensitivity checks.
The time horizon should be long enough to capture every meaningful difference between the options. For a chronic disease intervention, that often means a lifetime horizon. A two-year horizon on a prevention program captures the costs and misses most of the benefit, and the result is predictable before the model is built.
How to read an incremental cost-effectiveness ratio
The incremental cost-effectiveness ratio, or ICER, is the extra cost of an option divided by the extra health it produces, both compared with the next best alternative. The word incremental is the whole point. An ICER is never cost divided by effect for one option alone.
Four outcomes are possible when a new option is compared with the current one. It can cost less and produce more health, in which case it dominates and there is no ratio to compute. It can cost more and produce less, in which case it is dominated and should be dropped. Or it can cost more and produce more, or cost less and produce less, and the ICER tells you the price of the trade.
Three options for a screening program
Suppose the status quo costs $6.0 million and produces 2,400 QALYs across the target population. Option A costs $7.5 million and produces 2,450 QALYs. Option B costs $9.0 million and produces 2,470 QALYs. These figures are invented for illustration and describe no real program.
Option A compared with the status quo: $1.5 million extra for 50 extra QALYs, an ICER of $30,000 per QALY. Option B compared with the status quo: $3.0 million for 70 QALYs, about $42,900 per QALY. That second number is the trap. The right comparison for B is against A, the next best option: $1.5 million more for 20 more QALYs, an ICER of $75,000 per QALY. Moving from A to B buys health at more than twice the price of moving to A in the first place.
When several options are compared, rank them by cost, remove any that are dominated, then remove any that are extendedly dominated, meaning a combination of the options on either side would produce more health at the same cost. Compute each ICER against the next remaining option. The decision then depends on how much the decision-maker is willing to pay for a unit of health, and good analyses report results against several willingness-to-pay thresholds rather than one.
Sensitivity analysis tells you how much to trust it
Every input to a cost-effectiveness model is uncertain. Sensitivity analysis shows how the result moves when the inputs do.
One-way sensitivity analysis varies one input at a time across a plausible range while holding the rest fixed. The results are usually shown as a tornado diagram, with the input that moves the ICER most at the top. Threshold analysis asks how far an input would have to move for the decision to change. It is often the most useful single output for a decision-maker. Scenario analysis changes several related inputs together to represent a coherent alternative world, such as lower uptake combined with a shorter duration of effect.
Probabilistic sensitivity analysis assigns a probability distribution to every uncertain input and runs the model thousands of times, drawing each input from its distribution. The output is a cloud of results and a cost-effectiveness acceptability curve, which shows the probability that each option is the best choice at each willingness-to-pay threshold. If an option is preferred at your threshold in only half the simulations, the point estimate is hiding a coin flip.
Budget impact asks a different question
An intervention can be good value and still unaffordable. A cost-effectiveness analysis that finds a reasonable cost per QALY says nothing about whether this year's appropriation can carry it. Budget impact analysis answers that question directly.
It estimates the eligible population, the uptake over the first few years, the cost of the new intervention, the costs it displaces and the offsets that arrive within the period, then reports the net change in spending year by year. Budget impact is usually reported for one to five years from a single payer's perspective and is commonly left undiscounted, because a budget officer wants to know what the spending will be in each year, not its present value. It should always show the uptake ramp, because the answer in year one and year three can differ by a large multiple.
Return on investment and its traps
ROI is the analysis leadership asks for most and the one most often misread. It counts only benefits that can be expressed in money, compares them with cost, and reports a ratio or a payback period. When a program truly saves money, ROI makes the case cleanly. When it does not, ROI makes a good program look like a bad one.
Most health interventions improve health at a cost rather than saving money. A widely cited 2008 analysis in the New England Journal of Medicine found that while some preventive measures save money, most add costs, and many of those are still good value. A program office that commits to proving savings has set a bar that much worthwhile work cannot clear.
Two further traps. Savings often land in someone else's budget: a program funded by a public health agency may reduce hospital spending paid by Medicaid or private insurers. And projected savings are often not cashable. A reduced emergency department visit count does not reduce the hospital's staffing cost in the same year. Ask the analyst to separate savings to your budget, savings to other budgets and savings that will not appear as cash.
What to ask an analyst for
A short, specific request produces a better analysis than a long, general one. Before any modeling starts, agree on the following in writing.
- The decision the analysis supports, and the options being compared, including the status quo
- The analysis type, chosen for that decision
- The perspective, with an impact inventory listing which costs and effects are counted
- The time horizon and discount rate, fixed before results are seen
- The health outcome measure, and whether your statute restricts any of them
- One-way, threshold and probabilistic sensitivity analysis, with the threshold analysis reported up front
- A source for every input, with the year and population it came from
- A model file your own staff or a reviewer can open and rerun
The last item matters more than it looks. A model delivered only as a report cannot be updated when an input changes, and it cannot be checked. Ask for the working model, documented well enough that someone other than its author can run it.
Common misuses
Comparing ICERs from different studies as if they were a league table
Two ICERs from studies with different perspectives, discount rates, comparators and years are not comparable, even for the same disease. Ranking them side by side produces a list that looks precise and means little. Compare only within an analysis that held the settings constant.
Reporting an average cost-effectiveness ratio
Total cost divided by total effect for one option, compared against doing nothing, hides the incremental step that matters. The worked example above shows how an option that looks reasonable on average can be poor value compared with the next best choice.
Choosing the perspective after seeing the results
Switching from payer to societal perspective because the payer result was disappointing is a form of selective reporting, and experienced reviewers recognize it. Fix the perspective in advance and report the other one as a secondary analysis.
Treating a cost-of-illness figure as the size of the prize
The total cost of a condition is not what an intervention will save. Savings depend on which cases are prevented, which costs those cases would have incurred and when. Use cost of illness to establish scale, and cost-effectiveness or budget impact to support a choice.
Frequently asked questions
Cost-effectiveness reports what a unit of health gained costs compared with the alternative, usually per QALY or DALY, and accepts that good programs often cost money. Return on investment counts only monetary benefits and asks whether they exceed the cost. A program can be excellent value in cost-effectiveness terms and still have a negative ROI.
Report two: the perspective of the budget making the decision, and a societal perspective that shows the full picture. The first tells you what your program will spend and save. The second shows effects that land in other budgets or with patients and families, which often matter to appropriators and the public.
Prevention spends money now and produces health and savings later. Discounting reduces the present value of those future benefits, and short time horizons cut them off entirely. Use a horizon long enough to capture the benefits and report results at several discount rates.
It depends on whether a usable published model exists and how much new data is needed. Adapting an existing model with current inputs takes weeks. Building a new model with original data collection takes months. Agree on the decision, comparators and settings first, because changing them midway is the most common cause of delay.