Is the net effect of health care zero?
UPDATE: Nyman (2007) points out the RAND study discussed in this piece had a terrible flaw that undermines its argument. See the update below.
In the 1970s, the RAND Corporation picked out 7700 people in six cities and gave half of them free health care. Those lucky ones took advantage of it (spending 30-40% more on average) and they spent it on reasonable things (as judged by medical observers), but they didn’t seem to get any healthier. As the study put it:
> For the average participant, as well as for subgroups differing in income and initial health status, no significant effects were detected on eight other measures of health status and health habits. Confidence intervals for these eight measures were sufficiently narrow to rule out all but a minimal influence, favorable or adverse, of free care for the average participant.[†](http://www.rand.org/pubs/reports/R3055/)
The only exceptions: an improvement in vision (not too surprising that free glasses will help people see better) and an improvement in blood pressure. But for the latter, critics point out that you’re statistically likely to see improvement in _one_ metric in a sample of this size, even if none of the metrics actually improve.
The RAND study was by far the biggest study of this kind, but other studies find similar results. [One analysis][mc] found that regions whose Medicare programs give out more money (when the underlying healthiness of the residents is held constant) see no increase in survival rates. [A replication][va] found the same results in VA hospitals. [Cross-national comparisons][cn] find “the impact of public spending on health is … both numerically small and statistically insignificant”. [Correlational studies][cs] find “Environmental variables are far more important than medical care.” And there are more where that came from.
[mc]: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=161308
[va]: http://www3.interscience.wiley.com/cgi-bin/abstract/113336730/ABSTRACT?CRETRY=1&SRETRY=0
[cn]: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=10509822&dopt=Citation
[cs]: http://www.jstor.org/view/0022166x/ap010014/01a00020/0
There are two possible explanations. One is that, as Robin Hanson puts it, medicine is a scandal. “[T]he medical research literature must suffer from severe biases, such as fraud, funding bias, treatment selection bias, publication selection bias, leaky placebo effects, misapplied statistics, and so on. How else can we square the usual positive benefit found in medical publications with a net zero benefit?”
The other is that each individual treatment is effective on the particular grounds measured over the particular time period investigated, but that this only leaves people open to other health problems. Drug-coated stents, for example, are effective at opening blocked heart arteries, but they appear to also cause blood clots.[†][bc]
(Addendum by pde: there is an important third possibility, which is that the 30-40% of extra spending amongst the group that got free health cover simply didn’t result in any better treatment: they might have been charged more, given more hand-holding, and more treatments of unclear benefit, while the control group were getting equally good/bad access to the treatments that are extremely beneficial, such as early excision of melanoma or insulin for type 1 diabetes. It may be that social and cultural factors, rather than ability to pay, determine who gets access to good health care.)
[bc]: http://www.nytimes.com/2006/12/13/business/13leonhardt.html?ex=1179374400&en=1168571be8540ecf&ei=5070
UPDATE: Nyman (2007) points out that one reason this could be the case is that people in the subgroup that had to pay for their health care could voluntarily leave the study if they were sick, returning to their previous insurance regime where they may not have had to pay as much for treatment. And, indeed, he finds that 16 times as many people voluntarily left the pay subgroup as the free subgroup. This would seem to severely throw these findings into question.
As [Richard Lewontin argued many years ago][rl]:
> What is the evidence for the benefits of modern scientific medicine? Certainly we live a great deal longer than our ancestors. [... But a] very large fraction of the change [...] is a tremendous reduction in infant mortality. [... I]n 1860, the infant mortality rate in the U.S. was 13 percent–so the average life expectancy for the population as a whole was reduced considerably by this early death. The gravestones of people who died in the middle of the nineteenth century indicate a remarkable number of deaths at an old age. In fact, scientific medicine has done little to add years for people who have already reached their maturity. In the last 50 years, only four months have been added to the expected life span of a person who is already 60 years old.
>
> [...] As far as we can tell, the decrease in death rates from the infectious killers of the nineteenth century is a consequence of the general improvement in nutrition and is related to an increase in the real wage. In countries like Brazil today, infant mortality rises and falls with decreases and increases in the minimum wage. (42ff)
[rl]: http://books.theinfo.org/go/0060975199
Of course, universal health care is a lot more politically palatable than a universal living wage.
_(Thanks to jmc for suggesting this piece and to Robin Hanson and his site [Overcoming Bias](http://robinhanson.typepad.com/overcomingbias/) for collecting so many resources on it. Hanson is [starting a petition][p] to request a larger version of the RAND study.)_
[p]: http://www.overcomingbias.com/2007/05/rand_experiment.html
pde May 15
The crucial thing to include in a large replication of this study is a way to track the use and effects of the treatments that medicine is most confident about, within the two groups. Do even the uninsured receive them? Do other apparently unrelated health problems show up amongst those who receive them? Do net-harmful treatments simply counteract their benefits in a population?
Toby May 16
I think that Robin is stating the case far too strongly, and causing a bit of confusion over at Overcoming Bias, which has perhaps rubbed off on Aaron.
What the experiment would show is that additional health care is worth a negligible amount at the margins. Or, more precisely, that additional health care in the american system in the 1970s is worth a negligible amount at the margin. The first qualifier ‘at the margin’ is amazingly strong, as it is compatible with the claims of other economists that health improvements have been worth as much to us since the 1800’s as all other improvements combined. It may just be that we get all of them in the amount that people are willing to pay themselves for health care (and we would certainly expect to get much more than half the benefits in the first half of the money spent, as we go for efficiency first).
The qualifier about america and about the 1970s also seem quite strong. The US has considerably worse health statistics than many first world countries despite spending over twice as much as the next competitor. It isn’t surprising that they are not very efficient at their margin given that everyone else stopped spending far earlier.
Remember too, that RAND is not the most trustworthy source in general. I’m pretty sure they campaigned to nuke the USSR pre-emptively before they got their own nukes, for example. This example is of ethics rather than data collation, but we are right to be suspicious of this infamously politicized source.
Even if it showed everything it wanted to, there is also the redistributive benefit of public health care (it distributes wealth to the poor), though if really inefficient at the margins, this could be better done by a direct payment to everyone. However, scrapping the public spending in the US health system is unlikely to be accompanied by a compensating redistributive benefit, so I think it may be for the worse even if health care dollars are worth nothing at the margin (in the US, today).
This is not to say that the RAND experiment isn’t interesting. It is. However it must be taken very carefully.
Andrew Gelman Jun 12
Are you confusing total and marginal effects? See comment here:
http://www.stat.columbia.edu/~cook/movabletype/archives/2007/06/total_vs_margin.html
Robin Hanson Jun 12
Pde, yes, a repeat of the RAND Experiment should attend more to results conditional on treatment quality measures.
Toby, I distinguished in my comments between margin and average effects. We know less about the average than the margin, but that shouldn’t be much cause for encouragement about medicine.
Douglas Knight Jun 17
Here is another factor to explain the discrepancy between positive studies of individual treatments and neutral studies of net effects: the treatments have potential, but only if they’re done correctly. Doctors in studies may be more competent or may devote more attention because they know they’re being studied.
I think the first two possibilities are important, too, but pde’s third possibility (which is quite close to the marginal v average issue) seems to have been addressed by RAND’s methodology: they had doctors assess the treatment decisions after the fact and they didn’t think the expensive group’s treatment any less useful. My impression is that it’s hard to distinguish between the two sets of care: the extra care isn’t just hand-holding or poorly-practiced new techniques.
I guess it’s possible that institutions do a better job of judging treatment than doctors, but
I don’t think that’s a very useful hypothesis until specialized by a mechanism. A marginal benefit of zero is such a mechanism. I’d be happy to entertain others, but that’s the only one I see.
Toby:
Comparison with Europe, like the RAND study, suggests a 30% or more cut is possible. Most people call that infra-marginal. I’d also like to point out Singapore, which spends half as much as Europe (as a percent of GDP).
If you don’t trust RAND, try practically any other attempt to study this. The RAND study is always quoted because it was the biggest and best designed, not because it is unique.
Amy Chapman Jun 21
Chronically ill patients who receive the most intensive, aggressive, and expensive treatments fare no better than those who receive more conservative care. In fact, their outcomes are often worse. People who live in rural areas with fewer doctors live longer and healthier lives. These are facts established by strong studies and published in JAMA. Hospitals are dangerous places. Invasive tests of the kind showcased on programs like “House”, often are unnecessary and dangerous in the extreme. Often there are simple alternatives to those tests but they require tools like a stethoscope and don’t lead to a nice fat bill, so they aren’t tried. I have a friend who was in an auto accident in May. She was shaken but had no broken bones or lacerations. She was taken to a trauma center and released at the end of the day after numerous tests. The bill? Twenty-two thousand dollars. She earns 18,000 a year.
Isabelle Sep 12
It starts to seem like we really are a nation of enablers, albeit enablers who charge a high price. Although universal health care is not much better. I have a friend who lives in Canada who runs to the ER whenever she has a panic attack, however minor. She knows she will not be billed and will stand a good chance of getting her reassurance fix.
Pozycjonowanie Nov 26
[…] As far as we can tell, the decrease in death rates from the infectious killers of the nineteenth century is a consequence of the general improvement in nutrition and