Going Beyond the “Dartmouth Debate” to the Most Important Question: Why Are Outcomes at Some Hospitals So Much Better Than At Others?

We all have heard that “spending more” on health care does not necessarily lead to better care. In fact, in regions of the country where care is more intensive and more expensive, sometimes outcomes are worse. This is the basic thrust of what has become known as the “Dartmouth research,” and most medical researchers agree.

But a paper just published in the Annals of Internal Medicine suggests that specific types of higher hospital spending may lead to better outcomes. After examining the records of some 2.5 million patients admitted to 208 California hospitals from 1999 to 2008 a group of researchers from the University of Southern California and Harvard Medical School report that patients who received more costly and aggressive care were less likely to die while in the hospital.

Let me be clear: this study is not trying to prove that the Dartmouth research is “wrong." The investigators, led by John Romley of the Leonard D. Schaeffer Center for Health Policy and Economics at the University of Southern California, begin by acknowledging that “a convincing set of studies demonstrates that U.S. regions that spend more on medical care–using more specialists, diagnostic tests, imaging, and inpatient hospital care–have similar or poorer patient outcomes than areas that spend less. (Here they footnote the Dartmouth Atlas and this article by Dartmouth’s Elliott Fisher, et.al. 

                The Most Important Question

But this group set out to take a step beyond the Dartmouth research, examining the murky relationship between money and outcomes in more detail. While granting that Dartmouth’s investigation “of total medical spending across regions provides an informative overall picture,” they suggest that for certain patients “medical spending may lead to better patient outcomes in some circumstances (for example, inpatient hospital spending) but not in others.”

This is why Dartmouth economist Jon Skinner, is intrigued by their work: “We need to get beyond the simple ‘Is more better?’ question,” Skinner explains. “Finding that high-spending regions don’t necessarily achieve better outcomes is old news. Finding that Mayo or Intermountain get good results at low cost is also old news.” What we don’t know is why some hospitals are more successful. And this new article looking at inpatient spending and outcomes is a good first step in that direction.

                     What the Data Shows

Zeroing in on patients suffering from six common conditions—acute myocardial infarction; acute stroke, congestive heart failure; gastrointestinal hemorrhage; hip fracture and pneumonia—Romley, et. al., found “an association between high hospital spending and fewer inpatient deaths.”

What is striking is how widely spending varied across hospitals treating similar patients. For instance, between 2004 and 2008, the amount spent on patients who suffered from heart attacks was about $5,168 at the least expensive hospitals, while the total bill topped $21,000 at the costliest institutions. (Total spending included physician visits, hospital room charges, laboratory testing, diagnostic imaging, medication administration and procedures.) The tab for GI hemorrhages ran from roughly $5,000 to well over $21,000. (These figures reflect the cost of caring for patients who ultimately died in the hospital.)

The range of outcomes is almost as striking. After adjusting for risk factors such as age, socio-economic factors and co-morbidities, investigators found that patients who had suffered heart attacks and were admitted from 1999 and 2003 were 14% less likely to die in the more expensive hospitals; those who entered the hospital between 2004 and 2006 were 26% less likely to die before being discharged. Pneumonia patients who received costlier care were 27% less likely to die. Extra spending was not always associated with better patient outcomes. Patients admitted with hip fractures, and GI hemorrhages between 1999 and 2003 showed no benefit, but between 2004 and 2008, bigger outlays appeared to make a difference.

                Beware of the Headlines

It is all too easy to simplify these findings, especially in a headline. “High Cost California Hospitals Have Lower Mortalityannounces one blog. “More Hospital Spending May Save More Livesreports another. Even the L.A. Times could not resist telling readers what they want to hear: “Money Can Buy Better Health, Study Finds” The story begins by suggesting that the link between money and survival is clear-cut: “When it comes to heart attacks, strokes and some other types of medical emergencies that could land you in the hospital, you get what you pay for.” If you’re rich, you’re safe.

But read a little further, and the L.A. Times reporter acknowledges that while the researchers believe that “their results may show that certain kinds of acute-care spending are worthwhile, unfortunately, there weren't enough data to identify the specific interventions that made the difference…they speculated that the list would include ‘early and more frequent coronary revascularization’ for heart attack patients and ‘upper and lower endoscopies’ for patients with GI hemorrhages.” But they don’t really know.

In fact, Romley, et. al. don’t go quite as far as the story suggests: “Although our analysis demonstrates that intensive spending by hospitals is associated with lower mortality, it does not identify the specific costly interventions that high spending hospitals undertake to achieve this mortality benefit,” they write. “Prior work suggests that patients at moderate and high risk for death have lower mortality when admitted to hospitals in which larger proportions of patients spend time in the intensive care unit, undergo mechanical ventilation, or receive dialysis.”

But, they emphasize: “These interventions are probably only proxies for the additional costly diagnostic work-up and management that higher-spending hospitals may do.” In other words, ICU care, and the use of mechanical ventilation may only be “markers” for the care that led to survival, not the cause. (All medical research is limited in some ways; these investigators deserve a hat-tip for making what they do know—and what they don’t know—so clear.)

Moreover, the researchers claim only an “association” between higher spending and fewer inpatient deaths—this does not imply causality. As the Annal’s editors point out in a blue box near the top of the story: “This observational study cannot establish cause and effect.”

In other words, the fact that patients underwent more procedures or saw more specialists may or may not have led to fewer deaths.

Perhaps the more costly hospitals are simply better-staffed, and the nurse-patient ratio explains much of the difference. When more nurses are watching patients they are likely to spot complications sooner. And if the floor is fully staffed, nurses who are not trying to do the jobs of two people are less likely to make mistakes. Is it also possible that more “attending” physicians are supervising residents at hospitals that charge more per day. This too, could explain fewer inpatient mortalities.

In Health Services Research, Skinner and Elliott Fisher reply to another 2010 study which links more spending with better outcomes thirty days after surgery. There, they write: “We know that spending for specific clinical treatments appears to be associated with better outcomes—whether beta-blockers, aspirin, and reperfusion for heart attack patients” (Skinner and Staiger 2009), a greater supply of neonatologists in regions with low supply (Goodman et al. 2002), or intensive care for tourists . . . about whom little is known . . . admitted to emergency rooms in Florida (Doyle 2010). “These studies point to plausible clinical mechanisms by which more spending could yield benefits,” they add.

But, Fisher and Skinner write: “earlier research …suggests that additional spending devoted to treating patients with chronic disease in the hospital as opposed to the outpatient setting does not offer benefits. A recent study finds that hospitals with intensive end-of-life treatment patterns are also far more likely to use feeding tubes in patients with advanced dementia——a practice that is both expensive and clinically inappropriate.”(Teno et al. 2010).

They conclude: “For policy makers, the media, and even for some investigators, there is a temptation to over-interpret studies relating to spending and outcomes. Some might hope to conclude from some of our studies that to get better quality, we need only cut spending or reimbursement rates in high-cost regions or health systems. Others interpret positive associations between intensity and mortality as causal and conclude that lower intensity would cause higher mortality. We disagree with these interpretations.” Ultimately, Skinner speculates that “better outcomes probably have less to do with how much you spend than how you spend it.” Spending money to ensure coordinated care in the hospital will yield benefits to patients; spending more money on feeding tubes for patients with advanced dementia (as in Teno’s study above) will not.

Fisher agrees that “how you spend it” is key. “The papers that we published in the Annals of Internal Medicine show that in the high-cost hospitals, the money is often spent on frequent readmissions, seeing more specialists, and the attendant tests. We would rather avoid frequent readmissions. On the other hand, appropriate use of the ICU can save lives.

What Happened to Patients After They Left the Hospital?

Finally, it is important to keep in mind that the recent study done by Romley, et. al., only tracks how many patients survived while still in the hospital. What happened to them after they were discharged?

Again, the researchers are unusually candid: “Our findings should be interpreted with caution. The cost-effectiveness of hospital spending depends on its effect on inpatient and post discharge mortality, the latter of which we could not assess with our data. Hence, important questions about the efficacy and value of hospital care remain to be asked and answered.”

How did the patients who received the most intensive treatments fare after leaving the hospital? Presumably some went home; some landed in a nursing home; some found safe haven in a hospice. How many died two weeks or a month after discharge? There is a danger that, rather than prolonging life, in many cases, the high-intensity hospitals only prolonged the process of dying. Perhaps the money would have been better spent—and patients would have suffered less—if dollars invested in more hi-tech care had been spent on palliative care.

We know that, in some cases, more aggressive care helps patients survive just a little bit longer, long enough to make it home, where most patients prefer to die. As a 2010 study by Amber Barnato et. al. published in Medical Care suggests, the effects of intensive end of life care may be “fleeting.” After studying the records of more than a million patients in 169 Pennsylvania acute care hospitals, Barnato’s team concluded that, “Admission to higher end-of-life treatment intensity hospitals is associated with small gains in post admission survival. The marginal returns to intensity diminish for admission to hospitals [that offer] above average end-of-life treatment intensity and wane with time.” 

In other cases, patients are transferred to a nursing home when the hospital’s doctors realize that they have done all that they can do. It would be misleading to suggest that, for these patients, more spending improved outcomes. Many will never return to their normal lives; they simply will be warehoused in nursing homes. On the other hand, is it safe to assume that some patients discharged from the costliest institutions went home to play with their grandchildren and enjoyed another ten years of high-quality life? We just don’t have the numbers.

In the end, everyone agrees, more work needs to be done. Where should we start? “Step one is better clinical data to understand where the extra money is being spent, and what high-performance hospitals do differently,” Skinner commented. “But we also need to know what happens to patients after they leave the hospital – are they being seen by their primary care physician? Are they getting the rehab services they need?”

Ideally, we want to learn more about how to achieve the win/win that health care reformers aim for: higher quality care at a lower cost.

13 thoughts on “Going Beyond the “Dartmouth Debate” to the Most Important Question: Why Are Outcomes at Some Hospitals So Much Better Than At Others?

  1. Maggie, this is a GREAT post. I read your summary of the paper’s findings with great interest. I’m going to have to get a copy through work to read the whole thing.
    The question of what happens after the patient leaves the hospital is a pertinent one. The question of palliative care is a pertinent one.
    I remember caring for nursing home patients with dementia who had feeding tubes back in the 80’s. I remember watching as these non-verbal, immobile, incontinent patients balloon in size while on these feedings. Their suffering was crystal clear. It was a relief when they died, even while it was sad.
    I remember when my father had open heart surgery. I remember when I went to see him in ICU immediately after the surgery. He opened his eyes and gave me this pleading look as he struggled with the ET tube. He later told me, “Never again.” It broke my heart.
    Two days after his discharge, he had a syncopal episode from the metroprolol and fell on top of my mother (who has osteoporosis). I had raised the issue of his blood pressure prior to discharge, and was assured everything would be fine. It wasn’t. We simply got very, very lucky that there was no bad outcome from all of this.
    More money, more technology has its limits. We need to look at the human factor in terms of spending and outcomes.

  2. researchers are appropriately insatiable, but one might ask whether it is a wiser investment to implement what we already know — about check lists, low-dose aspirin for cardiac cases, etc.– rather than pushing the frontiers of knowledge to learn more that will then be ignored by many, if not most providers. it is true that there’s a whole lot we don’t know. but equally obvious is that a lot of what we know is not being practiced. in terms of making changes, the latter is apparently the more daunting task.

  3. I don’t think anyone — Dartmouth or international studies — has ever questioned that very aggressive management can and does lead to increased short term survival — hours, days, and weeks.
    The question is longer term survival and the even more complicated question of quality of life. Do the patients have more likelihood of being alive and having good quality of life in six weeks or six months after the acute event? Are they likely to be readmitted after a short time and require another round of expensive care, and then yet another until they finally die? A patient who leaves the hospital comatose and spends ten days dying in a nursing home is, technically, alive; but that outcome is probably actually worse for patients and their families than dying shortly after admission.
    Studies like this one are designed to be quick and easy by looking at hospital charts. More significant studies of outcomes after discharge are much more important although more difficult and costly to do. Without attention to those issues, this study is largely without value.

  4. Panacea, Jim Jaffe and Pat S.
    Panacea– Thank you.
    And yes, quality of life is terribly important. Our health care system tends to count “success” in terms of how long the patient lives, but we need to pay more attention to quality, especially as more of us live long enough to fall victim to dementia and Alzheimer’s.
    I agree. We know that there are things that can definitely reduce errors, save lives and improve the quality of care.
    We should invest more money in figuring out how to get doctors and hospitals to incorporate this knowledge into the way they practice medicine on a daily basis.
    The Institute for Health Care Improvment (IHI), co-founded by Don Berwick (who as you know is now director of Medicare & Medicaid) has been very successful when working with individual hospitals.
    Berwick says its largely a matter of tapping into a medical professional’s natural desire to work in a more rational environment where he or she can do a better job.
    If you show them medical evidence– and create a work environment where morale is “up” as people begin to belileve that they Can change and improve outcomes– more and more doctors and nurses will get on board.
    Financial carrots and sticks can also help get the hospital administration interested. But
    to change the culture, I really think you need to lift morale and persuade individuals that what they do will make a difference.
    I am hopeful that, under Berwick, many of Medicare’s pilot projects will manage to do this.
    Pat S.–
    Yes, the first thing I noted when I looked at this study is that it was only measuring how many people died while in the hospital.
    News and blog reports commenting on the study ig\\\\\\\\\\\\\\\\\
    Immediately, one wonders: what happened to patients after they left.
    The reserachers were working with the data set that was available.
    As you say, tracking what happened to patients 3 months after discharge is both difficult and expensive.
    I just don’t know enough to say how we should spend on that– or whether, as Jim Jaffe suggests in his comment, we
    should use that money to
    do the things that we KNOW will improve care- checklists. more hospital workers to “turn” patients so that they don’t develop bedsores (actualy the most expensive preventable hospital error (preventable in about 95% of caes–if you have the staff to do the turning . . .
    We also need to encourage relatives to speak out about intensive treatments that prolonged a loved one’s survival by days or weeks– and how much the patient suffered during that time.
    When we prolong the process of dying, we are not doing the patient a favor.
    This is an area where I think we need to change med school education.
    Death is not the enemy. Needless human suffering is the enemy. Allowing a patient a “good death” should be one goal of medicine. And, of course, the patient himself or herself should be involved in deciding what constitutes a “good death.”

  5. This is a great essay. But I think the “typical” American Hospital is so broken in so many ways that new models of the very definition of what a hospital is needs to emerge?
    The Planetree model was once promising but I have lost touch with them.
    Dr. Rick Lippin

  6. I think Pat S. hit the nail on the head. When we evaluate the cost of keeping people alive after significant medical events I’m not surprised that mortality goes down with more spending. The real evaluation needs to come later in the process when the patients are discharged. What are the downstream costs associated with more / more expensive (I think they might actually be 2 caatagories) care and is there a cost savings? Thanks for listening. –Chris

  7. Chris-
    I agree–Pat is absolutely right. The fact that this study doesn’t follow patients after they left the hospital is the elephant in the middle of the room.
    In many ways, Dartmouth’s “backward-looking studies” –which look at a group of paitents who have all died– give us a clearer picture.
    What’s interesting is that none of the stories that I read about the study picked up on this. Some reocngnized that the study couldln’t prove causality–or which part of the care might have helped.
    But they all seemed to accept the notion that the fact that the patient survived hospitalization was a major triumph.
    Even the study itself (which is generally very candid about its limits) alludes to the lack of post-discharge data only in the middle of a sentence, at the very end.
    I think doctors are simply trained to see “died or survived” as the great divide. Their job is defeat death–at least as long as the patient is their responsibilitiy.
    I don’t fault the rserachers. They were using the state of California’s data–in order to study a large pool– and there was no way these researchers had the money to track what happened to the patients after discharge.
    It probably would be possible to do this through Medicare data (or at least track 90% of them) , but it would a tedious, long process.
    That said, we really can’t talk about whether spending more or more intensive treatments (agreed, they could be separate categories) yields benefis unless we know what happened to the patients after they left the hopital–for at least 6 months.
    And “what happened” means more than “where did they wind up?” One needs to know something about their quailty of life, whether at home (a stroke victim trapped in his body, unable to talk or walk) or someone who has more or less gone back ot his former life)
    Is he in a nursing home (warehoused) or in an assisted living
    center (making friends.)?
    I’ve said this before, but as boomers age, I think we are going to beging to reocgnize that quality of life is so much more important than how long one lives.

  8. Dr. Rick & Mugeer–
    agree that our hospital model is broken.
    Recently I have been thinking that hospitals are the largest problem in the system both in in terms of squandering dollars and
    unintentionally hurting patients.
    That said, I agree with Don Berwick that most people working in most hospitals really want to help patients. . . But they are trapped in a broken system.
    And I’m afraid that most of the people who run non-rofit hospitals– the CEO, the COO, the board–have lost all sense of mission.
    Or, more accurately, they never had a commitment to healthcare as a “mission.”
    They are businessmen who see a hospital as a business, like any other.
    We need more M.D.s, MPHs, nurses and other medical professionals running hospitals.
    We need people who understand that providing healthcare is not at all like selling roller skates.
    Thank you.

  9. Thanks Maggie,
    I have long agreed with the notion that many (not all) health care professions feel trapped in broken systems.
    Some though are obscenely profiteering off the broken systems. I personally don’t view them as ” healthcare professionals”
    Dr. Rick Lippin

  10. Dr. Rick–
    I have to agree that not all health care professionals act like “professoinal>”
    Some would even tell you that medicine is a business, like any other, and that they are businessmen who, like other
    businessmen, must look out for themselves.
    But these physicians who would identify themsleves as self-intersted businessmne are a distinct minority.
    That said, ThaDoctors are not perfect; like the rest of us, they are fallible.
    Still, I think that the vast majority try to put their
    patients’ interests first.
    Though in some instances, they may be rationalizing, telling themselves that they are doing what is best for the patient when this isn’t entirely true . . .
    As I said, we’re all human. But if we did away with the perverse financial incentives in our health care system– and gave health care professionals a more rational workplace– I am quite certain that almost all would rise to the occasion.

  11. Maggie –
    Please help me to understand your emphasis on the section about this being a study that describes an association, not causation. You specifically quote the phrase: “Moreover, the researchers claim only an “association” between higher spending and fewer inpatient deaths—this does not imply causality. As the Annal’s editors point out in a blue box near the top of the story: “This observational study cannot establish cause and effect.”
    My question is: Is this a criticism of this study that explicitly does not apply to the ‘dartmouth research’ as you call it? Or is this really a level playing field, where again in yet another study the association dartmouth claims in their analyses cannot be validated by any other research group? Or do you think somehow the dartmouth research looks at causality?

  12. Epidoc-
    Dartmouth does look at causality. IF you look at the reserach you will find, for instance, explanations of how supply creates demand.
    When more beds are available, doctors quite naturally use them. In many cases, it’s not clear whether a patient suffering from congestive heart failure needs to be hospitalized, but if there are plenty of hospital beds it’s more convenient for the doctor to put the patient in the hospital where he can easily call in other doctors to consult, and where nurses will be watching her at night (and the doctor won’t be getting phone calls from anxious spouses in the middle of the night.)
    Similarly, when there are more specialists in a given town, they have more space open in their appointment book and see their patients more frequently. Again, the question as to how often a specialist needs to see a patient suffering from a chronic disease is a “grey area” –there are no hard and fast rules– so if he has more time,he will see them more often.
    We also know that in some hospitals where care is more intensive and aggressive, patients see more doctors, and at the same time, often forget to do the simplest things– i.e. give the heart attack victim an aspirin. Dartmouth researchers suggest that this may be because when so many doctors are consulting on a patient each of them thinks that someone else has done the obvious, simple thing . ..
    And the Dartmouth research has been confirmed.See, for example, Dr. Atul Gawande’s study of McAllen and El Paso http://www.newyorker.com/reporting/2009/06/01/090601fa_fact_gawande
    Long before Dartmouth began publishing its research–in 1959– Dr. Milton Roemer published
    “Roemer’s Law” which states that when there are more beds and specialists availabe they will be over-used. (“Build the beds and they will come.”)