Study asserts Hospital Compare errs on AMI mortality amounts at certain facilities

The Hospital Compare website, operated by the CMS (Centers for Medicare and Medicaid Services), is meant to assist people learn about the quality of hospitals, but a recent study emphasizes that the statistical methodology utilized by Hospital Compare underestimates heart attack or Acute Myocardial Infarction or AMI mortality amounts for small hospitals.

“The underestimation of AMI mortality amounts at small hospitals, as seen in Hospital Compare, contradicts initially established research and consistent findings that mortality rates are generally higher at low-volume hospitals,” stated Jeffrey Silber, MD, co-author of the study, professor of pediatrics at the Children's Hospital of Philadelphia and professor of health care management at The Wharton School.

The website is intended to assist sufferers and their families make decisions about providers by giving a side-by-side comparison between facilities in their place.

The study, released in the Journal of the American Statistical Association, makes the case that Hospital Compare’s statistical methodology—the random effects logit model—really shrinks mortality rates from small hospitals to resemble the national average.

“Hospital Compare’s finding of average risk at small facilities is a mistake because the current model isn’t precisely calibrated,” stated Edward George, professor of statistics at the University of Pennsylvania’s Wharton School and co-author of the study. “It is an error that has implications for sufferers.”

CMS officials weren’t immediately available for comment about the findings of the research.

Nevertheless, previous year, the agency released a statement saying it “designed the methodology to be inclusive of as many hospitals and as several measures as possible,” which “stops the methodology from limiting star rating calculations to certain types of hospitals based on characteristic or size.” Although, at the same time, CMS noted that it’ll “continue to re-evaluate and make any needed modifications to the methodology over time.”

“As a model for AMI hospital mortality amounts, we’ve discovered the hierarchical random effect logit model used by Hospital Compare to be inadequate, compared to alternatives that model hospital effects as a functions of hospital attributes,” summarize the authors, who add that “sufferers deserve to have the most accurate information available so they can make well-informed healthcare decisions.”