VA Center for Clinical Management Research
Benchmarking Hospital Quality
Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches
Hallie Prescott, MD, MSc (PI)
Tim Hofer, MD, MSc
Jeremy Sussman, MD, MS
Jack Iwashyna, MD, PhD
Sanjay Saint, MD, MPH
Wyndy Wiitala, PhD
Jane Forman, ScD, MHS
Kate Luginbill, MPH
Brenda Vincent, MPH
Jenny Burns, MHSA
Daniel Molling, MS
3/1/2018 – 8/31/2020
VA Office of Reporting, Analytics, Performance Improvement, and Deployment (RAPID); VA Inpatient Evaluation Center (IPEC)
Background:Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, comparing hospitals to identify under-performing hospitals is difficult because of inherent differences in patient case-mix and illness severity across hospitals (i.e. some hospitals take care of sicker patients, while other hospitals take care of older patients).
Clinicians consider the current approach to evaluating hospitals using regression methods, which adjust for differences in patients across hospitals, to be unfair, unclear, and unhelpful. Therefore, while substantial resources are devoted to hospital evaluations, the current return on this investment is limited because clinicians do not understand or trust the methods.
The National Academy of Medicine has recently called for investing in the science of hospital performance measurement and for increasing its transparency and validity. “Template matching” has been proposed as an alternative methodological approach for hospital evaluation that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings.
Objectives: In this IIR, we are testing the usefulness of template matching for comparing quality of care across VA’s diverse acute care hospitals—in terms of feasibility, accuracy, and interpretability. Specifically, the project will: (Aim1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (Aim2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (Aim3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. We expect that template matching will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons.