Racz and Sedransk (12) modeled risk-adjusted assessments utilizing Bayesian and frequentist indirect standardization methods, which they compared to the RAMR for "provider profiling." These methodologies produced very similar results although they found markedly fewer outlying hospitals when the random-effects assumption was applied to the hospitals.
In this paper, we will demonstrate that the RAMR is intrinsically a highly flawed performance measure and that, moreover, it is applied improperly.
Since many RAMR are zero, we added a small positive fraction (0.25) to all observed deaths to avoid having a zero variance term in the ztest calculation.
Figure SA1: Relationship between Overall Panel Rankings by RAMR and by Z Score.
We compared the average panel RAMR for these two groups of physicians: those with at least one black patient referred to CABG and those without any black CABG patients.
We first present the average panel RAMR for the black referrals and the average panel RAMR for white referrals and test for the difference between them using paired t-tests.
The report includes for each hospital and for each surgeon the number of cases, the RAMR, and a designation of outlier status based on a 95 percent confidence interval around the statewide average RAMR.
The RAMR is defined in the NYS report as the state average mortality rate multiplied by the ratio of a surgeon's observed mortality rate to the surgeon's predicted rate.
(1) Telephone interviews with and contracting data from the majority of MCOs licensed in NYS; (2) RAMR, quality outlier designation, and procedure volume for all cardiac surgeons, as reported in the Cardiac Surgery Reports.
Analyses of actual contracting patterns show that in aggregate, the hypothesis of random choice could be rejected with respect to high-quality outlier status and high procedure volume but not for RAMR or poor-quality outlier status.
Table 5 presents hospital risk-adjusted mortality rates and their ranks based on each of the eight models (note that hospitals are numbered according to model C1's RAMR rank).
After comparing the discrimination of the two models, the next step consisted of calculating risk-adjusted mortality rates (RAMRs) for each of New York's 31 hospitals in which CABG surgery is performed, and comparing RAMRs for the two models.