White Rose University Consortium logo
University of Leeds logo University of Sheffield logo York University logo

How robust are hospital ranks based on composite performance measures?

Jacobs, R., Goddard, M. and Smith, P.C. (2005) How robust are hospital ranks based on composite performance measures? Medical Care, 43 (12). pp. 1177-1184. ISSN 0025-7079

Full text not available from this repository.



Composite indices of healthcare performance are an aggregation of underlying individual performance measures. They are increasingly being used to rank healthcare organizations. Although composite indicators present the “big picture” in a way that is easy to interpret, misleading conclusions may be drawn if attention is not paid to key methodological issues in their construction.


We examine variability in performance measures in the context of the construction and use of composite measures. We illustrate how variability in the underlying data and the resulting composite may undermine the robustness of performance measures in health care. We also illustrate how variation in the methodological rules applied to aggregate the individual indicators can have an important impact on composite scores.


We use data for 117 English acute hospitals to illustrate the generic methodological issues. The variance in performance measures is partitioned into “controllable” and “uncontrollable” elements. We create a composite index from the underlying performance indicators and use Monte Carlo simulations to examine the robustness of the composite.


Random variation beyond the control of organizations gives rise to considerable uncertainty in hospital scores. Composites are also sensitive to changes made to the weighting system and to the aggregation rules. Some hospitals can jump almost half of the league table as a result of subtle changes.


Great care is warranted in interpreting the results of composite performance measures. Suggestions for their future development are made.

Item Type: Article
Institution: The University of York
Academic Units: The University of York > Centre for Health Economics (York)
Depositing User: York RAE Import
Date Deposited: 02 Apr 2009 15:38
Last Modified: 02 Apr 2009 15:38
Published Version: http://dx.doi.org/10.1097/01.mlr.0000185692.72905....
Status: Published
Publisher: J B Lippincott Co
Identification Number: 10.1097/01.mlr.0000185692.72905.4a
URI: http://eprints.whiterose.ac.uk/id/eprint/6917

Actions (repository staff only: login required)