Journal of Health Services Research & Policy > Volume 12, Number 3 > Pp. 173-180

This Article Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Lilford, R. Articles by Nicholl, J. Search for Related Content PubMed PubMed Citation Social Bookmarking                      What's this?

Review

Inter-rater reliability of case-note audit: a systematic review

Richard Lilford, Alex Edwards, Alan Girling, Timothy Hofer, Gian Luca Di Tanna, Jane Petty, Jon Nicholl

Department of Public Health & Epidemiology, University of Birmingham, Birmingham, UK; Department of Public Health & Epidemiology, University of Birmingham, Birmingham, UK; Department of Public Health & Epidemiology, University of Birmingham, Birmingham, UK; Department of Internal Medicine, University of Michigan Medical School, Ann Arbor, MI, USA; Department of Public Health & Epidemiology, University of Birmingham, Birmingham, UK; School of Psychology, University of Birmingham, UK; Medical Care Research Unit, University of Sheffield, Sheffield, UK

Objective: The quality of clinical care is often assessed by retrospective examination of case-notes (charts, medical records). Our objective was to determine the inter-rater reliability of case-note audit.

Methods: We conducted a systematic review of the inter-rater reliability of case-note audit. Analysis was restricted to 26 papers reporting comparisons of two or three raters making independent judgements about the quality of care.

Results: Sixty-six separate comparisons were possible, since some papers reported more than one measurement of reliability. Mean kappa values ranged from 0.32 to 0.70. These may be inflated due to publication bias. Measured reliabilities were found to be higher for case-note reviews based on explicit, as opposed to implicit, criteria and for reviews that focused on outcome (including adverse effects) rather than process errors. We found an association between kappa and the prevalence of errors (poor quality care), suggesting alternatives such as tetrachoric and polychoric correlation coefficients be considered to assess inter-rater reliability.

Conclusions: Comparative studies should take into account the relationship between kappa and the prevalence of the events being measured.

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

				R. Lilford The English Patient Safety Research Programme: a commissioner's tale J Health Serv Res Policy, January 1, 2010; 15(suppl_1): 1 - 3. [Full Text] [PDF]

				C. Brown and R. Lilford Evaluating service delivery interventions to enhance patient safety BMJ, December 17, 2008; 337(dec17_1): a2764 - a2764. [Full Text]

				C Brown, T Hofer, A Johal, R Thomson, J Nicholl, B D Franklin, and R J Lilford An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement Qual. Saf. Health Care, June 1, 2008; 17(3): 170 - 177. [Abstract] [Full Text] [PDF]