Brief Research Report
The Quality of Medical Record Review Studies in the International Emergency Medicine Literature

https://doi.org/10.1016/j.annemergmed.2004.11.011Get rights and content

Study objective

We assess the methodologic quality of studies using medical record review methodology in 4 international emergency medicine journals. A secondary aim was to compare methodology quality among these journals and across years.

Methods

This was an observational study of articles whose main methodology was medical record review published in Academic Emergency Medicine (AEM), Annals of Emergency Medicine (Annals), Emergency Medicine Journal (EMJ), and Emergency Medicine Australasia (EMA) between January 2002 and May 2004. Eligible articles were reviewed for reporting of a clear hypothesis or objective, training of abstractors, defined inclusion and exclusion criteria, use of a standard abstraction form, definition of important variables, monitoring of abstractor performance, blinding of abstractors to study hypothesis, reporting of interrater reliability, sample size or power calculation, reporting of ethics approval or waiver, and disclosure of funding source. The primary outcome was the proportion of articles meeting each criterion. Secondary outcomes were comparison of the proportions of articles meeting each criterion among journals and by years.

Results

One hundred seven articles were analyzed; 31 were published in AEM, 29 in Annals, 29 in EMJ, and 18 in EMA. A clear aim was reported in 93% of articles, standardized abstraction forms were reported in 51%, interrater reliability was reported in 25%, ethics approval or waiver was reported in 68%, and sample size or power calculation was reported in 10%.

Conclusion

Adherence to the quality criteria for medical record reviews was suboptimal, and there were significant differences among journals in overall methodologic quality.

Introduction

Documentation of information in medical records is of variable quality. However, many emergency medicine research studies rely on information extracted from routine medical records. It has been reported that up to 25% of published emergency medicine studies use this methodology.1

In 1996, Gilbert et al1 reported an analysis of medical record reviews published in 3 emergency medicine journals between 1989 and 1993 against methodologic quality indicators. Results were disappointing, particularly with respect to reporting of use of standardized abstraction forms and testing of interrater reliability. Recently, Worster et al,2 in an analysis of 20 articles each from 3 emergency medicine journals, reported minimal improvement in the quality of reporting of study methods. Both of these studies, however, were confined to emergency medicine journals published in North America.

The aim of this study was to assess the quality of methodologic reporting for studies using medical record review methodology in 4 international emergency medicine journals. A secondary aim was to compare methodology quality among these journals and across years.

Section snippets

Study Design

This was an observational study of manuscripts using medical record review as their main methodology published in Academic Emergency Medicine (AEM) and Annals of Emergency Medicine (Annals), published in North America, Emergency Medicine Journal (EMJ), published in the United Kingdom, and Emergency Medicine Australasia (formerly Emergency Medicine) (EMA), published in Australia. These were chosen to give broad international representation.

Data Collection and Processing

Two researchers (DK, AMK) independently searched issues

Results

One hundred fourteen articles were considered for inclusion. Researchers agreed without discussion that 100 were eligible, 7 were included after consensus discussion, and 7 were excluded after consensus discussion, resulting in a sample of 107. Thirty-one were published in AEM, 29 in Annals, 29 in EMJ, and 18 in EMA. Thirty-four were published in 2002, 51 in 2003, and 22 in 2004.

Interrater agreement was 65% for definition of inclusion and exclusion criteria, 79% for use of a standard

Limitations

There are some limitations that should be considered when our results are interpreted. We looked at only 4 English-language journals. Our results may not be generalizable to other journals that may have different submission requirements or publish articles from different research cultures. Authors may have included information about particular criteria in previous drafts, cover letters, or submission forms, but it may not have been published. The abstractors in this study were not blinded to

Discussion

Medical records are intended to document a clinical patient encounter. In truth, they are interpretations of clinical scenarios recorded by different observers who choose to record what they think is relevant or important. Missing data are common. Medical records are commonly in free-text format and often written by hand, adding the problems of legibility and interpretation.3 Despite these weaknesses, medical record review studies may be an appropriate method as pilot studies to inform planning

References (4)

There are more references available in the full text version of this article.

Cited by (0)

Author contributions: AMK and DB conceived the study. AMK designed the data collection instrument. AMK and DK identified qualifying papers. DK, DB, and TR performed the data collection and data entry. AMK and DK performed the data analysis. AMK and DB wrote the draft manuscript. All authors contributed to data interpretation and the final manuscript. AMK takes responsibility for the paper as a whole.

Funding and support: The authors report this study did not receive any outside funding or support.

Reprints not available from the authors.

View full text