Chapter 36: Critical Appraisal: Evaluating Research Reports

Introduction

The success of evidence-based practice will depend on how well we incorporate research findings into our clinical judgments and treatment decisions. We have a responsibility to evaluate research reports to determine whether the findings provide sufficient evidence to support the effectiveness of current practices or offer alternatives that will improve patient care.

The purpose of critical appraisal is to determine the scientific merit of a research report and its applicability to clinical decision making. Whether a published paper or conference presentation, we can evaluate a study for the general purpose of understanding current information in our areas of practice, or we may be focused on its application to a particular patient problem. In the latter case, the clinician will pose a clinical question using the PICO format to direct a search and identify appropriate research reports (see Chapter 5).

Clinicians often express that one of the foremost barriers to evidence-based practice is a lack of skill in searching for and appraising the literature.^1-3 To be a critical consumer, those who seek evidence must be able to retrieve and read published studies and make judgments about the validity and relevance of the information. The search process was covered in Chapter 6. The purpose of this chapter is to take the mystique out of appraisal by providing a structured framework to guide critical evaluation of individual published studies, and to foster informed judgments about the quality and usefulness of the research. This discussion will focus on studies for intervention, diagnosis, prognosis, and qualitative research, but concepts can be readily applied to most types of research. You can refer to appropriate chapters indicated in each section for background on relevant design and analysis elements.

Levels of Evidence

Published studies come in all shapes and sizes— with stronger and weaker designs, with well-controlled or biased procedures, using statistics appropriately or not, and drawing conclusions that may or may not be warranted. As we strive to use the literature to support clinical decisions, our goal is to identify studies with sufficient validity so that we can be confident in the application of results to our own practice.

The classification of levels of evidence can be used as an initial criterion to judge the rigor of a study’s design. Criteria reflect the degree of confidence one should have by considering the type of study.⁴ It will help to refer back to Chapter 5 to review these classifications for both quantitative and qualitative studies, which are broadly summarized in Table 36-1.⁵ Different criteria are used for different types of studies within each level. These criteria are not hard and fast, however, and any study may be judged stronger or weaker depending on its implementation and precision.