How to Read a Paper: The Basics of Evidence-Based Medicine

 

“Everyone is entitled to his [her] own opinion, but not to his [her] own facts.” —famously attributed to Daniel Patrick Moynihan, late US Senator 

 

In last month’s editorial, Senders et al invited naturopathic physicians and other holistic healthcare providers to “commit to establishing, practicing and refining your own research literacy on a daily basis.”¹ They presented a forceful argument about the power of knowledge and how embracing an evidence-based and evidence-informed approach to healthcare would lead to better patient care as well as to help strengthen our profession politically. They presented the “why” of their invitation, but did not discuss the “how,” which is left for these next 2 articles—a book review of the major textbook in the field, followed next month by a review of continuing medical education opportunities to develop this skill.

 

How to Read a Paper: The Basics of Evidence-Based Medicine was first published in 1996 by Trisha Greenhalgh, MD (currently serving as professor of primary health care and dean for research impact at Barts and the London Medical School). In the first edition, Professor Greenhalgh notes that the original goal was two-fold: first, to help students easily understand key concepts in evidence-based medicine (EBM) and second, to help colleagues who were resistant to EBM have a better appreciation for the logic and methodology of this approach. This year marks the publication of the fifth edition of her text (BMJ Books), which has become a core textbook in medical schools around the world, translated from English into at least 10 other languages. This “simple” text has informed nearly a generation of healthcare providers and is likely to do so for the foreseeable future.

At a slim 252 pages divided into 17 chapters, the book offers a solid introduction to core concepts in EBM, though for purposes of this review, I am condensing this discussion into 3 main sections. The first 5 chapters are introductory and cover key topics such as searching for literature, assessing the purpose and quality of a paper, and thinking about the appropriate statistics. The next 9 chapters are the meat (or kale—a better meal centerpiece) of the text, offering more in-depth training on how to read different types of papers (eg, randomized controlled trials, meta-analyses, economic evaluations). The final 3 chapters really cover the challenges of implementing EBM from a policy perspective, as well as discussing and affirming some of the major critiques of EBM.

The first chapters of How to Read a Paper are about setting the framework for engaging the literature critically and methodically. Starting with “Why Read Papers at All?,” Greenhalgh suggests that conscious reviews of the literature should ideally be to improve healthcare practice at the individual, institutional, or systemic level by avoiding older (and still common) methods of knowledge such as clinical anecdotes or individual papers. After establishing the importance of comprehensive, conscious literature reviews, the author uses the next several chapters to discuss how to frame research questions, how best to initiate a search, and key methodological and statistical issues to review to determine if the science behind the paper—in the context of other similar papers—actually supports the conclusions that are briefly written in the abstract.

 

In these first chapters, Greenhalgh also introduces “checklists” of questions to ask of a paper to determine if it is scientifically valid or relevant to your specific inquiry. These checklists are used throughout the rest of the text as well and are nicely compiled in the appendices of the book. As so many foundational concepts are explored in these introductory chapters, they should really be considered a “must-read” for anybody delving into this book. 

The next 9 chapters train readers in an evidence-based approach to assessing different types of clinically relevant papers: randomized controlled trials (RCTs) of simple and complex interventions, diagnostic testing, reviews and meta-analyses, qualitative research, economic analyses, and practice guidelines.

 

Given the realities of naturopathic medicine—and the oft-cited concerns about the difficulty of creating quality holistic health research—chapters 6 and 7 should not be glossed over. Chapter 6, “Papers that Report Trials of Drug Treatments and Other Simple Interventions,” begins with a blistering critique of how the pharmaceutical industry has coopted scientific research to support its marketing efforts. From there it proceeds through the critical information that a quality clinical trial must contain (including the differences between hard clinical outcomes and surrogate endpoints) and ends with a section boldly titled “Getting worthwhile evidence out of a pharmaceutical representative” with advice on how to “debunk sponsored clinical trials reports that attempt to blind you with statistics.” The same critical thinking skills should apply, of course, to nutraceutical representatives and trials.

 

Chapter 7, “Papers that Report Trials of Complex Interventions,” covers the challenges that researchers face in designing, implementing, and creating quality data when using complex, holistic, multifactorial approaches to health. Greenhalgh uses as her example a clinical trial from her own experience in which using yoga in the treatment of diabetes was tested. This particular trial started as an idea in her doctoral student’s mind, progressed through exploratory work (interviewing yoga teachers, diabetes educators, and patients), and finally developed into a pilot clinical trial giving diabetic patients twice-weekly classes. When the data were finally analyzed, however, her team was disheartened to learn that there had been no effect on the health status of the individuals with diabetes. The easy interpretation? Yoga has no effect on diabetes. The nuanced interpretation? Many of the participants failed to go to all of the sessions, and almost none of them did home exercises. From there, her research team did not proceed to a full-scale clinical trial but instead had to reconsider components of the trial and redesign a new pilot trial. Greenhalgh uses this trial to illustrate several key concepts in reviewing the literature on complex interventions: Was preliminary research done? Did the outcomes from a pilot study support the development of a larger study? If the study was negative, was the problem really with the intervention or was it in the implementation of the intervention? For any naturopathic physician who has ever been frustrated with “negative” reports about holistic interventions, this chapter is an essential tool to learn how to think through these questions when reading papers.

 

From the perspective of a busy clinician, the other chapters about specific types of papers might be optional based on one’s own interest. Chapters on meta-analyses and guidelines are helpful to consider after having read through the basic clinical trial chapters and include discussion of the strengths and weaknesses of these larger aggregations of data. The chapter on diagnostic testing is helpful if you are reviewing the literature (or lack thereof) for a new test or if you’re questioning the validity of regularly used tests, be they conventional (mammograms) or “alternative” (food sensitivity testing). Chapters on economic analysis, qualitative research (nonnumeric, often psychosocial in nature), questionnaires, and quality-improvement are probably of limited interest to most general clinicians, though even here there are areas of interest (naturopathic physicians might be especially interested in the chapter on qualitative research, which may explore how patients get engaged in their own healthcare). 

The final chapters in this book review the difficulties with institutional implementation of EBM, using and discussing evidence with patients (taking into account their own preferences), and ends with a critique of the entire enterprise of EBM. While the chapters on implementation are important for institutional leaders and individual clinicians, the critique—offered at the end of a text discussing the foundations of EBM—is the final “must-read” for any naturopathic physician reading this book.

 

We naturopathic physicians pride ourselves on developing individualized, holistic treatment plans. We are also quite aware that the vast majority of our interventions have never been (and probably will never be) subject to large, multisite RCTs in the same way that a pharmaceutical intervention might get tested. This creates 2 critiques of EBM—lack of tailoring to the patient and an overreliance on using only interventions that have been studied—that are described by Greenhalgh as examples of “What’s wrong with EBM when it’s done badly.” She notes, “The ‘best’ treatment is not necessarily the one shown to be most efficacious in RCTs but the one that fits a particular set of individual circumstances and aligns with the patient’s preferences and priorities.” While this comment is certainly in alignment with the principles of naturopathic medicine, it does not negate the preceding 16 chapters of her text. Instead it reminds us that we must constantly strive to master both the art and science of being a clinician.

Greenhalgh’s text wonderfully blends scientific rigor and skepticism with a deeply humanistic concern about supporting clinical science that improves the condition of our human family. It is neither a dry read nor overly technical, and the author weaves her personality and experiences into the subject matter nicely. For the clinician who wants to better engage with the literature, it will be an easy read over a few weeks. For the teachers and administrators at our universities, it should become as standard a text as Netter’s Atlas of Human Anatomy.

 

How to Read a Paper: The Basics of Evidence-Based Medicine, 5th Edition

Trisha Greenhalgh, MD

BMJ Books, 2014

ISBN: 1118800966