This article is part of the 2018 NMJ Oncology Special Issue. Download the full issue here.
Reference
Johnson SB, Park HS, Gross CP, Yu JB. Complementary medicine, refusal of conventional cancer therapy, and survival among patients with curable cancers. JAMA Oncol. 2018;4(10):1375-1381.
Design
A retrospective observational analysis of data obtained from the National Cancer Database (NCDB)
Participants
The initial cohort included 1,901,815 patients who were diagnosed with non-metastatic breast, prostate, lung, or colorectal cancer between 2004 and 2013. The NCDB is a clinical database that captures approximately 70% of all newly diagnosed cancers from more than 1,500 Commission on Cancer accredited centers in the United States and is a joint project of the Commission on Cancer of the American College of Surgeons and the American Cancer Society. Patients were selected from this cohort based on their use of complementary medicine during cancer treatment.
From the initial group of 1,901,815 patients with cancer, 258 people met the study’s definition of using complementary medicine (the CM group). Of the remaining patients, 1,032 served as the control group. The CM group consisted of 199 women and 59 men, with a mean age of 56 years. Patients were matched on age, clinical group stage, comorbidity score, insurance type, race/ethnicity, year of diagnosis, and cancer type to other patients in the database.
Study Medication and Dosage
Patients were defined as undergoing complementary medicine if they received “Other-Unproven: Cancer treatments administered by nonmedical personnel” in addition to any conventional cancer treatment (CCT) as noted in the patient record. Treatment refusal was defined as any NCDB-documented refusal of chemotherapy, radiotherapy, surgery, and/or hormonal therapy in the patient record.
Outcome Measures
Overall survival, adherence to treatment, and patient characteristics
Key Findings
Patients who chose CM did not have a longer delay to initiation of CCT but had higher refusal rates of surgery (7.0% vs 0.1%; P<0.001), chemotherapy (34.1% vs 3.2%; P<0.001), radiotherapy (53.0% vs 2.3%; P<0.001), and hormone therapy (33.7% vs 2.8%; P<0.001).
Use of CM was associated with poorer 5-year overall survival compared with no CM (82.2% [95% CI: 76.0%-87.0%] vs 86.6% [95% CI: 84.0%-88.9%]; P=0.001) and was independently associated with greater risk of death (hazard ratio: 2.08; 95% CI: 1.50-2.90) in a multivariate model that did not include treatment delay or refusal. However, there was no significant association between CM and survival once treatment delay or refusal was included in the model (hazard ratio: 1.39; 95% CI: 0.83-2.33).
In this study, patients who received CM were more likely to refuse additional CCT, and those who did had a higher risk of death.
Practice Implications
This could be an important study for our practices. It could inform how we counsel patients who are contemplating not following the standard of care and ignoring the advice of their medical oncologists. If the authors’ conclusions were correct we would be strongly lobbying in favor of conventional cancer therapy.
I write ‘could be’ because many of us remain skeptical about this study’s methodology. Put simply, the numbers do not add up.
Before considering the math though, we need to look at definitions.
In this research Johnson et al chose to use a definition of complementary medicine that differs from the way it has been defined in the medical literature to date. In Johnson’s view, “Complementary medicine (CM) is used in addition to conventional cancer therapy (CCT) and may be used as a substitute for adjuvant therapies.”
Past researchers have made a significant distinction between complementary medicine, which is used in addition to standard of care therapies, and alternative medicine, which is used instead of conventional therapies. The National Institutes of Health (NIH) also officially defines “complementary” and “alternative” medicines as separate entities.1 Johnson et al chose to consider use of “complementary therapies” as falling on a continuous spectrum, grouping together all practices whether they are used along with or instead of conventional cancer care.
The authors’ analysis of the data does not inform us whether those who use complementary medicine in an adjunctive and integrative manner with conventional care fare better or not.
Johnson et al also defined CM as “unproven cancer treatments administered by nonmedical personnel in addition to at least 1 conventional cancer treatment such as surgery, radiotherapy, chemotherapy and/or hormone therapy.” They go on to write, “Our work demonstrates that CM and alternative medicine likely represent entities along a continuum, rather than being distinct entities.”
Now, let us consider the numbers. Our professional experience is that a high percentage of cancer patients choose to include complementary therapies in their cancer treatment. This experience is borne out by published research that confirms our observations in practice. A 1998 review by Cassilith reported complementary and alternative medicine (CAM) usage at a high of 64% and a low of 7%.2,3 The NIH suggests that 30% of adults and 12% of kids with cancer are engaged in nonconventional cancer therapies.1 More recent estimates suggest use of CAM therapies is anywhere between 25% and 84%, and that this varies by part of the country, type of cancer, and criteria for defining CAM.4-12
Yet in Johnson et al’s data, only 258 cases out of nearly 2 million patients met the study’s criteria as using either complementary or alternative medicine. This calculates to less than 0.01% of the initial cohort. Even if we take the NIH’s conservative estimate that only 30% of adults with cancer engage in CAM practices, there should be half a million or more patients in the CM group; the incredibly low number of people meeting the study criteria is highly suspect for a flawed study design. Clearly, something is wrong early on in the process with how study participants were selected. Something about the selection process used by Johnson et al is far askew from all previous accounts of CAM usage. Of course, once we acknowledge this, we have to question all the study’s conclusions.
A clue that something may be amiss is mentioned in the study’s text: “The American College of Surgeons and the Commission on Cancer have not verified and are neither responsible for the analytic or statistical methods used herein, nor for the conclusions drawn from these data by investigators.” This is an odd sort of caveat to include.
One has to wonder just how peculiar a patient had to be to be selected into the CM group. We do know that they were unusually noncompliant, based on these data:
- 7% of them refused recommended surgery compared to 0.1% of the control group
- 34% of them refused chemotherapy compared to 3.2% of the control group
- 53% refused radiotherapy compared to 2.3% of the control group
- 33.7% refused hormone therapy compared to 2.8% of the control group
In our practice we see an occasional patient who refuses standard conventional treatment, but not many. These percentages stand out as unusually high. Again, this comes back to the initial definition of “complementary” medicine the authors used.
If we were to take the authors and their data at face value, we might conclude that of patients who chose complementary medicine, a high percentage will use it as an alternative treatment instead of conventional care and will experience greater chance of a poor outcome. CM was associated with increased risk of death when the data were analyzed, but if the data were categorized by which patients refused conventional treatment, the associations disappeared: “CM (vs no CM) no longer had a statistically significant association with the risk of death (HR, 1.39; 95% CI, 0.83-2.33).”
In other words what Johnson et al really measured is the effect of using CCT. Not using CCT was associated with a worse prognosis.
The authors’ analysis of the data does not inform us whether those who use complementary medicine in an adjunctive and integrative manner with conventional care fare better or not.
But all of that is only relevant if we trust the findings of the study, and many do not. In fact, some are suggesting that JAMA Oncology should retract the study.13
This is all unfortunate, because it would be valuable to have accurate data on the outcomes of CM in cancer care. Lately it seems that more and more patients come into our office having, as they claim, “done research on the internet.” They come in with the belief that a long list of natural medicines will effectively cure their cancer. Sadly, they are often mistaken in their conclusions and the faith they place in online information.
Do these patients who choose not to undergo standard of care treatments or conventional cancer care have a worse outcome than those patients who do as their oncologist suggests, though? It would be nice for us to know a definitive answer. According to Johnson et al, that answer is yes.
Yet common sense and simple math tell us that there is something fundamentally wrong with how Johnson et al selected their study participants—wrong enough that we need more than a grain of salt to swallow their findings.
Negative studies about alternative medicine get attention in the news media. There is something that draws us in; we are hardwired to want to say, “I told you so.” Or to gloat at someone else’s mistake or misfortune.
The New York Times chose an exaggerated headline to write about this study: “Alternative Cancer Treatments May Be Bad for Your Health: People who used herbs, acupuncture and other complementary treatments tended to die earlier than those who didn’t.” The author did point out in the course of the story that, “The complementary treatments did no harm when conventional treatment was carried out simultaneously.”14
But with that headline few readers will notice such a qualification.
