This is a follow up to one of my latest blog posts where I talked about the recent clinical trials for the new CGRP migraine treatments. As most people with migraine reading this will know, the trial results were promising. Across studies, participants receiving the CGRP treatment consistently reported fewer migraine days than their control-group counterparts. Though the effects are not that different from current treatments like Botox and Topamax, the side effect profiles are much better, at least in the short term. The new CGRP seems to have fewer side effects than previous migraine treatments.1
However, I wanted to revisit the concept of internal validity from that blog post. Recall that internal validity is something the developers of all kinds of research studies strive for. In an experiment, internal validity is essentially limiting as many of the confounding factors as possible so that you can say, with as much confidence as possible, that the treatment is what is having an effect on trial participants (if an is effect found) and not something else. In an effort to eliminate confounding factors, some of the CGRP for migraine trials had specific eligibility rules like participants could not continue to use Botox if that was part of their current treatment plan. In the study I attempted to join, participants couldn’t be using Botox and they needed to be on two (or fewer) oral migraine preventives.
The Institute for Clinical and Economic Review (ICER) published a report on the cost effectiveness of CGRP inhibitors.1 In the report, they review 11 CGRP trials that included episodic or chronic migraine patients. Of those studies, 5 reported requiring patients to
stop all preventives at the start of the study. The other 6 studies allowed continued, but stable dosing of current medications throughout the study (p. ES-5). Again, these decisions were made to help ensure that any effects that were found could be clearly attributed to the CGRP inhibitor and not another migraine preventive participants were using. Also of note, all CGRP inhibitor studies excluded patients who had not had a “therapeutic response” to more than two prior preventives (p. 35).
Why am I sharing all of this detail about who was excluded? Because this became very relevant to me being able to start Aimovig, the new CGRP treatment that was recently approved. You see, internal validity has a counterpart which you can probably deduce is called external validity. This is also a goal of most research studies. External validity in a study provides the basis on which the researchers can (or cannot) generalize their findings to other groups or settings. When I say “findings,” I don’t just mean whether a treatment worked, but also whether it is safe and anything else discovered in the course of the research.
For example, researchers who conduct a study on heart disease that only recruited men can only generalize to men. They can’t generalize their findings to women unless they can be very convincing that sex isn’t correlated with heart disease (i.e., sex and heart disease aren’t related). That’s because there could be very different mechanisms, manifestations, triggers, and treatments for heart disease in men and women. If they didn’t study women, then they really shouldn’t extrapolate. In this case, past research shows clear sex differences in heart disease.
The thing about internal and external validity is that it is hard to attain both in the same study. By maximizing one, you often sacrifice the other. Limiting study participants to only those who are using the treatment medication – in an effort to make it more likely to measure any treatment effect – means any effect that is measured can’t be generalized to the population of people with migraine who want to use the treatment medication with other preventive therapies because they weren’t in the study. Internal validity was improved, but at the expense of external validity. On the flip side, the more heterogeneous population you include in a study, the greater the external validity and generalizability,… but it could make it harder to measure an effect or conclude it is a result of the treatment and not another factor. I won’t get into the statistical reasons. You’ll hate me.
Coming back to the CGRP inhibitor trials (finally, I know, sorry!), these decisions have had a direct effect on me. Not only did it mean I was excluded from or opted out of the clinical trials, but it has delayed my access to the CGRP medication that was recently approved by the FDA. On May 14th, I received a round on Botox. On May 17th, the FDA approved Aimovig (erenumab), the first of the CGRP treatments for migraine. When I went for my Botox injections, no one in my doctor’s office made me aware of any rules they had made about patients on Botox receiving Aimovig. I was told to go ahead with my Botox injections. It would probably be months before anyone had access to the drug and that was if it was even approved! [The words of my doctor.]
Not only was Aimovig approved three days later, but within two weeks Amgen was making it available. I soon learned that my doctor’s office had made a rule that patients on Botox had to be off of it for 14 weeks before starting Aimovig. I was given a couple of reasons – regarding insurance, effectiveness – that I quickly investigated and provided counter-evidence that they weren’t applicable to me. I’m persistent like that. I also learned many other clinics were allowing their patients to use Aimovig with Botox, so I asked my doctor to reconsider. I hoped that while she was at a professional conference, she might be convinced otherwise by conversations with peers. Upon her return, we had a follow-up appointment. I was told the answer was still no. This time, the reason was that there was no evidence that Botox and Aimovig were safe to use simultaneously. That’s because no studies of Aimovig included Botox. In a nutshell, the inability to generalize the Aimovig study results to Botox patients is screwing me over. I can’t really refute that.
That said, thousands of patients are using both. The manufacturer has said there is no reason to think there will be contraindications. Other headache specialists I’ve chatted with don’t believe there is any reason to be concerned about interactions, but still, my clinic is playing it safe. So I’m on a countdown clock. Four weeks until Aimovig. In the meantime, the partial dose of Botox I had has long worn off and my baseline pain levels have increased. I’ll power through the next month and continue to silently pray to the migraine gods that the drug will help me.
Anyway, I thought this would make a dorky, but hopefully instructional illustration of a couple of principles of clinical trials and, more generally, research studies. Maybe it will give you something to think about as you read future study results. It’s hard to do good studies that tick all the boxes!