The External Validity Roadblock

This is a follow up to one of my latest blog posts where I talked about the recent clinical trials for the new CGRP migraine treatments. As most people with migraine reading this will know, the trial results were promising. Across studies, participants receiving the CGRP treatment consistently reported fewer migraine days than their control-group counterparts. Though the effects are not that different from current treatments like Botox and Topamax, the side effect profiles are much better, at least in the short term. The new CGRP seems to have fewer side effects than previous migraine treatments.1

However, I wanted to revisit the concept of internal validity from that blog post. Recall that internal validity is something the developers of all kinds of research studies strive for. In an experiment, internal validity is essentially limiting as many of the confounding factors as possible so that you can say, with as much confidence as possible, that the treatment is what is having an effect on trial participants (if an is effect found) and not something else. In an effort to eliminate confounding factors, some of the CGRP for migraine trials had specific eligibility rules like participants could not continue to use Botox if that was part of their current treatment plan. In the study I attempted to join, participants couldn’t be using Botox and they needed to be on two (or fewer) oral migraine preventives.

The Institute for Clinical and Economic Review (ICER) published a report on the cost effectiveness of CGRP inhibitors.1 In the report, they review 11 CGRP trials that included episodic or chronic migraine patients. Of those studies, 5 reported requiring patients to


stop all preventives at the start of the study. The other 6 studies allowed continued, but stable dosing of current medications throughout the study (p. ES-5). Again, these decisions were made to help ensure that any effects that were found could be clearly attributed to the CGRP inhibitor and not another migraine preventive participants were using. Also of note, all CGRP inhibitor studies excluded patients who had not had a “therapeutic response” to more than two prior preventives (p. 35).

Why am I sharing all of this detail about who was excluded? Because this became very relevant to me being able to start Aimovig, the new CGRP treatment that was recently approved. You see, internal validity has a counterpart which you can probably deduce is called external validity. This is also a goal of most research studies. External validity in a study provides the basis on which the researchers can (or cannot) generalize their findings to other groups or settings. When I say “findings,” I don’t just mean whether a treatment worked, but also whether it is safe and anything else discovered in the course of the research.

For example, researchers who conduct a study on heart disease that only recruited men can only generalize to men. They can’t generalize their findings to women unless they can be very convincing that sex isn’t correlated with heart disease (i.e., sex and heart disease aren’t related). That’s because there could be very different mechanisms, manifestations, triggers, and treatments for heart disease in men and women. If they didn’t study women, then they really shouldn’t extrapolate. In this case, past research shows clear sex differences in heart disease.

The thing about internal and external validity is that it is hard to attain both in the same study. By maximizing one, you often sacrifice the other. Limiting study participants to only those who are using the treatment medication – in an effort to make it more likely to measure any treatment effect – means any effect that is measured can’t be generalized to the population of people with migraine who want to use the treatment medication with other preventive therapies because they weren’t in the study. Internal validity was improved, but at the expense of external validity. On the flip side, the more heterogeneous population you include in a study, the greater the external validity and generalizability,… but it could make it harder to measure an effect or conclude it is a result of the treatment and not another factor. I won’t get into the statistical reasons. You’ll hate me.

Coming back to the CGRP inhibitor trials (finally, I know, sorry!), these decisions have had a direct effect on me. Not only did it mean I was excluded from or opted out of the clinical trials, but it has delayed my access to the CGRP medication that was recently approved by the FDA. On May 14th, I received a round on Botox. On May 17th, the FDA approved Aimovig (erenumab), the first of the CGRP treatments for migraine. When I went for my Botox injections, no one in my doctor’s office made me aware of any rules they had made about patients on Botox receiving Aimovig. I was told to go ahead with my Botox injections. It would probably be months before anyone had access to the drug and that was if it was even approved! [The words of my doctor.]

Not only was Aimovig approved three days later, but within two weeks Amgen was making it available. I soon learned that my doctor’s office had made a rule that patients on Botox had to be off of it for 14 weeks before starting Aimovig. I was given a couple of reasons – regarding insurance, effectiveness – that I quickly investigated and provided counter-evidence that they weren’t applicable to me. I’m persistent like that. I also learned many other clinics were allowing their patients to use Aimovig with Botox, so I asked my doctor to reconsider. I hoped that while she was at a professional conference, she might be convinced otherwise by conversations with peers. Upon her return, we had a follow-up appointment. I was told the answer was still no. This time, the reason was that there was no evidence that Botox and Aimovig were safe to use simultaneously. That’s because no studies of Aimovig included Botox. In a nutshell, the inability to generalize the Aimovig study results to Botox patients is screwing me over. I can’t really refute that.

That said, thousands of patients are using both. The manufacturer has said there is no reason to think there will be contraindications. Other headache specialists I’ve chatted calendar pagewith don’t believe there is any reason to be concerned about interactions, but still, my clinic is playing it safe. So I’m on a countdown clock. Four weeks until Aimovig. In the meantime, the partial dose of Botox I had has long worn off and my baseline pain levels have increased. I’ll power through the next month and continue to silently pray to the migraine gods that the drug will help me.

Anyway, I thought this would make a dorky, but hopefully instructional illustration of a couple of principles of clinical trials and, more generally, research studies. Maybe it will give you something to think about as you read future study results. It’s hard to do good studies that tick all the boxes!


Responding to the “Migraine Pose”

As many of you know, makeup artist Nam Vo posted a selfie to Instagram earlier this week with the original hashtag #migrainepose.

Nam Vo demonstrating her “migraine pose.” Courtesy of her Instagram account.

When the post was picked up by Elle and other news outlets, it set off a firestorm of responses by the migraine community. The American Migraine Foundation has sent a letter asking Elle “to use its reach and influence to demonstrate compassion and understanding for the millions of women around the world fighting to live a normal a life, while dealing with a devastating illness.” Likewise, hundreds of us have tweeted and emailed asking Elle and others to write a story from our perspective. I’m sharing the longest version of the letter I’ve circulated, though I have a 500 word (and 500 character! version), too.

To date (7/1/18), I don’t believe that any of us has had a response taking us up on the the offer to help author an article about the true impact of migraine and migraine stigma, but if we’re anything, we’re persistent, right?! Have you also written to news agencies and magazines? Have you posted your response publicly? I’d love to read it. Link to your response in the comments and let me know what you think about mine.




June was migraine and headache awareness month. The month is over, but I want to explain why our advocacy work is still so important and why the migraine community needs your help. We would like you to support us with an article about migraine and the real-life struggles of those of us who have it. There are dozens of us ready to help you write an article and we promise we can make it compelling after the events of the past few weeks.

I have chronic migraine. For the past three years, I’ve not had a true pain- or symptom-free day. Because of this, migraine has taken away my friends, my social life, my career, and my dreams. In place of those things, I advocate for a disease that, although it is common, is poorly understood and widely stigmatized. I spent most of the month of June feeling like I was doing good work. Then, earlier this week, a makeup artist named Nam Vo posted to Instagram a photo with the hashtag #migrainepose. In the photo, she posed with her hand pressed to her temple. She claimed this pose was the newest trend; done to accentuate cheekbones or manicures. Elle picked up the story, then Cosmopolitan Australia.

The migraine community clapped back. We flooded Twitter and Instagram with authentic photos of migraine. We described the pain, nausea, and the need to withdraw from light and sound. We talked about the events we’d missed, the meetings we’d blown, and the hours with our families we would never get back. We shared statistics. Like the fact that migraine affects about 38 million Americans and 1 billion people worldwide. Migraine is also the 6th highest cause of disability globally and the leading cause of disability among those aged 15-49. We offered to work with these magazines to reframe their stories about an insensitive pose to tell ours instead. So far, our offers have not been taken.

People with migraine face stigma. To start, it is an invisible illness, so our symptoms are hard to prove. We’re often doubted and suggested to be faking. Migraine is also frequently misperceived as just a headache. Everyone has had one of those. Some have even had a migraine. So even when we are believed, people often ask, “How bad can it be?” Migraine also affects women disproportionately. Let’s be clear, women’s pain is taken less seriously than men’s. Combined, this is basically the trifecta of medical stigma and I could go on.

We fight this stigma even when we’re facing professionals. Last month, the first new preventive medication for migraine was approved by the FDA. Earlier this month, the Institute for Clinical and Economic Review (ICER) met to review and vote after months of research and public input on the cost-effectiveness of these new drugs. ICER allowed a handful of patients and patient advocates to speak during the meeting, but none were allowed on the voting panel nor was a headache specialist. While the panel found some benefit of these new medications for patients like me – patients with chronic migraine – they were hesitant to endorse their use in patients with fewer migraines per month (those labeled “episodic”). That was bad enough. The kicker, however, was in the closing remarks when the meeting leader told those in attendance that people with migraine should be cautious to not “over-medicalize” migraine or forget lifestyle changes.

Again, let’s be clear. We’re talking about a serious neurobiological disorder. We’re also talking about the first migraine preventive developed for migraine. In the past, we’ve treated migraine with off-label therapies meant for high blood pressure, epilepsy, and depression. As another migraine advocate put it, I think it’s safe to say that having one FDA-approved migraine preventive is a far cry from over-medicalization. Furthermore, would we tell someone with other serious neurological disorders to treat it with yoga and acupuncture? We might suggest they include alternative therapies alongside their other preventive medications, but not likely replace them. The same holds for the “migraine pose.” We wouldn’t glamorize a cancer pose or an asthma pose.

De-stigmatizing migraine will only come if people with migraine are given a voice to share their experiences. So, again, this is where your help is needed. Thank you so much for sticking with me this far. I and other advocates are ready and willing to help you write an article to counter those posted about the migraine pose. Please, don’t hesitate to reach out.

With sincere thanks,

Beth Morton, PhD