Does expert opinion matter? A JAMA study reveals the role advisory committees play in FDA decisions

Licensed reprint: Pharmascan

In March 2023, the FDA's Peripheral and Central Nervous System Drugs Advisory Committee voted on whether the amyotrophic lateral sclerosis (ALS) drug tofersen should be approved. Although committee members disagreed during the discussion over whether the drug could effectively relieve patients' symptoms, prior trials had shown a downward trend in the protein neurofilament in patients' bodies. In the end, committee members voted unanimously in favor. One month later, tofersen successfully obtained accelerated approval.

For questions like this that can only be resolved by drawing on multiple areas of expertise, the FDA sometimes seeks advice from outside experts and treats their opinions as a reference for its final ruling. But just how much does the FDA base its rulings on the recommendations of its advisory-committee experts?

The FDA almost always agrees with the experts, but not in every situation

According to a recent study in the journal JAMA, over the past 12 years the FDA convened a total of 409 advisory committee meetings (human drugs). After further analysis of the meetings that involved a vote, the FDA's final decision matched the advisory committee's vote 88% of the time. The study tallied all FDA meetings held between 2010 and 2021, classifying the nature of the discussions into initial approval, supplemental indication approvals, safety actions, and accelerated approval withdrawals.

Lead author Dr. Joseph Ross Daval and his co-authors note in the paper that this indicates advisory committees play a key role in the FDA's decision-making process. The study's authors are all affiliated with Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts.

The high degree of agreement between advisory committee votes and the FDA's final decisions may be because the FDA is easily persuaded by the experts, because these experts are already able to predict the FDA's decision as they vote, or because the way the meetings are convened favors the FDA reaching the decision it wants. Moreover, this high degree of agreement did not vary over the years.

Although nearly 90% agreement almost implies that the advisory committee's outcome is equivalent to the FDA's final ruling, both historical cases and statistical data still tell us that the FDA retains an unpredictable element.

The most famous case in history is the Alzheimer's drug Aduhelm (aducanumab). At the time, the FDA's Peripheral and Central Nervous System Drugs Advisory Committee voted on the drug's efficacy. On the question of "whether it is reasonable to treat a single positive study result as the primary evidence that Aduhelm can effectively treat early-stage Alzheimer's symptoms," not a single committee member voted in favor; instead, 10 voted against and 1 was uncertain. In the end, the FDA still granted the drug accelerated approval, subject to another clinical trial. When the news broke, three of the advisory committee experts were dissatisfied that the FDA had cleared the drug; besides publicly questioning its efficacy, they ultimately resigned in protest. One of the departing members, Harvard Medical School professor Aaron S. Kesselheim, went further, calling it "probably the worst drug approval decision in recent U.S. history."

In fact, this was not an isolated case, and such inconsistencies are most likely to appear when the advisory committee votes negatively. Taking initial approval as an example, the study found that of 60 situations in which the advisory committee voted negatively, only 40 ultimately failed to be approved. In other words, when the advisory committee says "No" to the FDA, one third of the time the FDA disagrees. This finding stands in sharp contrast to situations where the advisory committee votes positively. Again taking initial approval as an example, of 147 situations in which the advisory committee voted positively, 142 were ultimately approved, an agreement rate of nearly 100%, meaning that when the advisory committee says "Yes" to the FDA, the FDA almost entirely follows these experts' opinions.

"The FDA is using advisory committees to identify barriers to approval while steering the agency toward more streamlined pathways for approving drugs," wrote Dr. Genevieve Kanter, associate professor at the USC Sol Price School of Public Policy, in an article on the study. The commentary was also published in JAMA.

What is the key question the FDA poses to its advisory committees?

"On the surface, the main question the FDA appears to pose to its advisory committees is: should we approve this drug?" Dr. Genevieve Kanter noted in the article. "But these findings tell us that, on the surface, the main question the FDA appears to be asking its advisory committees is: what is preventing the drug from being approved? Or: what different approaches do we have that could get this drug approved?"

Dr. Genevieve Kanter elaborated further in the article, using the Alzheimer's drug Aduhelm as an example. Although the advisory committee unanimously agreed that the drug had no effect on the most critical clinical endpoint, the FDA ultimately used a surrogate endpoint as the pathway for the drug's accelerated approval, without reconvening an advisory committee meeting. She argues that a starting point like this—"what different approaches do we have that could get this drug approved?"—may reflect pressure the FDA faced from patient groups, or ties to pharmaceutical-industry interest groups.

Dr. Genevieve Kanter also noted that some drugs not endorsed by advisory committees were ultimately approved, possibly because the drugmakers responded to the concerns raised at the meeting and went on to overcome those problems. Although this situation is not common, it is currently impossible to know how often it happens or why.

Among these inconsistent outcomes, experts who cast dissenting votes have suggested waiting for prospective results before deciding, thereby recommending delaying approval of certain drug indications or authorizations—for example, pembrolizumab for the supplemental indication of triple-negative breast cancer. The FDA approved that indication after obtaining the trial results, and did not reconvene another meeting. By contrast, both selinexor and olaparib received their initial approvals before trial results were available.

Typically, advisory committee meetings are held to help review divisions interpret data or results when questions or difficulties arise related to trial data. For the FDA, obtaining opinions from experts outside government—by encouraging patients, healthcare providers, and other interested parties to share their views at public hearings—can project the agency's effort to build an open and transparent government. Advisory committee meetings can be convened at any stage of the review process, and can also be held after a product has been approved and marketed, to help review divisions interpret data or results.

"But this pro-approval bias also signals reduced transparency and political interference," Dr. Genevieve Kanter criticized.

The number of FDA advisory committee meetings is declining

From 2010 to 2021, of the 409 meetings the FDA convened, the number peaked at 50 in 2012, then fell to 18 in both 2020 and 2021. The category with the largest decline was new drug applications. Although this may be because the COVID-19 pandemic affected the numbers in those two years, an overall downward trend is still evident.

Dr. Genevieve Kanter pointed out that FDA Commissioner Robert Califf had suggested not voting at advisory committee meetings, arguing that voting is not important and the results are not binding. Dr. Genevieve Kanter, however, believes voting broadly discusses complex medical issues and that the summaries benefit FDA officials and the public interest. She notes that what should be fixed is instead clarifying what the vote is really asking. She went further, suggesting that the final "pass/fail" vote could be abolished while retaining constructive votes on matters such as "efficacy" and "safety."

"As long as this kind of (pro-approval) bias reflects the public's desire for rapid access to new therapies and its ability to bear the corresponding high risks, it will not be a problem. Conversely, if it reflects the influence of pharmaceutical companies, this bias is deeply worrying—especially given the reduced frequency of advisory committee meetings and the abolition of the voting system following political intervention," Dr. Genevieve Kanter said.

Exactly why the FDA is convening advisory committees less often remains unknown. What can be confirmed, however, is that given the FDA's new drug approval numbers have not declined markedly over the past decade, and that the proportion of these meetings involving a vote has risen against the trend, the FDA does indeed increasingly seldom consult experts publicly—but once a meeting is convened, a vote is usually held, and the FDA's final decision still largely follows expert opinion.

"This study shows that committees play a key role in the FDA's decision-making process, but over time the FDA has sought independent expert opinion less often, even though it still follows that opinion. Nevertheless, advisory committees should be defined more clearly and openly within the current regulatory framework," the Joseph Ross Daval authors conclude at the end of the JAMA study.

The content of this article has been licensed by Pharmascan to PatientForce™ (Media-WIND Communications Co., Ltd. / Tax ID: 27903403). The intellectual property rights are jointly held by PatientForce™ and Pharmascan in accordance with the law. If you believe this article infringes any rights, please contact the relevant department of our company by email, and we will respond as soon as possible.