Transcript of BioCentury This Week TV Episode 176
Dr. Richard Moscicki, Deputy Director for Science Operations, Center for Drug Evaluation and Research, Food & Drug Administration
Dr. Leonard Freedman, President, Global Biological Standards Institute
Steve Edelson, Executive Editor, Scibx: Science-Business Exchange and Senior Editor, BioCentury Publications
PRODUCTS, COMPANIES, INSTITUTIONS AND PEOPLE MENTIONED
Vioxx, rofecoxib, Merck & Co. Inc.
Journal of American Medicine
National Institutes of Health
Francis Collins, Director, National Institutes of Health
Health and Human Services
PLoS One Reproducibility Collection
PLoS Reproducibility Initiative
Steve Usdin, Senior Editor
STEVE USDIN: Government regulators are in high demand at drug companies, but the door doesn't usually spin the other way. This week, we'll meet a physician researcher, who jumped from biotech to FDA. And last week, NIH director Francis Collins warned that the inability to reproduce research threatens medical progress. We'll talk about what's being done about it.
I'm Steve Usdin. Welcome to BioCentury This Week.
NARRATOR: Connecting patients, scientists, innovators, and policymakers to the future of medicine -- BioCentury This Week.
STEVE USDIN: Doctor Richard Moscicki had a 19 year career at Genzyme, a pioneer in developing drugs for rare diseases. He was Genzyme's chief medical officer and was responsible for regulatory and drug safety from 1996 to 2011. Along with his career in industry, Dr. Moscicki pursued an active research career as an immunologist at Massachusetts General Hospital. Last February, Dr. Moscicki made the leap from industry to government.
He was named to a newly created FDA position, Deputy Center Director for Science Operations at the Center for Drug Evaluation and Research, reporting to center director Janet Woodcock. Today, we're joined by Doctor Richard Moscicki. Doctor Moscicki, you spent about 20 years at Genzyme and in academic research.
You've been at FDA for about a year. What have you learned or what surprised you about the culture at FDA in government, compared to academia and industry?
RICHARD MOSCICKI: Hi, Steve. Indeed, when I first came about, six, seven months -- so it hasn't been quite a full year -- I really found that it's really the same world, in many ways. Doing drug development is doing drug development. It's just 180 degree prospective. So the people are just as bright, intelligent, dedicated to the mission of evaluating these drugs.
So that was a real pleasure to see and to experience. I think that the decision making I found was interesting, because it was often constrained by the laws and precedents that it all set. That gave it a different flavor than what you might experience in academia or in industry.
STEVE USDIN: One of the things that people say about FDA -- I don't know if it's true -- is that most people FDA, they're PhDs, they're physicians. They haven't had practical, hands on experience doing drug development, running clinical trials. I'm wondering if having had that experience has allowed you to kind of interject some of that experience into decision making at FDA.
RICHARD MOSCICKI: Well, I do think it helps inform the conversation. And I think it's appreciated, actually, in the conversations that I've participated in FDA, to be able to add that kind of practical consideration of what are the implications of the decisions that we are making, or the recommendations that we're making to sponsors.
STEVE USDIN: So one of the things that if you look at FDA and look back over the last five, 10 years, there was this pendulum swung toward drug safety after the withdrawal of Vioxx. There was a great deal of concern politically and in the public. And FDA seemed to respond to that and tighten its standards.
Then there was a concern that things had gotten too difficult, not enough drugs were getting on the market, people weren't getting the help that they needed. And it seemed that FDA changed and has emphasized streamlining drug development, breakthrough therapies, things like that. Recently, there's been some criticism of FDA in the Journal American Medical Association suggesting that it's letting drugs on the market too easily again.
Is it a pendulum? And is it going to inevitably swing back the other way?
RICHARD MOSCICKI: Well, I really think that it begins with standards. And the FDA has always maintained important standards. Whether the perceptions change, those standards still exist. And I think that's an important way to begin to look at it. How those are sometimes interpreted may be different by different individuals.
One thing I have learned is that the FDA is a large and complex organization made up of many individuals with many different opinions and active debate about which direction certain decisions might really go. I do think that FDA has responded to public health needs.
And that's what we're seeing today. The need to move certain drugs forward where there are serious conditions with unmet needs has to be addressed, for example, in antibiotics that are necessary as we see increasing drug resistance, in rare diseases, where the availability of a large number of patients just isn't there in order to conduct a normal kind of trials. And so I think FDA has been very thoughtful in trying to find solutions that will help drive these drugs forward and incentivize, in fact, the availability of drugs for these unmet needs.
STEVE USDIN: So one of the implicit criticisms in some of these JAM articles was the notion that if a drug is approved in less than two large randomized controlled trials, it's somehow a compromise. It compromises patient safety in and information about efficacy. Is that the way you see it?
RICHARD MOSCICKI: No, that's not the way I see it. And I don't think that's the way many of my colleagues see it. If you're going to approve a drug for a very large, broad population, then of course you need more information. You need to reduce the uncertainty regarding its risk. You need to understand the benefit that would counteract any such risk that you might identify.
But when we're talking about certain specific populations where unmet need really exists, we have to be able to balance the degree of uncertainty that we might have with the benefit that in fact, could be present for that population, and allow ourselves to learn more as the drug becomes more broadly used. And we have to be able to understand that.
I think understanding risk is an extremely important part of how we're going to best serve the public.
STEVE USDIN: We're going to talk more about that, and the idea of balancing risks and benefits with Dr. Moscicki when we return.
NARRATOR: Every month, BioCentury This Week will feature profiles in innovation, a special segment highlighting the stories of innovators whose work is improving lives and transforming the world of healthcare. You are watching BioCentury This Week.
STEVE USDIN: We're talking with FDA's Dr. Richard Moscicki. Dr. Moscicki, you came to FDA from Genzyme. That was kind of a surprise to people, to see somebody coming from industry. Are we seeing kind of a generational change? Are we going to see more people coming into senior positions at CDER, or the Center for Drugs?
RICHARD MOSCICKI: Well, I think we're really at an interesting moment, Steve, at FDA and at CDER. We're starting to see a turnover in leadership in senior positions within CDER, important positions. These are people who have many, many years of experience. And so I think we're at a moment where we need to be prepared to replenish that leadership, to look and renew the people who will continue to bring CDER forward into the future.
STEVE USDIN: Well, it's interesting, because I'm not going name the names, but people who are familiar with drug development -- everybody's got a couple of names, three names, in their mind of people they think, well, these are the people who are FDA. And a lot of them are very senior people, who have shaped drug development for a generation. What are the kind of skills you think that the next generation has to bring in? And what does FDA have to do to nurture people so that they can step up to that responsibility?
RICHARD MOSCICKI: Well, obviously there's going to be two different ways. One are those are bright young people, who will join FDA and work their way up and getting the experience to enter leadership positions. But the other is to look at people who've already achieved leadership in another venue, whether it be industry or academia, who have demonstrated their ability to have vision, their ability to manage.
I think just because sometimes you're a physician, doesn't mean you're a good manager. And the FDA is a large complex organization, where the leadership does require also managers. And so we need to bring those skill sets in, in order to accomplish the mission.
STEVE USDIN: And do you have the flexibility in hiring that makes it possible, really, to recruit a lot of senior people? My impression in the past has been the hiring process has become sclerotic, not necessarily because of FDA, but because you're part of HHS and the government.
RICHARD MOSCICKI: Well, I want to be politically correct in what I say here. But I do think we need to make more of an effort in the recruitment area. This is an area for FDA and for CDER that's really important to its future at this moment. And this is an area that we need to invest more in.
STEVE USDIN: So taking a step back also and looking at what this new leadership is going to be stepping into, the pace of science is moving tremendously. The pace of drug development is moving really quickly, also. If you had to pick out a couple of areas and say, well, these are the areas that are really going to be important in the future, in the near term future -- five, 10 years from now -- what would they be?
RICHARD MOSCICKI: Well, one of the areas, of course, is targeted therapies. That's already emerged and is going to probably even be more important as we move forward. Targeted therapies are those therapies, for example, where you use a special test, whether it be a biomarker or a genetic test or a susceptibility test to identify patients for either clinical trials or for the labeled indication.
And the hope is that by doing so, of course, you increase the chances that they will receive benefit and maybe a larger benefit and a larger treatment effect. And that also allows you to usually have a much smaller development program, which brings up its own issues, both positive and negative, positive on the side of lower cost for the developer, therefore greater ease to bring these products forward, but less information and maybe some greater uncertainty, as we were talking about before, in terms of the safety profile of the drug as one example. And we're even seeing this now among rare diseases.
For example, we're sub-setting rare diseases as with cystic fibrosis, as we've seen recently. So now you have subsets of rare diseases, which create an even greater challenge for regulatory authorities, because now we have these smaller sets of data in order to look at. And targeted therapies in the last three years have been about 28%, and they're growing.
If you look back at 1990, they were only about 8%, 9%. And I think last year, in some areas, it was as high as 45% of the approved new molecular entities.
STEVE USDIN: We haven't heard as much about drug shortages recently as we did, say, a year or so, a year and a half ago. Has that problem gone away?
RICHARD MOSCICKI: Well, no, of course it hasn't gone away. But it's been substantially improved, based on a number of approaches that have been taking place at FDA. It started, really, with a presidential directive a couple of years ago to make sure that sponsors who might face drug shortages would notify FDA early on of an impending shortage. And FDA recently, just last year, put forward its strategy for how to deal with this, and has been dealing with it now since that presidential directive.
And in fact, by making sure that we take into account the necessity of the drug for the public good and look at a number of alternative approaches and working with the sponsors, we've been able to reduce by almost 300 the number of possible shortages just in 2012, for example, looking at the statistics, with this kind of an approach.
STEVE USDIN: I wanted to ask you, when we talked about this on the show I think a year ago or so, one of the questions I asked is, what's the root cause of shortages? And there was some uncertainty at FDA and in industry about really what the root cause was. Have you kind of gone further on that?
RICHARD MOSCICKI: Well, there have been a number of analytics on this, but it looks like the biggest number of the problems are really manufacturing quality issues. And so at FDA, there's been, and at CDER in particular, we've been looking at how to change the approach to manufacturing quality, how to improve in general quality for the sponsors in their manufacturing and production methods, and also maybe rethinking into a new paradigm how to approach the way that we regulate quality, less of a focus on compliance as an issue, but more of a focus on real quality and effectiveness of production that allows you to even then develop metrics, for example, of how to measure, really, quality in the industry itself, and at the root of this, keeping the patient in mind as well, because that's really who we all want to serve.
And we need to have good quality medicines in order to serve them.
STEVE USDIN: We could talk all day. And I hope that you will come back on this show again and talk more about these things. Science is the foundation for medicine. Is that foundation cracking? We're going to talk about problems reproducing scientific results and how that threatens medical progress when we come back.
NARRATOR: Now in its 22nd year, visit biocentury.com for the most in-depth biotech news and analysis. And visit biocenturytv.com for exclusive free content.
STEVE USDIN: To discuss problems with reproducing biomedical science, I'm joined by Steve Edelson BioCentury's executive editor of SciBX. Steve, what's the problem with reproducibility, and what's the scale of the problem?
STEVE EDELSON: Hi, Steve. Thanks for having me. The problem with reproducibility is that it can't be done in many pre-clinical studies. What this means is that you have a lot of researchers going down dead end experiments that are doomed to fail before they even start. The numbers are really staggering. You've had NIH saying that there are hundreds of thousands of papers every year.
And of those, it's always been accepted that some number can't be reproduced. But a few years ago, two big biotech and pharma companies put some really distressing numbers around it. Bayer, in 2011, reported that about 25% of pre-clinical studies that tried to reproduce, it couldn't do it. Amgen, a year later, had even worse data.
They tried to replicate 53 seminal cancer papers, and they could only do it 11% of the time.
STEVE USDIN: So what are the consequences of that? One, why do scientists try to reproduce other people's results? And then what are the consequences for medicine if results can't be reproduced?
STEVE EDELSON: Well, people try to do it because it's part of the natural scientific process. It's self-correcting. You're out in this white space of new biology and finding novel targets. And it's accepted that some of the times, these targets are just not going to pan out. But when people rush to have their findings published, what you're really going to end up is with the system breaking down, and you're going to get researchers going down avenues that just should never been pursued in the first place if people took the time, the first time to get it right.
STEVE USDIN: So you mentioned people rushing to get things published and the need to get things right the first time. When we're talking about the papers that can't be reproduced, are we talking about people doing things in too much of a hurry? Are we talking about fraud? Are we talking about some other systemic problems?
STEVE EDELSON: We're not really talking about fraud. I think we could fill the entire show talking about the issues that lead to it. But Francis Collins is publishing a commentary this week in Nature that highlights the lack of reproducibility in pre-clinical research. And he said, of the hundreds of thousands of papers that are out there every year, only 12 cases of, like, actual misconduct are being pursued by HHS.
So fraud is really not the issue. One of the main issues is that researchers simply don't have adequate training in how to run a pre-clinical study. They're not using proper controls. Another issue is that academic labs may be using outdated reagents, outdated cell lines that have, like, no bearing on what the actual experience is in a human.
STEVE USDIN: NIH is spending literally billions of dollars every year to fund basic research in this country. The medical journals publish cutting edge medical work every week. Whose responsibility is it to try to fix this problem?
STEVE EDELSON: In a word, everyone. I think it has to start with the investigator, who has to just hold themselves to the standard that if you don't do the study right the first time, it's going to cause a gigantic ripple effect. And so it might delay your publication by a little bit.
It might cost 10% more of a total budget. But if you take the time to run your key experiments multiple times, if you follow proper procedures, like if you blind your pre-clinical studies, if you publish every piece of data as opposed to selectively publishing data, I think it's really going to benefit medicine as a whole.
STEVE USDIN: We've only got a few more seconds left. Quickly, what are some of the things that are being done to try to address the problem?
STEVE EDELSON: There are three main thrusts. I mentioned NIH, and what they're doing is developing a training module to train post-docs in how to run proper pre-clinical studies. They have many other initiatives as well. And by year end, they hope to select the best ones and pursue those.
There's something called the Reproducibility Initiative, which is a conglomeration of stakeholders that puts the onus on the academic to get their research reproduced by a third party. The academic has to foot the bill, but when their study is reproduced, they to get essentially a stamp of approval. And they get to publish their results in Plos One, showing that there were reproduced.
And finally, the Global Biological Standards Institute has settled on standards as the real battleground. And they are convening task forces to discuss which standards should be pursued.
STEVE USDIN: That's all the time we got today to talk with you, but we are going to be talking with Doctor Leonard Freedman of the Global Biological Standards Institute in just a minute, to talk about what can be done to improve reproducibility of science.
STEVE USDIN: To talk about the steps that can be taken to bolster reproducibility, I'm joined by Doctor Leonard Freedman, president of the Global Biological Standards Institute. Dr. Freedman, what is the Global Biological Standards Institute, and how are you trying to address reproducibility issues?
LEONARD FREEDMAN: Thanks, Steve. It's great to be here. We are a nonprofit thought advocacy education institute, really advocating a broader and more aggressive use of standards, primarily in basic research.
STEVE USDIN: So can you connect the dots? How does the use of standards lead to science that's more reproducible and more valid, more robust?
LEONARD FREEDMAN: Standards, you can't really separate, by definition, standards from reproducibility. Standards ensure that things are done in the same way, just like your iPhone will connect to my Galaxy phone because an agreed upon set of standards were used in the design of those phones. Remarkably, that's really never existed in basic biological research, for lots of reasons we can talk about.
But everybody's done things in their own way. And for a long time, that was OK, for reasons that are, I think, interesting in terms of the history of science and other sociological issues. It's become a huge issue, and rightfully so.
STEVE USDIN: One of the particular things that you're taking on, that, to me, is honestly shocking is that a lot of the human cancer cell lines that are being used in research that are then being used to try to develop therapies for cancer are not producing results that are relevant to humans. What's the problem there, and how do you solve it?
LEONARD FREEDMAN: So the problem, which, again, is really because of a lack of standards in how people of handled their own cell lines, in how they've assured their cell lines are authentic is that people have just taken cell lines from their friends down the hall, used those thinking that they're using, for example -- in a real example, in fact -- an esophageal cancer cell line, because they were interested in interesting biology affecting esophageal cancer, not authenticating, not insuring, that, in fact, that line was what it said it was, doing lots of experiments, spending lots of money, publishing papers, influencing perhaps pharmaceutical companies to take on particular lead compounds that might inhibit the growth of these tumor cell lines and then finding out that in fact they were skin cell line because of a lack of authentication, where a standard wasn't in place.
STEVE USDIN: So who imposes those standards? Who sets the standards? How do you ensure that scientists use those standards?
LEONARD FREEDMAN: No one, really. I think in the clinical world, it's clearly been very effectively laid out over the years, for obvious reasons, because it involves human beings. The FDA has done on an admirable job in terms of all the kinds of regulatory governances and standards, quite frankly, that are put on to the development, both clinical diagnostics, clinical therapies, and the release and maintenance of drugs. This just simply doesn't exist in the pre-clinical basic biological universe.
STEVE USDIN: So that suggests that approximately $30 billion a year of science that NIH funds every year, a lot of is not reproducible. If you were to fix this problem, we'd be getting a lot more scientific progress for the money we're spending, wouldn't we?
LEONARD FREEDMAN: And that's really one of the messages we're trying to convey to our scientific colleagues, that we should really regulate ourselves and that standards really are a very effective way of addressing this reproducibility issue.
STEVE USDIN: Well, thanks, Dr. Freedman. That's this week's show. I'd like to thank my guests Richard Moscicki, Steve Edelson, and Leonard Freedman. Remember to share your thoughts about today's show on Twitter. Join the conversation by using the hashtag #biocenturytv. I'm Steve Usdin. Thanks for watching.