Tuesday, April 1, 2014

A Call To Rein In Phase III Trials

Sooner or later the science will be done properly.  It should be blindingly obvious that the following is true:

1                    Doctors are trained technicians who can collect and electronically record critical data and monitor results.  After all they are called upon to do exactly this for clinical trials.   
2                    It can now be done for all medicines and all other curatives been used.  The patients can also be logically dragooned into properly building his profile for medication.  This provides a wealth of information which can also be confirmed by additional blood test as warranted.
3                    Thus all medications can now be effectively tracked on an ongoing basis over decades even.  This obviously provides a statistical gold mine were a wide range of filters can be used to eliminate false signals caused by under or over reporting.

This all means that all so called phase three trials are better done as trial releases with full monitoring by both doctor and patient.  This also encourages trust and data sharing which is very important.

This also means that problems will be quickly recognized, warnings issued and either limits soon established or even suspension ordered.

Yes the patient population is conducting the trial but it with eyes wide open and on the lookout for issues to report.  This also all happens in real timed.

Someone close to me has had experience with the Vioxx disaster where phase III was completed and a full release made.  I consider it likely that the phase III numbers got cooked there and that is what an open trial period eliminates.  The numbers themselves are too large, the number of doctors involved is too large and the patients are looking for problems.  Pass that sieve and you are surely good to go.

Even better, all 3000 plus herbals out there can also be monitored for simple statistical significance.  That might be important. I would love to crack open an herbal in which the benefit is spelled out and a level of significance is supplied.  A curative may be at the third standard deviation while an improver may be at the second standard deviation.  Most placebos will actually be used to establish the limits of a single standard deviation.

This pretty basic and we can be sure that most herbals do fit into the second standard deviation.  It needs to be known at least this rigorously.

A Call To Rein In Phase III Trials      

February 28, 2014
Posted by Derek

Here's a very nice perspective on what gets funded in drug research and why. Robert Kocher and Bryan Roberts bring their venture-capital viewpoint (Venrock) to the readers of the NEJM:

It is not mysterious why projects get funded. As venture-capital investors, we evaluate projects along four primary dimensions: development costs, selling costs, differentiation of the drug relative to current treatments, and incidence and prevalence of the targeted disease (see table). For a project to be attractive, it needs to be favorably reviewed on at least two of these dimensions. Many drugs designed for orphan diseases and cancers are good investments of scarce capital, since they tend to have relatively low development costs and selling costs and to be strongly differentiated from the current treatment options. Conversely, investors are less likely to fund drugs with much higher development and selling costs (e.g., drugs for type 2 diabetes or psychiatric disorders) and drugs that cannot be strongly differentiated from current treatment options — often because low-cost generics are available to treat the targeted condition — despite the condition's high incidence and prevalence (e.g., drugs for hypertension or hypercholesterolemia).

Since improving the rate of discovery is a rather knotty, multivariate problem, the authors turn to the economic back end of the process. They make the case for the FDA to move more towards conditional approvals, since no Phase III trial can be large enough (or long-running enough) to pick up on all the "long tail" adverse events that might be waiting out there. Current Phase III trials, they say, are often overpowered for efficacy but are still underpowered for rare events, so we're spending a lot of money rather inefficiently.

I think they've got a good point, but the FDA already gets enough flak as it is. Changing things in this way, if done too quickly (and frankly, if done too openly) would be seen by many as a bean-counting technique to shift the risk onto the paying customers. Can't you hear it now? But the world they describe would be a good one, if it's feasible:

We estimate that development costs for drugs could be reduced by as much as 90%, and the time required by 50%, if the threshold for initial approval were defined in terms of efficacy and fundamental safety. Cutting costs and time, while requiring high-quality and transparent patient registries for independent safety monitoring, would be a more informative and cost-effective approach. With the widespread adoption of electronic health records and the introduction of many low-cost data-analysis tools, it is now feasible to develop mandatory postmarketing surveillance programs that make thousand-patient trials obsolete. Large data sets would also inoculate drug makers against spurious claims such as the false association of pancreatitis with the glucagon-like peptide 1 (GLP-1) and dipeptidyl peptidase 4 (DPP-4) inhibitors. At the same time, it is essential to empower the FDA to quickly remove or restrict the use of drugs when safety signals emerge from the improved data and safety monitoring.

This moves beyond clinical science and into politics, which (as the cliché has it) is the art of the possible. Even if we agree that this move is desirable, is it possible, or not?


1. Sam Weller on February 28, 2014 10:47 AM writes...
Too much distrust out there to allow for this policy change.
From a layman perspective, this will look like nothing more than a money grab:
1. Pharmas will save 90% of costs, but will not discount the drugs accordingly.

2. More new molecules will be put on the market, and will create revenues for the pharmas, just to be pulled out later.

3. Shorter development time will mean more time under patent protection - more profits before generic competition kicks in.

4. More risk will be transferred to the patients.

2. dlib on February 28, 2014 11:13 AM writes...
The shadow of thalidomide is long....

3. Rob on February 28, 2014 11:43 AM writes...
Late last year the MHRA Expert Group on Innovation in the Regulation of Healthcare called more flexible licensing of medicines that address unmet needs - and Early Access to Medicines Scheme that would allow patients to access certain medicines at end Phase II.http://www.mhra.gov.uk/Opendocuments/OpenPDFdocuments/CON336728

4. Rob on February 28, 2014 11:44 AM writes...
Late last year the MHRA Expert Group on Innovation in the Regulation of Healthcare called more flexible licensing of medicines that address unmet needs - and Early Access to Medicines Scheme that would allow patients to access certain medicines at end Phase II.http://www.mhra.gov.uk/Opendocuments/OpenPDFdocuments/CON336728

5. watcher on February 28, 2014 11:45 AM writes...


6. Anon on February 28, 2014 11:50 AM writes...
One issue is that physicians often do not share results/labs with each other. Whether that is a patient going from a general practitioner to a specialist or changing general practitioners (or just simply seeing another because their current physician has a two week wait list). Anecdotally, I've seen this many times personally (and the physicians had EMR systems to make it easy), and I'm sure the readers here have also seen this.

Because of the lack of concern from physicians (or just laziness) information does not disseminate as quickly as it should. If it did, we would see better resolution in post marketing studies. We need to see better data from these studies if we are going to make a case for putting the drug on the market sooner. The FDA is not simply going to trust a company to keep an eye out. If this is the case, does this mean that pharma will be requesting all patients on this new drug to sign away access to their medical records? If that is the case I'm sure liability language would also be included (just as it would in a trial). Maybe it will save pharma from a few large lawsuits? If so, might that imply a lax environment for their reporting standards?

Additionally, we are talking about shifting costs away from pharma and towards insurance providers who will be resistant. A Kaiser like model would be needed on the insurance side.

I really don't see physicians or insurance providers putting in effort that will only help patients and not help their wallets.

7. Anon on February 28, 2014 11:52 AM writes...
From above, a good example of poor transfer of information in Vioxx.

8. Anonymous on February 28, 2014 12:49 PM writes...
Both # 1 and #5 are absolutely right. This is just money guys looking out for #1 in the most self-serving way possible. Maybe if George Merck was asking to try this approach and pharma companies were still viewed as “The Merchants of Life” it could be considered. However, multi-tens of billions of dollars in fines and litigation losses, over compensated ruinous executives, plus drug pricing only a big money hog could love have so tarnished the industry’s image that no sane regulator is going to get into bed with these sleaze-buckets. They need to try again. I have an idea. Let’s put the MDs and scientists back in charge with the mission of putting the patients way ahead of the company, stockholders and management. Yeah pharma might become a backwater sleepy industry again, but I would certainly invest in it for the long term.

9. Mike Parker on February 28, 2014 12:50 PM writes...
There might be some value for the health care system to shift focus from costly clinical trials to better patient monitoring post-marketing, but I find it very hard to believe that 90% of a drug's development costs could be eliminated without losing essential safety and efficacy data.

10. Samsonite on February 28, 2014 12:55 PM writes...
Can't this risk simply be covered by a regulatory policy and caveat emptor?

We have to risk peoples lives/well being either way, whether through volunteers in trials or volunteers in the marketplace.

If it was a requirement for new medications that have gone through a streamlined phase 3 trial to be clearly labelled as new to the market with potentially rare unknown adverse effects, who's really losing out?

We know by stage 3 anyway that we aren't going to be killing legions of people. We permit plenty of other risky activities in the marketplace, from motorbike driving(36 times more dangerous than cars) to tobacco and alcohol sales (that kill far more than a drug that's got to stage 3 trials is liable to do for no meaningful benefit).

Is the boogie man of a few harmful reactions in thousands really worth considerable delays and economic restrictions in new drugs coming to market? The potential benefit of reducing the cost/length of trials is massive, and the potential downside really extremely small if you take emotion and fear out of the equation.

In exchange for the money saved in trials we could easily require more extensive market follow up studies to help balance out the reduced trial data.

I don't see why in this one area of drugs that actually provide some utility we take a zero tolerance risk approach whilst allowing plenty of risk in recreational activities.

11. Sideline Chemist on February 28, 2014 1:15 PM writes...
Every (potential) change has to start with a proposal and that's what the folks at Venrock have done. Self serving--sure, but also potentially benefits everyone. I'm in agreement with #10 Samsonite that conditional approval is not inherently a bad idea, especially for a drug with limited or mixed efficacy signals in a Phase III trials. I'm thinking of rare diseases or cancers where the patient population is small to begin with, or an Alzheimer/Parkinson's/etc drug that showed some activity in a trial subpopulation. A conditional approval would allow for more rapid follow-up in broader or more select populations without having to go back and re-doing the trial design.

Moves the FDA from being the crossing guard before a drug gets onto the highway to more along the lines of being the State Police car on the side of the road watching for deviants. Lots of questions whether that's a good thing for society (pluses & minuses in both directions), lots more questions whether society would accept such a change, and lots more on how to make it actually work.

But the discussion has to start somewhere.

12. Hap on February 28, 2014 1:21 PM writes...
10: The riskiest activities we do have pretty well-known risks, and ones that have not been misrepresented. At least some, if not most, of the risky activities have risks within our control (in a car, don't speed, look around - smoking, duh...).

Medicines' risks are difficult enough to assess when likely side effects and profiles are known for doctors and patients even when they understand what the patient's needs are, and are probably more difficult for people who don't work with drugs for a living or in a related field. The risks are mostly random, rather than controllable (for example, other than by not drinking grapefruit juice, there isn't much of a way to avoid the rhabdomyolysis side effect for statins other than not take them). Lately, pharma has not been the world's most trusted industry - people (in lots of cases) don't trust that companies haven't hidden risks from them. People also probably inversely correlate cost to risk, but drugs don't work that way, and people feel unhappy accepting lots of risks from expensive things. I think the risk aversion is probably related to people's willingness to accept large risks voluntarily but to be highly intolerant of risks taken involuntarily (particularly those taken for someone else's benefit).
The costs for such a change in policy will come in the relative near term, while the benefits (more drugs and eventually more generics, which will probably lower costs some) will come later (after the first wave of drugs goes off patent). Given the bolus of older people likely to need medical care, I can see that being a problem as well.

13. Cellbio on February 28, 2014 1:33 PM writes...
While I tend to agree with the idea of conditional approval, this idea has the classic style of VCs or McK types who oversell the positives without discussing (or perhaps seeing? maybe entrepreneur's phenotype)the potential downsides.

Re cost savings, registry is not cheap and collecting high quality data on efficacy and adverse events can't be done with social media apps, so no way one is saving 90% of development costs with reduced power in Phase 3.

Second question relates to marketing limitations post-approval while essential data are collected. Limited? What are the claims granted? How is liability assessed? Do subscriptions come with informed consent like for trail participants?

Finally, will not the evils of pharma be magnified? Will marketers mobilize "patient advocacy" groups to be vocal upon review of post-marketing registry data? Did not the Genentech Avastin conditional approval give some insight into how personal opinions on how a drug "worked", delivered passionately by dying patients, obscure rational discussion of risk benefit across a population?

14. MTK on February 28, 2014 1:42 PM writes...
I'm not at all sure why the fact that these are VC's that are writing this makes a difference. These are essentially the same types of calculations that every Pharma and Biotech, or just about every other venture for that matter, does.

Basically, what is the size of investment, what are the risks, and what is the potential return on that investment?

And that investment can be time, money, effort, or opportunity costs.

The lowering of the size of an investment would obviously make risk more palatable, for any given return which in theory could lead to more breakthrough therapies. At the cost of more adverse effects certainly.

So the real question we should be asking is not a cynical "Oh sure, who's getting rich off that idea?" but rather an enquiring "Will this help more people than what we currently do?"

I honestly don't know, but I do agree with them that PhIII studies are generally overpowered for efficacy and underpowered for safety, so how do you change that if that's a problem?

15. biotechtoreador on February 28, 2014 1:48 PM writes...
"shift the risk onto the paying customers"
Isn't this what right wing nutjobs want? Shouldn't the all-knowing free market be left to decide which drugs are safe and which aren't? Nope, I don't see any problems with Uncle Bob in Springfield being able to choose a drug without adequate studies being done. I mean, if it's toxic people wouldn't buy it.....
Trump/Palin in 2016!!!!

16. Anonymous on February 28, 2014 2:03 PM writes...
I don't see any problem at all with this model - provided that the greater risk to patients is compensated by a significantly reduced price agreed upfront! So if pharma want lower risk they must be prepared to take a lower return. But having the option to choose between low risk-low return and high risk-high returns adds economic value overall.

17. MTK on February 28, 2014 2:23 PM writes...
well said

18. Anonymous on February 28, 2014 3:18 PM writes...
What about the idea of dynamic clinical trials that was floated by the FDA some time ago. Were not they supposed to make clinical trials less expensive and less long?

19. steve on February 28, 2014 6:36 PM writes...
I think these comments must be coming from academics. If one of the posters had $1M to invest, where would you invest it? Would you put it into a high risk drug with a risky long-term development that requires huge Ph3 data (like a diabetes drug) or one that has a better chance to succeed like an orphan? It's all theory until it's your money on the table.

20. Fat Old Man on February 28, 2014 6:58 PM writes...
I'm not a clinician but:
Shouldn't drugs tested in a superiority trial be higher cost than drugs in a non-inferiority trial? Superiority requires more patients than comparability. Did they get it backward or am I missing something?

21. Samsonite on February 28, 2014 7:01 PM writes...
I agree from a political perspective there's little incentive to make this policy change, however there's little political incentive to make any positive policy change so I'll talk about the argument on its own merits.

It's important to make a distinction between something unknown and something that exists between bounds of probability.

The risks of a drug that's made it to phase 3 clinical trials are not as you have suggested "random". In fact by phase 3 we have narrowed down the short term/acute side effects to fairly accurate numbers. The long term side effects also exist within probability bounds. We might discover some long term health risk that weren't anticipated, but it will be a matter of shaving years off peoples QALYs, not striking thousands dead in their prime. The potential magnitude of effect for things that have made it to phase 3 simply aren't that great or they would have showed up in earlier trials.

If people are intolerant of unknown risks, surely these people can just not take these medications that would be clearly labelled as still under investigation? The idea that any medicine is definitively safe after being put through trials is bogus anyway. There's no rubber stamp of safety in science, our knowledge continually develops and it sends an erroneous message to the public about the nature of risk and health to put trial data on a pedestal given all its limitations. Scientific understanding has shifted on a great many mass market drugs, from beta blockers to aspirin. Health is so conditional its impossible to say that drug is and will always be safe for everyone. What we can say is we know its not going to kill/seriously damage over X% of the people who take it and we understand enough about the mechanism of action to say whether its likely to be carcinogenic, teratogenic, neurotoxic etc.

There's a degree of randomness in all risk. There's plenty of surgeries where you have a random chance of dying. We don't say no one can have the surgery because you can't control the chance you might die, in fact patients are provided with very little detailed information on risks before a surgery and even surgeries with known high mortality rates are permitted under the right circumstances.

The argument of it being an unknown (therefore unacceptable) risk simply doesn't make sense to me if we're already allowing people to take known serious risks for no good reason. By releasing a drug to market we'll find out much sooner what the exact long term risks will be. we can always pull drugs from the market if they turn out to be unexpectedly harmful, as we have in the past and will do in future.

22. Derek Freyberg on February 28, 2014 7:57 PM writes...
"We estimate that development costs for drugs could be reduced by as much as 90%, and the time required by 50%, if the threshold for initial approval were defined in terms of efficacy and fundamental safety."

If we take what the authors state, namely that 40% of the cost of development is Phase 3 trials, then 60% of the cost is spent even before Phase 3 is entered, and the maximum that could be saved even by doing no Phase 3 studies is 40%. Thus the authors' conclusion that "as much as 90%" of teh development cost could be saved is disingenuous at best - they might as well have said "up to 100%" and it would have been essentially as true, i.e. not true in any reasonable sense.

As for time saving, Phase 3 trials are not quick, though in my experience sponsors try to move them as fast as they can - because they think they have a winner and the sooner they get it approved the better. But a lot of development time - if we define "development" as being from the start of the program to NDA approval, and I think that's how the authors intend it, since they talk about "from bench to bedside" - is spent in hit to lead to preclinical to Phase 1 to Phase 2 before we can even talk about saving of time by reducing the size of Phase 3.
For this, and for a number of the reasons mentioned by earlier commentors, I call BS on this article.

23. Cellbio on February 28, 2014 10:08 PM writes...
Not sure of others, but I have worked on 8 drugs, 3 remain in trials, two were approved.
If I had a million to invest, I would know that I could only get through a well done non-clin package, so would base my investment on the quality of the product candidate and its discrimination from the competition at that stage. The non-clin data would be designed to highlight the desirable advantage that the molecule had over the 100s to thousands found on the shelves of pharma.

To your point re huge Ph3 data, I would not invest in a program where the purchaser would need a 10,000 patient phase 3 to show gain a label advantage.

Orphan designation might mean the overall clinical dollars required would be lower since enrollment would be terribly long for rare diseases. It also offers market exclusivity advantages. It does not, however, offer any greater odds of success. Pharmacology and the complexity of human disease still rule supreme.

24. Hap on February 28, 2014 11:02 PM writes...
I wasn't arguing that unknown risks were necessarily untakeable, but that the significant risks people are normally willing to assume generally differ in significant ways from the risks with drugs. Other than the risks that Cellbio has brought up, I don't think making approval easier in this way and more contingent would necessarily be bad. I think that people's perception of risk and of pharma (and perhaps of VC investors, who would receive a lot of the benefits under this plan with few of the costs - patients, insurance companies, and pharma would bear most of them) make this hard.

25. Phage cocktail on February 28, 2014 11:38 PM writes...
Maybe i'm a cynic but does but "lower bar for getting medicines to patients" mean "increase type I error and soften endpoints"?
The vision sounds fine, but I suspect putting a specific protocol/stat plan together would be a bit more controversial.

26. Sofia on March 1, 2014 6:04 AM writes...
No. 16 wins the thread.

27. Cellbio on March 1, 2014 3:47 PM writes...
The most common failure in phase 3 is lack of efficacy, either compared to placebo or standard of care. Not sure how under-powering trials will help. Sure, adding subjects to enable accurate measure of smallish differences may not be a solution for approval or product launch into competitive spaces, but smaller trials will likely be a mess with the variation seen in trials.

The more I think about it, the more this proposal seems lacking to me. As for reduced price, does this remain steady, or can the company up the price when post-marketing studies clarify pharmacoeconomics. And since marketing claims are tied to what you roved in the clinic, how exactly to you state health and economic benefits in this model? In other words, price is lower because benefit is uncertain?

The most sense I could make of this is to allow more rapid approval with a period of reduced price off of a retail price that comes into effect with proof of risk/benefit. But again, the only case of this I know of (are there others) did not go smoothly once data clearly showed the R/B ratio was not confirmed.

No comments: