More Bad News from Neverland. The FDA Guidance on Nosocomial Pneumonia.

Image via Wikipedia

The FDA has just released its new Draft guidance on clinical trial design for antibacterial drugs in the treatment of hospital-acquired pneumonia (HAP) and ventilator-associated pneumonia (VAP).  As anticipated, they call for an endpoint of 28 day all cause mortality.  But the devil is in the details.  The FDA suggests that patients at high risk of dying (those with APACHE scores of 15 or greater) be enrolled to allow for a treated mortality rate of around 20%.  That would allow for a non-inferiority margin of 10%.  BUT – the analysis population is the microbiology intent to treat (MITT) population – that is, those patients enrolled and treated who have an identified bacterial pathogen at study entry. 

Once again, in the appendix to the guidance, the FDA has gone to extraordinary and unscientific lengths to discount the treatment effect of antibiotics in VAP such that they arrive at an infeasible trial design. They demonstrate that inappropriate therapy for VAP is associated with a 62% mortality and that appropriate therapy is associated with a 20% mortality.  The treatment effect of appropriate vs. inappropriate therapy is, therefore, 42%.  Of course, inappropriate therapy is not no therapy, and 42% as a treatment effect is therefore already conservative.  But, the FDA is not satisfied with that.  They go on to apply their 95/95 rule using the upper bound of the 95% confidence interval for treatment effect of inappropriate therapy and the lower bound for appropriate therapy yielding a treatment effect of 29% instead of 42%. They then discount the treatment effect of 29% by an additional 30% for good measure. The FDA justifies their additional 30% discount by claiming that this is necessary to correct for “uncertainties” of the historical database.  This brings the treatment effect or M1 down to 20%.  How convenient!  They now take 50% of that and call that a justified non-inferiority margin of 10%. (Is 10% starting to sound like a familiar number?)  But since the margin simply has to be smaller than the treatment effect, the margin could just as easily have been 19%. But of course the entire discounting argument is overly conservative and irrational.

So, as an amateur developer, I’ve been trying to understand the practical consequences of this design.  Lets take VAP since the FDA seems to focus on VAP in their guidance. In my calculations I have made the following assumptions:

Cure rate = 80% (20% mortality). 

Evaluability – as far as I can tell, in modern trials, the MITT population is about 45% of the enrolled population.

NI margin = 10%

90% power (to exclude the chance of falsely concluding inferiority as much as possible).

I calculate that one would need to enroll over 1200 patients per trial.  That would be at least 2400 patients in two trials.

I believe that the largest HAP/VAP trials ever performed were the ATTAIN-1 and -2 trials by Theravance/Astellas. They enrolled 1518 patients over about 28 months.  I don’t know how many centers were used. They powered their trials to show a 20% non-inferiority for treatment of MRSA infection compared to vancomycin.  Non-inferiority margins of 15-20% used to be common for VAP trials since these patients are so hard and so slow to enroll and since the trials are so expensive to run.  Those were the days when we actually considered feasibility of trial designs before we issued guidance. Two trials at 1200 patients per trial in VAP is, once again, totally infeasible.  Even just in terms of the time that would be required to complete such a trial.

I calculate, based on recent enrollment rates as cited by Steve Barriere of Theravance, that such a trial could take anywhere from 5 to 50 years to complete. In the current FDA guideline, the suggested severity of illness (APACHE score > 15) will slow enrollment.   The number of centers participating in the trial and, as a corollary, the number of competing trials will all affect enrollment rates and costs.  After even 5 years, the trial design and comparator might already be obsolete.

In terms of cost, VAP trials are probably the most expensive trials we currently undertake.  I’m guess that costs now are in the order of $30-50,000 or more per enrolled patient.  If the analysis population will now be the MITT population, these costs could be even higher.  Frank Tally estimated that in Cubist’s failed (for lack of enrollment) trial of daptomycin in the treament of bacteremia caused by vancomycin-resistant Enterococcus, their costs were $250,000 per evaluable patient.  They spent $20 million before jettisoning the trial.  At $50,000 per enrolled patient, a 2400 patient set of trials in VAP would cost $120 million.  No company, let me repeat that, NO COMPANY will take this on!  The costs rapidly outstrip any return on investment.

What about superiority trials as a solution?  We’ll address that in a future blog. 

Have a great day!

Footnote: As far as I can tell, Theravance never published the full results of their trials.  I can only find abstracts and a very brief and incomplete summary of their results on clintrials.gov. Now that it is clear what the FDA wants – will Theravance be able to provide it?

Ear Infections - Round 2

Image via Wikipedia

A recent article appeared in the Journal of the American Medical Association regarding the treatment of infections of the middle ear of children (acute otitis media or AOM).  The article caused a stir because the authors concluded that antibiotics have a limited benefit, that there was little difference among antibiotics, and that antibiotics also had side effects.  They seemed to indicate that expectant therapy would be a reasonable recommendation for dealing with these ear infections.   While I don’t argue with this conclusion, I think that it would be good to be clear on exactly which patients we are talking about and at what time we decide whether patients have benefited or not. 

First, the article in JAMA was a meta-analysis.  That is, it just reviewed results from previously published studies and kind of pooled the data.  For the antibiotic therapy of AOM, they chose studies that looked at clinical response to treatment after 14 days.  At 14 days, they showed that amoxicillin had a success rate of about 73% compared with 60% for placebo – a 13% treatment effect.  The antibiotics also had side effects like rash and diarrhea.  No other time points were examined.  A variety of antibiotics had similar results at 2 weeks.  Many news services picked up on these results to indicate that the use of antibiotics to treat ear infections is usually unnecessary and could represent an abuse of antibiotics that might, in turn, select for antibiotic resistance.

The meta-analysis published in JAMA was picked up by a number of news services including The Boston Globe, CNN, and others. They all echoed the authors’ conclusions that antibiotic treatment is of only modest benefit and is associated with side effects.

The studies that were reviewed did not use a standardized method of diagnosis of middle ear infection. One thing we have learned over the years is that a definitive diagnosis is important and can be easily made simply by looking at the eardrum through an otoscope, seeing that the drum is red, swollen, and immobile when you blow a little air on it. That has a 95% chance of correctly diagnosing a bacterial infection of the middle ear.  Thus, if some patients who did not actually have bacterial otitis media were admitted to the studies, the antibiotic effect would be lessened. 

On the other hand, the researchers who recently reported their results on clintrials.gov as I noted in my previous blog on otitis, carefully documented the presence of middle ear infection in each case.  They also looked at responses while patients were still on therapy (within 72 hours) and at 7 day endpoints for their trial. In the results from their study, which should probably be considered the definitive study, 23% of placebo treated patients were clinical failures compared to 3% of antibiotic treated subjects at the on therapy visit (within 72 hours).   16% of antibiotic-treated patients and 51% of placebo patients were failures at end of therapy (7 days).  These data establish a treatment effect of 11% during the first 72 hours and 35% during the 7 days of therapy.  This treatment effect being more than twice what was seen in the meta-analysis published in JAMA might be explained by the heterogeneity in the population studied and the 14 day endpoint used by these studies.

To conclude that antibiotics do not work well and are not justified in light of these most recent data seems premature. 

The Infectious Diseases Society Sounds Off on the FDA and Skin Infections

Well, boys and girls, it looks like at long last, the Infectious Diseases Society of America (IDSA) has taken off the gloves.  I guess they are now as frustrated as the rest of us with the FDA’s inability to provide a clear and feasible path forward for antibiotic development.  Their frustration takes the form of comments (you have to scroll down to the IDSA Comments section in this link) to the FDA’s latest guidance on clinical trial design for studies of acute bacterial infections of the skin and skin structures.  This is ironic, because I was just relieved that we finally had guidance that proposed, at least, a feasible design.  But the IDSA has a number of critical comments.

First, they note that in their view, “ There is no doubt that the lack of clear and feasible FDA guidances has contributed in a significant way to the growing crisis (of our poor antibiotic pipeline in the face of growing resistance).”  Their first specific criticism is of the crazy estimate of the treatment effect (M1) of antibiotics for skin infections of 12%.  As I explained in a previous blog, they use excessively conservative estimates and pseudoscientific discounting to arrive at this preposterous figure. Luckily, in discussions with sponsors, they have been ignoring their own calculations, probably because trials based on the assumption of a 12% treatment effect would still be infeasible. 

The IDSA then takes aim at the proposed endpoint of cessation of spread of the lesion at 72 hours arguing that since even the sulfonamides worked 99% of the time, such an endpoint would not discriminate among different therapies. They also note that for the practicing physician, this is a completely irrelevant endpoint.  The IDSA proposes a much more rational endpoint of clinical cure encompassing mortality at 28 days.

They also take issue with the lesion size requirement (>75 cm²) since that has never been validated anywhere. The IDSA takes the FDA to task in a number of other specific areas of the guidance as well.  They take aim at the use of the Snodgrass trial comparing sulfonamides to UV light from the 1930s as the modern totem to which all trials must compare. 

As I noted, this is the most aggressive stance I have seen by the IDSA vis a vis the FDA.  I guess that I have become so numb over the years that I was just grateful to have a trial that I thought industry could actually accomplish.  But the IDSA wants more.  They want the FDA to withdraw from the guidance their crazy stance on margins even though the FDA accepts margins larger than what they propose.  I suspect that the FDA’s proposal was more for political consumption than for sponsors.  See my blog on Markey and Grassley.

IDSA – Kudos!  And welcome to my world of fantasyland where science and the rational approach to solving problems disappears in the face of politics and pseudoscience. I only ask that as you and the FDA continue to work on issues of guidance for the study of antibiotics, you keep one concept in mind above all – feasibility.

FDA – these are the US physician experts in infectious diseases laying down the gauntlet. They are telling you that your approach to regulation of antibiotics has been misguided and inappropriate.  What are you going to do about it?

Next on this blog – Otitis and Antibiotics – Round 2.

Antibiotics, Incentives and Disincentives - continued.

Going back to that article by Andy Pollack in the NY Times, he includes a graphic from the FDA showing a substantial increase in the number of new antibiotics entering clinical development over the last few years. I think it is worth taking just a few minutes to put this in perspective.  It is absolutely true that the number of antibiotics entering the pipeline is increasing – especially in 2008 when a record 14 new molecules entered development in the US.  But, having looked at the same data from the FDA, I note the following.  The average for the decade between 1990-1999 was 4 new antibiotics entering clinical development per year.  For the last decade (2000-2009), it was 5.7 molecules per year – a 42% increase.  The worst period was the years between 1996 and 2003 when only 3.25 molecules entered development per year.  In general, only 1 in 5 of such molecules will make it all the way to the marketplace, and it takes say 7-10 years to get there once clinical trials start. So, for the next decade, at best, we can expect new antibiotic approvals to still be in the range of about 1 per year.  This is still 3-4 fold worse than during the 1980s.

The other point I wanted to emphasize is that these molecules are mostly entering at the earliest stage of development, Phase I.  A few are entering already at a later stage, Phase II.  But neither of these stages is really expensive.  So most are coming from biotech companies who will not have the means to develop the molecules beyond Phase II.  Phase II trials cost, say, $6-10 million.  Phase III trials, and you have to run two of these for each approved indication you want (with a few exceptions), cost on the order of $35 million when all is said and done.  Biotech investors cannot support this level of spend. Phase III trials must be funded either by public markets or by large pharmaceutical companies.  The former is still essentially non-existent.  The latter is a rapidly dwindling resource, especially for antibiotics.  So, incentives for both large and small companies are still needed, even if there are encouraging signs that the number of new antibiotics entering clinical development in the US is increasing.

An Op-Ed in a recent Boston Globe calls for an incentive combined with a sort of disincentive.  I have been unable to read the full article (since they actually want to charge me money!), but I think I understand what Drs. Kesselhem and Outterson are getting at.  What they claim is that pharmaceutical companies inappropriately market, or at best fail to appropriately market antibiotics such that abuse is encouraged to feed their bottom lines.  They argue that antibiotics are underpriced relative to their value to society.  I have one word from Drs. Kesselheim and Outterson – Zyvox.  Zyvox is an antibiotic that is priced to reflect its high value in the fight against MRSA and also to tie it to use only when an MRSA infection has been documented or at least is highly suspected.  Otherwise, no one would pay the price they charge.  The same is true for a number of antibiotics used in hospitals that are only available intravenously.  The argument that antibiotics are cheap usually refers to those used for infections in outpatients in indications that are, essentially, no longer considered valuable by the FDA – ear infections, bacterial bronchitis and acute bacterial sinusitis. In those infections, there is no longer a question of price since it is not feasible to develop antibiotics to treat them at the present time.

The idea of providing a disincentive, though, to pharmaceutical companies who abandon antibiotic discovery and development is not new.  Dr. Louis Rice suggested this a number of years ago.  He calls it both an antibiotic abuse tax – to make companies pay for the years they allowed patients and physicians to abuse antibiotics by using them when they were not necessary – and a disincentive for abandoning the field.  In this way, companies who continue to try and discovery and develop antibiotics would be exempt from the tax.  Combining this kind of disincentive with the kinds of incentives the London School of Economics has proposed for antibiotic discovery and development would be reasonable.  Although I think the incentive alone would be enough, the tax would help pay for the incentive – making it more palatable politically.  Then again, didn’t we just elect a very anti-tax congress?

Antibiotics and Incentives

A recent article by Andrew Pollack in the New York Times discussed the idea of incentives for the pharmaceutical industry to encourage them to develop new antibiotics. Mr. Pollack points out that while antibiotic resistance is rising, the antibiotic pipeline remains dismal with only two new antibiotics approved by the FDA in the last two years. On a more optimistic note, he also shows data from the FDA that the number of antibiotics in clinical trials has soared since 2003.  Some of these trials are being carried out by large pharmaceutical companies and will likely lead to marketed new drugs.  These include ceftazidime + NXL-104 and ceftaroline-NXL104 trials now being driven by Astra-Zeneca and Forest/Cerexa and Calixa-tazobactam from Cubist (a mid-sized company).  Unfortunately, many of the trials cited by the FDA data are early stage trials being funded by privately held, venture-capital-based biotechs.  These small companies will, probably, be unable to afford the phase III trials that will be required to register a new antibiotic.  They will therefore be dependent on one of two sources of funding; (1) the public markets or (2) large pharmaceutical companies.  But the public markets are virtually non-existent at least for now.  Trius’ public offering resulted in a vastly lowered share price compared to what the company had envisioned.  It is still not clear to me how they will actually pay for the two phase III trials required to register their drug, torezolid. Large pharmaceutical companies that are still interested in antibiotics is a rapidly disappearing species with only 5 of the 13 largest companies in the US and Europe still in the business of discovering antibiotics.  And when companies abandon the field, they lose their expertise.  They become unable to appropriately evaluate new opportunities.  This leads them either to make poor decisions or simply leads them away from in-licensing antibiotics. 

The Infectious Diseases Society of America has proposed a number of incentives for industry.  They also strongly support enactment of The STAAR Act (STRATEGIES TO ADDRESS ANTIMICROBIAL RESISTANCE ACT).  The STAAR Act, while an important step forward, does not really address key incentives to the industry for the development of new antibiotics active against resistant pathogens. Incentives supported by the IDSA do not really include those most likely to work, in my opinion.  The incentives that I think are most likely to work include (1) the wild card patent exclusivity, (2) the push-pull mechanism from the London School of Economics report and, perhaps most importantly, (3) a feasible path forward for new antibiotic development from the FDA and EMEA.  Although the latter might not be formally considered an “incentive”, we will have no antibiotics without it.  The wild card patent exclusivity would allow a company like Pfizer to obtain an additional 6 months to two years of patent exclusivity for a drug like Lipitor in return for marketing a new antibiotic active against resistant pathogens.  This has been shown by Spellberg and coworkers  to be cost effective given the societal costs of antibiotic-resistant infections.  The push-pull mechanism would provide support for development, say for the phase III trials required for registration, plus a guaranteed initial government purchase of a new antibiotic for resistant infections.  This would reduce the company’s risk and at the same time provide an initial sales spurt that would more than offset launch costs and provide an immediate return on investment for the company.  In my view, this is superior to the wild card option in that the costs to taxpayers would be less, but the incentive would still work quite well.

But all this remains a dream. So far, no one except me and the IDSA, supports giving money to pharmaceutical companies and we still do not have the kind of regulatory path forward that we need to even consider developing needed new antibiotics.  Somewhere, over the rainbow . . . 

Related articles

• Anti-antibiotics: Bugs, drugs and bureaucrats (newscientist.com)

Ear Infection - Antibiotics Work!

The data are in!  The NIH funded placebo-controlled trial of augmentin vs. placebo in children ages 6 months to two years with well documented otitis media (AOM) shows that antibiotics work at some cost of increased adverse events.  In this study, careful tympanography (assessing the swelling, redness and lack of movement of the tympanic membrane) was carried out to document that patients had acute otitis media usually caused by bacterial pathogens and not the confusing, and usually non-bacterially caused serous otitis.  A drawback of the study was that ear punctures were not performed – but even as it was, it took three years to complete enrollment.

The study enrolled 291 patients of whom 284 completed therapy. The only serious adverse events in the trial included one case of mastoiditis and one of pneumonia in the placebo arm. There was an increased incidence of diarrhea, diaper rash and thrush in the antibiotic treated group. Although the overall protocol-defined time to resolution of symptoms did not differ between groups, the antibiotic group did have a significantly faster rate of durable resolution of symptoms (symptoms resolved over two questioning periods = 24 hours) than the placebo group.  23% of placebo treated patients were clinical failures compared to 3% of antibiotic treated subjects at the on therapy visit (within 72 hours).   16% of antibiotic-treated patients and 51% of placebo patients were failures at end of therapy (7 days).  These data establish a treatment effect of 11% during the first 72 hours and 35% during the 7 days of therapy. The latter number should establish a reasonable NI margin for non-inferiority trials in otitis media with an endpoint of 7 days. Although not powered for this result, it is possible if not likely that antibiotics would prevent complications such as mastoiditis and pneumonia in patients with AOM. Since this study is contemporary, there is little justification for the usual discounting in defining the NI margin for a non-inferiority trial as usually practiced by the FDA. A 10% margin seems justified and reasonable.

Hopefully, this should be the last placebo-controlled trial needed for acute otitis media in young children.  The data show that when this diagnosis is established carefully, even in the absence of clear microbiological culture confirmation (which would have required puncture of the ear drum for culture), antibiotic therapy is superior to placebo.  It also confirms what most parents already knew – that symptoms resolve more quickly and completely when antibiotics are given than when they are withheld.  One plausible and likely explanation for the failure of previous trials to show a difference between antibiotic therapy and placebo in acute otitis media is the lack of a clear and consistent diagnosis using tympanography.

The current FDA and EMEA guidances require that placebo controlled trials be used to study the efficacy of antibiotics in acute otitis media.  The data provided by this NIH-sponsored trial belie the basis for the regulatory guidance by demonstrating a clear antibiotic effect.  Where do we go from here?  Is there a sponsor out there who would like to rediscuss phase 3 trial design with the FDA or EMEA?  Sanofi-Aventis?