No, this is not about cheap Z-packs (azithromycin) on the Internet. From an online dictionary – discount - to leave out of account; disregard. This is what the FDA is doing to antibiotics when they “discount” the antibiotic treatment effect in calculating their recommended non-inferiority margins. What is he talking about you ask!
In studying new antibiotics in clinical trials, we usually use a comparative design to show that the new antibiotic is at least not inferior to some already approved and active control antibiotic. The extent of non-inferiority that might be possible in the worst of all possible statistical cases is what we term the non-inferiority margin. So, for example, if we run two trials of a new antibiotic with the NI margin set at 15%, the chances of falsely concluding that the new antibiotic is not inferior to the control is less than 3%. Lately, the FDA has been conservative and has been asking for trials with an even more conservative margin of 10%. Well – you remember from antibiotics 101 that non-inferiority trials assume that the control antibiotic used is actually better than no antibiotic at all (placebo).
So, how does the FDA decide on the margin and why do we care? In the case of skin infections they are using the trials of sulfonamides carried out in the 1930s where the antibiotic was compared, not to placebo, but to UV light therapy that actually was shown to be somewhat effective in therapy of erysipelas, a serious skin infection. So the first level of discounting is that the FDA is not using a true placebo (no robust trials like that exist). In looking at those 75 year old data, they look at the difference between UV light and antibiotic treatment. But – in looking at this difference, they calculate a 95% confidence interval and then take the worst number (lowest difference or poorest treatment effect) that would occur one time in 40 trials. This unlikely number, which may grossly underestimate the treatment effect, is what they use. This is the second level of so-called discounting. Then, wanting to be extra-conservative and desiring to cover possible differences in patient population and trial design from the 75 year old trials until now, they discount the treatment effect still further by some arbitrary percentage. They assume arbitrarily that antibiotics today would perform worse than they did 75 years ago. This is a third level of discounting.
The non-inferiority margin cannot be greater than the treatment effect. Of course, the FDA has already reduced the treatment effect by using improbable numbers as shown above. So, knowing the treatment effect, the FDA then says, arbitrarily, that the margin cannot be more than 50% of the treatment effect. There is no particular scientific basis for 50%. It could be 90% or 10%. This is then the fourth level of discounting. This discounting gets us to the latest FDA-generated conundrum. Based on these sorts of statistical shenanigans, during the recent FDA/IDSA workshop on clinical trial design for anti-infectives, the FDA implied that a 5-6% margin would be feasible for skin infections and a 7% margin would be good for nosocomial pneumonia. Such trials are simply infeasible. The patient numbers required are too great. And they know it. And, as I also noted above, given that the chances of actually approving an antibiotic that is 10% inferior to the control antibiotic are less than 3% given two successful trials at the 15% margin, what are we thinking?
I just discussed this with George Talbot. He told me that he had spoken with Dr. Joe Toerner of FDA, who gave the August 2 presentation that summarized the FDA’s current thinking on determination of the non-inferiority margin for skin infection studies. (George and Joe are co-Chairs of the Foundation for the NIH Working Group on endpoints for skin and community-acquired pneumonia trials.) Joe noted that the new FDA guidance on non-inferiority trial design allows for some flexibility around the usual 50% discount of the treatment effect. I personally corresponded with Ed Cox on this and he also implied that the FDA would be flexible on the discounting in calculating the non-inferiority margins required for skin infection studies. Bottom line – the FDA may well continue to accept a 10% non-inferiority margin for skin infection trials.
So – all of us who want to have new antibiotics for our resistant infections – take heart! There is hope! But man! Talk about Sisyphus!
No comments:
Post a Comment