COVID-19 cases are reduced
by [49%, 29%, 16%] respectively when taken within ~[70, 94, 118] hours of
exposure (including shipping delay). The treatment delay-response relationship
is significant at p=0.002. The data is consistent with earlier treatment being
even more effective.
the most important
cases to consider are the treatment delay-response relationship and the
shortest delay to treatment (2+ days in this case). The shortest delay to
treatment is significant @94%
versus all placebo. (Treatment delay data is in the Supplementary
we expect an effective
treatment to be most beneficial early, with reducing benefit as treatment
is delayed. By simulation, assuming that cases occur randomly according
to the observed frequency, the probability that the results
follow the observed trend of earlier treatment being better, >10% absolute
benefit change between days, and >15% average benefit, is 0.2%. Since we have
performed 2 tests, conservative Bonferroni adjustment gives us p =
Statistical significance here has been confirmed by
Treatment is relatively late, ~70 to
140 hours after exposure, including the shipping delay. Enrollment was up to 4
days after exposure. The paper does not mention the shipping delay but partial
details are provided in the study protocol. They are not clear but indicate no
shipping on the weekends and a possible 12pm cutoff for same day dispensing
and mailing. Assuming that enrollments were evenly distributed between 6am and
12am each day, we get an average of ~46 hours shipping delay. Wiseman et al.
have found the delay may be up to 3.5 days. We have asked for shipping details
and will update with more accurate values when available. In any case the
treatment delay is quite long and there is no overlap with the more typical
delays used such as 0 - 36 hours for oseltamivir. Another source of treatment
delay is that the reported exposure may not have been the one that gave the
patient COVID-19 - people may have been exposed multiple times before the
Authors initially believed that 3 days
since exposure (excluding shipping delay) was the maximum delay of interest,
they modified this mid-trial to add an additional day delay. With the original
trial specification, they found a 30% reduction in cases, p=0.13. If the trial
was not ended early, and if the observed trend continued, 95% significance
would have been reached after about 420 patients per group, which is less than
the original trial specification of 621 patients per group.
The authors conclude "[treatment] did
not prevent illness compatible with COVID-19..", but as above this does not appear to match the data. In the context of
their chosen statistics, they could say: "the data suggests a benefit for
treatment, but when including the additional delay added mid-study, not
analyzing the expected trend for earlier intervention being more effective, and
with only 107 cases, we have not yet reached >95% statistical
Authors say that they halted the study
due to conditional power analysis, but if additional people have the same or
even slightly worse results, >95% statistical significance in their metric
will be reached, even when including their added 5+ days case. Further, the
data is consistent with the possibility that 0 and 1 day delayed treatment is
even more effective.
A note about power: it may seem that
with 821 participants the study should have relatively high power. The problem
is that only 107 had COVID-19, so the sample size is too small. Since
relatively few get COVID-19, the number that need to be treated to prevent a case
increases, and looks relatively high compared to other studies. But this is a
treatment for preventing death in a global pandemic with a current death toll
of , and the treatment being
studied is very inexpensive with very good and highly studied safety in
Only 75% of people reported taking the
medication as directed. Actual compliance could be lower. In the OFID podcast,
Dr. Boulware notes there were fake submissions with 555 numbers that were
removed, there may be more fake submissions that were not identified.
Authors test late post-exposure use,
primarily in healthcare volunteers. The primary outcome was having COVID-19
within 14 days. The primary outcome is not the most interesting in terms of
the pandemic where the main concern is mortality and morbidity.
Secondary outcomes of hospitalization
and death are more relevant. The study has a CFR of 0 and
IFR of 0. There was no mortality (or post COVID-19 recovery morbidity)
reported. They report 2 hospitalizations but do not provide details.
No serious side effects
were seen, even with the dosage used which is higher than typically
Authors had an objective to intervene
before the median incubation period of 5-6 days, but intervention is likely to be
more effective very early, as with Oseltamivir for example which must be taken
within 2 days (and is likely much more effective earlier). See also the NEJM editorial
"In a small-animal model of SARS-CoV-2, prevention of infection or more severe
disease was observed only when the antiviral agent was given before or shortly
Research shows the placebo used
(folate) may be protective for COVID-19 
Thanks to the authors
for their very important, innovative, and interesting study, hard work and
dedication. We hope they can revise their conclusions, and
we hope they can continue the study, perhaps for use within 24 - 48 hours, and
ideally with more fine-grained treatment delay information (hourly).
Additional notes from the NEJM editorial
trial has many limitations, acknowledged by the investigators.
trial methods did not allow consistent proof of exposure to SARS-CoV-2 or
consistent laboratory confirmation that the symptom complex that was reported
represented a SARS-CoV-2 infection. Indeed, the specificity of
participant-reported COVID-19 symptoms is low, so it is hard to be certain how
many participants in the trial actually had COVID-19. Adherence to the
interventions could not be monitored, and participants reported
less-than-perfect adherence, more notably in the group receiving [treatment].
In addition, those enrolled in the trial were younger (median age, 40 years)
and had fewer coexisting conditions than persons in whom severe COVID-19 is
most likely to develop, so enrollment of higher-risk participants might have
yielded a different result.
The trial design raises questions about the expected prevention benefits of
[the treatment]. Studies of postexposure prophylaxis are intended to
provide an intervention in the shortest possible time to prevent infection. In
a small-animal model of SARS-CoV-2 infection, prevention of infection or more
severe disease was observed only when the experimental antiviral agent was
given before or shortly after exposure. In the current trial, the long delay
between perceived exposure to SARS-CoV-2 and the initiation of
[treatment] (≥3 days in most participants) suggests that what was being
assessed was prevention of symptoms or progression of COVID-19, rather than
prevention of SARS-CoV-2 infection."
Note that author's comments also differ
from the published conclusion - for example in the OFID podcast Dr. Boulware
has said: "There’s probably two reasons – one is either it just doesn’t work,
or the other option is we just didn’t get it to them quick enough. So if you
read the tea leaves and look at the subgroup analyses, the people that got
enrolled within one or two days of exposure did better than the people that
did three or four days later." (8/13: we have removed a comment from Dr. Lewis
because it was deleted and Dr. Boulware indicates it was incorrect).
know how many people will get COVID-19 in the future, but based on deaths to date,
a treatment which is x% effective could have saved:
17% effective could have saved lives.
30% effective could have saved lives.
49% effective could have saved lives.
Please send us corrections, updates, or comments: