First Signal of Vaccine-Mediated Protection Against HIV Infection is Both Celebrated and Questioned
By Richard Jefferys
On September 24, 2009, the surprising announcemnet that a vaccine combination had shown efficacy in preventing HIV infection was splashed across the world media. The news emerged from a huge trial in Thailand, conducted under the aegis of the Thai Ministry of Public Health, the U.S. Military HIV Research Progam (USMHRP) and the National Institute of Allergy and Infectious Diseases (NIAID), which enrolled 16,402 mostly low-risk individuals (in the Rayong and Chon Buri provinces) and randomized them to receive a combination of two vaccines, ALVAC vCP1521 and AIDSVAX B/E, or placebo shots.
Modest Efficacy?
The term most commonly used to describe the result was modest; the reported efficacy was 31.2% and it achieved statistical significance by the thinnest of margins. Nevertheless, as the first-ever signal of efficacy in an HIV vaccine trial, the news was widely hailed as historic. In raw numbers, 74 out of 8,198 volunteers who received placebo immunizations became infected, compared to 51 out of 8,197 volunteers who received the vaccines. As a coprimary endpoint, the trial also measured viral load levels in participants who acquired HIV infection, but there was no difference between the vaccine and placebo groups. The reason the total participant numbers didn’t quite add up to 16,402 was because 7 volunteers who were randomized and immunized were excluded from the efficacy analyses after it was found that they were HIV-infected at the time of receiving their first vaccination; this type of exclusion is standard in prevention trials. (In the previous two efficacy of trials of AIDSVAX, there were 14 and 19 such exclusions, respectively.) The exclusion of these seven participants was not, however, made clear in the September 24 announcement; trial investigators Jerome Kim and Merlin Robb both stated in their presentations that 16,395 individuals were randomized. This issue would rear its head again when the results were eventually published in the New England Journal of Medicine (NEJM).
Not all responses to the initial announcement were euphoric. A great deal of skepticism about the vaccine candidates had been expressed prior to the trial’s start, with 22 scientists publishing a perspective piece in the journal Science arguing that it should not go ahead. TAG also had a letter published in Science, noting the design precluded any understanding of the contribution of the individual vaccines and suggesting that AIDSVAX should be dropped. For skeptical observers, the wide 95% confidence interval (CI) around the efficacy estimate was the most frequently cited cause for concern. Spanning 1.1% to 51.2%, the CI suggested both the marginal efficacy (a lower bound less than 1 indicates a lack of statistical significance) and a great deal of uncertainty about the reproducibility of the result. TAG issued a statement calling for caution in interpreting the findings until more details were available, and positing that the results might be better described as marginal rather than modest.
A Cloud of Controversy
A little over a week after the initial announcement, the views of the skeptics appeared to be bolstered when news began to leak out about additional analyses of the trial results that did not achieve statistical significance. Specifically, an analysis of only those individuals that fully complied with the protocol and received all immunizations on schedule—called the “per-protocol” analysis—reduced the number of infection endpoints from 125 to 86 and led to a lower efficacy estimate of 26.2% that was no longer statistically significant. These reports were widely cited as undermining confidence in the overall result and created the suspicion that the investigators had deliberately put their best analysis forward in the original announcement (which did not mention anything about the per-protocol finding).
The Data Revealed
Finally, after nearly a month of sturm und drang, a fuller report of the trial results was presented at the AIDS Vaccine 2009 conference in Paris and simultaneously published in a peer-reviewed paper in the NEJM on October 20. These presentations went a long way to clarifying the previous confusion. The apparent drop in efficacy in the per-protocol analysis was largely a consequence of the fact that the greatest difference in the numbers of infections between vaccine and placebo groups accrued during the first 12 months of the study, including during the period participants were receiving immunizations (the regimen involved ALVAC shots at weeks 0, 4, 12, and 24 with AIDSVAX added at weeks 12 and 24). Because the per-protocol analysis excluded any infection that occurred during the first six months, the efficacy estimate was consequently reduced. The loss of statistical significance reflected both this early difference in infection rates and the overall loss of statistical power associated with reducing the overall number of participants included in the analysis. Around one-third of the total trial population was excluded from the per-protocol analysis, due to issues such as immunizations not being received exactly on schedule.
There was one final wrinkle in the controversy over the results, however. The NEJM paper listed an “intent-to-treat” (ITT) analysis first, and this includes the seven individuals excluded from the September 24 announcement because they were infected at the time of first immunization. Because five of these individuals were in the vaccine group and two in the placebo group, their inclusion renders the efficacy result a nonsignificant trend (p = 0.08). But since exclusion of these cases is standard (a vaccine cannot protect someone already infected), it is unclear why the paper lists this analysis first. In all prior HIV vaccine efficacy trials, individuals later found to be infected at baseline have been excluded from the ITT analysis.
After the ITT, the next result reported in the paper is the nonsignificant per-protocol analysis, and the originally publicized statistically significant result is listed last; it’s called the “modified ITT” (mITT) and the p value is 0.039. To someone reading the paper who hasn’t read other vaccine efficacy results, it’s easy to get the impression that the mITT analysis is somehow less rigorous than the ITT analysis because it’s left until last, but this is not the case. In prior trials, the mITT was presented as the primary analysis with no controversy whatsoever, and it remains unclear why that didn’t occur here. Ultimately, it seems that what should have been a nonissue has added an extra layer of confusion to the story.
Yet with the publication and presentation of the results, the clouds of controversy and confusion are beginning to clear. The data do appear to support some marginal efficacy of the vaccines in preventing HIV infection, but there is a strong suggestion that this efficacy is short-lived. Subgroup analyses are also included in the NEJM paper, and while the numbers are too small to draw firm conclusions, infections among higher-risk participants were equivalent in the vaccine and placebo arms, indicating that the limited efficacy was concentrated among those at low and medium risk. Although the majority opinion among observers is that a small but real effect has been detected in the Thai trial, some statisticians have stressed that the possibility of a fluke result remains. This is the fourth HIV vaccine efficacy trial, and even though the result is statistically significant, there is around a 1 in 21 possibility of this occurring by a simple play of chance.
The Next Steps
There is unanimous agreement on one issue: the marginal efficacy observed in the Thai trial is not sufficient for licensure of the vaccines (the investigators set 50% as the minimum threshold). Discussions are being held regarding whether the vaccine should be offered to placebo recipients in the trial, but a decision has not yet been made. The priority for researchers is attempting to identify immune responses that may have been associated with protection (“correlates of immunity,” the holy grail for vaccine scientists), but the study was not powered with this in mind and samples are limited. At the AIDS Vaccine 2009 conference in Paris, investigator Nelson Michael explained that subcommittees have already been formed to look at various aspects of the immune response in the trial outcome; due to the apparent short-lived nature of the vaccine effect, there is particular interest in studying innate and antibody responses on the basis that they would be most likely to wane over time. The USMHRP will also solicit ideas from independent investigators via its website (http://www.hivresearch.org).
In the longer term, many additional issues will need to be addressed, including:
- The reproducibility of the finding (this does not mean repeating the trial, but instead attempting to reproduce the outcome in other settings, including animal models).
- The contribution (if any) of the AIDSVAX component, which failed to show even a hint of efficacy in two large prior trials. (Claims made by the manufacturer of efficacy in subgroups were entirely spurious.)
- The role of the subtype specificity of the vaccines. (Both constructs included envelope antigens from the most common circulating strain in Thailand, CRF01_ AE, formerly known as subtype E.)
- The relevance of the finding, if any, to higher-risk populations.
Currently it is uncertain if additional trials of ALVAC and AIDSVAX will occur, or whether similar vaccines that induce immune responses more consistently will take their place. Future issues of TAGline will provide updates as the story unfolds.
The Analyses:
| Modified intent-to-treat (mITT) | Strict intent-to-treat (ITT) | Pre-protocol | |
|---|---|---|---|
| Total number | 16,395 | 16,402 | 12,452 |
| Cases of HIV infection | 74 placebo, 51 vaccine | 76 placebo, 56 vaccine | 50 placebo, 36 vaccine |
| Efficacy (95% confidence interval) | 31.2% (95% CI, 1.1 to 51.2) | 26.4% (95% CI, -4.0 to 47.9) | 26.2% (95% CI, -13.3 to 51.9) |
| p value | 0.04 | 0.08 | 0.16 |
The Vaccines
ALVAC vCP1521 vector is a modified version of a bird virus called canarypox. While the natural form of the virus can be harmful to birds, it can only enter human cells and not replicate in them. ALVAC vCP1521 contains the gene encoding the HIV-1 gp120 protein from a virus codenamed 92TH023 that was isolated from a Thai individual in Bangkok. The virus was originally designated as belonging to subtype E, but it has since been recognized that this subtype is largely a circulating recombinant form now known by the name CRF01_AE. The 92TH023 virus isolate uses the CCR5 coreceptor to enter cells, like almost all primary HIV isolates. In the ALVAC vCP1521 construct, the 92TH023 gp120 protein is linked to a portion of the gp41 protein from the first HIV ever isolated, LAI (originally misnamed LAV). Part of the gp41 protein is deleted to make it easier to distinguish vaccine-induced antibodies from those induced by HIV infection. LAI belongs to subtype B and uses the CXCR4 coreceptor to enter cells. The other two HIV-1 proteins encoded by the ALVAC vector are Gag and Protease, also derived from LAI.
The AIDSVAX B/E vaccine contains two gp120 proteins. One is from the subtype B HIV-1 isolate MN, a CXCR4-using virus originally isolated from a child with AIDS-related complex (as it was then known) in 1984. The other gp120 is from a CRF01_AE virus isolate name A244 or CM244 that was obtained in northern Thailand (CM = Chang Mai) in 1990. The source of the isolate was a young Thai man who tested HIV-positive after being randomly selected for military service. Like 92TH023, CM244 is an R5-using isolate.
The Value of Volunteerism
Irrespective of the various controversies that have surrounded the trial, the commitment and dedication of the volunteers and supporting organizations deserves to be recognized and saluted. According to a presentation at the 2004 AIDS Conference in Bangkok, more than 70% of trial participants cited altruism as their primary motivation for participation.
Resources:
New England Journal of Medicine paper: http://content.nejm.org/cgi/content/full/NEJMoa0908492.
AIDS Vaccine 2009 Conference webcast and press conference: http://www.hivvaccineenterprise.org/conference/2009/webcasting.html.