And so begins the backlash from that supposedly remarkable AIDS vaccine research study. There’s already been some criticism, but now one huge AIDS group is helping raise the question, Did the U.S. Army, National Institute of Health, and the Thai government collude to spin the results of the study as overly positive?
From a release: “AIDS Healthcare Foundation (AHF), the largest non-profit HIV/AIDS healthcare
provider in the US […] expressed frustration and disappointment at the news that data that seriously undermines a widely reported Thai HIV vaccine trial were not released publicly when other, more favorable data from the vaccine trial were publicly reported late last month. The release of only partial-and favorable-data, which showed initial efficacy in approximately one third of the Thai vaccine trial participants, led to worldwide favorable publicity for the thought-to-be-promising vaccine effort, which was conducted by researchers from the US Army and Thailand. According to the Wall Street Journal (Oct. 10-11, 2009), a second heretofore suppressed or previously unreleased analysis of data from the same Thai vaccine trial suggested that the ‘modest protection highlighted by researchers might be a statistical fluke.’ The Journal reported that the additional data were available to the US and Thai researchers on September 24th when they announced the trial results to worldwide acclaim, but they chose not to release them. WSJ
also noted, ‘The incomplete disclosure raises the question of whether the Army, the Thai government and the U.S. National Institutes of Health-which helped fund the study-rushed to give a positive spin to what may turn out to be another inconclusive AIDS-vaccine effort.'”
Fitz
Do you have a link to a scholarly article on this?
AlanInUtah
Thats shitty!
B
Fitz asked, “Do you have a link to a scholarly article on this?”
http://blogs.wsj.com/health/2009/10/12/the-hiv-vaccine-and-science-by-press-release/
has the original article that AHF quoted, somewhat inaccurately.
What the Wall Street Journal article actually said was, “Researchers said last month that the vaccine lowered the risk of infection by about 31% — a ‘modest benefit,’ they said, but one that was statistically significant, suggesting the finding was not a fluke. Another slice of the data that was not released at the time — one that looked only at patients who received all of their shots as scheduled and had the full sequence of shots before becoming infected — suggested the vaccine was 26% effective, the WSJ reported this weekend. But that benefit was not statistically significant: There was a 16% chance that benefit may have been a fluke, and the cutoff for statistical significance is 5%.”
When you look at a subset of the data, which is what “another slice” refers to, it is hardly surprising that the statistical errors are higher. Also, the term “statistical significance” is widely misunderstood. There is no universal cutoff for it, as the Wall Street Journal implied. The level for statistical significance that one is willing to accept is really dependent on a judgment call that considers the cost of being wrong versus the benefit of being right given the odds for each outcome. What the second results showed is merely that there was not enough data to make a definitive statement regarding one particular subset of the data, given the level of significance that certain individuals wanted to see before claiming something as a fact.
Furthermore, unless you believe that failing to get the necessary number of shots on schedule increases the vaccine’s effectiveness, one would suspect that measured reduction in effectiveness to 26% from 31% is itself the result of statistical error – it can go either way.
The Wall Street Journal went on to say, “The study and its release points to some thorny issues regarding medical research and the press. If scientists release data suggesting a major medical advance, the press is going to report on the findings, even if the data haven’t been vetted in the peer-review process. And, if a deeper analysis points to a more nuanced picture — as appears to be the case in this instance — whatever follow-up stories that do appear are unlikely to get as much attention as the initial stories that were based only on the data scientists chose to make available.”
One might add the observation that organizations such as AHF have their own agendas and sometimes try to get attention by making a mountain out of a molehill.
ricky
i knew this stunk to high heaven when i first heard of it.
J.T.
B says: “One might add the observation that organizations such as AHF have their own agendas and sometimes try to get attention by making a mountain out of a molehill.”
Amen to that… AHF doesn’t do much that isn’t self-serving to the extreme. The amount of money they spend touting their own supposed accomplishments and protesting the US Government would be far better spent in actually getting HIV drugs to people who need them… right here in the USA.