Negative results are disappearing

[ConcertaParent]

Negative results are disappearing from most disciplines and countries

Abstract Concerns that the growing competition for funding and citations might distort science are frequently discussed, but have not been verified directly. Of the hypothesized problems, perhaps the most worrying is a worsening of positive-outcome bias. A system that disfavours negative results not only distorts the scientific literature directly, but might also discourage high-risk projects and pressure scientists to fabricate and falsify their data. This study analysed over 4,600 papers published in all disciplines between 1990 and 2007, measuring the frequency of papers that, having declared to have ‘‘tested’’ a hypothesis, reported a positive support for it. The overall frequency of positive supports has grown by over 22% between 1990 and 2007, with significant differences between disciplines and countries. The increase was stronger in the social and some biomedical disciplines. The United States had published, over the years, significantly fewer positive results than Asian countries (and particularly Japan) but more than European countries (and in particular the United Kingdom). Methodological artefacts cannot explain away these patterns, which support the hypotheses that research is becoming less pioneering and/or that the objectivity with which results are produced and published is decreasing.

:
In conclusion, it must be emphasised that the strongest increase in positive results was observed in disciplines — like Clinical Medicine, Pharmacology & Toxicology, Molecular Biology — where concerns for publication bias had a longer history and several initiatives to prevent and correct it have been attempted, including registration of clinical trials, enforcing guidelines for accurate reporting, and creating journals of negative results. This study suggests that such initiatives have not met their objectives so far, and the problem might be worsening.

[ConcertaParent]

http://www.youtube.com/watch?v=h4MhbkWJzKk

This is a drug that I myself have prescribed to patients. And I'm a very nerdy doctor. I hope I try to go out of my way to try and read and understand all the literature. I read the trials on this. They were all positive. They were all well-conducted. I found no flaw. Unfortunately, it turned out, that many of these trials were withheld. In fact, 76 percent of all of the trials that were done on this drug were withheld from doctors and patients. Now if you think about it, if I tossed a coin a hundred times, and I'm allowed to withhold from you the answers half the times, then I can convince you that I have a coin with two heads. If we remove half of the data, we can never know what the true effect size of these medicines is.

And this is not an isolated story. Around half of all of the trial data on antidepressants has been withheld, but it goes way beyond that.

[namazu]

There are some moves to counter this problem.

One is registration of clinical trials. That is, companies, researchers, etc. who want to have a chance at publishing results are being required to register the trial in publicly-accessible databases. That way, even if the results are negative and/or go unpublished, no one can pretend that the trial simply didn't happen (unlike before).

There are also journals dedicated to publishing negative results, or claiming at least to look upon papers with negative results favorably.

These are small steps forward, but good steps. The intense competition for research funds does promote varying degrees of gamesmanship, though, which is unfortunate.

[Amtram]

This is one of the things that PLoS is trying to address. Unfortunately, on the occasions when negative results are released to the public, the public automatically pounces on them and says, "SEE? I told you this is all a Big Pharma lie!!!" without understanding that the negative results under conditions A, B, and C don't undermine the positive results under condition D. Public misconception is a major factor in withholding negative results. (Sometimes it should be a major factor in withholding positive results, too, IMHO.)