http://www.sciencenet.cn/htmlnews/2009/9/223423.shtm 研究证实带正面结果的论文更受评审人青睐 说明同行评审也无法避免主观和偏见 有科学家曾一直怀疑,带有正面结果的论文(positive paper)更受评审人的青睐。如今,美国科学家的一项试验证实了这种论文评审中的偏见。 美国华盛顿大学的Seth Leopold捏造了2篇关于比较2种抗生素疗法的论文。它们一模一样,除了结果之外论文一称一种疗法比另一种好(正面结果);而论文二称两种疗法无差异。 Leopold和同事发现,这2篇论文提交给评审者后,评审人推荐发表论文一的可能性远远大于论文二。比如《骨与关节外科杂志美国卷》( JBJS )有100多位评审人被邀请评审其中一篇论文,结果98%的评审人推荐发表论文一,只有71%的人推荐发表论文二。更醒目的是,2篇论文方法明明一致,但评审人认为论文一的方法更好;而且评审人更容易发现论文二中隐藏的错误。 Leopold表示:这是基于证据医学的大问题,我们应该对得出正面结果的研究更挑剔一些。 国际出版伦理委员会主席Liz Wager说:这证实了同行评审是由带偏见、主观的人所完成的。所有人都希望新东西起作用,他们就是想相信这些。 Published online | Nature | doi:10.1038/news.2009.914 | Nature | doi:10.1038/news.2009.914 News Sneak test shows positive-paper bias Reviewers keener to give thumbs up to papers with positive results Nicola Jones VANCOUVER Reviewers were more critical of no-difference papers than positive papers. GETTY The bias towards positive results in journal publications has been confirmed through a cunning experiment. Seth Leopold of the University of Washington, Seattle, composed two versions of a fake paper comparing the relative benefits of two antibiotic treatments. They were identical except for one critical difference: one paper found that one treatment was better than the other, while the other found no difference between the two. Reviewers were far more likely to recommend the positive result for publication, Leopold and his colleagues found. Worse, reviewers graded the identical 'methods' section as better in the positive paper, and were more likely to find sneakily hidden errors in the 'no-difference' paper, presumably because they were feeling more negative and critical about the latter work. That's a major problem for evidence-based medicine, says Leopold, who presented the work on 11 September at the Sixth International Congress of Peer Review and Biomedical Publication in Vancouver, British Columbia. Such a bias can skew the medical literature towards good reviews of drugs, affecting consensus statements on recommended treatments. We should be more critical of positive studies, he says. Wanting to believe Previous studies have hinted at a 'positive outcome bias', just from the sheer number of papers that get published with positive versus 'no-difference' results. But it wasn't clear if there were some other aspects about 'no-difference' papers, such as methodological problems or a lack of impact, that might make editors turn up their noses. Leopold's study is the first experiment to attempt to pin this down. It just goes to show that peer review is done by biased, subjective people, says Liz Wager, managing director of the Sideview consultancy in Princes Risborough, UK, and chair of the UK-based Committee on Publication Ethics. Everyone wants the new stuff to work they want to believe. It just goes to show that peer review is done by biased, subjective people. Liz Wager Committee on Publication Ethics The two imaginary studies were of very high quality, conforming to all good standards of research, involving multiple study centres and oodles of good data. It's easy to make such a study if you don't have to actually do it, Leopold jokes. They compared two strategies of antibiotic treatment for surgery patients a single dose of drugs before surgery compared with a starter dose plus a 24-hour follow up of drugs. The relative benefit of these strategies is under debate by clinicians, so both a positive and a negative result should have equal impact on patient care both should have been equally interesting. But when more than 100 reviewers at the American edition of Journal of Bone and Joint Surgery (JBJS) were given one of the papers to assess, 98% of reviewers recommended the positive-result paper for publication, while only 71% recommended the nearly identical 'no-difference' paper. Strikingly, these reviewers also gave the entirely identical methods section a full point advantage (on a scale of one to ten) in the positive paper. There's no good explanation for that, says Leopold. That's dirty pool. Error catchers Five intentional small errors were sneaked into the papers, such as having slightly different numbers in a table compared with the text. Reviewers at the JBJS caught only an average of 0.3 errors per reviewer in the positive paper, but perked up their critical faculties to catch 0.7 errors per reviewer in the 'no difference' paper. ADVERTISEMENT