科学网-[汇集] “同行评议”阻碍原创与量化考核的局限性-杨正瓴的博文

[汇集] “同行评议”阻碍原创与量化考核的局限性

2022-6-28 14:31

阅读：2893

“一切真理开始时总是在少数人手里，总是受到多数人的压力。这是一个规律。”

不是由于有意压抑，只是由于鉴别不清，也会妨碍新生事物的成长。

为我国2070年开始的诺贝尔科学奖“井喷”清除障碍、铺平道路！

客观规律是客观的；独立于人而客观存在。

[汇集] “同行评议”阻碍原创与量化考核的局限性

米·阿·克拉契柯教授，化学博士，1925年毕业于基辅工业大学，苏联科学院普通与有机化学研究所研究员，1948年获斯大林奖。

1960-08-11在《对中国科学的某些意见在中国的工作情况》里写到：

“在科学作品方面，数量指标起不了特别的作用，无论如何也起不了决定性的作用，对每一个科学作品来说，最主要的是质量，也就是向自然界提出的问题的回答充分程度如何和所得答案的可靠程度如何。”

https://blog.sciencenet.cn/blog-1051670-730496.html , https://blog.sciencenet.cn/blog-107667-945476.html

1980年代，美国波士顿大学（Boston University）的约瑟夫·阿加西（Joseph Agassi）教授也说：“同行评议……只会导致普遍水平的下降。”

https://blog.sciencenet.cn/blog-107667-1243254.html (苏)A.H.卢克著,马约(署肖自力) 译. 科学史上的高水平人才[J]. 科学史译丛,1987(2):58-66. DOI：http://ir.ihns.ac.cn/handle/311051/6393 http://www.irgrid.ac.cn/handle/1471x/810443?mode=full&submit_simple=Show+full+item+record

1880年赫胥黎说：“历史告诫我们说，一种崭新的真理惯常的命运是：始于异端，终于迷信。”Thomas Huxley：History warns us that it is the customary fate of new truths to begin as heresies and to end as superstitions.

不幸的是，上述看法基本上被历史证实：

一、“同行评议”阻碍原创与量化考核的局限性

（1）潘教峰，王光辉，鲁晓. 基于五大价值导向的“破四唯”和“立新标”[J]. 科学通报, 2022, 67(3): 236-241.

DOI: 10.1360/TB-2021-1250

https://www.sciengine.com/CSB/article?doi=10.1360/TB-2021-1250&scroll=

近年来全球科学界开始反思定量评价体系对于科学发展的负面影响, 认为学术的独立性在现有的评价体系下被大大削弱了. 以定量指标为依据的评价方法更是一种判定性、工具性的方法, 而并没有体现对科学本身的发展性、促进性作用[6]. 有研究对241门细分学科的9000万篇论文的18亿次引用的定量分析发现, 大量论文的发表不但没有加快研究范式的更替, 反而巩固了经典研究[7]. 这也说明, 在定量评价导向下,大量科技论文其实是对现有知识体系的修修补补, 缺少真正具有开拓性、原创性的成果. 目前, 国际科学界通行的定量评价导向在体现客观性优势的同时, 也存在悖离科学本质价值的问题. 定量评价体系易停留在指标计算上, 无法回归科学本源, 无法体现价值创造, 无法甄别真正具有重大原创价值的成果.

[6] Henk J. ter Bogt, Robert W. Scapens. Performance management in universities: effects of the transition to more quantitative measurement systems [J]. European Accounting Review, 2012, 21(3): 451-497.

doi： 10.1080/09638180.2012.668323

https://www.tandfonline.com/doi/abs/10.1080/09638180.2012.668323?journalCode=rear20

[7] Johan S. G. Chu, James A. Evans. Slowed canonical progress in large fields of science [J]. PNAS, 2021, 118(41): e2021636118. October 12, 2021.

doi: 10.1073/pnas.2021636118

https://www.pnas.org/content/118/41/e2021636118

参见博文：2021-10-12，“同质化”论文暴增：“同行评议”阻碍“0到1”原创的新实证结果（PNAS）， https://blog.sciencenet.cn/blog-107667-1307679.html

（2）2022-01-14，废话的胜利：“精致而平庸”的论文是怎么发上顶级刊物的？

https://baijiahao.baidu.com/s?id=1721917651029877855&wfr=spider&for=pc

https://ishare.ifeng.com/c/s/v002Jp5DFzdH3lFRG5toTmcDqGnY9jd9wHZ--aSCNq19ynrA__

https://xw.qq.com/amphtml/20220114A0BWKK00

https://view.inews.qq.com/a/20220114A07CS700

Publish or Perish（要么发表，要么死亡）的学术生存法则，让我们在同行审稿人面前变得唯唯诺诺、战战兢兢，像一个受了委屈的小媳妇，敢怒不敢言，期待有朝一日媳妇熬成婆。

其结果就是此标准催生的无数“精致而平庸”的论文——先进的分析技术、晦涩的专业语言，似乎是为了掩盖研究本身的毫无意义。

[2-2] Dennis Tourish. The triumph of nonsense in management studies [J]. Academy of Management Learning & Education, 2020, 19(1): 99-109.

doi: 10.5465/amle.2019.0255

https://journals.aom.org/doi/10.5465/amle.2019.0255

（3）Michael Park, Erin Leahey, Russell Funk. Dynamics of Disruption in Science and echnology.

doi: 10.48550/arXiv.2106.11184

https://arxiv.org/abs/2106.11184

Although the number of new scientific discoveries and technological inventions has increased dramatically over the past century, there are growing concerns that progress is slowing. We analyze 25 million papers and 4 million patents across 6 decades and find that science and technology are becoming less disruptive of existing knowledge, a pattern that holds nearly universally across fields. We link this decline in disruptiveness to a narrowing in the utilization of existing knowledge. Diminishing quality of published science and changes in citation practices are unlikely to be responsible for this trend, suggesting that this pattern represents a fundamental shift in science and technology.

尽管在过去一个世纪中，新的科学发现和技术发明的数量急剧增加，但人们越来越担心进展正在放缓。我们分析了 6 个十年的 2500 万篇论文和 400 万项专利，发现科学技术对现有知识的破坏性越来越小，这种模式几乎在各个领域都普遍存在。我们将这种破坏性的下降与现有知识的利用范围缩小联系起来。已发表科学的质量下降和引用实践的变化不太可能是造成这种趋势的原因，这表明这种模式代表了科学和技术的根本转变。

（3-2）集智俱乐部，2021-07-06，分析了2500万篇论文后，发现科学正在变得越来越保守

http://app.myzaker.com/news/article.php?pk=60e44bff8e9f0951fc67fb6e

不止所有发表论文的平均突破性程度在下降，高水平的论文也是同样的趋势。选取 Nature、Science 和 PNAS 三则顶刊论文分析，发现其下降趋势比普通期刊更加明显，早在 1975/1980 年之后，PNAS 和 Nature 的平均突破性研究占比就早早躺平，似乎顶刊更加不愿意刊登争议性更大、但也有可能是突破性的研究。而图四右边诺奖三种自然科学型的获奖论文，其突破性得分平均甚至下降到了 0 以下。

参见博文：2021-07-24，Zenas 公理：2021年《Dynamics of Disruption in Science and Technology》里的图示， https://blog.sciencenet.cn/blog-107667-1296808.html

（4）刘进平,2021-05-24, 研究越假, 引用越多

http://blog.sciencenet.cn/blog-39731-1288105.html

[4-2] Marta Serra-Garcia, Uri Gneezy. Nonreplicable publications are cited more than replicable ones [J]. Science Advances, 2021, 7(21): eabd1705.

DOI: 10.1126/sciadv.abd1705

https://www.science.org/doi/10.1126/sciadv.abd1705#:~:text=Nonreplicable%20publications%20are%20cited%20more%20even%20after%20the,%20%20%E2%88%929.025%2A%2A%2A%20%209%20more%20rows%20

（5）科学网，2020-05-02，伪造同行评议，30余篇论文被撤！传统审稿模式再受质疑

http://news.sciencenet.cn/htmlnews/2020/5/439275.shtm

这项研究涉及发表在bioRxiv上近2500篇文章，以及arXiv上的1.2万篇文章。最终得出的结论也是：预印本与正式发表版本论文的差异很小。

[5-2] SCIENCE, JEFFREY BRAINARD, 2020-03-26, Do preprints improve with peer review? A little, one study suggests

https://www.science.org/news/2020/03/do-preprints-improve-peer-review-little-one-study-suggests

[5-3] Martin Klein, Peter Broadwell, Sharon E. Farb, Todd Grappone. Comparing published scientific journal articles to their pre-print versions[J]. International Journal on Digital Libraries, 2019, 20, pages335–350.

doi: 10.1007/s00799-018-0234-1

https://link.springer.com/article/10.1007/s00799-018-0234-1

（6）搜狐，2019-03-05，大组守旧，小组创新，基金项目难以颠覆！

https://www.sohu.com/a/299260876_120052083

[6-2] Wu Lingfei, Wang Dashun, James A. Evans. Large teams develop and small teams disrupt science and technology [J]. Nature, 2019, 566(7744): 378-382.

doi: 10.1038/s41586-019-0941-9

https://www.nature.com/articles/s41586-019-0941-9

To analyse the disruption of government-funded papers, we select 477,702 WOS papers that acknowledged funding from five major government agencies, published between 2004 and 2014. The acknowledged agencies include the National Science Foundation (NSF; 191,717 papers), National European Research Council and European Commission (ERC and EC; 81,296 papers), Natural Science Foundation of China (NSFC; 80,448 papers), German Research Foundation (DFG; 75,881 papers), and Japan Society for the Promotion of Science (JSPS; 58,275 papers). These papers are published across 7,325 journals. A paper may be funded by multiple agencies. The average disruption of these papers is −0.0024, ranking in the tail 31% of all WOS papers from the same period.

为了分析政府基金资助论文的颠覆指数，我们选择了 477,702 篇 WOS 论文，这些论文致谢了 2004 年至 2014 年间发表的五个主要政府机构的资助。致谢的机构包括国家科学基金会（NSF；191,717 篇论文）、国家欧洲研究委员会和欧盟委员会（ERC 和 EC；81,296 篇论文）、中国自然科学基金（NSFC；80,448 篇论文）、德国研究基金会（DFG；75,881 篇论文）和日本科学振兴会（JSPS；58,275 篇论文）。这些论文发表在 7,325 种期刊上。一篇论文可能由多个机构资助。这些论文的平均颠覆指数为 -0.0024，在同期所有 WOS 论文中排在尾部 31%。

（7）2014年 SCIENCE 杂志：“同行评议根本不能预测研究的成果。这令人非常不安。”

Jeffrey Mervis. Peering into peer review [J]. Science, 2014, 343(6171): 596-598. 7 Feb 2014.

DOI: 10.1126/science.343.6171.596

https://www.science.org/doi/full/10.1126/science.343.6171.596

"Peer review should be able to tell us what research projects will have the biggest impacts," Lauer contends. "In fact, we explicitly tell scientists it's one of the main criteria for review. But what we found is quite remarkable. Peer review is not predicting outcomes at all. And that's quite disconcerting."

“同行评审应该能够告诉我们哪些研究项目将产生最大的影响，”劳尔认为。“事实上，我们明确告诉科学家这是审查的主要标准之一。但我们的发现非常怪异。同行评审根本无法预测结果。这很令人不安。”

The problems of peer review, Lauer says, are those that afflict any system that relies on the judgments of experts. One eye-opener for Lauer was a 2006 book by Philip Tetlock, a psychologist at the University of Pennsylvania, titled Expert Political Judgment: How Good Is It? How Can We Know? The book describes how experts do little better than chance in predicting political events and also vastly overrate their prognosticating abilities. Its lessons apply to peer review as well, Tetlock says "There is high-impact research that has been rejected, and low-impact research that has been funded."

劳尔说，同行评审的问题是困扰任何依赖专家判断的系统的问题。让劳尔大开眼界的是宾夕法尼亚大学心理学家菲利普·泰特洛克（Philip Tetlock）在 2006 年出版的一本名为《专家预测判断：它有多好？我们怎么知道？这本书描述了专家们如何在预测政治事件方面做得比机会好，而且还大大高估了他们的预测能力。它的经验教训也适用于同行评审，Tetlock 说：“有高影响力的研究被拒绝，而低影响力的研究却得到了资助。”

相关背景知识：

Tetlock asks 284 world-renowned experts to make 27,451 verifiable predictions of the future. Many years later he then measured their accuracy and determined why some experts were more accurate than others.

Tetlock 邀请 284 位世界知名专家对未来做出 27,451 个可验证的预测。许多年后，他测量了它们的准确性，并确定了为什么一些专家比其他专家更准确。

Expert predictions of the future are no more accurate than random guesses!

The more confident an expert is in their own prediction, the less accurate the prediction turns out to be.

The overall amount of knowledge or prestige of the expert does not lead to more accurate predictions. It just leads to overconfidence.

Even after the predictions failed, most experts were unlikely to admit their error. This is particularly true with those who were most confident in their original prediction.

The more expertise the Hedgehog has, the less accurate the prediction, strongly suggesting that new information merely feed their bias.

专家对未来的预测并不比随机猜测更准确！

专家对自己的预测越有信心，预测结果就越不准确。

专家的知识或声望的总量不会导致更准确的预测。它只会导致过度自信。

即使在预测失败后，大多数专家也不太可能承认他们的错误。对于那些对他们最初的预测最有信心的人来说尤其如此。

刺猬的专业知识越多，预测就越不准确，这强烈表明新信息只会助长他们的偏见。

（8）科学网，2015-01-12，科学家分析同行评审有效性

http://news.sciencenet.cn/htmlpaper/201511219413977135306.shtm

同行评审在预测“良好的”论文方面是有效的，但可能难以识别出卓越和（或）突破性的研究。

研究人员还发现这3份医学期刊曾拒绝了许多之后获得高引用率的手稿，包括14篇引用数量最多的手稿，而这14篇手稿中的12篇是被编辑退稿的。

[1] Kyle Siler, Kirby Lee, Lisa Bero. Measuring the effectiveness of scientific gatekeeping [J]. Proceedings of The National Academy of Sciences of The United States of America, 2015, 112(2): 360-365. JAN 13 2015.

Doi: 10.1073/pnas.1418218112

https://www.pnas.org/doi/10.1073/pnas.1418218112

（9）Melinda Baldwin. Scientific autonomy, public accountability, and the rise of “peer review” in the Cold War United States [J]. Isis: Journal of the History of Science in Society, 2018, 109(3): 538-558.

doi: 10.1086/700070

https://www.journals.uchicago.edu/doi/abs/10.1086/700070

第 556 页：1975，

It was Proxmire, however, who expressed the most negative ideas about peer review. In his submitted testimony, he wrote, “I have received a number of letters pointing out that the peer review system serves to perpetuate the funding of established people, ideas and institutions.” Proxmire seemed to hold out little hope that peer review could be anything other than “incestuous.”

然而，Proxmire 对同行评审表达了最负面的看法。在提交的证词中，他写道：“我收到了许多信函，指出同行评议制度有助于为已建立地位的的人员、想法和机构提供资金。” Proxmire 似乎认为同行评审比“乱伦”还坏。

第 558 页：

In 2011, Great Britain’s House of Commons commissioned a report on the state of peer review and concluded that while peer review “is crucial to the reputation and reliability of scientific research,” many scientists believe the system stifles progress, is often biased, and that “there is little solid evidence on its efficacy.”

2011 年，英国下议院委托编写了一份关于同行评议状况的报告，并得出结论认为，虽然同行评议“对科学研究的声誉和可靠性至关重要”，但许多科学家认为该系统会扼杀进步，往往存在偏见，而且“关于它的功效几乎没有确凿的证据。”

第 558 页：

The more we have expected of peer review, the more its opportunities to disappoint have expanded. Peer review’s perceived failures may have their roots in the gap between modern expectations of refereeing and the more modest functions it was initially designed to fulfill.

我们对同行评审的期望越高，失望的机会就越大。同行评审的失败可能源于现代对裁判的期望与其最初旨在实现的更温和的职能之间的差距。

（10）科学网，2016-08-16，徐匡迪院士：对颠覆性创新意愿应宽容支持

https://news.sciencenet.cn/htmlnews/2016/8/353821.shtm

樊丽萍. “对颠覆性创新意愿应宽容支持”[N]. 文汇报, 2016-08-16 002版要闻

而在近期，以颠覆性技术取得创新成功的最经典案例，非埃隆·马斯克(Elon Mask)莫属。

相关背景知识：

[1] 新浪科技，2022-06-27，前副局长回忆：NASA曾把马斯克和 SpaceX 当笑话

https://finance.sina.com.cn/tech/2022-06-27/doc-imizmscu8904383.shtml

曾经担任美国宇航局副局长的高芬（Lori Garver）最近出版了回忆录，

高芬介绍说，在SpaceX发展早期，一些太空行业高管和美国宇航局的官员经常靠取笑SpaceX来作乐。

马斯克做了一些公开表态，称SpaceX能够改进美国宇航局的航天计划，***对马斯克的话非常不满，对着高芬大吼，“叫你们家小孩马斯克规矩点。”

（11）丁肇中申请书理论：

我有一个“理论”，绝对正确，可以告诉大家。

在加速器实验的发展史上，过去50年里面，尽管我们为了获得经费，要写一个申请报告书，设定一个目标，说服政府的人投钱做加速器实验，可是往往实际的发现跟原来的目标根本没有关系。

不要盲从专的结论。

要实现你的目标的话，最重要的是要有好奇心，对自己所做的事情有兴趣，不能因为别人反对你就停止。而且，你对意外的现象要有充分的准备。

人民日报，2000-05-02，第5版丁肇中：科学发现的几点体会

http://data.people.com.cn/rmrb/20000502/5

http://edu.people.com.cn/n/2013/1230/c1053-23978650.html

（12）2006年 Donald W. Braben：杀死同行评议，拯救人类文明！

https://blog.sciencenet.cn/blog-107667-1214979.html

唐纳德·布拉本（Donald W. Braben）教授，英国伦敦大学学院地球科学系的荣誉教授。目前在“泰晤士高等教育 Times Higher Education (THE)”工作。

（13）2017-05-25，William G. Kaelin Jr: Publish houses of brick, not mansions of straw 论文发表：要砖屋，不要稻草豪宅

William G. Kaelin Jr., 2019年诺贝尔医学奖（The Nobel Prize in Physiology or Medicine 2019）。

https://www.nature.com/articles/545387a

要“坚固的砖房 sturdy houses of brick”，不要“稻草堆砌的豪宅 grand mansions of straw”。

（14）更多的大数据实证研究

[1] Fortunato S, Bergstrom C T, Boerner K, et al. Science of science[J]. Science, 2018, 359(6379): eaao0185.

doi: 10.1126/science.aao0185

https://www.science.org/doi/10.1126/science.aao0185

https://science.sciencemag.org/content/sci/359/6379/eaao0185.full.pdf

[2] Stephan P, Veugelers R, Wang J. Blinkered by bibliometrics[J]. Nature, 2017, 544(7651): 411-412.

Reviewers are blinkered by bibliometrics

doi: 10.1038/544411a

https://www.nature.com/news/reviewers-are-blinkered-by-bibliometrics-1.21877

[3] Wang J, Veugelers R, Stephan P. Bias against novelty in science: A cautionary tale for users of bibliometric indicators[J]. Research Policy, 2017, 46(8): 1416-1436.

doi: 10.1016/j.respol.2017.06.006

https://linkinghub.elsevier.com/retrieve/pii/S0048733317301038

[4] Boudreau K J, Guinan E C, Lakhani K R, et al. Looking across and looking beyond the knowledge frontier: intellectual distance, novelty, and resource allocation in science[J]. Management Science, 2016, 62(10):2765-2783.

doi: 10.1287/mnsc.2015.2285

https://pubsonline.informs.org/doi/pdf/10.1287/mnsc.2015.2285

[5] Balietti S, Goldstone R L, Helbing D. Peer review and competition in the Art Exhibition Game[J]. Proceedings of the National Academy of Sciences of the United States of America, 2016, 113(30): 8414-8419.

doi: 10.1073/pnas.1603723113

https://www.pnas.org/content/113/30/8414

[6] Csiszar A. Peer review: Troubled from the start[J]. Nature, 2016, 532(7599): 306-308.

doi: 10.1038/532306a

https://www.nature.com/news/peer-review-troubled-from-the-start-1.19763

[7] Bromham L, Dinnage R, Hua X. Interdisciplinary research has consistently lower funding success[J]. Nature, 2016, 534(7609): 684-687.

doi: 10.1038/nature18315

https://www.nature.com/articles/nature18315

[8] Kyle Siler, Kirby Lee, Lisa Bero. Measuring the effectiveness of scientific gatekeeping [J]. Proceedings of The National Academy of Sciences of The United States of America, 2015, 112(2): 360-365. JAN 13 2015.

Doi: 10.1073/pnas.1418218112

https://www.pnas.org/doi/10.1073/pnas.1418218112

[9] Nicholson J M, Ioannidis J P A. Conform and be funded[J]. Nature, 2012, 492(7427): 34-36.

doi: 10.1038/492034a

https://www.nature.com/articles/492034a

（15）三件历史事实：牛顿、麦克斯韦，和“被淹没”

[1] 1673年31岁的牛顿（Isaac Newton），因为无法忍受出英国皇家学会（The Royal Society）会员们的“评议”，声称要退出皇家学会。

https://blog.sciencenet.cn/blog-107667-1328150.html

[2] 1860年，29岁的麦克斯韦（James Clerk Maxwell）被阿伯丁大学（University of Aberdeen）下岗，求职爱丁堡大学（University of Edinburgh）失败。

https://www.britannica.com/biography/James-Clerk-Maxwell

[3] A paper surfaces that was rejected by a Royal Society referee in 1845, outlining the kinetic theory of gases more than a decade before James Clerk Maxwell’s famous paper.

1845年，英国皇家学会同行评议拒稿了一篇气体动力学理论的论文。该文比麦克斯韦（James Clerk Maxwell）的著名论文早十多年。谁知道这位“被”拒稿作者的名字？

doi: 10.1038/532306a

https://www.nature.com/articles/532306a

二、最恐惧的是，把谬误当成真理。进而真诚地捍卫谬误，真诚地仇恨真理

我们所努力坚持的，是谬误？还是真理？

最恐惧的是，把谬误当成真理。进而真诚地捍卫谬误，真诚地仇恨真理。

马克·吐温：幽默的内在根源不是快乐，而是悲哀。

Mark Twain: The secret source of humor is not joy but sorrow.

福楼拜说：“大地有其边界，人类的愚蠢却没有尽头。”

Gustave Flaubert wrote that “Earth has its boundaries, but human stupidity is limitless.”

苏格拉底：“我唯一知道的，就是我的无知。”Socrates, "I know that I know nothing. = I know one thing, that I know nothing."

柏拉图：“我 - 同样无知 - 不相信[我知道任何事情]” “不知道自己无知，乃是双重的无知！”Plato,"On the other hand, I – equally ignorant – do not believe [that I know anything]."

三、让公平正义之光照耀每一个人！！

"One foul sentence doth more hurt, than many foul examples. For these do but corrupt the stream, the other corrupted the fountain." Francis Bacon, Of Judicature

“一次不公正的审判，其恶果甚至超过十次犯罪。因为犯罪虽是无视法律 —— 好比污染了水流，而不公正的审判则毁坏法律 —— 好比污染了水源。”——培根

谁促进了科技进步，谁阻碍了科技进步，都要有历史记录。评审人要承担相应的社会责任。

谁阻碍了科技进步，谁促进了科技进步，不是个人的隐私！！是必须公开的社会责任！！！

这样，阻碍人类文明进步的评审人，也会被人类的历史记住。

任其“逍遥法外”：真是对真理莫大的讽刺！

让公平正义之光照耀每一个人！！