University of Loughborough两位教授选取了全球101位图书情报领域的知名学者为样本，在全球范围内以问卷的方式做了同行评议，为每一位学者打分，最终得到一份同行评议调查结果。在没有利益导向的前提下，做这样的同行评议的结果基本趋于事实本身。这也是科学评价领域最尊崇的评价方式。但现实中，我们受到成本太高、利益驱使等因素的干扰，很难开展令人信服的同行评议。因此，我们看到了很多各类定量评价指标。那么定量评价指标的效力究竟怎样？

       我们课题组以University of Loughborough两位教授的同行评议结果为参照，基于Web of Science、Scopus和Google Scholar三个学术资源数据库，计算了每位学者的科学计量指标值，包括h指数以及h指数的衍生指数。然后，在定量指标与同行评议结果的比较中发现，定量指标与同行评议的相关性约0.5，中等强度相关。最后，我们认为，在同行评议不可行的情况，以定量评价指标作为替代在一定程度上是可行的。

       使用定量评价指标时，应警惕指标的导向性，避免盲目追求指标值而陷入误区，影响因子就是一个很好的反面例子。因此，2012年12月，多家学术机构与学者签署了《旧金山宣言》，支持停止将影响因子用于科学家个人学术工作的评价。但是，我个人认为：在中国当前的学术环境下，放弃定量指标而使用同行评议是不可行的。同行评议需要良好的社会信用体系与权力监督体系作支撑。相比而言，定量指标对社会环境的要求较低，更容易让人信服。有一天，大众开始理性地相信专家、人开始更改地相信人的时候，中国可以放弃定量评价指标而广泛地采用同行评议了。

       论文发表在《Journal of Informetrics》（SSCI收录），附标题与摘要：

Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments

This paper studies the correlations between peer review and citation indicators when evaluating research quality in library and information science (LIS). Forty-two LIS experts provided judgments on a 5-point scale of the quality of research published by 101 scholars; the median rankings resulting from these judgments were then correlated with h-, g- and H-index values computed using three different sources of citation data: Web of Science (WoS), Scopus and Google Scholar (GS). The two variants of the basic h-index correlated more strongly with peer judgment than did the h-index itself; citation data from Scopus was more strongly correlated with the expert judgments than was data from GS, which in turn was more strongly correlated than data from WoS; correlations from a carefully cleaned version of GS data were little different from those obtained using swiftly gathered GS data; the indices from the citation databases resulted in broadly similar rankings of the LIS academics; GS disadvantaged researchers in bibliometrics compared to the other two citation database while WoS disadvantaged researchers in the more technical aspects of information retrieval; and experts from the UK and other European countries rated UK academics with higher scores than did experts from the USA.