This book is a personal accountof a seemingly somewhat disillusioned bibliometrician on the uses andespecially on the misuses of bibliometrics in research evaluation. This is atranslation of an updated version of “Les Dérives de l'Évaluation de laRecherche: du bon usage de la bibliométrie” that was published in 2014. Anextensive review of the original book was published by Michel Zitt (2015) inASI. Here I review the English version published in 2016. Reviews are personaland inevitably subjective, thus Michel and I have slightly differentperspectives and emphases. The reader is invited to read both reviews. The book is not meant to becomprehensive, but is an opinionated essay. It is comprised of four chapters,starting with a historical overview, and then presenting the good, the bad, andthe ugly sides of bibliometrics and ends with suggested criteria for establishingthe validity of indicators to be used. The book is mainly intended forresearchers and decision makers who use bibliometrics and not so much forbibliometricians. Chapter one starts with thedefinition of the terms scientometrics and bibliometrics. Here, bibliometricsis viewed as a subset of scientometrics, unlike other prevailing definitions,where informetrics is the overarching term (Björneborn Ingwersen, 2004;Tague-Sutcliffe, 1992), with bibliometrics and scientometrics being intersectingbut nonoverlapping subsets of informetrics. Actually, it seems that the authoragrees with this other definition, when he describes the use of bibliometricsin the history of science, which is definitely not within the scope ofscientometrics, as scientometrics is about the measurement of the inputs andthe outputs of the scientific endeavor. Chapter one continues with ahistorical overview of the field, starting with Lotka in 1926, or even earlierwith psychologists observing the growth of the number of publications withintheir field. The overview continues concentrating on citation analysis, whichwas at first used as a means for journal selection by libraries, followed bystudies on the age of references in journal publications, and the citation half-life,and observations of the differences between disciplines. The potential ofcitation analysis was greatly enhanced by the establishment of EugeneGarfield's Institute of Scientific Information, which first published theScience Citation Index (SCI) in 1963 and was later extended to include theSocial Science Citation Index (SSCI) in 1973 and the Arts HumanitiesCitation Index in 1978. This section of the chapter also mentions Elsevier'sScopus, launched in 2004. It is pointed out that the electronic versions of thecitation databases greatly improve the ability to conduct large-scalebibliometric analyses. The next section discusses theemergence of the use of bibliometrics for science policy, mentioning theFrascati Manual (1962) and the National Science Foundation's (NSF) biannualScience and Engineering Indicators (first published in 1987). The increasinginterest in bibliometrics and research evaluation led to the establishment ofspecialty journals, like Research Policy and Scientometrics. For some reason, Icould not find any mention of the more recently established journal in thefield, the Journal of Informetrics. Chapter 2 provides an overview ofsome applications of bibliometrics categorized by Yves Gingras (YG) as “good.”It starts with an overview of the structure of the Web of Science, currentlyowned by Clarivate Analytics (but at the time of writing the book it was stillowned by Thomson Reuters) and the value of the citation databases forhistorians of science. There are some nice figures that demonstrate the growthin the average number of references and citations over time by majordisciplines. These highlight both the general growth trends over time and thedisciplinary differences, showing that in spite of the growth in the average numberof references in all disciplines, the average number of citations remained moreor less the same in the humanities, while it grew considerably in all otherdisciplines, especially in the biomedical sciences. An example of another“good” use of bibliometrics is studying the changes over time in the averagenumber of coauthors per paper. Here again, one can observe disciplinarydifferences, with only a very slow growth in the humanities, now about 12% arecoauthored in humanities compared to more than 90% of the papers in the naturaland biomedical sciences. YG also shows that there are disciplinary differenceswithin the sciences as well. Next, scientific networks are discussed both incollaboration and cocitation networks. Cocitation analysis is attributed toHenry Small without mentioning Irina Marshakova, who came up with the same ideaindependently at the same time. It should be noted that, on the one hand, theauthor declared that the book is mainly intended for researchers (not inbibliometrics) and decision makers, but on the other hand technical terms like“community detection” are used without any explanation. In the following section, severalmyths related to citation analysis are disproved, for example, that mostarticles are never cited, and it is shown that if a 5-year citation window isused then the differences between the uncitedness rates in the sciences versusthe social sciences diminish, and the uncitedness rate falls below 45% for alldisciplines except the humanities (about 90%) (Larivière, Archambault, Gingras, Wallace, 2008). There is a short discussion on negative citations and“sleeping beauties” (paper not cited or lowly cited for a long time, but thenbecoming central in new areas of research). An interesting myth that was falsifiedusing citation analysis is the claim that the Watson–Crick article on thestructure of the DNA was not cited in the first 10 years after publication. Acareful bibliometric analysis showed the contrary—in the years 1953–1970, itwas the most-cited Nature paper published in 1953 (Gingras, 2010). Thus, thischapter concentrated on useful applications of the citation indexes and ofcitation analysis. The third chapter in the bookdeals with the “bad,” that is, the huge increase in research evaluations. Mostscientific publications have undergone some kind of assessment since the 17thcentury, by members of the board/society or by external peer review. Aninteresting counterexample is that of Albert Einstein, who was accustomed to amore superficial review by the editorial members of European journals. When hereceived an anonymous negative report for one of his papers submitted to anAmerican journal, he was insulted, saying that he never allowed the editors toshow the paper to anyone before publication (Kennefick, 2005). In the 20thcentury, evaluation and granting committees became common, where suchcommittees rely on their own expertise supplemented with reports from externalreviewers. It is customary that members of the committee score the submission,and the average of the scores is used to rank submissions. YG criticizes thismethod, as there is no way to differentiate between a submission that receiveda grade of 3 from all members, and a submission that received a score of 5 fromhalf the members, and 1 from the other half, will receive the same averagescore; thus, the average score obviously cannot be interpreted as consensus. Asthis kind of peer review system became more criticized, funding agencies in the1970s started to use bibliometrics to assess the outcomes of funded research.Individuals, departments, and research units became objects of evaluation usingbibliometric measures in the 1980s. The trend of evaluations continues to growin the 21st century. One of the major criticisms in the chapter is about theevaluation of individual researchers and the use of the h-index. Unfortunately,the definition of the h-index in the book is not accurate, as it states thatthe h-index “is defined as being equal to the number N of articles published bya researcher that obtained at least N citations each.” What is missing from thedefinition is that N is maximal, that is, the researcher has not publishedN + 1 articles having N + 1 citations each. One of the objections to theh-index is that it cannot decrease, but the same is true for the number ofcitations a paper receives, thus I do not see why this is a claim against theh-index. There are several other well-known shortcomings of the h-index, whichare also mentioned in the book. The next object of criticism,unsurprisingly, is the impact factor. Here a list of well-known shortcomings ispresented, like the fact that there are fields where citations do not peakafter 2 years, and why the average is not a good measure for heavily skeweddistributions. Journal self-citations are also discussed in a more balancedmanner, stating that sometimes journal self-citations are inevitable when thereis a single specialty journal on the topic. In spite of these shortcomings,many decision makers rely on the impact factor of a researcher's publications,and researchers are rewarded based on the impact factors of the journals inwhich they publish and not by assessing the quality of their publications. University rankings are next inline. These deserve the criticism: ranking by a composite measure that includesthe number of international students or the number of Nobel Prize winners of auniversity and number of recent publications of the university (not normalizedeither by field or by the size of the faculty) does not make any sense, butstill these rankings are taken seriously by the institutions. One of the final points in thischapter is that most publications indexed in the citation databases are inEnglish, which greatly penalizes publications in local languages. To sum up,this chapter discussed the gray side of the use of bibliometrics, while thenext chapter provides some guidelines, while exposing the dark side of the useof bibliometrics. In the fourth chapter, YGdiscusses the nonapplicability of indicators, for example, internationalcollaboration in the humanities, where the norm is single authorship. He claimsthat adding books to the citation databases will not change much, as it is hardto imagine that someone highly cited in books is not cited in journals. Idisagree with this, as was shown by Cronin, Snyder, and Atkins (1997), that insociology there are two distinct populations of highly cited authors, one basedon journal citations and the other on book citations. YG dismisses altmetrics totally,and writes: “They thus implicitly accept the curious idea that we must knowimmediately what has had an impact or not, and the most likely way to know thisis to look at immediate Internet visibility on the new and rapidly multiplyingplatforms” (p. 68). I, of course, cannot agree with such a statement. I am wellaware of the limitations and challenges of altmetrics, but altmetric indicatorsprovide early signals, and thus supplement existing indicators like citationcounts. YG emphasizes the need tovalidate the indicators. Here we are in complete agreement. He proposes threecriteria as necessary conditions for validity of a good indicator: adequacy,that is, corresponding to the object or concept to be evaluated; sensitivity tothe changes of the object of evaluation over time; and homogeneity in itscomposition. I am not convinced that homogeneity is a necessary condition,especially if the values of the individual components are available as well.Further in the chapter YG adds a fourth criterion that the indicator should bea monotonically increasing function of the concept it measures. In my mind,this is not a necessary criterion; it depends on whether the indicator iscumulative or not. Citation counts are cumulative, but if the indicator is yearlycitations, downloads, mentions, etc., then these counts will not necessarily bemonotonically increasing. Next the author shows why theShanghai Ranking of universities (ARWU) and the h-index are not valid—neitherof them is homogenous—the h-index mixes citation and publication counts, andthe Shanghai Ranking mixes Nobel Prize winners with the number of articles inthe Web of Science. He also points to U-Multirank and to the Leiden ranking asbetter because they do not combine heterogeneous indicators—they display themside-by-side. However, in the Leiden ranking the user must select a singleindicator to rank by. The last part of the chapterdeals with manipulations, like dummy affiliations, editors requesting authorsto cite papers from the journal, and increasing the number of publications byaiming at middle-rank journals instead of top journals. In the conclusion, YGquestions why university leaders go along with the rankings when they areobviously flawed. This is a good question, but it seems to me that universityrankings are here to stay. To sum up, I am grateful that Iwas asked to review this book even if I disagree with some of YG'sobservations. Even though the book has an overall negative tone towards the useof bibliometrics in research evaluation, it is easy to read and the examplespresented are relevant and interesting. Overall, there is a lot of “food forthought” in this book. I recommend it. Judit Bar-Ilan Department of Information Science Bar-Ilan University, Israel E-mail: Judit.Bar-Ilan@biu.ac.il Bibliometrics and Research Evaluation: Uses and Abuses.YvesGingras. Cambridge, MA: MIT Press, 2016. 136 pp. $26.00 (hardcover). (ISBN:9780262035125)
单分子研究 theory and evaluation of single-molecule signals.pdf 另外的下载地址： http://bbs.bbioo.com/thread-132696-1-1.html Eli Barkai's Page http://faculty.biu.ac.il/~barkaie/ free papers and tex sources http://uk.arxiv.org/find/Computer+Science,Mathematics,Nonlinear+Sciences,Physics,Quantitative+Biology,Quantitative+Finance,Statistics/1/Barkai/0/1/0/all/8/0
国外： 1. Western Michigan University Evaluation Center The Evaluation Center’s mission is to advance the theory, practice, and utilization of evaluation. The Center’s principal activities are research, education, service, dissemination, and national and international leadership in evaluation. Providing evaluation, research, and capacity-building services to a broad array of University, public, community-based, national, and international organizations to assist them in assessing and improving their programs Conducting research on evaluation supported by federal grants to contribute to the evaluation knowledge base and to advancing theory and methodology of evaluation Engaging in academic leadership by publishing in the peer-reviewed literature, by presenting on cutting-edge evaluation issues, and through service to professional organizations and scholarly journals Administering the Interdisciplinary Ph.D. in Evaluation, which provides doctoral education and state-of-the-art research and evaluation opportunities for the next generation of evaluation scholars and practitioners. Values The core values of The Evaluation Center are excellence, integrity, service orientation, professionalism, innovation, diversity, and scholarship. http://www.wmich.edu/evalctr/home/ 2. American Evaluation Association The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of program evaluation, personnel evaluation, technology, and many other forms of evaluation. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. AEA has approximately 6800 members representing all 50 states in the US as well as over 60 foreign countries. The American Evaluation Association seeks to act in ways that embody our mission, vision, and values in pursuit of our defined policies and goals. MISSION: The American Evaluation Association’s mission is to improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of evaluation to the generation of theory and knowledge about effective human action. VISION: The American Evaluation Association’s vision is to foster an inclusive, diverse, and international community of practice positioned as a respected source of information for and about the field of evaluation. VALUES: The American Evaluation Association values excellence in evaluation practice, utilization of evaluation findings, and inclusion and diversity in the evaluation community. i. We value high quality, ethically defensible, culturally responsive evaluation practices that lead to effective and humane organizations and ultimately to the enhancement of the public good. ii. We value high quality, ethically defensible, culturally responsive evaluation practices that contribute to decision-making processes, program improvement, and policy formulation. iii. We value a global and international evaluation community and understanding of evaluation practices. iv. We value the continual development of evaluation professionals and the development of evaluators from under-represented groups. v. We value inclusiveness and diversity, welcoming members at any point in their career, from any context, and representing a range of thought and approaches. vi. We value efficient, effective, responsive, transparent, and socially responsible association operations. http://www.eval.org 3.the European Evaluation Society The primary goal of the European Evaluation Society is to promote theory, practice, and utilization of high quality evaluation especially, but not exclusively, within the European countries. This goal is obtained by bringing together academics and practitioners from all over Europe and from all professional sectors, thus creating a forum where all participants can benefit from the co-operation and bridge building opportunities of the organization. The society was founded in the Hague in 1994. The first official board was elected in autumn 1995 and started its work in January 1996. http://www.europeanevaluation.org/ 4.the UK Evaluation Society www.evaluation.org.uk 5. theCanadian Evaluation Society www.evaluationcanada.ca 6. The Australasian Evaluation Society www.aes.asn.au 7. the Source Evaluation Society www.sesnews.org 8. The Swiss Evaluation Society www.seval.ch/en/index.cfm 9.the Malaysian Evaluation Society www.mes.org.my 10. Finnish Evaluation Society (FES) http://www.finnishevaluationsociety.net/index.php?lk_id=6 11.theJapan Evaluation Society http://evaluationjp.org/english/index.html 12. Pilipinas Monitoring and Evaluation Society http://pmes.ph/ 13.Slovak Evaluation Society (SES) http://www.evaluacia.sk/en/ 14.African Evaluation Association http://www.afrea.org/home/index.cfm
I think I will just share two comments with you here. "Started class early - unacceptable as many students have class before. She didn answer any questions. RUDE! ..." "I really enjoyed learning from Dr. Yu. I want to say I appreciate her + her knowledge very much and apologize for some of the other students in the class. Thank you! I learned a lot." ps. I feel sorry that some students felt that way about me. I could not imagine how they must have suffered...
In the past, scientists who were searching for a new position had to give the total number of his published papers, usually attach a list of publications. Since the Thomson ISI Web of Science database became available, they had to give the total number of sitations or/and citations per paper. In 2005, Hirsch proposed an index h, defined as the number of papers with citation number = h, to characterize the scientific output of a researcher. Quickly, the Web of Science includes h index in its report. Nowadays, prominent scientists usually highlihgt their h indexes in their offical websites to demonstrate their accomplishments. However we usually see such two categories of scientists.
标签: evaluation
