ESEP 8:93-102 (2008)  -  doi:10.3354/esep00084

Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results

Lutz Bornmann1,*, Rüdiger Mutz1, Christoph Neuhaus1, Hans-Dieter Daniel1,2

1ETH Zurich, Professorship for Social Psychology and Research on Higher Education, Zähringerstr. 24, 8092 Zurich, Switzerland
2University of Zurich, Evaluation Office, Mühlegasse 21, 8001 Zurich, Switzerland

ABSTRACT: With the ready accessibility of bibliometric data and the availability of ready-to-use tools for generating bibliometric indicators for evaluation purposes, there is the danger of inappropriate use. Here we present standards of good practice for analyzing bibliometric data and presenting and interpreting the results. Comparisons drawn between research groups as to research performance are valid only if (1) the scientific impact of the research groups or their publications are looked at by using box plots, Lorenz curves, and Gini coefficients to represent distribution characteristics of data (in other words, going beyond the usual arithmetic mean value), (2) different reference standards are used to assess the impact of research groups, and the appropriateness of the reference standards undergoes critical examination, and (3) statistical analyses comparing citation counts take into consideration that citations are a function of many influencing factors besides scientific quality.


KEY WORDS: Citation counts · Research evaluation · Citation distribution · Reference standards · Lorenz curve · Gini coefficient · Box plots


Full text in pdf format 
Cite this article as: Bornmann L, Mutz R, Neuhaus C, Daniel HD (2008) Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci Environ Polit 8:93-102

Export citation
Mail this link - Contents Mailing Lists - RSS
- -