Bibliometric Evaluation vs. Informed Peer Review:Evidence from Italy

A relevant question for the organization of large scale research assessments is whether
bibliometric evaluation and informed peer review where reviewers know where the work was
published, yield similar results. It would suggest, for instance, that less costly bibliometric
evaluation might - at least partly - replace informed peer review, or that bibliometric
evaluation could reliably monitor research in between assessment exercises. We draw on our
experience of evaluating Italian research in Economics, Business and Statistics, where almost
12,000 publications dated 2004-2010 were assessed. A random sample from the available
population of journal articles shows that informed peer review and bibliometric analysis
produce similar evaluations of the same set of papers. Whether because of independent
convergence in assessment, or the influence of bibliometric information on the community of
reviewers, the implication for the organization of these exercises is that these two approaches
are substitutes.