Breaking down the relationship between disruption scores and citation counts

The emergence of the disruption score provides a new perspective that differs from traditional metrics of citations and novelty in research evaluation. Motivated by current studies on the differences among these metrics, we examine the relationship between disruption scores and citation counts. Intuitively, one would expect disruptive scientific work to be rewarded by high volumes of citations and, symmetrically, impactful work to also be disruptive. A number of recent studies have instead shown that such intuition is often at odds with reality. In this paper, we break down the relationship between impact and disruption with a detailed correlation analysis in two large data sets of publications in Computer Science and Physics. We find that highly disruptive papers tend to receive a higher number of citations than average. Contrastingly, the opposite is not true, as we do not find highly cited papers to be particularly disruptive. Notably, these results qualitatively hold even within individual scientific careers, as we find that—on average—an author’s most disruptive work tends to be well cited, whereas their most cited work does not tend to be disruptive. We discuss the implications of our findings in the context of academic evaluation systems, and show how they can contribute to reconcile seemingly contradictory results in the literature.