The Dayside : Tracking the dynamics of individual scientific impact

My title comes from the subtitle of a newly posted paper on arXiv by Raj Kumar Pan and Santo Fortunato of Finland’s Aalto University. In their paper, the two researchers introduce a new bibliometric index, which they call the author impact factor (AIF).

The traditional impact factor (IF) of a given journal, Pan and Fortunato remind their readers, is the average number of citations from papers published in year t to papers published in the journal in the two preceding years, t − 1 and t − 2. The AIF is similar in concept, but whereas the IF applies to journals, the AIF applies to individual researchers.

Your personal AIF is the average number of citations from papers published in year t to the papers that you published between year t − 1 and year t − 5. The longer evaluation window is needed to smooth year-to-year fluctuations.

Why introduce a new metric? Pan and Fortunato point first to the IF’s shortcomings. Even inNaturePhysical Review Letters, and other prestigious journals, the median number of citations to a journal’s papers is significantly lower than the mean number of citations, thanks to a few papers that garner lots of citations.

Philip Anderson's author impact factor spans six decades and features six prominent peaks. CREDIT: AIP Emilio Segre Visual Archives, Physics Today Collection

Philip Anderson’s author impact factor spans six decades and features six prominent peaks. CREDIT: AIP Emilio Segrè Visual Archives, Physics Today Collection

For example, last year Science had an impact factor of 31.027. But a paper that contributed to that impressive figure, “Sex determination in the social amoeba Dictyostelium discoideum from the journal’s 10 December 2010 issue, has been cited just twice (I didn’t count the citation from the commentary in the same issue). To find that lightly cited paper, I didn’t have to look hard. It was the first one I clicked on in the issue’s table of contents.

Another popular metric, the h-index, has different shortcomings, Pan and Fortunato contend. A given researcher’s h-index is n if he or she has published n papers that have each garnered at least n citations. Unlike the IF, the h-index applies to individuals, but its cumulative nature masks the ups and downs of a researcher’s publishing career. What’s more, the h-index tends to change from year to year at a low and not especially illuminating rate.

According to Pan and Fortunato, their AIF has two principal advantages over the IF and theh-index. First, because the AIF is calculated over a limited time span, it tracks the evolution of a researcher’s impact. Second, because the AIF is an average, researchers would be motivated to publish papers that are likely to garner a high number of citations. Because authors would be penalized for papers that garner few or no citations, the proliferation of papers of marginal significance would be arrested.

To support the case for adopting the AIF, Pan and Fortunato calculated and plotted the AIF for 12 Nobel science laureates, including the physicists Philip Anderson, David Gross, Wolfgang Ketterle, and Steven Weinberg. Whereas the four physicists’ h-indices have more or less the same profile, their AIFs have different numbers of peaks.

Weinberg's plot

In Weinberg’s plot, three big peaks are apparent. The first peak in the late 1960s arises from two of his early papers, “Pion scattering lengths,” (Physical Review Letters, volume 17, page 616, 1966) and “A model of leptons,” (Physical Review Letters, volume 19, page 1264, 1967). The second peak, in the early 1980s, presumably arises from CERN’s detection in 1983 of the W and Z bosons, whose existence Weinberg had predicted.

I’m less sure of the origin of Weinberg’s third peak, in the mid 1990s, but it could be from his second most-cited paper, “The cosmological constant problem,” Reviews of Modern Physics, volume 61, page 1 (1989). Although the accelerating expansion of the universe, which can be attributed to a cosmological constant, was discovered a decade after the paper appeared, observations made in the early and mid 1990s by the Hubble Space Telescope and other observatories had revealed serious deficiencies in the prevailing cosmological paradigm. The earliest citations to Weinberg’s paper came from particle and gravitational theorists, but by the mid 1990s citations from cosmologists were in the majority.

Whether the AIF catches on remains to be seen. Thomson Reuters, which calculates the traditional IF, could presumably build a software tool that researchers could use to calculate their own AIFs, as could Google Scholar. Publishers could build such a tool, too.

I for one would like to see my own AIF. Wouldn’t you like to see yours?

 

This post was originally posted on The Dayside, Charles Day’s blog on Physics Today‘s website.

Tagged with: , , ,
Posted in Bits and Bytes