Skip to Main Content

Scholarly Impact Research Guide

This guide provides information and instruction on the different types of metrics used to evaluate and locate one's scholarly metrics.

Criticisms of Impact Metrics

  • JIF is not a valid indicator of the quality of individual papers
  • Journal citations are usually skewed with a few articles accounting for the majority of citations. JIF uses an average making it more susceptible to outlier effects. 
  • Metrics are not normalized and can't be compared across fields
  • General journals have larger citation pools than specific journals
  • Review articles and meta-analyses receive on average, more citations possibly leading to the decline of innovative unique research
  • Higher JIF journals have longer author lists
  • JIF is highest in publishing areas with the largest number of researchers (ex: cancer research)
  • JIF is highest in areas where research outputs are the greatest
  • Has given rise to "citation cartels" where a group of editors ask authors to cite articles from each others journals
  • Negative findings or confirmatory studies, which are less frequently cited, won't be published. 
  • Funding may decrease for areas that have lower JIF
  • JIF uses a very brief two year window which is criticized as being two short to accurately account for differences in citation rates between publications
  • h-index is sexually biased since females don't produce as many papers 
  • Encourages self-citation
  • The literature will decrease in quality because the main incentive to publish will be to get higher impact metrics
  • Relying in impact metrics will increase the divide between the "haves" and the "have nots" in academia
  • Can lead to abuse of authorship credit
  • Inverse correlation between JIF and reliability of research findings

Casadevall, A., & Fang, F. C. (2014). Causes for the persistence of impact factor maniaMBio5(2), e00064-14.

Chapman, C. A., Bicca-Marques, J. C., Calvignac-Spencer, S., Fan, P., Fashing, P. J., Gogarten, J., ... & Chr. Stenseth, N. (2019). Games academics play and their consequences: How authorship, h-index and journal impact factors are shaping the future of academiaProceedings of the Royal Society B286(1916), 20192047.

Mech, E., Ahmed, M. M., Tamale, E., Holek, M., Li, G., & Thabane, L. (2020). Evaluating Journal Impact Factor: a systematic survey of the pros and cons, and overview of alternative measuresJournal of Venomous Animals and Toxins including Tropical Diseases26.

 

Best Bets

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.