Abstract
On the basis of the Impact Factor of Journal Citation Reports developed by ISI as a journal quality indicator, this paper puts forth an ordinal regression model to estimate the journal’s position by terciles. The set of explanatory variables includes the H-index of its Editor-in-chief, percentage of papers published in the journal that received external funding, average number of papers published yearly, and two factors concerning the scope and structure of the journal. The proposed model was applied to the field of Dentistry, Oral Surgery and Medicine, and led us to the conclusion that the above mentioned covariables alone had a significant input in the model, but not the factors. The essay performed on a sample of 30 Dentistry journals included in JCR provided a confirmatory correct classification rate (CCR) of 80%, with a predictive CCR of 75% on a sample of eight new journals not previously considered in the phase of model estimation.
References
Alberts, B. (2013). Impact factor distortions. Science, 340(6134), 787.
Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009). Differences in impact factor across fields and over time. Journal of the American Society for Information Science and Technology, 60(1), 27–34.
Anscombe, F. J. (1948). The validity of comparative experiments. Journal of the Royal Statistical Society, Series A, 111(3), 181–211.
Bollen, J., Van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLOS ONE, 4(6), 0006022. https://doi.org/10.1371/journal.pone.
Bordons, M., Fernandez, M. T., & Gomez, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance in a peripheral country. Scientometrics, 53(2), 195–206.
Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2012). Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32(7), 1861–1867.
Bornmann, L., Mutz, R., Marx, W., Schier, H., & Daniel, H. D. (2011). A multilevel modeling approach to investigating the predictive validity of editorial decisions: Do the editors of a high profile journal select manuscripts that are highly cited after publication? Journal of the Royal Statistical Society, Series A, 174(4), 857–879.
Bornmann, L., & Williams, R. (2017). Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on Researcher ID data. Journal of Informetrics, 11(3), 788–799.
Buela, G. (2003). Evaluating quality of articles and scientific journals. Proposal of weighted impact factor and a quality index? Psicothema, 15(1), 23–35.
Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae et Therapiae Experimentalis, 56(4), 223–226.
Falagas, M. E., Kouranos, V. D., Arencibia-Jorge, R., & Karageorgopoulos, D. E. (2008a). Comparison of SCImago journal rank indicator with journal impact factor. FASEB Journal, 22(8), 2623–2628.
Fassoulaki, A., Papilas, K., Paraskeva, A., & Patris, N. (2002). Impact factor bias and proposed adjustments for its determination. Acta Anaetesiologica Scandinavica, 46(7), 902–905.
Fassoulaki, A., Paraskeva, A., Papilas, K., & Karabinis, G. (2000). Self-citations in six anaesthesia journals and their significance in determining the impact factor. British Journal of Anaesthesia, 84(2), 266–269.
Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108–111.
Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161, 979–980.
Garfield, E. (2006). The history and meaning of the journal impact factor. Journal of the American Medical Association, 295(1), 90–93.
Hansson, S. (1995). Impact factor as a misleading tool in evaluation of medical journals. The Lancet, 346(8979), 906–906.
Kay, J., Memon, M., de Sa, D., Simunovic, N., Duong, A., Karlsson, J., et al. (2017). The H-index of editorial board members correlates positively with the impact factor of Sports Medicine journals. Orthopaedic Journal of Sports Medicine, 5(3), 2325967117694024. https://doi.org/10.1177/2325967117694024.
Kurmis, A. P. (2003). Understanding the limitations of the journal impact factor. The Journal of Bone and Joint Surgery, 85(12), 2449–2454.
Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. Scientometrics, 92(2), 355–365.
Leydesdorff, L., & Bornmann, L. (2011). How fractional counting of citations affects the impact factor: Normalization in terms of differences in citation potentials among fields of Science. Journal of the American Society for Information Science and Technology, 62(2), 217–229.
Leydesdorff, L., & Opthof, T. (2010). Scopus’s source normalized impact per paper (SNIP) versus a journal impact factor based on fractional counting of citations. Journal of the American Society for Information Science and Technology, 61(11), 2365–2369.
Lucena, C., Souza, E. M., Voinea, G. C., Pulgar, R., Valderrama, M. J., & De-Deus, G. (2017). A quality assessment of randomized controlled trial reports in Endodontics. International Endodontic Journal, 50(3), 237–250.
Malay, D. S. (2013). Impact factors and other measures of a journal’s influence. The Journal of Food and Ankle Surgery, 52(3), 285–287.
McVeigh, M. F., & Mann, S. J. (2009). The journal impact factor denominator defining citable (counted) items. Journal of the American Medical Association, 302(10), 1107–1109.
Moed, H. F. (2002). The impact-factors debate: The ISI’s uses and limits. Nature, 415(6873), 731–732.
Moed, H. F., & Vanleeuwen, T. N. (1995). Improving the accuracy of Institute for Scientific Information journal impact factors. Journal of the American Society for Information Science and Technology, 46(6), 461–467.
Mutz, R., & Daniel, H.-D. (2012a). The generalized propensity score methodology for estimating unbiased journal impact factors. Scientometrics, 92, 377–390.
Mutz, R., & Daniel, H.-D. (2012b). Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor. Journal of Informetrics, 6(2), 169–176.
Roberts, R. J. (2017). An obituary for the impact factor. Nature, 546(7660), 600.
Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association, 91(1), 42–46.
Seglen, P. O. (1997a). Citations and journal impact factors: Questionable indicators of research quality. Allergy, 52(11), 1050–1056.
Seglen, P. O. (1997b). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314(7079), 498–502.
Seglen, P. O. (1998). Citation rates and journal impact factors are not suitable for evaluation of research. Acta Orthopaedica Scandinavica, 69(3), 224–229.
Simons, K. (2008). The misused impact factor. Science, 322(5899), 165.
Valderrama, P., Escabias, M., Jiménez-Contreras, E. Valderrama, M. J. & Baca, P. (2017). Bibliometric variables determining the quality of a dentistry journal. In Skiadas, C. H. (Ed.), Proceedings of the 17th conference of the applied stochastic models and data analysis international society, pp. 825–831.
Van Noorden, R. (2016). Impact factor gets heavyweight rival. Nature, 540(7633), 325–326.
Vanclay, J. K. (2012). Impact factor: Outdated artefact or stepping-stone to journal certification? Scientometrics, 92(2), 211–238.
Varki, A. (2017). Rename the impact factor. Nature, 548(7668), 393.
Wagner, G., Prester, J., Roche, M., Benlian, A., & Schryen, G. (2006). Factors affecting the scientific impact of literature reviews: A scientometric study. In proceedings of the 37th international conference on information systems, Dublin 2016, Vol. 23, pp. 1659–1682.
Wilsdon, J., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. HEFCE. https://doi.org/10.13140/RG.2.1.4929.1363.
Zitt, M., & Small, H. (2008). Modifying the journal impact factor by fractional citation weighting: The audience factor. Journal of the American Society for Information Science and Technology, 59(11), 1856–1860.
Acknowledgements
This work was supported by MTM2017-88708-P of Secretaría de Estado de Investigación, Desarrollo e Innovación, Ministerio de Economía y Competitividad de España.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflicts of interest.
Additional information
Reports of my death are greatly exaggerated (Mark Twain).
Rights and permissions
About this article
Cite this article
Valderrama, P., Escabias, M., Jiménez-Contreras, E. et al. Proposal of a stochastic model to determine the bibliometric variables influencing the quality of a journal: application to the field of Dentistry. Scientometrics 115, 1087–1095 (2018). https://doi.org/10.1007/s11192-018-2707-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-018-2707-9