Skip to main content
SpringerLink
Log in

Editorial decisions with informed and uninformed reviewers

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

In this paper, the interaction of the reviewers is analyzed in the context of the open peer review. There, the reviewer group can exchange comments on the manuscript. This group of experts is called the informed reviewer group because, with the exchange of comments, each reviewer knows about the trends in the evaluation of the manuscript. Then, we show that, in a Bayesian Nash equilibrium, the probability that the editor chooses to accept or reject is the same. In this paper, this reviewer group is also compared to a group of uninformed reviewers in the classic review system. In this second case, the uninformed reviewers only communicate with the journal editor. They always believe that their recommendation is in the majority, and the probability that the uninformed reviewers abstain from making any recommendation increases as the number of reviewers grows. In some open access journals, the editor can obtain recommendations by both uninformed and informed reviewers groups. In this situation, the editor’s decision could depend on the order followed by the editor in the analysis of the recommendations of each reviewers group (informed and uninformed). A free online tool has been created to help the editor with this process.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Armstrong, J. S. (1982). Barriers to scientific contributions: The authors formula. Behavioral and Brain Sciences, 5(02), 197–199.

    Article  Google Scholar 

  • Bornmann, L. (2008). Scientific peer review. An analysis of the peer review process from the perspective of sociology of science theories. Human Architecture-Journal of the Sociology of Self-Knowledge, 6(2), 23–38.

    Google Scholar 

  • Bornmann, L. (2012). The Hawthorne effect in journal peer review. Scientometrics, 91, 857–862. https://doi.org/10.1007/s11192-011-0547-y.

    Article  Google Scholar 

  • Bornmann, L. (2014). Book Review: Gould, T.H.P. (2013). Do we still need peer review? An argument for change. Scarecrow Press: Plymouth, UK. Journal of the American Society for Information Science and Technology, 65(1), 209–213.

    Google Scholar 

  • Brown, R. J. C. (2004). The use of double anonymity in peer review: A decision whose time has come? Quality Assurance, 11, 103–109.

    Article  Google Scholar 

  • Bruza, P. D., Wang, Z., & Busemeyer, J. R. (2015). Quantum cognition: A new theoretical approach to psychology. Trends in Cognitive Science, 19(7), 383–393.

    Article  Google Scholar 

  • Garcia, J. A., Rodriguez-Sanchez, R., & Fdez-Valdivia, J. (2014). The principal-agent problem in peer review. Journal of the Association for Information Science and Technology, 66(2), 297–308. https://doi.org/10.1002/asi.23169.

    Article  Google Scholar 

  • Garcia, J. A., Rodriguez-Sanchez, R., & Fdez-Valdivia, J. (2015). Adverse selection of reviewers. Journal of the Association For Information Science and Technology, 66(6), 1252–1262. https://doi.org/10.1002/asi.23249.

    Article  Google Scholar 

  • Garcia, J. A., Rodriguez-Sanchez, R., & Fdez-Valdivia, J. (2016). Why the referees’ reports I receive as an editor are so much better than the reports I receive as an author? Scientometrics, 106(3), 967–986. https://doi.org/10.1007/s11192-015-1827-8.

    Article  Google Scholar 

  • Garcia, J. A., Rodriguez-Sanchez, R., & Fdez-Valdivia, J. (2017). Problems with open participation in peer review. Scientometrics, 112(3), 1881–1885. https://doi.org/10.1007/s11192-017-2445-4.

    Article  Google Scholar 

  • Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17.

    Article  Google Scholar 

  • Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6, 588. https://doi.org/10.12688/f1000research.11369.1.

    Article  Google Scholar 

  • Taylor, C. R., & Yildirim, H. (2010). Public information and electoral bias. Games and Economic Behavior, 68, 353–375.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research was sponsored by the Spanish Board for Science, Technology, and Innovation under Grant TIN2017-85542-P, and co-financed with European FEDER funds. Sincere thanks are due to the reviewers for their constructive suggestions.

Author information

Authors and Affiliations

  1. Departamento de Ciencias de la Computación e I. A., CITIC-UGR, Universidad de Granada, 18071, Granada, Spain

    Rosa Rodriguez-Sánchez, J. A. García & J. Fdez-Valdivia

Authors
  1. Rosa Rodriguez-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. A. García
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Fdez-Valdivia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rosa Rodriguez-Sánchez.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rodriguez-Sánchez, R., García, J.A. & Fdez-Valdivia, J. Editorial decisions with informed and uninformed reviewers. Scientometrics 117, 25–43 (2018). https://doi.org/10.1007/s11192-018-2875-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-018-2875-7

Keywords

Search

Navigation