Skip to main content
SpringerLink
Log in

A framework for systematic analysis of open access journals and its application in software engineering and information systems

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

This article is a contribution towards an understanding of open access (OA) publishing. It proposes an analysis framework of 18 core attributes, divided into the areas of bibliographic information, activity metrics, economics, accessibility, and predatory issues. The framework has been employed in a systematic analysis of 30 OA journals in software engineering (SE) and information systems (IS), which were selected from among 386 OA journals in Computer Science from the Directory of OA Journals. An analysis was performed on the sample of the journals, to provide an overview of the current situation of OA journals in the fields of SE and IS. The journals were then compared between-group, according to the presence of article processing charges. A within-group analysis was performed on the journals requesting article processing charges from authors, in order to understand what is the value added according to different price ranges. This article offers several contributions. It presents an overview of OA definitions and models. It provides an analysis framework born from the observation of data and the existing literature. It raises the need to study OA in the fields of SE and IS while offering a first analysis. Finally, it provides recommendations to readers of OA journals. This paper highlights several concerns still threatening the adoption of OA publishing in the fields of SE and IS. Among them, it is shown that high article processing charges are not sufficiently justified by the publishers, which often lack transparency and may prevent authors from adopting OA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Budapest (February 2002), Bethesda (June 2003), and Berlin (October 2003).

  2. The reader should note that this study was conducted months before the publication of Bohannon (2013) “Who’s afraid of peer review” in Science, and well before the publication of Beall (2013c) “The Open-Access Movement is Not Really about Open Access”. These two articles lent to long, sometimes flaming yet interesting debate between open access advocates, traditional publishers, and librarians. Beall (2013c) caused further disagreement on his list. Both these events are worthy of separate studies. However, this manuscript was already in a soon-to-be-published state when these events happened. Therefore, it does not take into account what these events generated. While none of this changes the results of this study, the authors further acknowledge that Beall’s list is a useful contribution but it should be employed with scientific skepticism because of its subjectivity and the recent personal views of its author.

  3. Some journals, for example, have a different price to publish review articles.

  4. As a side note, this manuscript lost two URLs during peer review.

  5. The query-related URL was http://www.doaj.org/doaj?func=subject&cpId=114&uiLanguage=en

References

  • ACM. (2012). The 2012 ACM computing classification system. Association for Computing Machinery. Retrieved August 01, 2013, from http://www.acm.org/about/class/2012.

  • Antelman, K. (2004). Do open-access articles have a greater research impact? College Research Libraries, 65(5), 372–382. Retrieved from http://crl.acrl.org/content/65/5/372.abstract.

  • Arunachalam, S. (2008). Open access to scientific knowledge. DESIDOC Journal of Library & Information Technology, 28(1), 7–14. Retrieved from http://www.publications.drdo.gov.in/ojs/index.php/djlit/article/view/147.

  • Bailey, C. (2008). Open access and libraries. Collection Management, 32(3), 351–383. doi:10.1300/J105v32n03_07.

    Google Scholar 

  • Beall, J. (2012a). Predatory publishers and opportunities for scholarly societies. In American Educational Research Association meeting, Washington, D.C. (Vol. 489, pp. 1–5). doi:10.1038/489179a.

  • Beall, J. (2012b). Predatory publishers are corrupting open access. Nature, 489(7415), 179. doi:10.1038/489179a.

    Article  Google Scholar 

  • Beall, J. (2013a). Criteria for determining predatory open-access publishers (2nd ed.). Scholarly Open Access. Retrieved May 21, 2013, from http://perma.cc/LKY4-UWHH.

  • Beall, J. (2013b). OA journal stops publishing, deletes Website. Scholarly Open Access. Retrieved August 01, 2013, from http://perma.cc/BU36-WSWM.

  • Beall, J. (2013c). The open-access movement is not really about open access. tripleC, 11(2), 589–597.

    Google Scholar 

  • Beall, J. (2013d). Potential, possible, or probable predatory scholarly open-access publishers. Scholarly Open Access. Retrieved January 28, 2014, from http://perma.cc/G45Q-88B5.

  • Björk, B. C. (2003). Open access to scientific publications - an analysis of the barriers to change. Information Research, 9(2), 170. Retrieved from http://perma.cc/X6GJ-R8FD.

  • Björk, B. C., & Solomon, D. (2012). Open access versus subscription journals: A comparison of scientific impact. BMC Medicine, 10(1), 73. doi:10.1186/1741-7015-10-73.

    Article  Google Scholar 

  • Bjørnshauge, L., Brage, R., Brage, S., & Jørgensen, L. (2013a). DOAJ. Directory of Open Access Journals. Retrieved May 20, 2013, from http://www.doaj.org/.

  • Bjørnshauge, L., Brage, R., Brage, S., & Jørgensen, L. (2013b). DOAJ announces new selection criteria. Directory of Open Access Journals. Retrieved August 06, 2013, from http://perma.cc/8CBJ-RFTN.

  • BOAI. (2002). Budapest open access initiative. Budapest Open Access Initiative. Retrieved May 27, 2013, from http://perma.cc/8BT3-KFNY.

  • Bohannon, J. (2013). Who’s afraid of peer review? Science (New York, N.Y.), 342(6154), 60–65. doi:10.1126/science.342.6154.60.

    Article  Google Scholar 

  • Boisvert, R. F., & Davidson, J. W. (2013). Positioning ACM for an open access future. Communications of the ACM, 56(2), 5. doi:10.1145/2408776.2408777.

    Article  Google Scholar 

  • Brown, P. O., Cabell, D., Chakravarti, A., Cohen, B., Delamothe, T., Eisen, M.,… Watson, L. (2003). Bethesda statement on open access publishing. Harvard Dash. Retrieved May 29, 2013, from http://dash.harvard.edu/handle/1/4725199.

  • Butler, D. (2012). Investigating journals: The dark side of publishing. Nature, 495(7442), 433–435. doi:10.1038/495433a.

    Article  Google Scholar 

  • CLOCKSS. (2013). Contribute to CLOCKSS. CLOCKSS. Retrieved August 06, 2013, from http://perma.cc/BS77-P23N.

  • Crawford, S., & Stucki, L. (1990). Peer review and the changing research record. Journal of the American Society for Information Science, 41(3), 223–228. doi:10.1002/(SICI)1097-4571(199004)41:3%3C223:AID-ASI14%3E3.0.CO;2-3.

    Google Scholar 

  • CrossRef. (2013). 2013 CrossRef Deposit Fees. CrossRef. Retrieved May 25, 2013, from http://perma.cc/XY86-JXYW.

  • Davis, P. (2009). Open access publisher accepts nonsense manuscript for dollars. The Scholarly Kitchen. Retrieved July 31, 2013, from http://perma.cc/UDK7-9QSU.

  • DOAJ. (2013). About. Directory of Open Access Journals. Retrieved January 27, 2014, from http://perma.cc/BR9R-KWSC.

  • Eysenbach, G. (2008). Black sheep among open access journals and publishers. Gunther Eysenbach Random Research Rants Blog. Retrieved July 31, 2013, from http://perma.cc/8SQU-JBLC.

  • Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. FASEB Journal: Official Publication of the Federation of American Societies for Experimental Biology, 22(2), 338–342.

    Article  Google Scholar 

  • Franceschet, M. (2010). The role of conference publications in CS. Communications of the ACM, 53(12), 129. doi:10.1145/1859204.1859234.

    Article  Google Scholar 

  • Graziotin, D. (2013, May 18). Comparison of open access software engineering and information systems journals. figshare. doi:10.6084/m9.figshare.704442.

  • Gumpenberger, C., Ovalle-Perandones, M.-A., & Gorraiz, J. (2012). On the impact of gold open access journals. Scientometrics, 96(1), 221–238. doi:10.1007/s11192-012-0902-7.

    Article  Google Scholar 

  • Gutman, S. A. (2011). Copyright in the age of digital scholarship. American Journal of Occupational Therapy, 65(2), 123–124. doi:10.5014/ajot.2011.000877.

    Article  Google Scholar 

  • Harnad, S., & Brody, T. (2004). Comparing the impact of open access (OA) vs. non-oa articles in the same journals. DLib Magazine, 10(6), 2–6. doi:10.1045/june2004-harnad.

  • Harnad, S., Brody, T., Vallières, F., Carr, L., Hitchcock, S., Gingras, Y.,… Hilf, E. R. (2008). The access/impact problem and the green and gold roads to open access: An update. Serials Review, 34(1), 36–40. doi:10.1016/j.serrev.2007.12.005

    Google Scholar 

  • Holmstrom, A. (2012). What happened to online articles published in K-theory (Springer journal)? Mathoverflow. Retrieved August 01, 2013, from http://perma.cc/46VC-46WG.

  • Houghton, J. W., & Oppenheim, C. (2010). The economic implications of alternative publishing models. Prometheus: Critical Studies in Innovation, 28(1), 41–54. doi:10.1080/08109021003676359.

  • IEEE. (2013). IEEE open-article processing charges. Institute of Electrical and Electornics Engineers. Retrieved May 20, 2013, from http://perma.cc/LZ56-ARGG.

  • ISSN. (2013). ISSN and electronic publications. ISSN International Centre. Retrieved August 01, 2013, from http://perma.cc/R2T3-4Z56.

  • Karen, C. (2013). Predatory publishers. Library Journal. Retrieved January 29, 2014, from http://perma.cc/PZZ9-KMKN.

  • Koehler, W. (2002). Web page change and persistence? A four-year longitudinal study. Journal of the American Society for Information Science and Technology, 53(2), 162–171. doi:10.1002/asi.10018.

    Article  Google Scholar 

  • Kuhn, T. S. (1970). The structure of scientific revolutions. In O. Neurath (Ed.), Philosophical review (Vol. II, p. 210). London: University of Chicago Press. doi:10.1119/1.1969660.

  • Laakso, M., Welling, P., Bukvova, H., Nyman, L., Björk, B. C., & Hedlund, T. (2011). The development of open access journal publishing from 1993 to 2009. PLoS ONE, 6(6), e20961. doi:10.1371/journal.pone.0020961.

    Article  Google Scholar 

  • Laika Spoetnik, J. (pseudonym). (2011). Jeffrey Beall’s list of predatory, open-access publishers. Laika’s MedLibLog.

  • Langston, M., & Tyler, J. (2004). Linking to journal articles in an online teaching environment: The persistent link, DOI, and OpenURL. The Internet and Higher Education, 7(1), 51–58. doi:10.1016/j.iheduc.2003.11.004.

    Article  Google Scholar 

  • Leubsdorf, C. (2012). Annotum: Launching a peer-reviewed journal online for free. Learned Publishing, 25(2), 8. doi:10.1087/20120204.

    Article  Google Scholar 

  • Ley, M. (2002). The DBLP computer science bibliography: Evolution, research issues, perspectives. In 9th international symposium on string processing and information retrieval (SPIRE 2013) (Vol. 2476, pp. 1–10). Lisbon, Portugal: Springer, Berlin. doi:10.1007/3-540-45735-6.

  • Liu, Z. (2006). Print vs. electronic resources: A study of user perceptions, preferences, and use. Information Processing and Management, 42(2), 583–592. doi:10.1016/j.ipm.2004.12.002.

    Article  Google Scholar 

  • Maniatis, P., Roussopoulos, M., Giuli, T. J., Rosenthal, D. S. H., & Baker, M. (2005). The LOCKSS peer-to-peer digital preservation system. ACM Transactions on Computer Systems, 23(1), 2–50. doi:10.1145/1047915.1047917.

    Article  Google Scholar 

  • Mattern, F. (2008). Bibliometric evaluation of computer science—Problems and pitfalls. In European computer science summit (Vol. 2008, p. 6). Zurich, Switzerland. Retrieved from http://www.inf.ethz.ch/department/IS/vs/publ/slides/Bibliometry-ECSS-Summit-08.pdf.

  • Max Planck Society. (2003). Berlin declaration. Open Access at the Max Planck Society. doi:10.1353/hrq.2005.0002.

  • Molloy, J. C. (2011). The open knowledge foundation: Open data means better science. PLoS Biology, 9(12), e1001195. doi:10.1371/journal.pbio.1001195.

    Article  Google Scholar 

  • Mosallahnezhad, R. (2007). REMOVED: Cooperative, compact algorithms for randomized algorithms. Applied Mathematics and Computation. doi:10.1016/j.amc.2007.03.011.

  • Navidi, W. (2010). Statistics for engineers and scientists. Education (3rd ed., Vol. 6). McGraw-Hill. doi:10.2307/2288012.

  • OASPA. (2013). Code of conduct. Open access scholarly publishers association. Retrieved from http://perma.cc/F874-Z72N.

  • Parks, R. P. (2002). The Faustian grip of academic publishing. Journal of Economic Methodology, 9(3), 317–335. doi:10.1080/1350178022000015122.

    Article  Google Scholar 

  • Piwowar, H. (2013). Altmetrics: Value all research products. Nature, 493(7431), 159. doi:10.1038/493159a.

    Google Scholar 

  • Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from http://perma.cc/D7YM-CCGB.

  • Pringle, J. (2013). Do Open Access journals have impact? Nature Web focus: Access to the literature. Retrieved June 05, 2013, from http://perma.cc/G26H-RP56.

  • Regazzi, J. (2004). The shifting sands of open access publishing, a publisher’s view. Serials Review, 30(4), 275–280. doi:10.1016/j.serrev.2004.09.010.

    Article  Google Scholar 

  • Reich, V., & Rosenthal, D. (2004). Preserving today’s scientific record for tomorrow. BMJ (Clinical research ed.), 328(7431), 61–62. doi:10.1136/bmj.328.7431.61.

    Article  Google Scholar 

  • Schroter, S., & Tite, L. (2006). Open access publishing and author-pays business models: A survey of authors’ knowledge and perceptions. Journal of the Royal Society of Medicine, 99(3), 141–148. doi:10.1258/jrsm.99.3.141.

    Article  Google Scholar 

  • Schroter, S., Tite, L., & Smith, R. (2005). Perceptions of open access publishing: Interviews with journal authors. BMJ (Clinical research ed.), 330(7494), 756. doi:10.1136/bmj.38359.695220.82.

    Article  Google Scholar 

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 497. doi:10.1136/bmj.314.7079.497.

    Article  Google Scholar 

  • SHERPA. (2002). RoMEO—Search—Publisher copyright policies & self-archiving. SHERPA Services based at the University of Nottingham. Retrieved May 29, 2013, from http://www.sherpa.ac.uk/romeo/search.php.

  • Smith, R. (1999). Opening up BMJ peer review. BMJ, 318(7175), 4–5. doi:10.1136/bmj.318.7175.4.

    Article  Google Scholar 

  • Sol, H. G. (1983). A feature analysis of information systems design methodologies: Methodological considerations. In Information Systems design methodologies: A feature analysis (pp. 1–7).

  • Song, X., & Osterweil, L. J. (1992). Toward objective, systematic design-method comparisons. IEEE Software, 9(3), 43–53. doi:10.1109/52.136166.

    Article  Google Scholar 

  • Sotudeh, H., & Horri, A. (2007). Tracking open access journals evolution: Some considerations in open access data collection validation. Journal of the American Society for Information Science and Technology, 58(11), 1578–1585. doi:10.1002/asi.20639.

    Article  Google Scholar 

  • Stribling, J., Krohn, M., & Aguayo, D. (2005). SCIgen—An automatic CS paper generator. PDOS Research. Retrieved July 31, 2013, from http://pdos.csail.mit.edu/scigen/.

  • Suber, P. (2008). Gratis and libre open access. SPARC Open Access Newsletter. Retrieved May 29, 2013, from http://perma.cc/AD8E8DZE.

  • Suber, P. (2009). Timeline of the open access movement. Open Access Directory. Retrieved July 29, 2013, from http://perma.cc/VYT6-5DWB.

  • Suber, P. (2012). Open Access (1st ed., pp. 1–255). Cambridge, MA: The MIT Press.

  • Swoger, B. (2013). It’s not about predators, it’s about journal quality. Scientific American Blogs. Retrieved January 29, 2014, from http://perma.cc/SCY9-BQV8.

  • Taylor, M. (2012). Crowdsourcing a database of “predatory OA journals.” Sauropod Vertebra Picture of the Week. Retrieved January 29, 2014, from http://perma.cc/629G-VBGS.

  • Van Noorden, R. (2012). Journal offers flat fee for “all you can publish”. Nature, 486(7402), 166. doi:10.1038/486166a.

    Article  Google Scholar 

  • Van Noorden, R. (2013). Open access: The true cost of science publishing. Nature, 495(7442), 426–429. doi:10.1038/495426a.

    Article  Google Scholar 

  • Vardi, M. Y. (2012). Predatory scholarly publishing. Communications of the ACM, 55(7), 5. doi:10.1145/2209249.2209250.

    Article  Google Scholar 

  • Vision, T. J. (2010). Open data and the social contract of scientific publishing. BioScience, 60(5), 330–331. doi:10.1525/bio.2010.60.5.2.

    Article  Google Scholar 

  • Wainer, J., Billa, C., & Goldenstein, S. (2011). Invisible work in standard bibliometric evaluation of computer science. Communications of the ACM, 54(5), 141. doi:10.1145/1941487.1941517.

    Article  Google Scholar 

  • Willinsky, J. (2005). Open journal systems: An example of open source software for journal management and publishing. Library Hi Tech, 23(4), 504–519. doi:10.1108/07378830510636300.

    Article  Google Scholar 

  • Willinsky, J. (2006). The access principle. electronic publishing (1st ed., Vol. 85, pp. 227–232). Cambridge, MA: MIT Press.

  • Worlock, K. (2013). The pros and cons of open access. Nature Web Focus: Access to the Literature. Retrieved June 05, 2013, from http://perma.cc/464S-3Z8K.

  • Young, B. (1988). International standard serial numbers. Serials: The Journal for the Serials Community, 1(2), 48–50. doi:10.1629/010248.

Download references

Acknowledgments

The authors are thankful to Elena Borgogno for her valuable help during the study and when writing this article. The authors would like to thank Christian Gumpenberger for the insightful comments he offered to improve the manuscript. Lastly, the authors are thankful to two anonymous reviewers for the several suggestions that significantly improved the article.

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Free University of Bozen-Bolzano, Piazza Domenicani 3, 39100, Bolzano, Italy

    Daniel Graziotin, Xiaofeng Wang & Pekka Abrahamsson

Authors
  1. Daniel Graziotin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofeng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pekka Abrahamsson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel Graziotin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Graziotin, D., Wang, X. & Abrahamsson, P. A framework for systematic analysis of open access journals and its application in software engineering and information systems. Scientometrics 101, 1627–1656 (2014). https://doi.org/10.1007/s11192-014-1278-7

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1278-7

Keywords

Search

Navigation