Abstract
In order to resolve questions frequently raised in the context of research evaluation about the citation rates of journal publications in relation to other types of publications, the total research output of substantial institutions or systems has to be brought under bibliographic control. That precondition has rarely been met: there are few published studies of the total range of publications of major research institutions, including books, book chapters, technical reports and published conference proceedings. The Research Evaluation and Policy Project (REPP) at the Australian National University (ANU) has established a database covering all the publications from the Institute of Advanced Studies (IAS), a fulltime research institution at the ANU, and has examined in detail citations in the journal literature accruing to all types of publications. The database contains a significant number of publications, nearly 30 000 items, and covers the sciences and the social sciences and humanities. This data enables us to examine whether the citation record of research publications appearing in journals indexed by the Institute for Scientific Information (ISI) is a useable surrogate for the citation record within ISI journals of all model of publication. We contend that, if certain preconditions are met, the choice of citation rate is not critical.
Similar content being viewed by others
References and notes
W. Shrum, Y. Shenhav, Science and technology in less developed countries, In:S. Jasanoff et al. (Eds),Handbook of Science and Technology Studies, Sage Publications Inc., California, 1995, pp. 627–651.
P. Bourke, L. Butler,Monitoring Research in the Periphery: Australia and the ISI indices. Research Evaluation and Policy Project Monograph Series No. 3, Research School of Social Sciences, The Australian National University, 1966.
National Board of Employment, Education and Training (NBEET).Quantitative Indicators of Australian Academic Research, Commissioned Report No. 27, Australian Government Publishing Service, Canberra, 1994.
The CPR database includes information on research output in both published and unpublished modes, but our analysis is restricted to published output only.
P. Bourke, L. Butler, 1996,op. cit. Monitoring Research in the Periphery: Australia and the ISI indices. Research Evaluation and Policy Project Monograph Series No. 3, Research School of Social Sciences, The Australian National University.
The fields listed here are from the Australian Standard Research Classification which was prepared by the Australian Bureau of Statistics for use in the measurement and analysis of research and experimental development in Australia.
P. Bourke, L. Butler, The use of bibliometric data in evaluating a research university: Issues and measures, In:M. Koenig, A. Bookstein (Eds),Fifth International Conference of the International Society for Scientometrics and Informetrics Proceedings — 1995. Learned Information Inc., NJ., 1995, pp. 93–102.
The Research School of Social Sciences and the Research School of Pacific and Asian Studies.
S. Maricic, Impact factors of peripheral journals,Journal of Documentation, 45(4) (1989) 330–331.
L. J. Zmaic, S. Maricic, V. Simeon, Visibility of peripheral journals through the Science Citation Index,Information Processing & Management, (1989) 25.
The six IAS research schools referred to in this table are the Research School of Chemistry (RSC), John Curtin School of Medical Research (JCSMR), Mount Stromlo and Siding Springs Observatories (MSSSO), Research School of Earth Sciences (RSES), Research School of Physical Sciences (RSPS) and Research School of Biological Sciences (RSBS).
It is interesting to look at the two most highly cited non-SCI journal publications. One is from RSBS and appeared in theJournal of Molecular and Applied Genetics in 1981. This journal appears to have become a source journal forCurrent Contents in 1984 and 1985. The other article was from JCSMR and appeared in the journalComprehensive Virology. This journal was a source journal for the SCI in 1983 and 1984. The details of these two highly cited non-SCI journal articles (or, more accurately, non-SCI at the time of their publication) suggests that the common story for the more highly cited Australian publications of this type may be that they are in journals on the fringe of being accepted for indexing by ISI.
In this particular instance the departments are not directly comparable as they undertake research in quite different fields; but if this analysis were of ‘like’ departments, it is easy to see the significance of the choice of citation rate.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bourke, P., Butler, L. Publication types, citation rates and evaluation. Scientometrics 37, 473–494 (1996). https://doi.org/10.1007/BF02019259
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02019259