Skip to main content
SpringerLink
Log in

Measuring Science

Capita Selecta of Current Main Issues

  • Chapter
Handbook of Quantitative Science and Technology Research

Abstract

After a review of developments in the quantitative study of science, particularly since the early 1970s, I focus on two current main lines of ‘measuring science’ based on bibliometric analysis. With the developments in the Leiden group as an example of daily practice, the measurement of research performance and, particularly, the importance of indicator standardisation are discussed, including aspects such as interdisciplinary relations, collaboration, ‘knowledge users’. Several important problems are addressed: language bias; timeliness; comparability of different research systems; statistical issues; and the ‘theory-invariance’ of indicators. Next, an introduction to the mapping of scientific fields is presented. Here basic concepts and issues of practical application of these ‘science maps’ are addressed. This contribution is concluded with general observations on current and near-future developments, including network-based approaches, necessary ‘next steps’ are formulated, and an answer is given to the question ‘Can science be measured?’

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 429.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 549.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Adam, D. (2002). The counting house. Nature, 415, 726–729.

    Google Scholar 

  • Albert, M.B., Avery, D., Narin, F., MacAllister, P. (1991). Direct validation of citation counts as indicators of industrially important patents. Research Policy, 20, 251–259.

    Article  Google Scholar 

  • Arunachalam, S., Srinivasan, R., Raman, V. (1994). International collaboration in science-participation by the Asian giants. Scientometrics, 30, 7–22.

    Article  Google Scholar 

  • Bar-Ilan, J. (2001). Data collection methods on the Web for informetric purposes — A review and analysis. Scientometrics, 50, 7–32.

    Article  Google Scholar 

  • Bayer, A.E., Smart, J.C., McLaughlin, G.W. (1990). Mapping intellectual structure of a scientific subfield through author cocitations. Journal of the American Society for Information Science, 41, 444–452.

    Article  Google Scholar 

  • Beaver, D. de B., Rosen, R. (1978). Studies in scientific collaboration, 1: Professional origins of scientific co-authorship. Scientometrics, 1, 65–84.

    Article  Google Scholar 

  • Björneborn, L., Ingwersen, P. (2001). Perspectives of webometrics. Scientometrics, 50, 65–82.

    Article  Google Scholar 

  • Borgman, C.L. (ed.) (1990). Scholarly Communication and Bibliometrics. Newbury Park: Sage.

    Google Scholar 

  • Braam, R.R., Moed, H.F., van Raan, A.F.J. (1991a). Mapping of science by combined cocitation and word analysis, I: Structural Aspects. Journal of the American Society for Information Science (JASIS), 42, 233–251, and, II: Dynamical Aspects. Journal of the American Society for Information Science (JASIS), 42, 252–266.

    Google Scholar 

  • Braun, T., Glänzel, W., Schubert, A. (1988). World flash on basic research — The newest version of the facts and figures on publication output and relative citation impact of 100 countries 1981–1985. Scientometrics, 13, 181–188.

    Google Scholar 

  • Braun, T., Glänzel, Grupp, H. (1995). The scientometric weight of 50 nations in 27 science areas, 1989–1993. 1: All fields combined, mathematics, engineering, chemistry and physics. Scientometrics, 33, 263–293; and 2: Life sciences. Scientometrics, 34, 207–237.

    Google Scholar 

  • Brooks, T.A. (1986). Evidence of complex citer motivations. Journal of the American Society for Information Science, 37, 34–36.

    Google Scholar 

  • Butler, L. (2003). Modifying publication practices in response to funding formulas. Research Evaluation, 17, 39–46.

    Google Scholar 

  • Callon, M., Bauin, S., Courtial, J.P., Turner, W. (1983). From translation to problematic networks: an introduction to co-word analysis. Social Science Information, 22, 191–235.

    Google Scholar 

  • Cole, S., Cole, J.R., Dietrich, L. (1978). Measuring the cognitive state of scientific disciplines. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op. cit..

    Google Scholar 

  • de Candolle, A. (1873, 2nd. edition 1885). Histoire des sciences et des savants depuis deux siècles. Genève/Basel: H.Georg. Reprint in 1987 by Fayard.

    Google Scholar 

  • Egghe L., Rousseau, R. (2000). The influence of publication delays on the observed aging distribution of scientific literature. Journal of the American Society for Information Science, 51, 158–165.

    Google Scholar 

  • Elkana, Y., Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.) (1978). Toward a metric of science: the advent of science indicators. New York: John Wiley.

    Google Scholar 

  • Etzkowitz, H., Leydesdorff, L. (2000). The dynamics of innovation: from National Systems and “Mode 2” to a Triple Helix of university-industry-government relations. Research Policy, 29, 109–123.

    Google Scholar 

  • Garfield, E. (1979). Is citation analysis a legitimate evaluation tool? Scientometrics, 1, 359–375.

    Article  Google Scholar 

  • Garfield, E. (1980). Premature discovery or delayed recognition — Why? Current Contents, 21, May 26, 5–10.

    Google Scholar 

  • Gilbert, G.N. (1978). Measuring the growth of science-review of indicators of scientific growth. Scientometrics, 1, 9–34.

    Article  Google Scholar 

  • Glänzel, W. (1996). A bibliometric approach to social sciences, national research performances in 6 selected social science areas, 1990–1992. Scientometrics, 35, 291–307.

    Google Scholar 

  • Glänzel, W. (2001). National characteristics in international scientific co-authorship relations. Scientometrics, 51, 69–115.

    Google Scholar 

  • Glänzel, W., Meyer, M. (2003). Patents cited in the scientific literature: An exploratory study of ‘reverse’ citation relations. Scientometrics, 58, 415–428.

    Google Scholar 

  • Glänzel, W., Schlemmer, B., Thijs, B. (2003). Better late than never? On the chance to become highly cited only beyond the standard bibliometric time horizon. Scientometrics, 58, 571–586

    Google Scholar 

  • Glänzel, W., Debackere, K. (2003). On the opportunities and limitations in using bibliometric indicators in a policy relevant context. In: Bibliometric analysis in science and research. Applications, Benefits and Limitations. Second Conference of the Central Library, Forschungszentrum Jülich, (pp. 225–236). (ISBN 3-89336-334-3).

    Google Scholar 

  • Gläser, J., Laudel, G. (2001). Integrating scientometric indicators into sociological studies: methodical and methodological problems. Scientometrics, 52, 411–434.

    Article  Google Scholar 

  • Grupp, H., Schmoch, U., Hinze, S. (2001). International alignment and scientific regard as macro-indicators for international comparisons of publications. Scientometrics, 51, 359–380.

    Article  Google Scholar 

  • Haitun, S.D. (1982). Stationary scientometric distributions. 1: Different approximations. Scientometrics, 4, 89–104.

    Google Scholar 

  • Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44, 193–215.

    Google Scholar 

  • Holton, G. (1978). Can science be measured? In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H. (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op. cit.

    Google Scholar 

  • Horrobin, D.F. (1990). The philosophical basis of peer review and the suppression of innovation. Journal of the American Medical Association (JAMA), 263, 1438–1441.

    Google Scholar 

  • Kamerlingh Onnes, H. (1882). De betekenis van kwantitatief onderzoek in de natuurkunde (The meaning of quantitative research in physics). Inaugural Address as Professor of Physics, Leiden University.

    Google Scholar 

  • Koenig, M.E.D. (1983). Bibliometric indicators versus expert opinion in assessing research performance. Journal of the American Society for Information Science, 34, 136–145.

    Google Scholar 

  • Kostoff, R.N. (1995). Federal research impact assessment — Axioms, approaches, applications. Scientometrics, 34, 163–206.

    Article  Google Scholar 

  • van Leeuwen, Th.N., Moed, H.F., Tijssen, R.J.W., Visser, M.S., van Raan, A.F.J. (2001). Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance. Scientometrics, 51, 335–346.

    Google Scholar 

  • van Leeuwen, Th.N. (2004). Second generation bibliometric analysis. Ph.D. Thesis Leiden University.

    Google Scholar 

  • Lewison, G. (2001). The quantity and quality of female researchers: a bibliometric study of Iceland. Scientometrics, 52, 29–43.

    Article  Google Scholar 

  • Lewison, G. (2002). Researchers’ and users’ perceptions of the relative standing of biomedical papers in different journals. Scientometrics, 53, 229–240.

    Article  Google Scholar 

  • Lotka, A.J. (1926). The frequency distribution of scientific productivity. J. Washington Acad. Sci., 16, 317–323.

    Google Scholar 

  • MacRoberts, M.H., MacRoberts, B.R. (1996). Problems of citation analysis. Scientometrics, 36, 435–444.

    Article  Google Scholar 

  • MacRoberts, M.H., MacRoberts, B.R. (1988). Author motivation for not giving citing influences — A methodological note. Journal of the American Society for Information Science, 39, 432–433.

    Article  Google Scholar 

  • Martin, B.R., Irvine, J. (1983). Assessing basic research: some partial indicators of scientific progress in radio astronomy. Research Policy, 12, 61–90.

    Article  Google Scholar 

  • May, R.M. (1997). The scientific wealth of nations. Science, 275, 793–796.

    Article  Google Scholar 

  • McCain, K.W. (1984). Longitudinal author cocitation mapping — The changing structure of macroeconomics. Journal of the American Society for Information Science, 35, 351–359.

    Google Scholar 

  • McCain, K.W. (1990). Mapping authors in intellectual space — A technical overview. Journal of the American Society for Information Science, 41, 433–443.

    Article  Google Scholar 

  • Melin, G., Persson, O. (1996). Studying research collaboration using co-authorships. Scientometrics, 36, 363–377.

    Article  Google Scholar 

  • Moed, H.F., van Leeuwen, Th.N. (1995). Improving the accuracy of the Institute for Scientific Information’s Journal Impact Factors. J. of the American Society for Information Science (JASIS), 46, 461–467.

    Google Scholar 

  • Moed, H.F., van Leeuwen, Th.N. (1996). Impact factors can mislead. Nature, 381, 186.

    Article  Google Scholar 

  • Moed, H.F., Luwel, M., Nederhof, A.J. (2002). Towards research performance measurement in the humanities. Library Trends, 50, 498–520.

    Google Scholar 

  • Moravcsik, M.J. (1975). Phenomenology and models of growth of science. Research Policy, 4, 80–86.

    Article  Google Scholar 

  • Moravcsik, M.J., Murugesan, P. (1979). Citation patterns in scientific revolutions. Scientometrics, 1, 161–169.

    Article  Google Scholar 

  • Moxham, H., Anderson, J. (1992). Peer review. A view from the inside. Science and Technology Policy, February 1992, 7–15.

    Google Scholar 

  • Narin, F. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity. Washington D.C.: National Science Foundation.

    Google Scholar 

  • Narin, F. (1978). Objectivity versus relevance in studies of scientific advance. Scientometrics, 1, 35–41.

    Article  Google Scholar 

  • Narin, F. (1994). Patent bibliometrics. Scientometrics, 30, 147–155.

    Article  Google Scholar 

  • Narin, F., Hamilton, K.S., Olivastro, D. (1997). The increasing linkage between US technology and public science. Research Policy, 26, 317–330.

    Article  Google Scholar 

  • National Science Board (1973). Science Indicators 1972. Washington DC: Government Printing Office.

    Google Scholar 

  • Nederhof, A.J. (1988). The validity and reliability of evaluation of scholarly performance. In: A.F.J. van Raan (ed). (1988), Handbook of Quantitative Studies of Science and Technology (pp. 193–228). Amsterdam: Elsevier/North-Holland, (ISBN 0-444-70537-6).

    Google Scholar 

  • Noma, E. (1982). An improved method for analysing square scientometric transaction matrices. Scientometrics, 4, 297–316.

    Google Scholar 

  • Noyons, E.C.M., van Raan, A.F.J. (1998). Monitoring scientific developments from a dynamic perspective: self-organized structuring to map neural network research. J. of the American Society for Information Science and Technology (JASIST), 49, 68–81.

    Google Scholar 

  • Noyons, E.C.M., Luwel, M., Moed, H.F. (1999). Combining mapping and citation analysis for evaluative bibliometric purpose. A bibliometric study on recent development in microelectronics. Journal of the American Society for Information Science and Technology (JASIST), 50, 115–131.

    Google Scholar 

  • Noyons, E.C.M. (1999). Bibliometric mapping as a science policy and research management tool. Ph.D. Thesis Leiden University. Leiden: DSWO Press (ISBN 90-6695-152-4).

    Google Scholar 

  • Noyons, E.C.M., Buter, R.K., van Raan, A.F.J., Schmoch, U., Heinze, T., Hinze, S., Rangnow, R. (2003). Mapping excellence in science and technology across Europe (Part 1: Life sciences, Part 2: Nanoscience and nanotechnology). Report to the European Commission by the Centre for Science and Technology Studies (CWTS), Leiden University, and the Fraunhofer Institute for Systems and Innovation Research (Fraunhofer-ISI), Karlsruhe.

    Google Scholar 

  • Noyons, E.C.M. (2004). Science Maps within in a Science Policy Context. This Handbook.

    Google Scholar 

  • OECD (1963). The measurement of scientific and technological activities, ‘Frascati Manual’, Paris: Organization for Economic Co-operation and Development (OECD).

    Google Scholar 

  • Peritz, B.C. (1983). A classification of citation roles for the social sciences and related fields. Scientometrics, 5, 303–312.

    Google Scholar 

  • Porter, A.L., Chubin. D.E. (1985). An indicator of cross-disciplinary research. Scientometrics, 8, 161–176.

    Article  Google Scholar 

  • De Solla Price, D.J. (1978). Toward a model for Science Indicators. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H., (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op.cit.

    Google Scholar 

  • De Solla Price, D.J. (1981). The analysis of scientometric matrices for policy implications. Scientometrics, 3, 47–53.

    Google Scholar 

  • Prime, C., Bassecoulard, E., Zitt, M. (2002). Co-citations and co-sitations: A cautionary view on an analogy. Scientometrics, 54, 291–308.

    Article  Google Scholar 

  • Prpić, K. (2002). Gender and productivity differentials in science. Scientometrics, 55, 27–58.

    Google Scholar 

  • van Raan, A.F.J. (ed). (1988). Handbook of Quantitative Studies of Science and Technology. Amsterdam: Elsevier/North-Holland (ISBN 0-444-70537-6).

    Google Scholar 

  • van Raan, A.F.J. (1990). Fractal dimension of co-citations. Nature, 347, 626.

    Google Scholar 

  • van Raan, A.F.J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36, 397–420.

    Article  Google Scholar 

  • van Raan, A.F.J. (1997). Scientometrics: State-of-the-Art. Scientometrics, 38, 205–218.

    Google Scholar 

  • van Raan, A.F.J. (1998). In matters of quantitative studies of science the fault of theorists is offering too little and asking too much. Scientometrics, 43, 129–139.

    Google Scholar 

  • van Raan, A.F.J. (2000a). The Pandora’s box of citation analysis: measuring scientific excellence, the last evil? In: B. Cronin and H. Barsky Atkins (eds.). The Web of Knowledge. A Festschrift in honor of Eugene Garfield. Ch. 15, p. 301–319. Medford (New Jersey): ASIS Monograph Series, 2000 (ISBN 1-57387-099-4).

    Google Scholar 

  • van Raan, A.F.J. (2000b). On growth, ageing, and fractal differentiation of science. Scientometrics 47, 347–362.

    Google Scholar 

  • van Raan, A.F.J. (2001). Two-step competition process leads to quasi power-law income distributions. Application to scientific publication and citation distributions. Physica A, 298, 530–536.

    Google Scholar 

  • van Raan, A.F.J., Noyons, E.C.M. (2002). Discovery of patterns of scientific and technological development and knowledge transfer. In W. Adamczak, A. Nase (Eds.), Gaining Insight from Research Information. Proceedings of the 6th International Conference on Current Research Information Systems, University of Kassel, August 29–31, 2002 (pp. 105–112). Kassel: University Press, (ISBN 3-933146-844).

    Google Scholar 

  • van Raan, A.F.J., van Leeuwen, Th.N. (2002). Assessment of the scientific basis of interdisciplinary, applied research. Application of bibliometric methods in nutrition and food research. Research Policy, 31, 611–632

    Google Scholar 

  • van Raan, A.F.J. (2003). Reference-based publication networks with episodic memories. E-print ArXiv cond-mat/0311318.

    Google Scholar 

  • van Raan, A.F.J. (2004). Sleeping Beauties in Science. Scientometrics, 59, 461–466.

    Google Scholar 

  • van Raan, A.F.J., van Leeuwen, Th.N. (2004). Statistical aspects of research group performance, journal impact, and peer judgement. To be published.

    Google Scholar 

  • Rinia, E.J., van Leeuwen, Th.N., van Vuren, H.G., van Raan, A.F.J. (1998). Comparative analysis of a set of bibliometric indicators and central peer review criteria. Evaluation of condensed matter physics in the Netherlands. Research Policy, 27, 95–107.

    Article  Google Scholar 

  • Rinia, E.J., van Leeuwen, Th.N., van Vuren, H.G., van Raan, A.F.J. (2001). Influence of interdisciplinarity on peer-review and bibliometric evaluations. Research Policy, 30, 357–361.

    Article  Google Scholar 

  • Rip, A., Courtial, J.P. (1984). Co-word maps of biotechnology — An example of cognitive scientometrics. Scientometrics, 6, 381–400.

    Article  Google Scholar 

  • Schmoch, U. (1993). Tracing the knowledge transfer from science to technology as reflected in patent indicators. Scientometrics, 26, 193–211.

    Article  Google Scholar 

  • Schwechheimer, H., Winterhager, M. (2001). Mapping interdisciplinary research fronts in neuroscience: a bibliometric view to retrograde amnesia. Scientometrics, 51, 311–318.

    Article  Google Scholar 

  • Schubert A., Glänzel, W. (1983). Statistical reliability of comparisons based on the citation impact of scientometric publications. Scientometrics, 5, 59–74.

    Google Scholar 

  • Seglen, P.O. (1992). The skewness of science. Journal of the American Society for Information Science, 43, 628–638.

    Article  Google Scholar 

  • Seglen, P.O. (1994). Causal relationship between article citedness and journal impact. Journal of the American Society for Information Science, 45, 1–11.

    Article  Google Scholar 

  • Small, H. (1973). Co-citation in the Scientific Literature: A New Measure of the Relationship Between Publications. Journal of the American Society for Information Science, 24, 265–269.

    Google Scholar 

  • Small, H., Greenlee, E. (1980). Citation context analysis of a co-citation cluster-recombinant DNA. Scientometrics, 2, 1980.

    Article  Google Scholar 

  • Small, H., Sweeney, E. (1985). Clustering the Science Citation Index using co-citations, I: A Comparison of Methods. Scientometrics, 7, 393–404.

    Article  Google Scholar 

  • Small, H., Sweeney, E., Greenlee, E. (1985). Clustering the Science Citation Index using cocitations, II: Mapping Science. Scientometrics, 8, 321–340.

    Article  Google Scholar 

  • Small, H. (1999). Visualizing science by citation mapping. Journal of the American Society for Information Science, 50, 799–813.

    Article  Google Scholar 

  • Swanson, D.R. (1986). Fish oil, Raynaud’s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30, 7–18.

    Google Scholar 

  • Swanson, D.R. (1987). Two medical literatures that are logically but not bibliographically connected. Journal of the American Society for Information Science, 38, 228–233.

    Article  Google Scholar 

  • Sullivan D., Koester, D., White, D.H., Kern, R. (1980). Understanding rapid theoretical change in particle physics — a month-by-month co-citation analysis. Scientometrics, 2, 309–319.

    Article  Google Scholar 

  • Thelwall, M., Smith, A. (2002). Interlinking between Asia-Pacific University Web sites. Scientometrics, 55, 363–376.

    Article  Google Scholar 

  • Thelwall, M., Harries, G. (2003). The connection between the research of a university and counts of links to its web pages: An investigation based upon a classification of the relationships of pages to the research of the host university. Journal of the American Society for Information Science, 54, 594–602.

    Google Scholar 

  • Vinkler, P. (1993). Research contribution, authorship and team cooperativeness. Scientometrics 26, 213–230.

    Article  Google Scholar 

  • Vinkler, P. (1998). Comparative investigation of frequency and strength of motives toward referencing, the reference threshold model-comments on theories of citation? Scientometrics, 43, 107–127.

    Google Scholar 

  • Vlàchy, J. (1979). Mobility in science. Bibliography of scientific career migration, field mobility, international academic circulation and brain drain. Scientometrics, 1, 201–228.

    Google Scholar 

  • Weingart, P. (2003). Evaluation of research performance: the danger if numbers. In: Bibliometric analysis in science and research. Applications, Benefits and Limitations. Second Conference of the Central Library, Forschungszentrum Jülich (pp. 7–19). (ISBN 3-89336-334-3).

    Google Scholar 

  • White, H.D., Griffith, B.C. (1981). Author cocitation— a literature measure of intellectual structure. Journal of the American Society for Information Science, 32, 163–171.

    Google Scholar 

  • White, H.D., McCain, K.W. (1998). Visualizing a discipline: An author co-citation analysis of information science, 1972—1995. Journal of the American Society for Information Science, 49, 327–355.

    Google Scholar 

  • Wouters, P.F. (1999), The Citation Culture, PhD thesis, University of Amsterdam.

    Google Scholar 

  • Ziman, J. (1978). From Parameters to Portents-and Back. In: Lederberg, J., Merton, R.K., Thackray, A., Zuckerman, H., (Eds.). Toward a metric of science: the advent of science indicators. New York: John Wiley Elkana et al., op.cit.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Science and Technology Studies (CWTS), Leiden University, the Netherlands

    Anthony F.J. van Raan

Authors
  1. Anthony F.J. van Raan
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Centre for Science and Technology Studies, University of Leiden, The Netherlands

    Henk F. Moed

  2. Steunpunt O&O Statistieken, K.U. Leuven, Belgium

    Wolfgang Glänzel

  3. Fraunhofer Institute for Systems and Innovation Research, Karlsruhe, Germany

    Ulrich Schmoch

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Kluwer Academic Publishers

About this chapter

Cite this chapter

van Raan, A.F. (2004). Measuring Science. In: Moed, H.F., Glänzel, W., Schmoch, U. (eds) Handbook of Quantitative Science and Technology Research. Springer, Dordrecht. https://doi.org/10.1007/1-4020-2755-9_2

Download citation

Publish with us

Policies and ethics

Search

Navigation