Abstract
Big Science accelerator complexes are no longer mere tools for nuclear and particle physics, but modern-day experimental resources for a wide range of natural sciences and often named instrumental to scientific and technological development for innovation and economic growth. Facilities compete on a global market to attract the best users and facilitate the best science, and advertise the achievement of their users as markers of quality and productivity. Thus a need has risen for (quantitative) quality assessment of science on the level of facilities. In this article, we examine some quantitative performance measurements frequently used by facilities to display quality: technical reliability, competition for access, and publication records. We report data from the world’s three largest synchrotron radiation facilities from the years 2004–2010, and discuss their meaning and significance by placing them in proper context. While we argue that quality is not possible to completely capture in these quantitative metrics, we acknowledge their apparent importance and, hence, we introduce and propose facilitymetrics as a new feature of the study of modern big science, and as a new empirical focus for scientometrical study, in the hope that future studies can contribute to a deeper, much-needed analysis of the topic.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Annual reports are found at: http://www.esrf.eu/UsersAndScience/Publications/Highlights (ESRF), http://www.aps.anl.gov/Science/Reports/ (APS), and http://www.spring8.or.jp/en/news_publications/publications/research_frontiers/ (Spring-8). Publication databases are found at http://vmis2.esrf.fr:9090/flora_illesrf/servlet/LoginServlet (ESRF), https://beam.aps.anl.gov/pls/apsweb/pub_v2_0006.review_start_page (APS), https://user.spring8.or.jp/uisearch/publication (Spring-8). Some data for the APS is unavailable in the APS annual reports and was retrieved by personal communication.
The most common utilizations are in solid state and condensed matter physics, chemistry and other materials-related sciences, whereby various spectroscopic methods are used; biology, biochemistry, medicine and other life sciences, using the radiation for crystallography and other diffraction studies. There are also a number of smaller areas of utility in environmental sciences, cultural studies and archaeometry (Hallonsten 2009, 91–96).
For example, it is commonly claimed that the continuity of various technological innovations have made possible a doubling of the peak intensity (highest achievable intensity) of the radiation every 24 months (!) (Frahm and Williams 2007).
Generally, proposals are judged only by their scientific quality and technical feasibility, but exceptions exist, for example the favoring of inexperienced applicants in order to make user communities less static, and attempts to achieve better gender equality (Hallonsten 2009, 187).
The technical performance parameters shown in Table 1 (energy, current, and emittance) are very straightforward measures of accelerator performance, i.e. a greater value normally means a better accelerator or a more capable synchrotron radiation facility. There are several other parameters determining accelerator performance, but these are severely more complicated to assess (Hallonsten 2009, 80–83).
Normally, it is required of users that they report publications based on previous beamtime to facilities in order to be awarded new time on their proposals.
For example, the European Organization for Nuclear Physics (CERN), the Joint Research Centre (JRC) for nuclear physics, the European Southern Observatory (ESO) and the European Space Research Organization (ESRO) (Krige 2003, 899).
This agreement should be properly placed in the broader context of the early-1980s renewed Franco-German collaboration on European level that produced the Framework Programmes, the Single European Act of 1985, and eventually, the Maastricht Treaty and the Euro currency (Hallonsten 2012, 304–305).
The shares were, eventually, distributed as follows: France (33 % of the construction costs; 27.5 % of the operations costs), the Federal Republic of Germany (23, 25.5 %); Italy (14, 15 %); United Kingdom (12, 14 %), the Benesync consortium comprising of Belgium and the Netherlands (6, 6 %); Spain (4, 4 %), the Nordsync consortium comprising of Denmark, Finland, Norway and Sweden (4, 4 %); and Switzerland (4, 4 %). The French contribution to the construction budget also included the site in Grenoble, ready to build on, free of charge (Hallonsten 2009, 218).
This is calculated by dividing proposal request with actual allocation. Exact measures vary somewhat (see Table 2); for the ESRF and APS the diagram shows an average of two measures (time requested/allocated, and number of proposals submitted/accepted). For SPring-8, only one measure is available, namely number of proposals submitted/accepted, and this value is hence represented directly in the diagram.
This explanation for the apparent performance lead of ESRF is reported by Hallonsten (2009, 234–235) in his comprehensive study of the facility. Though only representing the ESRF perspective on the matter, the study displays what seems to be a crucial difference between the APS and the ESRF in organizational structure, that appears to have an impact on scientific performance. While the ESRF management has always had a deliberate strategy to facilitate high-quality work by external users, a strategy sustained by the generous budget of the lab regulated in the founding documents, the APS was built up with heavy reliance on buy-in arrangements under the name Collaborating Access Teams (CATs). These arrangements secured vital scientific and technical competences in building up the lab but also incorporated organizational compartmentalization and insufficient coordination between units, which eventually led to ineffectiveness in technical maintenance and user operation (Hallonsten 2009, 130), a matter also recognized in a 1997 national review of US synchrotron radiation facilities (Birgeneau and Shen 1997, 101).
In fact, the clear lead of ESRF among the three is something that ESRF management takes every opportunity to advertise. The then-ESRF director-general is quoted in Hallonsten (2009), 234; saying: “for every two publications produced by the APS, the ESRF community produces three. We produce between two and three times as many publications as SPring-8. The budgets are broadly comparable. […] These publications, our different libraries have looked at them together, so we all agree that they correspond to the same thing. It’s not measuring elephants on one side and zebras on the other. So as far as we can tell, that is a fair thing”.
Cf. the concept of cumulative advantage proposed by Merton (1968) for the accumulation of credit in science.
References
Birgeneau, B., & Shen, Z.X. (1997). Report of the Basic Energy Sciences Advisory Committee Panel on D.O.E. synchrotron radiation sources and science. Washington, DC, US Department of Energy’s Office of Science’s Office of Basic Energy Science.
Doing, P. (2009). Velvet revolution at the synchrotron: biology, physics, and change in science. Cambridge: MIT Press.
Elzinga, A. (2012). Features of the current science policy regime: viewed in historical perspective. Science and Public Policy, 39(4), 416–428.
Frahm, R., & Williams, G. (2007). Twenty years of synchrotron radiation. Synchrotron Radiation News, 20(1), 2–3.
Hallonsten, O. (2009). Small science on big machines: politics and practices of synchrotron radiation laboratories. Dissertation, Lund University, Lund.
Hallonsten, O. (2012). Continuity and change in the politics of European scientific collaboration. Journal of Contemporary European Research, 8(3), 300–319.
Hallonsten, O., & Heinze, T. (2012). Institutional persistence through gradual adaptation: analysis of national laboratories in the USA and Germany. Science and Public Policy, 39(4), 450–463.
Holl, J. M. (1997). Argonne National Laboratory 1946–96. Chicago: University of Illinois Press.
Jacob, M., & Hallonsten, O. (2012). The persistence of big science and megascience in research and innovation policy. Science and Public Policy, 39(4), 411–415.
Krige, J. (2003). The Politics of European Scientific Collaboration. In J. Krige & D. Pestre (Eds.), Companion to Science in the Twentieth Century (pp. 897–918). London: Routledge.
Merton, R. K. (1968). The Matthew effect in science. Science, 159(3180), 56–63.
NUFO. (2009). Participation by industrial users in research at national user facilities: status, issues, and recommendations. Preliminary report prepared as a summary of a 11 June 2009, industrial usage workshop conducted under the auspices of the National User Facility Organization (NUFO) at Argonne National Laboratory, 3 Aug 2009.
Papon, P. (2004). European scientific cooperation and research infrastructures: past tendencies and future prospects. Minerva, 42(1), 61–76.
Sasaki, T. (1997). A prospect and retrospect—the Japanese case. Journal of Synchrotron Radiation, 4, 359–365.
Spring-8. (2012). History. SPring-8. http://www.spring8.or.jp/en/about_us/history/. Accessed 24 Sept 2012.
Westfall, C. (2008). Retooling for the future: launching the advanced light source at Lawrence’s Laboratory, 1980–1986. Historical Studies in the Natural Sciences, 38(4), 569–609.
Westfall, C. (2010). Surviving to tell the tale: Argonne’s intense pulsed neutron source from an ecosystem perspective. Historical Studies in the Natural Sciences, 40(3), 350–398.
Westfall, C. (2012). Institutional persistence and the material transformation of the US National Labs: the curious story of the advent of the advanced photon source. Science and Public Policy, 39(4), 439–449.
Acknowledgments
The author is grateful to Rick Fenner and Susan Strasser at the Advanced Photon Source, Argonne National Laboratory, for their assistance in retrieving some data unavailable in Annual Reports. The author also would like to thank Gustav Holmberg for the recurring fruitful discussions that eventually led to this article.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hallonsten, O. Introducing ‘facilitymetrics’: a first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide. Scientometrics 96, 497–513 (2013). https://doi.org/10.1007/s11192-012-0945-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-012-0945-9