Abstract
The ability to repeat experiments from a research study and obtain similar results is a corner stone in experiment-based scientific discovery. This essential feature has often been overlooked by the distributed computing and networking community. There are many reasons for that, such as the complexity of provisioning, configuring, and orchestrating the resources used by experiments, their multiple external dependencies, or the difficulty to seamlessly record these dependencies. This chapter describes a methodology based on well-established principles to plan, prepare and execute reproducible experiments. We propose and describe a family of tools, the LabWiki workspace, to support an experimenter’s workflow based on that methodology. This proposed workspace provides services and mechanisms for each step of an experiment-based study, while automatically capturing the necessary information to allow others to repeat, inspect, validate and modify prior experiments. Our LabWiki workspace builds on existing contributions, de-facto protocols, and model standards, which emerged from recent experimental facility initiatives. We use a real experiment as a thread to guide and illustrate the discussion throughout this chapter.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
Base SI units should be preferred whenever possible.
- 15.
Manpages for OML system components can be found at http://oml.mytestbed.net/doc.
- 16.
- 17.
We generally use APPNAME-oml2 as the binary’s name of OML-instrumented versions of upstream APPNAME utilities; the OEDL application description however only uses APPNAME for conciseness.
- 18.
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
References
Berman, M., et al.: GENI: a federated testbed for innovative network experiments. Comput. Netw. 61 (2014). Special issue on Future Internet Testbeds—Part I, pp. 5–23. ISSN: 1389–1286. doi:10.1016/j.bjp.2013.12.037
Chacon, S.: Pro Git, 1st edn. Apress, Berkely (2009). ISBN: 1430218339
Fisher, R.A.: The Design of Experiments. Hafner Publishing Company, New York (1937)
Gabriel, A., Capone, R.: Executable paper grand challenge workshop. Procedia Comput. Sci. 4 (2011). Proceedings of the International Conference on Computational Science, ICCS 2011, pp. 577–578. ISSN: 1877–0509. doi:10.1016/j.procs.2011.04.060
Gavras, A., et al.: Future internet research and experimentation: the FIRE initiative. SIGCOMM Comput. Commun. Rev. 37 (3), 89–92 (2007). ISSN: 0146–4833. doi:10.1145/1273445.1273460
Jourjon, G., Rakotoarivelo, T., Ott, M.: From learning to researcher, ease the shift through testbed, ser. LNICST, Berlin Heidelberg: Springer-Verlag 46, 496–505 (2010)
Kavoussanakis, K., et al.: BonFIRE: the clouds and services testbed. In: 2013 IEEE 5th International Conference on Cloud Computing Technology and Science (CloudCom), vol. 2, pp. 321–326 (2013). doi:10.1109/CloudCom.2013.156
Kim, Y.-H. et al.: Enabling iterative development and reproducible evaluation of network protocols. Comput. Netw. 63, 238–250 (2014)
Krishnamurthy, B., Willinger, W., Gill, P., Arlitt, M.: A socratic method for validation of measurement based networking research. Comput. Commun. 34, 43–53 (2011)
Langer, M.: iBooks Author: Publishing Your First eBook. Flying M Production, USA (2012)
Medina, A., Lakhina, A., Matta, I., Byers, J.: BRITE: an approach to universal topology generation. In: Proceedings of the International Workshop on Modeling, Analysis and Simulation of Computer and Telecommunications Systems, MASCOTS’01 (2001)
Mehani, O., Jourjon, G., Rakotoarivelo, T., Ott, M.: An instrumentation framework for the critical task of measurement collection in the future internet. Comput. Netw. 63 (2014). ISSN: 1389–1286. doi:10.1016/j.bjp.2014.01.007
Mehani, O., Taib, R., Itzstein, B.: Time calibration in experiments with networked sensors. In: Proceedings of IEEE of the 39th Local Computer Networks Conference (LCN) (2014). ISBN: 978-1-4799-3780-6/14
Mills, D., Martin, J., Burbank, J., Kasch, W.: Network time protocol version 4: protocol and algorithms specification. RFC 5905 (2010)
Paxson, V.: Strategies for sound internet measurement. In: The Internet Measurement Conference (IMC) (2004)
Rajasekar, A., et al.: iRODS primer: integrated rule-oriented data system. In: Synthesis Lectures on Information Concepts, Retrieval, and Services 2.1. Morgan and Claypool Publishers (2010). doi:10.2200/s00233ed1v01y200912icr012
Rakotoarivelo, T., Ott, M., Jourjon, G., Seskar, I.: OMF: a control and management framework for networking testbeds. SIGOPS Oper. Syst. Rev. 43 (4), 54–59 (2010). ISSN: 0163–5980. doi:10.1145/1713254.1713267
Rakotoarivelo, T., Jourjon, G., Ott, M.: Designing and orchestrating re-producible experiments on federated networking testbeds. Comput. Netw. 63 (2014). doi:http://dx.doi.org/10.1016/j.bjp.2013.12.033
Römer, K., Blum, P., Meier, L.: Time synchronization and calibration in wireless sensor networks. Handbook of Sensor Networks. Wiley, New York (2005). Chap. 7. doi:10.1002/047174414x.ch7
Vandenberghe, W., et al.: Architecture for the heterogeneous federation of future internet experimentation facilities. Future Network and Mobile Summit. (2013). ISBN: 978-1-905824-37-3
Wright, A.: Tablets over textbooks? Commun. ACM 55 (3), 17–17 (2012)
Zhuang, Y., et al.: Taking a walk on the wild side: teaching cloud computing on distributed research testbeds. In: Proceedings of the 45th ACM Technical Symposium on Computer Science Education. SIGCSE ’14. Atlanta, GA, pp. 535–540. ACM, New York (2014). ISBN: 978-1-4503-2605-6. doi:10.1145/2538862.2538931
Acknowledgements
NICTA is funded by the Australian Government through the Department of Communications and the Australian Research Council through the ICT Centre of Excellence Program. This material is based in part upon work supported by the GENI (Global Environment for Network Innovations) initiative under a National Science Foundation grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Rakotoarivelo, T., Jourjon, G., Mehani, O., Ott, M., Zink, M. (2016). A Walk Through the GENI Experiment Cycle. In: McGeer, R., Berman, M., Elliott, C., Ricci, R. (eds) The GENI Book. Springer, Cham. https://doi.org/10.1007/978-3-319-33769-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-33769-2_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-33767-8
Online ISBN: 978-3-319-33769-2
eBook Packages: Computer ScienceComputer Science (R0)