How to Build a Better Testbed: Lessons from a Decade of Network Experiments on Emulab

Abstract

The Emulab network testbed provides an environment in which researchers and educators can evaluate networked systems. Available to the public since 2000, Emulab is used by thousands of experimenters at hundreds of institutions around the world, and the research conducted on it has lead to hundreds of publications. The original Emulab facility at the University of Utah has been replicated at dozens of other sites.

The physical design of the Emulab facility, and many other testbeds like it, has been based on the facility operators’ expectations regarding user needs and behavior. If operators’ assumptions are incorrect, the resulting facility can exhibit inefficient use patterns and sub-optimal resource allocation. Our study, the first of its kind, gains insight into the needs and behaviors of networking researchers by analyzing more than 500,000 topologies from 13,000 experiments submitted to Emulab. Using this dataset, we re-visit the assumptions that went into the physical design of the Emulab facility and consider improvements to it. Through extensive simulations with real workloads, we evaluate alternative testbeds designs for their ability to improve testbed utilization and reduce hardware costs.