Abstract
Infrastructure-as-a-Service providers are offering their unused resources in the form of variable-priced virtual machines (VMs), known as “spot instances”, at prices significantly lower than their standard fixed-priced resources. To lease spot instances, users specify a maximum price they are willing to pay per hour and VMs will run only when the current price is lower than the user’s bid. This paper proposes a resource allocation policy that addresses the problem of running deadline-constrained compute-intensive jobs on a pool of composed solely of spot instances, while exploiting variations in price and performance to run applications in a fast and economical way. Our policy relies on job runtime estimations to decide what are the best types of VMs to run each job and when jobs should run. Several estimation methods are evaluated and compared, using trace-based simulations, which take real price variation traces obtained from Amazon Web Services as input, as well as an application trace from the Parallel Workload Archive. Results demonstrate the effectiveness of running computational jobs on spot instances, at a fraction (up to 60% lower) of the price that would normally cost on fixed priced resources.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hill, Z., Humphrey, M.: A Quantitative Analysis of High Performance Computing with Amazon’s EC2 Infrastructure: The Death of the Local Cluster? In: Proceedings of the 10th IEEE/ACM International Conference on Grid Computing (October 2009)
de Assuncao, M.D., di Costanzo, A., Buyya, R.: Evaluating the Cost-Benefit of Using Cloud Computing to Extend the Capacity of Clusters. In: Proceedings of the 18th ACM International Symposium on High Performance Distributed Computing. HPDC 2009. ACM, New York (2009)
Yi, S., Kondo, D., Andrzejak, A.: Reducing Costs of Spot Instances via Checkpointing in the Amazon Elastic Compute Cloud. In: 2010 IEEE 3rd International Conference on Cloud Computing, pp. 236–243. IEEE, Los Alamitos (2010)
Mattess, M., Vecchiola, C., Buyya, R.: Managing Peak Loads by Leasing Cloud Infrastructure Services from a Spot Market. In: Proceedings of the 10th IEEE International Conference on High Performance Computing and Communications, pp. 180–188. IEEE Computer Society, Los Alamitos (2010)
Andrzejak, A., Kondo, D., Yi, S.: Decision Model for Cloud Computing under SLA Constraints. Technical report, INRIA (2010)
Tsafrir, D.: Using inaccurate estimates accurately. In: Frachtenberg, E., Schwiegelshohn, U. (eds.) JSSPP 2010. LNCS, vol. 6253, pp. 208–221. Springer, Heidelberg (2010)
Downey, A.B.: A Model For Speedup of Parallel Programs. Technical report, Berkeley, CA, USA (1997)
Cirne, W., Berman, F.: A Model for Moldable Supercomputer Jobs. In: Proceedings of the 15th International Parallel and Distributed Processing Symposium. IEEE Computer Society, Los Alamitos (2001)
Buyya, R., Ranjan, R., Calheiros, R.: Modeling and Simulation of Scalable Cloud Computing Environments and the Cloudsim Toolkit: Challenges and Opportunities. In: Proceeding of the International Conference on High Performance Computing & Simulation, HPCS 2009, pp. 1–11. IEEE, Los Alamitos (2009)
Feitelson, D.: Parallel workloads archive, http://www.cs.huji.ac.il/labs/parallel/workload
Tsafrir, D., Etsion, Y., Feitelson, D.G.: Modeling User Runtime Estimates. In: Feitelson, D.G., Frachtenberg, E., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 1–35. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Voorsluys, W., Garg, S.K., Buyya, R. (2011). Provisioning Spot Market Cloud Resources to Create Cost-Effective Virtual Clusters. In: Xiang, Y., Cuzzocrea, A., Hobbs, M., Zhou, W. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2011. Lecture Notes in Computer Science, vol 7016. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24650-0_34
Download citation
DOI: https://doi.org/10.1007/978-3-642-24650-0_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24649-4
Online ISBN: 978-3-642-24650-0
eBook Packages: Computer ScienceComputer Science (R0)