Abstract
Cloud storage is the typical way for storing massive data in Big Data Era. Dynamic data balancing is important for cloud storage since it aims to improve the utilization of computing resource and the performance of data process. However, storage-load-aware data balancing, adopted by almost all existing cloud storage services and systems, is far less effective than access-load-aware one for typical cloud applications with hotspots of data. This paper focuses on the latter and puts forward a mechanism of dynamic data balancing for optimization of resource utilization. The mechanism detects the overloaded and underloaded physical nodes and virtual nodes by monitoring their utilization of resource. Then, it dynamically balances the access load among the nodes by pair, merge, mark, scale up and scale down operations. This mechanism is useful for the applications with hotspots in data. So it is a complementation of storage-load-aware data balancing. The results of experiments on Swift demonstrated the effectiveness of this mechanism.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Structured vs. Unstructured Data, http://www.robertprimmer.com/blog/structured-vs-unstructured.html
Amazon, Amazon S3, http://aws.amazon.com/s3
Google Cloud Storage, http://www.google.com/enterprise/cloud
Cloud Files, Cloud CDN, and Unlimited Online Storage, http://www.rackspace.com/cloud/public/files/
Openstack, http://www.openstack.org
Eucalyptus, http://www.eucalyptus.com
Nimbus, http://www.nimbusproject.org
DeCanadia, G., Hastorun, D., Jampani, M., et al.: Dynamo: Amazon’s Highly Available Key-value Store. In: 21st ACM SIGOPS Symposium on Operating Systems Principles, pp. 205–220. ACM Press, New York (2007)
Chang, F., Dean, J., Ghemawat, S., et al.: Bigtable: A Distributed Storage System for Structured Data. J. ACM Transaction on Comput. Syst. 26, 1–26 (2008)
MongoDB, http://www.mongodb.org/
Ghemawat, S., Gobioff, H., Leung, S.: The Google File System. In: 19th ACM SIGOPS Symposium on Operating Systems Principles, pp. 29–43. ACM Press, New York (2003)
Deng, Y., Lau, R.: Heat Diffusion Based Dynamic Load Balancing for Distributed Virtual Environments. In: 17th ACM Symposium on Virtual Reality Software and Technology, pp. 203–210. ACM Press, New York (2010)
Liu, Y., Wan, Y., Jin, Y.: Research on The Improvement of MongoDB Auto-Sharding in Cloud Environment. In: 7th International Conference on Computer Science & Education, Melbourne, VIC, Australia, pp. 851–854 (2012)
Pearce, O., Gambliny, T., Supinskiy, B., et al.: Quantifying the Effectiveness of Load Balance Algorithms. In: 26th ACM International Conference on Supercomputing, pp. 185–194. ACM Press, New York (2012)
Zhu, Y., Yu, Y., Wang, W., et al.: A Balanced Allocation Strategy for File Assignment in Parallel I/O Systems. In: 5th IEEE International Conference on Networking, Architecture and Storage, pp. 257–266. IEEE Press, New York (2010)
Bui, T.N., Deng, X., Zrncic, C.M.: An Improved Ant-Based Algorithm for the Degree-Constrained Minimum Spanning Tree Problem. J. IEEE Transactions on Evolutionary Computation 16, 266–278 (2012)
Lim, H.C., Babu, S., Chase, J.S.: Automated control for elastic storage. In: 7th International Conference on Autonomic Computing, Washington, DC, USA, pp. 1–10 (2010)
Qin, X., Zhang, W., Wang, W., et al.: Towards a Cost-Aware Data Migration Approach for Key-Value Stores. In: 2012 IEEE International Conference on Cluster Computing, pp. 551–556. IEEE Press, New York (2012)
Liu, Z., Lin, M., Wierman, A., et al.: Greening Geographical Load Balancing. In: Liu, Z., Lin, M., Wierman, A., et al. (eds.) 2011 ACM SIGMETRICS Joint International Conference on Measurement and Modeling of Computer Systems, pp. 233–244. ACM Press, New York (2011)
Lin, M., Wierman, A., Andrew, L.L.H., et al.: Dynamic Right-sizing for Power-proportional Data Centers. In: 2011 IEEE INFOCOM, pp. 1098–1106. IEEE Press, New York (2011)
Zhang, C., Chen, H., Gao, S.: ALARM: Autonomic Load-Aware Resource Management for P2P Key-value Stores in Cloud. In: 9th IEEE International Conference on Dependable, Autonomic and Secure Computing, pp. 404–410. IEEE Press, New York (2011)
Ban, Y., Chen, H., Wang, Z.: EALARM: An Enhanced Autonomic Load-Aware Resource Management. In: 7th IEEE International Symposium on Service-Oriented System Engineering, pp. 150–155. IEEE Press, New York (2013)
XenServer, http://www.citrix.com/products/xenserver/resources-and-support.html
Pylot, http://www.pylot.org/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chen, H., Wang, Z., Ban, Y. (2013). Access-Load-Aware Dynamic Data Balancing for Cloud Storage Service. In: Pathan, M., Wei, G., Fortino, G. (eds) Internet and Distributed Computing Systems. IDCS 2013. Lecture Notes in Computer Science, vol 8223. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41428-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-41428-2_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41427-5
Online ISBN: 978-3-642-41428-2
eBook Packages: Computer ScienceComputer Science (R0)