Skip to main content
SpringerLink
Log in

Dynamic Communication-Aware Scheduling with Uncertainty of Workflow Applications in Clouds

  • Conference paper
  • First Online:
High Performance Computer Applications (ISUM 2015)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 595))

Included in the following conference series:

  • 951 Accesses

Abstract

Cloud computing has emerged as a new approach to bring computing as a service, in both academia and industry. One of the challenging issues is scientific workflow execution, where the job scheduling problem becomes more complex, especially when communication processes are taken into account. To provide good performance, many algorithms have been designed for distributed environments. However, these algorithms are not adapted to the uncertain and dynamic nature of cloud computing. In this paper, we present a general view on scheduling problems in cloud computing with communication, and compare existed solutions based on three models of cloud applications named CU-DAG, EB-DAG and CA-DAG. We formulate the problem and review several workflow scheduling algorithms. We discuss the main difficulties of using existed application models in the domain of computations on clouds. Finally, we show that our CA-DAG approach, based on separate vertices for computing and communications, and introducing communication awareness, allows us to mitigate uncertainty in a more efficient way.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Robison, S.: HP Shane Robison Executive Viewpoint: The Next Wave: Everything as a Service. http://www.hp.com/hpinfo/execteam/articles/robison. Accessed 30 January 2014

  2. CSC: CSC cloud usage index latest report, Computer Sciences Corporation. http://www.csc.com/au/ds/39454/75790-csc_cloud_usage_index_latest_report. Accessed 20 January 2014

  3. Zhang, Q., Cheng, L., Boutaba, R.: Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1(1), 7–18 (2010)

    Article  Google Scholar 

  4. N. US Department of Commerce, Final Version of NIST Cloud Computing Definition Published. http://www.nist.gov/itl/csd/cloud-102511.cfm. Accessed 20 January 2014

  5. Hollinsworth, D.: The workflow reference model. In: Workflow Management Coalition, vol. TC00–1003 (1995)

    Google Scholar 

  6. Pandey, S., Wu, L., Guru, S.M., Buyya, R.: A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments. In: 2010 24th IEEE International Conference on Advanced Information Networking and Applications (AINA), pp. 400–407 (2010)

    Google Scholar 

  7. Kandula, S., Sengupta, S., Greenberg, A., Patel, P., Chaiken, R.: The nature of data center traffic: measurements & analysis. In: Proceedings of the 9th ACM SIGCOMM Conference on Internet Measurement Conference, New York, NY, USA, pp. 202–208 (2009)

    Google Scholar 

  8. AbdelBaky, M., Parashar, M., Kim, H., Jordan, K.E., Sachdeva, V., Sexton, J., Jamjoom, H., Shae, Z.Y., Pencheva, G., Tavakoli, T., Wheeler, M.F.: Enabling high-performance computing as a service. Computer 45(10), 72–80 (2012)

    Article  Google Scholar 

  9. Tchernykh, A., Schwiegelsohn, U., Alexandrov, V., Talbi, E.: Towards understanding uncertainty in cloud computing resource provisioning. SPU 2015 - solving problems with uncertainties (3rd Workshop). In: Conjunction with the 15th International Conference on Computational Science (ICCS 2015), Reykjavík, Iceland, 1–3 June 2015. Procedia Computer Science, Elsevier, vol. 51, pp. 1772–1781 (2015)

    Google Scholar 

  10. Tychinsky A.: Innovation Management of Companies: Modern Approaches, Algorithms, Experience. Taganrog Institute of Technology, Taganrog (2006). http://www.aup.ru/books/m87/

  11. Kliazovich, D., Pecero, J., Tchernykh, A., Bouvry, P., Khan, S., Zomaya, A.: CA-DAG: modeling communication-aware applications for scheduling in cloud computing. J. Grid Comput., 1–17 (2015). Springer, Netherlands

    Google Scholar 

  12. Tsafrir, D., Etsion, Y., Feitelson, D.G.: Backfilling using system-generated predictions rather than user runtime estimates. IEEE Trans. Parallel Distrib. Syst. 18(6), 789–803 (2007)

    Article  Google Scholar 

  13. Ramírez-Alcaraz, J.M., Tchernykh, A., Yahyapour, R., Schwiegelshohn, U., Quezada-Pina, A., González-García, J.L., Hirales-Carbajal, A.: Job allocation strategies with user run time estimates for online scheduling in hierarchical Grids. J. Grid Comput. 9(1), 95–116 (2011)

    Article  Google Scholar 

  14. Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications, vol. 63. Shaker, Ithaca (1999)

    Google Scholar 

  15. Kliazovich, D., Bouvry, P., Khan, S.U.: DENS: data center energy-efficient network-aware scheduling. Cluster Comput. 16(1), 65–75 (2013)

    Article  Google Scholar 

  16. Liu, K., Jin, H., Chen, J., Liu, X., Yuan, D., Yang, Y.: A compromised-time-cost scheduling algorithm in swindew-c for instance-intensive cost-constrained workflows on a cloud computing platform. Int. J. High Perform. Comput. Appl. 24(4), 445–456 (2010)

    Article  Google Scholar 

  17. Jin, J., Luo, J., Song, A., Dong, F., Xiong, R.: BAR: an efficient data locality driven task scheduling algorithm for cloud computing. In: 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2011), pp. 295–304 (2011)

    Google Scholar 

  18. Sonnek, J., Greensky, J., Reutiman, R., Chandra, A.: Starling: minimizing communication overhead in virtualized computing platforms using decentralized affinity-aware migration. In: 39th International Conference on Parallel Processing (ICPP 2010), pp. 228–237 (2010)

    Google Scholar 

  19. Pecero, J.E., Trystram, D., Zomaya, A.Y.: A new genetic algorithm for scheduling for large communication delays. In: Sips, H., Epema, D., Lin, H.-X. (eds.) Euro-Par 2009. LNCS, vol. 5704, pp. 241–252. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  20. Stage, A., Setzer, T.: Network-aware migration control and scheduling of differentiated virtual machine workloads. In: Proceedings of the 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing, pp. 9–14. IEEE Computer Society (2009)

    Google Scholar 

  21. Sinnen, O., Sousa, L.A.: Communication contention in task scheduling. IEEE Trans. Parallel Distrib. Syst. 16(6), 503–515 (2005)

    Article  Google Scholar 

  22. Volckaert, B., Thysebaert, P., De Leenheer, M., De Turck, F., Dhoedt, B., Demeester, P.: Network aware scheduling in grids. In: Proceedings of the 9th European Conference on Networks and Optical Communifications, p. 9 (2004)

    Google Scholar 

  23. Malawski, M., Juve, G., Deelman, E., Nabrzyski, J.: Cost-and deadline-constrained provisioning for scientific workflow ensembles in IaaS clouds. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, p. 22 (2012)

    Google Scholar 

  24. Yu, J., Buyya, R.: A taxonomy of workflow management systems for grid computing. J. Grid Comput. 3(3–4), 171–200 (2005)

    Article  Google Scholar 

  25. Tchernykh, A., Pecero, J., Barrondo, A., Schaeffer, E.: Adaptive energy efficient scheduling in peer-to-peer desktop grids. Future Gener. Comput. Systems 36, 209–220 (2014)

    Article  Google Scholar 

  26. Tchernykh, A., Lozano, L., Schwiegelshohn, U., Bouvry, P., Pecero, J.E., Nesmachnow, S., Drozdov, A.Y.: Online bi-objective scheduling for IaaS clouds with ensuring quality of service. J. Grid Comput., 1–18 (2015). Springer

    Google Scholar 

  27. Carbajal, A.H., Tchernykh, A., Yahyapour, R., Röblitz, T., Ramírez-Alcaraz, J.M., González-García, J.L.: Multiple workflow scheduling strategies with user run time estimates on a grid. J. Grid Comput. 10(2), 325–346 (2012). Springer-Verlag, New York, USA

    Article  Google Scholar 

  28. Quezada, A., Tchernykh, A., González, J., Hirales, A., Ramírez, J.-M., Schwiegelshohn, U., Yahyapour, R., Miranda, V.: Adaptive parallel job scheduling with resource admissible allocation on two level hierarchical grids. Future Gener. Comput. Syst. 28(7), 965–976 (2012)

    Article  Google Scholar 

  29. Rodriguez, A., Tchernykh, A., Ecker, K.: Algorithms for dynamic scheduling of unit execution time tasks. Eur. J. Oper. Res. 146(2), 403–416 (2003). Elsevier Science, North-Holland

    Article  MathSciNet  MATH  Google Scholar 

  30. Kianpisheh, S., Jalili, S., Charkari, N.M.: Predicting job wait time in grid environment by applying machine learning methods on historical information. Int. J. Grid Distrib. Comput. 5(3) (2012)

    Google Scholar 

  31. Iverson, M.A., Ozguner, F.; Follen, G.J.: Run-time statistical estimation of task execution times for heterogeneous distributed computing. In: Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing, 1996, pp. 263–270 (1996)

    Google Scholar 

  32. Ramirez-Velarde, R.V., Rodriguez-Dagnino, R.M.: From commodity computers to high-performance environments: scalability analysis using self-similarity, large deviations and heavy-tails. Concurrency Comput. Pract. Exp. 22, 1494–1515 (2010)

    Google Scholar 

Download references

Acknowledgment

This work is partially supported by CONACYT (Consejo Nacional de Ciencia y Tecnología, México), grant no. 178415. The work of D. Dzmitry Kliazovich is partly funded by National Research Fund, Luxembourg in the framework of ECO-CLOUD (C12/IS/3977641) project.

Author information

Authors and Affiliations

  1. CICESE Research Center, Ensenada, B.C., Mexico

    Vanessa Miranda & Andrei Tchernykh

  2. University of Luxembourg, Luxembourg, Luxembourg

    Dzmitry Kliazovich

Authors
  1. Vanessa Miranda
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrei Tchernykh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dzmitry Kliazovich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrei Tchernykh .

Editor information

Editors and Affiliations

  1. ABACUS Centro de Matemáticas Aplicadas, CINVESTAV-IPN, La Marquesa, Mexico

    Isidoro Gitler

  2. Instituto Nacional de Investigaciones Nu, La Marquesa, Mexico

    Jaime Klapp

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Miranda, V., Tchernykh, A., Kliazovich, D. (2016). Dynamic Communication-Aware Scheduling with Uncertainty of Workflow Applications in Clouds. In: Gitler, I., Klapp, J. (eds) High Performance Computer Applications. ISUM 2015. Communications in Computer and Information Science, vol 595. Springer, Cham. https://doi.org/10.1007/978-3-319-32243-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-32243-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-32242-1

  • Online ISBN: 978-3-319-32243-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation