Skip to main content
SpringerLink
Log in

Deadline-Constrained Workflow Scheduling in Volunteer Computing Systems

  • Conference paper
Algorithms and Architectures for Parallel Processing (ICA3PP 2013)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8285))

Abstract

One of the main challenges in volunteer computing systems is scheduling large-scale applications expressed as scientific workflows. This work aims to integrate partitioning scientific workflows and proximity-aware resource provisioning to increase the percentage of workflows that meet the deadline in peer-to-peer based volunteer computing systems. In the partitioning phase, a scientific workflow is partitioned into sub-workflows in order to minimize data dependencies among them. We utilize knowledge-free load balancing policy and proximity of resources to distribute sub-workflows on volunteer resources. Simulation results show that the proposed workflow scheduling system improves the percentage of scientific workflows that meet the deadline with average of 18% under a moderate workload.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Anderson, D.P., Cobb, J., Korpela, E., Lebofsky, M., Werthimer, D.: SETI@home: An experiment in public-resource computing. Commun. ACM 45, 56–61 (2002)

    Article  Google Scholar 

  2. EDGeS@Home project, http://home.edges-grid.eu

  3. Christensen, C., Aina, T., Stainforth, D.: The challenge of volunteer computing with lengthy climate model simulation. In: The 1st IEEE International Conference on e-Science and Grid Computing, pp. 8–15. IEEE Press, New York (2005)

    Google Scholar 

  4. Berriman, G.B., Deelman, E., et al.: Montage: a grid-enabled engine for delivering custom science grade mosaics on demand. In: 16th Annual Symposium Electronic Imaging Sci-ence and Technology, pp. 221–232. SPIE Press, California (2004)

    Google Scholar 

  5. Bharathi, S., Chervenak, A., Deelman, E., Mehta, G., Su, M.H., Vahi, K.: Characterization of Scientific Workflows. In: The Third Workshop on Workflows in Support of Large-scale Science, pp. 1–10. IEEE Press, New York (2008)

    Google Scholar 

  6. Basher, N., Mahanti, A., Williamson, C., Arlitt, M.: A comparative analysis of web and peer-to-peer traffic. In: The 17th International Conference on World Wide Web, pp. 287–296. ACM Press, New York (2008)

    Google Scholar 

  7. Livny, J., Teonadi, H., Livny, M., Waldor, M.K.: High-throughput, kingdom-wide prediction and annotation of bacterial Non-Coding RNAs. PLoS ONE 3, e3197 (2008)

    Google Scholar 

  8. Ghafarian, T., Deldari, H., Javadi, B., Yaghmaee, M.H., Buyya, R.: CycloidGrid: A prox-imity-aware P2P-based resource discovery architecture in volunteer computing systems. Future Gener. Comp. Sy. 29, 1583–1595 (2013)

    Article  Google Scholar 

  9. Di Gaspero, L., Schaerf, A.: Neighborhood portfolio approach for local search applied to timetabling problems. Journal of Mathematical Modeling and Algorithms 5, 65–89 (2006)

    Article  MATH  Google Scholar 

  10. Topcuouglu, H., Hariri, S., Wu, M.Y.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13, 260–274 (2002)

    Article  Google Scholar 

  11. Hordijk, A., der Laan, D.V.: Periodic routing to parallel queues and billiard sequences. Math. Method Oper. Res. 59, 173–192 (2004)

    Article  MATH  Google Scholar 

  12. Medina, A., Lakhina, A., Matta, I., Byers, J.: BRITE: an approach to universal topology generation. In: IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, pp. 346–353. IEEE Press, New York (2001)

    Google Scholar 

  13. Chen, W., Deelman, E.: Integration of workflow partitioning and resource provisioning. In: 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 764–768. IEEE Press, New York (2012)

    Google Scholar 

  14. Blythe, J., Jain, S., Deelman, E., Gil, Y., Vahi, K., Mandal, A., Kennedy, K.: Task scheduling strategies for workflow-based applications in Grids. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, pp. 759–767. IEEE Press, New York (2005)

    Google Scholar 

  15. Braun, T.D., Siegel, H.J., et al.: A Comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distr. Com. 61, 810–837 (2001)

    Article  Google Scholar 

  16. Duan, R., Prodan, R., Fahringer, T.: Run-time Optimization of grid workflow applications. In: 7th IEEE/ACM International Conference on Grid Computing, pp. 33–40. IEEE Press, New York (2005)

    Google Scholar 

  17. Wieczorek, M., Prodan, R., Fahringer, T.: Scheduling of scientific workflows in the ASKALON grid environment. SIGMOND Record 34, 56–62 (2005)

    Article  Google Scholar 

  18. Dong, F., Akl, S.: Two-phase computation and data scheduling algorithms for workflows in the grid. In: International Conference on Parallel Processing, p. 66. IEEE Press, New York (2007)

    Google Scholar 

  19. Kalayci, S., Dasgupta, G., Fong, L., Ezenwoye, O., Sadjadi, S.: Distributed and adaptive execution of condor DAGman workflows. In: The 23rd International Conference on Software Engineering and Knowledge Engineering, pp. 587–590. Knowledge Systems Institute, Illinois (2010)

    Google Scholar 

  20. Lin, C., Shih, C., Hsu, C.: Adaptive dynamic scheduling algorithms for mapping ongoing m-tasks to pr 2 grid. J. Inf. Sci. Eng. 26, 2107–2125 (2010)

    Google Scholar 

  21. Kumar, S., Das, S., Biswas, R.: Graph partitioning for parallel applications in heterogene-ous grid environments. In: International Parallel and Distributed Processing Symposium, pp. 66–72. IEEE Press, New York (2002)

    Google Scholar 

  22. Sanchis, L.A.: Multiple-way network partitioning. IEEE Trans. on Computers 38, 62–81 (1989)

    Article  MATH  Google Scholar 

  23. Sanchis, L.A.: Multiple-way network partitioning with different cost functions. IEEE Trans. on Computers 42, 1500–1504 (1993)

    Article  Google Scholar 

  24. Benlic, U., Hao, J.K.: An effective multilevel tabu search approach for balanced graph partitioning. Comput. Oper. Res. 38, 1066–1075 (2010)

    Article  MathSciNet  Google Scholar 

  25. Ghafarian, T., Deldari, H., Javadi, B., Buyya, R.: A proximity-aware load balancing in peer-to-peer-based volunteer computing systems. J. Supercomput. 65, 797–822 (2013)

    Article  Google Scholar 

  26. Malawski, M., Juve, G., Deelman, E., Nabrzyskiz, J.: Cost and Deadline-Constrained Pro-visioning for Scientific Workflow Ensembles in IaaS Clouds. In: The International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–11. IEEE Press, New York (2012)

    Google Scholar 

  27. Deelman, E., Callaghan, S., Field, E., Francoeur, H., Graves, R., Gupta, V., Jordan, T.H., Kesselman, C., Maechling, P., Mehta, G., Kaya, D.O., Vahi, K., Zhao, L.: Managing large-scale Workflow Execution from resource provisioning to provenance tracking: the CyberShake example. In: Proceedings of the Second IEEE international Conference on E-Science and Grid Computing, p. 14. IEEE Press, New York (2006)

    Google Scholar 

  28. Workflow Generator, https://conflence.pegasus.isi.edu/display/Pegasus/WorkflowGenerator

  29. Iosup, A., Sonmez, O., Anoep, S., Epema, D.: The performance of Bags-of-Tasks in large-scale distributed systems. In: The 17th International Symposium on High Performance Distributed Computing, pp. 97–108. ACM Press, New York (2008)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Khayyam Institute of Higher Education, Iran

    Toktam Ghafarian

  2. School of Computing, Engineering and Mathematics, University of Western Sydney, Australia

    Bahman Javadi

Authors
  1. Toktam Ghafarian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bahman Javadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Science, Cracow University of Technology, Warszawska 24, 31-155, Cracow, Poland

    Joanna Kołodziej

  2. Dipartimento di Ingegneria, Seconda Universita’ di Napoli, 81031, Aversa, CE, Italy

    Beniamino Di Martino

  3. DIMES and ICAR-CNR, c/o Università della Calabria, 87036, Rende, CS, Italy

    Domenico Talia

  4. College of Computing and Information Sciences, Rochester Institute of Technology, 14623, Rochester, NY, USA

    Kaiqi Xiong

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer International Publishing Switzerland

About this paper

Cite this paper

Ghafarian, T., Javadi, B. (2013). Deadline-Constrained Workflow Scheduling in Volunteer Computing Systems. In: Kołodziej, J., Di Martino, B., Talia, D., Xiong, K. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2013. Lecture Notes in Computer Science, vol 8285. Springer, Cham. https://doi.org/10.1007/978-3-319-03859-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-03859-9_4

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-03858-2

  • Online ISBN: 978-3-319-03859-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation