Skip to main content
SpringerLink
Log in

A Conflict Prevention Scheduling Strategy for Shared-State Scheduling in Large Scale Cluster

  • Conference paper
  • First Online:
Cloud Computing and Security (ICCCS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10039))

Included in the following conference series:

  • 1385 Accesses

Abstract

The scheduling strategy is being challenged by the scaling cloud size and more complicated application requirements. Omega provides a share state scheduling architecture to achieve flexible and scalable performance. However, there is few studies aim at the scheduling strategy for shared-state scheduling architecture. So it is worthy further research. In this paper, we present a conflict prevention scheduling strategy for shared-state scheduling architecture. Conflict prevention scheduling strategy gives a feasible solution to reduce conflict and improve efficient for parallel schedulers in shared-state scheduling architecture. We implement it in Omega’s public simulator, experiments results show that conflict prevention scheduling strategy is effective and can significantly improves the efficiency of scheduler with long decision time.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., Stoica, I.: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. In: Proceedings of the 5th European Conference on Computer Systems (EuroSys 2010), pp. 265–278. ACM (2010)

    Google Scholar 

  2. Hindman, B., Konwinski, A., Zaharia, M., Ghodsi, A., Joseph, A.D., Katz, R., Shenker, S., Stoica, I.: Mesos: a platform for fine-grained resource sharing in the data center. In: Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation (NSDI 2011), pp. 295–308. ACM (2011)

    Google Scholar 

  3. Vavilapalli, V.K., Murthy, A.C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Graves, T., Lowe, J., Shah, H., Seth, S., Saha, B., Curino, C., O’Malley, O., Radia, S., Reed, B., Baldeschwieler, E.: Apache Hadoop YARN: yet another resource negotiator. In: Proceedings of the 4th Symposium on Cloud, pp. 1–16. ACM (2013)

    Google Scholar 

  4. Schwarzkopf, M., Konwinski, A., Abd-El-Malek, M., Wilkes, J.: “Omega: flexible, scalable schedulers for large compute clusters. In: Proceedings of the 8th ACM European Conference on Computer Systems (EuroSys 2013), pp. 351–364. ACM (2013)

    Google Scholar 

  5. Ousterhout, K., Wendell, P., Zaharia, M., Stoica, I.: Sparrow: distributed, low latency scheduling. In: Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles (SOSP 2013), pp. 69–84. ACM (2013)

    Google Scholar 

  6. Boutin, E., Ekanayake, J., Lin, W., Shi, B., Zhou, J., Qian, Z., Wu, M., Zhou, L.: Apollo: scalable and coordinated scheduling for cloud-scale computing. In: Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2013), pp. 285–300. ACM (2013)

    Google Scholar 

  7. Verma, A., Pedrosa, L., Abd-El-Malek, M., Korupolu, M., Oppenheimer, D., Tune, E., Wilkes, J.: Large-scale cluster management at Google with Borg. In: Proceedings of the Tenth European Conference on Computer Systems (EuroSys 2015), p. 18. ACM (2015)

    Google Scholar 

  8. Delimitrou, C., Sanchez, D., Kozyrakis, C.: Tarcil: reconciling scheduling speed and quality in large shared clusters. In: Proceedings of the Sixth ACM Symposium on Cloud Computing (SoCC 2015), pp. 97–110. ACM (2015)

    Google Scholar 

  9. Mitzenmacher, M.: The power of two choices in randomized load balancing. IEEE Trans. Parallel Distrib. Comput. 12(10), 1094–1104 (2001)

    Article  Google Scholar 

  10. Park, G.: Eneralization of multiple choice balls-into-bins. In: Proceedings of the 30th Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, pp. 297–298. ACM (2011)

    Google Scholar 

  11. Ren, Y., Shen, J., Wang, J., Han, J., Lee, S.: Mutual verifiable provable data auditing in public cloud storage. J. Internet Technol. 16(2), 317–323 (2015)

    Google Scholar 

  12. Shen, J., Tan, H., Wang, J., Wang, J., Lee, S.: A novel routing protocol providing good transmission reliability in underwater sensor networks. J. Internet Technol. 16(1), 171–178 (2015)

    Google Scholar 

  13. Guo, P., Wang, J., Li, B., Lee, S.: A variable threshold-value authentication architecture for wireless mesh networks. J. Internet Technol. 15(6), 929–936 (2014)

    Google Scholar 

  14. Xia, Z., Wang, X., Sun, X., Wang, Q.: A secure and dynamic multi-keyword ranked search scheme over encrypted cloud data. IEEE Trans. Parallel Distrib. Syst. 27(2), 340–352 (2015)

    Article  MathSciNet  Google Scholar 

  15. Isard, M., Prabhakaran, V., Currey, J., Wieder, U., Talwar, K., Goldberg, A: Quincy: fair scheduling for distributed computing clusters. In: Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, pp. 261–276. ACM (2009)

    Google Scholar 

  16. Bradley, D., Clair, T.S., Farrellee, M., Guo, Z., Livny, M., Sfiligoi, I., Tannenbaum, T.: An update on the scalability limits of the condor batch system. J. Phys: Conf. Ser. 331(6), 062002 (2011)

    Google Scholar 

  17. Gale, D., Shapley, L.S.: College admissions and the stability of marriage. Am. Math. Mon. 69(1), 9–15 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  18. Kung, H.T., Robinson, J.T.: On optimistic methods for concurrency control. ACM Trans. Database Syst. (TODS) 6(2), 213–226 (1981)

    Article  Google Scholar 

  19. Papadimitriou, C.H.: Serializability of concurrent updates. J. ACM 26(4), 631–653 (1979)

    Article  MathSciNet  MATH  Google Scholar 

  20. Eswaran, K.P., Gray, J.N., Lorie, R.A., Traiger, I.L.: The notions of consistency and predicate locks in a database system. Commun. ACM 19(11), 624–633 (1976)

    Article  MathSciNet  MATH  Google Scholar 

  21. Chen, Y., Ganapathi, A.S., Griffith, R., Katz, R.H.: Design insights for MapReduce from diverse production workloads. Technical report UCB/EECS–2012–17, UC, Berkeley, January 2012

    Google Scholar 

  22. Kavulya, S., Tan, J., Gandhi, R., Narasimhan, P.: An analysis of traces from a production MapReduce cluster. In: Proceedings of 10th IEEE/ACM International Conference on (CCGrid 2010), pp. 94–103. IEEE (2010)

    Google Scholar 

  23. Reiss, C., Tumanov, A., Ganger, G.R., Katz, R.H., Kozuch, M.A.: Heterogeneity and dynamicity of clouds at scale: Google trace analysis. In: Proceedings of the Third ACM Symposium on Cloud Computing (SoCC 2012). ACM (2012)

    Google Scholar 

Download references

Acknowledgments

This work is supported by National Natural Science Foundation of China (No. 61363021, No. 61540061). We would like to thank the authors of Omega for their selflessness to public the simulator of Omega. This simulator make up for our lack of experimental environment.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yunnan University, Kunming, China

    Libo He, Zhenping Qiang & Lin Liu

  2. School of Software, Yunnan University, No. 2 Cuihu North Rd, Kunming, Yunnan, China

    Wei Zhou & Shaowen Yao

Authors
  1. Libo He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhenping Qiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaowen Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shaowen Yao .

Editor information

Editors and Affiliations

  1. University of Information Science and Technology, Nanjing, China

    Xingming Sun

  2. Michigan State University , EAST LANSING, Michigan, USA

    Alex Liu

  3. National Dong Hwa University , Shoufeng, Taiwan

    Han-Chieh Chao

  4. Purdue University , West Lafayette, Indiana, USA

    Elisa Bertino

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

He, L., Qiang, Z., Liu, L., Zhou, W., Yao, S. (2016). A Conflict Prevention Scheduling Strategy for Shared-State Scheduling in Large Scale Cluster. In: Sun, X., Liu, A., Chao, HC., Bertino, E. (eds) Cloud Computing and Security. ICCCS 2016. Lecture Notes in Computer Science(), vol 10039. Springer, Cham. https://doi.org/10.1007/978-3-319-48671-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48671-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48670-3

  • Online ISBN: 978-3-319-48671-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation