Skip to main content
SpringerLink
Log in

User Estimates Inaccuracy Study in HPC Scheduler

  • Conference paper
  • First Online:
Contemporary Complex Systems and Their Dependability (DepCoS-RELCOMEX 2018)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 761))

Included in the following conference series:

Abstract

The article is focused on user estimates inaccuracy study in high performance computing scheduler. The main goal of this survey is to study historical job submission data from the real supercomputer cenre and investigate accuracy of resources estimated by user on job submission. Based on this study 68.94% of jobs used less than 25% of their requested walltime, which indicates a high estimation inaccuracy. In case of estimation of amount of memory only 31.46% of jobs uses more than 75% of requested memory. Analysis of historical jobs from the further prediction of resources accuracy has been made. Similarities of the new submitted jobs with historical data were described by keys based on user names, software name, walltime and number of processors. Also mathematical model for jobs scheduling in HPC system, with a goal functions, has been proposed and verified.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bożejko, W.: Solving the flow shop problem by parallel programming. J. Parallel Distrib. Comput. 69(5), 470–481 (2009)

    Article  Google Scholar 

  2. Bożejko, W.: On single-walk parallelization of the job shop problem solving algorithms. Comput. Oper. Res. 39(9), 2258–2264 (2012)

    Article  MathSciNet  Google Scholar 

  3. Bożejko, W., Uchroński, M., Wodecki, M.: Block approach to the cyclic flow shop scheduling. Comput. Ind. Eng. 81, 158–166 (2015)

    Article  Google Scholar 

  4. Bożejko, W., Wodecki, M.: Parallel genetic algorithm for minimizing total weighted completion time. LNAI (L NCS), vol. 3070. Springer, Heidelberg (2004)

    Google Scholar 

  5. Chiang, S.H., Arpaci-Dusseau, A., Vernon, M.K.: The Impact of More Accurate Requested Runtimes on Production Job Scheduling Performance. In: Job scheduling strategies for parallel processing, 8th International Workshop, JSSPP 2002, Edinburgh, Scotland, UK, 24 July 2002 Revised Papers, pp. 103–127. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  6. Cirne, W., Berman, F.: A comprehensive model of the supercomputer workload. In: 2001 IEEE International Workshop on Workload Characterization 2001, WWC-4, pp. 140–148, December 2001

    Google Scholar 

  7. Lan, Z., Buettner, D., Desai, N., Tang, W.: Job scheduling with adjusted runtime estimates on production supercomputers. J. Parallel Distrib. Comput. 73, 926–938 (2013)

    Article  Google Scholar 

  8. Mu’alem, A.W., Feitelson, D.G.: Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Trans. Parallel Distrib. Syst. 12(6), 529–543 (2001)

    Article  Google Scholar 

  9. Rodrigo, G.P., Östberg, P.O., Elmroth, E., Antypas, K., Gerber, R., Ramakrishnan, L.: Towards understanding HPC users and systems: a NERSC case study. J. Parallel Distrib. Comput. 111, 206–221 (2018)

    Article  Google Scholar 

  10. Smith, W., Taylor, V., Foster, I.: Using run-time predictions to estimate queue wait times and improve scheduler performance. In: Job Scheduling Strategies for Parallel Processing: IPPS/SPDP’99 Workshop, JSSPP 1999, San Juan, Puerto Rico, 16 April 1999 Proceedings, pp. 202–219. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  11. Srinivasan, S., Kettimuthu, R., Subramani, V., Sadayappan, P.: Characterization of backfilling strategies for parallel job scheduling. In: International Conference on Parallel Processing Workshops 2002 Proceedings, pp. 514–519 (2002)

    Google Scholar 

  12. Ward, W.A., Mahood, C.L., West, J.E.: Scheduling jobs on parallel systems using a relaxed backfill strategy. In: Job Scheduling Strategies for Parallel Processing: 8th International Workshop, JSSPP 2002, Edinburgh, Scotland, UK, 24 July 2002, Revised Papers, pp. 88–102. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

Download references

Acknowledgement

The paper was partially supported by the National Science Centre of Poland, grant OPUS no. DEC 2017/25/B/ST7/02181.

Author information

Authors and Affiliations

  1. Wroclaw Centre for Networking and Supercomputing, Wyb. Wyspiańskiego 27, 50-370, Wrocław, Poland

    Mariusz Uchroński & Mateusz Tykierko

  2. Department of Control Systems and Mechatronics, Faculty of Electronics, Wrocław University of Science and Technology, Janiszewskiego 11/17, 50-372, Wrocław, Poland

    Wojciech Bożejko & Zdzisław Krajewski

  3. Telecommunications and Teleinformatics Department, Faculty of Electronics, Wrocław University of Technology, Janiszewskiego 11-17, 50-372, Wrocław, Poland

    Mieczysław Wodecki

Authors
  1. Mariusz Uchroński
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wojciech Bożejko
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zdzisław Krajewski
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mateusz Tykierko
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mieczysław Wodecki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariusz Uchroński .

Editor information

Editors and Affiliations

  1. Department of Computer Engineering, Wrocław University of Technology, Wrocław, Poland

    Wojciech Zamojski

  2. Department of Computer Engineering, Wrocław University of Technology, Wrocław, Poland

    Jacek Mazurkiewicz

  3. Department of Computer Engineering, Wrocław University of Technology, Wrocław, Poland

    Jarosław Sugier

  4. Department of Computer Engineering, Wrocław University of Technology, Wrocław, Poland

    Tomasz Walkowiak

  5. Polish Academy of Sciences, Systems Research Institute, Warsaw, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Uchroński, M., Bożejko, W., Krajewski, Z., Tykierko, M., Wodecki, M. (2019). User Estimates Inaccuracy Study in HPC Scheduler. In: Zamojski, W., Mazurkiewicz, J., Sugier, J., Walkowiak, T., Kacprzyk, J. (eds) Contemporary Complex Systems and Their Dependability. DepCoS-RELCOMEX 2018. Advances in Intelligent Systems and Computing, vol 761. Springer, Cham. https://doi.org/10.1007/978-3-319-91446-6_47

Download citation

Publish with us

Policies and ethics

Search

Navigation