Skip to main content
SpringerLink
Log in

On Grid Performance Evaluation Using Synthetic Workloads

  • Conference paper
Job Scheduling Strategies for Parallel Processing (JSSPP 2006)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 4376))

Included in the following conference series:

Abstract

Grid computing is becoming a common platform for solving large scale computing tasks. However, a number of major technical issues, including the lack of adequate performance evaluation approaches, hinder the grid computing’s further development. The requirements herefore are manifold; adequate approaches must combine appropriate performance metrics, realistic workload models, and flexible tools for workload generation, submission, and analysis. In this paper we present an approach to tackle this complex problem. First, we introduce a set of grid performance objectives based on traditional and grid-specific performance metrics. Second, we synthesize the requirements for realistic grid workload modeling, e.g. co-allocation, data and network management, and failure modeling. Third, we show how GrenchMark, an existing framework for generating, running, and analyzing grid workloads, can be extended to implement the proposed modeling techniques. Our approach aims to be an initial and necessary step towards a common performance evaluation framework for grid environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. The Parallel Workloads Archive Team. The parallel workloads archive logs (June 2006), Available at http://www.cs.huji.ac.il/labs/parallel/workload/logs.html

  2. Berman, F., et al.: Adaptive computing on the grid using apples. IEEE Trans. Parallel Distrib. Syst. 14(4), 369–382 (2003)

    Article  Google Scholar 

  3. Beume, N., et al.: Scheduling Algorithm Development based on Complex Owner Defined Objectives. Technical Report CI-190/05, University of Dortmund (January 2005)

    Google Scholar 

  4. Bucur, A.I.D., Epema, D.H.J.: The performance of processor co-allocation in multicluster systems. In: Proc. of the 3rd IEEE Int’l. Symp. on Cluster Computing and the Grid (CCGrid), pp. 302–309. IEEE Computer Society Press, Los Alamitos (2003)

    Chapter  Google Scholar 

  5. Bucur, A.I.D., Epema, D.H.J.: Trace-based simulations of processor co-allocation policies in multiclusters. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 70–79 (2003)

    Google Scholar 

  6. Buyya, R., Abramson, D., Venugopal, S.: The grid economy. In: Special Issue of the Proceedings of the IEEE on Grid Computing. IEEE Press, Los Alamitos (To appear, 2005)

    Google Scholar 

  7. Calzarossa, M., Serazzi, G.: A characterization of the variation in time of workload arrival patterns. IEEE Trans. Comput. 34(2), 156–162 (1985)

    Article  Google Scholar 

  8. Chapin, S.J., et al.: Benchmarks and standards for the evaluation of parallel job schedulers. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 1999, IPPS-WS 1999, and SPDP-WS 1999. LNCS, vol. 1659, pp. 67–90. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  9. Chun, B.N., et al.: Mirage: A microeconomic resource allocation system for sensornet testbeds. In: Proc. of 2nd IEEE Workshop on Embedded Networked Sensors (EmNetsII), IEEE Computer Society Press, Los Alamitos (2005)

    Google Scholar 

  10. Chun, G., et al.: Benchmark probes for grid assessment. In: Proc. of the 18th International Parallel and Distributed Processing Symposium (IPDPS) (2004)

    Google Scholar 

  11. Cirne, W., Berman, F.: A Comprehensive Model of the Supercomputer Workload. In: 4th Workshop on Workload Characterization (December 2001)

    Google Scholar 

  12. Cirne, W., Berman, F.: A model for moldable supercomputer jobs. In: Proc. of the 15th International Parallel and Distributed Processing Symposium (IPDPS), pp. 59–79 (2001)

    Google Scholar 

  13. Downey, A.B.: A parallel workload model and its implications for processor allocation. Cluster Computing 1(1), 133–145 (1998)

    Article  Google Scholar 

  14. Dumitrescu, C., Raicu, I., Foster, I.T.: Experiences in running workloads over grid3. In: Zhuge, H., Fox, G.C. (eds.) GCC 2005. LNCS, vol. 3795, pp. 274–286. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  15. Ernemann, C., Yahyapour, R.: Applying Economic Scheduling Methods to Grid Environments. In: Ernemann, C., Yahyapour, R. (eds.) Grid Resource Management - State of the Art and Future Trends, pp. 491–506. Kluwer Academic Publishers, Dordrecht (2003)

    Google Scholar 

  16. Ernemann, C., et al.: On advantages of grid computing for parallel job scheduling. In: Proc. of the 2nd IEEE Int’l. Symp. on Cluster Computing and the Grid (CCGrid), pp. 39–49. IEEE Computer Society Press, Los Alamitos (2002)

    Chapter  Google Scholar 

  17. Ernemann, C., Hamscher, V., Yahyapour, R.: Benefits of global grid computing for job scheduling. In: Proceedings of the 5th IEEE/ACM International Workshop on Grid Computing, Pittsburgh, November 2004, IEEE Computer Society, Los Alamitos (2004)

    Google Scholar 

  18. Ernemann, C., Song, B., Yahyapour, R.: Scaling of Workload Traces. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2003. LNCS, vol. 2862, pp. 166–183. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  19. Feitelson, D.G.: Packing Schemes for Gang Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1162, pp. 89–110. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  20. Feitelson, D.G.: The forgotten factor: facts on performance evaluation and its dependence on workloads. In: Monien, B., Feldmann, R.L. (eds.) Euro-Par 2002. LNCS, vol. 2400, pp. 49–60. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  21. Feitelson, D.G.: Workload Modeling for Performance Evaluation. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 114–141. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  22. Feitelson, D.G.: Metric and workload effects on computer systems evaluation. IEEE Computer 36(9), 18–25 (2003)

    Google Scholar 

  23. Feitelson, D.G.: Experimental analysis of the root causes of performance evaluation results: a backfilling case study. IEEE Transactions on Parallel and Distributed Systems 16(2), 175–182 (2005)

    Article  Google Scholar 

  24. Feitelson, D.G., Rudolph, L.: Metrics and benchmarking for parallel job scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1459, pp. 1–24. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  25. Feitelson, D.G., et al.: Theory and Practice in Parallel Job Scheduling. In: Feitelson, D.G., Rudolph, L. (eds.) Job Scheduling Strategies for Parallel Processing. LNCS, vol. 1291, pp. 1–34. Springer, Heidelberg (1997)

    Google Scholar 

  26. Foster, I., et al.: End-to-end quality of service for high-end applications. Computer Communications 27(14), 1375–1388 (2004)

    Article  Google Scholar 

  27. Frumkin, M.A., Van der Wijngaart, R.F.: Nas grid benchmarks: A tool for grid space exploration. In: Proc. of the 10th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 315–326 (2001)

    Google Scholar 

  28. Graham, R.L., et al.: Optimization and approximation in deterministic sequencing and scheduling: A survey. Annals of Discrete Mathematics 15, 287–326 (1979)

    Article  MathSciNet  Google Scholar 

  29. Hwang, S., Kesselman, C.: Gridworkflow: A flexible failure handling framework for the grid. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 126–137 (2003)

    Google Scholar 

  30. Iosup, A., et al.: How are real grids used? the analysis of four grid traces and its implications. In: The 7th IEEE/ACM International Conference on Grid Computing (Grid), Barcelona, ES, Sept. 28-29 (accepted, 2006)

    Google Scholar 

  31. Iosup, A., Epema, D.H.J.: GrenchMark: A framework for analyzing, testing, and comparing grids. In: Proc. of the 6th IEEE/ACM Int’l. Symp. on Cluster Computing and the GRID (CCGrid), May (accepted, 2006)

    Google Scholar 

  32. Iosup, A., et al.: Synthetic grid workloads with Ibis, KOALA, and GrenchMark. In: Dumke, R.R., Abran, A. (eds.) IWSM 2000. LNCS, vol. 2006, Springer, Heidelberg (2001)

    Google Scholar 

  33. Jackson, D., Snell, Q., Clement, M.: Core algorithms of the Maui scheduler. In: Feitelson, D.G., Rudolph, L. (eds.) JSSPP 2001. LNCS, vol. 2221, pp. 87–102. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  34. Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling (Winner of “1991 Best Advanced How-To Book, Systems” award from the Computer Press Association). Wiley-Interscience, New York (May 1991)

    MATH  Google Scholar 

  35. Kenyon, C., Cheliotis, G.: Architecture requirements for commercializing grid resources. In: Proc. of the 11th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 215–224 (2002)

    Google Scholar 

  36. Kola, G., Kosar, T., Livny, M.: Phoenix: Making data-intensive grid applications fault-tolerant. In: Buyya, R. (ed.) GRID, pp. 251–258. IEEE Computer Society Press, Los Alamitos (2004)

    Google Scholar 

  37. Kondo, D., et al.: Characterizing and evaluating desktop grids: An empirical study. In: Proc. of the 18th International Parallel and Distributed Processing Symposium (IPDPS) (2004)

    Google Scholar 

  38. Li, H., Groep, D., Wolters, L.: Workload characteristics of a multi-cluster supercomputer. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 176–194. Springer, Heidelberg (2005)

    Google Scholar 

  39. Lublin, U., Feitelson, D.G.: The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. Journal of Parallel and Distributed Computing 63(20), 1105–1122 (2003)

    Article  MATH  Google Scholar 

  40. Medeiros, R., et al.: Faults in grids: Why are they so bad and what can be done about it? In: Stockinger, H. (ed.) GRID, pp. 18–24. IEEE Computer Society Press, Los Alamitos (2003)

    Google Scholar 

  41. Medernach, E.: Workload analysis of a cluster in a grid environment. In: Feitelson, D.G., et al. (eds.) JSSPP 2005. LNCS, vol. 3834, pp. 36–61. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  42. Mohamed, H.H., Epema, D.H.J.: Experiences with the koala co-allocating scheduler in multiclusters. In: Proc. of the 5th IEEE/ACM Int’l Symp. on Cluster Computing and the GRID (CCGrid), May 2005, ACM Press, New York (2005)

    Google Scholar 

  43. Pinedo, M.: Scheduling: Theory, Algorithms, and Systems, 2nd edn. Prentice-Hall, Englewood Cliffs (2002)

    MATH  Google Scholar 

  44. Schroeder, W., et al.: Analysis of HPSS Performance Based on Per-file Transfer Logs. In: Proc. of the 16th IEEE Mass Storage Systems Symposium, San Diego, March 1999, pp. 103–115. IEEE Computer Society Press, Los Alamitos (1999)

    Google Scholar 

  45. Schwiegelshohn, U., Yahyapour, R.: Fairness in parallel job scheduling. Journal of Scheduling 3(5), 297–320 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  46. Shan, H., Oliker, L., Biswas, R.: Job superscheduler architecture and performance in computational grid environments. In: SC, pp. 44–54. ACM Press, New York (2003)

    Google Scholar 

  47. Song, B., Ernemann, C., Yahyapour, R.: Parallel Computer Workload Modeling with Markov Chains. In: Feitelson, D.G., Rudolph, L., Schwiegelshohn, U. (eds.) JSSPP 2004. LNCS, vol. 3277, pp. 47–62. Springer, Heidelberg (2005)

    Google Scholar 

  48. Song, B., Ernemann, C., Yahyapour, R.: User Group-based Workload Analysis and Modeling. In: Proc. of the 5th Int’l. Symp. on Cluster Computing and the Grid (CCGrid), IEEE Computer Society Press, Los Alamitos (2005)

    Google Scholar 

  49. Thain, D., et al.: Pipeline and batch sharing in grid workloads. In: Proc. of the 12th Intl. Symposium on High-Performance Distributed Computing (HPDC), pp. 152–161 (2003)

    Google Scholar 

  50. Theys, M.D., et al.: A Mathematical Model, Heuristic and Simulation Study for a Basic Data Staging Problem in a Heterogenous Networking Environment. In: Proc. of the 7th Heterogeneous Computing Workshop, Orlando, March 1998, pp. 115–122. IEEE, Los Alamitos (1998)

    Google Scholar 

  51. Tsouloupas, G., Dikaiakos, M.D.: GridBench: A workbench for grid benchmarking. In: Sloot, P.M.A., et al. (eds.) EGC 2005. LNCS, vol. 3470, pp. 211–225. Springer, Heidelberg (2005)

    Google Scholar 

  52. Wolski, R., Spring, N., Hayes, J.: The network weather service: A distributed resource performance forecasting service for metacomputing. Journal of Future Generation Computing Systems 15(5-6), 757–768 (1999)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering, Mathematics, and Computer Science, Delft University of Technology, The Netherlands

    Alexandru Iosup & Dick H. J. Epema

  2. Information Technology Section, Robotics Research Institute, Dortmund University, Germany

    Carsten Franke, Alexander Papaspyrou, Lars Schley, Baiyi Song & Ramin Yahyapour

Authors
  1. Alexandru Iosup
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dick H. J. Epema
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Carsten Franke
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Papaspyrou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lars Schley
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Baiyi Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Ramin Yahyapour
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Eitan Frachtenberg Uwe Schwiegelshohn

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this paper

Cite this paper

Iosup, A. et al. (2007). On Grid Performance Evaluation Using Synthetic Workloads. In: Frachtenberg, E., Schwiegelshohn, U. (eds) Job Scheduling Strategies for Parallel Processing. JSSPP 2006. Lecture Notes in Computer Science, vol 4376. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71035-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-71035-6_12

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-71034-9

  • Online ISBN: 978-3-540-71035-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation