Skip to main content
SpringerLink
Log in

Algorithms for Scalable Storage Servers

  • Conference paper
SOFSEM 2004: Theory and Practice of Computer Science (SOFSEM 2004)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 2932))

Abstract

We survey a set of algorithmic techniques that make it possible to build a high performance storage server from a network of cheap components. Such a storage server offers a very simple programming model. To the clients it looks like a single very large disk that can handle many requests in parallel with minimal interference between the requests. The algorithms use randomization, redundant storage, and sophisticated scheduling strategies to achieve this goal. The focus is on algorithmic techniques and open questions. The paper summarizes several previous papers and presents a new strategy for handling heterogeneous disks.

Partially supported by the Future and Emerging Technologies programme of the EU under contract number IST-1999-14186 (ALCOM-FT).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aerts, J., Korst, J., Verhaegh, W.: Complexity of Retrieval Problems. Technical Report NL-MS-20.899, Philips Research Laboratories (2000)

    Google Scholar 

  2. Ahuja, R.K., Magnanti, R.L., Orlin, J.B.: Network Flows. Prentice Hall, Englewood Cliffs (1993)

    Google Scholar 

  3. Alon, N., Azar, Y., Woeginger, G.J., Yadid, T.: Approximation Schemes for Scheduling. In: SODA, pp. 493–500 (1997)

    Google Scholar 

  4. Alvarez, G.A., Burkhard, W.A., Cristian, F.: Tolerating Multiple Failures in RAID Architectures with Optimal Storage and Uniform Declustering. In: Proceedings of the 24th Annual International Symposium on Computer Architecture (ISCA 1997), June 2–4. Computer Architecture News, vol. 25(2), pp. 62–72. ACM Press, New York (1997)

    Chapter  Google Scholar 

  5. Andrews, M., Bender, M.A., Zhang, L.: New Algorithms for the Disk Scheduling Problem. In: IEEE (ed.) 37th Annual Symposium on Foundations of Computer Science, pp. 550–559. IEEE Computer Society Press, Los Alamitos (1996)

    Google Scholar 

  6. Azar, Y., Broder, A.Z., Karlin, A.R., Upfal, E.: Balanced Allocations. In: 26th ACM Symposium on the Theory of Computing, pp. 593–602 (1994)

    Google Scholar 

  7. Barve, R.D., Grove, E.F., Vitter, J.S.: Simple Randomized Mergesort on Parallel Disks. Parallel Computing 23(4), 601–631 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  8. Berenbrink, P., Brinkmann, A., Scheideler, C.: Design of the PRESTO Multimedia Storage Network. In: International Workshop on Communication and Data Management in Large Networks, Paderborn Germany, October 5, pp. 2–12 (1999)

    Google Scholar 

  9. Berenbrink, P., Czumaj, A., Steger, A., Vöcking, B.: Balanced Allocations: The Heavily Loaded Case. In: 32th Annual ACM Symposium on Theory of Computing, pp. 745–754 (2000)

    Google Scholar 

  10. Blaum, M., Brady, J., Bruck, J., Menon, J.: EVENODD: An Optimal Scheme for Tolerating Double Disk Failures in RAID Architectures. In: Proceedings of the 21st Annual International Symposium on Computer Architecture, pp. 245–254 (1994)

    Google Scholar 

  11. Cain, J., Sanders, P., Wormald, N.: A Random Multigraph Process for Linear Time RDA Disk Scheduling (2003) (Manuscript in preparation)

    Google Scholar 

  12. Chen, L.T., Rotem, D.: Optimal Response Time Retrieval of Replicated Data (Extended abstract). In: 13th ACM Symposium on Principles of Database Systems, vol. 13, pp. 36–44. ACM Press, New York (1994)

    Google Scholar 

  13. Colarelli, D., Grunwald, D.: Massive Arrays of Idle Disks for Storage Archives. In: SC 2002 Conference CD, IEEE/ACM SIGARCH, Baltimore, MD (November 2002)

    Google Scholar 

  14. Dial, R.B.: Algorithm 360: Shortest-Path Forest with Topological Ordering. Communications of the ACM 12(11), 632–633 (1969)

    Article  Google Scholar 

  15. Gibson, G.A., Hellerstein, L., Karp, R.M., Katz, R.H., Patterson, D.A.: Coding Techniques for Handling Failures in Large Disk Arrays, CSD-88-477. Technical report, U. C. Berkley (1988)

    Google Scholar 

  16. Hutchinson, D.A., Sanders, P., Vitter, J.S.: Duality between Prefetching and Queued Writing with Parallel Disks. In: Meyer auf der Heide, F. (ed.) ESA 2001. LNCS, vol. 2161, pp. 62–73. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  17. Irani, S.: Competetive Analysis of Paging. In: Fiat, A. (ed.) Dagstuhl Seminar 1996. LNCS, vol. 1442, pp. 52–73. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  18. Karp, R.M., Luby, M., Meyer auf der Heide, F.: Efficient PRAM Simulation on a Distributed Memory Machine. In: 24th ACM Symp. on Theory of Computing, May 1992, pp. 318–326 (1992)

    Google Scholar 

  19. Korst, J.: Random Duplicate Assignment: An Alternative to Striping in Video Servers. In: ACM Multimedia, Seattle, pp. 219–226 (1997)

    Google Scholar 

  20. MacWilliams, F.J., Sloane, N.J.A.: Theory of Error-Correcting Codes. North-Holland, Amsterdam (1988)

    Google Scholar 

  21. Mehlhorn, K., Näher, S.: The LEDA Platform of Combinatorial and Geometric Computing. Cambridge University Press, Cambridge (1999)

    Google Scholar 

  22. Meyer, U., Sanders, P., Sibeyn, J. (eds.): Algorithms for Memory Hierarchies. LNCS, vol. 2625. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  23. Miller, E.L., Katz, R.H.: RAMA: An Easy-to-Use, High-Performance Parallel File System. Parallel Computing 23, 419–446 (1997)

    Article  MATH  Google Scholar 

  24. Muntz, R., Santos, J.R., Berson, S.: A Parallel Disk Storage System for Real-Time Multimedia Applications. International Journal of Intelligent Systems 13, 1137–1174 (1998)

    Article  Google Scholar 

  25. Naor, M., Reingold, O.: On the Construction of Pseudorandom Permutations: Luby-Rackoff Revisited. Journal of Cryptology: the Journal of the International Association for Cryptologic Research 12(1), 29–66 (1999)

    MATH  MathSciNet  Google Scholar 

  26. O’Neil, E., O’Neil, P., Weikum, G.: An Optimality Proof of the LRU-K Page Replacement Algorithm. JACM: Journal of the ACM 46 (1999)

    Google Scholar 

  27. Patterson, D., Gibson, G., Katz, R.: A Case for Redundant Arrays of Inexpensive Disks (RAID). In: Proceedings of ACM SIGMOD 1988, pp. 109–116 (1988)

    Google Scholar 

  28. Pittel, B., Spencer, J., Wormald, N.: Sudden Emergence of a Giant k-Core in Random Graph. J. Combinatorial Theory, Series B 67, 111–151 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  29. Rabin, M.O.: Efficient Dispersal of Information for Security, Load Balancing and Fault Tolerance. Journal of the ACM 36(2), 335–348 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  30. Ruemmler, C., Wilkes, J.: An Introduction to Disk Drive Modeling. IEEE Computer 27(3), 17–28 (1994)

    Google Scholar 

  31. Salem, K., Garcia-Molina, H.: Disk Striping. In: Proceedings of Data Engineering 1986 (1986)

    Google Scholar 

  32. Sanders, P.: Reconciling Simplicity and Realism in Parallel Disk Models. Parallel Computing 28(5), 705–723 (2002); Short version in 12th SODA, pp. 67–76 (2001)

    Article  MathSciNet  Google Scholar 

  33. Sanders, P.: Asynchronous Scheduling of Redundant Disk Arrays. IEEE Transactions on Computers 52(9), 1170–1184 (2003); Short version in 12th ACM Symposium on Parallel Algorithms and Architectures, pp. 89–98 (2000)

    Article  Google Scholar 

  34. Sanders, P., Egner, S., Korst, J.: Fast Concurrent Access to Parallel Disks. Algorithmica 1, 21–55 (2003), Short version in 11th SODA, pp. 849–858 (2000)

    MathSciNet  Google Scholar 

  35. Santos, J.R., Muntz, R.R., Ribeiro-Neto, B.: Comparing Random Data Allocation and Data Striping in Multimedia Servers. In: ACM SIGMETRICS, pp. 44–55 (2000)

    Google Scholar 

  36. Scheuermann, P., Weikum, G., Zabback, P.: Data Partitioning and Load Balancing in Parallel Disk Systems. VLDB Journal: Very Large Data Bases 7(1), 48–66 (1998)

    Article  Google Scholar 

  37. Schoenmakers, L.A.M.: A New Algorithm for the Recognition of Series Parallel Graphs. Technical Report CS-R9504, CWI – Centrum voor Wiskunde en Informatica, January 31 (1995)

    Google Scholar 

  38. Stone, H.S., Fuller, S.F.: On the Near-Optimality of the Shortest-Access-Time- First Drum Scheduling Discipline. Communications of the ACM 16(6), 352–353 (1973), Also published in/as: Technical Note No.12, DSL

    Article  MATH  Google Scholar 

  39. Tetzlaff, W., Flynn, R.: Block Allocation in Video Servers for Availability and Throughput. In: Proceedings Multimedia Computing and Networking (1996)

    Google Scholar 

  40. Tewari, R., Mukherjee, R., Dias, D.M., Vin, H.M.: Design and Performance Tradeoffs in Clustered Video Servers. In: Proceedings of the International Conference on Multimedia Computing and Systems, pp. 144–150 (1996)

    Google Scholar 

  41. Vitter, J.S.: External Memory Algorithms and Data Structures: Dealing with Massive Data. ACM Computing Surveys 33(2), 209–271 (2001)

    Article  Google Scholar 

  42. Vitter, J.S., Shriver, E.A.M.: Algorithms for Parallel Memory, I: Two Level Memories. Algorithmica 12(2/3), 110–147 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  43. Woeginger, G.J.: When Does a Dynamic Programming Formulation Guarantee the Existence of a Fully Polynomial Time Approximation Scheme (fptas). INFORMS Journal on Computing 12, 57–75 (2000)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Max Planck Institut für Informatik, Saarbrücken, Germany

    Peter Sanders

Authors
  1. Peter Sanders
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Faculty of Sciences, ILLC - Department of Mathematics and Computer Science, University of Amsterdam, Plantage Muidergracht 24, 1018 TV, Amsterdam, The Netherlands

    Peter Van Emde Boas

  2. Faculty of Mathematics and Physics, Charles University, Prague

    Jaroslav Pokorný

  3. Institute of Informatics and Software Engineering Faculty of Informatics and Information technologies, Slovak University of Technology, Ilkovičova 3, 842 16, Bratislava

    Mária Bieliková

  4. Institute of Computer Science, Academy of Sciences of the Czech Republic, Pod Vodárenskou věží 2, 182 07, Prague 8 Czech Republic

    Július Štuller

Rights and permissions

Reprints and permissions

Copyright information

© 2004 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sanders, P. (2004). Algorithms for Scalable Storage Servers. In: Van Emde Boas, P., Pokorný, J., Bieliková, M., Štuller, J. (eds) SOFSEM 2004: Theory and Practice of Computer Science. SOFSEM 2004. Lecture Notes in Computer Science, vol 2932. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24618-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-24618-3_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-20779-5

  • Online ISBN: 978-3-540-24618-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation