Skip to main content
SpringerLink
Log in

LZ78 Compression in Low Main Memory Space

  • Conference paper
  • First Online:
String Processing and Information Retrieval (SPIRE 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10508))

Included in the following conference series:

Abstract

We present the first algorithms that perform the LZ78 compression of a text of length n over alphabet \([1..\sigma ]\), whose output is z integers, using only \(O(z\lg \sigma )\) bits of main memory. The algorithms read the input text from disk in a single pass, and write the compressed output to disk. The text can also be decompressed within the same main memory usage, which is unprecedented too. The algorithms are based on hashing and, under some simplifying assumptions, run in O(n) expected time. We experimentally verify that our algorithms use 2–9 times less time and/or space than previously implemented LZ78 compressors.

This collaboration started during the Dagstuhl Seminar 16431, “Computation over Compressed Structured Data”. We also acknowledge the funding from Millennium Nucleus Information and Coordination in Networks ICM/FIC RC130003 (G.N.).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://pizzachili.dcc.uchile.cl/texts.

  2. 2.

    http://hgdownload.cse.ucsc.edu/goldenPath/hg18/bigZips/est.fa.gz.

References

  1. Arroyuelo, D., Davoodi, P., Satti, S.R.: Succinct dynamic cardinal trees. Algorithmica 74(2), 742–777 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  2. Arroyuelo, D., Navarro, G.: Space-efficient construction of Lempel-Ziv compressed text indexes. Inf. Comput. 209(7), 1070–1102 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  3. Arroyuelo, D., Navarro, G., Sadakane, K.: Stronger Lempel-Ziv based compressed text indexing. Algorithmica 62(1), 54–101 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. Clark, D.R.: Compact PAT trees. Ph.D. thesis, University of Waterloo, Canada (1996)

    Google Scholar 

  5. Ferrada, H., Navarro, G.: A Lempel-Ziv compressed structure for document listing. In: Kurland, O., Lewenstein, M., Porat, E. (eds.) SPIRE 2013. LNCS, vol. 8214, pp. 116–128. Springer, Cham (2013). doi:10.1007/978-3-319-02432-5_16

    Chapter  Google Scholar 

  6. Ferrada, H., Navarro, G.: Efficient compressed indexing for approximate top-k string retrieval. In: Moura, E., Crochemore, M. (eds.) SPIRE 2014. LNCS, vol. 8799, pp. 18–30. Springer, Cham (2014). doi:10.1007/978-3-319-11918-2_3

    Google Scholar 

  7. Ferragina, P., Manzini, G.: Indexing compressed texts. J. ACM 52(4), 552–581 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  8. Fischer, J., I, T., Köppl, D.: Lempel Ziv Computation in Small Space (LZ-CISS). In: Cicalese, F., Porat, E., Vaccaro, U. (eds.) CPM 2015. LNCS, vol. 9133, pp. 172–184. Springer, Cham (2015). doi:10.1007/978-3-319-19929-0_15

    Chapter  Google Scholar 

  9. Hardy, G.H., Wright, E.M.: An Introduction to the Theory of Numbers, 6th edn. Oxford University Press, Oxford (2008)

    MATH  Google Scholar 

  10. Jansson, J., Sadakane, K., Sung, W.: Linked dynamic tries with applications to LZ-compression in sublinear time and space. Algorithmica 71(4), 969–988 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  11. Köppl, D., Sadakane, K.: Lempel-Ziv Computation in Compressed Space (LZ-CICS). In: Proceedings of 26th Data Compression Conference, pp. 3–12 (2016)

    Google Scholar 

  12. Pagh, A., Pagh, R., Ruzic, M.: Linear probing with 5-wise independence. SIAM Rev. 53(3), 547–558 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  13. Patrascu, M., Thorup, M.: On the k-independence required by linear probing and minwise independence. ACM Trans. Algorithms 12(1) (2016). Article 8

    Google Scholar 

  14. Poyias, A., Puglisi, S.J., Raman, R.: m-Bonsai: a practical compact dynamic trie. In: Preliminary Version Proceedings of SPIRE 2015. LNCS, vol. 9309 (2017). CoRR abs/1704.05682. http://arxiv.org/abs/1704.05682,

  15. Russo, L.M.S., Oliveira, A.L.: A compressed self-index using a Ziv-Lempel dictionary. Inf. Retrieval 11(4), 359–388 (2008)

    Article  Google Scholar 

  16. Sadakane, K., Grossi, R.: Squeezing succinct data structures into entropy bounds. In: Proceedings of 17th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 1230–1239 (2006)

    Google Scholar 

  17. Welch, T.A.: A technique for high performance data compression. IEEE Comput. 17(6), 8–19 (1984)

    Article  Google Scholar 

  18. Ziv, J., Lempel, A.: A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23(3), 337–343 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  19. Ziv, J., Lempel, A.: Compression of individual sequences via variable length coding. IEEE Trans. Inf. Theory 24(5), 530–536 (1978)

    Article  MATH  Google Scholar 

Download references

Acknowledgements

We thank the reviewers for their insightful comments.

Author information

Authors and Affiliations

  1. Departamento de Informática, Universidad Técnica Federico Santa María, Vicuña Mackenna 3939, San Joaquín, Santiago, Chile

    Diego Arroyuelo

  2. LIRMM and IBC, 161 rue Ada, 34095, Montpellier Cedex 5, France

    Rodrigo Cánovas

  3. Deptartment of Computer Science, University of Chile, Beauchef 851, Santiago, Chile

    Gonzalo Navarro

  4. Department of Informatics, University of Leicester, F33 Computer Science Building, University Road, Leicester, UK

    Rajeev Raman

Authors
  1. Diego Arroyuelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rodrigo Cánovas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gonzalo Navarro
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rajeev Raman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gonzalo Navarro .

Editor information

Editors and Affiliations

  1. Università di Palermo, Palermo, Italy

    Gabriele Fici

  2. Università di Palermo, Palermo, Italy

    Marinella Sciortino

  3. Università di Pisa, Pisa, Italy

    Rossano Venturini

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Arroyuelo, D., Cánovas, R., Navarro, G., Raman, R. (2017). LZ78 Compression in Low Main Memory Space. In: Fici, G., Sciortino, M., Venturini, R. (eds) String Processing and Information Retrieval. SPIRE 2017. Lecture Notes in Computer Science(), vol 10508. Springer, Cham. https://doi.org/10.1007/978-3-319-67428-5_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-67428-5_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-67427-8

  • Online ISBN: 978-3-319-67428-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation