Skip to main content
SpringerLink
Log in

A Twin-Buffer Scheme for High-Throughput Logging

  • Conference paper
  • First Online:
Database Systems for Advanced Applications (DASFAA 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10828))

Included in the following conference series:

Abstract

For a transactional database system, the efficiency of logging is usually crucial to its performance. The emergence of new hardware, such as NVM and SSD, eliminated the traditional I/O bottleneck of logging and released the potential of multi-core CPUs. As a result, the parallelism of logging becomes important. We propose a parallel logging subsystem called TwinBuf and implemented it in PostgreSQL. This solution can make better use of multi-core CPUs, and is generally applicable to all kinds of storage devices, such as hard disk, SSD and NVM. TwinBuf adopts per-thread logging slots to parallelize logging, and a twin-log-buffer mechanism to make sure that logging can be performed in a non-stop manner. It performs group commit to minimize the persistence overheads. Experimental evaluation was conducted to demonstrate its advantages.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://en.wikipedia.org/wiki/Compare-and-swap.

References

  1. Improve scalability of wal insertions. https://github.com/postgres/postgres/commit/9a20a9b21baa819df1760b36f3c36f25d11fc27b

  2. Replace the xloginsert slots with regular lwlocks. https://github.com/postgres/postgres/commit/68a2e52bbaf98f136a96b3a0d734ca52ca440a95

  3. Alomari, M., Cahill, M., Fekete, A., Rohm, U.: The cost of serializability on platforms that use snapshot isolation. In: IEEE 24th International Conference on Data Engineering, 2008. ICDE 2008, pp. 576–585. IEEE (2008)

    Google Scholar 

  4. Chen, S.: Flashlogging: exploiting flash devices for synchronous logging performance. In: Proceedings of the 2009 ACM SIGMOD International Conference on Management of Data, pp. 73–86. ACM (2009)

    Google Scholar 

  5. Fang, R., Hsiao, H.-I., He, B., Mohan, C., Wang, Y.: High performance database logging using storage class memory. In: 2011 IEEE 27th International Conference on Data Engineering (ICDE), pp. 1221–1231. IEEE (2011)

    Google Scholar 

  6. Gao, S., Xu, J., He, B., Choi, B., Hu, H.: PCMLogging: reducing transaction logging overhead with PCM. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, pp. 2401–2404. ACM (2011)

    Google Scholar 

  7. Helland, P., Sammer, H., Lyon, J., Carr, R., Garrett, P., Reuter, A.: Group commit timers and high volume transaction systems. In: Gawlick, D., Haynie, M., Reuter, A. (eds.) HPTS 1987. LNCS, vol. 359, pp. 301–329. Springer, Heidelberg (1989). https://doi.org/10.1007/3-540-51085-0_52

    Chapter  Google Scholar 

  8. Huang, J., Schwan, K., Qureshi, M.K.: NVRAM-aware logging in transaction systems. Proc. VLDB Endow. 8(4), 389–400 (2014)

    Article  Google Scholar 

  9. Lee, S.-W., Moon, B., Park, C., Kim, J.-M., Kim, S.-W.: A case for flash memory SSD in enterprise database applications. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pp. 1075–1086. ACM (2008)

    Google Scholar 

  10. Mohan, C., Haderle, D., Lindsay, B., Pirahesh, H., Schwarz, P.: Aries: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging. ACM Trans. Database Syst. (TODS) 17(1), 94–162 (1992)

    Article  Google Scholar 

  11. Pelley, S., Wenisch, T.F., Gold, B.T., Bridge, B.: Storage management in the NVRAM era. Proc. VLDB Endow. 7(2), 121–132 (2013)

    Article  Google Scholar 

  12. Rafii, A., DuBois, D.: Performance tradeoffs of group commit logging. In: International CMG Conference, pp. 164–176 (1989)

    Google Scholar 

  13. Son, Y., Kang, H., Yeom, H.Y., Han, H.: A log-structured buffer for database systems using non-volatile memory. In: Proceedings of the Symposium on Applied Computing, pp. 880–886. ACM (2017)

    Google Scholar 

  14. Wang, T., Johnson, R.: Scalable logging through emerging non-volatile memory. Proc. VLDB Endow. 7(10), 865–876 (2014)

    Article  Google Scholar 

  15. Xiao-Feng, L.Z.-P.M., Da, Z.: HV-recovery: a high efficient recovery technique for flash-based database. Chin. J. Comput. 12, 007 (2010)

    Google Scholar 

  16. Xu, J., Swanson, S.: NOVA: a log-structured file system for hybrid volatile/non-volatile main memories. In: FAST, pp. 323–338 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MOE Key Laboratory of DEKE, Renmin University of China, Beijing, China

    Qingzhong Meng & Shan Wang

  2. School of Data Science & Engineering, East China Normal University, Shanghai, China

    Xuan Zhou

  3. Huawei Technologies Co., Ltd., Shenzhen, China

    Haiyan Huang & Xiaoli Liu

Authors
  1. Qingzhong Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Haiyan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoli Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qingzhong Meng .

Editor information

Editors and Affiliations

  1. Simon Fraser University, Burnaby, BC, Canada

    Jian Pei

  2. Aristotle University of Thessaloniki, Thessaloniki, Greece

    Yannis Manolopoulos

  3. University of Queensland, Brisbane, QLD, Australia

    Shazia Sadiq

  4. University of Western Australia, Crawley, WA, Australia

    Jianxin Li

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Meng, Q., Zhou, X., Wang, S., Huang, H., Liu, X. (2018). A Twin-Buffer Scheme for High-Throughput Logging. In: Pei, J., Manolopoulos, Y., Sadiq, S., Li, J. (eds) Database Systems for Advanced Applications. DASFAA 2018. Lecture Notes in Computer Science(), vol 10828. Springer, Cham. https://doi.org/10.1007/978-3-319-91458-9_45

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91458-9_45

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91457-2

  • Online ISBN: 978-3-319-91458-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation