Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines

Haubenschild M, Sauer C, Neumann T, Leis V (2020)

Publication Type: Conference contribution

Publication year: 2020

Publisher: Association for Computing Machinery

Pages Range: 877-892

Conference Proceedings Title: Proceedings of the ACM SIGMOD International Conference on Management of Data

Event location: Portland, OR US

ISBN: 9781450367356

DOI: 10.1145/3318464.3389716

Abstract

For decades, ARIES has been the standard for logging and recovery in database systems. ARIES offers important features like support for arbitrary workloads, fuzzy checkpoints, and transparent index recovery. Nevertheless, many modern in-memory database systems use more lightweight approaches that have less overhead and better multi-core scalability but only work well for the in-memory setting. Recently, a new class of high-performance storage engines has emerged, which exploit fast SSDs to achieve performance close to pure in-memory systems but also allow out-of-memory workloads. For these systems, ARIES is too slow whereas in-memory logging proposals are not applicable. In this work, we propose a new logging and recovery design that supports incremental and fuzzy checkpointing, index recovery, out-of-memory workloads, and low-latency transaction commits. Our continuous checkpointing algorithm guarantees bounded recovery time. Using per-thread logging with minimal synchronization, our implementation achieves near-linear scalability on multi-core CPUs. We implemented and evaluated these techniques in our LeanStore storage engine. For working sets that fit in main memory, we achieve performance close to that of an in-memory approach, even with logging, checkpointing, and dirty page writing enabled. For the out-of-memory scenario, we outperform a state-of-the-art ARIES implementation by a factor of two.

Authors with CRIS profile

Viktor Leis

Involved external institutions

Tableau Software, Inc. US United States (USA) (US) Technische Universität München (TUM) DE Germany (DE) Friedrich-Schiller-Universität Jena DE Germany (DE)

How to cite

APA:

Haubenschild, M., Sauer, C., Neumann, T., & Leis, V. (2020). Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines. In Proceedings of the ACM SIGMOD International Conference on Management of Data (pp. 877-892). Portland, OR, US: Association for Computing Machinery.

MLA:

Haubenschild, Michael, et al. "Rethinking Logging, Checkpoints, and Recovery for High-Performance Storage Engines." Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD 2020, Portland, OR Association for Computing Machinery, 2020. 877-892.

BibTeX: Download