Bibliographic Details
| Title: |
LeCo: Lightweight Compression via Learning Serial Correlations |
| Authors: |
Liu, Yihao, Zeng, Xinyu, Zhang, Huanchen |
| Publication Year: |
2023 |
| Collection: |
Computer Science |
| Subject Terms: |
Computer Science - Databases, Computer Science - Machine Learning |
| More Details: |
Lightweight data compression is a key technique that allows column stores to exhibit superior performance for analytical queries. Despite a comprehensive study on dictionary-based encodings to approach Shannon's entropy, few prior works have systematically exploited the serial correlation in a column for compression. In this paper, we propose LeCo (i.e., Learned Compression), a framework that uses machine learning to remove the serial redundancy in a value sequence automatically to achieve an outstanding compression ratio and decompression performance simultaneously. LeCo presents a general approach to this end, making existing (ad-hoc) algorithms such as Frame-of-Reference (FOR), Delta Encoding, and Run-Length Encoding (RLE) special cases under our framework. Our microbenchmark with three synthetic and six real-world data sets shows that a prototype of LeCo achieves a Pareto improvement on both compression ratio and random access speed over the existing solutions. When integrating LeCo into widely-used applications, we observe up to 5.2x speed up in a data analytical query in the Arrow columnar execution engine and a 16% increase in RocksDB's throughput. |
| Document Type: |
Working Paper |
| Access URL: |
http://arxiv.org/abs/2306.15374 |
| Accession Number: |
edsarx.2306.15374 |
| Database: |
arXiv |
