Table of Contents

• Methods / Tools / Softwares Compared
• Datasets Used
• SemantiPack Layers / General Compressors Implemented
• Hardwares Used
• Benchmark Result

Methods / Tools / Softwares Compared

• bzip2 bz2 v1.0.8 ( 1996- )
  • website: https://sourceware.org/bzip2/
  • repo: https://gitlab.com/bzip2/bzip2/
  • option flags used in this benchmark: -9
• bzip3 bz3 v1.5.3 ( 2022- )
  • repo: https://github.com/iczelia/bzip3
  • option flags used in this benchmark: -b 16
• Deflate gz v1.10 ( 1990- )
  • website: https://www.gzip.org/
  • repo: https://cgit.git.savannah.gnu.org/cgit/gzip.git
  • option flags used in this benchmark: -9
• LZMA2 xz v5.2.5 ( 2007- )
  • website: https://tukaani.org/xz/
  • repo: https://github.com/tukaani-project/xz
  • option flags used in this benchmark: -9eT0
• Zstandard zst v1.4.8 ( 2015- )
  • website: https://facebook.github.io/zstd/
  • repo: https://github.com/facebook/zstd
  • option flags used in this benchmark: -19T0
• Apache Parquet parquet v1.15.2 ( - )
  • website: https://parquet.apache.org/
  • repo: https://github.com/apache/parquet-format/
  • option flags used in this benchmark: --codec zstd --compression-level 22
  • note: Evaluated only for CSV datasets
• OpenZL zl ( 2025- )
  • website: https://openzl.org/
  • repo: https://github.com/facebook/openzl
  • option flags used in this benchmark: --compressor trained_for_individual_data.zlc or --profile csv as fall back
  • note: Some datasets could NOT be successfully processed by OpenZL; therefore, performance was evaluated using only the successfully processed datasets.
• Amazon Ion ion v0.9.1 ( along with LZMA2 )
  • website:
  • repo:
  • option flags used in this benchmark: cat --format binary
  • note: Evaluated only for JSON datasets
• JSON BinPack jbp v1.1.5
  • website: https://jsonbinpack.sourcemeta.com/
  • repo: https://github.com/sourcemeta/jsonbinpack
  • note: Evaluated only for JSON datasets

Datasets Used

Datasets Used