bzip3

bzip3 is designed as a more powerful, efficient, and modern spiritual successor to the classic BZip2. It aims to provide superior compression ratios and enhanced speed.

🛠 Technical Architecture

The tool achieves its performance through a sophisticated pipeline:

1. Burrows-Wheeler Transform (BWT): Utilizes suffix arrays for high-speed processing.
2. RLE with LZ+Prediction: Employs a pass based on LZ77-style string matching combined with PPM-style context modeling.
3. Entropy Coding: Uses an order-0 context mixing coder.

Note: Much like its predecessor, bzip3 is particularly effective when handling source code or plain text.

---

🚀 Installation

Depending on your preference, you can install the tool via a package manager or by compiling from the source.

• Option A: Manual Compilation (if cloning from Git):

  ./configure
  make
  sudo make install
  

• Option B: macOS Homebrew:

  brew install bzip3
  

---

📊 Performance Benchmarks

The Perl Source Test

To evaluate the tool, a dataset consisting of every released version of Perl5 was used. The files were gathered via wget, decompressed, and bundled into a single .tar archive.

Compression Command Examples:

• xz -T16 -9 -k all.tar
• bzip3 -e -b 511 -j 4 all.tar

Comparison Results:

Decompression Speed (WD Blue HDD):

• Zstandard: 3min 51s
• bzip3 (parallel): 4min 06s
• LZMA (xz): 4min 40s
• bzip2: 9min 22s

Long-Range Deduplication

By combining lrzip (for deduplication) with various back-ends, the following sizes were achieved:

---

⚠️ Disclaimers

[!IMPORTANT] I TAKE NO RESPONSIBILITY FOR ANY LOSS OF DATA ARISING FROM THE USE OF THIS PROGRAM/LIBRARY, HOWSOEVER CAUSED.

Compression inherently assumes that the process is reversible. While the author has invested significant effort into the design and testing of bzip3, the sheer complexity of these algorithms means that perfect software is guaranteed bugs may still exist in rare edge cases.

Warning: Do not use this software for critical data unless you accept the remote possibility that the data may become unrecoverable.

---

💻 System Compatibility & Specs

Performance Metrics: On x64 Linux using clang13, typical speeds are:

• Compression: $\approx 17 \text{ MiB/s}$ per thread.
• Decompression: $\approx 23 \text{ MiB/s}$ per thread.

Supported Architectures: bzip3 has been verified on:

• x86 / x86_64
• armv6 / armv7 / aarch64
• ppc64le
• mips / mips64
• sparc
• s390x

For more detailed results, please refer to etc/BENCHMARKS.md.

---

📜 Licensing

The project as a whole is licensed under LGPLv3. Below is the detailed breakdown of components: