chapter-tgz
Specially crafted .tar.gz with embedded chapter boundary information
What is it?
What it is
A Rust library for creating and consuming specially crafted.tar.gz files containing chapter boundaries.
Why it exists
To enable efficient O(1) access to specific points in a tar archive and allow parallel decompression of different chapters without stalling on previous entries.
Who should use it
Rust developers working with large compressed archivesSystems engineers needing high-performance file decompressionDevelopers implementing parallel data processing pipelinesUsers needing random access to specific segments of.tar.gz files
Who should avoid it
Developers working in languages other than RustUsers who do not require random access or parallel decompression for.tar.gz filesDevelopers who cannot use specialized.tar.gz files that use custom gzip blocks
How it works
A quick walkthrough in plain English
How chapter-tgz works
Step 1 of 3
You interact with it
Open chapter-tgz, send a request, or connect it to your stack.
Features
Advantages
- Skips intervening entries without full decompression
- Enables simultaneous reading of different file groups
- No breaking changes for existing software/files
- High performance for large datasets with non-sequential access
Disadvantages
- Requires specialized reader to benefit from chapter metadata
- Complexity in file creation compared to standard tar/gzip
Installation
FAQ
What is chapter-tgz and what problem does it solve?
chapter-tgz is a Rust library for creating and consuming .tar.gz files with chapter boundaries, enabling efficient skipping of entry groups in O(1) time and parallel decompression of independent chapters.
How does chapter-tgz achieve O(1) access to specific tar entries?
It encodes chapter boundaries as empty gzip blocks with unique Huffman codes, allowing direct access without decompressing intervening data.
Can chapters be processed in parallel?
Yes, chapters are independent, so different threads can decompress them simultaneously without stalling.
Is chapter-tgz compatible with existing tar.gz tools?
Yes, it maintains compatibility by storing chapter info in a way that standard readers/writers ignore, treating files as ordinary tar.gz.
How are chapters encoded in the file?
Chapter information is stored as valid but empty gzip blocks with peculiar Huffman code alphabets, ensuring backward compatibility.
How to create a tgz with chapters using chapter-tgz?
Use TgzWriter::new() and call create_chapter() to define chapter boundaries, then append data to each chapter.
Featured in Videos
YouTube tutorials and walkthroughs for chapter-tgz
Alternatives
Similar projects ranked by category, topics, and text overlap.