Files

T

Move Huffman decoding-table construction and X1/X2 single- and four-stream
decoding into Rust. The original C translation unit is now a declaration shim,
so unchanged C callers and tests resolve the public decoder ABI from the Rust
archive.

The portable implementation preserves the C workspace and error contracts,
retains the existing assembly-only internal fast loops, and mirrors the C
32-bit decode cadence so X2 writes remain within its required output window.
Forced X1/X2 modes follow the matching archive configuration from the build
integration.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets for default, forced X1, and forced X2 modes
- cargo build and cargo test for i686-unknown-linux-gnu in all three modes
- Strict normal, X1, and X2 C shim compilation
- Original fuzzer and fullbench compatibility targets

Refs: rust/README.md
Depends-on: d89ebb31

2026-07-10 20:48:51 +02:00

src

feat(rust): port Huffman decompression

2026-07-10 20:48:51 +02:00

Cargo.lock

feat(rust): add common primitive compatibility layer

2026-07-10 20:02:17 +02:00

Cargo.toml

build(rust): select archives by target and HUF mode

2026-07-10 20:46:23 +02:00

README.md

feat(rust): port Huffman decompression

2026-07-10 20:48:51 +02:00

README.md

Rust rewrite

This directory contains the in-progress Rust replacement for the zstd library and command-line program. During the migration, the crate is built as a static library and linked into the original C test programs. Production C translation units become declaration-only shims as their implementations move to Rust; the original C tests remain unchanged and provide compatibility coverage.

Component map

The crate is organized from low-level representation helpers toward the public zstd ABI:

Common primitives
- mem, bits, bitstream, and cpu implement byte-order, bitstream, and target-feature operations used by the codecs.
- errors, debug, xxhash, and zstd_common provide common exported ABI functions and state.
- common contains shared frame constants and internal data types.
Entropy coding
- entropy_common reads FSE normalized counts and Huffman statistics.
- fse_decompress builds FSE decoding tables and decodes FSE streams.
- fse_compress normalizes counts, writes FSE headers, builds compression tables, and encodes FSE streams.
- huf_decompress builds Huffman decoding tables and decodes X1 and X2 Huffman streams.
Compression primitives
- hist counts byte frequencies for FSE and Huffman compression.
Runtime support
- threading provides platform pthread wrappers required by zstd headers.
- pool implements the bounded worker pool used by multithreaded compression.

The remaining block compression, general decompression, dictionary, legacy, and CLI translation units are still C. They must move before the rewrite is complete. Keeping that boundary explicit prevents a passing hybrid build from being mistaken for the final all-Rust result.

Compatibility boundary

The public ABI continues to come from the existing headers under lib/. Exported Rust functions therefore use C layout and calling conventions. A C source file whose implementation has moved to Rust remains in the original makefile source list as a small shim so header configuration and platform preprocessor behavior stay available during the transition.

The test and program makefiles select an archive directory for the active C configuration: default, forced HUF X1/X2, and the matching Rust target for 32-bit C binaries. They rebuild the selected archive and relink the executable when a Rust source changes, so original C tests do not accidentally use a stale or configuration-incompatible implementation.

Validation

Run focused Rust checks from this directory:

cargo fmt --check
cargo clippy --all-targets -- -D warnings
cargo test --all-targets
cargo build --release

Then run original compatibility tests from the repository root, starting with the narrow target for the component being migrated. For example:

make -C tests fuzzer
./tests/fuzzer -i1 --no-big-tests

Broader tests/Makefile targets remain the authoritative integration gates as more of the library and CLI are rewritten.