Files

T

Move Huffman table construction, table serialization, and one- and
four-stream payload encoding from huf_compress.c into the Rust compatibility
archive.  The declaration-only C shim preserves the existing internal ABI,
so the remaining C compressor can call the migrated implementation unchanged.

The translation keeps CTable layouts, workspace checks, repeat-table
selection, and bitstream output compatible with the original encoder.  This
lets native C consumers exercise the Rust implementation through libzstd.

Test Plan:
- cargo clippy, cargo clippy --benches, and cargo clippy --tests
- cargo +nightly fmt and cargo test --all-targets (88 passed)
- cargo check --no-default-features --features compression
- rebuild and run tests fuzzer, zstreamtest, and invalidDictionaries
- compare tables, headers, and 1X/4X streams with a renamed pristine C build

Refs: rust/README.md component map

2026-07-10 21:54:23 +02:00

src

feat(rust): port Huffman compression

2026-07-10 21:54:23 +02:00

Cargo.lock

feat(rust): add common primitive compatibility layer

2026-07-10 20:02:17 +02:00

Cargo.toml

build(rust): link migrated code into native libraries

2026-07-10 21:41:22 +02:00

README.md

feat(rust): port Huffman compression

2026-07-10 21:54:23 +02:00

README.md

Rust rewrite

This directory contains the in-progress Rust replacement for the zstd library and command-line program. During the migration, the crate is built as a static library and linked into the original C test programs. Production C translation units become declaration-only shims as their implementations move to Rust; the original C tests remain unchanged and provide compatibility coverage.

Component map

The crate is organized from low-level representation helpers toward the public zstd ABI:

Common primitives
- mem, bits, bitstream, and cpu implement byte-order, bitstream, and target-feature operations used by the codecs.
- errors, debug, xxhash, and zstd_common provide common exported ABI functions and state.
- common contains shared frame constants and internal data types.
Entropy coding
- entropy_common reads FSE normalized counts and Huffman statistics.
- fse_decompress builds FSE decoding tables and decodes FSE streams.
- fse_compress normalizes counts, writes FSE headers, builds compression tables, and encodes FSE streams.
- huf_compress builds Huffman compression tables, writes table headers, and encodes one- and four-stream Huffman payloads.
- huf_decompress builds Huffman decoding tables and decodes X1 and X2 Huffman streams.
Compression primitives
- hist counts byte frequencies for FSE and Huffman compression.
- zstd_presplit chooses split points for full compression blocks.
Runtime support
- threading provides platform pthread wrappers required by zstd headers.
- pool implements the bounded worker pool used by multithreaded compression.
Dictionary support
- zstd_ddict owns, loads, copies, and references decode dictionaries.

The remaining block compression, general decompression, dictionary-building, legacy, and CLI translation units are still C. They must move before the rewrite is complete. Keeping that boundary explicit prevents a passing hybrid build from being mistaken for the final all-Rust result.

Compatibility boundary

The public ABI continues to come from the existing headers under lib/. Exported Rust functions therefore use C layout and calling conventions. A C source file whose implementation has moved to Rust remains in the original makefile source list as a small shim so header configuration and platform preprocessor behavior stay available during the transition.

The library, test, and program makefiles select an archive directory for the active C configuration: enabled compression/decompression modules, default or forced HUF X1/X2, and the matching Rust target for 32-bit C binaries. The native static archive flattens Rust object members rather than nesting a Rust archive, while the native shared library retains all migrated Rust exports. When the HUF mode changes, the test and program paths also rebuild cached C outputs before linking. This prevents original C tests from using a stale or configuration-incompatible implementation.

Validation

Run focused Rust checks from this directory:

cargo fmt --check
cargo clippy --all-targets -- -D warnings
cargo test --all-targets
cargo build --release

Then run original compatibility tests from the repository root, starting with the narrow target for the component being migrated. For example:

make -C tests fuzzer
./tests/fuzzer -i1 --no-big-tests
make -C tests test-rust-lib-smoke

Broader tests/Makefile targets remain the authoritative integration gates as more of the library and CLI are rewritten.