feat(rust): migrate high-level runtime paths

Move long-distance matching and high-level decompression from C shims into
Rust. The decoder now owns context, dictionary, parameter, one-shot, and
buffered streaming state while C retains allocation/configuration, legacy,
and trace leaves.

Move CLI parsing, safety policy, and dispatch into a separate Rust static
archive. Keeping it separate prevents library builds from retaining FIO
symbols, while C continues to own file opening, replacement, and I/O.
Program targets now select matching compression/decompression archives.

The remaining C boundary is intentional: high-level compression, optimal
parsing, dictionary building, legacy callbacks, and CLI file I/O still need
migration.

Test Plan:
- cargo test --all-targets (native and i686)
- cargo test --all-targets in rust/cli (native and i686)
- CLI crate compression-only and decompression-only feature tests
- native and i686 fuzzer/zstreamtest runs, plus legacy and dictionary tests
- ZSTD_C_PREDICT and ZSTD_HEAPMODE=0 fuzzer coverage
- library, dynamic-link, and program-target build/round-trip matrix

Refs: rust/README.md
This commit is contained in:
2026-07-11 09:03:41 +02:00
parent 031592da1e
commit fe7e24c770
14 changed files with 6886 additions and 4651 deletions
+25 -4
View File
@@ -37,6 +37,8 @@ zstd ABI:
including row-based and dictionary search variants.
- `zstd_opt_tree` maintains the binary-tree index used by optimal matching;
the dynamic-programming optimal parser itself remains in C for now.
- `zstd_ldm` implements long-distance-match parameter selection, table
maintenance, sequence generation, and sequence consumption.
- Runtime support
- `threading` provides platform pthread wrappers required by zstd headers.
- `pool` implements the bounded worker pool used by multithreaded compression.
@@ -45,11 +47,20 @@ zstd ABI:
- Block decompression
- `zstd_decompress_block` decodes literal and sequence sections, maintains
FSE/Huffman repeat state, and executes compressed-block sequences.
- `zstd_decompress` owns the public decompression context, one-shot,
dictionary, parameter, and streaming state machines. Its C shim retains
configuration-dependent context allocation plus legacy and trace leaves.
- Command-line frontend
- `zstd_cli` owns the Rust parser, safety policy, and dispatch. It is built
by the separate `cli/` static-library package only for program archives,
so library builds do not acquire program-only dependencies. The C
`fileio` backend still owns file opening, safe replacement, sparse writes,
metadata, and streaming I/O.
The optimal block matcher, high-level frame decompression, dictionary-building,
legacy, and CLI translation units are still C. They must move before the
rewrite is complete. Keeping that boundary explicit prevents a passing hybrid
build from being mistaken for the final all-Rust result.
The optimal block matcher, high-level frame compression, dictionary-building,
legacy decoding callbacks, and the CLI file-I/O backend are still C. They must
move before the rewrite is complete. Keeping that boundary explicit prevents a
passing hybrid build from being mistaken for the final all-Rust result.
## Compatibility boundary
@@ -79,6 +90,16 @@ cargo test --all-targets
cargo build --release
```
The program-only Rust archive has its own feature matrix and should be checked
from `rust/cli` as well:
```sh
cargo clippy --all-targets -- -D warnings
cargo test --all-targets
cargo test --no-default-features --features compression --all-targets
cargo test --no-default-features --features decompression --all-targets
```
Then run original compatibility tests from the repository root, starting with
the narrow target for the component being migrated. For example: