Add Rust parsing and dispatch for the default, Cover, FastCover, and legacy
dictionary-training modes, including their parameter validation, output
defaults, and help text. Keep file loading behind the existing narrow dibio
bridge while the dictionary algorithms are supplied by the Rust library.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/zstd_cli.rs
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/cli/Cargo.toml
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
Implement the Rust frontend's --show-default-cparams path, including source
and dictionary size discovery, strategy labels, verbose --zstd parameter
reporting, and the decompression-mode diagnostic. Keep the C parameter API
as the source of the selected values so the CLI remains aligned with the
active compression build.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- rustfmt +nightly --edition 2021 rust/src/zstd_cli.rs
- git diff --check
Depends-on: Rust CLI parser, file-selection, and C fileio bridge ports
Set the C file-I/O patch-from flag explicitly when the Rust frontend creates
preferences. The parser still rejects the unported option, but this keeps the
backend in ordinary dictionary mode instead of reading an unspecified field.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --all-features
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- git diff --check
Refs: rust/src/zstd_cli.rs, programs/fileio.c
Move recursive expansion, file-list inputs, output-directory dispatch, and
frame listing into the Rust command-line frontend while keeping directory
traversal and file I/O in the existing C helpers. Preserve the unsigned fast
compression-level parsing and add focused parser coverage for the new modes.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --all-features
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- git diff --check
Refs: rust/src/zstd_cli.rs, programs/util.c, programs/fileio.c
The C command-line parser accepts short-form levels through an unsigned
32-bit conversion before storing them in the signed compression-level field.
The Rust parser used signed parsing for both ordinary levels and --fast,
rejecting 4294967295 even though it represents the valid fast level -1.
Mirror the conversion and retain the fast-level clamp for the attached form.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --all-features
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- make -C programs zstd
- ./programs/zstd --fast -4294967295 tests/playTests.sh -o /tmp/zstd-fast-test
- ./programs/zstd --fast=4294967295 tests/playTests.sh -o /tmp/zstd-fast-test
Refs: tests/playTests.sh maximum fast-level cases
Move the implementation of programs/benchfn.c into rust/src/benchfn.rs and
wire benchmark mode (-b/-e/-i) into the Rust CLI frontend, which previously
rejected those options as not yet implemented. `zstd -b1 -i0 FILE` and range
runs like `zstd -b5e6 -i0 FILE` work again, including the synthetic-sample
benchmark when no file is given.
benchfn.rs is a faithful port of the run/timing state machine:
BMK_benchFunction keeps the exact loop accounting (first-loop blockResults
and errorFn checks, dstSize summed on the first loop only, 0xE5 warm-up of
result buffers, nbLoops minimum of 1) and BMK_benchTimedFn keeps the same
convergence behavior (x10 workload growth for short runs, budget-based
nbLoops estimation, runs below half the run budget re-tried rather than
reported, best qualifying run returned). Arithmetic that C leaves to
unsigned wrap-around uses wrapping operations so debug builds cannot panic
where release C would wrap.
ABI notes: BMK_runTime_t and BMK_runOutcome_t are returned by value across
the C boundary and BMK_benchParams_t is passed by value, so all three are
repr(C) mirrors of benchfn.h; their field offsets are pinned by const
asserts in Rust and matching C static asserts in the benchfn.c shim, which
is now declaration-only. BMK_timedFnState_t stays opaque, fits the 64-byte
BMK_timedFnState_shell (compile-time checked), and is malloc/free-managed
so creation and destruction remain interchangeable with C callers.
The CLI parses -b (bench mode), -e (range end, digits attach directly,
defaulting to 0 like readU32FromChar) and -i (duration in seconds), then
dispatches through a new ZSTD_rust_cli_bench bridge in the zstdcli.c shim.
The bridge exists because benchmark availability is a C preprocessor
property (ZSTD_NOBENCH): orchestration and reporting stay in C benchzstd.c,
stripped variants (zstd-small, zstd-compress, zstd-decompress) compile the
stub branch and report "benchmark mode is not available in this build", and
the Rust side never references benchmark symbols directly. Level clamping
against ZSTD_maxCLevel() happens in the bridge, where the symbol is
guaranteed to exist whenever benchmarking is compiled in. -T selects the
worker count, defaulting to single-threaded like the C bench path; -S
(separate files) and --priority=rt remain unimplemented.
Makefile updates only extend the Rust source prerequisite lists with
benchfn.rs; the helpers-archive plumbing from the timefn commit already
links fullbench(-lib/-dll/32) and paramgrill, the benchfn consumers among
the C tests. Original C test sources are untouched.
Known pre-existing issues, unchanged by this commit: tests/fullbench-lib
fails to link at the base commit too (libzstd.a precedes fullbench.c in its
link line), and the cli-tests basic/help.sh, compression/levels.sh,
compression/golden.sh, and decompression/pass-through.sh scripts fail
identically with a base-commit binary because the Rust CLI frontend is
still a partial reimplementation.
Test Plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets -- -D warnings
&& cargo test --all-targets && cargo build --release
- cd rust/cli && cargo fmt --check && cargo clippy --all-targets -- -D
warnings && cargo test --all-targets; repeat tests with
--no-default-features plus features compression / decompression / (none)
- make -C programs zstd; ./programs/zstd -b1 -i0 lib/common/xxhash.c;
./programs/zstd -b5e6 -i0 programs/fileio.c; ./programs/zstd -b1 -i0
(synthetic); echo roundtrip via zstd | zstd -d
- make -C programs zstd-small zstd-compress zstd-decompress zstd-nolegacy
zstd-dictBuilder; zstd-small -b reports benchmark unavailable; compress/
decompress roundtrip across the split binaries
- make -C tests fullbench fuzzer zstreamtest paramgrill decodecorpus
poolTests fullbench32 fuzzer32; ./tests/fullbench -i0 (exercises Rust
BMK_benchTimedFn from C); ./tests/fullbench32 -i0; ./tests/fuzzer -i1
--no-big-tests; ./tests/poolTests; make -C tests test-rust-lib-smoke
- cli-tests subset: basic/version.sh, compression/basic.sh,
compression/multiple-files.sh pass; failing scripts match the base commit
Refs: rust/README.md
The Rust CLI frontend rejected --single-thread, but tests/playTests.sh
uses it 28 times, so the flag is required before the original CLI test
suite can gate the migration.
Mirror the C zstdcli.c semantics: --single-thread pins zero workers and
sets a latch that suppresses the automatic core-count resolution, so
fileio receives nbWorkers == 0 and runs its single-thread streaming
mode (slightly different from -T1, which uses one worker thread). A
bare zero from -T0 or the zstdmt program name still auto-detects the
core count, and a later -T# overrides the pinned worker count while the
latch stays set, exactly as the C variable pair behaved.
Test plan:
- cd rust/cli && cargo clippy --all-targets -- -D warnings && cargo
test --all-targets (plus the compression-only and decompression-only
feature matrices); new parser tests cover the flag, the -T override
order, and rejection of an attached value.
- make -C programs zstd, then: --single-thread -3/-19 round-trips
against COPYING via cmp; --single-thread=1 fails with "does not take
an argument".
Move long-distance matching and high-level decompression from C shims into
Rust. The decoder now owns context, dictionary, parameter, one-shot, and
buffered streaming state while C retains allocation/configuration, legacy,
and trace leaves.
Move CLI parsing, safety policy, and dispatch into a separate Rust static
archive. Keeping it separate prevents library builds from retaining FIO
symbols, while C continues to own file opening, replacement, and I/O.
Program targets now select matching compression/decompression archives.
The remaining C boundary is intentional: high-level compression, optimal
parsing, dictionary building, legacy callbacks, and CLI file I/O still need
migration.
Test Plan:
- cargo test --all-targets (native and i686)
- cargo test --all-targets in rust/cli (native and i686)
- CLI crate compression-only and decompression-only feature tests
- native and i686 fuzzer/zstreamtest runs, plus legacy and dictionary tests
- ZSTD_C_PREDICT and ZSTD_HEAPMODE=0 fuzzer coverage
- library, dynamic-link, and program-target build/round-trip matrix
Refs: rust/README.md
ddidderr