Move mixed-format decompression status mapping and successful-file finalization
into Rust. C keeps the diagnostic and private FIO context callbacks, while
Rust decides pass-through/error behavior and invokes final accounting only for
a successful decoded file.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo test --manifest-path rust/cli/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C programs -j2 zstd
- make -B -C tests -j2 test-cli-tests
Move the hidden fullbench block-decoder entrypoints and the public
ZSTD_decompressBlock compatibility wrapper into Rust. The C translation unit
now only projects the configuration-dependent private DCtx fields into the
Rust block context, preserving the existing ABI and decoder feature modes.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-rust-lib-smoke
- make -B -C tests -j2 test-cli-tests
Move multithreaded input-range selection and overlap decisions into Rust using
an explicit scalar projection. C retains the round-buffer mutation, LDM
synchronization, prefix copying, and private job state, while Rust decides
whether a reusable section is available and where it begins.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- make -B -C tests -j2 test-rust-lib-smoke
Move the external-sequence-and-literals block loop into the Rust compression
module while preserving the C-owned CCtx initialization and sequence
conversion callback. Rust now owns block summaries, literal cursor movement,
entropy emission, block framing, repcode state updates, and completion
validation; C remains the private-context ABI adapter.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
Document that the Rust CLI now owns the gzip, xz/lzma, and LZ4 codec leaves as
well as MT job creation, while the generic frame-compression state machine and
remaining MT scheduling/outer-flush orchestration remain C-owned. Keep the
hybrid boundary explicit so compatibility-suite success is not mistaken for
completion of the full rewrite.
Test Plan:
- git diff --cached --check
- reviewed rust/README.md against the committed Rust/C ownership boundary
Document that the frame epilogue, produced-output flush policy, and gzip codec
leaf now execute in Rust. Narrow the remaining C-owned list to the generic
compression state machine, MT scheduling/outer flush orchestration, and the
xz/lzma/lz4 codec leaves so the README remains an accurate progress boundary.
Test Plan:
- git diff --cached --check
The migration map still described the optimal parser and benchmark orchestration
as C-owned, and its remaining-work paragraph did not distinguish the newly
ported high-level leaves from the still-private frame and MT orchestration.
That made the README under-report Rust ownership and overstate some remaining
C work.
Update the component map to identify `zstd_opt` price/parse ownership and Rust
benchmark orchestration, and describe the current C boundary as the remaining
frame setup/flush scheduling and format-specific codec leaves. Keep the
hybrid-build warning explicit until those larger orchestration layers move.
Test Plan:
- `git diff --cached --check` -- passed.
- Rust library and CLI Clippy default, benches, and tests -- passed before staging.
- Native library, program, fuzz, CLI, and upstream test matrices -- passed before staging.
`FIO_compressFilename_internal()` still selected the zstd, gzip, xz/lzma, and
lz4 paths in a C-owned format switch and updated aggregate byte accounting
around those callbacks. That left the mixed-format CLI dispatch loop outside
the Rust file-I/O orchestration even though the pools and decompression path
were already projected there.
Add a callback table for the codec leaves and move format selection, optional
codec status handling, post-success accounting, and progress-hook ordering to
Rust. The existing C codec implementations, diagnostics, elapsed-time
formatting, and resource ownership remain unchanged; unsupported optional
formats return explicit statuses so the C wrapper preserves its original
EXM_THROW messages.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- Native `test-cli-tests` -- all 41 passed.
- Native `test-zstd` -- passed, including async zstd/gzip/xz/lzma/lz4 round trips.
- `make -B -C programs -j2 zstd` and `make -B -C tests/fuzz -j2 all` -- passed.
Legacy detection, frame-size queries, one-shot decode, and buffered streaming
were still routed through the C header dispatcher even though every historical
v0.1 through v0.7 decoder is already ported under Rust feature gates. The C
implementation also duplicated the support-level boundary independently from
the Rust archive selection.
Move the dispatcher and stream lifecycle to Rust using the existing DCtx field
projection. Rust now selects the version, preserves the null-buffer and
position semantics, initializes and frees buffered legacy contexts, and calls
the frozen per-version modules. C retains only the configuration leaf that
reports ZSTD_LEGACY_SUPPORT plus the private DCtx layout and trace helpers, so
library, program, and test builds can still choose different support levels.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder,legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07 --all-targets -- --test-threads=1` -- 528 passed.
- Native `test-legacy`, `test-invalidDictionaries`, `test-decodecorpus`, and `test-zstd` -- passed.
- Strict C syntax checks with legacy support 0, 4, 5, and 7 -- passed.
- `cargo clippy` default, benches, and tests for library and CLI -- passed.
The multithreaded compressor still handled the terminal empty-job case in C,
including the decision to acquire a buffer, emit the final empty block, and
clear the consumed source range. That left a small but stateful branch outside
the Rust job helpers and made allocation-failure behavior difficult to test.
Project the terminal job and result through a stable C ABI, then let Rust own
that branch's assertions, buffer callback, block-header serialization, and
failure-preserving result. C remains responsible for its private job layout
and applies the returned buffer, source, and compressed-size fields exactly as
before.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- Native multithreaded cases in `test-fuzzer`, `test-zstream`, `test-cli-tests`, and `test-zstd` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1` -- passed.
The public and deprecated block APIs previously entered a C-owned
ZSTD_compressContinue_internal routine, which mixed frame-header sequencing,
window maintenance, block dispatch, and frame-size accounting with the private
CCtx match state. That made the high-level compression boundary both harder
to test and easy to diverge from the already-ported frame-chunk body.
Add an explicit C/Rust projection containing only scalar state and callback
slots. Rust now owns stage validation and transitions, header sequencing,
frame-versus-block dispatch, and consumed/produced progression, while C
callbacks retain access to the private match state and context-sensitive block
operations. The same path now covers compressContinue, deprecated block
compression, and compressEnd without exposing the unstable CCtx layout.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder,legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07 --all-targets -- --test-threads=1` -- 528 passed.
- Native `test-fuzzer`, `test-zstream`, `test-decodecorpus`, `test-cli-tests`, and `test-zstd` -- passed.
- `cargo clippy` default, benches, and tests for library and CLI -- passed.
Port sequence-decoder selection and offset-history policy into Rust. The
Rust block decoder now handles short/long selection, offset-table analysis,
history thresholds, and prefetch sequencing; C projects only the decoder mode
and private context fields required by the ABI.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0
- make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1
- make -B -C tests -j2 test-zstd
The current toolchain promoted two test-only patterns to hard errors under
`-D warnings`: integer-to-pointer casts used as non-null dictionary-handle
sentinels, and a repeat/take construction that Clippy now replaces with
`repeat_n`. The casts never represented addresses and were only placeholders
for ownership fields that the cleanup functions must clear. Use the standard
dangling-pointer constructor for those sentinels and the equivalent concise
iterator for the generated test input. Runtime behavior and the ABI are
unchanged, while the repository's Rust lint gate is warning-free again.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo +nightly fmt --manifest-path rust/cli/Cargo.toml` -- passed
Move the post-sequence-store body of ZSTD_compressBlock_internal behind a
small C/Rust state projection. C continues to build the sequence store and
handle the no-compress and sequence-producer error paths, while Rust now owns
sequence collection, entropy emission, the legacy non-first-frame RLE gate,
compressed-block confirmation, and offcode repeat cleanup. Remove the C
wrappers that became dead after those leaves moved behind the Rust body.
The first integration run exposed that leaving the old C finalization label in
place confirmed compressed block state twice, undoing Rust's pointer swap and
breaking a later sparse-file checksum. The C wrapper now returns directly for
the Rust-owned path and retains only its C-owned no-compress cleanup.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo check --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-zstd
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
- make -B -C tests/fuzz -j2 all
- make -B -C tests/fuzz -j2 sequence_compression_api
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings (pre-existing manual_repeat_n failure)
- cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings (pre-existing manual_repeat_n failure)
Move the post-split partition loop out of zstd_compress.c while keeping the
private CCtx, matchfinder sequence-store construction, and split discovery in
C. Rust now receives a layout-asserted projection containing sequence-store
views, block-state slots, workspace and scalar policy, then mirrors the C
loop's dRep/cRep histories, final-literal accounting, repeated single-block
serialization, and final dRep publication.
Remove the obsolete C sequence-store/count/chunk wrappers and the debug-only
C size-estimation path that was coupled to the old loop. Add a focused Rust
fixture covering partition payloads and final literals, and document the new
ownership boundary for the split search and emission paths.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1 (448 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings (passed)
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings (passed)
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (passed)
- cargo clippy --tests/--benches remain blocked only by the pre-existing manual_repeat_n lint in one_shot_promotes_nonfirst_rle_blocks
- make -B -C lib -j2 lib (passed)
- make -B -C tests -j2 test-zstd (passed)
- make -B -C tests -j2 test-cli-tests (41 passed)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (passed)
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer (252 passed)
- make -B -C tests/fuzz -j2 all and sequence_compression_api (passed)
Clarify that the decoder's private C layout remains configuration-sensitive in
C while Rust owns storage allocation, state-machine behavior, and the public
ABI. This keeps the module-level migration boundary consistent with the
allocator bridge now implemented in Rust.
Test Plan:
- `git diff --cached --check` -- passed
The component map still described decoder context allocation as a C-owned
responsibility after the allocation and custom-memory bridges moved into Rust.
Update the boundary description to distinguish the C-defined context layout and
configuration-sensitive platform, legacy, and trace leaves from Rust-owned
storage allocation and allocator dispatch.
Test Plan:
- `git diff --cached --check` -- passed
Document that the Rust dictionary-builder modules now own sample analysis,
training, and dictionary assembly, with the original C files retained only as
ABI/build shims. Remove the stale statement that only suffix-array
construction has migrated and keep the remaining C boundary focused on
compression orchestration, benchmark reporting, and format-specific CLI I/O.
Test Plan:
- git diff --cached --check
- make -C tests -j2 test-zstd (passed before this documentation-only commit)
Move the target-compressed-block body out of zstd_compress.c while keeping
sequence-store construction, matchfinding, context lifetime, and the outer
offcode repeat-mode cleanup in C. Rust now owns the target policy, repeated
block decision, Rust superblock call, compressed-state confirmation, and raw
fallback. The narrow C projection asserts the cross-language layout instead
of exposing ZSTD_CCtx.
Add focused tests for first-block RLE suppression, RLE output, superblock
errors, raw fallback, and state-slot swapping. Remove the C-only maybeRLE
wrapper that became unreachable after the extraction.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
The CLI already delegated one zstd frame at a time to Rust, but the C
FIO_decompressFrames loop still owned repeated-frame dispatch. That left
concatenated zstd streams split across the language boundary and made the
Rust frame helper unable to preserve the next mixed-format header itself.
Add a Rust multi-frame adapter that repeatedly probes four buffered bytes,
uses the existing one-frame decoder, accumulates decoded output and progress,
and leaves a following non-zstd header or short trailing input untouched for
C's format dispatcher. Keep C responsible for probing the first format,
non-zstd dispatch, diagnostics, cleanup, and final file accounting. Decoder
errors still preserve the current input for FIO_zstdErrorHelp().
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib fileio_asyncio -- --test-threads=1` -- 20 passed
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- `make -B -C tests -j2 test-cli-tests` -- all 41 passed
- `git diff --cached --check` -- passed
- Full root `cargo clippy --all-targets/--benches/--tests -- -D warnings` remains blocked by the pre-existing `manual_repeat_n` warning in `rust/src/zstd_compress.rs` test code; no unrelated change was made.
Move the per-block body of ZSTD_compressSequences_internal behind an explicit
Rust projection. The C wrapper still owns context initialization, public API
validation, frame headers, checksums, and the private CCtx layout. Rust now
owns block sizing and sequence transfer, sequence-store reset, entropy
compression, raw/RLE/compressed block selection, block headers, repcode/state
swapping, repeat-mode transition, and first-block handling.
The bridge passes only the sequence store, block-state pointer slots, workspace,
policy scalars, dictionary size, and the isFirstBlock slot. It does not pass a
CCtx or C callback across the ABI. The tests cover empty-block headers,
capacity errors, first-block RLE restrictions, and entropy fallback decisions.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -C tests -j2 test-fuzzer (covers ZSTD_compressSequences at fuzzer test 190)
- git diff --cached --check
Move the bounded multithreaded output-publication kernel into Rust. Rust now
calculates the flush amount, copies the selected job bytes, and returns the
updated output and job offsets through an explicit C-layout result. C retains
mutex and condition-variable handling, checksum insertion, job retirement,
progress accounting, and the terminal state machine.
The Rust helper validates the scalar bounds before copying and has focused
coverage for empty output space, offset partial flushes, repeated flushes,
complete flushes, sentinels, and invalid bounds.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (blocked by concurrent fileio worker formatting its uncommitted file)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (worker)
- git diff --cached --check
Update the Rust rewrite documentation to match the current implementation
boundary. The legacy decoder set is complete, and Rust now owns several
CLI file-I/O policy and resource leaves while C retains format dispatch,
metadata, and remaining orchestration. Keep the stated boundary explicit so
passing hybrid tests do not imply that the full rewrite is finished.
Test Plan:
- git diff --cached --check
ZSTDMT_compressionJob kept the chunking kernel in C while Rust already owned
most of the MT pool and progress helpers. The loop mixed compression calls
with mutex and condition-variable publication, making it difficult to test
its exact final-chunk and partial-tail behavior independently.
Move only the chunk-processing kernel into Rust. It calls the existing C
compression entry points through an explicit ABI, reports the final block size
or error in a repr(C) result, and invokes a C progress callback after each
successful intermediate chunk. C retains worker setup, serial and raw-sequence
state, synchronization, error publication, cleanup, and final completion
reporting. Mock compressors cover empty, exact, partial, progress, and error
boundaries without mirroring private MT job state in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress --
--test-threads=1` -- passed (44 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream` -- passed,
including multi-threaded dictionary/checksum coverage.
FIO_openDstFile still performed destination classification, platform-specific
file creation, stdio buffering, and overwrite preparation in C, even though
source opening and removal already used Rust filesystem leaves. That made the
CLI backend's most important output safety path split across two implementations.
Add a Rust destination leaf with a small status ABI. Rust owns sentinel and
same-file classification, regular-file detection, platform binary open and
fdopen, truncation, and buffering. C keeps the user-facing diagnostics,
sparse-mode preference mutation, overwrite prompt, and existing remove-file
wrapper; it performs a second Rust open after an accepted overwrite decision so
no prompt or policy is duplicated. Existing file contents are left untouched
until that C-owned decision is complete.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1`
-- passed (411 tests, including destination classification/open tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings`
-- passed.
- `cargo clippy --manifest-path rust/cli/Cargo.toml --lib
--no-default-features --features cli,compression,decompression,benchmark,
dict-builder -- -D warnings` -- passed.
- Nightly fmt checks for both Rust manifests -- passed.
- `make -B -C tests -j2 test-cli-tests` -- passed (41 tests).
Compression-parameter table selection and adjustment were already implemented
as Rust leaves, but the internal and public ZSTD_getCParams/ZSTD_getParams
helpers still assembled those results in the C translation unit. That left
source-size and dictionary-mode policy duplicated at the C/Rust boundary.
Add Rust implementations for the internal and public getter policies. The
internal helpers preserve a zero source size as a known empty input, while the
public helpers retain the API rule that zero means unknown. The C side keeps
only the build-specific exclusion-mask construction and thin ABI adapters, so
reduced builds continue to select the same available strategy cascade.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1`
-- passed (411 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings`
-- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- Focused tests cover public zero-to-unknown translation, internal zero
semantics, default frame parameters, and dictionary attachment modes.
The Rust simple-compression entry points already owned the dispatch, but
ZSTD_rust_compressCCtxStrategy still called the C parameter cascade to decide
whether a fast Rust frame path was safe. That left a small policy leaf in the
C implementation and made the dispatch boundary harder to test in isolation.
Implement the helper in Rust by using the existing Rust table-selection and
C-parameter-adjustment functions with the same source-size, dictionary-mode,
and automatic-adjustment inputs as the C code. Leave only the C ABI declaration
and add tests for the table thresholds, default-level behavior, and negative
fast-level behavior. The private context reset and fallback compression paths
remain unchanged.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress --
--test-threads=1` -- passed (234 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests -j2 test-rust-lib-smoke` -- passed.
- Full-target clippy remains blocked by pre-existing test-only
`manual_repeat_n` and `manual_dangling_ptr` warnings outside this seam.
The C benchmark translation unit still contained the complete BMK
implementation, so program builds and paramgrill could silently keep using a
second C implementation even though rust/src/benchzstd.rs already provided the
same ABI. Replace that duplicate with a declaration-only shim and retain
compile-time checks for the by-value result and parameter layouts.
The standalone test tools do not link the full CLI archive. Keep their
helpers archive free of the benchmark and trace-only objects, and add a
separate benchmark archive for paramgrill so the shim remains linkable without
pulling program-only trace dependencies into datagen and similar targets.
Test Plan:
- `cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark,dict-builder --lib -- --test-threads=1` -- 165 passed
- `make -B -C programs -j2 zstd` -- passed
- `./programs/zstd -b1 tests/hello` -- passed
- `make -B -C tests paramgrill` -- passed
- `./tests/paramgrill -S tests/hello` -- passed
- `make -C tests -j2 test-fullbench` -- passed
- targeted Rust clippy and nightly rustfmt checks -- passed
- full all-target clippy remains blocked by pre-existing test-only lints in `zstd_compress.rs` and `fileio_backend.rs`
The library reports an absolute maximum compression level of 22, but the CLI
advertises levels 1 through 19 unless `--ultra` is selected. The Rust frontend
used the absolute maximum for both help lines, diverging from the original C
contract and failing the help golden test. Separate the normal help ceiling
from the ultra ceiling and add a focused regression assertion.
Test Plan:
- `cargo fmt --manifest-path rust/cli/Cargo.toml -- --check` -- passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --lib -- -D warnings` -- passed
- CLI Rust tests -- 162 passed
- `make -C tests -j2 test-cli-tests` -- all 41 passed
- `git diff --check` -- passed
The v0.2 decoder implementation and C boundary were already Rust-owned, but
its public error predicate, frame-size sentinel behavior, and opaque streaming
context lifecycle lacked focused Rust coverage. Add tests for malformed input,
content-size error reporting, one-shot decoding, reset behavior, and idempotent
context release so future legacy changes are checked at the ABI boundary.
Test Plan:
- Focused v0.2 Cargo tests under both feature sets -- 6 passed each
- `make -C tests -j2 test-legacy` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- Scoped `git diff --check` -- passed
The CLI trace translation unit still contained the complete CSV writer and
strong callback implementations even though the Rust CLI archive already had
the equivalent trace module. Reduce the C file to its public-header shim and
make the Rust implementation authoritative, preserving the exact CSV header,
version assertion, timing fields, and no-trace build behavior. Focused tests
cover header creation and the version-mismatch assertion.
Test Plan:
- CLI tests: 161 default and 127 reduced-feature -- passed
- Focused trace tests: 3/3 in both configurations -- passed
- `make -B -C programs -j2` for all CLI variants -- passed
- Trace compression/decompression smoke test -- passed
- C shim compile with and without `ZSTD_NOTRACE` -- passed
- Clippy, nightly rustfmt, and `git diff --check` -- passed
The optimal block splitter stores up to 196 split points and then needs one
additional partition entry for the terminal sequence boundary consumed by the
C block-emission loop. The previous array declaration provided only the split
slots, so a maximally partitioned block could write the terminal boundary past
the projected Rust/C state. Reserve the extra entry and document the layout;
retain focused tests for equal-cost and estimation-error cases so the splitter
continues to avoid unnecessary or invalid partitions.
Test Plan:
- Focused block-split Rust tests -- 7 passed
- `make -B -C lib -j2 lib` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- Scoped `git diff --check` -- passed
The verbose compression-preference summary was still formatted by the C
file-I/O orchestration layer even though the preference object and the rest of
its policy helpers already lived in Rust. Move the option selection and exact
summary formatting into the Rust CLI archive, keeping the public C function as
a narrow assertion-and-dispatch shim. The Rust formatter preserves the legacy
option spellings, default memory limit, integer casts, and stderr output, with
a focused string-format regression test.
Test Plan:
- `cargo fmt --manifest-path rust/cli/Cargo.toml -- --check` -- passed
- `cargo test --manifest-path rust/cli/Cargo.toml --lib fileio_prefs -- --test-threads=1` -- 46 passed
- `git diff --check` -- passed
Move the active-stage check and pledged-size update behind a scalar Rust ABI
leaf while keeping the CCtx layout and public C entry point in C. Preserve the
stage_wrong error encoding and the unsigned wrap behavior at U64::MAX.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (80 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C programs -j2 zstd
- git diff --check
Route FIO_freeDict through the Rust filesystem backend for malloc-backed and
mapped dictionaries. Keep enum validation in C, pair Windows views with their
file handles, support the non-POSIX malloc fallback, and clear every ownership
field so repeated cleanup remains harmless.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --lib fileio_backend -- --test-threads=1 (25 passed)
- cargo clippy --manifest-path rust/cli/Cargo.toml --lib -- -D warnings
- rustfmt +nightly --edition 2021 rust/src/fileio_backend.rs --check
- make -C programs -j2 zstd
- git diff --check
Keep dictionary-training code out of compressor-only and decompressor-only CLI
archives, where its ZDICT symbols are intentionally absent. Add a dedicated
Rust CLI archive for zstd-dictBuilder, enable the builder feature only for the
full and dictionary-builder programs, and make reduced-feature help use the
feature-independent max-level helper.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --lib -- --test-threads=1 (158 passed)
- cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,dict-builder --lib -- --test-threads=1 (124 passed)
- cargo clippy --manifest-path rust/cli/Cargo.toml --lib -- -D warnings
- cargo clippy --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,dict-builder --lib -- -D warnings
- make -C programs -j2 zstd zstd-small zstd-frugal zstd-decompress zstd-compress zstd-dictBuilder
- git diff --check
Move the platform-specific dictionary mapping and release operations behind the
Rust CLI archive while keeping size policy, diagnostics, and FIO_Dict_t
ownership decisions in C. POSIX descriptors and Windows handles are cleaned up
on every failure path, and zero-length dictionaries use an owned sentinel so
the existing non-null dictionary invariant remains valid.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --lib fileio_backend (24 passed)
- cargo clippy --manifest-path rust/cli/Cargo.toml --lib -- -D warnings
- make -C programs -j2 zstd
- --mmap-dict compression/decompression round trip with programs/README.md
- programs/zstd --mmap-dict --test on the generated frame
- rustfmt +nightly --edition 2021 rust/src/fileio_backend.rs --check
- git diff --check
Keep the private ZSTD_window_t layout and pointer-difference calculation in C,
then route the size_t-to-U32 conversion and paired limit writes through a small
Rust ABI leaf. This removes the inline C window-clear implementation while
preserving its overflow behavior at both single-threaded and multithreaded
call sites.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (77 passed)
- make -C programs -j2 zstd
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C tests -j2 test-cli-tests (41 passed)
- make -C tests -j2 test-legacy test-invalidDictionaries test-decodecorpus test-rust-lib-smoke (passed)
- git diff --check
ddidderr