ZSTD_estimateCCtxSize_internal() evaluates four fixed source-size tiers and
must retain every C-owned parameter and sizing call, including raw error-shaped
size_t results. Store those four results in C, then delegate only the final
unsigned maximum comparison to a scalar Rust ABI helper. The outer monotonic
compression-level budget and the separate CStream estimation path remain
unchanged.
The Rust unit tests cover zero, ordinary, equal, SIZE_MAX, and encoded error
values to document that the helper compares raw size_t values without arithmetic
or error interpretation.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- Same clippy command with `--benches` -- passed before and after formatting
- Same clippy command with `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 328 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named plus 5,923 and 9,247 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build retains the pre-existing initializer-string warning at
tests/zstreamtest.c:1899.
ZSTD_literalsCompressionIsDisabled() previously switched on the private
ZSTD_CCtx_params layout in a header inline helper. Replace that switch with a
scalar ABI call that extracts only literalCompressionMode, strategy, and
targetLength in C. The Rust helper preserves the enable and disable results,
the fast-strategy auto rule, and C's debug assertion plus release fallthrough
for invalid modes. Keep callers and compression state ownership in C, and
cover the policy truth table and debug invalid-mode invariant in Rust.
Test Plan:
- Compression-feature clippy for the library, benches, and tests before and
after nightly formatting -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- passed (43 tests).
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed (84 named, 5,416, and 8,387
randomized cases).
- `git diff --check` and `git diff --cached --check` -- passed.
The MT rsync initialization previously computed the rolling-hash prime
power through the C-only ZSTD_rollingHash_primePower helper. Move only that
scalar exponentiation behind the existing Rust MT ABI boundary. The C caller
still owns the MT context assignment, RSYNC_LENGTH constant, and surrounding
initialization.
The Rust helper mirrors ZSTD_ipow with wrapping u64 multiplication and applies
u32 wrapping subtraction before widening the exponent, preserving the C
length == 0 behavior. Rolling-hash scanning, rotation, and all other MT state
remain unchanged.
Test Plan:
- Focused rolling_hash_prime_power Rust tests -- passed (2 tests).
- Required clippy/fmt sequence in normal, bench, and test modes -- passed.
- make -B -C lib -j2 lib -- passed.
- make -C tests test-rust-lib-smoke -- passed.
- tests/fuzzer -s4560 -t56 -i57 -v -- passed (57 tests).
- make -C tests -j2 test-zstream -- passed (84 named, 5,305 + 7,351 randomized).
- git diff --check and git diff --cached --check -- passed.
Keep ZSTD_getFrameProgression's multithreaded path and C-owned frame
construction unchanged while moving the single-thread ingested-value
calculation behind the existing scalar C-to-Rust boundary. The Rust helper
accepts the consumed U64 value and buffered size_t value, explicitly converts
the latter to u64, and uses wrapping addition to match C's unsigned
arithmetic. Focused tests cover zero, ordinary, and overflowing inputs.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` -- passed before and after formatting.
- The same clippy command with `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused Rust tests for `frame_progression_ingested` -- 3 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 5,800, and 9,479 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream run retains the pre-existing warning at
`tests/zstreamtest.c:1899` about an unterminated initializer string.
Keep ZSTD_endStream's streaming call, error propagation, multithreaded
minimal estimate, logging, and context state in C. Move only the final
single-thread estimate behind a scalar Rust ABI helper so the frame-ended
branch and raw checksum flag are explicit while size_t arithmetic wraps as
in C.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused end-stream Rust tests -- 4 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 6,457, and 8,692 randomized tests. The pre-existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep the private ZSTD_CCtx and public buffer structures in C while moving the
read-only stable-buffer validation policy behind a scalar ABI. The Rust helper
compares expected and current input pointers and positions without dereferencing
raw pointers, and uses wrapping subtraction for the stable output remainder.
The C wrapper still extracts the context and buffer fields, and the existing
compressStream2 FORWARD_IF_ERROR path remains responsible for propagating the
encoded stability error. The unused endOp parameter is no longer carried into
the validation helper.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed, 309 tests.
- All three requested clippy modes passed before and after `cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed, 84 named plus 5,333 and 8,072 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
- `make -C tests -j2 test-zstream32` -- unavailable because the i686 Rust target is not installed.
Keep ZSTD_selectBlockCompressor's compressor tables, callback pointers, and
final dictionary-mode table lookup in C while moving only the scalar choice of
row-based versus ordinary table index to the Rust parameter-policy module. The
Rust helper reuses the existing row-matchfinder policy, returns 0..2 for row
entries, and returns 3 plus the strategy for ordinary entries. C retains its
public/internal signature, strategy and row-mode assertions, numeric fast
strategy assumption, and all private callback ownership.
Test Plan:
- Rust clippy normal, benches, and tests before and after nightly formatting -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- 41 passed.
- `make -B -C lib -j2 lib` -- passed without new warnings.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests plus 6,708 and 8,980 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- The pre-existing `tests/zstreamtest.c:1899` unterminated-string warning remains.
Keep buffer-pool ownership, allocation, synchronization, and the Rust
pool's saturating accounting in their existing implementations. The C
wrapper still handles NULL, obtains sizeof(*bufPool), and queries the Rust
pool size; Rust now performs only the final size_t addition with wrapping
semantics. The sequence-pool sizing alias continues to call the same C
helper and therefore retains its behavior.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress` -- 29 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- 57 passed
- `make -C tests -j2 test-zstream` -- 84 named, 7,661, and 8,090
randomized tests passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build still emits the pre-existing unterminated-string warning
at tests/zstreamtest.c:1899.
Keep all context-sensitive workspace sizing in C, including intermediate
allocation calculations, match-state and LDM sizing, error propagation, and
diagnostics. Move only the final nine-component neededSpace aggregation
behind a scalar Rust ABI helper so its size_t wrapping behavior is explicit
and independently tested.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress` -- 186 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests and 6,536 plus 7,961 randomized cases passed; the existing unterminated-string warning remains.
- `git diff --cached --check` -- passed.
ZSTDMT_sizeof_CCtx previously performed the complete size_t sum in C,
although every component depends on C-owned MT state and sizing helpers.
Keep the NULL guard, pool and dictionary queries, job-table multiplication,
and round-buffer extraction in C. Pass those eight computed components to a
Rust ABI helper that performs the same ordered wrapping additions.
This keeps MT context layout and allocator ownership on the C side while
moving only the scalar arithmetic across the existing C/Rust boundary.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression`
- Same clippy command with `--benches` and `--tests`, nightly fmt, then all three clippy commands again -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 289 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,268, and 8,862 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains its pre-existing unterminated-string warning at
`tests/zstreamtest.c:1899`.
ZSTDMT_nextInputSizeHint only derives a size_t hint from targetSectionSize
and inBuff.filled. Keep the MT context, public/internal entry point, and
stream orchestration in C, but delegate those two scalars to Rust. The Rust
helper uses wrapping subtraction and preserves the zero-result fallback,
including C's wrapped behavior for invalid overfill.
Test Plan:
- The three compression clippy modes passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- Focused Rust tests passed: 5 `mt_next_input_size_hint` tests.
- `make -B -C lib -j2 lib` passed.
- `make -C tests test-rust-lib-smoke` passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` passed: 57 tests completed.
- `make -C tests -j2 test-zstream` passed: 84 named, 6,659, and 8,055
randomized tests completed.
- `git diff --check` and `git diff --cached --check` passed.
ZSTD_selectSequenceCopier combined block-delimiter validation with its
private C function-pointer dispatch. Move only the scalar mode policy behind
a Rust ABI shim, retaining the function-pointer switch and both sequence
transfer implementations in C. Rust preserves the 0..=1 debug validation and
the release fallback to no-block-delimiters for unsupported values.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` — 281 passed.
- Required clippy checks for the library, benches, and tests passed before and after `cargo +nightly fmt`.
- `make -B -C lib -j2 lib` and `make -B -C tests test-rust-lib-smoke` — passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `tests/fuzzer -s4560 -t56 -i57 -v` — passed.
- `make -C tests -j2 test-zstream` — 84 named, 7,298 standard randomized, and 9,379 new-API randomized cases passed.
- `git diff --check` and `git diff --cached --check` — passed.
Port the final strategy-dependent transform in
ZSTD_dedicatedDictSearch_getCParams to a Rust ABI leaf that accepts and
returns the complete repr(C) compression-parameter struct. Keep C responsible
for ZSTD_getCParams_internal table selection and adjustment, including the
createCDict mode and its existing call order, as well as all dictionary and
context construction. The Rust leaf changes only hashLog for greedy, lazy,
and lazy2; every other and unexpected strategy is preserved unchanged.
Remove the obsolete scalar hash-log C ABI now that the complete-parameter
boundary is used. The focused Rust tests cover every strategy class and
unexpected values, field preservation, wrapping adjustment boundaries, and
the existing reverse transform's subtraction/clamp behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- passed (37 tests)
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --benches` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- passed (84 named tests; randomized phases of 6,557 and 7,884 cases)
- `git diff --check` and `git diff --cached --check` -- passed
Keep ZSTD_assertEqualCParams as a C-local wrapper so both existing callers and
surrounding control flow remain unchanged, but move its seven-field invariant
to the Rust compression-parameter module. The ABI helper takes both
repr(C) parameter structs by value, compares each named field explicitly with
debug_assert_eq!, and remains exported in release builds so the C linkage is
stable while the checks compile out like C assert under NDEBUG.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and
after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- 36 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -B -C tests -j2 test-zstream` -- 84 named and 15,504 randomized
cases passed; the existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep ZSTD_CStream context access and caller-specific streaming behavior in C,
while moving the pure stable-versus-buffered input-buffer policy to a scalar
Rust ABI helper. The helper returns the public repr(C) buffer layout, preserves
stable-mode source, size, and position values including NULL sources, and
returns a zeroed buffer for every other mode.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- the same clippy command with `--benches` and `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 274 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named tests and 7,400 plus 8,519 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
ZSTD_sizeof_localDict only combines the optional local dictionary buffer
size with the already-owned C dictionary size. Keep dictionary ownership and
workspace sizing in C, but pass the three scalar inputs to Rust so the leaf
uses explicit C size_t wrapping semantics. The CCtx NULL sizing guard and
surrounding ownership and call structure remain unchanged.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` (before and after formatting) -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 272 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` and direct `rustLibSmoke` execution -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./fuzzer -s4560 -t56 -i57 -v` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
Port ZSTD_dedicatedDictSearch_revertCParams to a Rust leaf that accepts and
returns the complete repr(C) compression-parameter struct. Keep the existing
C pointer wrapper and call order before ZSTD_adjustCParams_internal, while
changing only hashLog for greedy, lazy, and lazy2 strategies.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression (and --benches/--tests, rerun after fmt)
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- make -B -C tests -j2 fuzzer
- tests/fuzzer -s4560 -t47 -i48 -v
- tests/fuzzer -s4560 -t57 -i1057 -v
- make -B -C tests -j2 test-zstream
- git diff --check
The public sequence compression paths still built compressed block headers
locally in C, leaving two byte-formatting implementations after the frame-chunk
path was migrated. Route both compressed-block branches through the existing
Rust serializer while retaining their C-side sequence, entropy, and capacity
logic.
Test Plan:
- `make -B -C lib -j2 lib` -- passed
- `make -B -C tests -j2 fuzzer` -- passed
- `tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- `git diff --cached --check` -- passed
The frame-chunk path still encoded compressed block headers directly in C even
though the equivalent single-block path had moved to the Rust serializer.
Route this remaining call site through the same ABI leaf so both paths retain
one byte-formatting implementation and identical RLE/compressed handling.
Test Plan:
- `make -B -C lib -j2 lib` -- passed
- `make -B -C tests -j2 fuzzer` -- passed
- `tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- `git diff --cached --check` -- passed
Keep ZSTD_invalidateRepCodes responsible for extracting the private previous
block state and checking that the match window has no external dictionary.
Delegate only the fixed three-entry repcode reset through a narrow pointer ABI,
with a C assertion preserving the Rust side's ZSTD_REP_NUM assumption.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress::tests::invalidate_rep_codes_clears_all_entries` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Required clippy, nightly fmt, and diff checks -- passed
ZSTD_copyCDictTableIntoCCtx only transformed tagged CDict match-table
entries or copied untagged entries; context-owned table selection remains in
C. Keep that policy and both call sites in C, forwarding the selected flag
through a narrow ABI. Rust removes the frozen eight-bit short-cache tag or
uses copy_nonoverlapping for C memcpy-equivalent copying without adding
validation.
Test Plan:
- Focused and full compression Rust tests passed (2 and 267 tests).
- `make -B -C lib -j2 lib` passed.
- `make -C tests -j2 fuzzer` passed.
- `./tests/fuzzer -s4560 -t47 -i48 -v` passed.
- Clippy normal/benches/tests before and after `cargo +nightly fmt` passed.
- `git diff --check` and `git diff --cached --check` passed.
Keep C responsible for extracting the cdict and CCtx scalar fields while
moving the attach-versus-copy mode decision into the Rust parameter module.
The Rust ABI short-circuits absent cdicts and reuses the established
attachment predicate, preserving unknown-size, force-attach, force-copy, and
force-window behavior.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression
cparam_mode_preserves_cdict_presence_and_attachment_policy -- passed
- make -B -C lib -j2 lib -- passed
- make -C tests -j2 fuzzer -- passed
- ./tests/fuzzer -s4560 -t47 -i48 -v -- passed
- Required clippy normal/benches/tests, nightly fmt, and diff checks -- passed
The public streaming-size functions still calculated their values in the C
compressor entry point. Keep those public symbols and their size_t ABI in C,
but route the pure formulas through zstd_compress_api.rs. Rust uses usize for
the C size_t-equivalent constants and reuses its compressBound implementation;
the fixed block and header sizes make the output helper exactly 131591 bytes.
Test Plan:
- Focused Rust cstream size test -- passed (1 test)
- make -B -C lib -j2 lib -- passed
- make -C tests -j2 fuzzer -- passed
- ./tests/fuzzer -s4560 -t47 -i48 -v -- passed
- Required clippy, nightly fmt, and staged diff checks -- passed
Keep the private CDict and CCtx parameter layouts in C while passing only the
five scalar inputs needed by the dictionary-attachment heuristic. Rust now
owns the exact per-strategy cutoffs and preserves unknown-size handling,
dedicated-search short-circuiting, force-attach and force-copy precedence, and
the force-window prohibition. The existing C helper and reset/copy call paths
remain unchanged.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression dictionary_attachment` -- passed (2 tests)
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Rust clippy for the library, benches, and tests before and after nightly formatting -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `git diff --cached --check` -- passed
Keep dedicated-dictionary strategy dispatch, parameter selection, and call-site
control flow in C while routing only the hashLog field transformations through
scalar Rust ABI helpers. The helpers use wrapping u32 arithmetic so overflow
and the existing subtract-then-clamp behavior remain identical to C.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression dedicated_dict_search_hash_log_adjustments_match_strategy_cases` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- clippy normal, benches, and tests before and after formatting -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` and `git diff --check` -- passed
Keep the private writeBlockHeader signature and its debug logging in C while
moving the pure block-type, size, and 24-bit little-endian encoding behind a
narrow Rust ABI. The Rust leaf preserves the RLE blockSize path for cSize == 1
and the compressed cSize path for every other value.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression block_header -- --nocapture` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Required clippy, nightly fmt, and diff checks -- passed
ZSTD_compressCCtx was selecting the Rust frame path for every compression
level. For source sizes that resolve to lazy or optimal strategies, that path
fell through to fast matching and could produce a different frame, including
the CCtx reuse regression caught by fuzzer test 56.
Query the source-size-dependent C parameters before preparing the context.
Keep fast and double-fast strategies on the existing Rust path, while routing
all other strategies through ZSTD_compress_usingDict with no dictionary so the
original simple API initializes requested parameters and C match state.
Test Plan:
- `make -B -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t56 -i56 -v` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 253 passed
- required clippy normal, benches, and tests checks -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
ZSTD_resolveExternalSequenceValidation() is a pure int identity used while
resolving CCtx parameters. Route its existing C wrapper through a Rust
extern "C" entry point in the parameter module, preserving the int-to-int
ABI and every call-site result. Add boundary coverage for signed int modes.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed (253 tests)
- `make -B -C lib -j2 lib` -- passed
- required clippy passes for the compression feature, benches, and tests -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` and `git diff --check` -- passed
The compression-parameter override helper previously lived in C and selectively
copied each nonzero field from a caller-provided override structure. Move that
pure field-wise operation into the Rust compression-parameter module, while
keeping the existing repr(C) structure and pointer-based ABI at both C call
sites. Zero-valued overrides continue to leave the current parameter untouched.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed (252 tests)
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- required clippy passes, nightly fmt, and `git diff --check` -- passed
Keep validation of the applied compression parameters in C and delegate the context-free minimum calculation through a narrow ABI leaf. Preserve the public getter and deprecated block-compression paths.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- make -C tests -j2 fuzzer
- tests/fuzzer -s4560 -t47 -i48 -v
Keep C ownership of the block-state structure while delegating the confirmed-block pointer swap through a narrow two-pointer ABI leaf. Preserve every existing confirmation call site and add direct pointer-swap coverage.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- tests/fuzzer -s4560 -t47 -i48 -v
- make -C tests -j2 test-zstream
Parse attached and separated --patch-from references, preserve the upstream ultra ceiling and conflict checks, and pass the reference path through the existing file-I/O dictionary ABI with patch mode enabled.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -C tests -j2 check
- small attached/separate patch-from round-trip and rejection smoke
Port the no-delimiter external-sequence block scanner, match splitting, repcode updates, validation, and sequence/literal storage behind a narrow C shim. Preserve trailing bytes as literals when the sequence buffer ends before the requested block, and remove the obsolete C validation wrapper.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (242 passed)
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression [--benches|--tests] (clean)
- cargo +nightly fmt --manifest-path rust/Cargo.toml (clean)
- make -B -C lib -j2 lib (passed)
- make -C tests -j2 test-zstream (84 deterministic; 5960 and 8307 randomized passed)
- make -C tests -j2 test-fuzzer (existing test 56 CCtx-reuse mismatch; also fails at 37eb75758)
- make -C tests -j2 test-zstd (existing Rust CLI --patch-from gap)
Port ZSTD_dictNCountRepeat and ZSTD_loadCEntropy to a dedicated Rust compression module. Preserve the existing C-facing ZSTD_loadCEntropy symbol through a narrow shim while keeping dictionary content and compression-context ownership in C.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --benches --tests
- make -B -C lib -j2 lib
- make -C tests -j2 test-invalidDictionaries
- make -C tests -j2 test-zstream
Move ZSTD_transferSequences_wBlockDelim's delimiter scan, literal copying, SeqDef storage, validation, and repcode bookkeeping into the Rust compression statistics module. Keep ZSTD_CCtx and block-state ownership in C through a narrow scalar/pointer shim, with ABI layout assertions for ZSTD_SequencePosition.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --benches --tests
- make -B -C lib -j2 lib
- make -C tests -j2 test-zstream
Block splitting still estimated literal and sequence section sizes through
three C helpers. Move the pure estimator into zstd_compress_stats.rs so its
histogram, HUF, and FSE cost calculations use the Rust implementations already
behind the compression ABI. Preserve the original literal headers, multi-stream
Huffman surcharge, sequence additional-bit costs, entropy-header accounting,
and fallback estimates; leave C responsible for split recursion and table
ownership.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 229 passed
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` plus benches/tests -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 test-zstream` -- passed, including 84 deterministic and 15,375 randomized cases
The explicit-delimiter sequence path still calculated each block summary in C,
with a separate AVX2 implementation and scalar fallback. Move that pure scan
to zstd_compress_stats.rs and keep a C wrapper for the existing orchestration.
The Rust leaf preserves delimiter inclusion, literal and match totals, and the
external-sequences error when no delimiter is present, while the C ABI is pinned
with a repr(C) layout check.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 227 passed
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` plus benches/tests -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 test-zstream` -- passed, including 84 deterministic and 15,464 randomized cases
The external-sequence block path still converted public ZSTD_Sequence values
in C, with separate scalar and AVX2 implementations. Move that pure format
conversion into zstd_compress_stats.rs so both native configurations share one
implementation. Preserve offBase encoding, u16 truncation, the rep-ignored
contract, and the long-length side-band marker; leave C responsible for block
state, repcode resolution, and long-length bookkeeping.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 225 passed
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` plus benches/tests -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 test-zstream` -- passed, including 84 deterministic and 14,768 randomized cases
Port ZSTD_seqStore_resolveOffCodes and its repcode-resolution helper to Rust. The C shim keeps Repcodes_t and SeqStore_t ownership in the compressor context while Rust preserves long-literal handling, mismatch materialization, and independent decoder/compressor history updates.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (208 tests)
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
- make -B -C lib -j2 lib
Replace the shared C raw and one-byte RLE fallback block serializers with Rust ABI leaves. Preserve the zstd block headers, payload copies, capacity errors, and all existing C compressor dispatch and context ownership; share the raw serializer with the Rust one-shot path.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (208 tests)
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
- make -B -C lib -j2 lib
Move malloc-backed dictionary file reads into the Rust CLI backend while keeping C responsible for stat metadata, patch-mode size policy, diagnostics, buffer-type selection, and eventual free(). The Rust leaf returns explicit open, size, allocation, and read statuses and publishes only fully read libc allocations.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml (126 tests)
- cargo clippy --manifest-path rust/cli/Cargo.toml
- make -B -C programs -j2 zstd
- make -C tests -j2 test-cli-tests (41 tests)
Port the scalar frame-finalization policy and byte serialization into the Rust compression leaf while keeping checksum finalization, tracing, and the compression context in C. Preserve the init-to-ongoing transition on partial failure and reset the stage only after a successful epilogue.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-zstream (84 deterministic, 6737 and 9012 fuzzer cases)
Route the CLI backend's safe file-removal leaf through a narrow Rust status ABI while preserving C diagnostics, Windows read-only handling, and the existing return convention.
Test Plan: cargo test --manifest-path rust/cli/Cargo.toml; cargo clippy --manifest-path rust/cli/Cargo.toml; cargo clippy --manifest-path rust/cli/Cargo.toml --benches; cargo clippy --manifest-path rust/cli/Cargo.toml --tests; make -B -C programs -j2 zstd; make -C tests -j2 test-cli-tests (41 tests); make -B -C programs -j2 zstd-small zstd-frugal.
Port the external-sequence window, offset, and minimum-match checks through a scalar C ABI leaf while retaining the C caller's control flow and diagnostics.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression; cargo clippy --manifest-path rust/Cargo.toml; cargo clippy --manifest-path rust/Cargo.toml --benches; cargo clippy --manifest-path rust/Cargo.toml --tests; make -B -C lib -j2 lib; make -B -C tests -j2 test-zstream (84 deterministic, 7063 and 8665 fuzzer cases).
Move ZSTD_isUpdateAuthorized behind the existing Rust parameter API while retaining the C context-stage checks, diagnostics, and mutation flow.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (187 tests)
- cargo clippy for library, benches, and tests
- make -B -C lib -j2 lib
Move the CLI's strategy-dependent cycle-log calculation behind a scalar Rust ABI shim while retaining the C assertion and call-site behavior.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark (116 tests)
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets with the full CLI feature set
- make -B -C programs -j2 zstd zstd-small zstd-frugal
- make -C tests -j2 test-cli-tests (41 scenarios)
Move hash-salt mixing and context-reset overflow predicates behind scalar Rust ABI shims. Keep the C-owned match state and window arithmetic in the C wrapper while preserving exact architecture-dependent thresholds.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (186 tests)
- cargo clippy for library, benches, tests, and CLI all-targets
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-zstream (84 deterministic tests; fuzzers 6451 and 9879 cases)
Move patch-from memory-limit sizing and rejection status selection behind a Rust ABI shim. Keep the C diagnostics and assertion in place, and leave the existing preference unchanged when input sizes are unknown or exceed the window limit.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark (113 tests)
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets with the CLI feature set
- make -B -C programs -j2 zstd zstd-small zstd-frugal
- make -C tests -j2 test-cli-tests (41 scenarios) and make -B -C tests -j2 test-zstream
Move target block-size and block-splitter policy predicates behind narrow Rust ABI shims while retaining the C debug logging, finalized-mode assertion, and surrounding control flow.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml and native library build
- make -B -C tests -j2 test-zstream (84 deterministic tests; fuzzers 6778 and 8299 cases)
Port the single-file and multiple-file summary decisions through the existing FIO context ABI, including the display-level lookup and the original multiple-file assertion invariant. Keep the C wrappers and call sites unchanged.
Test Plan: cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark (108 passed); CLI all-target clippy; make -B -C programs -j2 zstd zstd-small zstd-frugal; make -C tests -j2 test-cli-tests (41 passed).
Move construction of the external-dictionary and active-prefix ranges into a narrow Rust ABI while retaining the C window and logging surface. Preserve byte-range half-open overlap semantics and leave the input-range overlap wrapper available to its other C caller.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (180 passed); root and CLI clippy; make -B -C lib -j2 lib; make -C tests -j2 test-cli-tests (41 passed); make -B -C tests -j2 test-zstream (84 named tests plus 6,845 and 9,628 fuzz cases passed).
Port the 16-cell match-table reduction leaf while preserving the window-index threshold, wrapping subtraction, and btlazy2 unsorted marker. Keep ZSTD_reduceIndex and its strategy-specific table selection in C behind the narrow Rust ABI.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (180 passed); root clippy, bench clippy, and test clippy; make -B -C lib -j2 lib; make -B -C tests -j2 test-zstream (84 named tests plus 6,845 and 9,628 fuzz cases passed).
Mirror fileio.c's private fileInfo_t with a checked repr(C) ABI and aggregate totals, flags, and file counts through an output-pointer shim. Preserve zeroed non-aggregate fields and use wrapping arithmetic for the original integer operations.
Test Plan: cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark (106 passed); cargo clippy --all-targets with the same features; make -B -C programs -j2 zstd zstd-small zstd-frugal; make -C tests -j2 test-cli-tests (41 passed).
Port repcode offBase finalization and explicit-delimiter block sizing through the Rust compression ABI. Preserve wrapped sequence arithmetic, external-sequence error codes, and the no-delimiter target-size policy while keeping the C control flow intact.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (176 passed); cargo clippy root, benches, and tests; make -B -C lib -j2 lib; make -B -C tests -j2 test-zstream (84 named tests plus 6,230 and 8,598 fuzz cases passed).
Move sequence-store byte accounting, chunk derivation, and repcode resolution into the Rust compression module. Move MT raw-sequence buffer conversions into Rust while preserving the existing C adapters and ABI layouts.
Test Plan:\n- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression\n- cargo clippy --manifest-path rust/Cargo.toml\n- cargo clippy --manifest-path rust/Cargo.toml --benches\n- cargo clippy --manifest-path rust/Cargo.toml --tests\n- make -B -C lib -j2 lib\n- make -B -C tests -j2 test-cli-tests
Route file-I/O input/output buffer construction and the LZ4 block-size mapping through Rust ABI leaves while preserving the existing C helper names and public buffer layouts. Add layout, field, and formula coverage.
Test Plan: cargo test --manifest-path rust/cli/Cargo.toml --no-default-features --features cli,compression,decompression,benchmark; cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets --no-default-features --features cli,compression,decompression,benchmark; make -B -C programs -j2 zstd zstd-small zstd-frugal; make -B -C tests -j2 test-cli-tests
Route public CCtx parameter bounds through the Rust policy implementation while keeping build-specific MT limits supplied by the C configuration. Preserve clamping and validation behavior for migrated parameters and add focused Rust coverage.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression; cargo clippy --manifest-path rust/Cargo.toml; cargo clippy --manifest-path rust/Cargo.toml --benches; cargo clippy --manifest-path rust/Cargo.toml --tests; make -B -C lib -j2 lib; make -B -C tests -j2 test-cli-tests; make -B -C tests -j2 test-zstream
Move ZSTD_CCtxParams_init_advanced and its default-resolution policy into the Rust parameter API while retaining the C public ABI wrapper. Preserve validation, zeroing, and the existing row-match, block-splitter, LDM, and repcode defaults.
Test Plan:\n- cargo clippy --manifest-path rust/Cargo.toml\n- cargo clippy --manifest-path rust/Cargo.toml --benches\n- cargo clippy --manifest-path rust/Cargo.toml --tests\n- cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check\n- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params_api\n- make -B -C lib -j2 lib
Move CCtx_params allocation, initialization, and release behind Rust ABI helpers while preserving the public C wrappers and custom allocator contract. Keep parameter bounds and policy in the existing C implementation.
Test Plan:\n- cargo clippy --manifest-path rust/Cargo.toml\n- cargo clippy --manifest-path rust/Cargo.toml --benches\n- cargo clippy --manifest-path rust/Cargo.toml --tests\n- cargo +nightly fmt --manifest-path rust/Cargo.toml --all\n- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression\n- make -B -C lib -j2 lib\n- make -B -C tests -j2 test-rust-lib-smoke\n- make -B -C tests -j2 test-zstream\n- make -B -C tests -j2 test-cli-tests
Move the basename-preserving output-directory helper behind the existing Rust utility ABI. Keep the libc allocation contract and platform separator behavior intact while leaving the C file-I/O engines and higher-level naming policy unchanged.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml (93 passed)
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets
- make -B -C tests -j2 test-cli-tests (41 passed)
Implement the CLI trace callbacks and CSV logger in Rust, preserving the ZSTD_Trace ABI, timing fields, parameter reporting, mutex-protected writes, and no-op disabled behavior. Add the module to every Rust CLI archive and exclude the duplicate C trace object from Rust-backed program targets.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml (92 passed)
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets
- make -B -C programs -j2 zstd
- Rust trace smoke: programs/zstd -q --trace=/tmp/zstd-rust-trace-smoke.csv -c /tmp/levels-input
- make -B -C tests -j2 test-cli-tests (41 passed)
Match the original optimal-parser statistics for zero-possible symbols, complete the shortest-path fallback when the forward pass runs past its final position, and refresh the projected window state for the ultra2 second pass. Keep the simple Rust frame leaf restricted to the match finders it implements so lazy and optimal strategies retain their stateful lifecycle.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (135 passed)
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_opt (3 passed)
- make -B -C tests -j2 test-cli-tests (41 passed)
- make -B -C tests -j2 test-zstream (84 named tests and fuzzer suites passed)
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
Port the remaining high-level CLI option, validation, environment, terminal-safety, and file-stat compatibility paths into Rust while keeping the existing C file-I/O ABI boundary. Match upstream quiet-mode warnings and exact error text for size and level handling.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml (90 passed)
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
- make -B -C tests -j2 test-cli-tests (41 passed)
Route ordinary complete-input ZSTD_compressStream2 calls through the Rust frame compressor while retaining the C state machine for partial, advanced, and dictionary-backed streams. Track explicit maxBlockSize requests so the migrated path preserves the original byte-identical default behavior, and extend the C archive smoke test across context reuse and fallback cases.
Test Plan:
- cargo clippy --manifest-path rust/Cargo.toml
- cargo clippy --manifest-path rust/Cargo.toml --benches
- cargo clippy --manifest-path rust/Cargo.toml --tests
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (133 passed)
- make -B -C tests -j2 test-rust-lib-smoke and ./tests/rustLibSmoke
- make -B -C tests -j2 test-zstream (passed, including both APIs and maxBlockSize case 78)
Replace the program archive's C benchmark orchestration with the complete
Rust port while retaining the public benchzstd.h ABI and result/error
contracts. Include the module only in full CLI archives and remove only
benchzstd.c from the program source set, leaving C test targets on their
existing implementation.
Test Plan:
- cargo test --manifest-path /tmp/zstd-benchzstd.IHWpLD/Cargo.toml (6 passed)
- cargo check and cargo clippy --all-targets in the focused harness (passed)
- standalone release archive build and BMK_* symbol inspection (passed)
- make -C programs zstd (passed before concurrent CLI edits)
- make -C tests -B paramgrill (passed)
- native synthetic, multi-file, dictionary, and invalid-frame benchmark smoke tests (passed)
Replace programs/fileio_asyncio.c with a declaration-only shim and register
Rust's async file-I/O implementation in the library archive. Preserve the C
ABI for pool and job contexts, including DEBUGLEVEL pointer-based pthread
wrappers, and select separate Rust build configurations for those layouts.
Keep ordered-read queue state locked while waiting, drain queued work before
shutdown frees job buffers, and use platform large-file seeks for sparse writes.
Test Plan:
- `cargo test --no-default-features --lib fileio_asyncio` (7 passed).
- `cargo test --no-default-features --features debug-pthread --lib fileio_asyncio` (7 passed).
- `cargo check --all-targets`, clippy library/benches/tests, and nightly fmt check (passed).
- Threaded program build and README round-trip (passed).
- `make -C tests poolTests` and `./tests/poolTests` (passed).
- Current C shim object defines no `AIO_*` symbols; the Rust archive exports all 20 declarations.
- Full standalone all-target Rust tests and the default CLI archive rebuild remain affected by unrelated C/Rust integration and another worker's in-progress benchmark changes.
The multithreaded compressor already uses Rust-owned buffer and CCtx pools,
but zstdmt_compress.c still allocated and freed its job table directly. Move
the raw table storage and power-of-two sizing behind Rust's custom allocator
ABI. Keep descriptor field access and platform mutex/condition initialization
in C because those layouts remain private and platform-specific. The C
wrapper initializes and destroys synchronization primitives around the Rust
storage calls, preserving failure cleanup while leaving worker job setup,
scheduling, and stream entry points in C as the fallback implementation.
Keep the Rust storage-only ABI free of MT-only C references so single-threaded
archives can omit zstdmt_compress.c without acquiring new unresolved symbols.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress --lib` -- passed (5 tests).
- `make -B -C lib lib-mt` -- passed.
- `make -B -C tests -j2 fullbench zstreamtest poolTests` -- passed.
- `./poolTests` -- passed.
- `./fullbench -i1 -B1000 README.md` -- passed, including -T2 scenarios.
- Rebuilt `programs/zstd` and ran a `-T2` compress/decompress `cmp` round-trip
-- passed.
- `rustfmt` and `cargo +nightly fmt --manifest-path rust/Cargo.toml --all
-- --check` -- passed.
- Full `cargo clippy -D warnings` remains blocked by unrelated warnings in
the concurrent `rust/src/fileio_asyncio.rs` worktree changes.
- `./zstreamtest -T5s` reaches an unrelated single-thread maxBlockSize
assertion at `tests/zstreamtest.c:2157`; MT fullbench and CLI smoke pass.
Dispatch ordinary dictionary-free ZSTD_compress2 calls through the Rust frame
compressor while retaining the original C implementation for advanced
contexts. Prepare the C workspace first so allocation downsizing and
static-context limits remain observable.
Mark successful Rust frames so a following one-shot
ZSTD_compressStream2(..., ZSTD_e_end) call uses the same output path. Other
streaming directives and advanced parameters remain on the C implementation.
Test Plan:
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path
rust/Cargo.toml --no-default-features --features compression
zstd_compress::tests
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path
rust/Cargo.toml --no-default-features --features compression -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C tests fuzzer
- ./tests/fuzzer -t114 -i1000 --no-big-tests -s1
The basic fuzzer run now reaches the existing Rust one-shot compression-ratio
limit at test 113; the new stateful reuse tests pass before that point.
Move ZSTD_compressCCtx() onto the Rust frame-compression path while
retaining only the C-owned reset needed for its private context layout.
The simple API ignores advanced context parameters by contract, so the
Rust entry point can share the one-shot implementation after reset.
Preserve valid frames when the Rust superblock leaf declines a compressed
block by emitting a raw block instead. Match the fast-strategy literal
policy for negative levels so compatibility behavior remains intact.
Streaming, stateful ZSTD_compress2(), and multithreaded context
orchestration remain C-owned for a later slice. The full fuzzer currently
reaches an existing compress2() superblock-expansion failure after these
simple-API checks; that stateful path is intentionally out of scope here.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress::tests
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings
(library, benches, and tests)
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C programs zstd V=1
- make -B -C examples multiple_simple_compression
- ./multiple_simple_compression README.md ../rust/README.md
Move the frozen v0.7 decoder, dictionary handling, bufferless streaming,
and buffered streaming implementation into the Rust legacy module. Keep
the historical C translation unit as a declaration-only ABI shim so the
existing C callers and build selection remain unchanged.
The port preserves the v0.7 entropy and frame boundaries, while fixing the
Rust-side literal-tail bookkeeping and making buffered header loading revisit
the complete frame header before sizing its rolling buffers.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features decompression,legacy-v07 legacy::zstd_v07::tests
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings
- make -B -C programs zstd V=1
- decode a v0.7.5 fixture and compare its 200000-byte output
- make -C tests check V=1
Replace the v0.5 and v0.6 legacy decoder translation units with Rust modules
and keep only narrow C registration shims for the public legacy dispatch ABI.
Test Plan:
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/Cargo.toml --features compression,decompression,dict-builder,legacy-v05,legacy-v06,legacy-v07
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/Cargo.toml --all-targets --features compression,decompression,dict-builder,legacy-v05,legacy-v06,legacy-v07 -- -D warnings
- make -C tests check V=1
Delay the legacy dictionary segment copy until the effective content size is
known, preventing writes beyond the caller-provided dictionary capacity.
Test Plan:
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/cli/Cargo.toml
- make -C tests check V=1
Move filename tables, file-list expansion, core-count helpers, statistics,
and dictionary sample loading behind Rust implementations while retaining
the narrow C ABI used by the CLI.
Test Plan:
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/cli/Cargo.toml
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- make -B -C programs zstd V=1
- make -C tests check V=1
Add Rust parsing and dispatch for the default, Cover, FastCover, and legacy
dictionary-training modes, including their parameter validation, output
defaults, and help text. Keep file loading behind the existing narrow dibio
bridge while the dictionary algorithms are supplied by the Rust library.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/zstd_cli.rs
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/cli/Cargo.toml
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
Move the frozen v0.4 frame, entropy, streaming, and dictionary decoder
implementation to Rust while retaining the C translation unit as an ABI
anchor. Register the decoder behind the matching legacy feature and keep its
historical behavior isolated from newer formats.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/legacy/zstd_v04.rs rust/src/legacy/mod.rs
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path rust/Cargo.toml --no-default-features --features decompression,legacy-v04 legacy::zstd_v04
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/Cargo.toml --all-targets --no-default-features --features decompression,legacy-v04 -- -D warnings
- C/Rust v0.4 ABI and decompression build checks
- git diff --cached --check
Move FastCover parameter validation, corpus preparation, optimization, and
dictionary training into Rust. Retain the C source as a static-linking ABI
anchor and register the implementation with the compression dictionary-builder
feature set.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/dict_builder_fastcover.rs rust/src/lib.rs
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path rust/Cargo.toml --all-targets --no-default-features --features compression,dict-builder -- -D warnings
- FastCover focused Rust tests and deterministic C-reference harness
- git diff --cached --check
ddidderr