The Rust dispatch gate for ZSTD_compress2 and complete-input
ZSTD_compressStream2 calls was selecting compression parameters with an
unknown content size. For the 30-byte level-4 CCtx reuse case that chooses the
dfast strategy, while the actual source size selects greedy; the Rust frame
then emitted different output from the C-owned ZSTD_compressCCtx path.
Pass the actual source size through both helper ABIs. The one-shot and complete
stream entry points now select the same strategy as C, while the existing
sentinel keeps every non-fast/non-dfast configuration on the original C
fallback. The existing fuzzer test 56 exercises the regression without
modifying the shared C test harness.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4560 -t56 -i57 -v` --
passed; the 30-byte test-56 regression now completes.
- `./tests/fuzzer -s4142 -t63 -i64 -v` -- passed all 64 focused cases.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 4,385 first
randomized, and 8,137 new-API randomized cases.
- `make -C tests -j2 test-fuzzer` reaches test 113's flat-dictionary
efficiency assertion with concurrent unstaged LDM changes; test 56 passes.
The window overflow gate mixed two scalar decisions with C-owned pointer and
window state. The early correction test uses the block-start index and must
wait for dictionary invalidation, while the normal fallback uses the block-end
index and the platform-selected ZSTD_CURRENT_MAX threshold. Keeping those
subtractions and the ZSTD_window_t wrapper in C preserves the existing call
contract without making Rust depend on pointer width or private state.
Move the U32 cycle/MAX arithmetic, correction-count scaling, dictionary gate,
and frequent-policy decision into Rust. C passes the two pointer-derived
indices only through scalar results, and supplies the current-max threshold
and active build policy explicitly so 32/64-bit and fuzzing configurations
remain authoritative.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4142 -t63 -i64 -v` --
passed all 64 cases.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 5,477 first
randomized, and 6,902 new-API randomized cases; the existing unterminated
string initializer warning remains.
Keep the public ZSTD_getCParams() selection and all DISPLAYLEVEL output in
programs/fileio.c, but pass its by-value compression-parameter snapshot to a
Rust policy leaf. Rust now preserves the original source-size highbit and
window clamping rules, validates and updates the patch memory limit before
success outputs, derives cycleLog, updates comprParams->windowLog, and applies
automatic LDM without overwriting the explicit-LDM diagnostic semantics.
The Rust ABI returns the raw file window and the two diagnostic predicates so
C can retain its existing messages and optimal-parser note ordering. C layout
assertions cover the seven-word compression-parameter snapshot on both
platform-width variants; rejected unknown and oversized inputs leave all
outputs unchanged.
Test Plan:
- Rust clippy pre/post nightly-format matrix for library compression and CLI
cli,compression,decompression,benchmark features -- passed.
- `cargo test --manifest-path rust/cli/Cargo.toml --no-default-features
--features cli,compression,decompression,benchmark` -- 137 passed.
- `make -C programs -j2 zstd` -- passed.
- `make -C tests -j2 test-cli-tests` -- 41 passed.
- `python3 tests/cli-tests/run.py decompression/pass-through.sh` -- passed.
- Focused patch-from round trips, stream-size failure ordering, and automatic
long-mode diagnostics -- passed.
- `make -C tests -j2 test-rust-lib-smoke` -- passed.
- No i686 Rust target is installed; 32-bit policy bounds are covered by
size_of-based constants and C/Rust ABI assertions.
ZSTD_reduceIndex previously selected and reduced each match table through
separate C helpers, even though the 16-cell reduction leaf already lived in
Rust. Keep C responsible for match-state access, table allocation policy,
nullable table selection, btlazy2 strategy selection, and the surrounding
workspace/window/dictionary state transition. Add one narrow Rust ABI that
receives only the validated hash, chain, and hash3 table slices and delegates
to the existing reducer, preserving chain markers only when requested and
keeping threshold and U32 wrapping arithmetic unchanged. Focused tests cover
zero-sized optional tables, marker routing, and threshold behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress::tests::reduce_` -- 5 passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 344 passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,814 standard
randomized, and 8,955 new-API randomized cases passed.
- Required compression clippy library/benches/tests, nightly fmt, and the repeated three clippy checks -- passed.
Keep patch-from orchestration in C while extracting only the bounded window
selection and automatic-LDM decision into the Rust CLI leaf. The C caller
still computes the raw file window with FIO_highbit64, obtains cParams,
updates the memory limit, emits warnings and optimal-parser notes, and writes
comprParams->windowLog and prefs->ldmFlag. Rust receives only scalar policy
inputs and two output pointers, clamps the selected window to the target's
10..30/31 bounds, and compares the unclamped value strictly against cycleLog.
This preserves equality as no-LDM and keeps values above the maximum eligible
to trigger LDM. Focused unit tests cover both clamps and each comparison edge.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` with library, `--benches`, and `--tests`, before and
after formatting -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --
--config skip_children=true` and nightly rustfmt on the owned file -- passed
- `cargo test --manifest-path rust/cli/Cargo.toml --no-default-features
--features cli,compression,decompression,benchmark fileio_prefs` -- 36 passed
- `make -C programs -j2 zstd` -- passed
- `make -C tests -j2 test-cli-tests` -- 41 passed
- Focused patch-from round trip and automatic long-mode trigger -- passed
The Rust legacy streaming dispatcher has version-specific match arms. In
legacy-disabled builds those arms disappear, leaving the projected context,
dictionary sentinel, and decoder context parameters unused even though the
same parameters are required by enabled legacy formats. Prefix the shared
names with underscores while retaining their use in enabled branches, so the
no-legacy feature matrix remains warning-clean without changing the ABI or
runtime behavior.
Test Plan:
- cargo check --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder -- passed.
- cargo check --manifest-path rust/Cargo.toml --no-default-features --features decompression,legacy-v04,legacy-v05,legacy-v06,legacy-v07 -- passed.
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check -- passed.
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder --lib -- -D warnings -- passed.
- Same clippy command with decompression and legacy-v04 through legacy-v07 features -- passed.
- git diff --cached --check -- passed.
Move the legacy buffered-stream dispatcher out of the C decompression adapter
while keeping ZSTD_DCtx_s private and C-owned. Rust now consumes the existing
ZSTD_rustDctxView projection, detects the legacy version at input.pos, preserves
static-context rejection and stream-stage reset behavior, and handles legacy
context replacement through the version-gated v0.4-v0.7 Rust stream APIs.
The Rust boundary retains the C helper's NULL normalization, dictionary
lifetime sentinel, previous-version cleanup semantics, input/output position
updates, and version-unsupported and no-legacy-support fallbacks. The C legacy
free helper remains unchanged; only the dispatcher declaration and body were
removed from the adapter. Focused tests cover unsupported versions, stage
reset, context reuse/switching, and partial v0.4 input/output progress.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed
- Same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml`, followed by all three clippy commands -- passed
- Decompression/legacy clippy with `-D warnings` -- passed
- All 107 Rust tests with v0.4-v0.7 features -- passed
- Focused legacy dispatcher tests: 4 passed
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed
- `make -C tests -j2 test-legacy` -- built, then failed in the existing simple one-shot path with one v0.8 output byte mismatch (`e0` vs expected `e2`) before the streaming check; the new dispatcher symbol was not called on that failing path
The window-overflow helper mixed pointer arithmetic and ZSTD_window_t state
updates with the scalar U32 cycle calculation that determines the correction.
That made the arithmetic boundary depend on the host pointer implementation.
Add a narrow Rust ABI leaf that accepts only the current index, cycle log, and
maximum distance. Rust preserves the start-index adjustment, cycle alignment,
cycle-size versus max-distance selection, and explicit U32 wrapping. C derives
newCurrent from the returned correction so it can retain every invariant check,
pointer update, workspace/table transition, counter update, and dictionary
invalidation in the existing stateful wrapper.
Test Plan:
- Rust compression clippy, bench clippy, test clippy, nightly fmt, and the
repeated three clippy checks -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 340 passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `./tests/fuzzer -s4142 -t63 -i64 -v` -- focused overflow coverage passed.
- `make -C tests -j2 test-zstream` -- 84 deterministic, 5,018 standard
randomized, and 8,851 new-API randomized cases passed.
- `make -C tests -j2 test-fuzzer` reaches the pre-existing CCtx-reuse test 56
failure; a pristine 837e4f099 worktree reproduces it, while the focused
test-56 run passes.
Keep decompression resource ownership and file orchestration in C while the
Rust AIO module handles the pass-through byte-copy leaf through a two-pool ABI.
The Rust implementation derives the block limit from the actual read and write
job buffers, preventing a write-job overrun when their configured sizes differ.
It preserves the existing enqueue/reacquire, consume/refill, EOF, release, and
sparse-write ordering, with pool-backed coverage for synchronous and threaded
operation.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression --lib fileio_asyncio` -- passed, 9 tests.
- Required clippy/fmt sequence against `rust/Cargo.toml` with compression -- passed before and after nightly fmt.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression --lib` -- passed, 342 tests.
- `make -C programs -j2 zstd` -- passed.
- Manual 200 KiB and empty-input `programs/zstd -dc --pass-through` comparisons -- passed.
`dict_builder_zdict.rs` still declared the compressed-block reset, dictionary
entropy loader, and FastCOVER optimizer as C externs even though their Rust
implementations were already linked into the archive. That left unnecessary
Rust-to-C edges and kept a duplicate private FastCOVER `repr(C)` definition.
Call the Rust reset and entropy leaves directly, import the Rust FastCOVER
entry point, and use its canonical ABI parameter type. The eight opaque
compression-context APIs used by entropy analysis remain C-owned for a later
context port, so this change preserves the existing ownership boundary and
public layouts.
Test Plan:
- `cargo check --manifest-path rust/Cargo.toml --no-default-features --features compression,dict-builder` -- passed.
- Required `cargo clippy` runs for the library, benches, and tests, before and after nightly formatting -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` and `git diff --cached --check` -- passed.
- Rebuilt `libzstd_rs.a`; the old `ZSTD_loadCEntropy` and
`ZSTD_reset_compressedBlockState` names are absent from its undefined
symbols, while the Rust reset, entropy, and FastCOVER symbols are defined.
- `make -C tests -B -j2 zstreamtest` -- linked successfully; only the
pre-existing `tests/zstreamtest.c:1899` warning remains.
- `make -C tests -j2 test-zstream ZSTREAM_TESTTIME=-T5s` -- passed 84
deterministic, 220 legacy-API randomized, and 404 new-API randomized cases.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,dict-builder --lib` -- cannot link until the eight intentionally deferred opaque C compression-context symbols are ported.
- `./tests/fuzzer -v -T5s` stopped at existing test 56 from unrelated
concurrent compression edits before reaching dict-builder tests.
ZSTD_getCParamsFromCCtxParams previously kept its context-free selection,
LDM window override, nonzero-field override, and final adjustment orchestration
in C. Move that pipeline into a scalar Rust ABI leaf while keeping the private
ZSTD_CCtx_params snapshot in C. The ABI passes the compression parameters by
value, the source-size hint, mode switches, and the LDM default window log.
The active ZSTD_EXCLUDE_* definitions remain owned by the C preprocessor and
are encoded as a mask. Rust applies that mask before each adjustment stage,
including the existing standalone adjustment callers, so reduced compressor
builds retain the original fallback cascade without duplicating CCtx state or
compile-time configuration in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression`
-- passed, 337 tests.
- All three compression clippy commands passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic tests,
5,191 standard randomized cases, and 9,382 new-API randomized cases.
- `git diff --cached --check` -- passed.
Move the arithmetic aggregation in ZSTDMT_getFrameProgression across the
existing C/Rust boundary. C still owns MT state traversal, mutex locking,
jobIDMask ordering, nextJobID plus jobReady handling, error normalization,
and the flushed <= produced assertion. Rust constructs the base result and
folds each compact, C-normalized job snapshot into the complete repr(C)
ZSTD_frameProgression, preserving size_t-to-U64 wrapping and active-worker
counting without exposing private job or mutex layouts.
Focused Rust tests cover zero jobs, a ready job, completed jobs, normalized
error output, and mixed active/completed aggregation.
Test Plan:
- All three compression clippy commands passed before formatting.
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- passed.
- All three compression clippy commands passed after formatting.
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression -- 332 passed.
- make -B -C lib -j2 lib-mt -- passed.
- make -B -C lib -j2 lib-nomt -- passed.
- make -B -C tests -j2 test-zstream -- passed: 84 named, 6,004, and 8,392
randomized cases.
- git diff --check and git diff --cached --check -- passed.
The zstream build retained the pre-existing warning at
tests/zstreamtest.c:1899 about an unterminated initializer string.
Complete the single-thread branch of ZSTD_getFrameProgression behind a
narrow scalar ABI boundary. C still owns the public entry point, buffered
input extraction, assertions, and the multithreaded dispatch; Rust now owns
the six-field single-thread result construction. The Rust #[repr(C)] result
uses wrapping u64 conversion for consumed input plus buffered size, mirrors
produced into flushed, and zeros the MT-only fields.
This replaces the earlier ingested-only helper so the ABI no longer splits
one result across duplicate C and Rust construction paths. The unit tests now
cover the complete result and its u64 wrapping behavior.
Test Plan:
- All three requested compression clippy commands passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 327 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 5,330, and 9,772
randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains the pre-existing warning at
`tests/zstreamtest.c:1899` about an unterminated initializer string.
ZSTD_estimateCCtxSize_internal() evaluates four fixed source-size tiers and
must retain every C-owned parameter and sizing call, including raw error-shaped
size_t results. Store those four results in C, then delegate only the final
unsigned maximum comparison to a scalar Rust ABI helper. The outer monotonic
compression-level budget and the separate CStream estimation path remain
unchanged.
The Rust unit tests cover zero, ordinary, equal, SIZE_MAX, and encoded error
values to document that the helper compares raw size_t values without arithmetic
or error interpretation.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- Same clippy command with `--benches` -- passed before and after formatting
- Same clippy command with `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 328 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named plus 5,923 and 9,247 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build retains the pre-existing initializer-string warning at
tests/zstreamtest.c:1899.
ZSTD_literalsCompressionIsDisabled() previously switched on the private
ZSTD_CCtx_params layout in a header inline helper. Replace that switch with a
scalar ABI call that extracts only literalCompressionMode, strategy, and
targetLength in C. The Rust helper preserves the enable and disable results,
the fast-strategy auto rule, and C's debug assertion plus release fallthrough
for invalid modes. Keep callers and compression state ownership in C, and
cover the policy truth table and debug invalid-mode invariant in Rust.
Test Plan:
- Compression-feature clippy for the library, benches, and tests before and
after nightly formatting -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- passed (43 tests).
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed (84 named, 5,416, and 8,387
randomized cases).
- `git diff --check` and `git diff --cached --check` -- passed.
The MT rsync initialization previously computed the rolling-hash prime
power through the C-only ZSTD_rollingHash_primePower helper. Move only that
scalar exponentiation behind the existing Rust MT ABI boundary. The C caller
still owns the MT context assignment, RSYNC_LENGTH constant, and surrounding
initialization.
The Rust helper mirrors ZSTD_ipow with wrapping u64 multiplication and applies
u32 wrapping subtraction before widening the exponent, preserving the C
length == 0 behavior. Rolling-hash scanning, rotation, and all other MT state
remain unchanged.
Test Plan:
- Focused rolling_hash_prime_power Rust tests -- passed (2 tests).
- Required clippy/fmt sequence in normal, bench, and test modes -- passed.
- make -B -C lib -j2 lib -- passed.
- make -C tests test-rust-lib-smoke -- passed.
- tests/fuzzer -s4560 -t56 -i57 -v -- passed (57 tests).
- make -C tests -j2 test-zstream -- passed (84 named, 5,305 + 7,351 randomized).
- git diff --check and git diff --cached --check -- passed.
Keep ZSTD_getFrameProgression's multithreaded path and C-owned frame
construction unchanged while moving the single-thread ingested-value
calculation behind the existing scalar C-to-Rust boundary. The Rust helper
accepts the consumed U64 value and buffered size_t value, explicitly converts
the latter to u64, and uses wrapping addition to match C's unsigned
arithmetic. Focused tests cover zero, ordinary, and overflowing inputs.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` -- passed before and after formatting.
- The same clippy command with `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused Rust tests for `frame_progression_ingested` -- 3 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 5,800, and 9,479 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream run retains the pre-existing warning at
`tests/zstreamtest.c:1899` about an unterminated initializer string.
Keep ZSTD_endStream's streaming call, error propagation, multithreaded
minimal estimate, logging, and context state in C. Move only the final
single-thread estimate behind a scalar Rust ABI helper so the frame-ended
branch and raw checksum flag are explicit while size_t arithmetic wraps as
in C.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused end-stream Rust tests -- 4 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 6,457, and 8,692 randomized tests. The pre-existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep the private ZSTD_CCtx and public buffer structures in C while moving the
read-only stable-buffer validation policy behind a scalar ABI. The Rust helper
compares expected and current input pointers and positions without dereferencing
raw pointers, and uses wrapping subtraction for the stable output remainder.
The C wrapper still extracts the context and buffer fields, and the existing
compressStream2 FORWARD_IF_ERROR path remains responsible for propagating the
encoded stability error. The unused endOp parameter is no longer carried into
the validation helper.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed, 309 tests.
- All three requested clippy modes passed before and after `cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed, 84 named plus 5,333 and 8,072 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
- `make -C tests -j2 test-zstream32` -- unavailable because the i686 Rust target is not installed.
Keep ZSTD_selectBlockCompressor's compressor tables, callback pointers, and
final dictionary-mode table lookup in C while moving only the scalar choice of
row-based versus ordinary table index to the Rust parameter-policy module. The
Rust helper reuses the existing row-matchfinder policy, returns 0..2 for row
entries, and returns 3 plus the strategy for ordinary entries. C retains its
public/internal signature, strategy and row-mode assertions, numeric fast
strategy assumption, and all private callback ownership.
Test Plan:
- Rust clippy normal, benches, and tests before and after nightly formatting -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- 41 passed.
- `make -B -C lib -j2 lib` -- passed without new warnings.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests plus 6,708 and 8,980 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- The pre-existing `tests/zstreamtest.c:1899` unterminated-string warning remains.
Keep buffer-pool ownership, allocation, synchronization, and the Rust
pool's saturating accounting in their existing implementations. The C
wrapper still handles NULL, obtains sizeof(*bufPool), and queries the Rust
pool size; Rust now performs only the final size_t addition with wrapping
semantics. The sequence-pool sizing alias continues to call the same C
helper and therefore retains its behavior.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress` -- 29 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- 57 passed
- `make -C tests -j2 test-zstream` -- 84 named, 7,661, and 8,090
randomized tests passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build still emits the pre-existing unterminated-string warning
at tests/zstreamtest.c:1899.
Keep all context-sensitive workspace sizing in C, including intermediate
allocation calculations, match-state and LDM sizing, error propagation, and
diagnostics. Move only the final nine-component neededSpace aggregation
behind a scalar Rust ABI helper so its size_t wrapping behavior is explicit
and independently tested.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress` -- 186 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests and 6,536 plus 7,961 randomized cases passed; the existing unterminated-string warning remains.
- `git diff --cached --check` -- passed.
ZSTDMT_sizeof_CCtx previously performed the complete size_t sum in C,
although every component depends on C-owned MT state and sizing helpers.
Keep the NULL guard, pool and dictionary queries, job-table multiplication,
and round-buffer extraction in C. Pass those eight computed components to a
Rust ABI helper that performs the same ordered wrapping additions.
This keeps MT context layout and allocator ownership on the C side while
moving only the scalar arithmetic across the existing C/Rust boundary.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression`
- Same clippy command with `--benches` and `--tests`, nightly fmt, then all three clippy commands again -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 289 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,268, and 8,862 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains its pre-existing unterminated-string warning at
`tests/zstreamtest.c:1899`.
ZSTDMT_nextInputSizeHint only derives a size_t hint from targetSectionSize
and inBuff.filled. Keep the MT context, public/internal entry point, and
stream orchestration in C, but delegate those two scalars to Rust. The Rust
helper uses wrapping subtraction and preserves the zero-result fallback,
including C's wrapped behavior for invalid overfill.
Test Plan:
- The three compression clippy modes passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- Focused Rust tests passed: 5 `mt_next_input_size_hint` tests.
- `make -B -C lib -j2 lib` passed.
- `make -C tests test-rust-lib-smoke` passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` passed: 57 tests completed.
- `make -C tests -j2 test-zstream` passed: 84 named, 6,659, and 8,055
randomized tests completed.
- `git diff --check` and `git diff --cached --check` passed.
ZSTD_selectSequenceCopier combined block-delimiter validation with its
private C function-pointer dispatch. Move only the scalar mode policy behind
a Rust ABI shim, retaining the function-pointer switch and both sequence
transfer implementations in C. Rust preserves the 0..=1 debug validation and
the release fallback to no-block-delimiters for unsupported values.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` — 281 passed.
- Required clippy checks for the library, benches, and tests passed before and after `cargo +nightly fmt`.
- `make -B -C lib -j2 lib` and `make -B -C tests test-rust-lib-smoke` — passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `tests/fuzzer -s4560 -t56 -i57 -v` — passed.
- `make -C tests -j2 test-zstream` — 84 named, 7,298 standard randomized, and 9,379 new-API randomized cases passed.
- `git diff --check` and `git diff --cached --check` — passed.
Port the final strategy-dependent transform in
ZSTD_dedicatedDictSearch_getCParams to a Rust ABI leaf that accepts and
returns the complete repr(C) compression-parameter struct. Keep C responsible
for ZSTD_getCParams_internal table selection and adjustment, including the
createCDict mode and its existing call order, as well as all dictionary and
context construction. The Rust leaf changes only hashLog for greedy, lazy,
and lazy2; every other and unexpected strategy is preserved unchanged.
Remove the obsolete scalar hash-log C ABI now that the complete-parameter
boundary is used. The focused Rust tests cover every strategy class and
unexpected values, field preservation, wrapping adjustment boundaries, and
the existing reverse transform's subtraction/clamp behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- passed (37 tests)
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --benches` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- passed (84 named tests; randomized phases of 6,557 and 7,884 cases)
- `git diff --check` and `git diff --cached --check` -- passed
Keep ZSTD_assertEqualCParams as a C-local wrapper so both existing callers and
surrounding control flow remain unchanged, but move its seven-field invariant
to the Rust compression-parameter module. The ABI helper takes both
repr(C) parameter structs by value, compares each named field explicitly with
debug_assert_eq!, and remains exported in release builds so the C linkage is
stable while the checks compile out like C assert under NDEBUG.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and
after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- 36 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -B -C tests -j2 test-zstream` -- 84 named and 15,504 randomized
cases passed; the existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep ZSTD_CStream context access and caller-specific streaming behavior in C,
while moving the pure stable-versus-buffered input-buffer policy to a scalar
Rust ABI helper. The helper returns the public repr(C) buffer layout, preserves
stable-mode source, size, and position values including NULL sources, and
returns a zeroed buffer for every other mode.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- the same clippy command with `--benches` and `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 274 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named tests and 7,400 plus 8,519 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
ZSTD_sizeof_localDict only combines the optional local dictionary buffer
size with the already-owned C dictionary size. Keep dictionary ownership and
workspace sizing in C, but pass the three scalar inputs to Rust so the leaf
uses explicit C size_t wrapping semantics. The CCtx NULL sizing guard and
surrounding ownership and call structure remain unchanged.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` (before and after formatting) -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 272 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` and direct `rustLibSmoke` execution -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./fuzzer -s4560 -t56 -i57 -v` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
Port ZSTD_dedicatedDictSearch_revertCParams to a Rust leaf that accepts and
returns the complete repr(C) compression-parameter struct. Keep the existing
C pointer wrapper and call order before ZSTD_adjustCParams_internal, while
changing only hashLog for greedy, lazy, and lazy2 strategies.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression (and --benches/--tests, rerun after fmt)
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- make -B -C tests -j2 fuzzer
- tests/fuzzer -s4560 -t47 -i48 -v
- tests/fuzzer -s4560 -t57 -i1057 -v
- make -B -C tests -j2 test-zstream
- git diff --check
Keep ZSTD_invalidateRepCodes responsible for extracting the private previous
block state and checking that the match window has no external dictionary.
Delegate only the fixed three-entry repcode reset through a narrow pointer ABI,
with a C assertion preserving the Rust side's ZSTD_REP_NUM assumption.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress::tests::invalidate_rep_codes_clears_all_entries` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Required clippy, nightly fmt, and diff checks -- passed
ZSTD_copyCDictTableIntoCCtx only transformed tagged CDict match-table
entries or copied untagged entries; context-owned table selection remains in
C. Keep that policy and both call sites in C, forwarding the selected flag
through a narrow ABI. Rust removes the frozen eight-bit short-cache tag or
uses copy_nonoverlapping for C memcpy-equivalent copying without adding
validation.
Test Plan:
- Focused and full compression Rust tests passed (2 and 267 tests).
- `make -B -C lib -j2 lib` passed.
- `make -C tests -j2 fuzzer` passed.
- `./tests/fuzzer -s4560 -t47 -i48 -v` passed.
- Clippy normal/benches/tests before and after `cargo +nightly fmt` passed.
- `git diff --check` and `git diff --cached --check` passed.
Keep C responsible for extracting the cdict and CCtx scalar fields while
moving the attach-versus-copy mode decision into the Rust parameter module.
The Rust ABI short-circuits absent cdicts and reuses the established
attachment predicate, preserving unknown-size, force-attach, force-copy, and
force-window behavior.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression
cparam_mode_preserves_cdict_presence_and_attachment_policy -- passed
- make -B -C lib -j2 lib -- passed
- make -C tests -j2 fuzzer -- passed
- ./tests/fuzzer -s4560 -t47 -i48 -v -- passed
- Required clippy normal/benches/tests, nightly fmt, and diff checks -- passed
The public streaming-size functions still calculated their values in the C
compressor entry point. Keep those public symbols and their size_t ABI in C,
but route the pure formulas through zstd_compress_api.rs. Rust uses usize for
the C size_t-equivalent constants and reuses its compressBound implementation;
the fixed block and header sizes make the output helper exactly 131591 bytes.
Test Plan:
- Focused Rust cstream size test -- passed (1 test)
- make -B -C lib -j2 lib -- passed
- make -C tests -j2 fuzzer -- passed
- ./tests/fuzzer -s4560 -t47 -i48 -v -- passed
- Required clippy, nightly fmt, and staged diff checks -- passed
Keep the private CDict and CCtx parameter layouts in C while passing only the
five scalar inputs needed by the dictionary-attachment heuristic. Rust now
owns the exact per-strategy cutoffs and preserves unknown-size handling,
dedicated-search short-circuiting, force-attach and force-copy precedence, and
the force-window prohibition. The existing C helper and reset/copy call paths
remain unchanged.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression dictionary_attachment` -- passed (2 tests)
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Rust clippy for the library, benches, and tests before and after nightly formatting -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `git diff --cached --check` -- passed
Keep dedicated-dictionary strategy dispatch, parameter selection, and call-site
control flow in C while routing only the hashLog field transformations through
scalar Rust ABI helpers. The helpers use wrapping u32 arithmetic so overflow
and the existing subtract-then-clamp behavior remain identical to C.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression dedicated_dict_search_hash_log_adjustments_match_strategy_cases` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- clippy normal, benches, and tests before and after formatting -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` and `git diff --check` -- passed
Keep the private writeBlockHeader signature and its debug logging in C while
moving the pure block-type, size, and 24-bit little-endian encoding behind a
narrow Rust ABI. The Rust leaf preserves the RLE blockSize path for cSize == 1
and the compressed cSize path for every other value.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression block_header -- --nocapture` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- Required clippy, nightly fmt, and diff checks -- passed
ZSTD_compressCCtx was selecting the Rust frame path for every compression
level. For source sizes that resolve to lazy or optimal strategies, that path
fell through to fast matching and could produce a different frame, including
the CCtx reuse regression caught by fuzzer test 56.
Query the source-size-dependent C parameters before preparing the context.
Keep fast and double-fast strategies on the existing Rust path, while routing
all other strategies through ZSTD_compress_usingDict with no dictionary so the
original simple API initializes requested parameters and C match state.
Test Plan:
- `make -B -C tests -j2 fuzzer` -- passed
- `./tests/fuzzer -s4560 -t56 -i56 -v` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 253 passed
- required clippy normal, benches, and tests checks -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
ZSTD_resolveExternalSequenceValidation() is a pure int identity used while
resolving CCtx parameters. Route its existing C wrapper through a Rust
extern "C" entry point in the parameter module, preserving the int-to-int
ABI and every call-site result. Add boundary coverage for signed int modes.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed (253 tests)
- `make -B -C lib -j2 lib` -- passed
- required clippy passes for the compression feature, benches, and tests -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` and `git diff --check` -- passed
The compression-parameter override helper previously lived in C and selectively
copied each nonzero field from a caller-provided override structure. Move that
pure field-wise operation into the Rust compression-parameter module, while
keeping the existing repr(C) structure and pointer-based ABI at both C call
sites. Zero-valued overrides continue to leave the current parameter untouched.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed (252 tests)
- `make -B -C lib -j2 lib` -- passed
- `make -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t47 -i48 -v` -- passed
- required clippy passes, nightly fmt, and `git diff --check` -- passed
Keep validation of the applied compression parameters in C and delegate the context-free minimum calculation through a narrow ABI leaf. Preserve the public getter and deprecated block-compression paths.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- make -C tests -j2 fuzzer
- tests/fuzzer -s4560 -t47 -i48 -v
Keep C ownership of the block-state structure while delegating the confirmed-block pointer swap through a narrow two-pointer ABI leaf. Preserve every existing confirmation call site and add direct pointer-swap coverage.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression
- cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -B -C lib -j2 lib
- tests/fuzzer -s4560 -t47 -i48 -v
- make -C tests -j2 test-zstream
Parse attached and separated --patch-from references, preserve the upstream ultra ceiling and conflict checks, and pass the reference path through the existing file-I/O dictionary ABI with patch mode enabled.
Test Plan:
- cargo test --manifest-path rust/cli/Cargo.toml
- cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- make -C tests -j2 check
- small attached/separate patch-from round-trip and rejection smoke
ddidderr