Legacy detection, frame-size queries, one-shot decode, and buffered streaming
were still routed through the C header dispatcher even though every historical
v0.1 through v0.7 decoder is already ported under Rust feature gates. The C
implementation also duplicated the support-level boundary independently from
the Rust archive selection.
Move the dispatcher and stream lifecycle to Rust using the existing DCtx field
projection. Rust now selects the version, preserves the null-buffer and
position semantics, initializes and frees buffered legacy contexts, and calls
the frozen per-version modules. C retains only the configuration leaf that
reports ZSTD_LEGACY_SUPPORT plus the private DCtx layout and trace helpers, so
library, program, and test builds can still choose different support levels.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder,legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07 --all-targets -- --test-threads=1` -- 528 passed.
- Native `test-legacy`, `test-invalidDictionaries`, `test-decodecorpus`, and `test-zstd` -- passed.
- Strict C syntax checks with legacy support 0, 4, 5, and 7 -- passed.
- `cargo clippy` default, benches, and tests for library and CLI -- passed.
The multithreaded compressor still handled the terminal empty-job case in C,
including the decision to acquire a buffer, emit the final empty block, and
clear the consumed source range. That left a small but stateful branch outside
the Rust job helpers and made allocation-failure behavior difficult to test.
Project the terminal job and result through a stable C ABI, then let Rust own
that branch's assertions, buffer callback, block-header serialization, and
failure-preserving result. C remains responsible for its private job layout
and applies the returned buffer, source, and compressed-size fields exactly as
before.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- Native multithreaded cases in `test-fuzzer`, `test-zstream`, `test-cli-tests`, and `test-zstd` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1` -- passed.
The public and deprecated block APIs previously entered a C-owned
ZSTD_compressContinue_internal routine, which mixed frame-header sequencing,
window maintenance, block dispatch, and frame-size accounting with the private
CCtx match state. That made the high-level compression boundary both harder
to test and easy to diverge from the already-ported frame-chunk body.
Add an explicit C/Rust projection containing only scalar state and callback
slots. Rust now owns stage validation and transitions, header sequencing,
frame-versus-block dispatch, and consumed/produced progression, while C
callbacks retain access to the private match state and context-sensitive block
operations. The same path now covers compressContinue, deprecated block
compression, and compressEnd without exposing the unstable CCtx layout.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder,legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07 --all-targets -- --test-threads=1` -- 528 passed.
- Native `test-fuzzer`, `test-zstream`, `test-decodecorpus`, `test-cli-tests`, and `test-zstd` -- passed.
- `cargo clippy` default, benches, and tests for library and CLI -- passed.
Port sequence-decoder selection and offset-history policy into Rust. The
Rust block decoder now handles short/long selection, offset-table analysis,
history thresholds, and prefetch sequencing; C projects only the decoder mode
and private context fields required by the ABI.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0
- make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1
- make -B -C tests -j2 test-zstd
Move the post-sequence-store body of ZSTD_compressBlock_internal behind a
small C/Rust state projection. C continues to build the sequence store and
handle the no-compress and sequence-producer error paths, while Rust now owns
sequence collection, entropy emission, the legacy non-first-frame RLE gate,
compressed-block confirmation, and offcode repeat cleanup. Remove the C
wrappers that became dead after those leaves moved behind the Rust body.
The first integration run exposed that leaving the old C finalization label in
place confirmed compressed block state twice, undoing Rust's pointer swap and
breaking a later sparse-file checksum. The C wrapper now returns directly for
the Rust-owned path and retains only its C-owned no-compress cleanup.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo check --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-zstd
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
- make -B -C tests/fuzz -j2 all
- make -B -C tests/fuzz -j2 sequence_compression_api
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings (pre-existing manual_repeat_n failure)
- cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings (pre-existing manual_repeat_n failure)
Move the post-split partition loop out of zstd_compress.c while keeping the
private CCtx, matchfinder sequence-store construction, and split discovery in
C. Rust now receives a layout-asserted projection containing sequence-store
views, block-state slots, workspace and scalar policy, then mirrors the C
loop's dRep/cRep histories, final-literal accounting, repeated single-block
serialization, and final dRep publication.
Remove the obsolete C sequence-store/count/chunk wrappers and the debug-only
C size-estimation path that was coupled to the old loop. Add a focused Rust
fixture covering partition payloads and final literals, and document the new
ownership boundary for the split search and emission paths.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1 (448 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings (passed)
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings (passed)
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (passed)
- cargo clippy --tests/--benches remain blocked only by the pre-existing manual_repeat_n lint in one_shot_promotes_nonfirst_rle_blocks
- make -B -C lib -j2 lib (passed)
- make -B -C tests -j2 test-zstd (passed)
- make -B -C tests -j2 test-cli-tests (41 passed)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (passed)
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer (252 passed)
- make -B -C tests/fuzz -j2 all and sequence_compression_api (passed)
Move the target-compressed-block body out of zstd_compress.c while keeping
sequence-store construction, matchfinding, context lifetime, and the outer
offcode repeat-mode cleanup in C. Rust now owns the target policy, repeated
block decision, Rust superblock call, compressed-state confirmation, and raw
fallback. The narrow C projection asserts the cross-language layout instead
of exposing ZSTD_CCtx.
Add focused tests for first-block RLE suppression, RLE output, superblock
errors, raw fallback, and state-slot swapping. Remove the C-only maybeRLE
wrapper that became unreachable after the extraction.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
Move the per-block body of ZSTD_compressSequences_internal behind an explicit
Rust projection. The C wrapper still owns context initialization, public API
validation, frame headers, checksums, and the private CCtx layout. Rust now
owns block sizing and sequence transfer, sequence-store reset, entropy
compression, raw/RLE/compressed block selection, block headers, repcode/state
swapping, repeat-mode transition, and first-block handling.
The bridge passes only the sequence store, block-state pointer slots, workspace,
policy scalars, dictionary size, and the isFirstBlock slot. It does not pass a
CCtx or C callback across the ABI. The tests cover empty-block headers,
capacity errors, first-block RLE restrictions, and entropy fallback decisions.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -C tests -j2 test-fuzzer (covers ZSTD_compressSequences at fuzzer test 190)
- git diff --cached --check
Keep the MT chunk diagnostic after moving chunk iteration into Rust without
retaining a C local that is unused when debug logging is compiled out.
Test Plan:
- make -C lib -j2 lib
- git diff --cached --check
Move the bounded multithreaded output-publication kernel into Rust. Rust now
calculates the flush amount, copies the selected job bytes, and returns the
updated output and job offsets through an explicit C-layout result. C retains
mutex and condition-variable handling, checksum insertion, job retirement,
progress accounting, and the terminal state machine.
The Rust helper validates the scalar bounds before copying and has focused
coverage for empty output space, offset partial flushes, repeated flushes,
complete flushes, sentinels, and invalid bounds.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (blocked by concurrent fileio worker formatting its uncommitted file)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (worker)
- git diff --cached --check
ZSTDMT_compressionJob kept the chunking kernel in C while Rust already owned
most of the MT pool and progress helpers. The loop mixed compression calls
with mutex and condition-variable publication, making it difficult to test
its exact final-chunk and partial-tail behavior independently.
Move only the chunk-processing kernel into Rust. It calls the existing C
compression entry points through an explicit ABI, reports the final block size
or error in a repr(C) result, and invokes a C progress callback after each
successful intermediate chunk. C retains worker setup, serial and raw-sequence
state, synchronization, error publication, cleanup, and final completion
reporting. Mock compressors cover empty, exact, partial, progress, and error
boundaries without mirroring private MT job state in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress --
--test-threads=1` -- passed (44 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream` -- passed,
including multi-threaded dictionary/checksum coverage.
Compression-parameter table selection and adjustment were already implemented
as Rust leaves, but the internal and public ZSTD_getCParams/ZSTD_getParams
helpers still assembled those results in the C translation unit. That left
source-size and dictionary-mode policy duplicated at the C/Rust boundary.
Add Rust implementations for the internal and public getter policies. The
internal helpers preserve a zero source size as a known empty input, while the
public helpers retain the API rule that zero means unknown. The C side keeps
only the build-specific exclusion-mask construction and thin ABI adapters, so
reduced builds continue to select the same available strategy cascade.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1`
-- passed (411 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings`
-- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- Focused tests cover public zero-to-unknown translation, internal zero
semantics, default frame parameters, and dictionary attachment modes.
The Rust simple-compression entry points already owned the dispatch, but
ZSTD_rust_compressCCtxStrategy still called the C parameter cascade to decide
whether a fast Rust frame path was safe. That left a small policy leaf in the
C implementation and made the dispatch boundary harder to test in isolation.
Implement the helper in Rust by using the existing Rust table-selection and
C-parameter-adjustment functions with the same source-size, dictionary-mode,
and automatic-adjustment inputs as the C code. Leave only the C ABI declaration
and add tests for the table thresholds, default-level behavior, and negative
fast-level behavior. The private context reset and fallback compression paths
remain unchanged.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress --
--test-threads=1` -- passed (234 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests -j2 test-rust-lib-smoke` -- passed.
- Full-target clippy remains blocked by pre-existing test-only
`manual_repeat_n` and `manual_dangling_ptr` warnings outside this seam.
The optimal block splitter stores up to 196 split points and then needs one
additional partition entry for the terminal sequence boundary consumed by the
C block-emission loop. The previous array declaration provided only the split
slots, so a maximally partitioned block could write the terminal boundary past
the projected Rust/C state. Reserve the extra entry and document the layout;
retain focused tests for equal-cost and estimation-error cases so the splitter
continues to avoid unnecessary or invalid partitions.
Test Plan:
- Focused block-split Rust tests -- 7 passed
- `make -B -C lib -j2 lib` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- Scoped `git diff --check` -- passed
Move the active-stage check and pledged-size update behind a scalar Rust ABI
leaf while keeping the CCtx layout and public C entry point in C. Preserve the
stage_wrong error encoding and the unsigned wrap behavior at U64::MAX.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (80 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C programs -j2 zstd
- git diff --check
Keep the private ZSTD_window_t layout and pointer-difference calculation in C,
then route the size_t-to-U32 conversion and paired limit writes through a small
Rust ABI leaf. This removes the inline C window-clear implementation while
preserving its overflow behavior at both single-threaded and multithreaded
call sites.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (77 passed)
- make -C programs -j2 zstd
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C tests -j2 test-cli-tests (41 passed)
- make -C tests -j2 test-legacy test-invalidDictionaries test-decodecorpus test-rust-lib-smoke (passed)
- git diff --check
Include localDict.dict in the eligibility guard for the simple Rust compress2 route. By-reference dictionary loads populate this field without populating dictBuffer, so omitting it incorrectly allowed a dictionary-bearing context to bypass the C dictionary path.
Test Plan:
- ./tests/fuzzer -s9634 -v
- git diff --check
ZSTDMT_expandJobsTable previously kept worker-capacity comparison, old-table
synchronization teardown, replacement allocation, initialization failure cleanup,
and mask updates in C. That left the remaining job-table lifecycle orchestration
outside the Rust storage leaves. Add a narrow Rust ABI that exchanges only
opaque storage and scalar mask state and accepts C callbacks for the private
synchronization lifecycle.
The adapter frees the old table before replacement, reuses the existing Rust
create/free leaves with the caller's custom memory, destroys a partially
initialized replacement before freeing it, and updates jobIDMask only after
successful initialization. C still owns ZSTDMT_CCtx and job descriptors, so
worker-facing fields and platform synchronization stay outside the Rust ABI.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features
compression zstdmt_compress --lib` -- 38 passed.
- `cargo clippy`, `cargo clippy --benches`, and `cargo clippy --tests` -- passed.
- `cargo +nightly fmt --all -- --check` -- passed.
- `make -B -C lib lib-mt` -- passed.
- `make -B -C tests -j2 fullbench poolTests` -- passed.
- `./poolTests` and `./fullbench -i1 -B1000 ../README.md` -- passed.
Keep the C header's ZSTD_wildcopy signature as the enum-facing wrapper, but
move its over-copying implementation into rust/src/mem.rs. The wrapper passes
the enum's int representation to ZSTD_rust_wildcopy, where only the two valid
values are converted to ZstdOverlap. This leaves C's ZSTD_copy16/COPY16
helpers available to their direct compression callers and removes the now
unused COPY8 helper. ZSTD_copy16 is marked unused-safe for C translation units
that include the shared header without calling that direct helper.
The Rust leaf preserves the original first-copy behavior for zero and short
lengths, the source-before-destination 8-byte do-while path, the no-overlap
distance assertion, and the first-then-two-COPY16 loop. Its ABI contract does
not take ownership of caller buffers. ABI tests use 32-byte padded buffers and
exercise no-overlap plus offsets 8 and 15 across the boundary lengths, checking
both copied bytes and guard regions.
Test Plan:
- `cargo test mem::tests` -- passed (4 tests)
- `make lib-nomt` and `make lib-mt` -- passed
- `make -C tests test-zstream` -- passed
- `make -C tests test-fullbench` -- completed; its `-P0` run printed the
existing Scenario 17 diagnostic, but the target returned normally
- `cargo clippy`, `cargo clippy --benches`, `cargo clippy --tests`,
`cargo +nightly fmt`, then the same three clippy commands -- passed on the
final repeat
- `git diff --cached --check` -- passed
Keep the LDM limit-table update on the Rust compression path while leaving
match-state ownership in C. Previously, the C block wrapper combined pointer
subtraction, match-state access, and the bounded scalar update. The wrapper
now computes `curr` from `anchor - window.base`, passes `curr` and
`nextToUpdate` through the narrow U32 ABI, and stores Rust's result before the
existing fast-table dispatch.
The Rust leaf uses explicit wrapping arithmetic to preserve the C U32 behavior:
the strict `curr > nextToUpdate + 1024` threshold and the `MIN(512, ...)`
clamp. Focused tests cover the threshold, one-step update, clamp, nonzero
starting point, and arithmetic wraparound.
Test Plan:
- `cargo test zstd_ldm` -- default-feature test-binary link failed because
existing dict-builder C symbols are not linked.
- `cargo test --no-default-features --features compression zstd_ldm` -- passed
(7 tests).
- `make lib-nomt` -- passed.
- `make lib-mt` -- passed.
- `make -C tests test-zstream` -- passed; it emitted the existing
`tests/zstreamtest.c` unterminated-string warning.
- `cargo clippy`, `cargo clippy --benches`, `cargo clippy --tests`,
`cargo +nightly fmt`, then the same clippy sequence -- passed.
The Rust dispatch gate for ZSTD_compress2 and complete-input
ZSTD_compressStream2 calls was selecting compression parameters with an
unknown content size. For the 30-byte level-4 CCtx reuse case that chooses the
dfast strategy, while the actual source size selects greedy; the Rust frame
then emitted different output from the C-owned ZSTD_compressCCtx path.
Pass the actual source size through both helper ABIs. The one-shot and complete
stream entry points now select the same strategy as C, while the existing
sentinel keeps every non-fast/non-dfast configuration on the original C
fallback. The existing fuzzer test 56 exercises the regression without
modifying the shared C test harness.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4560 -t56 -i57 -v` --
passed; the 30-byte test-56 regression now completes.
- `./tests/fuzzer -s4142 -t63 -i64 -v` -- passed all 64 focused cases.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 4,385 first
randomized, and 8,137 new-API randomized cases.
- `make -C tests -j2 test-fuzzer` reaches test 113's flat-dictionary
efficiency assertion with concurrent unstaged LDM changes; test 56 passes.
The window overflow gate mixed two scalar decisions with C-owned pointer and
window state. The early correction test uses the block-start index and must
wait for dictionary invalidation, while the normal fallback uses the block-end
index and the platform-selected ZSTD_CURRENT_MAX threshold. Keeping those
subtractions and the ZSTD_window_t wrapper in C preserves the existing call
contract without making Rust depend on pointer width or private state.
Move the U32 cycle/MAX arithmetic, correction-count scaling, dictionary gate,
and frequent-policy decision into Rust. C passes the two pointer-derived
indices only through scalar results, and supplies the current-max threshold
and active build policy explicitly so 32/64-bit and fuzzing configurations
remain authoritative.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4142 -t63 -i64 -v` --
passed all 64 cases.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 5,477 first
randomized, and 6,902 new-API randomized cases; the existing unterminated
string initializer warning remains.
ZSTD_reduceIndex previously selected and reduced each match table through
separate C helpers, even though the 16-cell reduction leaf already lived in
Rust. Keep C responsible for match-state access, table allocation policy,
nullable table selection, btlazy2 strategy selection, and the surrounding
workspace/window/dictionary state transition. Add one narrow Rust ABI that
receives only the validated hash, chain, and hash3 table slices and delegates
to the existing reducer, preserving chain markers only when requested and
keeping threshold and U32 wrapping arithmetic unchanged. Focused tests cover
zero-sized optional tables, marker routing, and threshold behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress::tests::reduce_` -- 5 passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 344 passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,814 standard
randomized, and 8,955 new-API randomized cases passed.
- Required compression clippy library/benches/tests, nightly fmt, and the repeated three clippy checks -- passed.
Move the legacy buffered-stream dispatcher out of the C decompression adapter
while keeping ZSTD_DCtx_s private and C-owned. Rust now consumes the existing
ZSTD_rustDctxView projection, detects the legacy version at input.pos, preserves
static-context rejection and stream-stage reset behavior, and handles legacy
context replacement through the version-gated v0.4-v0.7 Rust stream APIs.
The Rust boundary retains the C helper's NULL normalization, dictionary
lifetime sentinel, previous-version cleanup semantics, input/output position
updates, and version-unsupported and no-legacy-support fallbacks. The C legacy
free helper remains unchanged; only the dispatcher declaration and body were
removed from the adapter. Focused tests cover unsupported versions, stage
reset, context reuse/switching, and partial v0.4 input/output progress.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed
- Same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml`, followed by all three clippy commands -- passed
- Decompression/legacy clippy with `-D warnings` -- passed
- All 107 Rust tests with v0.4-v0.7 features -- passed
- Focused legacy dispatcher tests: 4 passed
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed
- `make -C tests -j2 test-legacy` -- built, then failed in the existing simple one-shot path with one v0.8 output byte mismatch (`e0` vs expected `e2`) before the streaming check; the new dispatcher symbol was not called on that failing path
The window-overflow helper mixed pointer arithmetic and ZSTD_window_t state
updates with the scalar U32 cycle calculation that determines the correction.
That made the arithmetic boundary depend on the host pointer implementation.
Add a narrow Rust ABI leaf that accepts only the current index, cycle log, and
maximum distance. Rust preserves the start-index adjustment, cycle alignment,
cycle-size versus max-distance selection, and explicit U32 wrapping. C derives
newCurrent from the returned correction so it can retain every invariant check,
pointer update, workspace/table transition, counter update, and dictionary
invalidation in the existing stateful wrapper.
Test Plan:
- Rust compression clippy, bench clippy, test clippy, nightly fmt, and the
repeated three clippy checks -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 340 passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `./tests/fuzzer -s4142 -t63 -i64 -v` -- focused overflow coverage passed.
- `make -C tests -j2 test-zstream` -- 84 deterministic, 5,018 standard
randomized, and 8,851 new-API randomized cases passed.
- `make -C tests -j2 test-fuzzer` reaches the pre-existing CCtx-reuse test 56
failure; a pristine 837e4f099 worktree reproduces it, while the focused
test-56 run passes.
ZSTD_getCParamsFromCCtxParams previously kept its context-free selection,
LDM window override, nonzero-field override, and final adjustment orchestration
in C. Move that pipeline into a scalar Rust ABI leaf while keeping the private
ZSTD_CCtx_params snapshot in C. The ABI passes the compression parameters by
value, the source-size hint, mode switches, and the LDM default window log.
The active ZSTD_EXCLUDE_* definitions remain owned by the C preprocessor and
are encoded as a mask. Rust applies that mask before each adjustment stage,
including the existing standalone adjustment callers, so reduced compressor
builds retain the original fallback cascade without duplicating CCtx state or
compile-time configuration in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression`
-- passed, 337 tests.
- All three compression clippy commands passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic tests,
5,191 standard randomized cases, and 9,382 new-API randomized cases.
- `git diff --cached --check` -- passed.
Move the arithmetic aggregation in ZSTDMT_getFrameProgression across the
existing C/Rust boundary. C still owns MT state traversal, mutex locking,
jobIDMask ordering, nextJobID plus jobReady handling, error normalization,
and the flushed <= produced assertion. Rust constructs the base result and
folds each compact, C-normalized job snapshot into the complete repr(C)
ZSTD_frameProgression, preserving size_t-to-U64 wrapping and active-worker
counting without exposing private job or mutex layouts.
Focused Rust tests cover zero jobs, a ready job, completed jobs, normalized
error output, and mixed active/completed aggregation.
Test Plan:
- All three compression clippy commands passed before formatting.
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- passed.
- All three compression clippy commands passed after formatting.
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression -- 332 passed.
- make -B -C lib -j2 lib-mt -- passed.
- make -B -C lib -j2 lib-nomt -- passed.
- make -B -C tests -j2 test-zstream -- passed: 84 named, 6,004, and 8,392
randomized cases.
- git diff --check and git diff --cached --check -- passed.
The zstream build retained the pre-existing warning at
tests/zstreamtest.c:1899 about an unterminated initializer string.
Complete the single-thread branch of ZSTD_getFrameProgression behind a
narrow scalar ABI boundary. C still owns the public entry point, buffered
input extraction, assertions, and the multithreaded dispatch; Rust now owns
the six-field single-thread result construction. The Rust #[repr(C)] result
uses wrapping u64 conversion for consumed input plus buffered size, mirrors
produced into flushed, and zeros the MT-only fields.
This replaces the earlier ingested-only helper so the ABI no longer splits
one result across duplicate C and Rust construction paths. The unit tests now
cover the complete result and its u64 wrapping behavior.
Test Plan:
- All three requested compression clippy commands passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 327 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 5,330, and 9,772
randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains the pre-existing warning at
`tests/zstreamtest.c:1899` about an unterminated initializer string.
ZSTD_estimateCCtxSize_internal() evaluates four fixed source-size tiers and
must retain every C-owned parameter and sizing call, including raw error-shaped
size_t results. Store those four results in C, then delegate only the final
unsigned maximum comparison to a scalar Rust ABI helper. The outer monotonic
compression-level budget and the separate CStream estimation path remain
unchanged.
The Rust unit tests cover zero, ordinary, equal, SIZE_MAX, and encoded error
values to document that the helper compares raw size_t values without arithmetic
or error interpretation.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- Same clippy command with `--benches` -- passed before and after formatting
- Same clippy command with `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 328 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named plus 5,923 and 9,247 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build retains the pre-existing initializer-string warning at
tests/zstreamtest.c:1899.
ZSTD_literalsCompressionIsDisabled() previously switched on the private
ZSTD_CCtx_params layout in a header inline helper. Replace that switch with a
scalar ABI call that extracts only literalCompressionMode, strategy, and
targetLength in C. The Rust helper preserves the enable and disable results,
the fast-strategy auto rule, and C's debug assertion plus release fallthrough
for invalid modes. Keep callers and compression state ownership in C, and
cover the policy truth table and debug invalid-mode invariant in Rust.
Test Plan:
- Compression-feature clippy for the library, benches, and tests before and
after nightly formatting -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- passed (43 tests).
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed (84 named, 5,416, and 8,387
randomized cases).
- `git diff --check` and `git diff --cached --check` -- passed.
The MT rsync initialization previously computed the rolling-hash prime
power through the C-only ZSTD_rollingHash_primePower helper. Move only that
scalar exponentiation behind the existing Rust MT ABI boundary. The C caller
still owns the MT context assignment, RSYNC_LENGTH constant, and surrounding
initialization.
The Rust helper mirrors ZSTD_ipow with wrapping u64 multiplication and applies
u32 wrapping subtraction before widening the exponent, preserving the C
length == 0 behavior. Rolling-hash scanning, rotation, and all other MT state
remain unchanged.
Test Plan:
- Focused rolling_hash_prime_power Rust tests -- passed (2 tests).
- Required clippy/fmt sequence in normal, bench, and test modes -- passed.
- make -B -C lib -j2 lib -- passed.
- make -C tests test-rust-lib-smoke -- passed.
- tests/fuzzer -s4560 -t56 -i57 -v -- passed (57 tests).
- make -C tests -j2 test-zstream -- passed (84 named, 5,305 + 7,351 randomized).
- git diff --check and git diff --cached --check -- passed.
Keep ZSTD_getFrameProgression's multithreaded path and C-owned frame
construction unchanged while moving the single-thread ingested-value
calculation behind the existing scalar C-to-Rust boundary. The Rust helper
accepts the consumed U64 value and buffered size_t value, explicitly converts
the latter to u64, and uses wrapping addition to match C's unsigned
arithmetic. Focused tests cover zero, ordinary, and overflowing inputs.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` -- passed before and after formatting.
- The same clippy command with `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused Rust tests for `frame_progression_ingested` -- 3 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 5,800, and 9,479 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream run retains the pre-existing warning at
`tests/zstreamtest.c:1899` about an unterminated initializer string.
Keep ZSTD_endStream's streaming call, error propagation, multithreaded
minimal estimate, logging, and context state in C. Move only the final
single-thread estimate behind a scalar Rust ABI helper so the frame-ended
branch and raw checksum flag are explicit while size_t arithmetic wraps as
in C.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- Focused end-stream Rust tests -- 4 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed: 84 named, 6,457, and 8,692 randomized tests. The pre-existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep the private ZSTD_CCtx and public buffer structures in C while moving the
read-only stable-buffer validation policy behind a scalar ABI. The Rust helper
compares expected and current input pointers and positions without dereferencing
raw pointers, and uses wrapping subtraction for the stable output remainder.
The C wrapper still extracts the context and buffer fields, and the existing
compressStream2 FORWARD_IF_ERROR path remains responsible for propagating the
encoded stability error. The unused endOp parameter is no longer carried into
the validation helper.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed, 309 tests.
- All three requested clippy modes passed before and after `cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- passed, 84 named plus 5,333 and 8,072 randomized cases.
- `git diff --check` and `git diff --cached --check` -- passed.
- `make -C tests -j2 test-zstream32` -- unavailable because the i686 Rust target is not installed.
Keep ZSTD_selectBlockCompressor's compressor tables, callback pointers, and
final dictionary-mode table lookup in C while moving only the scalar choice of
row-based versus ordinary table index to the Rust parameter-policy module. The
Rust helper reuses the existing row-matchfinder policy, returns 0..2 for row
entries, and returns 3 plus the strategy for ordinary entries. C retains its
public/internal signature, strategy and row-mode assertions, numeric fast
strategy assumption, and all private callback ownership.
Test Plan:
- Rust clippy normal, benches, and tests before and after nightly formatting -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- 41 passed.
- `make -B -C lib -j2 lib` -- passed without new warnings.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `-s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests plus 6,708 and 8,980 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- The pre-existing `tests/zstreamtest.c:1899` unterminated-string warning remains.
Keep buffer-pool ownership, allocation, synchronization, and the Rust
pool's saturating accounting in their existing implementations. The C
wrapper still handles NULL, obtains sizeof(*bufPool), and queries the Rust
pool size; Rust now performs only the final size_t addition with wrapping
semantics. The sequence-pool sizing alias continues to call the same C
helper and therefore retains its behavior.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress` -- 29 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- 57 passed
- `make -C tests -j2 test-zstream` -- 84 named, 7,661, and 8,090
randomized tests passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build still emits the pre-existing unterminated-string warning
at tests/zstreamtest.c:1899.
Keep all context-sensitive workspace sizing in C, including intermediate
allocation calculations, match-state and LDM sizing, error propagation, and
diagnostics. Move only the final nine-component neededSpace aggregation
behind a scalar Rust ABI helper so its size_t wrapping behavior is explicit
and independently tested.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting.
- The same command with `--benches` and `--tests` -- passed before and after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress` -- 186 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named tests and 6,536 plus 7,961 randomized cases passed; the existing unterminated-string warning remains.
- `git diff --cached --check` -- passed.
ZSTDMT_sizeof_CCtx previously performed the complete size_t sum in C,
although every component depends on C-owned MT state and sizing helpers.
Keep the NULL guard, pool and dictionary queries, job-table multiplication,
and round-buffer extraction in C. Pass those eight computed components to a
Rust ABI helper that performs the same ordered wrapping additions.
This keeps MT context layout and allocator ownership on the C side while
moving only the scalar arithmetic across the existing C/Rust boundary.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression`
- Same clippy command with `--benches` and `--tests`, nightly fmt, then all three clippy commands again -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 289 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,268, and 8,862 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains its pre-existing unterminated-string warning at
`tests/zstreamtest.c:1899`.
ZSTDMT_nextInputSizeHint only derives a size_t hint from targetSectionSize
and inBuff.filled. Keep the MT context, public/internal entry point, and
stream orchestration in C, but delegate those two scalars to Rust. The Rust
helper uses wrapping subtraction and preserves the zero-result fallback,
including C's wrapped behavior for invalid overfill.
Test Plan:
- The three compression clippy modes passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- Focused Rust tests passed: 5 `mt_next_input_size_hint` tests.
- `make -B -C lib -j2 lib` passed.
- `make -C tests test-rust-lib-smoke` passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` passed: 57 tests completed.
- `make -C tests -j2 test-zstream` passed: 84 named, 6,659, and 8,055
randomized tests completed.
- `git diff --check` and `git diff --cached --check` passed.
ZSTD_selectSequenceCopier combined block-delimiter validation with its
private C function-pointer dispatch. Move only the scalar mode policy behind
a Rust ABI shim, retaining the function-pointer switch and both sequence
transfer implementations in C. Rust preserves the 0..=1 debug validation and
the release fallback to no-block-delimiters for unsupported values.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` — 281 passed.
- Required clippy checks for the library, benches, and tests passed before and after `cargo +nightly fmt`.
- `make -B -C lib -j2 lib` and `make -B -C tests test-rust-lib-smoke` — passed.
- `tests/fuzzer -s4560 -t47 -i48 -v` and `tests/fuzzer -s4560 -t56 -i57 -v` — passed.
- `make -C tests -j2 test-zstream` — 84 named, 7,298 standard randomized, and 9,379 new-API randomized cases passed.
- `git diff --check` and `git diff --cached --check` — passed.
Port the final strategy-dependent transform in
ZSTD_dedicatedDictSearch_getCParams to a Rust ABI leaf that accepts and
returns the complete repr(C) compression-parameter struct. Keep C responsible
for ZSTD_getCParams_internal table selection and adjustment, including the
createCDict mode and its existing call order, as well as all dictionary and
context construction. The Rust leaf changes only hashLog for greedy, lazy,
and lazy2; every other and unexpected strategy is preserved unchanged.
Remove the obsolete scalar hash-log C ABI now that the complete-parameter
boundary is used. The focused Rust tests cover every strategy class and
unexpected values, field preservation, wrapping adjustment boundaries, and
the existing reverse transform's subtraction/clamp behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression zstd_compress_params` -- passed (37 tests)
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --benches` -- passed before and after formatting
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression --tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- passed (84 named tests; randomized phases of 6,557 and 7,884 cases)
- `git diff --check` and `git diff --cached --check` -- passed
Keep ZSTD_assertEqualCParams as a C-local wrapper so both existing callers and
surrounding control flow remain unchanged, but move its seven-field invariant
to the Rust compression-parameter module. The ABI helper takes both
repr(C) parameter structs by value, compares each named field explicitly with
debug_assert_eq!, and remains exported in release builds so the C linkage is
stable while the checks compile out like C assert under NDEBUG.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- passed before and after formatting.
- The same clippy command with `--benches` and `--tests` -- passed before and
after formatting.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstd_compress_params` -- 36 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -B -C tests -j2 test-zstream` -- 84 named and 15,504 randomized
cases passed; the existing unterminated-string warning remains.
- `git diff --check` and `git diff --cached --check` -- passed.
Keep ZSTD_CStream context access and caller-specific streaming behavior in C,
while moving the pure stable-versus-buffered input-buffer policy to a scalar
Rust ABI helper. The helper returns the public repr(C) buffer layout, preserves
stable-mode source, size, and position values including NULL sources, and
returns a zeroed buffer for every other mode.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed before and after formatting
- the same clippy command with `--benches` and `--tests` -- passed before and after formatting
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 274 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed
- `make -C tests -j2 test-zstream` -- 84 named tests and 7,400 plus 8,519 randomized cases passed
- `git diff --check` and `git diff --cached --check` -- passed
ddidderr