Move the CDict-to-CCtx hash, chain, and row-match table copies into the
Rust reset orchestration. The C bridge now exposes table-field addresses
and scalar policy inputs while retaining private reset and hashTable3
operations; Rust loads destination pointers only after reset and strips
short-cache tags through the existing Rust leaf.
The destination chain policy must be read after the reset callback updates
appliedParams. Reading it before reset skipped DFAST chain copies and caused
the dictionary source-size zstream test to segfault in the double-fast
external-dictionary matcher. Focused tests cover tagged hash/chain copies,
row-match tag/salt copies, callback ordering, and reset-error short-circuiting.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Replace the remaining CCtx-copy custom-memory callback with explicit source and
destination pointers. Rust now copies the ABI-checked ZSTD_customMem payload in
its original position before reset, while C retains private context access and
the surrounding workspace and table operations.
Test Plan:
- ulimit -v 41943040 cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040 make -j1
- ulimit -v 41943040 make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040 make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Pass the CCtx compressed-block-state pointer slot and CDict source state
through the attach/copy bridges, then perform the final state copy in Rust
after reset has established the destination. Remove the redundant C memcpy
adapters while preserving dictionary attach/copy ordering and null/error
behavior.
Add focused tests for copied repcodes and entropy repeat state.
Test Plan:
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
- ulimit -v 41943040 && cargo fmt --manifest-path rust/Cargo.toml -- --check
- git diff --check
Route ZSTD_initStaticCDict() workspace-size calculation through the Rust
CDict sizing policy, including the by-reference/by-copy, resolved row
match-finder, and dedicated-search branches. Keep private layout constants
at the C boundary and retain C ownership of static workspace construction
and dictionary initialization.
Remove the obsolete C match-state sizing adapter now that no C path calls
it, avoiding a dead implementation after the sizing migration.
Test Plan:
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
- ulimit -v 41943040 && cargo fmt --manifest-path rust/Cargo.toml -- --check
- git diff --check
Move the advanced-CDict workspace-size formula and its by-reference,
by-copy, row-match-finder, and dedicated-search branches into the Rust
compression-parameter layer. Keep private C layout constants and the
already-resolved mode at the C boundary, where the allocator still owns
private object layout details.
Add a focused test covering the load-method and dedicated-search deltas
and the exported advanced-sizing boundary.
Test Plan:
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
- ulimit -v 41943040 && cargo fmt --manifest-path rust/Cargo.toml -- --check
- git diff --check
The transparent stream initializer previously called a C helper that both
inspected local-dictionary state and chose the no-dictionary, already-created,
and create/publish branches. Project that private state through a checked C/Rust
ABI so Rust owns the branch and callback order while C retains the requested
parameters, custom allocator, and CDict implementation details. Keep creation
and publication split so allocation failure cannot publish a partial dictionary.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml compress_stream_init_
- CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -j1
- make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
- CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
Move match-table selection and btlazy2 marker policy out of the C overflow
correction wrapper. Rust now decides whether chain and hash3 tables
participate using the strategy, row-matchfinder, dedicated-dictionary, and
log inputs, then delegates cell updates to the existing Rust reducer. C
retains only the private table pointers and scalar projection.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml zstd_compress::tests::reduce_index_for_match_state -- --nocapture
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Route match-state invalidation through a Rust leaf. C still computes the
private window pointer difference and passes the opaque dictionary-state
pointer, while Rust preserves the window-limit, next-index, dictionary,
optimal-parser, and dictionary-match-state reset order. Add a focused test
for all invalidated fields and use the concrete C pointer type at the ABI
boundary to avoid strict-aliasing warnings.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml zstd_compress::tests::invalidate_match_state -- --nocapture
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Move the bounded nextToUpdate catch-up arithmetic used after very long
matches into Rust. C continues to calculate the source index against the
private match-state window and retains all surrounding assertions and setup;
Rust owns the exact wrapping U32 threshold and 192-byte catch-up policy.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml limit_next_to_update --lib (1 passed)
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib (692 passed)
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && MAKEFLAGS=-j1 make -B -C programs -j1 zstd
- ulimit -v 41943040 && MAKEFLAGS=-j1 make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s (84 tests and both short fuzz rounds passed)
Move the scalar planning portion of ZSTD_resetCCtx_internal across an
explicit C/Rust boundary. Rust now computes window and block sizes, sequence
capacities, buffered I/O sizes, LDM and external-sequence reservations, the
index-reset policy, and the CCtx workspace estimate. C retains private
workspace resizing, reservations, pointer publication, and layout-dependent
callbacks.
Add ABI layout assertions and focused planner tests for buffered sizing, LDM
and index-reset policy, external sequence capacity, and invalid unadjusted LDM
inputs. Keep the migration boundary documented so the hybrid state remains
explicit while deeper CCtx reset and matchfinder operations stay in C.
Test Plan:
- `cargo fmt --manifest-path rust/Cargo.toml -- --check`
- capped focused planner tests: 3 passed
- capped full Rust library tests: 680 passed, 0 failed
- capped `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`: passed
- capped native `make -B -C programs -j1 zstd`: passed; only known fileio const-cast warnings
- capped `make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s`: 84 tests and both short fuzzer rounds passed; only known zstream initializer warning
The CCtx workspace estimator still assembled its private sizing formula in C,
even though the match-state and component-sum leaves already lived in Rust.
That left buffering, LDM, static-context, and external-sequence sizing policy
outside the rewrite boundary.
Move the complete size_t formula into a Rust entry point. C now supplies only
private object sizes, sanitizer redzone policy, and LDM scalar fields through a
small ABI record. Rust retains the C wrapping/alignment behavior and focused
branch coverage while the C adapter preserves the existing public estimator
surface.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --release estimate_cctx_workspace_size
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --release (667 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --release --all-targets -- -D warnings
- ulimit -v 41943040; make -B -C programs -j1 zstd
- ulimit -v 41943040; make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s (84 tests and both short fuzzer rounds)
ZSTD_createCDict_advanced2 previously selected compression parameters,
resolved dedicated-dictionary-search fallback, and ordered allocation and
initialization entirely in C. That left a large public dictionary boundary
outside the Rust rewrite and made the failure ordering implicit in the C
wrapper.
The Rust params API now owns advanced-CDict parameter preparation, including
the dedicated-search override/fallback and row-matchfinder resolution. A Rust
dictionary bridge owns the create -> init sequence and frees a created CDict
when initialization fails. The C shim retains only the private workspace and
custom-memory allocation, dictionary initialization, and teardown callbacks;
opaque parameter-field pointers and compile-time layout assertions preserve
the existing ABI. Focused probes cover policy publication and all callback
failure-ordering paths, and the migration README records the remaining C
boundary.
Test Plan:
- `rustfmt --check` and `cargo fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- `cargo test --manifest-path rust/Cargo.toml --release` -- 663 passed
- `cargo clippy --manifest-path rust/Cargo.toml --release --all-targets -- -D warnings` -- passed
- `make -B -C programs -j1 zstd` -- passed
- `make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s` -- 84 tests and both fuzz rounds passed
Move public ZSTD_copyCCtx pledge normalization and frame-parameter policy into
Rust while retaining the private workspace and match-table copy implementation
behind a C callback.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C programs -j1 zstd
- make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s
- focused copy_cctx unit tests
Move the shared CDict attach-versus-copy policy and callback dispatch into a
Rust-owned ABI leaf. Keep both workspace- and match-state-heavy reset
implementations in C behind opaque callbacks.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C programs -j1 zstd
- make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s
- focused reset_using_cdict unit tests
Move shared CDict initialization ordering and scalar publication into a
Rust-owned ABI leaf used by both dynamic and static CDict creation. Keep
content allocation/copy, entropy workspace reservation, match-state reset,
and dictionary insertion behind narrow C callbacks so private CDict and
workspace layouts remain C-owned.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C programs -j1 zstd
- make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s
- focused cdict_init unit test
ddidderr