Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
The new direct Rust file-I/O callback is unsafe because it receives a raw
NUL-terminated filename pointer. Add the required safety contract so CLI
clippy with `-D warnings` accepts the migrated ABI leaf.
Test Plan:
- `git diff --cached --check` -- passed.
- Capped CLI clippy rerun after this commit.
The new context-parameter projection test is compiled only in test builds,
but its constant was imported at module scope. Keep the import in the test
module so normal clippy builds remain warning-free under `-D warnings`.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
Import the no-attach-dictionary mode constant used by the new direct
ZSTD_getCParamsFromCCtxParams() projection test. The implementation was
correct, but the test module did not inherit that constant through the
existing parameter imports, so the root crate could not compile.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
The --list stdin callback was a stateless string predicate: it ignored its
opaque context and compared the input name with the fixed stdin marker. Move
that callback under its existing caller symbol into the Rust CLI archive, so
C no longer owns this policy leaf. The Rust implementation preserves the
callback ABI and returns the same 1/0 result for the marker and ordinary
names. File opening, frame parsing, diagnostics, human-readable formatting,
private file-info storage, and list orchestration remain C-owned. The optional
codec-version helpers remain untouched because their build-time library
configuration is not propagated to the Rust archive.
Test Plan:
- `clang -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder programs/fileio.c` -- passed.
- `rustfmt --edition 2021 --check --config skip_children=true rust/cli/src/lib.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, clippy, native builds, and upstream tests were not run per the explicit no-heavy-command constraint.
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.
Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
the requested lightweight-only scope.
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.
Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
The parameter policy leaves now export their C API names directly from Rust.
Two compatibility aliases were only used by the Rust unit test, while the
library and clippy builds correctly treated them as unused. Remove those dead
aliases and call the direct Rust-owned clamp symbol from the test so the
library, bench, and test targets share the same ABI surface without warning
under `-D warnings`.
Test Plan:
- Capped `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.
Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the Rust pass-through and frame-analysis leaves directly from their C
callbacks. The C translation unit still owns private resource projections,
filename suffix configuration, diagnostics, and format-specific operations;
this commit only removes two exact return-forwarding layers.
Test Plan:
- clang -fsyntax-only on programs/fileio.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.
Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.
Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
Keep the local buffer variables declared before the direct Rust buffer-policy
calls in the compression stream adapter, preserving the native C90 warning
profile after removing the old C forwarding helper.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed before this declaration-only cleanup; the next capped native gate will recheck it.
Rely on the existing fileio.h declaration now that the filename collision
checker is exported directly from Rust, avoiding a redundant C redeclaration
in the implementation file.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed; CLI lint and unit gates pending.
Move the pure filename-collision, input/output-buffer construction, and
compressed-destination-name helpers to direct Rust-owned caller symbols.
Leave C responsible for diagnostics, filesystem resources, suffix-list policy,
and the surrounding file-processing orchestration.
Test Plan: Pending capped full verification after this atomic ABI cleanup.
Remove the C forwarding wrappers around compression-parameter policy leaves
that are now implemented directly under their caller-facing ABI symbols.
Keep the private C parameter construction and resource logic in C, while Rust
continues to own the pure row-matchfinder, block-splitter, LDM, external
sequence/repcode, dictionary-tagging, block-size, and overflow predicates.
Test Plan: Pending capped full verification after the companion CLI leaf commit.
Export the file-summary predicates and largest-file-size scan directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving pointer contracts, assertions, return widths, and
filesystem behavior.
Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
Export ZSTD_useTargetCBlockSize and ZSTD_blockSplitterEnabled directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving diagnostics, ABI signatures, and parameter layout.
Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
Move reset, parameter copy, and default policy resolution from
ZSTD_makeCCtxParamsFromCParams into Rust through a pointer-based ABI function.
Keep the private LDM adjustment and its invariants in C while Rust owns the
stable parameter-object policy and focused reset/resolution coverage.
Test Plan:
- worker git diff and capped format checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
All parent verification is serial under a 40 GiB virtual-memory cap. Standalone
root Rust unit linking remains unavailable because the crate imports C-owned
bridge symbols.
Export FIO_LZ4_GetBlockSize_FromBlockId directly from Rust under the existing
caller symbol and remove the redundant C forwarding wrapper. Preserve the
block-ID formula and cover its boundary values in the focused Rust test.
Test Plan:
- worker capped nightly rustfmt check
- worker git diff --check
- full serial capped native and upstream suites to run at the next parent gate
Export FIO_highbit64 directly from Rust under the existing caller symbol and
remove the C forwarding wrapper. Preserve the nonzero-input contract and add
boundary tests for the scalar policy leaf.
Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests
All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
Move reset, copy, and default policy resolution from
ZSTD_CCtxParams_init_internal into Rust through the existing repr(C)
parameter mirror. C retains validation and diagnostics, while focused tests
cover reset behavior, policy resolution, and compression-level handling.
Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests
All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
Move the malloc-versus-mmap decision used by compression and decompression
resource construction into Rust. The C adapter continues to own platform
loaders, allocation handles, diagnostics, and dictionary I/O; Rust only
combines the explicit mmap preference, patch-mode size threshold, and explicit
disable override. Add enum-value ABI checks and focused policy coverage.
Also hoist the source-size declaration in the C adapter so the migration does
not introduce a C90 declaration-after-statement warning.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Move the strategy classification that decides whether LDM preparation fills
fast or double-fast tables into Rust. C retains the private match-state,
configuration-specific table-fill leaves, and the existing excluded-DFAST
assertion. Add enum-value ABI checks plus focused tests for valid and invalid
strategy classifications.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Move the fixed source-size tier traversal used by
ZSTD_estimateCCtxSize_internal() into Rust. C retains the authoritative
parameter lookup and per-tier workspace estimator behind callbacks, while Rust
owns the tier ordering and maximum-selection policy. The projection includes
ABI layout assertions and tests for callback order, level propagation, and
error-valued maxima.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Keep XXH64 state ownership and checksum calculation in the C MT scheduler,
but project the final four-byte frame write through Rust. Rust now owns the
little-endian serialization and destination-capacity policy, with ABI layout
assertions and flush-state error propagation preserving the existing cleanup
path. Focused tests cover byte order, exact capacity, invalid destinations,
and callback error handling.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran under a serial 40 GiB virtual-memory limit. The
full upstream suite passed, including large-file, multithreaded, API, fuzzer,
and zstream coverage. Standalone root Rust unit linking remains unavailable
because the crate imports C-owned bridge symbols without a Cargo build/link
setup.
Call the authoritative DDict constructor directly from the Rust decompression
policy after converting the shared repr(C) allocator projection. Remove the
C forwarding declaration and wrapper so dictionary creation no longer crosses
an unnecessary Rust-to-C-to-C boundary, while compile-time size and alignment
checks keep the allocator conversion tied to the ABI contract.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip, including the
parallel MT checksum seam that remains as the next commit. Standalone root
Rust unit linking remains unavailable because the crate imports C-owned bridge
symbols without a Cargo build/link setup.
Keep the heapmode guard, stack-owned ZSTD_DCtx, static initialization, and
private context layout in C. Once the context is initialized, Rust now owns the
one-shot stack-context policy: it clears the projected static-size marker and
dispatches the existing ZSTD_decompressDCtx implementation through the opaque
context pointer.
Add matching C/Rust view-layout assertions and focused callback tests covering
static-size clearing, argument forwarding, and null-context rejection.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; the integrated build and upstream C/Rust oracle supplied the runtime/link coverage.
Project the zstd window-too-large diagnostic inputs into Rust so Rust owns
error filtering and the concrete-versus-unsupported guidance choice. C keeps
frame-header parsing, window-log extraction, and the exact display text behind
a callback, preserving the existing diagnostics and fallback behavior.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1 (clean after C90 declaration cleanup)
- ulimit -v 41943040; make -j1 -C tests test (passed; the later cleanup only moved a declaration before statements)
Project the scalar single-thread hint inputs and keep the multithreaded
context behind a C callback so Rust owns the worker-count dispatch. Preserve
the existing buffer-mode assertion and both hint arithmetic leaves, with
focused tests covering MT selection and single-thread fallback.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Project the decompression-resource cleanup callbacks into Rust so the stable
teardown order remains dictionary, dstream, write pool, and read pool. C
retains ownership of dRess_t and each private resource implementation, with
its existing dstream CHECK behavior preserved. Focused tests cover order and
null or incomplete callback states.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Project the private stream state needed by the complete-input streaming fast
path into a stable C/Rust layout and let Rust own the eligibility predicate.
The C bridge keeps the CCtx private and retains the existing simple-level
selection leaf as a callback, while focused tests cover the accepted and
rejected stream states.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Move the scalar policy that selects the optimal parser for long-distance
matching into Rust. The C implementation continues to own match-state and
compressor dispatch, but the strategy threshold is now defined and tested at
the Rust seam.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
File-resource creation already delegates the general compression-parameter
policy to Rust, but the multithreaded parameters were still applied in C in a
separate branch. That left parameter ordering, optional overlap handling, and
error short-circuiting outside the Rust policy boundary. Add a narrow ABI
projection with C callbacks for CCtx mutation and diagnostics, and let Rust
apply worker count, job size, optional overlap, and rsyncable in the original
order. The C90 declaration layout and compile-time ABI assertions keep the
existing native program configurations intact.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- 179 passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed without the new C90 warning
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
The compression context still owns private dictionary storage and CDict
objects, but the teardown order is observable because each step can invoke an
allocator or clear state used by later steps. Keep those storage operations as
C callbacks while moving the sequencing policy into the Rust compression
module. The C bridge now contains only callback adapters and compile-time ABI
layout checks, and the Rust unit test pins the original five-step order.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
Move the pure multithreaded worker-parameter policy into Rust: only the first
job keeps the frame checksum flag, worker jobs disable LDM, and every worker
context runs with zero nested workers. Keep the complete C-owned parameter
structure and codec initialization in the adapter, crossing the ABI with a
small layout-checked scalar projection.
Test Plan:
- rustfmt --edition 2021 --check rust/src/zstdmt_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
Move the stable input and output expectation update into the Rust compression
module. Keep the private CCtx layout and debug logging in the C adapter while
passing only the buffer-mode scalars and projected buffer fields across the
ABI. Preserve C size_t wrapping for the output remainder and leave buffered
modes unchanged.
Test Plan:
- rustfmt --edition 2021 --check rust/src/zstd_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
Optimized dictionary scoring used the requested dictionary ID while building
its scoring CDict, so --dictID could change candidate quality and output
content. Parallel FastCover could also choose different equal-score candidates
based on completion order. Clear the ID only while constructing the scoring
CDict, restore it immediately afterward, and use deterministic (d, k)
tie-breakers in Cover and FastCover. The requested ID now remains output
metadata instead of influencing dictionary content or repeat-run stability.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml scoring_dictionary_id_is_restored_after_the_score --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
Move the non-threaded compression parameter sequence and adaptive window policy
out of FIO_createCResources. C continues to own context and resource
allocation, dictionary and pool setup, and the multithread-specific settings;
Rust drives the ordered parameter callbacks and stops on the first error.
The projection mirrors the C ABI and has focused order and short-circuit tests.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml create_c_resources --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
Move ZSTDMT_freeCCtx ownership branches and teardown ordering into Rust while
keeping the MT context, worker pool, synchronization, dictionary, and custom
allocator layouts private to C callbacks. Preserve the original factory and
round-buffer conditions, release jobs before destroying the pools, and keep
free-on-null compatible with the C API.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml free_cctx --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
Move the worker-count branch of ZSTD_toFlushNow into the Rust policy layer.
The C adapter retains the private CCtx and multithread context, exposing only
the worker count and a callback for the MT flush query. Single-threaded
contexts keep the existing zero result and the ABI projection is checked on
both sides.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml to_flush_now --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
Keep fileio's compression-resource teardown order in the Rust policy layer:
dictionary, write pool, read pool, then compression context. The C adapter
keeps cRess_t and each private resource layout local while exposing only
one callback projection to Rust, preserving the existing cleanup behavior.
The Rust entry point also treats a null state as a no-op and verifies the
projection layout at compile time.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml free_c_resources --lib
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
The Rust CLI clippy gate treats the hand-written alignment rounding in the
multi-file list projection layout test as a manual div-ceil implementation.
Use the standard integer operation so the test remains clear, equivalent, and
warning-free under the repository's deny-warnings policy.
Test Plan:
- `cargo fmt --manifest-path rust/cli/Cargo.toml --all` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
Move advanced and static CDict construction ordering into Rust while keeping
private workspace layout, object initialization, dictionary loading, and
allocator leaves in C callbacks. The Rust policy now validates parameters and
memory, computes the workspace contract, sequences allocation/reservation and
initialization, and preserves failure cleanup for both dynamic and static
workspaces. Add callback-order, allocation-failure, reservation-failure, and
initializer-failure tests around the ABI projections.
Test Plan:
- `cargo fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo build --manifest-path rust/Cargo.toml --release`
Move stdin and empty-input validation, header selection, per-file iteration,
status aggregation, and multi-file total selection into the Rust CLI policy
layer. The callback projection keeps C responsible for its private file-info
storage, file parsing, diagnostics, human-readable formatting, and per-file
listing. Only successful and frame-error records contribute to the aggregate,
matching the original early-return behavior for invalid or truncated inputs.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/fileio_prefs.rs` -- passed.
- `git diff --cached --check` -- passed.
- Cargo, native, and full test commands were not run per the explicit no-heavy-command constraint.
Keep the compression destination ABI assertion byte-accurate while expressing
the final callback offset in the form required by current Clippy. The total
projection size remains nine function-pointer slots.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --check
Move the residual ZSTD_resetCCtx_internal sequencing into a Rust policy
entrypoint. The Rust side now owns the plan-to-workspace-to-tail order and
scalar hand-off, while C retains private workspace checks, context
initialization, match-state reset, and pointer publication callbacks. Preserve
error short-circuiting and the existing static, dynamic, LDM, and index-reset
behavior.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- Full capped Rust, native, CLI, and upstream test suites to follow
Move the single-file compression destination lifecycle into the Rust file-I/O
policy layer while keeping C responsible for private resources, file handles,
diagnostics, metadata operations, and codec dispatch callbacks. Preserve the
original shared-destination fast path, temporary permissions, handler timing,
metadata-before-close ordering, close-error propagation, stdout cleanup guard,
and failed-artifact removal.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- Full capped Rust, native, CLI, and upstream test suites to follow
Use the pointer-width bit count for the decompression callback projection
assertion so the Rust archive remains clean under warnings-as-errors Clippy.
The assertion keeps the same byte offset on supported targets.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
`ZSTD_selectBlockCompressor` already delegated scalar strategy/index
calculation to the Rust parameter module, but C still owned table selection
and dictionary-mode offsets. Keep the configuration-dependent C function-
pointer tables and NULL entries for excluded compressors as private leaves,
and pass those tables through a narrow ABI state so Rust owns row versus
ordinary selection, dictionary-mode indexing, and selected callback
publication. The legacy C selector signature remains intact for the LDM
caller.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo/make/test/fuzzer verification not run per the enforced 40 GiB
serial verification limit.
Keep job table allocation, private synchronization callbacks, and
initialization-failure cleanup in one Rust adapter. The C wrapper now only
supplies pthread/object callbacks, preserving rounded capacity and the
destroy-before-custom-free ordering; table expansion reuses the same path.
Test Plan:
- rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs
- git diff --check and git diff --cached --check
- Static inspection only; cargo, make, tests, fuzzers, and native/heavy
commands were not run under the global 40 GiB cap.
Define the decompression trace projection with the exact unsigned-long-long
representation used by the public trace API. The generic U64 typedef can be
a different C type with the same width, which made the Rust trace view fail to
compile when assigned the DCtx trace field.
Test Plan:
- ulimit -v 41943040; make -j1
Use the type's bit width for the pointer-sized tail in the source-file
projection layout assertion. This is numerically identical to eight
pointer-sized bytes on supported targets while keeping the assertion
compatible with the repository's warnings-as-errors lint policy.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
Move ZSTD_loadDictionaryContent orchestration into a Rust-owned policy leaf while keeping C-private match-state, LDM, workspace, window, overflow, and matchfinder operations behind narrow callbacks. Preserve both dictionary suffix limits, LDM-before-main-table ordering, dictionary validity publication, strategy-specific loading, and final nextToUpdate publication without changing ZSTD_compress_insertDictionary dispatch semantics.
Add focused policy tests for the two suffix limits and matchfinder path selection. Preserve the existing global overflow helper as a C callback boundary.
Test Plan:
- rustfmt --check rust/src/zstd_compress_dictionary.rs
- git diff --check and git diff --cached --check
- Static diff/rg inspection of the Rust/C ABI, callback order, suffix truncation, and private-state ownership
- No cargo, make, native, fuzz, or heavy tests run by request
Keep weak-symbol lookup and the conditional traceCtx/DCtx layout in the C
adapter, but project only the trace slot, callback pointers, context pointer,
and dictionary metadata into Rust. Rust now decides begin/end eligibility and
builds the ABI-compatible trace record, preserving null callbacks, streaming
and size conversion semantics. Existing call sites remain in their
success-only branches, so checksum/error paths still emit no trace end.
Test Plan:
- rustfmt --edition 2021 --check rust/src/zstd_decompress.rs
- git diff --check and git diff --cached --check
- Static rg/diff inspection of weak callbacks, traceCtx lifecycle, dictionary
metadata, and success-only trace-end call sites
- No cargo, make, native, fuzz, or heavy tests run by request
Move the ordered policy from FIO_compressFilename_srcFile into a narrow Rust
projection: named-source stat and collision checks, compressed-file exclusion,
source open and close, size-based async selection, source attachment, and
successful-only --rm handling now run in Rust. The destination callback still
hands compression to the existing Rust compression loop.
Keep FIO_ctx_t, cRess_t, stat_t, FILE/pool handles, signal handling, and
CLI diagnostics behind opaque C callbacks, so the ABI carries only filenames,
scalar policy flags, and callback pointers. Focused Rust policy tests cover
named-file short circuits, stdin behavior, async threshold selection, and the
successful-compression removal gate.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/fileio_asyncio.rs` — passed
- `git diff --check` and path-limited `git diff --cached --check` — passed
- Static ABI/order/diff inspection only; policy tests were not executed
- cargo/make/native/fuzzer/heavy tests were not run per request
The Rust-owned frame-progression ring scan consumes ZSTDMT_jobProjection
through ZSTDMT_projectJob. That callback previously copied counters under
job_mutex but read source and prefix metadata after unlocking. Keep the
complete projection in one mutex-protected snapshot, matching the original
ZSTDMT_getFrameProgression access pattern without moving descriptor or
synchronization ownership out of C.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Static inspection only; no cargo, make, native binaries, tests, fuzzers,
or heavyweight commands were run.
Project the external sequence producer callback directly into the Rust
block-delimited sequence transfer leaf. Remove the redundant C forwarding
wrapper while preserving the C-facing sequence conversion API.
Test Plan:
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests fuzzer
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Remove the unreachable C target-sized and split-block adapters now that frame chunks use the direct Rust state projections. The C sequence-store wrapper was likewise unused after those adapters disappeared.
Keep the target and split callback slots as nullable Rust test seams and return a generic error if a caller selects a callback fallback without providing one. Production frame-chunk state now leaves both slots null because the direct projections are authoritative.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Replace the frame-chunk split-discovery callback with an explicit projection of the sequence stores, block-state pointer slots, entropy metadata, parameters, and workspace required by the Rust estimator. Rust now invokes the existing split estimator directly before the Rust-owned partition loop, while C retains only private block-split storage and projection assembly.
Keep the block-state pointer slots live through estimation instead of capturing entropy-table addresses during frame preparation. Sequence-store work can change which compressed-block state is active, and following the slots preserves the original C behavior for dictionary-backed streaming compression.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Project the target-sized block state into frame-chunk compression and invoke
Rust sequence-store construction followed by the existing target-block
post-build leaf. Refresh the per-block isFirstBlock snapshot to preserve the
former C adapter's behavior while leaving the split-block branch on its
existing callback path for a later seam.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib 'zstd_compress::tests::frame_chunk' -- --nocapture
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Route frame-chunk internal blocks through the projected build-sequence-store
and block-internal state, using a Rust-owned frame-chunk leaf while retaining
the target and split callbacks in C. Refresh the per-block isFirstBlock
snapshot for the direct path so its behavior remains identical to the former
C adapter, and remove the obsolete C frame-chunk and block-internal adapters.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Share the existing end-state projection between the public C wrapper and
Rust-owned compressed-dictionary orchestration. The CDict callback now reaches
ZSTD_rust_compressEnd through the private dispatcher without re-entering the
public C end wrapper, while the exported API keeps its compatibility wrapper.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Pass the existing overflow-correction projection directly through the
compressContinue ABI so Rust owns the correction decision and ordering while
C retains only the table/workspace callbacks. Remove the redundant C
forwarding adapter and keep the stack projection valid for synchronous
dispatch.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress::tests::compress_continue -- --nocapture
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Project the match-state and optional LDM window fields directly through the
compressContinue ABI so Rust can reuse the existing window-update leaf. Keep
forceNonContiguous and nextToUpdate updates visible to the following C
callbacks, preserve the match-before-LDM ordering, and remove the C window
update callback while retaining context-sensitive overflow and block calls.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress::tests::compress_continue -- --nocapture
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Remove the sequence-compression API's C frame-header callback and carry the
validated frame parameters as scalar state. Rust now serializes headers for
both sequence API variants through the same frame-header leaf, while C keeps
only CCtx initialization and checksum callbacks behind the boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress::tests::sequence_api -- --nocapture
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Remove the C frame-header callback from the Rust-owned compressContinue
orchestration. Project the applied frame parameters and dictionary ID as
scalars, so Rust can call the existing header serializer directly while
preserving pledged-size subtraction, stage transitions, and output accounting.
Route the sequence API's remaining header call directly to the same Rust leaf
and delete the redundant C wrapper.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress::tests::compress_continue -- --nocapture
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Project the end-frame stage, format flags, window log, and XXH64 state into
Rust so the Rust end-of-frame orchestrator computes the checksum and invokes
the existing Rust epilogue serializer directly. Remove the C epilogue and
checksum adapter while preserving callback order, output accounting, stage
transitions, and error precedence.
Test Plan:
- CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress::tests::compress_end -- --nocapture
- CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- CARGO_BUILD_JOBS=1 cargo test
- CARGO_BUILD_JOBS=1 make -j1
- CARGO_BUILD_JOBS=1 make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- CARGO_BUILD_JOBS=1 make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
All commands were run serially with a 40 GiB virtual-memory cap.
Project the prepared CDict window extent and destination match-state fields
through an explicit ABI state so Rust owns the post-reset attachment
operation. Preserve the original empty-dictionary and already-advanced
window branches, including window-limit clearing and loaded-dictionary
referential updates, while keeping private C pointer arithmetic and layout
ownership in the C adapter.
Test Plan:
- CARGO_BUILD_JOBS=1 cargo test --lib zstd_compress_dictionary::tests::reset_cctx_by_attaching_cdict -- --nocapture
- CARGO_BUILD_JOBS=1 cargo clippy --all-targets -- -D warnings
- CARGO_BUILD_JOBS=1 cargo test
- CARGO_BUILD_JOBS=1 make -j1
- CARGO_BUILD_JOBS=1 make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- CARGO_BUILD_JOBS=1 make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
All commands were run serially with a 40 GiB virtual-memory cap.
Move the final CDict-copy hashTable3 clearing operation into the Rust reset
orchestration. The C bridge now projects the post-reset destination table
and hash log while retaining only the private reset, workspace dirty/clean,
and metadata operations on the C side.
Preserve C's zero-log contract: hashLog3 == 0 means no table bytes are
cleared and a null hashTable3 is valid. Nonzero logs are bounds-checked before
Rust zeroes the table, keeping the operation between the dirty and clean
transitions.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Move the CDict-to-CCtx hash, chain, and row-match table copies into the
Rust reset orchestration. The C bridge now exposes table-field addresses
and scalar policy inputs while retaining private reset and hashTable3
operations; Rust loads destination pointers only after reset and strips
short-cache tags through the existing Rust leaf.
The destination chain policy must be read after the reset callback updates
appliedParams. Reading it before reset skipped DFAST chain copies and caused
the dictionary source-size zstream test to segfault in the double-fast
external-dictionary matcher. Focused tests cover tagged hash/chain copies,
row-match tag/salt copies, callback ordering, and reset-error short-circuiting.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040; make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Replace the remaining CCtx-copy custom-memory callback with explicit source and
destination pointers. Rust now copies the ABI-checked ZSTD_customMem payload in
its original position before reset, while C retains private context access and
the surrounding workspace and table operations.
Test Plan:
- ulimit -v 41943040 cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040 make -j1
- ulimit -v 41943040 make -j1 -C tests test-zstream ZSTREAM_TESTTIME=-T2s
- ulimit -v 41943040 make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
ddidderr