ZSTD_initStaticCDict previously resolved automatic row matching and computed the
minimum workspace in C before handing the result to the Rust construction
orchestrator. That left the public static-CDict sizing branch split from the
Rust validation and callback failure policy.
Make the Rust projection carry dictionary size, load method, compression
parameters, and the C-owned layout sizing inputs. Rust now resolves automatic
row matching, computes the dedicated-search workspace requirement, validates
workspace capacity, and passes the resolved mode into the initializer while
preserving create/reserve/move/init order. C retains opaque private cwksp and
CDict operations, allocator/layout behavior, and dictionary-content loading.
Add focused tests for computed sizing policy, capacity boundaries, callback
order, reservation and initialization failures, and null or unaligned
workspaces.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- GCC `-fsyntax-only` checks with multithreaded, non-multithreaded,
`ZSTD_DISABLE_ASM=1`, and `ZSTD_ADDRESS_SANITIZER=1` configurations -- passed
- Clang `-fsyntax-only` check with `ZSTD_MULTITHREAD` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, and large tests were not run per task constraints
The MT flush state machine already recognized worker and checksum errors in
Rust, but its single C error callback still bundled waiting for all workers
with releasing job resources. That kept teardown order in C and made the two
cleanup phases impossible to test independently.
Split the C callback into private wait and release leaves, then pass both to
Rust. Rust now owns the error cleanup sequence and always waits for workers
before releasing their resources on either error path. The MT context, job
resources, synchronization, and cleanup operations remain C-owned.
Test Plan:
- `rustup run nightly rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs`
-- passed under the 40 GiB virtual-memory cap.
- `cc -fsyntax-only -std=c99` with the zstd include paths for
`lib/compress/zstdmt_compress.c` -- passed under the cap.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native builds, fuzzers, and large tests were not run per the
worker OOM and scope constraints.
Move static CCtx construction sequencing out of the monolithic C leaf. Rust
now validates the existing public workspace contract, sequences workspace
creation, CCtx and auxiliary reservations, publication, and BMI2 setup, and
short-circuits callback-reported failures. C callbacks retain the private
cwksp and CCtx layouts plus the native workspace-size and CPU-feature rules.
Add matching ABI layout assertions and focused callback-order, reservation-
failure, workspace-capacity, and NULL-path unit coverage. Static-CDict and
heap-CCtx paths remain unchanged.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- GCC and Clang `-fsyntax-only` checks on `zstd_compress.c` -- passed
- GCC syntax checks with `ZSTD_DISABLE_ASM=1` and
`ZSTD_ADDRESS_SANITIZER=1` -- passed
- `git diff --check` -- passed
- Cargo, make, native suites, and fuzzers not run per request
The MT worker previously reported errors through one C callback and left a
combined C finish callback responsible for serial completion, private resource
release, output-size publication, consumed-size publication, and signaling.
That kept the branch and lifecycle policy on the C side of the existing Rust
stage scheduler.
Add a narrow finish projection whose C callbacks expose only those private
leaves and the source-size scalar. Rust now publishes an error before serial
completion on every failed resource or codec stage, releases the sequence and
CCtx resources in the original order, publishes the final block size only on
success, then publishes the consumed size and signals the job condition. The
failed path therefore normalizes any codec-reported last-block size to zero
without changing the C-owned mutex, descriptor, pool, or context layouts.
Focused tests cover successful final-block publication, resource failure,
codec failure, cleanup ordering, consumed-size publication, and error-path
normalization.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check -- lib/compress/zstdmt_compress.c rust/src/zstdmt_compress.rs` and `git diff --cached --check` -- passed
- Rust unit tests were added but not run because the request prohibited Cargo and heavy commands
The MT job callback added state publication before its Rust job-construction
call, which left the result declaration after executable statements and emitted
the repository's C90 mixed-declaration warning on every CLI/test build.
Declare the result alongside the other callback-local projections and assign it
at the existing call site. This is a warning-only cleanup with no runtime or
ABI change.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The prior capped suite reached `rustLibSmoke`; a fresh root library build is
required because its archive predates the display-hook fix
The first MT state-publication fix made the C count current before job
construction, but the Rust result still unconditionally copied its local
pre-job fill count back into `mtctx` after the callback returned. Preparing a
job clears `mtctx->inBuff.buffer` and `mtctx->inBuff.filled`, so the next stream
pass then observed a consumed range as if it were still active and returned a
generic error in the multi-file CLI test.
Keep the C-cleared state whenever job preparation has detached the active
range. Rust's filled count is published only while the C range remains active,
which covers the blocked/table-full path where no job reset occurred. The
callback still receives the current count before preparation, and the C90
projection declaration is kept before executable statements.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The capped upstream suite reproduced a generic error in the multi-file CLI
path before this correction; the full rerun is pending
The Rust MT stream scheduler now copies input into the reusable round buffer
through its local projection and publishes the final filled count when the
outer call returns. Job creation, however, is a C callback invoked during that
same scheduler call. It still inspected `mtctx->inBuff.filled` while preparing
the job, so the count lagged behind the bytes Rust had just copied and the
existing preparation assertion fired in the multiple-file CLI path.
Publish the callback's `srcSize` into the C context before constructing the job
projection. This restores the ordering of the former C copy callback: the
context count is current before job preparation, while the existing end-of-call
result publication remains available for paths that do not create a job.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reproduced the pre-fix assertion in
`ZSTDMT_prepareCompressionJob`; the capped full-suite rerun is pending
The advanced-CDict path previously kept a C callback that rebuilt the
ZSTD_rustCDictSizing projection and delegated the actual arithmetic back to
Rust. Replace that callback with a pointer to the explicit sizing projection,
so Rust owns the workspace-size policy while C continues to own private
zstd_cwksp allocation, object reservation, initialization, and teardown. The
projection replaces the old callback slot, preserving every later callback
offset and the synchronous lifetime of the C sizing values.
Update both sides' layout assertions, remove the redundant C helper, and make
the lifecycle tests assert that Rust's computed size reaches allocation and
workspace creation before the existing C-owned callbacks run. Custom-memory
validation and all other CDict callbacks remain unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo build/test/clippy, Make, fuzzers, and other heavy verification were intentionally not run per request.
The decoder context remains allocated and laid out by C because optional members
change with the active build configuration. Previously, the C adapter also
owned the platform leaf: its DYNAMIC_BMI2 branch queried CPU features and
assigned the private bmi2 member directly. That kept the policy in C and made
the Rust boundary depend on the context layout.
Move the leaf to Rust and give it an opaque scalar address plus the active
configuration value. The C adapter now only extracts the optional address when
that member exists and forwards it, while DYNAMIC_BMI2=0 passes NULL. Rust
performs the dynamic decision, queries CPU support, and writes the scalar. The
existing 68-word DCtx view is unchanged. Focused unit coverage verifies that
disabled dynamic dispatch leaves the scalar untouched and enabled dispatch
publishes the CPU feature result.
Test Plan:
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=0 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- `git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs` -- passed
The MT streaming scheduler previously delegated each bounded input-buffer copy
through a C-only callback. That callback mutated the C context as a side
effect, which kept a trivial data movement operation outside the Rust stream
path and made the result projection incomplete.
Copy the checked input range directly in Rust with copy_nonoverlapping, after
the existing available-input, destination-capacity, and non-null-source checks.
The C input-range callback remains responsible for exposing the reusable buffer,
while the Rust result now returns its updated filled count. C publishes that
count back to mtctx after the scheduler call. The result ABI has explicit
cross-language layout assertions, and the scheduler test verifies both copied
bytes and fill accounting. Job scheduling and all other MT callbacks are
unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo build/test, make, fuzzers, and other heavy checks were not run per task instructions
Make ZSTDMT_createCompressionJob match the exact callback type consumed by
ZSTDMT_rust_compressStreamGeneric. The callback now performs the existing
opaque and end-directive casts itself, so the redundant C forwarding shim can
be removed without changing scheduling, assertions, logging, or ABI behavior.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- No cargo, make, native, fuzz, or other heavy tests run per task scope.
Move the MT LDM overlap predicate behind the original
ZSTDMT_doesOverlapWindow() by-value ABI. Rust now owns the half-open range
calculation through a checked representation of Buffer and ZSTD_window_t;
C retains the synchronization and context orchestration around the callback.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.
Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
the requested lightweight-only scope.
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.
Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.
Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.
Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.
Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
Remove the C forwarding wrappers around compression-parameter policy leaves
that are now implemented directly under their caller-facing ABI symbols.
Keep the private C parameter construction and resource logic in C, while Rust
continues to own the pure row-matchfinder, block-splitter, LDM, external
sequence/repcode, dictionary-tagging, block-size, and overflow predicates.
Test Plan: Pending capped full verification after the companion CLI leaf commit.
Export ZSTD_useTargetCBlockSize and ZSTD_blockSplitterEnabled directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving diagnostics, ABI signatures, and parameter layout.
Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
Move reset, parameter copy, and default policy resolution from
ZSTD_makeCCtxParamsFromCParams into Rust through a pointer-based ABI function.
Keep the private LDM adjustment and its invariants in C while Rust owns the
stable parameter-object policy and focused reset/resolution coverage.
Test Plan:
- worker git diff and capped format checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
All parent verification is serial under a 40 GiB virtual-memory cap. Standalone
root Rust unit linking remains unavailable because the crate imports C-owned
bridge symbols.
Move reset, copy, and default policy resolution from
ZSTD_CCtxParams_init_internal into Rust through the existing repr(C)
parameter mirror. C retains validation and diagnostics, while focused tests
cover reset behavior, policy resolution, and compression-level handling.
Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests
All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
Move the strategy classification that decides whether LDM preparation fills
fast or double-fast tables into Rust. C retains the private match-state,
configuration-specific table-fill leaves, and the existing excluded-DFAST
assertion. Add enum-value ABI checks plus focused tests for valid and invalid
strategy classifications.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Move the fixed source-size tier traversal used by
ZSTD_estimateCCtxSize_internal() into Rust. C retains the authoritative
parameter lookup and per-tier workspace estimator behind callbacks, while Rust
owns the tier ordering and maximum-selection policy. The projection includes
ABI layout assertions and tests for callback order, level propagation, and
error-valued maxima.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Keep XXH64 state ownership and checksum calculation in the C MT scheduler,
but project the final four-byte frame write through Rust. Rust now owns the
little-endian serialization and destination-capacity policy, with ABI layout
assertions and flush-state error propagation preserving the existing cleanup
path. Focused tests cover byte order, exact capacity, invalid destinations,
and callback error handling.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran under a serial 40 GiB virtual-memory limit. The
full upstream suite passed, including large-file, multithreaded, API, fuzzer,
and zstream coverage. Standalone root Rust unit linking remains unavailable
because the crate imports C-owned bridge symbols without a Cargo build/link
setup.
Call the authoritative DDict constructor directly from the Rust decompression
policy after converting the shared repr(C) allocator projection. Remove the
C forwarding declaration and wrapper so dictionary creation no longer crosses
an unnecessary Rust-to-C-to-C boundary, while compile-time size and alignment
checks keep the allocator conversion tied to the ABI contract.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip, including the
parallel MT checksum seam that remains as the next commit. Standalone root
Rust unit linking remains unavailable because the crate imports C-owned bridge
symbols without a Cargo build/link setup.
Keep the heapmode guard, stack-owned ZSTD_DCtx, static initialization, and
private context layout in C. Once the context is initialized, Rust now owns the
one-shot stack-context policy: it clears the projected static-size marker and
dispatches the existing ZSTD_decompressDCtx implementation through the opaque
context pointer.
Add matching C/Rust view-layout assertions and focused callback tests covering
static-size clearing, argument forwarding, and null-context rejection.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; the integrated build and upstream C/Rust oracle supplied the runtime/link coverage.
Project the scalar single-thread hint inputs and keep the multithreaded
context behind a C callback so Rust owns the worker-count dispatch. Preserve
the existing buffer-mode assertion and both hint arithmetic leaves, with
focused tests covering MT selection and single-thread fallback.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Project the private stream state needed by the complete-input streaming fast
path into a stable C/Rust layout and let Rust own the eligibility predicate.
The C bridge keeps the CCtx private and retains the existing simple-level
selection leaf as a callback, while focused tests cover the accepted and
rejected stream states.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Move the scalar policy that selects the optimal parser for long-distance
matching into Rust. The C implementation continues to own match-state and
compressor dispatch, but the strategy threshold is now defined and tested at
the Rust seam.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
The compression context still owns private dictionary storage and CDict
objects, but the teardown order is observable because each step can invoke an
allocator or clear state used by later steps. Keep those storage operations as
C callbacks while moving the sequencing policy into the Rust compression
module. The C bridge now contains only callback adapters and compile-time ABI
layout checks, and the Rust unit test pins the original five-step order.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
Move the pure multithreaded worker-parameter policy into Rust: only the first
job keeps the frame checksum flag, worker jobs disable LDM, and every worker
context runs with zero nested workers. Keep the complete C-owned parameter
structure and codec initialization in the adapter, crossing the ABI with a
small layout-checked scalar projection.
Test Plan:
- rustfmt --edition 2021 --check rust/src/zstdmt_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
Move the stable input and output expectation update into the Rust compression
module. Keep the private CCtx layout and debug logging in the C adapter while
passing only the buffer-mode scalars and projected buffer fields across the
ABI. Preserve C size_t wrapping for the output remainder and leave buffered
modes unchanged.
Test Plan:
- rustfmt --edition 2021 --check rust/src/zstd_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
Move ZSTDMT_freeCCtx ownership branches and teardown ordering into Rust while
keeping the MT context, worker pool, synchronization, dictionary, and custom
allocator layouts private to C callbacks. Preserve the original factory and
round-buffer conditions, release jobs before destroying the pools, and keep
free-on-null compatible with the C API.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml free_cctx --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
Move the worker-count branch of ZSTD_toFlushNow into the Rust policy layer.
The C adapter retains the private CCtx and multithread context, exposing only
the worker count and a callback for the MT flush query. Single-threaded
contexts keep the existing zero result and the ABI projection is checked on
both sides.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml to_flush_now --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
Move advanced and static CDict construction ordering into Rust while keeping
private workspace layout, object initialization, dictionary loading, and
allocator leaves in C callbacks. The Rust policy now validates parameters and
memory, computes the workspace contract, sequences allocation/reservation and
initialization, and preserves failure cleanup for both dynamic and static
workspaces. Add callback-order, allocation-failure, reservation-failure, and
initializer-failure tests around the ABI projections.
Test Plan:
- `cargo fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo build --manifest-path rust/Cargo.toml --release`
Move the residual ZSTD_resetCCtx_internal sequencing into a Rust policy
entrypoint. The Rust side now owns the plan-to-workspace-to-tail order and
scalar hand-off, while C retains private workspace checks, context
initialization, match-state reset, and pointer publication callbacks. Preserve
error short-circuiting and the existing static, dynamic, LDM, and index-reset
behavior.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- Full capped Rust, native, CLI, and upstream test suites to follow
ddidderr