The CCtx reset orchestration already runs in Rust, but its C workspace
adapter still decided when capacity was too small or wasteful and maintained
the oversized-duration counter through C-only predicates. Move those factor,
threshold, overflow, and duration-update rules into Rust. C now supplies only
private workspace metrics and retains low-level allocation, reservation, and
layout operations.
Remove the obsolete C predicates so the workspace resize decision has one
Rust-owned implementation and remains covered at its boundary conditions.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/Cargo.toml --tests
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests invalidDictionaries
- ulimit -v 41943040; make -j1 -C tests test
The local-dictionary callback still contained the policy that selected
by-reference versus by-copy loading, rejected internal copies for static
CCtx instances, allocated and copied dictionary bytes, and published the
resulting local-dictionary state. Move that policy into the Rust compression
rewrite so the C callback is only an adapter around the private CCtx and
localDict layouts.
The adapter retains the C custom allocator and output slots, preserving the
existing ownership and free path while keeping the ABI boundary explicit.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/Cargo.toml --tests
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests invalidDictionaries
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -B -j1 -C tests fuzzer
- (cd tests && ulimit -v 41943040; ./fuzzer -v -i1 -s6572)
- ulimit -v 41943040; make -j1 -C tests test
The overflow-correction policy and scalar correction arithmetic were already
Rust-owned, but the stateful window rebase still lived in the C
ZSTD_window_correctOverflow() inline. That left pointer advancement, limit
clamping, and the correction counter as an untracked production C algorithm
on every compression path that protects matchfinder indices.
Replace that inline with a five-pointer projection. C continues to compute the
private source index and choose the correction context, while Rust now owns
the rebasing transition: it advances the regular and dictionary bases, clamps
both limits with the original wrapping U32 behavior, checks the correction
invariants in debug builds, and increments the overflow-correction counter.
The C/Rust projection has explicit layout assertions, and the Rust fixture
covers the pointer, limit, and counter updates together.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/Cargo.toml --tests
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests invalidDictionaries
- ulimit -v 41943040; make -j1 -C tests test
- git diff --check
Move the scalar portion of the private CCtx reset tail behind the existing
Rust reset adapter. The C side now publishes explicit pointers to the fields
that Rust must initialize, while retaining workspace ownership, match-state
reset, storage reset, and the XXH64 reset callback that cannot cross the
boundary safely. This removes the last opaque C initializer from this reset
phase without exposing the private ZSTD_CCtx layout to Rust.
The ABI record has compile-time offset and size assertions on both sides, and
the Rust unit fixture now checks the resolved compression parameters,
prefetch policy, pledged-size sentinel handling, counters, content-size flag,
block-size limit, stage, and dictionary metadata. The callback-order tests
continue to verify that compressed-block state reset precedes match-state and
storage cleanup, and that failures stop the tail at the correct callback.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/Cargo.toml --tests
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
- git diff --check
Remove the C-only DDict offset globals and route production
ZSTD_copyDDictParameters() through the existing C-produced DCtx view. This
keeps the private decoder layout and optional fuzzing fields owned by the
canonical projection while preserving the DDict entropy, prefix, window, and
checksum state updates. Keep the synthetic offset writer test-only so the
Rust unit tests remain independently linkable.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo check --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (207 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test
- git diff --check
The MT stream initializer previously exposed one C callback that waited for
all submitted jobs and then released their resources. That left the ordering
policy in C even though Rust already owned the surrounding initialization
sequence and the ring-order wait/release policies.
Expose separate wait-all and release-all callbacks at the Rust/C boundary and
invoke them in Rust in the original wait-before-release order. C retains the
worker synchronization, job descriptors, input buffer, and resource-pool
side effects behind the two callbacks, while the focused Rust initializer test
now makes the ordering explicit.
Test Plan:
- capped cargo +nightly fmt for the Rust workspace -- passed
- capped root cargo clippy --all-targets -- -D warnings -- passed
- capped serial make -j1 for the single-threaded library, MT library, and CLI -- passed
- git diff --cached --check -- passed
Transparent stream initialization can resolve an unknown source size to the
multithreaded path and update appliedParams.nbWorkers. The Rust stream2
fallback previously captured that value before initialization, then entered the
serial generic adapter while the context stage was still created. Unknown-size
stdin compression consequently returned StageWrong instead of producing a
frame.
Keep the worker-mode projection live through initialization so Rust observes
the applied value at the existing MT-versus-serial decision point. The ABI
layout assertions now cover the pointer projection, and a focused regression
test verifies that an initialization callback which selects MT dispatches the
MT step and preserves the expected callback order.
Test Plan:
- capped cargo +nightly fmt for the Rust workspace -- passed
- capped root and CLI cargo clippy with -D warnings -- passed
- capped serial make -j1 for single-threaded library, MT library, and CLI -- passed without the prior pointer-sign warning
- capped stdin compression/decompression round trip for unknown-size input -- passed
- capped original suite, including CLI tests, native tests, fuzzer, and both zstream phases -- passed; the final signedness-only ABI spelling cleanup was followed by a clean capped rebuild
- targeted root cargo test attempted but remains blocked at link time by pre-existing decompression bridge symbols (ZSTD_rust_dctx_trace_view, ZSTD_rust_dctx_view, and ZSTD_rust_block_context_init)
The public ZSTD_compressStream2_c entry point still contained the complete
fallback state machine even though its validation, initialization policy,
serial-versus-MT dispatch, and result accounting were already represented by
Rust policy helpers. That left the main streaming entry point as a large C
orchestration island and made the Rust rewrite boundary misleading.
Project the scalar stream state and private CCtx operations through a checked
C ABI, then let Rust own the fallback ordering. Rust now validates buffers and
the end directive, handles transparent one-shot completion and stable-input
deferral, performs the initialization and stability decisions, selects the
serial or MT path, and publishes the result policy. C retains the private
context layout, stream adapter, MT operations, checksum/epilogue side effects,
and trace/reset callbacks. ABI offset assertions and focused callback-order
and stable-input tests protect the projection without requiring a standalone
Rust test binary to link every C bridge symbol.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed before and after formatting.
- `git diff --cached --check` -- passed.
- Native `make -j1` and the original suite remain the next post-commit verification gates.
Project the private requested-parameter and context state into a fixed scalar ABI record, leaving only C's parameter-selection leaf and private layout access in the shim. Rust now owns the complete-input eligibility predicate, including advanced-state exclusions, LDM sentinel handling, and the final fast/double-fast strategy gate, while preserving the requested-level and INT_MIN results.
Add focused policy tests for fast and double-fast acceptance, representative advanced-state rejection, the accepted LDM sentinel, non-fast strategies, and null state.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test
Move the multithreaded compressStream2 outer coordinator into the Rust compression module. The C bridge now owns only the private ZSTDMT call and consumed/produced counters, while Rust preserves the progress loop, completion ordering, and error handoff.
Keep the MT context mutations, diagnostics, assertions, and buffer-expectation publication in C because they depend on the private context layout. Add callback-driven tests for continue/break, error completion, end trace/reset ordering, and malformed states.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
The public CCtx and CStream size-estimation APIs still selected row-matchfinder modes and walked compression levels in C, leaving a policy-level implementation behind the Rust workspace-sizing bridge. That also made the C layer responsible for the subtle raw-size_t MAX behavior used when an estimator returns an encoded error.
Move the row-mode selection and shared monotonic memory-budget policy into Rust. Keep the C callbacks responsible for constructing private ZSTD_CCtx_params values and for the distinct CCtx/CStream sizing formulas, so no private parameter layout crosses the ABI. Preserve the signed MIN(compressionLevel, 1) start and raw maxima exactly.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
The MT serial-reset bridge now declares its Rust scalar projection before
executing the diagnostic branch. This keeps the declaration ordering valid for
the C90-oriented compiler flags used by the native test builds while leaving
the Rust-owned policy and callback sequence unchanged.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed without the
new mixed-declaration warning.
- `git diff --cached --check` -- passed.
The multithreaded stream loop already delegated scalar result classification to
Rust, but its final error/end conditional still mixed completion ordering with
the private C context. That left trace and reset sequencing embedded beside the
MT counters and made the next Rust seam depend on exposing CCtx layout.
Rust now consumes the projected loop action and sequences the final completion
side effects through opaque C callbacks. Continue and break do nothing, worker
errors reset the session, and completed end operations trace before resetting.
The reset callback result is intentionally discarded. C retains the private
context mutations in callback adapters, and the callback invocation remains
inside the existing ZSTD_MULTITHREAD branch before FORWARD_IF_ERROR(flushMin).
C/Rust repr(C) projections and ABI assertions protect the callback/result
boundary, while focused tests cover all four loop actions and end ordering.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- single-thread and `-DZSTD_MULTITHREAD` C syntax checks -- passed; existing warnings only
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed
- focused Rust test filter was attempted serially but is blocked by unrelated in-progress `rust/src/zstdmt_compress.rs` test ABI edits; no full test suites were run
ZSTD_compressBegin_internal already routes attach-versus-reload policy through
Rust, but its C insertion callback still inspected a non-null ZSTD_CDict and
selected dictContent, dictContentSize, and dictContentType. That left source
selection coupled to the private CDict layout and kept the Rust bridge from
receiving the same selected source for direct and prepared dictionaries.
Project the three CDict content fields into the repr(C) begin state. Rust now
selects either those fields or the direct dictionary arguments before invoking
the C callback. The callback keeps the private CCtx insertion path and no
longer branches on CDict. ABI offsets and sizes are asserted on both sides,
with focused by-reference and forced-CDict tests covering pointer, size, type,
reset order, and dictionary-result publication.
Test Plan:
- `cargo check --manifest-path rust/Cargo.toml --lib --tests` -- passed under
`ulimit -v 41943040; CARGO_BUILD_JOBS=1`.
- Serial GCC/Clang syntax-only checks for `lib/compress/zstd_compress.c` --
passed under the same cap; only existing warnings were reported.
- `cargo clippy` for lib, benches, and tests with `-D warnings`, plus nightly
format check -- passed serially under the same cap.
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress_dictionary
-- --test-threads=1` -- compiled but standalone linking failed on the three
pre-existing C bridge symbols `ZSTD_rust_dctx_trace_view`,
`ZSTD_rust_dctx_view`, and `ZSTD_rust_block_context_init`.
- Full native, upstream, and fuzzer tests were not run per scope.
Add the existing heap-mode bridge declaration beside the other decompression shim prototypes so the new Rust policy seam remains warning-clean under the native C build.
Test Plan:
- git diff --cached --check
- capped full test suite passed before this warning-only fix; final capped native rebuild follows
Route the configured default maximum window size through the Rust policy layer while keeping the build-time C configuration value and decoder layout in C. The explicit projection preserves overridden builds exactly and gives the scalar policy a focused null-input contract and tests.
Test Plan:
- git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs
- capped serial cargo check, clippy, nightly fmt, C syntax checks, and focused default-window tests (passed in the worker)
CDict initialization still hard-coded the match-state reset policies in its
C callback, even though Rust already owned the surrounding content branch,
workspace reservation, and callback ordering. That left the reset target and
cleanup policy split across the language boundary and made the intended CDict
reset contract implicit.
Extend the existing repr(C) reset callback with the three private enum values.
Rust now selects make-clean, index-reset, and the CDict reset target before
calling the opaque C operation. C retains the private ZSTD_CDict, workspace,
and match-state layouts and only casts the projected values for
ZSTD_reset_matchState(). Focused by-reference and by-copy probes record the
values and preserve reserve, reset, and insert ordering.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstd_compress_dictionary.rs` -- passed.
- GCC and Clang syntax-only checks for `lib/compress/zstd_compress.c` under
`ulimit -v 41943040` -- passed.
- `cargo check --manifest-path rust/Cargo.toml --lib --tests` under the cap -- passed.
- Serial `cargo clippy` lib, benches, and tests passes with `-D warnings`,
plus `cargo +nightly fmt --all -- --check` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress_dictionary
-- --test-threads=1` compiled but could not link the standalone test binary
because three pre-existing C bridge symbols are unavailable outside the
native harness; no native, upstream, or fuzzer tests were run.
Move the Rust teardown projection declaration before the null guard so the MT source remains clean under the repository's C90 declaration rules. The initializer only copies pointers, scalars, the allocator, and the callback; the null guard still precedes the bridge call and preserves the existing no-op behavior for an absent job table.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test (passed before this warning-only fix)
Normal MT context teardown still let the C wrapper directly sequence
per-job synchronization destruction before Rust released the opaque job-table
storage, while failed create and expansion transactions already used a Rust
helper for that same order. Route the normal teardown through a repr(C)
projection so Rust owns the destroy-before-storage-free policy consistently.
The synchronization callback, descriptor storage, and custom allocator remain
opaque C-owned operations. Add matching C/Rust layout assertions and a focused
callback-order test.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, native tests, and heavy verification were not run per request
ZSTD_compress_advanced_internal() still encoded the one-shot begin-then-end
sequence in C after public advanced parameter validation had moved behind Rust.
That left the public compression boundary split between Rust policy and a C
orchestration body, and made the begin-error ordering implicit in the C path.
Add an explicit C-layout state with opaque begin and end callbacks. Rust now
owns the ordering, forwards the source size as the pledged size, propagates a
begin error before invoking the end callback, and returns the end result. The
callbacks retain the private ZSTD_CCtx and ZSTD_CCtx_params layouts, so the
change moves policy and sequencing without exposing window, workspace,
matchfinder, or context internals to Rust. The existing advanced and
using-dictionary wrappers continue to use the same C private operations.
Test Plan:
- `git diff --cached --check` -- passed.
- `rustfmt --check --edition 2021 rust/src/zstd_compress.rs` -- passed.
- Not run: Cargo, make, native tests, builds, or heavy verification per request.
`ZSTD_HEAPMODE` was still returned directly from C, and the stack entry
projection also copied the raw macro. That left the one-shot stack-versus-heap
classification in the C configuration shim even though Rust owns the public
decompression branch and stack action.
Keep the build-time override in C, but pass it through a one-field repr(C)
projection to Rust. Rust normalizes values below one to the stack mode and
values at or above one to the heap mode, preserving the existing behavior
while making the branch policy explicit. The C stack object, private DCtx
layout, platform initialization, and trace boundary remain C-owned; moving
those would either duplicate build configuration or cross the requested ABI
boundary. The stack projection now uses the same normalized result as the
one-shot branch.
ABI layout assertions cover the scalar projection, and focused Rust tests cover
negative, zero, positive, maximum, and missing-configuration inputs.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Cargo, Make, native tests, and heavy commands intentionally not run per the
task restriction; integration verification remains pending.
Keep the MT frame-progression callback under the ZSTD_MULTITHREAD preprocessor guard using an ordinary C declaration. The previous warning-cleanup edit accidentally prefixed the declaration with a preprocessor marker, which broke the multithreaded compilation path.
Test Plan:
- git diff --cached --check
- Capped native rebuild failed before this fix at the invalid directive; rerun pending
The MT frame-progression callback is only referenced when ZSTD_MULTITHREAD is enabled. Keep its definition under the same configuration guard so single-threaded library, test, and decode-corpus builds do not report an unused static function while the MT callback remains available to the Rust dispatch bridge.
Test Plan:
- git diff --cached --check
- Capped full tests passed before this warning-only guard
- Capped native rebuild and smoke rerun pending
The Rust-owned frame-progression dispatch invokes the existing C MT progression API through a mutable opaque callback context. Match that API's established signature in the adapter so the new boundary does not introduce a discarded-const qualifier warning while preserving the private MT context.
Test Plan:
- git diff --cached --check
- Capped native make rerun pending after this warning-only fix
ZSTD_NO_FORWARD_PROGRESS_MAX is a compile-time override that was returned
directly by the C adapter. Keep the active build value in C, where
configuration belongs, but pass it through an ABI-checked scalar projection to
Rust. Rust now owns the policy boundary and preserves the exact configured
threshold, while the existing stalled-stream comparison, error mapping, and
private DCtx layout remain unchanged.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, builds, native tests, and heavy verification were not run per
request.
Move the public ZSTD_getFrameProgression MT-versus-single-thread worker-count
branch into a Rust-owned scalar projection. C continues to compute the
single-thread scalar inputs and keeps the private ZSTDMT context behind a
callback; the compile-time ZSTD_MULTITHREAD adapter remains local to C. Add
C/Rust layout assertions and focused Rust tests for both dispatch paths and
null-state fallback.
Test Plan:
- `git diff --cached --check`
- Not run: Cargo, make, builds, native tests, and heavy verification per request.
Keep the opaque ZSTD_CCtx_params copy operation in C, but make Rust own the public advanced2 null-source validation and copy-before-preparation ordering. The bridge now rejects a missing source before any allocator or workspace callback, with explicit ABI assertions and focused order/null-input tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C build-time mutual-exclusion check and private decoder-context assembly in the C shim, but pass the selected force-short/force-long policy through Rust. Rust now owns normalization to the runtime, short, or long sequence decoder mode, with ABI assertions and focused null/build-policy tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the target-specific ZSTD_LEGACY_SUPPORT macro in the C translation unit, but pass its scalar value through a Rust policy bridge before legacy dispatch. Rust now accepts only the historical supported range 1 through 7 and normalizes all other values, including a missing projection, to disabled. ABI assertions and focused tests preserve the compile-time configuration contract.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the advanced CDict estimator and its private workspace sizing inputs in C, but route the public estimate through a Rust policy bridge. Rust now selects unknown-source/create-CDict parameters and the public by-copy load mode before invoking the opaque C estimator. Focused tests verify callback ordering, selected parameters, and the missing-policy-input guard.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C-owned compression job callback responsible for private buffer and job-state mutation, but return the range-active decision alongside its result. Rust now owns the outer scheduler policy that publishes or suppresses the reusable input count after job creation, including the error path. The focused seam test covers the callback-clears-range case without exposing the MT context layout.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Move the ready/wrap branch and synchronization ordering from ZSTDMT_tryGetInputRange into Rust. The Rust policy now waits for the prefix range, invokes the C-owned prefix move, waits for the selected source range, and publishes the buffer; C retains memmove, private buffer fields, round-buffer state, and LDM callbacks behind the projection. Add ABI assertions and focused wrapped/ordinary ordering tests.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification follows.
Move the post-call ZSTD_compressStream2_c policy into Rust: adapter errors now short-circuit before buffer publication, successful calls publish buffer expectations before calculating pending output, and the private C context remains behind a callback projection. Add ABI assertions and ordering tests for both paths.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification will run after the MT slice is integrated.
The multithreaded branch of `ZSTD_compressStream2_c` performed the complete
post-call result classification in C: it prioritized errors, recognized a
completed end directive, and applied different progress rules for continue,
flush, and end operations. Those decisions were scalar policy around a C-owned
MT call, but remained embedded beside private counters, reset, and tracing.
Rust now classifies one projected iteration and returns an explicit action for
continue, break, error, or completed end. The C shim still owns the MT call,
input/output accounting, error forwarding, private context mutation, reset,
and trace callback; the existing condition ordering and progress comparisons
are preserved. ABI layout assertions and focused tests cover error precedence,
completed end, progress-based continue termination, and pending/full output.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- Full capped Rust/native verification is the next serial step.
The stack-backed one-shot decompression entry point used to make its own
heap-mode rejection, static DCtx initialization, allocation-error mapping,
and dispatch ordering in C. That kept scalar control flow beside the private
DCtx layout even though Rust already owns the decompression policy and context
leaf.
Rust now consumes an ABI-checked scalar projection and callback set. It makes
the branch and ordering decisions, maps a failed static initializer to the
same memory-allocation error, and invokes the existing C-owned context leaf.
The C shim retains the local stack object, private DCtx layout, and
`ZSTD_initStaticDCtx` implementation. Recording-callback tests cover heap
rejection, initializer failure, and successful initialization-to-dispatch
ordering without fabricating the private context layout.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
compression seam is integrated.
ZSTDMT_updateCParams_whileCompressing already delegated parameter derivation to Rust, but C still owned the transition publication order and independently updated compressionLevel. Return a Rust-owned scalar result that carries the requested level with the derived compression parameters, while preserving the active frame window and all private MT context state in C.\n\nThe new projection and result have explicit C/Rust layout assertions. Focused Rust tests exercise unknown-size and explicit-size hints, LDM and override inputs, saved-window restoration, requested-level forwarding, and the exported ABI wrapper.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
ZSTD_compressStream2_c still owns private context mutation, diagnostics, buffer updates, and codec initialization, but its stable-input transparent-initialization decision was a remaining scalar policy island in C. Move the continuation validation, block-size threshold, format-specific progress hint, and wrapping size arithmetic behind a Rust projection. Preserve the original C error messages and return values while keeping all private context state on the C side.\n\nThe bridge uses explicit ABI layout assertions for the mixed pointer/size projection. Focused Rust tests cover buffered and non-continue calls, block-boundary initialization, empty and short stable input, both frame formats, matching continuation state, invalid source/position, and null bridge input.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings\n- Full capped native and original-test verification follows after the batch is committed.
The MT stream initializer and its later dictionary update both encoded the
same three-way choice in C: copy a supplied dictionary, attach a borrowed
CDict, or install a raw prefix. The old code also embedded the required
release/clear-before-attach ordering in each callback wrapper. Project the
presence and raw-content flags into Rust, where the branch and ordering are
now explicit and tested. C retains only the private CDict allocation,
prefix-storage, context publication, and destruction callbacks, so the
configured C layouts and allocator behavior remain unchanged.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `git diff --check` and `rustfmt --edition 2021 --check` -- passed before commit
- Capped native and original-test verification remains pending for the complete batch.
ZSTD_freeCCtx previously passed one C callback that bundled dictionary,
multithreaded context, workspace, and heap-object teardown. That left the
lifetime order inside private C control flow and made the public destruction
policy harder to audit. Project the four private operations separately and
let Rust enforce the original dictionary, MT-context, workspace, and
object order while preserving the static-context and workspace-embedded
object guards. The callbacks still own the C layouts and custom allocator
operations.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- Broader native and original-test verification remains pending for the complete batch.
Project the scalar ZSTD_compressStream2 end directive into Rust and keep C responsible for the private streaming context, existing parameter error, and diagnostic text. The Rust range policy preserves the original unsigned enum validation for continue, flush, and end.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Move LDM block-compressor strategy validation, selection ordering, and callback error propagation into Rust while retaining dictionary-mode lookup, MatchState mutation, and the actual codec callback in C. Invalid preflight inputs now use the existing generic error path, and valid strategies preserve the original optimal-parser boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Make Rust the owner of the compression-job preparation sequence: publish the descriptor, reset the reusable input state, and publish terminal frame state last. Keep all MT job and stream layouts in C behind three focused callbacks, and preserve the nonterminal and terminal branches exactly.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
Keep the private static-CDict initializer in C, but return its native size_t status through the callback boundary. Rust now owns the public success-or-NULL mapping, preserving the original workspace validation, callback order, and private layout ownership while making initializer failures explicit and unit-testable.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
The resource-acquisition projection calls its field `ldmEnabled`, but the
first implementation passed the three-state `ZSTD_ParamSwitch_e` value
unchanged. `ZSTD_ps_disable` is 2, so Rust interpreted disabled LDM as enabled
and rejected ordinary jobs whose raw sequence store was intentionally absent.
That surfaced as an allocation error on the small stdin compression path.
Convert the C enum to the same boolean semantic used by the Rust policy before
crossing the ABI boundary. The projection declaration also now precedes the
callback setup statements, keeping the native C90 warning-free style.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The first capped full upstream run failed at `test-zstd` on the small stdin path with the enum/boolean regression; the suite will be rerun after this fix.
ddidderr