Move the decompression resource creation order into the Rust CLI policy
layer: dictionary stat and patch-memory preparation, decoder allocation,
window/checksum configuration, dictionary reset/attachment, and asynchronous
pool creation now run through one Rust-owned sequence. Keep dRess_t,
stat_t, dictionary buffers, decoder context, pools, and exact allocation
errors in C callbacks.
Preserve the original patch-from and dictionary-reference behavior while
adding ABI layout assertions, scalar policy coverage, and error short-circuit
tests for the new boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/cli/Cargo.toml --all
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (207 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
Make the Rust policy layer own the ordered compression-resource lifecycle:
create the CCtx, prepare patch/dictionary state, create the write and read
pools, validate the dictionary, apply general and multithreaded parameters,
and finally load the dictionary. Keep cRess_t, FIO_Dict_t, file statistics,
AIO pools, and CLI diagnostics in C callbacks so the existing resource
ownership and error behavior remain local to the C backend.
Preserve adaptive window defaults and patch-from parameter adjustment while
adding ABI layout assertions and a lifecycle-order test for the new boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/cli/Cargo.toml --all
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (205 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
The MT stream initializer previously exposed one C callback that waited for
all submitted jobs and then released their resources. That left the ordering
policy in C even though Rust already owned the surrounding initialization
sequence and the ring-order wait/release policies.
Expose separate wait-all and release-all callbacks at the Rust/C boundary and
invoke them in Rust in the original wait-before-release order. C retains the
worker synchronization, job descriptors, input buffer, and resource-pool
side effects behind the two callbacks, while the focused Rust initializer test
now makes the ordering explicit.
Test Plan:
- capped cargo +nightly fmt for the Rust workspace -- passed
- capped root cargo clippy --all-targets -- -D warnings -- passed
- capped serial make -j1 for the single-threaded library, MT library, and CLI -- passed
- git diff --cached --check -- passed
Transparent stream initialization can resolve an unknown source size to the
multithreaded path and update appliedParams.nbWorkers. The Rust stream2
fallback previously captured that value before initialization, then entered the
serial generic adapter while the context stage was still created. Unknown-size
stdin compression consequently returned StageWrong instead of producing a
frame.
Keep the worker-mode projection live through initialization so Rust observes
the applied value at the existing MT-versus-serial decision point. The ABI
layout assertions now cover the pointer projection, and a focused regression
test verifies that an initialization callback which selects MT dispatches the
MT step and preserves the expected callback order.
Test Plan:
- capped cargo +nightly fmt for the Rust workspace -- passed
- capped root and CLI cargo clippy with -D warnings -- passed
- capped serial make -j1 for single-threaded library, MT library, and CLI -- passed without the prior pointer-sign warning
- capped stdin compression/decompression round trip for unknown-size input -- passed
- capped original suite, including CLI tests, native tests, fuzzer, and both zstream phases -- passed; the final signedness-only ABI spelling cleanup was followed by a clean capped rebuild
- targeted root cargo test attempted but remains blocked at link time by pre-existing decompression bridge symbols (ZSTD_rust_dctx_trace_view, ZSTD_rust_dctx_view, and ZSTD_rust_block_context_init)
FIO_openDstFile() still owned the complete destination policy loop in C even
though the filesystem opener and status/action classifier were already Rust
leaves. That left confirmation, sparse-mode adjustment, overwrite removal, and
retry ordering duplicated beside private FILE* and CLI state.
Project the C-owned callbacks for diagnostics, preference mutation, user
confirmation, stdout setup, and existing-file removal. Rust now owns the
retry/order state machine and calls the existing narrow opener for each
attempt; C keeps private preferences and context, exact diagnostics, and FILE*
ownership. The callback layout is asserted on both sides, and focused tests
cover status ordering, confirmation/removal retry, setvbuf preservation, and
invalid policy inputs.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed, including the single-thread library, MT library, and CLI binary.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed: 206 tests.
- `git diff --cached --check` -- passed.
The public ZSTD_compressStream2_c entry point still contained the complete
fallback state machine even though its validation, initialization policy,
serial-versus-MT dispatch, and result accounting were already represented by
Rust policy helpers. That left the main streaming entry point as a large C
orchestration island and made the Rust rewrite boundary misleading.
Project the scalar stream state and private CCtx operations through a checked
C ABI, then let Rust own the fallback ordering. Rust now validates buffers and
the end directive, handles transparent one-shot completion and stable-input
deferral, performs the initialization and stability decisions, selects the
serial or MT path, and publishes the result policy. C retains the private
context layout, stream adapter, MT operations, checksum/epilogue side effects,
and trace/reset callbacks. ABI offset assertions and focused callback-order
and stable-input tests protect the projection without requiring a standalone
Rust test binary to link every C bridge symbol.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed before and after formatting.
- `git diff --cached --check` -- passed.
- Native `make -j1` and the original suite remain the next post-commit verification gates.
Project the private requested-parameter and context state into a fixed scalar ABI record, leaving only C's parameter-selection leaf and private layout access in the shim. Rust now owns the complete-input eligibility predicate, including advanced-state exclusions, LDM sentinel handling, and the final fast/double-fast strategy gate, while preserving the requested-level and INT_MIN results.
Add focused policy tests for fast and double-fast acceptance, representative advanced-state rejection, the accepted LDM sentinel, non-fast strategies, and null state.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test
Document Rust ownership of compression-size estimator policy and the CLI adaptive feedback state machine, and describe the private C callbacks that remain at those boundaries.
Test Plan:
- Documentation-only change; git diff --cached --check
Move the multithreaded compressStream2 outer coordinator into the Rust compression module. The C bridge now owns only the private ZSTDMT call and consumed/produced counters, while Rust preserves the progress loop, completion ordering, and error handoff.
Keep the MT context mutations, diagnostics, assertions, and buffer-expectation publication in C because they depend on the private context layout. Add callback-driven tests for continue/break, error completion, end trace/reset ordering, and malformed states.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
Clippy rejects the hand-written round-up expression used to compute the callback-alignment offset in the Rust file-I/O projection. Use the equivalent div_ceil spelling so the ABI assertion remains clear and the root Rust targets pass with warnings denied.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
The zstd file-I/O loop still kept adaptive compression's state machine in C: progression deltas, refresh gating, job-completion checks, input counters, speed decisions, and level updates were interleaved with private FIO and ZSTD state. That left orchestration policy behind the existing scalar Rust predicates.
Move the adaptive state and callback order into Rust. C now supplies scalar progression snapshots, the clock gate, exact diagnostics, and the ignored CCtx parameter setter through callbacks; private FIO_prefs_t, ZSTD_CCtx, ZSTD_frameProgression, clocks, and progress formatting remain C-owned. Preserve the previous-progression publication before backlog evaluation, input counter reset points, and serial/MT level-clamp behavior.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
The public CCtx and CStream size-estimation APIs still selected row-matchfinder modes and walked compression levels in C, leaving a policy-level implementation behind the Rust workspace-sizing bridge. That also made the C layer responsible for the subtle raw-size_t MAX behavior used when an estimator returns an encoded error.
Move the row-mode selection and shared monotonic memory-budget policy into Rust. Keep the C callbacks responsible for constructing private ZSTD_CCtx_params values and for the distinct CCtx/CStream sizing formulas, so no private parameter layout crosses the ABI. Preserve the signed MIN(compressionLevel, 1) start and raw maxima exactly.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1
Update the migration boundary document to record the two latest multithreaded
policy moves. Rust now owns serial-reset LDM normalization, table sizing, and
publication plus final completion trace/reset ordering, while C still owns the
private job/context state and callbacks.
Test Plan:
- `git diff --cached --check` -- passed.
- Capped native build and full original test target were run before this
documentation-only commit and passed.
The MT serial-reset bridge now declares its Rust scalar projection before
executing the diagnostic branch. This keeps the declaration ordering valid for
the C90-oriented compiler flags used by the native test builds while leaving
the Rust-owned policy and callback sequence unchanged.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed without the
new mixed-declaration warning.
- `git diff --cached --check` -- passed.
The multithreaded stream loop already delegated scalar result classification to
Rust, but its final error/end conditional still mixed completion ordering with
the private C context. That left trace and reset sequencing embedded beside the
MT counters and made the next Rust seam depend on exposing CCtx layout.
Rust now consumes the projected loop action and sequences the final completion
side effects through opaque C callbacks. Continue and break do nothing, worker
errors reset the session, and completed end operations trace before resetting.
The reset callback result is intentionally discarded. C retains the private
context mutations in callback adapters, and the callback invocation remains
inside the existing ZSTD_MULTITHREAD branch before FORWARD_IF_ERROR(flushMin).
C/Rust repr(C) projections and ABI assertions protect the callback/result
boundary, while focused tests cover all four loop actions and end ordering.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- single-thread and `-DZSTD_MULTITHREAD` C syntax checks -- passed; existing warnings only
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed
- focused Rust test filter was attempted serially but is blocked by unrelated in-progress `rust/src/zstdmt_compress.rs` test ABI edits; no full test suites were run
Move the multi-input single-output decision sequence out of
FIO_multiFilesConcatWarning while preserving the C-owned CLI boundary. The
previous wrapper classified fatal remove cases, emitted the concatenation
warning, disabled --rm, reclassified the action, and then selected quiet
abort or confirmation in C. Add a repr(C) callback projection so Rust owns
only that scalar ordering while C continues to own exact diagnostics, the
confirmation prompt, FIO_prefs_t mutation, and all private state.
The Rust bridge re-runs the action after the C disable-remove callback with
the same has-output/remove arguments as the original wrapper. C and Rust
assert the callback layout, and focused tests cover callback order,
fatal/quiet paths, and ABI offsets. Existing scalar action tests remain in
place.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -B -C programs -j1 fileio.o`
— passed; only pre-existing suffixList C++-compat warnings appeared.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path
rust/cli/Cargo.toml --tests --no-deps` — passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 RUSTFLAGS='-C
link-arg=/tmp/zstd_rust_test_bridges.o' cargo test --manifest-path
rust/cli/Cargo.toml --lib 'fileio_prefs::tests::multi_files_concat_'`
— 8 passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo build --manifest-path
rust/cli/Cargo.toml --lib` — passed; the archive exports
`FIO_rust_multiFilesConcatWarning`.
- `cargo +nightly fmt --manifest-path rust/cli/Cargo.toml -- --check` — not
clean due pre-existing formatting drift in unchanged Rust code; no bulk
formatting was applied.
- Full native/upstream/fuzzer suites were not run by request.
ZSTD_compressBegin_internal already routes attach-versus-reload policy through
Rust, but its C insertion callback still inspected a non-null ZSTD_CDict and
selected dictContent, dictContentSize, and dictContentType. That left source
selection coupled to the private CDict layout and kept the Rust bridge from
receiving the same selected source for direct and prepared dictionaries.
Project the three CDict content fields into the repr(C) begin state. Rust now
selects either those fields or the direct dictionary arguments before invoking
the C callback. The callback keeps the private CCtx insertion path and no
longer branches on CDict. ABI offsets and sizes are asserted on both sides,
with focused by-reference and forced-CDict tests covering pointer, size, type,
reset order, and dictionary-result publication.
Test Plan:
- `cargo check --manifest-path rust/Cargo.toml --lib --tests` -- passed under
`ulimit -v 41943040; CARGO_BUILD_JOBS=1`.
- Serial GCC/Clang syntax-only checks for `lib/compress/zstd_compress.c` --
passed under the same cap; only existing warnings were reported.
- `cargo clippy` for lib, benches, and tests with `-D warnings`, plus nightly
format check -- passed serially under the same cap.
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress_dictionary
-- --test-threads=1` -- compiled but standalone linking failed on the three
pre-existing C bridge symbols `ZSTD_rust_dctx_trace_view`,
`ZSTD_rust_dctx_view`, and `ZSTD_rust_block_context_init`.
- Full native, upstream, and fuzzer tests were not run per scope.
Synchronize the ownership map with the default-window decompression policy, the Rust-owned single-file --list ordering, and CDict match-state reset policy moved in the latest migration cycle.
Test Plan:
- git diff --cached --check
Add the existing heap-mode bridge declaration beside the other decompression shim prototypes so the new Rust policy seam remains warning-clean under the native C build.
Test Plan:
- git diff --cached --check
- capped full test suite passed before this warning-only fix; final capped native rebuild follows
Move the single-file --list status gates, diagnostic-versus-metadata ordering, and aggregate projection into Rust. C retains file opening and frame analysis, the private fileInfo_t layout, exact diagnostics, and row formatting behind explicit callbacks, so the user-visible behavior remains unchanged while the policy boundary is auditable.
Test Plan:
- git diff --cached --check
- worker format, check, clippy, and serial C compilation (passed); focused Rust tests compiled but the standalone link hit the pre-existing C bridge-symbol gap
Route the configured default maximum window size through the Rust policy layer while keeping the build-time C configuration value and decoder layout in C. The explicit projection preserves overridden builds exactly and gives the scalar policy a focused null-input contract and tests.
Test Plan:
- git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs
- capped serial cargo check, clippy, nightly fmt, C syntax checks, and focused default-window tests (passed in the worker)
CDict initialization still hard-coded the match-state reset policies in its
C callback, even though Rust already owned the surrounding content branch,
workspace reservation, and callback ordering. That left the reset target and
cleanup policy split across the language boundary and made the intended CDict
reset contract implicit.
Extend the existing repr(C) reset callback with the three private enum values.
Rust now selects make-clean, index-reset, and the CDict reset target before
calling the opaque C operation. C retains the private ZSTD_CDict, workspace,
and match-state layouts and only casts the projected values for
ZSTD_reset_matchState(). Focused by-reference and by-copy probes record the
values and preserve reserve, reset, and insert ordering.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstd_compress_dictionary.rs` -- passed.
- GCC and Clang syntax-only checks for `lib/compress/zstd_compress.c` under
`ulimit -v 41943040` -- passed.
- `cargo check --manifest-path rust/Cargo.toml --lib --tests` under the cap -- passed.
- Serial `cargo clippy` lib, benches, and tests passes with `-D warnings`,
plus `cargo +nightly fmt --all -- --check` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress_dictionary
-- --test-threads=1` compiled but could not link the standalone test binary
because three pre-existing C bridge symbols are unavailable outside the
native harness; no native, upstream, or fuzzer tests were run.
Document the advanced one-shot begin/end seam, MT job-table teardown order, decompression heap-mode policy, and file-removal status policy so the migration boundary stays synchronized with the implementation.
Test Plan:
- git diff --cached --check
Move the Rust teardown projection declaration before the null guard so the MT source remains clean under the repository's C90 declaration rules. The initializer only copies pointers, scalars, the allocator, and the callback; the null guard still precedes the bridge call and preserves the existing no-op behavior for an absent job table.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test (passed before this warning-only fix)
Apply the repository formatter to the new MT job-table teardown projection and its focused callback-order test. This keeps the accepted lifecycle seam behavior unchanged while restoring the crate's formatting contract.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all
- git diff --cached --check
Normal MT context teardown still let the C wrapper directly sequence
per-job synchronization destruction before Rust released the opaque job-table
storage, while failed create and expansion transactions already used a Rust
helper for that same order. Route the normal teardown through a repr(C)
projection so Rust owns the destroy-before-storage-free policy consistently.
The synchronization callback, descriptor storage, and custom allocator remain
opaque C-owned operations. Add matching C/Rust layout assertions and a focused
callback-order test.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, native tests, and heavy verification were not run per request
ZSTD_compress_advanced_internal() still encoded the one-shot begin-then-end
sequence in C after public advanced parameter validation had moved behind Rust.
That left the public compression boundary split between Rust policy and a C
orchestration body, and made the begin-error ordering implicit in the C path.
Add an explicit C-layout state with opaque begin and end callbacks. Rust now
owns the ordering, forwards the source size as the pledged size, propagates a
begin error before invoking the end callback, and returns the end result. The
callbacks retain the private ZSTD_CCtx and ZSTD_CCtx_params layouts, so the
change moves policy and sequencing without exposing window, workspace,
matchfinder, or context internals to Rust. The existing advanced and
using-dictionary wrappers continue to use the same C private operations.
Test Plan:
- `git diff --cached --check` -- passed.
- `rustfmt --check --edition 2021 rust/src/zstd_compress.rs` -- passed.
- Not run: Cargo, make, native tests, builds, or heavy verification per request.
`ZSTD_HEAPMODE` was still returned directly from C, and the stack entry
projection also copied the raw macro. That left the one-shot stack-versus-heap
classification in the C configuration shim even though Rust owns the public
decompression branch and stack action.
Keep the build-time override in C, but pass it through a one-field repr(C)
projection to Rust. Rust normalizes values below one to the stack mode and
values at or above one to the heap mode, preserving the existing behavior
while making the branch policy explicit. The C stack object, private DCtx
layout, platform initialization, and trace boundary remain C-owned; moving
those would either duplicate build configuration or cross the requested ABI
boundary. The stack projection now uses the same normalized result as the
one-shot branch.
ABI layout assertions cover the scalar projection, and focused Rust tests cover
negative, zero, positive, maximum, and missing-configuration inputs.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Cargo, Make, native tests, and heavy commands intentionally not run per the
task restriction; integration verification remains pending.
Keep the MT frame-progression callback under the ZSTD_MULTITHREAD preprocessor guard using an ordinary C declaration. The previous warning-cleanup edit accidentally prefixed the declaration with a preprocessor marker, which broke the multithreaded compilation path.
Test Plan:
- git diff --cached --check
- Capped native rebuild failed before this fix at the invalid directive; rerun pending
Keep the migration boundary accurate after moving frame-progression dispatch, the decompression no-forward-progress threshold, and compression metadata-transfer selection into Rust. The README now states which scalar inputs Rust owns and which private callbacks, probes, and operations remain in C.
Test Plan:
- git diff --cached --check
- Capped clippy, CLI tests, native build, smoke test, and full upstream suite passed before this documentation-only commit
The MT frame-progression callback is only referenced when ZSTD_MULTITHREAD is enabled. Keep its definition under the same configuration guard so single-threaded library, test, and decode-corpus builds do not report an unused static function while the MT callback remains available to the Rust dispatch bridge.
Test Plan:
- git diff --cached --check
- Capped full tests passed before this warning-only guard
- Capped native rebuild and smoke rerun pending
The Rust-owned frame-progression dispatch invokes the existing C MT progression API through a mutable opaque callback context. Match that API's established signature in the adapter so the new boundary does not introduce a discarded-const qualifier warning while preserving the private MT context.
Test Plan:
- git diff --cached --check
- Capped native make rerun pending after this warning-only fix
ZSTD_NO_FORWARD_PROGRESS_MAX is a compile-time override that was returned
directly by the C adapter. Keep the active build value in C, where
configuration belongs, but pass it through an ABI-checked scalar projection to
Rust. Rust now owns the policy boundary and preserves the exact configured
threshold, while the existing stalled-stream comparison, error mapping, and
private DCtx layout remain unchanged.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, builds, native tests, and heavy verification were not run per
request.
Move the public ZSTD_getFrameProgression MT-versus-single-thread worker-count
branch into a Rust-owned scalar projection. C continues to compute the
single-thread scalar inputs and keeps the private ZSTDMT context behind a
callback; the compile-time ZSTD_MULTITHREAD adapter remains local to C. Add
C/Rust layout assertions and focused Rust tests for both dispatch paths and
null-state fallback.
Test Plan:
- `git diff --cached --check`
- Not run: Cargo, make, builds, native tests, and heavy verification per request.
Keep the private stat_t probe, destination opening, metadata syscalls, and format callbacks in C, but route the regular-source plus stdin/stdout exception policy through the Rust fileio preference layer. The Rust ABI bridge normalizes the scalar consumed by the existing destination lifecycle, with compile-time value assertions and focused tests covering regular, non-regular, stdin, stdout, and nonzero scalar inputs.
Test Plan: Not run by request; cargo, make, native tests, and heavy commands were intentionally avoided. Lightweight git diff --check passed.
Apply the repository formatter to the sequence-decoder policy bridge introduced in the preceding decompression refactor. The change is formatting-only.
Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
Keep the opaque ZSTD_CCtx_params copy operation in C, but make Rust own the public advanced2 null-source validation and copy-before-preparation ordering. The bridge now rejects a missing source before any allocator or workspace callback, with explicit ABI assertions and focused order/null-input tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C build-time mutual-exclusion check and private decoder-context assembly in the C shim, but pass the selected force-short/force-long policy through Rust. Rust now owns normalization to the runtime, short, or long sequence decoder mode, with ABI assertions and focused null/build-policy tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the target-specific ZSTD_LEGACY_SUPPORT macro in the C translation unit, but pass its scalar value through a Rust policy bridge before legacy dispatch. Rust now accepts only the historical supported range 1 through 7 and normalizes all other values, including a missing projection, to disabled. ABI assertions and focused tests preserve the compile-time configuration contract.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep FIO_removeFile responsible for the filesystem operation, exact diagnostics, and C return wrapper, but route its status classification through a Rust policy bridge. Unknown statuses are explicitly treated as failed removal. ABI value assertions and focused mapping tests preserve the C contract.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Apply the repository formatter to the dictionary-size and MT input-publication bridges introduced in the preceding atomic refactors. The change is formatting-only and keeps the semantic seams independently reviewable.
Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all
Keep the advanced CDict estimator and its private workspace sizing inputs in C, but route the public estimate through a Rust policy bridge. Rust now selects unknown-source/create-CDict parameters and the public by-copy load mode before invoking the opaque C estimator. Focused tests verify callback ordering, selected parameters, and the missing-policy-input guard.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C-owned compression job callback responsible for private buffer and job-state mutation, but return the range-active decision alongside its result. Rust now owns the outer scheduler policy that publishes or suppresses the reusable input count after job creation, including the error path. The focused seam test covers the callback-clears-range case without exposing the MT context layout.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Move the scalar InfoError-to-action mapping used by FIO_listFile into Rust and reuse the same classifier while aggregating multi-file list results. C retains file opening/parsing, exact diagnostics, metadata formatting, private fileInfo_t state, and the public result values; add layout and mapping coverage.
Test Plan: git diff --cached --check; focused Rust mapping test added; full capped verification will run after the MT and dictionary workers are integrated.
Apply rustfmt to the stream-result and MT input-range policy projections and their focused tests. This is mechanical only; it keeps the new Rust seams compliant with the repository formatter without changing behavior.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; git diff --cached --check.
Move the ready/wrap branch and synchronization ordering from ZSTDMT_tryGetInputRange into Rust. The Rust policy now waits for the prefix range, invokes the C-owned prefix move, waits for the selected source range, and publishes the buffer; C retains memmove, private buffer fields, round-buffer state, and LDM callbacks behind the projection. Add ABI assertions and focused wrapped/ordinary ordering tests.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification follows.
Move the post-call ZSTD_compressStream2_c policy into Rust: adapter errors now short-circuit before buffer publication, successful calls publish buffer expectations before calculating pending output, and the private C context remains behind a callback projection. Add ABI assertions and ordering tests for both paths.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification will run after the MT slice is integrated.
Move stdin-sentinel classification and source stat/open result policy into the Rust fileio backend while keeping C responsible for diagnostics, binary-mode setup, and FILE ownership. The bridge leaves stat layout and native file utilities behind the existing C ABI and adds a focused no-stat stdin test.
Test Plan: git diff --cached --check; focused Rust test added but full capped verification will run after the remaining workers are integrated.
The new MT-loop policy projection asserted its total size as eight times
`size_of::<usize>()`. Although that describes the padded C layout, clippy
interprets the multiplication as a manual bit-count expression and rejects it
under the repository's `-D warnings` policy.
Express the same ABI invariant from the final field offset plus its field size.
This keeps the assertion tied to the actual projected layout without changing
any runtime policy or C/Rust representation.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
ddidderr