Synchronize the ownership map with the default-window decompression policy, the Rust-owned single-file --list ordering, and CDict match-state reset policy moved in the latest migration cycle.
Test Plan:
- git diff --cached --check
Add the existing heap-mode bridge declaration beside the other decompression shim prototypes so the new Rust policy seam remains warning-clean under the native C build.
Test Plan:
- git diff --cached --check
- capped full test suite passed before this warning-only fix; final capped native rebuild follows
Move the single-file --list status gates, diagnostic-versus-metadata ordering, and aggregate projection into Rust. C retains file opening and frame analysis, the private fileInfo_t layout, exact diagnostics, and row formatting behind explicit callbacks, so the user-visible behavior remains unchanged while the policy boundary is auditable.
Test Plan:
- git diff --cached --check
- worker format, check, clippy, and serial C compilation (passed); focused Rust tests compiled but the standalone link hit the pre-existing C bridge-symbol gap
Route the configured default maximum window size through the Rust policy layer while keeping the build-time C configuration value and decoder layout in C. The explicit projection preserves overridden builds exactly and gives the scalar policy a focused null-input contract and tests.
Test Plan:
- git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs
- capped serial cargo check, clippy, nightly fmt, C syntax checks, and focused default-window tests (passed in the worker)
CDict initialization still hard-coded the match-state reset policies in its
C callback, even though Rust already owned the surrounding content branch,
workspace reservation, and callback ordering. That left the reset target and
cleanup policy split across the language boundary and made the intended CDict
reset contract implicit.
Extend the existing repr(C) reset callback with the three private enum values.
Rust now selects make-clean, index-reset, and the CDict reset target before
calling the opaque C operation. C retains the private ZSTD_CDict, workspace,
and match-state layouts and only casts the projected values for
ZSTD_reset_matchState(). Focused by-reference and by-copy probes record the
values and preserve reserve, reset, and insert ordering.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstd_compress_dictionary.rs` -- passed.
- GCC and Clang syntax-only checks for `lib/compress/zstd_compress.c` under
`ulimit -v 41943040` -- passed.
- `cargo check --manifest-path rust/Cargo.toml --lib --tests` under the cap -- passed.
- Serial `cargo clippy` lib, benches, and tests passes with `-D warnings`,
plus `cargo +nightly fmt --all -- --check` -- passed.
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress_dictionary
-- --test-threads=1` compiled but could not link the standalone test binary
because three pre-existing C bridge symbols are unavailable outside the
native harness; no native, upstream, or fuzzer tests were run.
Document the advanced one-shot begin/end seam, MT job-table teardown order, decompression heap-mode policy, and file-removal status policy so the migration boundary stays synchronized with the implementation.
Test Plan:
- git diff --cached --check
Move the Rust teardown projection declaration before the null guard so the MT source remains clean under the repository's C90 declaration rules. The initializer only copies pointers, scalars, the allocator, and the callback; the null guard still precedes the bridge call and preserves the existing no-op behavior for an absent job table.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test (passed before this warning-only fix)
Apply the repository formatter to the new MT job-table teardown projection and its focused callback-order test. This keeps the accepted lifecycle seam behavior unchanged while restoring the crate's formatting contract.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml --all
- git diff --cached --check
Normal MT context teardown still let the C wrapper directly sequence
per-job synchronization destruction before Rust released the opaque job-table
storage, while failed create and expansion transactions already used a Rust
helper for that same order. Route the normal teardown through a repr(C)
projection so Rust owns the destroy-before-storage-free policy consistently.
The synchronization callback, descriptor storage, and custom allocator remain
opaque C-owned operations. Add matching C/Rust layout assertions and a focused
callback-order test.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, native tests, and heavy verification were not run per request
ZSTD_compress_advanced_internal() still encoded the one-shot begin-then-end
sequence in C after public advanced parameter validation had moved behind Rust.
That left the public compression boundary split between Rust policy and a C
orchestration body, and made the begin-error ordering implicit in the C path.
Add an explicit C-layout state with opaque begin and end callbacks. Rust now
owns the ordering, forwards the source size as the pledged size, propagates a
begin error before invoking the end callback, and returns the end result. The
callbacks retain the private ZSTD_CCtx and ZSTD_CCtx_params layouts, so the
change moves policy and sequencing without exposing window, workspace,
matchfinder, or context internals to Rust. The existing advanced and
using-dictionary wrappers continue to use the same C private operations.
Test Plan:
- `git diff --cached --check` -- passed.
- `rustfmt --check --edition 2021 rust/src/zstd_compress.rs` -- passed.
- Not run: Cargo, make, native tests, builds, or heavy verification per request.
`ZSTD_HEAPMODE` was still returned directly from C, and the stack entry
projection also copied the raw macro. That left the one-shot stack-versus-heap
classification in the C configuration shim even though Rust owns the public
decompression branch and stack action.
Keep the build-time override in C, but pass it through a one-field repr(C)
projection to Rust. Rust normalizes values below one to the stack mode and
values at or above one to the heap mode, preserving the existing behavior
while making the branch policy explicit. The C stack object, private DCtx
layout, platform initialization, and trace boundary remain C-owned; moving
those would either duplicate build configuration or cross the requested ABI
boundary. The stack projection now uses the same normalized result as the
one-shot branch.
ABI layout assertions cover the scalar projection, and focused Rust tests cover
negative, zero, positive, maximum, and missing-configuration inputs.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Cargo, Make, native tests, and heavy commands intentionally not run per the
task restriction; integration verification remains pending.
Keep the MT frame-progression callback under the ZSTD_MULTITHREAD preprocessor guard using an ordinary C declaration. The previous warning-cleanup edit accidentally prefixed the declaration with a preprocessor marker, which broke the multithreaded compilation path.
Test Plan:
- git diff --cached --check
- Capped native rebuild failed before this fix at the invalid directive; rerun pending
Keep the migration boundary accurate after moving frame-progression dispatch, the decompression no-forward-progress threshold, and compression metadata-transfer selection into Rust. The README now states which scalar inputs Rust owns and which private callbacks, probes, and operations remain in C.
Test Plan:
- git diff --cached --check
- Capped clippy, CLI tests, native build, smoke test, and full upstream suite passed before this documentation-only commit
The MT frame-progression callback is only referenced when ZSTD_MULTITHREAD is enabled. Keep its definition under the same configuration guard so single-threaded library, test, and decode-corpus builds do not report an unused static function while the MT callback remains available to the Rust dispatch bridge.
Test Plan:
- git diff --cached --check
- Capped full tests passed before this warning-only guard
- Capped native rebuild and smoke rerun pending
The Rust-owned frame-progression dispatch invokes the existing C MT progression API through a mutable opaque callback context. Match that API's established signature in the adapter so the new boundary does not introduce a discarded-const qualifier warning while preserving the private MT context.
Test Plan:
- git diff --cached --check
- Capped native make rerun pending after this warning-only fix
ZSTD_NO_FORWARD_PROGRESS_MAX is a compile-time override that was returned
directly by the C adapter. Keep the active build value in C, where
configuration belongs, but pass it through an ABI-checked scalar projection to
Rust. Rust now owns the policy boundary and preserves the exact configured
threshold, while the existing stalled-stream comparison, error mapping, and
private DCtx layout remain unchanged.
Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, builds, native tests, and heavy verification were not run per
request.
Move the public ZSTD_getFrameProgression MT-versus-single-thread worker-count
branch into a Rust-owned scalar projection. C continues to compute the
single-thread scalar inputs and keeps the private ZSTDMT context behind a
callback; the compile-time ZSTD_MULTITHREAD adapter remains local to C. Add
C/Rust layout assertions and focused Rust tests for both dispatch paths and
null-state fallback.
Test Plan:
- `git diff --cached --check`
- Not run: Cargo, make, builds, native tests, and heavy verification per request.
Keep the private stat_t probe, destination opening, metadata syscalls, and format callbacks in C, but route the regular-source plus stdin/stdout exception policy through the Rust fileio preference layer. The Rust ABI bridge normalizes the scalar consumed by the existing destination lifecycle, with compile-time value assertions and focused tests covering regular, non-regular, stdin, stdout, and nonzero scalar inputs.
Test Plan: Not run by request; cargo, make, native tests, and heavy commands were intentionally avoided. Lightweight git diff --check passed.
Apply the repository formatter to the sequence-decoder policy bridge introduced in the preceding decompression refactor. The change is formatting-only.
Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
Keep the opaque ZSTD_CCtx_params copy operation in C, but make Rust own the public advanced2 null-source validation and copy-before-preparation ordering. The bridge now rejects a missing source before any allocator or workspace callback, with explicit ABI assertions and focused order/null-input tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C build-time mutual-exclusion check and private decoder-context assembly in the C shim, but pass the selected force-short/force-long policy through Rust. Rust now owns normalization to the runtime, short, or long sequence decoder mode, with ABI assertions and focused null/build-policy tests.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the target-specific ZSTD_LEGACY_SUPPORT macro in the C translation unit, but pass its scalar value through a Rust policy bridge before legacy dispatch. Rust now accepts only the historical supported range 1 through 7 and normalizes all other values, including a missing projection, to disabled. ABI assertions and focused tests preserve the compile-time configuration contract.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep FIO_removeFile responsible for the filesystem operation, exact diagnostics, and C return wrapper, but route its status classification through a Rust policy bridge. Unknown statuses are explicitly treated as failed removal. ABI value assertions and focused mapping tests preserve the C contract.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Apply the repository formatter to the dictionary-size and MT input-publication bridges introduced in the preceding atomic refactors. The change is formatting-only and keeps the semantic seams independently reviewable.
Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all
Keep the advanced CDict estimator and its private workspace sizing inputs in C, but route the public estimate through a Rust policy bridge. Rust now selects unknown-source/create-CDict parameters and the public by-copy load mode before invoking the opaque C estimator. Focused tests verify callback ordering, selected parameters, and the missing-policy-input guard.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Keep the C-owned compression job callback responsible for private buffer and job-state mutation, but return the range-active decision alongside its result. Rust now owns the outer scheduler policy that publishes or suppresses the reusable input count after job creation, including the error path. The focused seam test covers the callback-clears-range case without exposing the MT context layout.
Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
Move the scalar InfoError-to-action mapping used by FIO_listFile into Rust and reuse the same classifier while aggregating multi-file list results. C retains file opening/parsing, exact diagnostics, metadata formatting, private fileInfo_t state, and the public result values; add layout and mapping coverage.
Test Plan: git diff --cached --check; focused Rust mapping test added; full capped verification will run after the MT and dictionary workers are integrated.
Apply rustfmt to the stream-result and MT input-range policy projections and their focused tests. This is mechanical only; it keeps the new Rust seams compliant with the repository formatter without changing behavior.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; git diff --cached --check.
Move the ready/wrap branch and synchronization ordering from ZSTDMT_tryGetInputRange into Rust. The Rust policy now waits for the prefix range, invokes the C-owned prefix move, waits for the selected source range, and publishes the buffer; C retains memmove, private buffer fields, round-buffer state, and LDM callbacks behind the projection. Add ABI assertions and focused wrapped/ordinary ordering tests.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification follows.
Move the post-call ZSTD_compressStream2_c policy into Rust: adapter errors now short-circuit before buffer publication, successful calls publish buffer expectations before calculating pending output, and the private C context remains behind a callback projection. Add ABI assertions and ordering tests for both paths.
Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification will run after the MT slice is integrated.
Move stdin-sentinel classification and source stat/open result policy into the Rust fileio backend while keeping C responsible for diagnostics, binary-mode setup, and FILE ownership. The bridge leaves stat layout and native file utilities behind the existing C ABI and adds a focused no-stat stdin test.
Test Plan: git diff --cached --check; focused Rust test added but full capped verification will run after the remaining workers are integrated.
The new MT-loop policy projection asserted its total size as eight times
`size_of::<usize>()`. Although that describes the padded C layout, clippy
interprets the multiplication as a manual bit-count expression and rejects it
under the repository's `-D warnings` policy.
Express the same ABI invariant from the final field offset plus its field size.
This keeps the assertion tied to the actual projected layout without changing
any runtime policy or C/Rust representation.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
The multithreaded branch of `ZSTD_compressStream2_c` performed the complete
post-call result classification in C: it prioritized errors, recognized a
completed end directive, and applied different progress rules for continue,
flush, and end operations. Those decisions were scalar policy around a C-owned
MT call, but remained embedded beside private counters, reset, and tracing.
Rust now classifies one projected iteration and returns an explicit action for
continue, break, error, or completed end. The C shim still owns the MT call,
input/output accounting, error forwarding, private context mutation, reset,
and trace callback; the existing condition ordering and progress comparisons
are preserved. ABI layout assertions and focused tests cover error precedence,
completed end, progress-based continue termination, and pending/full output.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- Full capped Rust/native verification is the next serial step.
The stack-backed one-shot decompression entry point used to make its own
heap-mode rejection, static DCtx initialization, allocation-error mapping,
and dispatch ordering in C. That kept scalar control flow beside the private
DCtx layout even though Rust already owns the decompression policy and context
leaf.
Rust now consumes an ABI-checked scalar projection and callback set. It makes
the branch and ordering decisions, maps a failed static initializer to the
same memory-allocation error, and invokes the existing C-owned context leaf.
The C shim retains the local stack object, private DCtx layout, and
`ZSTD_initStaticDCtx` implementation. Recording-callback tests cover heap
rejection, initializer failure, and successful initialization-to-dispatch
ordering without fabricating the private context layout.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
compression seam is integrated.
The decompression callback in fileio used to classify result statuses and
select its display action with a C switch. That left scalar result policy in
the C frontend even though Rust already owns the decompression dispatch and
result classification, and it coupled the C callback to a diagnostic enum.
Rust now exposes an ABI-checked status-action classifier. It keeps the
original action ordering and preserves silent handling for statuses that have
no display diagnostic, while the C callback retains the exact diagnostic
strings, source-name formatting, and display operation. Invalid inputs map to
a silent fallback action so the public callback remains behavior-compatible.
Focused Rust tests cover every status class, invalid values, and the exported
ABI result.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
compression and decompression seams are integrated.
FIO_openDstFile retained the filesystem leaf in Rust but its C wrapper still encoded the status precedence for test mode, stdout, same-file protection, sparse-mode adjustment, overwrite prompting, removal, retry, and final errors. Centralize that pure action classification in Rust without moving FILE* handling, metadata, preference mutation, prompts, diagnostics, or retry callbacks across the ABI.\n\nThe action bridge validates status, confirmation, and sparse-state inputs and has focused tests for every destination outcome, sparse-first ordering, prompt acceptance/abort, quiet mode, overwrite mode, setvbuf/open failures, success, and invalid inputs.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
ZSTDMT_updateCParams_whileCompressing already delegated parameter derivation to Rust, but C still owned the transition publication order and independently updated compressionLevel. Return a Rust-owned scalar result that carries the requested level with the derived compression parameters, while preserving the active frame window and all private MT context state in C.\n\nThe new projection and result have explicit C/Rust layout assertions. Focused Rust tests exercise unknown-size and explicit-size hints, LDM and override inputs, saved-window restoration, requested-level forwarding, and the exported ABI wrapper.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
ZSTD_compressStream2_c still owns private context mutation, diagnostics, buffer updates, and codec initialization, but its stable-input transparent-initialization decision was a remaining scalar policy island in C. Move the continuation validation, block-size threshold, format-specific progress hint, and wrapping size arithmetic behind a Rust projection. Preserve the original C error messages and return values while keeping all private context state on the C side.\n\nThe bridge uses explicit ABI layout assertions for the mixed pointer/size projection. Focused Rust tests cover buffered and non-continue calls, block-boundary initialization, empty and short stable input, both frame formats, matching continuation state, invalid source/position, and null bridge input.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings\n- Full capped native and original-test verification follows after the batch is committed.
The MT stream initializer and its later dictionary update both encoded the
same three-way choice in C: copy a supplied dictionary, attach a borrowed
CDict, or install a raw prefix. The old code also embedded the required
release/clear-before-attach ordering in each callback wrapper. Project the
presence and raw-content flags into Rust, where the branch and ordering are
now explicit and tested. C retains only the private CDict allocation,
prefix-storage, context publication, and destruction callbacks, so the
configured C layouts and allocator behavior remain unchanged.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `git diff --check` and `rustfmt --edition 2021 --check` -- passed before commit
- Capped native and original-test verification remains pending for the complete batch.
The file-I/O wrapper already delegated dictionary loading to Rust but kept
all status interpretation in C. That duplicated the malloc and mmap status
families and made the platform-specific error branches part of the C policy
surface. Add a Rust classifier that maps the shared numeric loader statuses
to diagnostic actions, including the distinct mmap failure classes. Keep
metadata lookup, platform handles, ownership, and the exact EXM_THROW text
in C, where the configured platform APIs still belong. The classifier also
rejects out-of-range type/status values; the valid malloc and mmap enums
intentionally share numeric values and therefore cannot be distinguished
beyond their family tag.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed (192 tests)
- Broader native and original-test verification remains pending for the complete batch.
ZSTD_freeCCtx previously passed one C callback that bundled dictionary,
multithreaded context, workspace, and heap-object teardown. That left the
lifetime order inside private C control flow and made the public destruction
policy harder to audit. Project the four private operations separately and
let Rust enforce the original dictionary, MT-context, workspace, and
object order while preserving the static-context and workspace-embedded
object guards. The callbacks still own the C layouts and custom allocator
operations.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- Broader native and original-test verification remains pending for the complete batch.
Project the scalar ZSTD_compressStream2 end directive into Rust and keep C responsible for the private streaming context, existing parameter error, and diagnostic text. The Rust range policy preserves the original unsigned enum validation for continue, flush, and end.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Move LDM block-compressor strategy validation, selection ordering, and callback error propagation into Rust while retaining dictionary-mode lookup, MatchState mutation, and the actual codec callback in C. Invalid preflight inputs now use the existing generic error path, and valid strategies preserve the original optimal-parser boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Extract the mixed-format decompression finish status mapping into a pure Rust policy helper and keep the existing callback side effects, diagnostics, and return values in the ABI wrapper. The explicit enum makes the success, pass-through, report, and impossible-status branches auditable and testable without widening the private fileio boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Make Rust the owner of the compression-job preparation sequence: publish the descriptor, reset the reusable input state, and publish terminal frame state last. Keep all MT job and stream layouts in C behind three focused callbacks, and preserve the nonterminal and terminal branches exactly.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
Keep the private static-CDict initializer in C, but return its native size_t status through the callback boundary. Rust now owns the public success-or-NULL mapping, preserving the original workspace validation, callback order, and private layout ownership while making initializer failures explicit and unit-testable.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
Move the aggregate compression-result classification out of the C wrapper and into the Rust fileio module. Keep C responsible for the user-facing EXM_THROW diagnostics and the existing fallback assertion, so optional-format build guards and command-line behavior remain unchanged. The Rust classifier also makes unexpected statuses explicit without widening the format callback boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
The directory-source migration initially skipped its C probe for stdin because
stdin can never be a directory from the CLI's normal marker path. The original
C source-open callback nevertheless called `UTIL_isDirectory` for every source,
including the stdin marker, and file-stat trace tests rely on those calls being
observable.
Always invoke the private C directory probe before source opening and keep the
same positive-result rejection. This preserves the trace side effect for stdin
while retaining Rust ownership of the rejection decision and the old source
open ordering.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The capped upstream run exposed two stdin file-stat trace mismatches; the suite will be rerun after this compatibility fix.
The resource-acquisition projection calls its field `ldmEnabled`, but the
first implementation passed the three-state `ZSTD_ParamSwitch_e` value
unchanged. `ZSTD_ps_disable` is 2, so Rust interpreted disabled LDM as enabled
and rejected ordinary jobs whose raw sequence store was intentionally absent.
That surfaced as an allocation error on the small stdin compression path.
Convert the C enum to the same boolean semantic used by the Rust policy before
crossing the ABI boundary. The projection declaration also now precedes the
callback setup statements, keeping the native C90 warning-free style.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The first capped full upstream run failed at `test-zstd` on the small stdin path with the enum/boolean regression; the suite will be rerun after this fix.
MT worker execution used to combine CCtx acquisition, raw-sequence-store
acquisition, destination-buffer allocation, frame-header publication, and the
LDM resource check in one C callback. That made the resource order and
short-circuit policy another C-owned implementation boundary even though the
operations themselves must remain private to C.
Project the scalar LDM and destination-presence inputs into Rust and let Rust
own the ordering and error normalization. The C callbacks now only acquire
private resources, report readiness, allocate the destination buffer, and
publish the frame-header destination. The ordering remains compatible with the
worker cleanup path: both pool gets happen first, CCtx failure stops before
destination work, destination failure stops before publication, and the LDM
sequence-store check runs after publication.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- Capped GCC and Clang syntax-only checks for `lib/compress/zstdmt_compress.c` -- passed by the worker
- `git diff --cached --check` -- passed
- Native build and full upstream tests are deferred to the post-batch serial verification.
The decompression source-open callback previously both tested for a named
directory and returned the generic open failure. That left a source-resource
rejection in the C orchestration path even though Rust already owns the
per-file ordering.
Add a C-owned directory probe to the projection. Rust invokes it before source
opening for named inputs and rejects a positive result; the C callback retains
the private filesystem helper and exact directory diagnostic. Stdin bypasses
the probe as before, and optional codec callbacks plus the remaining source
open, asynchronous, and cleanup ordering are unchanged.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/fileio_asyncio.rs (passed)
- capped GCC syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- capped Clang syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- git diff --cached --check (passed)
- workspace cargo +nightly fmt --check was not clean because of an unrelated
rust/src/zstdmt_compress.rs formatting diff; that file was left untouched
- no cargo build/test, make, fuzzers, or upstream tests were run
- commit signing was disabled because the configured GPG prompt hung
CDict initialization already lets Rust choose by-reference versus by-copy
content, but the C bridge still selected the pointer-rounded content allocation
and the HUF entropy-workspace size. That kept advanced and static CDict
dictionary loading dependent on C policy even though the allocator itself can
remain private.
Compute the content reservation with the C-equivalent wrapping pointer
alignment in Rust and pass both that size and the HUF workspace size through the
existing callbacks. C now only forwards the sizes to
ZSTD_cwksp_reserve_object; private CDict/workspace layouts and allocator
behavior remain behind the ABI bridge. Focused probes cover alignment
boundaries, by-copy content, and both reservation sizes.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo builds/tests, make, fuzzers, and large upstream tests were not run per task constraints
- GPG signing was unavailable because the configured pinentry could not start;
this commit was created explicitly unsigned with `--no-gpg-sign`
The new MT compression-job begin entry point already copied the nullable ABI projection with as_ref().copied(), but the integration call retained an unnecessary dereference. Pass the copied projection directly so the production entry point compiles under all-target clippy without changing its callback policy.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
The worker context previously selected the cdict or raw-prefix initialization
path, applied non-first-job parameter updates, and published the frame-header
projection directly in C. That left the branch and failure order intertwined
with private CCtx and parameter layouts.
Project only the first-job and cdict flags into Rust. Rust now validates cdict
placement, selects the initialization path, stops on force-window or
prefix-policy errors, and publishes the header projection only after successful
initialization. C callbacks retain the private cdict, CCtx, parameter mutation,
and frame-header field operations, including the original pledged-size and
force-window behavior.
Focused tests cover cdict ordering, non-first parameter ordering, parameter and
initialization failures, header-publication suppression, and invalid cdict
placement.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstdmt_compress.rs` -- passed.
- Capped GCC syntax-only check for `zstdmt_compress.c` -- passed.
- Capped Clang syntax-only check for `zstdmt_compress.c` -- passed.
- `git diff --cached --check` -- passed.
- Cargo, native builds, fuzzers, and large tests were not run per assignment.
The gzip codec loop already returns distinct status values, but the C
wrapper still owned the status branch that selected the CLI failure path.
That kept a format-result policy in the implementation-bearing C file even
though the loop itself is Rust. Add a Rust diagnostic mapping for success,
initialization, deflate, finish, end, invalid projection, and unknown
statuses. C now branches on the Rust classification while retaining zlib
result values, EXM_THROW codes, and exact diagnostic strings. This keeps the
ABI and observable behavior unchanged and leaves unrelated source-exclusion
and zstd classification seams untouched.
Test Plan:
- Nightly rustfmt check on fileio_asyncio.rs -- passed.
- Targeted git diff checks -- passed.
- Capped GCC syntax-only compile of fileio.c with codec defines -- passed.
- Focused Rust mapping tests were added; Cargo/tests were not run per the
task's OOM constraint.
CDict initialization already routes content copying and dictionary insertion
through the Rust orchestrator, but its private C adapter still hard-coded the
full table-load method and the CDict table-fill purpose. That split left an
important advanced-CDict content-loading policy hidden in the C bridge and
made the callback contract less explicit.
Extend the private insertion callback with the two table-loading policy
values. Rust now selects the full-load and for-CDict modes after its content
branch and before invoking the opaque C operation. C retains only the private
CDict layout projection and forwards those selected values to the existing
content loader, preserving the original insertion order and behavior.
The focused initialization probes record both values in the callback and
assert them for by-reference and by-copy dictionaries alongside the existing
ordering and content-copy checks.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, fuzzers, and large tests were not run per task constraints
Use usize::BITS for the static-CDict ABI offset that Clippy identifies as a bit-width expression. Replace the two MT flush cleanup test closures that independently borrow one vector with Rc<RefCell> probes, preserving the explicit wait-before-release assertions while making the test harness compile.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
ZSTD_initStaticCDict previously resolved automatic row matching and computed the
minimum workspace in C before handing the result to the Rust construction
orchestrator. That left the public static-CDict sizing branch split from the
Rust validation and callback failure policy.
Make the Rust projection carry dictionary size, load method, compression
parameters, and the C-owned layout sizing inputs. Rust now resolves automatic
row matching, computes the dedicated-search workspace requirement, validates
workspace capacity, and passes the resolved mode into the initializer while
preserving create/reserve/move/init order. C retains opaque private cwksp and
CDict operations, allocator/layout behavior, and dictionary-content loading.
Add focused tests for computed sizing policy, capacity boundaries, callback
order, reservation and initialization failures, and null or unaligned
workspaces.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- GCC `-fsyntax-only` checks with multithreaded, non-multithreaded,
`ZSTD_DISABLE_ASM=1`, and `ZSTD_ADDRESS_SANITIZER=1` configurations -- passed
- Clang `-fsyntax-only` check with `ZSTD_MULTITHREAD` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, and large tests were not run per task constraints
The MT flush state machine already recognized worker and checksum errors in
Rust, but its single C error callback still bundled waiting for all workers
with releasing job resources. That kept teardown order in C and made the two
cleanup phases impossible to test independently.
Split the C callback into private wait and release leaves, then pass both to
Rust. Rust now owns the error cleanup sequence and always waits for workers
before releasing their resources on either error path. The MT context, job
resources, synchronization, and cleanup operations remain C-owned.
Test Plan:
- `rustup run nightly rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs`
-- passed under the 40 GiB virtual-memory cap.
- `cc -fsyntax-only -std=c99` with the zstd include paths for
`lib/compress/zstdmt_compress.c` -- passed under the cap.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native builds, fuzzers, and large tests were not run per the
worker OOM and scope constraints.
The zstd compression stream already runs its scheduling and accounting loop in
Rust, but the C callback still owned the result-status switch that selected
success, codec failure, incomplete input, or invalid projection handling. Move
that status-to-diagnostic policy into the Rust ABI module, matching the
existing LZMA and LZ4 diagnostic seams. C keeps the zstd-specific error-name
lookup, display text, and exception construction, so private codec details and
CLI diagnostics remain on the C side and the observable error behavior is
unchanged. Unknown statuses retain the projection-error fallback.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Capped GCC syntax-only check of `programs/fileio.c` with all CLI format
feature defines -- passed.
- Cargo, make, native builds, and large tests were not run per worker OOM rules.
Use usize::BITS for the byte offset that Clippy recognizes as a bit-width expression on both the static-CCtx and MT finish projections. Update the focused resource-failure test to provide the compression-job last-job argument required by the callback signature. This keeps the ABI checks architecture-independent while making all-target clippy compile the new tests.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
Move static CCtx construction sequencing out of the monolithic C leaf. Rust
now validates the existing public workspace contract, sequences workspace
creation, CCtx and auxiliary reservations, publication, and BMI2 setup, and
short-circuits callback-reported failures. C callbacks retain the private
cwksp and CCtx layouts plus the native workspace-size and CPU-feature rules.
Add matching ABI layout assertions and focused callback-order, reservation-
failure, workspace-capacity, and NULL-path unit coverage. Static-CDict and
heap-CCtx paths remain unchanged.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- GCC and Clang `-fsyntax-only` checks on `zstd_compress.c` -- passed
- GCC syntax checks with `ZSTD_DISABLE_ASM=1` and
`ZSTD_ADDRESS_SANITIZER=1` -- passed
- `git diff --check` -- passed
- Cargo, make, native suites, and fuzzers not run per request
The MT worker previously reported errors through one C callback and left a
combined C finish callback responsible for serial completion, private resource
release, output-size publication, consumed-size publication, and signaling.
That kept the branch and lifecycle policy on the C side of the existing Rust
stage scheduler.
Add a narrow finish projection whose C callbacks expose only those private
leaves and the source-size scalar. Rust now publishes an error before serial
completion on every failed resource or codec stage, releases the sequence and
CCtx resources in the original order, publishes the final block size only on
success, then publishes the consumed size and signals the job condition. The
failed path therefore normalizes any codec-reported last-block size to zero
without changing the C-owned mutex, descriptor, pool, or context layouts.
Focused tests cover successful final-block publication, resource failure,
codec failure, cleanup ordering, consumed-size publication, and error-path
normalization.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check -- lib/compress/zstdmt_compress.c rust/src/zstdmt_compress.rs` and `git diff --cached --check` -- passed
- Rust unit tests were added but not run because the request prohibited Cargo and heavy commands
The Rust source-file scheduler already owns the ordering around source
exclusion, but its suffix policy and diagnostic still depended on a C table
and helper. That left the policy leaf on the C side of the boundary and
made the Rust scheduler call back into a C-owned decision.
Move the complete 113-entry, case-sensitive suffix policy into the Rust CLI
file-I/O layer and export the exclusion callback through the existing C ABI.
The callback preserves leading dots, the stdin and NULL non-match behavior,
the 0/1 return policy, and the exact display-level-4 diagnostic while leaving
C resource, compression, and asynchronous callbacks unchanged. Focused
checks cover representative suffixes, case sensitivity, stdin/NULL handling,
and the display gate.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common programs/fileio.c` -- passed
- Exact extracted C/Rust suffix-list comparison -- passed; all 113 entries match
- `git diff --check` and `git diff --cached --check` -- passed
- Rust unit tests and Cargo/Make/native suites were not run per request
- `rustfmt +nightly --check --edition 2021 rust/src/fileio_prefs.rs` -- not clean
because it reports pre-existing formatting drift in unchanged code
The callback unit test switched its fake codec result by mutating the state used
for the successful invocation. Clippy cannot see the later read through the
extern function pointer and reported the assignment as unused under
`-D warnings`.
Use a separate fake state for the error invocation. The two scenarios remain
independent and still exercise both the success and error result paths without
suppressing a lint or changing production code.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` under `ulimit -v 41943040` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
The MT job callback added state publication before its Rust job-construction
call, which left the result declaration after executable statements and emitted
the repository's C90 mixed-declaration warning on every CLI/test build.
Declare the result alongside the other callback-local projections and assign it
at the existing call site. This is a warning-only cleanup with no runtime or
ABI change.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The prior capped suite reached `rustLibSmoke`; a fresh root library build is
required because its archive predates the display-hook fix
The first MT state-publication fix made the C count current before job
construction, but the Rust result still unconditionally copied its local
pre-job fill count back into `mtctx` after the callback returned. Preparing a
job clears `mtctx->inBuff.buffer` and `mtctx->inBuff.filled`, so the next stream
pass then observed a consumed range as if it were still active and returned a
generic error in the multi-file CLI test.
Keep the C-cleared state whenever job preparation has detached the active
range. Rust's filled count is published only while the C range remains active,
which covers the blocked/table-full path where no job reset occurred. The
callback still receives the current count before preparation, and the C90
projection declaration is kept before executable statements.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The capped upstream suite reproduced a generic error in the multi-file CLI
path before this correction; the full rerun is pending
The Rust MT stream scheduler now copies input into the reusable round buffer
through its local projection and publishes the final filled count when the
outer call returns. Job creation, however, is a C callback invoked during that
same scheduler call. It still inspected `mtctx->inBuff.filled` while preparing
the job, so the count lagged behind the bytes Rust had just copied and the
existing preparation assertion fired in the multiple-file CLI path.
Publish the callback's `srcSize` into the C context before constructing the job
projection. This restores the ordering of the former C copy callback: the
context count is current before job preparation, while the existing end-of-call
result publication remains available for paths that do not create a job.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reproduced the pre-fix assertion in
`ZSTDMT_prepareCompressionJob`; the capped full-suite rerun is pending
The Rust zstd stream callback initially called a display helper defined only in
programs/fileio.c. The CLI linked successfully, but the upstream C tests also
link the shared Rust archive without the CLI translation unit, leaving that
symbol unresolved even though those tests do not use the fileio projection.
Keep the diagnostic implementation in C, but pass it as an optional final field
of the zstd compression projection. Rust invokes it from the stream loop after
a successful codec call, using the same directive, input position, input size,
and produced-output count as the former C callback. Test projections can leave
the hook empty, so the reusable Rust archive has no dependency on CLI-only
symbols while the production CLI preserves its level-6 diagnostic.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reached the suite but failed before this fix on the
now-removed unresolved `FIO_rust_zstd_compressStreamDisplay` reference
The fileio zstd projection used to route its stream callback through a C
implementation that constructed the public input and output buffer views,
queried pending output, called the streaming codec, and copied four scalar
results back to the Rust-owned loop. That left the central codec operation in
the CLI C translation unit even though the surrounding stream loop was already
in Rust.
Move that callback into Rust while keeping the C CCtx opaque across the
boundary. Rust now builds the public ZSTD_inBuffer and ZSTD_outBuffer views,
invokes ZSTD_toFlushNow and ZSTD_compressStream2, publishes the original
positions and result, and preserves the existing success diagnostic through a
small C display callback. The C projection passes the real CCtx as codecOpaque
and retains the adaptive iteration context and all private CLI state. The
buffer structs are made public within the Rust crate so this seam can reuse the
existing C-compatible definitions without duplicating them.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Full capped native and Rust verification remains to be run after this seam
The advanced-CDict path previously kept a C callback that rebuilt the
ZSTD_rustCDictSizing projection and delegated the actual arithmetic back to
Rust. Replace that callback with a pointer to the explicit sizing projection,
so Rust owns the workspace-size policy while C continues to own private
zstd_cwksp allocation, object reservation, initialization, and teardown. The
projection replaces the old callback slot, preserving every later callback
offset and the synchronous lifetime of the C sizing values.
Update both sides' layout assertions, remove the redundant C helper, and make
the lifecycle tests assert that Rust's computed size reaches allocation and
workspace creation before the existing C-owned callbacks run. Custom-memory
validation and all other CDict callbacks remain unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo build/test/clippy, Make, fuzzers, and other heavy verification were intentionally not run per request.
The decoder context remains allocated and laid out by C because optional members
change with the active build configuration. Previously, the C adapter also
owned the platform leaf: its DYNAMIC_BMI2 branch queried CPU features and
assigned the private bmi2 member directly. That kept the policy in C and made
the Rust boundary depend on the context layout.
Move the leaf to Rust and give it an opaque scalar address plus the active
configuration value. The C adapter now only extracts the optional address when
that member exists and forwards it, while DYNAMIC_BMI2=0 passes NULL. Rust
performs the dynamic decision, queries CPU support, and writes the scalar. The
existing 68-word DCtx view is unchanged. Focused unit coverage verifies that
disabled dynamic dispatch leaves the scalar untouched and enabled dispatch
publishes the CPU feature result.
Test Plan:
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=0 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- `git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs` -- passed
The MT streaming scheduler previously delegated each bounded input-buffer copy
through a C-only callback. That callback mutated the C context as a side
effect, which kept a trivial data movement operation outside the Rust stream
path and made the result projection incomplete.
Copy the checked input range directly in Rust with copy_nonoverlapping, after
the existing available-input, destination-capacity, and non-null-source checks.
The C input-range callback remains responsible for exposing the reusable buffer,
while the Rust result now returns its updated filled count. C publishes that
count back to mtctx after the scheduler call. The result ABI has explicit
cross-language layout assertions, and the scheduler test verifies both copied
bytes and fill accounting. Job scheduling and all other MT callbacks are
unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo build/test, make, fuzzers, and other heavy checks were not run per task instructions
Make ZSTDMT_createCompressionJob match the exact callback type consumed by
ZSTDMT_rust_compressStreamGeneric. The callback now performs the existing
opaque and end-directive casts itself, so the redundant C forwarding shim can
be removed without changing scheduling, assertions, logging, or ABI behavior.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- No cargo, make, native, fuzz, or other heavy tests run per task scope.
The separate-file decompression callback only reached the existing Rust
FIO_rust_determineDstName ABI through a two-argument C wrapper that supplied
the file-static suffix table and display string. Call the Rust ABI directly at
both decompression call sites, keeping the same suffix inputs and return-value
handling while removing the redundant wrapper and forward declaration.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native tests, fuzzers, and other heavy verification were not run
per task constraints.
Move the MT LDM overlap predicate behind the original
ZSTDMT_doesOverlapWindow() by-value ABI. Rust now owns the half-open range
calculation through a checked representation of Buffer and ZSTD_window_t;
C retains the synchronization and context orchestration around the callback.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
The new direct Rust file-I/O callback is unsafe because it receives a raw
NUL-terminated filename pointer. Add the required safety contract so CLI
clippy with `-D warnings` accepts the migrated ABI leaf.
Test Plan:
- `git diff --cached --check` -- passed.
- Capped CLI clippy rerun after this commit.
The new context-parameter projection test is compiled only in test builds,
but its constant was imported at module scope. Keep the import in the test
module so normal clippy builds remain warning-free under `-D warnings`.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
Import the no-attach-dictionary mode constant used by the new direct
ZSTD_getCParamsFromCCtxParams() projection test. The implementation was
correct, but the test module did not inherit that constant through the
existing parameter imports, so the root crate could not compile.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
The --list stdin callback was a stateless string predicate: it ignored its
opaque context and compared the input name with the fixed stdin marker. Move
that callback under its existing caller symbol into the Rust CLI archive, so
C no longer owns this policy leaf. The Rust implementation preserves the
callback ABI and returns the same 1/0 result for the marker and ordinary
names. File opening, frame parsing, diagnostics, human-readable formatting,
private file-info storage, and list orchestration remain C-owned. The optional
codec-version helpers remain untouched because their build-time library
configuration is not propagated to the Rust archive.
Test Plan:
- `clang -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder programs/fileio.c` -- passed.
- `rustfmt --edition 2021 --check --config skip_children=true rust/cli/src/lib.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, clippy, native builds, and upstream tests were not run per the explicit no-heavy-command constraint.
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.
Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
the requested lightweight-only scope.
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.
Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
The parameter policy leaves now export their C API names directly from Rust.
Two compatibility aliases were only used by the Rust unit test, while the
library and clippy builds correctly treated them as unused. Remove those dead
aliases and call the direct Rust-owned clamp symbol from the test so the
library, bench, and test targets share the same ABI surface without warning
under `-D warnings`.
Test Plan:
- Capped `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.
Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the Rust pass-through and frame-analysis leaves directly from their C
callbacks. The C translation unit still owns private resource projections,
filename suffix configuration, diagnostics, and format-specific operations;
this commit only removes two exact return-forwarding layers.
Test Plan:
- clang -fsyntax-only on programs/fileio.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
ddidderr