The multithreaded branch of `ZSTD_compressStream2_c` performed the complete
post-call result classification in C: it prioritized errors, recognized a
completed end directive, and applied different progress rules for continue,
flush, and end operations. Those decisions were scalar policy around a C-owned
MT call, but remained embedded beside private counters, reset, and tracing.
Rust now classifies one projected iteration and returns an explicit action for
continue, break, error, or completed end. The C shim still owns the MT call,
input/output accounting, error forwarding, private context mutation, reset,
and trace callback; the existing condition ordering and progress comparisons
are preserved. ABI layout assertions and focused tests cover error precedence,
completed end, progress-based continue termination, and pending/full output.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- Full capped Rust/native verification is the next serial step.
The stack-backed one-shot decompression entry point used to make its own
heap-mode rejection, static DCtx initialization, allocation-error mapping,
and dispatch ordering in C. That kept scalar control flow beside the private
DCtx layout even though Rust already owns the decompression policy and context
leaf.
Rust now consumes an ABI-checked scalar projection and callback set. It makes
the branch and ordering decisions, maps a failed static initializer to the
same memory-allocation error, and invokes the existing C-owned context leaf.
The C shim retains the local stack object, private DCtx layout, and
`ZSTD_initStaticDCtx` implementation. Recording-callback tests cover heap
rejection, initializer failure, and successful initialization-to-dispatch
ordering without fabricating the private context layout.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
compression seam is integrated.
The decompression callback in fileio used to classify result statuses and
select its display action with a C switch. That left scalar result policy in
the C frontend even though Rust already owns the decompression dispatch and
result classification, and it coupled the C callback to a diagnostic enum.
Rust now exposes an ABI-checked status-action classifier. It keeps the
original action ordering and preserves silent handling for statuses that have
no display diagnostic, while the C callback retains the exact diagnostic
strings, source-name formatting, and display operation. Invalid inputs map to
a silent fallback action so the public callback remains behavior-compatible.
Focused Rust tests cover every status class, invalid values, and the exported
ABI result.
Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
compression and decompression seams are integrated.
FIO_openDstFile retained the filesystem leaf in Rust but its C wrapper still encoded the status precedence for test mode, stdout, same-file protection, sparse-mode adjustment, overwrite prompting, removal, retry, and final errors. Centralize that pure action classification in Rust without moving FILE* handling, metadata, preference mutation, prompts, diagnostics, or retry callbacks across the ABI.\n\nThe action bridge validates status, confirmation, and sparse-state inputs and has focused tests for every destination outcome, sparse-first ordering, prompt acceptance/abort, quiet mode, overwrite mode, setvbuf/open failures, success, and invalid inputs.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
ZSTDMT_updateCParams_whileCompressing already delegated parameter derivation to Rust, but C still owned the transition publication order and independently updated compressionLevel. Return a Rust-owned scalar result that carries the requested level with the derived compression parameters, while preserving the active frame window and all private MT context state in C.\n\nThe new projection and result have explicit C/Rust layout assertions. Focused Rust tests exercise unknown-size and explicit-size hints, LDM and override inputs, saved-window restoration, requested-level forwarding, and the exported ABI wrapper.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
ZSTD_compressStream2_c still owns private context mutation, diagnostics, buffer updates, and codec initialization, but its stable-input transparent-initialization decision was a remaining scalar policy island in C. Move the continuation validation, block-size threshold, format-specific progress hint, and wrapping size arithmetic behind a Rust projection. Preserve the original C error messages and return values while keeping all private context state on the C side.\n\nThe bridge uses explicit ABI layout assertions for the mixed pointer/size projection. Focused Rust tests cover buffered and non-continue calls, block-boundary initialization, empty and short stable input, both frame formats, matching continuation state, invalid source/position, and null bridge input.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings\n- Full capped native and original-test verification follows after the batch is committed.
The MT stream initializer and its later dictionary update both encoded the
same three-way choice in C: copy a supplied dictionary, attach a borrowed
CDict, or install a raw prefix. The old code also embedded the required
release/clear-before-attach ordering in each callback wrapper. Project the
presence and raw-content flags into Rust, where the branch and ordering are
now explicit and tested. C retains only the private CDict allocation,
prefix-storage, context publication, and destruction callbacks, so the
configured C layouts and allocator behavior remain unchanged.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `git diff --check` and `rustfmt --edition 2021 --check` -- passed before commit
- Capped native and original-test verification remains pending for the complete batch.
The file-I/O wrapper already delegated dictionary loading to Rust but kept
all status interpretation in C. That duplicated the malloc and mmap status
families and made the platform-specific error branches part of the C policy
surface. Add a Rust classifier that maps the shared numeric loader statuses
to diagnostic actions, including the distinct mmap failure classes. Keep
metadata lookup, platform handles, ownership, and the exact EXM_THROW text
in C, where the configured platform APIs still belong. The classifier also
rejects out-of-range type/status values; the valid malloc and mmap enums
intentionally share numeric values and therefore cannot be distinguished
beyond their family tag.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed (192 tests)
- Broader native and original-test verification remains pending for the complete batch.
ZSTD_freeCCtx previously passed one C callback that bundled dictionary,
multithreaded context, workspace, and heap-object teardown. That left the
lifetime order inside private C control flow and made the public destruction
policy harder to audit. Project the four private operations separately and
let Rust enforce the original dictionary, MT-context, workspace, and
object order while preserving the static-context and workspace-embedded
object guards. The callbacks still own the C layouts and custom allocator
operations.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- Broader native and original-test verification remains pending for the complete batch.
Project the scalar ZSTD_compressStream2 end directive into Rust and keep C responsible for the private streaming context, existing parameter error, and diagnostic text. The Rust range policy preserves the original unsigned enum validation for continue, flush, and end.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Move LDM block-compressor strategy validation, selection ordering, and callback error propagation into Rust while retaining dictionary-mode lookup, MatchState mutation, and the actual codec callback in C. Invalid preflight inputs now use the existing generic error path, and valid strategies preserve the original optimal-parser boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Extract the mixed-format decompression finish status mapping into a pure Rust policy helper and keep the existing callback side effects, diagnostics, and return values in the ABI wrapper. The explicit enum makes the success, pass-through, report, and impossible-status branches auditable and testable without widening the private fileio boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
Make Rust the owner of the compression-job preparation sequence: publish the descriptor, reset the reusable input state, and publish terminal frame state last. Keep all MT job and stream layouts in C behind three focused callbacks, and preserve the nonterminal and terminal branches exactly.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
Keep the private static-CDict initializer in C, but return its native size_t status through the callback boundary. Rust now owns the public success-or-NULL mapping, preserving the original workspace validation, callback order, and private layout ownership while making initializer failures explicit and unit-testable.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
Move the aggregate compression-result classification out of the C wrapper and into the Rust fileio module. Keep C responsible for the user-facing EXM_THROW diagnostics and the existing fallback assertion, so optional-format build guards and command-line behavior remain unchanged. The Rust classifier also makes unexpected statuses explicit without widening the format callback boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
The directory-source migration initially skipped its C probe for stdin because
stdin can never be a directory from the CLI's normal marker path. The original
C source-open callback nevertheless called `UTIL_isDirectory` for every source,
including the stdin marker, and file-stat trace tests rely on those calls being
observable.
Always invoke the private C directory probe before source opening and keep the
same positive-result rejection. This preserves the trace side effect for stdin
while retaining Rust ownership of the rejection decision and the old source
open ordering.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The capped upstream run exposed two stdin file-stat trace mismatches; the suite will be rerun after this compatibility fix.
The resource-acquisition projection calls its field `ldmEnabled`, but the
first implementation passed the three-state `ZSTD_ParamSwitch_e` value
unchanged. `ZSTD_ps_disable` is 2, so Rust interpreted disabled LDM as enabled
and rejected ordinary jobs whose raw sequence store was intentionally absent.
That surfaced as an allocation error on the small stdin compression path.
Convert the C enum to the same boolean semantic used by the Rust policy before
crossing the ABI boundary. The projection declaration also now precedes the
callback setup statements, keeping the native C90 warning-free style.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The first capped full upstream run failed at `test-zstd` on the small stdin path with the enum/boolean regression; the suite will be rerun after this fix.
MT worker execution used to combine CCtx acquisition, raw-sequence-store
acquisition, destination-buffer allocation, frame-header publication, and the
LDM resource check in one C callback. That made the resource order and
short-circuit policy another C-owned implementation boundary even though the
operations themselves must remain private to C.
Project the scalar LDM and destination-presence inputs into Rust and let Rust
own the ordering and error normalization. The C callbacks now only acquire
private resources, report readiness, allocate the destination buffer, and
publish the frame-header destination. The ordering remains compatible with the
worker cleanup path: both pool gets happen first, CCtx failure stops before
destination work, destination failure stops before publication, and the LDM
sequence-store check runs after publication.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- Capped GCC and Clang syntax-only checks for `lib/compress/zstdmt_compress.c` -- passed by the worker
- `git diff --cached --check` -- passed
- Native build and full upstream tests are deferred to the post-batch serial verification.
The decompression source-open callback previously both tested for a named
directory and returned the generic open failure. That left a source-resource
rejection in the C orchestration path even though Rust already owns the
per-file ordering.
Add a C-owned directory probe to the projection. Rust invokes it before source
opening for named inputs and rejects a positive result; the C callback retains
the private filesystem helper and exact directory diagnostic. Stdin bypasses
the probe as before, and optional codec callbacks plus the remaining source
open, asynchronous, and cleanup ordering are unchanged.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/fileio_asyncio.rs (passed)
- capped GCC syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- capped Clang syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- git diff --cached --check (passed)
- workspace cargo +nightly fmt --check was not clean because of an unrelated
rust/src/zstdmt_compress.rs formatting diff; that file was left untouched
- no cargo build/test, make, fuzzers, or upstream tests were run
- commit signing was disabled because the configured GPG prompt hung
CDict initialization already lets Rust choose by-reference versus by-copy
content, but the C bridge still selected the pointer-rounded content allocation
and the HUF entropy-workspace size. That kept advanced and static CDict
dictionary loading dependent on C policy even though the allocator itself can
remain private.
Compute the content reservation with the C-equivalent wrapping pointer
alignment in Rust and pass both that size and the HUF workspace size through the
existing callbacks. C now only forwards the sizes to
ZSTD_cwksp_reserve_object; private CDict/workspace layouts and allocator
behavior remain behind the ABI bridge. Focused probes cover alignment
boundaries, by-copy content, and both reservation sizes.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo builds/tests, make, fuzzers, and large upstream tests were not run per task constraints
- GPG signing was unavailable because the configured pinentry could not start;
this commit was created explicitly unsigned with `--no-gpg-sign`
The new MT compression-job begin entry point already copied the nullable ABI projection with as_ref().copied(), but the integration call retained an unnecessary dereference. Pass the copied projection directly so the production entry point compiles under all-target clippy without changing its callback policy.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
The worker context previously selected the cdict or raw-prefix initialization
path, applied non-first-job parameter updates, and published the frame-header
projection directly in C. That left the branch and failure order intertwined
with private CCtx and parameter layouts.
Project only the first-job and cdict flags into Rust. Rust now validates cdict
placement, selects the initialization path, stops on force-window or
prefix-policy errors, and publishes the header projection only after successful
initialization. C callbacks retain the private cdict, CCtx, parameter mutation,
and frame-header field operations, including the original pledged-size and
force-window behavior.
Focused tests cover cdict ordering, non-first parameter ordering, parameter and
initialization failures, header-publication suppression, and invalid cdict
placement.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstdmt_compress.rs` -- passed.
- Capped GCC syntax-only check for `zstdmt_compress.c` -- passed.
- Capped Clang syntax-only check for `zstdmt_compress.c` -- passed.
- `git diff --cached --check` -- passed.
- Cargo, native builds, fuzzers, and large tests were not run per assignment.
The gzip codec loop already returns distinct status values, but the C
wrapper still owned the status branch that selected the CLI failure path.
That kept a format-result policy in the implementation-bearing C file even
though the loop itself is Rust. Add a Rust diagnostic mapping for success,
initialization, deflate, finish, end, invalid projection, and unknown
statuses. C now branches on the Rust classification while retaining zlib
result values, EXM_THROW codes, and exact diagnostic strings. This keeps the
ABI and observable behavior unchanged and leaves unrelated source-exclusion
and zstd classification seams untouched.
Test Plan:
- Nightly rustfmt check on fileio_asyncio.rs -- passed.
- Targeted git diff checks -- passed.
- Capped GCC syntax-only compile of fileio.c with codec defines -- passed.
- Focused Rust mapping tests were added; Cargo/tests were not run per the
task's OOM constraint.
CDict initialization already routes content copying and dictionary insertion
through the Rust orchestrator, but its private C adapter still hard-coded the
full table-load method and the CDict table-fill purpose. That split left an
important advanced-CDict content-loading policy hidden in the C bridge and
made the callback contract less explicit.
Extend the private insertion callback with the two table-loading policy
values. Rust now selects the full-load and for-CDict modes after its content
branch and before invoking the opaque C operation. C retains only the private
CDict layout projection and forwards those selected values to the existing
content loader, preserving the original insertion order and behavior.
The focused initialization probes record both values in the callback and
assert them for by-reference and by-copy dictionaries alongside the existing
ordering and content-copy checks.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, fuzzers, and large tests were not run per task constraints
Use usize::BITS for the static-CDict ABI offset that Clippy identifies as a bit-width expression. Replace the two MT flush cleanup test closures that independently borrow one vector with Rc<RefCell> probes, preserving the explicit wait-before-release assertions while making the test harness compile.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
ZSTD_initStaticCDict previously resolved automatic row matching and computed the
minimum workspace in C before handing the result to the Rust construction
orchestrator. That left the public static-CDict sizing branch split from the
Rust validation and callback failure policy.
Make the Rust projection carry dictionary size, load method, compression
parameters, and the C-owned layout sizing inputs. Rust now resolves automatic
row matching, computes the dedicated-search workspace requirement, validates
workspace capacity, and passes the resolved mode into the initializer while
preserving create/reserve/move/init order. C retains opaque private cwksp and
CDict operations, allocator/layout behavior, and dictionary-content loading.
Add focused tests for computed sizing policy, capacity boundaries, callback
order, reservation and initialization failures, and null or unaligned
workspaces.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- GCC `-fsyntax-only` checks with multithreaded, non-multithreaded,
`ZSTD_DISABLE_ASM=1`, and `ZSTD_ADDRESS_SANITIZER=1` configurations -- passed
- Clang `-fsyntax-only` check with `ZSTD_MULTITHREAD` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, and large tests were not run per task constraints
The MT flush state machine already recognized worker and checksum errors in
Rust, but its single C error callback still bundled waiting for all workers
with releasing job resources. That kept teardown order in C and made the two
cleanup phases impossible to test independently.
Split the C callback into private wait and release leaves, then pass both to
Rust. Rust now owns the error cleanup sequence and always waits for workers
before releasing their resources on either error path. The MT context, job
resources, synchronization, and cleanup operations remain C-owned.
Test Plan:
- `rustup run nightly rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs`
-- passed under the 40 GiB virtual-memory cap.
- `cc -fsyntax-only -std=c99` with the zstd include paths for
`lib/compress/zstdmt_compress.c` -- passed under the cap.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native builds, fuzzers, and large tests were not run per the
worker OOM and scope constraints.
The zstd compression stream already runs its scheduling and accounting loop in
Rust, but the C callback still owned the result-status switch that selected
success, codec failure, incomplete input, or invalid projection handling. Move
that status-to-diagnostic policy into the Rust ABI module, matching the
existing LZMA and LZ4 diagnostic seams. C keeps the zstd-specific error-name
lookup, display text, and exception construction, so private codec details and
CLI diagnostics remain on the C side and the observable error behavior is
unchanged. Unknown statuses retain the projection-error fallback.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Capped GCC syntax-only check of `programs/fileio.c` with all CLI format
feature defines -- passed.
- Cargo, make, native builds, and large tests were not run per worker OOM rules.
Use usize::BITS for the byte offset that Clippy recognizes as a bit-width expression on both the static-CCtx and MT finish projections. Update the focused resource-failure test to provide the compression-job last-job argument required by the callback signature. This keeps the ABI checks architecture-independent while making all-target clippy compile the new tests.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
Move static CCtx construction sequencing out of the monolithic C leaf. Rust
now validates the existing public workspace contract, sequences workspace
creation, CCtx and auxiliary reservations, publication, and BMI2 setup, and
short-circuits callback-reported failures. C callbacks retain the private
cwksp and CCtx layouts plus the native workspace-size and CPU-feature rules.
Add matching ABI layout assertions and focused callback-order, reservation-
failure, workspace-capacity, and NULL-path unit coverage. Static-CDict and
heap-CCtx paths remain unchanged.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- GCC and Clang `-fsyntax-only` checks on `zstd_compress.c` -- passed
- GCC syntax checks with `ZSTD_DISABLE_ASM=1` and
`ZSTD_ADDRESS_SANITIZER=1` -- passed
- `git diff --check` -- passed
- Cargo, make, native suites, and fuzzers not run per request
The MT worker previously reported errors through one C callback and left a
combined C finish callback responsible for serial completion, private resource
release, output-size publication, consumed-size publication, and signaling.
That kept the branch and lifecycle policy on the C side of the existing Rust
stage scheduler.
Add a narrow finish projection whose C callbacks expose only those private
leaves and the source-size scalar. Rust now publishes an error before serial
completion on every failed resource or codec stage, releases the sequence and
CCtx resources in the original order, publishes the final block size only on
success, then publishes the consumed size and signals the job condition. The
failed path therefore normalizes any codec-reported last-block size to zero
without changing the C-owned mutex, descriptor, pool, or context layouts.
Focused tests cover successful final-block publication, resource failure,
codec failure, cleanup ordering, consumed-size publication, and error-path
normalization.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check -- lib/compress/zstdmt_compress.c rust/src/zstdmt_compress.rs` and `git diff --cached --check` -- passed
- Rust unit tests were added but not run because the request prohibited Cargo and heavy commands
The Rust source-file scheduler already owns the ordering around source
exclusion, but its suffix policy and diagnostic still depended on a C table
and helper. That left the policy leaf on the C side of the boundary and
made the Rust scheduler call back into a C-owned decision.
Move the complete 113-entry, case-sensitive suffix policy into the Rust CLI
file-I/O layer and export the exclusion callback through the existing C ABI.
The callback preserves leading dots, the stdin and NULL non-match behavior,
the 0/1 return policy, and the exact display-level-4 diagnostic while leaving
C resource, compression, and asynchronous callbacks unchanged. Focused
checks cover representative suffixes, case sensitivity, stdin/NULL handling,
and the display gate.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common programs/fileio.c` -- passed
- Exact extracted C/Rust suffix-list comparison -- passed; all 113 entries match
- `git diff --check` and `git diff --cached --check` -- passed
- Rust unit tests and Cargo/Make/native suites were not run per request
- `rustfmt +nightly --check --edition 2021 rust/src/fileio_prefs.rs` -- not clean
because it reports pre-existing formatting drift in unchanged code
The callback unit test switched its fake codec result by mutating the state used
for the successful invocation. Clippy cannot see the later read through the
extern function pointer and reported the assignment as unused under
`-D warnings`.
Use a separate fake state for the error invocation. The two scenarios remain
independent and still exercise both the success and error result paths without
suppressing a lint or changing production code.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` under `ulimit -v 41943040` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
The MT job callback added state publication before its Rust job-construction
call, which left the result declaration after executable statements and emitted
the repository's C90 mixed-declaration warning on every CLI/test build.
Declare the result alongside the other callback-local projections and assign it
at the existing call site. This is a warning-only cleanup with no runtime or
ABI change.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The prior capped suite reached `rustLibSmoke`; a fresh root library build is
required because its archive predates the display-hook fix
The first MT state-publication fix made the C count current before job
construction, but the Rust result still unconditionally copied its local
pre-job fill count back into `mtctx` after the callback returned. Preparing a
job clears `mtctx->inBuff.buffer` and `mtctx->inBuff.filled`, so the next stream
pass then observed a consumed range as if it were still active and returned a
generic error in the multi-file CLI test.
Keep the C-cleared state whenever job preparation has detached the active
range. Rust's filled count is published only while the C range remains active,
which covers the blocked/table-full path where no job reset occurred. The
callback still receives the current count before preparation, and the C90
projection declaration is kept before executable statements.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The capped upstream suite reproduced a generic error in the multi-file CLI
path before this correction; the full rerun is pending
The Rust MT stream scheduler now copies input into the reusable round buffer
through its local projection and publishes the final filled count when the
outer call returns. Job creation, however, is a C callback invoked during that
same scheduler call. It still inspected `mtctx->inBuff.filled` while preparing
the job, so the count lagged behind the bytes Rust had just copied and the
existing preparation assertion fired in the multiple-file CLI path.
Publish the callback's `srcSize` into the C context before constructing the job
projection. This restores the ordering of the former C copy callback: the
context count is current before job preparation, while the existing end-of-call
result publication remains available for paths that do not create a job.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reproduced the pre-fix assertion in
`ZSTDMT_prepareCompressionJob`; the capped full-suite rerun is pending
The Rust zstd stream callback initially called a display helper defined only in
programs/fileio.c. The CLI linked successfully, but the upstream C tests also
link the shared Rust archive without the CLI translation unit, leaving that
symbol unresolved even though those tests do not use the fileio projection.
Keep the diagnostic implementation in C, but pass it as an optional final field
of the zstd compression projection. Rust invokes it from the stream loop after
a successful codec call, using the same directive, input position, input size,
and produced-output count as the former C callback. Test projections can leave
the hook empty, so the reusable Rust archive has no dependency on CLI-only
symbols while the production CLI preserves its level-6 diagnostic.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reached the suite but failed before this fix on the
now-removed unresolved `FIO_rust_zstd_compressStreamDisplay` reference
The fileio zstd projection used to route its stream callback through a C
implementation that constructed the public input and output buffer views,
queried pending output, called the streaming codec, and copied four scalar
results back to the Rust-owned loop. That left the central codec operation in
the CLI C translation unit even though the surrounding stream loop was already
in Rust.
Move that callback into Rust while keeping the C CCtx opaque across the
boundary. Rust now builds the public ZSTD_inBuffer and ZSTD_outBuffer views,
invokes ZSTD_toFlushNow and ZSTD_compressStream2, publishes the original
positions and result, and preserves the existing success diagnostic through a
small C display callback. The C projection passes the real CCtx as codecOpaque
and retains the adaptive iteration context and all private CLI state. The
buffer structs are made public within the Rust crate so this seam can reuse the
existing C-compatible definitions without duplicating them.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Full capped native and Rust verification remains to be run after this seam
The advanced-CDict path previously kept a C callback that rebuilt the
ZSTD_rustCDictSizing projection and delegated the actual arithmetic back to
Rust. Replace that callback with a pointer to the explicit sizing projection,
so Rust owns the workspace-size policy while C continues to own private
zstd_cwksp allocation, object reservation, initialization, and teardown. The
projection replaces the old callback slot, preserving every later callback
offset and the synchronous lifetime of the C sizing values.
Update both sides' layout assertions, remove the redundant C helper, and make
the lifecycle tests assert that Rust's computed size reaches allocation and
workspace creation before the existing C-owned callbacks run. Custom-memory
validation and all other CDict callbacks remain unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo build/test/clippy, Make, fuzzers, and other heavy verification were intentionally not run per request.
The decoder context remains allocated and laid out by C because optional members
change with the active build configuration. Previously, the C adapter also
owned the platform leaf: its DYNAMIC_BMI2 branch queried CPU features and
assigned the private bmi2 member directly. That kept the policy in C and made
the Rust boundary depend on the context layout.
Move the leaf to Rust and give it an opaque scalar address plus the active
configuration value. The C adapter now only extracts the optional address when
that member exists and forwards it, while DYNAMIC_BMI2=0 passes NULL. Rust
performs the dynamic decision, queries CPU support, and writes the scalar. The
existing 68-word DCtx view is unchanged. Focused unit coverage verifies that
disabled dynamic dispatch leaves the scalar untouched and enabled dispatch
publishes the CPU feature result.
Test Plan:
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=0 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- `git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs` -- passed
The MT streaming scheduler previously delegated each bounded input-buffer copy
through a C-only callback. That callback mutated the C context as a side
effect, which kept a trivial data movement operation outside the Rust stream
path and made the result projection incomplete.
Copy the checked input range directly in Rust with copy_nonoverlapping, after
the existing available-input, destination-capacity, and non-null-source checks.
The C input-range callback remains responsible for exposing the reusable buffer,
while the Rust result now returns its updated filled count. C publishes that
count back to mtctx after the scheduler call. The result ABI has explicit
cross-language layout assertions, and the scheduler test verifies both copied
bytes and fill accounting. Job scheduling and all other MT callbacks are
unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo build/test, make, fuzzers, and other heavy checks were not run per task instructions
Make ZSTDMT_createCompressionJob match the exact callback type consumed by
ZSTDMT_rust_compressStreamGeneric. The callback now performs the existing
opaque and end-directive casts itself, so the redundant C forwarding shim can
be removed without changing scheduling, assertions, logging, or ABI behavior.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- No cargo, make, native, fuzz, or other heavy tests run per task scope.
The separate-file decompression callback only reached the existing Rust
FIO_rust_determineDstName ABI through a two-argument C wrapper that supplied
the file-static suffix table and display string. Call the Rust ABI directly at
both decompression call sites, keeping the same suffix inputs and return-value
handling while removing the redundant wrapper and forward declaration.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native tests, fuzzers, and other heavy verification were not run
per task constraints.
Move the MT LDM overlap predicate behind the original
ZSTDMT_doesOverlapWindow() by-value ABI. Rust now owns the half-open range
calculation through a checked representation of Buffer and ZSTD_window_t;
C retains the synchronization and context orchestration around the callback.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
The new direct Rust file-I/O callback is unsafe because it receives a raw
NUL-terminated filename pointer. Add the required safety contract so CLI
clippy with `-D warnings` accepts the migrated ABI leaf.
Test Plan:
- `git diff --cached --check` -- passed.
- Capped CLI clippy rerun after this commit.
The new context-parameter projection test is compiled only in test builds,
but its constant was imported at module scope. Keep the import in the test
module so normal clippy builds remain warning-free under `-D warnings`.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
Import the no-attach-dictionary mode constant used by the new direct
ZSTD_getCParamsFromCCtxParams() projection test. The implementation was
correct, but the test module did not inherit that constant through the
existing parameter imports, so the root crate could not compile.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
The --list stdin callback was a stateless string predicate: it ignored its
opaque context and compared the input name with the fixed stdin marker. Move
that callback under its existing caller symbol into the Rust CLI archive, so
C no longer owns this policy leaf. The Rust implementation preserves the
callback ABI and returns the same 1/0 result for the marker and ordinary
names. File opening, frame parsing, diagnostics, human-readable formatting,
private file-info storage, and list orchestration remain C-owned. The optional
codec-version helpers remain untouched because their build-time library
configuration is not propagated to the Rust archive.
Test Plan:
- `clang -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder programs/fileio.c` -- passed.
- `rustfmt --edition 2021 --check --config skip_children=true rust/cli/src/lib.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, clippy, native builds, and upstream tests were not run per the explicit no-heavy-command constraint.
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.
Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
the requested lightweight-only scope.
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.
Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
The parameter policy leaves now export their C API names directly from Rust.
Two compatibility aliases were only used by the Rust unit test, while the
library and clippy builds correctly treated them as unused. Remove those dead
aliases and call the direct Rust-owned clamp symbol from the test so the
library, bench, and test targets share the same ABI surface without warning
under `-D warnings`.
Test Plan:
- Capped `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.
Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the Rust pass-through and frame-analysis leaves directly from their C
callbacks. The C translation unit still owns private resource projections,
filename suffix configuration, diagnostics, and format-specific operations;
this commit only removes two exact return-forwarding layers.
Test Plan:
- clang -fsyntax-only on programs/fileio.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.
Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.
Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
Keep the local buffer variables declared before the direct Rust buffer-policy
calls in the compression stream adapter, preserving the native C90 warning
profile after removing the old C forwarding helper.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed before this declaration-only cleanup; the next capped native gate will recheck it.
Rely on the existing fileio.h declaration now that the filename collision
checker is exported directly from Rust, avoiding a redundant C redeclaration
in the implementation file.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed; CLI lint and unit gates pending.
Move the pure filename-collision, input/output-buffer construction, and
compressed-destination-name helpers to direct Rust-owned caller symbols.
Leave C responsible for diagnostics, filesystem resources, suffix-list policy,
and the surrounding file-processing orchestration.
Test Plan: Pending capped full verification after this atomic ABI cleanup.
Remove the C forwarding wrappers around compression-parameter policy leaves
that are now implemented directly under their caller-facing ABI symbols.
Keep the private C parameter construction and resource logic in C, while Rust
continues to own the pure row-matchfinder, block-splitter, LDM, external
sequence/repcode, dictionary-tagging, block-size, and overflow predicates.
Test Plan: Pending capped full verification after the companion CLI leaf commit.
Export the file-summary predicates and largest-file-size scan directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving pointer contracts, assertions, return widths, and
filesystem behavior.
Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
Export ZSTD_useTargetCBlockSize and ZSTD_blockSplitterEnabled directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving diagnostics, ABI signatures, and parameter layout.
Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
Move reset, parameter copy, and default policy resolution from
ZSTD_makeCCtxParamsFromCParams into Rust through a pointer-based ABI function.
Keep the private LDM adjustment and its invariants in C while Rust owns the
stable parameter-object policy and focused reset/resolution coverage.
Test Plan:
- worker git diff and capped format checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
All parent verification is serial under a 40 GiB virtual-memory cap. Standalone
root Rust unit linking remains unavailable because the crate imports C-owned
bridge symbols.
Export FIO_LZ4_GetBlockSize_FromBlockId directly from Rust under the existing
caller symbol and remove the redundant C forwarding wrapper. Preserve the
block-ID formula and cover its boundary values in the focused Rust test.
Test Plan:
- worker capped nightly rustfmt check
- worker git diff --check
- full serial capped native and upstream suites to run at the next parent gate
Export FIO_highbit64 directly from Rust under the existing caller symbol and
remove the C forwarding wrapper. Preserve the nonzero-input contract and add
boundary tests for the scalar policy leaf.
Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests
All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
Move reset, copy, and default policy resolution from
ZSTD_CCtxParams_init_internal into Rust through the existing repr(C)
parameter mirror. C retains validation and diagnostics, while focused tests
cover reset behavior, policy resolution, and compression-level handling.
Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests
All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
Move the malloc-versus-mmap decision used by compression and decompression
resource construction into Rust. The C adapter continues to own platform
loaders, allocation handles, diagnostics, and dictionary I/O; Rust only
combines the explicit mmap preference, patch-mode size threshold, and explicit
disable override. Add enum-value ABI checks and focused policy coverage.
Also hoist the source-size declaration in the C adapter so the migration does
not introduce a C90 declaration-after-statement warning.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Move the strategy classification that decides whether LDM preparation fills
fast or double-fast tables into Rust. C retains the private match-state,
configuration-specific table-fill leaves, and the existing excluded-DFAST
assertion. Add enum-value ABI checks plus focused tests for valid and invalid
strategy classifications.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Move the fixed source-size tier traversal used by
ZSTD_estimateCCtxSize_internal() into Rust. C retains the authoritative
parameter lookup and per-tier workspace estimator behind callbacks, while Rust
owns the tier ordering and maximum-selection policy. The projection includes
ABI layout assertions and tests for callback order, level propagation, and
error-valued maxima.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
Keep XXH64 state ownership and checksum calculation in the C MT scheduler,
but project the final four-byte frame write through Rust. Rust now owns the
little-endian serialization and destination-capacity policy, with ABI layout
assertions and flush-state error propagation preserving the existing cleanup
path. Focused tests cover byte order, exact capacity, invalid destinations,
and callback error handling.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran under a serial 40 GiB virtual-memory limit. The
full upstream suite passed, including large-file, multithreaded, API, fuzzer,
and zstream coverage. Standalone root Rust unit linking remains unavailable
because the crate imports C-owned bridge symbols without a Cargo build/link
setup.
Call the authoritative DDict constructor directly from the Rust decompression
policy after converting the shared repr(C) allocator projection. Remove the
C forwarding declaration and wrapper so dictionary creation no longer crosses
an unnecessary Rust-to-C-to-C boundary, while compile-time size and alignment
checks keep the allocator conversion tied to the ABI contract.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`
The integrated checks ran at the combined working-tree tip, including the
parallel MT checksum seam that remains as the next commit. Standalone root
Rust unit linking remains unavailable because the crate imports C-owned bridge
symbols without a Cargo build/link setup.
Keep the heapmode guard, stack-owned ZSTD_DCtx, static initialization, and
private context layout in C. Once the context is initialized, Rust now owns the
one-shot stack-context policy: it clears the projected static-size marker and
dispatches the existing ZSTD_decompressDCtx implementation through the opaque
context pointer.
Add matching C/Rust view-layout assertions and focused callback tests covering
static-size clearing, argument forwarding, and null-context rejection.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; the integrated build and upstream C/Rust oracle supplied the runtime/link coverage.
Project the zstd window-too-large diagnostic inputs into Rust so Rust owns
error filtering and the concrete-versus-unsupported guidance choice. C keeps
frame-header parsing, window-log extraction, and the exact display text behind
a callback, preserving the existing diagnostics and fallback behavior.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1 (clean after C90 declaration cleanup)
- ulimit -v 41943040; make -j1 -C tests test (passed; the later cleanup only moved a declaration before statements)
Project the scalar single-thread hint inputs and keep the multithreaded
context behind a C callback so Rust owns the worker-count dispatch. Preserve
the existing buffer-mode assertion and both hint arithmetic leaves, with
focused tests covering MT selection and single-thread fallback.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Project the decompression-resource cleanup callbacks into Rust so the stable
teardown order remains dictionary, dstream, write pool, and read pool. C
retains ownership of dRess_t and each private resource implementation, with
its existing dstream CHECK behavior preserved. Focused tests cover order and
null or incomplete callback states.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Project the private stream state needed by the complete-input streaming fast
path into a stable C/Rust layout and let Rust own the eligibility predicate.
The C bridge keeps the CCtx private and retains the existing simple-level
selection leaf as a callback, while focused tests cover the accepted and
rejected stream states.
All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.
Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
Move the scalar policy that selects the optimal parser for long-distance
matching into Rust. The C implementation continues to own match-state and
compressor dispatch, but the strategy threshold is now defined and tested at
the Rust seam.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
ddidderr