Move the aggregate compression-result classification out of the C wrapper and into the Rust fileio module. Keep C responsible for the user-facing EXM_THROW diagnostics and the existing fallback assertion, so optional-format build guards and command-line behavior remain unchanged. The Rust classifier also makes unexpected statuses explicit without widening the format callback boundary.
Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
The directory-source migration initially skipped its C probe for stdin because
stdin can never be a directory from the CLI's normal marker path. The original
C source-open callback nevertheless called `UTIL_isDirectory` for every source,
including the stdin marker, and file-stat trace tests rely on those calls being
observable.
Always invoke the private C directory probe before source opening and keep the
same positive-result rejection. This preserves the trace side effect for stdin
while retaining Rust ownership of the rejection decision and the old source
open ordering.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The capped upstream run exposed two stdin file-stat trace mismatches; the suite will be rerun after this compatibility fix.
The resource-acquisition projection calls its field `ldmEnabled`, but the
first implementation passed the three-state `ZSTD_ParamSwitch_e` value
unchanged. `ZSTD_ps_disable` is 2, so Rust interpreted disabled LDM as enabled
and rejected ordinary jobs whose raw sequence store was intentionally absent.
That surfaced as an allocation error on the small stdin compression path.
Convert the C enum to the same boolean semantic used by the Rust policy before
crossing the ABI boundary. The projection declaration also now precedes the
callback setup statements, keeping the native C90 warning-free style.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The first capped full upstream run failed at `test-zstd` on the small stdin path with the enum/boolean regression; the suite will be rerun after this fix.
MT worker execution used to combine CCtx acquisition, raw-sequence-store
acquisition, destination-buffer allocation, frame-header publication, and the
LDM resource check in one C callback. That made the resource order and
short-circuit policy another C-owned implementation boundary even though the
operations themselves must remain private to C.
Project the scalar LDM and destination-presence inputs into Rust and let Rust
own the ordering and error normalization. The C callbacks now only acquire
private resources, report readiness, allocate the destination buffer, and
publish the frame-header destination. The ordering remains compatible with the
worker cleanup path: both pool gets happen first, CCtx failure stops before
destination work, destination failure stops before publication, and the LDM
sequence-store check runs after publication.
Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- Capped GCC and Clang syntax-only checks for `lib/compress/zstdmt_compress.c` -- passed by the worker
- `git diff --cached --check` -- passed
- Native build and full upstream tests are deferred to the post-batch serial verification.
The decompression source-open callback previously both tested for a named
directory and returned the generic open failure. That left a source-resource
rejection in the C orchestration path even though Rust already owns the
per-file ordering.
Add a C-owned directory probe to the projection. Rust invokes it before source
opening for named inputs and rejects a positive result; the C callback retains
the private filesystem helper and exact directory diagnostic. Stdin bypasses
the probe as before, and optional codec callbacks plus the remaining source
open, asynchronous, and cleanup ordering are unchanged.
Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/fileio_asyncio.rs (passed)
- capped GCC syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- capped Clang syntax-only check of programs/fileio.c with all optional-format
macros enabled (passed)
- git diff --cached --check (passed)
- workspace cargo +nightly fmt --check was not clean because of an unrelated
rust/src/zstdmt_compress.rs formatting diff; that file was left untouched
- no cargo build/test, make, fuzzers, or upstream tests were run
- commit signing was disabled because the configured GPG prompt hung
CDict initialization already lets Rust choose by-reference versus by-copy
content, but the C bridge still selected the pointer-rounded content allocation
and the HUF entropy-workspace size. That kept advanced and static CDict
dictionary loading dependent on C policy even though the allocator itself can
remain private.
Compute the content reservation with the C-equivalent wrapping pointer
alignment in Rust and pass both that size and the HUF workspace size through the
existing callbacks. C now only forwards the sizes to
ZSTD_cwksp_reserve_object; private CDict/workspace layouts and allocator
behavior remain behind the ABI bridge. Focused probes cover alignment
boundaries, by-copy content, and both reservation sizes.
Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo builds/tests, make, fuzzers, and large upstream tests were not run per task constraints
- GPG signing was unavailable because the configured pinentry could not start;
this commit was created explicitly unsigned with `--no-gpg-sign`
The new MT compression-job begin entry point already copied the nullable ABI projection with as_ref().copied(), but the integration call retained an unnecessary dereference. Pass the copied projection directly so the production entry point compiles under all-target clippy without changing its callback policy.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
The worker context previously selected the cdict or raw-prefix initialization
path, applied non-first-job parameter updates, and published the frame-header
projection directly in C. That left the branch and failure order intertwined
with private CCtx and parameter layouts.
Project only the first-job and cdict flags into Rust. Rust now validates cdict
placement, selects the initialization path, stops on force-window or
prefix-policy errors, and publishes the header projection only after successful
initialization. C callbacks retain the private cdict, CCtx, parameter mutation,
and frame-header field operations, including the original pledged-size and
force-window behavior.
Focused tests cover cdict ordering, non-first parameter ordering, parameter and
initialization failures, header-publication suppression, and invalid cdict
placement.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstdmt_compress.rs` -- passed.
- Capped GCC syntax-only check for `zstdmt_compress.c` -- passed.
- Capped Clang syntax-only check for `zstdmt_compress.c` -- passed.
- `git diff --cached --check` -- passed.
- Cargo, native builds, fuzzers, and large tests were not run per assignment.
The gzip codec loop already returns distinct status values, but the C
wrapper still owned the status branch that selected the CLI failure path.
That kept a format-result policy in the implementation-bearing C file even
though the loop itself is Rust. Add a Rust diagnostic mapping for success,
initialization, deflate, finish, end, invalid projection, and unknown
statuses. C now branches on the Rust classification while retaining zlib
result values, EXM_THROW codes, and exact diagnostic strings. This keeps the
ABI and observable behavior unchanged and leaves unrelated source-exclusion
and zstd classification seams untouched.
Test Plan:
- Nightly rustfmt check on fileio_asyncio.rs -- passed.
- Targeted git diff checks -- passed.
- Capped GCC syntax-only compile of fileio.c with codec defines -- passed.
- Focused Rust mapping tests were added; Cargo/tests were not run per the
task's OOM constraint.
CDict initialization already routes content copying and dictionary insertion
through the Rust orchestrator, but its private C adapter still hard-coded the
full table-load method and the CDict table-fill purpose. That split left an
important advanced-CDict content-loading policy hidden in the C bridge and
made the callback contract less explicit.
Extend the private insertion callback with the two table-loading policy
values. Rust now selects the full-load and for-CDict modes after its content
branch and before invoking the opaque C operation. C retains only the private
CDict layout projection and forwards those selected values to the existing
content loader, preserving the original insertion order and behavior.
The focused initialization probes record both values in the callback and
assert them for by-reference and by-copy dictionaries alongside the existing
ordering and content-copy checks.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- `ulimit -v 41943040; gcc -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; clang -fsyntax-only -std=c99 -DXXH_NAMESPACE=ZSTD_ -DDEBUGLEVEL=0 -DZSTD_MULTITHREAD -DZSTD_LEGACY_SUPPORT=5 -DZSTD_NO_ASM=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/deprecated lib/compress/zstd_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, fuzzers, and large tests were not run per task constraints
Use usize::BITS for the static-CDict ABI offset that Clippy identifies as a bit-width expression. Replace the two MT flush cleanup test closures that independently borrow one vector with Rc<RefCell> probes, preserving the explicit wait-before-release assertions while making the test harness compile.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
ZSTD_initStaticCDict previously resolved automatic row matching and computed the
minimum workspace in C before handing the result to the Rust construction
orchestrator. That left the public static-CDict sizing branch split from the
Rust validation and callback failure policy.
Make the Rust projection carry dictionary size, load method, compression
parameters, and the C-owned layout sizing inputs. Rust now resolves automatic
row matching, computes the dedicated-search workspace requirement, validates
workspace capacity, and passes the resolved mode into the initializer while
preserving create/reserve/move/init order. C retains opaque private cwksp and
CDict operations, allocator/layout behavior, and dictionary-content loading.
Add focused tests for computed sizing policy, capacity boundaries, callback
order, reservation and initialization failures, and null or unaligned
workspaces.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed
- GCC `-fsyntax-only` checks with multithreaded, non-multithreaded,
`ZSTD_DISABLE_ASM=1`, and `ZSTD_ADDRESS_SANITIZER=1` configurations -- passed
- Clang `-fsyntax-only` check with `ZSTD_MULTITHREAD` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo, make, native builds, and large tests were not run per task constraints
The MT flush state machine already recognized worker and checksum errors in
Rust, but its single C error callback still bundled waiting for all workers
with releasing job resources. That kept teardown order in C and made the two
cleanup phases impossible to test independently.
Split the C callback into private wait and release leaves, then pass both to
Rust. Rust now owns the error cleanup sequence and always waits for workers
before releasing their resources on either error path. The MT context, job
resources, synchronization, and cleanup operations remain C-owned.
Test Plan:
- `rustup run nightly rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs`
-- passed under the 40 GiB virtual-memory cap.
- `cc -fsyntax-only -std=c99` with the zstd include paths for
`lib/compress/zstdmt_compress.c` -- passed under the cap.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native builds, fuzzers, and large tests were not run per the
worker OOM and scope constraints.
The zstd compression stream already runs its scheduling and accounting loop in
Rust, but the C callback still owned the result-status switch that selected
success, codec failure, incomplete input, or invalid projection handling. Move
that status-to-diagnostic policy into the Rust ABI module, matching the
existing LZMA and LZ4 diagnostic seams. C keeps the zstd-specific error-name
lookup, display text, and exception construction, so private codec details and
CLI diagnostics remain on the C side and the observable error behavior is
unchanged. Unknown statuses retain the projection-error fallback.
Test Plan:
- `rustfmt +nightly --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Capped GCC syntax-only check of `programs/fileio.c` with all CLI format
feature defines -- passed.
- Cargo, make, native builds, and large tests were not run per worker OOM rules.
Use usize::BITS for the byte offset that Clippy recognizes as a bit-width expression on both the static-CCtx and MT finish projections. Update the focused resource-failure test to provide the compression-job last-job argument required by the callback signature. This keeps the ABI checks architecture-independent while making all-target clippy compile the new tests.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --cached --check
Move static CCtx construction sequencing out of the monolithic C leaf. Rust
now validates the existing public workspace contract, sequences workspace
creation, CCtx and auxiliary reservations, publication, and BMI2 setup, and
short-circuits callback-reported failures. C callbacks retain the private
cwksp and CCtx layouts plus the native workspace-size and CPU-feature rules.
Add matching ABI layout assertions and focused callback-order, reservation-
failure, workspace-capacity, and NULL-path unit coverage. Static-CDict and
heap-CCtx paths remain unchanged.
Test Plan:
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- GCC and Clang `-fsyntax-only` checks on `zstd_compress.c` -- passed
- GCC syntax checks with `ZSTD_DISABLE_ASM=1` and
`ZSTD_ADDRESS_SANITIZER=1` -- passed
- `git diff --check` -- passed
- Cargo, make, native suites, and fuzzers not run per request
The MT worker previously reported errors through one C callback and left a
combined C finish callback responsible for serial completion, private resource
release, output-size publication, consumed-size publication, and signaling.
That kept the branch and lifecycle policy on the C side of the existing Rust
stage scheduler.
Add a narrow finish projection whose C callbacks expose only those private
leaves and the source-size scalar. Rust now publishes an error before serial
completion on every failed resource or codec stage, releases the sequence and
CCtx resources in the original order, publishes the final block size only on
success, then publishes the consumed size and signals the job condition. The
failed path therefore normalizes any codec-reported last-block size to zero
without changing the C-owned mutex, descriptor, pool, or context layouts.
Focused tests cover successful final-block publication, resource failure,
codec failure, cleanup ordering, consumed-size publication, and error-path
normalization.
Test Plan:
- `rustfmt --check --edition 2021 rust/src/zstdmt_compress.rs` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check -- lib/compress/zstdmt_compress.c rust/src/zstdmt_compress.rs` and `git diff --cached --check` -- passed
- Rust unit tests were added but not run because the request prohibited Cargo and heavy commands
The Rust source-file scheduler already owns the ordering around source
exclusion, but its suffix policy and diagnostic still depended on a C table
and helper. That left the policy leaf on the C side of the boundary and
made the Rust scheduler call back into a C-owned decision.
Move the complete 113-entry, case-sensitive suffix policy into the Rust CLI
file-I/O layer and export the exclusion callback through the existing C ABI.
The callback preserves leading dots, the stdin and NULL non-match behavior,
the 0/1 return policy, and the exact display-level-4 diagnostic while leaving
C resource, compression, and asynchronous callbacks unchanged. Focused
checks cover representative suffixes, case sensitivity, stdin/NULL handling,
and the display gate.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common programs/fileio.c` -- passed
- Exact extracted C/Rust suffix-list comparison -- passed; all 113 entries match
- `git diff --check` and `git diff --cached --check` -- passed
- Rust unit tests and Cargo/Make/native suites were not run per request
- `rustfmt +nightly --check --edition 2021 rust/src/fileio_prefs.rs` -- not clean
because it reports pre-existing formatting drift in unchanged code
The callback unit test switched its fake codec result by mutating the state used
for the successful invocation. Clippy cannot see the later read through the
extern function pointer and reported the assignment as unused under
`-D warnings`.
Use a separate fake state for the error invocation. The two scenarios remain
independent and still exercise both the success and error result paths without
suppressing a lint or changing production code.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` under `ulimit -v 41943040` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
The MT job callback added state publication before its Rust job-construction
call, which left the result declaration after executable statements and emitted
the repository's C90 mixed-declaration warning on every CLI/test build.
Declare the result alongside the other callback-local projections and assign it
at the existing call site. This is a warning-only cleanup with no runtime or
ABI change.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The prior capped suite reached `rustLibSmoke`; a fresh root library build is
required because its archive predates the display-hook fix
The first MT state-publication fix made the C count current before job
construction, but the Rust result still unconditionally copied its local
pre-job fill count back into `mtctx` after the callback returned. Preparing a
job clears `mtctx->inBuff.buffer` and `mtctx->inBuff.filled`, so the next stream
pass then observed a consumed range as if it were still active and returned a
generic error in the multi-file CLI test.
Keep the C-cleared state whenever job preparation has detached the active
range. Rust's filled count is published only while the C range remains active,
which covers the blocked/table-full path where no job reset occurred. The
callback still receives the current count before preparation, and the C90
projection declaration is kept before executable statements.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- The capped upstream suite reproduced a generic error in the multi-file CLI
path before this correction; the full rerun is pending
The Rust MT stream scheduler now copies input into the reusable round buffer
through its local projection and publishes the final filled count when the
outer call returns. Job creation, however, is a C callback invoked during that
same scheduler call. It still inspected `mtctx->inBuff.filled` while preparing
the job, so the count lagged behind the bytes Rust had just copied and the
existing preparation assertion fired in the multiple-file CLI path.
Publish the callback's `srcSize` into the C context before constructing the job
projection. This restores the ordering of the former C copy callback: the
context count is current before job preparation, while the existing end-of-call
result publication remains available for paths that do not create a job.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reproduced the pre-fix assertion in
`ZSTDMT_prepareCompressionJob`; the capped full-suite rerun is pending
The Rust zstd stream callback initially called a display helper defined only in
programs/fileio.c. The CLI linked successfully, but the upstream C tests also
link the shared Rust archive without the CLI translation unit, leaving that
symbol unresolved even though those tests do not use the fileio projection.
Keep the diagnostic implementation in C, but pass it as an optional final field
of the zstd compression projection. Rust invokes it from the stream loop after
a successful codec call, using the same directive, input position, input size,
and produced-output count as the former C callback. Test projections can leave
the hook empty, so the reusable Rust archive has no dependency on CLI-only
symbols while the production CLI preserves its level-6 diagnostic.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- `make -j1 -C tests test` reached the suite but failed before this fix on the
now-removed unresolved `FIO_rust_zstd_compressStreamDisplay` reference
The fileio zstd projection used to route its stream callback through a C
implementation that constructed the public input and output buffer views,
queried pending output, called the streaming codec, and copied four scalar
results back to the Rust-owned loop. That left the central codec operation in
the CLI C translation unit even though the surrounding stream loop was already
in Rust.
Move that callback into Rust while keeping the C CCtx opaque across the
boundary. Rust now builds the public ZSTD_inBuffer and ZSTD_outBuffer views,
invokes ZSTD_toFlushNow and ZSTD_compressStream2, publishes the original
positions and result, and preserves the existing success diagnostic through a
small C display callback. The C projection passes the real CCtx as codecOpaque
and retains the adaptive iteration context and all private CLI state. The
buffer structs are made public within the Rust crate so this seam can reuse the
existing C-compatible definitions without duplicating them.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Full capped native and Rust verification remains to be run after this seam
The advanced-CDict path previously kept a C callback that rebuilt the
ZSTD_rustCDictSizing projection and delegated the actual arithmetic back to
Rust. Replace that callback with a pointer to the explicit sizing projection,
so Rust owns the workspace-size policy while C continues to own private
zstd_cwksp allocation, object reservation, initialization, and teardown. The
projection replaces the old callback slot, preserving every later callback
offset and the synchronous lifetime of the C sizing values.
Update both sides' layout assertions, remove the redundant C helper, and make
the lifecycle tests assert that Rust's computed size reaches allocation and
workspace creation before the existing C-owned callbacks run. Custom-memory
validation and all other CDict callbacks remain unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `rustfmt +nightly --edition 2021 --check rust/src/zstd_compress_dictionary.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo build/test/clippy, Make, fuzzers, and other heavy verification were intentionally not run per request.
The decoder context remains allocated and laid out by C because optional members
change with the active build configuration. Previously, the C adapter also
owned the platform leaf: its DYNAMIC_BMI2 branch queried CPU features and
assigned the private bmi2 member directly. That kept the policy in C and made
the Rust boundary depend on the context layout.
Move the leaf to Rust and give it an opaque scalar address plus the active
configuration value. The C adapter now only extracts the optional address when
that member exists and forwards it, while DYNAMIC_BMI2=0 passes NULL. Rust
performs the dynamic decision, queries CPU support, and writes the scalar. The
existing 68-word DCtx view is unchanged. Focused unit coverage verifies that
disabled dynamic dispatch leaves the scalar untouched and enabled dispatch
publishes the CPU feature result.
Test Plan:
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=0 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `cc -fsyntax-only -Werror -DDYNAMIC_BMI2=1 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/decompress/zstd_decompress.c` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- `git diff --check -- lib/decompress/zstd_decompress.c rust/src/zstd_decompress.rs` -- passed
The MT streaming scheduler previously delegated each bounded input-buffer copy
through a C-only callback. That callback mutated the C context as a side
effect, which kept a trivial data movement operation outside the Rust stream
path and made the result projection incomplete.
Copy the checked input range directly in Rust with copy_nonoverlapping, after
the existing available-input, destination-capacity, and non-null-source checks.
The C input-range callback remains responsible for exposing the reusable buffer,
while the Rust result now returns its updated filled count. C publishes that
count back to mtctx after the scheduler call. The result ABI has explicit
cross-language layout assertions, and the scheduler test verifies both copied
bytes and fill accounting. Job scheduling and all other MT callbacks are
unchanged.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- Cargo build/test, make, fuzzers, and other heavy checks were not run per task instructions
Make ZSTDMT_createCompressionJob match the exact callback type consumed by
ZSTDMT_rust_compressStreamGeneric. The callback now performs the existing
opaque and end-directive casts itself, so the redundant C forwarding shim can
be removed without changing scheduling, assertions, logging, or ABI behavior.
Test Plan:
- `cc -fsyntax-only -Werror=incompatible-pointer-types -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- No cargo, make, native, fuzz, or other heavy tests run per task scope.
The separate-file decompression callback only reached the existing Rust
FIO_rust_determineDstName ABI through a two-argument C wrapper that supplied
the file-static suffix table and display string. Call the Rust ABI directly at
both decompression call sites, keeping the same suffix inputs and return-value
handling while removing the redundant wrapper and forward declaration.
Test Plan:
- `cc -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy programs/fileio.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, Make, native tests, fuzzers, and other heavy verification were not run
per task constraints.
Move the MT LDM overlap predicate behind the original
ZSTDMT_doesOverlapWindow() by-value ABI. Rust now owns the half-open range
calculation through a checked representation of Buffer and ZSTD_window_t;
C retains the synchronization and context orchestration around the callback.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
The new direct Rust file-I/O callback is unsafe because it receives a raw
NUL-terminated filename pointer. Add the required safety contract so CLI
clippy with `-D warnings` accepts the migrated ABI leaf.
Test Plan:
- `git diff --cached --check` -- passed.
- Capped CLI clippy rerun after this commit.
The new context-parameter projection test is compiled only in test builds,
but its constant was imported at module scope. Keep the import in the test
module so normal clippy builds remain warning-free under `-D warnings`.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
Import the no-attach-dictionary mode constant used by the new direct
ZSTD_getCParamsFromCCtxParams() projection test. The implementation was
correct, but the test module did not inherit that constant through the
existing parameter imports, so the root crate could not compile.
Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
The --list stdin callback was a stateless string predicate: it ignored its
opaque context and compared the input name with the fixed stdin marker. Move
that callback under its existing caller symbol into the Rust CLI archive, so
C no longer owns this policy leaf. The Rust implementation preserves the
callback ABI and returns the same 1/0 result for the marker and ordinary
names. File opening, frame parsing, diagnostics, human-readable formatting,
private file-info storage, and list orchestration remain C-owned. The optional
codec-version helpers remain untouched because their build-time library
configuration is not propagated to the Rust archive.
Test Plan:
- `clang -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder programs/fileio.c` -- passed.
- `rustfmt --edition 2021 --check --config skip_children=true rust/cli/src/lib.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, clippy, native builds, and upstream tests were not run per the explicit no-heavy-command constraint.
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.
Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.
Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
the requested lightweight-only scope.
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.
Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.
Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
The parameter policy leaves now export their C API names directly from Rust.
Two compatibility aliases were only used by the Rust unit test, while the
library and clippy builds correctly treated them as unused. Remove those dead
aliases and call the direct Rust-owned clamp symbol from the test so the
library, bench, and test targets share the same ABI surface without warning
under `-D warnings`.
Test Plan:
- Capped `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.
Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.
Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the Rust pass-through and frame-analysis leaves directly from their C
callbacks. The C translation unit still owns private resource projections,
filename suffix configuration, diagnostics, and format-specific operations;
this commit only removes two exact return-forwarding layers.
Test Plan:
- clang -fsyntax-only on programs/fileio.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.
Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.
Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.
Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
Keep the local buffer variables declared before the direct Rust buffer-policy
calls in the compression stream adapter, preserving the native C90 warning
profile after removing the old C forwarding helper.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed before this declaration-only cleanup; the next capped native gate will recheck it.
Rely on the existing fileio.h declaration now that the filename collision
checker is exported directly from Rust, avoiding a redundant C redeclaration
in the implementation file.
Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed; CLI lint and unit gates pending.
ddidderr