Commit Graph
11671 Commits
Author SHA1 Message Date
ddidderr d1a7a53e5b refactor(compress): remove pure C parameter forwarders
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.

Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
2026-07-20 11:34:31 +02:00
ddidderr 58606b18aa refactor(compress): expose stream size and level leaves from Rust
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.

Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
2026-07-20 11:18:44 +02:00
ddidderr c8603f2b3e refactor(compress): remove redundant MT worker setter
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.

Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
2026-07-20 11:17:09 +02:00
ddidderr 0394ac648b fix(cli): preserve C90 declaration order
Keep the local buffer variables declared before the direct Rust buffer-policy
calls in the compression stream adapter, preserving the native C90 warning
profile after removing the old C forwarding helper.

Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed before this declaration-only cleanup; the next capped native gate will recheck it.
2026-07-20 11:14:30 +02:00
ddidderr a1133ccaf4 fix(cli): use public collision checker declaration
Rely on the existing fileio.h declaration now that the filename collision
checker is exported directly from Rust, avoiding a redundant C redeclaration
in the implementation file.

Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed; CLI lint and unit gates pending.
2026-07-20 11:12:04 +02:00
ddidderr c750a91cff refactor(cli): expose filename and buffer leaves from Rust
Move the pure filename-collision, input/output-buffer construction, and
compressed-destination-name helpers to direct Rust-owned caller symbols.
Leave C responsible for diagnostics, filesystem resources, suffix-list policy,
and the surrounding file-processing orchestration.

Test Plan: Pending capped full verification after this atomic ABI cleanup.
2026-07-20 11:01:05 +02:00
ddidderr 03ac61063d refactor(compress): expose parameter policy leaves from Rust
Remove the C forwarding wrappers around compression-parameter policy leaves
that are now implemented directly under their caller-facing ABI symbols.
Keep the private C parameter construction and resource logic in C, while Rust
continues to own the pure row-matchfinder, block-splitter, LDM, external
sequence/repcode, dictionary-tagging, block-size, and overflow predicates.

Test Plan: Pending capped full verification after the companion CLI leaf commit.
2026-07-20 11:00:57 +02:00
ddidderr 5ccc6f89fb refactor(cli): expose summary policy leaves from Rust
Export the file-summary predicates and largest-file-size scan directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving pointer contracts, assertions, return widths, and
filesystem behavior.

Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
2026-07-20 10:36:53 +02:00
ddidderr ad1d27e2af refactor(compress): expose block policy leaves from Rust
Export ZSTD_useTargetCBlockSize and ZSTD_blockSplitterEnabled directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving diagnostics, ABI signatures, and parameter layout.

Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
2026-07-20 10:36:47 +02:00
ddidderr 2e0afa9c7d refactor(compress): move CCtx construction policy to Rust
Move reset, parameter copy, and default policy resolution from
ZSTD_makeCCtxParamsFromCParams into Rust through a pointer-based ABI function.
Keep the private LDM adjustment and its invariants in C while Rust owns the
stable parameter-object policy and focused reset/resolution coverage.

Test Plan:
- worker git diff and capped format checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests

All parent verification is serial under a 40 GiB virtual-memory cap. Standalone
root Rust unit linking remains unavailable because the crate imports C-owned
bridge symbols.
2026-07-20 10:21:46 +02:00
ddidderr 7692f7a6c5 refactor(cli): move LZ4 block-size policy leaf to Rust
Export FIO_LZ4_GetBlockSize_FromBlockId directly from Rust under the existing
caller symbol and remove the redundant C forwarding wrapper. Preserve the
block-ID formula and cover its boundary values in the focused Rust test.

Test Plan:
- worker capped nightly rustfmt check
- worker git diff --check
- full serial capped native and upstream suites to run at the next parent gate
2026-07-20 10:16:12 +02:00
ddidderr 9110af5f9e refactor(cli): move highbit policy leaf to Rust
Export FIO_highbit64 directly from Rust under the existing caller symbol and
remove the C forwarding wrapper. Preserve the nonzero-input contract and add
boundary tests for the scalar policy leaf.

Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests

All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
2026-07-20 10:12:02 +02:00
ddidderr 539608cac1 refactor(compress): move CCtx parameter init policy to Rust
Move reset, copy, and default policy resolution from
ZSTD_CCtxParams_init_internal into Rust through the existing repr(C)
parameter mirror. C retains validation and diagnostics, while focused tests
cover reset behavior, policy resolution, and compression-level handling.

Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests

All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
2026-07-20 10:11:48 +02:00
ddidderr ac4b7afca8 refactor(cli): move dictionary buffer selection policy to Rust
Move the malloc-versus-mmap decision used by compression and decompression
resource construction into Rust.  The C adapter continues to own platform
loaders, allocation handles, diagnostics, and dictionary I/O; Rust only
combines the explicit mmap preference, patch-mode size threshold, and explicit
disable override.  Add enum-value ABI checks and focused policy coverage.

Also hoist the source-size declaration in the C adapter so the migration does
not introduce a C90 declaration-after-statement warning.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:50:12 +02:00
ddidderr 7ea0cbd96e refactor(compress): move LDM fast-table policy to Rust
Move the strategy classification that decides whether LDM preparation fills
fast or double-fast tables into Rust.  C retains the private match-state,
configuration-specific table-fill leaves, and the existing excluded-DFAST
assertion.  Add enum-value ABI checks plus focused tests for valid and invalid
strategy classifications.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:49:57 +02:00
ddidderr 4409ef1e16 refactor(compress): move CCtx estimate tier policy to Rust
Move the fixed source-size tier traversal used by
ZSTD_estimateCCtxSize_internal() into Rust.  C retains the authoritative
parameter lookup and per-tier workspace estimator behind callbacks, while Rust
owns the tier ordering and maximum-selection policy.  The projection includes
ABI layout assertions and tests for callback order, level propagation, and
error-valued maxima.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:49:37 +02:00
ddidderr dd87b328a4 refactor(compress): move MT checksum serialization to Rust
Keep XXH64 state ownership and checksum calculation in the C MT scheduler,
but project the final four-byte frame write through Rust.  Rust now owns the
little-endian serialization and destination-capacity policy, with ABI layout
assertions and flush-state error propagation preserving the existing cleanup
path.  Focused tests cover byte order, exact capacity, invalid destinations,
and callback error handling.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran under a serial 40 GiB virtual-memory limit. The
full upstream suite passed, including large-file, multithreaded, API, fuzzer,
and zstream coverage. Standalone root Rust unit linking remains unavailable
because the crate imports C-owned bridge symbols without a Cargo build/link
setup.
2026-07-20 09:29:17 +02:00
ddidderr b550b14ceb refactor(decompress): remove redundant dictionary bridge
Call the authoritative DDict constructor directly from the Rust decompression
policy after converting the shared repr(C) allocator projection.  Remove the
C forwarding declaration and wrapper so dictionary creation no longer crosses
an unnecessary Rust-to-C-to-C boundary, while compile-time size and alignment
checks keep the allocator conversion tied to the ABI contract.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip, including the
parallel MT checksum seam that remains as the next commit. Standalone root
Rust unit linking remains unavailable because the crate imports C-owned bridge
symbols without a Cargo build/link setup.
2026-07-20 09:29:04 +02:00
ddidderr 6d3da898de refactor(compress): move MT parameter update policy to Rust
Move the next-job parameter derivation in
ZSTDMT_updateCParams_whileCompressing() behind a scalar Rust projection. Rust
now reproduces the existing CCtx-parameter derivation and restores the active
frame's saved windowLog; C retains private MT/CCtx layouts, logging, and final
writes.

Expose the preprocessor-derived block-compressor exclusion mask through the
private C boundary so reduced builds keep their authoritative configuration.
Add C/Rust layout assertions and focused tests for window preservation and
strategy-exclusion routing.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` (184 tests)
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; integrated build and upstream MT/API coverage passed.
2026-07-20 09:07:40 +02:00
ddidderr 870d6a2e40 refactor(decompress): move stack-context dispatch policy to Rust
Keep the heapmode guard, stack-owned ZSTD_DCtx, static initialization, and
private context layout in C. Once the context is initialized, Rust now owns the
one-shot stack-context policy: it clears the projected static-size marker and
dispatches the existing ZSTD_decompressDCtx implementation through the opaque
context pointer.

Add matching C/Rust view-layout assertions and focused callback tests covering
static-size clearing, argument forwarding, and null-context rejection.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; the integrated build and upstream C/Rust oracle supplied the runtime/link coverage.
2026-07-20 09:07:11 +02:00
ddidderr 2442fddc38 refactor(fileio): move separate-output routing policy to Rust
Move the separate-destination mode decision into the Rust file-iteration
boundary. The projection now carries the mirror-mode bit and one callback for
each destination policy, so Rust selects the callback once before preserving
the existing non-short-circuiting iteration and aggregate error behavior.

Keep path construction, mirror-directory diagnostics, resource ownership, and
the per-file compression leaf in the opaque C callbacks. Add C/Rust layout
assertions and focused tests proving mirror/flat routing and ordered error
aggregation.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets` (793 tests)
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` (184 tests)
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
2026-07-20 08:41:31 +02:00
ddidderr 675c3fc307 refactor(fileio): move zstd window diagnostic policy to Rust
Project the zstd window-too-large diagnostic inputs into Rust so Rust owns
error filtering and the concrete-versus-unsupported guidance choice. C keeps
frame-header parsing, window-log extraction, and the exact display text behind
a callback, preserving the existing diagnostics and fallback behavior.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1 (clean after C90 declaration cleanup)
- ulimit -v 41943040; make -j1 -C tests test (passed; the later cleanup only moved a declaration before statements)
2026-07-20 08:21:27 +02:00
ddidderr 69425cb8a4 refactor(compress): move MT input hint dispatch to Rust
Project the scalar single-thread hint inputs and keep the multithreaded
context behind a C callback so Rust owns the worker-count dispatch. Preserve
the existing buffer-mode assertion and both hint arithmetic leaves, with
focused tests covering MT selection and single-thread fallback.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 08:21:06 +02:00
ddidderr 22cb347f6e refactor(fileio): move decompression teardown order to Rust
Project the decompression-resource cleanup callbacks into Rust so the stable
teardown order remains dictionary, dstream, write pool, and read pool. C
retains ownership of dRess_t and each private resource implementation, with
its existing dstream CHECK behavior preserved. Focused tests cover order and
null or incomplete callback states.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 07:56:27 +02:00
ddidderr 9c46e18172 refactor(compress): move stream fast-path policy to Rust
Project the private stream state needed by the complete-input streaming fast
path into a stable C/Rust layout and let Rust own the eligibility predicate.
The C bridge keeps the CCtx private and retains the existing simple-level
selection leaf as a callback, while focused tests cover the accepted and
rejected stream states.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 07:56:08 +02:00
ddidderr 429694a250 refactor(mt): move stream reset ordering to Rust
Make Rust own the MT stream reset sequence through a narrow callback
projection. C retains the private buffer, job-ID, frame-flag, progress, and
sentinel mutations, preserving the original order without exposing private
layouts across the ABI.

Test Plan:
- rustfmt --edition 2021 --check rust/src/zstdmt_compress.rs
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (786 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 07:31:23 +02:00
ddidderr dc3c9c6c11 refactor(mt): move resize policy to Rust
Move MT resize callback ordering and short-circuit failure policy into Rust.
C retains the private factory, jobs table, buffer pool, CCtx pool, sequence pool,
and parameter storage behind narrow callbacks, including the original ignored
worker-parameter setter result.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (784 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 07:08:36 +02:00
ddidderr 2b2f9df5de refactor(mt): move context construction policy to Rust
Move MT context validation, worker clamping, resource-construction order, and
partial-failure cleanup policy into Rust. Keep the private context layout,
allocator-owned resources, pools, jobs table, and synchronization primitives in
C behind callbacks.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:44:03 +02:00
ddidderr a00ba4c2b7 refactor(cli): move codec diagnostic mapping to Rust
Move LZMA and LZ4 compression-status classification into Rust while retaining
user-facing diagnostic text, format callbacks, and the existing C ABI boundary.
Unknown statuses continue to select the generic diagnostic path.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:43:54 +02:00
ddidderr f7c04292d4 refactor(compress): move advanced CDict wrapper policy to Rust
Move advanced CDict parameter preparation and construction ordering into the Rust
policy layer while keeping advanced2 allocation, workspace, and private context
layout in C. Preserve the existing allocator and callback ABI at the seam.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:43:45 +02:00
ddidderr 25428b8cef refactor(compress): move LDM parser threshold to Rust
Move the scalar policy that selects the optimal parser for long-distance
matching into Rust.  The C implementation continues to own match-state and
compressor dispatch, but the strategy threshold is now defined and tested at
the Rust seam.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:22 +02:00
ddidderr 9cfbb7bebb refactor(cli): move decompression diagnostic mapping to Rust
Move the status-to-diagnostic policy for decompression failures into the
Rust fileio layer.  C retains the user-facing strings and callback display
logic, while Rust returns a stable diagnostic classification across the ABI
and leaves silent statuses silent.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:14 +02:00
ddidderr 0b56f667db refactor(compress): move heap CCtx initialization to Rust
Move the ordering-sensitive setup of a privately allocated compression
context behind a Rust callback coordinator while keeping allocation and
context storage in C.  The explicit callback ABI preserves zeroing, custom
allocator publication, BMI2 setup, and parameter-reset order, including
reset-error propagation.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:05 +02:00
ddidderr a67e6a59df refactor(cli): move MT resource parameter policy to Rust
File-resource creation already delegates the general compression-parameter
policy to Rust, but the multithreaded parameters were still applied in C in a
separate branch. That left parameter ordering, optional overlap handling, and
error short-circuiting outside the Rust policy boundary. Add a narrow ABI
projection with C callbacks for CCtx mutation and diagnostics, and let Rust
apply worker count, job size, optional overlap, and rsyncable in the original
order. The C90 declaration layout and compile-time ABI assertions keep the
existing native program configurations intact.

Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- 179 passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed without the new C90 warning
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
2026-07-20 06:03:41 +02:00
ddidderr 998c88c97b refactor(compress): move dictionary teardown ordering to Rust
The compression context still owns private dictionary storage and CDict
objects, but the teardown order is observable because each step can invoke an
allocator or clear state used by later steps. Keep those storage operations as
C callbacks while moving the sequencing policy into the Rust compression
module. The C bridge now contains only callback adapters and compile-time ABI
layout checks, and the Rust unit test pins the original five-step order.

Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
2026-07-20 06:02:52 +02:00
ddidderr 71a18ce315 refactor(mt): move per-job parameter policy to Rust
Move the pure multithreaded worker-parameter policy into Rust: only the first
job keeps the frame checksum flag, worker jobs disable LDM, and every worker
context runs with zero nested workers. Keep the complete C-owned parameter
structure and codec initialization in the adapter, crossing the ABI with a
small layout-checked scalar projection.

Test Plan:
- rustfmt --edition 2021 --check rust/src/zstdmt_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 05:42:31 +02:00
ddidderr 2d23838112 refactor(compress): move stable buffer policy to Rust
Move the stable input and output expectation update into the Rust compression
module. Keep the private CCtx layout and debug logging in the C adapter while
passing only the buffer-mode scalars and projected buffer fields across the
ABI. Preserve C size_t wrapping for the output remainder and leave buffered
modes unchanged.

Test Plan:
- rustfmt --edition 2021 --check rust/src/zstd_compress.rs
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 05:42:18 +02:00
ddidderr f3a2caa0e2 fix(dict-builder): stabilize optimized dictionary scoring
Optimized dictionary scoring used the requested dictionary ID while building
its scoring CDict, so --dictID could change candidate quality and output
content. Parallel FastCover could also choose different equal-score candidates
based on completion order. Clear the ID only while constructing the scoring
CDict, restore it immediately afterward, and use deterministic (d, k)
tie-breakers in Cover and FastCover. The requested ID now remains output
metadata instead of influencing dictionary content or repeat-run stability.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml scoring_dictionary_id_is_restored_after_the_score --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 05:22:25 +02:00
ddidderr bbe3a5dd74 refactor(fileio): move compression parameter policy to Rust
Move the non-threaded compression parameter sequence and adaptive window policy
out of FIO_createCResources.  C continues to own context and resource
allocation, dictionary and pool setup, and the multithread-specific settings;
Rust drives the ordered parameter callbacks and stops on the first error.
The projection mirrors the C ABI and has focused order and short-circuit tests.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml create_c_resources --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
2026-07-20 04:49:35 +02:00
ddidderr 907c3af89e refactor(mt): move context teardown order to Rust
Move ZSTDMT_freeCCtx ownership branches and teardown ordering into Rust while
keeping the MT context, worker pool, synchronization, dictionary, and custom
allocator layouts private to C callbacks.  Preserve the original factory and
round-buffer conditions, release jobs before destroying the pools, and keep
free-on-null compatible with the C API.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml free_cctx --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
2026-07-20 04:49:27 +02:00
ddidderr 6d164b7746 refactor(compress): move toFlushNow policy to Rust
Move the worker-count branch of ZSTD_toFlushNow into the Rust policy layer.
The C adapter retains the private CCtx and multithread context, exposing only
the worker count and a callback for the MT flush query.  Single-threaded
contexts keep the existing zero result and the ABI projection is checked on
both sides.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml to_flush_now --lib
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -j1
2026-07-20 04:49:18 +02:00
ddidderr dc83c98f06 refactor(fileio): move compression teardown order to Rust
Keep fileio's compression-resource teardown order in the Rust policy layer:
dictionary, write pool, read pool, then compression context.  The C adapter
keeps cRess_t and each private resource layout local while exposing only
one callback projection to Rust, preserving the existing cleanup behavior.
The Rust entry point also treats a null state as a no-op and verifies the
projection layout at compile time.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml free_c_resources --lib
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
2026-07-20 04:38:09 +02:00
ddidderr 66e7799f04 refactor(compress): move sequence generation policy to Rust
ZSTD_generateSequences() previously mixed public parameter validation,
temporary-output allocation, sequence-collector setup, compression, and cleanup
inside the C translation unit. Move that ordering into Rust behind callbacks so
private ZSTD_CCtx and SeqCollector layouts remain C-owned. The adapter retains
C allocation, context access, and the existing ZSTD_compress2 operation, while
Rust guarantees temporary-output cleanup after collector setup failures and
compression errors and preserves the original target-block-size and worker
parameter checks.

Focused callback tests cover validation-before-allocation, cleanup on setup
failure, and successful callback ordering/count publication.

Test Plan:
- `cargo fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml generate_sequences --lib` -- passed (3 tests)
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets` -- passed (762 tests)
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; make -j1` -- passed
2026-07-20 04:30:33 +02:00
ddidderr 53218c5537 fix(cli): use div_ceil for list ABI test
The Rust CLI clippy gate treats the hand-written alignment rounding in the
multi-file list projection layout test as a manual div-ceil implementation.
Use the standard integer operation so the test remains clear, equivalent, and
warning-free under the repository's deny-warnings policy.

Test Plan:
- `cargo fmt --manifest-path rust/cli/Cargo.toml --all` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
2026-07-20 04:16:54 +02:00
ddidderr 1c3bf6b708 refactor(compress): move CDict construction policy to Rust
Move advanced and static CDict construction ordering into Rust while keeping
private workspace layout, object initialization, dictionary loading, and
allocator leaves in C callbacks.  The Rust policy now validates parameters and
memory, computes the workspace contract, sequences allocation/reservation and
initialization, and preserves failure cleanup for both dynamic and static
workspaces.  Add callback-order, allocation-failure, reservation-failure, and
initializer-failure tests around the ABI projections.

Test Plan:
- `cargo fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo build --manifest-path rust/Cargo.toml --release`
2026-07-20 04:05:14 +02:00
ddidderr 371451d69c refactor(cli): move --list multi-file policy to Rust
Move stdin and empty-input validation, header selection, per-file iteration,
status aggregation, and multi-file total selection into the Rust CLI policy
layer.  The callback projection keeps C responsible for its private file-info
storage, file parsing, diagnostics, human-readable formatting, and per-file
listing.  Only successful and frame-error records contribute to the aggregate,
matching the original early-return behavior for invalid or truncated inputs.

Test Plan:
- `rustfmt --edition 2021 --check rust/src/fileio_prefs.rs` -- passed.
- `git diff --cached --check` -- passed.
- Cargo, native, and full test commands were not run per the explicit no-heavy-command constraint.
2026-07-20 04:03:41 +02:00
ddidderr a9e519db24 fix(fileio): satisfy compression projection lint
Keep the compression destination ABI assertion byte-accurate while expressing
the final callback offset in the form required by current Clippy. The total
projection size remains nine function-pointer slots.

Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- git diff --check
2026-07-20 03:34:23 +02:00
ddidderr 6c060c901f refactor(compress): move CCtx reset orchestration to Rust
Move the residual ZSTD_resetCCtx_internal sequencing into a Rust policy
entrypoint. The Rust side now owns the plan-to-workspace-to-tail order and
scalar hand-off, while C retains private workspace checks, context
initialization, match-state reset, and pointer publication callbacks. Preserve
error short-circuiting and the existing static, dynamic, LDM, and index-reset
behavior.

Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- Full capped Rust, native, CLI, and upstream test suites to follow
2026-07-20 03:32:56 +02:00
ddidderr 4b96b32118 refactor(fileio): move compression destination policy to Rust
Move the single-file compression destination lifecycle into the Rust file-I/O
policy layer while keeping C responsible for private resources, file handles,
diagnostics, metadata operations, and codec dispatch callbacks. Preserve the
original shared-destination fast path, temporary permissions, handler timing,
metadata-before-close ordering, close-error propagation, stdout cleanup guard,
and failed-artifact removal.

Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- git diff --check
- Full capped Rust, native, CLI, and upstream test suites to follow
2026-07-20 03:32:46 +02:00
ddidderr ef50115fcb fix(fileio): satisfy decompression ABI lint
Use the pointer-width bit count for the decompression callback projection
assertion so the Rust archive remains clean under warnings-as-errors Clippy.
The assertion keeps the same byte offset on supported targets.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
2026-07-20 03:04:17 +02:00