Commit Graph
11587 Commits
Author SHA1 Message Date
ddidderr 8919e0d292 refactor(mt): export window overlap policy from Rust
Move the MT LDM overlap predicate behind the original
ZSTDMT_doesOverlapWindow() by-value ABI. Rust now owns the half-open range
calculation through a checked representation of Buffer and ZSTD_window_t;
C retains the synchronization and context orchestration around the callback.

Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstdmt_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
2026-07-20 13:59:15 +02:00
ddidderr fe3587c637 refactor(ldm): expose parameter adjustment from Rust
Move the pure LDM parameter-defaulting policy behind the original
ZSTD_ldm_adjustParameters() ABI symbol. Rust now receives the existing C
parameter structs directly, applies the same 30/8/8 build constants, and
keeps the ASAN-sensitive table-size projection in C.

Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy verification intentionally deferred until the combined seam set is ready.
2026-07-20 13:57:15 +02:00
ddidderr b8e5932282 fix(cli): document stdin callback safety contract
The new direct Rust file-I/O callback is unsafe because it receives a raw
NUL-terminated filename pointer. Add the required safety contract so CLI
clippy with `-D warnings` accepts the migrated ABI leaf.

Test Plan:
- `git diff --cached --check` -- passed.
- Capped CLI clippy rerun after this commit.
2026-07-20 13:48:52 +02:00
ddidderr e18b05e920 fix(compress): keep test-only parameter import local
The new context-parameter projection test is compiled only in test builds,
but its constant was imported at module scope. Keep the import in the test
module so normal clippy builds remain warning-free under `-D warnings`.

Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
2026-07-20 13:38:08 +02:00
ddidderr 625b129876 fix(compress): import context parameter policy constant
Import the no-attach-dictionary mode constant used by the new direct
ZSTD_getCParamsFromCCtxParams() projection test. The implementation was
correct, but the test module did not inherit that constant through the
existing parameter imports, so the root crate could not compile.

Test Plan:
- `git diff --cached --check` -- passed.
- Root capped clippy rerun after this commit.
2026-07-20 13:37:31 +02:00
ddidderr 8e241eaa28 refactor(cli): move --list stdin predicate to Rust
The --list stdin callback was a stateless string predicate: it ignored its
opaque context and compared the input name with the fixed stdin marker. Move
that callback under its existing caller symbol into the Rust CLI archive, so
C no longer owns this policy leaf. The Rust implementation preserves the
callback ABI and returns the same 1/0 result for the marker and ordinary
names. File opening, frame parsing, diagnostics, human-readable formatting,
private file-info storage, and list orchestration remain C-owned. The optional
codec-version helpers remain untouched because their build-time library
configuration is not propagated to the Rust archive.

Test Plan:
- `clang -fsyntax-only -Iprograms -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder programs/fileio.c` -- passed.
- `rustfmt --edition 2021 --check --config skip_children=true rust/cli/src/lib.rs` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Cargo, clippy, native builds, and upstream tests were not run per the explicit no-heavy-command constraint.
2026-07-20 13:37:00 +02:00
ddidderr 3f081ee53b refactor(compress): export context parameter policy from Rust
Move ZSTD_getCParamsFromCCtxParams() out of its C forwarding body and
export the original symbol from the Rust parameter API. The Rust-side
ABI mirror already has layout checks, so it can project the same fields
to the existing scalar policy leaf without exposing additional context
state. Keep the exclusion mask in C and query it from Rust so reduced
builds retain their preprocessor-selected block-compressor behavior.

Test Plan:
- `cc -fsyntax-only -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -Ilib/legacy lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy Cargo, Make, native, and test-suite verification intentionally not run per request.
2026-07-20 13:34:20 +02:00
ddidderr 7174d2ec71 refactor(ldm): expose max sequence bound from Rust
The LDM sequence-space helper was already implemented in Rust, but C still
owned the public symbol and forwarded through a pointer-based Rust bridge.
Make the Rust parameter representation public and ABI-compatible for the
original by-value `ldmParams_t` call, then have Rust compare `enableLdm`
directly with `ZSTD_ps_enable` before applying the sequence bound. Remove only
the redundant C bridge declaration and wrapper; table sizing, state, and
sequence generation remain unchanged.

Test Plan:
- `git diff --check` -- passed.
- `git diff --cached --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_ldm.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per the requested lightweight-only scope.
2026-07-20 13:16:26 +02:00
ddidderr 66a163a78c refactor(compress): expose dictionary entropy leaf from Rust
The dictionary entropy-header loader was already implemented in Rust, but C
kept a private bridge declaration and retained `ZSTD_loadCEntropy` as a
forwarding wrapper. Export the original ABI name from Rust and retain a
crate-local `ZSTD_rust_loadCEntropy` alias for existing Rust callers. Remove
only the bridge declaration and forwarding body; dictionary-content
orchestration, callbacks, and private C layouts remain unchanged.

Test Plan:
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `git diff --check` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
- Cargo build/test, make, and other heavy checks were intentionally not run per
  the requested lightweight-only scope.
2026-07-20 13:15:57 +02:00
ddidderr ca9e383771 refactor(compress): expose compressed block reset from Rust
The compressed-block reset logic was already implemented in Rust, but the
Rust export used a private bridge name and C retained a forwarding wrapper
under the original ABI name. Export the original
`ZSTD_reset_compressedBlockState` symbol directly from Rust, keep the
`ZSTD_rust_resetCompressedBlockState` crate-local alias for existing Rust
callers, and remove only the redundant C declaration and body. Private
context-layout code and other C functions remain unchanged.

Test Plan:
- `git diff --check` -- passed.
- `cc -std=c99 -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dict -fsyntax-only lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- Cargo build/test and make were intentionally not run per the requested lightweight-only scope.
2026-07-20 13:00:39 +02:00
ddidderr 6bff89163f refactor(compress): expose sequence statistics leaves from Rust
The C definitions of ZSTD_get1BlockSummary() and ZSTD_resetSeqStore()
only forwarded their arguments to Rust, so they duplicated the ABI boundary
without contributing assertions, private-state adaptation, or C-only policy.
Make Rust provide the existing internal symbols directly and remove only the
corresponding C declarations and forwarding bodies. Keep crate-local aliases
for the current compressor-module imports; they are Rust name aliases and do
not add the former ABI symbols back. No other statistics leaf is included.

Test Plan:
- `ulimit -v 41943040; cc -Ilib -Ilib/common -Ilib/compress -Ilib/decompress -Ilib/dictBuilder -Ilib/legacy -Ilib/deprecated -Ilib/zstd -fsyntax-only lib/compress/zstd_compress.c` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --check` and `git diff --cached --check` -- passed
- No cargo builds, make commands, or tests run, per request
2026-07-20 12:42:03 +02:00
ddidderr b51b8f24d2 fix(compress): remove dead parameter aliases
The parameter policy leaves now export their C API names directly from Rust.
Two compatibility aliases were only used by the Rust unit test, while the
library and clippy builds correctly treated them as unused. Remove those dead
aliases and call the direct Rust-owned clamp symbol from the test so the
library, bench, and test targets share the same ABI surface without warning
under `-D warnings`.

Test Plan:
- Capped `cargo clippy --manifest-path rust/Cargo.toml -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings` -- passed.
- Capped `cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --cached --check` -- passed.
2026-07-20 12:26:13 +02:00
ddidderr 6d9b3d24b6 refactor(compress): expose pure parameter policy leaves from Rust
The compression-parameter check, clamp, and cycle-log helpers were still
translation-unit C bodies that only forwarded value arguments to Rust. That
left redundant C ownership and made the direct Rust ABI names differ from the
public or internal C symbols used by callers. Export the existing C names from
the Rust parameter module and remove the forwarding bodies. Crate-local aliases
retain the old Rust-side names for existing Rust callers; all private context,
configuration, assertion, and stateful adapters remain on the C side.

Test Plan:
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
2026-07-20 12:22:25 +02:00
ddidderr d2a43b5080 refactor(compress): expose CCtx parameter leaves from Rust
The three public CCtx parameter APIs were still implemented as C bodies that
only forwarded into Rust symbols. Move ownership of those leaves to the Rust
ABI by using the existing public C symbol names directly, while retaining the
same signatures, behavior, and linker-visible API names. The neighboring C
helpers remain because they carry C-owned configuration or private-state
adaptation rather than being pure public forwarders.

Test Plan:
- `cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed.
- `cc -fsyntax-only -Ilib -Icommon lib/compress/zstd_compress.c` -- passed.
- `git diff --check` and `git diff --cached --check` -- passed.
- Heavy cargo, make, and test commands were intentionally not run per scope.
2026-07-20 12:04:17 +02:00
ddidderr e1d98eb8b6 refactor(compress): expose LDM skip leaves from Rust
Keep the existing internal ZSTD_ldm_* ABI names while making the two
sequence-store skip operations direct Rust exports. Remove only the C
forwarding declarations and bodies; LDM state projection, configuration
constants, and block-compressor adapters remain in C.

Test Plan:
- clang -fsyntax-only on lib/compress/zstd_ldm.c
- git diff --check
- Full capped native/upstream suite pending after this commit
2026-07-20 11:49:48 +02:00
ddidderr db4ddda35b refactor(cli): remove pure fileio forwarders
Call the Rust pass-through and frame-analysis leaves directly from their C
callbacks. The C translation unit still owns private resource projections,
filename suffix configuration, diagnostics, and format-specific operations;
this commit only removes two exact return-forwarding layers.

Test Plan:
- clang -fsyntax-only on programs/fileio.c
- git diff --check
- Full capped native/upstream suite pending after this commit
2026-07-20 11:34:40 +02:00
ddidderr d1a7a53e5b refactor(compress): remove pure C parameter forwarders
Call the existing Rust parameter leaves directly from the compression
orchestration layer. This removes the redundant dedicated-dictionary
parameter reversal, C-parameter equality, and advanced CCtx-parameter
allocation wrappers while keeping all private context/resource callbacks and
debug assertion behavior in place.

Test Plan:
- clang -fsyntax-only on lib/compress/zstd_compress.c
- git diff --check
- Full capped native/upstream suite pending after this commit
2026-07-20 11:34:31 +02:00
ddidderr 58606b18aa refactor(compress): expose stream size and level leaves from Rust
Move the public stream-size and predefined compression-level leaves to their
caller-facing Rust symbols, removing the remaining C-only forwarding bodies.
Keep crate-local aliases where Rust implementation modules still use the old
internal names, while the public C ABI now lands directly in Rust.

Test Plan: Nightly Rust formatting and staged diff checks passed; capped native and upstream gates pending.
2026-07-20 11:18:44 +02:00
ddidderr c8603f2b3e refactor(compress): remove redundant MT worker setter
Call the existing Rust-backed public parameter setter directly from the MT
create and resize callbacks. Remove the C-only helper that added no policy or
resource ownership beyond that forwarding call.

Test Plan: Worker ran `clang -fsyntax-only`, `git diff --check`, and a symbol-reference audit; capped native and upstream gates pending.
2026-07-20 11:17:09 +02:00
ddidderr 0394ac648b fix(cli): preserve C90 declaration order
Keep the local buffer variables declared before the direct Rust buffer-policy
calls in the compression stream adapter, preserving the native C90 warning
profile after removing the old C forwarding helper.

Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed before this declaration-only cleanup; the next capped native gate will recheck it.
2026-07-20 11:14:30 +02:00
ddidderr a1133ccaf4 fix(cli): use public collision checker declaration
Rely on the existing fileio.h declaration now that the filename collision
checker is exported directly from Rust, avoiding a redundant C redeclaration
in the implementation file.

Test Plan: `ulimit -v 41943040; make -j1 -C tests test` passed; CLI lint and unit gates pending.
2026-07-20 11:12:04 +02:00
ddidderr c750a91cff refactor(cli): expose filename and buffer leaves from Rust
Move the pure filename-collision, input/output-buffer construction, and
compressed-destination-name helpers to direct Rust-owned caller symbols.
Leave C responsible for diagnostics, filesystem resources, suffix-list policy,
and the surrounding file-processing orchestration.

Test Plan: Pending capped full verification after this atomic ABI cleanup.
2026-07-20 11:01:05 +02:00
ddidderr 03ac61063d refactor(compress): expose parameter policy leaves from Rust
Remove the C forwarding wrappers around compression-parameter policy leaves
that are now implemented directly under their caller-facing ABI symbols.
Keep the private C parameter construction and resource logic in C, while Rust
continues to own the pure row-matchfinder, block-splitter, LDM, external
sequence/repcode, dictionary-tagging, block-size, and overflow predicates.

Test Plan: Pending capped full verification after the companion CLI leaf commit.
2026-07-20 11:00:57 +02:00
ddidderr 5ccc6f89fb refactor(cli): expose summary policy leaves from Rust
Export the file-summary predicates and largest-file-size scan directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving pointer contracts, assertions, return widths, and
filesystem behavior.

Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
2026-07-20 10:36:53 +02:00
ddidderr ad1d27e2af refactor(compress): expose block policy leaves from Rust
Export ZSTD_useTargetCBlockSize and ZSTD_blockSplitterEnabled directly from
Rust under their existing caller symbols. Remove the redundant C forwarding
wrappers while preserving diagnostics, ABI signatures, and parameter layout.

Test Plan:
- worker capped format, C syntax, and diff checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests
2026-07-20 10:36:47 +02:00
ddidderr 2e0afa9c7d refactor(compress): move CCtx construction policy to Rust
Move reset, parameter copy, and default policy resolution from
ZSTD_makeCCtxParamsFromCParams into Rust through a pointer-based ABI function.
Keep the private LDM adjustment and its invariants in C while Rust owns the
stable parameter-object policy and focused reset/resolution coverage.

Test Plan:
- worker git diff and capped format checks
- parent capped root clippy and native build
- parent capped upstream make -j1 -C tests test
- parent capped CLI clippy and tests

All parent verification is serial under a 40 GiB virtual-memory cap. Standalone
root Rust unit linking remains unavailable because the crate imports C-owned
bridge symbols.
2026-07-20 10:21:46 +02:00
ddidderr 7692f7a6c5 refactor(cli): move LZ4 block-size policy leaf to Rust
Export FIO_LZ4_GetBlockSize_FromBlockId directly from Rust under the existing
caller symbol and remove the redundant C forwarding wrapper. Preserve the
block-ID formula and cover its boundary values in the focused Rust test.

Test Plan:
- worker capped nightly rustfmt check
- worker git diff --check
- full serial capped native and upstream suites to run at the next parent gate
2026-07-20 10:16:12 +02:00
ddidderr 9110af5f9e refactor(cli): move highbit policy leaf to Rust
Export FIO_highbit64 directly from Rust under the existing caller symbol and
remove the C forwarding wrapper. Preserve the nonzero-input contract and add
boundary tests for the scalar policy leaf.

Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests

All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
2026-07-20 10:12:02 +02:00
ddidderr 539608cac1 refactor(compress): move CCtx parameter init policy to Rust
Move reset, copy, and default policy resolution from
ZSTD_CCtxParams_init_internal into Rust through the existing repr(C)
parameter mirror. C retains validation and diagnostics, while focused tests
cover reset behavior, policy resolution, and compression-level handling.

Test Plan:
- capped nightly rustfmt check
- capped root clippy with all targets and -D warnings
- capped native make -j1
- capped upstream make -j1 -C tests test
- capped CLI clippy and 185 CLI tests

All integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory cap. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols.
2026-07-20 10:11:48 +02:00
ddidderr ac4b7afca8 refactor(cli): move dictionary buffer selection policy to Rust
Move the malloc-versus-mmap decision used by compression and decompression
resource construction into Rust.  The C adapter continues to own platform
loaders, allocation handles, diagnostics, and dictionary I/O; Rust only
combines the explicit mmap preference, patch-mode size threshold, and explicit
disable override.  Add enum-value ABI checks and focused policy coverage.

Also hoist the source-size declaration in the C adapter so the migration does
not introduce a C90 declaration-after-statement warning.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:50:12 +02:00
ddidderr 7ea0cbd96e refactor(compress): move LDM fast-table policy to Rust
Move the strategy classification that decides whether LDM preparation fills
fast or double-fast tables into Rust.  C retains the private match-state,
configuration-specific table-fill leaves, and the existing excluded-DFAST
assertion.  Add enum-value ABI checks plus focused tests for valid and invalid
strategy classifications.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:49:57 +02:00
ddidderr 4409ef1e16 refactor(compress): move CCtx estimate tier policy to Rust
Move the fixed source-size tier traversal used by
ZSTD_estimateCCtxSize_internal() into Rust.  C retains the authoritative
parameter lookup and per-tier workspace estimator behind callbacks, while Rust
owns the tier ordering and maximum-selection policy.  The projection includes
ABI layout assertions and tests for callback order, level propagation, and
error-valued maxima.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip under a serial
40 GiB virtual-memory limit. Standalone root Rust unit linking remains
unavailable because the crate imports C-owned bridge symbols without a Cargo
build/link setup.
2026-07-20 09:49:37 +02:00
ddidderr dd87b328a4 refactor(compress): move MT checksum serialization to Rust
Keep XXH64 state ownership and checksum calculation in the C MT scheduler,
but project the final four-byte frame write through Rust.  Rust now owns the
little-endian serialization and destination-capacity policy, with ABI layout
assertions and flush-state error propagation preserving the existing cleanup
path.  Focused tests cover byte order, exact capacity, invalid destinations,
and callback error handling.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran under a serial 40 GiB virtual-memory limit. The
full upstream suite passed, including large-file, multithreaded, API, fuzzer,
and zstream coverage. Standalone root Rust unit linking remains unavailable
because the crate imports C-owned bridge symbols without a Cargo build/link
setup.
2026-07-20 09:29:17 +02:00
ddidderr b550b14ceb refactor(decompress): remove redundant dictionary bridge
Call the authoritative DDict constructor directly from the Rust decompression
policy after converting the shared repr(C) allocator projection.  Remove the
C forwarding declaration and wrapper so dictionary creation no longer crosses
an unnecessary Rust-to-C-to-C boundary, while compile-time size and alignment
checks keep the allocator conversion tied to the ABI contract.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets`

The integrated checks ran at the combined working-tree tip, including the
parallel MT checksum seam that remains as the next commit. Standalone root
Rust unit linking remains unavailable because the crate imports C-owned bridge
symbols without a Cargo build/link setup.
2026-07-20 09:29:04 +02:00
ddidderr 6d3da898de refactor(compress): move MT parameter update policy to Rust
Move the next-job parameter derivation in
ZSTDMT_updateCParams_whileCompressing() behind a scalar Rust projection. Rust
now reproduces the existing CCtx-parameter derivation and restores the active
frame's saved windowLog; C retains private MT/CCtx layouts, logging, and final
writes.

Expose the preprocessor-derived block-compressor exclusion mask through the
private C boundary so reduced builds keep their authoritative configuration.
Add C/Rust layout assertions and focused tests for window preservation and
strategy-exclusion routing.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` (184 tests)
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; integrated build and upstream MT/API coverage passed.
2026-07-20 09:07:40 +02:00
ddidderr 870d6a2e40 refactor(decompress): move stack-context dispatch policy to Rust
Keep the heapmode guard, stack-owned ZSTD_DCtx, static initialization, and
private context layout in C. Once the context is initialized, Rust now owns the
one-shot stack-context policy: it clears the projected static-size marker and
dispatches the existing ZSTD_decompressDCtx implementation through the opaque
context pointer.

Add matching C/Rust view-layout assertions and focused callback tests covering
static-size clearing, argument forwarding, and null-context rejection.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test` (all upstream tests completed successfully)
- Standalone Rust unit linking remains unavailable without the C-owned bridge symbols; the integrated build and upstream C/Rust oracle supplied the runtime/link coverage.
2026-07-20 09:07:11 +02:00
ddidderr 2442fddc38 refactor(fileio): move separate-output routing policy to Rust
Move the separate-destination mode decision into the Rust file-iteration
boundary. The projection now carries the mirror-mode bit and one callback for
each destination policy, so Rust selects the callback once before preserving
the existing non-short-circuiting iteration and aggregate error behavior.

Keep path construction, mirror-directory diagnostics, resource ownership, and
the per-file compression leaf in the opaque C callbacks. Add C/Rust layout
assertions and focused tests proving mirror/flat routing and ordered error
aggregation.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets` (793 tests)
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings`
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` (184 tests)
- `ulimit -v 41943040; make -j1`
- `ulimit -v 41943040; make -j1 -C tests test`
2026-07-20 08:41:31 +02:00
ddidderr 675c3fc307 refactor(fileio): move zstd window diagnostic policy to Rust
Project the zstd window-too-large diagnostic inputs into Rust so Rust owns
error filtering and the concrete-versus-unsupported guidance choice. C keeps
frame-header parsing, window-log extraction, and the exact display text behind
a callback, preserving the existing diagnostics and fallback behavior.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1 (clean after C90 declaration cleanup)
- ulimit -v 41943040; make -j1 -C tests test (passed; the later cleanup only moved a declaration before statements)
2026-07-20 08:21:27 +02:00
ddidderr 69425cb8a4 refactor(compress): move MT input hint dispatch to Rust
Project the scalar single-thread hint inputs and keep the multithreaded
context behind a C callback so Rust owns the worker-count dispatch. Preserve
the existing buffer-mode assertion and both hint arithmetic leaves, with
focused tests covering MT selection and single-thread fallback.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (790 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (184 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 08:21:06 +02:00
ddidderr 22cb347f6e refactor(fileio): move decompression teardown order to Rust
Project the decompression-resource cleanup callbacks into Rust so the stable
teardown order remains dictionary, dstream, write pool, and read pool. C
retains ownership of dRess_t and each private resource implementation, with
its existing dstream CHECK behavior preserved. Focused tests cover order and
null or incomplete callback states.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 07:56:27 +02:00
ddidderr 9c46e18172 refactor(compress): move stream fast-path policy to Rust
Project the private stream state needed by the complete-input streaming fast
path into a stable C/Rust layout and let Rust own the eligibility predicate.
The C bridge keeps the CCtx private and retains the existing simple-level
selection leaf as a callback, while focused tests cover the accepted and
rejected stream states.

All heavy verification was run serially with a 40 GiB virtual-memory cap and
one build job.

Test Plan:
- git diff --cached --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (788 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (181 passed)
- ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test
2026-07-20 07:56:08 +02:00
ddidderr 429694a250 refactor(mt): move stream reset ordering to Rust
Make Rust own the MT stream reset sequence through a narrow callback
projection. C retains the private buffer, job-ID, frame-flag, progress, and
sentinel mutations, preserving the original order without exposing private
layouts across the ABI.

Test Plan:
- rustfmt --edition 2021 --check rust/src/zstdmt_compress.rs
- git diff --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (786 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 07:31:23 +02:00
ddidderr dc3c9c6c11 refactor(mt): move resize policy to Rust
Move MT resize callback ordering and short-circuit failure policy into Rust.
C retains the private factory, jobs table, buffer pool, CCtx pool, sequence pool,
and parameter storage behind narrow callbacks, including the original ignored
worker-parameter setter result.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (784 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 07:08:36 +02:00
ddidderr 2b2f9df5de refactor(mt): move context construction policy to Rust
Move MT context validation, worker clamping, resource-construction order, and
partial-failure cleanup policy into Rust. Keep the private context layout,
allocator-owned resources, pools, jobs table, and synchronization primitives in
C behind callbacks.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:44:03 +02:00
ddidderr a00ba4c2b7 refactor(cli): move codec diagnostic mapping to Rust
Move LZMA and LZ4 compression-status classification into Rust while retaining
user-facing diagnostic text, format callbacks, and the existing C ABI boundary.
Unknown statuses continue to select the generic diagnostic path.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:43:54 +02:00
ddidderr f7c04292d4 refactor(compress): move advanced CDict wrapper policy to Rust
Move advanced CDict parameter preparation and construction ordering into the Rust
policy layer while keeping advanced2 allocation, workspace, and private context
layout in C. Preserve the existing allocator and callback ABI at the seam.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (782 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:43:45 +02:00
ddidderr 25428b8cef refactor(compress): move LDM parser threshold to Rust
Move the scalar policy that selects the optimal parser for long-distance
matching into Rust.  The C implementation continues to own match-state and
compressor dispatch, but the strategy threshold is now defined and tested at
the Rust seam.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:22 +02:00
ddidderr 9cfbb7bebb refactor(cli): move decompression diagnostic mapping to Rust
Move the status-to-diagnostic policy for decompression failures into the
Rust fileio layer.  C retains the user-facing strings and callback display
logic, while Rust returns a stable diagnostic classification across the ABI
and leaves silent statuses silent.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:14 +02:00
ddidderr 0b56f667db refactor(compress): move heap CCtx initialization to Rust
Move the ordering-sensitive setup of a privately allocated compression
context behind a Rust callback coordinator while keeping allocation and
context storage in C.  The explicit callback ABI preserves zeroing, custom
allocator publication, BMI2 setup, and parameter-reset order, including
reset-error propagation.

Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --all-targets (775 passed)
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets (179 passed)
- ulimit -v 41943040; make -j1
- ulimit -v 41943040; make -j1 -C tests test (all tests completed successfully)
2026-07-20 06:21:05 +02:00
ddidderr a67e6a59df refactor(cli): move MT resource parameter policy to Rust
File-resource creation already delegates the general compression-parameter
policy to Rust, but the multithreaded parameters were still applied in C in a
separate branch. That left parameter ordering, optional overlap handling, and
error short-circuiting outside the Rust policy boundary. Add a narrow ABI
projection with C callbacks for CCtx mutation and diagnostics, and let Rust
apply worker count, job size, optional overlap, and rsyncable in the original
order. The C90 declaration layout and compile-time ABI assertions keep the
existing native program configurations intact.

Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `cargo test --manifest-path rust/Cargo.toml --all-targets` -- 771 passed
- `cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- 179 passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- `make -j1` under `ulimit -v 41943040` -- passed without the new C90 warning
- `make -j1 -C tests test` under `ulimit -v 41943040` -- passed
2026-07-20 06:03:41 +02:00