Commit Graph
11661 Commits
Author SHA1 Message Date
ddidderr 19c3414095 refactor(compress): move advanced one-shot order to Rust
ZSTD_compress_advanced_internal() still encoded the one-shot begin-then-end
sequence in C after public advanced parameter validation had moved behind Rust.
That left the public compression boundary split between Rust policy and a C
orchestration body, and made the begin-error ordering implicit in the C path.

Add an explicit C-layout state with opaque begin and end callbacks. Rust now
owns the ordering, forwards the source size as the pledged size, propagates a
begin error before invoking the end callback, and returns the end result. The
callbacks retain the private ZSTD_CCtx and ZSTD_CCtx_params layouts, so the
change moves policy and sequencing without exposing window, workspace,
matchfinder, or context internals to Rust. The existing advanced and
using-dictionary wrappers continue to use the same C private operations.

Test Plan:
- `git diff --cached --check` -- passed.
- `rustfmt --check --edition 2021 rust/src/zstd_compress.rs` -- passed.
- Not run: Cargo, make, native tests, builds, or heavy verification per request.
2026-07-21 09:13:50 +02:00
ddidderr cb770812fd refactor(decompress): move heap mode policy to Rust
`ZSTD_HEAPMODE` was still returned directly from C, and the stack entry
projection also copied the raw macro. That left the one-shot stack-versus-heap
classification in the C configuration shim even though Rust owns the public
decompression branch and stack action.

Keep the build-time override in C, but pass it through a one-field repr(C)
projection to Rust. Rust normalizes values below one to the stack mode and
values at or above one to the heap mode, preserving the existing behavior
while making the branch policy explicit. The C stack object, private DCtx
layout, platform initialization, and trace boundary remain C-owned; moving
those would either duplicate build configuration or cross the requested ABI
boundary. The stack projection now uses the same normalized result as the
one-shot branch.

ABI layout assertions cover the scalar projection, and focused Rust tests cover
negative, zero, positive, maximum, and missing-configuration inputs.

Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Cargo, Make, native tests, and heavy commands intentionally not run per the
  task restriction; integration verification remains pending.
2026-07-21 09:12:42 +02:00
ddidderr b0b7441e94 fix(compress): correct MT callback declaration
Keep the MT frame-progression callback under the ZSTD_MULTITHREAD preprocessor guard using an ordinary C declaration. The previous warning-cleanup edit accidentally prefixed the declaration with a preprocessor marker, which broke the multithreaded compilation path.

Test Plan:

- git diff --cached --check

- Capped native rebuild failed before this fix at the invalid directive; rerun pending
2026-07-21 08:54:24 +02:00
ddidderr 04ebb65d0f docs(rust): record migrated policy seams
Keep the migration boundary accurate after moving frame-progression dispatch, the decompression no-forward-progress threshold, and compression metadata-transfer selection into Rust. The README now states which scalar inputs Rust owns and which private callbacks, probes, and operations remain in C.

Test Plan:

- git diff --cached --check

- Capped clippy, CLI tests, native build, smoke test, and full upstream suite passed before this documentation-only commit
2026-07-21 08:53:43 +02:00
ddidderr 71cad090cf fix(compress): gate MT frame callback by build feature
The MT frame-progression callback is only referenced when ZSTD_MULTITHREAD is enabled. Keep its definition under the same configuration guard so single-threaded library, test, and decode-corpus builds do not report an unused static function while the MT callback remains available to the Rust dispatch bridge.

Test Plan:

- git diff --cached --check

- Capped full tests passed before this warning-only guard

- Capped native rebuild and smoke rerun pending
2026-07-21 08:53:13 +02:00
ddidderr 6978a373c3 fix(compress): keep MT frame callback warning-clean
The Rust-owned frame-progression dispatch invokes the existing C MT progression API through a mutable opaque callback context. Match that API's established signature in the adapter so the new boundary does not introduce a discarded-const qualifier warning while preserving the private MT context.

Test Plan:

- git diff --cached --check

- Capped native make rerun pending after this warning-only fix
2026-07-21 08:42:09 +02:00
ddidderr b9f9c16340 refactor(decompress): move no-progress threshold policy to Rust
ZSTD_NO_FORWARD_PROGRESS_MAX is a compile-time override that was returned
directly by the C adapter. Keep the active build value in C, where
configuration belongs, but pass it through an ABI-checked scalar projection to
Rust. Rust now owns the policy boundary and preserves the exact configured
threshold, while the existing stalled-stream comparison, error mapping, and
private DCtx layout remain unchanged.

Test Plan:
- `git diff --cached --check` -- passed
- Cargo, make, builds, native tests, and heavy verification were not run per
  request.
2026-07-21 08:38:40 +02:00
ddidderr 11dc99899b refactor(compress): move frame progression dispatch to Rust
Move the public ZSTD_getFrameProgression MT-versus-single-thread worker-count
branch into a Rust-owned scalar projection. C continues to compute the
single-thread scalar inputs and keeps the private ZSTDMT context behind a
callback; the compile-time ZSTD_MULTITHREAD adapter remains local to C. Add
C/Rust layout assertions and focused Rust tests for both dispatch paths and
null-state fallback.

Test Plan:
- `git diff --cached --check`
- Not run: Cargo, make, builds, native tests, and heavy verification per request.
2026-07-21 08:38:26 +02:00
ddidderr e01f79fb1a refactor(cli): move compression metadata policy to Rust
Keep the private stat_t probe, destination opening, metadata syscalls, and format callbacks in C, but route the regular-source plus stdin/stdout exception policy through the Rust fileio preference layer. The Rust ABI bridge normalizes the scalar consumed by the existing destination lifecycle, with compile-time value assertions and focused tests covering regular, non-regular, stdin, stdout, and nonzero scalar inputs.

Test Plan: Not run by request; cargo, make, native tests, and heavy commands were intentionally avoided. Lightweight git diff --check passed.
2026-07-21 08:34:04 +02:00
ddidderr df30cb58d4 style(rust): format decoder policy bridge
Apply the repository formatter to the sequence-decoder policy bridge introduced in the preceding decompression refactor. The change is formatting-only.

Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check
2026-07-21 08:16:02 +02:00
ddidderr 0800f0d4c0 refactor(dict): move advanced-CDict copy policy to Rust
Keep the opaque ZSTD_CCtx_params copy operation in C, but make Rust own the public advanced2 null-source validation and copy-before-preparation ordering. The bridge now rejects a missing source before any allocator or workspace callback, with explicit ABI assertions and focused order/null-input tests.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 08:15:25 +02:00
ddidderr f44d9726c3 refactor(decompress): move sequence mode policy to Rust
Keep the C build-time mutual-exclusion check and private decoder-context assembly in the C shim, but pass the selected force-short/force-long policy through Rust. Rust now owns normalization to the runtime, short, or long sequence decoder mode, with ABI assertions and focused null/build-policy tests.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 08:13:08 +02:00
ddidderr f077d71e8b refactor(decompress): move legacy threshold policy to Rust
Keep the target-specific ZSTD_LEGACY_SUPPORT macro in the C translation unit, but pass its scalar value through a Rust policy bridge before legacy dispatch. Rust now accepts only the historical supported range 1 through 7 and normalizes all other values, including a missing projection, to disabled. ABI assertions and focused tests preserve the compile-time configuration contract.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 07:57:10 +02:00
ddidderr 78c4118a67 refactor(cli): move file-removal policy to Rust
Keep FIO_removeFile responsible for the filesystem operation, exact diagnostics, and C return wrapper, but route its status classification through a Rust policy bridge. Unknown statuses are explicitly treated as failed removal. ABI value assertions and focused mapping tests preserve the C contract.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 07:55:18 +02:00
ddidderr b89acb1fb5 style(rust): format migrated policy bridges
Apply the repository formatter to the dictionary-size and MT input-publication bridges introduced in the preceding atomic refactors. The change is formatting-only and keeps the semantic seams independently reviewable.

Test Plan: cargo +nightly fmt --manifest-path rust/Cargo.toml --all
2026-07-21 07:39:11 +02:00
ddidderr f1dcd68d5d refactor(dict): move CDict size policy to Rust
Keep the advanced CDict estimator and its private workspace sizing inputs in C, but route the public estimate through a Rust policy bridge. Rust now selects unknown-source/create-CDict parameters and the public by-copy load mode before invoking the opaque C estimator. Focused tests verify callback ordering, selected parameters, and the missing-policy-input guard.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 07:38:34 +02:00
ddidderr 8373de7917 refactor(mt): move input publication policy to Rust
Keep the C-owned compression job callback responsible for private buffer and job-state mutation, but return the range-active decision alongside its result. Rust now owns the outer scheduler policy that publishes or suppresses the reusable input count after job creation, including the error path. The focused seam test covers the callback-clears-range case without exposing the MT context layout.

Test Plan: Not run in this atomic commit; the capped serial Rust, native, smoke, and original test-suite verification follows.
2026-07-21 07:38:20 +02:00
ddidderr c522906d07 refactor(cli): move list status policy to Rust
Move the scalar InfoError-to-action mapping used by FIO_listFile into Rust and reuse the same classifier while aggregating multi-file list results. C retains file opening/parsing, exact diagnostics, metadata formatting, private fileInfo_t state, and the public result values; add layout and mapping coverage.

Test Plan: git diff --cached --check; focused Rust mapping test added; full capped verification will run after the MT and dictionary workers are integrated.
2026-07-21 07:35:53 +02:00
ddidderr 484a76757b style(rust): format newly migrated policy bridges
Apply rustfmt to the stream-result and MT input-range policy projections and their focused tests. This is mechanical only; it keeps the new Rust seams compliant with the repository formatter without changing behavior.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; git diff --cached --check.
2026-07-21 07:17:16 +02:00
ddidderr 3215458381 refactor(mt): move input-range commit policy to Rust
Move the ready/wrap branch and synchronization ordering from ZSTDMT_tryGetInputRange into Rust. The Rust policy now waits for the prefix range, invokes the C-owned prefix move, waits for the selected source range, and publishes the buffer; C retains memmove, private buffer fields, round-buffer state, and LDM callbacks behind the projection. Add ABI assertions and focused wrapped/ordinary ordering tests.

Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification follows.
2026-07-21 07:16:29 +02:00
ddidderr fc9aeee92b refactor(compress): move stream result policy to Rust
Move the post-call ZSTD_compressStream2_c policy into Rust: adapter errors now short-circuit before buffer publication, successful calls publish buffer expectations before calculating pending output, and the private C context remains behind a callback projection. Add ABI assertions and ordering tests for both paths.

Test Plan: git diff --cached --check; focused Rust tests added; full capped Rust/native/original-suite verification will run after the MT slice is integrated.
2026-07-21 07:15:23 +02:00
ddidderr 8ae0dfa49d refactor(cli): move source-open policy to Rust
Move stdin-sentinel classification and source stat/open result policy into the Rust fileio backend while keeping C responsible for diagnostics, binary-mode setup, and FILE ownership. The bridge leaves stat layout and native file utilities behind the existing C ABI and adds a focused no-stat stdin test.

Test Plan: git diff --cached --check; focused Rust test added but full capped verification will run after the remaining workers are integrated.
2026-07-21 07:14:54 +02:00
ddidderr 14e3f7c140 fix(compress): make MT loop layout assertion lint-clean
The new MT-loop policy projection asserted its total size as eight times
`size_of::<usize>()`. Although that describes the padded C layout, clippy
interprets the multiplication as a manual bit-count expression and rejects it
under the repository's `-D warnings` policy.

Express the same ABI invariant from the final field offset plus its field size.
This keeps the assertion tied to the actual projected layout without changing
any runtime policy or C/Rust representation.

Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
2026-07-21 06:54:32 +02:00
ddidderr 0faf3e4cf4 refactor(compress): move MT stream loop policy to Rust
The multithreaded branch of `ZSTD_compressStream2_c` performed the complete
post-call result classification in C: it prioritized errors, recognized a
completed end directive, and applied different progress rules for continue,
flush, and end operations. Those decisions were scalar policy around a C-owned
MT call, but remained embedded beside private counters, reset, and tracing.

Rust now classifies one projected iteration and returns an explicit action for
continue, break, error, or completed end. The C shim still owns the MT call,
input/output accounting, error forwarding, private context mutation, reset,
and trace callback; the existing condition ordering and progress comparisons
are preserved. ABI layout assertions and focused tests cover error precedence,
completed end, progress-based continue termination, and pending/full output.

Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_compress.rs` -- passed
- Full capped Rust/native verification is the next serial step.
2026-07-21 06:53:09 +02:00
ddidderr 35150c3cff refactor(decompress): move stack entry policy to Rust
The stack-backed one-shot decompression entry point used to make its own
heap-mode rejection, static DCtx initialization, allocation-error mapping,
and dispatch ordering in C. That kept scalar control flow beside the private
DCtx layout even though Rust already owns the decompression policy and context
leaf.

Rust now consumes an ABI-checked scalar projection and callback set. It makes
the branch and ordering decisions, maps a failed static initializer to the
same memory-allocation error, and invokes the existing C-owned context leaf.
The C shim retains the local stack object, private DCtx layout, and
`ZSTD_initStaticDCtx` implementation. Recording-callback tests cover heap
rejection, initializer failure, and successful initialization-to-dispatch
ordering without fabricating the private context layout.

Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/zstd_decompress.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
  compression seam is integrated.
2026-07-21 06:52:40 +02:00
ddidderr bc3c3b1ac6 refactor(cli): move decompression status action policy to Rust
The decompression callback in fileio used to classify result statuses and
select its display action with a C switch. That left scalar result policy in
the C frontend even though Rust already owns the decompression dispatch and
result classification, and it coupled the C callback to a diagnostic enum.

Rust now exposes an ABI-checked status-action classifier. It keeps the
original action ordering and preserves silent handling for statuses that have
no display diagnostic, while the C callback retains the exact diagnostic
strings, source-name formatting, and display operation. Invalid inputs map to
a silent fallback action so the public callback remains behavior-compatible.
Focused Rust tests cover every status class, invalid values, and the exported
ABI result.

Test Plan:
- `git diff --cached --check` -- passed
- `rustfmt --edition 2021 --check rust/src/fileio_asyncio.rs` -- passed
- Full capped Rust/native verification remains pending until the parallel
  compression and decompression seams are integrated.
2026-07-21 06:50:25 +02:00
ddidderr c075a7b2d9 refactor(cli): move destination action policy to Rust
FIO_openDstFile retained the filesystem leaf in Rust but its C wrapper still encoded the status precedence for test mode, stdout, same-file protection, sparse-mode adjustment, overwrite prompting, removal, retry, and final errors. Centralize that pure action classification in Rust without moving FILE* handling, metadata, preference mutation, prompts, diagnostics, or retry callbacks across the ABI.\n\nThe action bridge validates status, confirmation, and sparse-state inputs and has focused tests for every destination outcome, sparse-first ordering, prompt acceptance/abort, quiet mode, overwrite mode, setvbuf/open failures, success, and invalid inputs.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
2026-07-21 06:28:58 +02:00
ddidderr db5ae46f42 refactor(mt): move active parameter transition policy to Rust
ZSTDMT_updateCParams_whileCompressing already delegated parameter derivation to Rust, but C still owned the transition publication order and independently updated compressionLevel. Return a Rust-owned scalar result that carries the requested level with the derived compression parameters, while preserving the active frame window and all private MT context state in C.\n\nThe new projection and result have explicit C/Rust layout assertions. Focused Rust tests exercise unknown-size and explicit-size hints, LDM and override inputs, saved-window restoration, requested-level forwarding, and the exported ABI wrapper.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo fmt/clippy gates passed before staging\n- git diff --cached --check\n- Full capped native and original-test verification follows after the batch is committed.
2026-07-21 06:28:42 +02:00
ddidderr 85e9457fcc refactor(compress): move stable stream init policy to Rust
ZSTD_compressStream2_c still owns private context mutation, diagnostics, buffer updates, and codec initialization, but its stable-input transparent-initialization decision was a remaining scalar policy island in C. Move the continuation validation, block-size threshold, format-specific progress hint, and wrapping size arithmetic behind a Rust projection. Preserve the original C error messages and return values while keeping all private context state on the C side.\n\nThe bridge uses explicit ABI layout assertions for the mixed pointer/size projection. Focused Rust tests cover buffered and non-continue calls, block-boundary initialization, empty and short stable input, both frame formats, matching continuation state, invalid source/position, and null bridge input.\n\nTest Plan:\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings\n- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings\n- Full capped native and original-test verification follows after the batch is committed.
2026-07-21 06:28:29 +02:00
ddidderr b0e90c39e2 refactor(mt): move stream dictionary branch policy to Rust
The MT stream initializer and its later dictionary update both encoded the
same three-way choice in C: copy a supplied dictionary, attach a borrowed
CDict, or install a raw prefix. The old code also embedded the required
release/clear-before-attach ordering in each callback wrapper. Project the
presence and raw-content flags into Rust, where the branch and ordering are
now explicit and tested. C retains only the private CDict allocation,
prefix-storage, context publication, and destruction callbacks, so the
configured C layouts and allocator behavior remain unchanged.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `git diff --check` and `rustfmt --edition 2021 --check` -- passed before commit
- Capped native and original-test verification remains pending for the complete batch.
2026-07-20 19:47:51 +02:00
ddidderr ee6953b1d7 refactor(cli): classify dictionary load diagnostics in Rust
The file-I/O wrapper already delegated dictionary loading to Rust but kept
all status interpretation in C. That duplicated the malloc and mmap status
families and made the platform-specific error branches part of the C policy
surface. Add a Rust classifier that maps the shared numeric loader statuses
to diagnostic actions, including the distinct mmap failure classes. Keep
metadata lookup, platform handles, ownership, and the exact EXM_THROW text
in C, where the configured platform APIs still belong. The classifier also
rejects out-of-range type/status values; the valid malloc and mmap enums
intentionally share numeric values and therefore cannot be distinguished
beyond their family tag.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed (192 tests)
- Broader native and original-test verification remains pending for the complete batch.
2026-07-20 19:47:31 +02:00
ddidderr d714aeac23 refactor(compress): move heap CCtx release order to Rust
ZSTD_freeCCtx previously passed one C callback that bundled dictionary,
multithreaded context, workspace, and heap-object teardown. That left the
lifetime order inside private C control flow and made the public destruction
policy harder to audit. Project the four private operations separately and
let Rust enforce the original dictionary, MT-context, workspace, and
object order while preserving the static-context and workspace-embedded
object guards. The callbacks still own the C layouts and custom allocator
operations.

Test Plan:
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- Broader native and original-test verification remains pending for the complete batch.
2026-07-20 19:41:52 +02:00
ddidderr a152fe6498 refactor(cli): derive window error log in Rust
Move the historical ceil(log2(windowSize)) calculation used by
FIO_zstdErrorHelp into Rust. C retains frame-header parsing, private read-pool
access, the ABI slot, callbacks, diagnostics, and output formatting. The
zero-size and non-power-of-two behavior remains explicit and tested at the
u64 boundary.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed, 190 tests
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:29:56 +02:00
ddidderr e9850abbb7 refactor(mt): move terminal frame action policy to Rust
Classify the MT terminal-job flags in Rust while retaining the two private C
mutations as narrow callbacks. Descriptor setup, reusable-input reset, and
terminal frame publication keep their original order; nonterminal jobs remain
untouched, and checksum clearing still occurs only for the original flag
combination.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed, 190 tests
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:29:23 +02:00
ddidderr bab90f3622 refactor(compress): move end-stream result policy to Rust
Project the worker-count choice in ZSTD_endStream into Rust. Multithreaded
streams retain their minimal remaining-output estimate, while single-threaded
streams keep the precise end-stream arithmetic. C retains error forwarding,
private context reads, and the public wrapper.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/cli/Cargo.toml --all-targets` -- passed, 190 tests
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:29:01 +02:00
ddidderr d9515fcf96 refactor(cli): move zstd frame action policy to Rust
Project the zstd frame decoder status into a Rust action classification while
preserving C-owned diagnostics, error-help formatting, private resources, and
exact decoding return values. Unknown statuses retain the assertion fallback
and the existing frame-decoding error result.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:05:43 +02:00
ddidderr ffc52de643 refactor(decompress): move stack heap-mode policy to Rust
Move only the heap-mode branch decision for stack decompression into Rust
while retaining the C-owned stack context, static initialization, error codes,
and decoder projection. The Rust scalar policy preserves the historical
heapmode >= 1 rejection boundary.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:05:28 +02:00
ddidderr cea0a54ace refactor(compress): move stream buffer policy to Rust
Project the public ZSTD_compressStream2 buffer positions into Rust and
preserve the original output-first classification. C retains the
dstSize_tooSmall and srcSize_wrong mappings, exact diagnostics, private
context, and streaming state machine.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 ./tests/rustLibSmoke` -- passed
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 make -j1 -C tests test` -- passed, including large streaming, native, fuzzer, and zstream phases
2026-07-20 19:05:08 +02:00
ddidderr ccec75e149 refactor(mt): move serial reset error policy to Rust
Project the private serial-state reset result into Rust for the stream-initialization error decision while retaining C-owned allocation, serial-state mutation, dictionary setup, synchronization, callback order, and ABI layouts. Nonzero callback status preserves the existing memory_allocation error path.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings; cargo test --manifest-path rust/cli/Cargo.toml --all-targets (188 passed); cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
2026-07-20 18:44:38 +02:00
ddidderr 0da86ded9a refactor(cli): move pass-through selection policy to Rust
Move automatic decompression pass-through selection into a pure Rust scalar policy while preserving explicit preference values, stdout probing, overwrite semantics, assertions, diagnostics, and all C-owned file/resource callbacks. The C fileio boundary now supplies only the policy inputs before constructing the existing callback projection.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo clippy --manifest-path rust/cli/Cargo.toml --all-targets -- -D warnings; cargo test --manifest-path rust/cli/Cargo.toml --all-targets (188 passed); cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
2026-07-20 18:44:19 +02:00
ddidderr d6cda74b84 refactor(compress): move end directive validation to Rust
Project the scalar ZSTD_compressStream2 end directive into Rust and keep C responsible for the private streaming context, existing parameter error, and diagnostic text. The Rust range policy preserves the original unsigned enum validation for continue, flush, and end.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
2026-07-20 18:25:52 +02:00
ddidderr 03d0d4edac refactor(ldm): move block compressor policy to Rust
Move LDM block-compressor strategy validation, selection ordering, and callback error propagation into Rust while retaining dictionary-mode lookup, MatchState mutation, and the actual codec callback in C. Invalid preflight inputs now use the existing generic error path, and valid strategies preserve the original optimal-parser boundary.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
2026-07-20 18:25:45 +02:00
ddidderr 6caba856d4 refactor(cli): make decompression finish policy explicit in Rust
Extract the mixed-format decompression finish status mapping into a pure Rust policy helper and keep the existing callback side effects, diagnostics, and return values in the ABI wrapper. The explicit enum makes the success, pass-through, report, and impossible-status branches auditable and testable without widening the private fileio boundary.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; make -j1; ./tests/rustLibSmoke; make -j1 -C tests test (all shell tests, large streaming tests, native tester, fuzzer phases, and zstream tester passed).
2026-07-20 18:25:37 +02:00
ddidderr 701bc899ad refactor(mt): move job preparation order to Rust
Make Rust the owner of the compression-job preparation sequence: publish the descriptor, reset the reusable input state, and publish terminal frame state last. Keep all MT job and stream layouts in C behind three focused callbacks, and preserve the nonterminal and terminal branches exactly.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
2026-07-20 18:00:40 +02:00
ddidderr 0175fdb408 refactor(cdict): move static initializer error policy to Rust
Keep the private static-CDict initializer in C, but return its native size_t status through the callback boundary. Rust now owns the public success-or-NULL mapping, preserving the original workspace validation, callback order, and private layout ownership while making initializer failures explicit and unit-testable.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
2026-07-20 18:00:09 +02:00
ddidderr bfd61e6ed7 refactor(cli): move compression status classification to Rust
Move the aggregate compression-result classification out of the C wrapper and into the Rust fileio module. Keep C responsible for the user-facing EXM_THROW diagnostics and the existing fallback assertion, so optional-format build guards and command-line behavior remain unchanged. The Rust classifier also makes unexpected statuses explicit without widening the format callback boundary.

Test Plan: ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check; GCC and Clang syntax-only checks; make -j1 -C tests test (all 41 shell tests, fuzzer, zstd tester, and zstream tester passed).
2026-07-20 17:59:38 +02:00
ddidderr 0bd25053b7 fix(cli): preserve stdin directory probe traces
The directory-source migration initially skipped its C probe for stdin because
stdin can never be a directory from the CLI's normal marker path. The original
C source-open callback nevertheless called `UTIL_isDirectory` for every source,
including the stdin marker, and file-stat trace tests rely on those calls being
observable.

Always invoke the private C directory probe before source opening and keep the
same positive-result rejection. This preserves the trace side effect for stdin
while retaining Rust ownership of the rejection decision and the old source
open ordering.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The capped upstream run exposed two stdin file-stat trace mismatches; the suite will be rerun after this compatibility fix.
2026-07-20 17:26:36 +02:00
ddidderr a0aa1cd070 fix(mt): pass boolean LDM resource flag
The resource-acquisition projection calls its field `ldmEnabled`, but the
first implementation passed the three-state `ZSTD_ParamSwitch_e` value
unchanged. `ZSTD_ps_disable` is 2, so Rust interpreted disabled LDM as enabled
and rejected ordinary jobs whose raw sequence store was intentionally absent.
That surfaced as an allocation error on the small stdin compression path.

Convert the C enum to the same boolean semantic used by the Rust policy before
crossing the ABI boundary. The projection declaration also now precedes the
callback setup statements, keeping the native C90 warning-free style.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- `git diff --cached --check` -- passed
- The first capped full upstream run failed at `test-zstd` on the small stdin path with the enum/boolean regression; the suite will be rerun after this fix.
2026-07-20 17:23:01 +02:00
ddidderr 97469a2e55 feat(mt): move job resource acquisition policy to Rust
MT worker execution used to combine CCtx acquisition, raw-sequence-store
acquisition, destination-buffer allocation, frame-header publication, and the
LDM resource check in one C callback. That made the resource order and
short-circuit policy another C-owned implementation boundary even though the
operations themselves must remain private to C.

Project the scalar LDM and destination-presence inputs into Rust and let Rust
own the ordering and error normalization. The C callbacks now only acquire
private resources, report readiness, allocate the destination buffer, and
publish the frame-header destination. The ordering remains compatible with the
worker cleanup path: both pool gets happen first, CCtx failure stops before
destination work, destination failure stops before publication, and the LDM
sequence-store check runs after publication.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed
- `ulimit -v 41943040; cargo +nightly fmt --manifest-path rust/Cargo.toml --all -- --check` -- passed
- Capped GCC and Clang syntax-only checks for `lib/compress/zstdmt_compress.c` -- passed by the worker
- `git diff --cached --check` -- passed
- Native build and full upstream tests are deferred to the post-batch serial verification.
2026-07-20 17:20:12 +02:00
ddidderr 62a9db5f65 refactor(cli): move directory source policy into Rust
The decompression source-open callback previously both tested for a named
directory and returned the generic open failure. That left a source-resource
rejection in the C orchestration path even though Rust already owns the
per-file ordering.

Add a C-owned directory probe to the projection. Rust invokes it before source
opening for named inputs and rejects a positive result; the C callback retains
the private filesystem helper and exact directory diagnostic. Stdin bypasses
the probe as before, and optional codec callbacks plus the remaining source
open, asynchronous, and cleanup ordering are unchanged.

Test Plan:
- rustfmt +nightly --check --edition 2021 rust/src/fileio_asyncio.rs (passed)
- capped GCC syntax-only check of programs/fileio.c with all optional-format
  macros enabled (passed)
- capped Clang syntax-only check of programs/fileio.c with all optional-format
  macros enabled (passed)
- git diff --cached --check (passed)
- workspace cargo +nightly fmt --check was not clean because of an unrelated
  rust/src/zstdmt_compress.rs formatting diff; that file was left untouched
- no cargo build/test, make, fuzzers, or upstream tests were run
- commit signing was disabled because the configured GPG prompt hung
2026-07-20 17:16:24 +02:00