feat(compress): move stream2 fallback orchestration to Rust

The public ZSTD_compressStream2_c entry point still contained the complete
fallback state machine even though its validation, initialization policy,
serial-versus-MT dispatch, and result accounting were already represented by
Rust policy helpers. That left the main streaming entry point as a large C
orchestration island and made the Rust rewrite boundary misleading.

Project the scalar stream state and private CCtx operations through a checked
C ABI, then let Rust own the fallback ordering. Rust now validates buffers and
the end directive, handles transparent one-shot completion and stable-input
deferral, performs the initialization and stability decisions, selects the
serial or MT path, and publishes the result policy. C retains the private
context layout, stream adapter, MT operations, checksum/epilogue side effects,
and trace/reset callbacks. ABI offset assertions and focused callback-order
and stable-input tests protect the projection without requiring a standalone
Rust test binary to link every C bridge symbol.

Test Plan:
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo +nightly fmt --manifest-path rust/Cargo.toml --all` -- passed.
- `ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings` -- passed before and after formatting.
- `git diff --cached --check` -- passed.
- Native `make -j1` and the original suite remain the next post-commit verification gates.
This commit is contained in:
2026-07-21 13:08:28 +02:00
parent 5c6302a3c3
commit 6cf4898ab1
3 changed files with 762 additions and 150 deletions
+9 -5
View File
@@ -65,9 +65,13 @@ zstd ABI:
Rust while C retains the private contexts and mutation callbacks.
The public sequence APIs likewise use Rust-owned validation, frame-header,
checksum, and output-accounting orchestration around C-owned block state.
The public end-of-frame path and the `ZSTD_compress2_c` fallback also use
Rust-owned orchestration boundaries while C retains the context reset,
stream adapter, checksum/epilogue, and trace callbacks.
The public end-of-frame path, the `ZSTD_compress2_c` fallback, and the
`ZSTD_compressStream2_c` fallback also use Rust-owned orchestration
boundaries while C retains the context reset, stream adapter,
checksum/epilogue, and trace callbacks. The stream2 fallback keeps
transparent initialization/defer, buffer validation, serial-versus-MT
dispatch, and result accounting in Rust; C supplies only the private CCtx
callbacks and projected state.
Public advanced one-shot compression now uses the same Rust-owned
begin-then-end ordering boundary while C retains the private begin/end
callbacks.
@@ -151,8 +155,8 @@ zstd ABI:
destination-name construction, and the private file/resource/format
callbacks for these scheduler boundaries.
Rust already owns the file preference policy, filename decisions,
source/destination opening, dictionary buffers, asynchronous I/O pools,
and pass-through copy leaf.
source/destination opening and destination retry ordering, dictionary
buffers, asynchronous I/O pools, and pass-through copy leaf.
- `timefn` provides the monotonic nanosecond clock behind `UTIL_time_t`,
while `benchfn` owns the benchmark run/timing loop (`BMK_benchFunction`,
`BMK_benchTimedFn`) and `benchzstd` owns benchmark orchestration and