feat(mt): move frame progression job aggregation into Rust

Move the multithreaded frame-progression ring scan into Rust so the Rust
scheduler owns job-ID masking, ready-job range handling, error-output
normalization, and active-worker accounting. Keep C-owned job descriptors and
mutex-protected snapshots behind the narrow ZSTDMT_projectJob callback, which
preserves the private synchronization boundary while making the aggregation
policy directly testable in Rust.

Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml --lib (691 passed)
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && MAKEFLAGS=-j1 make -B -C programs -j1 zstd
- ulimit -v 41943040 && MAKEFLAGS=-j1 make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s (84 tests and both short fuzz rounds passed)
This commit is contained in:
2026-07-19 18:53:08 +02:00
parent fec11ae7d0
commit adaae03552
3 changed files with 141 additions and 23 deletions
+5 -2
View File
@@ -95,7 +95,9 @@ zstd ABI:
private to C while Rust owns input-retention scans, reusable input-range
overlap decisions, outer scheduling and end-directive adjustments,
job-creation decisions, compression-job stage sequencing and error flow,
and pending-output decisions through scalar job projections.
pending-output decisions through scalar job projections, and frame
progression's job-ring scan, error normalization, and active-worker
accounting.
- Dictionary support
- `zstd_ddict` owns, loads, copies, and references decode dictionaries.
- Legacy decoding
@@ -145,7 +147,8 @@ shared- and separate-destination multi-file compression schedulers,
shared- and separate-destination decompression scheduling,
single-threaded stream initialization and the buffered/stable stream state
machine, MT stream initialization, MT outer scheduling and flush policy, MT
compression-job stage sequencing and error flow, public sequence-API
compression-job stage sequencing and error flow, MT frame-progression job
aggregation, public sequence-API
orchestration, sequence-store and block policy, external-producer invocation
and success-path validation, external-sequence-store reset,
external-sequence/literals block loop, optional-format decompression loops,