Move the failed-job serial-state orchestration out of zstdmt_compress.c.
Rust now owns the lock, skip decision, serial-counter publication,
broadcast, LDM cleanup ordering, and final unlock. C retains the pthread
objects, private LDM window, and error/debug leaves behind callbacks. Add
ABI layout checks and focused tests for both skipped predecessors and later
jobs that only need the lock/unlock pair.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml zstdmt_compress::tests::serial_ensure_finished -- --nocapture
- ulimit -v 41943040 && make -j1
- ulimit -v 41943040 && make -j1 -C tests test-fuzzer FUZZERTEST=-T3s FUZZER_FLAGS=--no-big-tests
Move the serial turn increment, broadcast, and unlock ordering into a Rust
entry point. C retains the pthread condition variable and mutex operations
behind callbacks, while Rust preserves wrapping unsigned job-counter
semantics and the required broadcast-before-unlock contract. Add ABI layout
checks and focused tests for ordinary and wrapping advancement.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml zstdmt_compress::tests::serial_ -- --nocapture
- ulimit -v 41943040 && make -j1
Move the serial turn wait loop out of zstdmt_compress.c and into a Rust
entry point. Rust now owns the lock, condition-wait, and ready/skip ordering
policy while C retains pthread synchronization behind lock and wait callbacks.
The projection documents that the mutex remains locked on return for the
serial advance callback, preserving the existing worker synchronization
contract. Add ABI layout checks and focused tests for both waiting and skip
paths.
Test Plan:
- cargo fmt --manifest-path rust/Cargo.toml -- --check
- ulimit -v 41943040 && CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml zstdmt_compress::tests::serial_wait_for_turn -- --nocapture
- ulimit -v 41943040 && make -j1
Move the MT reusable-input wait decision and lock/overlap/condition-wait
ordering into Rust through a small callback projection. C retains ownership
of the pthread synchronization objects, private LDM window, and diagnostics.
Add layout assertions and callback-order tests for enabled and disabled LDM.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1; cargo test --manifest-path rust/Cargo.toml
- ulimit -v 41943040; CARGO_BUILD_JOBS=1; cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- ulimit -v 41943040; make -B -C programs -j1 zstd
- ulimit -v 41943040; make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s
Move the high-level ZSTDMT_initCStream_internal setup policy into Rust. Rust
now owns worker-count resizing decisions, job-size normalization, unfinished-
job draining order, overlap and section sizing, rsync setup, buffer sizing,
and stream reset sequencing through a scalar projection and callbacks.
Keep MT contexts, pools, job resources, dictionaries, buffers, synchronization,
and serial state private to C. C callbacks perform those private mutations while
Rust controls the transparent initialization flow and can test its normalization
and ordering independently of the private layouts.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo test --manifest-path rust/cli/Cargo.toml --all-targets -- --test-threads=1
- run the legacy Rust feature matrix and all six library/CLI clippy gates with
-D warnings
- run lib and program native rebuilds plus test-cli-tests,
test-rust-lib-smoke, and test-zstd with make -j1
- run fuzzer, zstream, and decode-corpus stress gates serially with
ulimit -v 41943040
Commit is intentionally unsigned because GPG pinentry hangs in this
non-interactive environment.
Move the high-level ZSTDMT compression-job stage sequence into Rust: resource
acquisition, per-job parameter preparation, serial sequence handling, context
initialization, external-sequence application, non-first frame-header repair,
chunk compression/error routing, tracing, and common finalization. Keep C-owned
job descriptors, pools, synchronization, codec contexts, serial state, and
cleanup behind callbacks so private worker state does not cross the boundary.
Test Plan:
- Rust library all-target tests: 517 passed, including MT job-order tests.
- Rust legacy feature matrix: 572 passed.
- Rust and CLI clippy, nightly fmt, native CLI tests (41), and library smoke.
- Native test-zstd, bounded fuzzer (319), zstream (152 + 297), and decode
corpus (1,647) all passed, including multi-GiB and MT round trips.
- All heavy checks ran serially with CARGO_BUILD_JOBS=1 or make -j1 and
ulimit -v 41943040 (40 GiB virtual memory).
- Commit is intentionally unsigned because configured GPG pinentry was
unavailable and hung during the signing attempt.
Move multithreaded input-range selection and overlap decisions into Rust using
an explicit scalar projection. C retains the round-buffer mutation, LDM
synchronization, prefix copying, and private job state, while Rust decides
whether a reusable section is available and where it begins.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- make -B -C tests -j2 test-rust-lib-smoke
The multithreaded compressor still handled the terminal empty-job case in C,
including the decision to acquire a buffer, emit the final empty block, and
clear the consumed source range. That left a small but stateful branch outside
the Rust job helpers and made allocation-failure behavior difficult to test.
Project the terminal job and result through a stable C ABI, then let Rust own
that branch's assertions, buffer callback, block-header serialization, and
failure-preserving result. C remains responsible for its private job layout
and applies the returned buffer, source, and compressed-size fields exactly as
before.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- Native multithreaded cases in `test-fuzzer`, `test-zstream`, `test-cli-tests`, and `test-zstd` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1` -- passed.
Keep the MT chunk diagnostic after moving chunk iteration into Rust without
retaining a C local that is unused when debug logging is compiled out.
Test Plan:
- make -C lib -j2 lib
- git diff --cached --check
Move the bounded multithreaded output-publication kernel into Rust. Rust now
calculates the flush amount, copies the selected job bytes, and returns the
updated output and job offsets through an explicit C-layout result. C retains
mutex and condition-variable handling, checksum insertion, job retirement,
progress accounting, and the terminal state machine.
The Rust helper validates the scalar bounds before copying and has focused
coverage for empty output space, offset partial flushes, repeated flushes,
complete flushes, sentinels, and invalid bounds.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (blocked by concurrent fileio worker formatting its uncommitted file)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (worker)
- git diff --cached --check
ZSTDMT_compressionJob kept the chunking kernel in C while Rust already owned
most of the MT pool and progress helpers. The loop mixed compression calls
with mutex and condition-variable publication, making it difficult to test
its exact final-chunk and partial-tail behavior independently.
Move only the chunk-processing kernel into Rust. It calls the existing C
compression entry points through an explicit ABI, reports the final block size
or error in a repr(C) result, and invokes a C progress callback after each
successful intermediate chunk. C retains worker setup, serial and raw-sequence
state, synchronization, error publication, cleanup, and final completion
reporting. Mock compressors cover empty, exact, partial, progress, and error
boundaries without mirroring private MT job state in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress --
--test-threads=1` -- passed (44 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream` -- passed,
including multi-threaded dictionary/checksum coverage.
Keep the private ZSTD_window_t layout and pointer-difference calculation in C,
then route the size_t-to-U32 conversion and paired limit writes through a small
Rust ABI leaf. This removes the inline C window-clear implementation while
preserving its overflow behavior at both single-threaded and multithreaded
call sites.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (77 passed)
- make -C programs -j2 zstd
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C tests -j2 test-cli-tests (41 passed)
- make -C tests -j2 test-legacy test-invalidDictionaries test-decodecorpus test-rust-lib-smoke (passed)
- git diff --check
ZSTDMT_expandJobsTable previously kept worker-capacity comparison, old-table
synchronization teardown, replacement allocation, initialization failure cleanup,
and mask updates in C. That left the remaining job-table lifecycle orchestration
outside the Rust storage leaves. Add a narrow Rust ABI that exchanges only
opaque storage and scalar mask state and accepts C callbacks for the private
synchronization lifecycle.
The adapter frees the old table before replacement, reuses the existing Rust
create/free leaves with the caller's custom memory, destroys a partially
initialized replacement before freeing it, and updates jobIDMask only after
successful initialization. C still owns ZSTDMT_CCtx and job descriptors, so
worker-facing fields and platform synchronization stay outside the Rust ABI.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features
compression zstdmt_compress --lib` -- 38 passed.
- `cargo clippy`, `cargo clippy --benches`, and `cargo clippy --tests` -- passed.
- `cargo +nightly fmt --all -- --check` -- passed.
- `make -B -C lib lib-mt` -- passed.
- `make -B -C tests -j2 fullbench poolTests` -- passed.
- `./poolTests` and `./fullbench -i1 -B1000 ../README.md` -- passed.
Move the arithmetic aggregation in ZSTDMT_getFrameProgression across the
existing C/Rust boundary. C still owns MT state traversal, mutex locking,
jobIDMask ordering, nextJobID plus jobReady handling, error normalization,
and the flushed <= produced assertion. Rust constructs the base result and
folds each compact, C-normalized job snapshot into the complete repr(C)
ZSTD_frameProgression, preserving size_t-to-U64 wrapping and active-worker
counting without exposing private job or mutex layouts.
Focused Rust tests cover zero jobs, a ready job, completed jobs, normalized
error output, and mixed active/completed aggregation.
Test Plan:
- All three compression clippy commands passed before formatting.
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- passed.
- All three compression clippy commands passed after formatting.
- cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression -- 332 passed.
- make -B -C lib -j2 lib-mt -- passed.
- make -B -C lib -j2 lib-nomt -- passed.
- make -B -C tests -j2 test-zstream -- passed: 84 named, 6,004, and 8,392
randomized cases.
- git diff --check and git diff --cached --check -- passed.
The zstream build retained the pre-existing warning at
tests/zstreamtest.c:1899 about an unterminated initializer string.
The MT rsync initialization previously computed the rolling-hash prime
power through the C-only ZSTD_rollingHash_primePower helper. Move only that
scalar exponentiation behind the existing Rust MT ABI boundary. The C caller
still owns the MT context assignment, RSYNC_LENGTH constant, and surrounding
initialization.
The Rust helper mirrors ZSTD_ipow with wrapping u64 multiplication and applies
u32 wrapping subtraction before widening the exponent, preserving the C
length == 0 behavior. Rolling-hash scanning, rotation, and all other MT state
remain unchanged.
Test Plan:
- Focused rolling_hash_prime_power Rust tests -- passed (2 tests).
- Required clippy/fmt sequence in normal, bench, and test modes -- passed.
- make -B -C lib -j2 lib -- passed.
- make -C tests test-rust-lib-smoke -- passed.
- tests/fuzzer -s4560 -t56 -i57 -v -- passed (57 tests).
- make -C tests -j2 test-zstream -- passed (84 named, 5,305 + 7,351 randomized).
- git diff --check and git diff --cached --check -- passed.
Keep buffer-pool ownership, allocation, synchronization, and the Rust
pool's saturating accounting in their existing implementations. The C
wrapper still handles NULL, obtains sizeof(*bufPool), and queries the Rust
pool size; Rust now performs only the final size_t addition with wrapping
semantics. The sequence-pool sizing alias continues to call the same C
helper and therefore retains its behavior.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features
--features compression` (before and after formatting) -- passed
- The same clippy command with `--benches` and `--tests` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml` -- passed
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress` -- 29 passed
- `make -B -C lib -j2 lib` -- passed
- `make -C tests test-rust-lib-smoke` -- passed
- `tests/fuzzer -s4560 -t56 -i57 -v` -- 57 passed
- `make -C tests -j2 test-zstream` -- 84 named, 7,661, and 8,090
randomized tests passed
- `git diff --check` and `git diff --cached --check` -- passed
The zstream build still emits the pre-existing unterminated-string warning
at tests/zstreamtest.c:1899.
ZSTDMT_sizeof_CCtx previously performed the complete size_t sum in C,
although every component depends on C-owned MT state and sizing helpers.
Keep the NULL guard, pool and dictionary queries, job-table multiplication,
and round-buffer extraction in C. Pass those eight computed components to a
Rust ABI helper that performs the same ordered wrapping additions.
This keeps MT context layout and allocator ownership on the C side while
moving only the scalar arithmetic across the existing C/Rust boundary.
Test Plan:
- `cargo clippy --manifest-path rust/Cargo.toml --no-default-features --features compression`
- Same clippy command with `--benches` and `--tests`, nightly fmt, then all three clippy commands again -- passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression` -- 289 passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests test-rust-lib-smoke` -- passed.
- `make -B -C tests -j2 fuzzer` and `tests/fuzzer -s4560 -t56 -i57 -v` -- passed.
- `make -C tests -j2 test-zstream` -- 84 named, 6,268, and 8,862 randomized cases passed.
- `git diff --check` and `git diff --cached --check` -- passed.
The zstream build retains its pre-existing unterminated-string warning at
`tests/zstreamtest.c:1899`.
ZSTDMT_nextInputSizeHint only derives a size_t hint from targetSectionSize
and inBuff.filled. Keep the MT context, public/internal entry point, and
stream orchestration in C, but delegate those two scalars to Rust. The Rust
helper uses wrapping subtraction and preserves the zero-result fallback,
including C's wrapped behavior for invalid overfill.
Test Plan:
- The three compression clippy modes passed before and after
`cargo +nightly fmt --manifest-path rust/Cargo.toml`.
- Focused Rust tests passed: 5 `mt_next_input_size_hint` tests.
- `make -B -C lib -j2 lib` passed.
- `make -C tests test-rust-lib-smoke` passed.
- `tests/fuzzer -s4560 -t56 -i57 -v` passed: 57 tests completed.
- `make -C tests -j2 test-zstream` passed: 84 named, 6,659, and 8,055
randomized tests completed.
- `git diff --check` and `git diff --cached --check` passed.
Move construction of the external-dictionary and active-prefix ranges into a narrow Rust ABI while retaining the C window and logging surface. Preserve byte-range half-open overlap semantics and leave the input-range overlap wrapper available to its other C caller.
Test Plan: cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression (180 passed); root and CLI clippy; make -B -C lib -j2 lib; make -C tests -j2 test-cli-tests (41 passed); make -B -C tests -j2 test-zstream (84 named tests plus 6,845 and 9,628 fuzz cases passed).
Move sequence-store byte accounting, chunk derivation, and repcode resolution into the Rust compression module. Move MT raw-sequence buffer conversions into Rust while preserving the existing C adapters and ABI layouts.
Test Plan:\n- cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression\n- cargo clippy --manifest-path rust/Cargo.toml\n- cargo clippy --manifest-path rust/Cargo.toml --benches\n- cargo clippy --manifest-path rust/Cargo.toml --tests\n- make -B -C lib -j2 lib\n- make -B -C tests -j2 test-cli-tests
The multithreaded compressor already uses Rust-owned buffer and CCtx pools,
but zstdmt_compress.c still allocated and freed its job table directly. Move
the raw table storage and power-of-two sizing behind Rust's custom allocator
ABI. Keep descriptor field access and platform mutex/condition initialization
in C because those layouts remain private and platform-specific. The C
wrapper initializes and destroys synchronization primitives around the Rust
storage calls, preserving failure cleanup while leaving worker job setup,
scheduling, and stream entry points in C as the fallback implementation.
Keep the Rust storage-only ABI free of MT-only C references so single-threaded
archives can omit zstdmt_compress.c without acquiring new unresolved symbols.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression zstdmt_compress --lib` -- passed (5 tests).
- `make -B -C lib lib-mt` -- passed.
- `make -B -C tests -j2 fullbench zstreamtest poolTests` -- passed.
- `./poolTests` -- passed.
- `./fullbench -i1 -B1000 README.md` -- passed, including -T2 scenarios.
- Rebuilt `programs/zstd` and ran a `-T2` compress/decompress `cmp` round-trip
-- passed.
- `rustfmt` and `cargo +nightly fmt --manifest-path rust/Cargo.toml --all
-- --check` -- passed.
- Full `cargo clippy -D warnings` remains blocked by unrelated warnings in
the concurrent `rust/src/fileio_asyncio.rs` worktree changes.
- `./zstreamtest -T5s` reaches an unrelated single-thread maxBlockSize
assertion at `tests/zstreamtest.c:2157`; MT fullbench and CLI smoke pass.
Move the reusable buffer and compression-context pools behind a Rust
implementation with a narrow C ABI adapter. The scheduler, job table, serial
LDM state, and stream orchestration remain in C until their private layouts
are ported.
Keep context types opaque across Rust modules so the pool bridge does not
depend on private C layout declarations. This also keeps the existing custom
allocator and pool replacement contracts intact.
Test Plan:
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo clippy --manifest-path
rust/Cargo.toml --all-targets -- -D warnings
- RUSTC_WRAPPER= CARGO_BUILD_RUSTC_WRAPPER= cargo test --manifest-path
rust/Cargo.toml zstdmt_compress
- make -B -C lib lib-mt
- git diff --cached --check
This function was seriously flawed:
* It didn't do output bounds checks
* It produced invalid sequences when an uncompressed or RLE block was emitted
* It produced invalid sequences when the block splitter was enabled
* It produced invalid sequences when ZSTD_c_targetCBlockSize was enabled
I've attempted to fix these issues, but this function is just a bad idea,
so I've marked it as deprecated and unsafe. We should replace it with
`ZSTD_extractSequences()` which operates on a compressed frame.
This PR introduces no functional changes. It attempts to change all
macros currently using `{ }` or some variant of that to to
`do { } while (0)`, and introduces trailing `;` where necessary.
There were no bugs found during this migration.
The bug in Visual Studios warning on this has been fixed since VS2015.
Additionally, we have several instances of `do { } while (0)` which have
been present for several releases, so we don't have to worry about
breaking peoples builds.
Fixes Issue #3830.
ddidderr
Yann Collet