Move ZSTD_CCtx_refThreadPool init-stage validation and pool-slot assignment
behind a Rust-owned ABI bridge while keeping the opaque pool and CCtx storage
in C. Preserve the stage error and avoid mutating the pool on rejection.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C programs -j1 zstd
- make -C tests -j1 test-zstream ZSTREAM_TESTTIME=-T1s
- focused ref_thread_pool unit tests
Move ZSTD_CCtx_reset directive handling into a Rust-owned state machine while keeping dictionary cleanup and parameter reset behind C callbacks. Preserve the original session ordering, stage error boundary, completion/max-block flag updates, and reset error propagation across the ABI bridge.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --all-targets -- -D warnings
- make -B -C programs -j1 zstd
- focused reset_cctx unit tests
Move ZSTD_initCStream_usingDict's public initialization policy into the Rust
projection. Rust now preserves reset, compression-level, and dictionary-load
ordering while C retains the private context and dictionary callbacks.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml init_cstream_using_dict -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move ZSTD_initCStream's public initialization policy into the Rust
projection. Rust now preserves the reset, CDict-clear, and compression-level
ordering while C retains the private context callbacks and ABI boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml init_cstream -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move ZSTD_initCStream_srcSize's public initialization policy into the Rust
projection. Rust now normalizes the pledged source size and preserves the
reset, dictionary-clear, compression-level, and pledge ordering while C
retains the private context callbacks.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml init_cstream_src_size -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move ZSTD_initCStream_usingCDict's reset and CDict-reference ordering into a
small Rust projection. Keep the opaque C context and dictionary operation
behind callbacks with an explicit layout-checked ABI boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml init_cstream_using_cdict -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move ZSTD_initCStream_usingCDict_advanced's reset, pledge, frame-parameter,
and CDict callback ordering into the Rust projection. Keep private CCtx and
CDict mutations in C callbacks and pass frame parameters as scalars across the
ABI boundary.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml init_cstream_using_cdict_advanced -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move the public CStream reset wrapper's pledge normalization, callback order,
and error short-circuiting into Rust. Keep the C-owned context reset and
pledged-size mutation behind opaque callbacks with an explicit ABI projection.
Test Plan:
- ulimit -v 41943040; CARGO_BUILD_JOBS=1 cargo test --manifest-path rust/Cargo.toml reset_cstream -- --test-threads=1
- ulimit -v 41943040; make -B -C programs -j1 zstd
- git diff --cached --check
Move the stable-buffer fallback behind ZSTD_compress2() into a Rust-owned
projection. Rust now owns reset/mode-switch/stream-end ordering and maps
remaining output to dstSize_tooSmall, while C retains the private CCtx reset,
mode mutation, and simple-arguments stream adapter callbacks.
Test Plan:
- cargo +nightly fmt --manifest-path rust/Cargo.toml
- cargo test --manifest-path rust/Cargo.toml compress2_fallback -- --test-threads=1 (2 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j1 lib
- make -B -C programs -j1 zstd
- all commands used CARGO_BUILD_JOBS=1 where applicable and
ulimit -v 41943040; no heavyweight jobs ran concurrently
Commit is intentionally unsigned because GPG pinentry hangs in this
non-interactive environment.
Move the high-level orchestration of ZSTD_compressSequences() and
ZSTD_compressSequencesAndLiterals() into Rust. Rust now owns the validation
precedence, CCtx initialization handoff, frame-header and checksum ordering,
block-loop dispatch, and output accounting for both public sequence APIs.
Keep the private CCtx, sequence-store, block-state, checksum, and conversion
layouts in C. C supplies scalar block projections and callbacks for private
initialization, frame-header emission, checksum operations, and sequence-state
preparation, preserving the ABI boundary and the existing codec leaves.
Test Plan:
- focused sequence API policy tests: 4 passed
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1:
533 passed
- cargo test --manifest-path rust/cli/Cargo.toml --all-targets --
--test-threads=1: 169 passed
- legacy compression/decompression/dictionary-builder feature matrix:
588 passed
- six library/CLI clippy gates with -D warnings
- capped serial native library/program rebuilds, 41 CLI tests, Rust library
smoke, and full test-zstd round trips
- capped serial stress gates: 278 fuzzer cases, 84+129+143 zstream cases,
and 1,601 decode-corpus cases
- every heavyweight command used CARGO_BUILD_JOBS=1 or make -j1 and
ulimit -v 41943040; no worker/native process remained afterward
Commit is intentionally unsigned because GPG pinentry hangs in this
non-interactive environment.
Move the high-level ZSTDMT_initCStream_internal setup policy into Rust. Rust
now owns worker-count resizing decisions, job-size normalization, unfinished-
job draining order, overlap and section sizing, rsync setup, buffer sizing,
and stream reset sequencing through a scalar projection and callbacks.
Keep MT contexts, pools, job resources, dictionaries, buffers, synchronization,
and serial state private to C. C callbacks perform those private mutations while
Rust controls the transparent initialization flow and can test its normalization
and ordering independently of the private layouts.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo test --manifest-path rust/cli/Cargo.toml --all-targets -- --test-threads=1
- run the legacy Rust feature matrix and all six library/CLI clippy gates with
-D warnings
- run lib and program native rebuilds plus test-cli-tests,
test-rust-lib-smoke, and test-zstd with make -j1
- run fuzzer, zstream, and decode-corpus stress gates serially with
ulimit -v 41943040
Commit is intentionally unsigned because GPG pinentry hangs in this
non-interactive environment.
Move the transparent ZSTD_CCtx_init_compressStream2 initialization policy into
Rust. Rust now owns the ordered local-dictionary, prefix, parameter-resolution,
pledged-size, worker-selection, and ordinary-buffering decisions through a
scalar projection and explicit callbacks.
Keep the private CCtx and parameter layouts, allocator and trace state, MT
context lifecycle, codec operations, reset behavior, and mutation details in C.
The C shim therefore remains the ABI and private-state boundary while the
high-level stream setup flow is testable in Rust without duplicating those
layouts.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo test --manifest-path rust/cli/Cargo.toml --all-targets -- --test-threads=1
- run the legacy Rust feature matrix and all six library/CLI clippy gates with
-D warnings
- run lib and program native rebuilds plus test-cli-tests,
test-rust-lib-smoke, and test-zstd with make -j1
- run fuzzer, zstream, and decode-corpus stress gates serially with
ulimit -v 41943040
Commit is intentionally unsigned because GPG pinentry hangs in this
non-interactive environment.
Move the high-level ZSTDMT compression-job stage sequence into Rust: resource
acquisition, per-job parameter preparation, serial sequence handling, context
initialization, external-sequence application, non-first frame-header repair,
chunk compression/error routing, tracing, and common finalization. Keep C-owned
job descriptors, pools, synchronization, codec contexts, serial state, and
cleanup behind callbacks so private worker state does not cross the boundary.
Test Plan:
- Rust library all-target tests: 517 passed, including MT job-order tests.
- Rust legacy feature matrix: 572 passed.
- Rust and CLI clippy, nightly fmt, native CLI tests (41), and library smoke.
- Native test-zstd, bounded fuzzer (319), zstream (152 + 297), and decode
corpus (1,647) all passed, including multi-GiB and MT round trips.
- All heavy checks ran serially with CARGO_BUILD_JOBS=1 or make -j1 and
ulimit -v 41943040 (40 GiB virtual memory).
- Commit is intentionally unsigned because configured GPG pinentry was
unavailable and hung during the signing attempt.
Move stage validation, dictionary clearing, and local/CDict/prefix assignment
selection for the three CCtx dictionary APIs into Rust. Keep the private CCtx
layouts, allocator and by-copy lifetime behavior, CDict ownership, reset
policy, and dictionary-content loader behind explicit C callbacks so the
language boundary carries policy rather than private state.
Test Plan:
- Rust library all-target tests: 517 passed.
- Rust legacy feature matrix: 572 passed.
- Rust and CLI clippy, nightly fmt, native CLI tests (41), and library smoke.
- Native test-zstd, bounded fuzzer (319), zstream (152 + 297), and decode
corpus (1,647) all passed.
- All heavy checks ran serially with CARGO_BUILD_JOBS=1 or make -j1 and
ulimit -v 41943040 (40 GiB virtual memory).
- Commit is intentionally unsigned because configured GPG pinentry was
unavailable and hung during the signing attempt.
Dictionary insertion still kept its high-level policy in
zstd_compress.c: short and missing dictionary handling, content-type
selection, magic and dictionary-ID processing, entropy-header loading, and
compressed-block-state reset were all interleaved with the private
match-state content loader.
Move that dispatch into ZSTD_rust_compressInsertDictionary. Rust now owns the
mode decisions, reset ordering, dictionary magic and ID semantics, entropy
loading, and error propagation. C retains only a narrow opaque callback for
ZSTD_loadDictionaryContent because that operation depends on private
ZSTD_MatchState_t, ldmState_t, workspace, and parameter layouts. The focused
Rust tests cover short/full errors, raw and auto callback selection, full
dictionary entropy loading, and noDictIDFlag behavior.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml dictionary -- --test-threads=1`
-- 21 passed under the 40 GiB virtual-memory cap.
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1`
-- 513 passed under the cap before the test-only clippy sentinel cleanup;
the focused dictionary suite passed again on the exact staged contents.
- `cargo clippy` lib/benches/tests for `rust` and `rust/cli`, with
`-D warnings`, and nightly formatting -- passed.
- Native library, CLI, full zstd, fuzzer, zstream, and decode-corpus targets
-- passed serially under the cap.
GPG signing was attempted but unavailable because no pinentry process was
available; this repository's preceding commits are unsigned, so this commit
uses the explicit unsigned fallback.
Move multithreaded input-range selection and overlap decisions into Rust using
an explicit scalar projection. C retains the round-buffer mutation, LDM
synchronization, prefix copying, and private job state, while Rust decides
whether a reusable section is available and where it begins.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- make -B -C tests -j2 test-rust-lib-smoke
Move the external-sequence-and-literals block loop into the Rust compression
module while preserving the C-owned CCtx initialization and sequence
conversion callback. Rust now owns block summaries, literal cursor movement,
entropy emission, block framing, repcode state updates, and completion
validation; C remains the private-context ABI adapter.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
The multithreaded compressor still handled the terminal empty-job case in C,
including the decision to acquire a buffer, emit the final empty block, and
clear the consumed source range. That left a small but stateful branch outside
the Rust job helpers and made allocation-failure behavior difficult to test.
Project the terminal job and result through a stable C ABI, then let Rust own
that branch's assertions, buffer callback, block-header serialization, and
failure-preserving result. C remains responsible for its private job layout
and applies the returned buffer, source, and compressed-size fields exactly as
before.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- Native multithreaded cases in `test-fuzzer`, `test-zstream`, `test-cli-tests`, and `test-zstd` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=1 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=0` -- passed.
- `make -B -C lib -j2 lib ZSTD_FORCE_DECOMPRESS_SEQUENCES_SHORT=0 ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG=1` -- passed.
The public and deprecated block APIs previously entered a C-owned
ZSTD_compressContinue_internal routine, which mixed frame-header sequencing,
window maintenance, block dispatch, and frame-size accounting with the private
CCtx match state. That made the high-level compression boundary both harder
to test and easy to diverge from the already-ported frame-chunk body.
Add an explicit C/Rust projection containing only scalar state and callback
slots. Rust now owns stage validation and transitions, header sequencing,
frame-versus-block dispatch, and consumed/produced progression, while C
callbacks retain access to the private match state and context-sensitive block
operations. The same path now covers compressContinue, deprecated block
compression, and compressEnd without exposing the unstable CCtx layout.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --all-targets -- --test-threads=1` -- 473 passed.
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features compression,decompression,dict-builder,legacy-v01,legacy-v02,legacy-v03,legacy-v04,legacy-v05,legacy-v06,legacy-v07 --all-targets -- --test-threads=1` -- 528 passed.
- Native `test-fuzzer`, `test-zstream`, `test-decodecorpus`, `test-cli-tests`, and `test-zstd` -- passed.
- `cargo clippy` default, benches, and tests for library and CLI -- passed.
Move the post-sequence-store body of ZSTD_compressBlock_internal behind a
small C/Rust state projection. C continues to build the sequence store and
handle the no-compress and sequence-producer error paths, while Rust now owns
sequence collection, entropy emission, the legacy non-first-frame RLE gate,
compressed-block confirmation, and offcode repeat cleanup. Remove the C
wrappers that became dead after those leaves moved behind the Rust body.
The first integration run exposed that leaving the old C finalization label in
place confirmed compressed block state twice, undoing Rust's pointer swap and
breaking a later sparse-file checksum. The C wrapper now returns directly for
the Rust-owned path and retains only its C-owned no-compress cleanup.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo check --manifest-path rust/Cargo.toml --lib
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-zstd
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
- make -B -C tests/fuzz -j2 all
- make -B -C tests/fuzz -j2 sequence_compression_api
- cargo clippy --manifest-path rust/Cargo.toml --tests -- -D warnings (pre-existing manual_repeat_n failure)
- cargo clippy --manifest-path rust/Cargo.toml --benches -- -D warnings (pre-existing manual_repeat_n failure)
Move the post-split partition loop out of zstd_compress.c while keeping the
private CCtx, matchfinder sequence-store construction, and split discovery in
C. Rust now receives a layout-asserted projection containing sequence-store
views, block-state slots, workspace and scalar policy, then mirrors the C
loop's dRep/cRep histories, final-literal accounting, repeated single-block
serialization, and final dRep publication.
Remove the obsolete C sequence-store/count/chunk wrappers and the debug-only
C size-estimation path that was coupled to the old loop. Add a focused Rust
fixture covering partition payloads and final literals, and document the new
ownership boundary for the split search and emission paths.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1 (448 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings (passed)
- cargo clippy --manifest-path rust/Cargo.toml -- -D warnings (passed)
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (passed)
- cargo clippy --tests/--benches remain blocked only by the pre-existing manual_repeat_n lint in one_shot_promotes_nonfirst_rle_blocks
- make -B -C lib -j2 lib (passed)
- make -B -C tests -j2 test-zstd (passed)
- make -B -C tests -j2 test-cli-tests (41 passed)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (passed)
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer (252 passed)
- make -B -C tests/fuzz -j2 all and sequence_compression_api (passed)
Move the target-compressed-block body out of zstd_compress.c while keeping
sequence-store construction, matchfinding, context lifetime, and the outer
offcode repeat-mode cleanup in C. Rust now owns the target policy, repeated
block decision, Rust superblock call, compressed-state confirmation, and raw
fallback. The narrow C projection asserts the cross-language layout instead
of exposing ZSTD_CCtx.
Add focused tests for first-block RLE suppression, RLE output, superblock
errors, raw fallback, and state-slot swapping. Remove the C-only maybeRLE
wrapper that became unreachable after the extraction.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -B -C tests -j2 test-fuzzer
Move the per-block body of ZSTD_compressSequences_internal behind an explicit
Rust projection. The C wrapper still owns context initialization, public API
validation, frame headers, checksums, and the private CCtx layout. Rust now
owns block sizing and sequence transfer, sequence-store reset, entropy
compression, raw/RLE/compressed block selection, block headers, repcode/state
swapping, repeat-mode transition, and first-block handling.
The bridge passes only the sequence store, block-state pointer slots, workspace,
policy scalars, dictionary size, and the isFirstBlock slot. It does not pass a
CCtx or C callback across the ABI. The tests cover empty-block headers,
capacity errors, first-block RLE restrictions, and entropy fallback decisions.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check
- cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1
- make -B -C lib -j2 lib
- make -B -C tests -j2 test-cli-tests
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream
- FUZZERTEST=-T5s make -C tests -j2 test-fuzzer (covers ZSTD_compressSequences at fuzzer test 190)
- git diff --cached --check
Keep the MT chunk diagnostic after moving chunk iteration into Rust without
retaining a C local that is unused when debug logging is compiled out.
Test Plan:
- make -C lib -j2 lib
- git diff --cached --check
Move the bounded multithreaded output-publication kernel into Rust. Rust now
calculates the flush amount, copies the selected job bytes, and returns the
updated output and job offsets through an explicit C-layout result. C retains
mutex and condition-variable handling, checksum insertion, job retirement,
progress accounting, and the terminal state machine.
The Rust helper validates the scalar bounds before copying and has focused
coverage for empty output space, offset partial flushes, repeated flushes,
complete flushes, sentinels, and invalid bounds.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress -- --test-threads=1
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check (blocked by concurrent fileio worker formatting its uncommitted file)
- ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream (worker)
- git diff --cached --check
ZSTDMT_compressionJob kept the chunking kernel in C while Rust already owned
most of the MT pool and progress helpers. The loop mixed compression calls
with mutex and condition-variable publication, making it difficult to test
its exact final-chunk and partial-tail behavior independently.
Move only the chunk-processing kernel into Rust. It calls the existing C
compression entry points through an explicit ABI, reports the final block size
or error in a repr(C) result, and invokes a C progress callback after each
successful intermediate chunk. C retains worker setup, serial and raw-sequence
state, synchronization, error publication, cleanup, and final completion
reporting. Mock compressors cover empty, exact, partial, progress, and error
boundaries without mirroring private MT job state in Rust.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstdmt_compress --
--test-threads=1` -- passed (44 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `ZSTREAM_TESTTIME=-T2s make -B -C tests -j2 test-zstream` -- passed,
including multi-threaded dictionary/checksum coverage.
Compression-parameter table selection and adjustment were already implemented
as Rust leaves, but the internal and public ZSTD_getCParams/ZSTD_getParams
helpers still assembled those results in the C translation unit. That left
source-size and dictionary-mode policy duplicated at the C/Rust boundary.
Add Rust implementations for the internal and public getter policies. The
internal helpers preserve a zero source size as a known empty input, while the
public helpers retain the API rule that zero means unknown. The C side keeps
only the build-specific exclusion-mask construction and thin ABI adapters, so
reduced builds continue to select the same available strategy cascade.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib -- --test-threads=1`
-- passed (411 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings`
-- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- Focused tests cover public zero-to-unknown translation, internal zero
semantics, default frame parameters, and dictionary attachment modes.
The Rust simple-compression entry points already owned the dispatch, but
ZSTD_rust_compressCCtxStrategy still called the C parameter cascade to decide
whether a fast Rust frame path was safe. That left a small policy leaf in the
C implementation and made the dispatch boundary harder to test in isolation.
Implement the helper in Rust by using the existing Rust table-selection and
C-parameter-adjustment functions with the same source-size, dictionary-mode,
and automatic-adjustment inputs as the C code. Leave only the C ABI declaration
and add tests for the table thresholds, default-level behavior, and negative
fast-level behavior. The private context reset and fallback compression paths
remain unchanged.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --lib zstd_compress --
--test-threads=1` -- passed (234 tests).
- `cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings` -- passed.
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed.
- `make -B -C lib -j2 lib` -- passed.
- `make -C tests -j2 test-rust-lib-smoke` -- passed.
- Full-target clippy remains blocked by pre-existing test-only
`manual_repeat_n` and `manual_dangling_ptr` warnings outside this seam.
The optimal block splitter stores up to 196 split points and then needs one
additional partition entry for the terminal sequence boundary consumed by the
C block-emission loop. The previous array declaration provided only the split
slots, so a maximally partitioned block could write the terminal boundary past
the projected Rust/C state. Reserve the extra entry and document the layout;
retain focused tests for equal-cost and estimation-error cases so the splitter
continues to avoid unnecessary or invalid partitions.
Test Plan:
- Focused block-split Rust tests -- 7 passed
- `make -B -C lib -j2 lib` -- passed
- `cargo +nightly fmt --manifest-path rust/Cargo.toml -- --check` -- passed
- Scoped `git diff --check` -- passed
Move the active-stage check and pledged-size update behind a scalar Rust ABI
leaf while keeping the CCtx layout and public C entry point in C. Preserve the
stage_wrong error encoding and the unsigned wrap behavior at U64::MAX.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (80 passed)
- cargo clippy --manifest-path rust/Cargo.toml --lib -- -D warnings
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C programs -j2 zstd
- git diff --check
Keep the private ZSTD_window_t layout and pointer-difference calculation in C,
then route the size_t-to-U32 conversion and paired limit writes through a small
Rust ABI leaf. This removes the inline C window-clear implementation while
preserving its overflow behavior at both single-threaded and multithreaded
call sites.
Test Plan:
- cargo test --manifest-path rust/Cargo.toml --lib zstd_compress::tests (77 passed)
- make -C programs -j2 zstd
- rustfmt +nightly --edition 2021 rust/src/zstd_compress.rs --check
- make -C tests -j2 test-cli-tests (41 passed)
- make -C tests -j2 test-legacy test-invalidDictionaries test-decodecorpus test-rust-lib-smoke (passed)
- git diff --check
Include localDict.dict in the eligibility guard for the simple Rust compress2 route. By-reference dictionary loads populate this field without populating dictBuffer, so omitting it incorrectly allowed a dictionary-bearing context to bypass the C dictionary path.
Test Plan:
- ./tests/fuzzer -s9634 -v
- git diff --check
ZSTDMT_expandJobsTable previously kept worker-capacity comparison, old-table
synchronization teardown, replacement allocation, initialization failure cleanup,
and mask updates in C. That left the remaining job-table lifecycle orchestration
outside the Rust storage leaves. Add a narrow Rust ABI that exchanges only
opaque storage and scalar mask state and accepts C callbacks for the private
synchronization lifecycle.
The adapter frees the old table before replacement, reuses the existing Rust
create/free leaves with the caller's custom memory, destroys a partially
initialized replacement before freeing it, and updates jobIDMask only after
successful initialization. C still owns ZSTDMT_CCtx and job descriptors, so
worker-facing fields and platform synchronization stay outside the Rust ABI.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features --features
compression zstdmt_compress --lib` -- 38 passed.
- `cargo clippy`, `cargo clippy --benches`, and `cargo clippy --tests` -- passed.
- `cargo +nightly fmt --all -- --check` -- passed.
- `make -B -C lib lib-mt` -- passed.
- `make -B -C tests -j2 fullbench poolTests` -- passed.
- `./poolTests` and `./fullbench -i1 -B1000 ../README.md` -- passed.
Keep the LDM limit-table update on the Rust compression path while leaving
match-state ownership in C. Previously, the C block wrapper combined pointer
subtraction, match-state access, and the bounded scalar update. The wrapper
now computes `curr` from `anchor - window.base`, passes `curr` and
`nextToUpdate` through the narrow U32 ABI, and stores Rust's result before the
existing fast-table dispatch.
The Rust leaf uses explicit wrapping arithmetic to preserve the C U32 behavior:
the strict `curr > nextToUpdate + 1024` threshold and the `MIN(512, ...)`
clamp. Focused tests cover the threshold, one-step update, clamp, nonzero
starting point, and arithmetic wraparound.
Test Plan:
- `cargo test zstd_ldm` -- default-feature test-binary link failed because
existing dict-builder C symbols are not linked.
- `cargo test --no-default-features --features compression zstd_ldm` -- passed
(7 tests).
- `make lib-nomt` -- passed.
- `make lib-mt` -- passed.
- `make -C tests test-zstream` -- passed; it emitted the existing
`tests/zstreamtest.c` unterminated-string warning.
- `cargo clippy`, `cargo clippy --benches`, `cargo clippy --tests`,
`cargo +nightly fmt`, then the same clippy sequence -- passed.
The Rust dispatch gate for ZSTD_compress2 and complete-input
ZSTD_compressStream2 calls was selecting compression parameters with an
unknown content size. For the 30-byte level-4 CCtx reuse case that chooses the
dfast strategy, while the actual source size selects greedy; the Rust frame
then emitted different output from the C-owned ZSTD_compressCCtx path.
Pass the actual source size through both helper ABIs. The one-shot and complete
stream entry points now select the same strategy as C, while the existing
sentinel keeps every non-fast/non-dfast configuration on the original C
fallback. The existing fuzzer test 56 exercises the regression without
modifying the shared C test harness.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4560 -t56 -i57 -v` --
passed; the 30-byte test-56 regression now completes.
- `./tests/fuzzer -s4142 -t63 -i64 -v` -- passed all 64 focused cases.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 4,385 first
randomized, and 8,137 new-API randomized cases.
- `make -C tests -j2 test-fuzzer` reaches test 113's flat-dictionary
efficiency assertion with concurrent unstaged LDM changes; test 56 passes.
The window overflow gate mixed two scalar decisions with C-owned pointer and
window state. The early correction test uses the block-start index and must
wait for dictionary invalidation, while the normal fallback uses the block-end
index and the platform-selected ZSTD_CURRENT_MAX threshold. Keeping those
subtractions and the ZSTD_window_t wrapper in C preserves the existing call
contract without making Rust depend on pointer width or private state.
Move the U32 cycle/MAX arithmetic, correction-count scaling, dictionary gate,
and frequent-policy decision into Rust. C passes the two pointer-derived
indices only through scalar results, and supplies the current-max threshold
and active build policy explicitly so 32/64-bit and fuzzing configurations
remain authoritative.
Test Plan:
- `cargo test --manifest-path rust/Cargo.toml --no-default-features
--features compression` -- 348 passed.
- Compression clippy for the library, benches, and tests, followed by
nightly fmt and the repeated three clippy checks -- passed.
- `make -C lib -j2 lib-mt` and `make -C lib -j2 lib-nomt` -- passed.
- `make -C tests -j2 fuzzer` and `./tests/fuzzer -s4142 -t63 -i64 -v` --
passed all 64 cases.
- `make -C tests -j2 test-zstream` -- passed 84 deterministic, 5,477 first
randomized, and 6,902 new-API randomized cases; the existing unterminated
string initializer warning remains.
ddidderr