Commit Graph
29 Commits
Author SHA1 Message Date
ddidderr 24488e4eaa feat(rust): port legacy v0.1 decoder
Port lib/legacy/zstd_v01.c (the frozen zstd v0.1 decoder) to
rust/src/legacy/zstd_v01.rs as the first legacy-format port on the new
scaffolding, and reduce the C file to a declaration-only shim that
keeps its header includes for configuration and platform preprocessor
behavior.

Frozen-decoder policy: zstd_v01.c embeds its own v0.1-era FSE and
Huff0 snapshot, distinct from every other release. The Rust port is a
line-by-line translation with the same table layouts (FSE_DTable as a
u32 header word plus packed newState/symbol/nbBits entries, the Huff0
u16 DTable with byte/nbBits pairs), the same arithmetic including
wrap-around and pointer-comparison quirks (e.g. the offset-vs-base
address check in ZSTD_execSequence), the same internal FSE error space
(size_t)-1..-7, and the same public ZSTD error codes. It reuses no
modern Rust entropy module; its only crate dependency is `errors`,
matching the C file's error_private.h include. The 32-bit-only reload
points are kept as compile-time conditions on usize::BITS.

Symbol takeover boundary: all nine ZSTDv01_* entry points from
zstd_v01.h now come from Rust as context-free #[no_mangle] extern "C"
functions (isError, decompress, decompressDCtx,
findFrameSizeInfoLegacy, createDCtx, freeDCtx, resetDCtx,
nextSrcSizeToDecompress, decompressContinue). zstd_legacy.h only uses
the first four for v0.1; streaming for v0.1-v0.3 intentionally returns
version_unsupported there, unchanged. The ZSTDv01_Dctx struct
definition moves entirely into Rust: C code only ever holds an opaque
pointer (zstd_v01.h forward-declares the type), and the context is
malloc/free-allocated exactly like the C version so create/free may
pair across the language boundary.

Byte-identity verification against the pristine pre-migration C build
(f8745da6, pure C, ZSTD_LEGACY_SUPPORT=1):

- Real v0.1 frames were generated by building the v0.1.0 git tag and
  compressing text, random, and 426 KB multi-block inputs. A one-shot
  ZSTD_decompress harness linked once against the pristine C libzstd.a
  and once against the Rust-backed libzstd.a produced bit-identical
  outputs for all frames.
- A direct ZSTDv01_* probe (one-shot decode, dst-too-small, truncated
  input, bad magic, findFrameSizeInfoLegacy, and the streaming
  continue loop) printed identical results, including exact error
  codes (-70 dstSize_tooSmall, -72 srcSize_wrong, -10 prefix_unknown)
  and identical dBound values.
- zstd -l -v on v0.1 files matches the pristine binary; CLI streaming
  decode of v0.1 fails with the same "Version not supported" in both,
  by design of zstd_legacy.h.

Unit tests embed three v0.1.0-generated fixtures (entropy-coded,
raw-block, and four-block frames) plus the truncation, bad-magic,
small-destination, and streaming-API cases, all asserting the exact C
error codes above. Note that `make -C tests test-legacy` only covers
v0.4+ frames, so the embedded fixtures and the harness comparison are
the actual v0.1 coverage.

Test plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets
  --features legacy-v01 -- -D warnings && cargo test --all-targets
  --features legacy-v01 (127 tests, 9 for v0.1)
- cargo clippy/test --no-default-features --features
  decompression,legacy-v01 (module builds standalone)
- make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests, also with
  ZSTD_LEGACY_SUPPORT=1 (mixed Rust v0.1 + C v0.2-0.7 link)
- make -C tests test-rust-lib-smoke && make -C tests test-legacy
- make -C programs zstd (default and ZSTD_LEGACY_SUPPORT=1); nm shows
  the nine ZSTDv01_* symbols provided by Rust at level 1
- make -C lib libzstd.a ZSTD_LEGACY_SUPPORT=0 (no legacy symbols) and
  meson -Dlegacy_level=1 shared library exporting all nine
2026-07-11 14:25:23 +02:00
ddidderr c0a1b0bae1 build(rust): add legacy feature scaffolding
The legacy decoders (lib/legacy/zstd_v01.c .. zstd_v07.c) are next in
the Rust migration. Each of those files is a frozen snapshot of the
FSE/Huff0 entropy coders and frame logic of one historical release, so
their ports must not reuse the modern Rust entropy modules and must not
share code with each other: outputs and error codes have to stay
byte-identical to the frozen C forever. This commit installs the
build-system scaffolding so seven per-version ports can land
independently, each adding only its own module file plus a one-line
registration in rust/src/legacy/mod.rs.

Cargo grows features legacy-v01 .. legacy-v07. They are never default
features: the C build defaults differ per build system, so each build
system passes the list explicitly, derived from its own legacy
configuration:

- lib/Makefile and programs/Makefile map ZSTD_LEGACY_SUPPORT=N to the
  features for versions N..7 (0 disables legacy), mirroring the
  ZSTD_LEGACY_FILES selection in lib/libzstd.mk.
- tests/Makefile always enables all seven features because its
  ZSTDLEGACY_FILES wildcard compiles every lib/legacy/*.c regardless of
  the dispatch level.
- build/meson maps legacy_level exactly like the makefiles; build/cmake
  enables all seven whenever ZSTD_LEGACY_SUPPORT is ON because it
  always compiles all seven C files (ZSTD_LEGACY_LEVEL only selects the
  C dispatch).

Every build system also encodes the legacy selection in the Rust target
directory name (e.g. c1-d1-default-legacy5), for the same reason the
HUF mode is encoded there: a cached archive built for one configuration
must never be linked into a build expecting another. In tests/Makefile
the legacy level additionally flows into the existing HUF C-mode stamp,
so the flat C test objects (which bake -DZSTD_LEGACY_SUPPORT into the
dispatch) are rebuilt whenever the level changes. In programs/Makefile
the compress-only, decompress-only, and CLI archives keep
level-independent directories (RUST_HUF_MODE) because they are only
linked into ZSTD_LEGACY_SUPPORT=0 program variants and carry no legacy
features.

A feature whose version has not been ported yet gates nothing: the
module registration in rust/src/legacy/mod.rs is added by each port,
so enabling e.g. legacy-v05 today simply leaves that decoder in C.
This is what makes mixed C/Rust legacy levels link cleanly while the
seven ports land in any order.

Test plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets
  -- -D warnings && cargo test --all-targets
- cargo clippy with --no-default-features --features
  decompression,legacy-v01 and with all seven legacy features
- make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests
- make -C tests test-rust-lib-smoke; make -C tests test-legacy
- make -C lib libzstd.a with ZSTD_LEGACY_SUPPORT=0, 1 and default (5)
- cmake configure and meson setup (including -Dlegacy_level=1) emit the
  expected --features lists and legacy-suffixed target directories
2026-07-11 14:24:47 +02:00
ddidderr 959e485201 feat(rust): port frame serialization
Continue the incremental zstd_compress.c migration with its frame
serialization leaves: ZSTD_writeFrameHeader(), the public
ZSTD_writeSkippableFrame() ABI, and ZSTD_writeLastEmptyBlock() now live
in rust/src/zstd_compress_frame.rs.

ZSTD_writeFrameHeader() reads five fields out of ZSTD_CCtx_params, whose
layout is private to zstd_compress.c and sensitive to build
configuration.  Rather than mirror that structure in Rust, the C
function keeps its original static signature and forwards the scalar
fields to ZSTD_rust_writeFrameHeader(), so no parameter-structure layout
crosses the language boundary.  The other two functions take only
pointer/size arguments and are exported directly, replacing their C
bodies outright.

Behavior differences are limited to hardening in release builds: the
Rust leaf clamps a window-size shift that C would leave undefined for a
malformed windowLog, and ZSTD_writeSkippableFrame() rejects a srcSize +
header overflow instead of comparing against a wrapped sum.  Both paths
are unreachable through validated callers, so compressed output is
byte-identical.

Remaining zstd_compress.c work: parameter selection and validation,
context lifecycle, block dispatch, dictionary loading, and the streaming
state machine.

Test plan:
- cd rust && cargo fmt --check && cargo clippy --all-targets -- -D
  warnings && cargo test --all-targets (118 tests, includes new frame
  header/skippable/last-block reference vectors)
- make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests
- make -C tests test-rust-lib-smoke
2026-07-11 09:16:06 +02:00
ddidderr fe7e24c770 feat(rust): migrate high-level runtime paths
Move long-distance matching and high-level decompression from C shims into
Rust. The decoder now owns context, dictionary, parameter, one-shot, and
buffered streaming state while C retains allocation/configuration, legacy,
and trace leaves.

Move CLI parsing, safety policy, and dispatch into a separate Rust static
archive. Keeping it separate prevents library builds from retaining FIO
symbols, while C continues to own file opening, replacement, and I/O.
Program targets now select matching compression/decompression archives.

The remaining C boundary is intentional: high-level compression, optimal
parsing, dictionary building, legacy callbacks, and CLI file I/O still need
migration.

Test Plan:
- cargo test --all-targets (native and i686)
- cargo test --all-targets in rust/cli (native and i686)
- CLI crate compression-only and decompression-only feature tests
- native and i686 fuzzer/zstreamtest runs, plus legacy and dictionary tests
- ZSTD_C_PREDICT and ZSTD_HEAPMODE=0 fuzzer coverage
- library, dynamic-link, and program-target build/round-trip matrix

Refs: rust/README.md
2026-07-11 09:03:41 +02:00
ddidderr 031592da1e feat(rust): port optimal binary-tree updates
Move the binary-tree table maintenance shared by optimal compression strategies
into Rust. A small C projection preserves the private match-state boundary;
the dynamic-programming parser and match selection remain C-owned for now.

The Rust implementation covers prefix and external-dictionary tree updates and
preserves the optional C predictor path. The component map now distinguishes
this migrated tree layer from the remaining optimal parser.

Test Plan:
- cargo test --all-targets
- cargo test --target i686-unknown-linux-gnu --all-targets
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt
- make -B -C tests -j2 fuzzer zstreamtest invalidDictionaries
- ./tests/fuzzer -s5346 -i1 --no-big-tests
- ./tests/zstreamtest -i3000 -s334462
- ./tests/invalidDictionaries
- make -B -C tests fuzzer32 MOREFLAGS=-DZSTD_C_PREDICT
- ./tests/fuzzer32 -s5346 -i1 --no-big-tests
- byte-compare C and Rust-control CLI output at levels 13, 16, 19, and 22
  for random, patterned, zero, and dictionary inputs with and without
  ZSTD_C_PREDICT

Refs: rust/README.md
2026-07-11 08:22:41 +02:00
ddidderr e5eebd4892 feat(rust): port compressed block decoding
Move literal decoding, sequence-table construction, FSE sequence decoding, and
sequence execution into Rust. The C shim retains ownership of the configured
decoder context and passes only the leaf state needed by the block codec.

Correct the two high offset-code bases while porting the tables. Their prior
values made valid large-window streams decode as corrupted data; the new unit
test fixes the exact values and a native zstream regression exercises them.
High-level frame and streaming context control remains C for now.

Test Plan:
- cargo test --all-targets
- cargo test --target i686-unknown-linux-gnu --all-targets
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt
- make -B -C tests -j2 fuzzer zstreamtest invalidDictionaries poolTests
- ./tests/fuzzer -s5346 -i1 --no-big-tests
- ./tests/zstreamtest -i3000 -s334462
- ./tests/invalidDictionaries
- timeout 20s stdbuf -oL ./tests/poolTests

Refs: rust/README.md
2026-07-11 08:07:40 +02:00
ddidderr 7f16a07375 feat(rust): port lazy block matching
Move greedy, lazy, lazy2, and binary-tree block matchers into Rust. A narrow
C state projection preserves configuration-specific context layout while Rust
owns hash-chain, row-based, attached-dictionary, and external-dictionary
searches and sequence-store updates.

The component map now records the lazy family as migrated; the optimized
matcher and high-level contexts remain outside this commit.

Test Plan:
- cargo test --all-targets
- cargo test --target i686-unknown-linux-gnu --all-targets
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt
- make -B -C tests -j2 fuzzer zstreamtest invalidDictionaries poolTests
- ./tests/fuzzer -s5346 -i1 --no-big-tests
- ./tests/zstreamtest -i3000 -s334462
- ./tests/invalidDictionaries
- timeout 20s stdbuf -oL ./tests/poolTests

Refs: rust/README.md
2026-07-11 08:06:30 +02:00
ddidderr f9adcf6aef feat(rust): port double-fast block matching
Move the two-table fast match finder into Rust while retaining a small C
projection of the private match-state. The Rust implementation handles normal,
attached-dictionary, and external-dictionary matching without exposing the
full C context ABI. The C entry points and build-time exclusion guards remain
unchanged for configured consumers.

Document the migrated matcher in the Rust component map and narrow the
remaining-C boundary accordingly.

Test Plan:
- cargo test --all-targets
- cargo test --target i686-unknown-linux-gnu --all-targets
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt
- make -B -C tests -j2 fuzzer zstreamtest invalidDictionaries poolTests
- ./tests/fuzzer -s5346 -i1 --no-big-tests
- ./tests/zstreamtest -i3000 -s334462
- ./tests/invalidDictionaries
- timeout 20s stdbuf -oL ./tests/poolTests

Refs: rust/README.md
2026-07-11 08:05:25 +02:00
ddidderr d316a14ec3 feat(rust): export public sequence helpers
Move the context-free ZSTD_sequenceBound and ZSTD_mergeBlockDelimiters APIs
into the Rust compression helper module. Preserve the public sequence ABI,
block-delimiter semantics, and unsigned literal-length wrapping behavior while
leaving context-driven sequence generation in C.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test zstd_compress_api::tests -- --nocapture
- cargo test --target i686-unknown-linux-gnu zstd_compress_api::tests

Refs: Rust compression-bound export 3e2635f4
2026-07-10 22:51:07 +02:00
ddidderr 3e2635f45d feat(rust): export compression bound from Rust
Move the context-free public ZSTD_compressBound ABI into Rust while leaving
the stateful compressor context in C. The Rust implementation preserves the
size-width-specific input limit, macro arithmetic, and srcSize_wrong error
contract used by C callers.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test zstd_compress_api::tests -- --nocapture
- cargo test --target i686-unknown-linux-gnu zstd_compress_api::tests
- make -B -C lib -j2 libzstd.a
- fuzzer -s5346 -i1 --no-big-tests

Refs: public ZSTD_COMPRESSBOUND macro in lib/zstd.h
2026-07-10 22:49:19 +02:00
ddidderr 74140c318f feat(rust): port fast block matching
Move fast hash-table filling and no-dictionary, external-dictionary, and
attached-CDict match loops into Rust. The C shim keeps `ZSTD_MatchState_t`
opaque, extracts only its needed fields, and asserts the mirrored SeqStore
leaf layout so existing compression dispatch remains ABI-compatible.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test zstd_fast::tests -- --nocapture
- cargo test --target i686-unknown-linux-gnu zstd_fast::tests -- --nocapture
- byte-identical C-control frames for dictionary and streaming variants
- fuzzer -s2066 -i5 --no-big-tests
- invalidDictionaries and zstreamtest -i1

Refs: Rust superblock port fde1d70c
2026-07-10 22:46:15 +02:00
ddidderr fde1d70cc5 feat(rust): port superblock compression
Move subblock partitioning, literal and sequence section emission, entropy
fallback, and repcode repair into Rust. Keep a narrow C wrapper to read the
opaque compression context and invoke the existing C entropy-stat builder;
the Rust side owns only verified C-shaped leaf layouts.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test zstd_compress_superblock::tests -- --nocapture
- cargo test --target i686-unknown-linux-gnu zstd_compress_superblock::tests
- byte-exact C control across 16 target-size and input-shape cases
- fuzzer, zstreamtest, invalidDictionaries, and bounded native test matrix

Refs: Rust sequence entropy port 887af204
2026-07-10 22:45:22 +02:00
ddidderr 887af204df feat(rust): port sequence entropy encoding
Move FSE table-costing, table construction, encoding selection, and sequence
bitstream emission into Rust while retaining the existing C internal-header
boundary. The shim preserves all five C ABI entry points, so the surrounding
compressor continues to consume C-shaped sequence and entropy-table storage.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test --all-targets
- cargo test --target i686-unknown-linux-gnu zstd_compress_sequences::tests
- C-vs-Rust differential across 163 payload cases on x86_64 and i686
- fuzzer, zstreamtest, and invalidDictionaries

Refs: Rust FSE compression port 5f9d607d
2026-07-10 22:44:39 +02:00
ddidderr da617508b9 test(pool): run C pool tests through the Rust implementation
Link poolTests against the multithreaded object set and Rust static archive,
so its C callbacks exercise the migrated pool rather than compiling a private
C implementation. Preserve ZSTD_MULTITHREAD for the test translation unit;
without it, its pthread test helpers become no-ops while the Rust pool runs
concurrently. Add a Rust regression for draining a small queue after reducing
the worker limit.

Test Plan:
- cargo clippy
- cargo clippy --benches
- cargo clippy --tests
- cargo +nightly fmt
- cargo test shrinking_the_limit_keeps_draining_a_small_queue -- --nocapture
- make -B -C tests poolTests && timeout 20s stdbuf -oL ./tests/poolTests

Refs: Rust pool migration 26b5e202
2026-07-10 22:38:07 +02:00
ddidderr 6a762660b7 feat(rust): port compression literal blocks
Move raw, RLE, and Huffman literal-section encoding into the Rust
compatibility archive.  The surrounding C block compressor continues to pass
its existing entropy workspace and Huffman-table state through the unchanged
internal ABI.

The port preserves literal header choices, compression-gain decisions,
repeat-table transitions, and 1X/4X encoder selection.  It lets ordinary C
compression paths exercise Rust code without widening this commit into the
remaining frame compressor.

Test Plan:
- cargo clippy, cargo clippy --benches, cargo clippy --tests, and nightly fmt
- cargo test --all-targets (94 passed) and feature-only cargo checks
- native fuzzer, zstreamtest, invalidDictionaries, and CLI round trip
- compare literals and next-table bytes with pristine C on x86_64 and i686

Refs: rust/README.md compression-primitives map
2026-07-10 22:07:27 +02:00
ddidderr 0a6a5f5267 feat(rust): port Huffman compression
Move Huffman table construction, table serialization, and one- and
four-stream payload encoding from huf_compress.c into the Rust compatibility
archive.  The declaration-only C shim preserves the existing internal ABI,
so the remaining C compressor can call the migrated implementation unchanged.

The translation keeps CTable layouts, workspace checks, repeat-table
selection, and bitstream output compatible with the original encoder.  This
lets native C consumers exercise the Rust implementation through libzstd.

Test Plan:
- cargo clippy, cargo clippy --benches, and cargo clippy --tests
- cargo +nightly fmt and cargo test --all-targets (88 passed)
- cargo check --no-default-features --features compression
- rebuild and run tests fuzzer, zstreamtest, and invalidDictionaries
- compare tables, headers, and 1X/4X streams with a renamed pristine C build

Refs: rust/README.md component map
2026-07-10 21:54:23 +02:00
ddidderr d784406374 build(rust): link migrated code into native libraries
Build feature-matched Rust archives for the native Make library path. Flatten
their members into libzstd.a so static consumers resolve C-to-Rust and
Rust-to-C references in one archive, and whole-archive link shared outputs so
every migrated public ABI export remains available.

Rust compression and decompression features now mirror the C partial-library
switches. This prevents compression-only artifacts from retaining DDict
references to decompression-only C helpers.

Add a C archive smoke test that round-trips data and exercises the DDict
lifecycle through lib/libzstd.a, rather than direct test-object links.

Test Plan:
- cargo fmt, strict Clippy, cargo test --all-targets, release build
- Rust checks with compression-only, decompression-only, and no features
- full, partial, forced-HUF, and 32-bit static/shared Make library probes
- test-rust-lib-smoke, fuzzer, zstreamtest, and invalidDictionaries

Refs: rust/README.md
2026-07-10 21:41:22 +02:00
ddidderr 24d01b92fb feat(rust): port compression block pre-splitting
Move ZSTD_splitBlock into Rust while retaining its caller-owned workspace
contract. The implementation preserves the C fingerprint sampling heuristic
and calls the migrated histogram primitive without adding a hot-path
allocation.

Reference-vector and randomized differential tests compare every split level
with the original C translation unit. The C source remains as a declaration
shim so existing source lists resolve the Rust ABI during the migration.

Test Plan:
- cargo fmt, cargo clippy, cargo clippy --benches, cargo clippy --tests
- cargo test --all-targets and cargo build --release
- cargo test/build --target i686-unknown-linux-gnu
- 264-case original-C/Rust differential harness
- C fuzzer, zstreamtest, and fuzzer32 smoke runs

Refs: rust/README.md
2026-07-10 21:28:56 +02:00
ddidderr ca70ca8061 feat(rust): port decode dictionaries
Move ZSTD_DDict allocation, ownership, construction, and public ABI exports
into Rust. The C decoder keeps ZSTD_loadDEntropy until its table loader moves.

The C shim now exposes C-computed DCtx field offsets. This keeps dictionary
state correct across conditional C layouts, including fuzz fields and 32-bit
static-BMI2 builds, without duplicating decoder-context layout in Rust.

Test Plan:
- cargo fmt, cargo clippy, cargo clippy --benches, cargo clippy --tests
- cargo test --all-targets and cargo build --release
- cargo test/build --target i686-unknown-linux-gnu
- C fuzzer, zstreamtest, invalidDictionaries, and fuzzer32 smoke runs
- fuzz-enabled and i686 -mbmi2 C/Rust dictionary roundtrip harnesses

Refs: rust/README.md
2026-07-10 21:20:50 +02:00
ddidderr 65afd94867 fix(build): rebuild C outputs on HUF mode changes
Mode-specific Rust archives exposed the test makefiles' flat C-object cache:
switching from default to forced HUF mode could relink stale default C objects
with a newly built forced Rust archive. That hybrid has incompatible decoder
table expectations and can corrupt dictionary decompression.

Track each makefile's effective HUF mode with an empty archive stamp. It is a
safe normal linker prerequisite, and every mode transition advances its mtime
so cached C objects and direct-source binaries rebuild while unchanged modes
remain incremental.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets
- Default -> forced-X1 -> default fuzzer builds without -B, each running
  ./fuzzer -s5346 -i1 --no-big-tests
- Equivalent zstd-small default/forced/default rebuild and --version checks

Refs: rust/README.md
Fixes: d89ebb31
2026-07-10 21:08:14 +02:00
ddidderr 27932113fd feat(rust): port Huffman decompression
Move Huffman decoding-table construction and X1/X2 single- and four-stream
decoding into Rust. The original C translation unit is now a declaration shim,
so unchanged C callers and tests resolve the public decoder ABI from the Rust
archive.

The portable implementation preserves the C workspace and error contracts,
retains the existing assembly-only internal fast loops, and mirrors the C
32-bit decode cadence so X2 writes remain within its required output window.
Forced X1/X2 modes follow the matching archive configuration from the build
integration.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets for default, forced X1, and forced X2 modes
- cargo build and cargo test for i686-unknown-linux-gnu in all three modes
- Strict normal, X1, and X2 C shim compilation
- Original fuzzer and fullbench compatibility targets

Refs: rust/README.md
Depends-on: d89ebb31
2026-07-10 20:48:51 +02:00
ddidderr d89ebb31e3 build(rust): select archives by target and HUF mode
Derive the Rust static-library configuration from the effective C HUF decoder
flags and place each mode in its own Cargo target directory. This prevents a
forced X1/X2 C build from silently reusing the default Rust archive.

The test and program makefiles now also build and link an i686 Rust archive
for their 32-bit targets, including zstd32. zstd-small retains its existing
default configuration while preserving an explicitly requested HUF mode.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets for default, forced X1, and forced X2 modes
- cargo build --release --target i686-unknown-linux-gnu for each mode
- make -n checks for default, forced, direct -D, and 32-bit test/CLI targets

Refs: rust/README.md
2026-07-10 20:46:23 +02:00
ddidderr 58a50db413 test(rust): make entropy tests portable to 32-bit
Use an explicitly 64-bit literal before narrowing it in the common-byte test,
and derive the FSE compression-bound expectation from `size_of::<usize>()`.
The tests now exercise the same public contracts on 32-bit and 64-bit targets
instead of embedding a 64-bit-only compile-time assumption.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --target i686-unknown-linux-gnu --lib
- cargo test --target i686-unknown-linux-gnu --lib --features \
  huf-force-decompress-x1
- cargo test --target i686-unknown-linux-gnu --lib --features \
  huf-force-decompress-x2

Refs: rust/README.md
2026-07-10 20:44:02 +02:00
ddidderr 5f9d607ddd feat(rust): port FSE compression primitives
Move FSE normalization, count-header writing, compression-table construction,
and stream encoding from C to Rust. The original C source remains as a
header-only shim, preserving the public header configuration while native
consumers receive the Rust archive exports.

This completes both codec directions for FSE in the migration crate and lets
the unchanged C compatibility tests exercise the Rust encoder implementation.

Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets
- cargo build --release
- Compile the C shim with strict warnings enabled
- 1,000-case pristine-C differential for tables, headers, and payloads
- ./tests/fuzzer -v -T10s
- ./tests/decodecorpus -t -T5

Refs: rust/README.md
2026-07-10 20:32:18 +02:00
ddidderr ebc43676b7 build(rust): rebuild the archive for C test and CLI links
Make Rust sources normal prerequisites of native C test executables and CLI
builds. A changed Rust module now rebuilds libzstd_rs.a and relinks the target,
instead of relying on a manually prepared archive that can silently be stale.

The archive is linked after C objects, allowing migrated C shims to resolve
their Rust ABI symbols while remaining compatible with the existing makefile
flows. The migration guide now documents this behavior.

Test Plan:
- make -B -C tests fuzzer
- ./tests/fuzzer -i1 --no-big-tests
- make -B -C programs zstd
- ./programs/zstd --version

Refs: rust/README.md
2026-07-10 20:11:42 +02:00
ddidderr 158cd680d9 feat(rust): port histogram counting primitives
Move byte histogram counting into Rust for FSE and Huffman compression callers.
The implementation preserves the C workspace contract, checked alphabet path,
fast path, empty-input behavior, and C ABI while the original C source becomes
a header-only compatibility shim.

Focused tests cover count accumulation, invalid alphabets, workspace failures,
and striped large inputs. Higher-level compression remains C for now.

Test Plan:
- cargo fmt --check
- cargo test --all-targets
- cargo clippy --all-targets -- -D warnings
- cargo build --release
- make -C tests fuzzer
- ./tests/fuzzer -i1 --no-big-tests
- compile hist.c with -Werror and -Wredundant-decls

Refs: rust/README.md
2026-07-10 20:06:25 +02:00
ddidderr 26b5e202ee feat(rust): port worker pool and pthread wrappers
Replace the common worker pool and debug pthread wrappers with ABI-compatible
Rust implementations. The pool preserves bounded-queue, resize, drain-on-free,
custom-allocator, and no-thread behavior while C supplies only the build-time
multithreading configuration bit.

This moves runtime support without changing C tests or public headers. Codec
and CLI implementations remain C while their dependencies are migrated.

Test Plan:
- cargo fmt --check
- cargo test --all-targets
- cargo clippy --all-targets -- -D warnings
- cargo build --release
- make -C tests poolTests
- ./tests/poolTests
- compile pool and threading shims with -Werror in MT and no-thread modes

Refs: rust/README.md
2026-07-10 20:05:35 +02:00
ddidderr a0f2b2a14c feat(rust): port FSE entropy decoding
Move FSE normalized-count parsing, Huffman statistics decoding, FSE table
construction, and FSE stream decompression into Rust. The C source files now
only retain the headers needed by the existing build configuration.

The implementation keeps the public C ABI and verifies C-generated balanced,
skewed, and Huffman-statistics streams. Compression and the higher-level frame
decoder remain C for now.

Test Plan:
- cargo fmt --check
- cargo test --all-targets
- cargo clippy --all-targets -- -D warnings
- cargo build --release
- compile both C shims with -Werror and -Wredundant-decls

Refs: rust/README.md
2026-07-10 20:04:26 +02:00
ddidderr 089f8e5b4d feat(rust): add common primitive compatibility layer
Introduce the Rust static library and move the shared byte, bitstream, CPU,
error, xxHash, debug, and public-common implementations into it. Thin C shims
preserve the existing header-driven C build while original tests link the Rust
archive.

This establishes the ABI-safe foundation for later codec and CLI ports;
entropy coding, runtime support, codecs, dictionaries, and the CLI remain C.
The top-down migration map documents that boundary and its validation path.

Test Plan:
- cargo fmt --check
- cargo test --all-targets
- cargo clippy --all-targets -- -D warnings
- cargo build --release

Refs: rust/README.md
2026-07-10 20:02:17 +02:00