Move long-distance matching and high-level decompression from C shims into
Rust. The decoder now owns context, dictionary, parameter, one-shot, and
buffered streaming state while C retains allocation/configuration, legacy,
and trace leaves.
Move CLI parsing, safety policy, and dispatch into a separate Rust static
archive. Keeping it separate prevents library builds from retaining FIO
symbols, while C continues to own file opening, replacement, and I/O.
Program targets now select matching compression/decompression archives.
The remaining C boundary is intentional: high-level compression, optimal
parsing, dictionary building, legacy callbacks, and CLI file I/O still need
migration.
Test Plan:
- cargo test --all-targets (native and i686)
- cargo test --all-targets in rust/cli (native and i686)
- CLI crate compression-only and decompression-only feature tests
- native and i686 fuzzer/zstreamtest runs, plus legacy and dictionary tests
- ZSTD_C_PREDICT and ZSTD_HEAPMODE=0 fuzzer coverage
- library, dynamic-link, and program-target build/round-trip matrix
Refs: rust/README.md
Mode-specific Rust archives exposed the test makefiles' flat C-object cache:
switching from default to forced HUF mode could relink stale default C objects
with a newly built forced Rust archive. That hybrid has incompatible decoder
table expectations and can corrupt dictionary decompression.
Track each makefile's effective HUF mode with an empty archive stamp. It is a
safe normal linker prerequisite, and every mode transition advances its mtime
so cached C objects and direct-source binaries rebuild while unchanged modes
remain incremental.
Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets
- Default -> forced-X1 -> default fuzzer builds without -B, each running
./fuzzer -s5346 -i1 --no-big-tests
- Equivalent zstd-small default/forced/default rebuild and --version checks
Refs: rust/README.md
Fixes: d89ebb31
Derive the Rust static-library configuration from the effective C HUF decoder
flags and place each mode in its own Cargo target directory. This prevents a
forced X1/X2 C build from silently reusing the default Rust archive.
The test and program makefiles now also build and link an i686 Rust archive
for their 32-bit targets, including zstd32. zstd-small retains its existing
default configuration while preserving an explicitly requested HUF mode.
Test Plan:
- cargo clippy; cargo clippy --benches; cargo clippy --tests
- cargo +nightly fmt, then repeat the Clippy checks
- cargo test --all-targets for default, forced X1, and forced X2 modes
- cargo build --release --target i686-unknown-linux-gnu for each mode
- make -n checks for default, forced, direct -D, and 32-bit test/CLI targets
Refs: rust/README.md
Make Rust sources normal prerequisites of native C test executables and CLI
builds. A changed Rust module now rebuilds libzstd_rs.a and relinks the target,
instead of relying on a manually prepared archive that can silently be stale.
The archive is linked after C objects, allowing migrated C shims to resolve
their Rust ABI symbols while remaining compatible with the existing makefile
flows. The migration guide now documents this behavior.
Test Plan:
- make -B -C tests fuzzer
- ./tests/fuzzer -i1 --no-big-tests
- make -B -C programs zstd
- ./programs/zstd --version
Refs: rust/README.md
the purpose of --ultra is to make the user explicitly opt-in
to generate very large window size (> 8 MB).
The agreement to generate very large window size is already implicit
when selecting --long or --patch-from.
Consequently, `--ultra ` is automatically enabled when `--long` or `--patch-from` is set.
Some older Android libc implementations don't support `fseeko` or `ftello`.
This commit adds a new compile-time macro `LIBC_NO_FSEEKO` as well as a usage in CMake for old Android APIs.
Do some include shuffling for `**.h` files within lib, programs, tests, and zlibWrapper.
`lib/legacy` and `lib/deprecated` are untouched.
`#include`s within `extern "C"` blocks in .cpp files are untouched.
todo: shuffling for `xxhash.h`
* Change CLI to employ multithreading by default
* Document changes to benchmarking, print number of threads for display level >= 4, and add lower bound of 1 for the default number of threads
following notification from @rainbowball.
fix#4186.
Note: there is currently no QNX compilation test in CI
so this is a "blind" fix,
and this target can be silently broken again in the future.
update the man page in troff format,
and the README with latest `--help` content and complementary details about benchmark mode.
also: display level 0 when doing decompression benchmark
After a regrettable update,
the benchmark module ended up reloading sources for every compression level.
While the delay itself is likely torelable,
the main issue is that the `--quiet` mode now also displays a loading summary between each compression line.
This wasn't the original intention, which is to produce a compact view of all compressions.
This is fixed in this version,
where sources are loaded only once, for all compression levels,
and loading summary is only displayed once.
The -Werror=missing-profile caused thread/zlib/lzma/lz4 detection failure
when build with profile-use, thus caused ZSTD_MULTITHREAD etc. is not
defined for profile-use, then there will be many profile mismatch information
in output and the final binary reports run error sometimes as below:
Error : ZSTD_CCtx_setParameter(ctx, ZSTD_c_nbWorkers, adv->nbWorkers) failed : Unsupported parameter
Signed-off-by: Xionghu Luo <xionghuluo@tencent.com>
Using '--single-thread' with '--patch-from' on compression levels above 15 will lead to significantly worse compression ratios.
Corrected the man page not suggest anymore to do this.
- switched the patter and input of $filter into the right places
- added pattern wildcard to MSYS_NT & CYGWIN_NT as they change with windows versions
- correctly identify MSYS2, even in an env like MINGW64
ddidderr
