ddidderr
da0e2c8380
`make all` exercises substantially more than the default zstd binary. It also builds compression-only and decompression-only archives, older contrib tools that compile program C shims directly, and the single-file amalgamations. The Rust migration had changed symbol ownership without updating every one of those feature and link boundaries. The first failure came from `fileio_asyncio`: it is always compiled, but its codec bindings unconditionally referenced both `zstd_compress` and `zstd_decompress`. A compression-only archive therefore required disabled decoder modules, and the decompression-only configuration had the symmetric problem. Gate the concrete codec imports, callback types, helpers, and exports with their Cargo features. Keep the format probe compilable without the legacy decoder predicate when decompression is disabled. Once that boundary compiled, the remaining `make all` paths exposed related integration gaps. Move the C decompression projection declarations out of the compression preprocessor block, while leaving destination callback types shared. Link the Rust CLI helpers archive into zlibWrapper, pzstd, and largeNbDicts, whose util/time/data-generator C files are now declaration shims rather than implementations. The generated single-file C sources also call Rust-owned symbols now. Build and link an appropriately featured Rust archive in their native smoke tests, and include the pool configuration shim in the decoder case. The full amalgamation combines `zstd_lazy.c` and `zstd_opt.c` into one translation unit, so guard their otherwise translation-unit-local dictionary mode enum against duplicate definition. A Rust-backed amalgamation is no longer a standalone Emscripten input. Remove the obsolete emcc/Docker path and report that limitation explicitly; restoring the WebAssembly smoke test requires a Rust WebAssembly archive and a defined cross-language amalgamation contract. Test Plan: - `cargo check --manifest-path rust/Cargo.toml --no-default-features --features compression` -- passed - `cargo check --manifest-path rust/Cargo.toml --no-default-features --features decompression` -- passed - `sh -n build/single_file_libs/build_decoder_test.sh build/single_file_libs/build_library_test.sh` -- passed - `make all` -- passed, including native single-file and seekable-format tests - `git diff --cached --check` -- passed
largeNbDicts
largeNbDicts is a benchmark test tool
dedicated to the specific scenario of
dictionary decompression using a very large number of dictionaries.
When dictionaries are constantly changing, they are always "cold",
suffering from increased latency due to cache misses.
The tool is created in a bid to investigate performance for this scenario, and experiment mitigation techniques.
Command line :
largeNbDicts [Options] filename(s)
Options :
-z : benchmark compression (default)
-d : benchmark decompression
-r : recursively load all files in subdirectories (default: off)
-B# : split input into blocks of size # (default: no split)
-# : use compression level # (default: 3)
-D # : use # as a dictionary (default: create one)
-i# : nb benchmark rounds (default: 6)
--nbBlocks=#: use # blocks for bench (default: one per file)
--nbDicts=# : create # dictionaries for bench (default: one per block)
-h : help (this text)
Advanced Options (see zstd.h for documentation) :
--dedicated-dict-search
--dict-content-type=#
--dict-attach-pref=#