ddidderr
039dec6c68
Continue the incremental zstd_compress.c migration with its sequence statistics and seqStore entropy-compression layer. The following now live in rust/src/zstd_compress_stats.rs: - ZSTD_seqToCodes(), exported under its original name because the C dictionary builder (zdict.c) and decodecorpus link against it, - ZSTD_buildSequencesStatistics() and its dummy variant, whose result struct no longer crosses the language boundary, - ZSTD_entropyCompressSeqStore_internal(), _wExtLitBuffer(), and ZSTD_entropyCompressSeqStore(), - ZSTD_buildBlockEntropyStats() with its literals/sequences helpers, - ZSTD_copyBlockSequences() and the ZSTD_updateRep() rules it shares with the superblock writer. Boundary: ZSTD_CCtx and ZSTD_CCtx_params stay private to C. These paths read exactly two parameter fields, so the C shims keep the original static/exported function names and forward the strategy and ZSTD_literalsCompressionIsDisabled() as int scalars alongside the seqStore and entropy-table leaves. The leaf layouts (SeqDef, SeqStore_t, ZSTD_hufCTables_t, ZSTD_fseCTables_t, entropy metadata, SeqCollector, ZSTD_Sequence) are pinned by compile-time asserts in zstd_compress.c and by both-pointer-width layout tests in Rust. ZSTD_buildSeqStore, block dispatch/splitting, and the block-size estimation helpers remain C for a later slice. The superblock module previously round-tripped through the C export of ZSTD_buildBlockEntropyStats and mirrored the entropy leaf structs privately. It now calls the crate-internal builder directly, and the shared struct definitions moved to zstd_compress_stats; consequently ZSTD_rust_compressSuperBlock() takes the two parameter scalars instead of an opaque ZSTD_CCtx_params pointer, extracted by its C shim. Its layout and repcode tests moved with the definitions. One C helper family gets no shim: the static ZSTD_entropyCompressSeqStore_wExtLitBuffer() had a single caller and was folded into the Rust implementation. Byte-identity was verified against the pre-change compressor: COPYING, datagen -g5000000 -s7, and datagen -g300000 -s21 -P90, each at levels 1/3/9/19 and --fast=5, plus a superblock-heavy pass at level 19 with --target-compressed-block-size=1024; all 18 frames are byte-identical, covering the repeat-mode state machine, longOffsets, RLE/raw fallback, and dstSize_tooSmall paths through the block splitter and superblock. Test plan: - cd rust && cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test --all-targets (131 tests: new reference vectors for seqToCodes, RLE table headers, empty-seqStore repeat copies, literal-stats type selection, and repcode resolution in copyBlockSequences) - cargo build --release --no-default-features --features compression and --features decompression - make -C tests fuzzer && ./tests/fuzzer -i1 --no-big-tests (covers ZSTD_generateSequences and ZDICT training over the Rust seqToCodes) - make -C tests test-rust-lib-smoke - ./tests/zstreamtest -i1 and ./tests/zstreamtest --newapi -t1 -i1 - ./tests/decodecorpus -t -T1s (decodecorpus links Rust seqToCodes) - /tmp/ref vs /tmp/got frame diff as described above: byte-identical
56 lines
1.4 KiB
Rust
56 lines
1.4 KiB
Rust
#![allow(clippy::missing_safety_doc)]
|
|
|
|
pub mod bits;
|
|
pub mod bitstream;
|
|
pub mod common;
|
|
pub mod cpu;
|
|
pub mod debug;
|
|
pub mod entropy_common;
|
|
pub mod errors;
|
|
#[cfg(feature = "compression")]
|
|
pub mod fse_compress;
|
|
pub mod fse_decompress;
|
|
#[cfg(feature = "compression")]
|
|
pub mod hist;
|
|
#[cfg(feature = "compression")]
|
|
pub mod huf_compress;
|
|
#[cfg(feature = "decompression")]
|
|
pub mod huf_decompress;
|
|
pub mod mem;
|
|
pub mod pool;
|
|
pub mod threading;
|
|
pub mod xxhash;
|
|
pub mod zstd_common;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_api;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_frame;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_literals;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_params;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_sequences;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_stats;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_compress_superblock;
|
|
#[cfg(feature = "decompression")]
|
|
pub mod zstd_ddict;
|
|
#[cfg(feature = "decompression")]
|
|
pub mod zstd_decompress;
|
|
#[cfg(feature = "decompression")]
|
|
pub mod zstd_decompress_block;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_double_fast;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_fast;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_lazy;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_ldm;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_opt_tree;
|
|
#[cfg(feature = "compression")]
|
|
pub mod zstd_presplit;
|