Commit Graph
27 Commits
Author SHA1 Message Date
ddidderr 24d01b92fb feat(rust): port compression block pre-splitting
Move ZSTD_splitBlock into Rust while retaining its caller-owned workspace
contract. The implementation preserves the C fingerprint sampling heuristic
and calls the migrated histogram primitive without adding a hot-path
allocation.

Reference-vector and randomized differential tests compare every split level
with the original C translation unit. The C source remains as a declaration
shim so existing source lists resolve the Rust ABI during the migration.

Test Plan:
- cargo fmt, cargo clippy, cargo clippy --benches, cargo clippy --tests
- cargo test --all-targets and cargo build --release
- cargo test/build --target i686-unknown-linux-gnu
- 264-case original-C/Rust differential harness
- C fuzzer, zstreamtest, and fuzzer32 smoke runs

Refs: rust/README.md
2026-07-10 21:28:56 +02:00
Yann Collet 37706a677c added a test
test both that the new parameter works as intended,
and that the over-split protection works as intended
2024-10-28 16:31:15 -07:00
Yann Collet e557abc8a0 new block splitting variant _fromBorders
less precise but still suitable for `fast` strategy.
2024-10-25 16:13:55 -07:00
Yann Collet da2c0dffd8 add faster block splitting heuristic, suitable for dfast strategy 2024-10-24 14:37:00 -07:00
Yann Collet 2366a87ddc fix minor visual conversion warning 2024-10-24 13:38:12 -07:00
Yann Collet 326c45bb8e complete sample11 with reduced fingerprint size 2024-10-24 13:17:56 -07:00
Yann Collet ca6e55cbf5 reduce splitBlock arguments 2024-10-24 13:17:56 -07:00
Yann Collet 94d7b07425 organize specialization at recordFingerprint level 2024-10-24 13:17:56 -07:00
Yann Collet 566763fdc9 new variant, sampling by 11 2024-10-24 13:17:56 -07:00
Yann Collet b68ddce818 rewrite fingerprint storage to no longer need 64-bit members
so that it can be stored using standard alignment requirement (sizeof(void*)).

Distance function still requires 64-bit signed multiplication though,
so it won't change the issue regarding the bug in ubsan for clang 32-bit on github ci.
2024-10-23 11:50:57 -07:00
Yann ColletandYann Collet 57239c4d3b fixed minor strict pedantic C90 issue 2024-10-23 11:50:57 -07:00
Yann Collet 18b1e67223 fixed extraneous return
strict C90 compliance test
2024-10-23 11:50:57 -07:00
Yann Collet 16450d0732 rewrite penalty update
suggested by @terrelln
2024-10-23 11:50:57 -07:00
Yann Collet 1ec5f9f1f6 changed loop exit condition so that there is no need to assert() within the loop. 2024-10-23 11:50:57 -07:00
Yann Collet 4662f6e646 renamed: FingerPrint => Fingerprint
suggested by @terrelln
2024-10-23 11:50:57 -07:00
Yann Collet a167571db5 added a faster block splitter variant
that samples 1 in 5 positions.

This variant is fast enough for lazy2 and btlazy2,
but it's less good in combination with post-splitter at higher levels (>= btopt).
2024-10-23 11:50:57 -07:00
Yann Collet 1c62e714ab minor split optimization
let's fill the initial stats directly into target fingerprint
2024-10-23 11:50:57 -07:00
Yann Collet 4685eafa81 fix alignment test
for non 64-bit systems
2024-10-23 11:50:56 -07:00
Yann Collet 73a6653653 ZSTD_splitBlock_4k() uses externally provided workspace
ideally, this workspace would be provided from the ZSTD_CCtx* state
2024-10-23 11:50:56 -07:00
Yann Collet 6dc52122e6 fixed c90 comment style 2024-10-23 11:50:56 -07:00
Yann Collet 8b3887f579 fixed kernel build 2024-10-23 11:50:56 -07:00
Yann Collet fa147cbb4d more ZSTD_memset() to apply 2024-10-23 11:50:56 -07:00
Yann Collet 6021b6663a minor C++-ism
though I really wonder if this is a property worth maintaining.
2024-10-23 11:50:56 -07:00
Yann Collet e2d7d08888 use ZSTD_memset()
for better portability on Linux kernel
2024-10-23 11:50:56 -07:00
Yann Collet 586ca96fec do not use new as variable name 2024-10-23 11:50:56 -07:00
Yann Collet 9e52789962 fixed strict C90 semantic 2024-10-23 11:50:56 -07:00
Yann Collet a5bce4ae84 XP: add a pre-splitter
instead of ingesting only full blocks, make an analysis of data, and infer where to split.
2024-10-23 11:50:56 -07:00