ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,485 Commits 17 Branches 77 Tags
7b94de78ff2c3634cd7ef32f0e16f4722c03578f
Commit Graph
11485 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 4662f6e646 renamed: FingerPrint => Fingerprint 
suggested by @terrelln
 2024-10-23 11:50:57 -07:00
Yann Collet ea85dc7af6 conservatively estimate over-splitting in presence of incompressible loss 
ensure data can never be expanded by more than 3 bytes per full block.
 2024-10-23 11:50:57 -07:00
Yann Collet 5ae34e4c96 ensure lastBlock is correctly determined 
reported by @terrelln
 2024-10-23 11:50:57 -07:00
Yann Collet 7bad787d8b made ZSTD_isPower2() an inline function 2024-10-23 11:50:57 -07:00
Yann Collet a167571db5 added a faster block splitter variant 
that samples 1 in 5 positions.

This variant is fast enough for lazy2 and btlazy2,
but it's less good in combination with post-splitter at higher levels (>= btopt).
 2024-10-23 11:50:57 -07:00
Yann Collet 1c62e714ab minor split optimization 
let's fill the initial stats directly into target fingerprint
 2024-10-23 11:50:57 -07:00
Yann Collet dac26eaeac updated regression test results 2024-10-23 11:50:57 -07:00
Yann ColletandYann Collet 4ce91cbf2b fixed workspace alignment on non 64-bit systems 2024-10-23 11:50:57 -07:00
Yann Collet cae8d13294 splitter workspace is now provided by ZSTD_CCtx* 2024-10-23 11:50:56 -07:00
Yann Collet 4685eafa81 fix alignment test 
for non 64-bit systems
 2024-10-23 11:50:56 -07:00
Yann Collet 433f4598ad fixed minor conversion warnings on Visual 2024-10-23 11:50:56 -07:00
Yann Collet 73a6653653 ZSTD_splitBlock_4k() uses externally provided workspace 
ideally, this workspace would be provided from the ZSTD_CCtx* state
 2024-10-23 11:50:56 -07:00
Yann Collet 7f015c2fd7 replaced uasan32 test by asan32 test 2024-10-23 11:50:56 -07:00
Yann Collet 31d48e9ffa fixing minor formatting issue in 32-bit mode with logs enabled 2024-10-23 11:50:56 -07:00
Yann Collet 76ad1d6903 fixed VS2010 solution 2024-10-23 11:50:56 -07:00
Yann Collet cdddcaaec9 new Makefile target mesonbuild 
for easier local testing
 2024-10-23 11:50:56 -07:00
Yann Collet 6939235f01 fixed meson build 2024-10-23 11:50:56 -07:00
Yann Collet 80a912dec1 fixed zstreamtest 2024-10-23 11:50:56 -07:00
Yann Collet 6dc52122e6 fixed c90 comment style 2024-10-23 11:50:56 -07:00
Yann Collet 20c3d176cd fix assert 2024-10-23 11:50:56 -07:00
Yann Collet 0d4b520657 only split full blocks 
short term simplification
 2024-10-23 11:50:56 -07:00
Yann Collet dd38c677eb fixed single-library build 2024-10-23 11:50:56 -07:00
Yann Collet 8b3887f579 fixed kernel build 2024-10-23 11:50:56 -07:00
Yann Collet f83ed087f6 fixed RLE detection test 2024-10-23 11:50:56 -07:00
Yann Collet 83a3402a92 fix overlap write scenario in presence of incompressible data 2024-10-23 11:50:56 -07:00
Yann Collet fa147cbb4d more ZSTD_memset() to apply 2024-10-23 11:50:56 -07:00
Yann Collet 6021b6663a minor C++-ism 
though I really wonder if this is a property worth maintaining.
 2024-10-23 11:50:56 -07:00
Yann Collet e2d7d08888 use ZSTD_memset() 
for better portability on Linux kernel
 2024-10-23 11:50:56 -07:00
Yann Collet 586ca96fec do not use new as variable name 2024-10-23 11:50:56 -07:00
Yann Collet 9e52789962 fixed strict C90 semantic 2024-10-23 11:50:56 -07:00
Yann Collet a5bce4ae84 XP: add a pre-splitter 
instead of ingesting only full blocks, make an analysis of data, and infer where to split.
 2024-10-23 11:50:56 -07:00
Yann ColletandGitHub dfaf5fafb9 Merge pull request #4174 from facebook/bench_loadOnce 
Modify benchmark to load sources only once
 2024-10-23 11:14:05 -07:00
Yann Collet f34bc9cee6 improve man page on benchmark mode 
update the man page in troff format,
and the README with latest `--help` content and complementary details about benchmark mode.

also: display level 0 when doing decompression benchmark
 2024-10-23 00:16:13 -07:00
Yann Collet 0079d515b1 Modify benchmark to only load sources once 
After a regrettable update,
the benchmark module ended up reloading sources for every compression level.

While the delay itself is likely torelable,
the main issue is that the `--quiet` mode now also displays a loading summary between each compression line.
This wasn't the original intention, which is to produce a compact view of all compressions.

This is fixed in this version,
where sources are loaded only once, for all compression levels,
and loading summary is only displayed once.
 2024-10-22 02:18:48 -07:00
Yann ColletandGitHub b880f20d52 Merge pull request #4171 from facebook/lvl3_ratio+ 
Improve compression ratio of levels 3 & 4
 2024-10-17 11:39:41 -07:00
Yann Collet 41d870fbbf updated regression tests results 2024-10-17 11:06:26 -07:00
Yann Collet ff8e98bebe enable regression tests at pull request time 
was transferred from circleci,
but was only triggered on push into dev,
i.e. after pull request is merged.
 2024-10-17 09:45:16 -07:00
Yann Collet 47d4f5662d rewrite code in the manner suggested by @terrelln 2024-10-17 09:37:23 -07:00
Yann Collet 61d08b0e42 fix test 
a margin of 4 is insufficient to guarantee compression success.
 2024-10-17 09:37:23 -07:00
Yann Collet 6326775166 slightly improved compression ratio at levels 3 & 4 
The compression ratio benefits are small but consistent, i.e. always positive.
On `silesia.tar` corpus, this modification saves ~75 KB at level 3.
The measured speed cost is negligible, i.e. below noise level, between 0 and -1%.
 2024-10-17 09:37:23 -07:00
Yann ColletandGitHub 18a42190c2 Merge pull request #4170 from facebook/dict_cSpeed 
Improve dictionary compression speed
 2024-10-16 17:36:49 -07:00
Yann Collet 730d2dce41 fix test 2024-10-15 18:44:40 -07:00
Yann Collet c2abfc5ba4 minor improvement to level 3 dictionary compression ratio 2024-10-15 17:58:33 -07:00
Yann Collet e63896eb58 small dictionary compression speed improvement 
not as good as small-blocks improvement,
but generally positive.
 2024-10-15 17:48:35 -07:00
Yann ColletandGitHub def3ee9548 Merge pull request #4167 from facebook/ci_m32test_faster 
attempt to make 32-bit tests faster
 2024-10-12 01:57:55 -07:00
Yann Collet e6740355e3 attempt parallel test running with -j 2024-10-11 18:01:28 -07:00
Yann Collet 6f2e29a234 measure if -O2 makes the test complete faster 2024-10-11 17:30:55 -07:00
Yann Collet 1024aa9252 attempt to make 32-bit tests faster 
this is the longest CI test, reaching ~40mn on last PR
 2024-10-11 16:24:25 -07:00
Yann ColletandGitHub 8c38bda935 Merge pull request #4165 from facebook/cspeed_cmov 
Improve compression speed on small blocks
 2024-10-11 16:20:19 -07:00
Yann Collet 8e5823b65c rename variable name 
findMatch -> matchFound
since it's a test, as opposed to an active search operation.
suggested by @terrelln
 2024-10-11 15:38:12 -07:00