ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,729 Commits 17 Branches 77 Tags
da0e2c8380325c8d4405b911e6defe6e3e41158d
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet b14d76d888 pinned dependency hash 2025-02-18 14:53:31 -08:00
Yann Collet 5c465fcabe harden github actions script Android NDK Build 2025-02-18 14:50:03 -08:00
Yann Collet c26bde119b update changelog for v1.5.7 2025-02-18 13:59:24 -08:00
Yann Collet 2a58b04752 disabled BTI test 
this test seems impossible on Ubuntu-24
 2025-02-11 13:20:15 -08:00
Yann Collet 85c39b78cf faster aarch64 test execution 2025-02-11 13:11:29 -08:00
Yann Collet 0b8119f0ad minor arm64 test script clarification 2025-02-11 12:58:35 -08:00
Yann Collet b73e06b83e updated FreeBSD VM to 14.2 
14.1 seems no longer supported by gcloud
 2025-02-11 12:52:59 -08:00
Yann Collet 2b7c661ad2 update arm64 test 
for ubuntu-24
 2025-02-11 12:41:38 -08:00
Yann Collet 75bcae1272 removing x32 tests 
this ABI is no longer supported by Ubuntu,
and there is a wider consensus that this ABI is on the way out,
with more and more distributions dropping it,
and lingering questions about support of x32 in the kernel.
 2025-02-11 11:37:18 -08:00
Yann Collet fc1baf3463 move qemu tests to ubuntu24 2025-02-11 09:57:18 -08:00
Yann Collet 815ca8c678 ubuntu 20 is being deprecated 
change ubuntu20 tests to use more modern ubuntu variants
 2025-02-11 09:19:08 -08:00
Yann Collet d2c562b803 update hrlog comment 2025-02-10 10:48:56 -08:00
Yann Collet 67fad95f79 derive hashratelog from hashlog when only hashlog is set 2025-02-10 10:46:37 -08:00
Yann Collet f86024ccd2 reinforce man page warning 
32-bit is unsuitable for --max
 2025-02-09 17:25:35 -08:00
Yann Collet 39d1d82fa8 adjusted mml heuristic 2025-02-09 17:24:12 -08:00
Yann Collet 41b7193757 added --max to test suite 2025-02-09 12:26:36 -08:00
Yann Collet 8ae1330708 add man entry for --max 2025-02-09 11:51:25 -08:00
Yann Collet 630b47a158 added a new --max command 
set parameters to maximum compression
(even more than -22)
 2025-02-09 11:38:44 -08:00
Yann Collet aebffd66ec --ultra automatically triggered with --long and --patch-from 
the purpose of --ultra is to make the user explicitly opt-in
to generate very large window size (> 8 MB).

The agreement to generate very large window size is already implicit
when selecting --long or --patch-from.

Consequently, `--ultra ` is automatically enabled when `--long` or `--patch-from` is set.
 2025-02-08 22:47:05 -08:00
Yann Collet 23e5f80390 Revert "pass dictionary loading method as parameter" 
This reverts commit 821fc567f93a415e9fbe856271ccd452ee7acf07.
 2025-02-05 18:47:26 -08:00
Yann Collet c7cd7dc04b better MT fluidity 
--patch-from no longer blocked on first job dictionary loading
 2025-02-05 18:42:00 -08:00
Yann Collet f11bd19c7f ensure cdict is properly reset to NULL 2025-02-05 18:42:00 -08:00
Yann Collet 7406d2b6eb skips the need to create a temporary cdict for --patch-from 
thus saving a bit of memory and a little bit of cpu time
 2025-02-05 18:42:00 -08:00
Yann Collet 220abe6da8 reduced memory usage 
by avoiding to duplicate in memory
a dictionary that was passed by reference.
 2025-02-05 18:42:00 -08:00
Yann Collet 85a44b233a always free .cdictLocal 2025-02-05 18:41:59 -08:00
Yann Collet e637fc64c5 update type naming convention 2025-02-05 18:41:59 -08:00
Yann Collet 34ba14437a minor boundary change 
improves compression ratio at low levels
 2025-02-05 18:41:59 -08:00
Yann Collet ffa66a6971 fix speed of --patch-from at high compression mode 2025-02-05 18:41:59 -08:00
Yann Collet 2949252923 fix minor conversion warnings 2025-02-05 17:01:19 -08:00
Yann Collet e87d15938c more %zu warnings fixes 2025-02-05 16:48:19 -08:00
Yann Collet 590c22454e fix more %zu warnings 2025-02-05 16:36:54 -08:00
Yann Collet f9c1850aa2 fix minor printf argument limitation on older mingw 
which do not support `%zu` fields
 2025-02-05 16:28:52 -08:00
Yann Collet e117d79e22 fix minor alignment warning 2025-02-05 16:13:58 -08:00
Yann Collet c39424ea87 fix minor alignment warning 
this is a prototype definition error:
`_mm_storeu_si128()` should accept a `void*` pointer,
since it explicitly states that it accepts unaligned addresses
yet requiring a `__m128i*` tells otherwise, and requires the compiler the enforce this alignment.
 2025-02-05 16:11:54 -08:00
Yann Collet 32dff04d32 fix one minor alignment warning 
seems like a prototype interface error:
input parameter should have been `const void*`,
since the documentation is explicit that input doesn't have to be aligned,
but `const __m256i*` makes the compiler enforce it.
 2025-02-05 15:46:44 -08:00
Yann Collet 5883ee6cc2 make visual studio tests error out on warnings 2025-02-05 15:38:39 -08:00
Yann Collet 54e9d46db4 added __clang__ to compiler-specific alignment attribute 
when clang is used within msvc, `__GNUC__` isn't defined,
so testing `__clang__` explicitly is required.
 2025-02-05 13:48:24 -08:00
Yann Collet bcf404c0ab changed C11 keyword to _Alignas 
so that it doesn't depend on #include
 2025-02-05 13:25:14 -08:00
Yann Collet 6e1d02f1f0 adding a Visual + ClangCL + AVX2 compilation test 2025-02-05 13:20:44 -08:00
Yann Collet 60f84f73fe bench: better result alignment 
when displaying ratios > x100
 2025-01-30 21:30:48 -08:00
Yann Collet 82346b92bb minor: generalize BitContainerType 
technically equivalent to `size_t`,
but it's the proper type for underlying register representation.

This makes it possible to control register type, and therefore size, independently from `size_t`,
which can be useful on systems where `size_t` is 32-bit, while the architecture supports 64-bit registers.
 2025-01-19 18:05:57 -08:00
Yann Collet d2d74616c0 also add -mbmi2 to the compilation test 2025-01-18 22:58:03 -08:00
Yann Collet 27d7940631 minor: cosmetic, indentation 2025-01-18 22:49:16 -08:00
Yann Collet 0501095898 added -DSTATIC_BMI2=1 for the -mavx2 test 2025-01-18 22:49:16 -08:00
Yann Collet f0b5f65bca fixed minor static function declaration issue 
in AVX2 mode only
 2025-01-18 22:49:16 -08:00
Yann Collet 35edbc20dc added avx2 (x64) compilation test 2025-01-18 22:49:16 -08:00
Yann Collet 9efb09749b added a CI test for x86 32-bit + avx2 combination 
which is expected to be quite rare, but nonetheless possible.

This test is initially expected to fail, before integration of #4248 fix
 2025-01-18 22:49:16 -08:00
Yann Collet e8de8085f4 minor: assert that state is not null 
replaces #4016
 2025-01-18 13:08:04 -08:00
Yann Collet 4f3311f245 CI: replaced shortest target by check 2025-01-18 12:24:11 -08:00
Yann Collet 78275149ea Added a final success message at end of tests 
so that a human reading the test log can determine everything was fine without consulting the shell error code.

Also: made `make check` slightly shorter by moving one longer test to `make test`
 2025-01-17 19:30:55 -08:00
Yann Collet 6f8e6f3c97 create new compilation macro ZSTD_ARCH_X86_AVX2 2025-01-15 17:11:27 -08:00
Yann Collet 0b96e6d42a fixed zlib test 
new versions of the library generate a different error code
 2025-01-15 16:31:00 -08:00
Yann Collet 43626f1ce0 zlib-wrapper: separate test components to better understand what is failing 2025-01-15 16:05:42 -08:00
Yann Collet d4ae5c3752 update installed lib64gcc version 2025-01-15 15:52:27 -08:00
Yann Collet 642157cc45 short-test-1: separate tests, giving each a name 
to better observe where the problem might be
 2025-01-15 15:39:20 -08:00
Yann Collet 80ff61de1d fixed cmake-build-and-test-check 
missing the liblzma library
 2025-01-15 15:28:23 -08:00
Yann Collet 196e76efe1 fix make-external-compressors "only-lzma" test 
missing the liblzma-dev package
 2025-01-15 15:24:27 -08:00
Yann Collet 0e819c9f93 fix meson-linux build test 
missing the liblzma package
 2025-01-15 15:20:28 -08:00
Yann Collet 908a95889b fix clang-pgo test 2025-01-15 15:05:44 -08:00
Yann Collet f5dbdac818 fixed macro name typo 2025-01-12 23:08:26 -08:00
Yann Collet 7b294caf46 add one valid test case 
ZSTD_compressSequencesAndLiterals() may return a specific error code
when data to compress is non-compressible.
 2024-12-23 19:43:17 -08:00
Yann Collet f8725e80cc added fuzzer test for compressSequencesAndLiterals() 
piggy-backing onto existing compressSequences() fuzzer test
 2024-12-23 18:42:51 -08:00
Yann Collet 47cbfc87a9 restore invocation of ZSTD_entropyCompressSeqStore() 
in the ZSTD_compressSequences() pipeline
 2024-12-20 10:37:01 -08:00
Yann Collet 522adc34eb minor: use MEM_writeLE24() 
so that an empty frame needs only 3 bytes of dstCapacity.
 2024-12-20 10:37:01 -08:00
Yann Collet b7a9e69d8d added parameter litCapacity 
to ZSTD_compressSequencesAndLiterals()
to enforce the litCapacity >= litSize+8 condition.
 2024-12-20 10:37:01 -08:00
Yann Collet 76445bb379 add a check, to return an error if Sequence validation is enabled 
since ZSTD_compressSequencesAndLiterals() doesn't support it.
 2024-12-20 10:37:01 -08:00
Yann Collet 52a9bc6fca fixed minor error in preparation of one fullbench scenario 2024-12-20 10:37:00 -08:00
Yann Collet ab0f1798e8 ensure that srcSize is controlled 2024-12-20 10:37:00 -08:00
Yann Collet b339efff2b add dedicated error code for special case 
ZSTD_compressSequencesAndLiterals() cannot produce an uncompressed block
 2024-12-20 10:37:00 -08:00
Yann Collet a80f55f47d added a test for ZSTD_compressSequencesAndLiterals 
checks that srcSize is present in the frame header
and bounds the window size.
 2024-12-20 10:37:00 -08:00
Yann Collet 0a54f6f288 ZSTD_compressSequencesAndLiterals requires srcSize as parameter 
this makes it possible to adjust windowSize to its tightest.
 2024-12-20 10:37:00 -08:00
Yann Collet ad023b392f fixed minor error in one benchmark scenario 2024-12-20 10:37:00 -08:00
Yann Collet b7b4e86347 fixed minor conversion warning 2024-12-20 10:37:00 -08:00
Yann Collet 12c47d3262 improved speed of the Sequences converter 2024-12-20 10:37:00 -08:00
Yann Collet 95ad9e47ff added benchmark for ZSTD_convertBlockSequences_wBlockDelim() 2024-12-20 10:37:00 -08:00
Yann Collet d48e330ae1 change name to ZSTD_convertSequences*() 2024-12-20 10:37:00 -08:00
Yann Collet 61ac8311e0 attempt to silence Visual Studio warning about fopen() 2024-12-20 10:37:00 -08:00
Yann Collet f617e86b71 fixed incorrect assert 2024-12-20 10:37:00 -08:00
Yann Collet 47edd0acf4 removed fullbench-dll project from visual solutions 2024-12-20 10:37:00 -08:00
Yann Collet 6f8c1046d0 update Visual Studio solutions 2024-12-20 10:37:00 -08:00
Yann Collet f0d0d95234 added tests 
check that ZSTD_compressAndLiterals() also controls that the `srcSize` field is exact.
 2024-12-20 10:37:00 -08:00
Yann Collet 31b5ef2539 ZSTD_compressSequencesAndLiterals() now supports multi-blocks frames. 2024-12-20 10:36:59 -08:00
Yann Collet 5164d44dab change advanced parameter name: ZSTD_c_repcodeResolution 
and updated its documentation.
Note: older name ZSTD_c_searchForExternalRepcodes remains supported via #define
 2024-12-20 10:36:59 -08:00
Yann Collet ca8bd83373 minor: cleaner function parameter repcodeResolution 2024-12-20 10:36:59 -08:00
Yann Collet d2d0fdac42 updated documentation on validateSequence 2024-12-20 10:36:59 -08:00
Yann Collet 1f6d6815c3 optimization: instantiate specialized version without Sequence checking code 
results in +4% compression speed,
thanks to removal of branches in the hot loop.
 2024-12-20 10:36:59 -08:00
Yann Collet a288751de7 minor optimization: only track seqPos->posInSrc when validateSequences is enabled 
note: very minor saving, no performance impact
 2024-12-20 10:36:59 -08:00
Yann Collet 8ab04097ed add the compressSequences() benchmark scenario 2024-12-20 10:36:59 -08:00
Yann Collet c050ae4fb8 fullbench: preparation functions are now in charge of allocating buffers 2024-12-20 10:36:59 -08:00
Yann Collet 8b7e1b795d minor variable renaming 2024-12-20 10:36:59 -08:00
Yann Collet c540976a4b minor: test reordering 2024-12-20 10:36:59 -08:00
Yann Collet 4c097b4939 refactor fullbench 
to make it easier to add new scenarios
 2024-12-20 10:36:58 -08:00
Yann Collet 09964c6276 prepare fullbench for a refactor 2024-12-20 10:36:58 -08:00
Yann Collet 0b013b2688 added unit tests to ZSTD_compressSequencesAndLiterals() 
seems to work as expected,
correctly control that `litSize` and `srcSize` are exactly correct.
 2024-12-20 10:36:58 -08:00
Yann Collet 14a21e43b3 produced ZSTD_compressSequencesAndLiterals() as a separate pipeline 
only supports explicit delimiter mode, at least for the time being
 2024-12-20 10:36:58 -08:00
Yann Collet bcb15091aa minor: more accurate variable scope 2024-12-20 10:36:58 -08:00
Yann Collet 047db4f1f8 ZSTD_SequenceCopier_f no returns the nb of bytes consumed from input 
which feels much more natural
 2024-12-20 10:36:58 -08:00
Yann Collet 4ef9d7d585 codemod: ZSTD_cParamMode_e -> ZSTD_CParamMode_e 2024-12-20 10:36:58 -08:00
Yann Collet 56cfb7816a codemod: ZSTD_paramSwitch_e -> ZSTD_ParamSwitch_e 2024-12-20 10:36:58 -08:00
Yann Collet 13b9296d79 minor simplification 2024-12-20 10:36:58 -08:00