ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,564 Commits 17 Branches 77 Tags
cd4dba74dea8a92f9e33d72fcb5b60224bc4e6c3
Commit Graph
4554 Commits
Author SHA1 Message Date
Elliot GorokhovskyandGitHub 7d970bd83c Implement one-shot fallback for magicless format (#3971) 2024-03-18 10:55:53 -04:00
Yann ColletandGitHub 0fcdc62c2d Merge pull request #3969 from facebook/v156_prep 
bump version number
 2024-03-15 10:26:49 -07:00
Yann ColletandGitHub dff5407cd0 Merge pull request #3966 from facebook/debug_lineNumber 
add line number to debug traces
 2024-03-15 10:26:06 -07:00
Yann Collet 686e7e4b4b updated version to v1.5.6 2024-03-14 15:38:14 -07:00
Elliot GorokhovskyandGitHub 559762da12 Remove duplicate and incorrect docs in zstd_decompress.c (#3967) 2024-03-14 15:55:01 -04:00
Yann Collet 9cc3304614 add line number to debug traces 2024-03-14 12:11:11 -07:00
Felix HandteandGitHub 515c07a131 Merge pull request #3964 from felixhandte/promote-tgt-c-blk-size-to-stable 
Promote `ZSTD_c_targetCBlockSize` Parameter to Stable API
 2024-03-14 13:06:46 -04:00
Yann ColletandGitHub 216099a73f Merge pull request #3962 from facebook/cover_lessIncludes 
reduce the amount of #include in cover.h
 2024-03-13 16:52:54 -07:00
W. Felix Handte 3613448fb8 Promote ZSTD_c_targetCBlockSize Parameter to Stable API 
This feature has demonstrated itself to be useful in web compression and we
want to encourage other folks to use it. But we currently make it difficult
to do so since it's locked away in the experimental API.

The API itself is really straightforward and I think it's fine to commit to
maintaining support / compatibility for this API even if in the future the
underlying implementation may continue to evolve.

Note that this commit changes its enum name and also its numeric value. Users
who respected the instructions of using the experimental API should be fine
with both of these changes since they should only have referred to it by the.

Conceivably someone could have done bad feature detection of this capability
by doing `#ifdef ZSTD_c_targetCBlockSize` which will now return false since
it's no longer a macro... but I think that's an acceptable hypothetical
breakage.
 2024-03-13 17:07:10 -04:00
Nick TerrellandNick Terrell ff0afbad58 [asm][aarch64] Mark that BTI and PAC are supported 
Mark that `huf_decompress_amd64.S` supports BTI and PAC, which it trivially does because it is empty for aarch64.

The issue only requested BTI markings, but it also makes sense to mark PAC, which is the only other feature.

Also run add a test for this mode to the ARM64 QEMU test. Before this PR it warns on `huf_decompress_amd64.S`, after it doesn't.

Fixes Issue #3841.
 2024-03-13 16:15:51 -04:00
Yann Collet c8ab027227 reduce the amount of includes in "cover.h" 2024-03-13 11:29:28 -07:00
Nick TerrellandNick Terrell 94c102038b [cpu] Backport fix for rbx clobbering on Windows with Clang 
Backport folly fix for rbx clobbering: https://github.com/facebook/folly/commit/f22f88b8b9d70160388f0f149bc9abaeb82c250b

This supercedes PR #3646.
 2024-03-13 09:45:40 -04:00
Yonatan KomornikandGitHub b20703f273 Updates ZSTD_RowFindBestMatch comment (#3947) 
Updates the comment on the head of `ZSTD_RowFindBestMatch` to make sure it's aligned with recent changes to the hash table.
 2024-03-12 15:10:07 -07:00
Elliot GorokhovskyandGitHub f65b9e27ce Exercise ZSTD_findDecompressedSize() in the simple decompression fuzzer (#3959) 
* Improve decompression fuzzer

* Fix legacy frame header fuzzer crash, add unit test
 2024-03-12 17:07:06 -04:00
Yann ColletandGitHub db996d253e Merge pull request #3933 from facebook/fix3819 
prevent XXH64 from being autovectorized by XXH512 by default
 2024-03-12 09:46:48 -07:00
Yann ColletandGitHub 3b573ba1f0 Merge pull request #3936 from facebook/debug5 
minor: fix incorrect debug level
 2024-03-12 09:45:20 -07:00
Yann ColletandGitHub fe7c645a7e Merge pull request #3937 from facebook/offset0 
new method to deal with offset==0 erroneous edge case
 2024-03-12 09:25:06 -07:00
acceptacrossandGitHub 8ba5bc4729 chore: fix some typos (#3949) 
Signed-off-by: acceptacross <csqcqs@gmail.com>
 2024-03-12 11:44:42 -04:00
Elliot GorokhovskyandGitHub e0872806df Use ZSTD_LEGACY_SUPPORT=5 in make test (#3943) 2024-03-12 10:08:26 -04:00
Yann ColletandGitHub 372fddf4e6 Merge pull request #3860 from likema/fix-xxhash-aix-51 
Fix building xxhash on AIX 5.1
 2024-03-09 15:34:02 -08:00
Yann Collet a9fb8d4c41 new method to deal with offset==0 
in this new method, when an `offset==0` is detected,
it's converted into (size_t)(-1), instead of 1.

The logic is that (size_t)(-1) is effectively an extremely large positive number,
which will not pass the offset distance test at next stage (`execSequence()`).
Checked the source code, and offset is always checked (as it should),
using a formula which is not vulnerable to arithmetic overflow:
```
RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart),
```

The benefit is that such a case (offset==0) is always detected as corrupted data
as opposed to relying on the checksum to detect the error.
 2024-03-08 15:26:06 -08:00
Yann Collet aed172a8fe minor: fix incorrect debug level 2024-03-08 14:29:44 -08:00
Yann Collet 007cda88ca prevent XXH64 from being autovectorized by XXH512 by default 
backport fix https://github.com/Cyan4973/xxHash/pull/924 from libxxhash
 2024-03-07 16:43:13 -08:00
Yann ColletandGitHub 8689633fdf Merge pull request #3840 from aimuz/fix-reserved 
lib/decompress: check for reserved bit corruption in zstd
 2024-03-05 13:40:12 -08:00
Yann ColletandGitHub e385c3dd46 Merge pull request #3753 from facebook/make2 
minor Makefile refactoring
 2024-03-03 19:13:00 -08:00
Yann Collet 8d31e8ec42 sizeBlockSequences() also tracks uncompressed size 
and only defines a sub-block boundary when
it believes that it is compressible.

It's effectively an optimization,
avoiding a compression cycle to reach the same conclusion.
 2024-02-26 14:31:12 -08:00
Yann Collet d23b95d21d minor refactor for clarity 
since we can ensure that nbSubBlocks>0
 2024-02-26 14:06:34 -08:00
Yann Collet 86db60752d optimization: bail out faster in presence of incompressible data 2024-02-26 13:27:59 -08:00
Yann Collet ef82b214ad nit: comment indentation 
as reported by @terrelln
 2024-02-26 13:23:59 -08:00
Yann Collet aa8592c532 minor: reformulate nbSubBlocks assignment 2024-02-26 13:21:14 -08:00
Yann Collet e0412c2062 fix extraneous semicolon ';' 
as reported by @terrelln
 2024-02-26 12:26:54 -08:00
Yann Collet 1fafd0c4ae fix minor visual static analyzer warning 
it's a false positive,
but change the code nonetheless to make it more obvious to the static analyzer.
 2024-02-25 19:45:32 -08:00
Yann Collet 038a8a906b targetCBlockSize: modified splitting strategy to generate blocks of more regular size 
notably avoiding to feature a larger first block
 2024-02-25 17:39:29 -08:00
Yann Collet f8372191f5 reduced minimum compressed block size 
with the intention to match the transport layer size,
such as Ethernet and 4G mobile networks.
 2024-02-24 01:59:16 -08:00
Yann Collet f77f634d41 update API documentation 2024-02-24 01:28:17 -08:00
Yann Collet 4b51526412 fix partial block uncompressed 2024-02-24 01:24:58 -08:00
Yann Collet 6719794379 fixed some regressionTests 
but not all
 2024-02-23 18:48:29 -08:00
Yann Collet 0591e7eea1 minor: fix overly cautious conversion warning 2024-02-23 16:05:09 -08:00
Yann Collet 3b40100058 fix long sequences (> 64 KB) 2024-02-23 15:35:12 -08:00
Yann Collet 6b11fc436c fix issue with incompressible sections 2024-02-23 14:53:56 -08:00
Yann Collet cc4530924b speed optimized version of targetCBlockSize 
note that the size of individual compressed blocks will vary more wildly with this modification.
But it seems good enough for a first test, and fix the speed regression issue.
Further refinements can be attempted later.
 2024-02-23 14:03:26 -08:00
Yann ColletandGitHub 621a263fb2 Merge pull request #3903 from gruenich/feature/reduce-scope-of-variables 
Reduce scope of variables
 2024-02-22 09:42:02 -08:00
Yann Collet e62e15df19 fix clangbuild 
notably -Wconversion and -Wdocumentation
 2024-02-20 22:43:22 -08:00
Christoph Grüninger b921f1aad6 Reduce scope of variables 
This improves readability, keeps variables local, and
prevents the unintended use (e.g. typo) later on.
Found by Cppcheck (variableScope)
 2024-02-11 22:00:03 +01:00
Yann Collet b0e8580dc7 fix fuzz issue 5131069967892480 2024-02-08 16:38:20 -08:00
Yann Collet 22574d848d fix issue 5921623844651008 
ossfuzz managed to create a scenario which triggers an `assert`.
This fixes it, by giving +1 more space for the backward search pass.
 2024-02-06 13:01:14 -08:00
Yann Collet b88c593d8f added or updated code comments 
as suggested by @terrelln,
to make the code of the optimal parser a bit more understandable.
 2024-02-05 18:32:25 -08:00
Yann Collet 6c35fb2e8c fix msan warnings 2024-02-05 01:21:06 -08:00
Yann Collet 641749fc09 fix uasan dictionary_stream_round_trip fuzz test 2024-02-05 00:36:10 -08:00
Yann Collet fe2e2ad36d use ZSTD_memcpy() 
which can be redirected in Linux kernel mode
 2024-02-03 19:57:38 -08:00