ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,977 Commits 17 Branches 77 Tags
2f3ee8b5309958a2bc1fc7477e703fd8195a31ea
Commit Graph
10977 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandYann Collet 2f3ee8b530 changed code compilation test to employ ZSTD_ARCH_X86_AVX2 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet debe3d20d9 removed unused branch 2025-01-15 17:11:27 -08:00
Yann Collet 6f8e6f3c97 create new compilation macro ZSTD_ARCH_X86_AVX2 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet e3181cfd32 minor code doc update 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet aa2cdf964f added compilation-time checks to ensure AVX2 code is valid 
since it depends on a specific definition of ZSTD_Sequence structure.
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 57a4554192 removed unused variable 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 4aaf9cefe9 fix minor conversion warning 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet db3d48823a no need for specialized variant 
the branch is not in the hot loop
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet cd53924eff removed erroneous #includes 
that were automatically added by the editor without notification
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet ed0a8b8be1 AVX2 version of ZSTD_get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet b6a4d5a8ba minor +10% speed improvement for scalar ZSTD_get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 8eb2587432 added benchmark for get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet bfc58f5ba2 generalize validation function 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 8d62164589 control long length within AVX2 implementation 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet d1f0e5fb97 fullbench can run a verification function 
compressSequencesAndLiterals: fixed long lengths in scalar mode
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 886720442f initial implementation (incomplete) 
needs to take care of long lengths > 65535
 2025-01-15 17:11:27 -08:00
Yann ColletandGitHub 72277079fb Merge pull request #4243 from facebook/fix_ci 
fix ci tests
 2025-01-15 17:10:26 -08:00
Yann Collet 0b96e6d42a fixed zlib test 
new versions of the library generate a different error code
 2025-01-15 16:31:00 -08:00
Yann Collet 43626f1ce0 zlib-wrapper: separate test components to better understand what is failing 2025-01-15 16:05:42 -08:00
Yann Collet d4ae5c3752 update installed lib64gcc version 2025-01-15 15:52:27 -08:00
Yann Collet 642157cc45 short-test-1: separate tests, giving each a name 
to better observe where the problem might be
 2025-01-15 15:39:20 -08:00
Yann Collet 80ff61de1d fixed cmake-build-and-test-check 
missing the liblzma library
 2025-01-15 15:28:23 -08:00
Yann Collet 196e76efe1 fix make-external-compressors "only-lzma" test 
missing the liblzma-dev package
 2025-01-15 15:24:27 -08:00
Yann Collet 0e819c9f93 fix meson-linux build test 
missing the liblzma package
 2025-01-15 15:20:28 -08:00
Yann Collet 908a95889b fix clang-pgo test 2025-01-15 15:05:44 -08:00
Yann ColletandGitHub 92d1a7d07c Merge pull request #4234 from MessyHack/dev 
[MSVC] Wrong define checked when building for 64bit with Visual Studio
 2025-01-14 14:33:25 -08:00
Yann ColletandGitHub 421f9c947d Merge pull request #4237 from facebook/typo_s 
fixed an obvious macro name typo
 2025-01-14 11:12:46 -08:00
Sean BartellandNick Terrell 80af41e08a [contrib/pzstd] Prevent hangs when there are errors 
When two threads are using a WorkQueue and the reader thread exits due
to an error, it must call WorkQueue::finish() to wake up the writer
thread. Otherwise, if the queue is full and the writer thread is waiting
for a free slot, it could hang forever.

This can happen in pratice when decompressing a large, corrupted file
that does not contain pzstd skippable frames.
 2025-01-13 17:24:41 -05:00
Yann Collet f5dbdac818 fixed macro name typo 2025-01-12 23:08:26 -08:00
MessyHack 42d704ad5e should check defined(_M_X64) not defined(_M_X86) when building with MSVC. 
_M_X86 is only defined under MSVC 32Bit
_M_X64 is only defined under MSVC 64Bit
 2025-01-10 22:47:48 -08:00
Victor ZhangandGitHub a610550e2c Merge pull request #4218 from facebook/externC 
Move #includes out of `extern "C"` blocks
 2025-01-07 10:06:08 -08:00
Yann ColletandGitHub f5e2996dd3 Merge pull request #4231 from sunpoet/dev 
Update MANDIR for FreeBSD
 2025-01-07 09:04:57 -08:00
Victor ZhangandGitHub 59c2e3335f Merge pull request #4230 from facebook/variedOutput 
Do not vary row matchfinder selection based on availability of SSE2/Neon
 2025-01-06 09:26:57 -08:00
Po-Chuan Hsieh 0fd521048d Update MANDIR for FreeBSD 
share/man became a valid path for manpage since Jan 2020. And we
converted the whole ports tree to share/man around last March.
 2025-01-05 22:26:08 +08:00
Victor Zhang d88651e604 Do not vary row matchfinder selection based on availability of SSE2/Neon 
Move towards a stronger guarantee of reproducibility by removing this small difference for machines without SSE2/Neon.
The SIMD behavior is now the default for all platforms.
 2025-01-03 09:35:18 -08:00
Nick TerrellandNick Terrell 1548bfc349 [opt] Fix too short of match getting generated 
The optimal parser with LDM enabled using minMatch > 3 could generate a match
length of 3 when minMatch >= 4. This is not allowed.

1. Fix the bug
2. Add validation logic to `ZSTD_buildSeqStore()` in debug mode for all block
   compressors that checks we never generate too short a match. This way we don't
   rely on the `generate_sequences` fuzzer to find this issue.

Credit to OSS-Fuzz
 2025-01-03 11:38:41 -05:00
Victor ZhangandGitHub 2759d9d52f Merge pull request #4229 from facebook/noFseek 
Support for libc variants without fseeko/ftello
 2025-01-02 22:42:00 -08:00
Victor Zhang dfb236b2aa chore: indentation alignment 2025-01-02 15:10:40 -08:00
Victor Zhang 6b046f5841 PR feedback 2025-01-02 15:05:58 -08:00
Victor Zhang 757e29e170 Oops 2025-01-02 14:17:24 -08:00
Victor Zhang 54c3d998a0 Support for libc variants without fseeko/ftello 
Some older Android libc implementations don't support `fseeko` or `ftello`.
This commit adds a new compile-time macro `LIBC_NO_FSEEKO` as well as a usage in CMake for old Android APIs.
 2025-01-02 14:02:10 -08:00
Yann ColletandGitHub f5d9d57fe6 Merge pull request #4228 from facebook/skippableMagicVariant 
ZSTD_getFrameHeader() for skippable frames
 2025-01-02 10:57:49 -08:00
Yann Collet 04a2a0219c update type names 
naming convention: Type names should start with a Capital letter (after the prefix)
 2024-12-29 14:25:33 -08:00
Yann ColletandGitHub cf5e53f618 Merge pull request #4227 from facebook/doc_skippableFrame 
clarify documentation for Frame-level methods invoked on a skippable frame
 2024-12-29 13:27:35 -08:00
Yann Collet a2ff6ea784 improve ZSTD_getFrameHeader on skippable frames 
now reports:
- the header size
- the magic variant (within @dictID field)
 2024-12-29 12:26:04 -08:00
Yann Collet f8a2b352d6 clarify doc on Frame-level methods when invoked on a skippable frame 
following discussion at #4226
 2024-12-29 02:41:09 -08:00
Yann ColletandGitHub 788926fe48 Merge pull request #4217 from facebook/ZSTD_compressSequencesAndLiterals 
ZSTD_compressSequencesAndLiterals
 2024-12-26 11:41:44 -08:00
Yann Collet 72ce56b527 fixed another invalid scenario 
compressSequencesAndLiterals() doesn't support sequence validation
 2024-12-23 21:15:50 -08:00
Yann Collet 7b294caf46 add one valid test case 
ZSTD_compressSequencesAndLiterals() may return a specific error code
when data to compress is non-compressible.
 2024-12-23 19:43:17 -08:00
Yann Collet f8725e80cc added fuzzer test for compressSequencesAndLiterals() 
piggy-backing onto existing compressSequences() fuzzer test
 2024-12-23 18:42:51 -08:00