ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,010 Commits 17 Branches 77 Tags
9fbed3330c032ec82b5a3ccbf38edabd61b80be6
Commit Graph
4766 Commits
Author SHA1 Message Date
Yann Collet a556559841 no longer limit automated BMI2 detection to x64 
this was previously no triggered in x86 32-bit mode,
due to a limitation in `bitstream.h`, that was fixed in #4248.

Now, `bmi2` will be automatically detected and triggered
at compilation time, if the corresponding instruction set is enabled,
even in 32-bit mode.

Also: updated library documentation, to feature STATIC_BMI2 build variable
 2025-01-19 00:08:57 -08:00
Yann Collet 27d7940631 minor: cosmetic, indentation 2025-01-18 22:49:16 -08:00
Yann Collet f0b5f65bca fixed minor static function declaration issue 
in AVX2 mode only
 2025-01-18 22:49:16 -08:00
Yann Collet 9efb09749b added a CI test for x86 32-bit + avx2 combination 
which is expected to be quite rare, but nonetheless possible.

This test is initially expected to fail, before integration of #4248 fix
 2025-01-18 22:49:16 -08:00
Yann ColletandGitHub a469e7c083 Merge pull request #4248 from pps83/dev-bzhi32 
Use _bzhi_u32 for 32-bit builds when building with STATIC_BMI2
 2025-01-18 22:48:24 -08:00
Yann ColletandGitHub 167b00495d Merge pull request #4246 from pps83/dev-asmx64-win 
[asm] Enable x86_64 asm for windows builds
 2025-01-18 20:03:16 -08:00
Pavel P fcd684b9b4 update sizeof check 2025-01-19 02:37:35 +02:00
Pavel P d60c4d75e9 remove unrelated changes 2025-01-19 02:36:00 +02:00
Pavel P 462484d5dc change to BitContainerType 2025-01-19 02:34:41 +02:00
Pavel P 26e5fb3614 handle 32bit size_t when building for x64 2025-01-18 23:37:50 +02:00
Pavel P 936927a427 handle 32bit size_t when building for x64 2025-01-18 23:30:55 +02:00
Yann Collet e8de8085f4 minor: assert that state is not null 
replaces #4016
 2025-01-18 13:08:04 -08:00
Pavel P ee17f4c6d2 Use _bzhi_u32 for 32-bit builds when building with STATIC_BMI2 
`_bzhi_u64` is available only for 64-bit builds, while `BIT_getLowerBits` expects `nbBits` to be less than `BIT_MASK_SIZE` (`BIT_MASK_SIZE` is 32)
 2025-01-18 21:33:04 +02:00
Pavel P 46e17b805b [asm] Enable x86_64 asm for windows builds 2025-01-18 05:33:08 +02:00
Yann ColletandGitHub 19025f3da0 Merge pull request #4238 from szsam/patch-1 
fix out-of-bounds array index access
 2025-01-15 17:56:41 -08:00
Yann ColletandYann Collet 87f0a4fbe0 restore full equation 
do not solve the equation, even though some members cancel each other,
this is done for clarity,
we'll let the compiler do the resolution at compile time.
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 8bff69af86 Alignment instruction ZSTD_ALIGNED() in common/compiler.h 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 2f3ee8b530 changed code compilation test to employ ZSTD_ARCH_X86_AVX2 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet debe3d20d9 removed unused branch 2025-01-15 17:11:27 -08:00
Yann Collet 6f8e6f3c97 create new compilation macro ZSTD_ARCH_X86_AVX2 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet e3181cfd32 minor code doc update 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet aa2cdf964f added compilation-time checks to ensure AVX2 code is valid 
since it depends on a specific definition of ZSTD_Sequence structure.
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 57a4554192 removed unused variable 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 4aaf9cefe9 fix minor conversion warning 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet db3d48823a no need for specialized variant 
the branch is not in the hot loop
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet cd53924eff removed erroneous #includes 
that were automatically added by the editor without notification
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet ed0a8b8be1 AVX2 version of ZSTD_get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet b6a4d5a8ba minor +10% speed improvement for scalar ZSTD_get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 8eb2587432 added benchmark for get1BlockSummary() 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 8d62164589 control long length within AVX2 implementation 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet d1f0e5fb97 fullbench can run a verification function 
compressSequencesAndLiterals: fixed long lengths in scalar mode
 2025-01-15 17:11:27 -08:00
Yann ColletandYann Collet 886720442f initial implementation (incomplete) 
needs to take care of long lengths > 65535
 2025-01-15 17:11:27 -08:00
Mingjie ShenandGitHub afff3d2cce return error if block delimiter is not found 2025-01-13 20:52:06 -05:00
Mingjie ShenandGitHub e490be895c fix out-of-bounds array index access 2025-01-13 16:39:34 -05:00
MessyHack 42d704ad5e should check defined(_M_X64) not defined(_M_X86) when building with MSVC. 
_M_X86 is only defined under MSVC 32Bit
_M_X64 is only defined under MSVC 64Bit
 2025-01-10 22:47:48 -08:00
Victor ZhangandGitHub a610550e2c Merge pull request #4218 from facebook/externC 
Move #includes out of `extern "C"` blocks
 2025-01-07 10:06:08 -08:00
Victor Zhang d88651e604 Do not vary row matchfinder selection based on availability of SSE2/Neon 
Move towards a stronger guarantee of reproducibility by removing this small difference for machines without SSE2/Neon.
The SIMD behavior is now the default for all platforms.
 2025-01-03 09:35:18 -08:00
Nick TerrellandNick Terrell 1548bfc349 [opt] Fix too short of match getting generated 
The optimal parser with LDM enabled using minMatch > 3 could generate a match
length of 3 when minMatch >= 4. This is not allowed.

1. Fix the bug
2. Add validation logic to `ZSTD_buildSeqStore()` in debug mode for all block
   compressors that checks we never generate too short a match. This way we don't
   rely on the `generate_sequences` fuzzer to find this issue.

Credit to OSS-Fuzz
 2025-01-03 11:38:41 -05:00
Yann Collet 04a2a0219c update type names 
naming convention: Type names should start with a Capital letter (after the prefix)
 2024-12-29 14:25:33 -08:00
Yann Collet a2ff6ea784 improve ZSTD_getFrameHeader on skippable frames 
now reports:
- the header size
- the magic variant (within @dictID field)
 2024-12-29 12:26:04 -08:00
Yann Collet f8a2b352d6 clarify doc on Frame-level methods when invoked on a skippable frame 
following discussion at #4226
 2024-12-29 02:41:09 -08:00
Yann Collet 47cbfc87a9 restore invocation of ZSTD_entropyCompressSeqStore() 
in the ZSTD_compressSequences() pipeline
 2024-12-20 10:37:01 -08:00
Yann Collet 522adc34eb minor: use MEM_writeLE24() 
so that an empty frame needs only 3 bytes of dstCapacity.
 2024-12-20 10:37:01 -08:00
Yann Collet b7a9e69d8d added parameter litCapacity 
to ZSTD_compressSequencesAndLiterals()
to enforce the litCapacity >= litSize+8 condition.
 2024-12-20 10:37:01 -08:00
Yann Collet 76445bb379 add a check, to return an error if Sequence validation is enabled 
since ZSTD_compressSequencesAndLiterals() doesn't support it.
 2024-12-20 10:37:01 -08:00
Yann Collet ab0f1798e8 ensure that srcSize is controlled 2024-12-20 10:37:00 -08:00
Yann Collet b339efff2b add dedicated error code for special case 
ZSTD_compressSequencesAndLiterals() cannot produce an uncompressed block
 2024-12-20 10:37:00 -08:00
Yann Collet 0a54f6f288 ZSTD_compressSequencesAndLiterals requires srcSize as parameter 
this makes it possible to adjust windowSize to its tightest.
 2024-12-20 10:37:00 -08:00
Yann Collet b7b4e86347 fixed minor conversion warning 2024-12-20 10:37:00 -08:00
Yann Collet 12c47d3262 improved speed of the Sequences converter 2024-12-20 10:37:00 -08:00