ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,844 Commits 17 Branches 77 Tags
44c54a3e319df11df1078c46e194ebd948825bd0
Commit Graph
3309 Commits
Author SHA1 Message Date
senhuang42 44c54a3e31 Addressing comments: more comments, cleanup, remove extra function, checksum logic 2020-08-24 16:14:19 -04:00
senhuang42 ffaa0df76d Document change in CLI for --no-check during decompression in --help menu 2020-08-24 09:49:12 -04:00
senhuang42 e3f5f9658a Added CLI tests for --no-check, fixed ignore checksum logic 2020-08-22 16:05:40 -04:00
senhuang42 20eb095882 Added unit test to fuzzer.c, changed definition param name 2020-08-22 13:26:33 -04:00
senhuang42 47685ac856 Move enum into zstd.h, and fix pesky switch() logic 2020-08-21 18:18:53 -04:00
senhuang42 08d3567ba8 Add function prototype 2020-08-21 16:51:43 -04:00
senhuang42 6a8dbdcd1f Modify decompression loop to gnore checksums if flag is enabled 2020-08-21 16:46:46 -04:00
senhuang42 2f39124342 Rename to ZSTD_d_forceIgnoreChecksum, add to DCtx, add function to set the advanced param 2020-08-21 16:23:39 -04:00
senhuang42 b5cddda073 Add new definition of ZSTD_d_forceSkipChecksum in experimental section 2020-08-21 15:59:03 -04:00
Antonio Bueno 77c97089fc Fixed Markdown warnings. No visible changes. 2020-08-19 12:36:28 +02:00
Felix Handte bb265da4ae Merge pull request #2270 from felixhandte/fix-doc-cctx-set-param 
Fix Documentation for ZSTD_CCtxParams_setParameter()
 2020-08-14 21:44:56 -04:00
W. Felix Handte 99746eea7e Fix Documentation for ZSTD_CCtxParams_setParameter() 
It does not only return 0 on success.
 2020-08-14 14:44:08 -04:00
Carl Woffenden 4c81fae146 Fix clang -Wcomma warning 2020-08-13 16:11:22 +02:00
Nick Terrell e3bda594ae Prefer __builtin_prefetch over inline asm 
Reorder the ifdefs for the PREFETCH macros so that the compiler builtin is
favored over the inline assembly for aarch64.
 2020-08-10 22:17:18 -07:00
Yann Collet 38e38546a4 Merge pull request #2258 from Niadb/dev 
Added STATIC_BMI2 for compile time detection of BMI2 on MSVC, when enabled various intrinsics are used
 2020-08-04 09:43:59 -07:00
Yann Collet 60b56e3b5f Merge pull request #2253 from facebook/histvec 
optimized histogram
 2020-08-02 14:08:42 -07:00
Nick Terrell f85a0f8bcf Merge pull request #2256 from helloguo/dev 
Optimize ZSTD_wildcopy
 2020-07-29 11:57:49 -07:00
Carl Woffenden 5d81d44e40 Fixed VS variable shadowing warning (and added test) 2020-07-29 12:33:39 +02:00
helloguo acb3dd9a68 Use ZSTD_copy16 instead of memcpy 2020-07-28 11:58:46 -07:00
Niadb a8ebc14035 Update bitstream.h 
Profiler showed some of these not being inlined on MSVC
 2020-07-28 11:17:04 -06:00
Niadb 216a63dcf7 Add files via upload 2020-07-28 02:52:52 -06:00
Niadb 493fd40dca Add files via upload 2020-07-28 02:52:15 -06:00
helloguo 82b0cd844f Optimize ZSTD_wildcopy 2020-07-27 22:08:52 -07:00
Yann Collet 8b9cdd2597 fixed overlapping count & workspace special case 2020-07-26 22:40:21 -07:00
Yann Collet 051232223f optimized histogram 
new version easier to vectorize
leads to smaller code and faster execution
notably at the last recombination stage
(basically, fixed cost per block).

Assembly inspected with godbolt

On my laptop, with `clang` and `-mavx2` :
2K block : 1280 MB/s -> 1550 MB/s
8K block : 1750 MB/s -> 1860 MB/s
 2020-07-26 22:24:22 -07:00
helloguo 6de87b3a74 fix preprocessor in ZSTD_wildcopy 2020-07-24 10:53:58 -07:00
Yann Collet c224367ede ensure workspace is large enough 
even when MAX_TABLELOG is reduced
 2020-07-16 20:33:50 -07:00
Yann Collet 21c273da84 import some minor fixes from FSE project 2020-07-16 20:25:15 -07:00
Yann Collet a44671b281 Revert "Fix -Wunused-variable under FUZZING_BUILD_MODE..." 2020-07-15 12:42:18 -07:00
Mitch Phillips 23b55d6b3e Fix -Wunused-variable under FUZZING_BUILD_MODE... 
Fuzzing build modes (FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION) doesn't
necessarily imply that assert() is enabled, according to the manual.

When the current do-nothing is expanded under -Wunused-variable (-Wall),
it results in unused variables in some of the FUZZING_BUILD_MODE...
blocks.

This patch extends the do-nothing to avoid the unused variable.
 2020-07-14 09:03:02 -07:00
Yann Collet 16b353b207 minor doc clarification regarding MT parameters 2020-07-11 02:16:52 -07:00
Yann Collet 2cdd33ae16 Merge pull request #2227 from yoshihitoh/single-file-dict-emscripten 
[contrib] Fix single-file compilation error on Emscripten build.
 2020-07-07 08:51:20 -07:00
dkcasset 82e7e2b47e Add variable for sed extended RE option (defaults to -E) 2020-06-29 13:44:23 -07:00
yoshihitoh c6548eac8e Rename static vars to avoid redefinition error. 2020-06-29 10:51:50 +09:00
dkcasset b0ed66ef92 Replace -E option with equivalent -r for older versions of sed 2020-06-26 10:43:28 -07:00
Nick Terrell 7afd5d85d3 Merge pull request #2218 from terrelln/assert-seq 
Fix unused variable warnings in fuzzing build mode without asserts
 2020-06-22 17:41:18 -07:00
Nick Terrell 081691a3aa Merge pull request #2217 from terrelln/cover-redundant 
[cover] Remove unnecessary mask and dedup hash functions
 2020-06-22 17:41:13 -07:00
Nick Terrell 370933fa20 Merge pull request #2209 from Niadb/dev 
Explicitly use __cdecl for qsort, to avoid warning when default calling convention is not __cdecl
 2020-06-22 17:41:03 -07:00
Nick Terrell cce0edfdbe Fix unused variable warnings in fuzzing build mode without asserts 
Fix unused vairable warnings when `FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION` is defined but asserts are disabled.

Fixes #2210.
 2020-06-22 12:56:57 -07:00
Nick Terrell 2312b819af [cover] Remove unnecessary mask and dedup hash functions 
* Remove the unnecessary mask, since `ZSTD_hash*()` already ensures
  the output is mod 2^h.
* Dedup the hash functions and use `ZSTD_hash*()` directly.
 2020-06-22 12:52:13 -07:00
Nick Terrell 78601d0806 Merge pull request #2212 from cwoffenden/single-file-dict 
Single-file libs now include dictBuilder
 2020-06-22 12:46:54 -07:00
Nick Terrell 1047097dad [superblock] Add defensive assert and bounds check 
The bound check condition should always be met because we selected `set_basic` as
our encoding type. But that code is very far away, so assert it is true so if it is
ever false we can catch it, and add a bounds check.

Fixes #2213.
 2020-06-22 10:21:38 -07:00
Carl Woffenden 38cdb6a072 Renamed cover and fast cover hash functions/vars 2020-06-22 11:54:24 +02:00
Carl Woffenden 4a9b7d136f Initial implementation (files added, macros fixed) 
Hashing functions still to fix.
 2020-06-22 10:31:36 +02:00
Niadb 74f65f624c Update compiler.h 
clean wording
 2020-06-19 09:51:00 -06:00
Niadb 8c115cbe23 Update compiler.h 
Added a comment explaining the purpose of the WIN_CDECL macro
 2020-06-19 09:48:35 -06:00
Niadb 2962fda93f Add files via upload 2020-06-19 03:34:05 -06:00
Niadb 405586d40a Add files via upload 2020-06-19 03:32:11 -06:00
Niadb a4c8aa5e02 Add files via upload 2020-06-19 03:31:47 -06:00
Nick Terrell 08981d2638 [lib] Allow compression dictionaries with missing symbols 
Allow compression to use dictionaries with missing symbols in their
entropy tables. We set the FSE repeat mode to check when there are
missing symbols, and set the FSE repeat mode to valid when all symbols
are present.

Note that when not all symbols are present, the heuristics which favor
dictionary tables for lower compression levels won't activate.

Tested by manually creating a dictionary with missing symbols of every
type, and validing that the compressor rejects it before this change,
and accepts it after this change. Also, I ran the `dictionary_loader`
fuzzer for >1 hour of CPU time without running into cases where
compression succeeds, but decompression fails.

Fixes #2174.
 2020-06-12 17:57:19 -07:00