ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,624 Commits 17 Branches 77 Tags
cd9d0a7e6e5aa770b377d738eb94de6671c5487e
Commit Graph
1736 Commits
Author SHA1 Message Date
senhuang42 1acf243540 Add a warning whenever (de)compressing multiple files into one source, or into stdout 2020-08-24 19:10:03 -04:00
Nick Terrell 52f33a1da5 Fix compiler warnings 2020-08-24 16:09:45 -07:00
senhuang42 a030560d62 Add new DCtx param: validateChecksum and update unit tests 2020-08-24 17:28:00 -04:00
Nick Terrell 1302f8d676 [fix] Always return dstSize_tooSmall when it is the case 2020-08-24 13:38:13 -07:00
senhuang42 44c54a3e31 Addressing comments: more comments, cleanup, remove extra function, checksum logic 2020-08-24 16:14:19 -04:00
Nick Terrell 8def0e5fd3 Fix up code after reading through 2020-08-24 12:24:45 -07:00
senhuang42 ffaa0df76d Document change in CLI for --no-check during decompression in --help menu 2020-08-24 09:49:12 -04:00
senhuang42 e3f5f9658a Added CLI tests for --no-check, fixed ignore checksum logic 2020-08-22 16:05:40 -04:00
senhuang42 20eb095882 Added unit test to fuzzer.c, changed definition param name 2020-08-22 13:26:33 -04:00
senhuang42 1b34b15e6b Adding CLI capability to invoke decompression with no checksum 2020-08-21 17:49:30 -04:00
senhuang42 6a8dbdcd1f Modify decompression loop to gnore checksums if flag is enabled 2020-08-21 16:46:46 -04:00
Nick Terrell 8f8bd2d1ac [regression] Update results.csv 2020-08-20 12:41:35 -07:00
Nick Terrell 575731b6db Use ncount=1 when < 4096 symbols 2020-08-18 16:47:53 -07:00
Nick Terrell 612e947c5e wire up bmi2 support 2020-08-17 16:35:28 -07:00
Nick Terrell a8006264cf small blocks benchmark 2020-08-14 18:57:20 -07:00
Yann Collet 23941eec04 added tests for newly enabled syntax 
for --patch-from origin
and --filelist list

Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.

For example :
    zstd file -of dest
used to be disallowed.

It's now allowed, and understood as:
    zstd file -o dest -f
 2020-07-17 13:31:15 -07:00
Xin Xie 9a8ccd4ba3 Add output-dir-mirror option 2020-06-24 22:12:11 -07:00
Bimba Shrestha de48f35306 adding --patch-from --stream-size test 2020-06-18 10:28:37 -07:00
Nick Terrell 08981d2638 [lib] Allow compression dictionaries with missing symbols 
Allow compression to use dictionaries with missing symbols in their
entropy tables. We set the FSE repeat mode to check when there are
missing symbols, and set the FSE repeat mode to valid when all symbols
are present.

Note that when not all symbols are present, the heuristics which favor
dictionary tables for lower compression levels won't activate.

Tested by manually creating a dictionary with missing symbols of every
type, and validing that the compressor rejects it before this change,
and accepts it after this change. Also, I ran the `dictionary_loader`
fuzzer for >1 hour of CPU time without running into cases where
compression succeeds, but decompression fails.

Fixes #2174.
 2020-06-12 17:57:19 -07:00
Bimba Shrestha e2838d9eb9 Spelling mistakes 2020-06-05 05:11:26 -05:00
Shaojing Li 847349195f fix the if statements in posix sh env 2020-06-03 11:36:38 -07:00
Shaojing Li 3a3da1712b check env variables and add default values 2020-06-03 10:49:21 -07:00
Bimba Shrestha b0f851675a [shellcheck] setting if unset 2020-06-02 09:12:50 -07:00
Bimba Shrestha 151deaf143 [shellcheck] adding quotes to expansion 2020-06-02 09:12:13 -07:00
Yann Collet 26b21e481f fix meson playTests.sh 2020-05-21 15:17:22 -07:00
Nick Terrell 651d3d73e0 [test] Update the ldm loadedDictEnd test to cover zstdmt 2020-05-19 16:14:14 -07:00
Nick TerrellandGitHub 0dcd3eec43 Merge pull request #2152 from terrelln/simple-rt-bound 
[fuzz] Expand the allowedExpansion
 2020-05-19 12:56:11 -07:00
Nick Terrell b82bf711fc [fuzz] Expand the allowedExpansion 2020-05-19 11:42:53 -07:00
Yann ColletandGitHub fdc56baa42 fix 22294 (#2151) 2020-05-18 21:05:10 -07:00
Nick TerrellandGitHub 9778f46014 Merge pull request #2150 from terrelln/ldm-dict-reset 
[ldm] Reset loadedDictEnd when the context is reset
 2020-05-18 18:33:01 -07:00
Nick Terrell 7b317b4876 [test] Test that the ldm dictionary gets invalidated on reset 2020-05-18 16:00:28 -07:00
Nick Terrell 87dbd6d4bf [test] Improve LDM forceMaxWindow test 2020-05-18 15:11:18 -07:00
W. Felix Handte d37fcf36eb Don't Use [[ in Shell Scripts 2020-05-18 15:06:56 -04:00
Bimba ShresthaandGitHub 255e5e3f56 [fuzz] Adding dictionary_stream_round_trip fuzzer (#2140) 
* Adding dictionary_stream_round_trip

* fixing memory leak
 2020-05-15 13:33:31 -07:00
Nick Terrell 608075abb2 [test][regression] Update results.csv 2020-05-14 17:06:39 -07:00
Nick Terrell bf0591e1e2 [test] Expose the LDM+MT+dict bug in a unit test 2020-05-14 12:06:55 -07:00
Bimba Shrestha 12071467d3 reverting docs and test 2020-05-13 15:22:07 -05:00
Nick TerrellandGitHub c3e921c639 Merge pull request #2131 from terrelln/raw-dict-fuzzer 
Fix rare scenario with lazy parser, dictionary, and repcodes
 2020-05-12 17:44:31 -07:00
Bimba Shrestha 0453cfa8f5 removing -f test (grep usage not supported on mac) 2020-05-12 15:18:43 -05:00
Nick Terrell 4b88bd3ee0 [lib][fuzz] Assert sequences are valid in round trip tests 2020-05-11 20:38:49 -07:00
Yann Collet e001715b3d fixed asan test 2020-05-11 20:35:47 -07:00
Yann Collet 20bd246045 blindfix for VS macro redefinition 2020-05-11 19:29:36 -07:00
Nick Terrell 1185dfb8d1 [fuzz] Add raw dictionary content fuzzer 2020-05-11 19:03:33 -07:00
Nick Terrell 301a62fe08 [fuzz] Fix compress bound for dictionary_round_trip 2020-05-11 19:00:52 -07:00
Yann Collet 91ad01218e updated initStatic tests 
differentiate small CCtx for small inputs
from full CCtx
from CStream contexts.

Ensure allocation & resize tests are more precise.
 2020-05-11 18:50:10 -07:00
Yann Collet 608f1bfc4c fixed context downsize with initStatic 
When context is created using initStatic,
no resize is possible.

fix : only bump oversizeDuration when !initStatic
 2020-05-11 18:16:38 -07:00
Yann Collet dd026ca505 re-inforced tests for initStaticCCtx 
ensure that `estimateCCtxSize()` works as intended.
 2020-05-09 11:30:45 -07:00
Nick TerrellandNick Terrell 5717bd39ee [lib] Fix NULL pointer dereference 
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.

This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.

* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.

Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.

Credit to OSS-Fuzz.
 2020-05-06 12:09:02 -07:00
Bimba ShresthaandGitHub e7df0d41bb Merge pull request #2095 from bimbashrestha/grep 
[bugs] zstdgrep/grep inconsistencies
 2020-05-06 11:18:15 -05:00
Bimba Shrestha 250184adf6 adding tests back 2020-05-05 16:51:06 -07:00