ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,141 Commits 17 Branches 77 Tags
db8efbfe7d5ce6df3e09ef2be93eb4234b32f511
Commit Graph
3084 Commits
Author SHA1 Message Date
Sen Huang d9646dcbb5 Fixed main compression logic changes 2019-11-14 19:39:09 -05:00
Sen Huang c26d32c91c Change superblock #include to be last 2019-11-14 13:12:17 -05:00
Sen Huang 13bb7500e8 Fix frame argument to compression 2019-11-05 16:15:55 -05:00
Sen Huang f2932fb5eb Fix more merge conflicts 2019-11-05 15:54:05 -05:00
Sen Huang 7ce891870c Fix merge conflicts 2019-11-05 15:51:25 -05:00
Bimba Shrestha 3fb5b106da Replacing some literals with constants 2019-11-05 10:26:57 -08:00
Nick Terrell 60205fec02 Fix 2 bugs in dictionary loading 
* Silently skip dictionaries less than 8 bytes, unless using `ZSTD_dct_fullDict`.
  This changes the compressor, which silently skips dictionaries <= 8 bytes.
* Allow repcodes that are equal to the dictionary content size, since it is in bounds.
 2019-11-01 16:52:07 -07:00
Sen Huang b9ede1c8c2 Make sure contentsize is known 2019-10-30 16:03:58 -04:00
Nick Terrell 9c1860861e Fix assert in ZSTD_safecopy 
In the case that `op >= oend_w` it is possible that `diff < 8` because
the two buffers could be adjacent.

Credit to OSS-Fuzz, which found the bug. It isn't reproducible because
it depends on the memory layout.
 2019-10-28 17:51:17 -07:00
Felix HandteandGitHub 01ec595b85 Merge pull request #1851 from felixhandte/pkg-config-prefix-fix 
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
 2019-10-28 14:24:56 -04:00
Yann Collet 74065da4c5 updated API inline doc and manual 
regarding ZSTD_CDict created without a dictBuffer.
 2019-10-28 11:15:41 -07:00
W. Felix Handte 74bd76c3ff In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time 
Addresses #1794. Instead of deriving the lib dir and include dir at
build-time, let's do it like everyone else does at pkg-config run-time.

This has the disadvantage that we can no longer override LIBDIR and
INCLUDEDIR in the Makefile and have that reflected in the .pc file.
 2019-10-25 15:07:31 -04:00
Yann ColletandGitHub c2140e9db0 Merge pull request #1845 from facebook/zbuff 
improve deprecation warning macro
 2019-10-25 09:59:00 -07:00
Yann ColletandGitHub a9a216a846 Merge pull request #1824 from senhuang42/new_path_for_cdict 
Avoid using CDict params when input is large.
 2019-10-23 12:04:40 -07:00
Yann Collet 63e435dda1 improve deprecation warning macro 
fix #1488

although, curiously enough, I was never able to reproduce the issue
(according to the bug report, it should be present while using gcc 4.8).
 2019-10-23 11:59:32 -07:00
Yann Collet f966cd080a added documentation on DYNAMIC_BMI2 build macro 2019-10-22 17:43:09 -07:00
Yann Collet 5d5c895b18 fix initCStream_advanced() for fast strategies 
Compression ratio of fast strategies (levels 1 & 2)
was seriously reduced, due to accidental disabling of Literals compression.

Credit to @QrczakMK, which perfectly described the issue, and implementation details,
making the fix straightforward.

Example : initCStream with level 1 on synthetic sample P50 :
Before : 5,273,976 bytes
After  : 3,154,678 bytes
ZSTD_compress (for comparison) : 3,154,550

Fix #1787.

To follow : refactor the test which was supposed to catch this issue (and failed)
 2019-10-22 15:01:38 -07:00
Yann Collet 111b0c53b0 update documentation on deprecated functions 
mostly : note that these functions will soon generate deprecation warnings
 2019-10-22 13:51:18 -07:00
Nick Terrell b1ec94e63c Fix ZSTD_f_zstd1_magicless for small data 
* Fix `ZSTD_FRAMEHEADERSIZE_PREFIX` and `ZSTD_FRAMEHEADERSIZE_MIN` to
  take a `format` parameter, so it is impossible to get the wrong size.
* Fix the places that called `ZSTD_FRAMEHEADERSIZE_PREFIX` without
  taking the format into account, which is now impossible by design.
* Call `ZSTD_frameHeaderSize_internal()` with `dctx->format`.
* The added tests catch both bugs in `ZSTD_decompressFrame()`.

Fixes #1813.
 2019-10-21 21:16:17 -07:00
Sen Huang c2e1e54f24 ((x or y) or z) == (x or y or z), remove brackets 2019-10-21 19:16:50 -04:00
Sen Huang 59c81aa31b Line up comments :) 2019-10-21 19:12:15 -04:00
Sen Huang dbda8c318a Trailing comma 2019-10-21 19:10:13 -04:00
Sen Huang 0c00455ea6 Merge branch 'dev' of github.com:senhuang42/zstd into new_path_for_cdict 2019-10-21 19:06:51 -04:00
Sen Huang 5b2f4ac1a8 merge 2019-10-21 19:02:52 -04:00
Sen Huang 2ab484a5f9 Fix bad merge 2019-10-21 18:55:17 -04:00
Nick TerrellandGitHub 919d1d8e93 Merge pull request #1831 from terrelln/zstdmt-bad-memset 
[zstdmt] Don't memset the jobDescription
 2019-10-21 15:53:57 -07:00
Sen Huang b6c3459d50 merge 2019-10-21 18:46:17 -04:00
Yann ColletandGitHub 6cf04c0344 Merge pull request #1834 from facebook/winFix 
Windows fixes
 2019-10-21 13:45:17 -07:00
Sen Huang 676f89902a Added multiplier, renamed new enum to something more useful 2019-10-21 15:36:12 -04:00
Sen Huang 1f3a51fb52 Updated forceAttachDict param bounds 2019-10-21 15:36:12 -04:00
Sen Huang 8f69c47643 Add enum to decision process 2019-10-21 15:36:12 -04:00
Sen Huang e4de8b098a Added support for forcing new CDict behavior and updated enum 2019-10-21 15:36:12 -04:00
Sen Huang 9294f4826b Changed to int from BYTE 2019-10-21 15:36:12 -04:00
Sen Huang f0fccc8847 Changed to int from BYTE 2019-10-21 15:36:12 -04:00
Sen Huang bb2df8c499 Trailing whitespace 2019-10-21 15:36:12 -04:00
Sen Huang cf51501d2f Fix test 2019-10-21 15:36:12 -04:00
Sen Huang ea3cb6988f Cast to BYTE to appease appveyor 2019-10-21 15:36:12 -04:00
Sen Huang a727a85a7e merge conflicts round 2 2019-10-21 15:36:12 -04:00
Sen Huang 053a35fd64 formatting 2019-10-21 15:35:33 -04:00
Sen Huang 3fa4daaa55 Fix error 2019-10-21 15:35:33 -04:00
Sen Huang 3328348c63 Add compressionlevel to cdict 2019-10-21 15:32:39 -04:00
Felix HandteandGitHub cf725630a6 Merge pull request #1795 from felixhandte/workspace-asan 
Add Poisoned Redzones to the Workspace When Compiling with ASAN
 2019-10-21 12:15:17 -04:00
Sen Huang e8aa3e486d Updated forceAttachDict param bounds 2019-10-20 22:01:08 -04:00
Sen Huang 6d297265f9 Add enum to decision process 2019-10-20 19:02:47 -04:00
Sen Huang 1daa898c93 Added support for forcing new CDict behavior and updated enum 2019-10-20 14:03:09 -04:00
Nick Terrell 0bc39bc3a0 [zstdmt] Don't memset the jobDescription 2019-10-18 15:05:51 -07:00
Nick Terrell 243824551f [threading] Add debug utilities 2019-10-18 15:05:34 -07:00
Yann Collet 1795133c45 refactored FIO_compressMultipleFilenames() prototype 
for consistency
 2019-10-17 15:32:03 -07:00
Yann ColletandGitHub 6446ffb277 Merge pull request #1827 from facebook/dm_Dct 
updated erroneous comments using ZSTD_dm_*
 2019-10-17 10:30:58 -07:00
Yann ColletandGitHub 19741c7d99 Merge pull request #1815 from facebook/zlibwrap 
make zlibWrapper strict ISO-C90 compatible
 2019-10-16 16:45:15 -07:00