ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,656 Commits 17 Branches 77 Tags
dev
Commit Graph
1395 Commits
Author SHA1 Message Date
Nick Terrell bfd943ace5 Fix buffer overrun in ZSTD_loadDictEntropyStats() 
The table log set by `FSE_readNCount()` was not checked in
`ZSTD_loadDictEntropyStats()`.  This caused `FSE_buildCTable()`
to stack/heap overflow in a few places.

The benchmarks look good, there is no obvious compression performance regression:

  > ./zstds/zstd.opt.0 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 271.6 MB/s , 716.8 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 204.8 MB/s , 671.1 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 156.8 MB/s , 658.6 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 136.4 MB/s , 665.3 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  98.9 MB/s , 647.0 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  65.2 MB/s , 670.4 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.9 MB/s , 688.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  32.4 MB/s , 711.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.1 MB/s , 718.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  22.2 MB/s , 721.8 MB/s

  > ./compress_zstds/zstd.opt.1 -i10 -b1 -e10 ~/bench/silesia.tar
   1#silesia.tar       : 211988480 ->  73656930 (2.878), 273.8 MB/s , 722.0 MB/s
   2#silesia.tar       : 211988480 ->  70162842 (3.021), 203.2 MB/s , 666.6 MB/s
   3#silesia.tar       : 211988480 ->  66997986 (3.164), 157.4 MB/s , 666.5 MB/s
   4#silesia.tar       : 211988480 ->  66002591 (3.212), 132.1 MB/s , 661.9 MB/s
   5#silesia.tar       : 211988480 ->  65008480 (3.261),  96.8 MB/s , 641.6 MB/s
   6#silesia.tar       : 211988480 ->  62979643 (3.366),  63.1 MB/s , 677.0 MB/s
   7#silesia.tar       : 211988480 ->  61974560 (3.421),  44.3 MB/s , 678.2 MB/s
   8#silesia.tar       : 211988480 ->  61028308 (3.474),  33.1 MB/s , 708.9 MB/s
   9#silesia.tar       : 211988480 ->  60416751 (3.509),  21.5 MB/s , 710.1 MB/s
  10#silesia.tar       : 211988480 ->  60174239 (3.523),  21.9 MB/s , 723.9 MB/s
 2016-10-17 16:55:52 -07:00
Yann Collet 2b361cf2f1 minor opt 2016-10-14 16:09:07 -07:00
Nick Terrell 3b9cdf9220 Fix ubsan failures (pass NULL to memcpy) 2016-10-12 20:54:42 -07:00
Yann Collet cf409a7e2a fixed : init*_advanced() followed by reset() with different pledgedSrcSiz 2016-09-26 16:41:05 +02:00
Yann Collet 97b378a6f8 Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize. 2016-09-21 17:20:19 +02:00
Yann Collet 993060e0f2 cli : better adaptation to small files 2016-09-21 16:46:08 +02:00
Yann Collet a6bdf55759 fixed memory leak 2016-09-15 17:02:06 +02:00
Yann Collet 644a8da88a fixed minor conversion warning 2016-09-15 16:16:21 +02:00
Yann Collet 4cb212938c introduced ZSTD_resetCStream() 2016-09-15 14:54:07 +02:00
Yann Collet fa0c09760c variable renaming 2016-09-15 14:11:01 +02:00
Yann Collet d7c6589df8 support ZSTD_sizeof_*() on NULL 
added ZSTD_sizeof_CDict()
 2016-09-15 02:57:27 +02:00
Yann Collet 64deef3bee Fixed srcSize=1 2016-09-14 00:16:07 +02:00
Yann Collet ac8bace6b1 support large skippable frames 2016-09-07 14:54:23 +02:00
Yann Collet 95d07d7447 introduced CHECK_E 2016-09-06 16:38:51 +02:00
Yann Collet 3e21ec5b01 introduced CHECK_F 2016-09-06 15:36:19 +02:00
Yann Collet 5c956d593c FORCE_INLINE common definition 2016-09-06 15:05:19 +02:00
Yann Collet edbcd9f5b2 fixed zbufftest 2016-09-06 14:30:57 +02:00
Yann Collet b624922b14 fixed checksum 2016-09-06 11:16:57 +02:00
Yann Collet a7737f6a60 improved compression on small files when using same parameters 2016-09-06 09:44:59 +02:00
Yann Collet 7ae67bb18a small compression speed gains with using_CDict 2016-09-06 06:28:05 +02:00
Yann Collet 855766d73d clarified dictionary in format description 2016-09-02 17:04:49 -07:00
Yann Collet 1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
David Lam e10f7f3dcb merge 2016-08-30 12:03:36 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 14200a20f0 Fixed issue #304, reported by @borzunov 2016-08-30 06:51:00 -07:00
David Lam da9d3b7057 Cleanup some errors in typedef comments and remove duplicated HOWTO from zbuff_decompress.c 2016-08-29 17:31:51 -07:00
Yann Collet 23b6e05d8e ZSTD_malloc() and ZSTD_free(), to simplify customMem 2016-08-28 21:05:43 -07:00
Yann Collet e19a9ef05d update compression level table 2016-08-26 20:02:49 +02:00
Yann Collet 87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
Yann Collet 0d59a6f73a removed debug strings 2016-08-25 22:42:46 +02:00
Yann Collet 5a02b69215 reinforced fix for huge files 2016-08-25 22:24:59 +02:00
Yann Collet 96bdd87de4 fixed : compression bug on very large files 2016-08-25 14:34:42 +02:00
inikep a3a47ec4d0 Merge remote-tracking branch 'refs/remotes/Cyan4973/dev' into Other 2016-08-24 21:25:49 +02:00
Yann Collet a2cdffe556 fixed wrong parameter issue 2016-08-24 19:42:15 +02:00
inikep e416e30019 remove unnecessary comments 2016-08-24 17:32:09 +02:00
inikep 4e90f6c1e0 removed ZSTD_LOG_ENCODE and ZSTD_LOG_BLOCK 2016-08-24 17:24:11 +02:00
inikep 8a36f8527c removed stats in debug mode 2016-08-24 17:19:12 +02:00
Yann Collet 24b68a5b23 update cLevel table for 256KB 2016-08-24 14:22:26 +02:00
Yann Collet c54692faeb improved level 3 2016-08-24 01:45:46 +02:00
Yann Collet 70e3b31306 fixed playtests on os-x 2016-08-23 01:18:06 +02:00
Yann Collet cb3276329a added sizeof CStream and DStream 2016-08-23 00:31:59 +02:00
Yann Collet c411902230 fixed g++ conversion warning 2016-08-17 01:50:54 +02:00
Yann Collet 53e17fbd5e updated streaming API 2016-08-17 01:39:22 +02:00
Yann Collet 104e5b072d added : streaming decompression API 2016-08-16 15:11:28 +02:00
Yann Collet 5a0c8e2439 new streaming API (compression) 2016-08-16 15:11:27 +02:00
inikep 98e08cbe34 fixed: tree not updated after finding very long rep matches 2016-08-10 15:00:30 +02:00
Yann Collet 280f9a8754 minor comment 2016-08-08 00:44:00 +02:00
Yann Collet 0763905f44 ZSTD_compress_usingCDict() correctly provides original size by default in frame header 
Fixed dictionary examples
 2016-08-03 01:57:57 +02:00
Yann Collet 346efccc35 fixed doc typo 2016-08-02 14:26:00 +02:00
Yann Collet 3b2bd1d11c zstd_opt uses same tables as zstd_compress 2016-07-30 13:21:41 +02:00