ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,624 Commits 17 Branches 77 Tags
cd9d0a7e6e5aa770b377d738eb94de6671c5487e
Commit Graph
9624 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell 785331acbd [paramgrill] Fix mingw build errors 2019-04-18 15:06:56 -07:00
Nick Terrell 579f3d7794 [legacy] Fix bug in ZSTD_decodeSeqHeaders() 2019-04-18 13:41:10 -07:00
Nick Terrell ac098c7f5f [legacy] Fix a bug in ZSTDv06_findFrameSizeInfoLegacy() 2019-04-18 13:33:26 -07:00
Nick Terrell 610a81ecf9 [fuzzer] Compile with legacy support 2019-04-18 12:44:55 -07:00
Nick Terrell cc669006dc [fuzzer] Size the decompression output buffer randomly 2019-04-18 12:44:21 -07:00
Nick TerrellandGitHub af3531ee35 Merge pull request #1590 from terrelln/frame-info-fuzz 
[fuzzer] Fuzz frame info functions
 2019-04-17 12:13:49 -07:00
Nick Terrell 58bcc328a4 [fuzz] Add a seedcorpora target for oss-fuzz 2019-04-17 12:13:06 -07:00
Nick Terrell ee130a9889 [libzstd] Check the size in readSkippableFrameSize() 2019-04-17 11:41:55 -07:00
Nick Terrell 5922f4e2ae [legacy] Return the right error code 2019-04-17 11:34:52 -07:00
Nick Terrell 09caa4d800 [fuzzer] Add a fuzzer for frame info functions 
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
 2019-04-17 11:29:42 -07:00
Nick Terrell 450feb0f95 [libzstd] Fix ZSTD_decompressBound() on bad skippable frames 
The function didn't verify that the skippable frame size is correct.
 2019-04-17 11:29:42 -07:00
Nick TerrellandGitHub 83b51e9f88 Merge pull request #1586 from facebook/dev 
v1.4.0
v1.4.0 		2019-04-16 15:37:24 -07:00
Nick TerrellandGitHub b85f7d7790 Merge pull request #1589 from terrelln/appveyorTest 
[libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx
 2019-04-16 12:32:19 -07:00
Nick Terrell a17fe4c9e5 [visual] Fix unreachable code warning 2019-04-16 11:32:35 -07:00
Yann ColletandGitHub 41ad46f1a9 Merge pull request #1588 from terrelln/release-prep 
[fullbench] Fix speed measurements
 2019-04-16 10:11:28 -07:00
Nick Terrell 82d5a9870e [visual] Compile with ZSTD_MULTITHREAD=1 
This branch `appveyorTests` runs the tests that `master` runs, so
passing Appveyor means that the tests are fixed.
 2019-04-15 23:04:46 -07:00
Nick Terrell de0499f7fa [libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx 
ZSTDMT was broken when compiled without ZSTD_MULTITHREAD defined,
because `ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, nbWorkerss)`
failed. It was detected by the MSVC test which runs the fuzzer with
multithreading disabled.

This is a very niche use case of a deprecated API, because the API is
inefficient and synchronous, since `threading.h` will be synchronous.
Users almost certainly don't want this, and anyone who tested their code
should realize that it is broken. Therefore, I think it is safe to
require `ZSTD_MULTITHREAD` to be defined to use ZSTDMT.
 2019-04-15 23:04:46 -07:00
Nick Terrell a48e32ca50 [fullbench] Fix measurements 2019-04-15 21:01:48 -07:00
Nick TerrellandGitHub a821aee3b1 Merge pull request #1587 from terrelln/release-prep 
[playTets] Don't pass status=none to dd
 2019-04-15 16:42:26 -07:00
Nick Terrell 6996804d05 [playTets] Don't pass status=none to dd 
The OS X test machine on TravisCI doesn't recognize the parameter
https://travis-ci.org/facebook/zstd/jobs/520517933.
 2019-04-15 16:21:46 -07:00
Nick TerrellandGitHub d86c7ad458 Merge pull request #1585 from terrelln/release-prep 
Prepare for the zstd-1.4.0 release
 2019-04-15 14:41:50 -07:00
Nick Terrell bef61221a3 Update the changelog for zstd-1.4.0 2019-04-15 14:33:46 -07:00
Nick Terrell 203aac5bc6 Update benchmarks in README 2019-04-15 13:57:14 -07:00
Josh SorefandNick Terrell a880ca239b Spelling (#1582) 
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
 2019-04-12 11:18:11 -07:00
Nick TerrellandGitHub 13fddac46d Merge pull request #1583 from lzutao/meson 
Fix tests and meson build
 2019-04-12 11:13:20 -07:00
Lzu Tao e31e7ca9ed travis: Make Mesonbuild non-optional 2019-04-12 23:43:34 +07:00
Lzu Tao 6c9b023f3e meson: Fix build 2019-04-12 21:28:36 +07:00
Lzu Tao fb6901b2c0 tests: Add missing header timefn.h 2019-04-12 21:28:17 +07:00
Yann ColletandGitHub 8ac2831f3d Merge pull request #1581 from facebook/benchfn 
benchfn's reduced dependencies
 2019-04-11 14:23:04 -07:00
Yann Collet 1e01560b83 fixed timespec_get() initialization bug on some targets 
not sure why, but msan fires an "unitialized variable" error
when time gets properly initialized by timespec_get().
Maybe in some cases, not all bytes of the structure are initialized ?
Or maybe msan fails to detect the initialization ?

Anyway, pre-initializing the variable before passing it to timespec_get() works.
 2019-04-11 13:46:30 -07:00
Yann Collet 058da605cb fixed minor conversion warning 2019-04-11 12:25:27 -07:00
Yann Collet f8e9bec73a fixed poolTests on Windows 
must use ZSTD_ prefix in front of pthread types
so that they get properly translated for Windows.
 2019-04-11 12:03:42 -07:00
Yann Collet 8f56fa2f58 fixed poolTests 
added poolTests to all
 2019-04-11 09:50:39 -07:00
Nick TerrellandGitHub 9563fb4bfc Merge pull request #1580 from terrelln/dict-fuzz 
[libzstd] Fix decompression dictionary bugs and clean up initialization
 2019-04-10 19:29:09 -07:00
Yann Collet 30c26ab726 fixed minor warning 
unused variable when assert() turned off in fileio.c
 2019-04-10 17:48:56 -07:00
Yann Collet fbdd30d68e fixed cmake build script for test programs 2019-04-10 17:47:01 -07:00
Nick Terrell aafe97b67d [libzstd] Switch dictUses to an enum 2019-04-10 16:50:35 -07:00
Yann Collet 526ec646b7 alternate static assert 
to circumvent Visual's C4804 warning
 2019-04-10 16:05:02 -07:00
Yann Collet 9703a59121 fixed minor conversion warning 2019-04-10 15:54:55 -07:00
Yann Collet 885476fb5b FreeBSD_11 specific fix 
C11 mandates the definition of timespec_get() and TIME_UTC.
However, FreeBSD11 announce C11 compliance, but does not provifr timespec_get(),
breaking link stage for benchfn.
Since it does not provide TIME_UTC either, which is also required by C11,
test this macro: this will automatically rule out FreeBSD 11 for this code path
(it will use the backup C90 path instead, based on clock_t).

The issue seeems fixed in FreeBSD 12.
 2019-04-10 15:22:18 -07:00
Yann Collet 2fa4f2e246 updated Visual projects 
added timefn
 2019-04-10 15:07:36 -07:00
Yann Collet 2c6b14ed22 fixed Windows header 
cmake build script: added timefn
 2019-04-10 14:54:13 -07:00
Yann Collet 3d346579d8 no more need for CLOCK_MONOTONIC 2019-04-10 14:16:39 -07:00
Yann Collet 36d2dfd846 moved C11 code path to timespec_get 2019-04-10 14:15:11 -07:00
Yann Collet 4765929271 fixed perror include 2019-04-10 14:04:11 -07:00
Yann Collet 70802cde6d fixed error message 
using stdlib's perror()
 2019-04-10 14:01:18 -07:00
Yann Collet 4b8185c7fc tried a blindfix for unix + c11 2019-04-10 13:26:27 -07:00
Nick Terrell 50b9c41196 [libzstd] Fix decompression dictionary bugs and clean up initialization 
Bugs:

* `ZSTD_DCtx_refPrefix()` didn't clear the dictionary after the first
  use. Fix and add a test case.
* `ZSTD_DCtx_reset()` always cleared the dictionary. Fix and add a test
  case.
* After calling `ZSTD_resetDStream()` you could no longer load a
  dictionary, since the stage was set to `zdss_loadHeader`. Fix and add
  a test case.

Cleanup:

* Make `ZSTD_initDStream*()` and `ZSTD_resetDStream()` wrap the new
 advanced API, and add test cases.
* Document the equivalent of these functions in the advanced API and
  document the unstable functions as deprecated.
 2019-04-10 12:59:02 -07:00
Yann Collet 59a7116cc2 benchfn dependencies reduced to only timefn 
benchfn used to rely on mem.h, and util,
which in turn relied on platform.h.
Using benchfn outside of zstd required to bring all these dependencies.

Now, dependency is reduced to timefn only.
This required to create a separate timefn from util,
and rewrite benchfn and timefn to no longer need mem.h.

Separating timefn from util has a wide effect accross the code base,
as usage of time functions is widespread.
A lot of build scripts had to be updated to also include timefn.
 2019-04-10 12:37:03 -07:00
Yann Collet 094c000904 Merge branch 'dev' into benchfn 2019-04-10 11:57:05 -07:00