Commit Graph
6506 Commits
Author SHA1 Message Date
Yann Collet 9719fd616c removed nextToUpdate3 from ZSTD_window
it's now a local variable of ZSTD_compressBlock_opt()
2019-05-28 16:18:12 -07:00
Yann Collet 33dabc8c80 get bt matches : made it a bit clearer which parameters are input and output 2019-05-28 16:11:32 -07:00
Yann Collet 327cf6fac1 nextToUpdate3 does not need to be maintained outside of zstd_opt.c
It's re-synchronized with nextToUpdate at beginning of each block.
It only needs to be tracked from within zstd_opt block parser.

Made the logic clear, so that no code tried to maintain this variable.

An even better solution would be to make nextToUpdate3
an internal variable of ZSTD_compressBlock_opt_generic().
That would make it possible to remove it from ZSTD_matchState_t,
thus restricting its visibility to only where it's actually useful.

This would require deeper changes though,
since the matchState is the natural structure to transport parameters into and inside the parser.
2019-05-28 15:26:52 -07:00
Yann Collet 6453f8158f complementary code comments
on variables used / impacted during maxDist check
2019-05-28 14:12:16 -07:00
Yann Collet 4baecdf72a added comments to better understand enforceMaxDist() 2019-05-28 13:15:48 -07:00
Felix HandteandGitHub 21bb78e908 Merge pull request #1618 from felixhandte/zstgrep-f-flag
Handle `-f` Flag in `zstgrep`
2019-05-24 20:59:11 -04:00
W. Felix Handte 6a0638048a Add Test 2019-05-24 17:21:44 -04:00
W. Felix Handte 61025d5b7d zstdgrep: Handle -f Flag 2019-05-24 16:55:59 -04:00
W. Felix Handte ff0be17cf7 Build Manual 2019-05-24 16:55:43 -04:00
Tyler-TranandNick Terrell cb47871a0a [dictBuilder] Be more specific than ERROR(generic) (#1616)
* Specify errors at a finer granularity than `ERROR(generic)`.
* Add tests for bad parameters in the dictionary builder.
2019-05-22 18:57:50 -07:00
Nick TerrellandGitHub b114477f2a Merge pull request #1613 from neheb/patch-1
fileio: Disable BACKTRACE under uClibc
2019-05-06 17:12:03 -07:00
Rosen PenevandGitHub 23b5ee1236 fileio: Disable BACKTRACE under uClibc
uClibc does not support BACKTRACE. It also defines __GLIBC__ and minor as both 2.
This fixes compilation under such a setup.
2019-05-06 15:02:47 -07:00
Nick TerrellandGitHub bfbef51afb Merge pull request #1610 from terrelln/pool-test
[test] Fix poolTests deadlock and flakiness
2019-05-02 15:13:50 -07:00
Nick Terrell 81f1f76479 [tests] Only sleep 10ms in poolTests 2019-05-02 14:35:25 -07:00
Nick Terrell 2953d45a0f [test] Fix poolTests deadlock and flakiness 2019-05-02 13:23:41 -07:00
lzutaoandNick Terrell 1d70bc3ba9 meson: Enable testing in CI (#1609)
* tests: Fix shellcheck warnings in playTests.sh

* tests: Do not use ../programs which is relative to tests dirs

This commit fixes error when running playTests.sh in Meson.
Mesonbuild runs out of tree, so ./datagen not in `zstd/tests` dir,
it lies in <mesonbuilddir>/tests. This leads to ../programs invalid.

* tests: Replace relative paths for zstd/tests dir

* playTests: Set shell options explicitly, not in shebang

* playTests: Replace echo -e with printf

* meson: Fix test-zstd

Use std=gnu99 to build and test just like `make test`.

* meson: Fix legacy test

* meson: Enable testing in CI

Run build under release mode for faster test time.

* meson: Increase timeout time for test-zstream
2019-05-02 12:35:37 -07:00
Nick TerrellandGitHub f8178ec74e Merge pull request #1602 from lzutao/meson
meson: Update default project version
2019-04-30 08:45:00 -07:00
Lzu Tao 5d900ff747 meson: Error out if fail to extracted version number 2019-04-30 13:12:00 +07:00
Nick TerrellandGitHub 69baaee3e4 Merge pull request #1604 from vejnar/dev
Make zstdcat to follow symbolic links
2019-04-29 17:02:42 -07:00
cev 3e1e49d775 add test for zstdcat and zcat on symlink 2019-04-29 18:47:47 -04:00
Nick TerrellandGitHub 9ef732f653 Merge pull request #1603 from sjnam/examples-test-fail
fails to "make test" in examples
2019-04-29 13:45:58 -07:00
cev c4a40dbf65 set followLinks option true to cat, zcat and gzcat programs 2019-04-27 19:06:29 -04:00
sjnam bee9e5f38c fix test fail 2019-04-28 05:05:45 +09:00
Lzu Tao 4107b734bd meson: Update default project version
* Update to use ninja v1.9.0 on CI
2019-04-27 01:42:22 +07:00
Nick TerrellandGitHub 585b5a1039 Merge pull request #1600 from myzhang1029/long-tell
Fix #1591 - Not building on MinGW-W64
2019-04-24 21:05:25 -07:00
myzhang1029 f837326f11 Fix #1591 - Not building on MinGW-W64
Add a static function LONG_TELL for the forth #if branch
2019-04-24 20:06:05 +08:00
Nick TerrellandGitHub 3d673f3275 Merge pull request #1598 from terrelln/decode-seq
[libzstd] Error if all sequence bits aren't consumed
2019-04-23 15:44:20 -07:00
Nick Terrell 5f228f8db2 [libzstd] Add a ZSTD_STATIC_ASSERT for BIT_DStream_status 2019-04-23 14:22:16 -07:00
Nick Terrell a892e25374 [libzstd] Error if all sequence bits aren't consumed 2019-04-23 14:07:36 -07:00
Nick TerrellandGitHub b758250eb1 Merge pull request #1595 from terrelln/legacy-fix
[legacy] Fix ZSTDv0*_decodeSequence()
2019-04-19 12:23:50 -07:00
Nick Terrell 0fd322f812 [legacy] Fix ZSTDv0*_decodeSequence()
* Version <= 0.5 could read beyond the end of `dumps`, which points into
  the input buffer.
* Check the validity of `dumps` before using it, if it is out of bounds
  return garbage values. There is no return code for this function.
* Introduce `MEM_readLE24()` for simplicity, since I don't want to trust
  that there is an extra byte after `dumps`.
2019-04-19 11:34:52 -07:00
Nick TerrellandGitHub 9ad7ea44ec Merge pull request #1594 from terrelln/legacy-fix
[legacy] Fix Huffman jump table reads in v01 and v05
2019-04-18 17:29:32 -07:00
Nick Terrell 2536771134 [legacy] Fix Huffman jump table reads in v01 and v05 2019-04-18 16:20:42 -07:00
Nick TerrellandGitHub a8db4bd3fe Merge pull request #1593 from terrelln/legacy-fix
[fuzzer] Run fuzzers in legacy mode and fix legacy code
2019-04-18 16:07:42 -07:00
Nick Terrell 785331acbd [paramgrill] Fix mingw build errors 2019-04-18 15:06:56 -07:00
Nick Terrell 579f3d7794 [legacy] Fix bug in ZSTD_decodeSeqHeaders() 2019-04-18 13:41:10 -07:00
Nick Terrell ac098c7f5f [legacy] Fix a bug in ZSTDv06_findFrameSizeInfoLegacy() 2019-04-18 13:33:26 -07:00
Nick Terrell 610a81ecf9 [fuzzer] Compile with legacy support 2019-04-18 12:44:55 -07:00
Nick Terrell cc669006dc [fuzzer] Size the decompression output buffer randomly 2019-04-18 12:44:21 -07:00
Nick TerrellandGitHub af3531ee35 Merge pull request #1590 from terrelln/frame-info-fuzz
[fuzzer] Fuzz frame info functions
2019-04-17 12:13:49 -07:00
Nick Terrell 58bcc328a4 [fuzz] Add a seedcorpora target for oss-fuzz 2019-04-17 12:13:06 -07:00
Nick Terrell ee130a9889 [libzstd] Check the size in readSkippableFrameSize() 2019-04-17 11:41:55 -07:00
Nick Terrell 5922f4e2ae [legacy] Return the right error code 2019-04-17 11:34:52 -07:00
Nick Terrell 09caa4d800 [fuzzer] Add a fuzzer for frame info functions
Add a fuzzer that fuzzes all helper functions that take compressed
input. This fuzzer caught one out of bounds read in
`ZSTD_decompressBound()`.
2019-04-17 11:29:42 -07:00
Nick Terrell 450feb0f95 [libzstd] Fix ZSTD_decompressBound() on bad skippable frames
The function didn't verify that the skippable frame size is correct.
2019-04-17 11:29:42 -07:00
Nick TerrellandGitHub b85f7d7790 Merge pull request #1589 from terrelln/appveyorTest
[libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx
2019-04-16 12:32:19 -07:00
Nick Terrell a17fe4c9e5 [visual] Fix unreachable code warning 2019-04-16 11:32:35 -07:00
Yann ColletandGitHub 41ad46f1a9 Merge pull request #1588 from terrelln/release-prep
[fullbench] Fix speed measurements
2019-04-16 10:11:28 -07:00
Nick Terrell 82d5a9870e [visual] Compile with ZSTD_MULTITHREAD=1
This branch `appveyorTests` runs the tests that `master` runs, so
passing Appveyor means that the tests are fixed.
2019-04-15 23:04:46 -07:00
Nick Terrell de0499f7fa [libzstd] Require ZSTD_MULTITHREAD to create a ZSTDMT_CCtx
ZSTDMT was broken when compiled without ZSTD_MULTITHREAD defined,
because `ZSTD_CCtx_setParameter(cctx, ZSTD_c_nbWorkers, nbWorkerss)`
failed. It was detected by the MSVC test which runs the fuzzer with
multithreading disabled.

This is a very niche use case of a deprecated API, because the API is
inefficient and synchronous, since `threading.h` will be synchronous.
Users almost certainly don't want this, and anyone who tested their code
should realize that it is broken. Therefore, I think it is safe to
require `ZSTD_MULTITHREAD` to be defined to use ZSTDMT.
2019-04-15 23:04:46 -07:00