ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,817 Commits 17 Branches 77 Tags
8f7ea1afeba1f3762e99413424df95b9faf4d2d8
Commit Graph
8817 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
TrianglesPCTandGitHub 8f7ea1afeb Update zstd_lazy.c 
Switch to other comment style
 2021-05-14 19:02:34 -06:00
TrianglesPCTandGitHub 0e071214b5 Update zstd_lazy.c 
switch to unaligned load as I don't know if buffer will always be aligned to 32 bytes, and compilers aside from MSVC might actually use aligned loads
 2021-05-14 17:03:30 -06:00
TrianglesPCTandGitHub 69ac124b12 Update zstd_lazy.c 2021-05-14 16:53:19 -06:00
TrianglesPCTandGitHub 0b9f4bb0ff Update zstd_lazy.c 
use 8bit
 2021-05-14 16:47:24 -06:00
TrianglesPCTandGitHub 77d54eb3b3 Add files via upload 2021-05-14 16:40:32 -06:00
TrianglesPCTandGitHub 52f44bb365 Add files via upload 
msvc
 2021-05-14 16:33:07 -06:00
TrianglesPCTandGitHub 25bda9053a Add files via upload 
msvc suport
avx2 path
 2021-05-14 16:32:04 -06:00
senandGitHub 06718087f8 Remove deprecate flag for vcx (#2647) 2021-05-13 19:33:48 -04:00
senandGitHub 40def70387 Add source level deprecation warning disabling to certain tests/utils (#2645) 2021-05-13 14:41:21 -04:00
Yann ColletandGitHub b57022eede Merge pull request #2644 from facebook/mesonFix 
Fixed meson test on travisCI
 2021-05-13 10:19:49 -07:00
Yann Collet 988beb3000 updated meson test 
hopefully, bionic will have a more recent version of python
required to install meson.
 2021-05-13 09:43:23 -07:00
Yann ColletandGitHub 705a62b612 Merge pull request #2643 from facebook/workers32 
reduce ZSTDMT_NBWORKERS_MAX in 32-bit mode
 2021-05-12 14:18:31 -07:00
senandGitHub a51e342ed1 Update CHANGELOG to include patch from fix (#2642) 2021-05-12 16:46:05 -04:00
Yann Collet 8fae35591e Merge branch 'dev' of github.com:facebook/zstd into dev 2021-05-12 13:12:30 -07:00
Olivier PerretandGitHub d4548c96cb fileio: clamp value of windowLog in patch-mode (#2637) 
With small enough input files, the inferred value of fileWindowLog could
be smaller than ZSTD_WINDOWLOG_MIN.

This can be reproduced like so:
$ echo abc > small
$ echo abcdef > small2
$ zstd --patch-from small small2 -o patch
previously, this would fail with the error "zstd: error 11 : Parameter is out of bound"
 2021-05-12 16:11:15 -04:00
Yann Collet cb0cad9b79 reduce Max nb Workers to 64 in 32-bit mode 
and restored limit to 256 when in 64-bit mode
(it was reduced to 200 to give more room for 32-bit).

This should fix test instability issues
using lot of threads in 32-bit environments.
 2021-05-12 13:10:25 -07:00
senandGitHub c730b8c5a3 Remove const data members in threadpooltest payload (#2639) (#2640) 2021-05-12 16:09:48 -04:00
senandGitHub b35c250bf3 Remove const data members in threadpooltest payload (#2639) 2021-05-12 12:56:57 -04:00
senandGitHub 01fe4796fb Add mt lib build to CL, shuffle around bugs section (#2638) 2021-05-12 11:31:31 -04:00
Yann ColletandGitHub 8a53a882f2 updated generated man pages for v1.5.0 (#2635) 2021-05-11 18:17:31 -04:00
senandGitHub 9c23ea9e2b Bump version to 1.5.0, rebuild documentation (#2634) 2021-05-11 16:32:09 -04:00
Yann ColletandGitHub 162f540402 Merge pull request #2633 from bmwiedemann/issue2632 
Avoid SIGBUS on armv6
 2021-05-11 11:17:52 -07:00
Bernhard M. Wiedemann 28d0120b5a Avoid SIGBUS on armv6 
When running armv6 userspace on armv8 hardware with a 64 bit Linux kernel,
the mode 2 caused SIGBUS (unaligned memory access).
Running all our arm builds in the build farm
only on armv8 simplifies administration a lot.

Depending on compiler and environment, this change might slow down
memory accesses (did not benchmark it). The original analysis is 6 years old.

Fixes #2632
 2021-05-11 17:51:03 +02:00
Yann ColletandGitHub 9fb5a0407c Merge pull request #2630 from facebook/gcc9 
improved gcc-9 and gcc-10 decoding speed
 2021-05-10 10:54:16 -07:00
Yann ColletandGitHub 334ac69db7 Merge pull request #2628 from skitt/libzstd-nomt-flags 
Apply flags to libzstd-nomt in libzstd style
 2021-05-08 00:21:59 -07:00
Yann Collet 439e58d060 improved gcc-9 and gcc-10 decoding speed 
the new alignment setting is better for gcc-9 and gcc-10
by about ~+5%.

Unfortunately, it's worse for essentially all other compilers.

Make the new alignment setting conditional to gcc-9+.
 2021-05-08 00:01:01 -07:00
Yann ColletandGitHub 5b6d38a99e Merge pull request #2547 from facebook/d_prefetch_refactor 
Refactor prefetching for the decoding loop
 2021-05-07 16:28:00 -07:00
Yann Collet 6755baf940 update decoder hot loop alignment 
This seems to bring an additional ~+1.2% decompression speed
on average across 10 compilers x 6 scenarios.
 2021-05-07 15:18:16 -07:00
Yann Collet 4d9caa4928 Merge branch 'd_prefetch_refactor' of github.com:facebook/zstd into d_prefetch_refactor 2021-05-07 11:30:44 -07:00
Yann Collet 1db5947591 improve decompression speed of long variant by ~+5% 
changed strategy,
now unconditionally prefetch the first 2 cache lines,
instead of cache lines corresponding to the first and last bytes of the match.

This better corresponds to cpu expectation,
which should auto-prefetch following cachelines on detecting the sequential nature of the read.

This is globally positive, by +5%,
though exact gains depend on compiler (from -2% to +15%).
The only negative counter-example is gcc-9.
 2021-05-07 11:26:14 -07:00
senandGitHub 13449d7ce1 Add PHONY targets to makefiles (#2629) 2021-05-07 14:03:19 -04:00
Nick TerrellandGitHub 66772efe73 Merge pull request #2627 from terrelln/timeout-fix 
[lib] Fix fuzzer timeouts by backing off overflow correction
 2021-05-07 10:55:26 -07:00
senandGitHub 9e94b7cac5 Assert no divison by 0, correct superblocks 0 sequences case (#2592) 2021-05-07 13:26:56 -04:00
Yann Collet a4d55c8748 Merge branch 'dev' into d_prefetch_refactor 2021-05-07 09:32:53 -07:00
Yann ColletandGitHub b4637d152a Merge pull request #2623 from facebook/fasterCygwin 
attempt to make Appveyor's Cygwin test faster
 2021-05-07 09:18:06 -07:00
senandGitHub d8d6e48a0a Add threadPool unit tests to fuzzer.c (#2604) 2021-05-07 11:13:44 -04:00
senandGitHub 91465e23b2 [1.5.0] Enable multithreading in lib build by default (#2584) 
* Update lib Makefile to have new targets

* Update lib/README.md for mt
 2021-05-07 11:13:30 -04:00
Stephen Kitt b2582de3c9 Apply flags to libzstd-nomt in libzstd style 
... for consistency (this doesn't actually change the build flags used
in practice, currently).

Signed-off-by: Stephen Kitt <steve@sk2.org>
 2021-05-07 13:25:27 +02:00
Nick Terrell c2555f8c6f [lib] Fix fuzzer timeouts by backing off overflow correction 
Linearly back off the frequency of overflow correction based on the
number of times the `ZSTD_window_t` has been overflow corrected. This
will still allow the fuzzer to quickly find overflow correction bugs,
while also keeping good speed for larger inputs.

Additionally, the `nbOverflowCorrections` variable can be useful for
debugging coredumps, since we can inspect the `ZSTD_CCtx` to see if
overflow correction has happened yet.

I've verified this fixes the timeouts in OSS-Fuzz (176 seconds -> 6
seconds). I've also verified that fuzzers and `fuzzer` and `zstreamtest`
still catch the row-hash overflow correction bug.
 2021-05-06 22:03:41 -07:00
Yann Collet 17b9e43c7d do not install g++ 2021-05-06 21:53:30 -07:00
Yann Collet ee425faaa7 Merge branch 'dev' into d_prefetch_refactor 2021-05-06 19:49:26 -07:00
Yann ColletandGitHub 0d05846952 Merge pull request #2626 from facebook/codingStyle1 
added a paragraph on coding style
 2021-05-06 19:46:05 -07:00
Nick TerrellandGitHub f36fbddbfa Merge pull request #2625 from terrelln/ubsan-failure 
[lib] Fix UBSAN warning in ZSTD_decompressSequences()
 2021-05-06 19:22:25 -07:00
Yann Collet f44c720fa8 added a paragraph on coding style 2021-05-06 18:40:25 -07:00
Yann Collet bd547232bc switch to clang 2021-05-06 16:07:44 -07:00
Yann Collet ee65162655 Merge branch 'dev' into fasterCygwin 2021-05-06 16:06:00 -07:00
Nick Terrell b052b583e5 [lib] Fix UBSAN warning in ZSTD_decompressSequences() 2021-05-06 15:31:30 -07:00
senandGitHub 698f261b35 [1.5.0] Deprecate some functions (#2582) 
* Add deprecated macro to zstd.h, mark certain functions as deprecated

* Remove ZSTD_compress.c dependencies on deprecated functions
 2021-05-06 17:59:32 -04:00
Nick TerrellandGitHub e7e4b74b2b Merge pull request #2600 from nickhutchinson/clang-cl 
Fix for excessive compiler warnings when building with clang-cl
 2021-05-06 12:43:15 -07:00
Nick TerrellandGitHub 2b82948e58 Merge pull request #2622 from terrelln/zdict-api 
[zdict] Add a FAQ to the top of zdict.h
 2021-05-06 12:42:56 -07:00