ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,908 Commits 17 Branches 77 Tags
45d707e908c302fee84bb0f6d34b12acc3d14e0f
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 61afa154cd improve tar compatibility 
This patch is supposed to improve compatibility with less featured tar variants
"when the tar program used does not support historical options (without hyphen) nor the '-z' option."

Patch proposed by Antonio Diaz Diaz
 2021-05-15 23:09:42 -07:00
Yann Collet 0b0b62d1cf minor mention of RFC8878 
more recent update
 2021-05-15 23:04:46 -07:00
Yann Collet 988beb3000 updated meson test 
hopefully, bionic will have a more recent version of python
required to install meson.
 2021-05-13 09:43:23 -07:00
Yann Collet 8fae35591e Merge branch 'dev' of github.com:facebook/zstd into dev 2021-05-12 13:12:30 -07:00
Yann Collet cb0cad9b79 reduce Max nb Workers to 64 in 32-bit mode 
and restored limit to 256 when in 64-bit mode
(it was reduced to 200 to give more room for 32-bit).

This should fix test instability issues
using lot of threads in 32-bit environments.
 2021-05-12 13:10:25 -07:00
Yann Collet 439e58d060 improved gcc-9 and gcc-10 decoding speed 
the new alignment setting is better for gcc-9 and gcc-10
by about ~+5%.

Unfortunately, it's worse for essentially all other compilers.

Make the new alignment setting conditional to gcc-9+.
 2021-05-08 00:01:01 -07:00
Yann Collet 6755baf940 update decoder hot loop alignment 
This seems to bring an additional ~+1.2% decompression speed
on average across 10 compilers x 6 scenarios.
 2021-05-07 15:18:16 -07:00
Yann Collet 4d9caa4928 Merge branch 'd_prefetch_refactor' of github.com:facebook/zstd into d_prefetch_refactor 2021-05-07 11:30:44 -07:00
Yann Collet 1db5947591 improve decompression speed of long variant by ~+5% 
changed strategy,
now unconditionally prefetch the first 2 cache lines,
instead of cache lines corresponding to the first and last bytes of the match.

This better corresponds to cpu expectation,
which should auto-prefetch following cachelines on detecting the sequential nature of the read.

This is globally positive, by +5%,
though exact gains depend on compiler (from -2% to +15%).
The only negative counter-example is gcc-9.
 2021-05-07 11:26:14 -07:00
Yann Collet 17b9e43c7d do not install g++ 2021-05-06 21:53:30 -07:00
Yann Collet ee425faaa7 Merge branch 'dev' into d_prefetch_refactor 2021-05-06 19:49:26 -07:00
Yann Collet f44c720fa8 added a paragraph on coding style 2021-05-06 18:40:25 -07:00
Yann Collet bd547232bc switch to clang 2021-05-06 16:07:44 -07:00
Yann Collet ee65162655 Merge branch 'dev' into fasterCygwin 2021-05-06 16:06:00 -07:00
Yann Collet 2e76bd7d10 attempt to make Appveyor's Cygwin test faster 
Cygwin is the longest Appveyor test
Appveyor is typically the CI which finish last
 2021-05-06 08:47:45 -07:00
Yann Collet df05b2ba7c fix --filelist compatibility with Windows cr+lf line ending 2021-05-05 18:01:55 -07:00
Yann Collet 9750f3c87b improved benchmark experience on Windows 
benchmark results are not progressively displayed on Windows terminal.
For long benchmark sessions, nothing is displayed,
until the end, where everything is flushed.

Force display to be flushed after each update.
Updates happen roughtly every second, or even less,
so it's not a substantial workload.
 2021-05-05 16:52:21 -07:00
Yann Collet 8cde167a27 Merge branch 'dev' into d_prefetch_refactor 2021-05-05 09:13:38 -07:00
Yann Collet 455fd1a067 updated documentation regarding minimum job size 2021-05-05 09:03:11 -07:00
Yann Collet 1026b9fa10 fix rsyncable mode 2021-05-04 15:59:27 -07:00
Yann Collet 8f86c29c06 allow jobSize to be as low as 512 KB 
previous lower limit was 1 MB.

Note : by default, the lowest job size is 2 MB, achieved at level 1.
Even lower job sizes can be achieved by manipulating this value directly,
or manually modifying window sizes to lower amounts.

Updated unit test to ensure that this new limit works fine
(test would fail with previous 1 MB limit).
 2021-05-04 11:02:55 -07:00
Yann Collet 8aafbd3604 Documented minimum version numbers 
Any stable API entry point introduced after v1.0
should be documented with its minimum version number.

Since PR fixes this requirement
updating mostly new entry points since v1.4.0
and newly introduced ones for future v1.5.0.
 2021-05-04 09:05:22 -07:00
Yann Collet 0f99a0d987 fix man page typo 
fix #2553 detected by @jwilk
 2021-03-24 05:55:04 -07:00
Yann Collet 9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Yann Collet f5434663ea Refactor prefetching for the decoding loop 
Following #2545,
I noticed that one field in `seq_t` is optional,
and only used in combination with prefetching.
(This may have contributed to static analyzer failure to detect correct initialization).

I then wondered if it would be possible to rewrite the code
so that this optional part is handled directly by the prefetching code
rather than delegated as an option into `ZSTD_decodeSequence()`.

This resulted into this refactoring exercise
where the prefetching responsibility is better isolated into its own function
and `ZSTD_decodeSequence()` is streamlined to contain strictly Sequence decoding operations.
Incidently, due to better code locality,
it reduces the need to send information around,
leading to simplified interface, and smaller state structures.
 2021-03-19 15:48:17 -07:00
Yann Collet 2fa4c8c405 added code comments for new API ZSTD_seekTable 2021-03-03 22:54:04 -08:00
Yann Collet 6e390ced1f Merge branch 'seekTable' of github.com:facebook/zstd into seekTable 2021-03-03 22:44:38 -08:00
Yann Collet 16ec1cf355 added test case for seekTable API 
and simple roundtrip test
 2021-03-03 18:56:23 -08:00
Yann Collet 6c0bfc468c fixed wrong assert condition 2021-03-03 15:30:55 -08:00
Yann Collet a1d7b9d654 fixed gcc conversion warnings 2021-03-03 15:17:12 -08:00
Yann Collet 24d59a655d Merge branch 'dev' into seekTable 2021-03-03 15:08:40 -08:00
Yann Collet ac95a30455 various minor style fixes 2021-03-02 16:03:18 -08:00
Yann Collet 029f974ddc strengthen compilation flags 2021-03-02 15:43:52 -08:00
Yann Collet c7e42e147b fixed const guarantees 
read-only objects are properly const-ified in parameters
 2021-03-02 15:24:30 -08:00
Yann Collet a80b10f5e6 fix potential leak on exit 2021-03-02 15:03:37 -08:00
Yann Collet 3d91ab74a1 disabled icc tests on Github Actions 2021-02-23 13:25:08 -08:00
Yann Collet 08fef036ec update FreeBSD image to latest 12.2 2021-02-23 10:40:44 -08:00
Yann Collet f2c0312889 removing signed integer overflow exception from ubsan tests 2021-02-19 16:30:06 -08:00
Yann Collet 9b0772177c fix ubsan test errors 
allows recovering from pointer overflow
 2021-02-19 10:42:05 -08:00
Yann Collet b9748757b0 fixed minor cast warning 2021-02-05 09:55:54 -08:00
Yann Collet 134be2731a removed flackey gcc6 tests 
from github actions.
replaced by equivalent "current" gcc tests when it makes sense
 2021-02-05 09:25:33 -08:00
Yann Collet 0bad3e5c0f parallel make build on linux 
fix #2474
 2021-01-18 11:33:03 -08:00
Yann Collet 3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet 2901b5e675 Merge branch 'dev' into cli-dll 2021-01-07 10:24:09 -08:00
Yann Collet cefdc023f7 The CLI can be linked to libzstd dynamic library 
invoking target zstd-dll
 2021-01-06 18:00:24 -08:00
Yann Collet 890d85bdb4 removed CLI dependency to legacy unsafe function 
makint the CLI ons step closer to being linkable to the dynamic library
 2021-01-06 16:19:42 -08:00
Yann Collet 9866148e22 removed redundant tests 
clang v3.8 tests are either flacky or redundant,
prefer using clang-latest.
 2021-01-06 15:40:20 -08:00
Yann Collet a0835b5bab Merge branch 'dev' into nomsanfuzz 2020-12-29 11:45:42 -08:00
Yann Collet ff2f888d56 fixed one more minor cast issue 
can't use address calculation with `void*`
 2020-12-29 11:44:37 -08:00
Yann Collet 3fd21d5438 Merge branch 'dev' into nomsanfuzz 2020-12-28 14:08:45 -08:00
Yann Collet 7f8be046b9 fixed minor warnings introduced in #2439 2020-12-28 14:07:31 -08:00
Yann Collet f1585fefa2 remove flackey msan ossfuzz test 
while waiting for it to be fixed
 2020-12-28 11:18:58 -08:00
Yann Collet f1225b186e try to fix 32-bit test on github actions 
for some reasons, this test fails at _installing_ 32-bit dependencies
using the exact same command that actually works in other tests !!?

It's unclear why it fails repeateadly for this test only.
Try another way to install dependencies to fix that.
 2020-12-21 15:47:20 -08:00
Yann Collet f2ac2b7bcf try to fix cross-compiler tests 2020-12-21 15:43:14 -08:00
Yann Collet 585196353d fix ppc64 build on circleci 2020-12-20 21:44:40 -08:00
Yann Collet 8233f55df8 verbose cross-compile tests on circleci 
for better diagnosis
 2020-12-20 21:31:58 -08:00
Yann Collet 52aa7f47dd updated clang+msan test 2020-12-20 18:29:36 -08:00
Yann Collet 9a9d3f76c4 fixed zstd+sanitizer build 2020-12-20 17:53:04 -08:00
Yann Collet f9884036c2 fixed zstd recipe 2020-12-20 17:19:23 -08:00
Yann Collet 9648bf027b try to keep libzstd.a "as is" once created 
to be compatible with scenarios such as
`make -j allmost`
 2020-12-20 17:10:57 -08:00
Yann Collet eacf1b3bc3 streamlined github action test 2020-12-20 15:03:59 -08:00
Yann Collet f78917cdd3 streamline zlibwrapper makefile 
making better usage of default build rules
 2020-12-20 12:53:30 -08:00
Yann Collet 7c495e8ea2 updated version number to v1.4.8 2020-12-18 15:52:11 -08:00
Yann Collet a7cb4af573 added emphasis on the alignment condition of workspace 
and made it a programming mistake (`assert()`)
rather than a runtime error.
 2020-12-18 15:04:09 -08:00
Yann Collet 7f09cdc8a8 removed duplicated release-only tests 
from travisCI
as they are already part of Github Actions
 2020-12-18 12:50:36 -08:00
Yann Collet 54cc01c5be removed tests duplicated between TravisCI and Github Actions 
reduce load on TravisCI
 2020-12-18 11:01:22 -08:00
Yann Collet 792fdf84f8 Merge branch 'check32' of github.com:facebook/zstd into check32 2020-12-17 15:45:22 -08:00
Yann Collet 3536e9d5ff removing tests using too much resources for 32-bit address space 2020-12-17 15:44:54 -08:00
Yann Collet d5eb7d1569 added pre-requisites for 32-bit tests in CI 2020-12-17 15:05:26 -08:00
Yann Collet c11db9c8b5 additional master->release switches (CI tests) 2020-12-17 15:01:04 -08:00
Yann Collet 4680d817c0 added a simple runtime test in CI for 32-bit binaries 2020-12-17 14:53:36 -08:00
Yann Collet ce34dc39a0 removed incorrect test 2020-12-16 23:53:13 -08:00
Yann Collet 0b39531d75 moving all references to release branch 
was previously `master`
 2020-12-16 23:00:35 -08:00
Yann Collet f647a759fe updated version number to v1.4.7 
and updated doc
 2020-12-15 20:53:05 -08:00
Yann Collet ecffa77e12 streamlined example recipe 
better use of default rules
 2020-12-15 02:07:08 -08:00
Yann Collet 8ab9bb9d26 added api issues reported by @animalize 2020-12-15 00:27:32 -08:00
Yann Collet 2e9ffdb468 Updated CHANGELOG for v1.4.7 2020-12-14 18:45:57 -08:00
Yann Collet c2939b96be refinement : only copy if binary is different 2020-12-10 14:31:35 -08:00
Yann Collet c56723ab03 replace final links by direct copy 
link can behave slightly differently from real binaries,
breaking a few scripts relying on "real binary" assumption.
 2020-12-10 13:25:08 -08:00
Yann Collet 2477968ee7 fix gcc-10 aliasing warning in zlib_wrapper 2020-12-04 20:25:01 -08:00
Yann Collet 5c0a3489a5 fix aliasing warning in decodecorpus 2020-12-04 19:21:40 -08:00
Yann Collet 6132df8dd3 fix gcc-10 strict aliasing warnings 
by exposing HUF_CElt declaration.
 2020-12-04 16:43:19 -08:00
Yann Collet 68c14bdff2 minor speed improvement to HUF_readCTable() 
faster by ~+1-2%
 2020-12-04 16:33:39 -08:00
Yann Collet 5de5c1d759 fixed fuzzer multithreading tests 2020-12-02 10:34:12 -08:00
Yann Collet db21d383b5 fixed fuzzer32 to support multithreading tests 
though it still fails on test33:
`test 33: superblock uncompressible data, too many nocompress superblocks`
 2020-12-02 09:13:55 -08:00
Yann Collet f69d8c027d removed fullbench-lib from tests/all 
this build works fine on all my systems,
but since to fail on CI environment.
Unclear why there is a difference.
This build test is not relevant anyway.
 2020-12-02 00:21:29 -08:00
Yann Collet 9f8b180d5d fixed API documentation 2020-12-02 00:15:07 -08:00
Yann Collet f8d0b46a9f streamline fuzzer 
from fuzzer32
 2020-12-01 23:44:16 -08:00
Yann Collet 37165f66b7 better usage of default build rules 2020-12-01 23:36:05 -08:00
Yann Collet 343a75d2ef simplified test makefile 
removed gzstd target:
relevant tests are unused and broken anyway
 2020-12-01 22:33:45 -08:00
Yann Collet 4b5d7e9ddb fix lz4 test messed by console detection 2020-11-30 06:47:16 -08:00
Yann Collet b15469203f added tests presumed to break lz4 console detection 2020-11-30 05:32:43 -08:00
Yann Collet 09e3bb95d2 Merge branch 'dev' into libzstd_autoconf_full 2020-10-28 10:53:08 -07:00
Yann Collet 0adce4631d Merge branch 'libzstd_autoconf_full' of github.com:facebook/zstd into libzstd_autoconf_full 2020-10-28 10:25:55 -07:00
Yann Collet ceccd7ae2d Merge branch 'dev' into libzstd_autoconf_full 2020-10-27 15:45:30 -07:00
Yann Collet 59ab4dcd32 Merge branch 'dev' into libzstd_autoconf_full 2020-10-26 15:17:39 -07:00
Yann Collet 456db0c377 make install only rebuild binaries if they don't exist 
Now `make` followed by `make install` doesn't rebuild binaries

also : only generated target directories if they don't already exist
 2020-10-23 16:46:49 -07:00
Yann Collet a6ee614a44 make zstd is now differentiated from zstd-nomt 
avoids mixing object files using different flags
 2020-10-23 16:08:21 -07:00
Yann Collet d3f1a9b5bd fix partial-build test 
sometimes, the scope difference is solely determined by the list of source files,
not by the flags.
 2020-10-22 21:36:09 -07:00
Yann Collet 98787a3018 Merge branch 'dev' into libzstd_autoconf 2020-10-22 18:55:30 -07:00