ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,376 Commits 17 Branches 77 Tags
26f89d47aa29afe16664dff6a14aca45383423ba
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 4b5d7e9ddb fix lz4 test messed by console detection 2020-11-30 06:47:16 -08:00
Yann Collet b15469203f added tests presumed to break lz4 console detection 2020-11-30 05:32:43 -08:00
Yann Collet 09e3bb95d2 Merge branch 'dev' into libzstd_autoconf_full 2020-10-28 10:53:08 -07:00
Yann Collet 0adce4631d Merge branch 'libzstd_autoconf_full' of github.com:facebook/zstd into libzstd_autoconf_full 2020-10-28 10:25:55 -07:00
Yann Collet ceccd7ae2d Merge branch 'dev' into libzstd_autoconf_full 2020-10-27 15:45:30 -07:00
Yann Collet 59ab4dcd32 Merge branch 'dev' into libzstd_autoconf_full 2020-10-26 15:17:39 -07:00
Yann Collet 456db0c377 make install only rebuild binaries if they don't exist 
Now `make` followed by `make install` doesn't rebuild binaries

also : only generated target directories if they don't already exist
 2020-10-23 16:46:49 -07:00
Yann Collet a6ee614a44 make zstd is now differentiated from zstd-nomt 
avoids mixing object files using different flags
 2020-10-23 16:08:21 -07:00
Yann Collet d3f1a9b5bd fix partial-build test 
sometimes, the scope difference is solely determined by the list of source files,
not by the flags.
 2020-10-22 21:36:09 -07:00
Yann Collet 98787a3018 Merge branch 'dev' into libzstd_autoconf 2020-10-22 18:55:30 -07:00
Yann Collet a912ef0952 can integrate later dynamic flags changes 
for example `libzstd-mt` is `differentiated from `libzstd`
 2020-10-22 18:48:06 -07:00
Yann Collet ce6cd07c33 updated build documentation 2020-10-22 12:31:23 -07:00
Yann Collet e3867fb735 fixed libzstd.dll compilation on mingw 
and zstd linking
 2020-10-22 11:52:19 -07:00
Yann Collet de985b1c1a fix appveyor test 
was mixing `*.o` object files from different compilation settings
 2020-10-16 14:31:40 -07:00
Yann Collet 314c7df170 minor : change test order 
to reduce a warning with `clang` on Windows
 2020-10-16 13:26:47 -07:00
Yann Collet abf5c0655c fix zlibwrap parallel build 
previous recipe would build object files directly within programs/
which could be in competition with other local builds happening in programs/ at the same time.

fixed by generating the relevant object file locally.
 2020-10-15 10:56:37 -07:00
Yann Collet dec1a78d3e minor fix casting for Visual 2020-09-14 11:46:23 -07:00
Yann Collet c91a0855f8 check endDirective in ZSTD_compressStream2() 
fix #2297
also :
- `assert()` `endDirective` in `ZSTD_compressStream_internal()`, for debug mode
- add relevant tests
 2020-09-14 10:56:08 -07:00
Yann Collet 8b9cdd2597 fixed overlapping count & workspace special case 2020-07-26 22:40:21 -07:00
Yann Collet 051232223f optimized histogram 
new version easier to vectorize
leads to smaller code and faster execution
notably at the last recombination stage
(basically, fixed cost per block).

Assembly inspected with godbolt

On my laptop, with `clang` and `-mavx2` :
2K block : 1280 MB/s -> 1550 MB/s
8K block : 1750 MB/s -> 1860 MB/s
 2020-07-26 22:24:22 -07:00
Yann Collet 15c5e20023 updated documentation 2020-07-23 12:10:57 -07:00
Yann Collet 610c41f90f improved 2-args macro to support both syntaxes 
both `--long-commmand=arg` and `--long-command arg`
are supported by macro `NEXT_FIELD()`
so that the long-command only needs to be listed once.

This extends the syntax to support new syntaxes like
`-o=FILE` or `-D=dict`,
though there is no need to advertise this capability for the time being.

Also : added `NEXT_UINT32()`,
which is wrapper around `NEXT_FIELD()`
to read integer parameters.
Use the wrapper to new field, such as `--memlimit`
which can now support both syntaxes too.
 2020-07-20 17:41:32 -07:00
Yann Collet 23941eec04 added tests for newly enabled syntax 
for --patch-from origin
and --filelist list

Also : removed some constrained syntax tests,
as the new argument parsing syntax is more permissive.

For example :
    zstd file -of dest
used to be disallowed.

It's now allowed, and understood as:
    zstd file -o dest -f
 2020-07-17 13:31:15 -07:00
Yann Collet 7280c1dafa extended 2-fields macro usage to number arguments 
leading to further simplications
 2020-07-17 13:09:23 -07:00
Yann Collet a7e5652672 CLI: simplified handling of separated-fields arguments 
this patch reduces complexity associated with
commands requiring a separated arguments
such as :
-o filename
-D dictionary
--output-dir-flat dir
--output-dir-mirror dir

It used to be a multi-stage process with explicit context,
it's now simplified as a single macro.

Thanks to this simplification,
separated arguments logic has also been extended to
--patch-from XXX
--filelist XXX
which extends existing capability using =XXX
--patch-from=XXX
--filelist=XXX

Separated argument is useful for filenames and directories,
as it benefits from shell expansion
such as ~/dir/file
where the ~ is automatically translated by the shell.
In contrast --long-command=FILE does not interpret FILE,
so ~/ is transmitted as is to the main() function,
generally resulting in incorrect file name.
 2020-07-17 12:46:36 -07:00
Yann Collet c224367ede ensure workspace is large enough 
even when MAX_TABLELOG is reduced
 2020-07-16 20:33:50 -07:00
Yann Collet 21c273da84 import some minor fixes from FSE project 2020-07-16 20:25:15 -07:00
Yann Collet 1dcc478796 fix uclibc's st_mtim 
reported and suggested by @ewildgoose (#1872)
 2020-07-13 14:16:33 -07:00
Yann Collet c1913ed8c1 added paragraph on static analysis pitfalls 2020-06-15 13:04:45 -07:00
Yann Collet 105b5839de added comment on $(Q) 2020-05-29 12:40:59 -07:00
Yann Collet 6ac6472988 fixed double-compilation of lib with make -j all 
also implemented verbose mode (make V=1)
 2020-05-29 12:09:51 -07:00
Yann Collet 11a392ce23 minor markdown formatting fix 2020-05-26 13:15:35 -07:00
Yann Collet bb3c9bf43a updated spec on dictID==0 
Specified decoder behavior on receiving a frame with dictID=0.

Pushed paragraph on reserved DictID ranges into the Dictionary Format section.
 2020-05-25 08:15:09 -07:00
Yann Collet 39a32f40c9 fixed default rule for lib/Makefile 
default rule is `lib-release`

`lib-release` wasn't working : it was just skipped.

Removing `lib-release` from the list of .PHONY targets fixes it.

Same for `lib-mt`.
 2020-05-25 06:50:45 -07:00
Yann Collet 082755bd3f do not install zbuff.h 
this API is deprecated, for a loong time now,
all related symbols will be removed in a future version (likely v1.5.0)
and the header file `zbuff.h` doesn't compile from `include/` anyway,
because it needs to be positioned one directory below `zstd.h`.

Also removed `cover.h` from `cmake` installer,
as it should have never been part of this list to begin with.
 2020-05-22 15:35:54 -07:00
Yann Collet 306e922f60 fixed environment variables 2020-05-21 18:06:43 -07:00
Yann Collet 5d0c8a0770 Merge branch 'dev' into visual_playTests 2020-05-21 16:45:02 -07:00
Yann Collet 26b21e481f fix meson playTests.sh 2020-05-21 15:17:22 -07:00
Yann Collet a63810e2ac try to fix PPC64 Qemu tests 2020-05-21 10:26:40 -07:00
Yann Collet ad0c0d1946 fixed appveyor long tests 
playTests changed variable name
 2020-05-21 00:53:57 -07:00
Yann Collet 14aaa5290a fixed zbuff test 
error introduced in https://github.com/facebook/zstd/blame/3b39ce6b527c5cfe395a187c75b77baf5cc7de02/.travis.yml#L192
 2020-05-21 00:22:52 -07:00
Yann Collet 18082ce85a updated CHANGELOG for v1.4.5 2020-05-15 17:19:00 -07:00
Yann Collet f22b371e31 updated documentation for v1.4.5 
API manual and man pages
 2020-05-14 16:17:03 -07:00
Yann Collet e001715b3d fixed asan test 2020-05-11 20:35:47 -07:00
Yann Collet 20bd246045 blindfix for VS macro redefinition 2020-05-11 19:29:36 -07:00
Yann Collet 76e726e3be updated documentation for ZSTD_estimate*() 
make it clearer that tighter memory estimation
can be provided using advanced functions
on the condition of a defined input size bound.
 2020-05-11 19:21:50 -07:00
Yann Collet 91ad01218e updated initStatic tests 
differentiate small CCtx for small inputs
from full CCtx
from CStream contexts.

Ensure allocation & resize tests are more precise.
 2020-05-11 18:50:10 -07:00
Yann Collet 608f1bfc4c fixed context downsize with initStatic 
When context is created using initStatic,
no resize is possible.

fix : only bump oversizeDuration when !initStatic
 2020-05-11 18:16:38 -07:00
Yann Collet 58227db405 Merge branch 'dev' into initStatic_tests 2020-05-11 16:51:13 -07:00
Yann Collet 37b2dafbed minor clarification on -o file 2020-05-08 14:20:47 -07:00
Yann Collet 5c67a4e737 Merge branch 'dev' of github.com:facebook/zstd into dev 2020-05-08 14:15:56 -07:00
Yann Collet e5366bee16 updated zstd -h 
to help readability.
Group arguments into categories,
try to give better visibility to more common arguments.
 2020-05-08 14:14:46 -07:00
Yann Collet d564d56440 document additional compilation macros 
ZSTD_NOBENCH, ZSTD_NODICT, ZSTD_NOCOMPRESS, ZSTD_NODECOMPRESS
 2020-05-08 13:35:59 -07:00
Yann Collet 4f2cb944a8 Merge branch 'dev' into clevel_doc 2020-05-08 11:59:45 -07:00
Yann Collet e873697625 updated documentation regarding ZSTD_CLEVEL 
make it clearer that it's useful for `tar --zstd`.
 2020-05-08 10:51:37 -07:00
Yann Collet 4f755bdafc faster cygwin test 
The cygwin test on Appveyor is way too long (>30 mn).
The main issue is the very long zstreamtest, because no time out has been programmed.
Ensures that both fuzzer and zstreamtest receive a 30sec slot.
Should reduce cygwin tests duration by -30mn.
 2020-05-05 14:08:35 -07:00
Yann Collet ef7d98e55a Merge branch 'dev' into hufc_wconv 2020-05-05 10:23:00 -07:00
Yann Collet c29fd7cd8b some more conversion warnings 
hunting down some static analyzer warnings
 2020-05-05 10:16:59 -07:00
Yann Collet c1b836f4c3 fix minor conversion warnings 2020-05-04 14:43:09 -07:00
Yann Collet 7ea2ae6649 added test linking user program to multi-threaded libzstd 2020-04-28 21:18:29 -07:00
Yann Collet f77fd5ced0 generalized pattern rules 2020-04-28 18:43:55 -07:00
Yann Collet c6ae2e83bc fix libzstd-mt underlinking issue 
fix #2045
When compiling `libzstd` in multithreading mode,
the `libzstd-mt` recipe would not include `-pthread`,
resulting in an underlinked dynamic library.

Added a test on Travis to check that the library is fully linked.

This makes it possible, in some future release,
to build a multi-threaded `libzstd` dynamic library by default
as it would no longer impact the build script of user programs.
 2020-04-28 18:29:20 -07:00
Yann Collet f17ac423b2 new tests created new artifacts 
they were not properly ignored
 2020-04-28 15:58:22 -07:00
Yann Collet 458a1a1723 minor refactor 
- fix a few comments
- reorder some parameters, to enforce "mutable references first"
- simplified fwriteSparse()
 2020-04-13 14:09:57 -07:00
Yann Collet d2464776d1 man page : added --verbose long command 
also updated `zstd.1`
stacking prior updates completed on `zstd.1.md`.
 2020-02-26 14:29:30 -08:00
Yann Collet 5c769e33f8 Merge branch 'dev' of github.com:facebook/zstd into dev 2020-01-24 15:04:02 -08:00
Yann Collet f34eab162f updated CHANGELOG 2020-01-24 14:12:25 -08:00
Yann Collet 3119256267 updated CHANGELOG 
one missing item in v1.4.4
 2020-01-07 09:51:22 -08:00
Yann Collet 5120883a9c bumped version number 
so that potential issue report do not confuse `dev` with latest release
 2019-12-03 17:06:42 -08:00
Yann Collet 4b4de7493f refactoring : remove global variable g_displayOut 
displaying into stderr or stdout is now an explicit operation,
no longer depending on some global state set elsewhere in the code.
 2019-12-02 16:19:03 -08:00
Yann Collet a49417b5af fix recent issue combining -r with empty list of input files 
This would resize the table of input filenames to zero,
delivering an empty table,
to which it was no longer possible to add stdin.
 2019-12-02 14:28:18 -08:00
Yann Collet d5b4a7ea58 removed scanbuild workaround 2019-11-26 17:46:57 -08:00
Yann Collet ecaf3ce103 reverted ppc qemu and scanbuild on trusty 
ppc cross-compilation seems broken outside of trusty
scan-build version changes with distrib version, resulting in a different list of false positives
 2019-11-26 16:44:56 -08:00
Yann Collet 96ee20758c assembleFNT() can no longer fail 2019-11-26 15:44:33 -08:00
Yann Collet aaab618ae9 pushed aside stdio.h too 
since only UTIL_DISPLAY() depended on it.
 2019-11-26 15:25:32 -08:00
Yann Collet 7543cd055c moved UTIL_DISPLAY() inside util.c 2019-11-26 15:21:58 -08:00
Yann Collet a684b82774 util: isolated some dependencies 
from *.h to *.c
so that they don't get transitively included
into users of util.h.
 2019-11-26 15:16:53 -08:00
Yann Collet f57be339ab travisCI : removed trusty generic statement 
trusty is out of long term support.
let's run (most) tests on "current" platform.
 2019-11-26 15:01:16 -08:00
Yann Collet 2d9fad4f52 fixed minor VS warning, on parameter difference 
complaining about a `const` property on one side but not the other.
 2019-11-26 14:53:37 -08:00
Yann Collet f622c0adf3 switched UTIL_refFilename() to an assert() 2019-11-26 14:48:23 -08:00
Yann Collet 46bdbb9c06 Merge branch 'dev' into ahmed_file 2019-11-26 14:33:04 -08:00
Yann Collet 49cacd858b fixed fifo test 2019-11-26 14:18:09 -08:00
Yann Collet c71bd45a3b Merge branch 'dev' into ahmed_file 2019-11-26 11:20:26 -08:00
Yann Collet 87ae7b940b travisCI: moved compatibility tests to dev 
they are fast enough, and we want to catch such incompatibility early
 2019-11-26 10:30:43 -08:00
Yann Collet 1d7adee68e moved meson and versions compatibility tests to master/cron 
also : joined c99 compatibility tests with gnu90 and c++ ones
 2019-11-26 10:28:21 -08:00
Yann Collet 228a6d28cd travis : added test durations 
as a guide when doing test selection
 2019-11-26 10:24:09 -08:00
Yann Collet 5e657aca90 silence scan-build false positive 
blind attempt
 2019-11-25 15:50:58 -08:00
Yann Collet 9a3de0a535 changed name from createX to assembleX 
shows that the resulting object just takes ownership of provided buffer.
 2019-11-25 15:34:55 -08:00
Yann Collet b1de3ecdd5 fix mode_t on Windows 2019-11-25 13:59:35 -08:00
Yann Collet 9a22140ef4 created UTIL_chmod() 
protecting "/dev/null" from having its permissions changed.

also : minor : improved consistency of util.h API
 2019-11-25 13:45:22 -08:00
Yann Collet 60db21a677 added decompression into /dev/null tests 2019-11-25 10:53:03 -08:00
Yann Collet 7aaac3f69c fix #1904 
/dev/null permissions were modified when using sudo rights.
This fixes this bug during decompression.

More importantly, this patch  adds a test, triggered in TravisCI,
ensuring unaltered /dev/null permissions.
 2019-11-25 10:35:36 -08:00
Yann Collet 762a0dfc45 minor update to travis script 
many tests used to be run on Trusty platform,
they are now run on _current_ platform,
and now many test titles are incorrect.
Fixed.
 2019-11-22 17:32:07 -08:00
Yann Collet 098b36e9ab clarifications for Block_Maximum_Size 
as a follow up of #1882
 2019-11-13 09:50:15 -08:00
Yann Collet 2ed5344e84 added target zstd-dictBuilder 
check that the dictBuilder can be compiled
with only lib/common and lib/compress as dependencies.
 2019-11-06 15:46:58 -08:00
Yann Collet 9df49dc50a Visual compiler bug work-around 2019-11-06 15:23:44 -08:00
Yann Collet be34969e5c minor man page fix 2019-11-06 15:07:35 -08:00
Yann Collet aea2ff5d8d fixed wrong assert() in regression driver 2019-11-06 14:56:21 -08:00
Yann Collet 485fec5665 removed experimental_dict_builders 
no longer required,
can still be accessed by going to back to an earlier commit or release (<= v1.4.4)
 2019-11-06 14:43:14 -08:00
Yann Collet a7e33e3e10 updated fuzz tests to use FileNamesTable* abstraction 2019-11-06 14:42:13 -08:00