Commit Graph
9543 Commits
Author SHA1 Message Date
Yann ColletandGitHub f1faab6720 Merge pull request #3112 from facebook/man2
updated man page, providing more details for --train mode
2022-04-19 15:36:47 -07:00
Yann Collet eb726c6a20 updated man pages
had to run the conversion script on Ubuntu, as it doesn't run correctly on macos anymore.
2022-04-13 18:57:27 -07:00
Yann Collet 0df2fd6088 updated man page, providing more details for --train mode
following questions from #3111.

Note : only the source markdown has been updated,
the actual man page zstd.1 still need to be processed.
2022-04-13 18:51:59 -07:00
Yann ColletandGitHub 460780f804 Merge pull request #3094 from dirkmueller/usage_cleanup
Split help in long and short version, cleanup formatting
2022-04-05 07:09:54 -07:00
Yann ColletandGitHub e4cd9bbd88 Merge pull request #3108 from paulmenzel/fix-typo-in-zstd.1
Remove superfluous *not* in description of `--long[=#]` in zstd(1)
2022-04-03 23:12:06 -07:00
Paul Menzel f133bc8c9c zstd.1: Remove superfluous *not* in description of --long[=#]
Resolves: https://github.com/facebook/zstd/issues/3101
2022-04-03 07:29:51 +02:00
Elliot GorokhovskyandGitHub 3e6bbdd847 Disable visual-2015 tests (#3106) 2022-03-31 12:26:20 -04:00
Yann ColletandGitHub 455c2c21e6 Merge pull request #3103 from facebook/fix45586
fix minor bug in sequence_compression_api tester
2022-03-30 10:08:57 -07:00
Yann Collet 678bfff4fe fix minor bug in sequence_compression_api tester
margin was merely slightly too short for extra splitting.
2022-03-29 16:45:09 -07:00
Dirk Müller 7fbe60d577 Split help in long and short version, cleanup formatting
Adopt the more standard Usage: formatting style
List short and long options alongside where available
Print lists as a table
Use command style description
2022-03-29 12:57:47 +02:00
Nick TerrellandGitHub f229daaf42 Merge pull request #3052 from dirkmueller/gzip_keep
Keep original file if -c or --stdout is given
2022-03-28 10:35:21 -07:00
Elliot GorokhovskyandGitHub 64efba4c5e Software pipeline for ZSTD_compressBlock_fast_dictMatchState (#3086)
* prefetch dict content inside loop

* ip0/ip1 pipeline

* add L2_4 prefetch to dms pipeline

* Remove L1 prefetch

* Remove L2 prefetching

* Reduce # of gotos

* Cosmetic fixes

* Check final position sometimes

* Track step size as in bc768bc

* Fix nits
2022-03-17 12:35:11 -04:00
Nick TerrellandGitHub eadb6c874f Merge pull request #3095 from dpelle/typo-and-grammar-fixes
Typo and grammar fixes
2022-03-14 09:17:21 -07:00
Dominique Pelle 3a64aa29a6 On more mistake (Node -> Note) 2022-03-13 00:08:55 +01:00
Dominique Pelle b772f53952 Typo and grammar fixes 2022-03-12 08:58:04 +01:00
Nick TerrellandGitHub 05fc7c78c8 Merge pull request #3088 from cyberknight777/dev
[contrib][linux] Fix a warning in zstd_reset_cstream()
2022-03-11 10:01:11 -08:00
Nick TerrellandGitHub c3a89ef60c Merge pull request #3093 from dirkmueller/cli_tests_fixup
Handle newer less versions in zstdless testing
2022-03-10 10:41:52 -08:00
Dirk Müller 7a3997c21a Handle newer less versions in zstdless testing
Newer less versions appear to have changed how stderr
and stdout are showing error messages. hardcode the
expected behavior to make the tests pass with any less version.

Also set locale to C so that the strings are matching.
2022-03-10 09:47:33 +01:00
Cyber Knight 498ac8238d [contrib][linux] Make zstd_reset_cstream() functionally identical to ZSTD_resetCStream()
- As referenced by Nick Terrelln ~ the ZSTD maintainer in the linux kernel, making zstd_reset_cstream() functionally identical to ZSTD_resetCStream() would be the perfect way to fix the warning without touching any core functions or breaking other parts of the code.

Suggested-by: Nick Terrell <terrelln@fb.com>
Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-10 15:32:13 +08:00
Nick TerrellandGitHub 6a8fba9e5f Merge pull request #3092 from terrelln/2022-03-09-decoder-errata-doc
[doc] Add decompressor errata document
2022-03-09 15:10:29 -08:00
Nick Terrell 696fa2524a [doc] Add decompressor errata document
Add a document that lists the known bugs in the decoder where valid
frames are rejected, along with the version that they were fixed.
2022-03-09 14:40:41 -08:00
Dirk Müller 3f4f8b04ed Keep original file if -c or --stdout is given
Set removeSrcFile back to false when -c or --stdout is used to improve
compatibility with gzip(1) behavior.

gzip(1) is removing the original file on compression unless --stdout or
/-c is used. zstd is defaulting to keep the file unless --rm is used or
when it is called via a gzip symlink, in which it is removing by
default. Specifying -c/--stdout turns this behavior off.
2022-03-09 23:38:59 +01:00
Felix HandteandGitHub 8814aa5bfa Merge pull request #3059 from dirkmueller/gzip_fast_best
Implement more gzip compatibility (#3037)
2022-03-08 15:32:06 -05:00
Yann ColletandGitHub db104f6e83 Merge pull request #3079 from niamster/cmake-compat-5
build:cmake: enable ZSTD legacy support by default
2022-03-08 00:21:23 -08:00
Cyber Knight 8ff20c25f3 [contrib][linux] Use ZSTD_CCtx_setPledgedSrcSize() instead of ZSTD_CCtx_reset()
- The previous patch throws the following warning:

 ../linux/lib/zstd/zstd_compress_module.c: In function ‘zstd_reset_cstream’:
../linux/lib/zstd/zstd_compress_module.c:136:34: error: enum conversion when passing argument 2 of ‘ZSTD_CCtx_reset’ is invalid in C++ [-Werror=c++-compat]
  136 |  return ZSTD_CCtx_reset(cstream, pledged_src_size);
      |                                  ^~~~~~~~~~~~~~~~
In file included from ../linux/include/linux/zstd.h:26,
                 from ../linux/lib/zstd/zstd_compress_module.c:15:
../linux/include/linux/zstd_lib.h:501:20: note: expected ‘ZSTD_ResetDirective’ {aka ‘enum <anonymous>’} but argument is of type ‘long long unsigned int’
  501 | ZSTDLIB_API size_t ZSTD_CCtx_reset(ZSTD_CCtx* cctx, ZSTD_ResetDirective reset);
      |                    ^~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Since we have a choice to either use ZSTD_CCtx_reset or ZSTD_CCtx_setPledgedSrcSize instead of ZSTD_resetCStream, let's switch to ZSTD_CCtx_setPledgedSrcSize to not have any unnecessary warns alongside the kernel build and CI test build.

Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-08 13:38:34 +08:00
Cyber Knight e470c940f6 [contrib][linux] Fix a warning in zstd_reset_cstream()
- This fixes the below warning:

../lib/zstd/zstd_compress_module.c: In function 'zstd_reset_cstream':
../lib/zstd/zstd_compress_module.c:136:9: warning: 'ZSTD_resetCStream' is deprecated [-Wdeprecated-declarations]
  136 |         return ZSTD_resetCStream(cstream, pledged_src_size);
      |         ^~~~~~
In file included from ../include/linux/zstd.h:26,
                 from ../lib/zstd/zstd_compress_module.c:15:
../include/linux/zstd_lib.h:2277:8: note: declared here
 2277 | size_t ZSTD_resetCStream(ZSTD_CStream* zcs, unsigned long long pledgedSrcSize);
      |        ^~~~~~~~~~~~~~~~~

ZSTD_resetCstream is deprecated and zstd_CCtx_reset is suggested to use hence let's switch to it.

Signed-off-by: Cyber Knight <cyberknight755@gmail.com>
2022-03-07 11:55:33 +08:00
Yann ColletandGitHub 930b5cecaa Merge pull request #3060 from rex4539/typos
Fix typos
2022-03-05 14:21:58 -08:00
Dimitris Apostolou cf1894b324 Fix typos 2022-03-05 23:47:25 +02:00
Xi RuoyaoandGitHub d109cef201 fix the assertion in readLinesFromFile (#3084)
* fix the assertion in readLinesFromFile

When the file is not terminated by endline, readLineFromFile will append
a '\0' for the last line.  In this case pos + lineLength == dstCapacity.

* test: don't print very long text garbage
2022-03-04 14:56:44 -05:00
Yann ColletandGitHub e8448a3d61 Merge pull request #3080 from TocarIP/tokarip/static_bmi2
Enable STATIC_BMI2 for gcc/clang
2022-03-03 12:49:40 -08:00
Ilya Tokar 7c3d1cb3ab Enable STATIC_BMI2 for gcc/clang
Some usage (e.g. BIT_getLowerBit) uses it without checking for MSVC,
so enabling for clang gives a small performance boost.
2022-03-03 15:03:54 -05:00
Elliot GorokhovskyandGitHub 0c386afbfd Fix CI failures by adding apt-get update to Github Actions (#3082)
* Fix CI failures by adding apt-get update to Makefile

* Fix travis failure caused by apt-get update

* Move apt-get update from Makefile to Github Actions .yml

* Revert .travis.yml change

* Fix typo
2022-03-03 14:52:06 -05:00
Nick TerrellandGitHub 41665679bf Merge pull request #3081 from terrelln/dict-load-zero
[programs] Fix infinite loop when empty input is passed to trainer
2022-03-02 12:58:51 -08:00
Nick Terrell da737c7ab8 [programs] Fix infinite loop when empty input is passed to trainer
When an empty input file was passed to the dictionary trainer, it would infinite loop.
The added test case exposes the bug, and is fixed with this PR.
2022-03-02 11:05:26 -08:00
Dmytro Milinevskyi 03bba1b0bf build:cmake: enable ZSTD legacy support by default 2022-03-01 18:29:47 +01:00
Elliot GorokhovskyandGitHub 87406b5f3b Merge pull request #3075 from TocarIP/tokarip/bzhi
Use helper function for bit manipulations.
2022-02-24 16:52:48 -05:00
Ilya Tokar 0178c12dd9 Use helper function for bit manipulations.
We already have BIT_getLowerBits, so use it. Benefits are 2fold:
1) Somewhat cleaner code
2) We are now using bzhi instructions, when available. Performance
delta is too small for microbenchmarks, but avoiding load still helps
larger applications, by reducing data cache pressure.
2022-02-23 17:59:56 -05:00
Elliot GorokhovskyandGitHub 621d798988 Merge pull request #3045 from embg/asm_refactor
Refactor bitwise intrinsics
2022-02-17 13:08:08 -05:00
Elliot Gorokhovsky 71d9dab76f Replace XOR with subtraction for readability 2022-02-16 16:49:42 -05:00
Elliot Gorokhovsky 856c7dc51d Fix fuzzer.c nits and replace CLZ fallback 2022-02-16 11:40:05 -05:00
Elliot Gorokhovsky 00f2acba36 Add back check to prevent Win32 static analysis issues 2022-02-15 11:41:09 -05:00
Elliot Gorokhovsky 7c674a0919 Add tests for bitwise intrinsics 2022-02-14 18:36:33 -05:00
Elliot Gorokhovsky 6994a9f99c bits.h refactor and bugfix 2022-02-14 16:59:55 -05:00
Elliot Gorokhovsky 529cd7b821 Fix nits 2022-02-14 14:24:50 -05:00
Elliot Gorokhovsky 796182652d Pull out software fallbacks 2022-02-14 11:16:03 -05:00
Elliot Gorokhovsky db2f4a6532 Move bitwise builtins into bits.h 2022-02-14 11:16:03 -05:00
Elliot GorokhovskyandGitHub 970460f67d Merge pull request #3063 from embg/largeNbDicts
Bugfix and new features for largeNbDicts benchmark
2022-02-11 13:55:35 -05:00
Elliot Gorokhovsky 762898f5e4 Bugfix and new features for largeNbDicts benchmark 2022-02-11 13:15:16 -05:00
Yann ColletandGitHub 3202c75148 Merge pull request #3054 from ooosssososos/patch-1
[trace] Add aarch64 to supported architectures for zstd_trace
2022-02-11 08:13:09 -08:00
Dirk Müller e653e97f77 Implement more gzip compatibility (#3037)
-n --no-name is the current behavior already, so we can implement
this as a noop.

--best is an alias for -9 in gzip

add basic cli tests.
2022-02-09 23:09:18 +01:00