Commit Graph
9624 Commits
Author SHA1 Message Date
W. Felix Handte 927859f5e8 Also Update Man Page Documentation 2021-01-11 17:55:58 -05:00
W. Felix Handte 8b6a4b5b7c Allow Input From Console When --force is Passed
Also update option flag documentation.
2021-01-11 17:53:20 -05:00
senandGitHub 7e6c53c7fe Merge pull request #2465 from senhuang42/cformat_2021
[license] Change year to 2021 compression_format.md
2021-01-11 12:35:34 -05:00
Nick TerrellandGitHub e7b782087a Merge pull request #2462 from lazka/cmake-fix-pc-escaping
cmake: use configure_file() for creating the .pc file
2021-01-11 12:30:13 -05:00
senhuang42 1d6d64afa3 Change year to 2021 for compression format file 2021-01-11 08:53:29 -05:00
Christoph Reiter 0766540b59 cmake: use configure_file() for creating the .pc file
Escaping in add_custom_target() seems to depend on the shell used in the cmake
generator and using Ninja on Windows, which uses cmd.exe, results in stray backslashes
in the .pc file.

Instead of going through escaping hell just use configure_file() with the existing
libzstd.pc.in file already used by the simple Makefile based build system.

This fixes the .pc file syntax when building zstd with CMake+Ninja+gcc on Windows.
2021-01-09 09:29:16 +01:00
yumeyaoandGitHub 821d9acd17 Fix visibility of symbols in .so (#2441)
Fix visibility of symbols in .so and add an alias for renamed API
2021-01-08 14:27:31 -08:00
senandGitHub 69085db61c Merge pull request #2446 from senhuang42/multiple_ddicts_v3
[RFC] Support references to multiple DDicts
2021-01-08 16:49:45 -05:00
Yann ColletandGitHub 33b73db33c Merge pull request #2457 from facebook/cli-dll
zstd CLI can now be linked to libzstd dynamic library
2021-01-07 17:10:13 -08:00
Yann ColletandGitHub c416015ab5 Merge pull request #2459 from ThomasWaldmann/fix-typos
fix typos (work done by Andrea Gelmini)
2021-01-07 16:19:10 -08:00
senhuang42 9ae0dd9336 Fix Visual and staticanalyze warnings 2021-01-07 17:58:37 -05:00
Thomas Waldmann 92a2b5ccc9 fixup: lits means literals 2021-01-07 23:30:42 +01:00
Yann Collet 3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet 2901b5e675 Merge branch 'dev' into cli-dll 2021-01-07 10:24:09 -08:00
Yann ColletandGitHub 7a620b190e Merge pull request #2455 from facebook/lessTests
removed redundant tests
2021-01-07 10:23:14 -08:00
Thomas Waldmann f9802d80a0 fix typos (work done by Andrea Gelmini) 2021-01-07 18:47:23 +01:00
senhuang42 4f7584e7a3 Allow freestanding lib script regex to detect XXH64( 2021-01-07 12:29:12 -05:00
senhuang42 17222654bf Add streaming decompression to unit test 2021-01-07 12:29:12 -05:00
senhuang42 c2c9b8a7ec Address comments, clean up interface/internals 2021-01-07 12:29:12 -05:00
senhuang42 22b7bff2bc Add unit test, improve documentation 2021-01-07 12:29:12 -05:00
senhuang42 ea52fc3606 Use XXHash for hash function, create a sensible public interface 2021-01-07 12:29:12 -05:00
senhuang42 7c1a79f232 Add debuglog statements 2021-01-07 12:29:11 -05:00
senhuang42 d1a6a9d285 Reference requested dict ID at decompression time 2021-01-07 12:29:11 -05:00
senhuang42 5a6d3eef2b Allocate memory for DDict hash set when parameter is set 2021-01-07 12:29:11 -05:00
senhuang42 fd5b608f1c Add parameter to control multiple DDicts 2021-01-07 12:29:11 -05:00
senhuang42 f933668d3f Implement hashset for dictIDs 2021-01-07 12:29:11 -05:00
Yann Collet cefdc023f7 The CLI can be linked to libzstd dynamic library
invoking target zstd-dll
2021-01-06 18:00:24 -08:00
Yann Collet 890d85bdb4 removed CLI dependency to legacy unsafe function
makint the CLI ons step closer to being linkable to the dynamic library
2021-01-06 16:19:42 -08:00
Yann Collet 9866148e22 removed redundant tests
clang v3.8 tests are either flacky or redundant,
prefer using clang-latest.
2021-01-06 15:40:20 -08:00
Yann ColletandGitHub f011f639de Merge pull request #2454 from facebook/cycleLog_noDeps
removed internal library dependency from CLI
2021-01-06 15:30:05 -08:00
Yann Collet 0d793a675a removed internal dependency from CLI
ZSTD_cycleLog() is a very short function,
creating a rather large dependency onto libzstd's internal just for it is overkill.
Prefer duplicating this 2-lines function.

This PR makes the zstd CLI one step closer to being linkable to the dynamic library (see #2450)
More steps are still needed to reach this goal.
2021-01-06 01:35:52 -08:00
Nick TerrellandGitHub a077a6a0a9 Merge pull request #2451 from terrelln/adjust-dict-2
Don't shrink window log when streaming with a dictionary
2021-01-04 22:27:30 -05:00
Nick Terrell 58476bcf7f Don't shrink window log in ZSTD_getCParams()
Treat ZSTD_getCParams() and ZSTD_adjustCParams() in the same way
we treat streaming compression. Choose parameters based on the
dictionary size + source size, and assume the source size is small
if unkown. But, don't shrink the window log down in
ZSTD_adjustCParams_internal().
2021-01-04 15:54:09 -08:00
Nick Terrell 9d31c704d5 Don't shrink window log when streaming with a dictionary
Fixes #2442.

1. When creating a dictionary keep the same behavior as before.
   Assume the source size is 513 bytes when adjusting parameters.
2. When calling ZSTD_getCParams() or ZSTD_adjustCParams() keep
   the same behavior as before.
3. When attaching a dictionary keep the same behavior of ignoring
   the dictionary size. When streaming this will select the
   largest parameters and not adjust them down. But, the CDict
   will use the correctly sized parameters, which seems like the
   right tradeoff.
4. When not attaching a dictionary (either forced not to, or
   using a prefix dictionary) we select parameters based on the
   dictionary size + source size, and assume the source size is
   small, which is the same behavior as before. But, now we don't
   adjust the window log (and hash and chain log) down when the
   source size is unknown.

When the source size is unknown all cdicts should attach, except
when the user disables attaching, or `forceWindow` is used. This
means that when streaming with a CDict we end up in the good case
where we get small CDict parameters, and large source parameters.

TODO: Add a streaming + dictionary regression test case.
2021-01-04 15:54:09 -08:00
Nick Terrell a98a6e2091 [test][regression] Add no source size with dictionary test
* Add a test that runs without a pledgedSrcSize and with a dictionary.
* Add github.tar data with uses the github dictionary while compressing
  github.tar, instead of each file individually.
2021-01-04 15:54:09 -08:00
Nick TerrellandGitHub e856052576 Merge pull request #2452 from terrelln/2021
[license] Update year to 2021
2021-01-04 18:23:18 -05:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann ColletandGitHub bc0a1e4d59 Merge pull request #2445 from facebook/nomsanfuzz
remove flackey msan ossfuzz test
2021-01-03 10:41:55 -08:00
Yann Collet a0835b5bab Merge branch 'dev' into nomsanfuzz 2020-12-29 11:45:42 -08:00
Yann Collet ff2f888d56 fixed one more minor cast issue
can't use address calculation with `void*`
2020-12-29 11:44:37 -08:00
Yann Collet 3fd21d5438 Merge branch 'dev' into nomsanfuzz 2020-12-28 14:08:45 -08:00
Yann Collet 7f8be046b9 fixed minor warnings introduced in #2439 2020-12-28 14:07:31 -08:00
Yann ColletandGitHub cfff4c1cd5 Merge pull request #2439 from senhuang42/skippable_frame_api
Generate skippable frame API
2020-12-28 11:22:07 -08:00
Yann ColletandGitHub 603829d3d2 Merge pull request #2444 from indygreg/dict-ifndef-guards
Add ifndef guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE
2020-12-28 11:19:58 -08:00
Yann Collet f1585fefa2 remove flackey msan ossfuzz test
while waiting for it to be fixed
2020-12-28 11:18:58 -08:00
Gregory Szorc dd1a7e41ee Add ifndef guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE
This ensures the symbols aren't redefined, which would result in a compiler
error.

I was getting redefined symbols for _LARGEFILE64_SOURCE when building for
32-bit x86 Linux on an older CentOS release in a CI environment. With this
change, I'm able to compile the single file library in this environment.

Closes #2443.
2020-12-26 10:02:45 -07:00
Yann ColletandGitHub d4f7d75f84 Merge pull request #2438 from facebook/makej
try to keep libzstd.a "as is" once created
2020-12-22 00:31:38 -08:00
Yann ColletandGitHub 20afffefbc Merge pull request #2440 from facebook/fixga32
try to fix 32-bit test on github actions
2020-12-21 16:26:50 -08:00
Yann Collet f1225b186e try to fix 32-bit test on github actions
for some reasons, this test fails at _installing_ 32-bit dependencies
using the exact same command that actually works in other tests !!?

It's unclear why it fails repeateadly for this test only.
Try another way to install dependencies to fix that.
2020-12-21 15:47:20 -08:00
Yann Collet f2ac2b7bcf try to fix cross-compiler tests 2020-12-21 15:43:14 -08:00