ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,656 Commits 17 Branches 77 Tags
dev
Commit Graph
10656 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell 7736549bea [bug-fix] Make simple single-pass functions ignore advanced parameters 
The simple compression functions are intended to ignore the advanced
parameters, but they were accidentally using them. All the
`ZSTD_parameters` were set correctly, but any extra parameters were
used as-is. E.g. `ZSTD_c_format`.

This PR makes all the simple single-pass functions listed below ignore
the advanced parameters, as intended.

* `ZSTD_compressCCtx()`
* `ZSTD_compress_usingDict()`
* `ZSTD_compress_usingCDict()`
* `ZSTD_compress_advanced()`
* `ZSTD_compress_usingCDict_advanced()`

It also adds a test case that ensures that each of these functions
ignore the advanced parameters.
 2021-02-12 19:11:23 -08:00
Nick TerrellandGitHub f39178b445 Merge pull request #2497 from terrelln/tracing 
[lib] Set appliedParams.compressionLevel correctly
 2021-02-12 17:34:07 -08:00
Nick Terrell c62eb05964 [lib] Set appliedParams.compressionLevel correctly 
Forward the correct compressionLevel to the appliedParams in all cases.
It was already correct for the advanced API, so only the old single-pass
functions needed to be fixed.

This compression level is unused by the library, but is set so that the
tracing framework can consume it.
 2021-02-12 15:00:14 -08:00
Nick TerrellandGitHub 3ac842d6cc Merge pull request #2496 from terrelln/tracing 
[trace] Minor fixes found during integration
 2021-02-12 10:48:30 -08:00
Nick Terrell f520f6dfbe [trace] Minor fixes found during integration 
* Mark `ZSTD_CCtx_getParameter()` as const
* Add `extern "C"` guards to `zstd_trace.h`
 2021-02-11 16:20:04 -08:00
Yann ColletandGitHub 8884cb887d Merge pull request #2483 from mpu/ldmgear 
New algorithms for the long distance matcher
 2021-02-11 08:38:23 -08:00
nia 74f85818a6 Use standard md5 tool on NetBSD. 
This avoids a GNU coreutils dependency.

-n is used to match the output format of coreutils:
http://man.netbsd.org/md5.1
 2021-02-11 10:50:11 +01:00
Quentin Carbonneaux 552efcac2d relocate large arrays from the stack to ldmState_t 2021-02-10 16:16:54 +01:00
Nick TerrellandGitHub a294a19990 Merge pull request #2490 from terrelln/tracing 
[trace] Keep track of a uint64_t tracing context
 2021-02-09 12:16:17 -08:00
Nick Terrell e59c9459a5 [trace] Keep track of a uint64_t tracing context 
The most common information that you want to track between begin() and
end() is the timestamp of the begin function, so you can measure the
duration of the (de)compression call. Allow the tracing library to put
this information inside the `ZSTD_TraceCtx`, so it doesn't need to keep
a global map in this case. If a single uint64_t is not enough, the
tracing library can return a unique identifier (like the context
pointer) instead, and use it as a key in a map.

This keeps the simple case simple.
 2021-02-09 11:37:05 -08:00
Quentin Carbonneaux e2ad174d73 fix some compiler warnings 2021-02-08 20:19:16 +01:00
Issam E. Maghniandfoo@bar.com 2636f53619 CMake: Enable only C for lib and programs projects 2021-02-07 19:39:04 -05:00
Nick TerrellandGitHub 856f38e6c5 Merge pull request #2482 from terrelln/tracing 
Add (de)compression tracing functionality
 2021-02-05 19:19:01 -08:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Nick TerrellandGitHub e926e9c4c3 Merge pull request #2488 from terrelln/continuity 
[zstd] Fix NULL pointer addition in ZSTD_checkContinuity()
 2021-02-05 16:08:02 -08:00
Nick Terrell f9b1e711ba [zstd] Fix NULL pointer addition in ZSTD_checkContinuity() 
Don't start a new section when `dstSize == 0` to avoid NULL
pointer addition.
 2021-02-05 12:18:06 -08:00
Yann ColletandGitHub 824dff4917 Merge pull request #2486 from facebook/nogcc6 
minor: removed flackey gcc6 tests from github actions
 2021-02-05 10:58:21 -08:00
Yann ColletandGitHub b5e990dc08 Merge pull request #2487 from facebook/fixcast 
fixed minor cast warning
 2021-02-05 10:57:55 -08:00
Yann Collet b9748757b0 fixed minor cast warning 2021-02-05 09:55:54 -08:00
Yann Collet 134be2731a removed flackey gcc6 tests 
from github actions.
replaced by equivalent "current" gcc tests when it makes sense
 2021-02-05 09:25:33 -08:00
Quentin Carbonneaux 874a590e5c deal safely with short inputs in ZSTD_ldm_generateSequences 
The fuzzer CI found this bug.
 2021-02-04 11:15:24 +01:00
Quentin Carbonneaux 9f327c02fd new core ldm algorithm 2021-02-03 22:24:07 +01:00
Nick TerrellandGitHub f5b3f64d3f Merge pull request #2464 from mpu/ldmfixes 
Simple performance improvements for ldm
 2021-01-22 12:41:22 -05:00
Quentin Carbonneaux aee3dc877f fix a variable name to reflect its nature 2021-01-22 02:24:19 -08:00
Quentin Carbonneaux d6e3de77dc fix warning and remove one more occurrence of makeEntryAndInsertByTag 2021-01-20 01:39:16 -08:00
Quentin Carbonneaux e0d5eca8fa fix forgotten numTagBits in getTagMask 2021-01-20 00:54:20 -08:00
Quentin Carbonneaux 1e65711ca5 a couple performance improvement changes for ldm 2021-01-20 00:54:20 -08:00
Yann ColletandGitHub 7e6729055a Merge pull request #2475 from facebook/parallel_build 
parallel make build on linux
 2021-01-19 10:19:41 -08:00
Yann Collet 0bad3e5c0f parallel make build on linux 
fix #2474
 2021-01-18 11:33:03 -08:00
Yann ColletandGitHub 7c2a17edf6 Merge pull request #2466 from felixhandte/force-stdin-console 
Allow Input From Console When `-f`/`--force` is Passed
 2021-01-18 11:03:45 -08:00
Nick TerrellandGitHub 649a766220 Merge pull request #2473 from terrelln/recovery 
[contrib][recovery] Add recovery_directory program
 2021-01-15 15:30:24 -05:00
Nick Terrell b45d22c851 [contrib][recovery] Add recovery_directory program 
This program takes a file with concatenated zstd frames and splits the
file up by frame. E.g. if `dir.zst` has 4 frames:

```
> ./recover_directory dir.zst recovery/file
Recovering 4 files...
Recovered recovery/file0
Recovered recovery/file1
Recovered recovery/file2
Recovered recovery/file3
Complete
```
 2021-01-15 08:45:22 -08:00
W. Felix Handte 927859f5e8 Also Update Man Page Documentation 2021-01-11 17:55:58 -05:00
W. Felix Handte 8b6a4b5b7c Allow Input From Console When --force is Passed 
Also update option flag documentation.
 2021-01-11 17:53:20 -05:00
senandGitHub 7e6c53c7fe Merge pull request #2465 from senhuang42/cformat_2021 
[license] Change year to 2021 compression_format.md
 2021-01-11 12:35:34 -05:00
Nick TerrellandGitHub e7b782087a Merge pull request #2462 from lazka/cmake-fix-pc-escaping 
cmake: use configure_file() for creating the .pc file
 2021-01-11 12:30:13 -05:00
senhuang42 1d6d64afa3 Change year to 2021 for compression format file 2021-01-11 08:53:29 -05:00
Christoph Reiter 0766540b59 cmake: use configure_file() for creating the .pc file 
Escaping in add_custom_target() seems to depend on the shell used in the cmake
generator and using Ninja on Windows, which uses cmd.exe, results in stray backslashes
in the .pc file.

Instead of going through escaping hell just use configure_file() with the existing
libzstd.pc.in file already used by the simple Makefile based build system.

This fixes the .pc file syntax when building zstd with CMake+Ninja+gcc on Windows.
 2021-01-09 09:29:16 +01:00
yumeyaoandGitHub 821d9acd17 Fix visibility of symbols in .so (#2441) 
Fix visibility of symbols in .so and add an alias for renamed API
 2021-01-08 14:27:31 -08:00
senandGitHub 69085db61c Merge pull request #2446 from senhuang42/multiple_ddicts_v3 
[RFC] Support references to multiple DDicts
 2021-01-08 16:49:45 -05:00
Yann ColletandGitHub 33b73db33c Merge pull request #2457 from facebook/cli-dll 
zstd CLI can now be linked to libzstd dynamic library
 2021-01-07 17:10:13 -08:00
Yann ColletandGitHub c416015ab5 Merge pull request #2459 from ThomasWaldmann/fix-typos 
fix typos (work done by Andrea Gelmini)
 2021-01-07 16:19:10 -08:00
senhuang42 9ae0dd9336 Fix Visual and staticanalyze warnings 2021-01-07 17:58:37 -05:00
Thomas Waldmann 92a2b5ccc9 fixup: lits means literals 2021-01-07 23:30:42 +01:00
Yann Collet 3324e87cff Added library version check 2021-01-07 10:37:27 -08:00
Yann Collet 2901b5e675 Merge branch 'dev' into cli-dll 2021-01-07 10:24:09 -08:00
Yann ColletandGitHub 7a620b190e Merge pull request #2455 from facebook/lessTests 
removed redundant tests
 2021-01-07 10:23:14 -08:00
Thomas Waldmann f9802d80a0 fix typos (work done by Andrea Gelmini) 2021-01-07 18:47:23 +01:00
senhuang42 4f7584e7a3 Allow freestanding lib script regex to detect XXH64( 2021-01-07 12:29:12 -05:00
senhuang42 17222654bf Add streaming decompression to unit test 2021-01-07 12:29:12 -05:00