senhuang42
0633bf17c3
Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression
2021-03-24 08:21:29 -07:00
senhuang42
eb1ee8686d
Refactor buildSequencesStatistics() to avoid pointer increment for superblocks
2021-03-24 08:21:29 -07:00
senhuang42
e2bb215117
Add unit tests and fuzzer param
2021-03-24 08:21:09 -07:00
senhuang42
de52de1347
Add recursive block split algorithm
2021-03-24 08:21:09 -07:00
senhuang42
f06f6626ed
Update function names for consistency
2021-03-24 08:20:54 -07:00
senhuang42
c56d6e49e8
Add block splitter to experimental params
2021-03-24 08:20:54 -07:00
senhuang42
2949a95224
Refactor block compression logic into single function
2021-03-24 08:20:54 -07:00
senhuang42
c05c090cc2
Centralize entropy statistics calculations to zstd_compress.c
2021-03-24 08:20:29 -07:00
senhuang42
386111adec
Add a nbSeq argument to compressSequences()
...
Refactor ZSTD_compressBlock_internal() to do the block header write within and add nbSeq argument to compressSequences()
2021-03-16 14:04:22 -07:00
senhuang42
98764493cf
Move block header write into compressBlock_internal()
2021-03-16 14:04:22 -07:00
Yann Collet and GitHub
3d6c903056
Merge pull request #2521 from animalize/doc_free
...
doc: ZSTD_free*() functions accept NULL pointer
2021-03-06 21:33:28 -08:00
Nick Terrell and GitHub
b5fd348a85
Merge pull request #2523 from terrelln/huf-stack-reduction
...
Add HUF_writeCTable_wksp() function
2021-03-05 12:35:09 -08:00
Nick Terrell
5df2a21f1e
Add HUF_writeCTable_wksp() function
...
This saves ~700 bytes of stack space in HUF_writeCTable.
2021-03-05 10:29:18 -08:00
Nick Terrell and GitHub
e50f88ca4c
Merge pull request #2522 from terrelln/stack-reduction
...
Reduce stack usage of ZSTD_buildCTable()
2021-03-04 20:55:58 -08:00
Nick Terrell
27498ff00f
Reduce stack usage of ZSTD_buildCTable()
...
It is a stack high-point for some compression strategies and has an easy
fix. This moves the normalized count into the entropy workspace.
2021-03-04 16:12:11 -08:00
animalize
0933775d79
doc: ZSTD_free*() functions accept NULL pointer
2021-03-04 12:02:52 +08:00
Yann Collet
029f974ddc
strengthen compilation flags
2021-03-02 15:43:52 -08:00
W. Felix Handte
d7db928f72
Bump Library Version 1.4.8 -> 1.4.9
2021-03-01 17:45:30 -05:00
Yann Collet and GitHub
61b63e9060
Merge pull request #2492 from niacat/dev
...
Use standard md5 tool on NetBSD.
2021-02-24 16:38:10 -08:00
Nick Terrell
7736549bea
[bug-fix] Make simple single-pass functions ignore advanced parameters
...
The simple compression functions are intended to ignore the advanced
parameters, but they were accidentally using them. All the
`ZSTD_parameters` were set correctly, but any extra parameters were
used as-is. E.g. `ZSTD_c_format`.
This PR makes all the simple single-pass functions listed below ignore
the advanced parameters, as intended.
* `ZSTD_compressCCtx()`
* `ZSTD_compress_usingDict()`
* `ZSTD_compress_usingCDict()`
* `ZSTD_compress_advanced()`
* `ZSTD_compress_usingCDict_advanced()`
It also adds a test case that ensures that each of these functions
ignore the advanced parameters.
2021-02-12 19:11:23 -08:00
Nick Terrell
c62eb05964
[lib] Set appliedParams.compressionLevel correctly
...
Forward the correct compressionLevel to the appliedParams in all cases.
It was already correct for the advanced API, so only the old single-pass
functions needed to be fixed.
This compression level is unused by the library, but is set so that the
tracing framework can consume it.
2021-02-12 15:00:14 -08:00
Nick Terrell
f520f6dfbe
[trace] Minor fixes found during integration
...
* Mark `ZSTD_CCtx_getParameter()` as const
* Add `extern "C"` guards to `zstd_trace.h`
2021-02-11 16:20:04 -08:00
Yann Collet and GitHub
8884cb887d
Merge pull request #2483 from mpu/ldmgear
...
New algorithms for the long distance matcher
2021-02-11 08:38:23 -08:00
nia
74f85818a6
Use standard md5 tool on NetBSD.
...
This avoids a GNU coreutils dependency.
-n is used to match the output format of coreutils:
http://man.netbsd.org/md5.1
2021-02-11 10:50:11 +01:00
Quentin Carbonneaux
552efcac2d
relocate large arrays from the stack to ldmState_t
2021-02-10 16:16:54 +01:00
Nick Terrell
e59c9459a5
[trace] Keep track of a uint64_t tracing context
...
The most common information that you want to track between begin() and
end() is the timestamp of the begin function, so you can measure the
duration of the (de)compression call. Allow the tracing library to put
this information inside the `ZSTD_TraceCtx`, so it doesn't need to keep
a global map in this case. If a single uint64_t is not enough, the
tracing library can return a unique identifier (like the context
pointer) instead, and use it as a key in a map.
This keeps the simple case simple.
2021-02-09 11:37:05 -08:00
Quentin Carbonneaux
e2ad174d73
fix some compiler warnings
2021-02-08 20:19:16 +01:00
Nick Terrell
54a4998a80
Add basic tracing functionality
2021-02-05 16:28:52 -08:00
Nick Terrell
f9b1e711ba
[zstd] Fix NULL pointer addition in ZSTD_checkContinuity()
...
Don't start a new section when `dstSize == 0` to avoid NULL
pointer addition.
2021-02-05 12:18:06 -08:00
Yann Collet
b9748757b0
fixed minor cast warning
2021-02-05 09:55:54 -08:00
Quentin Carbonneaux
874a590e5c
deal safely with short inputs in ZSTD_ldm_generateSequences
...
The fuzzer CI found this bug.
2021-02-04 11:15:24 +01:00
Quentin Carbonneaux
9f327c02fd
new core ldm algorithm
2021-02-03 22:24:07 +01:00
Quentin Carbonneaux
aee3dc877f
fix a variable name to reflect its nature
2021-01-22 02:24:19 -08:00
Quentin Carbonneaux
d6e3de77dc
fix warning and remove one more occurrence of makeEntryAndInsertByTag
2021-01-20 01:39:16 -08:00
Quentin Carbonneaux
e0d5eca8fa
fix forgotten numTagBits in getTagMask
2021-01-20 00:54:20 -08:00
Quentin Carbonneaux
1e65711ca5
a couple performance improvement changes for ldm
2021-01-20 00:54:20 -08:00
Yann Collet
0bad3e5c0f
parallel make build on linux
...
fix #2474
2021-01-18 11:33:03 -08:00
yumeyao and GitHub
821d9acd17
Fix visibility of symbols in .so ( #2441 )
...
Fix visibility of symbols in .so and add an alias for renamed API
2021-01-08 14:27:31 -08:00
sen and GitHub
69085db61c
Merge pull request #2446 from senhuang42/multiple_ddicts_v3
...
[RFC] Support references to multiple DDicts
2021-01-08 16:49:45 -05:00
Yann Collet and GitHub
33b73db33c
Merge pull request #2457 from facebook/cli-dll
...
zstd CLI can now be linked to libzstd dynamic library
2021-01-07 17:10:13 -08:00
Yann Collet and GitHub
c416015ab5
Merge pull request #2459 from ThomasWaldmann/fix-typos
...
fix typos (work done by Andrea Gelmini)
2021-01-07 16:19:10 -08:00
senhuang42
9ae0dd9336
Fix Visual and staticanalyze warnings
2021-01-07 17:58:37 -05:00
Thomas Waldmann
92a2b5ccc9
fixup: lits means literals
2021-01-07 23:30:42 +01:00
Yann Collet
2901b5e675
Merge branch 'dev' into cli-dll
2021-01-07 10:24:09 -08:00
Thomas Waldmann
f9802d80a0
fix typos (work done by Andrea Gelmini)
2021-01-07 18:47:23 +01:00
senhuang42
c2c9b8a7ec
Address comments, clean up interface/internals
2021-01-07 12:29:12 -05:00
senhuang42
22b7bff2bc
Add unit test, improve documentation
2021-01-07 12:29:12 -05:00
senhuang42
ea52fc3606
Use XXHash for hash function, create a sensible public interface
2021-01-07 12:29:12 -05:00
senhuang42
7c1a79f232
Add debuglog statements
2021-01-07 12:29:11 -05:00
senhuang42
d1a6a9d285
Reference requested dict ID at decompression time
2021-01-07 12:29:11 -05:00
senhuang42