Yann Collet
5164d44dab
change advanced parameter name: ZSTD_c_repcodeResolution
...
and updated its documentation.
Note: older name ZSTD_c_searchForExternalRepcodes remains supported via #define
2024-12-20 10:36:59 -08:00
Yann Collet
bbaba45589
change experimental parameter name
...
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
2024-10-31 13:43:40 -07:00
Elliot Gorokhovsky
a810e1eeb7
Provide an interface for fuzzing sequence producer plugins
2023-03-28 12:02:57 -07:00
Elliot Gorokhovsky and GitHub
ff42ed1582
Rename "External Matchfinder" to "Block-Level Sequence Producer" ( #3484 )
...
* change "external matchfinder" to "external sequence producer"
* migrate contrib/ to new naming convention
* fix contrib build
* fix error message
* update debug strings
* fix def of invalid sequences in zstd.h
* nit
* update CHANGELOG
* fix .gitignore
2023-02-09 17:01:17 -05:00
Elliot Gorokhovsky
7f8189ca57
add ZSTD_c_fastExternalSequenceParsing cctxParam
2023-02-01 09:09:53 -08:00
Elliot Gorokhovsky and GitHub
64052ef57d
Guard against invalid sequences from external matchfinders ( #3465 )
2023-01-31 13:55:48 -05:00
f593e54ee1
Enable if == 1 rather than if == 0
...
Co-authored-by: Nick Terrell <nickrterrell@gmail.com >
2023-01-20 11:41:53 -05:00
Elliot Gorokhovsky
3f9f568aa6
Fuzz the external matchfinder API
2023-01-19 13:33:25 -08:00
daniellerozenblit and GitHub
dc1c6cc5df
Merge pull request #3418 from daniellerozenblit/fuzz-max-block-size
...
Fuzz on maxBlockSize
2023-01-19 08:18:04 -05:00
Danielle Rozenblit
1fffcfe01d
update minimum threshold for max block size
2023-01-11 11:09:57 -08:00
Daniel Kutenin and Nick Terrell
ca2ff788df
Make the producer use the same amount of entropy
2023-01-11 10:09:19 -08:00
Daniel Kutenin and Nick Terrell
3ac0b91302
Fix fuzzing with ZSTD_MULTITHREAD
...
At Google we fuzz zstd without ZSTD_MULTITHREAD but we want inputs to be as much as reproducible. It allows us to test new fuzzing methods for our fuzz team internally and have more horsepower to find bugs
2023-01-11 10:09:19 -08:00
Danielle Rozenblit
fe08137d9a
resolve max block value in cctx and use when calculating the max block size
2023-01-09 07:53:53 -08:00
Danielle Rozenblit
908e812733
initial commit
2023-01-04 13:01:54 -08:00
W. Felix Handte
8927f985ff
Update Copyright Headers 'Facebook' -> 'Meta Platforms'
...
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
2022-12-20 12:37:57 -05:00
Elliot Gorokhovsky
bb3c01c853
Migrate other test usages of boolean LDM flag to paramSwitch enum
2022-11-21 16:20:38 -05:00
Daniel Kutenin and GitHub
1d153c923c
Make fuzzing work without ZSTD_MULTITHREAD ( #3291 )
2022-10-17 14:31:47 -07:00
Elliot Gorokhovsky
747e06f4f6
Add tests
2022-06-22 17:05:23 -04:00
Yann Collet
8da414231d
found a few more places which were dependent on seqStore offcode sumtype numeric representation
2021-12-28 17:03:24 -08:00
senhuang42
b5c35d7ea3
Use new paramSwitch enum for LCM, row matchfinder, and block splitter
2021-09-21 14:22:02 -04:00
Nick Terrell
172b4b6ac4
[lib] Add ZSTD_c_deterministicRefPrefix
...
This flag forces zstd to always load the prefix in ext-dict mode, even
if it happens to be contiguous, to force determinism. It also applies to
dictionaries that are re-processed.
A determinism test case is also added, which fails without
`ZSTD_c_deterministicRefPrefix` and passes with it set.
Question: Should this be the default behavior? It isn't in this PR.
2021-05-05 18:49:56 -07:00
senhuang42
f80dec66b0
Add DDS to oss fuzzer
2021-04-22 18:21:43 -04:00
Nick Terrell and Sen Huang
4694423c4f
Add and integrate lazy row hash strategy
2021-04-07 09:53:34 -07:00
Nick Terrell and Nick Terrell
a494308ae9
[copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
...
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
senhuang42
e2bb215117
Add unit tests and fuzzer param
2021-03-24 08:21:09 -07:00
Nick Terrell
66e811d782
[license] Update year to 2021
2021-01-04 17:53:52 -05:00
Nick Terrell and Nick Terrell
5717bd39ee
[lib] Fix NULL pointer dereference
...
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.
This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.
* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.
Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.
Credit to OSS-Fuzz.
2020-05-06 12:09:02 -07:00
Nick Terrell
ac58c8d720
Fix copyright and license lines
...
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized
The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.
The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
Sen Huang
e21a8bbecd
Fix FUZZ_rand32() bug
2019-11-05 16:43:24 -05:00
Sen Huang
f2932fb5eb
Fix more merge conflicts
2019-11-05 15:54:05 -05:00
Dario Pavlovic
92c58c4d5d
Use range instead of the generic uint32 method to use less bytes when generating necessary numbers.
2019-09-12 12:40:12 -07:00
Dario Pavlovic
0630d084cb
[Fuzz] Improve data generation #1723
...
Converting the rest of the tests to use the new data producer.
2019-09-10 16:14:43 -07:00
Nick Magerko
c7a24d7a14
Define ZSTD_SRCSIZEHINT_MIN as 0
2019-08-20 13:06:15 -07:00
Nick Magerko
ea9d35922c
Add size-hint to fuzz tests
2019-08-19 15:12:29 -07:00
Josh Soref and Nick Terrell
a880ca239b
Spelling ( #1582 )
...
* spelling: accidentally
* spelling: across
* spelling: additionally
* spelling: addresses
* spelling: appropriate
* spelling: assumed
* spelling: available
* spelling: builder
* spelling: capacity
* spelling: compiler
* spelling: compressibility
* spelling: compressor
* spelling: compression
* spelling: contract
* spelling: convenience
* spelling: decompress
* spelling: description
* spelling: deflate
* spelling: deterministically
* spelling: dictionary
* spelling: display
* spelling: eliminate
* spelling: preemptively
* spelling: exclude
* spelling: failure
* spelling: independence
* spelling: independent
* spelling: intentionally
* spelling: matching
* spelling: maximum
* spelling: meaning
* spelling: mishandled
* spelling: memory
* spelling: occasionally
* spelling: occurrence
* spelling: official
* spelling: offsets
* spelling: original
* spelling: output
* spelling: overflow
* spelling: overridden
* spelling: parameter
* spelling: performance
* spelling: probability
* spelling: receives
* spelling: redundant
* spelling: recompression
* spelling: resources
* spelling: sanity
* spelling: segment
* spelling: series
* spelling: specified
* spelling: specify
* spelling: subtracted
* spelling: successful
* spelling: return
* spelling: translation
* spelling: update
* spelling: unrelated
* spelling: useless
* spelling: variables
* spelling: variety
* spelling: verbatim
* spelling: verification
* spelling: visited
* spelling: warming
* spelling: workers
* spelling: with
2019-04-12 11:18:11 -07:00
Nick Terrell
7a1fde2957
[fuzzer] Add dictionary fuzzers
2019-04-08 21:07:28 -07:00
Yann Collet
39e28982cf
introduced constants ZSTD_STRATEGY_MIN and ZSTD_STRATEGY_MAX
2018-12-06 16:16:16 -08:00
Yann Collet
be9e561da4
changed ZSTD_c_compressionStrategy into ZSTD_c_strategy
...
also : fixed paramgrill, and limit conditions
2018-12-06 15:00:52 -08:00
Yann Collet
3583d19c4e
changed parameter names from ZSTD_p_* to ZSTD_c_*
...
for naming consistency
2018-12-05 17:26:02 -08:00
Yann Collet
41c7d0b1e1
changed hashEveryLog into hashRateLog
2018-11-21 14:36:57 -08:00
Yann Collet
2e7fd6a2cb
fixed remaining searchLength invocations
2018-11-20 15:13:27 -08:00
W. Felix Handte
2242330b26
Fix Fuzz Range
2018-11-12 13:01:14 -08:00
W. Felix Handte
01bb1c1016
Add CCtx Param Controlling Dict Attachment Behavior
2018-06-21 17:29:25 -04:00
Yann Collet
0b1833c2cb
fixed regressiontest
...
ZSTD_TARGETLEN_MIN no longer exists
since is would be tautological to check if an unsigned value is < 0.
2018-06-07 16:07:46 -07:00
Yann Collet
f15e2fd55e
fixed fuzz test
2018-03-21 16:45:15 -07:00
Nick Terrell
471aa385b3
[fuzz] Speed up round trip tests
...
* Enforce smaller maximum values for parameters
* Adjust parameters to the source size
The memory usage is reduced by about 5x, which makes the fuzzers run at
least twice as fast, even more so with ASAN/MSAN enabled.
2017-09-26 14:03:43 -07:00
Nick Terrell
39357c41cb
[fuzzer] Fuzz long range matching & new API
2017-09-14 14:48:08 -07:00
Yann Collet