senhuang42
f80dec66b0
Add DDS to oss fuzzer
2021-04-22 18:21:43 -04:00
sen and GitHub
3eb3845898
Merge pull request #2583 from senhuang42/remove_zbuff
...
[1.5.0] Remove ZBUFF
2021-04-20 17:05:32 -04:00
senhuang42
a423305e7b
Remove ZBUFF tests
2021-04-19 17:27:05 -04:00
senhuang42
3b98987496
Remove building of ZBUFF/deprecated folder by default
2021-04-19 17:12:00 -04:00
sen and GitHub
12c045f74d
Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix
...
Correct the block splitter mismatched repcodes detection.
2021-04-12 23:27:43 -04:00
sen and GitHub
ebd41ebe56
Merge pull request #2572 from senhuang42/row_hash_hashcache_one_off_error_fix
...
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
2021-04-12 15:38:53 -04:00
Sen Huang
8844f93957
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
2021-04-12 14:24:58 -04:00
Sen Huang
550f76f131
Correct the detection of mismatched repcodes
2021-04-09 09:08:51 -07:00
sen and GitHub
56421f34e4
Merge pull request #2494 from senhuang42/row_hash2
...
SIMD Row Based Matchfinder 🚀
2021-04-08 10:55:14 -04:00
Sen Huang
4d63d6e8aa
Update results.csv, add Row hash to regression test
2021-04-07 10:31:41 -07:00
Nick Terrell and Sen Huang
4694423c4f
Add and integrate lazy row hash strategy
2021-04-07 09:53:34 -07:00
sen and GitHub
f71aabb5b5
Move clevel override to after initLocalDict() ( #2571 )
2021-04-06 21:05:37 -04:00
sen and GitHub
f1e8b565c2
Maintain two repcode histories for block splitting, replace invalid repcodes ( #2569 )
2021-04-06 17:25:55 -04:00
sen and GitHub
e38124555e
Fix dictionary force reloading clevel selection ( #2570 )
...
* Move cdict clevel override to before localdict init
* Update results.csv after dict load changes
2021-04-06 15:35:09 -04:00
Nick Terrell and GitHub
8383fc828d
Merge pull request #2541 from ihsinme/patch-1
...
simple fix for using bit operator.
2021-04-02 13:01:09 -07:00
Niclas Rosenvik and Nick Terrell
e7647180cd
Stop complaining about hash tool not found
...
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
2021-04-02 13:00:19 -07:00
sen and GitHub
980f3bbf83
[cwksp] Align all allocated "tables" and "aligneds" to 64 bytes ( #2546 )
...
* Perform 64-byte alignment of wksp tables and aligneds internally
* Clean up cwskp_finalize() function to only do two allocs
* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations
* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned
* Back out aligned/table reservation order restriction
* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
2021-04-01 20:07:19 -04:00
sen and GitHub
255925c231
Fix repcode-related OSS-fuzz issues in block splitter ( #2560 )
...
* Do not emit last partitions of blocks as RLE/uncompressed
* Fix repcode updates within block splitter
* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature
* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
2021-03-31 15:14:59 -04:00
Nick Terrell and Nick Terrell
d334ad2ff4
[contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel()
2021-03-30 10:30:59 -07:00
Nick Terrell and Nick Terrell
a494308ae9
[copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
...
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
sen and GitHub
84ccb81e7c
Merge pull request #2561 from senhuang42/longlength_enum
...
Add enum for representing long length ID
2021-03-26 15:55:12 -04:00
Sen Huang
b1a43455f8
Add enum for representing long length ID
2021-03-26 10:41:09 -07:00
sen and GitHub
ab216bc2c5
Merge pull request #2559 from senhuang42/add_dict_regression_tests_backup
...
Add different dict modes to compression ratio regression test, update results.csv
2021-03-25 19:26:06 -04:00
Sen Huang
bbbd578f45
Update results.csv
2021-03-25 11:16:37 -07:00
sen and GitHub
4fe2e7ae14
Merge pull request #2558 from senhuang42/msan_block_splitter_fix
...
Fix block splitter minor MSAN warning.
2021-03-25 13:51:43 -04:00
Sen Huang
f27e326456
Restrict dictmode regression tests only to advanced API, fix some compiler warnings
2021-03-25 10:39:08 -07:00
Sen Huang
1cadf86b39
Add tests to regression tests for dict
2021-03-25 10:39:08 -07:00
sen and GitHub
b0407b9f0e
Merge pull request #2555 from senhuang42/default_clevel_func
...
Add ZSTD_defaultCLevel() function to public API
2021-03-25 13:07:28 -04:00
Sen Huang
ef4e26bda5
Add clang msan fuzz test to github actions
2021-03-25 09:11:43 -07:00
Sen Huang
2a907bf4aa
Move lastCountSize into a returned struct, fix MSAN error
2021-03-25 09:11:15 -07:00
Sen Huang
e398744a35
Add ZSTD_defaultCLevel() function to public API
2021-03-25 08:04:00 -07:00
Nick Terrell and GitHub
f8ac0ea7ef
Merge pull request #2539 from terrelln/linux-kernel-fixes
...
Fixes for the next linux kernel patch version
2021-03-24 10:34:29 -07:00
sen and GitHub
bf542c8a8d
Merge pull request #2447 from senhuang42/block_splitter_v2
...
Recursive block splitting
2021-03-24 12:27:22 -04:00
Yann Collet and GitHub
8889ca36fc
Merge pull request #2554 from facebook/fix2553
...
fix man page typo
2021-03-24 08:55:33 -07:00
Sen Huang
5b566ebe08
Rename *compressSequences*() functions for clarity
2021-03-24 08:21:29 -07:00
Sen Huang
0ef1f935b7
Add a fallback in case the total blocksize of split blocks exceeds raw block size
2021-03-24 08:21:29 -07:00
Sen Huang
c90e81a692
Enable block splitter by default when applicable
2021-03-24 08:21:29 -07:00
Sen Huang
e34332834a
Clean up various functions, add debuglogging for estimate vs. actual sizes
2021-03-24 08:21:29 -07:00
Sen Huang
41c3eae6d9
Fix various fuzzer failures: repcode history, superblocks
2021-03-24 08:21:29 -07:00
senhuang42
0633bf17c3
Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression
2021-03-24 08:21:29 -07:00
senhuang42
eb1ee8686d
Refactor buildSequencesStatistics() to avoid pointer increment for superblocks
2021-03-24 08:21:29 -07:00
senhuang42
e2bb215117
Add unit tests and fuzzer param
2021-03-24 08:21:09 -07:00
senhuang42
de52de1347
Add recursive block split algorithm
2021-03-24 08:21:09 -07:00
senhuang42
f06f6626ed
Update function names for consistency
2021-03-24 08:20:54 -07:00
senhuang42
c56d6e49e8
Add block splitter to experimental params
2021-03-24 08:20:54 -07:00
senhuang42
2949a95224
Refactor block compression logic into single function
2021-03-24 08:20:54 -07:00
senhuang42
c05c090cc2
Centralize entropy statistics calculations to zstd_compress.c
2021-03-24 08:20:29 -07:00
Yann Collet
0f99a0d987
fix man page typo
...
fix #2553 detected by @jwilk
2021-03-24 05:55:04 -07:00
sen and GitHub
c48889f097
Merge pull request #2538 from senhuang42/monotonicity_test
...
Add memory monotonicity test over srcSize
2021-03-22 16:54:34 -04:00
Nick Terrell and GitHub
ebc2dfa821
Merge pull request #2524 from terrelln/huf-stack-reduction
...
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
2021-03-22 12:37:54 -07:00
senhuang42