ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,701 Commits 17 Branches 77 Tags
0e2345b8594feccb9e63b8e400a94ef5bacb5a72
Commit Graph
3787 Commits
Author SHA1 Message Date
Nick Terrell 6cee3c2c4f [trace] Remove default definitions of weak symbols 
Instead of providing a default no-op implementation, check the symbols
for `NULL` before accessing them. Providing a default implementation
doesn't reliably work with dynamic linking. Depending on link order the
default implementations may not be overridden. By skipping the default
implementation, all link order issues are resolved. If the symbols
aren't provided the weak function will be `NULL`.
 2021-04-26 16:05:39 -07:00
senandGitHub 3e2fbfd056 Merge pull request #2579 from senhuang42/getcdictID_to_stable 
[1.5.0] Promote ZSTD_getDictID_fromCDict() into stable API
 2021-04-26 09:55:43 -04:00
felixhandteandsenhuang42 efa6dfa729 Apply DDS adjustments to avoid assert failures 2021-04-23 16:41:00 -04:00
senhuang42 3b98987496 Remove building of ZBUFF/deprecated folder by default 2021-04-19 17:12:00 -04:00
Sen Huang 9c1ca3c00b Moved ZSTD_getDictID_fromCDict() into stable API 2021-04-16 10:14:29 -07:00
senandGitHub 12c045f74d Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix 
Correct the block splitter mismatched repcodes detection.
 2021-04-12 23:27:43 -04:00
Sen Huang 8844f93957 Adjust nb elements to prefetch in ZSTD_row_fillHashCache() 2021-04-12 14:24:58 -04:00
Sen Huang 550f76f131 Correct the detection of mismatched repcodes 2021-04-09 09:08:51 -07:00
Sen Huang 4d63d6e8aa Update results.csv, add Row hash to regression test 2021-04-07 10:31:41 -07:00
Nick TerrellandSen Huang 4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
senandGitHub f71aabb5b5 Move clevel override to after initLocalDict() (#2571) 2021-04-06 21:05:37 -04:00
senandGitHub f1e8b565c2 Maintain two repcode histories for block splitting, replace invalid repcodes (#2569) 2021-04-06 17:25:55 -04:00
senandGitHub e38124555e Fix dictionary force reloading clevel selection (#2570) 
* Move cdict clevel override to before localdict init

* Update results.csv after dict load changes
 2021-04-06 15:35:09 -04:00
Nick TerrellandGitHub 8383fc828d Merge pull request #2541 from ihsinme/patch-1 
simple fix for using bit operator.
 2021-04-02 13:01:09 -07:00
senandGitHub 980f3bbf83 [cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546) 
* Perform 64-byte alignment of wksp tables and aligneds internally

* Clean up cwskp_finalize() function to only do two allocs

* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations

* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned

* Back out aligned/table reservation order restriction

* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
 2021-04-01 20:07:19 -04:00
senandGitHub 255925c231 Fix repcode-related OSS-fuzz issues in block splitter (#2560) 
* Do not emit last partitions of blocks as RLE/uncompressed

* Fix repcode updates within block splitter

* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature

* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
 2021-03-31 15:14:59 -04:00
Nick TerrellandNick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
senandGitHub 84ccb81e7c Merge pull request #2561 from senhuang42/longlength_enum 
Add enum for representing long length ID
 2021-03-26 15:55:12 -04:00
Sen Huang b1a43455f8 Add enum for representing long length ID 2021-03-26 10:41:09 -07:00
senandGitHub 4fe2e7ae14 Merge pull request #2558 from senhuang42/msan_block_splitter_fix 
Fix block splitter minor MSAN warning.
 2021-03-25 13:51:43 -04:00
senandGitHub b0407b9f0e Merge pull request #2555 from senhuang42/default_clevel_func 
Add ZSTD_defaultCLevel() function to public API
 2021-03-25 13:07:28 -04:00
Sen Huang 2a907bf4aa Move lastCountSize into a returned struct, fix MSAN error 2021-03-25 09:11:15 -07:00
Sen Huang e398744a35 Add ZSTD_defaultCLevel() function to public API 2021-03-25 08:04:00 -07:00
Nick TerrellandGitHub f8ac0ea7ef Merge pull request #2539 from terrelln/linux-kernel-fixes 
Fixes for the next linux kernel patch version
 2021-03-24 10:34:29 -07:00
senandGitHub bf542c8a8d Merge pull request #2447 from senhuang42/block_splitter_v2 
Recursive block splitting
 2021-03-24 12:27:22 -04:00
Sen Huang 5b566ebe08 Rename *compressSequences*() functions for clarity 2021-03-24 08:21:29 -07:00
Sen Huang 0ef1f935b7 Add a fallback in case the total blocksize of split blocks exceeds raw block size 2021-03-24 08:21:29 -07:00
Sen Huang c90e81a692 Enable block splitter by default when applicable 2021-03-24 08:21:29 -07:00
Sen Huang e34332834a Clean up various functions, add debuglogging for estimate vs. actual sizes 2021-03-24 08:21:29 -07:00
Sen Huang 41c3eae6d9 Fix various fuzzer failures: repcode history, superblocks 2021-03-24 08:21:29 -07:00
senhuang42 0633bf17c3 Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression 2021-03-24 08:21:29 -07:00
senhuang42 eb1ee8686d Refactor buildSequencesStatistics() to avoid pointer increment for superblocks 2021-03-24 08:21:29 -07:00
senhuang42 e2bb215117 Add unit tests and fuzzer param 2021-03-24 08:21:09 -07:00
senhuang42 de52de1347 Add recursive block split algorithm 2021-03-24 08:21:09 -07:00
senhuang42 f06f6626ed Update function names for consistency 2021-03-24 08:20:54 -07:00
senhuang42 c56d6e49e8 Add block splitter to experimental params 2021-03-24 08:20:54 -07:00
senhuang42 2949a95224 Refactor block compression logic into single function 2021-03-24 08:20:54 -07:00
senhuang42 c05c090cc2 Centralize entropy statistics calculations to zstd_compress.c 2021-03-24 08:20:29 -07:00
senandGitHub c48889f097 Merge pull request #2538 from senhuang42/monotonicity_test 
Add memory monotonicity test over srcSize
 2021-03-22 16:54:34 -04:00
Nick TerrellandGitHub ebc2dfa821 Merge pull request #2524 from terrelln/huf-stack-reduction 
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
 2021-03-22 12:37:54 -07:00
Nick Terrell 634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Sen Huang dff4a0e867 Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well 2021-03-21 16:15:31 -07:00
Nick Terrell 756bd59322 [huf][fse] Clean up workspaces 
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
 2021-03-17 16:50:37 -07:00
ihsinmeandGitHub a5bf09d764 simple fix for using bit operator. 
good day.
It seems to me that the developer intended to use a logical operator.
so I suggest a simple fix.
 2021-03-17 11:37:42 +03:00
Sen Huang 77ae664ba6 Fix ZSTD_dedicatedDictSearch_isSupported() requirements 2021-03-16 17:36:05 -07:00
senhuang42 386111adec Add a nbSeq argument to compressSequences() 
Refactor ZSTD_compressBlock_internal() to do the block header write within and add nbSeq argument to compressSequences()
 2021-03-16 14:04:22 -07:00
senhuang42 98764493cf Move block header write into compressBlock_internal() 2021-03-16 14:04:22 -07:00
Nick Terrell ea288e0d8e [lib] Bump zstd version number 2021-03-16 11:47:27 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro 
When defined, it disables tracing, and avoids including the header.
 2021-03-16 11:47:27 -07:00
Yann ColletandGitHub 3d6c903056 Merge pull request #2521 from animalize/doc_free 
doc: ZSTD_free*() functions accept NULL pointer
 2021-03-06 21:33:28 -08:00