ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8,691 Commits 17 Branches 77 Tags
f80dec66b0fdeeb0bb08ab4739a2549476e8cb4e
Commit Graph
3783 Commits
Author SHA1 Message Date
senhuang42 3b98987496 Remove building of ZBUFF/deprecated folder by default 2021-04-19 17:12:00 -04:00
senandGitHub 12c045f74d Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix 
Correct the block splitter mismatched repcodes detection.
 2021-04-12 23:27:43 -04:00
Sen Huang 8844f93957 Adjust nb elements to prefetch in ZSTD_row_fillHashCache() 2021-04-12 14:24:58 -04:00
Sen Huang 550f76f131 Correct the detection of mismatched repcodes 2021-04-09 09:08:51 -07:00
Sen Huang 4d63d6e8aa Update results.csv, add Row hash to regression test 2021-04-07 10:31:41 -07:00
Nick TerrellandSen Huang 4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
senandGitHub f71aabb5b5 Move clevel override to after initLocalDict() (#2571) 2021-04-06 21:05:37 -04:00
senandGitHub f1e8b565c2 Maintain two repcode histories for block splitting, replace invalid repcodes (#2569) 2021-04-06 17:25:55 -04:00
senandGitHub e38124555e Fix dictionary force reloading clevel selection (#2570) 
* Move cdict clevel override to before localdict init

* Update results.csv after dict load changes
 2021-04-06 15:35:09 -04:00
Nick TerrellandGitHub 8383fc828d Merge pull request #2541 from ihsinme/patch-1 
simple fix for using bit operator.
 2021-04-02 13:01:09 -07:00
senandGitHub 980f3bbf83 [cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546) 
* Perform 64-byte alignment of wksp tables and aligneds internally

* Clean up cwskp_finalize() function to only do two allocs

* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations

* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned

* Back out aligned/table reservation order restriction

* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
 2021-04-01 20:07:19 -04:00
senandGitHub 255925c231 Fix repcode-related OSS-fuzz issues in block splitter (#2560) 
* Do not emit last partitions of blocks as RLE/uncompressed

* Fix repcode updates within block splitter

* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature

* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
 2021-03-31 15:14:59 -04:00
Nick TerrellandNick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
senandGitHub 84ccb81e7c Merge pull request #2561 from senhuang42/longlength_enum 
Add enum for representing long length ID
 2021-03-26 15:55:12 -04:00
Sen Huang b1a43455f8 Add enum for representing long length ID 2021-03-26 10:41:09 -07:00
senandGitHub 4fe2e7ae14 Merge pull request #2558 from senhuang42/msan_block_splitter_fix 
Fix block splitter minor MSAN warning.
 2021-03-25 13:51:43 -04:00
senandGitHub b0407b9f0e Merge pull request #2555 from senhuang42/default_clevel_func 
Add ZSTD_defaultCLevel() function to public API
 2021-03-25 13:07:28 -04:00
Sen Huang 2a907bf4aa Move lastCountSize into a returned struct, fix MSAN error 2021-03-25 09:11:15 -07:00
Sen Huang e398744a35 Add ZSTD_defaultCLevel() function to public API 2021-03-25 08:04:00 -07:00
Nick TerrellandGitHub f8ac0ea7ef Merge pull request #2539 from terrelln/linux-kernel-fixes 
Fixes for the next linux kernel patch version
 2021-03-24 10:34:29 -07:00
senandGitHub bf542c8a8d Merge pull request #2447 from senhuang42/block_splitter_v2 
Recursive block splitting
 2021-03-24 12:27:22 -04:00
Sen Huang 5b566ebe08 Rename *compressSequences*() functions for clarity 2021-03-24 08:21:29 -07:00
Sen Huang 0ef1f935b7 Add a fallback in case the total blocksize of split blocks exceeds raw block size 2021-03-24 08:21:29 -07:00
Sen Huang c90e81a692 Enable block splitter by default when applicable 2021-03-24 08:21:29 -07:00
Sen Huang e34332834a Clean up various functions, add debuglogging for estimate vs. actual sizes 2021-03-24 08:21:29 -07:00
Sen Huang 41c3eae6d9 Fix various fuzzer failures: repcode history, superblocks 2021-03-24 08:21:29 -07:00
senhuang42 0633bf17c3 Change 1.3.4 bugfix to be cross-compatible with superblocks and normal compression 2021-03-24 08:21:29 -07:00
senhuang42 eb1ee8686d Refactor buildSequencesStatistics() to avoid pointer increment for superblocks 2021-03-24 08:21:29 -07:00
senhuang42 e2bb215117 Add unit tests and fuzzer param 2021-03-24 08:21:09 -07:00
senhuang42 de52de1347 Add recursive block split algorithm 2021-03-24 08:21:09 -07:00
senhuang42 f06f6626ed Update function names for consistency 2021-03-24 08:20:54 -07:00
senhuang42 c56d6e49e8 Add block splitter to experimental params 2021-03-24 08:20:54 -07:00
senhuang42 2949a95224 Refactor block compression logic into single function 2021-03-24 08:20:54 -07:00
senhuang42 c05c090cc2 Centralize entropy statistics calculations to zstd_compress.c 2021-03-24 08:20:29 -07:00
senandGitHub c48889f097 Merge pull request #2538 from senhuang42/monotonicity_test 
Add memory monotonicity test over srcSize
 2021-03-22 16:54:34 -04:00
Nick TerrellandGitHub ebc2dfa821 Merge pull request #2524 from terrelln/huf-stack-reduction 
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
 2021-03-22 12:37:54 -07:00
Nick Terrell 634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Sen Huang dff4a0e867 Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well 2021-03-21 16:15:31 -07:00
Nick Terrell 756bd59322 [huf][fse] Clean up workspaces 
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
 2021-03-17 16:50:37 -07:00
ihsinmeandGitHub a5bf09d764 simple fix for using bit operator. 
good day.
It seems to me that the developer intended to use a logical operator.
so I suggest a simple fix.
 2021-03-17 11:37:42 +03:00
Sen Huang 77ae664ba6 Fix ZSTD_dedicatedDictSearch_isSupported() requirements 2021-03-16 17:36:05 -07:00
senhuang42 386111adec Add a nbSeq argument to compressSequences() 
Refactor ZSTD_compressBlock_internal() to do the block header write within and add nbSeq argument to compressSequences()
 2021-03-16 14:04:22 -07:00
senhuang42 98764493cf Move block header write into compressBlock_internal() 2021-03-16 14:04:22 -07:00
Nick Terrell ea288e0d8e [lib] Bump zstd version number 2021-03-16 11:47:27 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro 
When defined, it disables tracing, and avoids including the header.
 2021-03-16 11:47:27 -07:00
Yann ColletandGitHub 3d6c903056 Merge pull request #2521 from animalize/doc_free 
doc: ZSTD_free*() functions accept NULL pointer
 2021-03-06 21:33:28 -08:00
Nick Terrell 3b1aba42cc [fse] Reduce stack usage of FSE_decompress_wksp() by 512 bytes 
* Move `counting` into the workspace
* Inrease `HUF_DECOMPRESS_WORKSPACE_SIZE` by 512 bytes
 2021-03-05 13:24:27 -08:00
Nick Terrell 0f18059a4e [huf] Reduce stack usage of HUF_readDTableX2 by ~460 bytes 
* Use `HUF_readStats_wksp()`
* Use workspace in `HUF_fillDTableX2*()`
* Clean up workspace usage to use a workspace struct
 2021-03-05 12:39:46 -08:00
Nick TerrellandGitHub b5fd348a85 Merge pull request #2523 from terrelln/huf-stack-reduction 
Add HUF_writeCTable_wksp() function
 2021-03-05 12:35:09 -08:00
Nick Terrell 5df2a21f1e Add HUF_writeCTable_wksp() function 
This saves ~700 bytes of stack space in HUF_writeCTable.
 2021-03-05 10:29:18 -08:00