ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,968 Commits 17 Branches 77 Tags
13c5ec3e413af3157b9bef5ed3eea42e5fb3ff18
Commit Graph
3391 Commits
Author SHA1 Message Date
W. Felix Handte 13c5ec3e41 Only Allow Dedicated Dict Search for Dicts Loaded in 1 Chunk 
The load algorithm requires we do it all in one go.
 2020-09-10 22:10:02 -04:00
W. Felix Handte 07793547e6 Fix Bug: Only Use DDSS Insertion on CDict MatchStates 
Previously, if DDSS was enabled on a CCtx and a dictionary was inserted into
the CCtx, the CCtx MatchState would be filled as a DDSS struct, causing
segfaults etc. This changes the check to use whether the MatchState is marked
as using the DDSS (which is only ever set for CDict MatchStates), rather than
looking at the CCtxParams.
 2020-09-10 18:51:52 -04:00
W. Felix Handte d214d8c859 Shorten Dict Mode Conditionals in Order to Improve Readability 2020-09-10 18:51:52 -04:00
W. Felix Handte f49c1563ff Force-Inline ZSTD_insertAndFindFirstIndex_internal() 
Without this, gcc was declining to inline the function in `ZSTD_noDict` mode,
resulting in a ~10% slowdown.
 2020-09-10 18:51:52 -04:00
W. Felix Handte cab86b074f Clean Up Search Function Selection 2020-09-10 18:51:52 -04:00
W. Felix Handte 2ffbde0d95 Fix -Wshorten-64-to-32 Error 2020-09-10 18:51:52 -04:00
W. Felix Handte 7b5d2f72ea Adjust Working Context Table Sizes Back Down 2020-09-10 18:51:52 -04:00
W. Felix Handte c09454e28f Add Warning Comment to ZSTD_createCDict_advanced2() Declaration 2020-09-10 18:51:52 -04:00
W. Felix Handte d332f57897 Permit Matching Against Lowest Valid Position 
This comparison was previously faulty: the lowest valid position is itself
valid, and we should therefore be allowed to match against it.
 2020-09-10 18:51:52 -04:00
W. Felix Handte a3659fe1ef Make ZSTD_dedicatedDictSearch_getCParams Wrap ZSTD_getCParams 
Fixes up bounds-checking, and lets us clean up what is at the moment an
unnecessary duplication of the default cparams tables.
 2020-09-10 18:51:52 -04:00
W. Felix Handte 7b9a755ac9 Remove Chain Limit on Hash Cache Entries; Slightly Improve Compression 
Entries in the hashTable chain cache aren't subject to the same aliasing that
the circular chain table is subject to. As such, we don't need to stop when we
cross the chain limit. We can delve deeper. :)
 2020-09-10 18:51:52 -04:00
W. Felix Handte e8b4011b52 Split Lookups in Hash Cache and Chain Table into Two Loops 
Sliiiight speedup.
 2020-09-10 18:51:52 -04:00
W. Felix Handte 9e83c782f8 Simplify DDS Hash Table Construction 
No need to walk the chainTable; we can just keep shifting the entries in the
hashTable.
 2020-09-10 18:51:52 -04:00
W. Felix Handte ad9f98ac3f Document the ZSTD_c_enableDedicatedDictSearch Parameter 2020-09-10 18:51:52 -04:00
W. Felix Handte 5390fee4f7 Rename and Move DD_BLOG Constant to ZSTD_LAZY_DDSS_BUCKET_LOG 2020-09-10 18:51:52 -04:00
W. Felix Handte 5e91ae27eb Prefetch First Batch of Match Positions; +11% Speed in Level 5 w/ 1 Dict 2020-09-10 18:51:52 -04:00
W. Felix Handte df386b3d8d Fix Off-By-One Error in Counting DDS Search Attempts 
This caused us to double-search the first position and fail to search the
last position in the chain, slowing down search and making it less effective.
 2020-09-10 18:51:52 -04:00
W. Felix Handte 914bfe7ee4 Init CCtx's Local Dict with CCtxParams 2020-09-10 18:51:52 -04:00
W. Felix Handte db2aa25252 Decision for Whether to Attach Should be Based on CDict Config, not CCtx 2020-09-10 18:51:52 -04:00
W. Felix Handte a494111385 Move Prefetch Before Insertion; Speed Up ~6% 2020-09-10 18:51:52 -04:00
W. Felix Handte eede46a47e Misc Refactor of DDS Search Code 2020-09-10 18:51:52 -04:00
W. Felix Handte f1b428fdac Rename enableDedicatedDictSearch to dedicatedDictSearch in MatchState 
This makes it clear that not only is the feature allowed here, we're actually
using it, as opposed to the CCtxParam field, in which it's enabled, but we may
or may not be using it.
 2020-09-10 18:51:52 -04:00
W. Felix Handte 41012193ad Always Init CDict's enableDedicatedDictSearch Field 2020-09-10 18:51:52 -04:00
W. Felix Handte 34b545acb0 Add a ZSTD_dedicatedDictSearch ZSTD_dictMode_e to Allow Const Propagation 
Speed +1.5%.
 2020-09-10 18:51:52 -04:00
W. Felix Handte beefdb0d3d Fix ZSTD_c_forceAttachDict Bounds 2020-09-10 18:51:52 -04:00
W. Felix Handte c204110eff Make ZSTD_c_enableDedicatedDictSearch an Experimental Param 2020-09-10 18:51:52 -04:00
W. Felix Handte ae4ebf6b8c TODO: Comment 2020-09-10 18:51:52 -04:00
W. Felix Handte def62e2d3e Fix Compilation Warnings 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 9c628238d3 creating ZSTD_createCDict_advanced_internal 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 0a9787c3e1 changing to int for consistency 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte e29bc3a009 using dict mls instead of src mls 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 145c2d12f9 add hashtable head prefetching 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 5d5507788d change method name for consistency 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte b30f71becf pass correct cparams 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte a3f6e4026e removing wrong comment 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 71fda0362f making cctxParams a pointer 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 628559d0e4 loading dict using new algorithm 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 22705f0c93 adding dedicatedDictSearch algorithm 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 31e581bf65 adding enableDedicatedDictSearch to matchState_t 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 50550a14ad adding dedicated dict load method to lazy 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 75b6360036 adding ZSTD_createCDict_advanced2 to zstd.h 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte b7dddbe89b always attach dict when using dedicatedDictSearch 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte e36a373df4 adding dedicatedDictSearch cParams helper methods 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte f10d4e313c adding ZSTD_dedicatedDictSearch_defaultCParameters variable 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte c497cb6716 Add ZSTD_c_enableDedicatedDictSearch Param 2020-09-10 18:51:52 -04:00
Nick Terrell da30a78c68 [lib] Bump version number to 1.4.6 2020-09-09 17:13:45 -07:00
Nick Terrell b92569a522 [doc] Document new build macros in lib/README.md 2020-09-09 17:13:16 -07:00
Nick Terrell aab4bf7b0d [linux-kernel] Add test that checks the ifdef hardwiring 2020-09-09 14:36:19 -07:00
Nick Terrell 79ded1b4a9 [lib] Add ZSTD_NO_UNUSED_FUNCTIONS macro to hide unused functions 
The unused function definitions are hidden behind a
`#ifndef ZSTD_NO_UNUSED_FUNCTIONS` check.

Initially hiding all functions which are unused and take up more than
2KB of stack space, because these will show up as warnings in the
Linux Kernel build system.
 2020-09-09 14:35:39 -07:00
Nick Terrell ac3a136b0a [lib] Replace 64-bit divisions with ZSTD_div64() 2020-09-09 14:35:39 -07:00