ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,783 Commits 17 Branches 77 Tags
c224367edec9052d22af101034d69bcecb22bcb7
Commit Graph
7783 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandGitHub 3934e010a2 Merge pull request #1197 from facebook/poolResize 
Thread Pool resize
 2018-06-22 14:20:07 -07:00
Yann Collet fbd5dfc1b1 changed POOL_resize() return type to int 
return is now just en error code.
This guarantee that `ctx` remains valid after POOL_resize().
Gets rid of internal POOL_free() operation.
 2018-06-22 12:14:59 -07:00
Yann ColletandGitHub d70c4a5074 Merge pull request #1201 from facebook/rfcUpdate 
updated Zstandard frame format
 2018-06-22 11:53:50 -07:00
Yann ColletandGitHub 1d5648ca10 Merge pull request #1196 from felixhandte/zstd-btopt-in-place-dict 
ZSTD_btopt: Support Searching the Dictionary Context In-Place
 2018-06-22 11:53:23 -07:00
Yann ColletandGitHub f6242d30b7 Merge pull request #1202 from facebook/barelyCompressible 
Increase threshold detection of poorly compressible data
 2018-06-22 11:52:52 -07:00
Yann Collet 698fd00afb huf: increase threshold detection of poorly compressible data 2018-06-21 18:32:38 -07:00
Yann Collet c1e6347717 fixed minor typos, detected by @terrelln 2018-06-21 18:08:11 -07:00
Yann Collet 243cd9d8bb add a cond_broadcast after resize 
to make sure all threads (notably newly available threads)
get awaken to immediately process potential items in the queue.
 2018-06-21 18:04:58 -07:00
Yann Collet 7639db939f updated Zstandard frame format 
adding clarifications from IETF RFC DISCUSS.
 2018-06-21 17:55:55 -07:00
George Lu ab26f24c9c benchFunction Timed Wrappers 
Add BMK_benchFunctionTimed
Add BMK_init_customResultCont..
Change benchMem to use benchFunctionTimed
Minor Fixes/Adjustments
 2018-06-21 16:23:55 -07:00
George Lu a8eea99ebe Incremental Display + Fn Separations 
Seperate syntheticTest and fileTableTest (now renamed as benchFiles)
Add incremental display to benchMem
Change to only iterMode for benchFunction
Make Synthetic test's compressibility configurable from cli (using -P#)
 2018-06-21 16:23:18 -07:00
Yann Collet 818e72b4d5 added extended POOL test 
abrupt end + downsizing with running jobs remaining in queue.

also : POOL_resize() requires numThreads >= 1
 2018-06-21 14:58:59 -07:00
W. Felix Handte 01bb1c1016 Add CCtx Param Controlling Dict Attachment Behavior 2018-06-21 17:29:25 -04:00
W. Felix Handte 3e91dc4d6a Add Repcode Bounds Check 2018-06-21 15:54:41 -04:00
W. Felix Handte 5bd3d4b7d2 Add Debug Log Statement 2018-06-21 15:54:07 -04:00
W. Felix Handte 3caba150c6 Fix dmsBtLow Test 2018-06-21 15:53:40 -04:00
Yann Collet 7d80ada5ca added a test for POOL (multithreading) 
ensuring all jobs in queue are nonetheless completed
when POOL is instructed to end abruptly (POOL_free())
 2018-06-21 12:24:36 -07:00
W. Felix Handte 5da9bbc38e Conceivably Dedup ZSTD_noDict and ZSTD_dictMatchState _insertBt1 Impls 
By reverting to the bool extDict flag, we call ZSTD_insertBt1 with the same
const args in both non-extDict dictModes.
 2018-06-21 11:20:01 -04:00
Yann Collet 6de249c1c6 fixed: bug when counting nb of active threads 
when queueSize > 1

also : added a test in testpool.c
       verifying resizing is effective.
 2018-06-20 18:28:49 -07:00
Yann Collet 6b48eb12c0 change control of threadLimit 
now limits maximum nb of active threads
even when queueSize > 1.
 2018-06-20 14:35:39 -07:00
Jennifer Liu a3cb97b038 set up sample config.yml 2018-06-20 14:23:14 -07:00
W. Felix Handte 5d81f71e83 Consistency in Guarding DMS-Only Variable Initializations 2018-06-20 16:54:53 -04:00
W. Felix Handte 9c14eafe3d Also Use matchLow for HC3 Match 2018-06-20 15:51:14 -04:00
W. Felix Handte 0a6cf7cd1d Minor Changes 2018-06-20 15:27:23 -04:00
W. Felix Handte ae1f3898a2 Remove Dead(!) HC3 DMS Lookup 2018-06-20 15:27:12 -04:00
Yann ColletandGitHub 93702a7a62 Merge pull request #1198 from facebook/msdebug 
made Visual Studio compatible with DEBUGLEVEL >= 2
 2018-06-20 12:26:31 -07:00
cyan4973 ae0b7ffa0a made Visual Studio compatible with DEBUGLEVEL >= 2 2018-06-20 09:45:02 -07:00
Yann Collet 62469c9f41 fixed wrong size in pthread struct transfer 2018-06-19 20:14:03 -07:00
Yann Collet 166901dc72 reduced POOL_resize() restriction 
It's not necessary to ensure that no job is ongoing.
The pool is only expanded, existing threads are preserved.
In case of error, the only option is to return NULL and terminate the thread pool anyway.
 2018-06-19 18:07:18 -07:00
Yann Collet 066fbbfe1c make zstdmt resize its context 
when nbThreads change.

Technically, it only expands.
But when instructed to use less threads,
the thread pool will limit nb of concurrent threads.
 2018-06-19 17:28:56 -07:00
Yann Collet 4567c57199 finalized POOL_resize() 
POOL_ctx* POOL_resize(POOL_ctx* ctx, size_t numThreads)

The function may fail, and returns a NULL pointer in this case.
 2018-06-19 16:03:12 -07:00
Yann ColletandGitHub 6768cf53fd Merge pull request #1190 from terrelln/ldm-adjust 
Adjust advanced parameters to source size
 2018-06-19 14:40:56 -07:00
Yann ColletandGitHub c0b6ce95b1 Merge pull request #1179 from supertopher/dev 
Improves UX for --list command's lack of support for pipes
 2018-06-19 14:36:30 -07:00
W. Felix Handte 03c39c540b Fix Incorrect Param 2018-06-19 15:36:33 -04:00
W. Felix Handte de639502aa Update Dict Attachment Cut-Offs 2018-06-19 15:36:13 -04:00
W. Felix Handte f0a13bcd68 Make Sure Position 0 Gets Into the Tree 2018-06-19 15:10:06 -04:00
Yann ColletandGitHub cb45b41f16 Merge pull request #1194 from facebook/no-opaqueapi 
removed specific --opaqueapi test
 2018-06-19 10:24:26 -07:00
W. Felix Handte 87fe4788a3 Fix Compression Ratio Regression #1 2018-06-19 13:01:21 -04:00
W. Felix Handte 4bb79f9c55 Misc Changes 2018-06-19 13:01:21 -04:00
W. Felix Handte 2091f34e9e Find Proper Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte 64348a15f1 Misc Fixes 2018-06-19 13:01:21 -04:00
W. Felix Handte ade8586ce6 Find mls == 3 Matches 2018-06-19 13:01:21 -04:00
W. Felix Handte ce743312e2 Fix Typo 2018-06-19 13:01:21 -04:00
W. Felix Handte a075864756 Switch != ZSTD_extDict to == ZSTD_noDict 2018-06-19 13:01:21 -04:00
W. Felix Handte 1e03377bde Implement RepCode Check 2018-06-19 13:01:21 -04:00
W. Felix Handte ccbf067973 Add _dictMatchState Functions 2018-06-19 13:01:21 -04:00
W. Felix Handte d5d8240967 Convert extDict Flag to dictMode Enum 2018-06-19 13:01:21 -04:00
W. Felix Handte 93c3184d44 Attach Dicts when Using ZSTD_btopt and ZSTD_btultra 2018-06-19 13:01:21 -04:00
Topher Lubaway 5bac1db28f Tests to verify piped input to --list exits 1 
I'm following the pattern that i saw in the rest of the test file
please tell me if i am using the wrong conventions
 2018-06-19 09:56:37 -07:00
Yann Collet 1c714fda3f introduced POOL_resize() 
not complete yet :
finalize behavior in case of unfinished expansion
 2018-06-18 20:46:39 -07:00