ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

policy change : ZSTDMT automatically caps nbThreads to ZSTDMT_NBTHREADS_MAX (#760)

Browse Source 
Previously, ZSTDMT would refuse to create the compressor.
Also : increased ZSTDMT_NBTHREADS_MAX to 256,
updated doc,
and added relevant test
This commit is contained in:
Yann Collet
2017-07-13 10:17:23 -07:00
parent 132e6efd76
commit 3a60efd3a9
4 changed files with 8 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/compress/zstdmt_compress.c
+3 -2
View File
@@ -9,7 +9,7 @@
/* ====== Tuning parameters ====== */ /* ====== Tuning parameters ====== */
#define ZSTDMT_NBTHREADS_MAX 128 #define ZSTDMT_NBTHREADS_MAX 256
#define ZSTDMT_OVERLAPLOG_DEFAULT 6 #define ZSTDMT_OVERLAPLOG_DEFAULT 6
@@ -407,7 +407,8 @@ ZSTDMT_CCtx* ZSTDMT_createCCtx_advanced(unsigned nbThreads, ZSTD_customMem cMem)
U32 nbJobs = nbThreads + 2; U32 nbJobs = nbThreads + 2;
DEBUGLOG(3, "ZSTDMT_createCCtx_advanced"); DEBUGLOG(3, "ZSTDMT_createCCtx_advanced");
if ((nbThreads < 1) | (nbThreads > ZSTDMT_NBTHREADS_MAX)) return NULL; if (nbThreads < 1) return NULL;
nbThreads = MIN(nbThreads , ZSTDMT_NBTHREADS_MAX);
if ((cMem.customAlloc!=NULL) ^ (cMem.customFree!=NULL)) if ((cMem.customAlloc!=NULL) ^ (cMem.customFree!=NULL))
/* invalid custom allocator */ /* invalid custom allocator */
return NULL; return NULL;
programs/zstd.1
+2 -2
View File
@@ -1,5 +1,5 @@
. .
.TH "ZSTD" "1" "June 2017" "zstd 1.3.0" "User Commands" .TH "ZSTD" "1" "July 2017" "zstd 1.3.1" "User Commands"
. .
.SH "NAME" .SH "NAME"
\fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files \fBzstd\fR \- zstd, zstdmt, unzstd, zstdcat \- Compress or decompress \.zst files
@@ -105,7 +105,7 @@ unlocks high compression levels 20+ (maximum 22), using a lot more memory\. Note
. .
.TP .TP
\fB\-T#\fR, \fB\-\-threads=#\fR \fB\-T#\fR, \fB\-\-threads=#\fR
Compress using \fB#\fR threads (default: 1)\. If \fB#\fR is 0, attempt to detect and use the number of physical CPU cores\. This modifier does nothing if \fBzstd\fR is compiled without multithread support\. Compress using \fB#\fR threads (default: 1)\. If \fB#\fR is 0, attempt to detect and use the number of physical CPU cores\. In all cases, the nb of threads is capped to ZSTDMT_NBTHREADS_MAX==256\. This modifier does nothing if \fBzstd\fR is compiled without multithread support\.
. .
.TP .TP
\fB\-D file\fR \fB\-D file\fR
programs/zstd.1.md
+1
View File
@@ -108,6 +108,7 @@ the last one takes effect.
* `-T#`, `--threads=#`: * `-T#`, `--threads=#`:
Compress using `#` threads (default: 1). Compress using `#` threads (default: 1).
If `#` is 0, attempt to detect and use the number of physical CPU cores. If `#` is 0, attempt to detect and use the number of physical CPU cores.
In all cases, the nb of threads is capped to ZSTDMT_NBTHREADS_MAX==256.
This modifier does nothing if `zstd` is compiled without multithread support. This modifier does nothing if `zstd` is compiled without multithread support.
* `-D file`: * `-D file`:
use `file` as Dictionary to compress or decompress FILE(s) use `file` as Dictionary to compress or decompress FILE(s)
tests/playTests.sh
+2 -1
View File
@@ -643,7 +643,8 @@ then
$ECHO "\n**** zstdmt long round-trip tests **** " $ECHO "\n**** zstdmt long round-trip tests **** "
roundTripTest -g99000000 -P99 "20 -T2" roundTripTest -g99000000 -P99 "20 -T2"
roundTripTest -g6000000000 -P99 "1 -T2" roundTripTest -g6000000000 -P99 "1 -T2"
fileRoundTripTest -g4193M -P98 " -T0" roundTripTest -g1500000000 -P97 "1 -T999"
fileRoundTripTest -g4195M -P98 " -T0"
else else
$ECHO "\n**** no multithreading, skipping zstdmt tests **** " $ECHO "\n**** no multithreading, skipping zstdmt tests **** "
fi fi