ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Don't shrink window log in ZSTD_getCParams()

Browse Source 
Treat ZSTD_getCParams() and ZSTD_adjustCParams() in the same way
we treat streaming compression. Choose parameters based on the
dictionary size + source size, and assume the source size is small
if unkown. But, don't shrink the window log down in
ZSTD_adjustCParams_internal().
This commit is contained in:
Nick Terrell
2021-01-04 15:54:09 -08:00
parent 9d31c704d5
commit 58476bcf7f
2 changed files with 27 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/compress/zstd_compress.c
+1 -5
View File
@@ -1189,17 +1189,13 @@ ZSTD_adjustCParams_internal(ZSTD_compressionParameters cPar,
assert(ZSTD_checkCParams(cPar)==0);
switch (mode) {
case ZSTD_cpm_unknown:
case ZSTD_cpm_noAttachDict:
/* If we don't know the source size, don't make any
* assumptions about it. We will already have selected
* smaller parameters if a dictionary is in use.
*/
break;
case ZSTD_cpm_unknown:
/* Keep the legacy behavior of assuming small source
* sizes when the cparam mode is unkown.
*/
/* fall-through */
case ZSTD_cpm_createCDict:
/* Assume a small source size when creating a dictionary
* with an unkown source size.
tests/fuzzer.c
+26
View File
@@ -3051,6 +3051,32 @@ static int basicUnitTests(U32 const seed, double compressibility)
free(dict);
}
DISPLAYLEVEL(3, "OK \n");
DISPLAYLEVEL(3, "test%3i : ZSTD_getCParams() + dictionary ", testNb++);
{
ZSTD_compressionParameters const medium = ZSTD_getCParams(1, 16*1024-1, 0);
ZSTD_compressionParameters const large = ZSTD_getCParams(1, 128*1024-1, 0);
ZSTD_compressionParameters const smallDict = ZSTD_getCParams(1, 0, 400);
ZSTD_compressionParameters const mediumDict = ZSTD_getCParams(1, 0, 10000);
ZSTD_compressionParameters const largeDict = ZSTD_getCParams(1, 0, 100000);
assert(!memcmp(&smallDict, &mediumDict, sizeof(smallDict)));
assert(!memcmp(&medium, &mediumDict, sizeof(medium)));
assert(!memcmp(&large, &largeDict, sizeof(large)));
}
DISPLAYLEVEL(3, "OK \n");
DISPLAYLEVEL(3, "test%3i : ZSTD_adjustCParams() + dictionary ", testNb++);
{
ZSTD_compressionParameters const cParams = ZSTD_getCParams(1, 0, 0);
ZSTD_compressionParameters const smallDict = ZSTD_adjustCParams(cParams, 0, 400);
ZSTD_compressionParameters const smallSrcAndDict = ZSTD_adjustCParams(cParams, 500, 400);
assert(smallSrcAndDict.windowLog == 10);
assert(!memcmp(&cParams, &smallDict, sizeof(cParams)));
}
DISPLAYLEVEL(3, "OK \n");
#endif
_end: