fixed fullbench, part 2

This commit is contained in:
Yann Collet
2017-07-06 02:48:00 -07:00
parent 9b2c1acfc0
commit 7758ed8458
2 changed files with 40 additions and 39 deletions
+40 -38
View File
@@ -73,27 +73,41 @@
or an errorCode if it fails (which can be tested using ZSTD_isError()). or an errorCode if it fails (which can be tested using ZSTD_isError()).
</p></pre><BR> </p></pre><BR>
<pre><b>unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize); <pre><b>#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
</b><p> NOTE: This function is planned to be obsolete, in favor of ZSTD_getFrameContentSize(). #define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
ZSTD_getFrameContentSize() works the same way, unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
returning the decompressed size of a single frame, </b><p> `src` should point to the start of a ZSTD encoded frame.
but distinguishes empty frames from frames with an unknown size, or errors. `srcSize` must be at least as large as the frame header.
hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
'src' is the start of a zstd compressed frame. @return : - decompressed size of the frame in `src`, if known
@return : content size to be decompressed, as a 64-bits value _if known_, 0 otherwise. - ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
note 1 : decompressed size is an optional field, it may not be present, typically in streaming mode. - ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
When `return==0`, data to decompress could be any size. note 1 : a 0 return value means the frame is valid but "empty".
note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode.
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
In which case, it's necessary to use streaming mode to decompress data. In which case, it's necessary to use streaming mode to decompress data.
Optionally, application can use ZSTD_decompress() while relying on implied limits. Optionally, application can rely on some implicit limit,
(For example, data may be necessarily cut into blocks <= 16 KB). as ZSTD_decompress() only needs an upper bound of decompressed size.
note 2 : decompressed size is always present when compression is done with ZSTD_compress() (For example, data could be necessarily cut into blocks <= 16 KB).
note 3 : decompressed size can be very large (64-bits value), note 3 : decompressed size is always present when compression is done with ZSTD_compress()
note 4 : decompressed size can be very large (64-bits value),
potentially larger than what local system can handle as a single memory segment. potentially larger than what local system can handle as a single memory segment.
In which case, it's necessary to use streaming mode to decompress data. In which case, it's necessary to use streaming mode to decompress data.
note 4 : If source is untrusted, decompressed size could be wrong or intentionally modified. note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
Always ensure result fits within application's authorized limits. Always ensure return value fits within application's authorized limits.
Each application can set its own limits. Each application can set its own limits.
note 5 : when `return==0`, if precise failure cause is needed, use ZSTD_getFrameHeader() to know more. note 6 : This function replaces ZSTD_getDecompressedSize()
</p></pre><BR>
<pre><b>unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);
</b><p> NOTE: This function is now obsolete, in favor of ZSTD_getFrameContentSize().
Both functions work the same way,
but ZSTD_getDecompressedSize() blends
"empty", "unknown" and "error" results in the same return value (0),
while ZSTD_getFrameContentSize() distinguishes them.
'src' is the start of a zstd compressed frame.
@return : content size to be decompressed, as a 64-bits value _if known and not empty_, 0 otherwise.
</p></pre><BR> </p></pre><BR>
<h3>Helper functions</h3><pre></pre><b><pre>int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b> <h3>Helper functions</h3><pre></pre><b><pre>int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b>
@@ -298,8 +312,8 @@ size_t ZSTD_decompressStream(ZSTD_DStream* zds, ZSTD_outBuffer* output, ZSTD_inB
<pre><b>size_t ZSTD_DStreamOutSize(void); </b>/*!< recommended size for output buffer. Guarantee to successfully flush at least one complete block in all circumstances. */<b> <pre><b>size_t ZSTD_DStreamOutSize(void); </b>/*!< recommended size for output buffer. Guarantee to successfully flush at least one complete block in all circumstances. */<b>
</b></pre><BR> </b></pre><BR>
<a name="Chapter10"></a><h2>START OF ADVANCED AND EXPERIMENTAL FUNCTIONS</h2><pre> The definitions in this section are considered experimental. <a name="Chapter10"></a><h2>START OF ADVANCED AND EXPERIMENTAL FUNCTIONS</h2><pre> The definitions in this section are considered experimental.
They should never be used with a dynamic library, as they may change in the future. They should never be used with a dynamic library, as prototypes may change in the future.
They are provided for advanced usages. They are provided for advanced scenarios.
Use them only in association with static linking. Use them only in association with static linking.
<BR></pre> <BR></pre>
@@ -348,26 +362,15 @@ static const ZSTD_customMem ZSTD_defaultCMem = { NULL, NULL, NULL };
<pre><b>size_t ZSTD_findFrameCompressedSize(const void* src, size_t srcSize); <pre><b>size_t ZSTD_findFrameCompressedSize(const void* src, size_t srcSize);
</b><p> `src` should point to the start of a ZSTD encoded frame or skippable frame </b><p> `src` should point to the start of a ZSTD encoded frame or skippable frame
`srcSize` must be at least as large as the frame `srcSize` must be at least as large as the frame
@return : the compressed size of the frame pointed to by `src`, @return : the compressed size of the first frame starting at `src`,
suitable to pass to `ZSTD_decompress` or similar, suitable to pass to `ZSTD_decompress` or similar,
or an error code if given invalid input. or an error code if input is invalid
</p></pre><BR>
<pre><b>#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
#define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
</b><p> `src` should point to the start of a ZSTD encoded frame.
`srcSize` must be at least as large as the frame header.
A value >= `ZSTD_frameHeaderSize_max` is guaranteed to be large enough.
@return : - decompressed size of the frame pointed to be `src` if known
- ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
- ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
</p></pre><BR> </p></pre><BR>
<pre><b>unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize); <pre><b>unsigned long long ZSTD_findDecompressedSize(const void* src, size_t srcSize);
</b><p> `src` should point the start of a series of ZSTD encoded and/or skippable frames </b><p> `src` should point the start of a series of ZSTD encoded and/or skippable frames
`srcSize` must be the _exact_ size of this series `srcSize` must be the _exact_ size of this series
(i.e. there should be a frame boundary exactly `srcSize` bytes after `src`) (i.e. there should be a frame boundary exactly at `srcSize` bytes after `src`)
@return : - decompressed size of all data in all successive frames @return : - decompressed size of all data in all successive frames
- if the decompressed size cannot be determined: ZSTD_CONTENTSIZE_UNKNOWN - if the decompressed size cannot be determined: ZSTD_CONTENTSIZE_UNKNOWN
- if an error occurred: ZSTD_CONTENTSIZE_ERROR - if an error occurred: ZSTD_CONTENTSIZE_ERROR
@@ -375,8 +378,6 @@ unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
note 1 : decompressed size is an optional field, that may not be present, especially in streaming mode. note 1 : decompressed size is an optional field, that may not be present, especially in streaming mode.
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size. When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
In which case, it's necessary to use streaming mode to decompress data. In which case, it's necessary to use streaming mode to decompress data.
Optionally, application can still use ZSTD_decompress() while relying on implied limits.
(For example, data may be necessarily cut into blocks <= 16 KB).
note 2 : decompressed size is always present when compression is done with ZSTD_compress() note 2 : decompressed size is always present when compression is done with ZSTD_compress()
note 3 : decompressed size can be very large (64-bits value), note 3 : decompressed size can be very large (64-bits value),
potentially larger than what local system can handle as a single memory segment. potentially larger than what local system can handle as a single memory segment.
@@ -385,7 +386,7 @@ unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
Always ensure result fits within application's authorized limits. Always ensure result fits within application's authorized limits.
Each application can set its own limits. Each application can set its own limits.
note 5 : ZSTD_findDecompressedSize handles multiple frames, and so it must traverse the input to note 5 : ZSTD_findDecompressedSize handles multiple frames, and so it must traverse the input to
read each contained frame header. This is efficient as most of the data is skipped, read each contained frame header. This is fast as most of the data is skipped,
however it does mean that all frame data must be present and valid. however it does mean that all frame data must be present and valid.
</p></pre><BR> </p></pre><BR>
@@ -483,14 +484,15 @@ size_t ZSTD_estimateDDictSize(size_t dictSize, unsigned byReference);
It is important that dictBuffer outlives CDict, it must remain read accessible throughout the lifetime of CDict It is important that dictBuffer outlives CDict, it must remain read accessible throughout the lifetime of CDict
</p></pre><BR> </p></pre><BR>
<pre><b>typedef enum { ZSTD_dm_auto=0, </b>/* dictionary is "full" if it starts with ZSTD_MAGIC_DICTIONARY, rawContent otherwize */<b> <pre><b>typedef enum { ZSTD_dm_auto=0, </b>/* dictionary is "full" if it starts with ZSTD_MAGIC_DICTIONARY, otherwise it is "rawContent" */<b>
ZSTD_dm_rawContent, </b>/* ensures dictionary is always loaded as rawContent, even if it starts with ZSTD_MAGIC_DICTIONARY */<b> ZSTD_dm_rawContent, </b>/* ensures dictionary is always loaded as rawContent, even if it starts with ZSTD_MAGIC_DICTIONARY */<b>
ZSTD_dm_fullDict </b>/* refuses to load a dictionary if it does not respect Zstandard's specification */<b> ZSTD_dm_fullDict </b>/* refuses to load a dictionary if it does not respect Zstandard's specification */<b>
} ZSTD_dictMode_e; } ZSTD_dictMode_e;
</b></pre><BR> </b></pre><BR>
<pre><b>ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictSize, <pre><b>ZSTD_CDict* ZSTD_createCDict_advanced(const void* dict, size_t dictSize,
unsigned byReference, ZSTD_dictMode_e dictMode, unsigned byReference, ZSTD_dictMode_e dictMode,
ZSTD_compressionParameters cParams, ZSTD_customMem customMem); ZSTD_compressionParameters cParams,
ZSTD_customMem customMem);
</b><p> Create a ZSTD_CDict using external alloc and free, and customized compression parameters </b><p> Create a ZSTD_CDict using external alloc and free, and customized compression parameters
</p></pre><BR> </p></pre><BR>
-1
View File
@@ -114,7 +114,6 @@ size_t local_ZSTD_decodeLiteralsBlock(void* dst, size_t dstSize, void* buff2, co
return ZSTD_decodeLiteralsBlock((ZSTD_DCtx*)g_zdc, buff2, g_cSize); return ZSTD_decodeLiteralsBlock((ZSTD_DCtx*)g_zdc, buff2, g_cSize);
} }
extern size_t ZSTD_getcBlockSize(const void* src, size_t srcSize, blockProperties_t* bpPtr);
extern size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeq, const void* src, size_t srcSize); extern size_t ZSTD_decodeSeqHeaders(ZSTD_DCtx* dctx, int* nbSeq, const void* src, size_t srcSize);
size_t local_ZSTD_decodeSeqHeaders(void* dst, size_t dstSize, void* buff2, const void* src, size_t srcSize) size_t local_ZSTD_decodeSeqHeaders(void* dst, size_t dstSize, void* buff2, const void* src, size_t srcSize)
{ {