Yann Collet
+82
-67
@@ -10,7 +10,7 @@
|
||||
<ol>
|
||||
<li><a href="#Chapter1">Introduction</a></li>
|
||||
<li><a href="#Chapter2">Version</a></li>
|
||||
<li><a href="#Chapter3">Simple API</a></li>
|
||||
<li><a href="#Chapter3">Simple Core API</a></li>
|
||||
<li><a href="#Chapter4">Explicit context</a></li>
|
||||
<li><a href="#Chapter5">Advanced compression API (Requires v1.4.0+)</a></li>
|
||||
<li><a href="#Chapter6">Advanced decompression API (Requires v1.4.0+)</a></li>
|
||||
@@ -74,7 +74,7 @@
|
||||
</b><p> Return runtime library version, like "1.4.5". Requires v1.3.0+.
|
||||
</p></pre><BR>
|
||||
|
||||
<a name="Chapter3"></a><h2>Simple API</h2><pre></pre>
|
||||
<a name="Chapter3"></a><h2>Simple Core API</h2><pre></pre>
|
||||
|
||||
<pre><b>size_t ZSTD_compress( void* dst, size_t dstCapacity,
|
||||
const void* src, size_t srcSize,
|
||||
@@ -88,38 +88,42 @@
|
||||
|
||||
<pre><b>size_t ZSTD_decompress( void* dst, size_t dstCapacity,
|
||||
const void* src, size_t compressedSize);
|
||||
</b><p> `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames.
|
||||
`dstCapacity` is an upper bound of originalSize to regenerate.
|
||||
If user cannot imply a maximum upper bound, it's better to use streaming mode to decompress data.
|
||||
@return : the number of bytes decompressed into `dst` (<= `dstCapacity`),
|
||||
or an errorCode if it fails (which can be tested using ZSTD_isError()).
|
||||
</b><p> `compressedSize` : must be the _exact_ size of some number of compressed and/or skippable frames.
|
||||
Multiple compressed frames can be decompressed at once with this method.
|
||||
The result will be the concatenation of all decompressed frames, back to back.
|
||||
`dstCapacity` is an upper bound of originalSize to regenerate.
|
||||
First frame's decompressed size can be extracted using ZSTD_getFrameContentSize().
|
||||
If maximum upper bound isn't known, prefer using streaming mode to decompress data.
|
||||
@return : the number of bytes decompressed into `dst` (<= `dstCapacity`),
|
||||
or an errorCode if it fails (which can be tested using ZSTD_isError()).
|
||||
</p></pre><BR>
|
||||
|
||||
<h3>Decompression helper functions</h3><pre></pre><b><pre></pre></b><BR>
|
||||
<pre><b>#define ZSTD_CONTENTSIZE_UNKNOWN (0ULL - 1)
|
||||
#define ZSTD_CONTENTSIZE_ERROR (0ULL - 2)
|
||||
unsigned long long ZSTD_getFrameContentSize(const void *src, size_t srcSize);
|
||||
</b><p> `src` should point to the start of a ZSTD encoded frame.
|
||||
`srcSize` must be at least as large as the frame header.
|
||||
hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
|
||||
@return : - decompressed size of `src` frame content, if known
|
||||
- ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
|
||||
- ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
|
||||
note 1 : a 0 return value means the frame is valid but "empty".
|
||||
note 2 : decompressed size is an optional field, it may not be present, typically in streaming mode.
|
||||
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
|
||||
In which case, it's necessary to use streaming mode to decompress data.
|
||||
Optionally, application can rely on some implicit limit,
|
||||
as ZSTD_decompress() only needs an upper bound of decompressed size.
|
||||
(For example, data could be necessarily cut into blocks <= 16 KB).
|
||||
note 3 : decompressed size is always present when compression is completed using single-pass functions,
|
||||
such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict().
|
||||
note 4 : decompressed size can be very large (64-bits value),
|
||||
potentially larger than what local system can handle as a single memory segment.
|
||||
In which case, it's necessary to use streaming mode to decompress data.
|
||||
note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
|
||||
Always ensure return value fits within application's authorized limits.
|
||||
Each application can set its own limits.
|
||||
note 6 : This function replaces ZSTD_getDecompressedSize()
|
||||
</b><p> `src` should point to the start of a ZSTD encoded frame.
|
||||
`srcSize` must be at least as large as the frame header.
|
||||
hint : any size >= `ZSTD_frameHeaderSize_max` is large enough.
|
||||
@return : - decompressed size of `src` frame content, if known
|
||||
- ZSTD_CONTENTSIZE_UNKNOWN if the size cannot be determined
|
||||
- ZSTD_CONTENTSIZE_ERROR if an error occurred (e.g. invalid magic number, srcSize too small)
|
||||
note 1 : a 0 return value means the frame is valid but "empty".
|
||||
note 2 : decompressed size is an optional field, it may not be present (typically in streaming mode).
|
||||
When `return==ZSTD_CONTENTSIZE_UNKNOWN`, data to decompress could be any size.
|
||||
In which case, it's necessary to use streaming mode to decompress data.
|
||||
Optionally, application can rely on some implicit limit,
|
||||
as ZSTD_decompress() only needs an upper bound of decompressed size.
|
||||
(For example, data could be necessarily cut into blocks <= 16 KB).
|
||||
note 3 : decompressed size is always present when compression is completed using single-pass functions,
|
||||
such as ZSTD_compress(), ZSTD_compressCCtx() ZSTD_compress_usingDict() or ZSTD_compress_usingCDict().
|
||||
note 4 : decompressed size can be very large (64-bits value),
|
||||
potentially larger than what local system can handle as a single memory segment.
|
||||
In which case, it's necessary to use streaming mode to decompress data.
|
||||
note 5 : If source is untrusted, decompressed size could be wrong or intentionally modified.
|
||||
Always ensure return value fits within application's authorized limits.
|
||||
Each application can set its own limits.
|
||||
note 6 : This function replaces ZSTD_getDecompressedSize()
|
||||
</p></pre><BR>
|
||||
|
||||
<pre><b>ZSTD_DEPRECATED("Replaced by ZSTD_getFrameContentSize")
|
||||
@@ -140,35 +144,21 @@ unsigned long long ZSTD_getDecompressedSize(const void* src, size_t srcSize);
|
||||
or an error code if input is invalid
|
||||
</p></pre><BR>
|
||||
|
||||
<h3>Helper functions</h3><pre></pre><b><pre></b>/* ZSTD_compressBound() :<b>
|
||||
* maximum compressed size in worst case single-pass scenario.
|
||||
* When invoking `ZSTD_compress()` or any other one-pass compression function,
|
||||
* it's recommended to provide @dstCapacity >= ZSTD_compressBound(srcSize)
|
||||
* as it eliminates one potential failure scenario,
|
||||
* aka not enough room in dst buffer to write the compressed frame.
|
||||
* Note : ZSTD_compressBound() itself can fail, if @srcSize > ZSTD_MAX_INPUT_SIZE .
|
||||
* In which case, ZSTD_compressBound() will return an error code
|
||||
* which can be tested using ZSTD_isError().
|
||||
*
|
||||
* ZSTD_COMPRESSBOUND() :
|
||||
* same as ZSTD_compressBound(), but as a macro.
|
||||
* It can be used to produce constants, which can be useful for static allocation,
|
||||
* for example to size a static array on stack.
|
||||
* Will produce constant value 0 if srcSize too large.
|
||||
*/
|
||||
#define ZSTD_MAX_INPUT_SIZE ((sizeof(size_t)==8) ? 0xFF00FF00FF00FF00ULL : 0xFF00FF00U)
|
||||
#define ZSTD_COMPRESSBOUND(srcSize) (((size_t)(srcSize) >= ZSTD_MAX_INPUT_SIZE) ? 0 : (srcSize) + ((srcSize)>>8) + (((srcSize) < (128<<10)) ? (((128<<10) - (srcSize)) >> 11) </b>/* margin, from 64 to 0 */ : 0)) /* this formula ensures that bound(A) + bound(B) <= bound(A+B) as long as A and B >= 128 KB */<b>
|
||||
size_t ZSTD_compressBound(size_t srcSize); </b>/*!< maximum compressed size in worst case single-pass scenario */<b>
|
||||
<h3>Compression helper functions</h3><pre></pre><b><pre></pre></b><BR>
|
||||
<pre><b>size_t ZSTD_compressBound(size_t srcSize); </b>/*!< maximum compressed size in worst case single-pass scenario */<b>
|
||||
</b></pre><BR>
|
||||
<h3>Error helper functions</h3><pre></pre><b><pre>#include "zstd_errors.h" </b>/* list of errors */<b>
|
||||
</b>/* ZSTD_isError() :<b>
|
||||
* Most ZSTD_* functions returning a size_t value can be tested for error,
|
||||
* using ZSTD_isError().
|
||||
* @return 1 if error, 0 otherwise
|
||||
*/
|
||||
unsigned ZSTD_isError(size_t code); </b>/*!< tells if a `size_t` function result is an error code */<b>
|
||||
const char* ZSTD_getErrorName(size_t code); </b>/*!< provides readable string from an error code */<b>
|
||||
int ZSTD_minCLevel(void); </b>/*!< minimum negative compression level allowed, requires v1.4.0+ */<b>
|
||||
int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b>
|
||||
int ZSTD_defaultCLevel(void); </b>/*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */<b>
|
||||
unsigned ZSTD_isError(size_t code); </b>/*!< tells if a `size_t` function result is an error code */<b>
|
||||
ZSTD_ErrorCode ZSTD_getErrorCode(size_t functionResult); </b>/* convert a result into an error code, which can be compared to error enum list */<b>
|
||||
const char* ZSTD_getErrorName(size_t code); </b>/*!< provides readable string from an error code */<b>
|
||||
int ZSTD_minCLevel(void); </b>/*!< minimum negative compression level allowed, requires v1.4.0+ */<b>
|
||||
int ZSTD_maxCLevel(void); </b>/*!< maximum compression level available */<b>
|
||||
int ZSTD_defaultCLevel(void); </b>/*!< default compression level, specified by ZSTD_CLEVEL_DEFAULT, requires v1.5.0+ */<b>
|
||||
</pre></b><BR>
|
||||
<a name="Chapter4"></a><h2>Explicit context</h2><pre></pre>
|
||||
|
||||
@@ -718,7 +708,7 @@ size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output);
|
||||
<a name="Chapter9"></a><h2>Streaming decompression - HowTo</h2><pre>
|
||||
A ZSTD_DStream object is required to track streaming operations.
|
||||
Use ZSTD_createDStream() and ZSTD_freeDStream() to create/release resources.
|
||||
ZSTD_DStream objects can be reused multiple times.
|
||||
ZSTD_DStream objects can be re-employed multiple times.
|
||||
|
||||
Use ZSTD_initDStream() to start a new decompression operation.
|
||||
@return : recommended first input size
|
||||
@@ -728,16 +718,21 @@ size_t ZSTD_endStream(ZSTD_CStream* zcs, ZSTD_outBuffer* output);
|
||||
The function will update both `pos` fields.
|
||||
If `input.pos < input.size`, some input has not been consumed.
|
||||
It's up to the caller to present again remaining data.
|
||||
|
||||
The function tries to flush all data decoded immediately, respecting output buffer size.
|
||||
If `output.pos < output.size`, decoder has flushed everything it could.
|
||||
But if `output.pos == output.size`, there might be some data left within internal buffers.,
|
||||
|
||||
However, when `output.pos == output.size`, it's more difficult to know.
|
||||
If @return > 0, the frame is not complete, meaning
|
||||
either there is still some data left to flush within internal buffers,
|
||||
or there is more input to read to complete the frame (or both).
|
||||
In which case, call ZSTD_decompressStream() again to flush whatever remains in the buffer.
|
||||
Note : with no additional input provided, amount of data flushed is necessarily <= ZSTD_BLOCKSIZE_MAX.
|
||||
@return : 0 when a frame is completely decoded and fully flushed,
|
||||
or an error code, which can be tested using ZSTD_isError(),
|
||||
or any other value > 0, which means there is still some decoding or flushing to do to complete current frame :
|
||||
the return value is a suggested next input size (just a hint for better latency)
|
||||
that will never request more than the remaining frame size.
|
||||
that will never request more than the remaining content of the compressed frame.
|
||||
|
||||
<BR></pre>
|
||||
|
||||
@@ -763,9 +758,10 @@ size_t ZSTD_freeDStream(ZSTD_DStream* zds); </b>/* accept NULL pointer */<b>
|
||||
Function will update both input and output `pos` fields exposing current state via these fields:
|
||||
- `input.pos < input.size`, some input remaining and caller should provide remaining input
|
||||
on the next call.
|
||||
- `output.pos < output.size`, decoder finished and flushed all remaining buffers.
|
||||
- `output.pos == output.size`, potentially uncflushed data present in the internal buffers,
|
||||
call ZSTD_decompressStream() again to flush remaining data to output.
|
||||
- `output.pos < output.size`, decoder flushed internal output buffer.
|
||||
- `output.pos == output.size`, unflushed data potentially present in the internal buffers,
|
||||
check ZSTD_decompressStream() @return value,
|
||||
if > 0, invoke it again to flush remaining data to output.
|
||||
Note : with no additional input, amount of data flushed <= ZSTD_BLOCKSIZE_MAX.
|
||||
|
||||
@return : 0 when a frame is completely decoded and fully flushed,
|
||||
@@ -1311,19 +1307,37 @@ ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr,
|
||||
|
||||
</p></pre><BR>
|
||||
|
||||
<pre><b></b><p> Generate sequences using ZSTD_compress2(), given a source buffer.
|
||||
<pre><b>ZSTD_DEPRECATED("For debugging only, will be replaced by ZSTD_extractSequences()")
|
||||
ZSTDLIB_STATIC_API size_t
|
||||
ZSTD_generateSequences(ZSTD_CCtx* zc,
|
||||
ZSTD_Sequence* outSeqs, size_t outSeqsSize,
|
||||
const void* src, size_t srcSize);
|
||||
</b><p> WARNING: This function is meant for debugging and informational purposes ONLY!
|
||||
Its implementation is flawed, and it will be deleted in a future version.
|
||||
It is not guaranteed to succeed, as there are several cases where it will give
|
||||
up and fail. You should NOT use this function in production code.
|
||||
|
||||
This function is deprecated, and will be removed in a future version.
|
||||
|
||||
Generate sequences using ZSTD_compress2(), given a source buffer.
|
||||
|
||||
@param zc The compression context to be used for ZSTD_compress2(). Set any
|
||||
compression parameters you need on this context.
|
||||
@param outSeqs The output sequences buffer of size @p outSeqsSize
|
||||
@param outSeqsSize The size of the output sequences buffer.
|
||||
ZSTD_sequenceBound(srcSize) is an upper bound on the number
|
||||
of sequences that can be generated.
|
||||
@param src The source buffer to generate sequences from of size @p srcSize.
|
||||
@param srcSize The size of the source buffer.
|
||||
|
||||
Each block will end with a dummy sequence
|
||||
with offset == 0, matchLength == 0, and litLength == length of last literals.
|
||||
litLength may be == 0, and if so, then the sequence of (of: 0 ml: 0 ll: 0)
|
||||
simply acts as a block delimiter.
|
||||
|
||||
@zc can be used to insert custom compression params.
|
||||
This function invokes ZSTD_compress2().
|
||||
|
||||
The output of this function can be fed into ZSTD_compressSequences() with CCtx
|
||||
setting of ZSTD_c_blockDelimiters as ZSTD_sf_explicitBlockDelimiters
|
||||
@return : number of sequences generated
|
||||
@returns The number of sequences generated, necessarily less than
|
||||
ZSTD_sequenceBound(srcSize), or an error code that can be checked
|
||||
with ZSTD_isError().
|
||||
|
||||
</p></pre><BR>
|
||||
|
||||
@@ -1512,13 +1526,14 @@ static
|
||||
#ifdef __GNUC__
|
||||
__attribute__((__unused__))
|
||||
#endif
|
||||
ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; </b>/**< this constant defers to stdlib's functions */<b>
|
||||
</b><p> These prototypes make it possible to pass your own allocation/free functions.
|
||||
ZSTD_customMem is provided at creation time, using ZSTD_create*_advanced() variants listed below.
|
||||
All allocation/free operations will be completed using these custom variants instead of regular <stdlib.h> ones.
|
||||
|
||||
</p></pre><BR>
|
||||
|
||||
<pre><b>ZSTD_customMem const ZSTD_defaultCMem = { NULL, NULL, NULL }; </b>/**< this constant defers to stdlib's functions */<b>
|
||||
</b></pre><BR>
|
||||
<pre><b>typedef struct POOL_ctx_s ZSTD_threadPool;
|
||||
ZSTDLIB_STATIC_API ZSTD_threadPool* ZSTD_createThreadPool(size_t numThreads);
|
||||
ZSTDLIB_STATIC_API void ZSTD_freeThreadPool (ZSTD_threadPool* pool); </b>/* accept NULL pointer */<b>
|
||||
|
||||
Reference in New Issue
Block a user