ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge branch 'dev' into adapt

Browse Source
This commit is contained in:
Yann Collet
2018-09-19 12:43:42 -07:00
parent c71c4f23d7 c8ff5200f9
commit 2f78228f65
78 changed files with 4962 additions and 1660 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/zstd_manual.html
+39 -17
View File
@@ -35,22 +35,34 @@
</ol>
<hr>
<a name="Chapter1"></a><h2>Introduction</h2><pre>
zstd, short for Zstandard, is a fast lossless compression algorithm,
targeting real-time compression scenarios at zlib-level and better compression ratios.
The zstd compression library provides in-memory compression and decompression functions.
The library supports compression levels from 1 up to ZSTD_maxCLevel() which is currently 22.
Levels >= 20, labeled `--ultra`, should be used with caution, as they require more memory.
zstd, short for Zstandard, is a fast lossless compression algorithm, targeting
real-time compression scenarios at zlib-level and better compression ratios.
The zstd compression library provides in-memory compression and decompression
functions.
The library supports regular compression levels from 1 up to ZSTD_maxCLevel(),
which is currently 22. Levels >= 20, labeled `--ultra`, should be used with
caution, as they require more memory. The library also offers negative
compression levels, which extend the range of speed vs. ratio preferences.
The lower the level, the faster the speed (at the cost of compression).
Compression can be done in:
- a single step (described as Simple API)
- a single step, reusing a context (described as Explicit context)
- unbounded multiple steps (described as Streaming compression)
The compression ratio achievable on small data can be highly improved using a dictionary in:
- a single step (described as Simple dictionary API)
- a single step, reusing a dictionary (described as Bulk-processing dictionary API)
Advanced experimental functions can be accessed using #define ZSTD_STATIC_LINKING_ONLY before including zstd.h.
Advanced experimental APIs shall never be used with a dynamic library.
They are not "stable", their definition may change in the future. Only static linking is allowed.
The compression ratio achievable on small data can be highly improved using
a dictionary. Dictionary compression can be performed in:
- a single step (described as Simple dictionary API)
- a single step, reusing a dictionary (described as Bulk-processing
dictionary API)
Advanced experimental functions can be accessed using
`#define ZSTD_STATIC_LINKING_ONLY` before including zstd.h.
Advanced experimental APIs should never be used with a dynamically-linked
library. They are not "stable"; their definitions or signatures may change in
the future. Only static linking is allowed.
<BR></pre>
<a name="Chapter2"></a><h2>Version</h2><pre></pre>
@@ -181,7 +193,8 @@ size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx);
</b><p> When compressing multiple messages / blocks with the same dictionary, it's recommended to load it just once.
ZSTD_createCDict() will create a digested dictionary, ready to start future compression operations without startup delay.
ZSTD_CDict can be created once and shared by multiple threads concurrently, since its usage is read-only.
`dictBuffer` can be released after ZSTD_CDict creation, since its content is copied within CDict
`dictBuffer` can be released after ZSTD_CDict creation, since its content is copied within CDict
Note : A ZSTD_CDict can be created with an empty dictionary, but it is inefficient for small data.
</p></pre><BR>
<pre><b>size_t ZSTD_freeCDict(ZSTD_CDict* CDict);
@@ -195,7 +208,9 @@ size_t ZSTD_freeDCtx(ZSTD_DCtx* dctx);
</b><p> Compression using a digested Dictionary.
Faster startup than ZSTD_compress_usingDict(), recommended when same dictionary is used multiple times.
Note that compression level is decided during dictionary creation.
Frame parameters are hardcoded (dictID=yes, contentSize=yes, checksum=no)
Frame parameters are hardcoded (dictID=yes, contentSize=yes, checksum=no)
Note : ZSTD_compress_usingCDict() can be used with a ZSTD_CDict created from an empty dictionary.
But it is inefficient for small data, and it is recommended to use ZSTD_compressCCtx().
</p></pre><BR>
<pre><b>ZSTD_DDict* ZSTD_createDDict(const void* dictBuffer, size_t dictSize);
@@ -965,16 +980,21 @@ size_t ZSTD_CCtx_refPrefix_advanced(ZSTD_CCtx* cctx,
const void* prefix, size_t prefixSize,
ZSTD_dictContentType_e dictContentType);
</b><p> Reference a prefix (single-usage dictionary) for next compression job.
Decompression need same prefix to properly regenerate data.
Prefix is **only used once**. Tables are discarded at end of compression job (ZSTD_e_end).
Decompression will need same prefix to properly regenerate data.
Compressing with a prefix is similar in outcome as performing a diff and compressing it,
but performs much faster, especially during decompression (compression speed is tunable with compression level).
Note that prefix is **only used once**. Tables are discarded at end of compression job (ZSTD_e_end).
@result : 0, or an error code (which can be tested with ZSTD_isError()).
Special: Adding any prefix (including NULL) invalidates any previous prefix or dictionary
Note 1 : Prefix buffer is referenced. It **must** outlive compression job.
Its contain must remain unmodified up to end of compression (ZSTD_e_end).
Note 2 : Referencing a prefix involves building tables, which are dependent on compression parameters.
Note 2 : If the intention is to diff some large src data blob with some prior version of itself,
ensure that the window size is large enough to contain the entire source.
See ZSTD_p_windowLog.
Note 3 : Referencing a prefix involves building tables, which are dependent on compression parameters.
It's a CPU consuming operation, with non-negligible impact on latency.
If there is a need to use same prefix multiple times, consider loadDictionary instead.
Note 3 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent).
Note 4 : By default, the prefix is treated as raw content (ZSTD_dm_rawContent).
Use ZSTD_CCtx_refPrefix_advanced() to alter dictMode.
</p></pre><BR>
@@ -1141,6 +1161,8 @@ size_t ZSTD_DCtx_refPrefix_advanced(ZSTD_DCtx* dctx,
const void* prefix, size_t prefixSize,
ZSTD_dictContentType_e dictContentType);
</b><p> Reference a prefix (single-usage dictionary) for next compression job.
This is the reverse operation of ZSTD_CCtx_refPrefix(),
and must use the same prefix as the one used during compression.
Prefix is **only used once**. Reference is discarded at end of frame.
End of frame is reached when ZSTD_DCtx_decompress_generic() returns 0.
@result : 0, or an error code (which can be tested with ZSTD_isError()).