ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Bugfixes for the External Matchfinder API (#3433)

Browse Source 
* external matchfinder bugfixes + tests

* small doc fix
This commit is contained in:
Elliot Gorokhovsky
2023-01-19 10:41:24 -05:00
committed by GitHub
parent dc1c6cc5df
commit bce0382c82
3 changed files with 88 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
lib/zstd.h
+18 -3
View File
@@ -2840,8 +2840,8 @@ ZSTDLIB_STATIC_API size_t ZSTD_insertBlock (ZSTD_DCtx* dctx, const void* bloc
* externalMatchState.
*
* *** LIMITATIONS ***
* External matchfinders are compatible with all zstd compression APIs. There are
* only two limitations.
* External matchfinders are compatible with all zstd compression APIs which respect
* advanced parameters. However, there are three limitations:
*
* First, the ZSTD_c_enableLongDistanceMatching cParam is not supported.
* COMPRESSION WILL FAIL if it is enabled and the user tries to compress with an
@@ -2863,7 +2863,11 @@ ZSTDLIB_STATIC_API size_t ZSTD_insertBlock (ZSTD_DCtx* dctx, const void* bloc
* APIs, work with the external matchfinder, but the external matchfinder won't
* receive any history from the previous block. Each block is an independent chunk.
*
* Long-term, we plan to overcome both limitations. There is no technical blocker to
* Third, multi-threading within a single compression is not supported. In other words,
* COMPRESSION WILL FAIL if ZSTD_c_nbWorkers > 0 and an external matchfinder is registered.
* Multi-threading across compressions is fine: simply create one CCtx per thread.
*
* Long-term, we plan to overcome all three limitations. There is no technical blocker to
* overcoming them. It is purely a question of engineering effort.
*/
@@ -2886,6 +2890,17 @@ typedef size_t ZSTD_externalMatchFinder_F (
* compressions. It will remain set until the user explicitly resets compression
* parameters.
*
* External matchfinder registration is considered to be an "advanced parameter",
* part of the "advanced API". This means it will only have an effect on
* compression APIs which respect advanced parameters, such as compress2() and
* compressStream(). Older compression APIs such as compressCCtx(), which predate
* the introduction of "advanced parameters", will ignore any external matchfinder
* setting.
*
* The external matchfinder can be "cleared" by registering a NULL external
* matchfinder function pointer. This removes all limitations described above in
* the "LIMITATIONS" section of the API docs.
*
* The user is strongly encouraged to read the full API documentation (above)
* before calling this function. */
ZSTDLIB_STATIC_API void