Expose function to add entropy tables to pre-built dictionary.
In some cases a custom dictionary building algorithm tailored for a specific input can be more effective than the one produced by `ZDICT_trainFromBuffer`, but with the current API it's not possible encode the entropy tables into the custom-built dictionary. This commit extracts the logic to add entropy tables to a dictionary from `ZDICT_trainFromBuffer` and exposes it as a function `ZDICT_addEntropyTablesFromBuffer`.
This commit is contained in:
@@ -52,6 +52,21 @@ extern "C" {
|
||||
size_t ZDICT_trainFromBuffer(void* dictBuffer, size_t dictBufferCapacity,
|
||||
const void* samplesBuffer, const size_t* samplesSizes, unsigned nbSamples);
|
||||
|
||||
/*! ZDICT_addEntropyTablesFromBuffer() :
|
||||
|
||||
Given a content-only dictionary (built for example from common strings in
|
||||
the input), add entropy tables computed from the memory buffer
|
||||
`samplesBuffer`, where `nbSamples` samples have been stored concatenated.
|
||||
Each sample size is provided into an orderly table `samplesSizes`.
|
||||
|
||||
The input dictionary is the last `dictContentSize` bytes of `dictBuffer`. The
|
||||
resulting dictionary with added entropy tables will written back to
|
||||
`dictBuffer`.
|
||||
@return : size of dictionary stored into `dictBuffer` (<= `dictBufferCapacity`).
|
||||
*/
|
||||
size_t ZDICT_addEntropyTablesFromBuffer(void* dictBuffer, size_t dictContentSize, size_t dictBufferCapacity,
|
||||
const void* samplesBuffer, const size_t* samplesSizes, unsigned nbSamples);
|
||||
|
||||
|
||||
/*-*************************************
|
||||
* Helper functions
|
||||
|
||||
Reference in New Issue
Block a user