added ability to split input files for dictionary training

using command -B#
This is the same behavior as benchmark module,
which can also split input into arbitrary size blocks, using -B#.
This commit is contained in:
Yann Collet
2017-09-14 16:45:10 -07:00
parent 77c137b3ae
commit 086b9597d9
5 changed files with 116 additions and 62 deletions
+3 -1
View File
@@ -184,6 +184,8 @@ Typical gains range from 10% (at 64KB) to x5 better (at <1KB).
Dictionary saved into `file` (default name: dictionary).
* `--maxdict=#`:
Limit dictionary to specified size (default: 112640).
* `-B#`:
Split input files in blocks of size # (default: no split)
* `--dictID=#`:
A dictionary ID is a locally unique ID that a decoder can use to verify it is
using the right dictionary.
@@ -373,7 +375,7 @@ The list of available _options_:
default value will likely result in a decrease in compression ratio.
The default value is `wlog - ldmhlog`.
### -B#:
Select the size of each compression job.
This parameter is available only when multi-threading is enabled.