improved speed of the Sequences converter

doc/zstd_manual.html

@@ -1084,7 +1084,7 @@ size_t ZSTD_sizeof_DDict(const ZSTD_DDict* ddict);
                                *
                                * Note: This field is optional. ZSTD_generateSequences() will calculate the value of
                                * 'rep', but repeat offsets do not necessarily need to be calculated from an external
-                               * sequence provider's perspective. For example, ZSTD_compressSequences() does not
+                               * sequence provider perspective. For example, ZSTD_compressSequences() does not
                                * use this 'rep' field at all (as of now).
                                */
 } ZSTD_Sequence;
@@ -1331,7 +1331,7 @@ ZSTDLIB_STATIC_API size_t ZSTD_getFrameHeader_advanced(ZSTD_frameHeader* zfhPtr,
 <pre><b>ZSTD_DEPRECATED("For debugging only, will be replaced by ZSTD_extractSequences()")
 ZSTDLIB_STATIC_API size_t
 ZSTD_generateSequences(ZSTD_CCtx* zc,
-           ZSTD_Sequence* outSeqs, size_t outSeqsSize,
+           ZSTD_Sequence* outSeqs, size_t outSeqsCapacity,
            const void* src, size_t srcSize);
 </b><p> WARNING: This function is meant for debugging and informational purposes ONLY!
  Its implementation is flawed, and it will be deleted in a future version.
@@ -1345,7 +1345,7 @@ ZSTD_generateSequences(ZSTD_CCtx* zc,
  @param zc The compression context to be used for ZSTD_compress2(). Set any
            compression parameters you need on this context.
  @param outSeqs The output sequences buffer of size @p outSeqsSize
- @param outSeqsSize The size of the output sequences buffer.
+ @param outSeqsCapacity The size of the output sequences buffer.
                     ZSTD_sequenceBound(srcSize) is an upper bound on the number
                     of sequences that can be generated.
  @param src The source buffer to generate sequences from of size @p srcSize.
@@ -1392,11 +1392,17 @@ ZSTD_compressSequences(ZSTD_CCtx* cctx,
     the block size derived from the cctx, and sequences may be split. This is the default setting.
 
     If ZSTD_c_blockDelimiters == ZSTD_sf_explicitBlockDelimiters, the array of ZSTD_Sequence is expected to contain
-    block delimiters (defined in ZSTD_Sequence). Behavior is undefined if no block delimiters are provided.
+    valid block delimiters (defined in ZSTD_Sequence). Behavior is undefined if no block delimiters are provided.
 
-    If ZSTD_c_validateSequences == 0, this function will blindly accept the sequences provided. Invalid sequences cause undefined
-    behavior. If ZSTD_c_validateSequences == 1, then if sequence is invalid (see doc/zstd_compression_format.md for
-    specifics regarding offset/matchlength requirements) then the function will bail out and return an error.
+    When ZSTD_c_blockDelimiters == ZSTD_sf_explicitBlockDelimiters, it's possible to decide generating repcodes
+    using the advanced parameter ZSTD_c_repcodeResolution. Repcodes will improve compression ratio, though the benefit
+    can vary greatly depending on Sequences. On the other hand, repcode resolution is an expensive operation.
+    By default, it's disabled at low (<10) compression levels, and enabled above the threshold (>=10).
+    ZSTD_c_repcodeResolution makes it possible to directly manage this processing in either direction.
+
+    If ZSTD_c_validateSequences == 0, this function blindly accepts the Sequences provided. Invalid Sequences cause undefined
+    behavior. If ZSTD_c_validateSequences == 1, then the function will detect invalid Sequences (see doc/zstd_compression_format.md for
+    specifics regarding offset/matchlength requirements) and then bail out and return an error.
 
     In addition to the two adjustable experimental params, there are other important cctx params.
     - ZSTD_c_minMatch MUST be set as less than or equal to the smallest match generated by the match finder. It has a minimum value of ZSTD_MINMATCH_MIN.
@@ -1414,19 +1420,21 @@ ZSTD_compressSequences(ZSTD_CCtx* cctx,
 <pre><b>ZSTDLIB_STATIC_API size_t
 ZSTD_compressSequencesAndLiterals(ZSTD_CCtx* cctx,
                       void* dst, size_t dstCapacity,
-                const ZSTD_Sequence* inSeqs, size_t inSeqsSize,
-                const void* literals, size_t litSize, size_t srcSize);
+                const ZSTD_Sequence* inSeqs, size_t nbSequences,
+                const void* literals, size_t litSize);
 </b><p> This is a variant of ZSTD_compressSequences() which,
  instead of receiving (src,srcSize) as input parameter, receives (literals,litSize),
- aka all literals already extracted and laid out into a single continuous buffer.
+ aka all the literals, already extracted and laid out into a single continuous buffer.
  This can be useful if the process generating the sequences also happens to generate the buffer of literals,
  thus skipping an extraction + caching stage.
- It's essentially a speed optimization when the right conditions are met,
- but it also is restricted by the following limitations:
+ It's a speed optimization, useful when the right conditions are met,
+ but it also features the following limitations:
  - Only supports explicit delimiter mode
  - Not compatible with frame checksum, which must disabled
- - Can fail when unable to compress sufficiently
- Also, to be valid, @litSize must be equal to the sum of all @.litLength fields in @inSeqs.
+ - Does not write the content size in frame header
+ - If any block is incompressible, will fail and return an error
+ - @litSize must be == sum of all @.litLength fields in @inSeqs. Any discrepancy will generate an error.
+ - the buffer @literals must be larger than @litSize by at least 8 bytes.
  @return : final compressed size, or a ZSTD error code.
  
 </p></pre><BR>