ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
10,883 Commits 17 Branches 77 Tags
f281497aef87a0e6459eef28f695c437a3dec42d
Commit Graph
10883 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandYann Collet f281497aef fullbench: new scenario: compressSequencesAndLiterals() 2024-12-20 10:36:59 -08:00
Yann ColletandYann Collet ac05ea89a5 fullbench: switch default generator to lorem ipsum 
which creates more "realistic" scenarios than former compressible noise.

The legacy data generator remains accessible,
it is triggered when requesting an explicit compressibility factor (-P#).
 2024-12-20 10:36:59 -08:00
Yann Collet 8ab04097ed add the compressSequences() benchmark scenario 2024-12-20 10:36:59 -08:00
Yann Collet c050ae4fb8 fullbench: preparation functions are now in charge of allocating buffers 2024-12-20 10:36:59 -08:00
Yann Collet 8b7e1b795d minor variable renaming 2024-12-20 10:36:59 -08:00
Yann Collet c540976a4b minor: test reordering 2024-12-20 10:36:59 -08:00
Yann Collet 4c097b4939 refactor fullbench 
to make it easier to add new scenarios
 2024-12-20 10:36:58 -08:00
Yann Collet 09964c6276 prepare fullbench for a refactor 2024-12-20 10:36:58 -08:00
Yann Collet 0b013b2688 added unit tests to ZSTD_compressSequencesAndLiterals() 
seems to work as expected,
correctly control that `litSize` and `srcSize` are exactly correct.
 2024-12-20 10:36:58 -08:00
Yann Collet 14a21e43b3 produced ZSTD_compressSequencesAndLiterals() as a separate pipeline 
only supports explicit delimiter mode, at least for the time being
 2024-12-20 10:36:58 -08:00
Yann Collet bcb15091aa minor: more accurate variable scope 2024-12-20 10:36:58 -08:00
Yann Collet 047db4f1f8 ZSTD_SequenceCopier_f no returns the nb of bytes consumed from input 
which feels much more natural
 2024-12-20 10:36:58 -08:00
Yann Collet 4ef9d7d585 codemod: ZSTD_cParamMode_e -> ZSTD_CParamMode_e 2024-12-20 10:36:58 -08:00
Yann Collet 56cfb7816a codemod: ZSTD_paramSwitch_e -> ZSTD_ParamSwitch_e 2024-12-20 10:36:58 -08:00
Yann Collet 13b9296d79 minor simplification 2024-12-20 10:36:58 -08:00
Yann Collet 2503b64345 fix minor artifact error in single_file_lib 2024-12-20 10:36:58 -08:00
Yann Collet e0f3aaee46 doc: add mention of frame checksum incompatibility 
for new prototype ZSTD_compressSequencesAndLiterals()
 2024-12-20 10:36:57 -08:00
Yann Collet 08edecb78c codemod: ZSTD_blockCompressor -> ZSTD_BlockCompressor_f 2024-12-20 10:36:57 -08:00
Yann Collet 25bef24c5c codemod: rawSeqStore_t -> RawSeqStore_t 2024-12-20 10:36:57 -08:00
Yann Collet 41c667c0fd codemod: repcodes_t -> Repcodes_t 2024-12-20 10:36:57 -08:00
Yann Collet 5df80acedb codemod: ZSTD_matchState_t -> ZSTD_MatchState_t 2024-12-20 10:36:57 -08:00
Yann Collet fa468944f2 codemod: ZSTD_buildSeqStore_e -> ZSTD_BuildSeqStore_e 2024-12-20 10:36:57 -08:00
Yann Collet 30671d77af codemod: ZSTD_sequencePosition -> ZSTD_SequencePosition 2024-12-20 10:36:57 -08:00
Yann Collet 5359d16d8d enable proper type 2024-12-20 10:36:57 -08:00
Yann Collet 03d95f9d13 fix proper type for .forceNonContiguous 2024-12-20 10:36:57 -08:00
Yann Collet 76dd3a98c4 scope: ZSTD_copySequencesToSeqStore*() are private to ZSTD_compress.c 
no need to publish them outside of this unit.
 2024-12-20 10:36:57 -08:00
Yann Collet 1ac79ba1b6 minor: simplify ZSTD_selectSequenceCopier 2024-12-20 10:36:56 -08:00
Yann Collet 894ea31281 codemod: ZSTD_sequenceCopier -> ZSTD_SequenceCopier_f 2024-12-20 10:36:56 -08:00
Yann Collet c97522f7fb codemod: ZSTD_sequenceFormat_e -> ZSTD_SequenceFormat_e 
since it's a type name.

Note: in contrast with previous names, this one is on the Public API side.
So there is a #define, so that existing programs using ZSTD_sequenceFormat_e still work.
 2024-12-20 10:36:56 -08:00
Yann Collet 0165eeb441 created ZSTD_entropyCompressSeqStore_wExtLitBuffer() 
can receive externally defined buffer of literals
 2024-12-20 10:36:56 -08:00
Yann Collet e9f8a119b4 ZSTD_entropyCompressSeqStore_internal() can accept an externally defined literals buffer 2024-12-20 10:36:56 -08:00
Yann Collet 0442e43aca codemod: ZSTD_defaultPolicy_e -> ZSTD_DefaultPolicy_e 2024-12-20 10:36:56 -08:00
Yann Collet 477a01067f codemod: symbolEncodingType_e -> SymbolEncodingType_e 2024-12-20 10:36:56 -08:00
Yann Collet 8d4506bc94 codemod: ZSTD_sequenceLength -> ZSTD_SequenceLength 2024-12-20 10:36:55 -08:00
Yann Collet a2245721ca codemod: seqStore_t -> SeqStore_t 
same idea, SeqStore_t is a type name, it should start with a Capital letter.
 2024-12-20 10:36:55 -08:00
Yann Collet 9671813375 codemod: seqDef -> SeqDef 
SeqDef is a type name, so it should start with a Capital letter.
It's an internal symbol, no impact on public API.
 2024-12-20 10:36:55 -08:00
Yann Collet b4a40a845f move Sequences definition to zstd_compress_internal.h 
they should not be in common/zstd_internal.h,
since these definitions are not shared beyond lib/compress/.
 2024-12-20 10:36:55 -08:00
Yann Collet a00f45a037 created ZSTD_storeSeqOnly() 
makes it possible to register a sequence without copying its literals.
 2024-12-20 10:36:04 -08:00
Yann Collet 125f05282b publish new symbol ZSTD_compressSequencesAndLiterals() 2024-12-20 10:36:04 -08:00
BentandNick Terrell 5a7f5c745c removed debug echos 2024-12-20 07:33:53 -08:00
BentandNick Terrell d06e8778bc added UNAME_TARGET_SYSTEM build flag to configure shared lib flags 2024-12-20 07:33:53 -08:00
Yann ColletandGitHub 0218c8de0f Merge pull request #4212 from diegonc/t/fix-test-on-gnuhurd 
tests: fix non-regular file test on GNU/Hurd
 2024-12-16 20:43:34 -08:00
Yann ColletandGitHub 2d8710c447 Merge pull request #4213 from kevinjzhang/CI 
Move sanitizer CI jobs back to ubuntu-latest
 2024-12-16 20:43:13 -08:00
Yann ColletandGitHub 5a1c5014cb Merge pull request #4214 from facebook/ldm_bucketLog 
minor: more accurate parameter for `ZSTD_ldm_insertEntry()`
 2024-12-16 20:43:00 -08:00
Yann Collet 50ca9984ad minor: more accurate parameter 
just pass ldm_bucketLog, instead of the entire ldm* state
 2024-12-14 11:57:14 -08:00
Victor ZhangandGitHub f7a8bb1263 Merge pull request #4210 from facebook/cm310 
Update cmake minimum requirement to 3.10
 2024-12-13 14:57:52 -08:00
Elliot GorokhovskyandKevin Zhang 7236e05b0a Revert "Pin tsan and msan CI jobs to ubuntu-20.04 (#3945)" 
This reverts commit ee6acaf26b.
 2024-12-13 10:44:54 -08:00
Victor Zhang 1198a582d3 Update VERSION_LESS usage to VERSION_GREATER_THAN 2024-12-12 11:39:40 -08:00
daniellerozenblitandGitHub 17beeb5d1a Change CLI to employ multithreading by default (#4211) 
* Change CLI to employ multithreading by default

* Document changes to benchmarking, print number of threads for display level >= 4, and add lower bound of 1 for the default number of threads
 2024-12-12 13:09:29 -05:00
Diego Nieto Cid 4a4786bef0 tests: fix non-regular file test on GNU/Hurd 
Since commit b21b03ca6 [1] the behaviour of writes to /dev/zero has been
fixed and now the non-regular file removal test no longer needs to be
done on /dev/random (which no longer works as random is not writable now).

[1] https://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=b21b03ca624b89caeedfe58430cea4b40317d39f
 2024-12-10 10:30:52 -03:00