 senhuang42
|
46824cb018
|
Add new sequence compress api params to cctx
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
48405b4633
|
Fix srcSize=0 edge case
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
022e6d81e7
|
Fix literals length calculation
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
3d26615c84
|
Adjust unit tests to agree with new sequence generation API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dad20b5ccb
|
Remove dstCapacity error check
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
b8e16a2057
|
Remove extraneous function in this API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
f29507c4fc
|
Add check comparing offset to window size
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
7a6e46a92f
|
Fix MSAN errors
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
cc2642bd17
|
Address edge case with endPosInSequence
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
fd10007174
|
Change debug levels to appropriate ones
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2db8441245
|
Add RLE support
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dfef298336
|
Fix various build warnings
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2bbdddf24e
|
Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
5fd69f8173
|
Add documentation for new api functions
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e8b7fdb64b
|
Refactor for enhanced code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c675fb46f1
|
Rename internal function compressSequences(), and promote new *_ext() functions to their actual name
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
013434e1e4
|
Add another API function to compress with existing CCTX
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c44ce29013
|
More adjustments to improve code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
48f67da854
|
Pull compressStream2() transparent initialization into its own function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c86151f53c
|
Add initial support for new ZSTD_Sequence mode
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e0f26afce9
|
Add sequence compression format param
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
f51af9a609
|
Always ensure sequenceRange updates properly, add more error forwarding
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
1a449688fd
|
Various minor logical refactors to improve clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e5fe485dcc
|
Fix cSize calculation for noCompressBlocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
6145ebb400
|
Rebased, roundtrips silesia.tar
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b5b61cc216
|
Refactor for better debugging info
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
293fad6b45
|
Corrections and edge-case fixes to be able to roundtrip dickens
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
7eb6fa7be4
|
Multi-block compression scaffolding - works on single-block files
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
75b01f34b9
|
Add support for uncompressible blocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e04da68157
|
Enable usage of ZSTD_sequenceRange for single-block compression
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
337fac216d
|
Add logic to handle ZSTD_sequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
85822ddd53
|
Add last literals handling like getSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
2cff8df1a2
|
Pull block compression out of main compressSequences() function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
cfced9344a
|
Implement ZSTD_updateSequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b116e1f211
|
Modify SequenceRange to have posInSequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
d99b675112
|
Add function definition for sequenceRange updater
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
74e95c05cc
|
Add ZSTD_SequenceRange to count ranges in array of ZSTD_Sequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
89f3848310
|
Add support for repcodes
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
3e930fd044
|
Code cleanup, add debuglog statments
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
086513b5b9
|
Implement first pass at compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
a9327b1e9b
|
Add initial function prototype for ZSTD_compressSequences_ext (to be renamed later)
|
2020-11-16 10:33:35 -05:00 |
|
|
sen
|
9736c497ce
|
Merge pull request #2393 from senhuang42/fix_sequence_extractions_api
Improve repcode handling in sequence extraction API
|
2020-11-15 18:29:52 -05:00 |
|
|
senhuang42
|
9d936d61d2
|
Reduce number of memcpy() calls
|
2020-11-13 19:43:30 -05:00 |
|
|
senhuang42
|
be4ac6c5bc
|
Use existing repcode update function to implement updates
|
2020-11-12 16:51:12 -05:00 |
|
|
senhuang42
|
674c9b9235
|
Add in proper block repcode histories
|
2020-11-12 15:34:37 -05:00 |
|
|
senhuang42
|
06c7f14066
|
Let block reps persist
|
2020-11-12 12:24:44 -05:00 |
|
|
senhuang42
|
396275068c
|
Fix incorrect repcode setting
|
2020-11-12 11:57:01 -05:00 |
|
|
senhuang42
|
1a8af0de73
|
Improve unit test
|
2020-11-12 11:09:09 -05:00 |
|
|
senhuang42
|
4d4fd2c55f
|
Overhaul repcode handling logic
|
2020-11-12 10:59:35 -05:00 |
|
|
sen
|
f62edf0fe9
|
Merge pull request #2381 from senhuang42/expand_sequence_extraction_api
Add enum to define ZSTD_Sequence type and update sequence extraction API
|
2020-11-06 13:00:31 -05:00 |
|
|
senhuang42
|
7d1dea070c
|
Update unit tests
|
2020-11-06 11:10:37 -05:00 |
|
|
senhuang42
|
779df995c6
|
Implement mergeGeneratedSequences()
|
2020-11-06 10:55:46 -05:00 |
|
|
senhuang42
|
51abd58208
|
Rename getSequences() to generateSequences()
|
2020-11-06 10:53:22 -05:00 |
|
|
senhuang42
|
261ea69661
|
Add new mergeGeneratedSequences() function
|
2020-11-06 10:52:34 -05:00 |
|
|
senhuang42
|
f782cac3d4
|
Change block delimiter removing to linear time approach
|
2020-11-02 17:06:20 -05:00 |
|
|
senhuang42
|
3c9b43da1d
|
Remove trailing comma
|
2020-11-02 11:53:58 -05:00 |
|
|
senhuang42
|
3434049c1f
|
Use ZSTD_memmove() instead of memmove()
|
2020-11-02 11:43:19 -05:00 |
|
|
senhuang42
|
c54a25b666
|
Revert compressibility change
|
2020-11-02 11:38:58 -05:00 |
|
|
senhuang42
|
d4d0346b40
|
Update name of enum, clarify documentation
|
2020-11-02 11:38:17 -05:00 |
|
|
senhuang42
|
9102f30dbf
|
Update unit test
|
2020-11-02 11:30:31 -05:00 |
|
|
senhuang42
|
e6178f837f
|
Revert unnecessary seqCollector adjustment
|
2020-11-02 10:59:20 -05:00 |
|
|
senhuang42
|
e8501e00b8
|
Fix incorrect index increment in merge algorithm
|
2020-11-02 10:58:41 -05:00 |
|
|
senhuang42
|
a36fdada57
|
Add algorithm to remove all delimiters
|
2020-11-02 10:46:52 -05:00 |
|
|
senhuang42
|
435a3a0428
|
Update seqCollector definition
|
2020-11-02 10:19:26 -05:00 |
|
|
senhuang42
|
3327932609
|
Update ZSTD_getSequences function signature
|
2020-11-02 10:17:59 -05:00 |
|
|
senhuang42
|
7397d0102f
|
Add new enum for different sequence formats for ingestion/extraction
|
2020-11-02 10:15:53 -05:00 |
|
|
sen
|
f54109c589
|
Merge pull request #2378 from senhuang42/free_cress_ptr
[minor] Pass cRess_t by const ptr instead of by value
|
2020-11-01 20:33:25 -05:00 |
|
|
sen
|
c37c714ef1
|
Merge pull request #2376 from senhuang42/clarify_sequence_extraction_api
Refine external ZSTD_Sequence API
|
2020-10-30 15:47:25 -04:00 |
|
|
senhuang42
|
f0da97642a
|
Specify that getSequences() will always emit block boundary sequences
|
2020-10-30 12:31:17 -04:00 |
|
|
senhuang42
|
536e89c723
|
Sequence extractor should update CBlockState
|
2020-10-30 12:13:19 -04:00 |
|
|
senhuang42
|
32cac2627a
|
Emit last literals of 0 size as well, to indicate block boundary
|
2020-10-29 16:41:17 -04:00 |
|
|
senhuang42
|
69bd5f0654
|
Correct literalsRead calculation to include longLength
|
2020-10-29 14:49:37 -04:00 |
|
|
sen
|
ff93440fc6
|
Merge pull request #2375 from senhuang42/ldm_oss_fuzz_testcase
Add a test case for LDM + opt parser with small uncompressible block
|
2020-10-29 09:32:05 -04:00 |
|
|
senhuang42
|
7198ebb213
|
Un-mix declarations and code
|
2020-10-28 18:51:03 -04:00 |
|
|
senhuang42
|
59624f3163
|
Remove implicit typecast to appease appVeyor windows build
|
2020-10-28 16:25:09 -04:00 |
|
|
senhuang42
|
60a52c29e6
|
Add check for allocation
|
2020-10-28 16:22:22 -04:00 |
|
|
senhuang42
|
3ed5d053d8
|
Clarify comments in zstd.h some more
|
2020-10-28 09:53:09 -04:00 |
|
|
senhuang42
|
169fc07aa1
|
Move test to appropriate location
|
2020-10-27 16:59:43 -04:00 |
|
|
senhuang42
|
db0b5d7d1e
|
Add test to fuzzer.c
|
2020-10-27 16:57:24 -04:00 |
|
|
sen
|
17b700d78a
|
Merge pull request #2366 from senhuang42/enable_ldm_by_default
Enable LDM by default if window size >= 128MB and strategy uses opt parser
|
2020-10-27 14:59:28 -04:00 |
|
|
senhuang42
|
3163909d14
|
Remove unused variable position
|
2020-10-27 12:58:12 -04:00 |
|
|
senhuang42
|
dc448563e9
|
Add test compatibility with last literals in sequences
|
2020-10-27 12:35:28 -04:00 |
|
|
senhuang42
|
1d221ecc03
|
Add support for representing last literals in the extracted seqs
|
2020-10-27 11:19:48 -04:00 |
|
|
senhuang42
|
9171f920cd
|
Improve documentation of seqStore_t
|
2020-10-27 10:50:22 -04:00 |
|
|
senhuang42
|
96b0ff7886
|
Improve documentation regarding various operations in copyBlockSequences
|
2020-10-27 10:36:06 -04:00 |
|
|
senhuang42
|
3a11c7eb03
|
Modify ZSTD_copyBlockSequences to agree with new API
|
2020-10-27 10:31:40 -04:00 |
|
|
senhuang42
|
761f40d1c6
|
Clarify and modify ZSTD_Sequence definition
|
2020-10-27 09:41:32 -04:00 |
|
|
senhuang42
|
8bdb32aebe
|
Add a function for LDM enable check
|
2020-10-20 13:46:02 -04:00 |
|
|
senhuang42
|
578e889ec1
|
Move ldm enable to compressStream2()
|
2020-10-20 13:04:45 -04:00 |
|
|
senhuang42
|
d28d8a1d72
|
Include LDM tables size for CCtx size estimation where relevant
|
2020-10-20 09:21:30 -04:00 |
|
|
senhuang42
|
81a2c02d8f
|
Move ldm no regression test to fuzzer longtests
|
2020-10-19 15:28:46 -04:00 |
|
|
senhuang42
|
b1c7fc5768
|
Add compatibility for multithreading
|
2020-10-19 12:07:06 -04:00 |
|
|
senhuang42
|
aad436da37
|
Document ldm enabled by default in zstd.h
|
2020-10-19 11:02:29 -04:00 |
|
|
senhuang42
|
df470e176b
|
Add unit test for no cctx requested params change
|
2020-10-19 10:52:41 -04:00 |
|
|
senhuang42
|
590f7f55f0
|
Add ldm enable condition in ZSTD_resetCCtx_internal
|
2020-10-19 10:26:17 -04:00 |
|
|
senhuang42
|
4d01979b62
|
Expose and call ZSTD_ldm_skipRawSeqStoreBytes()
|
2020-10-16 20:30:00 -04:00 |
|
|
senhuang42
|
adc53db6ed
|
Change make -j all check to make -j all && make check
|
2020-10-15 15:11:51 -04:00 |
|
|
senhuang42
|
42d037bdba
|
Add libregression build target, also fix make clean and .gitignore
|
2020-10-15 10:34:50 -04:00 |
|
|
senhuang42
|
f49926edf4
|
Change cycleLog adjustment to +3 from +4
|
2020-10-15 09:56:05 -04:00 |
|
|
senhuang42
|
ee84817fe7
|
Reset posInSequence when using ZSTD_referenceExternalSequences()
|
2020-10-14 22:06:08 -04:00 |
|