 senandGitHub
|
f8224ab116
|
Merge pull request #2418 from senhuang42/fix_paramgrill_warnings
Fix various paramgrill conversion warnings
|
2020-12-08 12:50:23 -05:00 |
|
|
senhuang42
|
b9ab6bc061
|
Fix various conversion warnings
|
2020-12-08 10:07:28 -05:00 |
|
|
senandGitHub
|
f34d2f4192
|
Merge pull request #2408 from senhuang42/seekable_hang_fix
Remove possibility of hanging when using seekable decompression
|
2020-12-07 08:46:27 -05:00 |
|
|
senhuang42
|
db249bef4b
|
Allow paramgrill in cmake on windows
|
2020-12-03 10:39:16 -05:00 |
|
|
senhuang42
|
260b85acf5
|
Fix MSVC 2019 warnings
|
2020-12-03 10:36:45 -05:00 |
|
|
senhuang42
|
a8693ddef7
|
Add seekable tests to CI
|
2020-12-03 09:25:45 -05:00 |
|
|
senhuang42
|
26f89d47aa
|
Clean up makefile for seekable tests
|
2020-12-03 09:25:04 -05:00 |
|
|
senhuang42
|
152b55879c
|
Add unit tests to seekable
|
2020-12-02 15:33:12 -05:00 |
|
|
senandGitHub
|
a6ee1b3047
|
Merge pull request #2407 from senhuang42/sequence_compression_nbseqcheck_fix
[oss-fuzz] Move sequence compression API nbSequences check
|
2020-12-02 14:40:02 -05:00 |
|
|
senhuang42
|
9db49a3989
|
Add a forward progress requirement bound to seekable streaming decompression
|
2020-12-02 12:24:16 -05:00 |
|
|
senhuang42
|
2cbd038528
|
Move max nb seq check to per-block
|
2020-12-02 12:11:32 -05:00 |
|
|
senandGitHub
|
993baba315
|
Merge pull request #2405 from senhuang42/sequence_compression_minmatch_fix
Sequence Compression API - make validator use cctx minMatch instead of global MINMATCH
|
2020-12-01 17:34:01 -05:00 |
|
|
senhuang42
|
3efe9c902b
|
Add sequence nb validation to compressSequences(), adjust minMatch comparisons
|
2020-12-01 10:54:45 -05:00 |
|
|
senhuang42
|
4c5f337248
|
Use cctx's minMatch instead of global MINMATCH, make fuzzer use validation
|
2020-11-30 15:41:20 -05:00 |
|
|
senandGitHub
|
e0ec32f1e3
|
Merge pull request #2398 from senhuang42/fix_seq_compression_oss_fuzz
[OSS-Fuzz] Make sequence compression fuzzer's generated minmatch to be same as CCtx's minmatch
|
2020-11-23 21:54:16 -05:00 |
|
|
senhuang42
|
23554ff25f
|
Force CCtx minmatch to be same as generated minmatch
|
2020-11-23 13:29:20 -05:00 |
|
|
senandGitHub
|
0a4dbe9cd2
|
Merge pull request #2394 from senhuang42/seq_compress_fuzzer
Fuzzer for sequence compression API
|
2020-11-22 13:07:46 -05:00 |
|
|
senhuang42
|
c502cd33e5
|
Fix generating 1 too few characters in random string generator
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
5b0c8f0a7c
|
Add appropriate bound to matchlengths, and reduce srcSize max
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
a73a07b189
|
Add a bound for matchlength dependent on window size
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
5c68c5e31e
|
Variety of minor fixups, reduce allocation, make deterministic
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
59c021f501
|
Add built binary to .gitignore
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
26bc0bfdf6
|
Add new fuzzer to build targets
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
ed575963c5
|
Implement new fuzzer for sequence compression
|
2020-11-20 16:58:25 -05:00 |
|
|
senandGitHub
|
c5fbd55dac
|
Merge pull request #2387 from senhuang42/compress_sequence_API
[RFC] New sequence compression API
|
2020-11-20 16:54:20 -05:00 |
|
|
senhuang42
|
7742f076b4
|
Add experimental param for sequence validation
|
2020-11-20 11:57:41 -05:00 |
|
|
senhuang42
|
0e32928b7d
|
Remove unnecessary repcode backup, apply style choices, use function pointer
|
2020-11-20 11:02:19 -05:00 |
|
|
senandGitHub
|
4e809dd572
|
Merge pull request #2395 from senhuang42/is_rle_speedup
10x speedup for ZSTD_isRLE()
|
2020-11-19 18:26:42 -05:00 |
|
|
senhuang42
|
dcbbf7c09f
|
Unroll isRLE loop
|
2020-11-19 12:38:13 -05:00 |
|
|
senhuang42
|
05c0229668
|
Clean up visual conversion warnings
|
2020-11-18 15:36:29 -05:00 |
|
|
senhuang42
|
3c4454769b
|
Improve documentation on ZSTD_compressSequences()
|
2020-11-18 09:52:24 -05:00 |
|
|
senhuang42
|
d6d7ba2a1f
|
Modification to offset validation to include entire sequence
|
2020-11-17 10:13:22 -05:00 |
|
|
senhuang42
|
8f3136a9c7
|
Fix assert edge case, improve documentation in zstd.h
|
2020-11-16 18:05:35 -05:00 |
|
|
senhuang42
|
f6baad87d6
|
Fix warnings and make validation enabled by default
|
2020-11-16 12:00:06 -05:00 |
|
|
senhuang42
|
55b90ef010
|
Fix unit tests to agree with new changes
|
2020-11-16 11:36:37 -05:00 |
|
|
senhuang42
|
7f563b0519
|
Add new sequence format as an experimental CCtx param
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
347824ad73
|
Overhaul logic to simplify, add in proper validations, fix match splitting
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
46824cb018
|
Add new sequence compress api params to cctx
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
48405b4633
|
Fix srcSize=0 edge case
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
022e6d81e7
|
Fix literals length calculation
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
3d26615c84
|
Adjust unit tests to agree with new sequence generation API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dad20b5ccb
|
Remove dstCapacity error check
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
b8e16a2057
|
Remove extraneous function in this API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
f29507c4fc
|
Add check comparing offset to window size
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
7a6e46a92f
|
Fix MSAN errors
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
cc2642bd17
|
Address edge case with endPosInSequence
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
fd10007174
|
Change debug levels to appropriate ones
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2db8441245
|
Add RLE support
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dfef298336
|
Fix various build warnings
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2bbdddf24e
|
Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
5fd69f8173
|
Add documentation for new api functions
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e8b7fdb64b
|
Refactor for enhanced code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c675fb46f1
|
Rename internal function compressSequences(), and promote new *_ext() functions to their actual name
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
013434e1e4
|
Add another API function to compress with existing CCTX
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c44ce29013
|
More adjustments to improve code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
48f67da854
|
Pull compressStream2() transparent initialization into its own function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c86151f53c
|
Add initial support for new ZSTD_Sequence mode
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e0f26afce9
|
Add sequence compression format param
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
f51af9a609
|
Always ensure sequenceRange updates properly, add more error forwarding
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
1a449688fd
|
Various minor logical refactors to improve clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e5fe485dcc
|
Fix cSize calculation for noCompressBlocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
6145ebb400
|
Rebased, roundtrips silesia.tar
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b5b61cc216
|
Refactor for better debugging info
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
293fad6b45
|
Corrections and edge-case fixes to be able to roundtrip dickens
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
7eb6fa7be4
|
Multi-block compression scaffolding - works on single-block files
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
75b01f34b9
|
Add support for uncompressible blocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e04da68157
|
Enable usage of ZSTD_sequenceRange for single-block compression
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
337fac216d
|
Add logic to handle ZSTD_sequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
85822ddd53
|
Add last literals handling like getSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
2cff8df1a2
|
Pull block compression out of main compressSequences() function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
cfced9344a
|
Implement ZSTD_updateSequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b116e1f211
|
Modify SequenceRange to have posInSequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
d99b675112
|
Add function definition for sequenceRange updater
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
74e95c05cc
|
Add ZSTD_SequenceRange to count ranges in array of ZSTD_Sequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
89f3848310
|
Add support for repcodes
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
3e930fd044
|
Code cleanup, add debuglog statments
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
086513b5b9
|
Implement first pass at compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
a9327b1e9b
|
Add initial function prototype for ZSTD_compressSequences_ext (to be renamed later)
|
2020-11-16 10:33:35 -05:00 |
|
|
senandGitHub
|
9736c497ce
|
Merge pull request #2393 from senhuang42/fix_sequence_extractions_api
Improve repcode handling in sequence extraction API
|
2020-11-15 18:29:52 -05:00 |
|
|
senhuang42
|
9d936d61d2
|
Reduce number of memcpy() calls
|
2020-11-13 19:43:30 -05:00 |
|
|
senhuang42
|
be4ac6c5bc
|
Use existing repcode update function to implement updates
|
2020-11-12 16:51:12 -05:00 |
|
|
senhuang42
|
674c9b9235
|
Add in proper block repcode histories
|
2020-11-12 15:34:37 -05:00 |
|
|
senhuang42
|
06c7f14066
|
Let block reps persist
|
2020-11-12 12:24:44 -05:00 |
|
|
senhuang42
|
396275068c
|
Fix incorrect repcode setting
|
2020-11-12 11:57:01 -05:00 |
|
|
senhuang42
|
1a8af0de73
|
Improve unit test
|
2020-11-12 11:09:09 -05:00 |
|
|
senhuang42
|
4d4fd2c55f
|
Overhaul repcode handling logic
|
2020-11-12 10:59:35 -05:00 |
|
|
senandGitHub
|
f62edf0fe9
|
Merge pull request #2381 from senhuang42/expand_sequence_extraction_api
Add enum to define ZSTD_Sequence type and update sequence extraction API
|
2020-11-06 13:00:31 -05:00 |
|
|
senhuang42
|
7d1dea070c
|
Update unit tests
|
2020-11-06 11:10:37 -05:00 |
|
|
senhuang42
|
779df995c6
|
Implement mergeGeneratedSequences()
|
2020-11-06 10:55:46 -05:00 |
|
|
senhuang42
|
51abd58208
|
Rename getSequences() to generateSequences()
|
2020-11-06 10:53:22 -05:00 |
|
|
senhuang42
|
261ea69661
|
Add new mergeGeneratedSequences() function
|
2020-11-06 10:52:34 -05:00 |
|
|
senhuang42
|
f782cac3d4
|
Change block delimiter removing to linear time approach
|
2020-11-02 17:06:20 -05:00 |
|
|
senhuang42
|
3c9b43da1d
|
Remove trailing comma
|
2020-11-02 11:53:58 -05:00 |
|
|
senhuang42
|
3434049c1f
|
Use ZSTD_memmove() instead of memmove()
|
2020-11-02 11:43:19 -05:00 |
|
|
senhuang42
|
c54a25b666
|
Revert compressibility change
|
2020-11-02 11:38:58 -05:00 |
|
|
senhuang42
|
d4d0346b40
|
Update name of enum, clarify documentation
|
2020-11-02 11:38:17 -05:00 |
|
|
senhuang42
|
9102f30dbf
|
Update unit test
|
2020-11-02 11:30:31 -05:00 |
|
|
senhuang42
|
e6178f837f
|
Revert unnecessary seqCollector adjustment
|
2020-11-02 10:59:20 -05:00 |
|
|
senhuang42
|
e8501e00b8
|
Fix incorrect index increment in merge algorithm
|
2020-11-02 10:58:41 -05:00 |
|
|
senhuang42
|
a36fdada57
|
Add algorithm to remove all delimiters
|
2020-11-02 10:46:52 -05:00 |
|