 senhuang42
|
444c4650a0
|
Add newline to end of cli help message
|
2021-02-17 12:30:42 -05:00 |
|
|
senandGitHub
|
7e6c53c7fe
|
Merge pull request #2465 from senhuang42/cformat_2021
[license] Change year to 2021 compression_format.md
|
2021-01-11 12:35:34 -05:00 |
|
|
senhuang42
|
1d6d64afa3
|
Change year to 2021 for compression format file
|
2021-01-11 08:53:29 -05:00 |
|
|
senandGitHub
|
69085db61c
|
Merge pull request #2446 from senhuang42/multiple_ddicts_v3
[RFC] Support references to multiple DDicts
|
2021-01-08 16:49:45 -05:00 |
|
|
senhuang42
|
9ae0dd9336
|
Fix Visual and staticanalyze warnings
|
2021-01-07 17:58:37 -05:00 |
|
|
senhuang42
|
4f7584e7a3
|
Allow freestanding lib script regex to detect XXH64(
|
2021-01-07 12:29:12 -05:00 |
|
|
senhuang42
|
17222654bf
|
Add streaming decompression to unit test
|
2021-01-07 12:29:12 -05:00 |
|
|
senhuang42
|
c2c9b8a7ec
|
Address comments, clean up interface/internals
|
2021-01-07 12:29:12 -05:00 |
|
|
senhuang42
|
22b7bff2bc
|
Add unit test, improve documentation
|
2021-01-07 12:29:12 -05:00 |
|
|
senhuang42
|
ea52fc3606
|
Use XXHash for hash function, create a sensible public interface
|
2021-01-07 12:29:12 -05:00 |
|
|
senhuang42
|
7c1a79f232
|
Add debuglog statements
|
2021-01-07 12:29:11 -05:00 |
|
|
senhuang42
|
d1a6a9d285
|
Reference requested dict ID at decompression time
|
2021-01-07 12:29:11 -05:00 |
|
|
senhuang42
|
5a6d3eef2b
|
Allocate memory for DDict hash set when parameter is set
|
2021-01-07 12:29:11 -05:00 |
|
|
senhuang42
|
fd5b608f1c
|
Add parameter to control multiple DDicts
|
2021-01-07 12:29:11 -05:00 |
|
|
senhuang42
|
f933668d3f
|
Implement hashset for dictIDs
|
2021-01-07 12:29:11 -05:00 |
|
|
senhuang42
|
5c41490bfe
|
Use pre-defined constants
|
2020-12-21 11:52:05 -05:00 |
|
|
senhuang42
|
339d8ba103
|
Add unit test
|
2020-12-21 11:33:27 -05:00 |
|
|
senhuang42
|
7e11bd012b
|
Implement skippable frame function
|
2020-12-21 11:13:22 -05:00 |
|
|
senandGitHub
|
f8224ab116
|
Merge pull request #2418 from senhuang42/fix_paramgrill_warnings
Fix various paramgrill conversion warnings
|
2020-12-08 12:50:23 -05:00 |
|
|
senhuang42
|
b9ab6bc061
|
Fix various conversion warnings
|
2020-12-08 10:07:28 -05:00 |
|
|
senandGitHub
|
f34d2f4192
|
Merge pull request #2408 from senhuang42/seekable_hang_fix
Remove possibility of hanging when using seekable decompression
|
2020-12-07 08:46:27 -05:00 |
|
|
senhuang42
|
db249bef4b
|
Allow paramgrill in cmake on windows
|
2020-12-03 10:39:16 -05:00 |
|
|
senhuang42
|
260b85acf5
|
Fix MSVC 2019 warnings
|
2020-12-03 10:36:45 -05:00 |
|
|
senhuang42
|
a8693ddef7
|
Add seekable tests to CI
|
2020-12-03 09:25:45 -05:00 |
|
|
senhuang42
|
26f89d47aa
|
Clean up makefile for seekable tests
|
2020-12-03 09:25:04 -05:00 |
|
|
senhuang42
|
152b55879c
|
Add unit tests to seekable
|
2020-12-02 15:33:12 -05:00 |
|
|
senandGitHub
|
a6ee1b3047
|
Merge pull request #2407 from senhuang42/sequence_compression_nbseqcheck_fix
[oss-fuzz] Move sequence compression API nbSequences check
|
2020-12-02 14:40:02 -05:00 |
|
|
senhuang42
|
9db49a3989
|
Add a forward progress requirement bound to seekable streaming decompression
|
2020-12-02 12:24:16 -05:00 |
|
|
senhuang42
|
2cbd038528
|
Move max nb seq check to per-block
|
2020-12-02 12:11:32 -05:00 |
|
|
senandGitHub
|
993baba315
|
Merge pull request #2405 from senhuang42/sequence_compression_minmatch_fix
Sequence Compression API - make validator use cctx minMatch instead of global MINMATCH
|
2020-12-01 17:34:01 -05:00 |
|
|
senhuang42
|
3efe9c902b
|
Add sequence nb validation to compressSequences(), adjust minMatch comparisons
|
2020-12-01 10:54:45 -05:00 |
|
|
senhuang42
|
4c5f337248
|
Use cctx's minMatch instead of global MINMATCH, make fuzzer use validation
|
2020-11-30 15:41:20 -05:00 |
|
|
senandGitHub
|
e0ec32f1e3
|
Merge pull request #2398 from senhuang42/fix_seq_compression_oss_fuzz
[OSS-Fuzz] Make sequence compression fuzzer's generated minmatch to be same as CCtx's minmatch
|
2020-11-23 21:54:16 -05:00 |
|
|
senhuang42
|
23554ff25f
|
Force CCtx minmatch to be same as generated minmatch
|
2020-11-23 13:29:20 -05:00 |
|
|
senandGitHub
|
0a4dbe9cd2
|
Merge pull request #2394 from senhuang42/seq_compress_fuzzer
Fuzzer for sequence compression API
|
2020-11-22 13:07:46 -05:00 |
|
|
senhuang42
|
c502cd33e5
|
Fix generating 1 too few characters in random string generator
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
5b0c8f0a7c
|
Add appropriate bound to matchlengths, and reduce srcSize max
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
a73a07b189
|
Add a bound for matchlength dependent on window size
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
5c68c5e31e
|
Variety of minor fixups, reduce allocation, make deterministic
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
59c021f501
|
Add built binary to .gitignore
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
26bc0bfdf6
|
Add new fuzzer to build targets
|
2020-11-20 16:58:25 -05:00 |
|
|
senhuang42
|
ed575963c5
|
Implement new fuzzer for sequence compression
|
2020-11-20 16:58:25 -05:00 |
|
|
senandGitHub
|
c5fbd55dac
|
Merge pull request #2387 from senhuang42/compress_sequence_API
[RFC] New sequence compression API
|
2020-11-20 16:54:20 -05:00 |
|
|
senhuang42
|
7742f076b4
|
Add experimental param for sequence validation
|
2020-11-20 11:57:41 -05:00 |
|
|
senhuang42
|
0e32928b7d
|
Remove unnecessary repcode backup, apply style choices, use function pointer
|
2020-11-20 11:02:19 -05:00 |
|
|
senandGitHub
|
4e809dd572
|
Merge pull request #2395 from senhuang42/is_rle_speedup
10x speedup for ZSTD_isRLE()
|
2020-11-19 18:26:42 -05:00 |
|
|
senhuang42
|
dcbbf7c09f
|
Unroll isRLE loop
|
2020-11-19 12:38:13 -05:00 |
|
|
senhuang42
|
05c0229668
|
Clean up visual conversion warnings
|
2020-11-18 15:36:29 -05:00 |
|
|
senhuang42
|
3c4454769b
|
Improve documentation on ZSTD_compressSequences()
|
2020-11-18 09:52:24 -05:00 |
|
|
senhuang42
|
d6d7ba2a1f
|
Modification to offset validation to include entire sequence
|
2020-11-17 10:13:22 -05:00 |
|
|
senhuang42
|
8f3136a9c7
|
Fix assert edge case, improve documentation in zstd.h
|
2020-11-16 18:05:35 -05:00 |
|
|
senhuang42
|
f6baad87d6
|
Fix warnings and make validation enabled by default
|
2020-11-16 12:00:06 -05:00 |
|
|
senhuang42
|
55b90ef010
|
Fix unit tests to agree with new changes
|
2020-11-16 11:36:37 -05:00 |
|
|
senhuang42
|
7f563b0519
|
Add new sequence format as an experimental CCtx param
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
347824ad73
|
Overhaul logic to simplify, add in proper validations, fix match splitting
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
46824cb018
|
Add new sequence compress api params to cctx
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
48405b4633
|
Fix srcSize=0 edge case
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
022e6d81e7
|
Fix literals length calculation
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
3d26615c84
|
Adjust unit tests to agree with new sequence generation API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dad20b5ccb
|
Remove dstCapacity error check
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
b8e16a2057
|
Remove extraneous function in this API
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
f29507c4fc
|
Add check comparing offset to window size
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
7a6e46a92f
|
Fix MSAN errors
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
cc2642bd17
|
Address edge case with endPosInSequence
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
fd10007174
|
Change debug levels to appropriate ones
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2db8441245
|
Add RLE support
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
dfef298336
|
Fix various build warnings
|
2020-11-16 10:49:17 -05:00 |
|
|
senhuang42
|
2bbdddf24e
|
Add test case to roundtrip using ZSTD_getSequences() and ZSTD_compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
5fd69f8173
|
Add documentation for new api functions
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e8b7fdb64b
|
Refactor for enhanced code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c675fb46f1
|
Rename internal function compressSequences(), and promote new *_ext() functions to their actual name
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
013434e1e4
|
Add another API function to compress with existing CCTX
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c44ce29013
|
More adjustments to improve code clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
48f67da854
|
Pull compressStream2() transparent initialization into its own function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
c86151f53c
|
Add initial support for new ZSTD_Sequence mode
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e0f26afce9
|
Add sequence compression format param
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
f51af9a609
|
Always ensure sequenceRange updates properly, add more error forwarding
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
1a449688fd
|
Various minor logical refactors to improve clarity
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e5fe485dcc
|
Fix cSize calculation for noCompressBlocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
6145ebb400
|
Rebased, roundtrips silesia.tar
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b5b61cc216
|
Refactor for better debugging info
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
293fad6b45
|
Corrections and edge-case fixes to be able to roundtrip dickens
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
7eb6fa7be4
|
Multi-block compression scaffolding - works on single-block files
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
75b01f34b9
|
Add support for uncompressible blocks
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
e04da68157
|
Enable usage of ZSTD_sequenceRange for single-block compression
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
337fac216d
|
Add logic to handle ZSTD_sequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
85822ddd53
|
Add last literals handling like getSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
2cff8df1a2
|
Pull block compression out of main compressSequences() function
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
cfced9344a
|
Implement ZSTD_updateSequenceRange
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
b116e1f211
|
Modify SequenceRange to have posInSequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
d99b675112
|
Add function definition for sequenceRange updater
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
74e95c05cc
|
Add ZSTD_SequenceRange to count ranges in array of ZSTD_Sequence
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
89f3848310
|
Add support for repcodes
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
3e930fd044
|
Code cleanup, add debuglog statments
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
086513b5b9
|
Implement first pass at compressSequences()
|
2020-11-16 10:49:16 -05:00 |
|
|
senhuang42
|
a9327b1e9b
|
Add initial function prototype for ZSTD_compressSequences_ext (to be renamed later)
|
2020-11-16 10:33:35 -05:00 |
|
|
senandGitHub
|
9736c497ce
|
Merge pull request #2393 from senhuang42/fix_sequence_extractions_api
Improve repcode handling in sequence extraction API
|
2020-11-15 18:29:52 -05:00 |
|
|
senhuang42
|
9d936d61d2
|
Reduce number of memcpy() calls
|
2020-11-13 19:43:30 -05:00 |
|
|
senhuang42
|
be4ac6c5bc
|
Use existing repcode update function to implement updates
|
2020-11-12 16:51:12 -05:00 |
|
|
senhuang42
|
674c9b9235
|
Add in proper block repcode histories
|
2020-11-12 15:34:37 -05:00 |
|