 senhuang42
|
81a2c02d8f
|
Move ldm no regression test to fuzzer longtests
|
2020-10-19 15:28:46 -04:00 |
|
|
senhuang42
|
adc53db6ed
|
Change make -j all check to make -j all && make check
|
2020-10-15 15:11:51 -04:00 |
|
|
senhuang42
|
42d037bdba
|
Add libregression build target, also fix make clean and .gitignore
|
2020-10-15 10:34:50 -04:00 |
|
|
senhuang42
|
f49926edf4
|
Change cycleLog adjustment to +3 from +4
|
2020-10-15 09:56:05 -04:00 |
|
|
senhuang42
|
d0550bb18f
|
Clarify argument names, fix DEBUGLOG() statements
|
2020-10-14 15:45:43 -04:00 |
|
|
senhuang42
|
3f99c9b38d
|
Adjust match backwards count args
|
2020-10-14 15:23:03 -04:00 |
|
|
senhuang42
|
bf0d559449
|
Introduce, implement, and call ZSTD_ldm_countBackwardsMatch_2segments()
|
2020-10-14 12:58:06 -04:00 |
|
|
senhuang42
|
467e4383b0
|
Merge branch 'dev' of github.com:senhuang42/zstd into change_ldm_mt_config
|
2020-10-14 10:17:50 -04:00 |
|
|
senhuang42
|
a39614de1f
|
Fix incorrect usage of cycleLog() in --patch-from
|
2020-10-13 13:00:27 -04:00 |
|
|
senhuang42
|
354b5f1c0a
|
Use cycleLog instead of chainLog to determine LDM jobLog
|
2020-10-12 16:09:59 -04:00 |
|
|
senhuang42
|
d6911b86be
|
Require LDM matches to be strictly greater in length
|
2020-10-09 12:56:18 -04:00 |
|
|
senhuang42
|
b9c8033cde
|
Define kNullRawSeqStore for every file
|
2020-10-07 19:02:41 -04:00 |
|
|
senhuang42
|
a6165c1b28
|
Change matchState_t::ldmSeqStore to pointer
|
2020-10-07 14:13:57 -04:00 |
|
|
senhuang42
|
abce708a56
|
Move posInSequence correction to correct location
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
0c515590d8
|
Replace offCode of largest match if ldm's offCode is superior
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
0fac8e07e1
|
Refactor usage of ms->ldmSeqStore so that it is not modified during compressBlock(), and simplify skipRawSeqStoreBytes
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
a5500cf2af
|
Refactor separate ldm variables all into one struct
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
0731b94e7c
|
Use kNullRawSeqStore constant in zstdmt_compress.c
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
0325d878f2
|
Remove bubbling down matches with longer offCode and same matchLen
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
031b7ec15f
|
Disable LDM minMatch adjustment when using opt parser
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
ddf8a3f1b9
|
Enable inclusion of mid-flight LDMs in opt parser
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
88f72ed942
|
Correct incorrect offcode calculation
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
e96ea5d147
|
Fix static analyze fuzzer.c error
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
d8b43a4202
|
Add explicit conversion of size_t to U32
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
b8bfc4e63d
|
Add cSize regression test to fuzzer.c
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
c87d2e5866
|
Prefix new static ldm helpers with ZSTD_opt
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
429dec4f42
|
Add DEBUGLOG() calls in ldm helpers
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
10647924f1
|
Make function descriptions more accurate
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
1a687b3fcb
|
Improve documentation of relevant structs
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
37617e23d7
|
Correct matchLength calculation and remove unnecessary functions
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
7dee62c287
|
Reset ldmSeqStore after initStats_ultra() pass for btultra2
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
0718aa70df
|
Refactor existing functions to use posInSequence
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
7348b40a87
|
Adjustments to ldm_calculateMatchRange() to calculate bounds correctly
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
a1ef2db5b2
|
Add ldm_calculateMatchRange() function
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
ef823e0299
|
Remove rawSeqStore.base and add rawSeqStore.posInSequence
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
cfd2aec1b7
|
Add unit tests into playTests.sh
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
4793ae3b84
|
Prevent duplicate LDMs from being inserted
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
65f9cfeeec
|
Add extra bounds check to prevent heap access after free ASAN error
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
bff5785fd5
|
Address mixed variables C90 warning
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
724b94ed18
|
ldm_getNextMatch fixed return values
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
ea92fb3a68
|
Cleanups, add comments and explanations
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
78da2e1808
|
Fixed sifting algorithm
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
6ccd97fc96
|
Fixed end of match boundary update issues
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
28394b64f2
|
Add proper bounds check on adding ldms
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
a2f2b58d04
|
Add a function ldm_voidSequences()
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
9c3c7cd20e
|
Fix function argument to getNextMatch()
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
c8b8572b38
|
Adjustments to no longer segfault on nci
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
f57c7e6bbf
|
Add base adjustment correction
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
5df9b5e05f
|
Add initial getNextMatch() in opt parser
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
f8ce7cabc3
|
Added more debugging
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
84009a076a
|
Add re-copying of ldmSeqStore after processing
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
42395a70c2
|
Add debug statements, flesh out functions
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
dd3dd199bb
|
Get zstd to build with new functions and callsites, fix arguments
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
766c4a8c28
|
Implement part of ldm_maybeAddLdm()
|
2020-10-07 13:56:25 -04:00 |
|
|
senhuang42
|
84777059d2
|
Implement ldm_getNextMatch()
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
28c74bf591
|
Implement basic splitSequence and skipSequence functions
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
634ab7830d
|
Flesh out required args for ldm_handleLdm()
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
db70761032
|
Add callsites to appropriate locations in ..opt_generic()
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
aea61e3c91
|
Add ldm helper function declarations into opt parser
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
35d9f488f5
|
Modify codepath to use opt parser exclusively if the compression level is high enough
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
e1ae398ad5
|
Add rawSeqStore to match state
|
2020-10-07 13:56:24 -04:00 |
|
|
senhuang42
|
7259b258d1
|
Add callsites to zstdcli.c and tests to playTests.sh
|
2020-10-07 13:47:38 -04:00 |
|
|
senhuang42
|
93cd9d8a6e
|
Add hasStdoutOutput setter to fileio.h
|
2020-10-07 13:44:25 -04:00 |
|
|
senhuang42
|
dd3dac9ce0
|
Fixed logic for stdout output
|
2020-10-07 13:43:27 -04:00 |
|
|
senhuang42
|
1ebe360d0f
|
Add new stdoutOutput field
|
2020-10-07 13:42:34 -04:00 |
|
|
senhuang42
|
ce56810a32
|
Modify error messages on console input/output
|
2020-09-28 12:15:18 -04:00 |
|
|
senhuang42
|
02422db841
|
Fix Stdin typo
|
2020-09-25 11:51:35 -04:00 |
|
|
senhuang42
|
9f7212a48b
|
Update unit tests
|
2020-09-24 16:44:33 -04:00 |
|
|
senhuang42
|
88f4410390
|
Add more useful failure message when stdin is an input
|
2020-09-24 16:29:12 -04:00 |
|
|
senhuang42
|
93d63eaeb8
|
Expand UTIL_requireUserConfirmation to include stdin input check
|
2020-09-24 15:58:06 -04:00 |
|
|
senhuang42
|
432186cbea
|
Add FIO_determineHasStdinInput() function and member to fCtx
|
2020-09-24 15:55:30 -04:00 |
|
|
senhuang42
|
0e8ac6b995
|
Add fCtx to FIO_openDstFile()
|
2020-09-24 15:49:30 -04:00 |
|
|
senhuang42
|
21cd640b93
|
Add unit tests to guard against bad stdin
|
2020-09-22 14:55:41 -04:00 |
|
|
senhuang42
|
7aa3da1cd7
|
Use IS_CONSOLE macro to detect that we're indeed using a console
|
2020-09-22 14:15:52 -04:00 |
|
|
senhuang42
|
8adeb9f1e6
|
Updated to repcode documentation to reflect dict content size
|
2020-09-22 13:24:27 -04:00 |
|
|
senhuang42
|
9dcfe4d7b7
|
Update documentation about repcodes in dictionaries
|
2020-09-22 13:02:26 -04:00 |
|
|
senhuang42
|
07034952df
|
Add -f to .zstd decompression CLI test
|
2020-09-18 13:01:45 -04:00 |
|
|
senhuang42
|
6b6cc80196
|
Support .zstd suffix only for decompression
|
2020-09-18 12:49:51 -04:00 |
|
|
senhuang42
|
29b8f2eb08
|
Fix multifile decompression summary
|
2020-09-16 10:38:13 -04:00 |
|
|
senhuang42
|
1d5c6fda9f
|
Add filename truncations to decompression
|
2020-09-16 10:28:45 -04:00 |
|
|
senhuang42
|
ab0d332563
|
Add display clears
|
2020-09-15 15:53:32 -04:00 |
|
|
senhuang42
|
cad6bf99af
|
Add padding to remove previous line's leftovers, and keep the printed line to around the same size
|
2020-09-15 13:01:46 -04:00 |
|
|
senhuang42
|
17b56f934e
|
Coding style cleanup
|
2020-09-11 11:42:12 -04:00 |
|
|
senhuang42
|
801513b5e7
|
Modify params rather than cctx->requestedParams
|
2020-09-11 11:41:10 -04:00 |
|
|
senhuang42
|
64bd68e44b
|
Adjust ZSTD_createCDict_byReference() function, and check for cdict when using compressStream2
|
2020-09-10 13:42:26 -04:00 |
|
|
senhuang42
|
a71963c7b8
|
nbThreads instead of numThreads
|
2020-09-09 12:40:00 -04:00 |
|
|
senhuang42
|
0a170b20a8
|
Add ZSTD_NUMTHREADS tests to playTests.sh
|
2020-09-08 10:34:50 -04:00 |
|
|
senhuang42
|
243c8dfb41
|
Add include guards for init_numThreads()
|
2020-09-08 09:26:16 -04:00 |
|
|
senhuang42
|
cc29492c5f
|
Update documentation for environment variable
|
2020-09-08 08:42:46 -04:00 |
|
|
senhuang42
|
136a0673f6
|
Add quick documentation for ZSTD_NUMTHREADS in the code
|
2020-09-07 18:19:31 -04:00 |
|
|
senhuang42
|
888c385a49
|
Change param name to ZSTDCLI_NUMTHREADS_DEFAULT
|
2020-09-07 18:11:42 -04:00 |
|
|
senhuang42
|
972e063219
|
Change default num_threads value
|
2020-09-07 18:09:48 -04:00 |
|
|
senhuang42
|
5123496104
|
Adjust function signatures
|
2020-09-07 13:13:05 -04:00 |
|
|
senhuang42
|
28a9dc78c3
|
Always update bytes status in FIO_ctx_t
|
2020-09-03 20:23:30 -04:00 |
|
|
senhuang42
|
60ee0519ca
|
Remove extraneous FIO_ctx_t functions
|
2020-09-03 20:22:56 -04:00 |
|
|
senhuang42
|
48bca10792
|
Address memory leak in CLI from fCtx
|
2020-09-03 10:14:04 -04:00 |
|
|
senhuang42
|
23feaecbe5
|
Fix FIO_removeMultiFilesWarning conflict
|
2020-09-03 09:49:13 -04:00 |
|
|
senhuang42
|
49949f1d51
|
Merge branch 'dev' into multifile_status_update_fio_refactor
|
2020-09-03 09:44:47 -04:00 |
|
|
senhuang42
|
202b2954a2
|
Remove comma-separated decls
|
2020-09-03 09:28:40 -04:00 |
|
|
senhuang42
|
a480b02044
|
Adjust comments and struct to better reflect FIO_ctx_t
|
2020-09-03 09:26:30 -04:00 |
|