ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,039 Commits 17 Branches 77 Tags
a5f2c45528032ed20c33e0f8cd2c163a800a0017
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
senandGitHub 9d2a45a705 Merge pull request #2778 from senhuang42/opt_inlining_revert 
Revert opt outlining change
 2021-09-15 14:22:10 -04:00
Sen Huang bd84e4a9d3 Revert opt outlining change 2021-09-15 09:08:41 -07:00
senandGitHub 29f595ef76 Merge pull request #2773 from senhuang42/huffman_bugix 
Fix dictionary training huffman segfault and small speed improvement
 2021-09-13 21:02:42 -04:00
Sen Huang d45d0ad9d8 Update regression test 2021-09-13 12:41:02 -04:00
Sen Huang 4a498fb9c3 Add a dictionary training large corpus test 2021-09-13 12:29:17 -04:00
Sen Huang 1daf3c8dbc Use 32 buckets for log2 bucketing in huffman sort 2021-09-13 12:29:16 -04:00
senandGitHub bcc68275f1 Merge pull request #2769 from senhuang42/typo_fix 
[easy] Fix patch-from help msg typo
 2021-09-07 11:09:55 -04:00
senhuang42 30fe49af4e Fix patch-from help msg typo 2021-09-07 10:08:35 -04:00
senandGitHub 71076b7a01 Merge pull request #2763 from senhuang42/opt_compiletime 
Improve compile speed and binary size in `opt`
 2021-09-02 11:59:02 -04:00
senandGitHub 4d61f10e23 Merge pull request #2761 from senhuang42/fse_wksp_fix 
Add 8 bytes to FSE_buildCTable wksp
 2021-09-01 17:09:45 -04:00
Sen Huang d88c1d95ce Remove inlining for opt 2021-09-01 16:58:57 -04:00
senhuang42 414e24becf Add 8 bytes to FSE workspace 2021-09-01 15:56:33 -04:00
senandGitHub 1f3fc1936c Merge pull request #2753 from senhuang42/better_error_msg 
[easy] Fix zstd bench error message
 2021-08-23 20:37:48 -04:00
senhuang42 dce48f53df Fix benchzstd error message 2021-08-23 19:10:16 -04:00
senandGitHub ae998544de Merge pull request #2750 from senhuang42/sb_compress 
Improve branch misses on FSE symbol spreading
 2021-08-20 12:47:24 -04:00
senhuang42 da095ed899 Improve branch misses on FSE symbol spreading 2021-08-18 10:22:22 -07:00
senandGitHub 6a258043f5 Merge pull request #2692 from senhuang42/rebalance_clevel 
[RFC] Rebalance compression levels
 2021-08-06 12:51:31 -04:00
Sen Huang 539b3aab9b Optimize 32-bit VecMask_next() 2021-08-04 17:14:58 -04:00
senhuang42 e411040ea1 Add 64 row entry support for lazy 2021-08-04 16:19:12 -04:00
senhuang42 31820e032c Rebalance clevels for lazy 2021-08-04 16:18:52 -04:00
senandGitHub f9b034086a Merge pull request #2732 from senhuang42/huf_sort_improvement 
[HUF] Improve Huffman sorting algorithm
 2021-08-04 16:07:56 -04:00
senhuang42 aa1957477b Improve Huffman sorting algorithm 2021-08-04 12:43:34 -04:00
senandGitHub 5c46f62006 Merge pull request #2677 from senhuang42/ci_overhaul_2 
[CI][2/2] Migrate CI tests which (currently) fail
 2021-08-02 09:55:49 -04:00
Sen Huang 5ec7897a26 Fix static analyzer warnings 2021-07-29 09:11:12 -07:00
Sen Huang 43aad811db Add VS2019 test, migrate minimal decompressor macros to GHActions 2021-07-29 08:49:36 -07:00
senandGitHub d90bc0e0b6 Merge pull request #2720 from senhuang42/remove_folder 
Remove folder when done with test
 2021-06-30 14:37:21 -04:00
senhuang42 f5f6cc2e48 Remove folder when done with test 2021-06-30 13:09:00 -04:00
senandGitHub 45d707e908 Merge pull request #2715 from senhuang42/sequence_api_3 
[RFC] Add internal API for converting ZSTD_Sequence into seqStore
 2021-06-24 13:02:11 -04:00
senhuang42 76466dfadf Add simple API for converting ZSTD_Sequence into seqStore 2021-06-23 12:10:48 -04:00
senandGitHub d5f3568c4b Merge pull request #2697 from senhuang42/entropy_repeat_fix 
[bug] Fix entropy repeat mode bug
 2021-06-10 16:39:17 +03:00
senandGitHub a21b9036fe Merge pull request #2678 from senhuang42/big_endian_no_intrinsics_fuzztest 
Fuzzer test with no intrinsics on S390x (big endian)
 2021-06-09 18:46:52 +03:00
senandGitHub dd33ec9db0 Merge pull request #2700 from gauthamkrishna9991/update_readme_travis_link 
Update README for Travis CI Badge
 2021-06-09 10:41:35 +03:00
senandGitHub 5fb3884f33 Merge pull request #2691 from senhuang42/per_pr_regressiontest 
Make regression test run on every PR
 2021-06-07 12:44:31 +03:00
Sen Huang 2ff5c7b59f Add no intrinsics fuzztest, rowhash compression size test, and S390X to travis 2021-06-07 00:54:53 -07:00
senhuang42 88acf0ac65 Make regression test run on every PR 2021-06-07 00:47:56 -07:00
Sen Huang 923e5ad3f5 Fix entropy repeat mode bug 2021-06-07 00:32:03 -07:00
senandGitHub 0a96d00064 Merge pull request #2684 from senhuang42/old_api_ldm_blocksplit 
Add ldm and block splitter auto-enable to old api
 2021-06-03 10:10:27 +03:00
senandGitHub 18d02cbf2e Merge pull request #2686 from senhuang42/arm64fuzztest 
Add arm64 fuzz test to travis
 2021-05-27 13:50:26 -04:00
senhuang42 56b7dd121c Add arm64 fuzz test to travis 2021-05-26 14:56:47 -04:00
senandGitHub 2ee2cf9cdf Merge pull request #2682 from senhuang42/armbuild_fixtest 
Make GH Actions CI tests run apt-get update before apt-get install
 2021-05-26 14:14:43 -04:00
senhuang42 d278bede33 Update apt-get prior to tests that install packages 2021-05-26 13:38:36 -04:00
senandGitHub 414bcf239b Merge pull request #2675 from senhuang42/ci_overhaul 
[CI][1/2] Re-do the github actions workflows, migrate various travis and appveyor tests.
 2021-05-26 13:35:53 -04:00
senhuang42 939276cd0c Add ldm and block splitter auto-enable to old api 2021-05-24 13:09:32 -04:00
senhuang42 bb0cd722b6 Migrate travis CI tests 2021-05-21 09:59:57 -04:00
senandGitHub d92fef0f0a Merge pull request #2667 from senhuang42/arm_tests_ci 
[CI] Add ARM tests back into CI
 2021-05-17 14:29:20 -04:00
senandGitHub 9fe1072229 Merge pull request #2668 from senhuang42/update_ci_platforms 
[CI] Fix zlib-wrapper test
 2021-05-17 14:29:04 -04:00
senhuang42 5a75417d2b [ci] Add ARM tests back into CI 2021-05-17 13:10:57 -04:00
senhuang42 38ffe9658e [ci] Use *-latest for platforms to test on 2021-05-17 13:04:55 -04:00
senandGitHub 06718087f8 Remove deprecate flag for vcx (#2647) 2021-05-13 19:33:48 -04:00
senandGitHub 40def70387 Add source level deprecation warning disabling to certain tests/utils (#2645) 2021-05-13 14:41:21 -04:00
senandGitHub a51e342ed1 Update CHANGELOG to include patch from fix (#2642) 2021-05-12 16:46:05 -04:00
senandGitHub c730b8c5a3 Remove const data members in threadpooltest payload (#2639) (#2640) 2021-05-12 16:09:48 -04:00
senandGitHub b35c250bf3 Remove const data members in threadpooltest payload (#2639) 2021-05-12 12:56:57 -04:00
senandGitHub 01fe4796fb Add mt lib build to CL, shuffle around bugs section (#2638) 2021-05-12 11:31:31 -04:00
senandGitHub 9c23ea9e2b Bump version to 1.5.0, rebuild documentation (#2634) 2021-05-11 16:32:09 -04:00
senandGitHub 13449d7ce1 Add PHONY targets to makefiles (#2629) 2021-05-07 14:03:19 -04:00
senandGitHub 9e94b7cac5 Assert no divison by 0, correct superblocks 0 sequences case (#2592) 2021-05-07 13:26:56 -04:00
senandGitHub d8d6e48a0a Add threadPool unit tests to fuzzer.c (#2604) 2021-05-07 11:13:44 -04:00
senandGitHub 91465e23b2 [1.5.0] Enable multithreading in lib build by default (#2584) 
* Update lib Makefile to have new targets

* Update lib/README.md for mt
 2021-05-07 11:13:30 -04:00
senandGitHub 698f261b35 [1.5.0] Deprecate some functions (#2582) 
* Add deprecated macro to zstd.h, mark certain functions as deprecated

* Remove ZSTD_compress.c dependencies on deprecated functions
 2021-05-06 17:59:32 -04:00
senandGitHub 6030cdfede Add --progress flag (#2595) 2021-05-06 14:50:28 -04:00
senandGitHub d6be7659b0 Add seekable roundtrip fuzzer (#2617) 2021-05-06 10:08:21 -04:00
Sen Huang e6c8a5dd40 Fix incorrect usages of repIndex across all strategies 2021-05-04 19:50:55 -04:00
senandGitHub cc31bb8b66 Merge pull request #2598 from senhuang42/reduce_index_rowhash_fix 
Fix chaintable check to include rowhash in ZSTD_reduceIndex()
 2021-05-03 17:34:39 -04:00
senandGitHub 4c5cc345fb Merge pull request #2581 from senhuang42/lcm_stable 
[1.5.0] Promote ZSTD_c_literalCompressionMode to stable params
 2021-05-03 11:59:19 -04:00
senandGitHub cdc979ddb3 Merge pull request #2580 from senhuang42/defaultclevel_to_stable 
[1.5.0] Promote ZSTD_defaultCLevel() into stable API
 2021-05-03 11:59:05 -04:00
senhuang42 61fe571af6 Fix chaintable check to include rowhash in ZSTD_reduceIndex() 2021-04-30 19:52:04 -04:00
senandGitHub f8afc66573 Merge pull request #2588 from senhuang42/update_regressiontest 
[regressiontest] Update results.csv
 2021-04-27 12:40:07 -04:00
senhuang42 33abda4400 Update results.csv 2021-04-26 15:55:23 -04:00
senandGitHub 3e2fbfd056 Merge pull request #2579 from senhuang42/getcdictID_to_stable 
[1.5.0] Promote ZSTD_getDictID_fromCDict() into stable API
 2021-04-26 09:55:43 -04:00
Sen Huang 3c595a4a79 Add literalCompressionMode to stable cParams 2021-04-26 09:55:06 -04:00
senandGitHub 14c11c7459 Merge pull request #2586 from senhuang42/dds_fuzz 
Add DDS to oss fuzzer
 2021-04-24 12:39:16 -04:00
senhuang42 f80dec66b0 Add DDS to oss fuzzer 2021-04-22 18:21:43 -04:00
senandGitHub 3eb3845898 Merge pull request #2583 from senhuang42/remove_zbuff 
[1.5.0] Remove ZBUFF
 2021-04-20 17:05:32 -04:00
senhuang42 a423305e7b Remove ZBUFF tests 2021-04-19 17:27:05 -04:00
senhuang42 3b98987496 Remove building of ZBUFF/deprecated folder by default 2021-04-19 17:12:00 -04:00
Sen Huang c5869677d9 Moved ZSTD_defaultCLevel() into stable API 2021-04-16 10:15:40 -07:00
Sen Huang 9c1ca3c00b Moved ZSTD_getDictID_fromCDict() into stable API 2021-04-16 10:14:29 -07:00
senandGitHub 12c045f74d Merge pull request #2574 from senhuang42/repcode_mismatch_detector_fix 
Correct the block splitter mismatched repcodes detection.
 2021-04-12 23:27:43 -04:00
senandGitHub ebd41ebe56 Merge pull request #2572 from senhuang42/row_hash_hashcache_one_off_error_fix 
Adjust nb elements to prefetch in ZSTD_row_fillHashCache()
 2021-04-12 15:38:53 -04:00
Sen Huang 8844f93957 Adjust nb elements to prefetch in ZSTD_row_fillHashCache() 2021-04-12 14:24:58 -04:00
Sen Huang 550f76f131 Correct the detection of mismatched repcodes 2021-04-09 09:08:51 -07:00
senandGitHub 56421f34e4 Merge pull request #2494 from senhuang42/row_hash2 
SIMD Row Based Matchfinder 🚀
 2021-04-08 10:55:14 -04:00
Sen Huang 4d63d6e8aa Update results.csv, add Row hash to regression test 2021-04-07 10:31:41 -07:00
senandGitHub f71aabb5b5 Move clevel override to after initLocalDict() (#2571) 2021-04-06 21:05:37 -04:00
senandGitHub f1e8b565c2 Maintain two repcode histories for block splitting, replace invalid repcodes (#2569) 2021-04-06 17:25:55 -04:00
senandGitHub e38124555e Fix dictionary force reloading clevel selection (#2570) 
* Move cdict clevel override to before localdict init

* Update results.csv after dict load changes
 2021-04-06 15:35:09 -04:00
senandGitHub 980f3bbf83 [cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546) 
* Perform 64-byte alignment of wksp tables and aligneds internally

* Clean up cwskp_finalize() function to only do two allocs

* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations

* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned

* Back out aligned/table reservation order restriction

* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
 2021-04-01 20:07:19 -04:00
senandGitHub 255925c231 Fix repcode-related OSS-fuzz issues in block splitter (#2560) 
* Do not emit last partitions of blocks as RLE/uncompressed

* Fix repcode updates within block splitter

* Add a entropytables confirm function, redo ZSTD_confirmRepcodesAndEntropyTables() for better function signature

* Add a repcode updater to block splitter, no longer need to force emit compressed blocks
 2021-03-31 15:14:59 -04:00
senandGitHub 84ccb81e7c Merge pull request #2561 from senhuang42/longlength_enum 
Add enum for representing long length ID
 2021-03-26 15:55:12 -04:00
Sen Huang b1a43455f8 Add enum for representing long length ID 2021-03-26 10:41:09 -07:00
senandGitHub ab216bc2c5 Merge pull request #2559 from senhuang42/add_dict_regression_tests_backup 
Add different dict modes to compression ratio regression test, update results.csv
 2021-03-25 19:26:06 -04:00
Sen Huang bbbd578f45 Update results.csv 2021-03-25 11:16:37 -07:00
senandGitHub 4fe2e7ae14 Merge pull request #2558 from senhuang42/msan_block_splitter_fix 
Fix block splitter minor MSAN warning.
 2021-03-25 13:51:43 -04:00
Sen Huang f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Sen Huang 1cadf86b39 Add tests to regression tests for dict 2021-03-25 10:39:08 -07:00
senandGitHub b0407b9f0e Merge pull request #2555 from senhuang42/default_clevel_func 
Add ZSTD_defaultCLevel() function to public API
 2021-03-25 13:07:28 -04:00
Sen Huang ef4e26bda5 Add clang msan fuzz test to github actions 2021-03-25 09:11:43 -07:00
Sen Huang 2a907bf4aa Move lastCountSize into a returned struct, fix MSAN error 2021-03-25 09:11:15 -07:00
Sen Huang e398744a35 Add ZSTD_defaultCLevel() function to public API 2021-03-25 08:04:00 -07:00