ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,691 Commits 17 Branches 77 Tags
13a271c420f7225f14c40795c120e5a95122d915
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Bimba ShresthaandW. Felix Handte 80053bdae3 updating cold benchmark 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 9c628238d3 creating ZSTD_createCDict_advanced_internal 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 0a9787c3e1 changing to int for consistency 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte e29bc3a009 using dict mls instead of src mls 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 145c2d12f9 add hashtable head prefetching 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 5d5507788d change method name for consistency 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte b30f71becf pass correct cparams 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte a3f6e4026e removing wrong comment 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 71fda0362f making cctxParams a pointer 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 628559d0e4 loading dict using new algorithm 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 22705f0c93 adding dedicatedDictSearch algorithm 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 31e581bf65 adding enableDedicatedDictSearch to matchState_t 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 50550a14ad adding dedicated dict load method to lazy 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte 75b6360036 adding ZSTD_createCDict_advanced2 to zstd.h 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte b7dddbe89b always attach dict when using dedicatedDictSearch 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte e36a373df4 adding dedicatedDictSearch cParams helper methods 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte f10d4e313c adding ZSTD_dedicatedDictSearch_defaultCParameters variable 2020-09-10 18:51:52 -04:00
Bimba ShresthaandW. Felix Handte c497cb6716 Add ZSTD_c_enableDedicatedDictSearch Param 2020-09-10 18:51:52 -04:00
Bimba Shrestha a944fc8620 removing icc from travis 2020-08-13 12:47:04 -05:00
Bimba Shrestha b413dac52b adding icc to actions 2020-08-13 12:45:28 -05:00
Bimba Shrestha 2598af451f removing min decomp 2020-07-08 11:11:45 -07:00
Bimba ShresthaandGitHub f03192c122 Merge pull request #2208 from bimbashrestha/arm-actions 
[actions] adding arm to actions
 2020-06-18 17:18:31 -05:00
Bimba Shrestha 4d6e4a2978 adding arm to actions 2020-06-18 12:28:37 -07:00
Bimba ShresthaandGitHub c496fd261d Merge pull request #2206 from bimbashrestha/stream-size 
[cli] Allowing stdin on --patch-from using --stream-size
 2020-06-18 13:10:11 -05:00
Bimba Shrestha de48f35306 adding --patch-from --stream-size test 2020-06-18 10:28:37 -07:00
Bimba Shrestha b06710889f error when not using --stream-size on pipe 2020-06-18 09:31:06 -07:00
Bimba Shrestha 6653321ba3 passing streamSrcSize when set 2020-06-18 09:28:18 -07:00
Bimba ShresthaandGitHub 4f59be7410 Merge pull request #2205 from bimbashrestha/cmake 
Removing de-duplication of cmake flags
 2020-06-15 19:41:55 -05:00
Bimba Shrestha 0fa9406ae2 removing deduplicatin of flags 2020-06-15 14:58:53 -07:00
Bimba ShresthaandGitHub 4c144cf9c5 Merge pull request #2193 from bimbashrestha/typos 
Fixing some typos
 2020-06-05 12:00:04 -05:00
Bimba Shrestha e2838d9eb9 Spelling mistakes 2020-06-05 05:11:26 -05:00
Bimba ShresthaandGitHub 448658a662 Merge pull request #2187 from bimbashrestha/bourne 
[ci] Adding bourne shell compatibility test
 2020-06-02 13:07:36 -05:00
Bimba ShresthaandGitHub 00ac0415ac Merge pull request #2186 from bimbashrestha/dll-install 
[cmake] Fix dll install location
 2020-06-02 12:32:12 -05:00
Bimba Shrestha eee59be6a8 excluding ls | grep warning 2020-06-02 09:15:24 -07:00
Bimba Shrestha b0f851675a [shellcheck] setting if unset 2020-06-02 09:12:50 -07:00
Bimba Shrestha 151deaf143 [shellcheck] adding quotes to expansion 2020-06-02 09:12:13 -07:00
Bimba Shrestha 12185ec5e7 adding shellcheck bourne test 2020-06-02 09:08:16 -07:00
Bimba Shrestha 0ce5f70ca8 newline lints (ignore) 2020-06-02 09:07:45 -07:00
Bimba Shrestha e75db971fc DLL install fix for https://github.com/facebook/zstd/issues/2160 
Credit goes to @xantares

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
 2020-06-02 07:45:06 -07:00
Bimba ShresthaandGitHub 7f248af036 Merge pull request #2180 from facebook/generic-release 
[actions] Migrating 2nd batch of tests from Travis to Github Actions
 2020-05-26 16:30:11 -05:00
Bimba ShresthaandGitHub 4ae5cb1ec9 Create generic-release.yml 2020-05-26 15:12:00 -05:00
Bimba ShresthaandGitHub b1a269437a Merge pull request #2137 from facebook/generic-dev 
[actions] Migrating 1st batch of tests from Travis to Github Actions
 2020-05-26 13:03:17 -05:00
Bimba ShresthaandGitHub c96151a25d Merge pull request #2178 from bimbashrestha/quick-fix 
Change --patch-from limit from 4gb to 2gb
 2020-05-26 12:26:22 -05:00
Bimba Shrestha f847909cf2 Using ZSTD_MAX_WINDOWLOG instead of INT_MAX 2020-05-26 09:23:26 -07:00
Bimba Shrestha 01b38ef162 Change --patch-from limit from 4gb to 2gb 2020-05-26 06:52:30 -07:00
Bimba ShresthaandGitHub 255e5e3f56 [fuzz] Adding dictionary_stream_round_trip fuzzer (#2140) 
* Adding dictionary_stream_round_trip

* fixing memory leak
 2020-05-15 13:33:31 -07:00
Bimba ShresthaandGitHub a49a4852c2 Migrating first batch of tests from travis 
# Dev PR jobs that still have to be migrated from travis 
# 
# icc (need self-hosted)
# arm/qemu-arm (need self-hosted)
# versionTag 
# valgrindTest (keeps failing for some reason. need investigation)
# staticAnalyze (need trusty so need self-hosted)
# pcc-fuzz: (need trusty so need self-hosted)
# arm-build-test (need self-hosted)
 2020-05-14 12:35:37 -05:00
Bimba ShresthaandGitHub 0208eeccd6 Merge pull request #2135 from bimbashrestha/grep 
Revert to old zstdgrep because of potential license issue
 2020-05-13 16:08:59 -05:00
Bimba Shrestha 12071467d3 reverting docs and test 2020-05-13 15:22:07 -05:00
Bimba Shrestha 9b13077cfb Revert to old zstdgrep 2020-05-13 15:12:14 -05:00
Bimba Shrestha 0453cfa8f5 removing -f test (grep usage not supported on mac) 2020-05-12 15:18:43 -05:00
Bimba ShresthaandGitHub e7d2391e9a [doc] measuring performance docs (#2117) 
* performance measuring docs

* spelling

* combining advanced and simple section

* zstd benchmark title change
 2020-05-11 18:55:01 -07:00
Bimba ShresthaandGitHub 0c04e641e9 Merge pull request #2127 from bimbashrestha/docs1 
[doc] documenting non-supported grep options for zstdgrep to README
 2020-05-11 12:51:35 -05:00
Bimba Shrestha ef467fd428 documenting non-supported grep options for zstdgrep 2020-05-11 12:22:45 -05:00
Bimba ShresthaandGitHub 602dae3509 Merge pull request #2119 from bimbashrestha/mingw 
[build] Adding $TARGET_SYSTEM for cross-compilation Fedora fix
 2020-05-08 10:54:06 -05:00
Bimba Shrestha df9e5b6f4c adding 2020-05-07 22:07:40 -05:00
Bimba ShresthaandGitHub 625924774e Merge pull request #2115 from bimbashrestha/docs 
[doc] Adding static analysis section and note
 2020-05-07 12:03:33 -05:00
Bimba Shrestha 1b1cc3d1b7 adding some static-anslysis docs 2020-05-07 10:38:06 -05:00
Bimba ShresthaandGitHub e7df0d41bb Merge pull request #2095 from bimbashrestha/grep 
[bugs] zstdgrep/grep inconsistencies
 2020-05-06 11:18:15 -05:00
Bimba Shrestha 250184adf6 adding tests back 2020-05-05 16:51:06 -07:00
Bimba Shrestha 6ff16b4246 adding help text 2020-05-05 10:03:57 -07:00
Bimba Shrestha a06a508bf3 fixing inconsistencies 2020-05-05 09:56:15 -07:00
Bimba ShresthaandGitHub 0301ef5d04 [bench] Extending largeNbDicts to compression (#2089) 
* adding cdict_collection_t

* adding shuffleCDictionaries()

* adding compressInstructions

* adding compress()

* integrating compression into bench()

* copy paste error fix

* static analyzer uninit value complaint fix

* changing to control

* removing assert

* changing to control

* moving memcpy to seperate function

* fixing static analyzer complaint

* another hacky solution attempt

* Copying createbuffer logic
 2020-05-04 10:42:22 -07:00
Bimba ShresthaandGitHub 0eaaa4d3a7 Merge pull request #2101 from bimbashrestha/meson 
[build] fix meson build issue
 2020-05-01 12:52:37 -05:00
Bimba Shrestha 834a8f838a meson msvc build fix 2020-05-01 09:04:09 -05:00
Bimba ShresthaandGitHub 6b4a3e019f Merge pull request #2088 from bimbashrestha/bug 
[bug] Making compressStream2 fail when passing rawContent but claiming fullDict
 2020-04-23 14:16:56 -05:00
Bimba ShresthaandGitHub 861a73744d Merge pull request #2086 from bimbashrestha/quick3 
[bug] passing cParams.hashLog instead of comprParams->hashLog for ldm trigger
 2020-04-23 13:02:57 -05:00
Bimba Shrestha 1875f616ce passing dictContentType instead of rawContent every time 2020-04-21 22:29:35 -07:00
Bimba Shrestha f7a7409a49 adding fail test when passing wrong fullDict using refPrefix 2020-04-21 22:26:48 -07:00
Bimba Shrestha dba02245bf bash to shell conversion 2020-04-21 20:31:11 -07:00
Bimba Shrestha 659ff85cf5 changing to cParams.hashLog 2020-04-21 21:12:50 -05:00
Bimba Shrestha 0b107188b8 adding test for long mode trigger 2020-04-21 21:09:49 -05:00
Bimba ShresthaandGitHub 38a6d2a8f2 Merge pull request #2085 from bimbashrestha/quick2 
[doc-bug] Updating the recommended max chainlog for patch-from
 2020-04-20 14:09:23 -05:00
Bimba Shrestha b640802ff8 updating chainlog bound 2020-04-20 11:34:24 -07:00
Bimba ShresthaandGitHub c6311584aa Merge pull request #2083 from bimbashrestha/quick1 
[nit] Adding missing newline at end of patch-from recommendations
 2020-04-20 13:24:34 -05:00
Bimba Shrestha 587a20a12a adding newline 2020-04-20 10:25:58 -07:00
Bimba ShresthaandGitHub 5b0a452cac Adding --long support for --patch-from (#1959) 
* adding long support for patch-from

* adding refPrefix to dictionary_decompress

* adding refPrefix to dictionary_loader

* conversion nit

* triggering log mode on chainLog < fileLog and removing old threshold

* adding refPrefix to dictionary_round_trip

* adding docs

* adding enableldm + forceWindow test for dict

* separate patch-from logic into FIO_adjustParamsForPatchFromMode

* moving memLimit adjustment to outside ifdefs (need for decomp)

* removing refPrefix gate on dictionary_round_trip

* rebase on top of dev refPrefix change

* making sure refPrefx + ldm is < 1% of srcSize

* combining notes for patch-from

* moving memlimit logic inside fileio.c

* adding display for optimal parser and long mode trigger

* conversion nit

* fuzzer found heap-overflow fix

* another conversion nit

* moving FIO_adjustMemLimitForPatchFromMode outside ifndef

* making params immutable

* moving memLimit update before createDictBuffer call

* making maxSrcSize unsigned long long

* making dictSize and maxSrcSize params unsigned long long

* error on files larger than 4gb

* extend refPrefix test to include round trip

* conversion to size_t

* making sure ldm is at least 10x better

* removing break

* including zstd_compress_internal and removing redundant macros

* exposing ZSTD_cycleLog()

* using cycleLog instead of chainLog

* add some more docs about user optimizations

* formatting
 2020-04-17 15:58:53 -05:00
Bimba ShresthaandGitHub c0d4b2b5a3 Merge pull request #2075 from bimbashrestha/dict_fuzzer_ref 
[bug] handling case where prefix is NULL or 0 sized in refPrefix_advanced
 2020-04-07 17:37:19 -05:00
Bimba Shrestha 1658ae75cd handling nil case for refprefix 2020-04-07 14:41:53 -07:00
Bimba Shrestha 794f03459e adding refPrefix 2020-04-06 22:57:49 -07:00
Bimba ShresthaandGitHub 2847d1a606 Merge pull request #2069 from bimbashrestha/doversize 
[memory opt] Downsize decompression buffers when large for too long
 2020-04-06 16:06:37 -05:00
Bimba Shrestha 31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba ShresthaandGitHub ebc422b0b7 Merge pull request #2068 from bimbashrestha/quick 
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
 2020-04-03 19:28:42 -05:00
Bimba Shrestha 0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha 0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha 3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba ShresthaandGitHub 009a92f749 Merge pull request #2067 from bimbashrestha/res-leak 
[bug] adding fclose before return
 2020-04-03 15:51:59 -05:00
Bimba Shrestha 1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
Bimba Shrestha d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
Bimba Shrestha 936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha 05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha d0412f3aba no percentage on readsize == 0 2020-04-03 12:10:02 -07:00
Bimba Shrestha bf1856c26f removing max(1, ..) 2020-04-03 12:07:17 -07:00
Bimba ShresthaandGitHub 40414f984a Merge pull request #2036 from bimbashrestha/edist 
[contrib] Edit distance match finder
 2020-03-23 12:53:44 -05:00
Bimba ShresthaandGitHub 91a62bad47 Merge pull request #2047 from bimbashrestha/doc 
[doc] targetLen -> targetLength
 2020-03-23 08:27:54 -05:00
Bimba Shrestha 22e7123a9a targetLen -> targetLength 2020-03-23 06:25:08 -07:00
Bimba Shrestha dbdeca1412 Edit distance match finder 2020-03-13 09:39:33 -05:00
Bimba ShresthaandGitHub 66607d0eac Merge pull request #2033 from bimbashrestha/icc 
[opt] Small icc level 1 compression speed gain using #pragma vector
 2020-03-10 20:42:19 -05:00