ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,465 Commits 17 Branches 77 Tags
ae47d503559659ef6e0878d30fdc7a6ed584ed49
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Bimba Shrestha ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba Shrestha 936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha 05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha 40414f984a Merge pull request #2036 from bimbashrestha/edist 
[contrib] Edit distance match finder
 2020-03-23 12:53:44 -05:00
Bimba Shrestha 91a62bad47 Merge pull request #2047 from bimbashrestha/doc 
[doc] targetLen -> targetLength
 2020-03-23 08:27:54 -05:00
Bimba Shrestha 22e7123a9a targetLen -> targetLength 2020-03-23 06:25:08 -07:00
Bimba Shrestha dbdeca1412 Edit distance match finder 2020-03-13 09:39:33 -05:00
Bimba Shrestha 66607d0eac Merge pull request #2033 from bimbashrestha/icc 
[opt] Small icc level 1 compression speed gain using #pragma vector
 2020-03-10 20:42:19 -05:00
Bimba Shrestha 5c05d419e8 Merge pull request #2031 from bimbashrestha/bitscan 
_BitScan{Reverse,Forward} add check for undefined
 2020-03-10 16:22:25 -05:00
Bimba Shrestha a89c45bdbd Typo 2020-03-10 15:19:48 -05:00
Bimba Shrestha 43fc88f443 Adding comment and remvoing ivdep 2020-03-10 14:57:27 -05:00
Bimba Shrestha 10f915f041 Merge pull request #2032 from bimbashrestha/no-size 
Adding --[no-]content-size to the cli
 2020-03-09 18:14:05 -05:00
Bimba Shrestha 9388dac2c0 Setting ZSTD_c_contentSizeFlag instead of UNKNOWN filesize 2020-03-09 15:40:18 -05:00
Bimba Shrestha 84d6bd4c7f Adding documentation 2020-03-09 14:50:49 -05:00
Bimba Shrestha fae64b3390 Adding test for --[no-]content-size 2020-03-09 14:44:38 -05:00
Bimba Shrestha 43874730f4 Adding --content-size 2020-03-09 14:19:05 -05:00
Bimba Shrestha 6d8e761ba3 Inverting content size prefs bool 2020-03-09 14:16:10 -05:00
Bimba Shrestha 03ee18d9c5 Adding (void)noContentSize 2020-03-09 14:00:52 -05:00
Bimba Shrestha 167244a64f --no-content-size cli 2020-03-09 13:07:29 -05:00
Bimba Shrestha dba3abc95a Missed returns 2020-03-05 12:20:59 -08:00
Bimba Shrestha a75e5f2ffc bitscan add undef check 2020-03-05 11:52:15 -08:00
Bimba Shrestha bc0e10ae78 Merge pull request #2030 from bimbashrestha/quick 
Removing no-tree-vectorize for intel
 2020-03-05 10:41:06 -08:00
Bimba Shrestha 85d0efd619 Removing no-tree-vectorize for intel 2020-03-05 10:02:48 -08:00
Bimba Shrestha 4c72a1a9c2 adding vector to main loop 2020-03-05 09:55:38 -08:00
Bimba Shrestha 11c1e5c9bc Merge pull request #2027 from bimbashrestha/travis-icc 
[ci] Adding icc make check to travis
 2020-03-03 17:01:47 -08:00
Bimba Shrestha 9405af6fa0 Merge pull request #2026 from bimbashrestha/issues-temp 
Add issue templates to zstd
 2020-03-03 15:52:41 -08:00
Bimba Shrestha c7060bb711 Removing mmx compiler flags 2020-03-03 15:46:11 -08:00
Bimba Shrestha 1bc56dfa4c Adding icc make check to travis ci 2020-03-03 15:22:35 -08:00
Bimba Shrestha fa9c0a4d78 Merge pull request #2025 from bimbashrestha/doc 
[doc] Fixing ZSTD_c_compressionLevel confusing note
 2020-03-03 13:43:29 -08:00
Bimba Shrestha cba46e9b7b Fixing ZSTD_c_compressionLevel confusing note 2020-03-03 13:12:02 -08:00
Bimba Shrestha 8974906129 Merge pull request #2019 from bimbashrestha/quick 
--show-default-cparams Adding hashLog
 2020-02-27 14:15:34 -08:00
Bimba Shrestha 23165b387e Adding hashLog to --show-default-cparams 2020-02-27 13:56:56 -08:00
Bimba Shrestha 0a00397e19 [cirrus-ci] Removing pkg -y update and using 11.3-snap instead of 11.2 (#2018) 
* Replacing with boilerplate from cirrus

* Adding task name and adding back freebsd 11.x

* Changing from 11.3 to freebsd-11-3-snap

* Rearanging the order of matrix and instance
 2020-02-27 13:53:03 -08:00
Bimba Shrestha c9ce6db069 Merge pull request #1999 from bket/fix_have_lz4 
Fix forgotten portable header prefix
 2020-02-20 10:55:28 -08:00
Bimba Shrestha b7b9d3fd34 Merge pull request #2002 from bimbashrestha/docs 
[doc] Adding notes about CI and an outline of collaboration workflow
 2020-02-20 10:54:04 -08:00
Bimba Shrestha 8f8c53d995 Remove redundant section and typo 2020-02-18 14:21:19 -08:00
Bimba Shrestha 042b28bd2c Outlining our current workflow 2020-02-18 13:03:52 -08:00
Bimba Shrestha 76c52b6b7c Adding steps for setting up travis and appveyor (and some general notes) 2020-02-16 11:31:17 -05:00
Bimba Shrestha 06a57cf57e [build-issue] More portable header prefix usage (#) (#1987) 
* make 4.3 build issue fix

* Changing header name and adding comment
 2020-02-06 14:10:51 -08:00
Bimba Shrestha f627e1a1ae Displaying level in --show-default-cparams (#1991) 
* Displaying level in --show-default-cparams

* Displaying actual enum value instead of level
 2020-02-06 13:51:43 -08:00
Bimba Shrestha 6a4258a08a Removing symbols already in unit tests and adding some new unit tests for missing symbols (#1985) 
* Removing symbols that are not being tested

* Removing symbols used in zstdcli, fileio, dibio and benchzstd

* Removing symbols used in zbuff and add test-zbuff to travis

* Removing remaining symbols and adding unit tests instead

* Removing symbols test entirely
 2020-02-05 16:55:00 -08:00
Bimba Shrestha 0cbafe3ce6 Merge pull request #1982 from bimbashrestha/quick 
Using appliedParams instead of "supplied" params in compressBegin()
 2020-02-03 16:48:21 -08:00
Bimba Shrestha 075783c1f0 Merge pull request #1980 from bimbashrestha/params 
Adding --show-default-cparams to cli
 2020-02-03 14:43:55 -08:00
Bimba Shrestha f1de05f348 Merge pull request #1981 from bimbashrestha/bug-fix 
Benchzstd multiple file decompression enable
 2020-02-03 14:40:26 -08:00
Bimba Shrestha d1389ac3ff Adding assert(NB_STRATEGIES == upperLimit) 2020-02-03 09:52:39 -08:00
Bimba Shrestha 80c26117a9 Line-wrapping 2020-02-03 09:38:16 -08:00
Bimba Shrestha ee8a712af3 Using appliedParams instead of supplied params 2020-01-31 15:49:07 -08:00
Bimba Shrestha f33baa21c6 Removing assert and changing ratio cSize 2020-01-31 11:54:14 -08:00
Bimba Shrestha 19473390d1 Adding (void)ZSTD_strategyMap 2020-01-31 10:54:02 -08:00
Bimba Shrestha 38d010eea0 Adding static assert and using it in cli 2020-01-31 10:47:17 -08:00
Bimba Shrestha 0fa6c90e89 Moving to same category as -v 2020-01-31 10:38:19 -08:00
Bimba Shrestha f4ae6c0b94 Moving documentation and elaborating 2020-01-30 17:54:42 -08:00
Bimba Shrestha 9d9cfa9ef9 Adding assert and using ZSTD_STRATEGY_MAX as ZSTD_NB_STRATEGIES 2020-01-30 17:54:01 -08:00
Bimba Shrestha 6ad2f0d7d6 Adding back one cast and using UTIL_FILESIZE_UNKNOWN to check unknown file size 2020-01-30 17:16:02 -08:00
Bimba Shrestha 6bf05a53ab Documenting in the man file 2020-01-30 17:11:38 -08:00
Bimba Shrestha c203341292 Removing more casts 2020-01-30 17:07:58 -08:00
Bimba Shrestha 2a0771cfdb Removing unnecessary cast 2020-01-30 17:05:33 -08:00
Bimba Shrestha b0ceab4dc0 Moving default cparams back, checking for unknown file size, using getCParams() 2020-01-30 16:52:07 -08:00
Bimba Shrestha b660ef87f8 Convertion nit U32 -> size_t 2020-01-30 14:29:04 -08:00
Bimba Shrestha 71754a26f8 Update comment 2020-01-30 14:14:27 -08:00
Bimba Shrestha 2f10019b92 Adding --show-default-cparams (show cparams before compressing 2020-01-30 14:12:03 -08:00
Bimba Shrestha 3b10060b71 Merge pull request #1978 from bimbashrestha/quick 
[Typo][fix] make benchmarking typo fix
 2020-01-28 22:11:51 -08:00
Bimba Shrestha 9b049836c9 Typo baseline_build -> baseline_label 2020-01-28 21:39:20 -08:00
Bimba Shrestha 8fe562a770 [automated_benchmarking] Make arguments optional and add --dict argument (#1968) 
* Make arugments optional and add --dict argument

* Removing accidental print statement

* Change to more likely scenario for dictionary compression benchmark
 2020-01-28 11:29:43 -08:00
Bimba Shrestha f25a6e9f8f Adding new cli endpoint --patch-from= (#1940) 
* Adding new cli endpoint --diff-from=

* Appveyor conversion nit

* Using bool set trick instead of direct set

* Removing --diff-from and only leaving --diff-from=#

* Throwing error when both dictFileName vars are set

* Clean up syntax

* Renaming diff-from to patch-from

* Revering comma separated syntax clean up

* Updating playtests with patch-from

* Uncommenting accidentally commented

* Updating remaining docs and var names to be patch-from instead of diff-from

* Constifying

* Using existing log2 function and removing newly created one

* Argument order (moving prefs to end)

* Using comma separated syntax

* Moving to outside #ifndef
 2020-01-10 14:25:24 -08:00
Bimba Shrestha eb76f786bc [bench] Automated benchmarking script (#1906) 
* Initial revised automated benchmarking script

* Updating nb_iterations and making loop infinite

* Allowing benchmarking params to be changed from cli

* Renaming old speed test

* Removing numpy dependency for cli

* Change filename and benchmakr on pr level

* Moving build outside loop and adding iterations param

* Moving benchmarking to seperate travis ci test

* Fixing typo and using unused variable

* Added mode labels and updated README accordingly

* Adding new mode 'current' that compraes facebook:dev against current hash

* Typo

* Reverting previous accidental diff

* Typo

* Adding frequency config variable to prevent github from blacklisting

* Added new argument for frequency of fetching new prs

* Updating documentation
 2020-01-06 14:19:11 -08:00
Bimba Shrestha b1f53b1a10 [fuzz] Dividing by targetCBlockSize instead of blockSize for nbBlocks fit (#1936) 
* Adding fail logging for superblock flow

* Dividing by targetCBlockSize instead of blockSize

* Adding new const and using more acurate formula for nbBlocks

* Only do dstCapacity check if using superblock

* Remvoing disabling logic

* Updating test to make it catch more extreme case of previou bug

* Also updating comment

* Only taking compressEnd shortcut on non-superblock
 2020-01-03 16:53:51 -08:00
Bimba Shrestha ef1684f29a [doc] Adding -M# decompression operation modifier documentation (#1938) 
* Adding -M# operation modifier documentation

* Adding long name, typo and update desc
 2020-01-03 16:36:47 -08:00
Bimba Shrestha 56415efc76 Constifying, malloc check and naming nit 2019-12-17 17:16:51 -08:00
Bimba Shrestha 989ce13e19 One more type conversion 2019-12-13 16:50:21 -08:00
Bimba Shrestha 4399eed42e Adding explict cast to satisfy appveyor ci 2019-12-13 16:38:11 -08:00
Bimba Shrestha db5124ef6e More void* issues. Just replacing with BYTE* 2019-12-13 16:24:49 -08:00
Bimba Shrestha 49b2bf7106 'void* size issue' fix 2019-12-13 16:06:57 -08:00
Bimba Shrestha 5225dcfc0f Adding bool to check if enough room left for noCompress superblocks 2019-12-13 15:47:28 -08:00
Bimba Shrestha e3cd2785e2 Add test to catch too many noCompress superblocks on streaming 2019-12-13 15:31:29 -08:00
Bimba Shrestha e1913dc87f Making const, removing unnecessary indent, changing parameter order 2019-12-04 15:51:17 -08:00
Bimba Shrestha 2ec556fec2 Moving init/end functions, moving compressSuperBlock inside body() 2019-12-04 15:23:13 -08:00
Bimba Shrestha ffb0463041 Refactor 2019-12-04 14:52:27 -08:00
Bimba Shrestha 49c6d49247 [fuzz] msan uninitialized unsigned value (#1908) 
Fixes new fuzz issue

Credit to OSS-Fuzz

* Initializing unsigned value

* Initialilzing to 1 instead of 0 because its more conservative

* Unconditionoally setting to check first and then checking zero

* Moving bool to before block for c90

* Move check set before block
 2019-12-04 10:02:17 -08:00
Bimba Shrestha 1fc9352f81 Using bss var instead of creating new bool 2019-12-02 21:39:06 -08:00
Bimba Shrestha 1f681d8592 Merge branch 'oss' of https://github.com/bimbashrestha/zstd into oss 2019-11-27 10:56:54 -08:00
Bimba Shrestha a3a3c62b81 [fuzz] Only set HUF_repeat_valid if loaded table has all non-zero weights (#1898) 
Fixes a fuzz issue where dictionary_round_trip failed because the compressor was generating corrupt files thanks to zero weights in the table.

* Only setting loaded dict huf table to valid on non-zero

* Adding hasNoZeroWeights test to fse tables

* Forbiding nbBits != 0 when weight == 0

* Reverting the last commit

* Setting table log to 0 when weight == 0

* Small (invalid) zero weight dict test

* Small (valid) zero weight dict test

* Initializing repeatMode vars to check before zero check

* Removing FSE changes to seperate pr

* Reverting accidentally changed file

* Negating bool, using unsigned, optimization nit
 2019-11-26 12:24:19 -08:00
Bimba Shrestha d4e17d0776 Negating bool, updating bool on inner branches 2019-11-26 12:17:43 -08:00
Bimba Shrestha 826b555463 Merge branch 'dev' into oss 2019-11-22 17:29:33 -08:00
Bimba Shrestha 707a12c419 Test enough room for checksum in superblock 2019-11-22 17:25:36 -08:00
Bimba Shrestha 10bce1919e Mixed declration fix 2019-11-21 13:08:27 -08:00
Bimba Shrestha 0451accab1 Checking noCompressBlock explicitly for rep code confirmation 2019-11-21 13:06:26 -08:00
Bimba Shrestha 8f0c2d04c8 Going back to original flow but removing else return 2019-11-19 10:03:07 -08:00
Bimba Shrestha b032a6e1d9 Updated some more outdated bounds 2019-11-19 09:55:13 -08:00
Bimba Shrestha 80586f5e80 Reversing condition order and forwarding error 2019-11-18 13:53:55 -08:00
Bimba Shrestha dade64428f Output regular uncompressed block when compressSequences fails 2019-11-18 08:43:14 -08:00
Bimba Shrestha 2d5d961a60 Typo in comment 2019-11-15 19:00:53 -08:00
Bimba Shrestha dba767c0bb Leaving room for checksum 2019-11-15 18:44:51 -08:00
Bimba Shrestha 982830b081 Fixing hashLog error 2^26=64Mib 2019-11-14 10:48:55 -08:00
Bimba Shrestha 3fb5b106da Replacing some literals with constants 2019-11-05 10:26:57 -08:00
Bimba Shrestha 28d6773d75 Merge branch 'named_pipes' of https://github.com/bimbashrestha/zstd into named_pipes 2019-10-29 09:46:28 -07:00
Bimba Shrestha 4a9eca4b9d Removing merge side effect 2019-10-29 09:45:28 -07:00
Bimba Shrestha da73b11b14 Merge branch 'dev' into named_pipes 2019-10-29 01:17:13 -04:00
Bimba Shrestha 66f580ca73 Removing Visual08 and Visual10 tests 2019-10-28 22:09:34 -07:00