ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,973 Commits 17 Branches 77 Tags
b6c3459d50dac9d64cdc3bdbddbb9fed60b5b548
Commit Graph
3032 Commits
Author SHA1 Message Date
Sen Huang b6c3459d50 merge 2019-10-21 18:46:17 -04:00
Sen Huang 676f89902a Added multiplier, renamed new enum to something more useful 2019-10-21 15:36:12 -04:00
Sen Huang 1f3a51fb52 Updated forceAttachDict param bounds 2019-10-21 15:36:12 -04:00
Sen Huang 8f69c47643 Add enum to decision process 2019-10-21 15:36:12 -04:00
Sen Huang e4de8b098a Added support for forcing new CDict behavior and updated enum 2019-10-21 15:36:12 -04:00
Sen Huang 9294f4826b Changed to int from BYTE 2019-10-21 15:36:12 -04:00
Sen Huang f0fccc8847 Changed to int from BYTE 2019-10-21 15:36:12 -04:00
Sen Huang bb2df8c499 Trailing whitespace 2019-10-21 15:36:12 -04:00
Sen Huang cf51501d2f Fix test 2019-10-21 15:36:12 -04:00
Sen Huang ea3cb6988f Cast to BYTE to appease appveyor 2019-10-21 15:36:12 -04:00
Sen Huang a727a85a7e merge conflicts round 2 2019-10-21 15:36:12 -04:00
Sen Huang 053a35fd64 formatting 2019-10-21 15:35:33 -04:00
Sen Huang 3fa4daaa55 Fix error 2019-10-21 15:35:33 -04:00
Sen Huang 3328348c63 Add compressionlevel to cdict 2019-10-21 15:32:39 -04:00
Sen Huang e8aa3e486d Updated forceAttachDict param bounds 2019-10-20 22:01:08 -04:00
Sen Huang 6d297265f9 Add enum to decision process 2019-10-20 19:02:47 -04:00
Sen Huang 1daa898c93 Added support for forcing new CDict behavior and updated enum 2019-10-20 14:03:09 -04:00
Sen Huang 4455f00cb8 Changed to int from BYTE 2019-10-16 15:06:02 -04:00
Sen Huang 4f7d26b0ee Changed to int from BYTE 2019-10-16 15:05:29 -04:00
Sen Huang cf00ea367a Trailing whitespace 2019-10-16 10:31:27 -04:00
Sen Huang 8cb2174446 Fix test 2019-10-16 10:29:31 -04:00
Sen Huang 5e901b6f32 Cast to BYTE to appease appveyor 2019-10-15 13:58:44 -04:00
Sen Huang 5c010c9d2d merge conflicts round 2 2019-10-15 13:10:05 -04:00
Sen Huang a06b51879c merge conflict 2019-10-15 12:58:50 -04:00
Sen Huang 23dac23a49 formatting 2019-10-15 12:44:48 -04:00
Sen Huang 0c8df5c928 Fix error 2019-10-15 12:28:23 -04:00
Sen Huang a65eb39f9d Add compressionlevel to cdict 2019-10-15 10:22:06 -04:00
Yann ColletandGitHub fb77afc626 Merge pull request #1760 from bimbashrestha/extract_sequences_api 
Adding api for extracting sequences from seqstore
 2019-10-10 13:11:18 -07:00
Bimba Shrestha 36528b96c4 Manually moving instead of memcpy on decoder and using genBuffer() 2019-10-03 09:26:51 -07:00
Bimba Shrestha 61ec4c2e7f Cleaning sequence parsing logic 2019-10-03 06:42:40 -07:00
Yann Collet ad2a2785f7 bump version number to v1.4.4 
so that future reports on `dev` branch use this number instead
 2019-09-24 15:15:33 -07:00
Bimba Shrestha c04245b257 Replacing assert with memory_allocation error code throw 2019-09-23 15:42:16 -07:00
Bimba Shrestha be0bebd24e Adding test and null check for malloc 2019-09-23 15:08:18 -07:00
Dávid Bolvanský 1ab1a40c9c Fixed one more place 2019-09-23 21:32:56 +02:00
Dávid Bolvanský 1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
Nick TerrellandGitHub 7451c6578c Merge pull request #1804 from terrelln/wild-and-fast 
Optimize (de)compression and fix wildcopy overread
 2019-09-21 17:04:36 -07:00
Nick Terrell 5cb7615f1f Add UNUSED_ATTR to ZSTD_storeSeq() 2019-09-20 21:37:13 -07:00
Nick Terrell 5dc0a1d659 HINT_INLINE ZSTD_storeSeq() 
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
 2019-09-20 16:39:27 -07:00
Bimba Shrestha f3c4fd17e3 Passing in dummy dst buffer of compressbound(srcSize) 2019-09-20 15:50:58 -07:00
Felix HandteandGitHub c047fcf7bf Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
Nick Terrell 44c65da97e Remove literals overread in ZSTD_storeSeq() for ~neutral perf 2019-09-20 12:23:25 -07:00
W. Felix Handte f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Nick Terrell fde217df04 Fix bounds check in ZSTD_storeSeq() 2019-09-20 08:25:12 -07:00
Nick Terrell 67b1f5fc72 Fix too strict assert 2019-09-20 01:23:35 -07:00
Nick Terrell ddab2a94e8 Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy() 2019-09-20 00:56:20 -07:00
Nick Terrell cdad7fa512 Widen ZSTD_wildcopy to 32 bytes 2019-09-20 00:52:15 -07:00
Nick TerrellandNick Terrell efd37a64ea Optimize decompression and fix wildcopy overread 
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.
 2019-09-19 21:07:14 -07:00
Bimba Shrestha ae6d0e64ae Addressing comments 2019-09-19 15:25:20 -07:00
Yann ColletandGitHub 3cac061db5 Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2 
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…
 2019-09-18 16:32:37 -07:00
Bimba Shrestha 6e9f6813bb adding bit container size 2019-09-18 13:49:45 -07:00