ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,624 Commits 17 Branches 77 Tags
cd9d0a7e6e5aa770b377d738eb94de6671c5487e
Commit Graph
9624 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Dávid Bolvanský 1ab1a40c9c Fixed one more place 2019-09-23 21:32:56 +02:00
Dávid Bolvanský 1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
Nick TerrellandGitHub 7451c6578c Merge pull request #1804 from terrelln/wild-and-fast 
Optimize (de)compression and fix wildcopy overread
 2019-09-21 17:04:36 -07:00
Nick Terrell 5cb7615f1f Add UNUSED_ATTR to ZSTD_storeSeq() 2019-09-20 21:37:13 -07:00
Nick Terrell 5dc0a1d659 HINT_INLINE ZSTD_storeSeq() 
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
 2019-09-20 16:39:27 -07:00
Bimba Shrestha f3c4fd17e3 Passing in dummy dst buffer of compressbound(srcSize) 2019-09-20 15:50:58 -07:00
Felix HandteandGitHub c047fcf7bf Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
Nick Terrell 44c65da97e Remove literals overread in ZSTD_storeSeq() for ~neutral perf 2019-09-20 12:23:25 -07:00
W. Felix Handte f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Nick Terrell fde217df04 Fix bounds check in ZSTD_storeSeq() 2019-09-20 08:25:12 -07:00
Nick Terrell 67b1f5fc72 Fix too strict assert 2019-09-20 01:23:35 -07:00
Nick Terrell e068bd01df [tests] Fix decodecorpus 2019-09-20 01:09:47 -07:00
Nick Terrell ddab2a94e8 Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy() 2019-09-20 00:56:20 -07:00
Nick Terrell cdad7fa512 Widen ZSTD_wildcopy to 32 bytes 2019-09-20 00:52:15 -07:00
Nick TerrellandNick Terrell efd37a64ea Optimize decompression and fix wildcopy overread 
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.
 2019-09-19 21:07:14 -07:00
Bimba Shrestha ae6d0e64ae Addressing comments 2019-09-19 15:25:20 -07:00
Nick TerrellandGitHub 0e76000dee Merge pull request #1801 from terrelln/int-max 
[test] Test the bounds of ZSTD_c_srcSizeHint
 2019-09-19 11:10:13 -07:00
Yann ColletandGitHub 3cac061db5 Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2 
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…
 2019-09-18 16:32:37 -07:00
Bimba Shrestha 6e9f6813bb adding bit container size 2019-09-18 13:49:45 -07:00
Bimba Shrestha f9b6abb896 Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week) 2019-09-18 13:29:05 -07:00
Yann ColletandGitHub bfff5b30a4 Merge pull request #1756 from mgrice/dev 
Improvements in zstd decode performance
 2019-09-18 11:35:50 -07:00
Nick Terrell 51990246c3 [test] Test the bounds of ZSTD_c_srcSizeHint 2019-09-18 11:05:08 -07:00
Sergey Ponomarev dafe796e39 #1790 short tar's extensions tgz, txz, tlz4m .tzst should be decompressed with .tar suffix 2019-09-18 09:23:10 +03:00
Sergey Ponomarev bfb4d830b2 FIO_determineDstName: extract dstFileNameEndPos variable 2019-09-18 09:21:00 +03:00
Bimba Shrestha 3cacc0a30b Casting void pointer to ZSTD_Sequence pointer 2019-09-17 17:44:08 -07:00
Yann ColletandGitHub 5329de1f1e Merge pull request #1798 from facebook/refac_fast 
minor refactor of ZSTD_fast
 2019-09-17 14:54:23 -07:00
Yann Collet 243200e5bf minor refactor of ZSTD_fast 
- reduced variables lifetime
- more accurate code comments
 2019-09-17 14:02:57 -07:00
Felix HandteandGitHub dd2838eeb4 Merge pull request #1783 from felixhandte/mtime-nsec 
Set Mod Time Nanoseconds
 2019-09-17 13:30:21 -04:00
Bimba Shrestha 76fea3fb99 Resolving appveyor test failure implicit conversion 2019-09-16 14:02:23 -07:00
Bimba Shrestha 5b038f128f Merge branch 'extract_sequences_api' of https://github.com/bimbashrestha/zstd into extract_sequences_api 2019-09-16 13:35:49 -07:00
Bimba Shrestha 1f93be0f6d Handling memory leak and potential side effect 2019-09-16 13:35:45 -07:00
Bimba ShresthaandGitHub a874435478 Merge branch 'dev' into extract_sequences_api 2019-09-16 13:29:59 -07:00
Felix HandteandGitHub 2164a130f3 Merge pull request #1780 from felixhandte/workspace-efficiency-3 
Avoid Clearing Tables Even When Changing CParams
 2019-09-16 14:37:05 -04:00
W. Felix Handte 72ea79cacd Don't Include sanitizer/msan_interface.h, Since Not All Platforms Provide It 
Instead, explicitly declare the functions we use.
 2019-09-16 12:08:03 -04:00
Bimba Shrestha bff6072e3a Bailing early when collecting sequences and documentation 2019-09-16 08:26:21 -07:00
Dmitri Shubin 4607f80f5a Don't hardcode installation directories in CMakeLists.txt 
Use paths provided by GNUInstallDirs module instead.
 2019-09-16 11:36:37 +03:00
Nick TerrellandGitHub 282ac22b8a Merge pull request #1791 from terrelln/doc-up 
[libzstd] Improve advanced API docs
 2019-09-15 14:50:55 -07:00
Nick Terrell fbeaf6989e [libzstd] Improve advanced API docs 2019-09-15 12:41:24 -07:00
Sergey Ponomarev 59f369a6da Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst) 2019-09-14 21:30:15 +03:00
Sergey Ponomarev a101721f4e Use one strstr() call instead of chain of strcmp() 2019-09-14 21:26:27 +03:00
Sergey Ponomarev 7d9cd22e21 #754 Add a hint about -o option 2019-09-14 21:23:47 +03:00
Sergey Ponomarev 8cc815a941 #754 sufixlist->SUFFIX_LIST 2019-09-14 21:15:24 +03:00
Sergey Ponomarev b804dd3e5b #754 move sufixlist upper and improve error message on missing suffix 2019-09-14 21:14:43 +03:00
Sergey Ponomarev a4ca246eca build/cmake/README.md: improve 2019-09-14 21:10:42 +03:00
Nick TerrellandGitHub f2941db4a9 Merge pull request #1789 from terrelln/larger-fuzz 
[fuzz] Fix leak in block_round_trip
 2019-09-13 14:13:34 -07:00
Nick Terrell d721fcf3ee [fuzz] Fix leak in block_round_trip 2019-09-13 10:32:38 -07:00
Yann ColletandGitHub 09b1844d9b Merge pull request #1784 from bimbashrestha/fse_block_bound_err 
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
 2019-09-12 19:09:27 -07:00
Nick TerrellandGitHub 23c5df1052 Merge pull request #1785 from terrelln/larger-fuzz 
[fuzz] Generate seed data up to 256KB
 2019-09-12 17:21:10 -07:00
Bimba Shrestha fe9af338ed Added assert to BIT_flushBits() 2019-09-12 15:35:27 -07:00
Nick Terrell 7c4578160e [fuzz] Generate seed data up to 256KB 2019-09-12 15:02:01 -07:00