ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,910 Commits 17 Branches 77 Tags
0ffae7e44061bd50aa76cff8b310d6da2f802ddc
Commit Graph
3000 Commits
Author SHA1 Message Date
W. Felix Handte 0ffae7e440 Stop Allocating Extra Space for Table Redzones 2019-10-10 13:40:16 -04:00
W. Felix Handte a07037b784 Don't Try to Redzone the Tables 2019-10-10 13:40:16 -04:00
W. Felix Handte 0cc481ef66 Fix Workspace Size Calculation 2019-10-10 13:40:16 -04:00
W. Felix Handte b6c0a02a17 Fix ZSTD_sizeof_matchState() Calculation 2019-10-10 13:40:16 -04:00
W. Felix Handte 8cffd6ed08 Avoid ASAN Failure in ZSTD_cwksp_free() 2019-10-10 13:40:16 -04:00
W. Felix Handte ef0b5707c5 Refactor Freeing CCtxes / CDicts Inside Workspaces 2019-10-10 13:40:16 -04:00
W. Felix Handte 143b296cf6 Surround Workspace Allocs with Dead Zone 2019-10-10 13:40:16 -04:00
W. Felix Handte 19a0955ec9 Add ZSTD_cwksp_alloc_size() to Help Calculate Needed Workspace Size 2019-10-10 13:40:16 -04:00
W. Felix Handte da88c35d41 Stop Assuming Tables are Adjacent 2019-10-10 13:40:16 -04:00
W. Felix Handte 35c30d6ca7 Poison Unused Workspace Memory 2019-10-10 13:40:16 -04:00
W. Felix Handte edb6d884a5 Detect Whether We're Being Compiled with ASAN 2019-10-10 13:40:16 -04:00
W. Felix Handte dc1fb684bf Remove Unused MEM_SKIP_MSAN Macro 2019-10-10 13:40:16 -04:00
Yann Collet ad2a2785f7 bump version number to v1.4.4 
so that future reports on `dev` branch use this number instead
 2019-09-24 15:15:33 -07:00
Dávid Bolvanský 1ab1a40c9c Fixed one more place 2019-09-23 21:32:56 +02:00
Dávid Bolvanský 1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
Nick TerrellandGitHub 7451c6578c Merge pull request #1804 from terrelln/wild-and-fast 
Optimize (de)compression and fix wildcopy overread
 2019-09-21 17:04:36 -07:00
Nick Terrell 5cb7615f1f Add UNUSED_ATTR to ZSTD_storeSeq() 2019-09-20 21:37:13 -07:00
Nick Terrell 5dc0a1d659 HINT_INLINE ZSTD_storeSeq() 
Clang on Mac wasn't inlining `ZSTD_storeSeq()` in level 1, which was
causing a 5% performance regression. This fixes it.
 2019-09-20 16:39:27 -07:00
Felix HandteandGitHub c047fcf7bf Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
Nick Terrell 44c65da97e Remove literals overread in ZSTD_storeSeq() for ~neutral perf 2019-09-20 12:23:25 -07:00
W. Felix Handte f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Nick Terrell fde217df04 Fix bounds check in ZSTD_storeSeq() 2019-09-20 08:25:12 -07:00
Nick Terrell 67b1f5fc72 Fix too strict assert 2019-09-20 01:23:35 -07:00
Nick Terrell ddab2a94e8 Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy() 2019-09-20 00:56:20 -07:00
Nick Terrell cdad7fa512 Widen ZSTD_wildcopy to 32 bytes 2019-09-20 00:52:15 -07:00
Nick TerrellandNick Terrell efd37a64ea Optimize decompression and fix wildcopy overread 
* Bump `WILDCOPY_OVERLENGTH` to 16 to fix the wildcopy overread.
* Optimize `ZSTD_wildcopy()` by removing unnecessary branches and
  unrolling the loop.
* Extract `ZSTD_overlapCopy8()` into its own function.
* Add `ZSTD_safecopy()` for `ZSTD_execSequenceEnd()`. It is
  optimized for single long sequences, since that is the important
  case that can end up in `ZSTD_execSequenceEnd()`. Without this
  optimization, decompressing a block with 1 long match goes
  from 5.7 GB/s to 800 MB/s.
* Refactor `ZSTD_execSequenceEnd()`.
* Increase the literal copy shortcut to 16.
* Add a shortcut for offset >= 16.
* Simplify `ZSTD_execSequence()` by pushing more cases into
  `ZSTD_execSequenceEnd()`.
* Delete `ZSTD_execSequenceLong()` since it is exactly the
  same as `ZSTD_execSequence()`.

clang-8 seeds +17.5% on silesia and +21.8% on enwik8.
gcc-9 sees +12% on silesia and +15.5% on enwik8.

TODO: More detailed measurements, and on more datasets.

Crdit to OSS-Fuzz for finding the wildcopy overread.
 2019-09-19 21:07:14 -07:00
Yann ColletandGitHub 3cac061db5 Merge pull request #1802 from bimbashrestha/rle_block_bound_fix_pt2 
Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from las…
 2019-09-18 16:32:37 -07:00
Bimba Shrestha 6e9f6813bb adding bit container size 2019-09-18 13:49:45 -07:00
Bimba Shrestha f9b6abb896 Adding 4 blocks to FSE_BLOCKBOUND() in lib/common (different from last week) 2019-09-18 13:29:05 -07:00
Yann ColletandGitHub bfff5b30a4 Merge pull request #1756 from mgrice/dev 
Improvements in zstd decode performance
 2019-09-18 11:35:50 -07:00
Yann Collet 243200e5bf minor refactor of ZSTD_fast 
- reduced variables lifetime
- more accurate code comments
 2019-09-17 14:02:57 -07:00
Felix HandteandGitHub 2164a130f3 Merge pull request #1780 from felixhandte/workspace-efficiency-3 
Avoid Clearing Tables Even When Changing CParams
 2019-09-16 14:37:05 -04:00
W. Felix Handte 72ea79cacd Don't Include sanitizer/msan_interface.h, Since Not All Platforms Provide It 
Instead, explicitly declare the functions we use.
 2019-09-16 12:08:03 -04:00
Nick Terrell fbeaf6989e [libzstd] Improve advanced API docs 2019-09-15 12:41:24 -07:00
Yann ColletandGitHub 09b1844d9b Merge pull request #1784 from bimbashrestha/fse_block_bound_err 
Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND()
 2019-09-12 19:09:27 -07:00
Bimba Shrestha fe9af338ed Added assert to BIT_flushBits() 2019-09-12 15:35:27 -07:00
Bimba Shrestha 43da5bf27e Rearranging assert and allowing 4 extra for FSE_BLOCKBOUND() 2019-09-12 14:43:50 -07:00
W. Felix Handte 20c69077d1 Shrink Table Valid End During Alloc Alignment / Phase Change 2019-09-11 17:14:59 -04:00
W. Felix Handte 51d90668ba Add Assertions to Confirm that Workspace Pointers are Correctly Ordered 2019-09-11 17:14:59 -04:00
W. Felix Handte a10c191613 __msan_poison() Workspace When Preparing for Re-Use 2019-09-11 17:14:45 -04:00
W. Felix Handte 7c57e2b9ca Zero h3size When h3log is 0 
This led to a nasty edgecase, where index reduction for modes that don't use
the h3 table would have a degenerate table (size 4) allocated and marked clean,
but which would not be re-indexed.
 2019-09-11 13:14:26 -04:00
W. Felix Handte bc020eec92 Also Shrink Clean Table Area When Reducing Indices 2019-09-11 11:40:57 -04:00
W. Felix Handte 1999b2ed9b Update DEBUGLOG Statements 2019-09-11 11:21:00 -04:00
W. Felix Handte 13e29a56de Shrink Clean Table Area When Copying Table Contents into Context 
The source matchState is potentially at a lower current index, which means
that any extra table space not overwritten by the copy may now contain
invalid indices. The simple solution is to unconditionally shrink the valid
table area to just the area overwritten.
 2019-09-11 11:18:45 -04:00
W. Felix Handte edb3ad053e Comments 2019-09-10 18:25:45 -04:00
W. Felix Handte f31ef28ff8 Only Reset Indexing in ZSTD_resetCCtx_internal() When Necessary 2019-09-10 18:25:45 -04:00
W. Felix Handte 9968a53e91 Remove No-Longer-Used Continuation Functions 2019-09-10 18:25:45 -04:00
W. Felix Handte 1b28e80416 Remove Fast Continue Path in ZSTD_resetCCtx_internal() 2019-09-10 18:25:45 -04:00
W. Felix Handte ad16eda5e4 ZSTD_reset_matchState Optionally Doesn't Restart Indexing 2019-09-10 18:25:45 -04:00
W. Felix Handte 5b10bb5ec3 Rename ZSTD_compResetPolicy_e Values and Add Comment 2019-09-10 18:25:45 -04:00