ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9,290 Commits 17 Branches 77 Tags
8a724f91cbe256e981e018f9a1002e8430cd6c51
Commit Graph
9290 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 8a724f91cb minor changelog update 2021-12-16 01:07:15 -08:00
Yann Collet 7a347769dc minor changelog update 2021-12-15 21:45:30 -08:00
Elliot GorokhovskyandGitHub c5f1e826ca Merge pull request #2925 from embg/dict_training_sample_limit_size 
Allow user to specify memory limit for dictionary training
 2021-12-15 15:58:17 -05:00
Yann Collet 622fee6140 minor changelog update, for clarity 2021-12-15 11:56:11 -08:00
Yann ColletandGitHub eaf786242d Merge pull request #2929 from facebook/sse_row_lazy 
simplify SSE implementation of row_lazy match finder
 2021-12-15 11:47:15 -08:00
Felix HandteandGitHub d5933a4009 Merge pull request #2930 from nolange/reduce_tables_to_8bit 
Reduce tables to 8bit
 2021-12-15 12:43:25 -05:00
Norbert Lange 2fbb1d10c1 Reduce bit tables to 8bit 
This saves some 1.7Kb in rodata section (x86_64, zstd tool),
while assembler code stays the same except
the type of a few load/extend instructions.

Should not have negative performance implications.
 2021-12-14 23:47:57 +01:00
Norbert Lange 99923dfc1a Add typedefs for 8bit (un)signed 
To make code more expressive, add U8 and S8 typedefs
 2021-12-14 23:47:57 +01:00
binhdvoandGitHub 64205b7832 Fix performance degradation with -m32 (#2926) 2021-12-14 15:53:50 -05:00
Elliot Gorokhovsky 71c0c07c19 Allow user to specify memory limit for dictionary training 2021-12-14 14:29:01 -05:00
Felix HandteandGitHub 5e2fede604 Merge pull request #2921 from felixhandte/neg-lvl-stagger-step 
Stagger Stepping in Negative Levels
 2021-12-14 14:13:57 -05:00
Yann Collet 05430b25a8 roll SSE implementation of row_lazy match finder 
mostly for maintenance convenience.

Performance wise, there is very little change,
slightly faster for slog 3 & 4,
neutral or very slightly negative for slot 5 & 6.
 2021-12-14 10:44:23 -08:00
W. Felix Handte 450fca9704 Update Regression Tests w/ New Sizes 2021-12-13 17:29:32 -05:00
W. Felix Handte 82a49c88f9 Increment Step by 1 not 2 
I couldn't find a good way to spread `ip0` and `ip1` apart when we accelerate
due to incompressible inputs. (The methods I tried slowed things down quite a
bit.)

Since we aren't splaying ip0 and ip1 apart (which would be like `0_1_2_3_`, as
opposed to the `01__23__` we were actually doing), it's a big ambitious to
increment `step` by 2. Instead, let's increment it by 1, which has the benefit
sliiightly improving compression. Speed remains pretty much unchanged.
 2021-12-13 16:59:33 -05:00
Nick TerrellandGitHub 3e2a70b6fb Merge pull request #2905 from 15596858998/dev_1205 
add test case
 2021-12-13 13:45:23 -08:00
W. Felix Handte 6ca5f42402 Rewrite step to Track Increment Between Pairs of Positions 
The position updates are rewritten from `ip[N] = ip[N-1] + step` to be
`ip[N] = ip[N-2] + step`. This lets us only deal with the asymmetric spacing
of gaps at setup and then we only have to keep a single `step` variable.

This seems to work quite well on GCC and Clang!
 2021-12-13 14:48:26 -05:00
W. Felix Handte b8434cb754 Allow Templating ZSTD_fast Matchfinders on Acceleration (Lvl < -1) 2021-12-13 14:46:57 -05:00
Felix HandteandGitHub 65404fe14a Merge pull request #2923 from IAL32/patch-1 
typo: Small spelling mistake in example
 2021-12-13 13:15:21 -05:00
zx123123andGitHub c69d13eb99 Update playTests.sh 2021-12-13 08:58:42 +08:00
Adrian CastroandGitHub e0f9dc0dde typo: Small spelling mistake in example 
Just a couple of characters:
`main` -> `may`
 2021-12-11 12:02:23 +01:00
Yann ColletandGitHub 252ef866fb Merge pull request #2922 from facebook/x32 
x32 compatibility
 2021-12-11 00:12:10 -08:00
Yann Collet e1ab2200ff fixed x32 compatibility 2021-12-10 21:02:17 -08:00
Yann Collet c94cda283c added x32 compatibility test 2021-12-10 20:56:20 -08:00
W. Felix Handte ace6a7e746 Decompose step into Two Variables 
This avoids an additional addition, at the cost of an additional variable.
 2021-12-10 16:44:23 -05:00
W. Felix Handte 22501cd283 Stagger Application of stepSize in ZSTD_fast 
This replicates the behavior of @terrelln's `ZSTD_fast` implementation. That
is, it always looks at adjacent pairs of positions, and only applies the
acceleration every other position. This produces a more fine-grained
acceleration.
 2021-12-10 16:44:23 -05:00
Yann Collet 4cc5e2818a complete changelog with #2885 2021-12-09 09:53:45 -08:00
Yann ColletandGitHub c077b530a0 Merge pull request #2917 from facebook/change151 
Update changelog for v1.5.1
 2021-12-09 08:45:34 -08:00
Felix HandteandGitHub 0c26d98c0d Merge pull request #2910 from felixhandte/reject-irregular-dicts 
Reject Irregular Dictionary Files
 2021-12-09 11:44:37 -05:00
Yann Collet 3d738307b4 Update changelog for v1.5.1 2021-12-08 16:55:38 -08:00
Yann ColletandGitHub 57383d2317 Merge pull request #2914 from facebook/xxhash081 
updated xxHash to latest v0.8.1
 2021-12-08 16:48:46 -08:00
Yann Collet 3ce265fea8 remove offending static assert lines 
no idea why visual + clang-cl + appveyor don't like them,
I've not been able to reproduce the issue locally,
but these static assert are very unlikely to deliver a useful signal,
I can't imagine a situation where they will be wrong,
and if they are, then a ton of other things will be broken way before reaching that point.
 2021-12-08 15:05:17 -08:00
Nick TerrellandGitHub 8b40095b3f Merge pull request #2916 from terrelln/issue-2906 
Remove possible NULL pointer addition
 2021-12-08 16:51:10 -05:00
Yann Collet 16241b7d26 altered copyright title 2021-12-08 13:18:41 -08:00
W. Felix Handte 9985e10fda Reject Irregular Dictionary Files 
I hadn't seen #2890, so I wrote my own version. I like this approach a little
better, since it does an explicit check for a regular file, rather than
passing a magic value.

Addresses #2874.
 2021-12-08 16:17:04 -05:00
Yann Collet a9cd6164d7 removed declarations of XXH3 symbols when XXH_NO_XXH3 is defined 
on top of implementations, which were already scoped out.
 2021-12-08 12:56:16 -08:00
Yann Collet 27e706de88 replaces malloc / free / memcpy by Zstandard's version 2021-12-08 12:51:04 -08:00
Nick Terrell b94407b6cf Remove possible NULL pointer addition 
Refactor `ZSTDMT_isOverlapped()` to do NULL checks before computing the
end pointer.

Fixes #2906.
 2021-12-08 12:40:40 -08:00
Nick TerrellandGitHub 859e0500ab Merge pull request #2915 from terrelln/oss-fuzz-build-fix 
Fix oss-fuzz build
 2021-12-08 15:32:49 -05:00
Felix HandteandGitHub 33562c55de Merge pull request #2912 from felixhandte/pkg-config-fix 
Fix Up #2659; Build libzstd.pc Whenever Building the Lib on Unix
 2021-12-08 15:22:56 -05:00
Nick Terrell aa7729c9f3 Fix oss-fuzz build 
Disable assembly when dataflow sanitizer is enabled.

This regressed in PR #2893, which accidentally removed the check for
dataflow sanitizer.
 2021-12-08 11:01:52 -08:00
Yann Collet fb3522a3fe fixed very minor cast warning under cygwin 2021-12-08 09:48:56 -08:00
W. Felix Handte 9f1dee8fa5 Fix Up #2659; Build libzstd.pc Whenever Building the Lib on Unix 2021-12-08 12:43:34 -05:00
Yann Collet 1c7d2c4dd5 updated xxHash to latest v0.8.1 
with minor modifications directly embedded in source :
- does not compile XXH3
- namespace emulation (ZSTD_ prefix)

Incidentally fix #2824
 2021-12-07 21:16:15 -08:00
binhdvoandGitHub 38dfc4699e Imply -q when stderr is not a tty (#2884) 
* Imply -q when stderr is not a tty
 2021-12-07 16:56:19 -05:00
Felix HandteandGitHub 9118ee04c2 Merge pull request #2659 from ericonr/pc 
[lib] Fix libzstd.pc for lib-mt builds
 2021-12-07 14:18:38 -05:00
Nick TerrellandGitHub b6b4c9a3da Merge pull request #2907 from Hello71/armv6-fix-legacy 
Apply FORCE_MEMORY_ACCESS=1 to legacy
 2021-12-06 15:41:22 -05:00
Nick TerrellandGitHub e7b0ae385e Merge pull request #2890 from 15596858998/dec_1201 
fixbug CLI's -D fails when the argument is not a regular file
 2021-12-06 13:18:15 -05:00
Alex Xu (Hello71) 3d773d7013 Apply FORCE_MEMORY_ACCESS=1 to legacy 
See #2633, #2881.
 2021-12-05 22:51:44 -05:00
15596858998 ba7e8f1051 add test case 2021-12-05 19:12:52 +08:00
Nick TerrellandGitHub 486472c453 Merge pull request #2893 from terrelln/issue-2789 
[asm] Share portability macros and restrict ASM further
 2021-12-03 14:07:30 -05:00