ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,725 Commits 17 Branches 77 Tags
e3d1f7057dd8cddf28bb3d40fba1ea3dc3f94de4
Commit Graph
11725 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 1e7fa242f4 minor refactor zstd_fast 
make hot variables more local
 2024-10-07 11:22:40 -07:00
Yann ColletandGitHub da23998e9a Merge pull request #4160 from facebook/fix_nightly 
fix dependency for nightly github actions tests
 2024-10-03 21:02:39 -07:00
Yann Collet b84653fc83 fix dependency for nightly github actions tests 2024-10-03 15:10:16 -07:00
Yann ColletandGitHub b7e1eef048 Merge pull request #4159 from facebook/spec_refactor_fse 
specification update
 2024-10-03 14:54:16 -07:00
Yann Collet a8b86d024a refactor documentation of the FSE decoding table build process 2024-10-02 23:09:06 -07:00
Yann ColletandGitHub 75b0f5f4f5 Merge pull request #4153 from artem/fix-meson-includes 
meson: Do not export private headers in libzstd_dep to avoid name clash
 2024-10-02 16:51:44 -07:00
Yann ColletandGitHub dda3cdfdec Merge pull request #4156 from facebook/rm_circleci 
removing nightly tests built on circleci
 2024-10-02 16:51:15 -07:00
Yann ColletandGitHub 751bf1ffd8 Merge pull request #4157 from facebook/fix_result_c 
fix incorrect pointer manipulation
 2024-10-02 16:50:45 -07:00
Yann ColletandGitHub dcc8fd0472 Merge pull request #4158 from facebook/benchzstd_fclose 
fix missing fclose()
 2024-10-02 16:49:43 -07:00
Yann Collet 8edd147686 fix missing fclose() 
fix #4151
 2024-10-01 09:52:45 -07:00
Yann Collet de6cc98e07 fix incorrect pointer manipulation 
fix #4155
 2024-10-01 09:25:26 -07:00
Yann Collet 3d5d3f5630 removing nightly tests built on circleci 2024-09-30 21:38:29 -07:00
Yann ColletandGitHub 27bf1362fe Merge pull request #4154 from dearblue/freebsd-14.1 
Update FreeBSD VM image to 14.1
 2024-09-30 11:54:32 -07:00
Artem Labazov ccc02a9a77 meson: Fix contrib and tests build 2024-09-30 18:05:57 +03:00
Artem LabazovandArtem Labazov d2d49a1161 meson: Do not export private headers in libzstd_dep to avoid name clash 
This way libzstd_dep does not override, for instance, <xxhash.h>
 2024-09-30 17:03:42 +03:00
dearblue a3b5c4521c Update FreeBSD VM image to 14.1 
FreeBSD 14.0 will reach the end of life on 2024-09-30.
The updated 14.1 is scheduled to end-of-life on 2025-03-31.

ref. https://www.freebsd.org/releases/14.2R/schedule/
 2024-09-30 22:45:17 +09:00
Yann ColletandGitHub 984d11a4d1 Merge pull request #4146 from facebook/dictBench_Doc 
update documentation: specify that Dictionary can be used for benchmark
 2024-09-27 13:44:42 -07:00
Yann ColletandGitHub d2212c680a Merge pull request #4013 from elasota/spec-clarify-offset-code-overflow 
Specify that decoders may reject non-zero probabilities for larger offset codes than implementation supports
 2024-09-27 13:42:32 -07:00
Yann Collet 039f404faa update documentation to specify that Dictionary can be used for benchmark 
fix #4139
 2024-09-25 16:56:01 -07:00
inventor500andNick Terrell 9215de52c7 Included suggestion from @neheb 2024-09-25 09:51:05 -07:00
inventor500andNick Terrell a8b544d460 Fixed warning when compiling pzstd with CPPFLAGS=-Wunused-result and CXXFLAGS=-std=c++17 2024-09-25 09:51:05 -07:00
Yann ColletandGitHub bc96d4b077 Merge pull request #4119 from xionghul/dev 
Fix zstd-pgo run error
 2024-09-24 17:55:43 -07:00
Yann ColletandGitHub d27a4cd4ac Merge pull request #4143 from facebook/fix_dictsizemin_dic 
fix doc nit: ZDICT_DICTSIZE_MIN
 2024-09-24 17:55:25 -07:00
Ilya Tokar e8fce38954 Optimize compression by avoiding unpredictable branches 
Avoid unpredictable branch. Use conditional move to generate the address
that is guaranteed to be safe and compare unconditionally.
Instead of

if (idx < limit && x[idx] == val ) // mispredicted idx < limit branch

Do

addr = cmov(safe,x+idx)
if (*addr == val && idx < limit) // almost always false so well predicted

Using microbenchmarks from https://github.com/google/fleetbench,
I get about ~10% speed-up:

name                                                                                          old cpu/op   new cpu/op    delta
BM_ZSTD_COMPRESS_Fleet/compression_level:-7/window_log:15                                     1.46ns ± 3%   1.31ns ± 7%   -9.88%  (p=0.000 n=35+38)
BM_ZSTD_COMPRESS_Fleet/compression_level:-7/window_log:16                                     1.41ns ± 3%   1.28ns ± 3%   -9.56%  (p=0.000 n=36+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:-5/window_log:15                                     1.61ns ± 1%   1.43ns ± 3%  -10.70%  (p=0.000 n=30+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:-5/window_log:16                                     1.54ns ± 2%   1.39ns ± 3%   -9.21%  (p=0.000 n=37+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:-3/window_log:15                                     1.82ns ± 2%   1.61ns ± 3%  -11.31%  (p=0.000 n=37+40)
BM_ZSTD_COMPRESS_Fleet/compression_level:-3/window_log:16                                     1.73ns ± 3%   1.56ns ± 3%   -9.50%  (p=0.000 n=38+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:-1/window_log:15                                     2.12ns ± 2%   1.79ns ± 3%  -15.55%  (p=0.000 n=34+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:-1/window_log:16                                     1.99ns ± 3%   1.72ns ± 3%  -13.70%  (p=0.000 n=38+38)
BM_ZSTD_COMPRESS_Fleet/compression_level:0/window_log:15                                      3.22ns ± 3%   2.94ns ± 3%   -8.67%  (p=0.000 n=38+40)
BM_ZSTD_COMPRESS_Fleet/compression_level:0/window_log:16                                      3.19ns ± 4%   2.86ns ± 4%  -10.55%  (p=0.000 n=40+38)
BM_ZSTD_COMPRESS_Fleet/compression_level:1/window_log:15                                      2.60ns ± 3%   2.22ns ± 3%  -14.53%  (p=0.000 n=40+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:1/window_log:16                                      2.46ns ± 3%   2.13ns ± 2%  -13.67%  (p=0.000 n=39+36)
BM_ZSTD_COMPRESS_Fleet/compression_level:2/window_log:15                                      2.69ns ± 3%   2.46ns ± 3%   -8.63%  (p=0.000 n=37+39)
BM_ZSTD_COMPRESS_Fleet/compression_level:2/window_log:16                                      2.63ns ± 3%   2.36ns ± 3%  -10.47%  (p=0.000 n=40+40)
BM_ZSTD_COMPRESS_Fleet/compression_level:3/window_log:15                                      3.20ns ± 2%   2.95ns ± 3%   -7.94%  (p=0.000 n=35+40)
BM_ZSTD_COMPRESS_Fleet/compression_level:3/window_log:16                                      3.20ns ± 4%   2.87ns ± 4%  -10.33%  (p=0.000 n=40+40)

I've also measured the impact on internal workloads and saw similar
~10% improvement in performance, measured by cpu usage/byte of data.
 2024-09-20 16:07:01 -04:00
Yann Collet 7a48dc230c fix doc nit: ZDICT_DICTSIZE_MIN 
fix #4142
 2024-09-19 09:50:30 -07:00
Yann ColletandGitHub 20707e3718 Merge pull request #4129 from facebook/mitigate_32bit 
Limit range of operations on Indexes in 32-bit mode
 2024-08-22 11:00:50 -07:00
Yann Collet 09cb37cbb1 Limit range of operations on Indexes in 32-bit mode 
and use unsigned type.
This reduce risks that an operation produces a negative number when crossing the 2 GB limit.
 2024-08-21 11:03:43 -07:00
Yann ColletandGitHub ad038d8768 Merge pull request #4128 from facebook/dependabot/github_actions/github/codeql-action-3.26.2 
Bump github/codeql-action from 3.25.1 to 3.26.2
 2024-08-19 13:54:06 -07:00
dependabot[bot]andGitHub ec0c41414d Bump github/codeql-action from 3.25.1 to 3.26.2 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.25.1 to 3.26.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/c7f9125735019aa87cfc361530512d50ea439c71...429e1977040da7a23b6822b13c129cd1ba93dbb2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-08-19 05:37:33 +00:00
Yann ColletandGitHub a761013b03 Merge pull request #4122 from facebook/dependabot/github_actions/actions/setup-java-4 
Bump actions/setup-java from 3 to 4
 2024-08-11 23:07:43 -07:00
dependabot[bot]andGitHub aed3c7540a Bump actions/setup-java from 3 to 4 
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 3 to 4.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-08-12 05:16:05 +00:00
Yann ColletandGitHub 1eb32ff594 Merge pull request #4115 from Adenilson/leak01 
[zstd][leak] Avoid memory leak on early return of ZSTD_generateSequence
 2024-08-09 14:09:17 -07:00
Yann ColletandGitHub ee1fc7ee5c Merge pull request #4114 from Adenilson/trace01 
[riscv] Enable support for weak symbols
 2024-08-09 14:08:57 -07:00
Yann ColletandGitHub 10e2a8042c Merge pull request #4113 from facebook/fix4110 
formatString_u() can display numbers > 100
 2024-08-08 17:19:08 -07:00
Xionghu Luo bf4a43fcd4 Fix zstd-pgo run error 
The -Werror=missing-profile caused thread/zlib/lzma/lz4 detection failure
when build with profile-use, thus caused ZSTD_MULTITHREAD etc. is not
defined for profile-use, then there will be many profile mismatch information
in output and the final binary reports run error sometimes as below:

Error : ZSTD_CCtx_setParameter(ctx, ZSTD_c_nbWorkers, adv->nbWorkers) failed : Unsupported parameter

Signed-off-by: Xionghu Luo <xionghuluo@tencent.com>
 2024-08-08 15:13:27 +08:00
Yann ColletandGitHub 9a5c74b2ca Merge pull request #4109 from facebook/actionsTest 
added android-ndk-build
 2024-08-07 18:17:18 -07:00
Adenilson Cavalcanti a40bad8ec0 [zstd][leak] Avoid memory leak on early return of ZSTD_generateSequence 
Sanity checks on a few of the context parameters (i.e. workers and block size)
may prompt an early return on ZSTD_generateSequences.

Allocating the destination buffer past those return points avoids a potential
memory leak.

This patch should fix issue #4112.
 2024-08-06 18:01:20 -07:00
Adenilson Cavalcanti 6dbd49bcd0 [riscv] Enable support for weak symbols 
Both gcc and clang support weak symbols on RISC-V, therefore
let's enable it.

This should fix issue #4069.
 2024-08-06 16:55:32 -07:00
Yann Collet 14b8d398fd fix c90 comment style 2024-08-06 12:47:30 -07:00
Yann Collet 89451cafbd formatString_u() can display numbers > 100 
fixes #4110
 2024-08-06 11:44:37 -07:00
Yann ColletandGitHub 6b16169ccf Merge pull request #4111 from facebook/dependabot/github_actions/msys2/setup-msys2-2.24.1 
Bump msys2/setup-msys2 from 2.24.0 to 2.24.1
 2024-08-05 00:39:06 -07:00
dependabot[bot]andGitHub 688a815c86 Bump msys2/setup-msys2 from 2.24.0 to 2.24.1 
Bumps [msys2/setup-msys2](https://github.com/msys2/setup-msys2) from 2.24.0 to 2.24.1.
- [Release notes](https://github.com/msys2/setup-msys2/releases)
- [Changelog](https://github.com/msys2/setup-msys2/blob/main/CHANGELOG.md)
- [Commits](https://github.com/msys2/setup-msys2/compare/5df0ca6cbf14efcd08f8d5bd5e049a3cc8e07fd2...ddf331adaebd714795f1042345e6ca57bd66cea8)

---
updated-dependencies:
- dependency-name: msys2/setup-msys2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2024-08-05 05:17:43 +00:00
Yann ColletandGitHub fdfb2aff39 Merge pull request #4102 from proppy/decodecorpus-ext 
tests/decodecorpus: add more advanced options
 2024-07-31 01:43:15 -07:00
Yann Collet cb784edf5d added android-ndk-build 2024-07-30 11:34:49 -07:00
Yann ColletandGitHub d4b176db0f Merge pull request #4107 from Adenilson/ndk01 
[zstd][android] Fix build with NDK r27
 2024-07-30 11:34:27 -07:00
Adenilson Cavalcanti c3c28c4d5a [zstd][android] Fix build with NDK r27 
The NDK cross compiler declares the target as __linux (which is
not technically incorrect), which triggers the enablement of _GNU_SOURCE
in the newly added code that requires the presence of qsort_r() used
in the COVER dictionary code.

Even though the NDK uses llvm/libc, it doesn't declare qsort_r()
in the stdlib.h header.

The build fix is to only activate the _GNU_SOURCE macro if the OS is
*not* Android, as then we will fallback to the C90 compliant code.

This patch should solve the reported issue number #4103.
 2024-07-29 17:13:58 -07:00
Yann ColletandGitHub 43b713a2ce Merge pull request #4094 from RubenKelevra/patchfrom_singlethread_man_update 
clarify when to use '--single-thread' with '--patch-from'
 2024-07-29 09:34:55 -07:00
Yann ColletandGitHub 81b52f31e3 Merge pull request #4106 from facebook/dependabot/github_actions/msys2/setup-msys2-2.24.0 
Bump msys2/setup-msys2 from 2.23.0 to 2.24.0
 2024-07-29 09:34:17 -07:00
Yann ColletandGitHub adcb543fb8 Merge pull request #4087 from jclab-joseph/fix/genhtml-windows 
gen_html: Fix build error with mingw
 2024-07-29 09:18:27 -07:00
Yann ColletandGitHub b768e09393 Merge pull request #4101 from uilianries/readme/conan 
[docs] Add instruction how to install zstd using Conan
 2024-07-29 09:17:51 -07:00