10656 Commits
Author SHA1 Message Date
Yann ColletandGitHub 9f42fa0a04 Merge pull request #4021 from pstef/dev
Fix zlibWrapper build
2024-04-09 11:02:49 -07:00
Christian Marangi f1f1ae369a Provide variant pkg-config file for multi-threaded static lib
Multi-threaded static library require -pthread to correctly link and works.
The pkg-config we provide tho only works with dynamic multi-threaded library
and won't provide the correct libs and cflags values if lib-mt is used.

To handle this, introduce an env variable MT to permit advanced user to
install and generate a correct pkg-config file for lib-mt or detect if
lib-mt target is called.

With MT env set on calling make install-pc, libzstd.pc.in is a
pkg-config file for a multi-threaded static library.

On calling make lib-mt, a libzstd.pc is generated for a multi-threaded
static library as it's what asked by the user by forcing it.

libzstd.pc is changed to PHONY to force regeneration of it on calling
lib targets or install-pc to handle case where the same directory is
used for mixed compilation.

This was notice while migrating from meson to make build system where
meson generates a correct .pc file while make doesn't.

Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
2024-04-08 17:44:52 +02:00
dependabot[bot]andGitHub 7968c661af Bump github/codeql-action from 3.24.9 to 3.24.10
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.9 to 3.24.10.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/1b1aada464948af03b950897e5eb522f92603cc2...4355270be187e1b672a7a1c7c7bae5afdc1ab94a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-04-08 05:47:58 +00:00
Piotr Paweł Stefaniak 71def59890 Fix zlibWrapper build
Just after a clone I'm getting this:

~/zstd/zlibWrapper$ cc -c zstd_zlibwrapper.o gz*.c -lz -lzstd -DSTDC
gzwrite.c: In function ‘gz_write’:
gzwrite.c:226:43: error: ‘z_uInt’ undeclared (first use in this
                         function); did you mean ‘uInt’?
  226 |             state.state->strm.avail_in = (z_uInt)n;
      |                                           ^~~~~~
      |                                           uInt
gzwrite.c:226:43: note: each undeclared identifier is reported only
                        once for each function it appears in
gzwrite.c:226:50: error: expected ‘;’ before ‘n’
  226 |             state.state->strm.avail_in = (z_uInt)n;
      |                                                  ^
      |                                                  ;

z_uInt is never used directly, zconf.h redefines uInt to z_uInt under
the condition that Z_PREFIX is set. All examples use uInt, and the type
of avail_in is also uInt.

In this commit I modify the cast to refer to the same type as the type
of lvalue.

Arguably, the real fix here is to handle possible overflows, but that's
beyond the scope of this commit.
2024-04-07 13:04:26 +02:00
Yann ColletandGitHub 87af5fb2df Merge pull request #3996 from facebook/fix_nodejs_version_warning
fix nodejs deprecation warning
2024-04-05 23:06:21 -07:00
Yann ColletandGitHub 6f7f59b232 Merge pull request #4012 from elasota/spec-remove-fse-overflow
Remove specification text stating that probability overflow is invalid
2024-04-05 23:06:06 -07:00
Yann ColletandGitHub 3fd4f3bf04 Merge pull request #4011 from facebook/fix4005
decompression errors always display the full origin filename
2024-04-05 23:05:53 -07:00
BadWolf fd5f8106a5 fix missing include folder for resource compiler 2024-04-06 00:05:58 +02:00
elasota 8cff66f2f5 Remove text specifying probability overflow as invalid, the variable-size value encoding scheme makes this impossible. 2024-04-01 20:08:42 -04:00
Yann Collet a2f145f059 decompression errors always display the full origin filename
instead of the truncated size-limited version.
2024-04-01 11:49:21 -07:00
Yuriy ChernyshovandNick Terrell 72c16b187d Fix building on windows-x86 if clang already includes
[D101338](https://reviews.llvm.org/D101338) landed in 2021, so clang16 should have it
2024-04-01 09:53:08 -07:00
Yann ColletandGitHub e3566d62ac Merge pull request #3994 from sunpoet/dev
Use md5sum rather than gmd5sum for FreeBSD
2024-03-31 23:09:46 -07:00
Yann ColletandGitHub 4b5a266d7b Merge pull request #3997 from facebook/readme_bench_156
updated benchmarks for v1.5.6
2024-03-31 13:17:01 -07:00
Yann Collet e0ee0fccf8 updated benchmarks for v1.5.6 2024-03-27 18:19:31 -07:00
Yann ColletandGitHub aaad8e2e8c Merge pull request #3988 from facebook/dependabot/github_actions/github/codeql-action-3.24.9
Bump github/codeql-action from 3.24.7 to 3.24.9
2024-03-27 18:05:42 -07:00
Yann Collet ebf24b7b77 fix nodejs deprecation warning
by updating msys2 action
2024-03-27 16:10:15 -07:00
Yann ColletandGitHub 80a9a2dfcd Merge pull request #3987 from ManuelBlanc/msbuild_vswhere
Use vswhere to find MSBuild; add VS2022 support
2024-03-27 15:48:40 -07:00
Po-Chuan Hsieh 103a85e6f6 Use md5sum rather than gmd5sum for FreeBSD
Reference:	https://man.freebsd.org/cgi/man.cgi?query=md5sum
2024-03-27 20:53:52 +08:00
Nick TerrellandNick Terrell 81a5e5d438 [fuzz] Turn off -Werror by default
This was causing OSS-Fuzz errors, due to compiler differences.
* Fix the issue
* Also turn off -Werror so we don't fail fuzzer builds for warnings
* Turn on -Werror in our CI
2024-03-26 16:34:36 -07:00
dependabot[bot]andGitHub 101e601c79 Bump github/codeql-action from 3.24.7 to 3.24.9
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.7 to 3.24.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/3ab4101902695724f9365a384f86c1074d94e18c...1b1aada464948af03b950897e5eb522f92603cc2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-25 05:31:42 +00:00
ManuelBlanc 9c442d6fc2 Improve MSBuild search; add latest option 2024-03-23 18:34:07 -04:00
ManuelBlanc 65ab6c267e Use vswhere to find MSBuild; add VS2022 support
See:
https://github.com/microsoft/vswhere/wiki/Find-MSBuild
2024-03-23 09:13:49 -04:00
Yann ColletandGitHub 78f732cfdd Merge pull request #3985 from facebook/scorecard_permission
try to silence some scorecard warnings
2024-03-21 13:56:20 -07:00
Yann Collet 273d1279ca try to silence some scorecard warnings 2024-03-21 13:25:48 -07:00
Yann ColletandGitHub c6e5257240 Merge pull request #3977 from facebook/doc_advanced
Doc update
2024-03-21 12:33:15 -07:00
Elliot GorokhovskyandGitHub dc1f7b560b fix -Werror=pointer-arith in fuzzers (#3983) 2024-03-21 15:16:38 -04:00
Yann ColletandGitHub 6679d0ca7b Merge pull request #3982 from embg/fuzzer_readme
Document the process for adding a new fuzzer
2024-03-21 11:10:03 -07:00
Yann ColletandGitHub 76fbb5f3d4 Merge pull request #3975 from terrelln/2024-03-18-cmake-docs
[cmake] Emit warnings for contradictory build settings
2024-03-21 10:41:54 -07:00
Yann ColletandGitHub 1d3f664fce Merge pull request #3979 from yoniko/Werror-fuzz
Fail on errors when building fuzzers
2024-03-21 10:41:34 -07:00
Yann ColletandGitHub 86caab507f Merge pull request #3981 from terrelln/2024-03-19-generate-sequences
Fix & fuzz ZSTD_generateSequences
2024-03-21 10:41:21 -07:00
Nick Terrell 731f4b70fc Fix & fuzz ZSTD_generateSequences
This function was seriously flawed:
* It didn't do output bounds checks
* It produced invalid sequences when an uncompressed or RLE block was emitted
* It produced invalid sequences when the block splitter was enabled
* It produced invalid sequences when ZSTD_c_targetCBlockSize was enabled

I've attempted to fix these issues, but this function is just a bad idea,
so I've marked it as deprecated and unsafe. We should replace it with
`ZSTD_extractSequences()` which operates on a compressed frame.
2024-03-21 07:18:05 -07:00
Elliot GorokhovskyandGitHub 741b87bbe1 Fuzzing and bugfixes for magicless-format decoding (#3976)
* fuzzing and bugfixes for magicless format

* reset dctx before each decompression

* do not memcmp empty buffers

* nit: decompressor errata
2024-03-20 19:22:34 -04:00
Elliot Gorokhovsky f62b2663b9 Add docs on how to add a new fuzzer 2024-03-19 14:05:23 -07:00
Yonatan Komornik 3487a60950 Fail on errors when building fuzzers
Fails on errors when building fuzzers with `fuzz.py` (adds `Werror`).
Currently allows `declaration-after-statement`, `c++-compat` and
`deprecated` as they are abundant in code (some fixes to
`declaration-after-statement` are presented in this commit).
2024-03-18 15:51:28 -07:00
Yonatan KomornikandGitHub 6a0052a409 Fix bugs in simple decompression fuzzer (#3978)
Fixes 2 issue in `simple_decompress.c`:
1. Wrong type used for storing the results of `ZSTD_findDecompressedSize` resulting in never matching to `ZSTD_CONTENTSIZE_ERROR` or `ZSTD_CONTENTSIZE_UNKNOWN`.

2. Experimental API is used (`ZSTD_findDecompressedSize`) without defining `ZSTD_STATIC_LINKING_ONLY`.
2024-03-18 15:36:40 -07:00
Yann Collet 6f1215b874 fix ZSTD_TARGETCBLOCKSIZE_MIN test
when requested CBlockSize is too low,
bound it to the minimum
instead of returning an error.
2024-03-18 14:10:08 -07:00
Yann Collet c5da438dc0 fix typo 2024-03-18 12:33:22 -07:00
Yann Collet 3d18d9a9ce updated API manual 2024-03-18 12:30:54 -07:00
Yann Collet 902c7ec1fe add doc on CCtx UB state 2024-03-18 12:30:35 -07:00
Yann Collet 5d82c2b57c add a paragraph on UB DCtx state after error 2024-03-18 12:17:41 -07:00
Yann Collet f5728da365 update targetCBlockSize documentation 2024-03-18 12:04:02 -07:00
Nick Terrell 42b02f5185 [cmake] Emit warnings for contradictory build settings
Document that the `ZSTD_BUILD_{SHARED,STATIC}` take precedence over `BUILD_SHARED_LIBS` when exactly one is ON.

Thanks to @teo-tsirpanis for pointing out the potentially confusing behavior.
2024-03-18 09:31:22 -07:00
Alexander KanavinandNick Terrell cd4dba74de pzstd: use c++14 without conditions
Doing this check with a direct c++ snippet is prone to portability problems:

- \043 is not portable between shells: dash expands it to #,
bash does not;

- using # directly works with make 4.3 but does not with make 4.2.

Let's just use the c++ version that covers both the code and the gtest.
2024-03-18 12:15:26 -04:00
Elliot GorokhovskyandGitHub 7d970bd83c Implement one-shot fallback for magicless format (#3971) 2024-03-18 10:55:53 -04:00
Nick TerrellandNick Terrell a595e5812a [cmake] Fix up PR #3716
* Make a variable `PublicHeaders` for Zstd's public headers
* Add `PublicHeaders` to `Headers`, which was missing
* Only export `${LIBRARY_DIR}` publicly, not `common/`
* Switch the `target_include_directories()` to `INTERFACE` because zstd uses relative includes internally, so doesn't need any include directories to build
* Switch installation to use the `PublicHeaders` variable, and test that the right headers are installed
2024-03-18 09:18:51 -04:00
dsviandNick Terrell 79cd0ff712 Makes it possible to use the lib through FetchContent or ExternalProject_Add 2024-03-18 09:18:51 -04:00
Yann ColletandGitHub 3008f74ee2 Merge pull request #3973 from facebook/dependabot/github_actions/github/codeql-action-3.24.7
Bump github/codeql-action from 3.24.6 to 3.24.7
2024-03-18 00:44:55 -07:00
Yann ColletandGitHub f79c1c476e Merge pull request #3972 from facebook/dependabot/github_actions/actions/cache-4
Bump actions/cache from 3 to 4
2024-03-18 00:44:12 -07:00
dependabot[bot]andGitHub 9dca0602f4 Bump github/codeql-action from 3.24.6 to 3.24.7
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.6 to 3.24.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/8a470fddafa5cbb6266ee11b37ef4d8aae19c571...3ab4101902695724f9365a384f86c1074d94e18c)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 05:37:56 +00:00
dependabot[bot]andGitHub 88301b58c1 Bump actions/cache from 3 to 4
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 05:37:52 +00:00