ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,485 Commits 17 Branches 77 Tags
7b94de78ff2c3634cd7ef32f0e16f4722c03578f
Commit Graph
11485 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 1f83b7cfc4 fix a minor inefficiency in compress_superblock 
and in `decodecorpus`:
the specific case `nbSeq=127` can be represented using the 1-byte format.
Note that both the 1-byte and the 2-bytes formats are valid to represent this case,
so there was no "error", produced data remains valid,
it's just that the 1-byte format is more efficient.

fix #3667

Credit to @ip7z for finding this issue.
 2023-06-05 09:51:52 -07:00
Gregor Jasny 7e09f07b32 Fix Intel Xcode builds with assembly 
When forcing the source file language to `C`, Xcode enforces
the file to be compiled as `C` by appending `-x c` to the
compiler command line.

For now try to limit the damage and only enforce the language
if the ASM and C compilers differ.

Reproducer (CMake `3.26.4`, Xcode `14.3`):
```
cmake -S build/cmake -B _b -GXcode -DCMAKE_OSX_ARCHITECTURES=x86_64
cmake --build _b
```

Fix: #3622
 2023-06-02 12:43:59 +02:00
Yann Collet 94a2f2791f changed LLU suffix into ULL for Visual 2012 and lower 
both suffixes are supposed to be valid,
but for some reason, Visual 2012 and lower only support ULL.
 2023-05-31 13:29:53 -07:00
Yann Collet 5108c9ac97 Fixed a bug in the educational decoder 
Credit to Igor Pavlov
 2023-05-27 11:22:30 -07:00
Tao HeandNick Terrell 5059618295 Add options in Makefile to cmake 
Signed-off-by: Tao He <sighingnow@gmail.com>
 2023-05-26 13:45:21 -07:00
Nick TerrellandNick Terrell d01a2c6929 Fix UBSAN issue (zero addition to NULL) 
Fix UBSAN issue that came up internally.
 2023-05-26 13:43:47 -07:00
Yann ColletandGitHub 23a0643ef1 Merge pull request #3631 from facebook/example_streaming_mt 
minor : update streaming_compression example
 2023-05-23 20:38:28 -04:00
Duncan HornandNick Terrell 1b994cbc57 Get zstd working with ARM64EC on Windows 2023-05-23 18:40:31 -04:00
Felix HandteandGitHub ed6953772f Merge pull request #3623 from felixhandte/compile-out-compressors 
Allow Build-Time Exclusion of Individual Compression Strategies
 2023-05-22 17:38:10 -04:00
W. Felix Handte 1b65803fe7 Reorder Definitions in zstd_opt.c to Group Under Macro Guards (Slightly) 2023-05-22 12:41:48 -04:00
W. Felix Handte 59c7b2a492 Reorder Definitions in zstd_lazy.c to Group Under Macro Guards 2023-05-22 12:37:03 -04:00
W. Felix Handte 5490c75dda Also Allow/Document/Test Excluding dfast and Up 2023-05-04 12:31:41 -04:00
W. Felix Handte cc1ffe0bd6 Add Documentation to lib/README.md 2023-05-04 12:20:02 -04:00
W. Felix Handte eb9227935e Also Reorganize Zstd Opt Declarations 2023-05-04 12:18:58 -04:00
W. Felix Handte d09f195ceb Remove blockCompressor NULL Checks 2023-05-04 12:18:58 -04:00
W. Felix Handte b7add1dd67 Abort if Unsupported Parameters Used 2023-05-04 12:18:58 -04:00
W. Felix Handte f242f5be8f Re-Order Lazy Declarations; Minimize ifndefs 2023-05-04 12:18:58 -04:00
W. Felix Handte 698af84fcf Add CI Test for Excluding Matchfinders 2023-05-04 12:18:58 -04:00
W. Felix Handte bae174960b Add ZSTD_LIB_EXCLUDE_COMPRESSORS_DFAST_AND_UP Build Variable 2023-05-04 12:18:58 -04:00
W. Felix Handte 39b7946b95 Define Macros for Possibly-Present Functions; Use Them Rather than Ifdef Guards 2023-05-04 12:18:58 -04:00
W. Felix Handte b12e8cb3e7 Merge Ultra and Ultra2 Exclusion 
Ultra2 does not exist for dict compression, and so uses ultra. So ultra must
be present if ultra2 is.
 2023-05-04 12:18:58 -04:00
W. Felix Handte 6761e1c949 Tweak Ultra/Opt Guards 2023-05-04 12:18:58 -04:00
W. Felix Handte 16bbd7437c Avoid Ratio Regression Tests When Compressors are Excluded 2023-05-04 12:18:58 -04:00
W. Felix Handte 5a75956001 Adjust Strategy in CParams to Avoid Using Excluded Block Compressors 2023-05-04 12:18:58 -04:00
W. Felix Handte 50cdf84f58 Macro-Exclude Block Compressors from Declaration/Definition 2023-05-04 12:18:58 -04:00
W. Felix Handte 81b86a2024 NULL Out Block Compressor Table Entries When Excluded 
Don't check about excluding `ZSTD_fast`. It's always included so that we know
we can resolve downwards and hit a strategy that's present.
 2023-05-04 12:18:58 -04:00
W. Felix Handte cbf3e26316 Allow ZSTD_selectBlockCompressor() to Return NULL 
Return an error rather than segfaulting.
 2023-05-04 12:18:58 -04:00
Felix HandteandGitHub 7806d80338 Merge pull request #3634 from facebook/dependabot/github_actions/github/codeql-action-2.3.2 
Bump github/codeql-action from 2.3.0 to 2.3.2
 2023-05-01 08:06:07 -07:00
dependabot[bot]andGitHub 2a5076d264 Bump github/codeql-action from 2.3.0 to 2.3.2 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.3.0 to 2.3.2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/b2c19fb9a2a485599ccf4ed5d65527d94bc57226...f3feb00acb00f31a6f60280e6ace9ca31d91c76a)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-05-01 06:04:02 +00:00
Yann Collet 6ec18aed31 minor : update streaming_compression example 
display a warning when requesting multi-threading
while linking to a library that doesn't support multi-threading.
 2023-04-26 12:45:23 -07:00
Elliot GorokhovskyandGitHub 0525d1cec6 Merge pull request #3629 from facebook/dependabot/github_actions/github/codeql-action-2.3.0 
Bump github/codeql-action from 2.2.11 to 2.3.0
 2023-04-25 22:58:25 -04:00
dependabot[bot]andGitHub be489f78df Bump github/codeql-action from 2.2.11 to 2.3.0 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.11 to 2.3.0.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/d186a2a36cc67bfa1b860e6170d37fb9634742c7...b2c19fb9a2a485599ccf4ed5d65527d94bc57226)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-24 06:03:24 +00:00
Yann ColletandGitHub e256e43274 Merge pull request #3624 from danlark1/patch-4 
Disable unused variable warning in msan configurations
 2023-04-20 10:46:20 -07:00
Daniel KuteninandGitHub 4c25ea329b Disable unused variable warning in msan configurations 2023-04-20 11:14:08 +01:00
Yann ColletandGitHub 189653a9c1 Merge pull request #3614 from mredig/darwin-fat 
add makefile entry to build fat binary on macos
 2023-04-19 15:06:36 -07:00
Yann ColletandGitHub 504c4a1f36 Merge pull request #3620 from facebook/errata_128k 
[doc] add decoder errata paragraph
 2023-04-19 11:31:16 -07:00
Yann Collet 0d6954b4cc added golden file for the new decompressor erratum 2023-04-19 00:24:35 -07:00
Nick TerrellandNick Terrell 61efb2a047 Add ZSTD_d_maxBlockSize parameter 
Reduces memory when blocks are guaranteed to be smaller than allowed by
the format. This is useful for streaming compression in conjunction with
ZSTD_c_maxBlockSize.

This PR saves 2 * (formatMaxBlockSize - paramMaxBlockSize) when streaming.
Once it is rebased on top of PR #3616 it will save
3 * (formatMaxBlockSize - paramMaxBlockSize).
 2023-04-17 22:06:44 -07:00
Yann ColletandGitHub ed313342f1 Merge pull request #3621 from facebook/remove_CIs 
removed travis & appveyor scripts
 2023-04-17 17:54:40 -07:00
Nick TerrellandNick Terrell 0abf2baef9 Reduce streaming decompression memory by 128KB 
The split literals buffer patch increased streaming decompression memory
by 64KB (shrunk lit buffer from 128KB to 64KB, and added 128KB). This
patch removes the added 128KB buffer, because it isn't necessary.

The buffer was there because the literals compression code didn't know
the true `blockSizeMax` of the frame, and always put split literals so
they ended 128KB - 32 from the beginning of the block. Instead, we can
pass down the true `blockSizeMax` and ensure that the split literals
end up at `blockSizeMax - 32` from the beginning of the block. We
already reserve a full `blockSizeMax` bytes in streaming mode, so we
won't be overwriting the extDict window.
 2023-04-17 16:31:02 -07:00
Yann Collet 05434fe9a5 removed travis & appveyor scripts 
we don't employ these CI systems anymore
 2023-04-17 15:50:31 -07:00
Yann Collet a29b6ed251 added decoder errata paragraph 
for compressed blocks of size exactly 128 KB
which used to be disallowed by the spec
but have become allowed in more recent version of the spec.

While this limitation is fixed in decoders v1.5.4+,
implementers should refrain from generating such block with their custom encoder
as they could be misclassified as corrupted by older decoder versions.
 2023-04-17 15:43:27 -07:00
Yann ColletandGitHub 7afe0b3bea Merge pull request #3619 from facebook/dependabot/github_actions/actions/checkout-3.5.2 
Bump actions/checkout from 3.5.0 to 3.5.2
 2023-04-16 23:50:36 -07:00
dependabot[bot]andGitHub 803e65f935 Bump actions/checkout from 3.5.0 to 3.5.2 
Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/8f4b7f84864484a7bf31766abe9204da3cbe65b3...8e5e7e5ab8b370d6c329ec480221332ada57f0ab)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-17 06:02:27 +00:00
Michael Redig 0a794163f4 add makefile entry to build fat binary on macos 2023-04-13 18:43:06 -05:00
Nick TerrellandNick Terrell e72e13ac6c [oss-fuzz] Fix simple_round_trip fuzzer with overlapping decompression 
When `ZSTD_c_maxBlockSize` is set, we weren't computing the
decompression margin correctly, leading to `dstSize_tooSmall` errors.
Fix that computation.

This is just a bug in the fuzzer, not a bug in the library itself.

Credit to OSS-Fuzz
 2023-04-13 10:14:29 -07:00
Felix HandteandGitHub 0f255ff4fa Merge pull request #3606 from facebook/dependabot/github_actions/github/codeql-action-2.2.11 
Bump github/codeql-action from 2.2.9 to 2.2.11
 2023-04-10 08:32:54 -07:00
Yann ColletandGitHub 632baf62e3 Merge pull request #3607 from facebook/dependabot/github_actions/cygwin/cygwin-install-action-4 
Bump cygwin/cygwin-install-action from 3 to 4
 2023-04-10 01:30:20 -07:00
dependabot[bot]andGitHub d9582a0cb8 Bump cygwin/cygwin-install-action from 3 to 4 
Bumps [cygwin/cygwin-install-action](https://github.com/cygwin/cygwin-install-action) from 3 to 4.
- [Release notes](https://github.com/cygwin/cygwin-install-action/releases)
- [Commits](https://github.com/cygwin/cygwin-install-action/compare/f5e0f048310c425e84bc789f493a828c6dc80a25...006ad0b0946ca6d0a3ea2d4437677fa767392401)

---
updated-dependencies:
- dependency-name: cygwin/cygwin-install-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-10 06:00:18 +00:00
dependabot[bot]andGitHub dc88f7b8a0 Bump github/codeql-action from 2.2.9 to 2.2.11 
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.9 to 2.2.11.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/04df1262e6247151b5ac09cd2c303ac36ad3f62b...d186a2a36cc67bfa1b860e6170d37fb9634742c7)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2023-04-10 06:00:15 +00:00