Commit Graph
8654 Commits
Author SHA1 Message Date
Nick Terrell b9302410bf [linux-kernel] Don't inline function in zstd_opt.c
The optimal parser is unlikely to be used in the linux kernel in
practice. There is no reason these functions should be force inlined,
since we aren't gaining anything, and are losing build size.

| Compiler | Before (Bytes) | After (Bytes) | Delta (Bytes) |
|----------|----------------|---------------|---------------|
| gcc-11   |        1142090 |        952754 |       -189336 |
| clang-12 |        1228402 |        976290 |       -252112 |

This is a temporary solution pending the resolution of Issue #2862 in
the `dev` branch.
2021-11-15 20:37:58 -08:00
Nick Terrell 0118fe65ff Fix unused variable warning
`litLengthSum` is unused when asserts are disabled. Already fixed in `dev` by
PR#2838. Found by the Kernel test robot [0].

[0] https://lore.kernel.org/linux-mm/202111120312.833wII4i-lkp@intel.com/T/

Reported-by: kernel test robot <lkp@intel.com>
2021-11-15 16:55:59 -08:00
Nick Terrell 608bacf6cb Backport zstd patch from LKML
Credit to Nathan Chancellor for the bug fix and Nick Desaulniers for the
bug report.

Link: ClangBuiltLinux/linux#1486
Link: https://lore.kernel.org/all/20211021202353.2356400-1-nathan@kernel.org/
2021-11-11 12:18:35 -08:00
Nick Terrell c0c38ba1db [binary-tree] Fix underflow of nbCompares
Fix underflow of `nbCompares` by switching to an `int` and comparing
`nbCompares > 0`. This is a minimal fix, because I don't want to change
the logic. These loops seem to be doing `nbCompares + 1` comparisons.

The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:57:35 -07:00
Nick Terrell 5a3e16f0c2 [ldm] Fix ZSTD_c_ldmHashRateLog bounds check
There is no minimum value check, so the parameter could be negative.
Switch to the standard pattern of using `BOUNDCHECK()`.

The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:43:57 -07:00
Nick Terrell 2c94f9fc61 [nit] Fix buggy indentation
The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:43:33 -07:00
Nick Terrell 695181c2e0 [multiple-ddicts] Fix NULL checks
The bug was reported by Dan Carpenter and found by Smatch static
checker.

https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-11 15:42:45 -07:00
Nick Terrell 20821a46f4 [lib] Make lib compatible with -Wfall-through excepting legacy
Switch to a macro `ZSTD_FALLTHROUGH;` instead of a comment. On supported
compilers this uses an attribute, otherwise it becomes a comment.

This is necessary to be compatible with clang's `-Wfall-through`, and
gcc's `-Wfall-through=2` which don't support comments. Without this the
linux build emits a bunch of warnings.
2021-09-23 11:54:14 -07:00
Nick Terrell 1715601e55 [contrib][linux] Reduce stack usage by 80 bytes
Instead of calling `ZSTD_compress_advanced()` and
`ZSTD_initCStream_advanced()`, which each take a `ZSTD_parameters` by
value, use the new advanced API.

Stack usage went from 2024 -> 1944.
2021-09-22 18:18:20 -07:00
Nick Terrell 07d7ebe448 [contrib][linux] Fix up SPDX license identifiers
Correctly identify that we are GPL v2+ or BSD 3 clause, as pointed out
in issue #2663.
2021-09-22 15:06:08 -07:00
Nick Terrell 67a426c322 [linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.

E.g.

```
/** Before */

/* After */

/**
 * Before
 */

/*
 * After
 */

/***********************************
 * Before
 ***********************************/

/* *********************************
 * After
 ***********************************/
```
2021-04-29 15:53:38 -07:00
Nick Terrell 4432dac93b [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:37:45 -07:00
Nick Terrell de9de869a3 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:37:39 -07:00
Nick Terrell c1a244d534 Merge pull request #2539 from terrelln/linux-kernel-fixes
Fixes for the next linux kernel patch version
2021-03-24 16:29:38 -07:00
senandGitHub c48889f097 Merge pull request #2538 from senhuang42/monotonicity_test
Add memory monotonicity test over srcSize
2021-03-22 16:54:34 -04:00
Nick TerrellandGitHub ebc2dfa821 Merge pull request #2524 from terrelln/huf-stack-reduction
[huf] Reduce stack usage of HUF_readDTableX2 by ~972 bytes
2021-03-22 12:37:54 -07:00
Nick Terrell 634bfd339f [FSE] Clean up workspace using dynamically sized struct 2021-03-22 11:07:07 -07:00
Yann ColletandGitHub 8b97931ae1 Merge pull request #2550 from facebook/vvv_srcSize
fix #2549
2021-03-22 10:44:44 -07:00
Sen Huang dff4a0e867 Make ZSTD_estimateCCtxSize_internal() loop through all srcSize parameter sets as well 2021-03-21 16:15:31 -07:00
Yann Collet 9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Yann ColletandGitHub 515807182f Merge pull request #2548 from SupervisedThinking/build_fix
meson: fix build by adding missing files
2021-03-19 15:20:14 -07:00
SupervisedThinking edf2b1176d meson: fix build by adding missing files
fixes https://github.com/facebook/zstd/issues/2519
2021-03-19 19:52:45 +01:00
senandGitHub eace4abc25 Merge pull request #2540 from senhuang42/fix_dds_supported
Fix dedicated dict search isSupported() requirements.
2021-03-17 23:25:53 -04:00
Nick Terrell 756bd59322 [huf][fse] Clean up workspaces
* Move `counting` to a struct in `FSE_decompress_wksp_body()`
* Fix error code in `FSE_decompress_wksp_body()`
* Rename a variable in `HUF_ReadDTableX2_Workspace`
2021-03-17 16:50:37 -07:00
Sen Huang 77ae664ba6 Fix ZSTD_dedicatedDictSearch_isSupported() requirements 2021-03-16 17:36:05 -07:00
Nick Terrell ea288e0d8e [lib] Bump zstd version number 2021-03-16 11:47:27 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell 7222614a19 [contrib][freestanding] Remove tracing support
Remove tracing support from `freestanding.py` to keep things simple.
2021-03-16 11:47:27 -07:00
Nick Terrell e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
2021-03-16 11:47:22 -07:00
Nick Terrell 49a9e070f5 [contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
2021-03-16 11:40:24 -07:00
Nick Terrell d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
2021-03-16 11:40:24 -07:00
senandGitHub 413b3198b0 Merge pull request #2534 from foxeng/fix-seek-descriptor-check
Fix seek table descriptor check when loading
2021-03-16 13:09:00 -04:00
Sen Huang b9dd821441 Add mem monotonicity test over srcSize 2021-03-16 08:24:26 -07:00
Felix HandteandGitHub d1a6d080cf Merge pull request #2535 from felixhandte/gha-release-artifacts
Add GitHub Action to Automatically Publish Release Tarballs
2021-03-15 13:58:27 -04:00
W. Felix Handte d2b7f2e27a Allow a Passphrase on the Key 2021-03-15 12:48:53 -04:00
W. Felix Handte eed64d75f4 Maintain Artifact Name Backwards Compatibility
When the tag is `v1.2.3`, name the artifacts `zstd-1.2.3.tar*` rather than
`zstd-v1.2.3.tar*`. When the tag doesn't match, use the full tag.
2021-03-15 11:59:31 -04:00
Fotis Xenakis 316f3d2224 Run generic-dev:gcc-8-asan-ubsan-testzstd on latest Ubuntu 2021-03-13 11:42:47 +02:00
W. Felix Handte a51511e7a7 Remove CircleCI Artifact Generation 2021-03-12 17:35:11 -05:00
Fotis Xenakis 3c6f5d5eca Fix seekable test to provide valid descriptor 2021-03-13 00:00:08 +02:00
Fotis Xenakis 21697b9c9e Fix seek table descriptor check when loading 2021-03-12 23:07:15 +02:00
W. Felix Handte 5d1fec8ce1 Add GitHub Action to Automatically Publish Release Tarballs
This commit introduces a GitHub action that is triggered on release creation,
which creates the release tarball, compresses it, hashes it, signs it, and
attaches all of those files to the release.
2021-03-12 12:33:58 -05:00
senandGitHub a3feed8dcd Merge pull request #2517 from PaulBone/num_cores
Make the number of physical CPU cores detection more robust
2021-03-08 11:14:34 -05:00
Yann ColletandGitHub 3d6c903056 Merge pull request #2521 from animalize/doc_free
doc: ZSTD_free*() functions accept NULL pointer
2021-03-06 21:33:28 -08:00
Nick Terrell 3b1aba42cc [fse] Reduce stack usage of FSE_decompress_wksp() by 512 bytes
* Move `counting` into the workspace
* Inrease `HUF_DECOMPRESS_WORKSPACE_SIZE` by 512 bytes
2021-03-05 13:24:27 -08:00
Nick Terrell 0f18059a4e [huf] Reduce stack usage of HUF_readDTableX2 by ~460 bytes
* Use `HUF_readStats_wksp()`
* Use workspace in `HUF_fillDTableX2*()`
* Clean up workspace usage to use a workspace struct
2021-03-05 12:39:46 -08:00
Nick TerrellandGitHub b5fd348a85 Merge pull request #2523 from terrelln/huf-stack-reduction
Add HUF_writeCTable_wksp() function
2021-03-05 12:35:09 -08:00
Nick Terrell 5df2a21f1e Add HUF_writeCTable_wksp() function
This saves ~700 bytes of stack space in HUF_writeCTable.
2021-03-05 10:29:18 -08:00
Nick TerrellandGitHub e50f88ca4c Merge pull request #2522 from terrelln/stack-reduction
Reduce stack usage of ZSTD_buildCTable()
2021-03-04 20:55:58 -08:00
Nick Terrell 27498ff00f Reduce stack usage of ZSTD_buildCTable()
It is a stack high-point for some compression strategies and has an easy
fix. This moves the normalized count into the entropy workspace.
2021-03-04 16:12:11 -08:00
Yann ColletandGitHub c4d54ab9bf Merge pull request #2518 from facebook/seekTable
New direct seekTable access methods
2021-03-04 15:29:23 -08:00