Commit Graph
689 Commits
Author SHA1 Message Date
Nick Terrell 1eba76a2d1 [linux-kernel] Don't add -O3 to CFLAGS
It is no longer necessary to get good performance, there is only a small
speed difference between -O2 and -O3, so just stick to the default of
-O2.

This also fixes the stack space usage on parisc. The compiler was buggy
for -O3 and used ~3KB of stack space for several functions. With -O2 the
problem is completely resolved, and stack space is back to a few hundred
bytes.

Additionally, we get a large code size win on gcc:

| Compiler | Before (Bytes) | After (Bytes) | Delta (Bytes) |
|----------|----------------|---------------|---------------|
| gcc-11   |         952754 |        738954 |       -213800 |
| clang-12 |         976290 |        938826 |        -37464 |
2021-11-16 14:29:35 -08:00
Nick Terrell b9302410bf [linux-kernel] Don't inline function in zstd_opt.c
The optimal parser is unlikely to be used in the linux kernel in
practice. There is no reason these functions should be force inlined,
since we aren't gaining anything, and are losing build size.

| Compiler | Before (Bytes) | After (Bytes) | Delta (Bytes) |
|----------|----------------|---------------|---------------|
| gcc-11   |        1142090 |        952754 |       -189336 |
| clang-12 |        1228402 |        976290 |       -252112 |

This is a temporary solution pending the resolution of Issue #2862 in
the `dev` branch.
2021-11-15 20:37:58 -08:00
Nick Terrell 20821a46f4 [lib] Make lib compatible with -Wfall-through excepting legacy
Switch to a macro `ZSTD_FALLTHROUGH;` instead of a comment. On supported
compilers this uses an attribute, otherwise it becomes a comment.

This is necessary to be compatible with clang's `-Wfall-through`, and
gcc's `-Wfall-through=2` which don't support comments. Without this the
linux build emits a bunch of warnings.
2021-09-23 11:54:14 -07:00
Nick Terrell 1715601e55 [contrib][linux] Reduce stack usage by 80 bytes
Instead of calling `ZSTD_compress_advanced()` and
`ZSTD_initCStream_advanced()`, which each take a `ZSTD_parameters` by
value, use the new advanced API.

Stack usage went from 2024 -> 1944.
2021-09-22 18:18:20 -07:00
Nick Terrell 07d7ebe448 [contrib][linux] Fix up SPDX license identifiers
Correctly identify that we are GPL v2+ or BSD 3 clause, as pointed out
in issue #2663.
2021-09-22 15:06:08 -07:00
Nick Terrell 67a426c322 [linux-kernel] Replace kernel-style comments
Replace kernel-style comments with regular comments.

E.g.

```
/** Before */

/* After */

/**
 * Before
 */

/*
 * After
 */

/***********************************
 * Before
 ***********************************/

/* *********************************
 * After
 ***********************************/
```
2021-04-29 15:53:38 -07:00
Nick Terrell 4432dac93b [contrib][linux-kernel] Add zstd_min_clevel() and zstd_max_clevel() 2021-03-30 10:37:45 -07:00
Nick Terrell de9de869a3 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:37:39 -07:00
Nick Terrell c1a244d534 Merge pull request #2539 from terrelln/linux-kernel-fixes
Fixes for the next linux kernel patch version
2021-03-24 16:29:38 -07:00
Nick Terrell cd1551d261 [lib][tracing] Add ZSTD_NO_TRACE macro
When defined, it disables tracing, and avoids including the header.
2021-03-16 11:47:27 -07:00
Nick Terrell 7222614a19 [contrib][freestanding] Remove tracing support
Remove tracing support from `freestanding.py` to keep things simple.
2021-03-16 11:47:27 -07:00
Nick Terrell e4b914e663 [contrib][linux] Expose zstd headers to avoid duplication
Expose the zstd headers in `include/linux` to avoid struct duplication.
This makes the member names not follow Kernel style guidelines, and
exposes the zstd symbols. But, the LMKL reviewers are okay with that.
2021-03-16 11:47:22 -07:00
Nick Terrell 49a9e070f5 [contrib][linux-kernel] Update test include stubs
Update the test include stubs so they are able to run the current zstd
version in the kernel, so I can compare stack usage.
2021-03-16 11:40:24 -07:00
Nick Terrell d2dd35ae65 [contrib][linux-kernel] Fix unaligned.h
Fix the `unaligned.h` shim in the tests that was causing corruption in
the tests. Note that this is a problem with the test shim, not the
kernel code.
2021-03-16 11:40:24 -07:00
Fotis Xenakis 3c6f5d5eca Fix seekable test to provide valid descriptor 2021-03-13 00:00:08 +02:00
Fotis Xenakis 21697b9c9e Fix seek table descriptor check when loading 2021-03-12 23:07:15 +02:00
Yann Collet 2fa4c8c405 added code comments for new API ZSTD_seekTable 2021-03-03 22:54:04 -08:00
Yann Collet 6e390ced1f Merge branch 'seekTable' of github.com:facebook/zstd into seekTable 2021-03-03 22:44:38 -08:00
Yann Collet 16ec1cf355 added test case for seekTable API
and simple roundtrip test
2021-03-03 18:56:23 -08:00
Yann Collet 713d4953f7 fixed gcc-7 conversion warning 2021-03-03 18:00:41 -08:00
Yann Collet 6c0bfc468c fixed wrong assert condition 2021-03-03 15:30:55 -08:00
Yann Collet a1d7b9d654 fixed gcc conversion warnings 2021-03-03 15:17:12 -08:00
Yann Collet 24d59a655d Merge branch 'dev' into seekTable 2021-03-03 15:08:40 -08:00
Yann Collet ac95a30455 various minor style fixes 2021-03-02 16:03:18 -08:00
Yann Collet 029f974ddc strengthen compilation flags 2021-03-02 15:43:52 -08:00
Yann Collet c7e42e147b fixed const guarantees
read-only objects are properly const-ified in parameters
2021-03-02 15:24:30 -08:00
Yann Collet a80b10f5e6 fix potential leak on exit 2021-03-02 15:03:37 -08:00
Sen Huang 527a20c3cd Fix seekable decompress hanging 2021-03-02 14:30:03 -08:00
Martin LindsayandSen Huang 3cbdbb888b ZSTD_seekable_decompress() example that hangs. 2021-03-02 14:25:17 -08:00
Yann ColletandGitHub ce6d1b9376 Merge pull request #2113 from mdittmer/expose-seek-table
[contrib] Support seek table-only API
2021-03-02 10:50:47 -08:00
Stephen Kitt adb54293ab Stop using deprecated reset?Stream functions
These are replaced by the corresponding context resets. When
converting resetCStream, CCtx_setPledgedSrcSize isn't called if the
source size is "unknown".

This helps reduce the reliance on "static only" symbols, as well as
reducing the use of deprecated functions.

Signed-off-by: Stephen Kitt <steve@sk2.org>
2021-02-23 21:56:01 +01:00
Nick Terrell 54a4998a80 Add basic tracing functionality 2021-02-05 16:28:52 -08:00
Nick Terrell b45d22c851 [contrib][recovery] Add recovery_directory program
This program takes a file with concatenated zstd frames and splits the
file up by frame. E.g. if `dir.zst` has 4 frames:

```
> ./recover_directory dir.zst recovery/file
Recovering 4 files...
Recovered recovery/file0
Recovered recovery/file1
Recovered recovery/file2
Recovered recovery/file3
Complete
```
2021-01-15 08:45:22 -08:00
senhuang42 4f7584e7a3 Allow freestanding lib script regex to detect XXH64( 2021-01-07 12:29:12 -05:00
Nick Terrell 66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
Yann Collet 0b39531d75 moving all references to release branch
was previously `master`
2020-12-16 23:00:35 -08:00
senandGitHub f34d2f4192 Merge pull request #2408 from senhuang42/seekable_hang_fix
Remove possibility of hanging when using seekable decompression
2020-12-07 08:46:27 -05:00
senhuang42 26f89d47aa Clean up makefile for seekable tests 2020-12-03 09:25:04 -05:00
senhuang42 152b55879c Add unit tests to seekable 2020-12-02 15:33:12 -05:00
Nick Terrell 10f87d2f43 [contrib][linux] Add SPDX-License-Identifier 2020-12-02 12:00:44 -08:00
Nick Terrell d0cc2e3868 [contrib][linux] Fix make import-upstream 2020-12-02 11:43:28 -08:00
Nick Terrell 3256c00c6e [contrib][linux] Follow style guide more closely 2020-12-02 11:07:38 -08:00
Nick Terrell 2c570e7f87 [contrib][linux] Add a make target to import upstream 2020-12-02 10:42:33 -08:00
senhuang42 9db49a3989 Add a forward progress requirement bound to seekable streaming decompression 2020-12-02 12:24:16 -05:00
Nick Terrell 87b38156be [contrib][linux-kernel] Fix test warnings 2020-12-02 01:12:42 -08:00
Nick Terrell f58e9a9830 [contrib][linux] Add wrapper API
Add the kernel wrapper API. This keeps the same API and semantics as the
existing kernel API with name changes to be more kernel style and avoid
symbol collisions with zstd.
2020-12-02 01:08:08 -08:00
Nick Terrell 6a1e526ea7 [lib] Add ZSTD_COMPRESS_HEAPMODE tuning parameter 2020-09-24 19:42:04 -07:00
Nick Terrell b841387218 [freestanding] Improve macro resolution to handle #if X 2020-09-24 19:42:04 -07:00
Nick Terrell caecd8c211 Allow user to override ASAN/MSAN detection
Rename ADDRESS_SANITIZER -> ZSTD_ADDRESS_SANITIZER and same for
MEMORY_SANITIZER. Also set it to 0/1 instead of checking for defined.
This allows the user to override ASAN/MSAN detection for platforms that
don't support it.
2020-09-24 19:42:04 -07:00
Nick Terrell 683150e59f [linux-kernel] Avoid including <string.h> in the tests 2020-09-24 19:42:04 -07:00