Commit Graph
9353 Commits
Author SHA1 Message Date
W. Felix Handte 1778222446 Update the Swift Package Definition to Reflect Move 2022-01-05 18:32:56 -05:00
W. Felix Handte 8dd943e42c Improve Module Map File
This commit makes several changes:

1. It adds modules for the dictionary builder and errors headers.
2. It captures all of the macros that are used to configure these headers.
   When the headers are imported as modules and one of these macros is defined
   the compiler issues a warning that it needs to be defined on the CLI.
3. It promotes the modulemap file into the root of the lib directory.
   Experimentation shows that clang's `-fimplicit-module-maps` will find the
   modulemap when placed here, but not when it's put in a subdirectory.
2022-01-05 18:32:53 -05:00
Felix HandteandGitHub 7e679511a8 Merge pull request #2964 from felixhandte/noexecstack-all-archs
Mark Huffman Decoder Assembly `noexecstack` on All Architectures
2022-01-05 16:52:39 -05:00
W. Felix Handte ff5d1daf33 Clean Up Debugging Statements 2022-01-05 16:13:00 -05:00
W. Felix Handte ef1f9e80ff Restrict GNU-stack Note to GNU Assemblers 2022-01-05 16:03:32 -05:00
W. Felix Handte b12edddb37 Write GNU-stack Section on All ELF Architectures
Previously we did this only on Linux, which missed other Unices.
2022-01-05 15:44:40 -05:00
W. Felix Handte 4620ce6a9a Makefiles: Add noexecstack Options to Compilation and Linking
Hopefully this marks the binary artifacts `noexecstack` even on platforms
where binaries default to true.
2022-01-05 15:12:31 -05:00
Nick TerrellandGitHub c592b463ca Merge pull request #2972 from terrelln/meson-fix
[meson] Explicitly disable assembly for non clang/gcc copmilers
2022-01-05 09:27:33 -08:00
Yann ColletandGitHub 3c2c3fbe4a Merge pull request #2969 from facebook/fix2966_part1
fix performance issue in scenario #2966 (part 1)
2022-01-04 17:03:10 -08:00
Nick Terrell c4f5116e95 [meson] Explicitly disable assembly for non clang/gcc copmilers
After merging #2951 I realized that we will want to explicitly disable
assembly when we aren't including the assembly source file. Otherwise,
if some non clang/gcc compiler does support assembly, it would fail to
build.
2022-01-04 16:05:59 -08:00
Nick TerrellandGitHub 6c50042355 Merge pull request #2951 from eli-schwartz/msvc-no-asm
meson: fix MSVC support
2022-01-04 15:56:35 -08:00
W. Felix Handte 35208f702f Add Test Validating Stack is not Executable in playTests.sh 2022-01-04 16:57:35 -05:00
Yann Collet 41ad7332dd Updated expression for better readability 2022-01-04 09:07:11 -08:00
Yann Collet a0b9520e38 fullbench: added compress_freshCCtx scenario 2022-01-02 08:52:33 -08:00
Yann Collet 213dc6110f fixed fullbench freshCCtx scenario 2022-01-01 23:46:49 -08:00
Yann Collet 8c53e526db fix performance issue in scenario #2966 (part 1)
When re-using a compression state, across multiple successive compressions,
the state should minimize the amount of allocation and initialization required.

This mostly matters in situations where initialization is an overwhelming task
compared to compression itself.
This can happen when the amount to compress is small,
while the compression state was given the impression that it would be much larger,
aka, streaming mode without providing a srcSize hint.

This lean-initialization optimization was broken in 980f3bbf83 .

This commit fixes it, making this scenario once again on par with v1.4.9.

Note that this does not completely fix #2966,
since another heavy initialization, specific to row mode,
is also happening (and was not present in v1.4.9).
This will be fixed in a separate commit.
2021-12-31 15:16:19 -08:00
Yann Collet 6211bfee5e fixed backup prototype for POOL_sizeof() 2021-12-30 14:33:21 -08:00
Yann Collet b1978d60ee POOL_sizeof() only needs a const read-only reference 2021-12-30 14:08:51 -08:00
W. Felix Handte 9a9d1ec6f4 Mark Huffman Decoder Assembly noexecstack on All Architectures
Apparently, even when the assembly file is empty (because
`ZSTD_ENABLE_ASM_X86_64_BMI2` is false), it still is marked as possibly
needing an executable stack and so the whole library is marked as such. This
commit applies a simple patch for this problem by moving the noexecstack
indication outside the macro guard.

This commit builds on #2857.

This commit addresses #2963.
2021-12-29 17:47:12 -08:00
Yann ColletandGitHub fb14e2288e Merge pull request #2962 from facebook/seqStore_off
Refactor offset+repcode sumtype
2021-12-29 16:07:36 -08:00
Yann Collet ad7c9fc11e use ZSTD_memcpy(), for proper redirection within Linux Kernel 2021-12-28 17:41:47 -08:00
Yann Collet 8da414231d found a few more places which were dependent on seqStore offcode sumtype numeric representation 2021-12-28 17:03:24 -08:00
Yann ColletandGitHub 4f63cb9755 Merge pull request #2960 from fwessels/patch-1
Fix mini typo
2021-12-28 16:07:40 -08:00
Yann ColletandGitHub 0e83d1523b Merge pull request #2954 from facebook/storeSeq_ml
storeSeq & mlBase : clarity refactoring
2021-12-28 15:42:21 -08:00
Yann Collet de9f52e945 regroup all mentions of ZSTD_REP_MOVE within zstd_compress_internal.h 2021-12-28 13:47:57 -08:00
Yann Collet a34ccad9a6 fixed minor conversion warnings 2021-12-28 13:21:22 -08:00
Yann Collet 92a08eec72 abstracted storeSeq() sumtype numeric representation from zstd_lazy.c 2021-12-28 12:23:39 -08:00
Yann Collet e909fa627f abstracted storeSeq() sumtype numeric representation from zstd_opt.c 2021-12-28 12:14:33 -08:00
Yann Collet 681c81f06c abstracted storeSeq() sumtype numeric representation from decodecorpus.c 2021-12-28 11:58:33 -08:00
Yann Collet 6fa640ef70 separate newRep() from updateRep()
the new contracts seems to make more sense :
updateRep() updates an array of repeat offsets _in place_,
while newRep() generates a new structure with the updated repeat-offset array.

Most callers are actually expecting the in-place variant,
and a limited sub-section, in `zstd_opt.c` mainly, prefer `newRep()`.
2021-12-28 11:52:33 -08:00
Yann Collet 321583ccf5 fixed minor typecast warnings 2021-12-28 11:38:21 -08:00
Yann Collet b7630a474b abstracted usage of offBase sumtype within zstd_lazy.c 2021-12-28 10:59:47 -08:00
Yann Collet 435f5a2e6d fixed regression test assert
optLdm->offset might be == 0 in invalid case.
Only use STORE_OFFSET() after validating it's a correct case.
2021-12-28 09:55:31 -08:00
Frank WesselsandGitHub 7ee35bad6b Fix mini typo 2021-12-28 09:04:28 -08:00
Yann Collet 2068889146 created STORED_*() macros
to act on values stored / expressed in the sumtype numeric representation required by `storedSeq()`.

This makes it possible to abstract away this representation by using the macros to extract these values.

First user : ZSTD_updateRep() .
2021-12-28 06:59:07 -08:00
Yann Collet 1aed962216 introduce macros STORE_OFFSET() and STORE_REPCODE()
this meant to abstract the sumtype representation required
to transfert `offcode` to `ZSTD_storeSeq()`.

Unfortunately, the sumtype numeric representation is currently a leaky abstraction
that has permeated many other parts of the code,
especially within `zstd_lazy.c` and also within `zstd_opt.c` and `zstd_compress.c`.

While this PR makes a good job a transfering a large nb of call sites
to using the new macros, there are still a few sites where this transformation is more complex,
or where the numeric representation itself it used "as is".

One of the problematics area is the decision to use the numeric format of the sumtype
within the match finders of `zstd_lazy`.

This commit doesn't change the behavior, it only introduces and employes the macros,
but eventually the resulting code remains identical.

At target, if the numeric representation of the sumtype can be completely abstracted
and no other part of the code depends on it,
it will be possible to move it towards something slightly more efficient.
2021-12-23 22:03:30 -08:00
Yann Collet bec7bbb5a4 Merge branch 'dev' into seqStore_off 2021-12-23 18:03:17 -08:00
Yann Collet aeff128331 change seqDef.offset into seqDef.offBase
to better reflect the value stored in this field.
2021-12-23 17:56:08 -08:00
Yann Collet 75525fcb9f library optimization flag can be selected on command line again
`CFLAGS=-O0 make`
will now use `-O0` instead of enforcing `-O3`
which used to be the behavior before introduction of `libzstd.mk`.

This should result in faster tests,
since a few tests depend on this capability for faster roundtrips.
2021-12-23 17:43:12 -08:00
Yann Collet e145b58cfd changed seqDef.matchLength into seqDef.mlBase
since this is effectively what is stored in this field (== matchLength - MINMATCH).
This makes it clearer what needs to be done when reading from / writing to this field.
2021-12-23 13:39:46 -08:00
Yann Collet b77fcac61f change ZSTD_storeSeq() interface to accept matchLength
instead of mlBase.

This removes the need to do `- MINMATCH` at every call site.

The new interface contract is checked with an `assert()`.
2021-12-23 12:03:33 -08:00
Eli Schwartz 29e44bc554 meson: fix MSVC support
Regression from commit a5f2c45528. It is
not possible to unconditionally add the asm sources, since not all
compilers understand the .s file extension.

Specifically for meson, only compilers inheriting from the GNU mixin
will allow a .s file at configure time.

zstd doesn't support asm for MSVC for the same basic reason; if/when
Windows asm support is added, it would involve preprocessing with nasm,
most likely.
2021-12-22 20:20:30 -05:00
Yann ColletandGitHub eebf24bd2a Merge pull request #2947 from MehdiChinoune/patch-1
Fix zstd-static output name with MINGW/Clang
2021-12-22 11:13:40 -08:00
مهدي شينون (Mehdi Chinoune)andMehdi Chinoune 14a0eaf73b Fix zstd-static output name with MINGW/Clang 2021-12-22 10:16:41 +01:00
Yann Collet 41153071a0 updated manual 2021-12-21 08:52:50 -08:00
Yann Collet abc694f53e update man pages 2021-12-20 14:24:06 -08:00
Yann Collet 1219b7f3e3 updated changelog for v1.5.1 2021-12-20 12:28:27 -08:00
Yann ColletandGitHub 24960134d9 Merge pull request #2940 from facebook/revert-2885-limit-level-32bit-systems
Revert "Limit `ZSTD_maxCLevel` to 21 for 32-bit binaries."
2021-12-20 12:27:28 -08:00
Yann ColletandGitHub a9e43b37d0 Revert "Limit ZSTD_maxCLevel to 21 for 32-bit binaries." 2021-12-20 11:43:14 -08:00
Yann ColletandGitHub 85bd1e663b Merge pull request #2934 from facebook/lazy_rebalance
rebalance lazy compression levels
2021-12-20 11:37:44 -08:00