ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,230 Commits 17 Branches 77 Tags
2d4cff69c439d40b2bb39300f003522da8f2c6ab
Commit Graph
11230 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Dave VasilevskyandYann Collet 2d4cff69c4 seekable_format: Make parallel_compression use memory properly 
Previously, parallel_compression would only handle each job's results
after ALL jobs were successfully queued. This caused all src/dst
buffers to remain in memory until then!

It also polled to check whether a job completed, which is racy without
any memory barrier.

Now, we flush results as a side effect of completing a job. Completed
frames are placed in an ordered linked-list, and any eligible frames
are flushed. This may be zero or multiple frames, depending on the
order in which jobs finish.

This design also makes it simple to support streaming input, so that
is now available. Just pass `-` as the filename, and stdin/stdout will
be used for I/O.
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet f5b6531902 seekable_format: Link against multi-threaded libzstd.a 
Some of these examples are intended to be parallel, and don't make
sense to link against single-threaded libzstd.

The filename of mt and nomt libzstd are identical, so it's still
possible to link against the single-threaded one, just harder.
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 6b0039abcf seekable_format: Build with $(MAKE) 
This passes make flags, such as `-jN` for building in parallel, to
the underlying make.
 2025-05-07 22:01:49 -07:00
Thomas DevoogdtandYann Collet 769723aee2 [cmake] only require a CXX compiler when tests are build 
Signed-off-by: Thomas Devoogdt <thomas@devoogdt.com>
 2025-05-06 12:20:26 -07:00
Nobuhiro IwamatsuandYann Collet 2d224dc745 Add License variable to pkg-config file 
The pkg-config file has License variable that allows you to set the license for
the software. This sets 'BSD-3-Clause OR GPL-2.0-only' to License.

Ref: https://github.com/pkgconf/pkgconf/blob/master/man/pc.5#L116
Signed-off-by: Nobuhiro Iwamatsu <iwamatsu@nigauri.org>
 2025-05-06 12:16:28 -07:00
Yann ColletandGitHub e26dde3dcf Merge pull request #4378 from Cyan4973/fix_zstreamtest_minor 
fix minor warning in zstreamtest
 2025-04-29 14:01:14 -07:00
Yann ColletandGitHub 453e252015 Merge pull request #4371 from ecordonnier-sc/eco/fix-macos-duplicate-lc-rpath 
Fix duplicate LC_RPATH error on MacOS
 2025-04-29 14:00:41 -07:00
Yann Collet 7f907d5c23 fix minor warning in zstreamtest 2025-04-29 09:56:00 -07:00
Etienne Cordonnier 8929d3b09f Fix duplicate LC_RPATH error on MacOS 
After the update to MacOS 15.4, the dynamic loader dyld treats duplicated LC_RPATH as an error.
The `FLAGS` variable already contains `LDFLAGS`, thus using both `FLAGS` and `LDFLAGS`
duplicates all `LDFLAGS`, including `-Wl,rpath` parameters.

The duplicate LC_RPATH causes this kind of errors:

```
dyld[29361]: Library not loaded: @loader_path/../lib/libzstd.1.dylib
      Referenced from: <7131C877-3CF0-33AC-AA05-257BA4FDD770> /Users/foobar/...
      Reason: tried: '/Users/foobar/..../lib/libzstd.1.dylib' (duplicate LC_RPATH '/usr/mypath.../lib')
```

Closes https://github.com/facebook/zstd/issues/4369

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
 2025-04-18 15:59:06 +02:00
Yann ColletandGitHub d654fca786 Merge pull request #4362 from facebook/dependabot/github_actions/actions/setup-java-4.7.1 
Bump actions/setup-java from 4.7.0 to 4.7.1
 2025-04-14 03:19:07 -07:00
dependabot[bot]andGitHub 28ffb100ab Bump actions/setup-java from 4.7.0 to 4.7.1 
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4.7.0 to 4.7.1.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/3a4f6e1af504cf6a31855fa899c6aa5355ba6c12...c5195efecf7bdfc987ee8bae7a71cb8b11521c00)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-version: 4.7.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-14 05:21:33 +00:00
Yann ColletandGitHub f5d4da09f5 Merge pull request #4353 from afq984/sparse 
Check regular file for sparse support after opening
 2025-04-07 03:48:41 -07:00
Li-Yu Yu 2a12bbaf90 Update cli-tests/file-stat tests 2025-03-28 22:46:26 +00:00
Li-Yu Yu 3bd5aa3404 Check regular file for sparse support after opening 
A regular file may be created by the open call.
Checking after opening allows sparseFileSupport even
if dstFileName does not already exist.
 2025-03-28 21:23:10 +00:00
Yann ColletandGitHub c5926fbab8 Merge pull request #4349 from Cyan4973/devfd 
Support process substitution for `--filelist=`
 2025-03-28 10:35:45 -07:00
Yann Collet 2f9627863f update error message 2025-03-28 10:02:37 -07:00
Yann Collet 0bdeb1d204 fix test 2025-03-28 09:18:17 -07:00
Yann Collet 8626da73b6 add error message clarification for inputs as process substitution 2025-03-28 09:18:17 -07:00
Yann Collet a293cdcb85 added CI test 2025-03-28 09:18:17 -07:00
Yann Collet 0bde39a9b3 visual studio fix 2025-03-28 09:18:17 -07:00
Yann Collet 94cfa0b5a0 minor: restore some code comment 2025-03-28 09:18:17 -07:00
Yann Collet 76c2fdc7b7 better naming 
and more narrow scope of local variables
 2025-03-28 09:18:17 -07:00
Yann Collet 7630870b47 second implementation, better structured for improved maintenance 2025-03-28 09:18:17 -07:00
Yann Collet 165e52ce62 first implementation supporting Process Substitution 2025-03-28 09:18:17 -07:00
Yann ColletandGitHub 7f0519dcfd Merge pull request #4346 from Cyan4973/wintime 
fix a risk of overflow on a time counter on Windows
 2025-03-27 21:07:57 -07:00
Yann ColletandGitHub 23e8812f5b Merge pull request #4350 from Cyan4973/patchapply 
add --patch-apply command
 2025-03-27 09:57:20 -07:00
Yann Collet eb168a0afc add --patch-apply command 
as an equivalent for `-d --patch-from`.
Requested by @sergeevabc in #2173.
 2025-03-25 14:50:39 -07:00
Yann ColletandGitHub 12ea5f6e30 Merge pull request #4314 from facebook/dependabot/github_actions/ossf/scorecard-action-2.4.1 
Bump ossf/scorecard-action from 2.4.0 to 2.4.1
 2025-03-24 18:33:51 -07:00
Yann ColletandGitHub ec40252066 Merge pull request #4345 from Cyan4973/block128 
[doc] minor clarification for maximum block size
 2025-03-24 18:33:01 -07:00
Yann ColletandGitHub 64dc08f47f Merge pull request #4344 from Cyan4973/assert1 
add an assert
 2025-03-24 18:32:37 -07:00
Yann Collet 334af72280 update zstd --fast benchmark result 
use --fast=4, instead of --fast=3,
to make it more representative of a lzo equivalent scenario.
 2025-03-23 12:06:58 -07:00
Yann Collet c59f7ac51e update benchmark result 
brotli uses -1 level, like all other compressors
 2025-03-23 12:02:51 -07:00
Yann Collet 33f4d40d1e fix a risk of overflow on a time counter on Windows 
closes #4126
 2025-03-23 11:42:41 -07:00
Yann Collet bea1e942f7 [doc] minor clarification for maximum block size 
closes #4339
 2025-03-22 23:52:45 -07:00
Yann Collet 2fec3989c1 add an assert 
to help static analyzers understand there is no overflow risk there.
 2025-03-22 18:23:31 -07:00
Yann ColletandGitHub 6775af5b2e Merge pull request #4341 from clan/pragma 
lib/zstd.h: don't put pragma after static
 2025-03-20 19:49:08 -07:00
Z. Liu cd8ca9d92e lib/zstd.h: move pragma before static 
otherwise will cause dev-python/zstandard build failed when compiling with
clang as reported at https://bugs.gentoo.org/950259

the root cause is pycparser, which is unfixed since reported 2.5 years
ago, :(

Signed-off-by: Z. Liu <zhixu.liu@gmail.com>
 2025-03-20 03:40:42 +00:00
Yann ColletandGitHub eca205fc78 Merge pull request #4336 from Cyan4973/qsort_v2 
fix #4312 - musl compilation compatibility
 2025-03-12 10:31:54 -07:00
Yann Collet 4d53e27144 removed OpenBSD specificity 2025-03-12 09:55:14 -07:00
Yann Collet ddcb41a282 updated documentation 2025-03-11 14:10:35 -07:00
Yann Collet a9b8fef2e8 add support for C11 Annex K qsort_s() 
standard defined re-entrant variant of qsort().
Unfortunately, Annex K is optional.
 2025-03-11 14:10:35 -07:00
Yann Collet dcf675886b re-design qsort() selection in cover 
centralizes auto detection tests,
then distribute the outcome in all the places where it's active.
 2025-03-11 14:10:35 -07:00
Yann ColletandYann Collet 51b6e79f65 fix #4312 
and upgraded the test so that it would fail, both at compile time and at run time, without the fix
 2025-03-11 14:10:35 -07:00
Yann ColletandYann Collet 9a57bdc0bd attempt to reduce length of long cli tests by invoking -T0 2025-03-11 14:10:35 -07:00
Nick TerrellandNick Terrell 68dfd14a8c [linux] Opt out of row based match finder for the kernel 
The row based match finder is slower without SIMD. We used to detect the
presence of SIMD to set the lower bound to 17, but that breaks
determinism. Instead, specifically opt into it for the kernel, because
it is one of the rare cases that doesn't have SIMD support.
 2025-03-11 16:18:59 -04:00
Yann ColletandGitHub b27ba56a6d Merge pull request #4335 from Cyan4973/dec_mt_warning 
warn when requesting decompression with multiple threads
 2025-03-11 11:51:15 -07:00
Yann Collet 19541b4d1a fix minor unused variable warning 2025-03-11 00:17:45 -07:00
Yann Collet c583c2c39e warn when requesting decompression with multiple threads 
restore #2918 fix
 2025-03-10 22:50:00 -07:00
Yann ColletandGitHub f9a6031963 Merge pull request #4334 from Cyan4973/fix4332 
fix #4332: setting ZSTD_NBTHREADS=0 via environment variable
 2025-03-10 21:12:24 -08:00
Yann Collet 3c4096c83e fixed ShellCheck warning 2025-03-10 19:11:44 -07:00