ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,245 Commits 17 Branches 77 Tags
39c091bc9e16d428407d3205364bce117afcb3aa
Commit Graph
11245 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandGitHub 39c091bc9e Merge pull request #4397 from xiaoge1001/free 
Fix several locations with potential memory leak
 2025-06-09 10:06:36 -07:00
shixuantong de8d9e8914 Fix several locations with potential memory leak 2025-06-09 21:23:23 +08:00
Yann ColletandGitHub ec8ad2e552 Merge pull request #4384 from xiaoge1001/dev 
update `--rm` cmd help info
 2025-06-08 12:39:26 -07:00
Yann ColletandGitHub 5b6fbf5f96 Merge pull request #4392 from mugitya03/MLK 
Fix potential memory leak in function `benchMem`
 2025-06-08 12:38:31 -07:00
Yann ColletandGitHub e23d0a9616 Merge pull request #4399 from zijianli1234/dev 
Improve speed of convertSequences() and get1BlockSummary() using RVV
 2025-06-08 12:38:02 -07:00
Yann ColletandGitHub bd894054c0 Merge pull request #4401 from mugitya03/MLK-1 
Release resources before returning
 2025-06-04 12:49:38 -07:00
jinyaoguo dd4cee9190 Release resources before returning 
In main, resources were freed on the success path but not in the error path.
This change ensures all allocated resources are released before returning.
 2025-06-03 15:28:11 -04:00
李子建 d95123f2e6 Improve speed of ZSTD_compressSequencesAndLiterals() using RVV 2025-06-02 17:21:02 +08:00
shixuantong 4bd5654e72 update --rm cmd help info 
Starting from cee6bec9fa, --rm is ignored when the output is `stdout`.
 2025-05-30 06:49:52 +08:00
jinyaoguo 4618b255ea Fix memory leak in function benchMem 
`speedPerRound` is allocated at the start of benchMem to collect per-round speeds,
but is never freed, causing a leak on each invocation.
 2025-05-25 15:21:23 -04:00
Dave VasilevskyandYann Collet f9938c217d lz4: Remove ancient test helpers 
Building lz4 as root was causing `make clean` to fail with permission
errors.

We used to have to install lz4 from source back in Ubuntu 14.04, but
nowadays the installed lz4 is fine. Get rid of ancient helpers and
cruft!
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 448a09ff78 seekable_format: Fix conversion warnings in parallel_compression 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 13cb7a10ae seekable_format: Add test for parallel_compression memory usage 
Use ulimit to fail the test if we use O(filesize) memory, rather than
O(threads).
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 01c973de8d seekable_format: Fix race in parallel_processing 
There was no memory barrier between writing and reading `done`, which
would allow reordering to cause races. With so little data to handle
after each job completes, we might as well just join.
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 6fc8455a72 seekable_format: Cleanup POOL in parallel_compression 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 2d4cff69c4 seekable_format: Make parallel_compression use memory properly 
Previously, parallel_compression would only handle each job's results
after ALL jobs were successfully queued. This caused all src/dst
buffers to remain in memory until then!

It also polled to check whether a job completed, which is racy without
any memory barrier.

Now, we flush results as a side effect of completing a job. Completed
frames are placed in an ordered linked-list, and any eligible frames
are flushed. This may be zero or multiple frames, depending on the
order in which jobs finish.

This design also makes it simple to support streaming input, so that
is now available. Just pass `-` as the filename, and stdin/stdout will
be used for I/O.
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet f5b6531902 seekable_format: Link against multi-threaded libzstd.a 
Some of these examples are intended to be parallel, and don't make
sense to link against single-threaded libzstd.

The filename of mt and nomt libzstd are identical, so it's still
possible to link against the single-threaded one, just harder.
 2025-05-07 22:01:49 -07:00
Dave VasilevskyandYann Collet 6b0039abcf seekable_format: Build with $(MAKE) 
This passes make flags, such as `-jN` for building in parallel, to
the underlying make.
 2025-05-07 22:01:49 -07:00
Thomas DevoogdtandYann Collet 769723aee2 [cmake] only require a CXX compiler when tests are build 
Signed-off-by: Thomas Devoogdt <thomas@devoogdt.com>
 2025-05-06 12:20:26 -07:00
Nobuhiro IwamatsuandYann Collet 2d224dc745 Add License variable to pkg-config file 
The pkg-config file has License variable that allows you to set the license for
the software. This sets 'BSD-3-Clause OR GPL-2.0-only' to License.

Ref: https://github.com/pkgconf/pkgconf/blob/master/man/pc.5#L116
Signed-off-by: Nobuhiro Iwamatsu <iwamatsu@nigauri.org>
 2025-05-06 12:16:28 -07:00
Yann ColletandGitHub e26dde3dcf Merge pull request #4378 from Cyan4973/fix_zstreamtest_minor 
fix minor warning in zstreamtest
 2025-04-29 14:01:14 -07:00
Yann ColletandGitHub 453e252015 Merge pull request #4371 from ecordonnier-sc/eco/fix-macos-duplicate-lc-rpath 
Fix duplicate LC_RPATH error on MacOS
 2025-04-29 14:00:41 -07:00
Yann Collet 7f907d5c23 fix minor warning in zstreamtest 2025-04-29 09:56:00 -07:00
Etienne Cordonnier 8929d3b09f Fix duplicate LC_RPATH error on MacOS 
After the update to MacOS 15.4, the dynamic loader dyld treats duplicated LC_RPATH as an error.
The `FLAGS` variable already contains `LDFLAGS`, thus using both `FLAGS` and `LDFLAGS`
duplicates all `LDFLAGS`, including `-Wl,rpath` parameters.

The duplicate LC_RPATH causes this kind of errors:

```
dyld[29361]: Library not loaded: @loader_path/../lib/libzstd.1.dylib
      Referenced from: <7131C877-3CF0-33AC-AA05-257BA4FDD770> /Users/foobar/...
      Reason: tried: '/Users/foobar/..../lib/libzstd.1.dylib' (duplicate LC_RPATH '/usr/mypath.../lib')
```

Closes https://github.com/facebook/zstd/issues/4369

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
 2025-04-18 15:59:06 +02:00
Yann ColletandGitHub d654fca786 Merge pull request #4362 from facebook/dependabot/github_actions/actions/setup-java-4.7.1 
Bump actions/setup-java from 4.7.0 to 4.7.1
 2025-04-14 03:19:07 -07:00
dependabot[bot]andGitHub 28ffb100ab Bump actions/setup-java from 4.7.0 to 4.7.1 
Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4.7.0 to 4.7.1.
- [Release notes](https://github.com/actions/setup-java/releases)
- [Commits](https://github.com/actions/setup-java/compare/3a4f6e1af504cf6a31855fa899c6aa5355ba6c12...c5195efecf7bdfc987ee8bae7a71cb8b11521c00)

---
updated-dependencies:
- dependency-name: actions/setup-java
  dependency-version: 4.7.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-04-14 05:21:33 +00:00
Yann ColletandGitHub f5d4da09f5 Merge pull request #4353 from afq984/sparse 
Check regular file for sparse support after opening
 2025-04-07 03:48:41 -07:00
Li-Yu Yu 2a12bbaf90 Update cli-tests/file-stat tests 2025-03-28 22:46:26 +00:00
Li-Yu Yu 3bd5aa3404 Check regular file for sparse support after opening 
A regular file may be created by the open call.
Checking after opening allows sparseFileSupport even
if dstFileName does not already exist.
 2025-03-28 21:23:10 +00:00
Yann ColletandGitHub c5926fbab8 Merge pull request #4349 from Cyan4973/devfd 
Support process substitution for `--filelist=`
 2025-03-28 10:35:45 -07:00
Yann Collet 2f9627863f update error message 2025-03-28 10:02:37 -07:00
Yann Collet 0bdeb1d204 fix test 2025-03-28 09:18:17 -07:00
Yann Collet 8626da73b6 add error message clarification for inputs as process substitution 2025-03-28 09:18:17 -07:00
Yann Collet a293cdcb85 added CI test 2025-03-28 09:18:17 -07:00
Yann Collet 0bde39a9b3 visual studio fix 2025-03-28 09:18:17 -07:00
Yann Collet 94cfa0b5a0 minor: restore some code comment 2025-03-28 09:18:17 -07:00
Yann Collet 76c2fdc7b7 better naming 
and more narrow scope of local variables
 2025-03-28 09:18:17 -07:00
Yann Collet 7630870b47 second implementation, better structured for improved maintenance 2025-03-28 09:18:17 -07:00
Yann Collet 165e52ce62 first implementation supporting Process Substitution 2025-03-28 09:18:17 -07:00
Yann ColletandGitHub 7f0519dcfd Merge pull request #4346 from Cyan4973/wintime 
fix a risk of overflow on a time counter on Windows
 2025-03-27 21:07:57 -07:00
Yann ColletandGitHub 23e8812f5b Merge pull request #4350 from Cyan4973/patchapply 
add --patch-apply command
 2025-03-27 09:57:20 -07:00
Yann Collet eb168a0afc add --patch-apply command 
as an equivalent for `-d --patch-from`.
Requested by @sergeevabc in #2173.
 2025-03-25 14:50:39 -07:00
Yann ColletandGitHub 12ea5f6e30 Merge pull request #4314 from facebook/dependabot/github_actions/ossf/scorecard-action-2.4.1 
Bump ossf/scorecard-action from 2.4.0 to 2.4.1
 2025-03-24 18:33:51 -07:00
Yann ColletandGitHub ec40252066 Merge pull request #4345 from Cyan4973/block128 
[doc] minor clarification for maximum block size
 2025-03-24 18:33:01 -07:00
Yann ColletandGitHub 64dc08f47f Merge pull request #4344 from Cyan4973/assert1 
add an assert
 2025-03-24 18:32:37 -07:00
Yann Collet 334af72280 update zstd --fast benchmark result 
use --fast=4, instead of --fast=3,
to make it more representative of a lzo equivalent scenario.
 2025-03-23 12:06:58 -07:00
Yann Collet c59f7ac51e update benchmark result 
brotli uses -1 level, like all other compressors
 2025-03-23 12:02:51 -07:00
Yann Collet 33f4d40d1e fix a risk of overflow on a time counter on Windows 
closes #4126
 2025-03-23 11:42:41 -07:00
Yann Collet bea1e942f7 [doc] minor clarification for maximum block size 
closes #4339
 2025-03-22 23:52:45 -07:00
Yann Collet 2fec3989c1 add an assert 
to help static analyzers understand there is no overflow risk there.
 2025-03-22 18:23:31 -07:00