ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,549 Commits 17 Branches 77 Tags
b0592024e2f5700a75e2daecb86ae12ad746ded7
Commit Graph
6549 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandGitHub b0592024e2 Merge pull request #1644 from chungy/chmod_600 
[programs] set chmod 600 after opening destination file
 2019-06-13 17:25:31 -07:00
Felix HandteandGitHub 344f942573 Merge pull request #1620 from michaelforney/test-no-threads 
Skip --adapt and --rsyncable tests when built without thread support
 2019-06-11 15:05:25 -04:00
Nick TerrellandGitHub 81276c4ee4 Merge pull request #1646 from terrelln/oss-fuzz 
[fuzz] Remove max_len from the options
 2019-06-10 22:13:16 -07:00
Yann ColletandGitHub c601622aa7 Merge pull request #1619 from j301scott/dev 
CMake: Check for existing custom target 'uninstall'
 2019-06-10 11:56:49 -07:00
Nick Terrell 6810dd6191 [fuzz] Remove max_len from the options 2019-06-10 11:05:45 -07:00
Felix HandteandGitHub e79dbb8074 Merge pull request #1642 from Absotively/dev 
VS2010 project settings improvements
 2019-06-10 10:39:25 -04:00
Mike Swanson af80f6dfac [programs] Don’t try to chmod a dst file if it can’t be opened 
Repairs an oversight in my last commit, thanks @Cyan4973
 2019-06-09 01:52:45 -07:00
Mike Swanson 3968160a91 [programs] set chmod 600 after opening destination file 
This resolves a race condition where zstd or unzstd may expose read
permissions beyond the original file allowed.  Mode 600 is used
temporarily during the compression and decompression write stage
and the new file inherits the original file’s mode at the end.

Fixes #1630
 2019-06-08 21:54:02 -07:00
Jen Pollock 176a543dde In VS2010+, turn off assembler output for libzstd & libzstd-dll, and don't export functions from libzstd 2019-06-07 17:21:51 -06:00
Nick TerrellandGitHub d06c15cc89 Merge pull request #1635 from terrelln/opt-opt 
[libzstd] Optimize ZSTD_insertBt1() for repetitive data
 2019-06-06 13:15:23 -07:00
Nick Terrell f3800ba073 [regression] Update results.csv 2019-06-05 20:40:52 -07:00
Nick TerrellandNick Terrell cdb9481e38 [libzstd] Optimize ZSTD_insertBt1() for repetitive data 
We would only skip at most 192 bytes at a time before this diff.
This was added to optimize long matches and skip the middle of the
match. However, it doesn't handle the case of repetitive data.

This patch keeps the optimization, but also handles repetitive data
by taking the max of the two return values.

```
> for n in $(seq 9); do echo strategy=$n; dd status=none if=/dev/zero bs=1024k count=1000 | command time -f %U ./zstd --zstd=strategy=$n >/dev/null; done
strategy=1
0.27
strategy=2
0.23
strategy=3
0.27
strategy=4
0.43
strategy=5
0.56
strategy=6
0.43
strategy=7
0.34
strategy=8
0.34
strategy=9
0.35
```

At level 19 with multithreading the compressed size of `silesia.tar` regresses 300 bytes, and `enwik8` regresses 100 bytes.
In single threaded mode `enwik8` is also within 100 bytes, and I didn't test `silesia.tar`.

Fixes Issue #1634.
 2019-06-05 20:34:00 -07:00
Yann ColletandGitHub c0eb6c9c81 Merge pull request #1632 from ephiepark/dev 
CircleCI : use custom Docker Image with pre-installed dependencies
 2019-06-05 12:30:21 -07:00
Felix HandteandGitHub 015985c1f9 Merge pull request #1633 from yk-tanigawa/patch-1 
Update README.md
 2019-06-05 14:55:19 -04:00
Yosuke TanigawaandGitHub f46a3603c9 Update README.md 
Fix minor typo in the documentation.
 2019-06-05 09:11:21 -07:00
Ephraim Park 286394aa9b Merge branch 'dev' of https://github.com/ephiepark/zstd into dev 2019-06-04 21:14:20 -07:00
Ephraim Park c96535f67e circleci : remove install dependencies command from config.yml 2019-06-04 21:13:40 -07:00
Ephraim Park 3e9572bfa9 circleci : use custom docker image with installations 2019-06-04 21:13:40 -07:00
Ephraim ParkandGitHub f1bf281715 Merge pull request #2 from ephiepark/circleci 
CircleCI : use custom Docker Image with pre-installed dependencies
 2019-06-04 20:53:29 -07:00
Yann ColletandGitHub c6ca65ab78 Merge pull request #1631 from ephiepark/dev 
zstdcli : expose cpu load indicator for each file on -vv mode
 2019-06-04 15:14:35 -07:00
Yann ColletandGitHub 61d912f3ff Merge pull request #1629 from facebook/streamSize 
Added comments on I/O buffer sizes for streaming
 2019-06-04 12:38:43 -07:00
Ephraim Park a6a7f08eaf circleci : remove install dependencies command from config.yml 2019-06-04 12:18:33 -07:00
Ephraim Park a371df534d circleci : use custom docker image with installations 2019-06-04 11:05:42 -07:00
Yann Collet b3af1873a0 better title formatting for html documentation 
must pay attention to /** and /*! patterns.
 2019-06-04 10:35:40 -07:00
Yann Collet b5c98fbfd0 Added comments on I/O buffer sizes for streaming 
It seems this is still a confusing topic,
as in https://github.com/klauspost/compress/issues/109 .
 2019-06-04 10:26:16 -07:00
Ephraim Park e498bb64b6 zstdcli : align output message with previous message 2019-06-04 09:42:18 -07:00
Ephraim Park a38601f05a zstdcli : remove extra semicolon 2019-06-04 09:25:16 -07:00
Ephraim Park ef2c836955 zstdcli : trying to fix declaration after statement 2019-06-04 09:23:49 -07:00
Ephraim Park 5fe974261b zstdcli : moving cpu load calculation from FIO_compressFilename_dstFile to FIO_compressFilename_internal 2019-06-04 09:04:35 -07:00
Ephraim Park 2841c79e87 zstdcli : fixing mixed declarations and code error 2019-06-03 17:36:09 -07:00
Ephraim Park 43d463eccb zstdcli : moving start time and cpu initialization after potential prompt 2019-06-03 17:31:14 -07:00
Ephraim Park 5029ebce2f zstdcli : exposing cpu load indicator for each file on -vv mode 2019-06-03 14:38:40 -07:00
Yann ColletandGitHub b13a9207f9 Merge pull request #1623 from facebook/fullbench 
fullbench minor improvements
 2019-05-31 14:40:19 -07:00
Yann Collet 78056a3e9c consistency : changed constant name to kSampleSizeDefault 
as suggested by @terrelln.

added <assert.h> include
 2019-05-30 10:13:56 -07:00
Yann Collet ab3625cffd define CONTROL, to ensure check cannot be disabled 
assert() can be disabled with NDEBUG
 2019-05-29 16:18:22 -07:00
Yann Collet 904d4da239 fullbench : minor refactoring, for readability 2019-05-29 16:08:49 -07:00
Yann Collet ed38b645db fullbench: pass proper parameters in scenario 43 2019-05-29 15:26:06 -07:00
Yann ColletandGitHub b84274da0f Merge pull request #1622 from facebook/nextToUpdate3 
nextToUpdate3 does not need to be maintained outside of zstd_opt.c
 2019-05-28 18:26:21 -07:00
Yann Collet c63081623f fullbench 43: ensure context is freed after each usage 2019-05-28 17:27:52 -07:00
Yann Collet eb6b199038 fullbench: added streaming with fresh CCtx scenario 
worst case situation, where context must be recreated every time
but without knowledge of the input size
thus sizing the context for some large input.
 2019-05-28 16:50:49 -07:00
Yann Collet 4c4149452c fullbench can now select sample size with command -B# 2019-05-28 16:37:03 -07:00
Yann Collet 9719fd616c removed nextToUpdate3 from ZSTD_window 
it's now a local variable of ZSTD_compressBlock_opt()
 2019-05-28 16:18:12 -07:00
Yann Collet 33dabc8c80 get bt matches : made it a bit clearer which parameters are input and output 2019-05-28 16:11:32 -07:00
Yann Collet 327cf6fac1 nextToUpdate3 does not need to be maintained outside of zstd_opt.c 
It's re-synchronized with nextToUpdate at beginning of each block.
It only needs to be tracked from within zstd_opt block parser.

Made the logic clear, so that no code tried to maintain this variable.

An even better solution would be to make nextToUpdate3
an internal variable of ZSTD_compressBlock_opt_generic().
That would make it possible to remove it from ZSTD_matchState_t,
thus restricting its visibility to only where it's actually useful.

This would require deeper changes though,
since the matchState is the natural structure to transport parameters into and inside the parser.
 2019-05-28 15:26:52 -07:00
Yann Collet 6453f8158f complementary code comments 
on variables used / impacted during maxDist check
 2019-05-28 14:12:16 -07:00
Yann Collet 4baecdf72a added comments to better understand enforceMaxDist() 2019-05-28 13:15:48 -07:00
Michael Forney 1bc77ee174 Skip --adapt and --rsyncable tests when built without thread support 
These options require ZSTD_MULTITHREAD.
 2019-05-26 03:20:37 -07:00
Jonathan Scott 3f5737afd7 CMake: Check for existing custom target 'uninstall' 2019-05-25 17:30:04 -07:00
Felix HandteandGitHub 21bb78e908 Merge pull request #1618 from felixhandte/zstgrep-f-flag 
Handle `-f` Flag in `zstgrep`
 2019-05-24 20:59:11 -04:00
W. Felix Handte 6a0638048a Add Test 2019-05-24 17:21:44 -04:00