ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6,401 Commits 17 Branches 77 Tags
641e5943093a776ee4bbe58a28aef472025a03e7
Commit Graph
62 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Nick Terrell 641e594309 [libzstd] Remove ZSTDMT from the shared object 
* Remove ZSTDMT from the shared object by default.
* Provide a macro `ZSTD_LEGACY_MULTITHREADED_API` to override it.
* Document it in `lib/README.md`.
 2019-04-07 18:47:52 -07:00
Nick TerrellandGitHub d5910a5d94 Merge pull request #1574 from terrelln/examples 
Stabilize ZSTD_getDictID_*() functions and clean up examples
 2019-04-05 23:29:32 -07:00
Nick TerrellandGitHub a63aaaa2cc Merge pull request #1573 from terrelln/regression 
[regression] Update results.csv for level 1 change
 2019-04-05 11:06:35 -07:00
Nick TerrellandGitHub dbc8a59a0a Merge pull request #1569 from terrelln/stable 
Stabilize the advanced API
 2019-04-05 10:47:48 -07:00
Nick TerrellandGitHub 72a3fbc0e4 Merge pull request #1562 from terrelln/2fast 
[libzstd] Speed up single segment zstd_fast by 5%
 2019-04-03 18:08:15 -07:00
Nick TerrellandGitHub 56261001ea Merge pull request #1567 from terrelln/examples2 
[examples] Update streaming_decompression.c
 2019-04-03 11:27:49 -07:00
Nick TerrellandGitHub 882ceb86bc Merge pull request #1566 from terrelln/examples 
[examples] Update multiple_streaming_compression.c
 2019-04-02 17:13:10 -07:00
Nick TerrellandGitHub 425ce5547c Merge pull request #1563 from terrelln/dms-sep 
[libzstd] Split out zstd_fast dict match state function
 2019-03-29 16:19:21 -06:00
Nick TerrellandGitHub 6625f3b390 Merge pull request #1561 from shakeelrao/fix-typo 
Update comments in zstd.h and fileio.c
 2019-03-28 23:42:16 -06:00
Nick TerrellandGitHub dcc6c7e9ae Merge pull request #1556 from terrelln/dictbuilder 
[cover] Improvements for small or homogeneous data
 2019-03-25 15:08:32 -07:00
Nick TerrellandGitHub 440f390cba Merge pull request #1557 from terrelln/examples 
[examples] Update streaming_compression to the new API
 2019-03-25 15:07:35 -07:00
Nick TerrellandGitHub 7186a50775 Merge pull request #1559 from shakeelrao/reject-dict 
[CLI] ensure dictionary and input file are different
 2019-03-25 15:06:58 -07:00
Nick TerrellandGitHub d97605ad85 Merge pull request #1558 from nehaljwani/fix-version-soversion-libzstd 
[libzstd] Specify soversion and version corectly for CMake build
 2019-03-23 13:32:39 -07:00
Nick TerrellandGitHub 0c7668cd06 Merge pull request #1555 from terrelln/load-dict 
[lib] Allow ZSTD_CCtx_loadDictionary() to be called before parameters are set
 2019-03-21 17:52:57 -07:00
Nick TerrellandGitHub 041bc0bdb7 Merge pull request #1551 from terrelln/stream-wrap 
Refactor old stream API to completely wrap the advanced API
 2019-03-21 16:10:00 -07:00
Nick TerrellandGitHub 0dd3588acc Merge pull request #1553 from shakeelrao/legacy-bound 
Add legacy support to decompressBound
 2019-03-19 10:53:12 -07:00
Nick TerrellandGitHub f52a7d8faa Merge pull request #1547 from shakeelrao/fix-error 
Fix incorrect error code in ZSTD_errorFrameSizeInfo
 2019-03-15 10:57:49 -07:00
Nick TerrellandGitHub 45139e9fd5 Merge pull request #1550 from terrelln/cparams-cdict 
[libzstd] Allow compression parameters to be set with a cdict
 2019-03-13 19:17:22 -07:00
Nick TerrellandGitHub ea3b81f02c Merge pull request #1549 from terrelln/ref-cdict-free 
[libzstd] Free local cdict when referencing cdict
 2019-03-13 16:08:54 -07:00
Nick TerrellandGitHub 13cc7c610e Merge pull request #1543 from shakeelrao/dev 
Provide a function to estimate decompressed size.
 2019-03-04 08:00:47 -08:00
Nick TerrellandGitHub 8b7fa49590 Merge pull request #1541 from terrelln/fuzz 
[fuzz] Add --enable-fuzzer for clang fuzzing
 2019-02-28 11:25:27 -08:00
Nick TerrellandGitHub be3bd70c57 Merge pull request #1532 from terrelln/cctx-params 
[libzstd] Rename ZSTD_CCtxParam_* to ZSTD_CCtxParams_*
 2019-02-20 10:46:46 -08:00
Nick TerrellandGitHub a36d9302ce Merge pull request #1531 from terrelln/appveyorTest 
Fixes for Windows release
 2019-02-19 19:42:07 -08:00
Nick TerrellandGitHub eb3a7a3827 Merge pull request #1530 from terrelln/param-order 
Clean up parameter code
 2019-02-19 14:40:10 -08:00
Nick TerrellandGitHub 54e9412ddd Merge pull request #1525 from terrelln/enable-huf-flag 
Fix optimal parser prices with uncompressed literals
 2019-02-15 20:21:08 -05:00
Nick TerrellandGitHub 4c870b70fc Merge pull request #1522 from terrelln/enable-huf-flag 
Add ZSTD_c_literalCompressionMode flag
 2019-02-13 19:02:32 -05:00
Nick TerrellandGitHub ae7380d2af Merge pull request #1519 from terrelln/play-bsd 
[cirrus] Run playTests.sh
 2019-02-13 16:21:14 -05:00
Nick TerrellandGitHub 32a98bafdc Merge pull request #1521 from bket/lstat 
Detection of symbolic links on OpenBSD
 2019-02-11 20:38:49 -05:00
Nick TerrellandGitHub a17369c7dc Merge pull request #1518 from bket/non-portable 
'head -c BYTES' is non portable
 2019-02-11 15:43:03 -05:00
Nick TerrellandGitHub 197a5737c8 Merge pull request #1516 from terrelln/dict-doc 
[zdict] Improve documentation
 2019-02-01 19:04:05 -05:00
Nick TerrellandGitHub 5c1148a6a5 Merge pull request #1515 from thatsafunnyname/patch-1 
-Wformat-security not needed with -Wformat=2
 2019-02-01 13:41:27 -05:00
Nick TerrellandGitHub fafa322456 Merge pull request #1511 from lwhsu/cirrus-ci 
Change Cirrus CI's badge to dev branch
 2019-01-30 16:53:13 -05:00
Nick TerrellandGitHub 79463a65e9 Merge pull request #1501 from lwhsu/cirrus-ci 
Add Cirrus-CI config and build status badge for testing on FreeBSD
 2019-01-30 14:15:02 -05:00
Nick TerrellandGitHub 87204a2afd Merge pull request #1467 from terrelln/regression 
[regression] Update results.csv
 2018-12-20 18:14:13 -08:00
Nick TerrellandGitHub 3a4634f2af Merge pull request #1459 from terrelln/destroy 
[zstdcli] Refuse to overwrite input file
 2018-12-18 17:03:54 -08:00
Nick TerrellandGitHub 7f6d11e1e2 Merge pull request #1450 from terrelln/no-progress 
[zstdcli] Add --no-progress flag
 2018-12-14 11:50:59 -08:00
Nick TerrellandGitHub 75fa3f2eb7 Merge pull request #1446 from terrelln/overflow 
[libzstd] Fix infinite loop in decompression
 2018-12-13 16:21:15 -08:00
Nick TerrellandGitHub 54ca4b3c5d Merge pull request #1445 from terrelln/regression 
[regression] add more methods
 2018-12-12 11:16:54 -08:00
Nick TerrellandGitHub c631fdec9f Merge pull request #1434 from terrelln/regression 
[regression] Add dictionary support
 2018-12-03 10:39:12 -08:00
Nick TerrellandGitHub 787532b91a Merge pull request #1418 from ryandesign/cpuid-i386 
Fix i386 build failure "Junk character 13"
 2018-11-16 11:01:46 -08:00
Nick TerrellandGitHub f15f1bfefb Merge pull request #1416 from terrelln/rsync 
- Factor out LDM's hash function for reuse
- Add rsyncable mode to zstdmt and expose it via the advanced API
- Fix `-B`/`--block-size` to actually set the job size
- Add rsyncable tests to `zstreamtest` and `playTests.sh`

Tested by:

```
> cat A.100MB B.100MB C.100MB D.100MB E.100MB | zstd --rsyncable -fo src/file.zst
/*stdin*\            : 48.22%   (524288000 => 252837782 bytes, src/file.zst)
> rsync -rc --stats src devbigvm:/data/users/terrelln/rsync-test
Total bytes sent: 252868779
total size is 252837782  speedup is 1.00
> echo test > test
> cat test A.100MB test B.100MB test C.100MB test D.100MB test E.100MB | zstd --rsyncable -fo src/file.zst
/*stdin*\            : 48.23%   (524288025 => 252838025 bytes, src/unicorn.tar.zst)
> rsync -rc --stats src devbigvm:/data/users/terrelln/rsync-test
Total bytes sent: 4605696
total size is 252838025  speedup is 53.60
```

Close #1155.
 2018-11-14 18:43:12 -08:00
Nick TerrellandGitHub ed39c0e04b Merge pull request #1382 from jonesz/ticket-compiler-rt-fuzzer 
Fix libFuzzer location in makefile.
 2018-10-22 10:48:56 -07:00
Nick TerrellandGitHub e06f91a169 Merge pull request #1343 from terrelln/faster-ci 
[zstreamtest] Reduce memory of newapi tests
 2018-09-27 19:29:02 -07:00
Nick TerrellandGitHub f884e9dd8e Merge pull request #1321 from bket/non-portable 
'head -c BYTES' is non-portable.
 2018-09-17 14:40:27 -07:00
Nick TerrellandGitHub 8f27e8cf3d Merge pull request #1322 from azat-archive/seekable-fixes-pull 
Fixes read write past end of input buffer.
 2018-09-17 11:04:51 -07:00
Nick TerrellandGitHub d492ef4e07 Merge pull request #1306 from jennifermliu/dev 
Change default f of FASTCOVER to 20
 2018-09-04 18:03:36 -07:00
Nick TerrellandGitHub 7a02df8dbe Merge pull request #1302 from jennifermliu/splitpoint 
Always use splitPoint=1.0 for non-optimize cover and fastcover and some minor changes on comments
 2018-09-04 16:59:41 -07:00
Nick TerrellandGitHub 1420129fda Merge pull request #1258 from jennifermliu/fastCover 
Use CDict instead of CCtx in analyzeEntropy
 2018-07-31 14:31:05 -07:00
Nick TerrellandGitHub fc685d6b94 Merge pull request #1257 from jennifermliu/fastCover 
Increment frequency for every dmer occurence within same sample
 2018-07-30 16:03:18 -07:00
Nick TerrellandGitHub 92633789f6 Merge pull request #1254 from jennifermliu/fastCover 
Save segmentFreqs in ctx instead of malloc and memset in SelectSegment
 2018-07-27 18:22:52 -07:00
Nick TerrellandGitHub 3abfbee215 Merge pull request #1250 from jennifermliu/fastCover 
Add Fast Cover Dictionary Builder
 2018-07-27 15:11:57 -07:00
Nick TerrellandGitHub 7974628587 Merge pull request #1252 from GeorgeLu97/bench-quiet-display-fix 
Fixes bug
 2018-07-26 16:12:15 -07:00
Nick TerrellandGitHub 77068a8447 Merge pull request #1246 from jennifermliu/benchmark 
Benchmark dictionary builders
 2018-07-20 18:09:31 -07:00
Nick TerrellandGitHub 2e47865d7d Merge pull request #1245 from terrelln/disable-arm 
Revert "attempt to re-enable arm64 tests"
 2018-07-19 15:32:53 -07:00
Nick TerrellandGitHub 4d1ad5cdb2 Merge pull request #1238 from jennifermliu/random 
Add random dictionary builder
 2018-07-19 13:52:15 -07:00
Nick TerrellandGitHub 45821fac0c Merge pull request #1225 from jennifermliu/dev 
Split samples when building dictionary for COVER
 2018-07-13 13:26:15 -07:00
Nick TerrellandGitHub aa2415402d Merge pull request #1235 from ldv-alt/dev 
tests: use /dev/zero instead of /dev/random on all systems except GNU/Hurd
 2018-07-12 16:56:54 -07:00
Nick TerrellandGitHub b426bcc097 [zstdmt] Fix jobsize bugs (#1205) 
[zstdmt] Fix jobsize bugs

* `ZSTDMT_serialState_reset()` should use `targetSectionSize`, not `jobSize` when sizing the seqstore.
  Add an assert that checks that we sized the seqstore using the right job size.
* `ZSTDMT_compressionJob()` should check if `rawSeqStore.seq == NULL`.
* `ZSTDMT_initCStream_internal()` should not adjust `mtctx->params.jobSize` (clamping to MIN/MAX is okay).
 2018-06-25 15:21:08 -07:00
Nick TerrellandGitHub 06b70179da Work around bug in zstd decoder (#1147) 
Work around bug in zstd decoder

Pull request #1144 exercised a new path in the zstd decoder that proved to
be buggy. Avoid the extremely rare bug by emitting an uncompressed block.
 2018-05-23 18:02:30 -07:00
Nick TerrellandGitHub 3c3f59e68f Enforce pledgeSrcSize whenever known (#1106) 
The test fails before the patch and passes after.

Fixes #1095.
 2018-04-12 16:02:03 -07:00
Nick TerrellandGitHub aa4dbd09a1 Pull job/overlap log logic into common function (#1055) 
Prepares for LDM integration by separating the job size and overlap logic
into helper functions.
 2018-03-19 15:56:36 -07:00
Nick TerrellandGitHub 90d38f6a53 Merge pull request #945 from terrelln/dev 
Fix cdict compressor repcodes
 2017-12-13 14:24:21 -08:00