ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,725 Commits 17 Branches 77 Tags
e3d1f7057dd8cddf28bb3d40fba1ea3dc3f94de4
Commit Graph
11725 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann Collet 241c57adff playtest : minor cosmetic changes 2017-10-16 14:01:42 -07:00
Nick Terrell 7f961ba6cd Don't allow default tables to repeat 
It isn't useful in any case to repeat default tables.
Saves a few bytes on Silesia, since we don't trigger the dictionary
heuristic.

Before: 211988480 => 73651998 bytes
After:  211988480 => 73651721 bytes
 2017-10-16 11:37:56 -07:00
Yann ColletandGitHub b7977e348d Merge pull request #889 from terrelln/bug-fix 
Fix invalid use of dictionary offcode table
 2017-10-14 01:29:42 -07:00
Yann Collet fc8d293460 dictionary compression use correct file size estimation 
when determining compression parameters
to compress one file only.

For multiple files, it still "bets" that files are going to be small.

There was also a bug recently added in ZSTD_CCtx_loadDictionary_advanced()
making it incapable to use pledgedSrcSize to determine compression parameters.
 2017-10-14 01:21:43 -07:00
Yann Collet 5eed8e7a55 changed API comments to invite using macro ZSTD_CONTENTSIZE_UNKNOWN 
to mean "pledgedSrcSize is not known at init time" instead of `0`.
Note that, a few prototypes created and documented with `0` to mean "unknown" still interpret "0" as unknown,
to avoid breaking 3rd party applications which depend on this behavior.
But this value is no longer recommended to mean "unknown".

In some future version, it might be possible to switch "0" to mean "empty",
as is already the case for several prototypes.
The advantage is that pledgedSrcSize field would have same behavior accross entire API,
making it easier to reason about.

Note that all concerned prototypes belong to the "experimental" API section.

srcSize is controlled at end of compression,
so if someone uses "0" to mean "unknown" while it effectively means "empty",
this is immediately caught by the compression function, which generates an error code : ZSTD_ERROR_srcSize_wrong
 2017-10-14 00:32:06 -07:00
Yann Collet 9ef32b3cf1 minor : zstd -l -v display each file name 2017-10-14 00:02:32 -07:00
Yann Collet 43914f6a20 add test for presence of decompressed size field in header 
previous test was "display" only, it wouldn't trigger an error in case of missing header.
 2017-10-13 23:47:01 -07:00
Yann Collet beb9b4b398 fixed ZSTDMT_initCStream() when contentSizeFlag==1 by default 
and a wrong test in zstreamtest --mt
 2017-10-13 19:09:30 -07:00
Yann Collet 213ef3b510 fixed ZSTD_initCStream_advanced() behavior, which depends on contentSizeFlag, 
and a stream fuzzer test, which was incorrect
(relied on 0 being unconditionnally transformed into `ZSTD_CONTENTSIZE_UNKNOWN`)
 2017-10-13 19:01:58 -07:00
Yann Collet 3c1e3f8ec9 contentSizeFlag enabled by default would also fail for streaming and MT operations 
fixed
 2017-10-13 18:32:06 -07:00
Yann Collet fb44516641 ensure fParams.contentSizeFlag starts at 1 
such default was failing for ZSTD_compressBegin/ZSTD_compressContinue
fixed too
 2017-10-13 17:39:13 -07:00
Yann Collet dd18d73e7e fileio: content size is enabled by default 2017-10-13 16:32:18 -07:00
Nick Terrell ced6e6189c Add DEBUGLOG() that prints FSE encoding types 2017-10-13 14:55:23 -07:00
Nick Terrell 24ac2dbd2a Fix invalid use of dictionary offcode table 
Fixes #888.
 2017-10-13 12:47:03 -07:00
Yann Collet df6b844b4a Merge branch 'dev' of github.com:facebook/zstd into dev 2017-10-13 02:36:40 -07:00
Yann Collet a9e5705077 minor code formatting 
added a trace during sequence encoding
 2017-10-13 02:36:16 -07:00
Yann ColletandGitHub 5da6693fb6 Merge pull request #885 from terrelln/dev 
[doc] Add images for release notes
 2017-10-09 16:50:09 -07:00
Nick Terrell 6d8778d8e7 [doc] Add images for release notes 2017-10-09 16:44:31 -07:00
Yann Collet 7f6a783862 fixed a small error in decodeCorpus 
a compressed block must be strictly smaller than its decompressed size.
 2017-10-07 15:19:52 -07:00
Yann Collet 4252621e26 playtests: do not use cat on large files 
some target have limitation making cat incompatible with large files
(namely debian hurd-i386)
 2017-10-05 20:21:59 -07:00
Yann ColletandGitHub ba55dc8b5e Merge pull request #882 from terrelln/list 
[zstdcli] Add window size to verbose list
 2017-10-04 13:39:30 -07:00
Nick Terrell 6dd958eea2 [zstdcli] Add window size to verbose list 
```
> zstd --list -v file1 file2 file3
*** zstd command line interface 64-bits v1.3.2, by Yann Collet ***
Window Size: 512.00 KB (524288 B)
Compressed Size: 0.02 KB (19 B)
Check: XXH64

Window Size: 8192.00 KB (8388608 B)
Compressed Size: 0.02 KB (19 B)
Check: XXH64

Window Size: 512.00 KB (524288 B)
Compressed Size: 0.01 KB (15 B)
Check: None

```
 2017-10-04 12:26:28 -07:00
Yann ColletandGitHub c85141caed Merge pull request #881 from terrelln/dev 
Ensure dictionary Huff table can encode any symbol
 2017-10-03 14:49:07 -07:00
Nick Terrell a86a7097ec Ensure dictionary Huff table can encode any symbol 
* Ensure that the dictionary Huffman CTable has maxSymbolValue 255.
* Fix a stack buffer overflow during compression dictionary loading.
 2017-10-03 13:22:13 -07:00
Yann Collet 67478f4cb0 fixed minor conversion warnings for printf 
in debug mode
 2017-10-02 17:28:57 -07:00
Yann Collet 9b166d2291 Merge branch 'dev' of github.com:facebook/zstd into dev 2017-10-02 16:34:26 -07:00
Yann Collet 3b27ed41fd Merge branch 'srcSize' into dev 2017-10-02 16:34:14 -07:00
Yann Collet 7e00df4a49 bumped version number 
and updated NEWS in anticipation for release
 2017-10-02 16:27:25 -07:00
Yann ColletandGitHub 004fd34fd9 Merge pull request #876 from facebook/srcSize 
CLI Fix : srcSize written in frame headers when compressing multiple files
 2017-10-02 15:02:05 -07:00
Yann ColletandGitHub 7304a63c12 Merge pull request #879 from terrelln/block-size 
[libzstd] Set CLEVEL_CUSTOM correctly
 2017-10-02 14:51:47 -07:00
Nick Terrell 86e83e926f [libzstd] Set CLEVEL_CUSTOM correctly 
In `ZSTD_compressBegin_advanced()`, `ZSTD_parameters` are used to set the
compression parameters, but the level didn't get set to `CLEVEL_CUSTOM`, so
`ZSTD_compressBlock()` used the wrong parameters when checking the source
size.
 2017-10-02 13:43:30 -07:00
Yann Collet 4946993f87 removed isRegularFile parameter 
no longer useful : size of src is determined for each file.
 2017-10-02 12:29:25 -07:00
Yann Collet 7f580f9ee8 interruption handler and variable are static 2017-10-02 11:39:05 -07:00
Yann Collet 0d58aaf6f0 /contrib: fixed license header 
removed last reference to PATENTS file
 2017-10-02 02:07:17 -07:00
Yann Collet fe5444bc66 removed the statement for all versions of Visual Studio 2017-10-02 02:02:16 -07:00
Yann Collet 51d82d5516 same error in Visual Studio 2012 ... 2017-10-02 01:12:40 -07:00
Yann Collet ed7ae4c9bd The issue also impacts Visual Studio 2010 2017-10-02 00:45:28 -07:00
Yann Collet 6e7ba3df2f added (void)sig to avoid compilers complaining that sig is not used. 2017-10-02 00:19:47 -07:00
Yann Collet 82bc200f82 conditionnally removed invocation that generates a buggy warning with Visual Studio 2008 2017-10-02 00:02:24 -07:00
Yann Collet 5e4ad557bc Merge branch 'dev' of github.com:facebook/zstd into dev 2017-10-01 15:33:57 -07:00
Yann Collet bd18095edc blindfix for Visual : minor casting issue 
should not happen since SIGIGN is provided by <signal.h>,
so it should work "ouf of the box"
 2017-10-01 15:32:48 -07:00
Yann ColletandGitHub b600b5bafd Merge pull request #877 from facebook/compressBound 
added ZSTD_COMPRESSBOUND() as a macro
 2017-10-01 14:59:38 -07:00
Yann Collet 00fc1ba8dd cli: add Ctrl-C support, requested by @mike155 in #854 
Now, pressing Ctrl-C during compression or decompression
will erase operation artefact (unfinished destination file)
before leaving execution.
 2017-10-01 12:10:26 -07:00
Yann Collet 5db19b8685 added comment on ZSTD_COMPRESSBOUND() 
as requested by @terrelln
 2017-10-01 11:32:38 -07:00
Yann Collet 6e930c13d1 Merge branch 'dev' into compressBound 2017-10-01 11:24:02 -07:00
Yann Collet 76ac0b2d99 macro compatible with scenario where windowSize = 1024 (minimum) 2017-09-30 15:34:44 -07:00
Yann Collet dc404119e5 ZSTD_adjustCParams_internal : minor optimization 2017-09-30 15:02:40 -07:00
Yann ColletandGitHub 96bb29aa14 Merge pull request #878 from terrelln/adjust 
Don't `size -= 1` in ZSTD_adjustCParams()
 2017-09-30 14:52:54 -07:00
Nick Terrell c5d6dde502 Don't size -= 1 in ZSTD_adjustCParams() 
The window size could end up too small if the source size is 2^n + 1.

Credit to OSS-Fuzz
 2017-09-30 14:20:06 -07:00
Yann Collet ee1ed78fcb fix proper naming on FSE_createCTable() arguments in fse.h 2017-09-30 11:08:50 -07:00