Commit Graph
5183 Commits
Author SHA1 Message Date
Yann Collet 3cdb342bd5 disable aarch64 test on travis CI
there's a bug on travis' ld
> collect2: error: ld terminated with signal 11 [Segmentation fault], core dumped

I cannot reproduce it.
Note that travis' ld version is 2.24
while the one in my Linux VM is 2.26.1.
ld is part of binutils, so it's pretty hard to change.

I would expect the bug to no longer be triggered after some random code change.
To be re-enabled later.
2018-06-07 12:18:47 -07:00
Yann Collet 36922b1369 added ld -v to aarch64fuzz
to better understand the travis CI linking bug
2018-06-06 18:15:57 -07:00
Yann Collet f1ea383f45 context can be sized down even with constant parameters
when parameters are "equivalent",
the context is re-used in continue mode,
hence needed workspace size is not recalculated.
This incidentally also evades the size-down check and action.

This patch intercepts the "continue mode"
so that the size-down check and action is actually triggered.
2018-06-06 15:04:12 -07:00
Yann Collet e5e17d009f changed member name to workSpaceOversizedDuration 2018-06-06 15:00:27 -07:00
Yann Collet f7392f3dc9 added test case 2018-06-05 14:53:28 -07:00
Yann Collet 3d523c741b added workSpaceTooLarge and workSpaceWasteful
also :
slightly increased speed of test fuzzer.16
2018-06-05 11:42:48 -07:00
Yann ColletandGitHub bdb673666f Merge pull request #1162 from facebook/corruptionFix
Fixed a corruption bug
2018-06-05 06:50:29 -07:00
Yann Collet 357c648c3f changed a few variable names
to unify naming convention
2018-06-04 17:10:50 -07:00
Yann ColletandGitHub 54f05641e3 Merge pull request #1161 from GeorgeLu97/paramgrillformat
Added zstdcli-style format for compression parameters in paramgrill
2018-06-04 14:56:23 -07:00
George Lu 3f054dceb4 forgot \n, ; 2018-06-04 13:38:37 -07:00
George Lu ddf143ba6a Update usage_advanced 2018-06-04 10:16:05 -07:00
Yann Collet 2108decb41 Fixed a nasty corruption bug
recently introduce into the new dictionary mode.
The bug could be reproduced with this command :
./zstreamtest -v --opaqueapi --no-big-tests -s4092 -t639

error was in function ZSTD_count_2segments() :
the beginning of the 2nd segment corresponds to prefixStart
and not the beginning of the current block (istart == src).
This would result in comparing the wrong byte.
2018-06-01 18:54:34 -07:00
George Lu 5e586aa025 -O# with no file fails 2018-06-01 18:02:56 -07:00
George Lu e355f0a580 Added Level Option 2018-06-01 14:27:53 -07:00
Yann ColletandGitHub d3615c28db Merge pull request #1159 from GeorgeLu97/suffixlist
Unknown Suffix Error
2018-06-01 14:00:10 -07:00
George Lu ae6d1fd3fa Add Error Print 2018-06-01 13:54:08 -07:00
Yann ColletandGitHub 2b5b2ea7df Merge pull request #1160 from GeorgeLu97/formatzstd
Added --format=zstd
2018-06-01 12:46:03 -07:00
George Lu 110ec9079d Remove echo 2018-06-01 12:45:02 -07:00
George Lu c9b4d20f02 Added new --zstd= format 2018-06-01 12:39:39 -07:00
George Lu 41249bf34b Modified Tests
Changed format as per suggestion and added second test
2018-06-01 10:54:51 -07:00
George Lu 53ea32cabe Suffix list test 2018-06-01 10:43:06 -07:00
Yann ColletandGitHub 143fc9ff6c Merge pull request #1157 from facebook/decompressedSize
minor : improved zstd.h API code comment
2018-06-01 10:28:17 -07:00
George Lu cfc3451dcc Added Test Case 2018-06-01 09:52:25 -07:00
George Lu 8984cc93d6 update display 2018-05-31 18:04:05 -07:00
George Lu 547096d672 update man 2018-05-31 18:03:52 -07:00
George Lu c9b1068298 removed strcats 2018-05-31 17:47:29 -07:00
George Lu cec205c842 copy paste 2018-05-31 17:39:36 -07:00
Yann ColletandGitHub 7c33b48221 Merge pull request #1151 from felixhandte/zstd-dfast-in-place-dict-goto
ZSTD_dfast: Support Searching the Dictionary Context In-Place (Alternate `goto` Implementation)
2018-05-31 17:37:09 -07:00
George Lu 5ff30fe2e5 Unknown Suffix Error
Changed so only compiled formats are printed in list of supported extensions
2018-05-31 16:13:36 -07:00
George Lu 140f59d38e Added --format=zstd
title
2018-05-31 15:29:35 -07:00
W. Felix Handte 48deab92de Allow Different Dict Attachment Cut-Offs for Different Strategies 2018-05-31 17:37:44 -04:00
W. Felix Handte f86796639e Remove Incorrect and Extraneous Repcode Bounds Check 2018-05-31 17:02:29 -04:00
Yann Collet 9b979d0e33 minor : improved API code comment
Extend guarantee that ZSTD_getFrameContentSize() will delivering the decompressed size
to any single-pass compression function.

Answer #1156
2018-05-31 11:12:18 -07:00
Yann ColletandGitHub ac4f7ead3b Merge pull request #1154 from facebook/altOpt
minor optimal parser optimization
2018-05-30 13:05:51 -07:00
Yann Collet 809f2f9322 minor update of literal cost function
just assert() there is no negative cost evaluation for literals
2018-05-29 15:34:50 -07:00
Yann Collet 463a0fe38b simplified optimal parser
removed "cached" structure.
prices are now saved in the optimal table.

Primarily done for simplification.
Might improve speed by a little.
But actually, and surprisingly, also improves ratio in some circumstances.
2018-05-29 14:07:25 -07:00
Yann ColletandGitHub bb6eaf6495 Merge pull request #1153 from facebook/dynThreshold
changed dynamic fse threshold for offset
2018-05-26 08:43:45 -07:00
Yann Collet e916c365a1 fixed minor visual warning 2018-05-25 20:43:09 -07:00
Yann ColletandGitHub c6ccf435ae Merge pull request #1152 from facebook/lowCompression
btultra accepts blocks with poorer compression ratio
2018-05-25 17:46:11 -07:00
Yann Collet a7fdceeccd changed dynamic fse threshold for offset
recent experienced showed that
default distribution table for offset
can get it wrong pretty quickly with the nb of symbols,
while it remains a reasonable choice much longer for lengths symbols.

Changed the formula,
so that dynamic threshold is now 32 symbols for offsets.
It remains at 64 symbols for lengths.

Detection based on defaultNormLog
2018-05-25 17:41:16 -07:00
Yann Collet 4b3a36d5d8 Merge branch 'dev' into lowCompression 2018-05-25 15:45:03 -07:00
Yann ColletandGitHub 744f36aac4 Merge pull request #1141 from facebook/staticDictCost
Random stuff on High Compression mode
2018-05-25 15:43:32 -07:00
Yann Collet 5f177f1c53 btultra accepts blocks with poorer compression ratio
zstd rejects blocks which do not compress by at least a certain amount.
In which case, such block is simply emitted uncompressed (even if a little bit of compression could be achieved).
This is better for decompression speed, hence for energy.

The logic is controlled by ZSTD_minGain().
The rule is applied uniformly, at all compression levels.

This change makes btultra accepts blocks with poor compression ratios.
We presume that users of btultra mode prefers compression ratio over some decompress speed gains.

The threshold for minimum gain is lowered for btultra
from s>>6 (~1.5% minimum gain)
to s>>7 (~0.8% minimum gain).

This is a prudent change.
Not sure if it's large enough.
2018-05-25 15:19:52 -07:00
Yann Collet e2c0e3d437 slightly nudge choices towards less sequences
also slightly improve some strange detrimental corner cases.
2018-05-25 14:52:21 -07:00
W. Felix Handte 5b292b5685 Check Long + 1 Matches in Both Prefix and Dict in Bothe Short Match Paths 2018-05-25 13:13:57 -04:00
W. Felix Handte 88b733b380 Interleave Prefix and Dict Searches 2018-05-25 13:13:57 -04:00
W. Felix Handte 1850025156 Refactor ZSTD_dfast to Use gotos 2018-05-25 13:13:57 -04:00
W. Felix Handte 43606f9c83 ... When I Said "HashTable", I Meant "ChainTable" 2018-05-25 13:13:28 -04:00
W. Felix Handte ec7efe88f5 Fix Off-By-One Error 2018-05-25 13:13:28 -04:00
W. Felix Handte 2bfe43267e Disallow Too-Long Repcodes When Using an Attached Dict 2018-05-25 13:13:28 -04:00