ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5,161 Commits 17 Branches 77 Tags
2b5b2ea7df5b5a12e321e7cf41d6e6a6239a455d
Commit Graph
5161 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Yann ColletandGitHub 2b5b2ea7df Merge pull request #1160 from GeorgeLu97/formatzstd 
Added --format=zstd
 2018-06-01 12:46:03 -07:00
George Lu 41249bf34b Modified Tests 
Changed format as per suggestion and added second test
 2018-06-01 10:54:51 -07:00
Yann ColletandGitHub 143fc9ff6c Merge pull request #1157 from facebook/decompressedSize 
minor : improved zstd.h API code comment
 2018-06-01 10:28:17 -07:00
George Lu cfc3451dcc Added Test Case 2018-06-01 09:52:25 -07:00
George Lu 8984cc93d6 update display 2018-05-31 18:04:05 -07:00
George Lu 547096d672 update man 2018-05-31 18:03:52 -07:00
Yann ColletandGitHub 7c33b48221 Merge pull request #1151 from felixhandte/zstd-dfast-in-place-dict-goto 
ZSTD_dfast: Support Searching the Dictionary Context In-Place (Alternate `goto` Implementation)
 2018-05-31 17:37:09 -07:00
George Lu 140f59d38e Added --format=zstd 
title
 2018-05-31 15:29:35 -07:00
W. Felix Handte 48deab92de Allow Different Dict Attachment Cut-Offs for Different Strategies 2018-05-31 17:37:44 -04:00
W. Felix Handte f86796639e Remove Incorrect and Extraneous Repcode Bounds Check 2018-05-31 17:02:29 -04:00
Yann Collet 9b979d0e33 minor : improved API code comment 
Extend guarantee that ZSTD_getFrameContentSize() will delivering the decompressed size
to any single-pass compression function.

Answer #1156
 2018-05-31 11:12:18 -07:00
Yann ColletandGitHub ac4f7ead3b Merge pull request #1154 from facebook/altOpt 
minor optimal parser optimization
 2018-05-30 13:05:51 -07:00
Yann Collet 809f2f9322 minor update of literal cost function 
just assert() there is no negative cost evaluation for literals
 2018-05-29 15:34:50 -07:00
Yann Collet 463a0fe38b simplified optimal parser 
removed "cached" structure.
prices are now saved in the optimal table.

Primarily done for simplification.
Might improve speed by a little.
But actually, and surprisingly, also improves ratio in some circumstances.
 2018-05-29 14:07:25 -07:00
Yann ColletandGitHub bb6eaf6495 Merge pull request #1153 from facebook/dynThreshold 
changed dynamic fse threshold for offset
 2018-05-26 08:43:45 -07:00
Yann Collet e916c365a1 fixed minor visual warning 2018-05-25 20:43:09 -07:00
Yann ColletandGitHub c6ccf435ae Merge pull request #1152 from facebook/lowCompression 
btultra accepts blocks with poorer compression ratio
 2018-05-25 17:46:11 -07:00
Yann Collet a7fdceeccd changed dynamic fse threshold for offset 
recent experienced showed that
default distribution table for offset
can get it wrong pretty quickly with the nb of symbols,
while it remains a reasonable choice much longer for lengths symbols.

Changed the formula,
so that dynamic threshold is now 32 symbols for offsets.
It remains at 64 symbols for lengths.

Detection based on defaultNormLog
 2018-05-25 17:41:16 -07:00
Yann Collet 4b3a36d5d8 Merge branch 'dev' into lowCompression 2018-05-25 15:45:03 -07:00
Yann ColletandGitHub 744f36aac4 Merge pull request #1141 from facebook/staticDictCost 
Random stuff on High Compression mode
 2018-05-25 15:43:32 -07:00
Yann Collet 5f177f1c53 btultra accepts blocks with poorer compression ratio 
zstd rejects blocks which do not compress by at least a certain amount.
In which case, such block is simply emitted uncompressed (even if a little bit of compression could be achieved).
This is better for decompression speed, hence for energy.

The logic is controlled by ZSTD_minGain().
The rule is applied uniformly, at all compression levels.

This change makes btultra accepts blocks with poor compression ratios.
We presume that users of btultra mode prefers compression ratio over some decompress speed gains.

The threshold for minimum gain is lowered for btultra
from s>>6 (~1.5% minimum gain)
to s>>7 (~0.8% minimum gain).

This is a prudent change.
Not sure if it's large enough.
 2018-05-25 15:19:52 -07:00
Yann Collet e2c0e3d437 slightly nudge choices towards less sequences 
also slightly improve some strange detrimental corner cases.
 2018-05-25 14:52:21 -07:00
W. Felix Handte 5b292b5685 Check Long + 1 Matches in Both Prefix and Dict in Bothe Short Match Paths 2018-05-25 13:13:57 -04:00
W. Felix Handte 88b733b380 Interleave Prefix and Dict Searches 2018-05-25 13:13:57 -04:00
W. Felix Handte 1850025156 Refactor ZSTD_dfast to Use gotos 2018-05-25 13:13:57 -04:00
W. Felix Handte 43606f9c83 ... When I Said "HashTable", I Meant "ChainTable" 2018-05-25 13:13:28 -04:00
W. Felix Handte ec7efe88f5 Fix Off-By-One Error 2018-05-25 13:13:28 -04:00
W. Felix Handte 2bfe43267e Disallow Too-Long Repcodes When Using an Attached Dict 2018-05-25 13:13:28 -04:00
W. Felix Handte b97ad3f457 Port Changes Made to ZSTD_fast to ZSTD_dfast 2018-05-25 13:13:28 -04:00
W. Felix Handte 2313cca1b7 Implement Second Repcode Check 2018-05-25 13:13:28 -04:00
W. Felix Handte 0998f10813 Implement First Repcode Check 2018-05-25 13:13:28 -04:00
W. Felix Handte 50c5b2bb90 Find Dict Hash Table Matches 2018-05-25 13:13:28 -04:00
W. Felix Handte 7a25f7ef5b Existing Repcode Check Only Applies to noDict Case 2018-05-25 13:13:28 -04:00
W. Felix Handte 8b241da4df Properly Initialize Repcode Values 2018-05-25 13:13:28 -04:00
W. Felix Handte 7097a03749 Add Necessary Dict Variables 2018-05-25 13:13:28 -04:00
W. Felix Handte aacbbf4f9a Rename 'lowest' to 'localLowest' to Prepare to Introduce Dict Indices 2018-05-25 13:13:28 -04:00
W. Felix Handte c10d1b4011 Skeleton for In-Place Impl for ZSTD_dfast 2018-05-25 13:13:28 -04:00
Yann Collet f6ad59ab5c Merge branch 'dev' into staticDictCost 2018-05-24 16:21:02 -07:00
Yann ColletandGitHub a7c75740b3 Merge pull request #1149 from terrelln/fuzz-py 
Small fixes to fuzz.py
 2018-05-24 15:56:09 -07:00
Nick Terrell fdd4d8510f Improve compiler detection to work on Mac 2018-05-24 14:21:12 -07:00
Nick Terrell ac852abb8b Define BIT_DEBUG for --debug 2018-05-24 14:21:12 -07:00
Nick Terrell 2a9975f77b Increase the maximum file size 2018-05-24 14:21:12 -07:00
Nick Terrell e712a3a0a3 Small fixes to fuzz.py 2018-05-24 14:21:12 -07:00
Yann ColletandGitHub 47d2331ed8 Merge pull request #1150 from facebook/fracFse 
fix corner case when requiring cost of an FSE symbol
 2018-05-24 14:19:30 -07:00
Yann Collet b5ef32fea7 Merge branch 'dev' into fracFse 2018-05-24 14:09:49 -07:00
Yann Collet 776128d16f fix corner case when requiring cost of an FSE symbol 
ensure that, when frequency[symbol]==0,
result is (tableLog + 1) bits
with both upper-bit and fractional-bit estimates.

Also : enable BIT_DEBUG in /tests
 2018-05-24 13:59:11 -07:00
Yann ColletandGitHub 08c5be5db3 Merge pull request #1117 from felixhandte/zstd-fast-in-place-dict 
ZSTD_fast: Support Searching the Dictionary Context In-Place
 2018-05-23 19:32:25 -07:00
Nick TerrellandGitHub 06b70179da Work around bug in zstd decoder (#1147) 
Work around bug in zstd decoder

Pull request #1144 exercised a new path in the zstd decoder that proved to
be buggy. Avoid the extremely rare bug by emitting an uncompressed block.
 2018-05-23 18:02:30 -07:00
Yann ColletandGitHub 712318a244 Merge pull request #1146 from terrelln/fse-fix 
[zstd] Fix decompression edge case
 2018-05-23 16:41:42 -07:00
Nick Terrell f2d0924b87 Variable declarations 2018-05-23 14:58:58 -07:00