Yann Collet and GitHub
8049556928
Merge pull request #778 from terrelln/bad-huff
...
[libzstd] Fix bug in Huffman decompresser
2017-08-07 14:05:58 -07:00
Yann Collet and GitHub
6d0a5c5728
Merge pull request #781 from stellamplau/qSize
...
Allow queueSize=0 in pool.c
2017-08-07 13:55:19 -07:00
Nick Terrell
abe12b3399
[libzstd] Fix bug in Huffman decompresser
...
The zstd format specification doesn't enforce that Huffman compressed
literals (including the table) have to be smaller than the uncompressed
literals. The compressor will never Huffman compress literals if the
compressed size is larger than the uncompressed size. The decompresser
doesn't accept Huffman compressed literals with 4 streams whose compressed
size is at least as large as the uncompressed size.
* Make the decompresser accept Huffman compressed literals whose size
increases.
* Add a test case that exposes the bug. The compressed file has to be
statically generated, since the compressor won't normally produce files
that expose the bug.
2017-08-07 12:37:48 -07:00
Stella Lau
e1abc2a367
Switch the sleep function to UTIL_sleepMilli
2017-08-07 11:49:13 -07:00
Stella Lau
1e366f9dea
Add test for deadlock
2017-08-02 11:27:50 -07:00
Stella Lau
73ba58955f
Signal after finishing job when queueSize=0
2017-08-01 20:12:06 -07:00
Stella Lau
1d76da1d87
Replace marker with queueEmpty variable and update pool.h comment
2017-08-01 12:30:16 -07:00
Stella Lau
5adceeed01
Allow queueSize=0 in pool.c and update poolTests
2017-07-31 10:10:16 -07:00
Yann Collet and GitHub
ee27f6937c
Merge pull request #758 from stellamplau/ldm
...
Add working prototype of a long distance matcher
2017-07-28 17:07:21 -07:00
Stella Lau
8fae41c412
Return error code in verify() and minor code cleanup
2017-07-27 17:14:05 -07:00
Stella Lau
1294a4a897
Fix typo
2017-07-27 15:49:46 -07:00
Stella Lau
627621839c
Add checks in initialization code
2017-07-27 15:37:37 -07:00
Stella Lau
c105f605e6
Update README
2017-07-27 11:11:35 -07:00
Yann Collet and GitHub
e1222544be
Merge pull request #753 from paulcruz74/adapt-approach-3
...
adaptive compression v1
2017-07-27 10:00:10 -07:00
Paul Cruz
2320e7378a
remove unused variable, add documentation for context fields
2017-07-26 17:02:47 -07:00
Stella Lau
9eaf3d22d0
Allow HASH_ONLY_EVERY_LOG to be configured in ldm.h
2017-07-26 16:43:25 -07:00
Paul Cruz
ab5a78547e
fix leaky abstraction regarding measuring completion
2017-07-26 16:40:05 -07:00
Paul Cruz
715f36ca81
added definitions for conversion constants, moved forced compression check to top of adaptCompressionLevel, used ZSTD_BLOCKSIZE_MAX
2017-07-26 15:52:15 -07:00
Paul Cruz
6c1c1242fc
set the window log value before performing compression
2017-07-26 14:29:59 -07:00
Stella Lau
40759bade9
Add README and clean up code
2017-07-26 13:24:03 -07:00
Paul Cruz
a959cc881a
moved reset of completion to right after wait
2017-07-26 10:34:48 -07:00
Paul Cruz
305d5ee70f
change to >= convergence counter
2017-07-26 10:20:29 -07:00
Paul Cruz
be92a38d6a
decrease completion requirements for change, move create thread wait, merge cases where compression thread should wait
2017-07-26 10:05:10 -07:00
Stella Lau
e9161637b2
Allow parameters to be modified from a separate file
2017-07-25 18:13:27 -07:00
Paul Cruz
0b18d21e03
building on readme, added another help tip in the menu
2017-07-25 17:47:02 -07:00
Paul Cruz
7cc74e0b27
adding more to readme
2017-07-25 16:55:16 -07:00
Paul Cruz
8dbb07d822
updated progress bar with better representation of time, added const
2017-07-25 16:03:43 -07:00
Paul Cruz
9a132707af
changing time units to seconds
2017-07-25 15:26:26 -07:00
Stella Lau
629c300118
Rename and remove unneeded files
2017-07-25 15:17:36 -07:00
Paul Cruz
31a9ed9883
updated const values, added more comments
2017-07-25 14:53:40 -07:00
Yann Collet and GitHub
6466fd3400
Merge pull request #776 from terrelln/dev
...
[libzstd] Fix CHECK_V_F() macro
2017-07-25 14:39:32 -07:00
Paul Cruz
5cfbf609a4
removed old debug statements no longer being used
2017-07-25 14:31:48 -07:00
Paul Cruz
0882cd1981
progress bar -- don't print num jobs, time elapsed shown in seconds
2017-07-25 14:26:55 -07:00
Paul Cruz
310c12d07e
moved debug statements to a compiler flag
2017-07-25 14:08:39 -07:00
Nick Terrell
ae20d413da
[libzstd] Fix CHECK_V_F macros
2017-07-25 12:52:01 -07:00
Paul Cruz
e02c79f833
started using decrease cooldown so that compression level would not decrease several times in a row
2017-07-25 11:16:27 -07:00
Paul Cruz
85d7c919f6
created independent function for controlling how completion relates to compression level change
2017-07-25 10:32:14 -07:00
Paul Cruz
6f1e260edd
added mechanism for getting rid of spikes
2017-07-25 10:01:10 -07:00
Paul Cruz
700758d676
added help statement for -p, switched it to hide progress bar now that progress bar is default
2017-07-24 16:26:20 -07:00
Paul Cruz
df3754b6ed
add quiet option, make progress bar default
2017-07-24 16:19:07 -07:00
Stella Lau
0295a27133
Experiment with not using a checksum
2017-07-24 15:26:44 -07:00
Paul Cruz
4dc83ca64c
compression thread should take measurements independently based on whether or not the create/write thread will actually bottleneck performance
2017-07-24 15:14:58 -07:00
Paul Cruz
0ee3f8c2f8
adding more debug
2017-07-24 15:06:11 -07:00
Paul Cruz
8328f8192a
updating debug statements again
2017-07-24 14:40:23 -07:00
Paul Cruz
d3d759301f
changing position of endline for debug
2017-07-24 13:47:39 -07:00
Stella Lau
08a6e9a141
Minor code cleanup
2017-07-24 13:22:00 -07:00
Stella Lau
6eefa32911
Deduplicate code
2017-07-24 12:40:59 -07:00
Stella Lau
8ed9220102
Experiment with eviction policies and minor code cleanup
2017-07-24 12:05:43 -07:00
Paul Cruz
e508f632d6
updated comments and debug statements
2017-07-24 11:01:36 -07:00
Stella Lau
eb16da647d
Minor clean up
2017-07-24 10:20:53 -07:00
