72 Commits
Author SHA1 Message Date
Nick Terrell 5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
Yann Collet 4cf0093571 restored bonus rule 2017-03-26 14:51:00 -07:00
Yann Collet 69017bf253 Merge branch 'dev' into LegacyDictBuilder 2017-03-26 14:39:13 -07:00
Yann Collet 582760818f minor refactor
add const
changed if for easier to add new conditions
2017-03-26 03:04:56 -07:00
Yann Collet 858f72eeb8 fixed dictBuilder issue
dictionary loading would fail during entropy analysis
2017-03-26 02:50:00 -07:00
Yann Collet ecee9f2ef8 fixed conversion warnings 2017-03-26 00:59:14 -07:00
Yann Collet 4c41d37fcc changed test for new syntax
--dictID= and --maxdict=
2017-03-24 18:36:56 -07:00
Yann Collet d41f707e88 minor improvement : remove duplicates with 1 char prefix difference 2017-03-24 17:56:45 -07:00
Yann Collet 96aa3019b2 changed advanced commands --maxdict= and --dictID=
now works with the `=` variant, which is the recommended one.
Old variant `--dictID #` still works, for compatibility with existing scripts.
Long term objective is to remove the old variant..
2017-03-24 16:04:29 -07:00
Yann Collet 9da3b215ec Ensure all limits derived from same constants
Now uses ZDICT_DICTSIZE_MIN and ZDICT_CONTENTSIZE_MIN
from zdict.h.

Also : reduced values to 256 and 128 respectively
2017-03-24 15:02:09 -07:00
Yann Collet f332ece468 dictBuilder fails to create dictionary on certain input
Properly expressed with an error code (see zstd_errors.h)
and a cli return code != 0
2017-03-23 16:24:02 -07:00
Sean Purcell 042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
Yann Collet aca113f4f5 fixed ZSTD_sizeof_?Dict() 2016-12-23 22:25:03 +01:00
Nick Terrell bcbe77e994 ZDICT_finalizeDictionary() flipped comparison
`ZDICT_finalizeDictionary()` had a flipped comparison.
I also allowed `dictBufferCapacity == dictContentSize`.
It might be the case that the user wants to fill the dictionary
completely up, and then let zstd take exactly the space it needs
for the entropy tables.
2016-12-22 18:01:14 -08:00
Yann Collet d76d1a9ef0 added ZDICT_finalizeDictionary() 2016-12-22 20:18:43 +01:00
Yann Collet 0819abe3c1 added ZSTD_createDDict_byReference() body 2016-12-21 19:25:15 +01:00
Yann Collet 1496c3dc47 Fix : size estimation when some samples are very large 2016-12-18 11:58:23 +01:00
Yann Collet d46ecb58a5 added dll compilation tests 2016-12-17 16:28:12 +01:00
Yann Collet 0a5a5fb7fd Fix #418 : printing selected segments in zdict debug mode can segfault with certain pathological patterns 2016-11-02 13:57:55 -07:00
Yann Collet 52c1bf93fe improved dicitonary segment merge 2016-10-18 16:34:58 -07:00
Yann Collet 2b361cf2f1 minor opt 2016-10-14 16:09:07 -07:00
Yann Collet df6797447f update dictionary builder warning comments 2016-09-27 15:14:32 +02:00
Yann Collet 47094ea66b added comment on filePos 2016-09-26 18:03:33 +02:00
Yann Collet 97b378a6f8 Streaming : dictionary compression on multiple files / segments can correctly provide srcSize into header (when provided) using pledgedSrcSize. 2016-09-21 17:20:19 +02:00
Yann Collet d56dbc02d3 removed g_displayLevel 2016-09-02 17:28:41 -07:00
Yann Collet 855766d73d clarified dictionary in format description 2016-09-02 17:04:49 -07:00
Yann Collet d725427a3c g_time => local displayTime 2016-09-02 15:32:39 -07:00
Yann Collet 4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
Yann Collet 3b15f1f10f minor refactor 2016-08-30 09:58:50 -07:00
Yann Collet da3fbcb302 Added ZDICT_getDictID() 2016-08-19 14:23:58 +02:00
Yann Collet 49d105cfcf better warning and error messages in case of dictionary training failure (#292) 2016-08-18 15:02:11 +02:00
Yann Collet e9b414d825 fixed msan warning (#281) 2016-08-11 22:09:09 +02:00
Yann Collet e0b4a2d40f fixed dictionary generation, reported by Bartosz Taudul 2016-08-03 03:36:03 +02:00
Yann Collet bf2bc112bb bench : controlled display update when loading lot of files 2016-08-02 23:48:13 +02:00
Yann Collet f116e87f59 fixed analyzer warning 2016-08-01 19:15:18 +02:00
Yann Collet 3ca750372d updated doc (#269) 2016-08-01 02:26:20 +02:00
Yann Collet 70a9ff4af3 fixed too large selectivity level, reported by Ilona Papava 2016-07-30 01:09:14 +02:00
Yann Collet c0ce4f1211 slightly improved compression speed 2016-07-30 00:55:13 +02:00
Yann Collet ed57d8530a new seqStore 2016-07-29 21:22:17 +02:00
Yann Collet 4110534886 ZSTD_maxCLevel() is promoted to "stable" API (#254, by @FrancescAlted) 2016-07-27 15:09:11 +02:00
Yann Collet 55a8bea0b5 fixed dictionary generation 2016-07-27 14:48:47 +02:00
Yann Collet c154d9d6a2 better support for large dictionaries (> 128 KB) 2016-07-27 14:37:00 +02:00
Yann Collet 07626dfa51 improved dictbuilder notifications on selectivity 2016-07-27 13:28:46 +02:00
Yann Collet f796f7ab45 removed fastscan mode 2016-07-27 12:53:54 +02:00
Yann Collet dd25a27702 added tutorial warning messages for dictBuilder 2016-07-27 12:43:09 +02:00
Yann Collet cf05b9d477 ZSTD_getBlockSizeMax() 2016-07-18 16:52:10 +02:00
Yann Collet e557fd5e92 minor compression level corrections 2016-07-17 16:21:37 +02:00
Yann Collet f6ff53cd4e implemented dictID reserved ranges 2016-07-15 17:03:38 +02:00
Yann Collet 961b6a0e34 ZSTD_compressBlock() limits block size depending on windowLog parameter 2016-07-15 11:58:49 +02:00
Yann Collet e9ed5cdc94 fixed minor coverity warning 2016-07-14 21:02:57 +02:00