Commit Graph
1460 Commits
Author SHA1 Message Date
Yann Collet f16f4497ca added ZSTD_estimateDStreamSize() 2017-05-09 16:18:17 -07:00
Yann Collet 542c9dfcf8 changed name frameParams into frameHeader
ZSTD_frameParams => ZSTD_frameHeader
ZSTD_getFrameParams() -> ZSTD_getFrameHeader()

The new naming is more distinctive from ZSTD_frameParameters,
which is used during compression.

ZSTD_frameHeader is clearer in its intention to described frame header content.
It also implies we are decoding a ZSTD frame, hence we are at decoding stage.
2017-05-09 15:46:07 -07:00
Yann Collet 5a36c069e7 regroup memory usage function declarations
in a single paragraph in zstd.h, for clarity
2017-05-09 15:11:30 -07:00
Yann Collet fa8dadb294 separated ZSTD_estimateCStreamSize() from ZSTD_estimateCCtxSize()
for clarity
2017-05-08 18:24:16 -07:00
Yann Collet 51652522a2 bumped version number 2017-05-08 17:52:46 -07:00
Yann Collet a1d6704d7f added ZSTD_estimateCDictSize() and ZSTD_estimateDDictSize()
it complements ZSTD_estimateCCtxSize()
for the special case of ZSTD_initCStream_usingDict()
2017-05-08 17:51:49 -07:00
Yann Collet 7855366598 Updated ZSTD_freeCCtx()
which can also contain streaming buffers now.
Redirected ZSTD_freeCStream() towards it.
2017-05-08 17:15:00 -07:00
Yann Collet fc5145955a updated ZSTD_estimateCCtxSize()
added a parameter streaming,
to estimate memory allocation size
when the CCtx is used for streaming (CStream).

Note : this function is not able to estimate
memory cost of a potential internal CDict
which can only happen when starting with ZSTD_initCStream_usingDict()
2017-05-08 17:07:59 -07:00
Yann Collet 791d744279 Updated ZSTD_sizeof_CCtx()
can now contain buffers if object used as CStream.
ZSTD_sizeof_CStream() is now just a thin wrapper of ZSTD_sizeof_CCtx().
2017-05-08 16:17:30 -07:00
Yann Collet 0be6fd3429 merged CCtx and CStream as a single same object
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
Yann Collet a00e9599f1 removed -g from DEBUGFLAGS
It inflates binary sizes, which is negative for the Windows build.
It also makes it impossible to check if 2 different source codes
get nonetheless compiled to the same binary,
since checksum will be different, due to integrated source code.
2017-05-04 17:24:29 -07:00
Yann Collet 606c04c228 Merge branch 'dev' of github.com:facebook/zstd into dev 2017-05-02 12:13:52 -07:00
Yann ColletandGitHub 072484a3bf Merge pull request #683 from terrelln/odev
[CLI] Make cover the default dictionary builder
2017-05-02 12:13:23 -07:00
Nick Terrell f376d47c11 [CLI] Switch dictionary builder on CLI to cover 2017-05-02 11:18:27 -07:00
Nick Terrell 020b960e13 [cover] Make optimization faster 2017-05-02 11:02:48 -07:00
Nick Terrell f2d9ef1dc0 [cover] Optimize case where d <= 8 2017-05-02 11:02:43 -07:00
Nick Terrell 865918dd04 Fix typo in zdict.h 2017-05-02 11:02:37 -07:00
Yann Collet b184589c4c minor code refactoring for clarity 2017-05-01 11:35:47 -07:00
Yann Collet 33c38b0925 fixed const in prototype, that Visual doesn't accept 2017-05-01 11:12:30 -07:00
Yann Collet f39a6731ec sync bitstream.h from fse library 2017-05-01 09:56:03 -07:00
Yann Collet 202082f285 sync bitstream from FSE project
add assert into unsafe *_fast() variants
2017-04-28 17:00:31 -07:00
Yann Collet 89f50deec7 minor code refactoring
clearer tables
2017-04-28 16:52:36 -07:00
Yann Collet 68a7d3d49a added HUF_PUBLIC_API macro to huf.h
to make it possible to control symbol visibility.
Also : better separation and comments between "public" and "static" sections
2017-04-28 12:46:48 -07:00
Yann ColletandGitHub a51cab6e68 Merge pull request #678 from facebook/apiChange
Breaking API Change around CDict
2017-04-28 10:02:45 -07:00
Yann Collet 29297c6751 Changed default level 18 (large input)
Previous -18 : 4.7 MB/s, R:3.833
New -18 : 5.1 MB/s. R:3.825

It's a better fit within -17 (6.8 MB/s) and -19 (4.0 MB/s)
The new level 18 also uses significantly less memory.
And, it makes a good transition between level 17 (mml5)
and level 19 (mml3).
Up to now, there was no level with mml4.

(note : minmatch setting can have a large impact on some (specific) datasets)
2017-04-27 17:44:01 -07:00
Yann Collet a92cbb7004 Added a secondary test, checking dictID presence after setting noDictIdFLag=1 2017-04-27 15:08:56 -07:00
Yann Collet d3694e6c70 removed C4204 2017-04-27 14:29:35 -07:00
Yann Collet 1c3ab0c77f fixed init error on Visual 2008 2017-04-27 12:57:11 -07:00
Yann Collet 8b669535f8 bumped version number to v1.2.0 2017-04-27 12:50:20 -07:00
Yann Collet 77bf59ef50 added ZSTD_initCStream_usingCDict_advanced() 2017-04-27 11:43:04 -07:00
Yann Collet f4bd857d81 created ZSTD_compress_usingCDict_advanced() 2017-04-27 11:31:55 -07:00
Yann Collet 69a54d138a fixed compilation warning : declaration-after-statement 2017-04-27 01:11:26 -07:00
Yann Collet 31533bacce Changed ZSTD_createCDict_advanced()
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.

Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
Yann Collet 768df129d2 changed ZSTD_compressBegin_usingCDict()
No longer takes `pledgedSrcSize` as argument
this is in line with similar functions ZSTD_compress_usingCDict()
and ZSTD_initCStream_usingCDict().
2017-04-26 15:42:10 -07:00
Yann Collet e42afbc6fa Comply with suggested comments by @terrelln
created FSE_CTABLE_SIZE() and FSE_DTABLE_SIZE()
2017-04-26 11:39:35 -07:00
Yann Collet 7271203bdb transferred entropy scratch space from CCtx into workSpace
Saved 6 KB
2017-04-20 23:21:19 -07:00
Yann Collet a408645f50 made some room for entropy scratch space 2017-04-20 23:09:39 -07:00
Yann Collet 71aaa32c3c transferred FSE tables from CCtx into workspace
Saved 5 KB from CCtx
2017-04-20 23:03:38 -07:00
Yann Collet 71ddeb67b1 made room in workspace for FSE tables
still need to be transferred from CCtx into workspace
2017-04-20 22:54:54 -07:00
Yann Collet a34a39c183 changed size evaluation of entropy tables
so that memcpy() does no longer depends on fse pointer being a static table
2017-04-20 18:26:25 -07:00
Yann Collet 7bb60b17d8 init entropy table pointers only once
per workSpace resize
2017-04-20 17:38:56 -07:00
Yann Collet e6fa70a0a1 reorganized ZSTD_resetCCtx_internal()
clearer separation between variables and buffers
clearer buffers category
kept static buffers at the beginning, favoring cache locality
(it will be easier to add FSE tables there later)

This break a few assumptions that hashTable was always at the beginning.
This is fixed.
And remaining assumptions (namely that tables stand next to each other in memory)
are now tested with assert.
2017-04-20 17:28:31 -07:00
Yann Collet c17e020c9a disable assert when compiling paramgrill
paramgrill is a benchmark calibration function.
Speed accuracy is critical, it cannot be altered by assert.
2017-04-20 12:50:02 -07:00
Yann Collet 16f9c572fc Merge branch 'dev' into compressionFlow 2017-04-20 11:16:40 -07:00
Yann Collet e348dad305 minor long line reformatting 2017-04-20 11:14:13 -07:00
Yann Collet e847730452 slightly refined README comments on lib-mt 2017-04-18 23:15:28 -07:00
Yann Collet 2c5514c759 fixed ZSTDMT_initCStream_advanced()
Must use the new ZSTD_compressBegin_usingCDict_advanced()
to enforce correct frame parameters
2017-04-18 22:52:41 -07:00
Sean Purcell 98cf7fcb2a Update README 2017-04-18 17:03:37 -07:00
Yann Collet a4cab80183 added ZSTD_copyCCtx_internal()
which respects provided fParams.
2017-04-18 14:54:54 -07:00
Sean Purcell ca6fae7808 Add MT enabled targets for libzstd 2017-04-18 14:13:01 -07:00