Commit Graph
2760 Commits
Author SHA1 Message Date
shakeelrao 95dfd48143 update formatting 2019-03-01 23:11:15 -08:00
shakeelrao 1e08c49f75 add stylistic changes 2019-03-01 18:29:35 -08:00
shakeelrao 2bb5eec711 update missing error case to CONTENTSIZE_ERROR 2019-03-01 00:12:16 -08:00
shakeelrao 44ae395b3e change nbBlocks to size_t for consistency 2019-03-01 00:05:59 -08:00
shakeelrao 03026c3b1d change compressedBound to ULL 2019-03-01 00:03:50 -08:00
shakeelrao 8930c3c79b implement API-level changes 2019-02-28 22:55:18 -08:00
shakeelrao dce9a09772 initialize local vars in decompressBound 2019-02-28 03:01:21 -08:00
shakeelrao 515c506b4c switch frameBound type to ULL 2019-02-28 02:10:17 -08:00
shakeelrao d0a3f25697 change return type to ULL 2019-02-28 01:52:01 -08:00
shakeelrao c9d674b60d Remove autogenerated test file 2019-02-28 01:29:04 -08:00
shakeelrao 97d3d28dab Fix decl-after-stmnt build error 2019-02-28 01:24:54 -08:00
shakeelrao 820af1e078 Provide an API function to estimate decompressed size.
Introduces a new utility function `ZSTD_findFrameCompressedSize_internal` which
is equivalent to `ZSTD_findFrameCompressSize`, but accepts an additional output
parameter `bound` that computes an upper-bound for the compressed data in the frame.

The new API function is named `ZSTD_decompressBound` to be consistent with
`zstd_compressBound` (the inverse operation). Clients will now be able to compute an upper-bound for
their compressed payloads instead of guessing a large size.

Implements https://github.com/facebook/zstd/issues/1536.
2019-02-28 00:42:49 -08:00
Nick TerrellandGitHub be3bd70c57 Merge pull request #1532 from terrelln/cctx-params
[libzstd] Rename ZSTD_CCtxParam_* to ZSTD_CCtxParams_*
2019-02-20 10:46:46 -08:00
Nick Terrell 7ad7ba3178 [libzstd] Rename ZSTD_CCtxParam_* to ZSTD_CCtxParams_* 2019-02-19 17:44:52 -08:00
Nick Terrell 9f9630f455 [Windows] Don't use a .def file 2019-02-19 16:52:38 -08:00
Nick Terrell 0c86d23467 [Windows] Move public headers to include/ 2019-02-19 15:49:48 -08:00
Nick Terrell f4abba02ba [libzstd] Clean up parameter code
* Move all ZSTDMT parameter setting code to ZSTD_CCtxParams_*Parameter().
  ZSTDMT now calls these functions, so we can keep all the logic in the
  same place.
* Clean up `ZSTD_CCtx_setParameter()` to only add extra checks where needed.
* Clean up `ZSTDMT_initJobCCtxParams()` by copying all parameters by default,
  and then zeroing the ones that need to be zeroed. We've missed adding several
  parameters here, and it makes more sense to only have to update it if you
  change something in ZSTDMT.
* Add `ZSTDMT_cParam_clampBounds()` to clamp a parameter into its valid
  range. Use this to keep backwards compatibility when setting ZSTDMT parameters,
  which clamp into the valid range.
2019-02-19 13:22:37 -08:00
Nick Terrell 3d7377b874 [libzstd] Handle uncompressed literals 2019-02-15 14:58:11 -08:00
Nick Terrell f9513115e4 [libzstd] Add ZSTD_c_literalCompressionMode flag
It controls the literals compression. It is either
`auto`, `huffman`, or `uncompressed`. It defaults to
`auto`, which is the current behavior.
2019-02-13 14:59:22 -08:00
Nick TerrellandGitHub 197a5737c8 Merge pull request #1516 from terrelln/dict-doc
[zdict] Improve documentation
2019-02-01 19:04:05 -05:00
Nick Terrell 21616d8a77 [zdict] Improve documentation 2019-02-01 15:19:32 -08:00
Peter (Stig) EdwardsandGitHub 894bbda44c -Wformat-security not needed with -Wformat=2 2019-02-01 09:31:02 +00:00
W. Felix Handte 501eb25102 Rename FORWARD_ERROR -> FORWARD_IF_ERROR 2019-01-29 12:56:07 -05:00
W. Felix Handte 429987c9a6 Add Comment 2019-01-28 17:35:31 -05:00
W. Felix Handte 2179ce00e1 Remove CHECK_E Macro 2019-01-28 17:33:13 -05:00
W. Felix Handte 03e040a966 Replace Uses of CHECK_E with RETURN_ERROR_IF(*_isError(... 2019-01-28 17:33:01 -05:00
W. Felix Handte 7ebd897157 Remove CHECK_F Macro 2019-01-28 17:16:32 -05:00
W. Felix Handte 64bb6640f2 Replace CHECK_F Uses in zstdmt_compress.c and zstd_ddict.c 2019-01-28 17:15:57 -05:00
W. Felix Handte cafc3b1bcb Also Convert zstd_compress.c 2019-01-28 17:05:18 -05:00
W. Felix Handte 324e9654d3 Add grep-able String to Error Macros 2019-01-28 12:50:36 -05:00
W. Felix Handte 32fed9c7be Switch CHECK_F Calls to FORWARD_ERROR 2019-01-28 12:45:34 -05:00
W. Felix Handte 800c87fed0 Switch Unconditional RETURN_ERROR_IF Calls to RETURN_ERROR 2019-01-28 12:45:34 -05:00
W. Felix Handte a3538bbc6f Add RETURN_ERROR and FORWARD_ERROR Macros 2019-01-28 12:45:26 -05:00
W. Felix Handte c823237d7b Convert Checks in zstd_decompress.c to RETURN_ERROR_IF 2019-01-28 12:23:14 -05:00
W. Felix Handte ea031f4ea2 Convert Checks in zstd_decompress_block.c to RETURN_ERROR_IF 2019-01-28 11:56:39 -05:00
W. Felix Handte 54fa31f03b Add RETURN_ERROR_IF Macro That Logs Debug Information When Check Fails 2019-01-28 11:43:33 -05:00
Yann Collet f9e4f89252 improved comments for adjustCParams() and getCParams() 2019-01-02 12:18:40 -08:00
Yann Collet 0fb4b21d1a updated libzstd documentation 2018-12-25 03:10:07 -08:00
Yann ColletandGitHub e980ba212f Merge pull request #1471 from facebook/nofloat
guard functions using floating point for debug mode only
2018-12-23 12:35:51 -08:00
Yann ColletandGitHub aae5bc538a Merge pull request #1470 from facebook/U32
fix confusion between unsigned <-> U32
2018-12-23 12:35:39 -08:00
Yann Collet c9dfb7e445 guard functions using floating point for debug mode only
they are only used to print debug messages.
Requested in #1386,
2018-12-22 09:09:40 -08:00
Yann Collet ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
Yann Collet c8d1fda982 update aarch64 test to xenial
in an attempt to circumvent the `ld` bug
2018-12-21 15:08:48 -08:00
Yann ColletandGitHub 8f35c7f94c Merge pull request #1466 from facebook/noDictPresent
fixed : better error message
2018-12-20 19:01:27 -08:00
Yann ColletandGitHub 41b45b84a1 Merge pull request #1465 from facebook/noFilePresent
fixed : detection of non-existing file
2018-12-20 17:21:04 -08:00
Yann Collet ed2fb6bd57 fixed : better error message when dictionary missing
during benchmark.
Also : refactored ZSTD_fillHashTable(),
just for readability (it does the same thing)
2018-12-20 17:20:07 -08:00
Yann ColletandGitHub e4ae24c229 Merge pull request #1420 from felixhandte/zstd-decompress-minimal
Various Macros to Allow Building Extremely Minimal Decoder Library
2018-12-20 15:17:37 -08:00
Yann Collet 95784c654c fixed shadowing of stat variable
some standard lib declares a `stat` variable at global scope
shadowing local declarations ....
2018-12-20 14:56:44 -08:00
Yann Collet ffba142406 fixed file identity detection in 32-bit mode
also :
some library decided to use `index` as a global variable declared in standard header
shadowing the ones used in fastcover.c  :(
2018-12-20 14:30:30 -08:00
W. Felix Handte 91b7309115 Mask Off Unused Functions When ZSTD_FORCE_DECOMPRESS_SEQUENCES_LONG 2018-12-20 12:20:34 -08:00