Commit Graph
7510 Commits
Author SHA1 Message Date
Nick Terrell 77a2945c43 Add some comments 2020-04-27 20:04:04 -07:00
Nick Terrell f33de06c3e [lib] Fix single-pass mode for empty frames 2020-04-27 20:04:01 -07:00
Nick Terrell a4ff217baf [lib] Add ZSTD_d_stableOutBuffer 2020-04-27 18:09:44 -07:00
Nick Terrell b104f8e3eb [zstd] Fix typo in ZSTD_dParameter 2020-04-27 12:12:28 -07:00
Bimba ShresthaandGitHub 6b4a3e019f Merge pull request #2088 from bimbashrestha/bug
[bug] Making compressStream2 fail when passing rawContent but claiming fullDict
2020-04-23 14:16:56 -05:00
Bimba ShresthaandGitHub 861a73744d Merge pull request #2086 from bimbashrestha/quick3
[bug] passing cParams.hashLog instead of comprParams->hashLog for ldm trigger
2020-04-23 13:02:57 -05:00
Bimba Shrestha 1875f616ce passing dictContentType instead of rawContent every time 2020-04-21 22:29:35 -07:00
Bimba Shrestha f7a7409a49 adding fail test when passing wrong fullDict using refPrefix 2020-04-21 22:26:48 -07:00
Bimba Shrestha dba02245bf bash to shell conversion 2020-04-21 20:31:11 -07:00
Bimba Shrestha 659ff85cf5 changing to cParams.hashLog 2020-04-21 21:12:50 -05:00
Bimba Shrestha 0b107188b8 adding test for long mode trigger 2020-04-21 21:09:49 -05:00
Bimba ShresthaandGitHub 38a6d2a8f2 Merge pull request #2085 from bimbashrestha/quick2
[doc-bug] Updating the recommended max chainlog for patch-from
2020-04-20 14:09:23 -05:00
Bimba Shrestha b640802ff8 updating chainlog bound 2020-04-20 11:34:24 -07:00
Bimba ShresthaandGitHub c6311584aa Merge pull request #2083 from bimbashrestha/quick1
[nit] Adding missing newline at end of patch-from recommendations
2020-04-20 13:24:34 -05:00
Bimba Shrestha 587a20a12a adding newline 2020-04-20 10:25:58 -07:00
Bimba ShresthaandGitHub 5b0a452cac Adding --long support for --patch-from (#1959)
* adding long support for patch-from

* adding refPrefix to dictionary_decompress

* adding refPrefix to dictionary_loader

* conversion nit

* triggering log mode on chainLog < fileLog and removing old threshold

* adding refPrefix to dictionary_round_trip

* adding docs

* adding enableldm + forceWindow test for dict

* separate patch-from logic into FIO_adjustParamsForPatchFromMode

* moving memLimit adjustment to outside ifdefs (need for decomp)

* removing refPrefix gate on dictionary_round_trip

* rebase on top of dev refPrefix change

* making sure refPrefx + ldm is < 1% of srcSize

* combining notes for patch-from

* moving memlimit logic inside fileio.c

* adding display for optimal parser and long mode trigger

* conversion nit

* fuzzer found heap-overflow fix

* another conversion nit

* moving FIO_adjustMemLimitForPatchFromMode outside ifndef

* making params immutable

* moving memLimit update before createDictBuffer call

* making maxSrcSize unsigned long long

* making dictSize and maxSrcSize params unsigned long long

* error on files larger than 4gb

* extend refPrefix test to include round trip

* conversion to size_t

* making sure ldm is at least 10x better

* removing break

* including zstd_compress_internal and removing redundant macros

* exposing ZSTD_cycleLog()

* using cycleLog instead of chainLog

* add some more docs about user optimizations

* formatting
2020-04-17 15:58:53 -05:00
Yann ColletandGitHub 88ecdc939b Merge pull request #2080 from facebook/minor_fileio_refactor
minor refactor of `fileio.c`
2020-04-13 20:47:34 -07:00
Yann Collet 458a1a1723 minor refactor
- fix a few comments
- reorder some parameters, to enforce "mutable references first"
- simplified fwriteSparse()
2020-04-13 14:09:57 -07:00
Nick TerrellandGitHub 5fcbc484c8 Merge pull request #2040 from caoyzh/dev-2
Optimize by prefetching on aarch64
2020-04-08 13:14:47 -07:00
Bimba ShresthaandGitHub c0d4b2b5a3 Merge pull request #2075 from bimbashrestha/dict_fuzzer_ref
[bug] handling case where prefix is NULL or 0 sized in refPrefix_advanced
2020-04-07 17:37:19 -05:00
Bimba Shrestha 1658ae75cd handling nil case for refprefix 2020-04-07 14:41:53 -07:00
Yann ColletandGitHub dde98d833d Merge pull request #2065 from cwoffenden/single-file-lib
Single-file decoder script can now (optionally) create an encoder
2020-04-07 10:17:05 -07:00
Carl Woffenden 3eaa525138 Fixed clash when projects are already using xxHash
Undefing XXH_* macros allows the `.c` to build standalone without clashes. Removing `xxhash.c` and only including the header is the correct usage (according to `XXH_PRIVATE_API`).
2020-04-07 18:17:59 +02:00
Carl WoffendenandGitHub 4e6027f8ca Fixed-up single-file libs directory 2020-04-07 13:35:19 +02:00
Carl Woffenden d01e7c2fad Renamed directory 2020-04-07 13:34:19 +02:00
Carl Woffenden 59812afac7 Improved documentation 2020-04-07 13:19:52 +02:00
Carl Woffenden a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
Carl Woffenden 7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
Carl Woffenden edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
Bimba Shrestha 794f03459e adding refPrefix 2020-04-06 22:57:49 -07:00
Bimba ShresthaandGitHub 2847d1a606 Merge pull request #2069 from bimbashrestha/doversize
[memory opt] Downsize decompression buffers when large for too long
2020-04-06 16:06:37 -05:00
Bimba Shrestha 31e76f1ed4 adding test for dctx size reduction 2020-04-04 08:49:24 -07:00
Bimba ShresthaandGitHub ebc422b0b7 Merge pull request #2068 from bimbashrestha/quick
[bug] Fixing Incorrect/inconsistent reported compression percentage for empty files
2020-04-03 19:28:42 -05:00
Bimba Shrestha 0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
Bimba Shrestha 0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
Bimba Shrestha 3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
Bimba Shrestha ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
Bimba Shrestha a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
Bimba ShresthaandGitHub 009a92f749 Merge pull request #2067 from bimbashrestha/res-leak
[bug] adding fclose before return
2020-04-03 15:51:59 -05:00
Bimba Shrestha 1d267dc5d6 returning on null check 2020-04-03 13:39:02 -07:00
Bimba Shrestha d598c88fb3 adding fclose before return 2020-04-03 13:31:47 -07:00
Bimba Shrestha 936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
Bimba Shrestha 05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
Bimba Shrestha d0412f3aba no percentage on readsize == 0 2020-04-03 12:10:02 -07:00
Bimba Shrestha bf1856c26f removing max(1, ..) 2020-04-03 12:07:17 -07:00
Carl Woffenden 88da79b42e Minor typo 2020-04-03 20:50:54 +02:00
Carl Woffenden 0442be5aa4 Added missing -pthread flag (was breaking on BSD) 2020-04-03 20:34:47 +02:00
Carl Woffenden 7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
Carl WoffendenandGitHub 7202184ee0 Fixes decompressor when using -Wshorten-64-to-32 (#2062)
Spotted on iOS when building with `-Wshorten-64-to-32` (since `__builtin_expect` returns a `long`).
2020-04-03 02:55:29 -07:00
Nick TerrellandGitHub 1665462573 Merge pull request #2054 from terrelln/license-fix
Standardize and fix copyright and licenses
2020-03-27 11:00:01 -07:00