ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,130 Commits 17 Branches 77 Tags
b3c9fc27b4596beb7452ae71647a97eb7fad08cd
Commit Graph
7130 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Clement Courbet b3c9fc27b4 Optimized loop bounds to allow the compiler to unroll the loop. 
This has no measurable impact on large files but improves small file
decompression by ~1-2% for 10kB, benchmarked with:

head -c 10000 silesia.tar > /tmp/test
make CC=/usr/local/bin/clang-9 BUILD_STATIC=1 && ./lzbench -ezstd -t1,5 /tmp/test
 2019-11-15 08:27:05 +01:00
Nick TerrellandGitHub 332aade370 Merge pull request #1864 from terrelln/dict-fix 
Fix 2 bugs in dictionary loading
 2019-11-01 20:01:12 -07:00
Nick Terrell 60205fec02 Fix 2 bugs in dictionary loading 
* Silently skip dictionaries less than 8 bytes, unless using `ZSTD_dct_fullDict`.
  This changes the compressor, which silently skips dictionaries <= 8 bytes.
* Allow repcodes that are equal to the dictionary content size, since it is in bounds.
 2019-11-01 16:52:07 -07:00
Nick Terrell 75e7c0d107 [fuzz] Add dictionary_loader fuzzer 
* Adds the fuzzer
* Adds an additional `InputType` for the fuzzer

I ran the fuzzer for about 10 minutes and it found 2 bugs:

* Catches the original bug without any help
* Catches an additional bug with 8-byte dictionaries
 2019-11-01 15:54:24 -07:00
Yann Collet d770a2a89f fixed zlibwrapper build script 2019-10-31 10:47:06 -07:00
Yann ColletandGitHub 94dcce2bd8 Merge pull request #1860 from senhuang42/streaming_compression_using_cdict 
Disable reloading dictionary when there is no known srcSize
 2019-10-30 16:27:58 -07:00
Sen Huang b9ede1c8c2 Make sure contentsize is known 2019-10-30 16:03:58 -04:00
Felix HandteandGitHub f9d867e79f Merge pull request #1857 from 0x123456789A/link-shared 
Add CMake flag for linking programs dynamically
 2019-10-30 15:43:11 -04:00
Yann ColletandGitHub 47034cd6c3 Merge pull request #1850 from bimbashrestha/named_pipes 
Gating named pipe support on windows
 2019-10-30 12:18:58 -07:00
Yann Collet ae3d35dc8e updated CHANGELOG for v1.4.4 2019-10-30 12:16:45 -07:00
Yann ColletandGitHub 4ef97aaea7 Merge pull request #1855 from shashank0791/shtavi-filter-precompressed-files 
Added "--exclude-compressed" flag feature
 2019-10-30 12:14:41 -07:00
Shashank Tavildar 3c1649f139 Removed the optimization check 2019-10-29 15:59:20 -07:00
Shashank Tavildar 6d0b7bd6ce Changed extension comparision logic, added new test cases 2019-10-29 13:36:09 -07:00
Shashank Tavildar c5060997e9 Added check to perform comparison only if extension is present 2019-10-29 12:56:04 -07:00
Shashank Tavildar 9ab6a747d4 Created utility function to extract extension from filename, fixed tests 2019-10-29 12:27:54 -07:00
Shashank Tavildar 5e6dbad6c1 Deleting test directory before creation to pass travis-ci test 2019-10-29 09:54:54 -07:00
Bimba Shrestha 28d6773d75 Merge branch 'named_pipes' of https://github.com/bimbashrestha/zstd into named_pipes 2019-10-29 09:46:28 -07:00
Bimba Shrestha 4a9eca4b9d Removing merge side effect 2019-10-29 09:45:28 -07:00
0x123456789AandGitHub 9db11e3e42 Fixed check for building programs statically 2019-10-29 14:41:32 +01:00
0x123456789AandGitHub 57a311d3b7 Consider ZSTD_PROGRAMS_LINK_SHARED 
Actually consider ZSTD_PROGRAMS_LINK_SHARED in programs CMakeLists
 2019-10-29 14:33:50 +01:00
0x123456789AandGitHub 26ffc18634 Introduce ZSTD_PROGRAMS_LINK_SHARED 
The CMake variable ZSTD_PROGRAMS_LINK_SHARED indicactes wether or not to link the zstd programs dynamically or statically.
 2019-10-29 14:31:02 +01:00
Shashank Tavildar 850ba66139 Minor fixes in test 2019-10-28 22:24:01 -07:00
Bimba ShresthaandGitHub da73b11b14 Merge branch 'dev' into named_pipes 2019-10-29 01:17:13 -04:00
Bimba Shrestha 66f580ca73 Removing Visual08 and Visual10 tests 2019-10-28 22:09:34 -07:00
Nick TerrellandGitHub 5c0bdde922 Merge pull request #1856 from terrelln/assert-fix 
Fix assert in ZSTD_safecopy
 2019-10-28 21:44:22 -07:00
Shashank Tavildar 0f2bff2faf Addressing comments, removing cyclic dependency with header file, updating tests 2019-10-28 18:21:47 -07:00
Nick Terrell 9c1860861e Fix assert in ZSTD_safecopy 
In the case that `op >= oend_w` it is possible that `diff < 8` because
the two buffers could be adjacent.

Credit to OSS-Fuzz, which found the bug. It isn't reproducible because
it depends on the memory layout.
 2019-10-28 17:51:17 -07:00
Shashank Tavildar 0e9a37daeb Fixing tests and moving flag to advanced usage 2019-10-28 15:22:26 -07:00
Shashank Tavildar f422914619 Resolving local merge conflicts 2019-10-28 15:15:13 -07:00
Shashank Tavildar 02433e0b15 Addressing comments: -Created a list of extensions defined in fileio.h, -Updated test 2019-10-28 14:54:54 -07:00
Shashank Tavildar 48f856640e Added --exclude-compressed flag feature that skips compression of precompressed files 2019-10-28 13:57:40 -07:00
Yann Collet 6d5e0f1c9f Merge branch 'dev' of github.com:facebook/zstd into dev 2019-10-28 12:57:23 -07:00
Yann Collet faddd2aa1d updated CHANGELOG for v1.4.4 2019-10-28 12:57:05 -07:00
Felix HandteandGitHub 01ec595b85 Merge pull request #1851 from felixhandte/pkg-config-prefix-fix 
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
 2019-10-28 14:24:56 -04:00
Yann Collet 74065da4c5 updated API inline doc and manual 
regarding ZSTD_CDict created without a dictBuffer.
 2019-10-28 11:15:41 -07:00
Yann ColletandGitHub b4037b18ef Merge pull request #1848 from felixhandte/tzst-ext-tmp 
Support Decompressing Short Tar Extensions
 2019-10-28 10:08:20 -07:00
Yann ColletandGitHub 66a96caf22 Merge pull request #1847 from facebook/zlibwrap_visual 
fix zlibWrapper for Visual Studio
 2019-10-25 17:42:42 -07:00
Shashank Tavildar 55ee7d56e4 Added --exclude-compressed flag feature that skips compression of precompressed files 2019-10-25 15:49:11 -07:00
Bimba Shrestha 5e59d51991 Merge branch 'master' of https://github.com/bimbashrestha/zstd into named_pipes 2019-10-25 15:43:41 -07:00
Bimba Shrestha 857268b32c Gating named pipe support in hedaer file 2019-10-25 15:15:28 -07:00
Bimba Shrestha 0b52d878b2 Cleaning up gate and adding comment to flag 2019-10-25 14:06:50 -07:00
W. Felix Handte 74bd76c3ff In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time 
Addresses #1794. Instead of deriving the lib dir and include dir at
build-time, let's do it like everyone else does at pkg-config run-time.

This has the disadvantage that we can no longer override LIBDIR and
INCLUDEDIR in the Makefile and have that reflected in the .pc file.
 2019-10-25 15:07:31 -04:00
Bimba ShresthaandGitHub fb509a2a7b Merge pull request #8 from bimbashrestha/appveyorTest 
Appveyor test
 2019-10-25 12:07:05 -07:00
Bimba Shrestha 8adecc73b0 Running playtests.sh on PRs too 2019-10-25 12:04:54 -07:00
Bimba Shrestha 5b45ff4f1c Gating named file support on windows 2019-10-25 11:32:38 -07:00
W. Felix Handte 4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
Yann ColletandGitHub c2140e9db0 Merge pull request #1845 from facebook/zbuff 
improve deprecation warning macro
 2019-10-25 09:59:00 -07:00
W. Felix Handte 24499036ba Add Tests 2019-10-24 20:19:10 -04:00
W. Felix Handte 91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Yann Collet 42a22af78b fix zlibWrapper for Visual Studio 
As per https://github.com/facebook/zstd/issues/1800#issuecomment-545945050,
Visual does not support `ssize_t` type,
which is an issue for `gzread.c`.

Added a work around, suggested by @bluenlive

Note : I have not been able to confirm the problem,
so this is a blind fix.
This seems safe outside of Visual, since it is gated by _MSC_VER macro.
 2019-10-24 15:19:05 -07:00