ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
7,385 Commits 17 Branches 77 Tags
f627e1a1ae76b956aa060c1463aa8020dda0e168
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Felix HandteandGitHub 3a9a079ed1 Merge pull request #1956 from PhoebeHui/dev 
Add vcpkg installation instructions
 2020-01-21 11:54:00 -05:00
Felix HandteandGitHub 68160372be Merge pull request #1920 from felixhandte/fix-mtim-again 
Use statbuf->st_mtim Again
 2020-01-03 16:07:16 -05:00
Felix HandteandYann Collet 6f4341c432 Fix playTests.sh Under QEMU (#1923) 2019-12-26 11:16:23 -08:00
W. Felix Handte e2a99db2dd Also Define _ATFILE_SOURCE 2019-12-23 13:24:28 -05:00
W. Felix Handte 5666835ea7 Add Comment 2019-12-23 13:24:28 -05:00
W. Felix Handte 5af8cb7aea Use statbuf->st_mtim Again 2019-12-23 13:24:28 -05:00
W. Felix Handte 976c055a9a Add Travis Test that Builds ARM on Trusty 2019-12-23 13:24:28 -05:00
W. Felix Handte 5f46894184 Fix playTests.sh Under QEMU 2019-12-23 13:24:28 -05:00
Felix HandteandGitHub 07ad866754 Merge pull request #1932 from felixhandte/diagnose-corruption-dicts 
Add Support for Dictionaries in Corruption Diagnosis Tool
 2019-12-19 12:08:28 -05:00
W. Felix Handte fe454c0a76 Add Support for Dictionaries 2019-12-18 11:54:39 -08:00
W. Felix Handte 18f03516b3 Print Summary of Errors Encountered 2019-12-18 11:29:53 -08:00
W. Felix Handte 4584c88a99 Don't Rely on Implicit Return in Main 2019-12-17 11:26:18 -08:00
W. Felix Handte 722149cf2b Easy: Update Comment on ZSTD_initDStream() 2019-11-19 01:57:15 -05:00
W. Felix Handte b5fb2e7c12 Add Tool to Diagnose Whether Corrupt Blobs are Plausibly Bit-Flips 
I spend an increasing amount of my time looking at "Corrupted block detected"
failures in decompression. Not infrequently, I suspect that it is the result
of hardware failure, and that the blob has become bit-flipped or otherwise
corrupted somewhere along the line.

For that reason I was motivated to write a little tool to inspect blobs that
fail to decompress, to try modifying them, and then check whether they
decompress successfully. This seems like potentially a generally useful tool,
so I figured it might be worth putting in `contrib/`.
 2019-11-19 01:53:36 -05:00
Felix HandteandGitHub 5688447758 Merge pull request #1873 from felixhandte/make-overlap-log-multithread-only 
Fix #1861: Restrict overlapLog Parameter When Not Built With Multithreading
 2019-11-06 16:56:37 -05:00
Felix HandteandGitHub ba4613602f Merge pull request #1843 from moozzyk/issue-1637 
Take ZSTD_parameters as a const pointer
 2019-11-06 16:56:14 -05:00
Felix HandteandGitHub 32f989c39a Merge pull request #1869 from bket/openbsd 
'head -c BYTES' is non-portable.
 2019-11-06 16:54:03 -05:00
Felix HandteandGitHub cba922f3e6 Merge pull request #1870 from bket/portable 
'date -r FILE' is non-portable.
 2019-11-06 16:52:24 -05:00
W. Felix Handte c13f81905a Fix #1861: Restrict overlapLog Parameter When Not Built With Multithreading 
This parameter is unused in single-threaded compression. We should make it
behave like the other multithread-only parameters, for which we only accept
zero when we are not built with multithreading.
 2019-11-06 16:05:02 -05:00
Felix HandteandGitHub f9d867e79f Merge pull request #1857 from 0x123456789A/link-shared 
Add CMake flag for linking programs dynamically
 2019-10-30 15:43:11 -04:00
Felix HandteandGitHub 01ec595b85 Merge pull request #1851 from felixhandte/pkg-config-prefix-fix 
In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time
 2019-10-28 14:24:56 -04:00
W. Felix Handte 74bd76c3ff In pkg-config File, Derive Lib and Include Dir from Prefix at Use-Time 
Addresses #1794. Instead of deriving the lib dir and include dir at
build-time, let's do it like everyone else does at pkg-config run-time.

This has the disadvantage that we can no longer override LIBDIR and
INCLUDEDIR in the Makefile and have that reflected in the .pc file.
 2019-10-25 15:07:31 -04:00
W. Felix Handte 4eccc82875 Minor Fixes 2019-10-25 13:59:19 -04:00
W. Felix Handte 24499036ba Add Tests 2019-10-24 20:19:10 -04:00
W. Felix Handte 91c3f545cc Fix Build; Refactor 2019-10-24 20:18:57 -04:00
Felix HandteandGitHub 30b27ee0c9 Merge pull request #1790 from stokito/tzst-ext 
Add short tar's extensions .tgz (.tar.gz), .txz (.tar.xz), .tzst (.tar.zst)
 2019-10-24 17:50:22 -04:00
Felix HandteandGitHub 506e1a1176 Merge branch 'tzst-ext-tmp' into tzst-ext 2019-10-24 17:49:34 -04:00
Felix HandteandGitHub cf725630a6 Merge pull request #1795 from felixhandte/workspace-asan 
Add Poisoned Redzones to the Workspace When Compiling with ASAN
 2019-10-21 12:15:17 -04:00
W. Felix Handte ede31da2ea Fix CCtx Size Estimation 2019-10-10 15:02:08 -04:00
W. Felix Handte bd6a20b8a0 Expand Default Redzone Size 2019-10-10 13:45:55 -04:00
W. Felix Handte 2c80a9f8ac Check if CCtx in Workspace after Null Check 2019-10-10 13:40:16 -04:00
W. Felix Handte b6987acbbf Declare the ASAN Functions We Need, Don't Include the Header 2019-10-10 13:40:16 -04:00
W. Felix Handte 0ffae7e440 Stop Allocating Extra Space for Table Redzones 2019-10-10 13:40:16 -04:00
W. Felix Handte a07037b784 Don't Try to Redzone the Tables 2019-10-10 13:40:16 -04:00
W. Felix Handte 0cc481ef66 Fix Workspace Size Calculation 2019-10-10 13:40:16 -04:00
W. Felix Handte b6c0a02a17 Fix ZSTD_sizeof_matchState() Calculation 2019-10-10 13:40:16 -04:00
W. Felix Handte 8cffd6ed08 Avoid ASAN Failure in ZSTD_cwksp_free() 2019-10-10 13:40:16 -04:00
W. Felix Handte ef0b5707c5 Refactor Freeing CCtxes / CDicts Inside Workspaces 2019-10-10 13:40:16 -04:00
W. Felix Handte 143b296cf6 Surround Workspace Allocs with Dead Zone 2019-10-10 13:40:16 -04:00
W. Felix Handte 19a0955ec9 Add ZSTD_cwksp_alloc_size() to Help Calculate Needed Workspace Size 2019-10-10 13:40:16 -04:00
W. Felix Handte da88c35d41 Stop Assuming Tables are Adjacent 2019-10-10 13:40:16 -04:00
W. Felix Handte 35c30d6ca7 Poison Unused Workspace Memory 2019-10-10 13:40:16 -04:00
W. Felix Handte edb6d884a5 Detect Whether We're Being Compiled with ASAN 2019-10-10 13:40:16 -04:00
W. Felix Handte dc1fb684bf Remove Unused MEM_SKIP_MSAN Macro 2019-10-10 13:40:16 -04:00
Felix HandteandGitHub c047fcf7bf Merge pull request #1806 from felixhandte/estimate-cctx-doc 
Update Comment on `ZSTD_estimateCCtxSize()`
 2019-09-20 15:36:00 -04:00
W. Felix Handte f7d9b36835 Update Comment on ZSTD_estimateCCtxSize() 2019-09-20 14:11:29 -04:00
Felix HandteandGitHub dd2838eeb4 Merge pull request #1783 from felixhandte/mtime-nsec 
Set Mod Time Nanoseconds
 2019-09-17 13:30:21 -04:00
Felix HandteandGitHub 2164a130f3 Merge pull request #1780 from felixhandte/workspace-efficiency-3 
Avoid Clearing Tables Even When Changing CParams
 2019-09-16 14:37:05 -04:00
W. Felix Handte 72ea79cacd Don't Include sanitizer/msan_interface.h, Since Not All Platforms Provide It 
Instead, explicitly declare the functions we use.
 2019-09-16 12:08:03 -04:00
W. Felix Handte e1ec8004cc Formatting and Clean Up 2019-09-12 16:27:05 -04:00
W. Felix Handte 5a9baae9cf Set M-Time Nanoseconds 2019-09-12 11:50:33 -04:00
Felix HandteandGitHub 6ae1ec96bc Merge pull request #1708 from neheb/dev 
zstd: Don't use utime on Linux
 2019-09-12 11:44:31 -04:00
W. Felix Handte 20c69077d1 Shrink Table Valid End During Alloc Alignment / Phase Change 2019-09-11 17:14:59 -04:00
W. Felix Handte 51d90668ba Add Assertions to Confirm that Workspace Pointers are Correctly Ordered 2019-09-11 17:14:59 -04:00
W. Felix Handte a10c191613 __msan_poison() Workspace When Preparing for Re-Use 2019-09-11 17:14:45 -04:00
W. Felix Handte 194c542598 Fix Memory Leak in Test 2019-09-11 14:25:30 -04:00
W. Felix Handte ff67c62458 Fix Compilation Error (uint32_t -> size_t) 2019-09-11 13:59:09 -04:00
W. Felix Handte 5707c8a9d5 Speed Up Test a Little 2019-09-11 13:23:59 -04:00
W. Felix Handte ed4c2c60c3 Add Fuzzer Test Case for Index Reduction 2019-09-11 13:17:19 -04:00
W. Felix Handte 7c57e2b9ca Zero h3size When h3log is 0 
This led to a nasty edgecase, where index reduction for modes that don't use
the h3 table would have a degenerate table (size 4) allocated and marked clean,
but which would not be re-indexed.
 2019-09-11 13:14:26 -04:00
W. Felix Handte bc020eec92 Also Shrink Clean Table Area When Reducing Indices 2019-09-11 11:40:57 -04:00
W. Felix Handte 1999b2ed9b Update DEBUGLOG Statements 2019-09-11 11:21:00 -04:00
W. Felix Handte 13e29a56de Shrink Clean Table Area When Copying Table Contents into Context 
The source matchState is potentially at a lower current index, which means
that any extra table space not overwritten by the copy may now contain
invalid indices. The simple solution is to unconditionally shrink the valid
table area to just the area overwritten.
 2019-09-11 11:18:45 -04:00
W. Felix Handte edb3ad053e Comments 2019-09-10 18:25:45 -04:00
W. Felix Handte f31ef28ff8 Only Reset Indexing in ZSTD_resetCCtx_internal() When Necessary 2019-09-10 18:25:45 -04:00
W. Felix Handte 9968a53e91 Remove No-Longer-Used Continuation Functions 2019-09-10 18:25:45 -04:00
W. Felix Handte 1b28e80416 Remove Fast Continue Path in ZSTD_resetCCtx_internal() 2019-09-10 18:25:45 -04:00
W. Felix Handte ad16eda5e4 ZSTD_reset_matchState Optionally Doesn't Restart Indexing 2019-09-10 18:25:45 -04:00
W. Felix Handte 5b10bb5ec3 Rename ZSTD_compResetPolicy_e Values and Add Comment 2019-09-10 18:25:45 -04:00
W. Felix Handte 0492b9a9ec Accept ZSTD_indexResetPolicy_e Param in ZSTD_reset_matchState() 2019-09-10 18:25:45 -04:00
W. Felix Handte 14c5471d5e Introduce ZSTD_indexResetPolicy_e Enum 2019-09-10 18:25:45 -04:00
W. Felix Handte 17b6da2e0f Track Usable Table Space in Compression Workspace 2019-09-10 18:25:37 -04:00
W. Felix Handte a9d373f093 Remove Empty lib/compress/zstd_cwksp.c 2019-09-10 16:03:13 -04:00
W. Felix Handte 81208fd7c2 Forward Declare ZSTD_cwksp_available_space to Fix Build 2019-09-09 19:10:09 -04:00
W. Felix Handte 91bf1babd1 Inline Workspace Functions 2019-09-09 18:53:53 -04:00
W. Felix Handte 0db3ffe7ee Forward resetCCtx Errors when Using CDict 2019-09-09 16:47:19 -04:00
W. Felix Handte eb6f69d978 Fix sizeof_CCtx and sizeof_CDict Calculations for Statically Init'ed Objects 2019-09-09 16:45:17 -04:00
W. Felix Handte e3703825a8 Fix workspaceTooSmall Calculation 2019-09-09 15:12:14 -04:00
W. Felix Handte 0a65a67901 Shorten &zc->workspace -> ws in ZSTD_resetCCtx_internal() 2019-09-09 14:59:09 -04:00
W. Felix Handte 1120e4d962 Clean Up TODOs and Comments pt. II 2019-09-09 14:04:39 -04:00
W. Felix Handte c60e1c3be5 Nit 2019-09-09 13:34:08 -04:00
W. Felix Handte 7d7b665c90 Pull Phase Advance Logic Out into Internal Function 2019-09-09 13:34:08 -04:00
W. Felix Handte 8549ae9f1d Hide Workspace Movement Behind Helper Function 2019-09-09 13:34:08 -04:00
W. Felix Handte 2405c03bcd Fix DEBUGLOG Statement Levels 2019-09-09 13:34:08 -04:00
W. Felix Handte 7100d24221 Fix Rescale Continue Special Case 2019-09-09 13:34:08 -04:00
W. Felix Handte 7321e4c9f3 Remove Unused noRealloc CRP Value 2019-09-09 13:34:08 -04:00
W. Felix Handte 901bba4ca6 Re-Implement Workspace Shrinking when Oversized 2019-09-09 13:34:08 -04:00
W. Felix Handte e8cc137477 Add New File to Visual Studio Projects 2019-09-09 13:34:08 -04:00
W. Felix Handte 4e9b1341cf Add New File to Meson Build 2019-09-09 13:34:08 -04:00
W. Felix Handte 881bcd80ca Cleanup from Move 2019-09-09 13:34:08 -04:00
W. Felix Handte b511a84adc Move Workspace Functions to Their Own File 2019-09-09 13:34:08 -04:00
W. Felix Handte 077a2d7dc9 Rename 2019-09-09 13:34:08 -04:00
W. Felix Handte ebd162194f Clean Up TODOs and Comments 2019-09-09 13:34:08 -04:00
W. Felix Handte 2abe0145b1 Improve Comments a Bit 2019-09-09 13:34:08 -04:00
W. Felix Handte 7a2416a863 Allocate CDict in Workspace (Rather than in Separate Allocation) 2019-09-09 13:34:08 -04:00
W. Felix Handte 65057cf009 Rewrite ZSTD_initStaticCCtx to Alloc CCtx in Workspace 2019-09-09 13:34:08 -04:00
W. Felix Handte 58b69ab15c Only the CCtx Itself Needs to be Cleared during Static CCtx Init 2019-09-09 13:34:08 -04:00
W. Felix Handte 88c2fcd0ee Align Alloc Pointer When Transitioning from Buffers to Aligned Allocs 2019-09-09 13:34:08 -04:00
W. Felix Handte e936b73889 Remove Overly-Restrictive Assert 2019-09-09 13:34:08 -04:00
W. Felix Handte 75d574368b When Loading Dict By Copy, Always Put it in the Workspace 2019-09-09 13:34:08 -04:00