ddidderr/zstd-rs
1
0 Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,366 Commits 17 Branches 77 Tags
fbd5ab70272095255948255368ac4f2f64aca780
Commit Graph
1084 Commits
Author SHA1 Message Date
Yann Collet fbd5ab7027 minor fix : no longer use fake srcSize during resource creation 
srcSize is read and provided at each file, not at resource creation.
This used to be useful with older API, because it could not re-adapt parameters between sessions.

At some point, it will be better to remove the old code, and only keep the new_api.
It works fine by now.
 2017-09-29 19:40:27 -07:00
Yann Collet db1668a43b fix : srcSize written in frame header when multiple files compressed 
This information used to be disabled when nbFiles>1.
It was badly initialized later in the code, resulting in an error.
 2017-09-29 18:05:18 -07:00
Yann Collet 54a827fff0 Merge branch 'dev' into newFormats 
Fixed conflicts in zstdmt_compress.c
 2017-09-27 16:39:40 -07:00
Yann Collet 3182ea2e64 Merge pull request #866 from facebook/list 
improved --list display
 2017-09-27 16:34:29 -07:00
Nick Terrell c233bdbaee Increase maximum window size 
* Maximum window size in 32-bit mode is 1GB, since allocations for 2GB fail
  on my Mac.
* Maximum window size in 64-bit mode is 2GB, since that is the largest
  power of 2 that works with the overflow prevention.
* Allow `--long=windowLog` to set the window log, along with
  `--zstd=wlog=#`. These options also set the window size during
  decompression, but don't override `--memory=#` if it is set.
* Present a helpful error message when the window size is too large during
  decompression.
* The long range matcher defaults to a hash log 7 less than the window log,
  which keeps it at 20 for window log 27.
* Keep the default long range matcher window size and the default maximum
  window size at 27 for the API and CLI.
* Add tests that use the maximum window size and hash size for compression
  and decompression.
 2017-09-26 14:00:01 -07:00
Yann Collet 3095ca8c56 fixed minor conversion warnings for g++ on Linux 
U64 is not considered equivalent to unsigned long long
 2017-09-26 13:53:50 -07:00
Yann Collet 56f1f0e3dd write summary for --list on multiple files 2017-09-26 11:21:36 -07:00
Yann Collet 62568c9a42 added capability to generate magic-less frames 
decoder not implemented yet
 2017-09-25 14:26:26 -07:00
Yann Collet b0c0e3a3fb Merge pull request #853 from terrelln/blog 
[zstdcli] Fix LDM advanced options parsing
 2017-09-18 15:21:23 -07:00
Nick Terrell 1fe762e236 [zstdcli] Fix LDM advanced options parsing 2017-09-18 14:49:35 -07:00
Yann Collet 92889709f9 fix #851 : sudo zstd -t file.zst changes /dev/null permissions 
reported by @mike155
 2017-09-18 13:41:54 -07:00
Yann Collet 1722055799 add comment on using -B# to split input file for dictionary training 2017-09-15 16:23:50 -07:00
Yann Collet c68d17f2da ensures that sampleSizes table is large enough 
as recommended by @terrelln
 2017-09-15 15:31:31 -07:00
Yann Collet 25a60488dd fixed 64-to-32 conversion warnings 2017-09-15 11:55:13 -07:00
Yann Collet a9694231ca fixed minor conversion warning 2017-09-15 10:16:26 -07:00
Yann Collet 086b9597d9 added ability to split input files for dictionary training 
using command -B#
This is the same behavior as benchmark module,
which can also split input into arbitrary size blocks, using -B#.
 2017-09-14 16:45:10 -07:00
Yann Collet 77c137b3ae minor comment refactor 2017-09-14 15:12:57 -07:00
Yann Collet f1571dad8f Merge pull request #838 from stellamplau/ldm-mergeDev 
Add long distance matcher
 2017-09-13 13:24:08 -07:00
Yann Collet 8f26dc3f9c blindfix for Visual 
LARGE_INTEGER is not an integer :
https://msdn.microsoft.com/en-us/library/windows/desktop/aa383713(v=vs.85).aspx

Do not take any risk with the structure definition :
use int init = 0; like Mac code
 2017-09-12 21:21:17 -07:00
Yann Collet bc41c7f0eb fixed minor prototype warning 2017-09-12 19:32:26 -07:00
Yann Collet c95c0c9725 modified util::time API 
for easier invocation.
- no longer expose frequency timer :
it's either useless, or stored internally in a static variable (init is only necessary once).
- UTIL_getTime() provides result by function return.
 2017-09-12 18:12:46 -07:00
Nick Terrell 6ab4d5e904 [bench] Use higher resolution timer on POSIX 
The timer used was only accurate up to 0.01 seconds. This timer is accurate up to 1 ns.
It is a monotonic timer that measures the real time difference, not on CPU time.
 2017-09-12 16:46:44 -07:00
Stella Lau eb3327c10a Merge branch 'dev' of https://github.com/facebook/zstd into ldm-mergeDev 2017-09-11 15:00:01 -07:00
Yann Collet 3128e03be6 updated license header 
to clarify dual-license meaning as "or"
 2017-09-08 00:09:23 -07:00
Yann Collet baa37c3362 programs/Makefile : better support for GNU conventions 
see https://www.gnu.org/prep/standards/html_node/Command-Variables.html
 2017-09-06 16:53:59 -07:00
Stella Lau eeff55dfa8 Merge remote-tracking branch 'upstream/dev' into ldm-mergeDev 2017-09-06 15:56:32 -07:00
Stella Lau 8c33cfe0bc Add ldm documentation in README 2017-09-06 15:21:01 -07:00
Stella Lau c706de5395 Rename and add short ldm parameters in cli 2017-09-05 21:11:18 -07:00
Stella Lau 643d28c701 Add ldm options to 'man zstd' 2017-09-05 11:27:15 -07:00
Stella Lau 67d4a6161c Add ldmBucketSizeLog param 2017-09-02 21:55:29 -07:00
Stella Lau a1f04d518d Move hashEveryLog to cctxParams and update cli 2017-09-01 15:05:47 -07:00
Yann Collet 0558850735 bench stops immediately on decoding error 2017-09-01 11:46:15 -07:00
Stella Lau 17d8e0bdcc Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate 2017-09-01 10:19:38 -07:00
Stella Lau 8081becadc Add long distance matching as a CCtxParam 2017-09-01 09:18:58 -07:00
Yann Collet d7ad99b2ab Merge branch 'longRangeMatcher' into dev 2017-08-31 18:08:37 -07:00
Yann Collet c7818fc676 Merge branch 'modTests' into dev 
fixed conflict
 2017-08-31 17:00:16 -07:00
Yann Collet 4299c27132 improved console log of utils.h 
removed a warning when compiling on Windows
 2017-08-31 16:58:47 -07:00
Yann Collet 8e298382a8 changed target allarch into allzstd 
allzstd contains only zstd-related tests.
allmost = allzstd + zwrapper tests (which require zlib)
 2017-08-31 14:30:52 -07:00
Stella Lau 6a546efb8c Add long distance matcher 
Move last literals section to ZSTD_block_internal
 2017-08-31 12:53:19 -07:00
Yann Collet b0cb081dc8 last batch of header files changed to reflect new license (#825) 
only remains to update contrib/linux-kernel (@terrelln)
 2017-08-31 12:20:50 -07:00
Stella Lau c88fb9267f Replace 'byReference' with enum 2017-08-29 11:55:02 -07:00
Bernhard M. Wiedemann cf689b84f9 Sort input file list 
in order to make builds reproducible
in spite of indeterministic filesystem readdir order.
See https://reproducible-builds.org/ for why this is good.
 2017-08-26 17:08:00 +02:00
Stella Lau 6f1a21c7e9 Remove formatting-only changes 2017-08-23 10:24:19 -07:00
Yann Collet 232d62b637 fixed a few headers that were too hastily copy/pasted during last license change 2017-08-21 11:24:32 -07:00
Stella Lau 91b30dbe84 Remove test parameter 2017-08-21 10:09:06 -07:00
Stella Lau f181f33bdf Disable tests and refactor 2017-08-21 01:59:08 -07:00
Stella Lau 023b24e6d4 Add cctx param tests 2017-08-20 22:55:07 -07:00
Yann Collet 1c108c811e cli : Display supported formats on -vV command 
Requested and inspired by patch from @ib (#771)
 2017-08-19 13:33:50 -07:00
Yann Collet 2ecd34ee5e fixed unused variables warnings 2017-08-19 01:23:49 -07:00
Yann Collet 23706fb743 updated doc on compilation variables 2017-08-19 01:14:36 -07:00