ddidderr/zstd-rs
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
11,725 Commits 17 Branches 77 Tags
e3d1f7057dd8cddf28bb3d40fba1ea3dc3f94de4
Commit Graph
2369 Commits
Author SHA1 Message Date
W. Felix Handte 9b67219b1e Fix Integer Constants; Fix Comparison 2021-06-10 12:53:07 -04:00
W. Felix Handte bbb81c8801 Avoid snprintf() in Preparing Human-Readable Sizes; Improve Formatting 
This produces the following formatting:

   Size    | `zstd` | `ls -lh`
---------- | ------ | --------
1          | 1      | 1
12         | 12     | 12
123        | 123    | 123
1234       | 1.21K  | 1.3K
12345      | 12.1K  | 13K
123456     | 121K   | 121K
1234567    | 1.18M  | 1.2M
12345678   | 11.8M  | 12M
123456789  | 118M   | 118M
1234567890 | 1.15G  | 1.2G
999        | 999    | 999
1000       | 1000   | 1000
1001       | 1001   | 1001
1023       | 1023   | 1023
1024       | 1.000K | 1.0K
1025       | 1.00K  | 1.1K
999999     | 977K   | 977K
1000000    | 977K   | 977K
1000001    | 977K   | 977K
1023999    | 1000K  | 1000K
1024000    | 1000K  | 1000K
1024001    | 1000K  | 1001K
1048575    | 1024K  | 1.0M
1048576    | 1.000M | 1.0M
1048577    | 1.00M  | 1.1M

This was produced with the following invocation:

```
for N in 1 12 123 1234 12345 123456 1234567 12345678 123456789 1234567890 999 1000 1001 1023 1024 1025 999999 1000000 1000001 1023999 1024000 1024001 1048575 1048576 1048577; do
  head -c $N /dev/urandom > r$N
done
./zstd -i1 -b1 -S r1 r12 r123 r1234 r12345 r123456 r1234567 r12345678 r123456789 r1234567890 r999 r1000 r1001 r1023 r1024 r1025 r999999 r1000000 r1000001 r1023999 r1024000 r1024001 r1048575 r1048576 r1048577
```
 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 8e0a9695d7 Attempt to fix a failing test with help from @aqrit 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 1eb852854b Some fixes to address things @felixhandte found 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 376a2730a8 Try enabling the BIG strings now the unsigned long long is in effect 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 20b9b00b41 Try unsigned long long 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 64385ef7cb Update humanSize() to skip the big numbers (it requires 64 bit) 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 1ef6f3d079 Use unsigned long instead to help with some tests 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte e5fc830795 human_size() should use size_t 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 35576e63ce Convert tabs to spaces 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 77001f00fb Use human_size() on the "multiple files compressed" output also 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 894698d3b6 Use human_size() in the benchmark output also 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 4e0d9f1cc8 Move the variable declarations to the top 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte eefdbcd93a Make the variable types match 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte b6b23dfe64 Convert names to CamelCase 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte b70175e5ec Put the human_size() function in util.c 2021-06-10 12:53:07 -04:00
Scott BakerandW. Felix Handte 26fab1d963 Make the CLI output the file sizes in human readable format 2021-06-10 12:53:07 -04:00
binhdvoandGitHub 325952f878 Revert "Add support for --long-param flag, fix #2104" 2021-06-09 15:35:43 -04:00
Binh Vo 6583fa3f0a Add support for --long-param flag 2021-06-09 14:07:52 -04:00
Binh Vo d2f31b6627 Fix --progress flag to properly control progress display and default progress display on when using -v 2021-06-08 17:24:38 -04:00
Binh Vo 1e17184ad0 Add documentation for --patch-from 2021-06-03 11:12:27 -04:00
Samuli Piippo fa2a6d4746 Makefile: fix build for mingw 
Add ${EXT} to required places to make install succeed for mingw build.
 2021-05-27 16:09:37 +03:00
Yann Collet 8fae35591e Merge branch 'dev' of github.com:facebook/zstd into dev 2021-05-12 13:12:30 -07:00
Olivier PerretandGitHub d4548c96cb fileio: clamp value of windowLog in patch-mode (#2637) 
With small enough input files, the inferred value of fileWindowLog could
be smaller than ZSTD_WINDOWLOG_MIN.

This can be reproduced like so:
$ echo abc > small
$ echo abcdef > small2
$ zstd --patch-from small small2 -o patch
previously, this would fail with the error "zstd: error 11 : Parameter is out of bound"
 2021-05-12 16:11:15 -04:00
Yann Collet cb0cad9b79 reduce Max nb Workers to 64 in 32-bit mode 
and restored limit to 256 when in 64-bit mode
(it was reduced to 200 to give more room for 32-bit).

This should fix test instability issues
using lot of threads in 32-bit environments.
 2021-05-12 13:10:25 -07:00
Yann ColletandGitHub 8a53a882f2 updated generated man pages for v1.5.0 (#2635) 2021-05-11 18:17:31 -04:00
senandGitHub 13449d7ce1 Add PHONY targets to makefiles (#2629) 2021-05-07 14:03:19 -04:00
senandGitHub 6030cdfede Add --progress flag (#2595) 2021-05-06 14:50:28 -04:00
Yann ColletandGitHub 2f7bbd6539 Merge pull request #2620 from facebook/winFilelist 
fix --filelist compatibility with Windows cr+lf line ending
 2021-05-06 11:35:16 -07:00
Yann Collet df05b2ba7c fix --filelist compatibility with Windows cr+lf line ending 2021-05-05 18:01:55 -07:00
Yann Collet 9750f3c87b improved benchmark experience on Windows 
benchmark results are not progressively displayed on Windows terminal.
For long benchmark sessions, nothing is displayed,
until the end, where everything is flushed.

Force display to be flushed after each update.
Updates happen roughtly every second, or even less,
so it's not a substantial workload.
 2021-05-05 16:52:21 -07:00
W. Felix Handte 4f9c6fdb7f Attempt to Fix Windows Build Error 2021-05-05 13:13:56 -04:00
W. Felix Handte da61918c75 Also Pass Mode Bits in on Windows 
I think in some unix emulation environments on Windows, (cygwin?) mode bits
are somehow respected. So we might as well pass them in. Can't hurt.
 2021-05-05 13:10:34 -04:00
W. Felix Handte 45c4918ccf Fix Build for Windows 2021-05-05 13:10:34 -04:00
W. Felix Handte 1fb10ba831 Don't Block Removing File on Being Able to Read It 
`open()`'s mode bits are only applied to files that are created by the call.
If the output file already exists, but is not readable, the `fopen()` would
fail, preventing us from removing it, which would mean that the file would
not end up with the correct permission bits.

It's not clear to me why the `fopen()` is there at all. `UTIL_isRegularFile()`
should be sufficient, AFAICT.
 2021-05-05 13:10:34 -04:00
W. Felix Handte b87f97b3ea Create Files with Desired Permissions; Avoid chmod(); Remove UTIL_chmod() 2021-05-05 13:10:34 -04:00
Felix HandteandGitHub 2d10544b84 Merge pull request #2613 from felixhandte/allow-block-device 
Allow Reading from Block Devices with `--force`
 2021-05-05 13:06:32 -04:00
Yann Collet 455fd1a067 updated documentation regarding minimum job size 2021-05-05 09:03:11 -07:00
W. Felix Handte 33f3e293e8 Allow Reading from Block Devices with --force 2021-05-04 16:25:26 -04:00
Nick TerrellandGitHub a8ecf4ff88 Merge pull request #2597 from terrelln/public-headers 
[1.5.0] Move `zstd_errors.h` and `zdict.h` to `lib/` root
 2021-05-04 11:28:41 -07:00
W. Felix Handte ee122baacf Detect Presence of md5 on Darwin 
This fixes #2568.
 2021-05-04 12:33:19 -04:00
Nick Terrell 09149beaf8 [1.5.0] Move zstd_errors.h and zdict.h to lib/ root 
`zstd_errors.h` and `zdict.h` are public headers, so they deserve to be
in the root `lib/` directory with `zstd.h`, not mixed in with our private
headers.
 2021-04-30 15:13:54 -07:00
Nick TerrellandSen Huang 4694423c4f Add and integrate lazy row hash strategy 2021-04-07 09:53:34 -07:00
Niclas RosenvikandNick Terrell e7647180cd Stop complaining about hash tool not found 
If build_dir is set the zstd build complains about md5sum not being found.
Fix this by checking if build_dir is set before checking and using the hash tool
just like in lib/Makefile .
 2021-04-02 13:00:19 -07:00
Nick TerrellandNick Terrell a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files 
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
 2021-03-30 10:30:43 -07:00
Sen Huang f27e326456 Restrict dictmode regression tests only to advanced API, fix some compiler warnings 2021-03-25 10:39:08 -07:00
Yann Collet 0f99a0d987 fix man page typo 
fix #2553 detected by @jwilk
 2021-03-24 05:55:04 -07:00
Yann Collet 9fb4a42c7b fix #2549 2021-03-20 17:29:41 -07:00
Paul BoneandPaul Bone 4d6c78fb89 Only set numPhysicalCores if ratio is valid 2021-03-03 10:59:00 +11:00
Paul BoneandPaul Bone eb1a09df61 If cpuinfo parsing fails fallback to sysconf 2021-03-03 10:58:51 +11:00