Commit Graph
30 Commits
Author SHA1 Message Date
Yann ColletandGitHub 7607b96ed9 Merge pull request #3385 from jonpalmisc/improve_help
Improve help/usage (`-h`, `-H`) formatting
2022-12-21 10:56:09 -08:00
Jon Palmisciano 125bcde17c Update tests to expect new CLI help output 2022-12-20 20:13:00 -05:00
W. Felix Handte 8927f985ff Update Copyright Headers 'Facebook' -> 'Meta Platforms'
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
  sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
2022-12-20 12:37:57 -05:00
Nick TerrellandNick Terrell 15f32ad74c [fileio] Separate parameter adaption from display update rate
Split the logic for parameter adaption from the logic to update the display rate.
This decouples the two updates, so changes to display updates don't affect
parameter adaption.

Also add a test case that checks that parameter adaption actually happens.

This fixes Issue #3353, where --adapt is broken when --no-progress is passed.
2022-12-14 17:08:21 -08:00
Nick TerrellandNick Terrell fbff7827fa Refactor progress bar & summary line logic
* Centralize the logic about whether to print the progress bar or not in
  the `*_PROGRESS()` macros.
* Centralize the logc about whether to print the summary line or not in
  `FIO_shouldDisplayFileSummary()` and
  `FIO_shouldDisplayMultipleFileSummary()`.
* Make `--progress` work for non-zstd (de)compressors.
* Clean up several edge cases in compression and decompression progress
  printing along the way. E.g. wrong log level, or missing summary line.

One thing I don't like about stdout mode, which sets the display level
to 1, is that warnings aren't displayed. After this PR, we could change
stdout mode from lowering the display level, to defaulting to implied
`--no-progress`. But, I think that deserves a separate PR.
2022-12-14 15:44:09 -08:00
Danielle Rozenblit 9b9ad5af3f use .ignore for stderr output in window-resize test case 2022-12-13 13:52:54 -08:00
Danielle Rozenblit 32bb667138 added test to cli-tests 2022-12-13 12:45:41 -08:00
358787764f Fix for zstd CLI accepts bogus values for numeric parameters (#3268)
* add checks to mal-formed numeric values for memory and memlimit parameters

Signed-off-by: Ly Cao <lycao@fb.com>

* changed errorMsg to a literal string instead of static string in main

* moved bogus numeric error to NEXT_UINT32 + add macro NEXT_TSIZE

Signed-off-by: Ly Cao <lycao@fb.com>

Signed-off-by: Ly Cao <lycao@fb.com>
Co-authored-by: Ly Cao <lycao@fb.com>
2022-09-21 13:20:01 -07:00
Chris BurgessandGitHub 3b4e47092e Document pass-through behavior (#3242)
Adds documentation to help and man pages for legacy pass-through behavior
when force is set and destination is stdout. Documents --pass-through in
man pages
2022-08-15 10:29:54 -07:00
Nick TerrellandGitHub 03cc84fddb Add explicit --pass-through flag and default to enabled for *cat (#3223)
Fixes #3211.

Adds the `--[no-]pass-through` flag which enables/disables pass-through mode.

* `zstdcat`, `zcat`, and `gzcat` default to `--pass-through`.
  Pass-through mode can be disabled by passing `--no-pass-through`.
* All other binaries default to not setting pass-through mode.
  However, we preserve the legacy behavior of enabling pass-through
  mode when writing to stdout with `-f` set, unless pass-through
  mode is explicitly disabled with `--no-pass-through`.

Adds a new test for this behavior that should codify the behavior we want.
2022-08-04 17:15:59 -07:00
d0dcc9d775 fix issue #3144 (#3226)
* fix issue #3144

* add test case for verbose-wlog

Co-authored-by: zengyijing <yijingzeng@fb.com>
2022-08-04 13:51:14 -07:00
Nick TerrellandGitHub a70ca2bd7d Fix off-by-one error in superblock mode (#3221)
Fixes #3212.

Long literal and match lengths had an off-by-one error in ZSTD_getSequenceLength.
Fix the off-by-one error, and add a golden compression test that catches the bug.
Also run all the golden tests in the cli-tests framework.
2022-08-03 11:28:39 -07:00
Elliot GorokhovskyandGitHub 28ceb63503 Merge pull request #3220 from embg/issue3200
Disallow empty string as argument for --output-dir-flat and --output-dir-mirror
2022-08-01 14:04:57 -04:00
Yonatan KomornikandGitHub ae4670466c stdin multiple file fixes (#3222)
* Fixes for https://github.com/facebook/zstd/issues/3206 - bugs when handling stdin as part of multiple files.

* new line at end of multiple-files.sh
2022-07-29 16:13:07 -07:00
Elliot Gorokhovsky f9f27de91c Disallow empty output directory 2022-07-29 14:48:33 -07:00
Tom WangandGitHub d4a5bc4efc Add warning when multi-thread decompression is requested (#3208)
When user pass in argument for both decompression and multi-thread, print a warning message
to indicate that multi-threaded decompression is not supported.

* Add warning when multi-thread decompression is requested
* add test case for multi-threaded decoding warning
   Expectation is for -d -T0 we will not throw any warning,
   and see warning for any other -d -T(>1) inputs
2022-07-29 12:51:58 -07:00
Dirk Müller 7fbe60d577 Split help in long and short version, cleanup formatting
Adopt the more standard Usage: formatting style
List short and long options alongside where available
Print lists as a table
Use command style description
2022-03-29 12:57:47 +02:00
Nick TerrellandGitHub f229daaf42 Merge pull request #3052 from dirkmueller/gzip_keep
Keep original file if -c or --stdout is given
2022-03-28 10:35:21 -07:00
Dirk Müller 7a3997c21a Handle newer less versions in zstdless testing
Newer less versions appear to have changed how stderr
and stdout are showing error messages. hardcode the
expected behavior to make the tests pass with any less version.

Also set locale to C so that the strings are matching.
2022-03-10 09:47:33 +01:00
Dirk Müller 3f4f8b04ed Keep original file if -c or --stdout is given
Set removeSrcFile back to false when -c or --stdout is used to improve
compatibility with gzip(1) behavior.

gzip(1) is removing the original file on compression unless --stdout or
/-c is used. zstd is defaulting to keep the file unless --rm is used or
when it is called via a gzip symlink, in which it is removing by
default. Specifying -c/--stdout turns this behavior off.
2022-03-09 23:38:59 +01:00
Felix HandteandGitHub 8814aa5bfa Merge pull request #3059 from dirkmueller/gzip_fast_best
Implement more gzip compatibility (#3037)
2022-03-08 15:32:06 -05:00
Nick Terrell da737c7ab8 [programs] Fix infinite loop when empty input is passed to trainer
When an empty input file was passed to the dictionary trainer, it would infinite loop.
The added test case exposes the bug, and is fixed with this PR.
2022-03-02 11:05:26 -08:00
Dirk Müller e653e97f77 Implement more gzip compatibility (#3037)
-n --no-name is the current behavior already, so we can implement
this as a noop.

--best is an alias for -9 in gzip

add basic cli tests.
2022-02-09 23:09:18 +01:00
936ae8a3a0 Move zstdgrep and zstdless tests to cli-tests (#3057)
* Move zstdgrep and zstdless tests to cli-tests

Co-authored-by: Binh Vo <binhvo@fb.com>
2022-02-09 11:21:33 -05:00
Nick Terrell 169f8c11ff [cli-tests] Fix zstd symlinks
The zstd symlinks, notably `zstdcat`, weren't working as expected
because only the `tests/cli-tests/bin/zstd` wrapper was symlinked. We
still invoked `zstd` with the name `zstd`. The fix is to create a
directory of zstd symlinks in `tests/cli-tests/bin/symlinks` for each
name that zstd recognizes. And when `tets/cli-tests/bin/zstd` is
invoked, it selects the correct symlink to call.

See the test `zstd-cli/zstdcat.sh` for an example of how it would work.
2022-02-07 15:20:42 -08:00
Yann ColletandGitHub be0a4b4df6 Merge pull request #3046 from rex4539/typos
Fix typos
2022-02-04 07:50:25 -08:00
Yonatan KomornikandGitHub 4bba97b4cb Macos playtest envvars fix (#3035)
* playtests.sh: fix for a bug in macos' /bin/sh that persists temporary env vars when introduced before function calls
* cli-tests/run.py: Do not use existing ZSTD* envvars
2022-02-03 18:42:20 -08:00
Dimitris Apostolou 4c4d403ecb Fix typos 2022-02-02 19:32:31 +02:00
Nick Terrell 1fc42de86a [CI] Hook cli-tests up to CI
Add cli-tests to `make test`. This adds a `python3` dependency to `make
test`, but not `make check`. We could make this dependency optional by
skipping the tests if `python3` is not present.
2022-01-27 13:56:59 -08:00
Nick Terrell f3096ff6d1 [test] Add new CLI testing platform
Adds the new CLI testing platform that I'm proposing.
See the added `README.md` for details.
2022-01-27 13:56:59 -08:00