From 222916a5d31385231f3a5a70c6da756a15d99e39 Mon Sep 17 00:00:00 2001 From: Nick Terrell Date: Fri, 30 Oct 2020 13:54:30 -0700 Subject: [PATCH 1/2] [regression] Update results.csv https://github.com/facebook/zstd/pull/2339 removes the single-pass zstdmt API. This changes the compressed size, because we no longer take the # of threads into account when deciding the job size. --- tests/regression/results.csv | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/regression/results.csv b/tests/regression/results.csv index 60d43b6ad..979b1d250 100644 --- a/tests/regression/results.csv +++ b/tests/regression/results.csv @@ -212,8 +212,8 @@ silesia.tar, level 16, advanced silesia.tar, level 19, advanced one pass, 4281605 silesia.tar, no source size, advanced one pass, 4861425 silesia.tar, long distance mode, advanced one pass, 4848098 -silesia.tar, multithreaded, advanced one pass, 4860781 -silesia.tar, multithreaded long distance mode, advanced one pass, 4847398 +silesia.tar, multithreaded, advanced one pass, 4861508 +silesia.tar, multithreaded long distance mode, advanced one pass, 4853186 silesia.tar, small window log, advanced one pass, 7101530 silesia.tar, small hash log, advanced one pass, 6587951 silesia.tar, small chain log, advanced one pass, 4943307 @@ -221,7 +221,7 @@ silesia.tar, explicit params, advanced silesia.tar, uncompressed literals, advanced one pass, 5129458 silesia.tar, uncompressed literals optimal, advanced one pass, 4320927 silesia.tar, huffman literals, advanced one pass, 5347335 -silesia.tar, multithreaded with advanced params, advanced one pass, 5129777 +silesia.tar, multithreaded with advanced params, advanced one pass, 5129555 github, level -5, advanced one pass, 205285 github, level -5 with dict, advanced one pass, 46718 github, level -3, advanced one pass, 190643 @@ -304,8 +304,8 @@ silesia.tar, level 16, advanced silesia.tar, level 19, advanced one pass small out, 4281605 silesia.tar, no source size, advanced one pass small out, 4861425 silesia.tar, long distance mode, advanced one pass small out, 4848098 -silesia.tar, multithreaded, advanced one pass small out, 4860781 -silesia.tar, multithreaded long distance mode, advanced one pass small out, 4847398 +silesia.tar, multithreaded, advanced one pass small out, 4861508 +silesia.tar, multithreaded long distance mode, advanced one pass small out, 4853186 silesia.tar, small window log, advanced one pass small out, 7101530 silesia.tar, small hash log, advanced one pass small out, 6587951 silesia.tar, small chain log, advanced one pass small out, 4943307 @@ -313,7 +313,7 @@ silesia.tar, explicit params, advanced silesia.tar, uncompressed literals, advanced one pass small out, 5129458 silesia.tar, uncompressed literals optimal, advanced one pass small out, 4320927 silesia.tar, huffman literals, advanced one pass small out, 5347335 -silesia.tar, multithreaded with advanced params, advanced one pass small out, 5129777 +silesia.tar, multithreaded with advanced params, advanced one pass small out, 5129555 github, level -5, advanced one pass small out, 205285 github, level -5 with dict, advanced one pass small out, 46718 github, level -3, advanced one pass small out, 190643 From a446fa33dcd29f582a2292766d9b1c7ea79a9143 Mon Sep 17 00:00:00 2001 From: Nick Terrell Date: Fri, 30 Oct 2020 13:55:52 -0700 Subject: [PATCH 2/2] [regression] Add README explaining the test --- tests/regression/README.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) create mode 100644 tests/regression/README.md diff --git a/tests/regression/README.md b/tests/regression/README.md new file mode 100644 index 000000000..bb36b1ddd --- /dev/null +++ b/tests/regression/README.md @@ -0,0 +1,28 @@ +# Regression tests + +The regression tests run zstd in many scenarios and ensures that the size of the compressed results doesn't change. This helps us ensure that we don't accidentally regress zstd's compression ratio. + +These tests get run every night by CircleCI. If the job fails you can read the diff printed by the job to ensure the change isn't a regression. If all is well you can download the `results.csv` artifact and commit the new results. Or you can rebuild it yourself following the instructions below. + +## Rebuilding results.csv + +From the root of the zstd repo run: + +``` +# Build the zstd binary +make clean +make -j zstd + +# Build the regression test binary +cd tests/regression +make clean +make -j test + +# Run the regression test +./test --cache data-cache --zstd ../../zstd --output results.csv + +# Check results.csv to ensure the new results are okay +git diff + +# Then submit the PR +```