Commit Graph
1590 Commits
Author SHA1 Message Date
Nigel Tao 85be5b9f8a Delete the obsolete lzw.decoder.suffixes array
This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

Avoiding the zero-initialization of the 32768-byte array avoids a fixed
cost, regardless of e.g. the size of the decoded GIF image. A fixed
absolute improvement is a bigger relative improvement for smaller GIFs.

Compared to the previous commit:

name                                     old speed      new speed      delta

wuffs_lzw_decode_20k/clang5               241MB/s ± 0%   247MB/s ± 0%   +2.71%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang5              401MB/s ± 1%   407MB/s ± 0%   +1.42%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc7                 235MB/s ± 0%   238MB/s ± 1%   +1.21%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc7                382MB/s ± 0%   384MB/s ± 0%     ~     (p=0.095 n=5+5)

wuffs_gif_decode_1k_bw/clang5             235MB/s ± 1%   318MB/s ± 0%  +35.34%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang5          132MB/s ± 0%   154MB/s ± 0%  +16.47%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang5          678MB/s ± 0%   697MB/s ± 1%   +2.75%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang5       181MB/s ± 0%   186MB/s ± 0%   +2.65%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang5               226MB/s ± 0%   226MB/s ± 1%     ~     (p=0.690 n=5+5)
wuffs_gif_decode_100k_artificial/clang5   506MB/s ± 0%   508MB/s ± 1%     ~     (p=0.222 n=5+5)
wuffs_gif_decode_100k_realistic/clang5    203MB/s ± 1%   203MB/s ± 0%     ~     (p=0.310 n=5+5)
wuffs_gif_decode_1000k/clang5             206MB/s ± 0%   206MB/s ± 0%   -0.26%  (p=0.032 n=5+5)
wuffs_gif_decode_anim_screencap/clang5   1.04GB/s ± 1%  1.03GB/s ± 0%     ~     (p=0.151 n=5+5)

wuffs_gif_decode_1k_bw/gcc7               255MB/s ± 1%   398MB/s ± 0%  +56.43%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc7            137MB/s ± 1%   161MB/s ± 0%  +17.63%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc7            593MB/s ± 0%   604MB/s ± 1%   +1.83%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc7         182MB/s ± 2%   187MB/s ± 1%   +2.85%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc7                 226MB/s ± 0%   227MB/s ± 0%   +0.56%  (p=0.032 n=5+5)
wuffs_gif_decode_100k_artificial/gcc7     488MB/s ± 0%   492MB/s ± 0%   +0.80%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc7      203MB/s ± 0%   201MB/s ± 0%   -1.29%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc7               207MB/s ± 0%   203MB/s ± 0%   -2.17%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc7     1.03GB/s ± 0%  1.04GB/s ± 0%   +0.67%  (p=0.008 n=5+5)
2019-02-03 15:02:16 +11:00
Nigel Tao f877fb2c59 Use the workbuf instead of lzw.decoder.suffixes
This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

name                                     old speed      new speed      delta

wuffs_lzw_decode_20k/clang5               255MB/s ± 0%   241MB/s ± 0%   -5.63%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang5              418MB/s ± 1%   401MB/s ± 1%   -4.02%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc7                 255MB/s ± 1%   235MB/s ± 0%   -7.88%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc7                444MB/s ± 1%   382MB/s ± 0%  -13.90%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang5             239MB/s ± 2%   235MB/s ± 1%   -1.89%  (p=0.016 n=5+5)
wuffs_gif_decode_1k_color/clang5          135MB/s ± 0%   132MB/s ± 0%   -1.86%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang5          692MB/s ± 1%   678MB/s ± 0%   -2.08%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang5       186MB/s ± 0%   181MB/s ± 0%   -2.58%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang5               237MB/s ± 1%   226MB/s ± 0%   -4.46%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang5   536MB/s ± 0%   506MB/s ± 0%   -5.53%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang5    219MB/s ± 0%   203MB/s ± 1%   -7.11%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang5             222MB/s ± 0%   206MB/s ± 0%   -7.25%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang5   1.07GB/s ± 0%  1.04GB/s ± 1%   -3.11%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc7               262MB/s ± 0%   255MB/s ± 1%   -2.63%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc7            142MB/s ± 1%   137MB/s ± 1%   -3.38%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc7            596MB/s ± 1%   593MB/s ± 0%     ~     (p=0.151 n=5+5)
wuffs_gif_decode_10k_indexed/gcc7         185MB/s ± 0%   182MB/s ± 2%   -1.63%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc7                 230MB/s ± 0%   226MB/s ± 0%   -1.78%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc7     490MB/s ± 1%   488MB/s ± 0%     ~     (p=0.095 n=5+5)
wuffs_gif_decode_100k_realistic/gcc7      203MB/s ± 1%   203MB/s ± 0%     ~     (p=0.421 n=5+5)
wuffs_gif_decode_1000k/gcc7               207MB/s ± 0%   207MB/s ± 0%     ~     (p=1.000 n=5+5)
wuffs_gif_decode_anim_screencap/gcc7     1.04GB/s ± 0%  1.03GB/s ± 0%   -0.83%  (p=0.008 n=5+5)
2019-02-03 14:57:34 +11:00
Nigel Tao d1f1340d47 Add a nothing built-in literal 2019-02-03 14:01:19 +11:00
Nigel Tao 49627b46f2 Flatten the lzw.decoder.suffixes array
This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

name                                     old speed      new speed      delta

wuffs_lzw_decode_20k/clang5               265MB/s ± 0%   255MB/s ± 0%  -3.85%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang5              452MB/s ± 1%   418MB/s ± 1%  -7.62%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc7                 263MB/s ± 1%   255MB/s ± 1%  -3.21%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc7                482MB/s ± 2%   444MB/s ± 1%  -7.85%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang5             241MB/s ± 1%   239MB/s ± 2%    ~     (p=0.310 n=5+5)
wuffs_gif_decode_1k_color/clang5          141MB/s ± 1%   135MB/s ± 0%  -4.30%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang5          721MB/s ± 0%   692MB/s ± 1%  -4.00%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang5       194MB/s ± 1%   186MB/s ± 0%  -4.03%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang5               246MB/s ± 1%   237MB/s ± 1%  -3.60%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang5   551MB/s ± 0%   536MB/s ± 0%  -2.73%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang5    227MB/s ± 0%   219MB/s ± 0%  -3.74%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang5             231MB/s ± 0%   222MB/s ± 0%  -3.88%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang5   1.10GB/s ± 0%  1.07GB/s ± 0%  -2.35%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc7               265MB/s ± 1%   262MB/s ± 0%  -1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc7            145MB/s ± 1%   142MB/s ± 1%  -2.52%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc7            622MB/s ± 1%   596MB/s ± 1%  -4.18%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc7         194MB/s ± 0%   185MB/s ± 0%  -4.81%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc7                 234MB/s ± 1%   230MB/s ± 0%  -1.80%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc7     506MB/s ± 1%   490MB/s ± 1%  -3.14%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc7      210MB/s ± 1%   203MB/s ± 1%  -2.89%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc7               213MB/s ± 0%   207MB/s ± 0%  -2.66%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc7     1.06GB/s ± 0%  1.04GB/s ± 0%  -2.38%  (p=0.008 n=5+5)
2019-02-03 13:39:43 +11:00
Nigel Tao 6729715c48 Work around memfd_create linker issues 2019-02-02 17:09:14 +11:00
Nigel Tao 54a112a130 Skip generating wuffs_base__utility typed fields 2019-02-02 16:47:04 +11:00
Nigel Tao dcf91f6c7e Move wuffs_base__utility__null_slice_u8 to private 2019-02-02 16:44:41 +11:00
Nigel Tao 852a979d2f Remove the cgen receiver for base.utility methods 2019-02-02 16:43:07 +11:00
Nigel Tao b9cbce9779 Add decoder.workbuf_len methods 2019-02-02 16:34:40 +11:00
Nigel Tao de5b64a846 Rename max_incl_workbuf_len constants 2019-02-02 16:29:54 +11:00
Nigel Tao 5f7c807729 Generate macros for public scalar consts 2019-02-02 16:17:10 +11:00
Nigel Tao 4af1b88ef8 Make cgen's public constants' always static 2019-02-02 16:02:38 +11:00
Nigel Tao e45baffd51 Trim example/library's buffer size 2019-02-02 15:56:39 +11:00
Nigel Tao 022d221266 Tweak cgen's constants' format 2019-02-02 15:55:25 +11:00
Nigel Tao be2d96bb27 Add public max_incl_workbuf_len constants 2019-02-02 15:51:04 +11:00
Nigel Tao c9830ceb9a Add a workbuf argument to decode_io_writer 2019-02-02 15:35:05 +11:00
Nigel Tao 91f7f4b70a Avoid -Wunused-function warnings with clang 2019-02-02 14:15:52 +11:00
Nigel Tao 70958ea914 Avoid "defined but not used" for consts 2019-02-02 13:34:50 +11:00
Nigel Tao 11a6819be9 Add a deflate distance-code-31 test 2019-02-02 13:04:09 +11:00
Nigel Tao e9709992e2 Have the fuzzers check for internal errors 2019-02-02 12:08:44 +11:00
Nigel Tao 92f3958a28 Reduce the deflate decoder Huffman table size
name                                             old speed     new speed     delta

wuffs_deflate_decode_1k/clang5                   138MB/s ± 0%  140MB/s ± 1%  +1.16%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang5                  228MB/s ± 1%  229MB/s ± 1%    ~     (p=0.222 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang5   267MB/s ± 1%  267MB/s ± 1%    ~     (p=0.690 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang5  225MB/s ± 0%  225MB/s ± 0%    ~     (p=0.310 n=5+5)

wuffs_deflate_decode_1k/gcc7                     157MB/s ± 0%  157MB/s ± 0%    ~     (p=0.730 n=4+5)
wuffs_deflate_decode_10k/gcc7                    269MB/s ± 0%  269MB/s ± 1%    ~     (p=0.222 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc7     321MB/s ± 0%  321MB/s ± 1%    ~     (p=0.548 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc7    254MB/s ± 0%  254MB/s ± 0%    ~     (p=0.421 n=5+5)

wuffs_gzip_decode_10k/clang5                     205MB/s ± 0%  212MB/s ± 0%  +3.50%  (p=0.016 n=5+4)
wuffs_gzip_decode_100k/clang5                    238MB/s ± 0%  245MB/s ± 0%  +3.31%  (p=0.008 n=5+5)

wuffs_gzip_decode_10k/gcc7                       247MB/s ± 0%  248MB/s ± 0%  +0.47%  (p=0.016 n=5+5)
wuffs_gzip_decode_100k/gcc7                      293MB/s ± 1%  293MB/s ± 0%    ~     (p=0.690 n=5+5)

wuffs_zlib_decode_10k/clang5                     210MB/s ± 0%  210MB/s ± 0%    ~     (p=0.056 n=5+5)
wuffs_zlib_decode_100k/clang5                    244MB/s ± 0%  244MB/s ± 0%    ~     (p=0.556 n=4+5)

wuffs_zlib_decode_10k/gcc7                       247MB/s ± 0%  249MB/s ± 1%  +0.77%  (p=0.008 n=5+5)
wuffs_zlib_decode_100k/gcc7                      288MB/s ± 0%  288MB/s ± 0%    ~     (p=1.000 n=5+5)
2019-02-02 08:46:40 +11:00
Nigel Tao 670b45be53 Add some "horizontal rule" comment lines 2019-02-02 08:10:08 +11:00
Nigel Tao fc66207b59 Add script/mmap-ring-buffer.c 2019-02-02 08:01:01 +11:00
Nigel Tao 8bbdcc4ce1 Allow scalar constants in type expressions 2019-02-02 07:53:59 +11:00
Nigel Tao 278200f560 Fix example for initialize method name change 2019-01-26 16:59:01 +11:00
Nigel Tao 705a84723a For "x shift y", check that y < (x's bit-size) 2019-01-26 16:51:49 +11:00
Nigel Tao 65c2f4971f Rename some initialize methods to prepare or set 2019-01-26 15:46:42 +11:00
Nigel Tao ecccafab09 Add check for interleaved coroutines
It's not an optimization per se, but it seems to help:

name                                             old speed      new speed      delta

wuffs_deflate_decode_1k/clang5                    138MB/s ± 1%   150MB/s ± 1%   +8.96%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang5                   223MB/s ± 0%   243MB/s ± 0%   +9.25%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang5    263MB/s ± 2%   289MB/s ± 0%  +10.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang5   220MB/s ± 1%   243MB/s ± 0%  +10.42%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k/gcc7                      158MB/s ± 0%   157MB/s ± 0%     ~     (p=0.056 n=5+5)
wuffs_deflate_decode_10k/gcc7                     268MB/s ± 0%   268MB/s ± 0%     ~     (p=1.000 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc7      312MB/s ± 0%   320MB/s ± 0%   +2.62%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc7     254MB/s ± 0%   253MB/s ± 0%   -0.48%  (p=0.016 n=5+5)
2019-01-26 15:33:43 +11:00
Nigel Tao d076b6e800 Remove hand-written decode_huffman_fast type pun
Type punning is undefined behavior.
2019-01-26 10:48:14 +11:00
Nigel Tao b31ddaa052 Make hand-written decode_huffman_fast standalone 2019-01-26 10:19:53 +11:00
Nigel Tao a27b24099e Raise example/zcat's buffer size from 16K to 128K
This matches the 128K buffer size (ZSTD_DStreamOutSize returns
ZSTD_BLOCKSIZE_MAX, defined to be 1<<17 in
https://github.com/facebook/zstd/blob/dev/lib/zstd.h) as shipped as
/usr/bin/zstdcat on Debian.

Decoding a 156 MiB gzipped file:
$ /usr/bin/time ./gen/bin/example-zcat < linux-4.20.4.tar.gz > /dev/null

Before:
2.14user 0.13system 0:02.28elapsed 100%CPU (0avgtext+0avgdata 1076maxresident)k
0inputs+0outputs (0major+75minor)pagefaults 0swaps

After:
2.03user 0.05system 0:02.09elapsed 99%CPU (0avgtext+0avgdata 1460maxresident)k
0inputs+0outputs (0major+135minor)pagefaults 0swaps

Before/After elapsed ratio: 2.28 / 2.09 = 1.09.
2019-01-26 09:32:55 +11:00
Nigel Tao 89fef4cf5f Align cgen's private_impl.placeholder 2019-01-26 09:28:50 +11:00
Nigel Tao f4c5d953ec Have build-all.sh check as a .h file, not just .c 2019-01-26 09:21:18 +11:00
Nigel Tao fda3c4c986 wuffs gen -version=0.2.0-alpha.29 2019-01-19 15:00:39 +11:00
Nigel Tao dec519d9b4 Remove image_config.workbuf_len 2019-01-19 14:49:27 +11:00
Nigel Tao 989c611711 Allow C++ methods even when !WUFFS_IMPLEMENTATION
Also disallow copy and assign.
2019-01-19 14:38:19 +11:00
Nigel Tao f8a619f47d Update lang/check tests for coroutine_resumed 2019-01-19 12:36:27 +11:00
Nigel Tao 6326479e81 Have hand-written decode_huffman_fast copy history
This makes it pass "wuffs test -skipgen std/deflate".
2019-01-19 12:13:09 +11:00
Nigel Tao c6a48b3731 Call decode_huffman_fast more often
name                                             old speed     new speed     delta

wuffs_deflate_decode_1k/clang5                   137MB/s ± 0%  139MB/s ± 0%   +1.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang5                  219MB/s ± 0%  224MB/s ± 0%   +2.59%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang5   257MB/s ± 0%  265MB/s ± 0%   +3.41%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang5  181MB/s ± 0%  220MB/s ± 0%  +21.72%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k/gcc7                     157MB/s ± 0%  156MB/s ± 1%   -0.63%  (p=0.032 n=5+5)
wuffs_deflate_decode_10k/gcc7                    263MB/s ± 1%  269MB/s ± 0%   +2.39%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc7     310MB/s ± 1%  314MB/s ± 0%   +1.27%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc7    158MB/s ± 1%  254MB/s ± 0%  +61.29%  (p=0.008 n=5+5)
2019-01-19 10:50:40 +11:00
Nigel Tao 2fd03f42b5 Prepare to call decode_huffman_fast more often 2019-01-19 10:33:58 +11:00
Nigel Tao 1af543866c Add an implicit coroutine_resumed variable 2019-01-13 11:41:58 +11:00
Nigel Tao 89c136ebe4 Add token.ID.IsCannotAssignTo 2019-01-13 11:15:17 +11:00
Nigel Tao b18209c7dd Re-order some token.ID numbers 2019-01-13 11:11:39 +11:00
Nigel Tao 109d6f04bc Add std/deflate many_big_read benchmarks 2019-01-13 10:08:40 +11:00
Nigel Tao cf52415f9f Have hand-written decode_huffman_fast obey limits
This makes it pass "wuffs test -skipgen std/deflate".
2019-01-13 09:00:37 +11:00
Nigel Tao dea71ad324 Update wuffs-deflate-decoder-decode-huffman.c 2019-01-13 08:33:40 +11:00
Nigel Tao 031783f9c8 Add C compiler version to test and bench output 2019-01-12 14:52:30 +11:00
Nigel Tao e071e171cc Have bench-go-gif process multiple images 2019-01-12 14:38:17 +11:00
Nigel Tao c3a07db892 Add wuffs_gif_decode_20k benchmark
That GIF image's LZW-compressed data is the same as that used for the
wuffs_lzw_decode_20k benchmark. Comparing the two speeds gives some
indication of the relative overhead of the framing format, as well as
decoding to a richer data structure (2D table instead of a 1D array,
potentially to BGRA instead of palette-index, amongst other things).

name                        speed
wuffs_gif_decode_20k/clang  246MB/s ± 1%
wuffs_gif_decode_20k/gcc    234MB/s ± 1%
wuffs_lzw_decode_20k/clang  272MB/s ± 0%
wuffs_lzw_decode_20k/gcc    269MB/s ± 0%
2019-01-12 14:17:03 +11:00
Nigel Tao 419d403257 Build benchmarks with "-Wall -Werror" 2019-01-12 12:37:52 +11:00