Commit Graph
44 Commits
Author SHA1 Message Date
Nigel Tao 314b1209ed Let std/lzw mix automatic and manual flushes 2018-12-29 14:23:27 +11:00
Nigel Tao 093268e179 Rename the !?? effect to ? 2018-12-29 13:57:57 +11:00
Nigel Tao d26fa09d07 Have wuffsfmt align the "T"s in "x T" fields 2018-12-27 15:09:52 +11:00
Nigel Tao 1ceb304ad9 Have wuffsfmt align the "T"s in "var x T" blocks 2018-12-27 14:19:20 +11:00
Nigel Tao 9bcdd1e028 Place std/{gif,lzw} vars at the top of functions 2018-12-26 19:35:28 +11:00
Nigel Tao 7405307173 Propagate tighter bounds checks after assignments 2018-12-26 16:52:55 +11:00
Nigel Tao 00c5f77c19 Rename lzw's flush_j field 2018-12-26 11:15:22 +11:00
Nigel Tao c2826e87e3 Factor lzw's decode into read and write halves
name                                    old speed      new speed      delta

wuffs_lzw_decode_20k/clang               203MB/s ± 3%   240MB/s ± 1%  +17.93%  (p=0.000 n=49+5)
wuffs_lzw_decode_100k/clang              334MB/s ± 4%   413MB/s ± 0%  +23.75%  (p=0.000 n=50+5)

wuffs_lzw_decode_20k/gcc                 235MB/s ± 2%   254MB/s ± 2%   +8.40%  (p=0.000 n=45+5)
wuffs_lzw_decode_100k/gcc                395MB/s ± 2%   455MB/s ± 2%  +15.06%  (p=0.000 n=45+5)

wuffs_gif_decode_1k_bw/clang             229MB/s ± 1%   240MB/s ± 0%   +4.51%  (p=0.000 n=49+5)
wuffs_gif_decode_1k_color/clang          120MB/s ± 1%   134MB/s ± 1%  +11.44%  (p=0.000 n=46+5)
wuffs_gif_decode_10k_bgra/clang          557MB/s ± 1%   649MB/s ± 0%  +16.56%  (p=0.000 n=47+5)
wuffs_gif_decode_10k_indexed/clang       149MB/s ± 1%   174MB/s ± 1%  +17.11%  (p=0.000 n=48+5)
wuffs_gif_decode_100k_artificial/clang   459MB/s ± 1%   514MB/s ± 1%  +11.95%  (p=0.000 n=47+5)
wuffs_gif_decode_100k_realistic/clang    175MB/s ± 1%   204MB/s ± 0%  +16.80%  (p=0.000 n=47+5)
wuffs_gif_decode_1000k/clang             178MB/s ± 1%   207MB/s ± 0%  +16.21%  (p=0.000 n=46+5)
wuffs_gif_decode_anim_screencap/clang    961MB/s ± 1%  1060MB/s ± 1%  +10.35%  (p=0.000 n=46+5)

wuffs_gif_decode_1k_bw/gcc               244MB/s ± 1%   261MB/s ± 0%   +6.86%  (p=0.000 n=47+5)
wuffs_gif_decode_1k_color/gcc            133MB/s ± 1%   145MB/s ± 0%   +9.44%  (p=0.000 n=48+5)
wuffs_gif_decode_10k_bgra/gcc            543MB/s ± 2%   593MB/s ± 1%   +9.22%  (p=0.000 n=49+5)
wuffs_gif_decode_10k_indexed/gcc         166MB/s ± 1%   185MB/s ± 0%  +11.55%  (p=0.000 n=47+5)
wuffs_gif_decode_100k_artificial/gcc     464MB/s ± 1%   512MB/s ± 0%  +10.51%  (p=0.000 n=47+5)
wuffs_gif_decode_100k_realistic/gcc      190MB/s ± 1%   211MB/s ± 0%  +11.05%  (p=0.000 n=47+5)
wuffs_gif_decode_1000k/gcc               193MB/s ± 1%   215MB/s ± 0%  +11.54%  (p=0.000 n=44+5)
wuffs_gif_decode_anim_screencap/gcc     1.00GB/s ± 1%  1.07GB/s ± 0%   +7.76%  (p=0.000 n=45+5)
2018-12-26 11:08:37 +11:00
Nigel Tao 2f4c931b42 Use peek_uxx_as_uyy more often 2018-12-23 09:12:56 +11:00
Nigel Tao 179f7afb9b Have lzw rewind in and flush out on a bad code
name                                    old speed      new speed      delta

wuffs_lzw_decode_20k/clang               214MB/s ± 0%   204MB/s ± 2%  -4.62%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang              369MB/s ± 0%   337MB/s ± 2%  -8.72%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                 232MB/s ± 1%   236MB/s ± 0%  +1.99%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc                384MB/s ± 1%   396MB/s ± 1%  +3.00%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang             236MB/s ± 0%   230MB/s ± 0%  -2.67%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang          118MB/s ± 1%   120MB/s ± 0%  +1.10%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang          562MB/s ± 0%   562MB/s ± 0%    ~     (p=0.690 n=5+5)
wuffs_gif_decode_10k_indexed/clang       149MB/s ± 0%   149MB/s ± 0%    ~     (p=0.421 n=5+5)
wuffs_gif_decode_100k_artificial/clang   487MB/s ± 0%   459MB/s ± 1%  -5.63%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang    183MB/s ± 0%   174MB/s ± 0%  -4.52%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang             185MB/s ± 0%   177MB/s ± 1%  -4.61%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang   1.02GB/s ± 0%  0.97GB/s ± 0%  -5.30%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc               250MB/s ± 0%   249MB/s ± 0%  -0.63%  (p=0.016 n=5+5)
wuffs_gif_decode_1k_color/gcc            133MB/s ± 0%   133MB/s ± 1%    ~     (p=0.151 n=5+5)
wuffs_gif_decode_10k_bgra/gcc            543MB/s ± 1%   543MB/s ± 0%    ~     (p=0.548 n=5+5)
wuffs_gif_decode_10k_indexed/gcc         167MB/s ± 1%   166MB/s ± 1%  -0.83%  (p=0.032 n=5+5)
wuffs_gif_decode_100k_artificial/gcc     465MB/s ± 1%   465MB/s ± 1%    ~     (p=0.841 n=5+5)
wuffs_gif_decode_100k_realistic/gcc      190MB/s ± 0%   190MB/s ± 1%    ~     (p=0.222 n=5+5)
wuffs_gif_decode_1000k/gcc               192MB/s ± 1%   193MB/s ± 0%    ~     (p=0.421 n=5+5)
wuffs_gif_decode_anim_screencap/gcc     1.00GB/s ± 1%  1.00GB/s ± 1%    ~     (p=0.841 n=5+5)
2018-12-22 14:45:42 +11:00
Nigel Tao 010df42d75 Have std/lzw always rewind excess bits
I'm not sure of the underlying reason for the lzw/gcc benchmark anomaly
(the gif/gcc numbers don't change), but it's the same mystery as commit
ecb3995 `Tweak how std/lzw yields "short read"`.

name                                    old speed      new speed      delta

wuffs_lzw_decode_20k/clang               223MB/s ± 0%   214MB/s ± 0%   -3.96%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang              385MB/s ± 0%   369MB/s ± 0%   -4.15%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                 186MB/s ± 1%   232MB/s ± 1%  +24.41%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc                224MB/s ± 1%   384MB/s ± 1%  +71.81%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang             238MB/s ± 1%   236MB/s ± 0%     ~     (p=0.222 n=5+5)
wuffs_gif_decode_1k_color/clang          124MB/s ± 0%   118MB/s ± 1%   -4.75%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang          596MB/s ± 0%   562MB/s ± 0%   -5.71%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang       159MB/s ± 0%   149MB/s ± 0%   -6.42%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang   492MB/s ± 1%   487MB/s ± 0%   -0.98%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang    192MB/s ± 0%   183MB/s ± 0%   -5.07%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang             195MB/s ± 0%   185MB/s ± 0%   -4.93%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang   1.01GB/s ± 1%  1.02GB/s ± 0%   +1.25%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc               247MB/s ± 0%   250MB/s ± 0%   +1.26%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc            131MB/s ± 2%   133MB/s ± 0%     ~     (p=0.151 n=5+5)
wuffs_gif_decode_10k_bgra/gcc            541MB/s ± 1%   543MB/s ± 1%     ~     (p=0.548 n=5+5)
wuffs_gif_decode_10k_indexed/gcc         167MB/s ± 0%   167MB/s ± 1%     ~     (p=0.730 n=4+5)
wuffs_gif_decode_100k_artificial/gcc     469MB/s ± 0%   465MB/s ± 1%   -0.78%  (p=0.016 n=4+5)
wuffs_gif_decode_100k_realistic/gcc      190MB/s ± 0%   190MB/s ± 0%   -0.34%  (p=0.032 n=4+5)
wuffs_gif_decode_1000k/gcc               193MB/s ± 1%   192MB/s ± 1%     ~     (p=0.556 n=4+5)
wuffs_gif_decode_anim_screencap/gcc      995MB/s ± 1%   995MB/s ± 1%     ~     (p=0.841 n=5+5)
2018-12-22 14:05:34 +11:00
Nigel Tao 9311f7a694 Rename some std/lzw methods 2018-12-16 10:23:53 +11:00
Nigel Tao 7a02605144 Remove the std/lzw decoder.available method 2018-12-16 10:21:55 +11:00
Nigel Tao 622a76d4c0 Avoid buffer copies from LZW to GIF
name                                    old speed     new speed      delta

wuffs_lzw_decode_20k/clang              213MB/s ± 0%   224MB/s ± 1%   +4.89%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang             359MB/s ± 1%   393MB/s ± 2%   +9.36%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                188MB/s ± 0%   187MB/s ± 0%   -0.70%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc               223MB/s ± 2%   224MB/s ± 0%     ~     (p=0.690 n=5+5)

wuffs_gif_decode_1k_bw/clang            227MB/s ± 1%   240MB/s ± 0%   +5.89%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang         117MB/s ± 0%   125MB/s ± 0%   +6.16%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang         567MB/s ± 1%   599MB/s ± 0%   +5.56%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang      152MB/s ± 0%   159MB/s ± 1%   +4.76%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang  455MB/s ± 1%   504MB/s ± 0%  +10.93%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang   178MB/s ± 2%   194MB/s ± 0%   +9.00%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang            179MB/s ± 1%   197MB/s ± 0%   +9.81%  (p=0.016 n=5+4)
wuffs_gif_decode_anim_screencap/clang   952MB/s ± 0%  1050MB/s ± 1%  +10.29%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc              243MB/s ± 1%   252MB/s ± 0%   +3.71%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc           130MB/s ± 0%   133MB/s ± 1%   +2.69%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc           537MB/s ± 0%   547MB/s ± 1%   +1.93%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc        163MB/s ± 0%   167MB/s ± 1%   +2.40%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc    458MB/s ± 0%   471MB/s ± 0%   +2.90%  (p=0.016 n=5+4)
wuffs_gif_decode_100k_realistic/gcc     186MB/s ± 0%   191MB/s ± 0%   +3.08%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc              189MB/s ± 0%   194MB/s ± 0%   +2.70%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc     976MB/s ± 0%  1010MB/s ± 0%   +3.49%  (p=0.008 n=5+5)
2018-12-16 09:57:15 +11:00
Nigel Tao 5aede827ad Roll back Tweak how std/lzw yields "short read"
See the previous commit for the rationale.
2018-12-16 08:23:34 +11:00
Nigel Tao ecb399508c Tweak how std/lzw yields "short read"
This fixes a std/lzw performance regression (for gcc only, not clang),
introduced in 33a4557 "Collect GIF's LZW data into larger batches".

This commit restores the std/lzw performance, but it also makes the
std/gif benchmarks worse. As LZW is not really used outside of GIF, this
commit will immediately be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

name                                    old speed     new speed     delta

wuffs_lzw_decode_20k/clang              213MB/s ± 1%  205MB/s ± 1%   -3.90%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang             359MB/s ± 1%  351MB/s ± 0%   -2.14%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                189MB/s ± 0%  229MB/s ± 0%  +21.37%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc               223MB/s ± 0%  390MB/s ± 1%  +74.77%  (p=0.016 n=4+5)

wuffs_gif_decode_1k_bw/clang            225MB/s ± 1%  229MB/s ± 0%   +1.52%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang         118MB/s ± 1%  118MB/s ± 1%     ~     (p=0.841 n=5+5)
wuffs_gif_decode_10k_bgra/clang         569MB/s ± 0%  565MB/s ± 0%   -0.66%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang      152MB/s ± 1%  150MB/s ± 1%   -1.25%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang  454MB/s ± 1%  459MB/s ± 0%   +1.05%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang   179MB/s ± 0%  178MB/s ± 1%     ~     (p=0.222 n=5+5)
wuffs_gif_decode_1000k/clang            180MB/s ± 1%  181MB/s ± 0%   +0.79%  (p=0.016 n=5+5)
wuffs_gif_decode_anim_screencap/clang   951MB/s ± 0%  958MB/s ± 0%   +0.66%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc              238MB/s ± 0%  240MB/s ± 0%   +0.91%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc           130MB/s ± 1%  127MB/s ± 0%   -2.25%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc           537MB/s ± 1%  518MB/s ± 2%   -3.59%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc        164MB/s ± 1%  157MB/s ± 1%   -4.43%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc    454MB/s ± 2%  447MB/s ± 0%     ~     (p=0.095 n=5+5)
wuffs_gif_decode_100k_realistic/gcc     186MB/s ± 1%  179MB/s ± 1%   -3.28%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc              188MB/s ± 1%  182MB/s ± 0%   -3.06%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc     971MB/s ± 1%  941MB/s ± 0%   -3.11%  (p=0.008 n=5+5)
2018-12-16 08:17:29 +11:00
Nigel Tao 33a4557d98 Collect GIF's LZW data into larger batches
name                                    old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang            224MB/s ± 1%  227MB/s ± 0%  +0.98%  (p=0.000 n=10+9)
wuffs_gif_decode_1k_color/clang         116MB/s ± 1%  118MB/s ± 1%  +2.26%  (p=0.000 n=10+10)
wuffs_gif_decode_10k_bgra/clang         541MB/s ± 0%  568MB/s ± 1%  +4.97%  (p=0.000 n=9+10)
wuffs_gif_decode_10k_indexed/clang      144MB/s ± 0%  152MB/s ± 0%  +5.61%  (p=0.000 n=9+10)
wuffs_gif_decode_100k_artificial/clang  438MB/s ± 1%  454MB/s ± 1%  +3.56%  (p=0.000 n=9+10)
wuffs_gif_decode_100k_realistic/clang   171MB/s ± 0%  179MB/s ± 0%  +4.64%  (p=0.000 n=8+10)
wuffs_gif_decode_1000k/clang            174MB/s ± 0%  180MB/s ± 0%  +3.90%  (p=0.000 n=9+10)
wuffs_gif_decode_anim_screencap/clang   897MB/s ± 1%  943MB/s ± 0%  +5.10%  (p=0.000 n=9+9)

wuffs_gif_decode_1k_bw/gcc              250MB/s ± 1%  241MB/s ± 1%  -3.26%  (p=0.000 n=10+10)
wuffs_gif_decode_1k_color/gcc           132MB/s ± 1%  130MB/s ± 1%  -2.15%  (p=0.000 n=10+10)
wuffs_gif_decode_10k_bgra/gcc           534MB/s ± 1%  535MB/s ± 1%    ~     (p=0.211 n=9+10)
wuffs_gif_decode_10k_indexed/gcc        162MB/s ± 1%  163MB/s ± 0%    ~     (p=0.105 n=10+10)
wuffs_gif_decode_100k_artificial/gcc    460MB/s ± 0%  456MB/s ± 0%  -0.80%  (p=0.000 n=10+10)
wuffs_gif_decode_100k_realistic/gcc     186MB/s ± 0%  186MB/s ± 1%  -0.33%  (p=0.001 n=9+10)
wuffs_gif_decode_1000k/gcc              189MB/s ± 1%  188MB/s ± 1%    ~     (p=0.053 n=9+10)
wuffs_gif_decode_anim_screencap/gcc     981MB/s ± 1%  971MB/s ± 1%  -0.95%  (p=0.000 n=10+10)
2018-12-15 14:10:10 +11:00
Nigel Tao 762b320b5e Make the return value mandatory; add ok 2018-11-24 15:09:28 +11:00
Nigel Tao 9e5aec5d25 Change "$foo" to base."$foo" 2018-11-24 13:36:09 +11:00
Nigel Tao 875362ea07 Change status "?foo" to just "?foo" 2018-11-24 12:36:42 +11:00
Nigel Tao f60723f624 Simplify cgen output for copy_from_slice 2018-10-21 18:03:54 +11:00
Nigel Tao 479326cbad Rename a LZW decoder field 2018-10-21 16:07:36 +11:00
Nigel Tao b0c72867e6 Update std/lzw/README.md for algorithmic detail 2018-10-21 16:03:47 +11:00
Nigel Tao 3338d6d1bc Write LZW output 8 bytes at a time, not 1
The benchmarks are mixed, but I think it's still a net win.

----
name                                    old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang            258MB/s ± 0%  253MB/s ± 1%    -2.04%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang         159MB/s ± 0%  116MB/s ± 0%   -26.76%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang              167MB/s ± 0%  138MB/s ± 0%   -17.42%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang  205MB/s ± 0%  439MB/s ± 0%  +114.53%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang   151MB/s ± 0%  164MB/s ± 0%    +8.41%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang            155MB/s ± 0%  165MB/s ± 0%    +6.50%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang   255MB/s ± 0%  921MB/s ± 0%  +261.50%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc              316MB/s ± 2%  267MB/s ± 1%   -15.78%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc           184MB/s ± 1%  134MB/s ± 0%   -27.57%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc                196MB/s ± 0%  161MB/s ± 0%   -17.50%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc    287MB/s ± 0%  449MB/s ± 1%   +56.22%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc     182MB/s ± 0%  182MB/s ± 1%      ~     (p=0.690 n=5+5)
wuffs_gif_decode_1000k/gcc              187MB/s ± 1%  185MB/s ± 1%      ~     (p=0.056 n=5+5)
wuffs_gif_decode_anim_screencap/gcc     397MB/s ± 0%  973MB/s ± 0%  +145.42%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang              170MB/s ± 0%  191MB/s ± 1%   +12.52%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang             242MB/s ± 0%  312MB/s ± 2%   +29.00%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                220MB/s ± 0%  227MB/s ± 0%    +3.31%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc               317MB/s ± 0%  369MB/s ± 1%   +16.53%  (p=0.008 n=5+5)
----

LZW is the compression method used by GIF images. Roughly speaking, LZW
is a sequence of codes, and each code emits either 1 byte (a literal
code) or multiple bytes (a copy code). The speed gain in this commit
comes from, when emitting an N-byte copy code, this now requires N/8
iterations instead of N iterations, as each iteration now writes 8 bytes
instead of 1. Literal codes, both before and after this commit, requires
a single iteration.

Decoding should therefore get faster or slower depending on whether it's
copy-code-heavy or literal-code-heavy respectively. For what it's worth,
copy-code-heavy GIF images also usually compress better.

Running script/extract-giflzw.go (or examining the cached output at
test/data/analysis/giflzw-emission-lengths.txt) on the anim_screencap
(which got faster) and 1k_color (which got slower) cases (also known as
gifplayer-muybridge.gif and hippopotamus.regular.gif) shows that the
proportion of literal codes agrees with that explanation.

----
gifplayer-muybridge.gif (all frames)
 Percent   Count  Emission_Length_Bucket
   12.78   34658           1 (Literal Code)
   14.05   38106           2
   13.25   35937           3
   10.59   28707           4
    7.85   21286           5
    6.08   16481           6
    4.85   13156           7
    3.60    9764           8
    9.34   25318    9 ..  16
    4.89   13257   17 ..  32
    4.78   12954   33 ..  64
    4.72   12806   65 .. 128
    3.02    8190  127 .. 256
    0.20     550  256+

hippopotamus.regular.gif (all frames)
 Percent   Count  Emission_Length_Bucket
   82.66     672           1 (Literal Code)
   13.04     106           2
    2.83      23           3
    0.86       7           4
    0.37       3           5
    0.25       2           6
    0.00       0           7
    0.00       0           8
    0.00       0    9 ..  16
    0.00       0   17 ..  32
    0.00       0   33 ..  64
    0.00       0   65 .. 128
    0.00       0  127 .. 256
    0.00       0  256+
----

The new algorithm also adds more CPU and RAM overhead, regardless of
whether the input GIF image's LZW data is copy-code-heavy or
literal-code-heavy. For example, the suffix table grows by 8x, from 4KiB
to 32 KiB.

The gifplayer-muybridge.gif image is also larger (472 x 298) and the
hippopotamus.regular.gif image is also smaller (36 x 28). Higher fixed
costs may have larger (negative) relative effects on decoding smaller
images.

To repeat, I think it's still a net win.

An upcoming commit will update the std/lzw/README.md file to provide
more detail on the algorithm.

Thanks to Tushar Khandelwal for the idea.
2018-10-21 14:48:40 +11:00
Nigel Tao 6945bdc75c Fill the LZW bit buffer less often
name                             old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang     292MB/s ± 1%  256MB/s ± 1%  -12.29%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang  143MB/s ± 0%  159MB/s ± 0%  +10.95%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang       150MB/s ± 1%  165MB/s ± 0%  +10.24%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/clang      152MB/s ± 0%  150MB/s ± 0%   -1.19%  (p=0.016 n=4+5)
wuffs_gif_decode_1000k/clang     156MB/s ± 1%  153MB/s ± 1%   -1.60%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc       250MB/s ± 1%  330MB/s ± 0%  +31.99%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc    172MB/s ± 1%  184MB/s ± 1%   +7.40%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc         189MB/s ± 0%  193MB/s ± 0%   +2.31%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/gcc        166MB/s ± 0%  180MB/s ± 0%   +8.56%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc       171MB/s ± 0%  186MB/s ± 0%   +8.19%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang       168MB/s ± 3%  168MB/s ± 0%     ~     (p=0.222 n=5+5)
wuffs_lzw_decode_100k/clang      252MB/s ± 1%  239MB/s ± 0%   -4.89%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc         209MB/s ± 0%  217MB/s ± 0%   +4.28%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc        307MB/s ± 0%  314MB/s ± 0%   +2.27%  (p=0.008 n=5+5)
2018-10-20 07:22:23 +11:00
Nigel Tao 24e7ee0104 Flush LZW output if stalled on input
name                             old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang     295MB/s ± 0%  292MB/s ± 1%   -0.89%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang  163MB/s ± 0%  143MB/s ± 0%  -12.15%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang       179MB/s ± 0%  150MB/s ± 1%  -16.37%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/clang      155MB/s ± 0%  152MB/s ± 0%   -1.84%  (p=0.016 n=5+4)
wuffs_gif_decode_1000k/clang     159MB/s ± 0%  156MB/s ± 1%   -2.26%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc       313MB/s ± 0%  250MB/s ± 1%  -20.20%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc    173MB/s ± 0%  172MB/s ± 1%   -1.01%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc         184MB/s ± 1%  189MB/s ± 0%   +2.34%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/gcc        173MB/s ± 1%  166MB/s ± 0%   -3.88%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc       179MB/s ± 1%  171MB/s ± 0%   -4.07%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang       185MB/s ± 0%  168MB/s ± 3%   -9.34%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang      260MB/s ± 0%  252MB/s ± 1%   -3.36%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc         211MB/s ± 0%  209MB/s ± 0%   -1.01%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc        300MB/s ± 0%  307MB/s ± 0%   +2.16%  (p=0.008 n=5+5)
2018-10-14 22:35:42 +11:00
Nigel Tao ff92d22ea6 Read LZW input bits 4 bytes at a time, not 1
name                             old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang     306MB/s ± 0%  295MB/s ± 0%   -3.62%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang  156MB/s ± 0%  163MB/s ± 0%   +4.64%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang       153MB/s ± 0%  179MB/s ± 0%  +17.34%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/clang      147MB/s ± 0%  155MB/s ± 0%   +5.13%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang     151MB/s ± 1%  159MB/s ± 0%   +5.34%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc       319MB/s ± 0%  313MB/s ± 0%   -1.79%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc    165MB/s ± 0%  173MB/s ± 0%   +4.82%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc         171MB/s ± 1%  184MB/s ± 1%   +7.48%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/gcc        161MB/s ± 1%  173MB/s ± 1%   +7.32%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc       166MB/s ± 0%  179MB/s ± 1%   +7.53%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang       167MB/s ± 0%  185MB/s ± 0%  +10.85%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang      239MB/s ± 0%  260MB/s ± 0%   +8.96%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc         192MB/s ± 0%  211MB/s ± 0%   +9.47%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc        295MB/s ± 0%  300MB/s ± 0%   +1.65%  (p=0.008 n=5+5)
2018-10-14 22:14:02 +11:00
Nigel Tao 4e27b1280e Write LZW decoder output in batches
name                             old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang     309MB/s ± 0%  306MB/s ± 0%   -0.93%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang  160MB/s ± 0%  156MB/s ± 0%   -2.26%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang       145MB/s ± 0%  153MB/s ± 0%   +5.21%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/clang      131MB/s ± 1%  147MB/s ± 0%  +12.15%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang     134MB/s ± 0%  151MB/s ± 1%  +12.83%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc       339MB/s ± 0%  319MB/s ± 0%   -5.99%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc    186MB/s ± 0%  165MB/s ± 0%  -10.93%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc         164MB/s ± 0%  171MB/s ± 1%   +4.19%  (p=0.008 n=5+5)
wuffs_gif_decode_100k/gcc        143MB/s ± 0%  161MB/s ± 1%  +12.32%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/gcc       146MB/s ± 0%  166MB/s ± 0%  +13.61%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang       142MB/s ± 0%  167MB/s ± 0%  +17.51%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang      186MB/s ± 1%  239MB/s ± 0%  +28.68%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc         162MB/s ± 0%  192MB/s ± 0%  +18.63%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc        235MB/s ± 1%  295MB/s ± 0%  +25.59%  (p=0.008 n=5+5)
2018-10-14 17:53:20 +11:00
Nigel Tao cae1bfbc7d Remove LZW "cyclical prefix chain" internal error
name                             old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang     289MB/s ± 1%  309MB/s ± 0%   +6.94%  (p=0.000 n=50+5)
wuffs_gif_decode_1k_color/clang  159MB/s ± 1%  160MB/s ± 0%     ~     (p=0.080 n=48+5)
wuffs_gif_decode_10k/clang       145MB/s ± 1%  145MB/s ± 0%     ~     (p=0.371 n=49+5)
wuffs_gif_decode_100k/clang      125MB/s ± 0%  131MB/s ± 1%   +4.84%  (p=0.000 n=46+5)
wuffs_gif_decode_1000k/clang     128MB/s ± 1%  134MB/s ± 0%   +5.22%  (p=0.000 n=47+5)

wuffs_gif_decode_1k_bw/gcc       341MB/s ± 1%  339MB/s ± 0%   -0.48%  (p=0.001 n=49+5)
wuffs_gif_decode_1k_color/gcc    184MB/s ± 1%  186MB/s ± 0%   +1.10%  (p=0.000 n=50+5)
wuffs_gif_decode_10k/gcc         162MB/s ± 0%  164MB/s ± 0%   +1.76%  (p=0.000 n=50+5)
wuffs_gif_decode_100k/gcc        139MB/s ± 0%  143MB/s ± 0%   +3.49%  (p=0.000 n=49+5)
wuffs_gif_decode_1000k/gcc       142MB/s ± 0%  146MB/s ± 0%   +3.08%  (p=0.000 n=49+5)

wuffs_lzw_decode_20k/clang       132MB/s ± 0%  142MB/s ± 0%   +7.91%  (p=0.000 n=46+5)
wuffs_lzw_decode_100k/clang      170MB/s ± 0%  186MB/s ± 1%   +9.18%  (p=0.000 n=46+5)

wuffs_lzw_decode_20k/gcc         156MB/s ± 0%  162MB/s ± 0%   +3.74%  (p=0.000 n=47+5)
wuffs_lzw_decode_100k/gcc        214MB/s ± 0%  235MB/s ± 1%  +10.03%  (p=0.000 n=47+5)
2018-10-14 11:06:38 +11:00
Nigel Tao 6236e7205f Remove packageid 2018-08-12 21:44:49 +10:00
Nigel Tao eec0fce647 Change error "?foo" to status "?foo" 2018-08-12 21:31:56 +10:00
Nigel Tao 203b45e869 Remove status values 2018-08-12 19:36:29 +10:00
Nigel Tao b385c30b4f Add a "?" prefix to non-built-in status messages 2018-08-12 14:07:40 +10:00
Nigel Tao eca5565821 Removed closed-for-read/write status codes 2018-08-04 09:34:39 +10:00
Nigel Tao 2100a9f837 Use !?? to denote coroutines 2018-07-29 13:57:13 +10:00
Nigel Tao ca06083f8a Rename the implicit in variable to args 2018-07-29 10:26:22 +10:00
Nigel Tao 9919d78245 Change func out-type from struct to bare type 2018-07-29 10:02:06 +10:00
Nigel Tao 92b566367d Optimize some more more std/gif if blocks
name                             old speed     new speed     delta

wuffs_lzw_decode_20k/clang       136MB/s ± 0%  134MB/s ± 1%  -1.27%  (p=0.000 n=9+9)
wuffs_lzw_decode_100k/clang      175MB/s ± 0%  175MB/s ± 0%    ~     (p=0.863 n=9+9)

wuffs_lzw_decode_20k/gcc         168MB/s ± 0%  177MB/s ± 1%  +5.79%  (p=0.000 n=9+8)
wuffs_lzw_decode_100k/gcc        242MB/s ± 0%  262MB/s ± 3%  +8.27%  (p=0.000 n=10+10)

wuffs_gif_decode_1k_bw/clang     280MB/s ± 1%  294MB/s ± 1%  +4.88%  (p=0.000 n=9+9)
wuffs_gif_decode_1k_color/clang  158MB/s ± 0%  158MB/s ± 0%    ~     (p=0.724 n=8+9)
wuffs_gif_decode_10k/clang       146MB/s ± 0%  146MB/s ± 0%    ~     (p=0.387 n=9+9)
wuffs_gif_decode_100k/clang      125MB/s ± 0%  125MB/s ± 1%  +0.18%  (p=0.040 n=9+9)
wuffs_gif_decode_1000k/clang     128MB/s ± 0%  128MB/s ± 1%  +0.38%  (p=0.008 n=8+9)

wuffs_gif_decode_1k_bw/gcc       323MB/s ± 1%  306MB/s ± 2%  -5.20%  (p=0.000 n=10+10)
wuffs_gif_decode_1k_color/gcc    167MB/s ± 2%  164MB/s ± 1%  -1.71%  (p=0.002 n=10+10)
wuffs_gif_decode_10k/gcc         151MB/s ± 0%  152MB/s ± 0%  +0.67%  (p=0.000 n=9+10)
wuffs_gif_decode_100k/gcc        134MB/s ± 0%  136MB/s ± 1%  +1.16%  (p=0.001 n=9+9)
wuffs_gif_decode_1000k/gcc       138MB/s ± 0%  140MB/s ± 1%  +1.18%  (p=0.000 n=9+9)
2018-06-30 08:49:23 +10:00
Nigel Tao 9cd37adfe8 Optimize some more std/gif if blocks
name                             old speed     new speed     delta

wuffs_lzw_decode_20k/clang       124MB/s ± 0%  136MB/s ± 0%   +9.77%  (p=0.000 n=9+9)
wuffs_lzw_decode_100k/clang      164MB/s ± 1%  175MB/s ± 0%   +6.66%  (p=0.000 n=10+9)

wuffs_lzw_decode_20k/gcc         167MB/s ± 0%  168MB/s ± 0%   +0.38%  (p=0.000 n=8+9)
wuffs_lzw_decode_100k/gcc        247MB/s ± 0%  242MB/s ± 0%   -2.16%  (p=0.000 n=9+10)

wuffs_gif_decode_1k_bw/clang     266MB/s ± 1%  280MB/s ± 1%   +5.36%  (p=0.000 n=10+9)
wuffs_gif_decode_1k_color/clang  122MB/s ± 0%  158MB/s ± 0%  +29.46%  (p=0.000 n=10+8)
wuffs_gif_decode_10k/clang       115MB/s ± 1%  146MB/s ± 0%  +27.00%  (p=0.000 n=10+9)
wuffs_gif_decode_100k/clang      113MB/s ± 0%  125MB/s ± 0%  +10.46%  (p=0.000 n=9+9)
wuffs_gif_decode_1000k/clang     115MB/s ± 1%  128MB/s ± 0%  +11.24%  (p=0.000 n=9+8)

wuffs_gif_decode_1k_bw/gcc       298MB/s ± 1%  323MB/s ± 1%   +8.23%  (p=0.000 n=9+10)
wuffs_gif_decode_1k_color/gcc    163MB/s ± 1%  167MB/s ± 2%   +2.10%  (p=0.000 n=10+10)
wuffs_gif_decode_10k/gcc         152MB/s ± 0%  151MB/s ± 0%   -0.55%  (p=0.000 n=9+9)
wuffs_gif_decode_100k/gcc        134MB/s ± 0%  134MB/s ± 0%     ~     (p=0.606 n=8+9)
wuffs_gif_decode_1000k/gcc       138MB/s ± 1%  138MB/s ± 0%     ~     (p=0.422 n=9+9)
2018-06-30 08:40:24 +10:00
Nigel Tao 91bf555431 Optimize some std/gif if blocks
name                             old speed     new speed     delta

wuffs_lzw_decode_20k/clang       123MB/s ± 1%  124MB/s ± 0%   +0.91%  (p=0.000 n=46+9)
wuffs_lzw_decode_100k/clang      156MB/s ± 1%  164MB/s ± 1%   +5.07%  (p=0.000 n=46+10)

wuffs_lzw_decode_20k/gcc         167MB/s ± 2%  167MB/s ± 0%     ~     (p=0.933 n=46+8)
wuffs_lzw_decode_100k/gcc        243MB/s ± 3%  247MB/s ± 0%   +1.56%  (p=0.000 n=47+9)

wuffs_gif_decode_1k_bw/clang     263MB/s ± 1%  266MB/s ± 1%   +1.02%  (p=0.000 n=46+10)
wuffs_gif_decode_1k_color/clang  147MB/s ± 1%  122MB/s ± 0%  -16.63%  (p=0.000 n=46+10)
wuffs_gif_decode_10k/clang       134MB/s ± 2%  115MB/s ± 1%  -14.02%  (p=0.000 n=49+10)
wuffs_gif_decode_100k/clang      115MB/s ± 1%  113MB/s ± 0%   -1.73%  (p=0.000 n=44+9)
wuffs_gif_decode_1000k/clang     117MB/s ± 1%  115MB/s ± 1%   -2.10%  (p=0.000 n=46+9)

wuffs_gif_decode_1k_bw/gcc       291MB/s ± 1%  298MB/s ± 1%   +2.46%  (p=0.000 n=44+9)
wuffs_gif_decode_1k_color/gcc    138MB/s ± 1%  163MB/s ± 1%  +18.50%  (p=0.000 n=45+10)
wuffs_gif_decode_10k/gcc         134MB/s ± 2%  152MB/s ± 0%  +13.37%  (p=0.000 n=47+9)
wuffs_gif_decode_100k/gcc        121MB/s ± 1%  134MB/s ± 0%  +11.18%  (p=0.000 n=46+8)
wuffs_gif_decode_1000k/gcc       124MB/s ± 2%  138MB/s ± 1%  +11.07%  (p=0.000 n=47+9)
2018-06-30 08:33:23 +10:00
Nigel Tao 634789722d Assign values, not just messages, to status codes 2018-06-10 15:48:29 +10:00
Nigel Tao 8dbfad9509 Type check used-package structs and funcs 2018-06-03 10:49:24 +10:00
Nigel Tao 6b640bea86 Rename some std/lzw error messages 2018-06-03 09:28:57 +10:00
Nigel Tao 6d510bcfcb Spin out a std/lzw package 2018-06-02 14:15:09 +10:00