3375 Commits
Author SHA1 Message Date
Nigel Tao 1e03a0c84f wuffs gen -version=0.3.0 v0.3.0 2023-01-26 17:18:14 +11:00
Nigel Tao c698d81b4d Update README.md, doc/changelog.md 2023-01-26 17:17:12 +11:00
Nigel Tao 95164c7a3f Update doc/binary-size.md 2023-01-26 15:56:07 +11:00
Nigel Tao 255d3a0a29 Fix C++ "init-statement in selection" warning 2023-01-26 15:39:23 +11:00
Nigel Tao e915543f09 Fix jsonfindptrs "internal error" false positive 2022-12-08 15:07:42 +11:00
Nigel Tao dee65c0e61 wuffs gen -version=0.3.0-rc.3 v0.3.0-rc.3 2022-10-28 17:29:05 +11:00
Nigel Tao c27250f52f Add -Wall to build-example.sh 2022-10-28 17:17:09 +11:00
Nigel Tao 9445c8b9d0 Guard header declarations on the MODULE_ETC macros 2022-10-28 17:11:14 +11:00
Nigel Tao b5ccd9e8e7 wuffs gen -version=0.3.0-rc.2 v0.3.0-rc.2 2022-10-20 11:23:06 +11:00
Nigel Tao b0e9efff3f Avoid (NULL + 0) in derived io_buffer variables
This addresses a "runtime error: applying zero offset to null pointer"
UBSAN (Undefined Behavior Sanitizer) warning using clang 15.0.1:
https://logs.chromium.org/logs/skia/5e005cc1b1981011/+/steps/dm/0/stdout

The offending line of code was "io1_a_dst = io0_a_dst + a_dst->meta.wi"
in wuffs_lzw__decoder__write_to and "io1_a_dst = NULL + 0" was Undefined
Behavior even though io1_a_dst was never dereferenced.

This commit avoids initializing io1_a_dst to (NULL + 0) when io0_a_dst
is NULL, falling back to initializing it with just NULL.

As discussed in https://reviews.llvm.org/D67122 while (nullptr + 0) is
Defined Behavior according to the C++ spec, (NULL + 0) is Undefined
Behavior (by omission) in C11 6.5.6/8: "If both the pointer operand and
the result point to elements of the same array object, or one past the
last element of the array object, the evaluation shall not produce an
overflow; otherwise, the behavior is undefined."

The benchmarks (an excerpt of the full suite is below) seem quite
sensitive to this simple change that's outside of hot loops, sometimes
much better and sometimes much worse. I don't know why.

name                                               old speed       new speed      delta

wuffs_bzip2_decode_10k/clang11                    63.1MB/s ± 0%   58.4MB/s ± 0%   -7.43%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/clang11                   49.4MB/s ± 0%   46.2MB/s ± 0%   -6.35%  (p=0.008 n=5+5)

wuffs_bzip2_decode_10k/gcc10                      60.9MB/s ± 0%   56.3MB/s ± 0%   -7.59%  (p=0.016 n=5+4)
wuffs_bzip2_decode_100k/gcc10                     49.6MB/s ± 0%   47.0MB/s ± 0%   -5.15%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/clang11          196MB/s ± 0%    195MB/s ± 1%   -0.63%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/clang11          226MB/s ± 0%    224MB/s ± 0%   -0.95%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_full_init/clang11         409MB/s ± 0%    420MB/s ± 0%   +2.84%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/clang11         418MB/s ± 0%    431MB/s ± 0%   +2.97%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang11    517MB/s ± 1%    542MB/s ± 0%   +4.78%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang11   330MB/s ± 0%    338MB/s ± 0%   +2.45%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/gcc10            188MB/s ± 0%    177MB/s ± 0%   -5.38%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/gcc10            218MB/s ± 0%    209MB/s ± 0%   -4.11%  (p=0.016 n=4+5)
wuffs_deflate_decode_10k_full_init/gcc10           402MB/s ± 0%    407MB/s ± 1%   +1.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/gcc10           413MB/s ± 0%    419MB/s ± 1%   +1.64%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc10      520MB/s ± 0%    532MB/s ± 0%   +2.25%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc10     330MB/s ± 0%    335MB/s ± 0%   +1.56%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang11                     781MB/s ± 0%    645MB/s ± 0%  -17.46%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang11        176MB/s ± 0%    181MB/s ± 0%   +3.11%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang11        215MB/s ± 0%    223MB/s ± 0%   +3.46%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang11                  786MB/s ± 0%    801MB/s ± 0%   +1.94%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang11               208MB/s ± 0%    211MB/s ± 0%   +1.44%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang11                       261MB/s ± 0%    252MB/s ± 0%   -3.43%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang11           578MB/s ± 0%    583MB/s ± 0%   +0.86%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang11            224MB/s ± 0%    223MB/s ± 0%   -0.79%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.73%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.69%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang11           1.31GB/s ± 0%   1.29GB/s ± 0%   -1.06%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc10                       650MB/s ± 0%    631MB/s ± 0%   -3.03%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/gcc10          169MB/s ± 0%    168MB/s ± 0%   -0.51%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc10          202MB/s ± 0%    202MB/s ± 0%   -0.18%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc10                    766MB/s ± 0%    790MB/s ± 0%   +3.08%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc10                 202MB/s ± 0%    208MB/s ± 0%   +3.16%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc10                         250MB/s ± 0%    258MB/s ± 0%   +3.20%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc10             563MB/s ± 0%    566MB/s ± 0%   +0.57%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc10              217MB/s ± 0%    220MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.43%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc10             1.30GB/s ± 0%   1.30GB/s ± 0%   +0.34%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang11                       293MB/s ± 0%    292MB/s ± 0%   -0.41%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang11                      486MB/s ± 0%    537MB/s ± 0%  +10.49%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc10                         258MB/s ± 0%    276MB/s ± 0%   +7.19%  (p=0.016 n=4+5)
wuffs_lzw_decode_100k/gcc10                        512MB/s ± 0%    527MB/s ± 0%   +3.08%  (p=0.016 n=4+5)
2022-10-20 11:18:09 +11:00
Nigel Tao 8b7f82142c Update doc/roadmap.md 2022-10-17 19:13:19 +11:00
Nigel Tao b245252e1e Update doc/benchmarks.md 2022-09-04 00:54:21 +10:00
Nigel Tao bad2693efb Add lib/litonlylzma.FileFormatXz 2022-08-24 22:46:01 +10:00
Nigel Tao be1f605108 Run gofmt 2022-08-21 21:41:06 +10:00
Nigel Tao 315b2e5262 wuffs gen -version=0.3.0-rc.1 v0.3.0-rc.1 2022-07-10 10:30:15 +10:00
Nigel Tao fac934b272 Fix lib/readerat ReadSeeker.Seek offset bug
Fixes #80
2022-06-29 11:40:26 +10:00
Nigel Tao b8d44be783 Touch fuzz/c/std/bzip2_fuzzer.c again
This undoes commit c1283e49 now that issue #79 was closed.

Updates #79
2022-06-24 13:32:04 +10:00
Nigel Tao d0f0591f64 Add lib/litonlylzma 2022-06-24 13:22:36 +10:00
Nigel Tao c1283e4989 Touch fuzz/c/std/bzip2_fuzzer.c
Updates #79
2022-06-20 19:13:40 +10:00
Nigel Tao c226ed60f5 Use memcpy-based peeks and pokes on Microsoft C
I don't have a Windows machine readily available, but according to
https://godbolt.org/z/q4MfjzTPh and the https://imgur.com/UD5a7MF
profile mentioned in #72, this could improve inner loop performance.

Updates #72
2022-06-12 17:18:49 +10:00
Nigel Tao 45a79923e6 Tighten std/bzip2 Huffman leaf node bit packing 2022-06-10 14:31:53 +10:00
Nigel Tao 67aa63297c Rename std/bzip2 num_nodes variable 2022-06-10 14:06:27 +10:00
Nigel Tao 8c33dd503b Tighten std/bzip2 huffman_trees array size 2022-06-10 14:02:17 +10:00
Nigel Tao 5d7b113172 Optimize imageviewer xcb_clear_area calls 2022-06-08 17:22:29 +10:00
Nigel Tao 886e7f3e8f Simplify fuzzers' io_buffer construction 2022-06-08 17:08:49 +10:00
Nigel Tao fbc686f9b7 Add memrandomize to fuzz/c/fuzzlib 2022-06-08 17:06:59 +10:00
Nigel Tao 9256fe5e54 Have std/bzip2 check the number of sections
Updates #79
2022-06-08 16:41:46 +10:00
Nigel Tao e1b43cd767 Add test/data/artificial-bzip2/abraca.txt.bz2 2022-06-08 14:04:09 +10:00
Nigel Tao d90dde4049 Add .allstar/binary_artifacts.yaml
Fixes #78
2022-06-06 14:17:57 +10:00
Nigel Tao 5392d503e3 wuffs gen -version=0.3.0-beta.15 v0.3.0-beta.15 2022-06-03 14:16:18 +10:00
Nigel Tao 3ff4bf4674 Add script/print-image-metadata.cc
Updates #58
2022-06-03 14:15:13 +10:00
Nigel Tao ebe080c9ec Fix gif metadata when skipping decode_image_config 2022-06-03 11:43:05 +10:00
Nigel Tao 8feb2709fa Have imageviewer use wuffs_base__i32__{min,max} 2022-06-03 11:28:39 +10:00
Nigel Tao 65afe919b5 Fix std/gif "@end of data" for 0-frame GIFs
This fixes a regression introduced by 8d03a67e "Re-do image decoders'
call sequence state machine".

build-all.sh's script/print-nia-checksums.sh test once again matches
test/nia-checksums-of-data.txt:
-4c6d02c3 test/data/artificial-gif/no-frames.gif
+ba6e48f8 test/data/artificial-gif/no-frames.gif
2022-06-02 16:48:06 +10:00
Nigel Tao 21acc78c24 Have build-all.sh note 99/11 in C99/C++11 2022-06-02 16:31:28 +10:00
Nigel Tao c5e41f0e65 Let std/png report metadata after the first frame
Fixes #58
2022-06-02 16:05:24 +10:00
Nigel Tao 4c5d313120 Let example/imageviewer scroll 2022-05-31 12:29:05 +10:00
Nigel Tao 89411bc91d Add lib/nie Go library for decoding NIE images 2022-05-28 16:26:59 +10:00
Nigel Tao 8d03a67e6e Re-do image decoders' call sequence state machine 2022-05-28 15:06:56 +10:00
Nigel Tao a2e3a5a272 Update doc/std/compression-decoders.md for bzcat 2022-05-28 14:23:25 +10:00
Nigel Tao f700e00df8 Update doc/std/compression-decoders.md for bzip2 2022-05-28 10:11:34 +10:00
Nigel Tao a56dbb3e3b Tweak script/print-average-pixel.cc output 2022-05-26 14:28:30 +10:00
Nigel Tao 0f98004914 Add script/print-average-pixel.cc 2022-05-25 16:59:13 +10:00
Nigel Tao f55274aaa7 Fix -Werror=conversion in decode_bzip2.wuffs 2022-05-24 13:58:48 +10:00
Nigel Tao 3e93ffe175 Type check const's array length and elem count 2022-05-24 13:56:24 +10:00
Nigel Tao e6793fc501 Have std/bzip2 support QUIRK_IGNORE_CHECKSUM 2022-05-24 12:31:43 +10:00
Nigel Tao 09102fb1e8 Tweak std/bzip2 decoder.flush_fast applicability 2022-05-24 12:25:47 +10:00
Nigel Tao 6a3e43f495 Tweak std/bzip2 flush_* basic blocks
name                             old speed      new speed      delta

wuffs_bzip2_decode_10k/clang11   65.2MB/s ± 0%  65.2MB/s ± 0%    ~     (p=0.056 n=5+5)
wuffs_bzip2_decode_100k/clang11  52.4MB/s ± 0%  51.5MB/s ± 0%  -1.67%  (p=0.016 n=5+4)

wuffs_bzip2_decode_10k/gcc10     60.6MB/s ± 0%  61.7MB/s ± 0%  +1.81%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/gcc10    49.2MB/s ± 0%  49.6MB/s ± 0%  +0.86%  (p=0.008 n=5+5)
2022-05-24 12:01:30 +10:00
Nigel Tao bed155bd6f Optimize std/bzip2 flush_repeat_count checks
name                             old speed      new speed      delta

wuffs_bzip2_decode_10k/clang11   60.5MB/s ± 0%  65.2MB/s ± 0%   +7.85%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/clang11  45.7MB/s ± 1%  52.4MB/s ± 0%  +14.68%  (p=0.008 n=5+5)

wuffs_bzip2_decode_10k/gcc10     58.5MB/s ± 0%  60.6MB/s ± 0%   +3.67%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/gcc10    46.3MB/s ± 0%  49.2MB/s ± 0%   +6.31%  (p=0.008 n=5+5)
2022-05-24 11:56:23 +10:00
Nigel Tao 66f194f7e0 Introduce std/bzip2 decoder.flush_fast method
Compared to the recent commit 623290ef "Cache std/bzip2 Huffman tree
lookup":

name                             old speed      new speed      delta

wuffs_bzip2_decode_10k/clang11   60.5MB/s ± 0%  60.5MB/s ± 0%    ~     (p=1.000 n=5+5)
wuffs_bzip2_decode_100k/clang11  46.7MB/s ± 1%  45.7MB/s ± 1%  -2.05%  (p=0.008 n=5+5)

wuffs_bzip2_decode_10k/gcc10     58.7MB/s ± 0%  58.5MB/s ± 0%    ~     (p=0.222 n=5+5)
wuffs_bzip2_decode_100k/gcc10    46.9MB/s ± 0%  46.3MB/s ± 0%  -1.46%  (p=0.008 n=5+5)
2022-05-24 11:50:48 +10:00