Commit Graph
626 Commits
Author SHA1 Message Date
Nigel Tao a41149e73c std/jpeg: implement triangle-filter upsampling
"script/confirm-matches-djpeg.sh test/data/*.jpeg" is now all matches.
2023-05-21 22:05:38 +10:00
Nigel Tao 94df43c530 std/jpeg: prepare for triangle-filter upsampling 2023-05-21 22:04:45 +10:00
Nigel Tao 5ab65d0923 cgen: generate 123u instead of 123 (for std/jpeg)
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59018
2023-05-16 22:19:43 +10:00
Nigel Tao 85b244aa49 std/netpbm: add new package 2023-05-16 16:29:58 +10:00
Nigel Tao 7514dede23 std/jpeg: decode color images, not just gray
Updates #42
2023-05-14 23:03:33 +10:00
Nigel Tao 8cfe304f3c std/jpeg: implement Inverse DCT
Updates #42
2023-05-08 21:51:47 +10:00
Nigel Tao 0bbc4c0083 std/jpeg: add new package
This is just a placeholder decoder implementation (and all of the
required plumbing) that produces a 1x1 reddish-purple image. Follow-up
commits will add flesh to the metaphorical bones.

Updates #42
2023-04-21 18:33:00 +10:00
Nigel Tao 4834cb97c8 base: deprecate ETC__PIXEL_FORMAT__NUM_PLANES_MAX 2023-04-16 11:12:06 +10:00
Nigel Tao e126c998e0 std: limit supported image dimensions to 0xFF_FFFF 2023-04-16 08:57:55 +10:00
Nigel Tao 6e84987d33 std: change set_quirk_enabled to set_quirk 2023-04-14 21:42:15 +10:00
Nigel Tao f3caf09d77 lang: add read-only type decorators 2023-04-10 15:29:12 +10:00
Nigel Tao 6e8349129c base: remove deprecated names 2023-04-09 16:12:49 +10:00
Nigel Tao 00d5e35865 floatconv: optimize parse_number_f64 fallback
The time taken running script/manual-test-parse-number-f64.cc on the
parse-number-fxx-test-data suite falls from 3.565s to 2.491s (a 1.43x
ratio; the inverse is 0.70 = 100% - 30%).

Fixes #113
2023-04-08 14:56:32 +10:00
Nigel Tao 4d4c3c4c3c floatconv: tweak Simple Decimal Conversion
This tweaks the code and comments added in commit 36d7748e "Merge pull
request #90 from sarastro-nl/improvement/optimize-SDC" and commit
d46220ca "optimize SDC". Both of those two commit messages are short but
see the discussion re algorithmic changes on PR 87 (yes, 87, not 90):
https://github.com/google/wuffs/pull/87

Also run clang-format.

Also update doc/changelog.md

Compared to before commit d46220ca "optimize SDC", when running
script/manual-test-parse-number-f64.cc and disabling both the
etc__parse_number_f64_eisel_lemire function and the "If both man and
(10 ** exp10) are exactly representable by a double" fast path, the
tests still pass but the number of etc__high_prec_dec__small_lshift and
etc__high_prec_dec__small_rshift calls drop:

Function    Before      After       Ratio
lshift      31174584    25762258    82.64% = 100% - 17.36%
rshift      21592604    20982502    97.17% = 100% -  2.83%

The time taken drops from 15.984s to 13.710s (85.77% = 100% - 14.23%).

To repeat, these are the improvements after disabling the Eisel-Lemire
and other fast paths. Simple Decimal Conversion is the rarely invoked
fallback parse_number_f64 algorithm. The practical improvement (with
Eisel-Lemire etc enabled) is smaller.
2023-02-04 14:48:25 +11:00
Nigel Tao 36d7748ed5 Merge pull request #90 from sarastro-nl/improvement/optimize-SDC
optimize SDC
2023-02-04 10:30:29 +11:00
Nigel Tao 3705e0d93a cgen: io_reader.is_closed doesn't need derived var 2023-01-31 23:33:02 +11:00
Nigel Tao d7c48ae76b cgen: have io_limit update meta.closed 2023-01-31 22:05:20 +11:00
sarastro-nl d46220ca00 optimize SDC 2022-11-21 22:10:19 +01:00
Nigel Tao 9445c8b9d0 Guard header declarations on the MODULE_ETC macros 2022-10-28 17:11:14 +11:00
Nigel Tao b0e9efff3f Avoid (NULL + 0) in derived io_buffer variables
This addresses a "runtime error: applying zero offset to null pointer"
UBSAN (Undefined Behavior Sanitizer) warning using clang 15.0.1:
https://logs.chromium.org/logs/skia/5e005cc1b1981011/+/steps/dm/0/stdout

The offending line of code was "io1_a_dst = io0_a_dst + a_dst->meta.wi"
in wuffs_lzw__decoder__write_to and "io1_a_dst = NULL + 0" was Undefined
Behavior even though io1_a_dst was never dereferenced.

This commit avoids initializing io1_a_dst to (NULL + 0) when io0_a_dst
is NULL, falling back to initializing it with just NULL.

As discussed in https://reviews.llvm.org/D67122 while (nullptr + 0) is
Defined Behavior according to the C++ spec, (NULL + 0) is Undefined
Behavior (by omission) in C11 6.5.6/8: "If both the pointer operand and
the result point to elements of the same array object, or one past the
last element of the array object, the evaluation shall not produce an
overflow; otherwise, the behavior is undefined."

The benchmarks (an excerpt of the full suite is below) seem quite
sensitive to this simple change that's outside of hot loops, sometimes
much better and sometimes much worse. I don't know why.

name                                               old speed       new speed      delta

wuffs_bzip2_decode_10k/clang11                    63.1MB/s ± 0%   58.4MB/s ± 0%   -7.43%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/clang11                   49.4MB/s ± 0%   46.2MB/s ± 0%   -6.35%  (p=0.008 n=5+5)

wuffs_bzip2_decode_10k/gcc10                      60.9MB/s ± 0%   56.3MB/s ± 0%   -7.59%  (p=0.016 n=5+4)
wuffs_bzip2_decode_100k/gcc10                     49.6MB/s ± 0%   47.0MB/s ± 0%   -5.15%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/clang11          196MB/s ± 0%    195MB/s ± 1%   -0.63%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/clang11          226MB/s ± 0%    224MB/s ± 0%   -0.95%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_full_init/clang11         409MB/s ± 0%    420MB/s ± 0%   +2.84%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/clang11         418MB/s ± 0%    431MB/s ± 0%   +2.97%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang11    517MB/s ± 1%    542MB/s ± 0%   +4.78%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang11   330MB/s ± 0%    338MB/s ± 0%   +2.45%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/gcc10            188MB/s ± 0%    177MB/s ± 0%   -5.38%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/gcc10            218MB/s ± 0%    209MB/s ± 0%   -4.11%  (p=0.016 n=4+5)
wuffs_deflate_decode_10k_full_init/gcc10           402MB/s ± 0%    407MB/s ± 1%   +1.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/gcc10           413MB/s ± 0%    419MB/s ± 1%   +1.64%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc10      520MB/s ± 0%    532MB/s ± 0%   +2.25%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc10     330MB/s ± 0%    335MB/s ± 0%   +1.56%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang11                     781MB/s ± 0%    645MB/s ± 0%  -17.46%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang11        176MB/s ± 0%    181MB/s ± 0%   +3.11%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang11        215MB/s ± 0%    223MB/s ± 0%   +3.46%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang11                  786MB/s ± 0%    801MB/s ± 0%   +1.94%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang11               208MB/s ± 0%    211MB/s ± 0%   +1.44%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang11                       261MB/s ± 0%    252MB/s ± 0%   -3.43%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang11           578MB/s ± 0%    583MB/s ± 0%   +0.86%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang11            224MB/s ± 0%    223MB/s ± 0%   -0.79%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.73%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.69%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang11           1.31GB/s ± 0%   1.29GB/s ± 0%   -1.06%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc10                       650MB/s ± 0%    631MB/s ± 0%   -3.03%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/gcc10          169MB/s ± 0%    168MB/s ± 0%   -0.51%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc10          202MB/s ± 0%    202MB/s ± 0%   -0.18%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc10                    766MB/s ± 0%    790MB/s ± 0%   +3.08%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc10                 202MB/s ± 0%    208MB/s ± 0%   +3.16%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc10                         250MB/s ± 0%    258MB/s ± 0%   +3.20%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc10             563MB/s ± 0%    566MB/s ± 0%   +0.57%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc10              217MB/s ± 0%    220MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.43%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc10             1.30GB/s ± 0%   1.30GB/s ± 0%   +0.34%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang11                       293MB/s ± 0%    292MB/s ± 0%   -0.41%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang11                      486MB/s ± 0%    537MB/s ± 0%  +10.49%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc10                         258MB/s ± 0%    276MB/s ± 0%   +7.19%  (p=0.016 n=4+5)
wuffs_lzw_decode_100k/gcc10                        512MB/s ± 0%    527MB/s ± 0%   +3.08%  (p=0.016 n=4+5)
2022-10-20 11:18:09 +11:00
Nigel Tao c226ed60f5 Use memcpy-based peeks and pokes on Microsoft C
I don't have a Windows machine readily available, but according to
https://godbolt.org/z/q4MfjzTPh and the https://imgur.com/UD5a7MF
profile mentioned in #72, this could improve inner loop performance.

Updates #72
2022-06-12 17:18:49 +10:00
Nigel Tao c5e41f0e65 Let std/png report metadata after the first frame
Fixes #58
2022-06-02 16:05:24 +10:00
Nigel Tao be047533e1 Add bounds check to wuffs_base__make_slice_uNN_ij 2022-05-23 11:07:31 +10:00
Nigel Tao 84ef91dda8 Tweak cgen for an_array[i .. j] 2022-05-17 00:00:25 +10:00
Nigel Tao f1350825cc Fix cgen for an_array[i .. j] 2022-05-16 23:39:44 +10:00
Nigel Tao 787ff21cc7 Let magic_number_guess_fourcc sniff compression 2022-05-13 11:01:28 +10:00
Nigel Tao 97eb1290f5 Tweak liveness analysis for if-else chains 2022-01-25 19:40:22 +11:00
Nigel Tao 14f5859311 Prohibit iterate loops inside coroutines 2022-01-25 15:22:56 +11:00
Nigel Tao 79ce1a7b0e Clear livenesses after jumps and returns 2022-01-25 15:04:07 +11:00
Nigel Tao 63d8564b3c Tweak liveness analysis for "while true" loops 2022-01-25 14:21:49 +11:00
Nigel Tao 84d0576419 Tweak liveness analysis for return statements 2022-01-25 14:15:32 +11:00
Nigel Tao 6614ed0e90 Tweak liveness analysis for while loops 2022-01-25 13:58:37 +11:00
Nigel Tao 54646d826d Fix liveness analysis typo for break/continue 2022-01-25 12:56:48 +11:00
Nigel Tao c82793a984 Tighten magic_number_guess_fourcc TGA heuristic
Prior to this commit, test/data/cbor-rfc-7049-examples.cbor was
incorrectly sniffed as TGA. /usr/bin/file did this too.

$ file test/data/cbor-rfc-7049-examples.cbor
test/data/cbor-rfc-7049-examples.cbor: Targa image data - Map 59395 x 26
x 15 +6169 +6500 - 2-bit alpha - interleave "@\033"

Updates #67
2022-01-15 14:32:51 +11:00
Nigel Tao 4c09c77bb9 Add std/tga package
This is just a placeholder decoder implementation  (and all of the
required plumbing) that produces a 1x1 reddish-purple image. Follow-up
commits will add flesh to the metaphorical bones.

TGA has no magic identifier, so MIME sniffers have to use heuristics.
The test/data/cbor-rfc-7049-examples.cbor file (which is CBOR binary
data, not a TGA image) is mistakenly sniffed as TGA. The /usr/bin/file
program makes the same classification (a false positive).

$ file test/data/cbor-rfc-7049-examples.cbor
test/data/cbor-rfc-7049-examples.cbor: Targa image data - Map 59395 x 26
x 15 +6169 +6500 - 2-bit alpha - interleave "@\033"

Some test/3pdata/tgasuite/*.tga test files, which are invalid TGA
images, are also false positives for now.

Updates #67
2022-01-14 22:53:20 +11:00
Nigel Tao 792d05bd9c Have magic_number_guess_fourcc use prefix_closed 2022-01-14 15:26:21 +11:00
Nigel Tao e5ead0218e Rename the magic-submodule "prefix" arguments 2022-01-14 15:16:20 +11:00
Nigel Tao f0412122be Add magic_number_guess_fourcc prefix_closed arg 2022-01-14 15:12:14 +11:00
Nigel Tao 546af5a7c3 Let magic_number_guess_fourcc sniff TGA
Updates #67
2022-01-14 14:40:58 +11:00
Nigel Tao b64b2ff09a Tighten magic_number_guess_fourcc ICO heuristic 2022-01-14 10:54:55 +11:00
Nigel Tao dda01f1a77 Add WUFFS_BASE__FOURCC__QOI definition 2022-01-02 09:10:51 +11:00
Nigel Tao 4c04ac1127 Replace FOURCC__{WP8,WP8L} with FOURCC__WEBP
Fixes #61
2021-12-15 15:53:14 +11:00
Nigel Tao 72d4a9e769 Have std/png pass through EXIF data
Fixes #59
2021-11-26 19:14:30 +11:00
Nigel Tao ccbec18330 Re-order pixconv "Extract... components" variables 2021-11-21 21:20:59 +11:00
Nigel Tao 55cad41d6c Have nonpremul SRC_OVER keep transparent red/blue 2021-11-21 21:16:10 +11:00
Nigel Tao f5d6ce3797 Have etc__limited_copy_etc args match call sites 2021-11-20 22:34:25 +11:00
Nigel Tao 161e4730d8 Tweak default WUFFS_VERSION_PRE_RELEASE_LABEL 2021-11-18 19:18:34 +11:00
Nigel Tao d5066c1aae Add wuffs_aux::private_impl::ErrorMessages type 2021-11-18 17:30:11 +11:00
Nigel Tao bfce0e0a16 Let wuffs_aux::DecodeImage re-use raw metadata buf 2021-11-17 12:44:21 +11:00
Nigel Tao 6737565c20 Let std/png decode iTXt, tEXt and zTXt chunks
Updates #55
2021-11-17 11:56:49 +11:00