563 Commits
Author SHA1 Message Date
Nigel Tao 4d4c3c4c3c floatconv: tweak Simple Decimal Conversion
This tweaks the code and comments added in commit 36d7748e "Merge pull
request #90 from sarastro-nl/improvement/optimize-SDC" and commit
d46220ca "optimize SDC". Both of those two commit messages are short but
see the discussion re algorithmic changes on PR 87 (yes, 87, not 90):
https://github.com/google/wuffs/pull/87

Also run clang-format.

Also update doc/changelog.md

Compared to before commit d46220ca "optimize SDC", when running
script/manual-test-parse-number-f64.cc and disabling both the
etc__parse_number_f64_eisel_lemire function and the "If both man and
(10 ** exp10) are exactly representable by a double" fast path, the
tests still pass but the number of etc__high_prec_dec__small_lshift and
etc__high_prec_dec__small_rshift calls drop:

Function    Before      After       Ratio
lshift      31174584    25762258    82.64% = 100% - 17.36%
rshift      21592604    20982502    97.17% = 100% -  2.83%

The time taken drops from 15.984s to 13.710s (85.77% = 100% - 14.23%).

To repeat, these are the improvements after disabling the Eisel-Lemire
and other fast paths. Simple Decimal Conversion is the rarely invoked
fallback parse_number_f64 algorithm. The practical improvement (with
Eisel-Lemire etc enabled) is smaller.
2023-02-04 14:48:25 +11:00
Nigel Tao 36d7748ed5 Merge pull request #90 from sarastro-nl/improvement/optimize-SDC
optimize SDC
2023-02-04 10:30:29 +11:00
Nigel Tao 3705e0d93a cgen: io_reader.is_closed doesn't need derived var 2023-01-31 23:33:02 +11:00
Nigel Tao d7c48ae76b cgen: have io_limit update meta.closed 2023-01-31 22:05:20 +11:00
sarastro-nl d46220ca00 optimize SDC 2022-11-21 22:10:19 +01:00
Nigel Tao 9445c8b9d0 Guard header declarations on the MODULE_ETC macros 2022-10-28 17:11:14 +11:00
Nigel Tao b0e9efff3f Avoid (NULL + 0) in derived io_buffer variables
This addresses a "runtime error: applying zero offset to null pointer"
UBSAN (Undefined Behavior Sanitizer) warning using clang 15.0.1:
https://logs.chromium.org/logs/skia/5e005cc1b1981011/+/steps/dm/0/stdout

The offending line of code was "io1_a_dst = io0_a_dst + a_dst->meta.wi"
in wuffs_lzw__decoder__write_to and "io1_a_dst = NULL + 0" was Undefined
Behavior even though io1_a_dst was never dereferenced.

This commit avoids initializing io1_a_dst to (NULL + 0) when io0_a_dst
is NULL, falling back to initializing it with just NULL.

As discussed in https://reviews.llvm.org/D67122 while (nullptr + 0) is
Defined Behavior according to the C++ spec, (NULL + 0) is Undefined
Behavior (by omission) in C11 6.5.6/8: "If both the pointer operand and
the result point to elements of the same array object, or one past the
last element of the array object, the evaluation shall not produce an
overflow; otherwise, the behavior is undefined."

The benchmarks (an excerpt of the full suite is below) seem quite
sensitive to this simple change that's outside of hot loops, sometimes
much better and sometimes much worse. I don't know why.

name                                               old speed       new speed      delta

wuffs_bzip2_decode_10k/clang11                    63.1MB/s ± 0%   58.4MB/s ± 0%   -7.43%  (p=0.008 n=5+5)
wuffs_bzip2_decode_100k/clang11                   49.4MB/s ± 0%   46.2MB/s ± 0%   -6.35%  (p=0.008 n=5+5)

wuffs_bzip2_decode_10k/gcc10                      60.9MB/s ± 0%   56.3MB/s ± 0%   -7.59%  (p=0.016 n=5+4)
wuffs_bzip2_decode_100k/gcc10                     49.6MB/s ± 0%   47.0MB/s ± 0%   -5.15%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/clang11          196MB/s ± 0%    195MB/s ± 1%   -0.63%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/clang11          226MB/s ± 0%    224MB/s ± 0%   -0.95%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_full_init/clang11         409MB/s ± 0%    420MB/s ± 0%   +2.84%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/clang11         418MB/s ± 0%    431MB/s ± 0%   +2.97%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang11    517MB/s ± 1%    542MB/s ± 0%   +4.78%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang11   330MB/s ± 0%    338MB/s ± 0%   +2.45%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k_full_init/gcc10            188MB/s ± 0%    177MB/s ± 0%   -5.38%  (p=0.008 n=5+5)
wuffs_deflate_decode_1k_part_init/gcc10            218MB/s ± 0%    209MB/s ± 0%   -4.11%  (p=0.016 n=4+5)
wuffs_deflate_decode_10k_full_init/gcc10           402MB/s ± 0%    407MB/s ± 1%   +1.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k_part_init/gcc10           413MB/s ± 0%    419MB/s ± 1%   +1.64%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc10      520MB/s ± 0%    532MB/s ± 0%   +2.25%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc10     330MB/s ± 0%    335MB/s ± 0%   +1.56%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang11                     781MB/s ± 0%    645MB/s ± 0%  -17.46%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang11        176MB/s ± 0%    181MB/s ± 0%   +3.11%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang11        215MB/s ± 0%    223MB/s ± 0%   +3.46%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang11                  786MB/s ± 0%    801MB/s ± 0%   +1.94%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang11               208MB/s ± 0%    211MB/s ± 0%   +1.44%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang11                       261MB/s ± 0%    252MB/s ± 0%   -3.43%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang11           578MB/s ± 0%    583MB/s ± 0%   +0.86%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang11            224MB/s ± 0%    223MB/s ± 0%   -0.79%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.73%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang11           229MB/s ± 0%    225MB/s ± 0%   -1.69%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang11           1.31GB/s ± 0%   1.29GB/s ± 0%   -1.06%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc10                       650MB/s ± 0%    631MB/s ± 0%   -3.03%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/gcc10          169MB/s ± 0%    168MB/s ± 0%   -0.51%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc10          202MB/s ± 0%    202MB/s ± 0%   -0.18%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc10                    766MB/s ± 0%    790MB/s ± 0%   +3.08%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/gcc10                 202MB/s ± 0%    208MB/s ± 0%   +3.16%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc10                         250MB/s ± 0%    258MB/s ± 0%   +3.20%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc10             563MB/s ± 0%    566MB/s ± 0%   +0.57%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc10              217MB/s ± 0%    220MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.49%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc10             220MB/s ± 0%    224MB/s ± 0%   +1.43%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc10             1.30GB/s ± 0%   1.30GB/s ± 0%   +0.34%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang11                       293MB/s ± 0%    292MB/s ± 0%   -0.41%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang11                      486MB/s ± 0%    537MB/s ± 0%  +10.49%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc10                         258MB/s ± 0%    276MB/s ± 0%   +7.19%  (p=0.016 n=4+5)
wuffs_lzw_decode_100k/gcc10                        512MB/s ± 0%    527MB/s ± 0%   +3.08%  (p=0.016 n=4+5)
2022-10-20 11:18:09 +11:00
Nigel Tao c226ed60f5 Use memcpy-based peeks and pokes on Microsoft C
I don't have a Windows machine readily available, but according to
https://godbolt.org/z/q4MfjzTPh and the https://imgur.com/UD5a7MF
profile mentioned in #72, this could improve inner loop performance.

Updates #72
2022-06-12 17:18:49 +10:00
Nigel Tao c5e41f0e65 Let std/png report metadata after the first frame
Fixes #58
2022-06-02 16:05:24 +10:00
Nigel Tao be047533e1 Add bounds check to wuffs_base__make_slice_uNN_ij 2022-05-23 11:07:31 +10:00
Nigel Tao 84ef91dda8 Tweak cgen for an_array[i .. j] 2022-05-17 00:00:25 +10:00
Nigel Tao f1350825cc Fix cgen for an_array[i .. j] 2022-05-16 23:39:44 +10:00
Nigel Tao 787ff21cc7 Let magic_number_guess_fourcc sniff compression 2022-05-13 11:01:28 +10:00
Nigel Tao 97eb1290f5 Tweak liveness analysis for if-else chains 2022-01-25 19:40:22 +11:00
Nigel Tao 14f5859311 Prohibit iterate loops inside coroutines 2022-01-25 15:22:56 +11:00
Nigel Tao 79ce1a7b0e Clear livenesses after jumps and returns 2022-01-25 15:04:07 +11:00
Nigel Tao 63d8564b3c Tweak liveness analysis for "while true" loops 2022-01-25 14:21:49 +11:00
Nigel Tao 84d0576419 Tweak liveness analysis for return statements 2022-01-25 14:15:32 +11:00
Nigel Tao 6614ed0e90 Tweak liveness analysis for while loops 2022-01-25 13:58:37 +11:00
Nigel Tao 54646d826d Fix liveness analysis typo for break/continue 2022-01-25 12:56:48 +11:00
Nigel Tao c82793a984 Tighten magic_number_guess_fourcc TGA heuristic
Prior to this commit, test/data/cbor-rfc-7049-examples.cbor was
incorrectly sniffed as TGA. /usr/bin/file did this too.

$ file test/data/cbor-rfc-7049-examples.cbor
test/data/cbor-rfc-7049-examples.cbor: Targa image data - Map 59395 x 26
x 15 +6169 +6500 - 2-bit alpha - interleave "@\033"

Updates #67
2022-01-15 14:32:51 +11:00
Nigel Tao 4c09c77bb9 Add std/tga package
This is just a placeholder decoder implementation  (and all of the
required plumbing) that produces a 1x1 reddish-purple image. Follow-up
commits will add flesh to the metaphorical bones.

TGA has no magic identifier, so MIME sniffers have to use heuristics.
The test/data/cbor-rfc-7049-examples.cbor file (which is CBOR binary
data, not a TGA image) is mistakenly sniffed as TGA. The /usr/bin/file
program makes the same classification (a false positive).

$ file test/data/cbor-rfc-7049-examples.cbor
test/data/cbor-rfc-7049-examples.cbor: Targa image data - Map 59395 x 26
x 15 +6169 +6500 - 2-bit alpha - interleave "@\033"

Some test/3pdata/tgasuite/*.tga test files, which are invalid TGA
images, are also false positives for now.

Updates #67
2022-01-14 22:53:20 +11:00
Nigel Tao 792d05bd9c Have magic_number_guess_fourcc use prefix_closed 2022-01-14 15:26:21 +11:00
Nigel Tao e5ead0218e Rename the magic-submodule "prefix" arguments 2022-01-14 15:16:20 +11:00
Nigel Tao f0412122be Add magic_number_guess_fourcc prefix_closed arg 2022-01-14 15:12:14 +11:00
Nigel Tao 546af5a7c3 Let magic_number_guess_fourcc sniff TGA
Updates #67
2022-01-14 14:40:58 +11:00
Nigel Tao b64b2ff09a Tighten magic_number_guess_fourcc ICO heuristic 2022-01-14 10:54:55 +11:00
Nigel Tao dda01f1a77 Add WUFFS_BASE__FOURCC__QOI definition 2022-01-02 09:10:51 +11:00
Nigel Tao 4c04ac1127 Replace FOURCC__{WP8,WP8L} with FOURCC__WEBP
Fixes #61
2021-12-15 15:53:14 +11:00
Nigel Tao 72d4a9e769 Have std/png pass through EXIF data
Fixes #59
2021-11-26 19:14:30 +11:00
Nigel Tao ccbec18330 Re-order pixconv "Extract... components" variables 2021-11-21 21:20:59 +11:00
Nigel Tao 55cad41d6c Have nonpremul SRC_OVER keep transparent red/blue 2021-11-21 21:16:10 +11:00
Nigel Tao f5d6ce3797 Have etc__limited_copy_etc args match call sites 2021-11-20 22:34:25 +11:00
Nigel Tao 161e4730d8 Tweak default WUFFS_VERSION_PRE_RELEASE_LABEL 2021-11-18 19:18:34 +11:00
Nigel Tao d5066c1aae Add wuffs_aux::private_impl::ErrorMessages type 2021-11-18 17:30:11 +11:00
Nigel Tao bfce0e0a16 Let wuffs_aux::DecodeImage re-use raw metadata buf 2021-11-17 12:44:21 +11:00
Nigel Tao 6737565c20 Let std/png decode iTXt, tEXt and zTXt chunks
Updates #55
2021-11-17 11:56:49 +11:00
Nigel Tao 07c053af6f Have reset! LEAVE_INTERNAL_BUFFERS_UNINITIALIZED 2021-11-16 13:16:43 +11:00
Nigel Tao d40bf000a9 Rename wuffs_base__table_u8__row as etc__row_u32 2021-11-15 17:26:51 +11:00
Nigel Tao 0045ce49f3 Rename table.row as table.row_u32 2021-11-15 10:49:42 +11:00
Nigel Tao 962a515dd2 Add wuffs_base__table_u8__subtable_ij 2021-11-15 09:58:02 +11:00
Nigel Tao 54bdcc1b57 Update wuffs_aux::DecodeImageArgFlags comment 2021-11-14 19:57:18 +11:00
Nigel Tao f854646586 Fix -Wshadow warnings 2021-11-05 17:51:22 +11:00
Nigel Tao f148f5c5b3 Make "// - etc" bullet point comments consistent 2021-11-05 16:50:38 +11:00
Nigel Tao 0854344580 Add wuffs_aux::sync_io::DynIOBuffer 2021-11-04 22:58:45 +11:00
Nigel Tao 16b51dd2a4 Fix -Wshadow warnings 2021-10-31 16:27:47 +11:00
Nigel Tao badce3ffa4 Tweak magic_number_guess_fourcc negative codes 2021-10-31 16:27:17 +11:00
Nigel Tao 8f9a4f40c2 Rename METADATA_RAW as METADATA_RAW_PASSTHROUGH 2021-10-31 15:17:46 +11:00
Nigel Tao c754f81634 Have wuffs_aux::DecodeImage report ICCP, XMP 2021-10-31 15:09:55 +11:00
Nigel Tao a9a6332be2 Have etc__magic_number_guess_fourcc find ICO 2021-10-22 15:51:44 +11:00