Commit Graph
3009 Commits
Author SHA1 Message Date
Nigel Tao b645180357 Drop pixel_swizzler__bgr__etc d[3] assignment
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=32451
2021-03-26 13:51:27 +11:00
Nigel Tao 77bbe685bf Update doc/roadmap.md 2021-03-25 23:05:54 +11:00
Nigel Tao 3f262b2121 Drop obsolete etc_8_byte_chunks_fast comment 2021-03-25 22:43:54 +11:00
Nigel Tao ad40102c5a Change mimic marker symbol from "!!" to "¿" 2021-03-25 22:22:27 +11:00
Nigel Tao bb2e9e90b5 Change cgen marker symbol from "!!" to "¡" 2021-03-25 22:17:26 +11:00
Nigel Tao 61635213cd Add "‼" to "WUFFS C HEADER ENDS HERE" comment 2021-03-25 14:11:23 +11:00
Nigel Tao 5adb0d3c80 Add "WUFFS MULTI-FILE SECTION" comments 2021-03-25 14:02:04 +11:00
Nigel Tao 7a73708a56 Have script/bench-*-png convert to BGRA
On a mid-range x86_64 laptop:

Compared to the previous commit:

go_decode_19k_8bpp                      39.3MB/s ± 1%  39.4MB/s ± 1%     ~     (p=0.548 n=5+5)
go_decode_40k_24bpp                     46.5MB/s ± 1%  46.7MB/s ± 1%     ~     (p=0.095 n=5+5)
go_decode_77k_8bpp                      78.3MB/s ± 0%  78.3MB/s ± 0%     ~     (p=0.286 n=4+5)
go_decode_552k_32bpp_verify_checksum     120MB/s ± 1%   118MB/s ± 0%     ~     (p=0.056 n=5+5)
go_decode_4002k_24bpp                   50.7MB/s ± 0%  50.5MB/s ± 0%   -0.58%  (p=0.016 n=5+4)

rust_decode_19k_8bpp                    88.3MB/s ± 0%  89.8MB/s ± 0%   +1.76%  (p=0.008 n=5+5)
rust_decode_40k_24bpp                    133MB/s ± 0%   122MB/s ± 0%   -8.58%  (p=0.008 n=5+5)
rust_decode_77k_8bpp                     180MB/s ± 0%   158MB/s ± 0%  -12.04%  (p=0.008 n=5+5)
rust_decode_552k_32bpp_verify_checksum   146MB/s ± 0%   136MB/s ± 0%   -6.59%  (p=0.016 n=4+5)
rust_decode_4002k_24bpp                  134MB/s ± 0%   122MB/s ± 0%   -9.02%  (p=0.008 n=5+5)

Compared to libpng (the C library):

libpng_decode_19k_8bpp                                 58.0MB/s ± 0%  1.00x
libpng_decode_40k_24bpp                                73.1MB/s ± 0%  1.00x
libpng_decode_77k_8bpp                                  177MB/s ± 0%  1.00x
libpng_decode_552k_32bpp_verify_checksum                146MB/s ± 0%  1.00x
libpng_decode_4002k_24bpp                               104MB/s ± 0%  1.00x

go_decode_19k_8bpp/go1.16                              39.4MB/s ± 1%  0.68x
go_decode_40k_24bpp/go1.16                             46.7MB/s ± 1%  0.64x
go_decode_77k_8bpp/go1.16                              78.3MB/s ± 0%  0.44x
go_decode_552k_32bpp_verify_checksum/go1.16             118MB/s ± 0%  0.81x
go_decode_4002k_24bpp/go1.16                           50.5MB/s ± 0%  0.49x

rust_decode_19k_8bpp/rust1.48                          89.8MB/s ± 0%  1.55x
rust_decode_40k_24bpp/rust1.48                          122MB/s ± 0%  1.67x
rust_decode_77k_8bpp/rust1.48                           158MB/s ± 0%  0.89x
rust_decode_552k_32bpp_verify_checksum/rust1.48         136MB/s ± 0%  0.93x
rust_decode_4002k_24bpp/rust1.48                        122MB/s ± 0%  1.17x
2021-03-25 12:38:02 +11:00
Nigel Tao 5dd7ede975 Add more png benchmarks for alternative mimic libs
On a mid-range x86_64 laptop:

"libpng" (Debian's /usr/lib/x86_64-linux-gnu/libpng16.so) numbers are
used as a 1.00x baseline for the other libraries.

(†): libpng's "simplified API" doesn't provide a way to ignore the
checksum. We copy the "verify_checksum" numbers for a 1.00x baseline.
Some of the other libraries don't have the option to ignore or verify
the checksum, always doing only one or the other.

----

libpng_decode_19k_8bpp                            58.0MB/s ± 0%  1.00x
libpng_decode_40k_24bpp                           73.1MB/s ± 0%  1.00x
libpng_decode_77k_8bpp                             177MB/s ± 0%  1.00x
libpng_decode_552k_32bpp_ignore_checksum           146MB/s ± 0%  (†)
libpng_decode_552k_32bpp_verify_checksum           146MB/s ± 0%  1.00x
libpng_decode_4002k_24bpp                          104MB/s ± 0%  1.00x

----

wuffs_decode_19k_8bpp/clang9                       131MB/s ± 0%  2.26x
wuffs_decode_40k_24bpp/clang9                      153MB/s ± 0%  2.09x
wuffs_decode_77k_8bpp/clang9                       472MB/s ± 0%  2.67x
wuffs_decode_552k_32bpp_ignore_checksum/clang9     370MB/s ± 0%  2.53x
wuffs_decode_552k_32bpp_verify_checksum/clang9     357MB/s ± 0%  2.45x
wuffs_decode_4002k_24bpp/clang9                    156MB/s ± 0%  1.50x

wuffs_decode_19k_8bpp/gcc10                        136MB/s ± 1%  2.34x
wuffs_decode_40k_24bpp/gcc10                       162MB/s ± 0%  2.22x
wuffs_decode_77k_8bpp/gcc10                        486MB/s ± 0%  2.75x
wuffs_decode_552k_32bpp_ignore_checksum/gcc10      388MB/s ± 0%  2.66x
wuffs_decode_552k_32bpp_verify_checksum/gcc10      373MB/s ± 0%  2.55x
wuffs_decode_4002k_24bpp/gcc10                     164MB/s ± 0%  1.58x

----

libspng_decode_19k_8bpp/clang9                    59.3MB/s ± 0%  1.02x
libspng_decode_40k_24bpp/clang9                   78.4MB/s ± 0%  1.07x
libspng_decode_77k_8bpp/clang9                     189MB/s ± 0%  1.07x
libspng_decode_552k_32bpp_ignore_checksum/clang9   236MB/s ± 0%  1.62x
libspng_decode_552k_32bpp_verify_checksum/clang9   203MB/s ± 0%  1.39x
libspng_decode_4002k_24bpp/clang9                  110MB/s ± 0%  1.06x

libspng_decode_19k_8bpp/gcc10                     59.6MB/s ± 0%  1.03x
libspng_decode_40k_24bpp/gcc10                    77.5MB/s ± 0%  1.06x
libspng_decode_77k_8bpp/gcc10                      189MB/s ± 0%  1.07x
libspng_decode_552k_32bpp_ignore_checksum/gcc10    223MB/s ± 0%  1.53x
libspng_decode_552k_32bpp_verify_checksum/gcc10    194MB/s ± 0%  1.33x
libspng_decode_4002k_24bpp/gcc10                   109MB/s ± 0%  1.05x

----

lodepng_decode_19k_8bpp/clang9                    65.1MB/s ± 0%  1.12x
lodepng_decode_40k_24bpp/clang9                   72.1MB/s ± 0%  0.99x
lodepng_decode_77k_8bpp/clang9                     222MB/s ± 0%  1.25x
lodepng_decode_552k_32bpp_ignore_checksum/clang9               skipped
lodepng_decode_552k_32bpp_verify_checksum/clang9   162MB/s ± 0%  1.11x
lodepng_decode_4002k_24bpp/clang9                 70.5MB/s ± 0%  0.68x

lodepng_decode_19k_8bpp/gcc10                     61.1MB/s ± 0%  1.05x
lodepng_decode_40k_24bpp/gcc10                    62.5MB/s ± 1%  0.85x
lodepng_decode_77k_8bpp/gcc10                      176MB/s ± 0%  0.99x
lodepng_decode_552k_32bpp_ignore_checksum/gcc10                skipped
lodepng_decode_552k_32bpp_verify_checksum/gcc10    139MB/s ± 0%  0.95x
lodepng_decode_4002k_24bpp/gcc10                  62.3MB/s ± 0%  0.60x

----

stbimage_decode_19k_8bpp/clang9                   75.1MB/s ± 1%  1.29x
stbimage_decode_40k_24bpp/clang9                  84.6MB/s ± 0%  1.16x
stbimage_decode_77k_8bpp/clang9                    234MB/s ± 0%  1.32x
stbimage_decode_552k_32bpp_ignore_checksum/clang9  162MB/s ± 0%  1.11x
stbimage_decode_552k_32bpp_verify_checksum/clang9              skipped
stbimage_decode_4002k_24bpp/clang9                80.7MB/s ± 0%  0.78x

stbimage_decode_19k_8bpp/gcc10                    73.3MB/s ± 0%  1.26x
stbimage_decode_40k_24bpp/gcc10                   81.8MB/s ± 0%  1.12x
stbimage_decode_77k_8bpp/gcc10                     214MB/s ± 0%  1.21x
stbimage_decode_552k_32bpp_ignore_checksum/gcc10   145MB/s ± 0%  0.99x
stbimage_decode_552k_32bpp_verify_checksum/gcc10               skipped
stbimage_decode_4002k_24bpp/gcc10                 79.7MB/s ± 0%  0.77x

----

go_decode_19k_8bpp/go1.16                         39.3MB/s ± 1%  0.68x
go_decode_40k_24bpp/go1.16                        46.5MB/s ± 1%  0.64x
go_decode_77k_8bpp/go1.16                         78.3MB/s ± 0%  0.44x
go_decode_552k_32bpp_ignore_checksum/go1.16                    skipped
go_decode_552k_32bpp_verify_checksum/go1.16        120MB/s ± 1%  0.82x
go_decode_4002k_24bpp/go1.16                      50.7MB/s ± 0%  0.49x

----

rust_decode_19k_8bpp/rust1.48                     88.3MB/s ± 0%  1.52x
rust_decode_40k_24bpp/rust1.48                     133MB/s ± 0%  1.82x
rust_decode_77k_8bpp/rust1.48                      180MB/s ± 0%  1.02x
rust_decode_552k_32bpp_ignore_checksum/rust1.48                skipped
rust_decode_552k_32bpp_verify_checksum/rust1.48    146MB/s ± 0%  1.00x
rust_decode_4002k_24bpp/rust1.48                   134MB/s ± 0%  1.29x
2021-03-24 23:36:48 +11:00
Nigel Tao dba914c5bd Link pixel_swizzler tests to pixel_swizzler_fuzzer 2021-03-22 23:26:52 +11:00
Nigel Tao 13f9ecec68 Have pixel_swizzler_fuzzer use no-r/w guard pages 2021-03-22 23:20:30 +11:00
Nigel Tao 41bfcabd3a Add fuzz/c/std/pixel_swizzler_fuzzer.c 2021-03-22 14:02:39 +11:00
Nigel Tao a5de5fc1cd Link pixel_swizzler tests to aux's SelectPixfmt 2021-03-21 22:22:13 +11:00
Nigel Tao df5bb599a7 Swizzler support src_pixfmt = BGR_565
Binary size, before:
 59472 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
 62072 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
102936 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
102904 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
 66064 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
 68768 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
109256 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
109208 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2021-03-21 22:15:33 +11:00
Nigel Tao 15b04e65f3 Swizzler support src_pixfmt = RGBA_PREMUL
Binary size, before:
 55392 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
 57880 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
 96816 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
 96736 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
 59472 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
 62072 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
102936 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
102904 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2021-03-21 21:49:53 +11:00
Nigel Tao 17ae882a96 Swizzler support src_pixfmt = BGRA_PREMUL
Binary size, before:
48632 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
50784 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
82936 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
82920 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
55392 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
57880 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
96816 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
96736 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2021-03-21 21:27:51 +11:00
Nigel Tao bdb45036a9 Swizzler support dst_pixfmt BGRA_NONPREMUL_4X16LE
Binary size, before:
41496 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
43104 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
72344 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
72328 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
48632 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
50784 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
82936 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
82920 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2021-03-20 17:01:41 +11:00
Nigel Tao 3b6b5ad486 Swizzler support dst_pixfmt = RGBA_{NON,}PREMUL
Binary size, before:
37856 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
39552 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
64536 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
64488 gen/lib/c/gcc-static/wuffs-base-pixconv.o

After:
41496 gen/lib/c/clang-9-dynamic/wuffs-base-pixconv.lo
43104 gen/lib/c/clang-9-static/wuffs-base-pixconv.o
72344 gen/lib/c/gcc-dynamic/wuffs-base-pixconv.lo
72328 gen/lib/c/gcc-static/wuffs-base-pixconv.o
2021-03-19 18:52:56 +11:00
Nigel Tao f49c382029 wuffs gen -version=0.3.0-alpha.21 2021-03-18 12:11:20 +11:00
Nigel Tao 8d90971782 Bump "MSVC+X64 needs /arch:AVX" from error to warn 2021-03-18 12:09:22 +11:00
Nigel Tao cbc072e776 Include <immintrin.h> etc for MSVC+X64 2021-03-18 11:50:05 +11:00
Nigel Tao b24e046670 Replace SIMD CRC-32 k6' magic constant with Px'
This is just a change of name (and a change of comment), not a change of
value.

From my colleague, Eric Biggers:

The number is not k6' = (x^32 mod P(x) << 32)' << 1 = 0x1DB710640, but
rather P(x)' = 0x1DB710641. This value represents the generator
polynomial of CRC-32 as a 33-bit value, using the CRC-32 bit ordering
convention (lowest order bit = coefficient of highest order polynomial
term). The lowest order bit is the coefficient of x^32, which by
definition is 1 for CRC-32. P(x) is used as a multiplicand in Barrett
reduction, which is the step which reduces a 64-bit value to the final
32-bit remainder. See "Algorithm 1" in the Gopal paper.

The reason that last bit doesn't actually matter is because the output
of the Barrett reduction is supposed to be fully reduced mod P(x), so
only 32 bits (the bits that represent the coefficients of x^0..x^31) are
supposed to be able to be nonzero. Therefore, in practice
implementations will just truncate the result to 32 bits. Incorrectly
using 0x1DB710640 rather than P(x) = 0x1DB710641 only causes the result
to be wrong by a factor of x^32, so it doesn't affect those 32 bits.
2021-03-18 11:21:51 +11:00
Nigel Tao 16b4b7554b Link to SIMD CRC-32 k6' magic constant blog post 2021-03-18 11:06:21 +11:00
Nigel Tao 8d1650117f wuffs gen -version=0.3.0-alpha.20 2021-03-17 12:08:15 +11:00
Nigel Tao 9914f7314d Add WUFFS_BASE__MAYBE_ATTRIBUTE_TARGET 2021-03-17 12:03:06 +11:00
Nigel Tao 073d3e0150 wuffs gen -version=0.3.0-alpha.19 2021-03-16 12:22:28 +11:00
Nigel Tao 85bdcd1e1e Fix script/benchstat-ratio.go when there's no '/' 2021-03-13 22:40:37 +11:00
Nigel Tao 3e5afc3e52 Remove png_fuzzer "smoke alarm test"
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=31984
2021-03-13 13:39:35 +11:00
Nigel Tao ee62582609 Tweak some mimic bench names for benchstat-ratio 2021-03-13 13:31:41 +11:00
Nigel Tao 9e4c87b6ab Add script/benchstat-ratio.go 2021-03-13 00:11:50 +11:00
Nigel Tao cbc1367a0f Add more pixel_buffer fill_rect cases 2021-03-11 23:57:41 +11:00
Nigel Tao a04fda42d6 Update doc/std/image-decoders.md 2021-03-11 23:04:53 +11:00
Nigel Tao 385c34930a Check 8-byte-chunk copy distance precondition 2021-03-11 22:54:06 +11:00
Nigel Tao 5767f030a3 Add DecodeImageCallbacks::SelectPixfmt 2021-03-11 16:25:16 +11:00
Nigel Tao 441e96a8d4 Add pixel_buffer.set_color_u32_fill_rect 2021-03-11 15:53:09 +11:00
Nigel Tao 2e0f2a68be Add doc/std/README.md "Modules and Dependencies" 2021-03-11 12:38:58 +11:00
Nigel Tao 1daace6342 Add DecodeImageCallbacks::OnImageFormat def impl 2021-03-11 12:04:00 +11:00
Nigel Tao 54175156bd Add more wuffs_aux::DecodeImage comments 2021-03-11 11:28:50 +11:00
Nigel Tao 49c5692c32 Add png_fuzzer "smoke alarm test" 2021-03-11 11:05:20 +11:00
Nigel Tao f2ed33375c Gate wuffs_aux on WUFFS_BASE__HAVE_UNIQUE_PTR 2021-03-11 10:52:49 +11:00
Nigel Tao 1b3a87d831 Fix cgen.go comment typo 2021-03-11 10:25:18 +11:00
Nigel Tao 62986fde71 Let example/convert-to-nia handle I/O redirects 2021-03-10 23:31:23 +11:00
Nigel Tao e9c6f7fd55 Let example/convert-to-nia speak PNG 2021-03-10 23:29:51 +11:00
Nigel Tao 02041e84f8 Narrow the arg range for base.uxx.{high,low}_bits 2021-03-10 14:28:06 +11:00
Nigel Tao e16fbafdca Add TODOs for using MSVC intrinsics 2021-03-10 14:22:11 +11:00
Nigel Tao 334974f56e Add WUFFS_BASE__HAVE_EQ_DELETE 2021-03-10 14:05:09 +11:00
Nigel Tao d05cae1e1a Pull struct definitions out of extern "C"
This avoids
https://developercommunity.visualstudio.com/t/Cryptic-error-message:-error-C2065:-__/1363621
2021-03-10 13:49:01 +11:00
Nigel Tao 210021e584 Drop obsolete "#if defined(_MSC_VER)" 2021-03-10 09:25:52 +11:00
Nigel Tao 312810f5a3 Fix some -Werror=sign-conversion bugs 2021-03-09 00:39:26 +11:00
Nigel Tao 69e32e8579 Add WUFFS_BASE__HAVE_UNIQUE_PTR macro 2021-03-09 00:30:06 +11:00