Commit Graph
71 Commits
Author SHA1 Message Date
Nigel Tao 82ea1985b4 base: add optional_u63 type 2023-11-19 12:02:24 +11:00
Nigel Tao 5407f39386 base: use wuffs_base__empty_slice_u8 more often 2023-11-02 22:43:01 +11:00
Nigel Tao 9a68ec70c6 base: use wuffs_private_impl__etc names 2023-11-02 22:30:10 +11:00
Nigel Tao dd75f3215a base: tweak etc_with_non_null_address var names 2023-10-24 20:47:23 +11:00
Nigel Tao f81deb6b6e base: have empty_foo() functions use non-NULL ptr
Fixes #135
2023-10-19 19:31:59 +11:00
Nigel Tao e0636e3255 base: add bitvec256 type 2023-10-12 21:26:53 +11:00
Nigel Tao ae8531f7f8 cgen: generate "foo(void)" instead of "foo()"
This was discovered by "gcc -Wstrict-prototypes".

Updates #132
2023-10-07 11:36:53 +11:00
Patrick Georgi f563f32f82 Add SPDX identifier to license headers
SPDX is an ISO standard for describing licenses in a computer readable
format, which is useful when applying license compliance tools and the
like to source code.

In addition to ensuring that the Wuffs C distribution is properly
annotated to the benefit of projects using Wuffs and SPDX-supporting
tools, add it everywhere else where there's a license in this repo for
consistency, too.
2023-10-06 20:52:36 +02:00
Nigel Tao 74a6ea2fcd all: relicense from Apache2 to dual Apache2/MIT
The LICENSE has changed from a single license (Apache2) to a dual
license (Apache2 or MIT, at your option).

If you've previously used Wuffs under the Apache2 license, you are able
to keep doing so. No further action required by you.

The two files listed below (previous releases) are exceptions, as they
are older snapshots, frozen in time. Their license has not changed:
release/c/wuffs-v0.2.c
release/c/wuffs-v0.3.c

Every name in the AUTHORS file, other than Google Inc., has signed
Google's CLA (Contributor License Agreement).
2023-10-06 10:00:59 +11:00
Nigel Tao bffbad69a4 cgen: halve the sizeof wuffs_foo__bar::unique_ptr 2023-09-24 21:22:22 +10:00
Nigel Tao eca57ef5b3 base: add status.is_truncated_input_error
This just adds a C/C++ function/method. We may or may not add the
corresponding Wuffs method in a future commit.
2023-08-24 16:11:15 +10:00
Nigel Tao 5dfc913cb0 base: let cpu_arch__have_x86_etc return true 2023-07-23 21:48:33 +10:00
Nigel Tao e2b9566fc5 base: restrict x86_avx2 to 64-bit (not 32-bit) x86
32-bit C/C++ compilers don't always compile the AVX intrinsics. Per the
60773 bug below,
> clang++ -m32 etc etc fuzz/c/std/jpeg_fuzzer.c
> undefined reference to `_mm_cvtsi128_si64'
> undefined reference to `_mm256_extract_epi64'

See also the "#ifdef __x86_64__" guards in
https://github.com/gcc-mirror/gcc/blob/4a3895903c29ed85da6fcb886f31ff23d4c6e935/gcc/config/i386/emmintrin.h#L228-L231
https://github.com/llvm/llvm-project/blob/08fab9ebecf72a682279d75489dc2460121cbeed/clang/lib/Headers/emmintrin.h#L3502-L3515

Note that -m32 is not the same as -mx32
> $ touch empty.c
> $ gcc       -dM -E empty.c | grep -E '__i386__|__x86_64__'
> #define __x86_64__ 1
> $ gcc -m32  -dM -E empty.c | grep -E '__i386__|__x86_64__'
> #define __i386__ 1
> $ gcc -mx32 -dM -E empty.c | grep -E '__i386__|__x86_64__'
> #define __x86_64__ 1

Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=60773
2023-07-23 21:20:35 +10:00
Nigel Tao e7df7cafac cgen: add WUFFS_BASE__GENERATED_C_CODE macro
This disables UBSan's "unsigned-integer-overflow" runtime checks.

Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59966
2023-06-20 17:04:35 +10:00
Nigel Tao 6e8349129c base: remove deprecated names 2023-04-09 16:12:49 +10:00
Nigel Tao c226ed60f5 Use memcpy-based peeks and pokes on Microsoft C
I don't have a Windows machine readily available, but according to
https://godbolt.org/z/q4MfjzTPh and the https://imgur.com/UD5a7MF
profile mentioned in #72, this could improve inner loop performance.

Updates #72
2022-06-12 17:18:49 +10:00
Nigel Tao be047533e1 Add bounds check to wuffs_base__make_slice_uNN_ij 2022-05-23 11:07:31 +10:00
Nigel Tao 84ef91dda8 Tweak cgen for an_array[i .. j] 2022-05-17 00:00:25 +10:00
Nigel Tao f0412122be Add magic_number_guess_fourcc prefix_closed arg 2022-01-14 15:12:14 +11:00
Nigel Tao 161e4730d8 Tweak default WUFFS_VERSION_PRE_RELEASE_LABEL 2021-11-18 19:18:34 +11:00
Nigel Tao 962a515dd2 Add wuffs_base__table_u8__subtable_ij 2021-11-15 09:58:02 +11:00
Nigel Tao 0460bfcd7a Add FourCC comment 2021-10-17 23:18:58 +11:00
Nigel Tao e317f04bfd Add defined(__clang__) to defined(__GNUC__) checks 2021-10-17 12:14:54 +11:00
Nigel Tao 7f9f37ce5a Have examples use WUFFS_CONFIG__STATIC_FUNCTIONS
Compile time:
$ time ./build-example.sh
Before: user 1m15.750s
After:  user 0m50.343s

Binary size (excluding the statically linked example-toy-genlib):
$ ls -l gen/bin/example-* | awk '{print $5 " " $9}'

Before:
212624 gen/bin/example-cbor-to-json
284192 gen/bin/example-convert-to-nia
344744 gen/bin/example-crc32
184304 gen/bin/example-gifplayer
324952 gen/bin/example-imageviewer
209632 gen/bin/example-json-to-cbor
241432 gen/bin/example-jsonfindptrs
196320 gen/bin/example-jsonptr
309768 gen/bin/example-sdl-imageviewer
200704 gen/bin/example-zcat

After:
 85280 gen/bin/example-cbor-to-json
242216 gen/bin/example-convert-to-nia
344744 gen/bin/example-crc32
 53328 gen/bin/example-gifplayer
282840 gen/bin/example-imageviewer
 95152 gen/bin/example-json-to-cbor
122720 gen/bin/example-jsonfindptrs
 60448 gen/bin/example-jsonptr
267672 gen/bin/example-sdl-imageviewer
 64792 gen/bin/example-zcat

Before total: 2508672
After  total: 1619192
2021-10-04 12:35:32 +11:00
Nigel Tao a66a083e82 Have X86_FAMILY allow 32-bit x86, not just x86_64
Fixes #53
2021-08-22 22:37:48 +10:00
Nigel Tao cb2e49ab09 Rename CPU arch macro s/X86_64/X86_FAMILY/ 2021-08-22 22:23:49 +10:00
Nigel Tao 1d19157668 Remove After editing... run "go generate" lines 2021-08-22 12:57:39 +10:00
Nigel Tao b9a10d051a Fix -Werror=conversion in fundamental-public.h 2021-06-28 10:10:13 +10:00
Nigel Tao d8796aa6d9 Add wuffs_base__uxx__rotate_etc functions 2021-06-28 09:59:32 +10:00
Nigel Tao 91857f4c4b Disable cpu_arch specific code for Native Client 2021-06-24 22:40:54 +10:00
Nigel Tao 08797bc713 Have "x86_avx2" require PCLMUL, POPCNT and SSE4.2
The std/crc32 benchmark numbers get worse. The benches still run AVX2
code, but it runs slower for an unknown reason. The std/gzip and std/png
benchmark numbers are mixed.

Still, it's cleaner for Wuffs' "x86_avx2" to be a superset of
"x86_sse42" (and apart from PCLMUL, they're now similar to Clang 12 and
GCC 11's "x86-64-v3" and "x86-64-v2" micro-architecture feature levels).
Maybe future compiler versions can claw back the std/crc32 performance.

On a mid-range x86_64 laptop (2016, Skylake):

name                                                               old speed      new speed      delta

wuffs_crc32_ieee_10k/clang9                                        8.07GB/s ± 0%  7.61GB/s ± 1%   -5.72%  (p=0.000 n=8+8)
wuffs_crc32_ieee_100k/clang9                                       11.0GB/s ± 1%   9.7GB/s ± 3%  -11.45%  (p=0.000 n=7+8)

wuffs_crc32_ieee_10k/gcc10                                         9.77GB/s ± 1%  7.72GB/s ± 1%  -21.00%  (p=0.000 n=10+9)
wuffs_crc32_ieee_100k/gcc10                                        12.7GB/s ± 2%  11.9GB/s ± 2%   -5.84%  (p=0.000 n=9+9)

wuffs_gzip_decode_10k/clang9                                        217MB/s ± 2%   221MB/s ± 0%   +1.80%  (p=0.000 n=10+8)
wuffs_gzip_decode_100k/clang9                                       280MB/s ± 1%   283MB/s ± 0%   +1.38%  (p=0.000 n=9+8)

wuffs_gzip_decode_10k/gcc10                                         215MB/s ± 1%   220MB/s ± 1%   +2.52%  (p=0.000 n=10+9)
wuffs_gzip_decode_100k/gcc10                                        272MB/s ± 0%   274MB/s ± 1%   +0.74%  (p=0.000 n=9+10)

wuffs_png_decode_image_19k_8bpp/clang9                              136MB/s ± 0%   142MB/s ± 0%   +3.95%  (p=0.000 n=8+8)
wuffs_png_decode_image_40k_24bpp/clang9                             171MB/s ± 1%   172MB/s ± 0%   +0.84%  (p=0.000 n=10+8)
wuffs_png_decode_image_77k_8bpp/clang9                              492MB/s ± 3%   501MB/s ± 2%   +1.88%  (p=0.009 n=10+10)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9            456MB/s ± 1%   461MB/s ± 1%   +1.04%  (p=0.000 n=9+9)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9            439MB/s ± 1%   440MB/s ± 3%     ~     (p=0.173 n=8+10)
wuffs_png_decode_image_4002k_24bpp/clang9                           172MB/s ± 1%   174MB/s ± 1%   +1.29%  (p=0.005 n=10+10)

wuffs_png_decode_image_19k_8bpp/gcc10                               152MB/s ± 0%   149MB/s ± 1%   -1.92%  (p=0.000 n=9+9)
wuffs_png_decode_image_40k_24bpp/gcc10                              184MB/s ± 0%   186MB/s ± 1%   +1.37%  (p=0.000 n=8+9)
wuffs_png_decode_image_77k_8bpp/gcc10                               550MB/s ± 1%   542MB/s ± 2%   -1.46%  (p=0.000 n=10+10)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10             473MB/s ± 1%   478MB/s ± 1%   +1.23%  (p=0.000 n=9+10)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10             453MB/s ± 1%   458MB/s ± 1%   +0.91%  (p=0.000 n=9+10)
wuffs_png_decode_image_4002k_24bpp/gcc10                            185MB/s ± 2%   188MB/s ± 0%   +1.82%  (p=0.000 n=9+8)
2021-06-22 13:27:25 +10:00
Nigel Tao b8f87debe3 Tweak wuffs_base__cpu_arch__have_foo_bar impl 2021-04-29 17:52:10 +10:00
Nigel Tao 8ccaf81e96 Fix s/bit_BMI2/bit_AVX2/ typo 2021-04-26 22:52:27 +10:00
Nigel Tao 2dddd388bc Add crc32.ieee_hasher.up_x86_avx2
name                                                      old speed      new speed      delta

wuffs_crc32_ieee_10k/clang9                               8.42GB/s ± 1%  8.12GB/s ± 1%   -3.55%  (p=0.000 n=9+8)
wuffs_crc32_ieee_100k/clang9                              11.0GB/s ± 1%  10.9GB/s ± 1%     ~     (p=0.931 n=9+9)

wuffs_crc32_ieee_10k/gcc10                                9.20GB/s ± 0%  9.71GB/s ± 0%   +5.54%  (p=0.000 n=8+10)
wuffs_crc32_ieee_100k/gcc10                               11.9GB/s ± 2%  13.1GB/s ± 1%  +10.22%  (p=0.000 n=10+8)

wuffs_gzip_decode_10k/clang9                               219MB/s ± 1%   218MB/s ± 2%     ~     (p=0.497 n=9+10)
wuffs_gzip_decode_100k/clang9                              283MB/s ± 2%   281MB/s ± 1%     ~     (p=0.063 n=10+10)

wuffs_gzip_decode_10k/gcc10                                224MB/s ± 1%   223MB/s ± 1%     ~     (p=0.489 n=9+9)
wuffs_gzip_decode_100k/gcc10                               285MB/s ± 4%   290MB/s ± 0%   +1.68%  (p=0.004 n=10+8)

wuffs_png_decode_image_19k_8bpp/clang9                     138MB/s ± 2%   138MB/s ± 1%     ~     (p=0.156 n=10+9)
wuffs_png_decode_image_40k_24bpp/clang9                    171MB/s ± 2%   172MB/s ± 1%     ~     (p=0.971 n=10+10)
wuffs_png_decode_image_77k_8bpp/clang9                     495MB/s ± 4%   504MB/s ± 1%   +1.79%  (p=0.000 n=9+9)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9   457MB/s ± 4%   463MB/s ± 0%   +1.35%  (p=0.002 n=9+9)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9   435MB/s ± 5%   445MB/s ± 1%   +2.31%  (p=0.000 n=10+9)
wuffs_png_decode_image_4002k_24bpp/clang9                  172MB/s ± 2%   174MB/s ± 1%     ~     (p=0.053 n=10+9)

wuffs_png_decode_image_19k_8bpp/gcc10                      157MB/s ± 1%   157MB/s ± 1%     ~     (p=0.549 n=9+10)
wuffs_png_decode_image_40k_24bpp/gcc10                     187MB/s ± 1%   186MB/s ± 1%     ~     (p=0.182 n=10+9)
wuffs_png_decode_image_77k_8bpp/gcc10                      554MB/s ± 1%   559MB/s ± 0%   +0.84%  (p=0.000 n=9+9)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10    502MB/s ± 1%   500MB/s ± 0%   -0.38%  (p=0.011 n=10+10)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10    481MB/s ± 1%   478MB/s ± 0%   -0.46%  (p=0.002 n=8+9)
wuffs_png_decode_image_4002k_24bpp/gcc10                   189MB/s ± 2%   190MB/s ± 0%   +0.62%  (p=0.008 n=10+9)
2021-04-09 23:12:32 +10:00
Nigel Tao a7ef82f7db Add deflate.decoder.decode_huffman_bmi2
Binary size, before:
21864 gen/lib/c/clang-9-dynamic/wuffs-std-deflate.lo
21488 gen/lib/c/clang-9-static/wuffs-std-deflate.o
21136 gen/lib/c/gcc-dynamic/wuffs-std-deflate.lo
21536 gen/lib/c/gcc-static/wuffs-std-deflate.o

After:
24592 gen/lib/c/clang-9-dynamic/wuffs-std-deflate.lo
24368 gen/lib/c/clang-9-static/wuffs-std-deflate.o
23320 gen/lib/c/gcc-dynamic/wuffs-std-deflate.lo
23928 gen/lib/c/gcc-static/wuffs-std-deflate.o

I've upgraded my kernel and compilers recently, so the absolute Wuffs
benchmark numbers below aren't directly comparable to those reported in
previous commits, but the relative changes still seems valid.

On a mid-range x86_64 laptop:

name                                                      old speed      new speed      delta

wuffs_deflate_decode_1k_full_init/clang9                   107MB/s ± 1%   104MB/s ± 1%   -2.36%  (p=0.000 n=8+9)
wuffs_deflate_decode_1k_part_init/clang9                   123MB/s ± 1%   121MB/s ± 1%   -1.78%  (p=0.000 n=10+9)
wuffs_deflate_decode_10k_full_init/clang9                  221MB/s ± 2%   223MB/s ± 1%   +0.86%  (p=0.035 n=10+9)
wuffs_deflate_decode_10k_part_init/clang9                  228MB/s ± 0%   230MB/s ± 1%   +0.59%  (p=0.003 n=8+10)
wuffs_deflate_decode_100k_just_one_read/clang9             276MB/s ± 0%   279MB/s ± 1%   +1.12%  (p=0.001 n=8+9)
wuffs_deflate_decode_100k_many_big_reads/clang9            174MB/s ± 1%   178MB/s ± 5%   +2.38%  (p=0.028 n=9+10)

wuffs_deflate_decode_1k_full_init/gcc10                    103MB/s ± 3%   103MB/s ± 1%     ~     (p=0.105 n=10+10)
wuffs_deflate_decode_1k_part_init/gcc10                    123MB/s ± 1%   122MB/s ± 1%   -0.94%  (p=0.001 n=9+10)
wuffs_deflate_decode_10k_full_init/gcc10                   216MB/s ± 2%   235MB/s ± 1%   +8.83%  (p=0.000 n=10+10)
wuffs_deflate_decode_10k_part_init/gcc10                   223MB/s ± 1%   242MB/s ± 1%   +8.45%  (p=0.000 n=9+9)
wuffs_deflate_decode_100k_just_one_read/gcc10              270MB/s ± 4%   300MB/s ± 4%  +11.08%  (p=0.000 n=9+10)
wuffs_deflate_decode_100k_many_big_reads/gcc10             179MB/s ± 1%   193MB/s ± 2%   +7.97%  (p=0.000 n=10+10)

wuffs_gzip_decode_10k/clang9                               218MB/s ± 3%   220MB/s ± 1%     ~     (p=0.549 n=10+9)
wuffs_gzip_decode_100k/clang9                              270MB/s ± 0%   269MB/s ± 2%     ~     (p=0.730 n=9+9)

wuffs_gzip_decode_10k/gcc10                                208MB/s ± 1%   227MB/s ± 1%   +8.87%  (p=0.000 n=9+10)
wuffs_gzip_decode_100k/gcc10                               267MB/s ± 1%   283MB/s ± 1%   +6.05%  (p=0.000 n=10+9)

wuffs_zlib_decode_10k/clang9                               214MB/s ± 2%   226MB/s ± 1%   +5.69%  (p=0.000 n=10+9)
wuffs_zlib_decode_100k/clang9                              259MB/s ± 6%   272MB/s ± 2%   +5.05%  (p=0.000 n=9+10)

wuffs_zlib_decode_10k/gcc10                                211MB/s ± 1%   227MB/s ± 1%   +7.53%  (p=0.000 n=10+10)
wuffs_zlib_decode_100k/gcc10                               262MB/s ± 1%   282MB/s ± 1%   +7.69%  (p=0.000 n=10+9)

wuffs_png_decode_image_19k_8bpp/clang9                     144MB/s ± 0%   148MB/s ± 1%   +2.68%  (p=0.000 n=9+9)
wuffs_png_decode_image_40k_24bpp/clang9                    166MB/s ± 1%   170MB/s ± 1%   +2.38%  (p=0.000 n=9+9)
wuffs_png_decode_image_77k_8bpp/clang9                     515MB/s ± 1%   525MB/s ± 1%   +1.91%  (p=0.000 n=9+8)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/clang9   401MB/s ± 1%   400MB/s ± 2%     ~     (p=0.913 n=9+9)
wuffs_png_decode_image_552k_32bpp_verify_checksum/clang9   387MB/s ± 1%   388MB/s ± 1%     ~     (p=0.546 n=9+9)
wuffs_png_decode_image_4002k_24bpp/clang9                  170MB/s ± 3%   175MB/s ± 1%   +3.02%  (p=0.000 n=9+10)

wuffs_png_decode_image_19k_8bpp/gcc10                      143MB/s ± 2%   154MB/s ± 3%   +8.32%  (p=0.000 n=9+9)
wuffs_png_decode_image_40k_24bpp/gcc10                     174MB/s ± 2%   182MB/s ± 1%   +4.42%  (p=0.000 n=10+9)
wuffs_png_decode_image_77k_8bpp/gcc10                      513MB/s ± 2%   552MB/s ± 3%   +7.66%  (p=0.000 n=10+9)
wuffs_png_decode_image_552k_32bpp_ignore_checksum/gcc10    408MB/s ± 1%   426MB/s ± 2%   +4.43%  (p=0.000 n=9+10)
wuffs_png_decode_image_552k_32bpp_verify_checksum/gcc10    390MB/s ± 4%   410MB/s ± 2%   +5.05%  (p=0.000 n=10+9)
wuffs_png_decode_image_4002k_24bpp/gcc10                   177MB/s ± 1%   186MB/s ± 1%   +5.13%  (p=0.000 n=9+9)
2021-04-09 00:06:33 +10:00
Nigel Tao c86ac5baef Add wuffs_base__table__flattened_length 2021-04-03 11:01:46 +11:00
Nigel Tao 314d265fe8 Spin out wuffs_aux Alloc{Pixbuf,Workbuf}Result 2021-04-02 23:57:18 +11:00
Nigel Tao 97e30c29da Add some more #pragma's for GCC to ignore warnings 2021-04-02 19:21:54 +11:00
Nigel Tao 38d7548718 Fix WUFFS_BASE__TABLE comment typo 2021-04-01 23:15:18 +11:00
Nigel Tao bb2e9e90b5 Change cgen marker symbol from "!!" to "¡" 2021-03-25 22:17:26 +11:00
Nigel Tao 8d90971782 Bump "MSVC+X64 needs /arch:AVX" from error to warn 2021-03-18 12:09:22 +11:00
Nigel Tao cbc072e776 Include <immintrin.h> etc for MSVC+X64 2021-03-18 11:50:05 +11:00
Nigel Tao 9914f7314d Add WUFFS_BASE__MAYBE_ATTRIBUTE_TARGET 2021-03-17 12:03:06 +11:00
Nigel Tao e16fbafdca Add TODOs for using MSVC intrinsics 2021-03-10 14:22:11 +11:00
Nigel Tao 2dfeee722c Add wuffs_base__cpu_arch__have_x86_sse42 for MSC 2021-03-09 00:15:35 +11:00
Nigel Tao a0228720e3 Have wuffs_aux's CopyIn compact the dst buffer 2021-03-04 16:02:56 +11:00
Nigel Tao 6050cbd319 Add wuffs_base__magic_number_guess_fourcc 2021-02-27 13:45:57 +11:00
Nigel Tao ce97a7ccd9 Rename wuffs_base__cpu_arch__have_sse42 2021-02-18 13:44:33 +11:00
Nigel Tao bbc720c261 Delete wuffs_base__cpu_arch__x86_64__capabilities 2021-02-18 13:30:28 +11:00
Nigel Tao 3338b5eafa Add arm_crc32,arm_neon cpu_arch conditions 2021-02-13 17:01:09 +11:00