Commit Graph
102 Commits
Author SHA1 Message Date
Nigel Tao 8bbdcc4ce1 Allow scalar constants in type expressions 2019-02-02 07:53:59 +11:00
Nigel Tao ecccafab09 Add check for interleaved coroutines
It's not an optimization per se, but it seems to help:

name                                             old speed      new speed      delta

wuffs_deflate_decode_1k/clang5                    138MB/s ± 1%   150MB/s ± 1%   +8.96%  (p=0.008 n=5+5)
wuffs_deflate_decode_10k/clang5                   223MB/s ± 0%   243MB/s ± 0%   +9.25%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_just_one_read/clang5    263MB/s ± 2%   289MB/s ± 0%  +10.19%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/clang5   220MB/s ± 1%   243MB/s ± 0%  +10.42%  (p=0.008 n=5+5)

wuffs_deflate_decode_1k/gcc7                      158MB/s ± 0%   157MB/s ± 0%     ~     (p=0.056 n=5+5)
wuffs_deflate_decode_10k/gcc7                     268MB/s ± 0%   268MB/s ± 0%     ~     (p=1.000 n=5+5)
wuffs_deflate_decode_100k_just_one_read/gcc7      312MB/s ± 0%   320MB/s ± 0%   +2.62%  (p=0.008 n=5+5)
wuffs_deflate_decode_100k_many_big_reads/gcc7     254MB/s ± 0%   253MB/s ± 0%   -0.48%  (p=0.016 n=5+5)
2019-01-26 15:33:43 +11:00
Nigel Tao 89fef4cf5f Align cgen's private_impl.placeholder 2019-01-26 09:28:50 +11:00
Nigel Tao 989c611711 Allow C++ methods even when !WUFFS_IMPLEMENTATION
Also disallow copy and assign.
2019-01-19 14:38:19 +11:00
Nigel Tao 20c474b32a Add a genlinenum flag 2018-12-28 16:26:21 +11:00
Nigel Tao 983cebd893 Have cgen zero-initialize all local variables 2018-12-27 11:54:33 +11:00
Nigel Tao d552f88738 Make Iterate contain Assign's not Var's 2018-12-27 10:35:27 +11:00
Nigel Tao 04c91f1410 Generate .c files instead of .h files
The files' contents haven't changed (other than some relative #include's
that refer to other .c files).

Some build systems have difficulty recognizing .h files as things to be
compiled, instead of just things referred to by things to be compiled.
2018-12-22 15:29:00 +11:00
Nigel Tao badbdfc487 Tweak io_bind syntax 2018-12-09 09:13:29 +11:00
Nigel Tao 6c9dca47ee Add a base.pixel_swizzler type 2018-12-08 13:25:53 +11:00
Nigel Tao 86b8408223 Refactor cgen's base/base-{private,public}.h 2018-12-08 10:46:02 +11:00
Nigel Tao b162747934 Remove cgen's obsolete replacementPolicy concept 2018-12-01 22:41:41 +11:00
Nigel Tao 8f1240f20e Mark cgen's replaceNothing as unused 2018-12-01 22:36:32 +11:00
Nigel Tao c84732f3d3 Quote built-in statuses 2018-11-24 13:42:18 +11:00
Nigel Tao 9e5aec5d25 Change "$foo" to base."$foo" 2018-11-24 13:36:09 +11:00
Nigel Tao 6ebe99ced9 Don't suspend/resume unnecessary local variables
name                                    old speed      new speed      delta

wuffs_adler32_10k/clang                 2.42GB/s ± 1%  2.41GB/s ± 0%  -0.43%  (p=0.000 n=49+10)
wuffs_adler32_100k/clang                2.42GB/s ± 1%  2.41GB/s ± 1%  -0.56%  (p=0.000 n=48+10)

wuffs_adler32_10k/gcc                   3.22GB/s ± 1%  3.21GB/s ± 0%  -0.35%  (p=0.000 n=48+10)
wuffs_adler32_100k/gcc                  3.22GB/s ± 0%  3.20GB/s ± 0%  -0.69%  (p=0.000 n=48+10)

wuffs_crc32_ieee_10k/clang              2.86GB/s ± 0%  2.85GB/s ± 0%  -0.44%  (p=0.000 n=48+9)
wuffs_crc32_ieee_100k/clang             2.88GB/s ± 1%  2.87GB/s ± 0%  -0.45%  (p=0.000 n=49+10)

wuffs_crc32_ieee_10k/gcc                3.35GB/s ± 1%  3.35GB/s ± 0%    ~     (p=0.094 n=48+9)
wuffs_crc32_ieee_100k/gcc               3.38GB/s ± 1%  3.38GB/s ± 1%    ~     (p=0.371 n=45+10)

wuffs_deflate_decode_1k/clang            147MB/s ± 1%   137MB/s ± 1%  -6.84%  (p=0.000 n=50+10)
wuffs_deflate_decode_10k/clang           241MB/s ± 0%   227MB/s ± 0%  -5.74%  (p=0.000 n=48+9)
wuffs_deflate_decode_100k/clang          269MB/s ± 0%   258MB/s ± 0%  -3.92%  (p=0.000 n=49+9)

wuffs_deflate_decode_1k/gcc              159MB/s ± 0%   161MB/s ± 1%  +1.51%  (p=0.000 n=49+10)
wuffs_deflate_decode_10k/gcc             269MB/s ± 0%   276MB/s ± 0%  +2.51%  (p=0.000 n=49+10)
wuffs_deflate_decode_100k/gcc            320MB/s ± 0%   316MB/s ± 0%  -1.29%  (p=0.000 n=46+10)

wuffs_gif_decode_1k_bw/clang             252MB/s ± 1%   264MB/s ± 0%  +4.56%  (p=0.000 n=49+10)
wuffs_gif_decode_1k_color/clang          127MB/s ± 1%   122MB/s ± 1%  -3.84%  (p=0.000 n=47+10)
wuffs_gif_decode_10k/clang               152MB/s ± 0%   148MB/s ± 0%  -3.12%  (p=0.000 n=47+9)
wuffs_gif_decode_100k_artificial/clang   431MB/s ± 0%   443MB/s ± 1%  +2.95%  (p=0.000 n=42+10)
wuffs_gif_decode_100k_realistic/clang    170MB/s ± 0%   173MB/s ± 0%  +1.39%  (p=0.000 n=48+10)
wuffs_gif_decode_1000k/clang             172MB/s ± 0%   174MB/s ± 0%  +1.27%  (p=0.000 n=48+10)
wuffs_gif_decode_anim_screencap/clang    917MB/s ± 1%   949MB/s ± 0%  +3.44%  (p=0.000 n=47+9)

wuffs_gif_decode_1k_bw/gcc               265MB/s ± 1%   278MB/s ± 2%  +4.87%  (p=0.000 n=48+10)
wuffs_gif_decode_1k_color/gcc            135MB/s ± 0%   140MB/s ± 2%  +4.10%  (p=0.000 n=46+10)
wuffs_gif_decode_10k/gcc                 157MB/s ± 1%   166MB/s ± 0%  +5.84%  (p=0.000 n=48+8)
wuffs_gif_decode_100k_artificial/gcc     446MB/s ± 1%   459MB/s ± 1%  +2.89%  (p=0.000 n=48+10)
wuffs_gif_decode_100k_realistic/gcc      176MB/s ± 0%   186MB/s ± 2%  +5.82%  (p=0.000 n=46+10)
wuffs_gif_decode_1000k/gcc               178MB/s ± 0%   189MB/s ± 1%  +6.11%  (p=0.000 n=45+9)
wuffs_gif_decode_anim_screencap/gcc      962MB/s ± 1%   989MB/s ± 1%  +2.83%  (p=0.000 n=48+9)

wuffs_gzip_decode_10k/clang              212MB/s ± 0%   208MB/s ± 0%  -1.67%  (p=0.000 n=47+10)
wuffs_gzip_decode_100k/clang             240MB/s ± 0%   236MB/s ± 0%  -1.68%  (p=0.000 n=47+10)

wuffs_gzip_decode_10k/gcc                252MB/s ± 0%   246MB/s ± 0%  -2.41%  (p=0.000 n=47+10)
wuffs_gzip_decode_100k/gcc               290MB/s ± 0%   288MB/s ± 0%  -0.70%  (p=0.000 n=44+10)

wuffs_lzw_decode_20k/clang               202MB/s ± 0%   202MB/s ± 0%    ~     (p=0.069 n=48+8)
wuffs_lzw_decode_100k/clang              332MB/s ± 1%   334MB/s ± 1%  +0.54%  (p=0.000 n=46+10)

wuffs_lzw_decode_20k/gcc                 215MB/s ± 1%   223MB/s ± 0%  +3.60%  (p=0.000 n=49+9)
wuffs_lzw_decode_100k/gcc                361MB/s ± 1%   368MB/s ± 0%  +2.05%  (p=0.000 n=47+10)

wuffs_zlib_decode_10k/clang              218MB/s ± 0%   206MB/s ± 0%  -5.53%  (p=0.000 n=48+9)
wuffs_zlib_decode_100k/clang             244MB/s ± 0%   234MB/s ± 0%  -4.14%  (p=0.000 n=45+10)

wuffs_zlib_decode_10k/gcc                252MB/s ± 0%   252MB/s ± 0%    ~     (p=0.960 n=49+10)
wuffs_zlib_decode_100k/gcc               290MB/s ± 0%   292MB/s ± 0%  +0.52%  (p=0.000 n=49+10)
2018-11-10 17:04:40 +11:00
Nigel Tao 8eb9781a9d Have gen/c code use relative includes
For example, gen/c/wuffs-std-gif.h now #include's "./wuffs-std-lzw.h"
instead of "std/lzw.h", which removes the need to configure an -I
directory when compiling it.
2018-11-03 11:03:11 +11:00
Nigel Tao 61d03537c2 De-duplicate the gen/c/* files
The gen/c/* files are not checked in, due to the .gitignore file

In Wuffs version 0.1, they were checked in, and programs were expected
to #include them directly. Another goal was for them to be
self-contained, without any #include lines to other Wuffs files, so that
they would be easy to build, without having to configure an -I dir to a
C compiler.

They therefore inlined what would otherwise be an #include. For example,
the generated std/gif.h code would inline the contents of the std/lzw.h
file. The genrelease program would have to de-duplicate these inlines in
order to produce a clean, monolithic release/c/* file.

In Wuffs version 0.2 (the latest version), programs don't #include
gen/c/*. Instead, they #include the monolithic release/c/*. Avoiding the
need to configure an -I directory isn't as important as it used to be.

This change removes that inlining, using the simpler #include line, and
the genrelease program is then simpler because it doesn't need to
de-duplicate.
2018-10-27 12:50:01 +11:00
Nigel Tao 51a0b347ed Add WUFFS_BASE__LOW_BITS_MASK__UXX macros 2018-10-21 21:53:44 +11:00
Nigel Tao 3338d6d1bc Write LZW output 8 bytes at a time, not 1
The benchmarks are mixed, but I think it's still a net win.

----
name                                    old speed     new speed     delta

wuffs_gif_decode_1k_bw/clang            258MB/s ± 0%  253MB/s ± 1%    -2.04%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/clang         159MB/s ± 0%  116MB/s ± 0%   -26.76%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/clang              167MB/s ± 0%  138MB/s ± 0%   -17.42%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang  205MB/s ± 0%  439MB/s ± 0%  +114.53%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang   151MB/s ± 0%  164MB/s ± 0%    +8.41%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k/clang            155MB/s ± 0%  165MB/s ± 0%    +6.50%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang   255MB/s ± 0%  921MB/s ± 0%  +261.50%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc              316MB/s ± 2%  267MB/s ± 1%   -15.78%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color/gcc           184MB/s ± 1%  134MB/s ± 0%   -27.57%  (p=0.008 n=5+5)
wuffs_gif_decode_10k/gcc                196MB/s ± 0%  161MB/s ± 0%   -17.50%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc    287MB/s ± 0%  449MB/s ± 1%   +56.22%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc     182MB/s ± 0%  182MB/s ± 1%      ~     (p=0.690 n=5+5)
wuffs_gif_decode_1000k/gcc              187MB/s ± 1%  185MB/s ± 1%      ~     (p=0.056 n=5+5)
wuffs_gif_decode_anim_screencap/gcc     397MB/s ± 0%  973MB/s ± 0%  +145.42%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/clang              170MB/s ± 0%  191MB/s ± 1%   +12.52%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/clang             242MB/s ± 0%  312MB/s ± 2%   +29.00%  (p=0.008 n=5+5)

wuffs_lzw_decode_20k/gcc                220MB/s ± 0%  227MB/s ± 0%    +3.31%  (p=0.008 n=5+5)
wuffs_lzw_decode_100k/gcc               317MB/s ± 0%  369MB/s ± 1%   +16.53%  (p=0.008 n=5+5)
----

LZW is the compression method used by GIF images. Roughly speaking, LZW
is a sequence of codes, and each code emits either 1 byte (a literal
code) or multiple bytes (a copy code). The speed gain in this commit
comes from, when emitting an N-byte copy code, this now requires N/8
iterations instead of N iterations, as each iteration now writes 8 bytes
instead of 1. Literal codes, both before and after this commit, requires
a single iteration.

Decoding should therefore get faster or slower depending on whether it's
copy-code-heavy or literal-code-heavy respectively. For what it's worth,
copy-code-heavy GIF images also usually compress better.

Running script/extract-giflzw.go (or examining the cached output at
test/data/analysis/giflzw-emission-lengths.txt) on the anim_screencap
(which got faster) and 1k_color (which got slower) cases (also known as
gifplayer-muybridge.gif and hippopotamus.regular.gif) shows that the
proportion of literal codes agrees with that explanation.

----
gifplayer-muybridge.gif (all frames)
 Percent   Count  Emission_Length_Bucket
   12.78   34658           1 (Literal Code)
   14.05   38106           2
   13.25   35937           3
   10.59   28707           4
    7.85   21286           5
    6.08   16481           6
    4.85   13156           7
    3.60    9764           8
    9.34   25318    9 ..  16
    4.89   13257   17 ..  32
    4.78   12954   33 ..  64
    4.72   12806   65 .. 128
    3.02    8190  127 .. 256
    0.20     550  256+

hippopotamus.regular.gif (all frames)
 Percent   Count  Emission_Length_Bucket
   82.66     672           1 (Literal Code)
   13.04     106           2
    2.83      23           3
    0.86       7           4
    0.37       3           5
    0.25       2           6
    0.00       0           7
    0.00       0           8
    0.00       0    9 ..  16
    0.00       0   17 ..  32
    0.00       0   33 ..  64
    0.00       0   65 .. 128
    0.00       0  127 .. 256
    0.00       0  256+
----

The new algorithm also adds more CPU and RAM overhead, regardless of
whether the input GIF image's LZW data is copy-code-heavy or
literal-code-heavy. For example, the suffix table grows by 8x, from 4KiB
to 32 KiB.

The gifplayer-muybridge.gif image is also larger (472 x 298) and the
hippopotamus.regular.gif image is also smaller (36 x 28). Higher fixed
costs may have larger (negative) relative effects on decoding smaller
images.

To repeat, I think it's still a net win.

An upcoming commit will update the std/lzw/README.md file to provide
more detail on the algorithm.

Thanks to Tushar Khandelwal for the idea.
2018-10-21 14:48:40 +11:00
Nigel Tao 8b1ffec576 Make struct impls private (opaque pointers)
This gives us flexibility to change sizeof(wuffs_foo__bar__struct)
across Wuffs versions, while retaining ABI / API compatibility.

In particular, this lets us add and remove fields.
2018-10-06 10:33:47 +10:00
Nigel Tao 26a448450e Move C++ method impls inside struct definitions 2018-10-06 10:06:07 +10:00
Nigel Tao 5b82dd905a Separate C struct declaration and definition 2018-10-06 09:45:03 +10:00
Nigel Tao e26ba24fbf Add wuffs_foo__bar__struct names 2018-10-06 09:41:50 +10:00
Nigel Tao dc17398d64 Generate "WUFFS C HEADER ENDS HERE"
Future commits will mean more than one "#ifdef WUFFS_IMPLEMENTATION".
2018-10-06 09:12:04 +10:00
Nigel Tao c4edf6ac25 Add sizeof__wuffs_foo__bar functions 2018-09-23 09:26:08 +10:00
Nigel Tao 9c04249bfe Add warnings as another status code category 2018-08-26 23:22:45 +10:00
Nigel Tao f836d3b743 Rename "check_wuffs_version called twice" 2018-08-19 17:08:36 +10:00
Nigel Tao 9367cd800b Remove the (pointer-typed) private_impl.status 2018-08-19 11:32:31 +10:00
Nigel Tao c1af1e2c82 Rename $(etc) to [etc] 2018-08-18 10:11:43 +10:00
Nigel Tao eec0fce647 Change error "?foo" to status "?foo" 2018-08-12 21:31:56 +10:00
Nigel Tao 203b45e869 Remove status values 2018-08-12 19:36:29 +10:00
Nigel Tao c499f03bc7 Delete obsolete wuffs_foo__status__string funcs 2018-08-12 19:10:24 +10:00
Nigel Tao b840c80040 Make the status type a const char*, not a struct 2018-08-12 16:05:14 +10:00
Nigel Tao e0311e3bb2 Don't generate header defs for private statuses 2018-08-12 14:08:04 +10:00
Nigel Tao b385c30b4f Add a "?" prefix to non-built-in status messages 2018-08-12 14:07:40 +10:00
Nigel Tao 0ed0ead057 Generate named strings for status messages 2018-08-12 11:28:13 +10:00
Nigel Tao 1462084531 Add a "?" or "$" prefix to status messages 2018-08-12 10:50:22 +10:00
Nigel Tao 2a32a86a6c Make the status type a struct, not an int32_t 2018-08-11 13:27:29 +10:00
Nigel Tao 9c0b2d2f5c Rename ioptr_etc cgen variables 2018-08-11 08:32:49 +10:00
Nigel Tao fd43e777bc Have check_wuffs_version return a status code 2018-08-05 22:29:54 +10:00
Nigel Tao 98b2aa9837 Move some Go packages from lang/foo to lib/foo 2018-08-04 09:17:08 +10:00
Nigel Tao db04cf346f Rename Struct.Optional to Struct.Classy 2018-08-04 08:57:27 +10:00
Nigel Tao 5cca9471ec Have cgen look for !?? methods not ? methods 2018-07-29 14:01:57 +10:00
Nigel Tao 0a47966e72 Introduce finer-grained effects 2018-07-29 13:40:49 +10:00
Nigel Tao 4e70498189 Spin out base/image-public.h 2018-07-21 08:02:58 +10:00
Nigel Tao fa95380a23 Have WUFFS_INCLUDE_GUARD cover the IMPLEMENTATION 2018-07-14 11:17:54 +10:00
Nigel Tao d73353024c Make clang-format place func names at line start 2018-07-14 10:33:06 +10:00
Nigel Tao 8a6e177701 Add WUFFS_CONFIG__STATIC_FUNCTIONS 2018-07-14 10:16:10 +10:00
Nigel Tao f2e683ab69 Have gen's "single file C library" be .h not .c 2018-07-14 09:26:10 +10:00