Commit Graph
128 Commits
Author SHA1 Message Date
Nigel Tao 4e55710afc Call the set_limit method 2018-04-28 14:36:16 +10:00
Nigel Tao a2cf739e52 Make an io_{reader,writer}'s buffer private 2018-04-27 21:02:48 +10:00
Nigel Tao 8827e222b1 Add wuffs_base__io_buffer__{reader,writer} funcs 2018-04-27 20:54:44 +10:00
Nigel Tao eecdaba4e8 Rename crc32.ieee to crc32.ieee_hasher 2018-04-27 20:23:45 +10:00
Nigel Tao 035b138cb1 Spin out a std/adler32 package 2018-04-27 17:20:10 +10:00
Nigel Tao 3ab749be0d Test checksumming fragmented input 2018-04-26 13:11:05 +10:00
Nigel Tao 97aab37462 Rename buf1, reader1, etc to io_buffer, io_reader
Also remove the unused Buf1, Buf2 and Usize tokens.
2018-04-05 20:49:48 +10:00
Nigel Tao 82a48796c8 Add pixel_format, image_buffer, etc. definitions 2018-04-05 17:09:33 +10:00
Nigel Tao ef899b0492 Make gzip checksum check length, not just CRC32 2018-03-28 12:36:31 +11:00
Nigel Tao a8e40fedc9 Implement the gzip checksum
Also fix a bug in ast.Raw.SetPackage, discovered because type checking
failed to show that "[] u8" and "[] u8" are equivalent types.
2018-03-28 12:30:00 +11:00
Nigel Tao 86d0ade206 Implement gzip decode 2018-03-28 11:10:51 +11:00
Nigel Tao 9cc6964c30 Stub out a new std/gzip package 2018-03-28 10:22:46 +11:00
Nigel Tao 061729f7b2 Remove redundant crc32 benchmarks 2018-03-25 22:02:13 +11:00
Nigel Tao a2949fe28d Add Adler32 and CRC32/IEEE tests for pi's digits 2018-03-25 21:54:07 +11:00
Nigel Tao 3a7ccfe9b4 Implement crc32's ieee.update 2018-03-25 18:09:51 +11:00
Nigel Tao 0ebfb75699 Add crc32 ieee test cases 2018-03-25 17:20:31 +11:00
Nigel Tao 008bce65f2 Have crc32's ieee.update return u32 2018-03-25 17:04:14 +11:00
Nigel Tao 917737ec97 Stub out a new std/crc32 package 2018-03-25 16:58:23 +11:00
Nigel Tao 4675efdcc5 Fix deflate decode when the distance is 32768 2018-03-23 12:49:42 +11:00
Nigel Tao 4806f6160d Run clang-format-5.0 2018-03-23 09:51:13 +11:00
Nigel Tao f11e1f97c6 Add deflate-distance-32768 test case 2018-03-21 12:32:51 +11:00
Nigel Tao d1399fac19 Rename test/testdata to test/data 2018-03-16 10:48:33 +11:00
Nigel Tao ea2dec09df Test read_uxxbe across an I/O short read 2018-02-22 12:23:52 +11:00
Nigel Tao 446d4662db Let the zlib decoder ignore the checksum 2018-02-16 08:51:28 +11:00
Nigel Tao fb2430e765 Add some GIF per-frame fields 2018-02-15 12:41:13 +11:00
Nigel Tao 0d33d3ef3a Decode a GIF local color table 2018-02-11 15:30:18 +11:00
Nigel Tao 584fc631a8 Decode a GIF's loop count 2018-02-10 17:14:49 +11:00
Nigel Tao ded38a043a Add testdata/animated-red-blue.gif 2018-02-10 15:09:29 +11:00
Nigel Tao d3770b9972 Add an end of data concept 2018-02-10 12:13:26 +11:00
Nigel Tao 7e90967717 Fill in a GIF image_config 2018-02-09 16:05:04 +11:00
Nigel Tao d750ea7653 Add an image_config concept 2018-02-09 14:46:37 +11:00
Nigel Tao de9000cc70 Add a call sequence concept 2018-02-08 09:06:19 +11:00
Nigel Tao f5c6204f09 Strip the prefix from base status messages 2017-12-27 14:45:54 +11:00
Nigel Tao 7e47f4cc7a Delegate etc__status__string to used packages 2017-12-27 13:49:05 +11:00
Nigel Tao 4877279ee7 Be able to test and bench the miniz library
For the record, benchstat numbers for 'old' being zlib-the-library and
'new' being miniz are below. Neither 'old' or 'new' is Wuffs code.

name                             old speed     new speed     delta
mimic_deflate_decode_1k/clang    217MB/s ± 1%  175MB/s ± 1%  -19.54%  (p=0.008 n=5+5)
mimic_deflate_decode_10k/clang   262MB/s ± 2%  222MB/s ± 0%  -15.34%  (p=0.008 n=5+5)
mimic_deflate_decode_100k/clang  281MB/s ± 1%  262MB/s ± 0%   -6.86%  (p=0.008 n=5+5)
mimic_deflate_decode_1k/gcc      220MB/s ± 1%  169MB/s ± 0%  -23.12%  (p=0.008 n=5+5)
mimic_deflate_decode_10k/gcc     267MB/s ± 0%  266MB/s ± 0%     ~     (p=0.222 n=5+5)
mimic_deflate_decode_100k/gcc    283MB/s ± 0%  282MB/s ± 1%     ~     (p=0.690 n=5+5)
mimic_zlib_decode_10k/clang      243MB/s ± 1%  197MB/s ± 1%  -18.68%  (p=0.008 n=5+5)
mimic_zlib_decode_100k/clang     287MB/s ± 1%  229MB/s ± 1%  -20.13%  (p=0.008 n=5+5)
mimic_zlib_decode_10k/gcc        243MB/s ± 0%  240MB/s ± 6%     ~     (p=0.730 n=5+5)
mimic_zlib_decode_100k/gcc       288MB/s ± 1%  260MB/s ± 0%   -9.51%  (p=0.008 n=5+5)
2017-12-15 11:33:35 +11:00
Nigel Tao 2e12a41414 Make std/gif's LZW decoder private 2017-12-08 16:18:35 +11:00
Nigel Tao afb879b000 Rename deflate-related testdata 2017-12-04 10:49:11 +11:00
Nigel Tao 8e2255916e Drop the "flate_" in "deflate.flate_decoder" 2017-12-04 10:35:08 +11:00
Nigel Tao 1388f84e5c Rename std/flate to std/deflate 2017-12-04 10:27:48 +11:00
Nigel Tao d168e6977b Rename test/c/mimiclib/flate.c 2017-12-03 14:34:41 +11:00
Nigel Tao 7e73a5ac5f Call initializers for out-of-pkg struct fields 2017-12-03 14:22:49 +11:00
Nigel Tao 6fec22c6d7 Move zlib tests from std/flate to std/zlib 2017-12-03 14:08:52 +11:00
Nigel Tao dd3f0b9ed4 Add a placeholder std/zlib package 2017-12-02 11:21:30 +11:00
Nigel Tao 56a2a926c5 Add comment about inlining C relative includes 2017-12-01 11:36:12 +11:00
Nigel Tao 3c708dfe59 Rename PUFFS to WUFFS, part 7 of N 2017-11-30 18:02:59 +11:00
Nigel Tao 79a94558be Rename PUFFS to WUFFS, part 5 of N 2017-11-30 16:37:20 +11:00
Nigel Tao 1910ca31cf Run the mimic checksum benchmarks for longer. 2017-11-08 12:46:57 +11:00
Nigel Tao 13731f989f Add a reader1.unread_u8 method.
Also re-enable a number of std/flate tests and benchmarks.

It's not an optimization per se, but as a data point, here are some
benchmark numbers between 36a93fa and this commit. 36a93fa is "36a93fa
Add a TODO about impure functions", which is one commit before "f968950
Read bits up front in decode_huffman_fast", and f968950 was the commit
that made decode_huffman_fast read more than one byte at a time.

name                             old speed      new speed       delta

puffs_adler32_10k/clang          2.45GB/s ± 0%   2.45GB/s ± 0%     ~     (p=0.684 n=10+10)
puffs_adler32_100k/clang         2.45GB/s ± 0%   2.46GB/s ± 0%   +0.07%  (p=0.019 n=10+10)
puffs_flate_decode_1k/clang       124MB/s ± 0%    132MB/s ± 0%   +6.06%  (p=0.000 n=10+10)
puffs_flate_decode_10k/clang      145MB/s ± 0%    174MB/s ± 0%  +20.19%  (p=0.000 n=9+10)
puffs_flate_decode_100k/clang     168MB/s ± 0%    208MB/s ± 0%  +23.31%  (p=0.000 n=10+10)
puffs_zlib_decode_10k/clang       136MB/s ± 0%    162MB/s ± 0%  +18.94%  (p=0.000 n=10+9)
puffs_zlib_decode_100k/clang      158MB/s ± 0%    192MB/s ± 0%  +21.58%  (p=0.000 n=10+9)

puffs_adler32_10k/gcc            3.26GB/s ± 0%   3.26GB/s ± 0%     ~     (p=0.631 n=10+10)
puffs_adler32_100k/gcc           3.27GB/s ± 0%   3.27GB/s ± 0%     ~     (p=0.237 n=10+8)
puffs_flate_decode_1k/gcc        95.3MB/s ± 0%  105.5MB/s ± 0%  +10.67%  (p=0.000 n=10+10)
puffs_flate_decode_10k/gcc        110MB/s ± 0%    140MB/s ± 0%  +27.32%  (p=0.000 n=9+9)
puffs_flate_decode_100k/gcc       121MB/s ± 0%    150MB/s ± 0%  +24.02%  (p=0.000 n=8+10)
puffs_zlib_decode_10k/gcc         112MB/s ± 0%    120MB/s ± 0%   +6.96%  (p=0.000 n=10+9)
puffs_zlib_decode_100k/gcc        123MB/s ± 0%    134MB/s ± 0%   +8.93%  (p=0.000 n=9+8)
2017-11-05 15:59:02 +11:00
Nigel Tao f6e072fe77 Add some more "inconsistent n_bits" checks.
This uncovered a bug in the decode_huffman_fast function. A follow up commit
will actually fix the bug.

For what it's worth, some benchmark numbers are below, but they're
possibly bogus, as we might be decoding incorrect (and truncated) output
for some inputs. Everything since "f968950 Read bits up front in
decode_huffman_fast" landed possibly has bogus numbers too.

name                           old speed     new speed     delta
puffs_flate_decode_100k/clang  194MB/s ± 0%  204MB/s ± 1%   +5.02%  (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc    180MB/s ± 3%  202MB/s ± 1%  +12.09%  (p=0.008 n=5+5)
2017-11-05 09:23:49 +11:00
Nigel Tao cf61ca11c3 Run the adler benchmarks for longer.
A future commit could set each benchmark's iteration count (call it N)
dynamically, but for now, the iteration counts are hard-coded. The rough
guiding principle is those N iterations should take more or less one
second, on my modern desktop CPU.

Prior to this commit, the adler benchmarks ran for the same N as the
zlib benchmarks. Each zlib decode involves exactly one adler checksum
computation, so both having the same N makes it very easy to calculate a
rough estimate of what fraction of zlib decoding is spent in adler code.
But in having the same N, the adler benchmarks run for a tiny fraction
of a second (and would run for an even tinier fraction if we further
optimize the adler code, e.g. with SIMD).

This commit raises that N for the adler benchmarks (but not for the zlib
benchmarks), which should hopefully make the adler numbers a little more
reliable. It will make that very easy calculation slightly less easy,
but it'd still be trivial.

The benchmark numbers do change. I'm not entirely sure why, but keep in
mind that this commit only changes the benchmark harness code, not the
actual std/flate code running in that harness.

name                           old speed      new speed      delta

puffs_adler32_10k/clang        2.45GB/s ± 0%  2.40GB/s ± 1%  -2.00%  (p=0.008 n=5+5)
puffs_adler32_100k/clang       2.44GB/s ± 0%  2.40GB/s ± 1%  -1.67%  (p=0.008 n=5+5)
puffs_flate_decode_1k/clang     133MB/s ± 0%   131MB/s ± 1%  -1.37%  (p=0.008 n=5+5)
puffs_flate_decode_10k/clang    174MB/s ± 0%   171MB/s ± 1%  -1.66%  (p=0.008 n=5+5)
puffs_flate_decode_100k/clang   205MB/s ± 0%   202MB/s ± 0%  -1.28%  (p=0.016 n=5+4)
puffs_zlib_decode_10k/clang     162MB/s ± 0%   159MB/s ± 1%  -1.94%  (p=0.008 n=5+5)
puffs_zlib_decode_100k/clang    188MB/s ± 0%   185MB/s ± 1%  -1.60%  (p=0.008 n=5+5)

puffs_adler32_10k/gcc          3.25GB/s ± 0%  3.26GB/s ± 0%  +0.25%  (p=0.016 n=4+5)
puffs_adler32_100k/gcc         3.25GB/s ± 0%  3.27GB/s ± 0%  +0.39%  (p=0.008 n=5+5)
puffs_flate_decode_1k/gcc       132MB/s ± 0%   132MB/s ± 0%    ~     (p=0.151 n=5+5)
puffs_flate_decode_10k/gcc      170MB/s ± 0%   170MB/s ± 0%    ~     (p=0.738 n=5+5)
puffs_flate_decode_100k/gcc     207MB/s ± 0%   207MB/s ± 0%    ~     (p=0.841 n=5+5)
puffs_zlib_decode_10k/gcc       161MB/s ± 1%   162MB/s ± 0%    ~     (p=0.841 n=5+5)
puffs_zlib_decode_100k/gcc      194MB/s ± 0%   194MB/s ± 0%    ~     (p=0.841 n=5+5)
2017-11-05 07:16:04 +11:00