Commit Graph
131 Commits
Author SHA1 Message Date
Nigel Tao 97aab37462 Rename buf1, reader1, etc to io_buffer, io_reader
Also remove the unused Buf1, Buf2 and Usize tokens.
2018-04-05 20:49:48 +10:00
Nigel Tao 82a48796c8 Add pixel_format, image_buffer, etc. definitions 2018-04-05 17:09:33 +10:00
Nigel Tao ef899b0492 Make gzip checksum check length, not just CRC32 2018-03-28 12:36:31 +11:00
Nigel Tao a8e40fedc9 Implement the gzip checksum
Also fix a bug in ast.Raw.SetPackage, discovered because type checking
failed to show that "[] u8" and "[] u8" are equivalent types.
2018-03-28 12:30:00 +11:00
Nigel Tao 153d1276a1 Read checksum bytes even if ignoring its value 2018-03-28 11:21:03 +11:00
Nigel Tao 72449fa0ec Rename checksum fields and variables 2018-03-28 11:15:13 +11:00
Nigel Tao 86d0ade206 Implement gzip decode 2018-03-28 11:10:51 +11:00
Nigel Tao 9cc6964c30 Stub out a new std/gzip package 2018-03-28 10:22:46 +11:00
Nigel Tao 008bce65f2 Have crc32's ieee.update return u32 2018-03-25 17:04:14 +11:00
Nigel Tao 917737ec97 Stub out a new std/crc32 package 2018-03-25 16:58:23 +11:00
Nigel Tao 26445adcdf Rename deflate's "distance minus 1" variable 2018-03-23 12:54:46 +11:00
Nigel Tao abdf46045f Upgrade to clang 5.0
The spurious changes to gen/ will hopefully be fixed by a clang-format
5.x revision.
2018-03-21 13:08:16 +11:00
Nigel Tao 446d4662db Let the zlib decoder ignore the checksum 2018-02-16 08:51:28 +11:00
Nigel Tao f82c33db2f Recognize a GIF "ANIMEXTS1.0" extension 2018-02-15 14:35:01 +11:00
Nigel Tao fb2430e765 Add some GIF per-frame fields 2018-02-15 12:41:13 +11:00
Nigel Tao 0d33d3ef3a Decode a GIF local color table 2018-02-11 15:30:18 +11:00
Nigel Tao 584fc631a8 Decode a GIF's loop count 2018-02-10 17:14:49 +11:00
Nigel Tao d3770b9972 Add an end of data concept 2018-02-10 12:13:26 +11:00
Nigel Tao d750ea7653 Add an image_config concept 2018-02-09 14:46:37 +11:00
Nigel Tao de9000cc70 Add a call sequence concept 2018-02-08 09:06:19 +11:00
Nigel Tao fecd7b631c Re-organize some status codes 2017-12-27 13:15:37 +11:00
Nigel Tao 2e12a41414 Make std/gif's LZW decoder private 2017-12-08 16:18:35 +11:00
Nigel Tao 8e2255916e Drop the "flate_" in "deflate.flate_decoder" 2017-12-04 10:35:08 +11:00
Nigel Tao 2755a61ae9 Rename std/deflate's packageid 2017-12-04 10:29:17 +11:00
Nigel Tao 1388f84e5c Rename std/flate to std/deflate 2017-12-04 10:27:48 +11:00
Nigel Tao 530605a16b Delete std/flate's zlib code (moved to std/zlib) 2017-12-03 14:31:35 +11:00
Nigel Tao 6913a1ff00 Copy/paste zlib code from std/flate to std/zlib 2017-12-03 13:32:42 +11:00
Nigel Tao 0a32c01014 Allow referring to types from used packages 2017-12-03 13:21:23 +11:00
Nigel Tao 9867cd5e3a Generate inline headers for use declarations 2017-12-02 14:54:58 +11:00
Nigel Tao c5b5efd7c8 Prepare for generating use declarations 2017-12-02 14:17:24 +11:00
Nigel Tao dd3f0b9ed4 Add a placeholder std/zlib package 2017-12-02 11:21:30 +11:00
Nigel Tao 849db0faa2 Make C hex constants' capitalization consistent 2017-12-01 15:35:41 +11:00
Nigel Tao 65dcf6a5b9 Add a yield keyword 2017-12-01 15:24:48 +11:00
Nigel Tao 3c708dfe59 Rename PUFFS to WUFFS, part 7 of N 2017-11-30 18:02:59 +11:00
Nigel Tao eda3130628 Rename PUFFS to WUFFS, part 6 of N 2017-11-30 16:43:39 +11:00
Nigel Tao 30b54616a8 Check copy_from_history32's distance. 2017-11-09 12:53:19 +11:00
Nigel Tao 0bc6f792fb Remove some suspensions from decode_huffman_fast.
name                           old speed      new speed      delta

puffs_adler32_10k/clang        2.41GB/s ± 1%  2.39GB/s ± 2%  -0.88%  (p=0.022 n=9+10)
puffs_adler32_100k/clang       2.41GB/s ± 1%  2.39GB/s ± 2%    ~     (p=0.095 n=10+9)
puffs_flate_decode_1k/clang     136MB/s ± 2%   134MB/s ± 1%  -0.86%  (p=0.036 n=9+8)
puffs_flate_decode_10k/clang    172MB/s ± 0%   175MB/s ± 0%  +1.96%  (p=0.000 n=8+9)
puffs_flate_decode_100k/clang   206MB/s ± 0%   209MB/s ± 1%  +1.38%  (p=0.000 n=8+9)
puffs_zlib_decode_10k/clang     160MB/s ± 1%   163MB/s ± 2%  +1.73%  (p=0.001 n=9+9)
puffs_zlib_decode_100k/clang    189MB/s ± 1%   188MB/s ± 6%    ~     (p=0.143 n=10+10)

puffs_adler32_10k/gcc          3.27GB/s ± 0%  3.20GB/s ± 2%  -2.03%  (p=0.000 n=8+10)
puffs_adler32_100k/gcc         3.27GB/s ± 0%  3.21GB/s ± 1%  -1.89%  (p=0.000 n=8+10)
puffs_flate_decode_1k/gcc       141MB/s ± 1%   139MB/s ± 1%  -1.66%  (p=0.000 n=9+10)
puffs_flate_decode_10k/gcc      200MB/s ± 0%   209MB/s ± 1%  +4.73%  (p=0.000 n=10+10)
puffs_flate_decode_100k/gcc     242MB/s ± 0%   237MB/s ± 0%  -2.19%  (p=0.000 n=9+10)
puffs_zlib_decode_10k/gcc       188MB/s ± 0%   195MB/s ± 1%  +3.71%  (p=0.000 n=10+10)
puffs_zlib_decode_100k/gcc      225MB/s ± 0%   221MB/s ± 0%  -1.84%  (p=0.000 n=8+9)
2017-11-06 16:59:27 +11:00
Nigel Tao 3099991d7f Add an "invalid I/O operation" error code. 2017-11-05 13:53:35 +11:00
Nigel Tao aa1f0078db Add decode_huffman_fast.
It's not fast yet, and the immediate impact is indeed a significant
performance loss, but follow-up commits should make it faster.

name                           old speed      new speed      delta

puffs_adler32_10k/clang        2.41GB/s ± 0%  2.40GB/s ± 1%     ~     (p=0.222 n=5+5)
puffs_adler32_100k/clang       2.41GB/s ± 1%  2.39GB/s ± 1%   -0.97%  (p=0.016 n=5+5)
puffs_flate_decode_1k/clang     116MB/s ± 5%   115MB/s ± 0%     ~     (p=0.310 n=5+5)
puffs_flate_decode_10k/clang    145MB/s ± 0%   139MB/s ± 0%   -4.18%  (p=0.016 n=4+5)
puffs_flate_decode_100k/clang   164MB/s ± 5%   158MB/s ± 1%     ~     (p=0.151 n=5+5)
puffs_zlib_decode_10k/clang     135MB/s ± 2%   131MB/s ± 0%   -2.84%  (p=0.008 n=5+5)
puffs_zlib_decode_100k/clang    154MB/s ± 1%   148MB/s ± 2%   -4.09%  (p=0.016 n=4+5)

puffs_adler32_10k/gcc          3.13GB/s ± 4%  3.19GB/s ± 0%   +1.94%  (p=0.032 n=5+5)
puffs_adler32_100k/gcc         3.14GB/s ± 3%  3.20GB/s ± 3%     ~     (p=0.056 n=5+5)
puffs_flate_decode_1k/gcc       122MB/s ± 1%    94MB/s ± 1%  -22.84%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc      143MB/s ± 0%   110MB/s ± 0%  -23.24%  (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc     158MB/s ± 0%   118MB/s ± 0%  -24.98%  (p=0.008 n=5+5)
puffs_zlib_decode_10k/gcc       136MB/s ± 0%   119MB/s ± 1%  -12.63%  (p=0.008 n=5+5)
puffs_zlib_decode_100k/gcc      150MB/s ± 1%   124MB/s ± 2%  -17.17%  (p=0.008 n=5+5)
2017-11-04 14:45:11 +11:00
Nigel Tao ded044da41 Add a flate end_of_block field. 2017-11-04 14:12:38 +11:00
Nigel Tao 4810db3994 Rename decode_huffman to decode_huffman_slow. 2017-11-04 13:38:24 +11:00
Nigel Tao c3420dd138 Refactor std/flate/*.puffs code. 2017-11-04 13:31:39 +11:00
Nigel Tao f8146cf1c6 Remove more PUFFS_USE_NO_OP_PERFORMANCE_HACKS. 2017-11-03 10:15:43 +11:00
Nigel Tao e3822f9951 Re-introduce puffs_base__limit1.
It's not immediately obvious why this data structure performs differently, but
on net, the benchmark numbers seem better. It also won't require any
PUFFS_USE_NO_OP_PERFORMANCE_HACKS or "redundant but mysteriously performant"
use_limit fields.

name                             old speed      new speed      delta

puffs_adler32_10k/clang          2.46GB/s ± 0%  2.43GB/s ± 3%     ~     (p=0.513 n=5+10)
puffs_adler32_100k/clang         2.46GB/s ± 0%  2.44GB/s ± 2%     ~     (p=0.241 n=5+10)
puffs_flate_decode_1k/clang       122MB/s ± 1%   119MB/s ± 2%   -1.86%  (p=0.013 n=5+10)
puffs_flate_decode_10k/clang      147MB/s ± 0%   142MB/s ± 2%   -3.47%  (p=0.001 n=5+10)
puffs_flate_decode_100k/clang     161MB/s ± 0%   161MB/s ± 4%     ~     (p=0.635 n=4+10)
puffs_zlib_decode_10k/clang       139MB/s ± 0%   135MB/s ± 1%   -2.69%  (p=0.001 n=5+10)
puffs_zlib_decode_100k/clang      151MB/s ± 0%   152MB/s ± 3%     ~     (p=1.000 n=5+10)

puffs_adler32_10k/gcc            3.21GB/s ± 1%  3.18GB/s ± 2%     ~     (p=0.310 n=5+10)
puffs_adler32_100k/gcc           3.22GB/s ± 0%  3.19GB/s ± 2%     ~     (p=0.075 n=5+10)
puffs_flate_decode_1k/gcc         121MB/s ± 4%   126MB/s ± 1%   +4.54%  (p=0.002 n=5+8)
puffs_flate_decode_10k/gcc        144MB/s ± 1%   141MB/s ± 1%   -2.52%  (p=0.001 n=5+9)
puffs_flate_decode_100k/gcc       154MB/s ± 1%   158MB/s ± 0%   +2.39%  (p=0.001 n=5+9)
puffs_zlib_decode_10k/gcc         137MB/s ± 2%   135MB/s ± 1%     ~     (p=0.055 n=5+10)
puffs_zlib_decode_100k/gcc        146MB/s ± 2%   150MB/s ± 0%   +2.87%  (p=0.001 n=5+10)

puffs_gif_lzw_decode_20k/clang    132MB/s ± 0%   125MB/s ± 0%   -5.24%  (p=0.001 n=5+10)
puffs_gif_lzw_decode_100k/clang   176MB/s ± 0%   164MB/s ± 0%   -6.82%  (p=0.001 n=5+9)
puffs_gif_decode_1k/clang         363MB/s ± 0%   339MB/s ± 1%   -6.58%  (p=0.002 n=4+10)
puffs_gif_decode_10k/clang        138MB/s ± 0%   133MB/s ± 1%   -3.77%  (p=0.001 n=5+10)
puffs_gif_decode_100k/clang       125MB/s ± 0%   119MB/s ± 0%   -4.79%  (p=0.001 n=5+10)
puffs_gif_decode_1000k/clang      127MB/s ± 0%   121MB/s ± 0%   -5.04%  (p=0.001 n=5+10)

puffs_gif_lzw_decode_20k/gcc      141MB/s ± 0%   140MB/s ± 2%     ~     (p=1.000 n=5+10)
puffs_gif_lzw_decode_100k/gcc     182MB/s ± 0%   180MB/s ± 2%   -1.56%  (p=0.001 n=5+10)
puffs_gif_decode_1k/gcc           363MB/s ± 0%   409MB/s ± 3%  +12.71%  (p=0.001 n=5+10)
puffs_gif_decode_10k/gcc          127MB/s ± 0%   161MB/s ± 3%  +26.15%  (p=0.001 n=5+10)
puffs_gif_decode_100k/gcc         112MB/s ± 0%   136MB/s ± 2%  +21.95%  (p=0.001 n=5+10)
puffs_gif_decode_1000k/gcc        114MB/s ± 0%   140MB/s ± 2%  +22.78%  (p=0.001 n=5+10)
2017-11-03 10:09:57 +11:00
Nigel Tao e5db8233fc Re-introduce writer1.use_limit.
Nobody uses it yet, as we only decode and not encode, but it's there for
symmetry.

No benchmark changes, as nobody uses it yet.
2017-11-02 16:46:03 +11:00
Nigel Tao 7d8085f891 Re-introduce reader1.use_limit.
The std/flate benchmarks get better. The std/gif benchmarks get worse,
but that's possibly because the PUFFS_USE_NO_OP_PERFORMANCE_HACKS hacks
are fragile.

name                             old speed      new speed      delta

puffs_adler32_10k/clang          2.45GB/s ± 0%  2.45GB/s ± 0%     ~     (p=0.310 n=5+5)
puffs_adler32_100k/clang         2.46GB/s ± 0%  2.46GB/s ± 0%     ~     (p=0.690 n=5+5)
puffs_flate_decode_1k/clang       113MB/s ± 0%   113MB/s ± 0%     ~     (p=0.690 n=5+5)
puffs_flate_decode_10k/clang      140MB/s ± 0%   140MB/s ± 0%     ~     (p=0.222 n=5+5)
puffs_flate_decode_100k/clang     151MB/s ± 0%   151MB/s ± 0%     ~     (p=0.095 n=5+5)
puffs_zlib_decode_10k/clang       132MB/s ± 1%   132MB/s ± 0%     ~     (p=0.548 n=5+5)
puffs_zlib_decode_100k/clang      143MB/s ± 0%   143MB/s ± 0%     ~     (p=0.730 n=5+4)

puffs_adler32_10k/gcc            3.26GB/s ± 0%  3.26GB/s ± 0%     ~     (p=0.548 n=5+5)
puffs_adler32_100k/gcc           3.27GB/s ± 0%  3.27GB/s ± 0%     ~     (p=0.841 n=5+5)
puffs_flate_decode_1k/gcc         108MB/s ± 0%   125MB/s ± 0%  +14.97%  (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc        128MB/s ± 0%   148MB/s ± 0%  +15.48%  (p=0.016 n=4+5)
puffs_flate_decode_100k/gcc       136MB/s ± 0%   158MB/s ± 0%  +16.40%  (p=0.008 n=5+5)
puffs_zlib_decode_10k/gcc         123MB/s ± 0%   141MB/s ± 0%  +14.70%  (p=0.016 n=4+5)
puffs_zlib_decode_100k/gcc        131MB/s ± 0%   151MB/s ± 0%  +15.60%  (p=0.016 n=5+4)

puffs_gif_lzw_decode_20k/clang    131MB/s ± 1%   128MB/s ± 3%   -2.04%  (p=0.032 n=5+5)
puffs_gif_lzw_decode_100k/clang   172MB/s ± 1%   173MB/s ± 1%     ~     (p=0.151 n=5+5)
puffs_gif_decode_1k/clang         351MB/s ± 3%   341MB/s ± 5%     ~     (p=0.310 n=5+5)
puffs_gif_decode_10k/clang        138MB/s ± 1%   134MB/s ± 3%   -3.34%  (p=0.008 n=5+5)
puffs_gif_decode_100k/clang       122MB/s ± 2%   122MB/s ± 2%     ~     (p=1.000 n=5+5)
puffs_gif_decode_1000k/clang      126MB/s ± 1%   122MB/s ± 4%   -3.07%  (p=0.016 n=5+5)

puffs_gif_lzw_decode_20k/gcc      140MB/s ± 3%   140MB/s ± 3%     ~     (p=0.421 n=5+5)
puffs_gif_lzw_decode_100k/gcc     187MB/s ± 2%   184MB/s ± 1%     ~     (p=0.056 n=5+5)
puffs_gif_decode_1k/gcc           401MB/s ± 1%   357MB/s ± 1%  -10.88%  (p=0.008 n=5+5)
puffs_gif_decode_10k/gcc          154MB/s ± 1%   126MB/s ± 1%  -18.42%  (p=0.008 n=5+5)
puffs_gif_decode_100k/gcc         131MB/s ± 2%   109MB/s ± 0%  -17.37%  (p=0.008 n=5+5)
puffs_gif_decode_1000k/gcc        135MB/s ± 1%   111MB/s ± 1%  -17.65%  (p=0.008 n=5+5)
2017-11-02 16:35:15 +11:00
Nigel Tao ed6ce6fd54 Add PUFFS_USE_NO_OP_PERFORMANCE_HACKS.
name                             old speed      new speed      delta

puffs_adler32_10k/gcc            3.26GB/s ± 0%  3.26GB/s ± 0%     ~     (p=0.222 n=5+5)
puffs_adler32_100k/gcc           3.27GB/s ± 0%  3.27GB/s ± 0%   -0.12%  (p=0.032 n=5+5)
puffs_flate_decode_1k/gcc         108MB/s ± 0%   108MB/s ± 0%     ~     (p=0.841 n=5+5)
puffs_flate_decode_10k/gcc        128MB/s ± 0%   128MB/s ± 0%     ~     (p=0.057 n=4+4)
puffs_flate_decode_100k/gcc       136MB/s ± 0%   136MB/s ± 0%     ~     (p=0.222 n=5+5)
puffs_zlib_decode_10k/gcc         123MB/s ± 0%   123MB/s ± 0%     ~     (p=0.730 n=5+4)
puffs_zlib_decode_100k/gcc        130MB/s ± 0%   131MB/s ± 0%     ~     (p=1.000 n=5+5)

puffs_gif_lzw_decode_20k/gcc      137MB/s ± 4%   140MB/s ± 3%     ~     (p=0.095 n=5+5)
puffs_gif_lzw_decode_100k/gcc     177MB/s ± 3%   187MB/s ± 2%   +5.75%  (p=0.008 n=5+5)
puffs_gif_decode_1k/gcc           348MB/s ± 0%   401MB/s ± 1%  +15.34%  (p=0.016 n=4+5)
puffs_gif_decode_10k/gcc          121MB/s ± 5%   154MB/s ± 1%  +27.90%  (p=0.008 n=5+5)
puffs_gif_decode_100k/gcc         106MB/s ± 4%   131MB/s ± 2%  +24.35%  (p=0.008 n=5+5)
puffs_gif_decode_1000k/gcc        109MB/s ± 2%   135MB/s ± 1%  +24.03%  (p=0.008 n=5+5)
2017-11-02 14:51:29 +11:00
Nigel Tao aa396e7992 Change cgen's limit from a count to a pointer. 2017-10-27 10:28:35 +11:00
Nigel Tao 3485950a3f Remove the obsolete puffs_base__limit1 concept. 2017-10-27 07:51:57 +11:00
Nigel Tao 8ff453c752 Re-do how limits work. 2017-10-26 16:59:03 +11:00