Nigel Tao
e2ce9f1b1d
Add a writer1.is_marked method.
2017-11-09 10:38:24 +11:00
Nigel Tao
0bc6f792fb
Remove some suspensions from decode_huffman_fast.
...
name old speed new speed delta
puffs_adler32_10k/clang 2.41GB/s ± 1% 2.39GB/s ± 2% -0.88% (p=0.022 n=9+10)
puffs_adler32_100k/clang 2.41GB/s ± 1% 2.39GB/s ± 2% ~ (p=0.095 n=10+9)
puffs_flate_decode_1k/clang 136MB/s ± 2% 134MB/s ± 1% -0.86% (p=0.036 n=9+8)
puffs_flate_decode_10k/clang 172MB/s ± 0% 175MB/s ± 0% +1.96% (p=0.000 n=8+9)
puffs_flate_decode_100k/clang 206MB/s ± 0% 209MB/s ± 1% +1.38% (p=0.000 n=8+9)
puffs_zlib_decode_10k/clang 160MB/s ± 1% 163MB/s ± 2% +1.73% (p=0.001 n=9+9)
puffs_zlib_decode_100k/clang 189MB/s ± 1% 188MB/s ± 6% ~ (p=0.143 n=10+10)
puffs_adler32_10k/gcc 3.27GB/s ± 0% 3.20GB/s ± 2% -2.03% (p=0.000 n=8+10)
puffs_adler32_100k/gcc 3.27GB/s ± 0% 3.21GB/s ± 1% -1.89% (p=0.000 n=8+10)
puffs_flate_decode_1k/gcc 141MB/s ± 1% 139MB/s ± 1% -1.66% (p=0.000 n=9+10)
puffs_flate_decode_10k/gcc 200MB/s ± 0% 209MB/s ± 1% +4.73% (p=0.000 n=10+10)
puffs_flate_decode_100k/gcc 242MB/s ± 0% 237MB/s ± 0% -2.19% (p=0.000 n=9+10)
puffs_zlib_decode_10k/gcc 188MB/s ± 0% 195MB/s ± 1% +3.71% (p=0.000 n=10+10)
puffs_zlib_decode_100k/gcc 225MB/s ± 0% 221MB/s ± 0% -1.84% (p=0.000 n=8+9)
2017-11-06 16:59:27 +11:00
Nigel Tao
058938006a
Remove some suspensions from decode_huffman_fast.
...
name old speed new speed delta
puffs_adler32_10k/clang 2.45GB/s ± 0% 2.41GB/s ± 1% -1.82% (p=0.000 n=9+9)
puffs_adler32_100k/clang 2.45GB/s ± 0% 2.41GB/s ± 1% -1.74% (p=0.000 n=9+10)
puffs_flate_decode_1k/clang 133MB/s ± 0% 136MB/s ± 2% +1.93% (p=0.003 n=9+9)
puffs_flate_decode_10k/clang 181MB/s ± 0% 172MB/s ± 0% -5.18% (p=0.000 n=9+8)
puffs_flate_decode_100k/clang 204MB/s ± 0% 206MB/s ± 0% +0.63% (p=0.000 n=10+8)
puffs_zlib_decode_10k/clang 168MB/s ± 0% 160MB/s ± 1% -4.82% (p=0.000 n=10+9)
puffs_zlib_decode_100k/clang 189MB/s ± 0% 189MB/s ± 1% ~ (p=0.143 n=10+10)
puffs_adler32_10k/gcc 3.26GB/s ± 0% 3.27GB/s ± 0% +0.12% (p=0.000 n=10+8)
puffs_adler32_100k/gcc 3.27GB/s ± 0% 3.27GB/s ± 0% +0.18% (p=0.000 n=10+8)
puffs_flate_decode_1k/gcc 130MB/s ± 0% 141MB/s ± 1% +8.71% (p=0.000 n=10+9)
puffs_flate_decode_10k/gcc 203MB/s ± 0% 200MB/s ± 0% -1.72% (p=0.000 n=9+10)
puffs_flate_decode_100k/gcc 241MB/s ± 0% 242MB/s ± 0% +0.54% (p=0.000 n=8+9)
puffs_zlib_decode_10k/gcc 188MB/s ± 0% 188MB/s ± 0% +0.26% (p=0.000 n=9+10)
puffs_zlib_decode_100k/gcc 223MB/s ± 0% 225MB/s ± 0% +0.93% (p=0.000 n=9+8)
2017-11-06 15:25:19 +11:00
Nigel Tao
13731f989f
Add a reader1.unread_u8 method.
...
Also re-enable a number of std/flate tests and benchmarks.
It's not an optimization per se, but as a data point, here are some
benchmark numbers between 36a93fa and this commit. 36a93fa is "36a93fa
Add a TODO about impure functions", which is one commit before "f968950
Read bits up front in decode_huffman_fast", and f968950 was the commit
that made decode_huffman_fast read more than one byte at a time.
name old speed new speed delta
puffs_adler32_10k/clang 2.45GB/s ± 0% 2.45GB/s ± 0% ~ (p=0.684 n=10+10)
puffs_adler32_100k/clang 2.45GB/s ± 0% 2.46GB/s ± 0% +0.07% (p=0.019 n=10+10)
puffs_flate_decode_1k/clang 124MB/s ± 0% 132MB/s ± 0% +6.06% (p=0.000 n=10+10)
puffs_flate_decode_10k/clang 145MB/s ± 0% 174MB/s ± 0% +20.19% (p=0.000 n=9+10)
puffs_flate_decode_100k/clang 168MB/s ± 0% 208MB/s ± 0% +23.31% (p=0.000 n=10+10)
puffs_zlib_decode_10k/clang 136MB/s ± 0% 162MB/s ± 0% +18.94% (p=0.000 n=10+9)
puffs_zlib_decode_100k/clang 158MB/s ± 0% 192MB/s ± 0% +21.58% (p=0.000 n=10+9)
puffs_adler32_10k/gcc 3.26GB/s ± 0% 3.26GB/s ± 0% ~ (p=0.631 n=10+10)
puffs_adler32_100k/gcc 3.27GB/s ± 0% 3.27GB/s ± 0% ~ (p=0.237 n=10+8)
puffs_flate_decode_1k/gcc 95.3MB/s ± 0% 105.5MB/s ± 0% +10.67% (p=0.000 n=10+10)
puffs_flate_decode_10k/gcc 110MB/s ± 0% 140MB/s ± 0% +27.32% (p=0.000 n=9+9)
puffs_flate_decode_100k/gcc 121MB/s ± 0% 150MB/s ± 0% +24.02% (p=0.000 n=8+10)
puffs_zlib_decode_10k/gcc 112MB/s ± 0% 120MB/s ± 0% +6.96% (p=0.000 n=10+9)
puffs_zlib_decode_100k/gcc 123MB/s ± 0% 134MB/s ± 0% +8.93% (p=0.000 n=9+8)
2017-11-05 15:59:02 +11:00
Nigel Tao
55e0bc513b
Don't suspend for read_u8 if data is available.
...
It's not obvious why the benchmarks get worse when the code looks
better, but the long term goal is for all of decode_huffman_fast to not
suspend, and this is one step along that path.
name old speed new speed delta
puffs_adler32_10k/clang 2.40GB/s ± 1% 2.40GB/s ± 1% ~ (p=0.841 n=5+5)
puffs_adler32_100k/clang 2.40GB/s ± 1% 2.39GB/s ± 2% ~ (p=0.421 n=5+5)
puffs_flate_decode_1k/clang 131MB/s ± 1% 128MB/s ± 0% -2.51% (p=0.008 n=5+5)
puffs_flate_decode_10k/clang 171MB/s ± 1% 167MB/s ± 2% -2.86% (p=0.008 n=5+5)
puffs_flate_decode_100k/clang 202MB/s ± 0% 193MB/s ± 2% -4.64% (p=0.016 n=4+5)
puffs_zlib_decode_10k/clang 159MB/s ± 1% 157MB/s ± 0% -1.65% (p=0.008 n=5+5)
puffs_zlib_decode_100k/clang 185MB/s ± 1% 180MB/s ± 1% -3.11% (p=0.008 n=5+5)
puffs_adler32_10k/gcc 3.26GB/s ± 0% 3.26GB/s ± 0% ~ (p=0.421 n=5+5)
puffs_adler32_100k/gcc 3.27GB/s ± 0% 3.26GB/s ± 0% ~ (p=0.151 n=5+5)
puffs_flate_decode_1k/gcc 132MB/s ± 0% 129MB/s ± 0% -2.29% (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc 170MB/s ± 0% 164MB/s ± 0% -3.13% (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc 207MB/s ± 0% 185MB/s ± 0% -10.45% (p=0.016 n=5+4)
puffs_zlib_decode_10k/gcc 162MB/s ± 0% 157MB/s ± 0% -2.95% (p=0.008 n=5+5)
puffs_zlib_decode_100k/gcc 194MB/s ± 0% 175MB/s ± 0% -9.78% (p=0.016 n=5+4)
2017-11-05 08:03:21 +11:00
Nigel Tao
aa1f0078db
Add decode_huffman_fast.
...
It's not fast yet, and the immediate impact is indeed a significant
performance loss, but follow-up commits should make it faster.
name old speed new speed delta
puffs_adler32_10k/clang 2.41GB/s ± 0% 2.40GB/s ± 1% ~ (p=0.222 n=5+5)
puffs_adler32_100k/clang 2.41GB/s ± 1% 2.39GB/s ± 1% -0.97% (p=0.016 n=5+5)
puffs_flate_decode_1k/clang 116MB/s ± 5% 115MB/s ± 0% ~ (p=0.310 n=5+5)
puffs_flate_decode_10k/clang 145MB/s ± 0% 139MB/s ± 0% -4.18% (p=0.016 n=4+5)
puffs_flate_decode_100k/clang 164MB/s ± 5% 158MB/s ± 1% ~ (p=0.151 n=5+5)
puffs_zlib_decode_10k/clang 135MB/s ± 2% 131MB/s ± 0% -2.84% (p=0.008 n=5+5)
puffs_zlib_decode_100k/clang 154MB/s ± 1% 148MB/s ± 2% -4.09% (p=0.016 n=4+5)
puffs_adler32_10k/gcc 3.13GB/s ± 4% 3.19GB/s ± 0% +1.94% (p=0.032 n=5+5)
puffs_adler32_100k/gcc 3.14GB/s ± 3% 3.20GB/s ± 3% ~ (p=0.056 n=5+5)
puffs_flate_decode_1k/gcc 122MB/s ± 1% 94MB/s ± 1% -22.84% (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc 143MB/s ± 0% 110MB/s ± 0% -23.24% (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc 158MB/s ± 0% 118MB/s ± 0% -24.98% (p=0.008 n=5+5)
puffs_zlib_decode_10k/gcc 136MB/s ± 0% 119MB/s ± 1% -12.63% (p=0.008 n=5+5)
puffs_zlib_decode_100k/gcc 150MB/s ± 1% 124MB/s ± 2% -17.17% (p=0.008 n=5+5)
2017-11-04 14:45:11 +11:00
Nigel Tao
4810db3994
Rename decode_huffman to decode_huffman_slow.
2017-11-04 13:38:24 +11:00
Nigel Tao
558d8ddbbd
Restrict tPrefix temporary variables's scope.
2017-11-03 14:00:35 +11:00
Nigel Tao
acec19e969
Add genFilenameLineComments.
2017-11-03 12:28:15 +11:00
Nigel Tao
e3822f9951
Re-introduce puffs_base__limit1.
...
It's not immediately obvious why this data structure performs differently, but
on net, the benchmark numbers seem better. It also won't require any
PUFFS_USE_NO_OP_PERFORMANCE_HACKS or "redundant but mysteriously performant"
use_limit fields.
name old speed new speed delta
puffs_adler32_10k/clang 2.46GB/s ± 0% 2.43GB/s ± 3% ~ (p=0.513 n=5+10)
puffs_adler32_100k/clang 2.46GB/s ± 0% 2.44GB/s ± 2% ~ (p=0.241 n=5+10)
puffs_flate_decode_1k/clang 122MB/s ± 1% 119MB/s ± 2% -1.86% (p=0.013 n=5+10)
puffs_flate_decode_10k/clang 147MB/s ± 0% 142MB/s ± 2% -3.47% (p=0.001 n=5+10)
puffs_flate_decode_100k/clang 161MB/s ± 0% 161MB/s ± 4% ~ (p=0.635 n=4+10)
puffs_zlib_decode_10k/clang 139MB/s ± 0% 135MB/s ± 1% -2.69% (p=0.001 n=5+10)
puffs_zlib_decode_100k/clang 151MB/s ± 0% 152MB/s ± 3% ~ (p=1.000 n=5+10)
puffs_adler32_10k/gcc 3.21GB/s ± 1% 3.18GB/s ± 2% ~ (p=0.310 n=5+10)
puffs_adler32_100k/gcc 3.22GB/s ± 0% 3.19GB/s ± 2% ~ (p=0.075 n=5+10)
puffs_flate_decode_1k/gcc 121MB/s ± 4% 126MB/s ± 1% +4.54% (p=0.002 n=5+8)
puffs_flate_decode_10k/gcc 144MB/s ± 1% 141MB/s ± 1% -2.52% (p=0.001 n=5+9)
puffs_flate_decode_100k/gcc 154MB/s ± 1% 158MB/s ± 0% +2.39% (p=0.001 n=5+9)
puffs_zlib_decode_10k/gcc 137MB/s ± 2% 135MB/s ± 1% ~ (p=0.055 n=5+10)
puffs_zlib_decode_100k/gcc 146MB/s ± 2% 150MB/s ± 0% +2.87% (p=0.001 n=5+10)
puffs_gif_lzw_decode_20k/clang 132MB/s ± 0% 125MB/s ± 0% -5.24% (p=0.001 n=5+10)
puffs_gif_lzw_decode_100k/clang 176MB/s ± 0% 164MB/s ± 0% -6.82% (p=0.001 n=5+9)
puffs_gif_decode_1k/clang 363MB/s ± 0% 339MB/s ± 1% -6.58% (p=0.002 n=4+10)
puffs_gif_decode_10k/clang 138MB/s ± 0% 133MB/s ± 1% -3.77% (p=0.001 n=5+10)
puffs_gif_decode_100k/clang 125MB/s ± 0% 119MB/s ± 0% -4.79% (p=0.001 n=5+10)
puffs_gif_decode_1000k/clang 127MB/s ± 0% 121MB/s ± 0% -5.04% (p=0.001 n=5+10)
puffs_gif_lzw_decode_20k/gcc 141MB/s ± 0% 140MB/s ± 2% ~ (p=1.000 n=5+10)
puffs_gif_lzw_decode_100k/gcc 182MB/s ± 0% 180MB/s ± 2% -1.56% (p=0.001 n=5+10)
puffs_gif_decode_1k/gcc 363MB/s ± 0% 409MB/s ± 3% +12.71% (p=0.001 n=5+10)
puffs_gif_decode_10k/gcc 127MB/s ± 0% 161MB/s ± 3% +26.15% (p=0.001 n=5+10)
puffs_gif_decode_100k/gcc 112MB/s ± 0% 136MB/s ± 2% +21.95% (p=0.001 n=5+10)
puffs_gif_decode_1000k/gcc 114MB/s ± 0% 140MB/s ± 2% +22.78% (p=0.001 n=5+10)
2017-11-03 10:09:57 +11:00
Nigel Tao
4b572702f9
Remove the return error "foo" special case.
2017-10-27 14:39:48 +11:00
Nigel Tao
9a8508857f
Fix the "limitt" typo.
2017-10-27 11:10:25 +11:00
Nigel Tao
cbc8c39375
Delete old limit- and mark-related code.
2017-10-27 10:57:32 +11:00
Nigel Tao
8ff453c752
Re-do how limits work.
2017-10-26 16:59:03 +11:00
Nigel Tao
a9ba7e85ac
Add reader1.limitt and writer1.limitt methods.
...
The "limitt" typo will be renamed to "limit" once we remove the keyword
named "limit".
2017-10-26 15:06:59 +11:00
Nigel Tao
b886e3a1ea
Add reader1.mark and reader1.since_mark methods.
2017-10-26 14:39:38 +11:00
Nigel Tao
5052b16f2d
Rename writer1.slice to writer1.since_mark.
2017-10-26 13:03:19 +11:00
Nigel Tao
474a0d2ed7
Make mark a method, not a keyword.
2017-10-26 12:31:27 +11:00
Nigel Tao
36f5ae9bf4
Move the "mark in.dst" into the flate decode loop.
2017-10-26 11:20:42 +11:00
Nigel Tao
21bbbd422f
Generate code for "mark in.dst".
...
The generated code is ugly and the language design is not finalized, but
the flate_backref_crosses_blocks test (testing a correctness bug) now
passes. We can fix the ugliness in subsequent commits.
2017-10-22 14:44:53 +11:00
Nigel Tao
83c8faef4d
Reorder some switch cases.
2017-10-22 14:01:26 +11:00
Nigel Tao
ac793d33bc
Generate placeholder code for "mark in.dst".
2017-10-22 13:57:37 +11:00
Nigel Tao
f29c4e278f
Follow the pkg__type__method convention for base.
2017-10-21 15:59:08 +11:00
Nigel Tao
6815375de5
Generate double '_'s for pkg__type__method names.
2017-10-21 15:10:25 +11:00
Nigel Tao
2639f8f46a
Rename std/flate's decoder to flate_decoder.
2017-10-21 13:05:19 +11:00
Nigel Tao
62935c41d7
Prepare for calculating the zlib checksum.
2017-10-21 11:55:58 +11:00
Nigel Tao
245e3ec7b9
Unroll iterate loops.
...
name old speed new speed delta
puffs_adler32_10k/clang 2.12GB/s ± 1% 2.42GB/s ± 1% +14.24% (p=0.008 n=5+5)
puffs_adler32_100k/clang 2.12GB/s ± 3% 2.43GB/s ± 0% +14.29% (p=0.008 n=5+5)
mimic_adler32_10k/clang 2.93GB/s ± 2% 2.96GB/s ± 0% ~ (p=0.310 n=5+5)
mimic_adler32_100k/clang 2.89GB/s ± 2% 2.91GB/s ± 0% ~ (p=1.000 n=5+5)
puffs_adler32_10k/gcc 2.17GB/s ± 1% 3.23GB/s ± 0% +48.59% (p=0.008 n=5+5)
puffs_adler32_100k/gcc 2.18GB/s ± 1% 3.23GB/s ± 0% +48.34% (p=0.008 n=5+5)
mimic_adler32_10k/gcc 2.97GB/s ± 0% 2.96GB/s ± 1% ~ (p=1.000 n=5+5)
mimic_adler32_100k/gcc 2.91GB/s ± 1% 2.91GB/s ± 1% ~ (p=0.548 n=5+5)
2017-10-20 17:28:21 +11:00
Nigel Tao
b024cb1d8c
Add some support for iterate assertions.
2017-10-20 16:17:20 +11:00
Nigel Tao
2cd6ebed8e
Add a "deref" unary operator.
2017-10-20 15:44:30 +11:00
Nigel Tao
51bd52239b
Add an "iterate" keyword.
2017-10-20 15:33:31 +11:00
Nigel Tao
50369c782c
Add a skeleton of an adler.update method.
2017-10-19 18:58:17 +11:00
Nigel Tao
c99fdaf63d
Actually copy from the flate decoder's history.
2017-10-13 14:50:34 +11:00
Nigel Tao
a33f744da5
Drop the "?" from "writer1.copy_from_slice?".
2017-10-13 11:44:53 +11:00
Nigel Tao
0384c5e4a8
Rename the copy_from_xxx methods for consistency.
2017-10-13 10:59:47 +11:00
Nigel Tao
d4372cb5b8
Change license to Apache 2.
...
Also remove explicit PATENTS file. The new LICENSE file already
discusses patents.
Note that the Puffs project has not yet been publicly released.
2017-10-12 11:17:41 +11:00
Nigel Tao
5a7f9e6cde
Remove some duplicated short_read_src code.
2017-10-07 15:36:16 +11:00
Nigel Tao
d275b379a3
Remove the idea of limited reads and writes.
2017-10-07 15:32:00 +11:00
Nigel Tao
a12292d094
Have "return ok" set coro_susp_point = 0.
2017-10-07 14:43:25 +11:00
Nigel Tao
585ea2024d
Drop the "?" from "copy_from32?".
...
Also add a PUFFS_COROUTINE_SUSPENSION_POINT_MAYBE_SUSPEND macro.
Also add a loop to decoder.decode, as the code generator was previously
(incorrectly) not generating a coro_susp_point at the end of the
function.
2017-10-07 14:34:31 +11:00
Nigel Tao
2efdf5fab6
Add a lang/builtin package.
2017-10-07 11:37:10 +11:00
Nigel Tao
010c4db54f
Allow explicit returns of built-in status codes.
2017-10-06 17:34:58 +11:00
Nigel Tao
2e883fcc23
Drop the "?" from "copy_history32?".
...
This makes the caller, not the implementation, responsible for detecting
short reads and writes. This (the lack of "?") is consistent with the
slice.copy_from method.
name old speed new speed delta
puffs_flate_decode_1k/clang 117MB/s ± 6% 120MB/s ± 0% +2.31% (p=0.016 n=5+5)
puffs_flate_decode_10k/clang 153MB/s ± 1% 155MB/s ± 0% +1.16% (p=0.008 n=5+5)
puffs_flate_decode_100k/clang 168MB/s ± 1% 177MB/s ± 0% +5.41% (p=0.008 n=5+5)
puffs_flate_decode_1k/gcc 127MB/s ± 1% 143MB/s ± 0% +12.15% (p=0.008 n=5+5)
puffs_flate_decode_10k/gcc 145MB/s ± 0% 180MB/s ± 1% +24.00% (p=0.008 n=5+5)
puffs_flate_decode_100k/gcc 163MB/s ± 0% 212MB/s ± 1% +29.90% (p=0.008 n=5+5)
2017-10-06 17:02:09 +11:00
Nigel Tao
96a9ab4126
Have copy_history32 compare to the wstart pointer.
2017-10-02 16:08:03 +11:00
Nigel Tao
6f132a74e2
Remove the 'mark' concept, no longer necessary.
2017-10-02 14:02:13 +11:00
Nigel Tao
ead5e45ae7
Refactor cgen of expressions-as-statements.
2017-09-28 17:32:05 +10:00
Nigel Tao
f1643be221
Have copy_from return the number of bytes copied.
2017-09-28 17:27:34 +10:00
Nigel Tao
7b6f7384d1
Take more care with a NULL reader1 or writer1 buf.
2017-09-23 15:32:42 +10:00
Nigel Tao
fd5d44b3fc
Add a mark method and type.
2017-09-23 13:17:37 +10:00
Nigel Tao
a9f12c23a4
Generate C code for an_array[:].
2017-09-23 10:51:04 +10:00
Nigel Tao
f107ec155d
Rename puffs_u16be to puffs_base_load_u16be.
2017-09-23 09:29:45 +10:00