Nigel Tao
6c13877688
Drop WUFFS_JSON__QUIRK_ALLOW_BACKSLASH_X_AS_BYTES
...
We can always add it back later if there's demand for it.
2020-08-30 16:15:35 +10:00
Nigel Tao
4284369bb8
Add TOKEN__VBD__FILLER__PUNCTUATION
2020-08-17 16:48:30 +10:00
Nigel Tao
2de1dfec97
Fix test/data/cbor*tokens losing DEFINITELY_ASCII
...
The change to the golden file was mistakenly left out of 2397d029 "Fix
std/cbor UTF-8 strings aren't always ASCII".
2020-08-12 23:32:40 +10:00
Nigel Tao
0091a2132d
Have std/cbor reject tags before an 0xFF stop code
2020-08-10 21:39:37 +10:00
Nigel Tao
9497b174c9
Add test_wuffs_cbor_decode_{in,}valid
2020-08-10 21:22:47 +10:00
Nigel Tao
0b7eaf291b
Add example/cbor-to-json
2020-08-09 23:03:42 +10:00
Nigel Tao
1adbc9547f
Split std/json's QUIRK_ALLOW_BACKSLASH_X in two
2020-08-06 23:28:07 +10:00
Nigel Tao
1e96ef200c
Add test/data/cbor-rfc-7049-examples.*
2020-07-27 22:00:28 +10:00
Nigel Tao
d6a10dfe8a
Add io_buffer.{read,writ}er_slice methods
2020-07-27 11:47:47 +10:00
Nigel Tao
702c7b299f
Let wuffs_base__utf_8__etc take ptr-to-const-u8
2020-07-22 15:42:54 +10:00
Nigel Tao
4e193599ac
Add std/cbor
2020-07-15 14:08:37 +10:00
Nigel Tao
168f60a781
Add example/jsonptr -output-format=cbor flag
2020-07-14 15:57:05 +10:00
Nigel Tao
a3931d5215
Rename IEEE 754 from_f64_to_u{16,32} functions
2020-07-12 21:06:44 +10:00
Nigel Tao
7bf7cf28b9
Add IEEE 754 from_f64_to_u{16,32} functions
2020-07-12 17:01:14 +10:00
Nigel Tao
0899b82c42
Add IEEE 754 from_u{16,32}_to_f64 functions
2020-07-12 13:23:04 +10:00
Nigel Tao
4d449dc663
Rename ieee_754_bit_representation functions
2020-07-12 11:00:47 +10:00
Nigel Tao
0a9075f81c
Move test_wuffs_strconv_parse_number_f64_options
2020-07-11 13:17:58 +10:00
Nigel Tao
37bc8cdaa5
Refactor parse_number_f64 related tests
2020-07-11 13:16:00 +10:00
Nigel Tao
c5c9885db4
Add PARSE_NUMBER_XXX__ALLOW_UNDERSCORES
2020-07-11 13:10:14 +10:00
Nigel Tao
e82bc8ec7b
Add PARSE_NUMBER_XXX__ALLOW_MULTIPLE_LEADING_Z...
2020-07-11 12:57:25 +10:00
Nigel Tao
4d61a056e0
Add PARSE_NUMBER_FXX__REJECT_INF_AND_NAN
2020-07-11 12:34:40 +10:00
Nigel Tao
e0c5de9d8d
Add PARSE_NUMBER_FXX__DECIMAL_SEPARATOR_IS_A_COMMA
2020-07-11 12:14:39 +10:00
Nigel Tao
e0ca5a4c3c
Have base_16 codec return a transform__output
2020-07-09 21:09:19 +10:00
Nigel Tao
309a5ba231
Add base_64 codec to wuffs_base
2020-07-09 20:42:17 +10:00
Nigel Tao
84bb3afd87
Lower json.DECODER_NUMBER_LENGTH_MAX_INCL to 99
...
Commit b9f69264a0 "Raise
json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047" might have regressed the
std/json benchmarks.
2020-07-07 23:29:30 +10:00
Nigel Tao
b9f69264a0
Raise json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047
2020-07-07 23:08:11 +10:00
Nigel Tao
6b7ce30e32
Add wuffs_base__parse_number_xxx options arg
2020-07-07 16:19:46 +10:00
Nigel Tao
832edf7227
Remove the medium_prec_bin type
...
It's obsolete now that we have an Eisel algorithm implementation.
Binary size, before:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
28744 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28968 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31176 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31176 gen/lib/c/gcc-static/wuffs-base-f64conv.o
Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.031s
real 0m0.975s
$ time ./a.out < citylots.json
Before/After:
real 0m0.807s
real 0m0.729s
2020-07-07 16:07:35 +10:00
Nigel Tao
86093c8194
Have parse_number_f64 use the Eisel algorithm
...
The algorithm is discussed at
https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/
----
Binary size, before:
27848 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28040 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
30392 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
30392 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o
The binary size difference does not account for the powers-of-10 table,
which was expanded in a previous commit in anticipation of this one. A
future commit will also remove some of the other f64 parser code used
before implementing the Eisel algorithm.
----
Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.231s
real 0m1.031s
The difference with and without -parse-number-f64 measures how much time
is spent in the parse_number_f64 function. Before: 424ms, after: 224ms.
$ time ./a.out < citylots.json
real 0m0.807s
Similarly, for canada.json from github.com/miloyip/nativejson-benchmark
$ time ./a.out -parse-number-f64 < canada.json
Before/After:
real 0m0.023s
real 0m0.017s
$ time ./a.out < canada.json
real 0m0.013s
Time spent in parse_number_f64, before: 10ms, after: 4ms.
----
name old time/op new time/op delta
wuffs_strconv_parse_number_f64_1_lsh53_add0/clang5 77.0ns ± 0% 37.0ns ± 0% -51.95% (p=0.029 n=4+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/clang5 690ns ± 0% 725ns ± 2% +4.95% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/clang5 92.0ns ± 0% 136.0ns ± 2% +47.83% (p=0.016 n=4+5)
wuffs_strconv_parse_number_f64_pi_short/clang5 35.0ns ± 3% 18.0ns ± 0% -48.57% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_1_lsh53_add0/gcc7 67.0ns ± 0% 34.0ns ± 0% -49.25% (p=0.000 n=5+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/gcc7 712ns ± 0% 760ns ± 2% +6.72% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/gcc7 88.4ns ± 1% 116.2ns ± 3% +31.45% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_short/gcc7 32.6ns ± 2% 21.4ns ± 7% -34.36% (p=0.008 n=5+5)
2020-07-07 15:27:51 +10:00
Nigel Tao
267bfc8176
Deepen parse_number_f64's powers_of_10 table
...
Deepen means changing the mantissa from 64 bits to 128 bits. We also
adjust the exponent bias, in anticipation of a future commit.
Binary size, before:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
27336 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
27528 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
28856 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
28856 gen/lib/c/gcc-static/wuffs-base-f64conv.o
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.432s
real 0m1.421s
citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 19:03:37 +10:00
Nigel Tao
53da68fe4b
Refine parse_number_f64's powers_of_10 table
...
Refine means changing the base-10 exponents from "-348 ..= +340 stepping
by 8" to "-326 ..= +310 stepping by 1".
Binary size, before:
15992 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
16232 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
17384 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
17384 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.455s
real 0m1.432s
citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 13:58:49 +10:00
Nigel Tao
19b3ce24a3
Optimize high_prec_dec__round_just_enough
...
name old time/op new time/op delta
wuffs_strconv_render_number_f64_just_enough_fractions/clang5 56.4µs ± 1% 56.3µs ± 1% ~ (p=1.000 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/clang5 4.12µs ± 2% 1.00µs ± 1% -75.65% (p=0.008 n=5+5)
wuffs_strconv_render_number_f64_just_enough_fractions/gcc7 54.3µs ± 0% 54.2µs ± 0% ~ (p=0.151 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/gcc7 4.16µs ± 0% 1.01µs ± 0% -75.77% (p=0.000 n=4+5)
2020-07-03 23:44:37 +10:00
Nigel Tao
58678cb342
Add wuffs_base__render_number_f64 benches
2020-07-03 23:16:53 +10:00
Nigel Tao
b9afc26387
Add TOKEN__VBD__STRING__CHAIN_ETC
2020-07-03 13:36:13 +10:00
Nigel Tao
a6031b6d2e
Add wuffs_base__utf_8__next_from_end
2020-07-02 15:08:54 +10:00
Nigel Tao
89ceb9fabc
Add wuffs_base__render_number_f64
2020-07-01 12:03:20 +10:00
Nigel Tao
d40a386211
Add wuffs_base__render_number_{i,u}64
2020-06-21 21:57:48 +10:00
Nigel Tao
f45f01f4f3
Update clang-format from v5 to v9
2020-06-05 14:06:56 +10:00
Nigel Tao
d7c9e9bbc9
De-duplicate the wuffs_gif_decode test function
2020-06-01 14:33:19 +10:00
Nigel Tao
e6b7c51557
Inline do_bench_gif_decode
2020-06-01 14:23:59 +10:00
Nigel Tao
829519880a
Drop unnecessary memcpy in do_bench_image_decode
...
This doesn't affect the std/bmp or std/gif implementations per se, only
the code that benches them. Prior to this commit, the source file was
first decoded into a pixel buffer and then always copied from the
(2-dimensional) wuffs_base__pixel_buffer to a (1-dimensional)
wuffs_base_io_buffer. That copy is useful for *tests*, which compare
that io_buffer's contents to a known good result (i.e. a separate I/O
buffer filled with the contents of a golden file), but it isn't
necessary for *benches*. Dropping that unnecessary copy reduces the
total time per iteration, and therefore increases the reported speed.
name old speed new speed delta
wuffs_bmp_decode_40k/clang5 2.09GB/s ± 2% 2.48GB/s ± 0% +18.78% (p=0.008 n=5+5)
wuffs_bmp_decode_40k/gcc7 1.73GB/s ± 1% 1.93GB/s ± 0% +11.54% (p=0.008 n=5+5)
wuffs_gif_decode_1k_bw/clang5 291MB/s ± 0% 376MB/s ± 1% +29.14% (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang5 95.3MB/s ± 1% 103.2MB/s ± 1% +8.37% (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang5 117MB/s ± 0% 130MB/s ± 0% +10.68% (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang5 469MB/s ± 0% 489MB/s ± 1% +4.39% (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang5 126MB/s ± 1% 130MB/s ± 1% +3.50% (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang5 152MB/s ± 0% 156MB/s ± 1% +2.77% (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang5 318MB/s ± 0% 329MB/s ± 0% +3.53% (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang5 137MB/s ± 0% 139MB/s ± 0% +1.40% (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang5 138MB/s ± 0% 140MB/s ± 1% +1.45% (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang5 139MB/s ± 0% 140MB/s ± 1% +1.03% (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang5 661MB/s ± 0% 747MB/s ± 1% +13.08% (p=0.008 n=5+5)
mimic_gif_decode_1k_bw/clang5 92.0MB/s ± 0% 91.9MB/s ± 1% ~ (p=0.841 n=5+5)
mimic_gif_decode_1k_color/clang5 45.5MB/s ± 0% 45.7MB/s ± 0% +0.45% (p=0.032 n=5+4)
mimic_gif_decode_10k_indexed/clang5 53.5MB/s ± 0% 53.7MB/s ± 0% +0.31% (p=0.032 n=5+4)
mimic_gif_decode_20k/clang5 55.5MB/s ± 0% 55.1MB/s ± 2% ~ (p=0.690 n=5+5)
mimic_gif_decode_100k_artificial/clang5 88.8MB/s ± 0% 89.0MB/s ± 1% ~ (p=0.151 n=5+5)
mimic_gif_decode_100k_realistic/clang5 53.5MB/s ± 1% 53.7MB/s ± 0% +0.31% (p=0.024 n=5+5)
mimic_gif_decode_1000k/clang5 54.6MB/s ± 0% 54.6MB/s ± 1% ~ (p=0.222 n=5+5)
mimic_gif_decode_anim_screencap/clang5 106MB/s ± 0% 109MB/s ± 0% +2.59% (p=0.008 n=5+5)
wuffs_gif_decode_1k_bw/gcc7 284MB/s ± 0% 356MB/s ± 0% +25.33% (p=0.016 n=4+5)
wuffs_gif_decode_1k_color_full_init/gcc7 82.1MB/s ± 0% 95.3MB/s ± 0% +16.08% (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc7 97.5MB/s ± 0% 117.4MB/s ± 0% +20.39% (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc7 345MB/s ± 0% 396MB/s ± 0% +14.71% (p=0.016 n=4+5)
wuffs_gif_decode_10k_indexed/gcc7 104MB/s ± 0% 121MB/s ± 0% +16.38% (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc7 131MB/s ± 0% 149MB/s ± 0% +13.55% (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc7 293MB/s ± 0% 321MB/s ± 1% +9.77% (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc7 116MB/s ± 1% 130MB/s ± 1% +12.10% (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc7 117MB/s ± 1% 132MB/s ± 1% +12.53% (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc7 117MB/s ± 0% 132MB/s ± 1% +12.20% (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc7 615MB/s ± 2% 711MB/s ± 2% +15.53% (p=0.008 n=5+5)
mimic_gif_decode_1k_bw/gcc7 91.4MB/s ± 0% 91.7MB/s ± 1% ~ (p=0.095 n=5+5)
mimic_gif_decode_1k_color/gcc7 46.3MB/s ± 0% 46.2MB/s ± 2% ~ (p=0.841 n=5+5)
mimic_gif_decode_10k_indexed/gcc7 51.9MB/s ± 1% 51.7MB/s ± 1% ~ (p=0.310 n=5+5)
mimic_gif_decode_20k/gcc7 54.2MB/s ± 0% 54.2MB/s ± 1% ~ (p=0.421 n=5+5)
mimic_gif_decode_100k_artificial/gcc7 88.4MB/s ± 1% 88.7MB/s ± 1% ~ (p=0.222 n=5+5)
mimic_gif_decode_100k_realistic/gcc7 53.7MB/s ± 0% 53.7MB/s ± 0% ~ (p=0.206 n=5+5)
mimic_gif_decode_1000k/gcc7 53.8MB/s ± 2% 54.0MB/s ± 2% ~ (p=0.690 n=5+5)
mimic_gif_decode_anim_screencap/gcc7 106MB/s ± 0% 109MB/s ± 0% +2.79% (p=0.008 n=5+5)
2020-06-01 13:59:50 +10:00
Nigel Tao
4b9b971436
Harmonize do_bench_{gif,image}_decode
2020-06-01 13:27:46 +10:00
Nigel Tao
2784613b49
Add n_bytes_out arg to gif bench functions
2020-06-01 13:14:14 +10:00
Nigel Tao
7a85f67b22
Add std/bmp benchmark
2020-06-01 13:03:03 +10:00
Nigel Tao
54196c4e80
Rename pixel_swizzler.swizzle_interleaved
2020-05-29 16:54:05 +10:00
Nigel Tao
6537a817b7
Roll back the two previous "@I/O redirect" commits
...
See the previous commit for the rationale.
2020-05-29 11:27:51 +10:00
Nigel Tao
844fc9b5ee
Temporarily avoid using base."@I/O redirect"
...
This commit (and the next) will soon be followed by a rollback.
2020-05-29 09:53:30 +10:00
Nigel Tao
dcb81a85bb
Prepare to decode BMP embedding a JPEG or PNG
2020-05-27 13:37:54 +10:00
Nigel Tao
1239280ad8
Roll back 'dst_palette into the pixel_swizzler'
...
See the previous commit for the rationale.
2020-05-26 14:13:28 +10:00
Nigel Tao
2d8ec9ace9
Roll dst_palette into the pixel_swizzler struct
...
This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.
name old speed new speed delta
wuffs_gif_decode_1k_bw/clang5 286MB/s ± 1% 251MB/s ± 0% -12.11% (p=0.000 n=10+9)
wuffs_gif_decode_1k_color_full_init/clang5 96.0MB/s ± 0% 88.9MB/s ± 0% -7.35% (p=0.000 n=10+10)
wuffs_gif_decode_1k_color_part_init/clang5 119MB/s ± 0% 108MB/s ± 0% -9.06% (p=0.000 n=10+8)
wuffs_gif_decode_10k_bgra/clang5 469MB/s ± 1% 438MB/s ± 0% -6.50% (p=0.000 n=10+8)
wuffs_gif_decode_10k_indexed/clang5 125MB/s ± 1% 117MB/s ± 1% -6.78% (p=0.000 n=9+9)
wuffs_gif_decode_20k/clang5 153MB/s ± 0% 139MB/s ± 0% -9.40% (p=0.000 n=9+9)
wuffs_gif_decode_100k_artificial/clang5 324MB/s ± 0% 304MB/s ± 0% -6.22% (p=0.000 n=10+9)
wuffs_gif_decode_100k_realistic/clang5 139MB/s ± 1% 125MB/s ± 1% -9.69% (p=0.000 n=9+10)
wuffs_gif_decode_1000k_full_init/clang5 141MB/s ± 0% 126MB/s ± 0% -10.40% (p=0.000 n=10+10)
wuffs_gif_decode_1000k_part_init/clang5 141MB/s ± 1% 126MB/s ± 0% -10.50% (p=0.000 n=10+8)
wuffs_gif_decode_anim_screencap/clang5 655MB/s ± 0% 617MB/s ± 1% -5.91% (p=0.000 n=9+9)
wuffs_gif_decode_1k_bw/gcc7 299MB/s ± 0% 295MB/s ± 0% -1.18% (p=0.000 n=10+8)
wuffs_gif_decode_1k_color_full_init/gcc7 85.2MB/s ± 1% 88.2MB/s ± 0% +3.58% (p=0.000 n=10+10)
wuffs_gif_decode_1k_color_part_init/gcc7 104MB/s ± 0% 103MB/s ± 0% -0.39% (p=0.001 n=9+10)
wuffs_gif_decode_10k_bgra/gcc7 349MB/s ± 2% 352MB/s ± 0% +0.77% (p=0.000 n=9+10)
wuffs_gif_decode_10k_indexed/gcc7 106MB/s ± 0% 106MB/s ± 2% ~ (p=0.762 n=8+10)
wuffs_gif_decode_20k/gcc7 133MB/s ± 0% 133MB/s ± 0% ~ (p=0.258 n=9+9)
wuffs_gif_decode_100k_artificial/gcc7 302MB/s ± 0% 301MB/s ± 1% -0.41% (p=0.035 n=10+10)
wuffs_gif_decode_100k_realistic/gcc7 118MB/s ± 1% 118MB/s ± 1% ~ (p=0.095 n=9+10)
wuffs_gif_decode_1000k_full_init/gcc7 120MB/s ± 1% 119MB/s ± 0% -0.34% (p=0.007 n=10+10)
wuffs_gif_decode_1000k_part_init/gcc7 119MB/s ± 1% 119MB/s ± 1% ~ (p=0.138 n=10+10)
wuffs_gif_decode_anim_screencap/gcc7 640MB/s ± 0% 640MB/s ± 0% ~ (p=0.905 n=9+10)
2020-05-26 14:07:30 +10:00