Commit Graph
542 Commits
Author SHA1 Message Date
Nigel Tao ddac2c50ee Add more {parse,render}_number_f64 test cases 2020-09-15 21:17:33 +10:00
Nigel Tao 57d47c633c Fix etc__high_prec_dec__to_f64 shift amount
Shifting 0.299 left by 2 doesn't put us in [½ .. 1].
2020-09-08 16:43:31 +10:00
Nigel Tao 6c13877688 Drop WUFFS_JSON__QUIRK_ALLOW_BACKSLASH_X_AS_BYTES
We can always add it back later if there's demand for it.
2020-08-30 16:15:35 +10:00
Nigel Tao 4284369bb8 Add TOKEN__VBD__FILLER__PUNCTUATION 2020-08-17 16:48:30 +10:00
Nigel Tao 2de1dfec97 Fix test/data/cbor*tokens losing DEFINITELY_ASCII
The change to the golden file was mistakenly left out of 2397d029 "Fix
std/cbor UTF-8 strings aren't always ASCII".
2020-08-12 23:32:40 +10:00
Nigel Tao 0091a2132d Have std/cbor reject tags before an 0xFF stop code 2020-08-10 21:39:37 +10:00
Nigel Tao 9497b174c9 Add test_wuffs_cbor_decode_{in,}valid 2020-08-10 21:22:47 +10:00
Nigel Tao 0b7eaf291b Add example/cbor-to-json 2020-08-09 23:03:42 +10:00
Nigel Tao 1adbc9547f Split std/json's QUIRK_ALLOW_BACKSLASH_X in two 2020-08-06 23:28:07 +10:00
Nigel Tao 1e96ef200c Add test/data/cbor-rfc-7049-examples.* 2020-07-27 22:00:28 +10:00
Nigel Tao d6a10dfe8a Add io_buffer.{read,writ}er_slice methods 2020-07-27 11:47:47 +10:00
Nigel Tao 702c7b299f Let wuffs_base__utf_8__etc take ptr-to-const-u8 2020-07-22 15:42:54 +10:00
Nigel Tao 4e193599ac Add std/cbor 2020-07-15 14:08:37 +10:00
Nigel Tao 168f60a781 Add example/jsonptr -output-format=cbor flag 2020-07-14 15:57:05 +10:00
Nigel Tao a3931d5215 Rename IEEE 754 from_f64_to_u{16,32} functions 2020-07-12 21:06:44 +10:00
Nigel Tao 7bf7cf28b9 Add IEEE 754 from_f64_to_u{16,32} functions 2020-07-12 17:01:14 +10:00
Nigel Tao 0899b82c42 Add IEEE 754 from_u{16,32}_to_f64 functions 2020-07-12 13:23:04 +10:00
Nigel Tao 4d449dc663 Rename ieee_754_bit_representation functions 2020-07-12 11:00:47 +10:00
Nigel Tao 0a9075f81c Move test_wuffs_strconv_parse_number_f64_options 2020-07-11 13:17:58 +10:00
Nigel Tao 37bc8cdaa5 Refactor parse_number_f64 related tests 2020-07-11 13:16:00 +10:00
Nigel Tao c5c9885db4 Add PARSE_NUMBER_XXX__ALLOW_UNDERSCORES 2020-07-11 13:10:14 +10:00
Nigel Tao e82bc8ec7b Add PARSE_NUMBER_XXX__ALLOW_MULTIPLE_LEADING_Z... 2020-07-11 12:57:25 +10:00
Nigel Tao 4d61a056e0 Add PARSE_NUMBER_FXX__REJECT_INF_AND_NAN 2020-07-11 12:34:40 +10:00
Nigel Tao e0c5de9d8d Add PARSE_NUMBER_FXX__DECIMAL_SEPARATOR_IS_A_COMMA 2020-07-11 12:14:39 +10:00
Nigel Tao e0ca5a4c3c Have base_16 codec return a transform__output 2020-07-09 21:09:19 +10:00
Nigel Tao 309a5ba231 Add base_64 codec to wuffs_base 2020-07-09 20:42:17 +10:00
Nigel Tao 84bb3afd87 Lower json.DECODER_NUMBER_LENGTH_MAX_INCL to 99
Commit b9f69264a0 "Raise
json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047" might have regressed the
std/json benchmarks.
2020-07-07 23:29:30 +10:00
Nigel Tao b9f69264a0 Raise json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047 2020-07-07 23:08:11 +10:00
Nigel Tao 6b7ce30e32 Add wuffs_base__parse_number_xxx options arg 2020-07-07 16:19:46 +10:00
Nigel Tao 832edf7227 Remove the medium_prec_bin type
It's obsolete now that we have an Eisel algorithm implementation.

Binary size, before:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
28744 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28968 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31176 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31176 gen/lib/c/gcc-static/wuffs-base-f64conv.o

Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.031s
real	0m0.975s
$ time ./a.out                   < citylots.json
Before/After:
real    0m0.807s
real	0m0.729s
2020-07-07 16:07:35 +10:00
Nigel Tao 86093c8194 Have parse_number_f64 use the Eisel algorithm
The algorithm is discussed at
https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/

----

Binary size, before:
27848 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28040 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
30392 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
30392 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o

The binary size difference does not account for the powers-of-10 table,
which was expanded in a previous commit in anticipation of this one. A
future commit will also remove some of the other f64 parser code used
before implementing the Eisel algorithm.

----

Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.231s
real	0m1.031s

The difference with and without -parse-number-f64 measures how much time
is spent in the parse_number_f64 function. Before: 424ms, after: 224ms.
$ time ./a.out                   < citylots.json
real	0m0.807s

Similarly, for canada.json from github.com/miloyip/nativejson-benchmark
$ time ./a.out -parse-number-f64 < canada.json
Before/After:
real	0m0.023s
real	0m0.017s
$ time ./a.out                   < canada.json
real	0m0.013s
Time spent in parse_number_f64, before: 10ms, after: 4ms.

----

name                                                old time/op  new time/op   delta
wuffs_strconv_parse_number_f64_1_lsh53_add0/clang5  77.0ns ± 0%   37.0ns ± 0%  -51.95%  (p=0.029 n=4+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/clang5   690ns ± 0%    725ns ± 2%   +4.95%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/clang5       92.0ns ± 0%  136.0ns ± 2%  +47.83%  (p=0.016 n=4+5)
wuffs_strconv_parse_number_f64_pi_short/clang5      35.0ns ± 3%   18.0ns ± 0%  -48.57%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_1_lsh53_add0/gcc7    67.0ns ± 0%   34.0ns ± 0%  -49.25%  (p=0.000 n=5+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/gcc7     712ns ± 0%    760ns ± 2%   +6.72%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/gcc7         88.4ns ± 1%  116.2ns ± 3%  +31.45%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_short/gcc7        32.6ns ± 2%   21.4ns ± 7%  -34.36%  (p=0.008 n=5+5)
2020-07-07 15:27:51 +10:00
Nigel Tao 267bfc8176 Deepen parse_number_f64's powers_of_10 table
Deepen means changing the mantissa from 64 bits to 128 bits. We also
adjust the exponent bias, in anticipation of a future commit.

Binary size, before:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
27336 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
27528 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
28856 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
28856 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.432s
real	0m1.421s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 19:03:37 +10:00
Nigel Tao 53da68fe4b Refine parse_number_f64's powers_of_10 table
Refine means changing the base-10 exponents from "-348 ..= +340 stepping
by 8" to "-326 ..= +310 stepping by 1".

Binary size, before:
15992 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
16232 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
17384 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
17384 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real	0m1.455s
real	0m1.432s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 13:58:49 +10:00
Nigel Tao 19b3ce24a3 Optimize high_prec_dec__round_just_enough
name                                                               old time/op  new time/op  delta
wuffs_strconv_render_number_f64_just_enough_fractions/clang5       56.4µs ± 1%  56.3µs ± 1%     ~     (p=1.000 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/clang5  4.12µs ± 2%  1.00µs ± 1%  -75.65%  (p=0.008 n=5+5)
wuffs_strconv_render_number_f64_just_enough_fractions/gcc7         54.3µs ± 0%  54.2µs ± 0%     ~     (p=0.151 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/gcc7    4.16µs ± 0%  1.01µs ± 0%  -75.77%  (p=0.000 n=4+5)
2020-07-03 23:44:37 +10:00
Nigel Tao 58678cb342 Add wuffs_base__render_number_f64 benches 2020-07-03 23:16:53 +10:00
Nigel Tao b9afc26387 Add TOKEN__VBD__STRING__CHAIN_ETC 2020-07-03 13:36:13 +10:00
Nigel Tao a6031b6d2e Add wuffs_base__utf_8__next_from_end 2020-07-02 15:08:54 +10:00
Nigel Tao 89ceb9fabc Add wuffs_base__render_number_f64 2020-07-01 12:03:20 +10:00
Nigel Tao d40a386211 Add wuffs_base__render_number_{i,u}64 2020-06-21 21:57:48 +10:00
Nigel Tao f45f01f4f3 Update clang-format from v5 to v9 2020-06-05 14:06:56 +10:00
Nigel Tao d7c9e9bbc9 De-duplicate the wuffs_gif_decode test function 2020-06-01 14:33:19 +10:00
Nigel Tao e6b7c51557 Inline do_bench_gif_decode 2020-06-01 14:23:59 +10:00
Nigel Tao 829519880a Drop unnecessary memcpy in do_bench_image_decode
This doesn't affect the std/bmp or std/gif implementations per se, only
the code that benches them. Prior to this commit, the source file was
first decoded into a pixel buffer and then always copied from the
(2-dimensional) wuffs_base__pixel_buffer to a (1-dimensional)
wuffs_base_io_buffer. That copy is useful for *tests*, which compare
that io_buffer's contents to a known good result (i.e. a separate I/O
buffer filled with the contents of a golden file), but it isn't
necessary for *benches*. Dropping that unnecessary copy reduces the
total time per iteration, and therefore increases the reported speed.

name                                        old speed      new speed       delta

wuffs_bmp_decode_40k/clang5                 2.09GB/s ± 2%   2.48GB/s ± 0%  +18.78%  (p=0.008 n=5+5)

wuffs_bmp_decode_40k/gcc7                   1.73GB/s ± 1%   1.93GB/s ± 0%  +11.54%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/clang5                291MB/s ± 0%    376MB/s ± 1%  +29.14%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_full_init/clang5  95.3MB/s ± 1%  103.2MB/s ± 1%   +8.37%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/clang5   117MB/s ± 0%    130MB/s ± 0%  +10.68%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/clang5             469MB/s ± 0%    489MB/s ± 1%   +4.39%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_indexed/clang5          126MB/s ± 1%    130MB/s ± 1%   +3.50%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/clang5                  152MB/s ± 0%    156MB/s ± 1%   +2.77%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/clang5      318MB/s ± 0%    329MB/s ± 0%   +3.53%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/clang5       137MB/s ± 0%    139MB/s ± 0%   +1.40%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/clang5      138MB/s ± 0%    140MB/s ± 1%   +1.45%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/clang5      139MB/s ± 0%    140MB/s ± 1%   +1.03%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/clang5       661MB/s ± 0%    747MB/s ± 1%  +13.08%  (p=0.008 n=5+5)

mimic_gif_decode_1k_bw/clang5               92.0MB/s ± 0%   91.9MB/s ± 1%     ~     (p=0.841 n=5+5)
mimic_gif_decode_1k_color/clang5            45.5MB/s ± 0%   45.7MB/s ± 0%   +0.45%  (p=0.032 n=5+4)
mimic_gif_decode_10k_indexed/clang5         53.5MB/s ± 0%   53.7MB/s ± 0%   +0.31%  (p=0.032 n=5+4)
mimic_gif_decode_20k/clang5                 55.5MB/s ± 0%   55.1MB/s ± 2%     ~     (p=0.690 n=5+5)
mimic_gif_decode_100k_artificial/clang5     88.8MB/s ± 0%   89.0MB/s ± 1%     ~     (p=0.151 n=5+5)
mimic_gif_decode_100k_realistic/clang5      53.5MB/s ± 1%   53.7MB/s ± 0%   +0.31%  (p=0.024 n=5+5)
mimic_gif_decode_1000k/clang5               54.6MB/s ± 0%   54.6MB/s ± 1%     ~     (p=0.222 n=5+5)
mimic_gif_decode_anim_screencap/clang5       106MB/s ± 0%    109MB/s ± 0%   +2.59%  (p=0.008 n=5+5)

wuffs_gif_decode_1k_bw/gcc7                  284MB/s ± 0%    356MB/s ± 0%  +25.33%  (p=0.016 n=4+5)
wuffs_gif_decode_1k_color_full_init/gcc7    82.1MB/s ± 0%   95.3MB/s ± 0%  +16.08%  (p=0.008 n=5+5)
wuffs_gif_decode_1k_color_part_init/gcc7    97.5MB/s ± 0%  117.4MB/s ± 0%  +20.39%  (p=0.008 n=5+5)
wuffs_gif_decode_10k_bgra/gcc7               345MB/s ± 0%    396MB/s ± 0%  +14.71%  (p=0.016 n=4+5)
wuffs_gif_decode_10k_indexed/gcc7            104MB/s ± 0%    121MB/s ± 0%  +16.38%  (p=0.008 n=5+5)
wuffs_gif_decode_20k/gcc7                    131MB/s ± 0%    149MB/s ± 0%  +13.55%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_artificial/gcc7        293MB/s ± 0%    321MB/s ± 1%   +9.77%  (p=0.008 n=5+5)
wuffs_gif_decode_100k_realistic/gcc7         116MB/s ± 1%    130MB/s ± 1%  +12.10%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_full_init/gcc7        117MB/s ± 1%    132MB/s ± 1%  +12.53%  (p=0.008 n=5+5)
wuffs_gif_decode_1000k_part_init/gcc7        117MB/s ± 0%    132MB/s ± 1%  +12.20%  (p=0.008 n=5+5)
wuffs_gif_decode_anim_screencap/gcc7         615MB/s ± 2%    711MB/s ± 2%  +15.53%  (p=0.008 n=5+5)

mimic_gif_decode_1k_bw/gcc7                 91.4MB/s ± 0%   91.7MB/s ± 1%     ~     (p=0.095 n=5+5)
mimic_gif_decode_1k_color/gcc7              46.3MB/s ± 0%   46.2MB/s ± 2%     ~     (p=0.841 n=5+5)
mimic_gif_decode_10k_indexed/gcc7           51.9MB/s ± 1%   51.7MB/s ± 1%     ~     (p=0.310 n=5+5)
mimic_gif_decode_20k/gcc7                   54.2MB/s ± 0%   54.2MB/s ± 1%     ~     (p=0.421 n=5+5)
mimic_gif_decode_100k_artificial/gcc7       88.4MB/s ± 1%   88.7MB/s ± 1%     ~     (p=0.222 n=5+5)
mimic_gif_decode_100k_realistic/gcc7        53.7MB/s ± 0%   53.7MB/s ± 0%     ~     (p=0.206 n=5+5)
mimic_gif_decode_1000k/gcc7                 53.8MB/s ± 2%   54.0MB/s ± 2%     ~     (p=0.690 n=5+5)
mimic_gif_decode_anim_screencap/gcc7         106MB/s ± 0%    109MB/s ± 0%   +2.79%  (p=0.008 n=5+5)
2020-06-01 13:59:50 +10:00
Nigel Tao 4b9b971436 Harmonize do_bench_{gif,image}_decode 2020-06-01 13:27:46 +10:00
Nigel Tao 2784613b49 Add n_bytes_out arg to gif bench functions 2020-06-01 13:14:14 +10:00
Nigel Tao 7a85f67b22 Add std/bmp benchmark 2020-06-01 13:03:03 +10:00
Nigel Tao 54196c4e80 Rename pixel_swizzler.swizzle_interleaved 2020-05-29 16:54:05 +10:00
Nigel Tao 6537a817b7 Roll back the two previous "@I/O redirect" commits
See the previous commit for the rationale.
2020-05-29 11:27:51 +10:00
Nigel Tao 844fc9b5ee Temporarily avoid using base."@I/O redirect"
This commit (and the next) will soon be followed by a rollback.
2020-05-29 09:53:30 +10:00
Nigel Tao dcb81a85bb Prepare to decode BMP embedding a JPEG or PNG 2020-05-27 13:37:54 +10:00