Commit Graph
64 Commits
Author SHA1 Message Date
Nigel Tao 86093c8194 Have parse_number_f64 use the Eisel algorithm
The algorithm is discussed at
https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/

----

Binary size, before:
27848 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28040 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
30392 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
30392 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o

The binary size difference does not account for the powers-of-10 table,
which was expanded in a previous commit in anticipation of this one. A
future commit will also remove some of the other f64 parser code used
before implementing the Eisel algorithm.

----

Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.231s
real	0m1.031s

The difference with and without -parse-number-f64 measures how much time
is spent in the parse_number_f64 function. Before: 424ms, after: 224ms.
$ time ./a.out                   < citylots.json
real	0m0.807s

Similarly, for canada.json from github.com/miloyip/nativejson-benchmark
$ time ./a.out -parse-number-f64 < canada.json
Before/After:
real	0m0.023s
real	0m0.017s
$ time ./a.out                   < canada.json
real	0m0.013s
Time spent in parse_number_f64, before: 10ms, after: 4ms.

----

name                                                old time/op  new time/op   delta
wuffs_strconv_parse_number_f64_1_lsh53_add0/clang5  77.0ns ± 0%   37.0ns ± 0%  -51.95%  (p=0.029 n=4+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/clang5   690ns ± 0%    725ns ± 2%   +4.95%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/clang5       92.0ns ± 0%  136.0ns ± 2%  +47.83%  (p=0.016 n=4+5)
wuffs_strconv_parse_number_f64_pi_short/clang5      35.0ns ± 3%   18.0ns ± 0%  -48.57%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_1_lsh53_add0/gcc7    67.0ns ± 0%   34.0ns ± 0%  -49.25%  (p=0.000 n=5+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/gcc7     712ns ± 0%    760ns ± 2%   +6.72%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/gcc7         88.4ns ± 1%  116.2ns ± 3%  +31.45%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_short/gcc7        32.6ns ± 2%   21.4ns ± 7%  -34.36%  (p=0.008 n=5+5)
2020-07-07 15:27:51 +10:00
Nigel Tao 267bfc8176 Deepen parse_number_f64's powers_of_10 table
Deepen means changing the mantissa from 64 bits to 128 bits. We also
adjust the exponent bias, in anticipation of a future commit.

Binary size, before:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
27336 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
27528 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
28856 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
28856 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.432s
real	0m1.421s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 19:03:37 +10:00
Nigel Tao 53da68fe4b Refine parse_number_f64's powers_of_10 table
Refine means changing the base-10 exponents from "-348 ..= +340 stepping
by 8" to "-326 ..= +310 stepping by 1".

Binary size, before:
15992 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
16232 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
17384 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
17384 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real	0m1.455s
real	0m1.432s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 13:58:49 +10:00
Nigel Tao 19b3ce24a3 Optimize high_prec_dec__round_just_enough
name                                                               old time/op  new time/op  delta
wuffs_strconv_render_number_f64_just_enough_fractions/clang5       56.4µs ± 1%  56.3µs ± 1%     ~     (p=1.000 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/clang5  4.12µs ± 2%  1.00µs ± 1%  -75.65%  (p=0.008 n=5+5)
wuffs_strconv_render_number_f64_just_enough_fractions/gcc7         54.3µs ± 0%  54.2µs ± 0%     ~     (p=0.151 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/gcc7    4.16µs ± 0%  1.01µs ± 0%  -75.77%  (p=0.000 n=4+5)
2020-07-03 23:44:37 +10:00
Nigel Tao 58678cb342 Add wuffs_base__render_number_f64 benches 2020-07-03 23:16:53 +10:00
Nigel Tao b9afc26387 Add TOKEN__VBD__STRING__CHAIN_ETC 2020-07-03 13:36:13 +10:00
Nigel Tao a6031b6d2e Add wuffs_base__utf_8__next_from_end 2020-07-02 15:08:54 +10:00
Nigel Tao 89ceb9fabc Add wuffs_base__render_number_f64 2020-07-01 12:03:20 +10:00
Nigel Tao d40a386211 Add wuffs_base__render_number_{i,u}64 2020-06-21 21:57:48 +10:00
Nigel Tao f45f01f4f3 Update clang-format from v5 to v9 2020-06-05 14:06:56 +10:00
Nigel Tao d83bd8d036 Add std/json quirk_allow_ascii_control_codes 2020-04-07 17:50:47 +10:00
Nigel Tao 6f67bbc4b2 Add std/json quirk_allow_backslash_new_line 2020-04-07 17:24:31 +10:00
Nigel Tao 64dbd00b07 Rename wuffs_base__make_etc_buffer_{read,writ}er 2020-04-02 22:21:36 +11:00
Nigel Tao 462f86675d Add an "extended token" concept 2020-04-02 11:20:45 +11:00
Nigel Tao d9d022b2df Fix std/json test/bench naming convention typos 2020-03-31 22:49:19 +11:00
Nigel Tao 17944aae67 Optimize wuffs_base__parse_number_f64
name                                          old time/op   new time/op   delta

strconv_parse_number_f64_1_lsh53_add0/clang5    100ns ± 0%    100ns ± 1%     ~     (p=1.000 n=4+5)
strconv_parse_number_f64_1_lsh53_add1/clang5    741ns ± 1%    749ns ± 0%   +1.05%  (p=0.024 n=5+5)
strconv_parse_number_f64_pi_long/clang5         153ns ± 1%    153ns ± 0%     ~     (p=0.238 n=5+4)
strconv_parse_number_f64_pi_short/clang5       62.0ns ± 2%   45.4ns ± 3%  -26.77%  (p=0.008 n=5+5)

strconv_parse_number_f64_1_lsh53_add0/gcc7     96.6ns ± 4%   95.8ns ± 1%     ~     (p=0.794 n=5+5)
strconv_parse_number_f64_1_lsh53_add1/gcc7      757ns ± 0%    752ns ± 0%   -0.63%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/gcc7           151ns ± 3%    162ns ± 0%   +7.43%  (p=0.016 n=5+4)
strconv_parse_number_f64_pi_short/gcc7         55.0ns ± 0%   49.0ns ± 2%  -10.91%  (p=0.000 n=4+5)
2020-03-31 22:20:01 +11:00
Nigel Tao d587c9e4fa Add medium-precision-binary floating point impl
name                                          old time/op   new time/op   delta

strconv_parse_number_f64_1_lsh53_add0/clang5    218ns ± 1%    100ns ± 0%  -54.13%  (p=0.016 n=5+4)
strconv_parse_number_f64_1_lsh53_add1/clang5    716ns ± 2%    741ns ± 1%   +3.46%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/clang5         438ns ± 2%    153ns ± 1%  -65.19%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_short/clang5        174ns ± 2%     62ns ± 2%  -64.37%  (p=0.008 n=5+5)

strconv_parse_number_f64_1_lsh53_add0/gcc7      221ns ± 1%     97ns ± 4%  -56.25%  (p=0.008 n=5+5)
strconv_parse_number_f64_1_lsh53_add1/gcc7      742ns ± 1%    757ns ± 0%   +2.02%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/gcc7           477ns ± 1%    151ns ± 3%  -68.40%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_short/gcc7          191ns ± 1%     55ns ± 0%  -71.14%  (p=0.000 n=5+4)
2020-03-31 22:18:54 +11:00
Nigel Tao 3f0d66285f Add bench_strconv_parse_number_f64_etc 2020-03-28 11:16:06 +11:00
Nigel Tao b72dd16f9b Have test/* global variable names start with g_ 2020-03-27 17:12:38 +11:00
Nigel Tao f3146c2ea9 Add base.token_decoder.workbuf_len 2020-03-26 08:47:42 +11:00
Nigel Tao 3bfed6d812 Rename std/json quirk_replace_invalid_utf_8 2020-03-24 17:44:47 +11:00
Nigel Tao 5d5a17cc9c Add test/data/json-quirks.json 2020-03-24 17:13:07 +11:00
Nigel Tao 60e928f7c9 Fix -Wmaybe-uninitialized in test/c/std/json 2020-03-24 14:04:41 +11:00
Nigel Tao 93882564fd Rename std/json quirk_allow_final_comma 2020-03-24 14:00:21 +11:00
Nigel Tao d7c72373f3 Add std/json quirk_allow_inf_nan_numbers 2020-03-24 13:58:38 +11:00
Nigel Tao b43e2493a2 Add std/json quirk_allow_final_comma 2020-03-22 21:42:15 +11:00
Nigel Tao eb06ed77ba Add std/json quirk_allow_comment_etc 2020-03-22 21:07:12 +11:00
Nigel Tao 502c8eff56 Add std/json quirk_allow_backslash_x 2020-03-21 21:42:30 +11:00
Nigel Tao c6b2aded30 Add wuffs_base__hexadecimal__decodeN 2020-03-20 22:48:06 +11:00
Nigel Tao 83e7da0a6a Add std/json quirk_allow_backslash_capital_u 2020-03-19 23:20:45 +11:00
Nigel Tao 97f7f28218 Add std/json backslash-related quirks 2020-03-19 15:56:21 +11:00
Nigel Tao 31cfaaeb4e Add std/json quirk_replace_invalid_utf_8 2020-03-19 10:37:18 +11:00
Nigel Tao c5c10e95b7 Remove some superfluous empty // comments 2020-03-18 21:58:19 +11:00
Nigel Tao 4b186b0b6b Add std/json quirk_allow_trailing_etc 2020-03-18 16:42:06 +11:00
Nigel Tao 3a75d9746e Add std/json quirk_allow_leading_etc 2020-03-18 13:15:04 +11:00
Nigel Tao 791437bb47 Have std/json return base."@end of data" 2020-03-17 21:21:39 +11:00
Nigel Tao 9fd96e8e44 Add io_buffer and token_buffer methods 2020-03-17 13:38:37 +11:00
Nigel Tao 982f79f6dd Add test_strconv_parse_number_f64 test case 2020-03-13 09:44:02 +11:00
Nigel Tao 499ca132f6 Add wuffs_base__parse_number_f64 2020-03-11 11:41:32 +11:00
Nigel Tao fdac24a425 Rename FOO_BUFFER_SIZE to FOO_BUFFER_ARRAY_SIZE 2020-03-06 21:53:08 +11:00
Nigel Tao a5184edd6a Fix std/json's checking of number length too long
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21051
2020-03-06 21:05:44 +11:00
Nigel Tao b5662b2906 Fix test_strconv_utf_8_next array size 2020-03-05 10:22:34 +11:00
Nigel Tao 757a4bd68c Add more test_strconv_parse_number_u64 test cases 2020-03-04 22:18:56 +11:00
Nigel Tao 43f0a87a6d Test std/json emit pending strings on bad UTF-8 2020-03-02 13:59:46 +11:00
Nigel Tao 57b1dd69b3 Raise bench_wuffs_json_decode_217k_stringy iters 2020-02-29 10:48:09 +11:00
Nigel Tao 809220e6cc Add wuffs_base__utf_8__next 2020-02-27 14:41:45 +11:00
Nigel Tao 8cd45054fa Fix U+D800 typo in std/json test 2020-02-27 10:28:20 +11:00
Nigel Tao 193527bd2d Add script/print-json-token-debug-format.c 2020-02-26 22:47:31 +11:00
Nigel Tao 550ef90114 Have std/json reject invalid UTF-8 2020-02-26 16:26:16 +11:00
Nigel Tao 3b17d5b28a Have std/json reject some invalid UTF-8 2020-02-26 14:36:23 +11:00