The time taken running script/manual-test-parse-number-f64.cc on the
parse-number-fxx-test-data suite falls from 3.565s to 2.491s (a 1.43x
ratio; the inverse is 0.70 = 100% - 30%).
Fixes#113
Before this commit, line-comment token chains included the final "\n"
the way block-comments included the final "*/". The "\n" was mandatory.
However, when parsing with both QUIRK_ALLOW_COMMENT_LINE and
QUIRK_ALLOW_TRAILING_FILLER, input like "123//xyz" (without a "\n"
before the EOF) was rejected before and accepted after this commit.
So that line-comments are consistent, in that they either always or
never include the final "\n", they now never include the final "\n".
An input fragment (parsed with QUIRK_ALLOW_COMMENT_LINE) like
"//abc\n\t\t" was, before this commit, parsed as a 6 byte comment token
and a 2 byte other-filler token. That split is now 5+3.
It's obsolete now that we have an Eisel algorithm implementation.
Binary size, before:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
28744 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28968 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31176 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31176 gen/lib/c/gcc-static/wuffs-base-f64conv.o
Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.031s
real 0m0.975s
$ time ./a.out < citylots.json
Before/After:
real 0m0.807s
real 0m0.729s
The algorithm is discussed at
https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/
----
Binary size, before:
27848 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28040 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
30392 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
30392 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o
The binary size difference does not account for the powers-of-10 table,
which was expanded in a previous commit in anticipation of this one. A
future commit will also remove some of the other f64 parser code used
before implementing the Eisel algorithm.
----
Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.231s
real 0m1.031s
The difference with and without -parse-number-f64 measures how much time
is spent in the parse_number_f64 function. Before: 424ms, after: 224ms.
$ time ./a.out < citylots.json
real 0m0.807s
Similarly, for canada.json from github.com/miloyip/nativejson-benchmark
$ time ./a.out -parse-number-f64 < canada.json
Before/After:
real 0m0.023s
real 0m0.017s
$ time ./a.out < canada.json
real 0m0.013s
Time spent in parse_number_f64, before: 10ms, after: 4ms.
----
name old time/op new time/op delta
wuffs_strconv_parse_number_f64_1_lsh53_add0/clang5 77.0ns ± 0% 37.0ns ± 0% -51.95% (p=0.029 n=4+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/clang5 690ns ± 0% 725ns ± 2% +4.95% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/clang5 92.0ns ± 0% 136.0ns ± 2% +47.83% (p=0.016 n=4+5)
wuffs_strconv_parse_number_f64_pi_short/clang5 35.0ns ± 3% 18.0ns ± 0% -48.57% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_1_lsh53_add0/gcc7 67.0ns ± 0% 34.0ns ± 0% -49.25% (p=0.000 n=5+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/gcc7 712ns ± 0% 760ns ± 2% +6.72% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/gcc7 88.4ns ± 1% 116.2ns ± 3% +31.45% (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_short/gcc7 32.6ns ± 2% 21.4ns ± 7% -34.36% (p=0.008 n=5+5)
Deepen means changing the mantissa from 64 bits to 128 bits. We also
adjust the exponent bias, in anticipation of a future commit.
Binary size, before:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o
After:
27336 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
27528 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
28856 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
28856 gen/lib/c/gcc-static/wuffs-base-f64conv.o
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real 0m1.432s
real 0m1.421s
citylots.json is from github.com/zemirco/sf-city-lots-json