Commit Graph
104 Commits
Author SHA1 Message Date
Nigel Tao 054c6c964b strconv: drop base from wuffs_base__private_impl 2023-10-29 21:59:52 +11:00
Patrick Georgi f563f32f82 Add SPDX identifier to license headers
SPDX is an ISO standard for describing licenses in a computer readable
format, which is useful when applying license compliance tools and the
like to source code.

In addition to ensuring that the Wuffs C distribution is properly
annotated to the benefit of projects using Wuffs and SPDX-supporting
tools, add it everywhere else where there's a license in this repo for
consistency, too.
2023-10-06 20:52:36 +02:00
Nigel Tao 74a6ea2fcd all: relicense from Apache2 to dual Apache2/MIT
The LICENSE has changed from a single license (Apache2) to a dual
license (Apache2 or MIT, at your option).

If you've previously used Wuffs under the Apache2 license, you are able
to keep doing so. No further action required by you.

The two files listed below (previous releases) are exceptions, as they
are older snapshots, frozen in time. Their license has not changed:
release/c/wuffs-v0.2.c
release/c/wuffs-v0.3.c

Every name in the AUTHORS file, other than Google Inc., has signed
Google's CLA (Contributor License Agreement).
2023-10-06 10:00:59 +11:00
Nigel Tao 38205517d4 test: fix some errors in "compile C as C++" 2023-06-20 17:25:56 +10:00
Nigel Tao 6e84987d33 std: change set_quirk_enabled to set_quirk 2023-04-14 21:42:15 +10:00
Nigel Tao 00d5e35865 floatconv: optimize parse_number_f64 fallback
The time taken running script/manual-test-parse-number-f64.cc on the
parse-number-fxx-test-data suite falls from 3.565s to 2.491s (a 1.43x
ratio; the inverse is 0.70 = 100% - 30%).

Fixes #113
2023-04-08 14:56:32 +10:00
Nigel Tao 3474af988b std/{cbor,json}: add test for empty input
Updates #96
2023-02-01 21:43:25 +11:00
Nigel Tao a55e0a02a2 Fix test/c/std/json.c -Wunused-but-set-variable 2022-02-08 22:08:14 +11:00
Nigel Tao d5075c7e94 Push "int i" into "for (int i = etc; etc; etc)" 2021-11-02 14:07:58 +11:00
Nigel Tao 75faea996d Add parse "122.416294033786585" test case 2021-08-22 16:02:44 +10:00
Nigel Tao 77f855206d Have JSON line-comment tokens exclude the "\n"
Before this commit, line-comment token chains included the final "\n"
the way block-comments included the final "*/". The "\n" was mandatory.

However, when parsing with both QUIRK_ALLOW_COMMENT_LINE and
QUIRK_ALLOW_TRAILING_FILLER, input like "123//xyz" (without a "\n"
before the EOF) was rejected before and accepted after this commit.

So that line-comments are consistent, in that they either always or
never include the final "\n", they now never include the final "\n".

An input fragment (parsed with QUIRK_ALLOW_COMMENT_LINE) like
"//abc\n\t\t" was, before this commit, parsed as a 6 byte comment token
and a 2 byte other-filler token. That split is now 5+3.
2021-07-20 14:17:09 +10:00
Nigel Tao 40ffa334ee Add more json allow_trailing_comments test cases 2021-07-18 23:27:56 +10:00
Nigel Tao ad40102c5a Change mimic marker symbol from "!!" to "¿" 2021-03-25 22:22:27 +11:00
Nigel Tao 2f78804e62 Use more inclusive language 2021-01-23 19:50:47 +11:00
Nigel Tao a1c22caa15 Rename load and store to peek and poke 2021-01-17 22:44:39 +11:00
Nigel Tao 2b17cf72ca Fix uint64_t vs size_t when compiling with -m32 2021-01-08 23:07:07 +11:00
Nigel Tao ba3818cb6b Add more parse_number_f64 test cases 2020-09-28 12:51:45 +10:00
Nigel Tao cd4cbc9102 Tweak JSON trailing comment/new-line quirks 2020-09-22 23:03:49 +10:00
Nigel Tao fa50f4de67 Add json QUIRK_ALLOW_TRAILING_COMMENT 2020-09-21 11:24:22 +10:00
Nigel Tao ddac2c50ee Add more {parse,render}_number_f64 test cases 2020-09-15 21:17:33 +10:00
Nigel Tao 57d47c633c Fix etc__high_prec_dec__to_f64 shift amount
Shifting 0.299 left by 2 doesn't put us in [½ .. 1].
2020-09-08 16:43:31 +10:00
Nigel Tao 6c13877688 Drop WUFFS_JSON__QUIRK_ALLOW_BACKSLASH_X_AS_BYTES
We can always add it back later if there's demand for it.
2020-08-30 16:15:35 +10:00
Nigel Tao 1adbc9547f Split std/json's QUIRK_ALLOW_BACKSLASH_X in two 2020-08-06 23:28:07 +10:00
Nigel Tao 702c7b299f Let wuffs_base__utf_8__etc take ptr-to-const-u8 2020-07-22 15:42:54 +10:00
Nigel Tao a3931d5215 Rename IEEE 754 from_f64_to_u{16,32} functions 2020-07-12 21:06:44 +10:00
Nigel Tao 7bf7cf28b9 Add IEEE 754 from_f64_to_u{16,32} functions 2020-07-12 17:01:14 +10:00
Nigel Tao 0899b82c42 Add IEEE 754 from_u{16,32}_to_f64 functions 2020-07-12 13:23:04 +10:00
Nigel Tao 4d449dc663 Rename ieee_754_bit_representation functions 2020-07-12 11:00:47 +10:00
Nigel Tao 0a9075f81c Move test_wuffs_strconv_parse_number_f64_options 2020-07-11 13:17:58 +10:00
Nigel Tao 37bc8cdaa5 Refactor parse_number_f64 related tests 2020-07-11 13:16:00 +10:00
Nigel Tao c5c9885db4 Add PARSE_NUMBER_XXX__ALLOW_UNDERSCORES 2020-07-11 13:10:14 +10:00
Nigel Tao e82bc8ec7b Add PARSE_NUMBER_XXX__ALLOW_MULTIPLE_LEADING_Z... 2020-07-11 12:57:25 +10:00
Nigel Tao 4d61a056e0 Add PARSE_NUMBER_FXX__REJECT_INF_AND_NAN 2020-07-11 12:34:40 +10:00
Nigel Tao e0c5de9d8d Add PARSE_NUMBER_FXX__DECIMAL_SEPARATOR_IS_A_COMMA 2020-07-11 12:14:39 +10:00
Nigel Tao e0ca5a4c3c Have base_16 codec return a transform__output 2020-07-09 21:09:19 +10:00
Nigel Tao 309a5ba231 Add base_64 codec to wuffs_base 2020-07-09 20:42:17 +10:00
Nigel Tao 84bb3afd87 Lower json.DECODER_NUMBER_LENGTH_MAX_INCL to 99
Commit b9f69264a0 "Raise
json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047" might have regressed the
std/json benchmarks.
2020-07-07 23:29:30 +10:00
Nigel Tao b9f69264a0 Raise json.DECODER_NUMBER_LENGTH_MAX_INCL to 2047 2020-07-07 23:08:11 +10:00
Nigel Tao 6b7ce30e32 Add wuffs_base__parse_number_xxx options arg 2020-07-07 16:19:46 +10:00
Nigel Tao 832edf7227 Remove the medium_prec_bin type
It's obsolete now that we have an Eisel algorithm implementation.

Binary size, before:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
28744 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28968 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31176 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31176 gen/lib/c/gcc-static/wuffs-base-f64conv.o

Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.031s
real	0m0.975s
$ time ./a.out                   < citylots.json
Before/After:
real    0m0.807s
real	0m0.729s
2020-07-07 16:07:35 +10:00
Nigel Tao 86093c8194 Have parse_number_f64 use the Eisel algorithm
The algorithm is discussed at
https://lemire.me/blog/2020/03/10/fast-float-parsing-in-practice/

----

Binary size, before:
27848 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
28040 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
30392 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
30392 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
29416 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
29688 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
31792 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
31792 gen/lib/c/gcc-static/wuffs-base-f64conv.o

The binary size difference does not account for the powers-of-10 table,
which was expanded in a previous commit in anticipation of this one. A
future commit will also remove some of the other f64 parser code used
before implementing the Eisel algorithm.

----

Processing citylots.json from github.com/zemirco/sf-city-lots-json
$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.231s
real	0m1.031s

The difference with and without -parse-number-f64 measures how much time
is spent in the parse_number_f64 function. Before: 424ms, after: 224ms.
$ time ./a.out                   < citylots.json
real	0m0.807s

Similarly, for canada.json from github.com/miloyip/nativejson-benchmark
$ time ./a.out -parse-number-f64 < canada.json
Before/After:
real	0m0.023s
real	0m0.017s
$ time ./a.out                   < canada.json
real	0m0.013s
Time spent in parse_number_f64, before: 10ms, after: 4ms.

----

name                                                old time/op  new time/op   delta
wuffs_strconv_parse_number_f64_1_lsh53_add0/clang5  77.0ns ± 0%   37.0ns ± 0%  -51.95%  (p=0.029 n=4+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/clang5   690ns ± 0%    725ns ± 2%   +4.95%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/clang5       92.0ns ± 0%  136.0ns ± 2%  +47.83%  (p=0.016 n=4+5)
wuffs_strconv_parse_number_f64_pi_short/clang5      35.0ns ± 3%   18.0ns ± 0%  -48.57%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_1_lsh53_add0/gcc7    67.0ns ± 0%   34.0ns ± 0%  -49.25%  (p=0.000 n=5+4)
wuffs_strconv_parse_number_f64_1_lsh53_add1/gcc7     712ns ± 0%    760ns ± 2%   +6.72%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_long/gcc7         88.4ns ± 1%  116.2ns ± 3%  +31.45%  (p=0.008 n=5+5)
wuffs_strconv_parse_number_f64_pi_short/gcc7        32.6ns ± 2%   21.4ns ± 7%  -34.36%  (p=0.008 n=5+5)
2020-07-07 15:27:51 +10:00
Nigel Tao 267bfc8176 Deepen parse_number_f64's powers_of_10 table
Deepen means changing the mantissa from 64 bits to 128 bits. We also
adjust the exponent bias, in anticipation of a future commit.

Binary size, before:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
27336 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
27528 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
28856 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
28856 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real    0m1.432s
real	0m1.421s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 19:03:37 +10:00
Nigel Tao 53da68fe4b Refine parse_number_f64's powers_of_10 table
Refine means changing the base-10 exponents from "-348 ..= +340 stepping
by 8" to "-326 ..= +310 stepping by 1".

Binary size, before:
15992 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
16232 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
17384 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
17384 gen/lib/c/gcc-static/wuffs-base-f64conv.o

After:
22232 gen/lib/c/clang-5.0-dynamic/wuffs-base-f64conv.lo
22408 gen/lib/c/clang-5.0-static/wuffs-base-f64conv.o
23816 gen/lib/c/gcc-dynamic/wuffs-base-f64conv.lo
23816 gen/lib/c/gcc-static/wuffs-base-f64conv.o

$ g++ -O3 script/process-json-numbers.c
$ time ./a.out -parse-number-f64 < citylots.json
Before/After:
real	0m1.455s
real	0m1.432s

citylots.json is from github.com/zemirco/sf-city-lots-json
2020-07-05 13:58:49 +10:00
Nigel Tao 19b3ce24a3 Optimize high_prec_dec__round_just_enough
name                                                               old time/op  new time/op  delta
wuffs_strconv_render_number_f64_just_enough_fractions/clang5       56.4µs ± 1%  56.3µs ± 1%     ~     (p=1.000 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/clang5  4.12µs ± 2%  1.00µs ± 1%  -75.65%  (p=0.008 n=5+5)
wuffs_strconv_render_number_f64_just_enough_fractions/gcc7         54.3µs ± 0%  54.2µs ± 0%     ~     (p=0.151 n=5+5)
wuffs_strconv_render_number_f64_just_enough_small_integers/gcc7    4.16µs ± 0%  1.01µs ± 0%  -75.77%  (p=0.000 n=4+5)
2020-07-03 23:44:37 +10:00
Nigel Tao 58678cb342 Add wuffs_base__render_number_f64 benches 2020-07-03 23:16:53 +10:00
Nigel Tao b9afc26387 Add TOKEN__VBD__STRING__CHAIN_ETC 2020-07-03 13:36:13 +10:00
Nigel Tao a6031b6d2e Add wuffs_base__utf_8__next_from_end 2020-07-02 15:08:54 +10:00
Nigel Tao 89ceb9fabc Add wuffs_base__render_number_f64 2020-07-01 12:03:20 +10:00
Nigel Tao d40a386211 Add wuffs_base__render_number_{i,u}64 2020-06-21 21:57:48 +10:00
Nigel Tao f45f01f4f3 Update clang-format from v5 to v9 2020-06-05 14:06:56 +10:00