Commit Graph
555 Commits
Author SHA1 Message Date
Nigel Tao 496e88b6c5 Remove wuffs_base__token__link_prev 2020-04-09 22:11:08 +10:00
Nigel Tao 478d1b817c Add doc/note/tokens.md 2020-04-08 23:31:18 +10:00
Nigel Tao d83bd8d036 Add std/json quirk_allow_ascii_control_codes 2020-04-07 17:50:47 +10:00
Nigel Tao 6f67bbc4b2 Add std/json quirk_allow_backslash_new_line 2020-04-07 17:24:31 +10:00
Nigel Tao 2db72dabe5 Re-assign JSON quirk numbers 2020-04-07 17:17:54 +10:00
Nigel Tao 695a72a37c Roll back 'factor out json.decoder.decode_string'
See the previous commit for the rationale.
2020-04-04 11:32:35 +11:00
Nigel Tao 86d3b89f9a Factor out json.decoder.decode_string
This commit will soon be followed by a rollback, but it is committed
anyway so that we can refer to these numbers in the git log.

name                                    old speed     new speed     delta

wuffs_json_decode_1k/clang5             607MB/s ± 0%  543MB/s ± 1%  -10.68%  (p=0.008 n=5+5)
wuffs_json_decode_21k_formatted/clang5  330MB/s ± 2%  320MB/s ± 3%     ~     (p=0.056 n=5+5)
wuffs_json_decode_26k_compact/clang5    468MB/s ± 2%  434MB/s ± 1%   -7.26%  (p=0.008 n=5+5)
wuffs_json_decode_217k_stringy/clang5   391MB/s ± 1%  363MB/s ± 0%   -7.03%  (p=0.008 n=5+5)

wuffs_json_decode_1k/gcc7               715MB/s ± 0%  583MB/s ± 0%  -18.38%  (p=0.016 n=5+4)
wuffs_json_decode_21k_formatted/gcc7    403MB/s ± 0%  350MB/s ± 0%  -12.94%  (p=0.016 n=5+4)
wuffs_json_decode_26k_compact/gcc7      534MB/s ± 0%  420MB/s ± 1%  -21.41%  (p=0.008 n=5+5)
wuffs_json_decode_217k_stringy/gcc7     435MB/s ± 0%  359MB/s ± 0%  -17.44%  (p=0.008 n=5+5)
2020-04-04 11:32:35 +11:00
Nigel Tao 2db88dd98e Fix -Wshadow warnings 2020-04-03 22:13:56 +11:00
Nigel Tao 1a1f36ce13 Disable clang's -Wunreachable-code warning 2020-04-03 21:57:49 +11:00
Nigel Tao ecd3285567 Simplify a std/json while as an if 2020-04-03 12:35:18 +11:00
Nigel Tao 64dbd00b07 Rename wuffs_base__make_etc_buffer_{read,writ}er 2020-04-02 22:21:36 +11:00
Nigel Tao 462f86675d Add an "extended token" concept 2020-04-02 11:20:45 +11:00
Nigel Tao 7edf71414f Fix std/json replace_invalid_unicode copy/pasto
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21465
2020-03-31 22:37:43 +11:00
Nigel Tao e2adb87cdd Fix etc__parse_number_f64 goto-crosses-init 2020-03-31 22:28:19 +11:00
Nigel Tao 17944aae67 Optimize wuffs_base__parse_number_f64
name                                          old time/op   new time/op   delta

strconv_parse_number_f64_1_lsh53_add0/clang5    100ns ± 0%    100ns ± 1%     ~     (p=1.000 n=4+5)
strconv_parse_number_f64_1_lsh53_add1/clang5    741ns ± 1%    749ns ± 0%   +1.05%  (p=0.024 n=5+5)
strconv_parse_number_f64_pi_long/clang5         153ns ± 1%    153ns ± 0%     ~     (p=0.238 n=5+4)
strconv_parse_number_f64_pi_short/clang5       62.0ns ± 2%   45.4ns ± 3%  -26.77%  (p=0.008 n=5+5)

strconv_parse_number_f64_1_lsh53_add0/gcc7     96.6ns ± 4%   95.8ns ± 1%     ~     (p=0.794 n=5+5)
strconv_parse_number_f64_1_lsh53_add1/gcc7      757ns ± 0%    752ns ± 0%   -0.63%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/gcc7           151ns ± 3%    162ns ± 0%   +7.43%  (p=0.016 n=5+4)
strconv_parse_number_f64_pi_short/gcc7         55.0ns ± 0%   49.0ns ± 2%  -10.91%  (p=0.000 n=4+5)
2020-03-31 22:20:01 +11:00
Nigel Tao d587c9e4fa Add medium-precision-binary floating point impl
name                                          old time/op   new time/op   delta

strconv_parse_number_f64_1_lsh53_add0/clang5    218ns ± 1%    100ns ± 0%  -54.13%  (p=0.016 n=5+4)
strconv_parse_number_f64_1_lsh53_add1/clang5    716ns ± 2%    741ns ± 1%   +3.46%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/clang5         438ns ± 2%    153ns ± 1%  -65.19%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_short/clang5        174ns ± 2%     62ns ± 2%  -64.37%  (p=0.008 n=5+5)

strconv_parse_number_f64_1_lsh53_add0/gcc7      221ns ± 1%     97ns ± 4%  -56.25%  (p=0.008 n=5+5)
strconv_parse_number_f64_1_lsh53_add1/gcc7      742ns ± 1%    757ns ± 0%   +2.02%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_long/gcc7           477ns ± 1%    151ns ± 3%  -68.40%  (p=0.008 n=5+5)
strconv_parse_number_f64_pi_short/gcc7          191ns ± 1%     55ns ± 0%  -71.14%  (p=0.000 n=5+4)
2020-03-31 22:18:54 +11:00
Nigel Tao f3146c2ea9 Add base.token_decoder.workbuf_len 2020-03-26 08:47:42 +11:00
Nigel Tao 40d200cdab Drop cached expr MBounds when re-checking asserts
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21371
2020-03-26 08:14:43 +11:00
Nigel Tao 58351d44da Make endwhile mandatory for nameless loops 2020-03-25 21:40:00 +11:00
Nigel Tao 3c38f671fa Remove empty "if (coro_susp_point) {}" 2020-03-25 21:24:00 +11:00
Nigel Tao 3bfed6d812 Rename std/json quirk_replace_invalid_utf_8 2020-03-24 17:44:47 +11:00
Nigel Tao 59eab68360 Fix -Werror=conversion in hexadecimal__decodeN 2020-03-24 17:22:17 +11:00
Nigel Tao 2ca0ef7bd5 Rename some std/json variables 2020-03-24 14:02:08 +11:00
Nigel Tao 93882564fd Rename std/json quirk_allow_final_comma 2020-03-24 14:00:21 +11:00
Nigel Tao d7c72373f3 Add std/json quirk_allow_inf_nan_numbers 2020-03-24 13:58:38 +11:00
Nigel Tao b43e2493a2 Add std/json quirk_allow_final_comma 2020-03-22 21:42:15 +11:00
Nigel Tao eb06ed77ba Add std/json quirk_allow_comment_etc 2020-03-22 21:07:12 +11:00
Nigel Tao 21f6a5df4a Add std/json CLASS_COMMENT 2020-03-21 22:33:39 +11:00
Nigel Tao 502c8eff56 Add std/json quirk_allow_backslash_x 2020-03-21 21:42:30 +11:00
Nigel Tao c6b2aded30 Add wuffs_base__hexadecimal__decodeN 2020-03-20 22:48:06 +11:00
Nigel Tao 83e7da0a6a Add std/json quirk_allow_backslash_capital_u 2020-03-19 23:20:45 +11:00
Nigel Tao 97f7f28218 Add std/json backslash-related quirks 2020-03-19 15:56:21 +11:00
Nigel Tao b86373aff5 Add io_reader.peek_u64le_at 2020-03-19 11:33:22 +11:00
Nigel Tao 31cfaaeb4e Add std/json quirk_replace_invalid_utf_8 2020-03-19 10:37:18 +11:00
Nigel Tao 4b186b0b6b Add std/json quirk_allow_trailing_etc 2020-03-18 16:42:06 +11:00
Nigel Tao 3a75d9746e Add std/json quirk_allow_leading_etc 2020-03-18 13:15:04 +11:00
Nigel Tao 791437bb47 Have std/json return base."@end of data" 2020-03-17 21:21:39 +11:00
Nigel Tao 9fd96e8e44 Add io_buffer and token_buffer methods 2020-03-17 13:38:37 +11:00
Nigel Tao a3827275c0 Add std/json quirks 2020-03-12 16:47:26 +11:00
Nigel Tao c3ca77a539 Add WUFFS_BASE__TOKEN__VBD__NUMBER__FORMAT_ETC 2020-03-11 12:06:50 +11:00
Nigel Tao 499ca132f6 Add wuffs_base__parse_number_f64 2020-03-11 11:41:32 +11:00
Nigel Tao a5184edd6a Fix std/json's checking of number length too long
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21051
2020-03-06 21:05:44 +11:00
Nigel Tao e65c861207 Extend std/deflate ringbuffer to simplify copies
name                                             old speed     new speed     delta

wuffs_deflate_decode_1k_full_init/clang5         155MB/s ± 1%  154MB/s ± 0%   -0.98%  (p=0.000 n=10+9)
wuffs_deflate_decode_1k_part_init/clang5         190MB/s ± 0%  193MB/s ± 1%   +1.49%  (p=0.000 n=10+9)
wuffs_deflate_decode_10k_full_init/clang5        240MB/s ± 0%  249MB/s ± 0%   +3.68%  (p=0.000 n=9+10)
wuffs_deflate_decode_10k_part_init/clang5        247MB/s ± 0%  257MB/s ± 1%   +4.03%  (p=0.000 n=10+10)
wuffs_deflate_decode_100k_just_one_read/clang5   292MB/s ± 1%  291MB/s ± 0%   -0.34%  (p=0.043 n=10+9)
wuffs_deflate_decode_100k_many_big_reads/clang5  246MB/s ± 0%  250MB/s ± 0%   +1.77%  (p=0.000 n=9+10)

wuffs_deflate_decode_1k_full_init/gcc9           166MB/s ± 1%  165MB/s ± 1%   -0.48%  (p=0.009 n=10+10)
wuffs_deflate_decode_1k_part_init/gcc9           208MB/s ± 1%  212MB/s ± 0%   +1.85%  (p=0.000 n=10+9)
wuffs_deflate_decode_10k_full_init/gcc9          248MB/s ± 1%  281MB/s ± 0%  +13.05%  (p=0.000 n=9+10)
wuffs_deflate_decode_10k_part_init/gcc9          256MB/s ± 0%  290MB/s ± 0%  +13.34%  (p=0.000 n=10+10)
wuffs_deflate_decode_100k_just_one_read/gcc9     336MB/s ± 0%  333MB/s ± 0%   -0.93%  (p=0.000 n=9+8)
wuffs_deflate_decode_100k_many_big_reads/gcc9    262MB/s ± 0%  280MB/s ± 1%   +6.99%  (p=0.000 n=9+10)
2020-03-05 12:35:03 +11:00
Nigel Tao 771d4f9d98 Add std/json lut_decimal_digits
name                                    old speed      new speed      delta

wuffs_json_decode_1k/clang5             1.13GB/s ± 0%  1.12GB/s ± 1%  -1.01%  (p=0.002 n=10+10)
wuffs_json_decode_21k_formatted/clang5   576MB/s ± 1%   595MB/s ± 1%  +3.29%  (p=0.000 n=10+10)
wuffs_json_decode_26k_compact/clang5     873MB/s ± 1%   907MB/s ± 1%  +3.81%  (p=0.000 n=10+10)
wuffs_json_decode_217k_stringy/clang5    727MB/s ± 0%   750MB/s ± 0%  +3.14%  (p=0.000 n=10+8)

wuffs_json_decode_1k/gcc9               1.10GB/s ± 1%  1.11GB/s ± 1%  +0.43%  (p=0.035 n=10+10)
wuffs_json_decode_21k_formatted/gcc9     628MB/s ± 0%   618MB/s ± 1%  -1.59%  (p=0.000 n=10+10)
wuffs_json_decode_26k_compact/gcc9       892MB/s ± 0%   888MB/s ± 1%  -0.43%  (p=0.023 n=10+10)
wuffs_json_decode_217k_stringy/gcc9      724MB/s ± 1%   733MB/s ± 1%  +1.28%  (p=0.000 n=10+10)
2020-03-05 11:17:57 +11:00
Nigel Tao 07110398f1 Optimize std/json string decoding
name                                    old speed     new speed      delta

wuffs_json_decode_1k/clang5             729MB/s ± 1%  1130MB/s ± 0%  +54.99%  (p=0.000 n=10+10)
wuffs_json_decode_21k_formatted/clang5  541MB/s ± 1%   576MB/s ± 1%   +6.44%  (p=0.000 n=10+10)
wuffs_json_decode_26k_compact/clang5    603MB/s ± 0%   873MB/s ± 1%  +44.72%  (p=0.000 n=9+10)
wuffs_json_decode_217k_stringy/clang5   607MB/s ± 0%   727MB/s ± 0%  +19.78%  (p=0.000 n=10+10)

wuffs_json_decode_1k/gcc9               871MB/s ± 0%  1104MB/s ± 1%  +26.74%  (p=0.000 n=10+10)
wuffs_json_decode_21k_formatted/gcc9    641MB/s ± 1%   628MB/s ± 0%   -2.03%  (p=0.000 n=10+10)
wuffs_json_decode_26k_compact/gcc9      746MB/s ± 1%   892MB/s ± 0%  +19.54%  (p=0.000 n=10+10)
wuffs_json_decode_217k_stringy/gcc9     736MB/s ± 1%   724MB/s ± 1%   -1.68%  (p=0.000 n=9+10)
2020-03-05 11:00:56 +11:00
Nigel Tao 2178dcd4da Fix wuffs_base__utf_8__encode conversion warning 2020-03-04 22:34:11 +11:00
Nigel Tao 757a4bd68c Add more test_strconv_parse_number_u64 test cases 2020-03-04 22:18:56 +11:00
Nigel Tao b9ad34fad3 Add wuffs_base__utf_8__encode 2020-03-03 12:44:01 +11:00
Nigel Tao 43f0a87a6d Test std/json emit pending strings on bad UTF-8 2020-03-02 13:59:46 +11:00
Nigel Tao 19967648ef Add std/json while.string_loop_outer loop name 2020-03-01 22:39:02 +11:00