Commit Graph
2679 Commits
Author SHA1 Message Date
Nigel Tao 75e1efcc4a Fix std/cbor src.meta.ri for too much recursion
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25145
2020-08-24 00:39:54 +10:00
Nigel Tao 75682548f5 Add example/jsonptr -output-inf-nan-numbers flag 2020-08-22 21:40:18 +10:00
Nigel Tao ade016523b Optimize example/jsonptr string conversion
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.687s
After:
real    0m1.677s
Ratio: 1.01x
2020-08-21 15:57:51 +10:00
Nigel Tao 3f688b2677 Optimize example/jsonptr token VBD switch
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.765s
After:
real    0m1.687s
Ratio: 1.05x
2020-08-21 15:51:48 +10:00
Nigel Tao 592560e0f1 Upgrade default clang C compiler from v5 to v9 2020-08-21 14:18:26 +10:00
Nigel Tao 6344181b96 Add look-up table for JSON ASCII escapes
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.799s
After:
real    0m1.765s
Ratio: 1.02x
2020-08-21 14:11:31 +10:00
Nigel Tao 1f3912aadf Fix std/cbor decoding multi-byte 0-length strings
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25041
2020-08-20 11:41:23 +10:00
Nigel Tao 50db4a47fb Inline example/jsonptr's handle_token
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.911s
After:
real    0m1.799s
Ratio: 1.06x
2020-08-20 11:31:28 +10:00
Nigel Tao 2104205d25 Let example/jsonptr output comments
Performance (on vanilla, comment-free JSON) slips a little.

On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.863s
After:
real    0m1.911s
Ratio: 0.97x   Inverse: 1.03x
2020-08-20 11:14:29 +10:00
Nigel Tao 6b86cbcb25 Optimize example/jsonptr's write_dst
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.949s
After:
real    0m1.863s
Ratio: 1.05x
2020-08-19 15:53:06 +10:00
Nigel Tao fc304139af Add WUFFS_BASE__TOKEN__VBD__FILLER__COMMENT_ANY 2020-08-19 09:40:02 +10:00
Nigel Tao 991bd51306 Rename some example/jsonptr variables 2020-08-19 09:38:16 +10:00
Nigel Tao 0a0c7d68aa Optimize example/*json*'s indentation output
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m3.612s
After:
real    0m1.949s
Ratio: 1.85x
2020-08-19 00:02:30 +10:00
Nigel Tao 1864d9c01a Add internal/cgen/auxiliary/README.md 2020-08-18 09:23:22 +10:00
Nigel Tao 4284369bb8 Add TOKEN__VBD__FILLER__PUNCTUATION 2020-08-17 16:48:30 +10:00
Nigel Tao 67503b5bda wuffs gen -version=0.3.0-alpha.11 2020-08-17 10:02:41 +10:00
Nigel Tao ca1113620b Let wuffs_aux's MemoryInput take a char* ptr 2020-08-17 10:00:19 +10:00
Nigel Tao 2742a4f4be Have wuffs_aux take lvalue (not rvalue) references 2020-08-17 00:02:49 +10:00
Nigel Tao d690ce7dce Fix cbor_fuzzer for extended-token CBOR tags
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24946
2020-08-16 23:57:21 +10:00
Nigel Tao e2398d580f Add doc/note/hermeticity.md 2020-08-16 23:40:50 +10:00
Nigel Tao 3c3d58bfe7 Rename io_writer.history_available 2020-08-13 23:29:59 +10:00
Nigel Tao be8542e313 Rename I/O available methods to length 2020-08-13 23:26:08 +10:00
Nigel Tao 79b923c322 Rename io_reader.available_valid_utf_8 2020-08-13 23:15:28 +10:00
Nigel Tao 0291a47dc8 Remove example/jsonptr's ability to speak CBOR
Originally, the example/jsonptr program read JSON from stdin and wrote
JSON to stdout. Over the course of numerous recent commits, it gained
the ability to read either CBOR or JSON and write either CBOR or JSON.
(CBOR is, very roughly, a binary file format similar to the JSON text
file format). While a useful feature, the extra code hurt JSON-to-JSON
performance, the original motivating purpose of that program. This
commit reverts that feature, re-gaining original performance. (The
example/cbor-to-json and example/json-to-cbor programs were recently
added to convert between CBOR and JSON, covering that feature loss).

On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real	0m5.095s
After:
real	0m3.583s
Ratio: 1.42x

$ sloccount example/jsonptr/jsonptr.cc | grep ^cpp
Before this commit:
cpp:           1508 (100.00%)
After:
cpp:            869 (100.00%)
Ratio: 1.74x

This commit brings jsonptr.cc approximately to its state just before
168f60a7 "Add example/jsonptr -output-format=cbor flag" was committed,
although it keeps the flags added since then for allowing comments and
inf/NaN numbers.
2020-08-13 22:58:59 +10:00
Nigel Tao d384e1e458 Avoid wuffs_aux macro's unused variable token_ptr 2020-08-13 00:13:12 +10:00
Nigel Tao b21a82bbcd Add some section breaks to auxiliary/json.cc 2020-08-13 00:09:28 +10:00
Nigel Tao b3438434ca Add json QUIRK_JSON_POINTER_ALLOW_TILDE_R_TILDE_N 2020-08-13 00:06:56 +10:00
Nigel Tao 16fadb25cf Fix std/cbor meta.ri decoding F8XX for small XX
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24819
2020-08-12 23:45:44 +10:00
Nigel Tao 2de1dfec97 Fix test/data/cbor*tokens losing DEFINITELY_ASCII
The change to the golden file was mistakenly left out of 2397d029 "Fix
std/cbor UTF-8 strings aren't always ASCII".
2020-08-12 23:32:40 +10:00
Nigel Tao 8aac6767bb Add example/jsonfindptrs -query=STR flag 2020-08-12 23:18:37 +10:00
Nigel Tao ab6f895007 Extract WUFFS_AUX__DECODE_JSON__GET_THE_NEXT_TOKEN 2020-08-11 22:10:46 +10:00
Nigel Tao da3504833e Have Decode{Cbor,Json} quirks default to empty 2020-08-11 13:46:50 +10:00
Nigel Tao c7b90256b3 Bump fuzzlib hash size from 32 to 64
Also have json_fuzzer choose at most one of the BACKSLASH_X quirks.
2020-08-10 22:56:47 +10:00
Nigel Tao 28ef6a7862 Add fuzz/c/std/cbor_fuzzer.c 2020-08-10 22:29:08 +10:00
Nigel Tao 2397d02997 Fix std/cbor UTF-8 strings aren't always ASCII 2020-08-10 22:27:40 +10:00
Nigel Tao 0091a2132d Have std/cbor reject tags before an 0xFF stop code 2020-08-10 21:39:37 +10:00
Nigel Tao 9497b174c9 Add test_wuffs_cbor_decode_{in,}valid 2020-08-10 21:22:47 +10:00
Nigel Tao ca5da1fb9d Have example C++ programs use override specifier 2020-08-10 15:30:12 +10:00
Nigel Tao 62c4feb268 Let wuffs_aux::DecodeJson use BACKSLASH_X_AS_BYTES 2020-08-10 15:22:05 +10:00
Nigel Tao 6d81764e14 Tweak wuffs_aux::DecodeJsonCallbacks 2020-08-09 23:15:41 +10:00
Nigel Tao 484c1d5fc3 Delete unused MAX_INDENT definition 2020-08-09 23:11:06 +10:00
Nigel Tao 0b7eaf291b Add example/cbor-to-json 2020-08-09 23:03:42 +10:00
Nigel Tao 9dd8c32683 Re-do doc/note/tokens.md's pjtdf example output 2020-08-07 13:18:53 +10:00
Nigel Tao 0a5940d399 Tweak example/jsonfindptrs -input-etc flags 2020-08-07 13:15:41 +10:00
Nigel Tao 9132dd6e2e Fix json_fuzzer for QUIRK_ALLOW_BACKSLASH_X rename 2020-08-06 23:59:29 +10:00
Nigel Tao 7a5e89115d Fix -Werror=conversion in std/json decode_tokens 2020-08-06 23:55:03 +10:00
Nigel Tao 934136b09c Add token_buffer.{read,writ}er_slice methods 2020-08-06 23:51:13 +10:00
Nigel Tao 82a96cb139 Rename token_buffer.{read,writ}er_available 2020-08-06 23:41:36 +10:00
Nigel Tao 1adbc9547f Split std/json's QUIRK_ALLOW_BACKSLASH_X in two 2020-08-06 23:28:07 +10:00
Nigel Tao d7e0133c0b Tweak doc/note/auxiliary-code.md 2020-08-06 09:32:17 +10:00