Commit Graph
2695 Commits
Author SHA1 Message Date
Nigel Tao a06a89c080 Add link to previous commit (BACKSLASH_X_AS_BYTES) 2020-08-30 21:14:11 +10:00
Nigel Tao 6c13877688 Drop WUFFS_JSON__QUIRK_ALLOW_BACKSLASH_X_AS_BYTES
We can always add it back later if there's demand for it.
2020-08-30 16:15:35 +10:00
Nigel Tao 6bc3f5716d Add "#json: bad new-line in a string" 2020-08-29 23:23:15 +10:00
Nigel Tao 8d779ae1cd Add jsonfindptrs -only-parse-dont-output flag 2020-08-29 23:10:08 +10:00
Nigel Tao 5396dbd17d Have jsonfindptrs use C++17's std::variant
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonfindptrs < citylots.json > /dev/null
Before this commit:
real    0m7.333s
After:
real    0m5.970s
Ratio: 1.23x

Using "/usr/bin/time -v" instead of "time"
Before this commit:
Maximum resident set size (kbytes): 2556992
After:
Maximum resident set size (kbytes): 1214812
Ratio: 2.10x
2020-08-29 22:23:15 +10:00
Nigel Tao 9d69939f2c Have jsonfindptrs factor out escape_needed
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonfindptrs < citylots.json > /dev/null
Before this commit:
real    0m7.550s
After:
real    0m7.333s
Ratio: 1.03x
2020-08-29 10:56:37 +10:00
Nigel Tao b6c01b3464 Have jsonfindptrs cache std::to_string(i)
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonfindptrs < citylots.json > /dev/null
Before this commit:
real    0m8.822s
After:
real    0m7.550s
Ratio: 1.17x
2020-08-29 10:46:04 +10:00
Nigel Tao 8098d96113 Have jsonfindptrs re-use the output string buffer
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonfindptrs < citylots.json > /dev/null
Before this commit:
real    0m9.774s
After:
real    0m8.822s
Ratio: 1.11x
2020-08-29 10:41:05 +10:00
Nigel Tao feded888dd Have jsonfindptrs not flush stdout after each line
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonfindptrs < citylots.json > /dev/null
Before this commit:
real	0m21.104s
After:
real	0m9.774s
Ratio: 2.16x
2020-08-29 10:33:41 +10:00
Nigel Tao 65b0b59529 Have hermeticity.md mention thread-local storage 2020-08-28 09:05:52 +10:00
Nigel Tao 6069ebd619 Rename {read,writ}er_io_position 2020-08-27 23:32:53 +10:00
Nigel Tao 378ceb2537 Add bullet points to doc/note/hermeticity.md 2020-08-26 15:48:32 +10:00
Nigel Tao 3832b04c93 Tweak wuffs_base__io_buffer_meta.pos comment 2020-08-26 10:29:17 +10:00
Nigel Tao 63e6796896 Add "4Ki tokens is 32KiB" comment 2020-08-26 00:00:32 +10:00
Nigel Tao 7fd3bb661a Fix -Wswitch in example/jsonfindptrs 2020-08-24 21:34:53 +10:00
Nigel Tao 525d4bcc34 Add hexadecimal limits to doc/note/base38-etc 2020-08-24 00:45:42 +10:00
Nigel Tao 75e1efcc4a Fix std/cbor src.meta.ri for too much recursion
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25145
2020-08-24 00:39:54 +10:00
Nigel Tao 75682548f5 Add example/jsonptr -output-inf-nan-numbers flag 2020-08-22 21:40:18 +10:00
Nigel Tao ade016523b Optimize example/jsonptr string conversion
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.687s
After:
real    0m1.677s
Ratio: 1.01x
2020-08-21 15:57:51 +10:00
Nigel Tao 3f688b2677 Optimize example/jsonptr token VBD switch
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.765s
After:
real    0m1.687s
Ratio: 1.05x
2020-08-21 15:51:48 +10:00
Nigel Tao 592560e0f1 Upgrade default clang C compiler from v5 to v9 2020-08-21 14:18:26 +10:00
Nigel Tao 6344181b96 Add look-up table for JSON ASCII escapes
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.799s
After:
real    0m1.765s
Ratio: 1.02x
2020-08-21 14:11:31 +10:00
Nigel Tao 1f3912aadf Fix std/cbor decoding multi-byte 0-length strings
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25041
2020-08-20 11:41:23 +10:00
Nigel Tao 50db4a47fb Inline example/jsonptr's handle_token
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.911s
After:
real    0m1.799s
Ratio: 1.06x
2020-08-20 11:31:28 +10:00
Nigel Tao 2104205d25 Let example/jsonptr output comments
Performance (on vanilla, comment-free JSON) slips a little.

On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.863s
After:
real    0m1.911s
Ratio: 0.97x   Inverse: 1.03x
2020-08-20 11:14:29 +10:00
Nigel Tao 6b86cbcb25 Optimize example/jsonptr's write_dst
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m1.949s
After:
real    0m1.863s
Ratio: 1.05x
2020-08-19 15:53:06 +10:00
Nigel Tao fc304139af Add WUFFS_BASE__TOKEN__VBD__FILLER__COMMENT_ANY 2020-08-19 09:40:02 +10:00
Nigel Tao 991bd51306 Rename some example/jsonptr variables 2020-08-19 09:38:16 +10:00
Nigel Tao 0a0c7d68aa Optimize example/*json*'s indentation output
On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real    0m3.612s
After:
real    0m1.949s
Ratio: 1.85x
2020-08-19 00:02:30 +10:00
Nigel Tao 1864d9c01a Add internal/cgen/auxiliary/README.md 2020-08-18 09:23:22 +10:00
Nigel Tao 4284369bb8 Add TOKEN__VBD__FILLER__PUNCTUATION 2020-08-17 16:48:30 +10:00
Nigel Tao 67503b5bda wuffs gen -version=0.3.0-alpha.11 2020-08-17 10:02:41 +10:00
Nigel Tao ca1113620b Let wuffs_aux's MemoryInput take a char* ptr 2020-08-17 10:00:19 +10:00
Nigel Tao 2742a4f4be Have wuffs_aux take lvalue (not rvalue) references 2020-08-17 00:02:49 +10:00
Nigel Tao d690ce7dce Fix cbor_fuzzer for extended-token CBOR tags
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24946
2020-08-16 23:57:21 +10:00
Nigel Tao e2398d580f Add doc/note/hermeticity.md 2020-08-16 23:40:50 +10:00
Nigel Tao 3c3d58bfe7 Rename io_writer.history_available 2020-08-13 23:29:59 +10:00
Nigel Tao be8542e313 Rename I/O available methods to length 2020-08-13 23:26:08 +10:00
Nigel Tao 79b923c322 Rename io_reader.available_valid_utf_8 2020-08-13 23:15:28 +10:00
Nigel Tao 0291a47dc8 Remove example/jsonptr's ability to speak CBOR
Originally, the example/jsonptr program read JSON from stdin and wrote
JSON to stdout. Over the course of numerous recent commits, it gained
the ability to read either CBOR or JSON and write either CBOR or JSON.
(CBOR is, very roughly, a binary file format similar to the JSON text
file format). While a useful feature, the extra code hurt JSON-to-JSON
performance, the original motivating purpose of that program. This
commit reverts that feature, re-gaining original performance. (The
example/cbor-to-json and example/json-to-cbor programs were recently
added to convert between CBOR and JSON, covering that feature loss).

On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real	0m5.095s
After:
real	0m3.583s
Ratio: 1.42x

$ sloccount example/jsonptr/jsonptr.cc | grep ^cpp
Before this commit:
cpp:           1508 (100.00%)
After:
cpp:            869 (100.00%)
Ratio: 1.74x

This commit brings jsonptr.cc approximately to its state just before
168f60a7 "Add example/jsonptr -output-format=cbor flag" was committed,
although it keeps the flags added since then for allowing comments and
inf/NaN numbers.
2020-08-13 22:58:59 +10:00
Nigel Tao d384e1e458 Avoid wuffs_aux macro's unused variable token_ptr 2020-08-13 00:13:12 +10:00
Nigel Tao b21a82bbcd Add some section breaks to auxiliary/json.cc 2020-08-13 00:09:28 +10:00
Nigel Tao b3438434ca Add json QUIRK_JSON_POINTER_ALLOW_TILDE_R_TILDE_N 2020-08-13 00:06:56 +10:00
Nigel Tao 16fadb25cf Fix std/cbor meta.ri decoding F8XX for small XX
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24819
2020-08-12 23:45:44 +10:00
Nigel Tao 2de1dfec97 Fix test/data/cbor*tokens losing DEFINITELY_ASCII
The change to the golden file was mistakenly left out of 2397d029 "Fix
std/cbor UTF-8 strings aren't always ASCII".
2020-08-12 23:32:40 +10:00
Nigel Tao 8aac6767bb Add example/jsonfindptrs -query=STR flag 2020-08-12 23:18:37 +10:00
Nigel Tao ab6f895007 Extract WUFFS_AUX__DECODE_JSON__GET_THE_NEXT_TOKEN 2020-08-11 22:10:46 +10:00
Nigel Tao da3504833e Have Decode{Cbor,Json} quirks default to empty 2020-08-11 13:46:50 +10:00
Nigel Tao c7b90256b3 Bump fuzzlib hash size from 32 to 64
Also have json_fuzzer choose at most one of the BACKSLASH_X quirks.
2020-08-10 22:56:47 +10:00
Nigel Tao 28ef6a7862 Add fuzz/c/std/cbor_fuzzer.c 2020-08-10 22:29:08 +10:00