Commit Graph
2658 Commits
Author SHA1 Message Date
Nigel Tao be8542e313 Rename I/O available methods to length 2020-08-13 23:26:08 +10:00
Nigel Tao 79b923c322 Rename io_reader.available_valid_utf_8 2020-08-13 23:15:28 +10:00
Nigel Tao 0291a47dc8 Remove example/jsonptr's ability to speak CBOR
Originally, the example/jsonptr program read JSON from stdin and wrote
JSON to stdout. Over the course of numerous recent commits, it gained
the ability to read either CBOR or JSON and write either CBOR or JSON.
(CBOR is, very roughly, a binary file format similar to the JSON text
file format). While a useful feature, the extra code hurt JSON-to-JSON
performance, the original motivating purpose of that program. This
commit reverts that feature, re-gaining original performance. (The
example/cbor-to-json and example/json-to-cbor programs were recently
added to convert between CBOR and JSON, covering that feature loss).

On a mid-range x86_64 laptop, processing the 181 MiB citylots.json file
from github.com/zemirco/sf-city-lots-json:

$ time gen/bin/example-jsonptr < citylots.json > /dev/null
Before this commit:
real	0m5.095s
After:
real	0m3.583s
Ratio: 1.42x

$ sloccount example/jsonptr/jsonptr.cc | grep ^cpp
Before this commit:
cpp:           1508 (100.00%)
After:
cpp:            869 (100.00%)
Ratio: 1.74x

This commit brings jsonptr.cc approximately to its state just before
168f60a7 "Add example/jsonptr -output-format=cbor flag" was committed,
although it keeps the flags added since then for allowing comments and
inf/NaN numbers.
2020-08-13 22:58:59 +10:00
Nigel Tao d384e1e458 Avoid wuffs_aux macro's unused variable token_ptr 2020-08-13 00:13:12 +10:00
Nigel Tao b21a82bbcd Add some section breaks to auxiliary/json.cc 2020-08-13 00:09:28 +10:00
Nigel Tao b3438434ca Add json QUIRK_JSON_POINTER_ALLOW_TILDE_R_TILDE_N 2020-08-13 00:06:56 +10:00
Nigel Tao 16fadb25cf Fix std/cbor meta.ri decoding F8XX for small XX
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=24819
2020-08-12 23:45:44 +10:00
Nigel Tao 2de1dfec97 Fix test/data/cbor*tokens losing DEFINITELY_ASCII
The change to the golden file was mistakenly left out of 2397d029 "Fix
std/cbor UTF-8 strings aren't always ASCII".
2020-08-12 23:32:40 +10:00
Nigel Tao 8aac6767bb Add example/jsonfindptrs -query=STR flag 2020-08-12 23:18:37 +10:00
Nigel Tao ab6f895007 Extract WUFFS_AUX__DECODE_JSON__GET_THE_NEXT_TOKEN 2020-08-11 22:10:46 +10:00
Nigel Tao da3504833e Have Decode{Cbor,Json} quirks default to empty 2020-08-11 13:46:50 +10:00
Nigel Tao c7b90256b3 Bump fuzzlib hash size from 32 to 64
Also have json_fuzzer choose at most one of the BACKSLASH_X quirks.
2020-08-10 22:56:47 +10:00
Nigel Tao 28ef6a7862 Add fuzz/c/std/cbor_fuzzer.c 2020-08-10 22:29:08 +10:00
Nigel Tao 2397d02997 Fix std/cbor UTF-8 strings aren't always ASCII 2020-08-10 22:27:40 +10:00
Nigel Tao 0091a2132d Have std/cbor reject tags before an 0xFF stop code 2020-08-10 21:39:37 +10:00
Nigel Tao 9497b174c9 Add test_wuffs_cbor_decode_{in,}valid 2020-08-10 21:22:47 +10:00
Nigel Tao ca5da1fb9d Have example C++ programs use override specifier 2020-08-10 15:30:12 +10:00
Nigel Tao 62c4feb268 Let wuffs_aux::DecodeJson use BACKSLASH_X_AS_BYTES 2020-08-10 15:22:05 +10:00
Nigel Tao 6d81764e14 Tweak wuffs_aux::DecodeJsonCallbacks 2020-08-09 23:15:41 +10:00
Nigel Tao 484c1d5fc3 Delete unused MAX_INDENT definition 2020-08-09 23:11:06 +10:00
Nigel Tao 0b7eaf291b Add example/cbor-to-json 2020-08-09 23:03:42 +10:00
Nigel Tao 9dd8c32683 Re-do doc/note/tokens.md's pjtdf example output 2020-08-07 13:18:53 +10:00
Nigel Tao 0a5940d399 Tweak example/jsonfindptrs -input-etc flags 2020-08-07 13:15:41 +10:00
Nigel Tao 9132dd6e2e Fix json_fuzzer for QUIRK_ALLOW_BACKSLASH_X rename 2020-08-06 23:59:29 +10:00
Nigel Tao 7a5e89115d Fix -Werror=conversion in std/json decode_tokens 2020-08-06 23:55:03 +10:00
Nigel Tao 934136b09c Add token_buffer.{read,writ}er_slice methods 2020-08-06 23:51:13 +10:00
Nigel Tao 82a96cb139 Rename token_buffer.{read,writ}er_available 2020-08-06 23:41:36 +10:00
Nigel Tao 1adbc9547f Split std/json's QUIRK_ALLOW_BACKSLASH_X in two 2020-08-06 23:28:07 +10:00
Nigel Tao d7e0133c0b Tweak doc/note/auxiliary-code.md 2020-08-06 09:32:17 +10:00
Nigel Tao b0dcdc33e6 Add Mike Kaufman to A+C 2020-08-06 09:20:16 +10:00
Nigel Tao 724ce98540 Update some refs to the old aux directory name 2020-08-06 09:17:54 +10:00
Nigel Tao 5b081ca0c8 Merge pull request #34 from mike-kaufman/user/mkaufman/fix-aux-directory
rename "aux" directory to "auxiliary" to prevent checkout errors on windows.
2020-08-06 09:10:33 +10:00
Mike Kaufman c8c2144579 updating refs to new auxiliary folder 2020-08-05 09:16:35 -07:00
Mike Kaufman b66cf6b4b1 rename internal/cgen/aux to internal/cgen/auxiliary 2020-08-05 09:11:03 -07:00
Nigel Tao 6ea09e0de4 Remove #ifdef MODULE_ETC guards for aux headers 2020-08-05 22:54:53 +10:00
Nigel Tao cf8884d44c Rename example/library to example/toy-genlib 2020-08-05 11:55:51 +10:00
Nigel Tao 50bfab9998 Tweak example/*/*.cc compile instructions 2020-08-05 11:39:09 +10:00
Nigel Tao f557dad608 Add example/json-to-cbor 2020-08-04 23:45:46 +10:00
Nigel Tao cf6c578973 Add aux library 2020-08-04 23:43:38 +10:00
Nigel Tao a4f2bbb249 Update cgen/base run "go generate" comment 2020-07-28 14:15:24 +10:00
Nigel Tao cb130ba83e Re-organize internal/cgen/data's base files list 2020-07-28 14:09:50 +10:00
Nigel Tao 0a68f6332a Have jsonptr pass through CBOR floating points 2020-07-28 10:39:32 +10:00
Nigel Tao c96b31c7bf Suppress jsonptr byte string output when in query 2020-07-27 23:07:49 +10:00
Nigel Tao 1e96ef200c Add test/data/cbor-rfc-7049-examples.* 2020-07-27 22:00:28 +10:00
Nigel Tao 77c7551b26 Fix jsonptr whitespace and commas for CBOR tags 2020-07-27 21:35:11 +10:00
Nigel Tao 35c4e9532b Fix jsonptr's JSON output for CBOR simple values 2020-07-27 18:01:05 +10:00
Nigel Tao 983a74f257 Let jsonptr accept integer CBOR map keys 2020-07-27 15:56:56 +10:00
Nigel Tao ee6927fcdb Have jsonptr pass around slice_u8 more often 2020-07-27 12:08:33 +10:00
Nigel Tao d6a10dfe8a Add io_buffer.{read,writ}er_slice methods 2020-07-27 11:47:47 +10:00
Nigel Tao ea53245441 Have jsonptr base64url encode CBOR byte strings 2020-07-27 00:03:00 +10:00