Commit Graph
7 Commits
Author SHA1 Message Date
Nigel Tao 77f855206d Have JSON line-comment tokens exclude the "\n"
Before this commit, line-comment token chains included the final "\n"
the way block-comments included the final "*/". The "\n" was mandatory.

However, when parsing with both QUIRK_ALLOW_COMMENT_LINE and
QUIRK_ALLOW_TRAILING_FILLER, input like "123//xyz" (without a "\n"
before the EOF) was rejected before and accepted after this commit.

So that line-comments are consistent, in that they either always or
never include the final "\n", they now never include the final "\n".

An input fragment (parsed with QUIRK_ALLOW_COMMENT_LINE) like
"//abc\n\t\t" was, before this commit, parsed as a 6 byte comment token
and a 2 byte other-filler token. That split is now 5+3.
2021-07-20 14:17:09 +10:00
Nigel Tao 4284369bb8 Add TOKEN__VBD__FILLER__PUNCTUATION 2020-08-17 16:48:30 +10:00
Nigel Tao 1adbc9547f Split std/json's QUIRK_ALLOW_BACKSLASH_X in two 2020-08-06 23:28:07 +10:00
Nigel Tao b9afc26387 Add TOKEN__VBD__STRING__CHAIN_ETC 2020-07-03 13:36:13 +10:00
Nigel Tao 496e88b6c5 Remove wuffs_base__token__link_prev 2020-04-09 22:11:08 +10:00
Nigel Tao 7edf71414f Fix std/json replace_invalid_unicode copy/pasto
Updates https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=21465
2020-03-31 22:37:43 +11:00
Nigel Tao 5d5a17cc9c Add test/data/json-quirks.json 2020-03-24 17:13:07 +11:00