Have JSON line-comment tokens exclude the "\n"

Before this commit, line-comment token chains included the final "\n"
the way block-comments included the final "*/". The "\n" was mandatory.

However, when parsing with both QUIRK_ALLOW_COMMENT_LINE and
QUIRK_ALLOW_TRAILING_FILLER, input like "123//xyz" (without a "\n"
before the EOF) was rejected before and accepted after this commit.

So that line-comments are consistent, in that they either always or
never include the final "\n", they now never include the final "\n".

An input fragment (parsed with QUIRK_ALLOW_COMMENT_LINE) like
"//abc\n\t\t" was, before this commit, parsed as a 6 byte comment token
and a 2 byte other-filler token. That split is now 5+3.
This commit is contained in:
Nigel Tao committed 2021-07-20 14:17:09 +10:00
1 parent 40ffa334ee
commit 77f855206d
6 files changed
+41 -50

No files matched your search

+1 -1
View File
@@ -3451,7 +3451,7 @@ test_wuffs_json_decode_quirk_allow_trailing_comments() {
"306 /*foo*/ \n", //
"307 /*foo*/ \n\n", //
"308/*bar\nbaz*/\n\n", //
"309 // qux\n", // TODO: drop the "\n".
"309 // qux", //
"310 // qux\n", //
"311 // qux\n\n", //
"312 /*c0*/ /*c1*/\n\n", //