Have JSON line-comment tokens exclude the "\n"

Before this commit, line-comment token chains included the final "\n"
the way block-comments included the final "*/". The "\n" was mandatory.

However, when parsing with both QUIRK_ALLOW_COMMENT_LINE and
QUIRK_ALLOW_TRAILING_FILLER, input like "123//xyz" (without a "\n"
before the EOF) was rejected before and accepted after this commit.

So that line-comments are consistent, in that they either always or
never include the final "\n", they now never include the final "\n".

An input fragment (parsed with QUIRK_ALLOW_COMMENT_LINE) like
"//abc\n\t\t" was, before this commit, parsed as a 6 byte comment token
and a 2 byte other-filler token. That split is now 5+3.
This commit is contained in:
Nigel Tao committed 2021-07-20 14:17:09 +10:00
1 parent 40ffa334ee
commit 77f855206d
6 files changed
+41 -50

No files matched your search

Binary file not shown.