diff --git a/example/jsonfindptrs/jsonfindptrs.cc b/example/jsonfindptrs/jsonfindptrs.cc new file mode 100644 index 00000000..4e26d0e9 --- /dev/null +++ b/example/jsonfindptrs/jsonfindptrs.cc @@ -0,0 +1,747 @@ +// Copyright 2020 The Wuffs Authors. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// https://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ---------------- + +/* +jsonfindptrs reads UTF-8 JSON from stdin and writes every node's JSON Pointer +(RFC 6901) to stdout. + +See the "const char* usage" string below for details. + +---- + +This program uses Wuffs' JSON decoder at a relatively high level, building +in-memory representations of JSON 'things' (e.g. numbers, strings, objects). +After the entire input has been converted, walking the tree prints the output +(in sorted order). The core conversion mechanism is to call JsonThing::parse, +which consumes a variable number of tokens (the output of Wuffs' JSON decoder). +JsonThing::parse can call itself recursively, as JSON values can nest. + +This approach is centered around JSON things. Each JSON thing comprises one or +more JSON tokens. + +An alternative, lower-level approach is in the sibling example/jsonptr program. +Neither approach is better or worse per se, but when studying this program, be +aware that there are multiple ways to use Wuffs' JSON decoder. + +The two programs, jsonfindptrs and jsonptr, also demonstrate different +trade-offs with regard to JSON object duplicate keys. The JSON spec permits +different implementations to allow or reject duplicate keys. It is not always +clear which approach is safer. Rejecting them is certainly unambiguous, and +security bugs can lurk in ambiguous corners of a file format, if two different +implementations both silently accept a file but differ on how to interpret it. +On the other hand, in the worst case, detecting duplicate keys requires O(N) +memory, where N is the size of the (potentially untrusted) input. + +This program (jsonfindptrs) rejects duplicate keys. + +---- + +This example program differs from most other example Wuffs programs in that it +is written in C++, not C. + +$CXX jsonfindptrs.cc && ./a.out < ../../test/data/github-tags.json; rm -f a.out + +for a C++ compiler $CXX, such as clang++ or g++. +*/ + +#include +#include +#include +#include +#include +#include +#include + +// Wuffs ships as a "single file C library" or "header file library" as per +// https://github.com/nothings/stb/blob/master/docs/stb_howto.txt +// +// To use that single file as a "foo.c"-like implementation, instead of a +// "foo.h"-like header, #define WUFFS_IMPLEMENTATION before #include'ing or +// compiling it. +#define WUFFS_IMPLEMENTATION + +// Defining the WUFFS_CONFIG__MODULE* macros are optional, but it lets users of +// release/c/etc.c whitelist which parts of Wuffs to build. That file contains +// the entire Wuffs standard library, implementing a variety of codecs and file +// formats. Without this macro definition, an optimizing compiler or linker may +// very well discard Wuffs code for unused codecs, but listing the Wuffs +// modules we use makes that process explicit. Preprocessing means that such +// code simply isn't compiled. +#define WUFFS_CONFIG__MODULES +#define WUFFS_CONFIG__MODULE__BASE +#define WUFFS_CONFIG__MODULE__JSON + +// If building this program in an environment that doesn't easily accommodate +// relative includes, you can use the script/inline-c-relative-includes.go +// program to generate a stand-alone C++ file. +#include "../../release/c/wuffs-unsupported-snapshot.c" + +#define TRY(error_msg) \ + do { \ + std::string z = error_msg; \ + if (!z.empty()) { \ + return z; \ + } \ + } while (false) + +static const char* usage = + "Usage: jsonfindptrs -flags input.json\n" + "\n" + "Flags:\n" + " -o=NUM -max-output-depth=NUM\n" + " -s -strict-json-pointer-syntax\n" + "\n" + "The input.json filename is optional. If absent, it reads from stdin.\n" + "\n" + "----\n" + "\n" + "jsonfindptrs reads UTF-8 JSON from stdin and writes every node's JSON\n" + "Pointer (RFC 6901) to stdout.\n" + "\n" + "For example, given RFC 6901 section 5's sample input\n" + "(https://tools.ietf.org/rfc/rfc6901.txt), this command:\n" + " jsonfindptrs rfc-6901-json-pointer.json\n" + "will print:\n" + " \n" + " /\n" + " / \n" + " /a~1b\n" + " /c%d\n" + " /e^f\n" + " /foo\n" + " /foo/0\n" + " /foo/1\n" + " /g|h\n" + " /i\\j\n" + " /k\"l\n" + " /m~0n\n" + "\n" + "The first three lines are (1) a 0-byte \"\", (2) a 1-byte \"/\" and (3)\n" + "a 2-byte \"/ \". Unlike a file system, the \"/\" JSON Pointer does not\n" + "identify the root. Instead, \"\" is the root and \"/\" is the child (the\n" + "value in a key-value pair) of the root whose key is the empty string.\n" + "Similarly, \"/xyz\" and \"/xyz/\" are two different nodes.\n" + "\n" + "----\n" + "\n" + "The JSON specification (https://json.org/) permits implementations that\n" + "allow duplicate keys, but this one does not. Conversely, it prints keys\n" + "in sorted order, but the overall output is not necessarily sorted\n" + "lexicographically. For example, \"/a/9\" would come before \"/a/10\",\n" + "and \"/b/c\", a child of \"/b\", would come before \"/b+\".\n" + "\n" + "This JSON implementation also rejects integer values outside ±M, where\n" + "M is ((1<<53)-1), also known as JavaScript's Number.MAX_SAFE_INTEGER.\n" + "\n" + "----\n" + "\n" + "The -s or -strict-json-pointer-syntax flag restricts the output lines\n" + "to exactly RFC 6901, with only two escape sequences: \"~0\" and \"~1\"\n" + "for \"~\" and \"/\". Without this flag, this program also lets \"~n\"\n" + "and \"~r\" escape the New Line and Carriage Return ASCII control\n" + "characters, which can work better with line oriented Unix tools that\n" + "assume exactly one value (i.e. one JSON Pointer string) per line. With\n" + "this flag, the program will fail if the input JSON's object keys contain\n" + "\"\\u000A\" or \"\\u000D\".\n" + "\n" + "----\n" + "\n" + "The JSON specification permits implementations to set their own maximum\n" + "input depth. This JSON implementation sets it to 1024.\n" + "\n" + "The -o=NUM or -max-output-depth=NUM flag gives the maximum (inclusive)\n" + "output depth. JSON containers ([] arrays and {} objects) can hold other\n" + "containers. A bare -o or -max-output-depth is equivalent to -o=1,\n" + "analogous to the Unix ls command. The flag's absence is equivalent to an\n" + "unlimited output depth, analogous to the Unix find command (and hence\n" + "the name of this program: jsonfindptrs)."; + +// ---- + +struct { + int remaining_argc; + char** remaining_argv; + + uint32_t max_output_depth; + bool strict_json_pointer_syntax; +} flags = {0}; + +std::string // +parse_flags(int argc, char** argv) { + flags.max_output_depth = 0xFFFFFFFF; + + int c = (argc > 0) ? 1 : 0; // Skip argv[0], the program name. + for (; c < argc; c++) { + char* arg = argv[c]; + if (*arg++ != '-') { + break; + } + + // A double-dash "--foo" is equivalent to a single-dash "-foo". As special + // cases, a bare "-" is not a flag (some programs may interpret it as + // stdin) and a bare "--" means to stop parsing flags. + if (*arg == '\x00') { + break; + } else if (*arg == '-') { + arg++; + if (*arg == '\x00') { + c++; + break; + } + } + + if (!strcmp(arg, "o") || !strcmp(arg, "max-output-depth")) { + flags.max_output_depth = 1; + continue; + } else if (!strncmp(arg, "o=", 2) || + !strncmp(arg, "max-output-depth=", 16)) { + while (*arg++ != '=') { + } + wuffs_base__result_u64 u = wuffs_base__parse_number_u64( + wuffs_base__make_slice_u8((uint8_t*)arg, strlen(arg))); + if (wuffs_base__status__is_ok(&u.status) && (u.value <= 0xFFFFFFFF)) { + flags.max_output_depth = (uint32_t)(u.value); + continue; + } + return usage; + } + if (!strcmp(arg, "s") || !strcmp(arg, "strict-json-pointer-syntax")) { + flags.strict_json_pointer_syntax = true; + continue; + } + + return usage; + } + + flags.remaining_argc = argc - c; + flags.remaining_argv = argv + c; + return ""; +} + + // ---- + +#ifndef SRC_BUFFER_ARRAY_SIZE +#define SRC_BUFFER_ARRAY_SIZE (4 * 1024) +#endif +#ifndef TOKEN_BUFFER_ARRAY_SIZE +#define TOKEN_BUFFER_ARRAY_SIZE (1 * 1024) +#endif + +class TokenStream { + public: + struct Result { + std::string status_msg; + wuffs_base__token token; + // src_data is a sub-slice of m_src (a slice is a pointer-length pair). + // Calling TokenStream::peek or TokenStream::next may change the backing + // array's contents, so handling a TokenStream::Result may require copying + // this src_data slice's contents. + wuffs_base__slice_u8 src_data; + + Result(std::string s) + : status_msg(s), + token(wuffs_base__make_token(0)), + src_data(wuffs_base__empty_slice_u8()) {} + + Result(std::string s, wuffs_base__token t, wuffs_base__slice_u8 d) + : status_msg(s), token(t), src_data(d) {} + }; + + TokenStream(int input_file_descriptor) + : m_status(wuffs_base__make_status(nullptr)), + m_src(wuffs_base__make_io_buffer( + wuffs_base__make_slice_u8(m_src_array, SRC_BUFFER_ARRAY_SIZE), + wuffs_base__empty_io_buffer_meta())), + m_tok(wuffs_base__make_token_buffer( + wuffs_base__make_slice_token(m_tok_array, TOKEN_BUFFER_ARRAY_SIZE), + wuffs_base__empty_token_buffer_meta())), + m_input_file_descriptor(input_file_descriptor), + m_curr_token_end_src_index(0) { + m_status = + m_dec.initialize(sizeof__wuffs_json__decoder(), WUFFS_VERSION, 0); + } + + Result peek() { return peek_or_next(false); } + Result next() { return peek_or_next(true); } + + private: + Result peek_or_next(bool next) { + while (m_tok.meta.ri >= m_tok.meta.wi) { + if (m_status.repr == nullptr) { + // No-op. + } else if (m_status.repr == wuffs_base__suspension__short_read) { + if (m_curr_token_end_src_index != m_src.meta.ri) { + return Result( + "TokenStream: internal error: inconsistent src indexes"); + } + const char* z = read_src(); + m_curr_token_end_src_index = m_src.meta.ri; + if (z) { + return Result(z); + } + } else if (m_status.repr == wuffs_base__suspension__short_write) { + m_tok.compact(); + } else { + return Result(m_status.message()); + } + + m_status = m_dec.decode_tokens(&m_tok, &m_src); + } + + wuffs_base__token t = m_tok.data.ptr[m_tok.meta.ri]; + size_t i = m_curr_token_end_src_index; + uint64_t n = t.length(); + if ((m_src.meta.ri < i) || ((m_src.meta.ri - i) < n)) { + return Result("TokenStream: internal error: inconsistent src indexes"); + } + if (next) { + m_tok.meta.ri++; + m_curr_token_end_src_index += n; + } + return Result("", t, wuffs_base__make_slice_u8(m_src.data.ptr + i, n)); + } + + const char* // + read_src() { + if (m_src.meta.closed) { + return "main: internal error: read requested on a closed source"; + } + m_src.compact(); + if (m_src.meta.wi >= m_src.data.len) { + return "main: src buffer is full"; + } + while (true) { + ssize_t n = read(m_input_file_descriptor, m_src.data.ptr + m_src.meta.wi, + m_src.data.len - m_src.meta.wi); + if (n >= 0) { + m_src.meta.wi += n; + m_src.meta.closed = n == 0; + break; + } else if (errno != EINTR) { + return strerror(errno); + } + } + return nullptr; + } + + wuffs_base__status m_status; + wuffs_base__io_buffer m_src; + wuffs_base__token_buffer m_tok; + int m_input_file_descriptor; + // m_curr_token_end_src_index is the m_src.data.ptr index of the end of the + // current token. An invariant is that (m_curr_token_end_src_index <= + // m_src.meta.ri). + size_t m_curr_token_end_src_index; + + wuffs_base__token m_tok_array[TOKEN_BUFFER_ARRAY_SIZE]; + uint8_t m_src_array[SRC_BUFFER_ARRAY_SIZE]; + wuffs_json__decoder m_dec; +}; + +// ---- + +class JsonThing { + public: + struct Result; + + using Vector = std::vector; + + // We use a std::map in this example program to avoid dependencies outside of + // the C++ standard library. If you're copy/pasting this JsonThing code, + // consider a more efficient data structure such as an absl::btree_map. + // + // See CppCon 2014: Chandler Carruth "Efficiency with Algorithms, Performance + // with Data Structures" at https://www.youtube.com/watch?v=fHNmRkzxHWs + using Map = std::map; + + enum class Kind { + Null, + Bool, + Int64, + Float64, + String, + Array, + Object, + } kind = Kind::Null; + + struct Value { + bool b = false; + int64_t i = 0; + double f = 0; + std::string s; + Vector a; + Map o; + } value; + + static JsonThing::Result parse(TokenStream& ts); + + private: + static JsonThing::Result parse_array(TokenStream& ts); + static JsonThing::Result parse_literal(TokenStream::Result tsr); + static JsonThing::Result parse_number(TokenStream::Result tsr); + static JsonThing::Result parse_object(TokenStream& ts); + static JsonThing::Result parse_string(TokenStream& ts, + TokenStream::Result tsr); +}; + +struct JsonThing::Result { + std::string status_msg; + JsonThing thing; + + Result(std::string s) : status_msg(s), thing(JsonThing()) {} + + Result(std::string s, JsonThing t) : status_msg(s), thing(t) {} +}; + +JsonThing::Result // +JsonThing::parse(TokenStream& ts) { + while (true) { + TokenStream::Result tsr = ts.next(); + if (!tsr.status_msg.empty()) { + return Result(std::move(tsr.status_msg)); + } + + uint64_t vbc = tsr.token.value_base_category(); + uint64_t vbd = tsr.token.value_base_detail(); + switch (vbc) { + case WUFFS_BASE__TOKEN__VBC__FILLER: + continue; + case WUFFS_BASE__TOKEN__VBC__STRUCTURE: + if (vbd & WUFFS_BASE__TOKEN__VBD__STRUCTURE__PUSH) { + if (vbd & WUFFS_BASE__TOKEN__VBD__STRUCTURE__TO_LIST) { + return parse_array(ts); + } else if (vbd & WUFFS_BASE__TOKEN__VBD__STRUCTURE__TO_DICT) { + return parse_object(ts); + } + } + break; + case WUFFS_BASE__TOKEN__VBC__STRING: + return parse_string(ts, tsr); + case WUFFS_BASE__TOKEN__VBC__LITERAL: + return parse_literal(tsr); + case WUFFS_BASE__TOKEN__VBC__NUMBER: { + return parse_number(tsr); + } + } + + return Result("main: internal error: unexpected token"); + } +} + +JsonThing::Result // +JsonThing::parse_array(TokenStream& ts) { + JsonThing jt; + jt.kind = Kind::Array; + while (true) { + TokenStream::Result tsr = ts.peek(); + if (!tsr.status_msg.empty()) { + return Result(std::move(tsr.status_msg)); + } + uint64_t vbc = tsr.token.value_base_category(); + uint64_t vbd = tsr.token.value_base_detail(); + if (vbc == WUFFS_BASE__TOKEN__VBC__FILLER) { + ts.next(); + continue; + } else if ((vbc == WUFFS_BASE__TOKEN__VBC__STRUCTURE) && + (vbd & WUFFS_BASE__TOKEN__VBD__STRUCTURE__POP)) { + ts.next(); + break; + } + + JsonThing::Result jtr = JsonThing::parse(ts); + if (!jtr.status_msg.empty()) { + return Result(std::move(jtr.status_msg)); + } + jt.value.a.push_back(std::move(jtr.thing)); + } + return Result("", jt); +} + +JsonThing::Result // +JsonThing::parse_literal(TokenStream::Result tsr) { + uint64_t vbd = tsr.token.value_base_detail(); + if (vbd & WUFFS_BASE__TOKEN__VBD__LITERAL__NULL) { + JsonThing jt; + jt.kind = Kind::Null; + return Result("", jt); + } else if (vbd & WUFFS_BASE__TOKEN__VBD__LITERAL__FALSE) { + JsonThing jt; + jt.kind = Kind::Bool; + jt.value.b = false; + return Result("", jt); + } else if (vbd & WUFFS_BASE__TOKEN__VBD__LITERAL__TRUE) { + JsonThing jt; + jt.kind = Kind::Bool; + jt.value.b = true; + return Result("", jt); + } + return Result("main: internal error: unexpected token"); +} + +JsonThing::Result // +JsonThing::parse_number(TokenStream::Result tsr) { + // Parsing the number from its string representation (converting from "123" + // to 123) isn't necessary for the jsonfindptrs program, but if you're + // copy/pasting this JsonThing code, here's how to do it. + uint64_t vbd = tsr.token.value_base_detail(); + if (vbd & WUFFS_BASE__TOKEN__VBD__NUMBER__FORMAT_TEXT) { + if (vbd & WUFFS_BASE__TOKEN__VBD__NUMBER__CONTENT_INTEGER_SIGNED) { + static constexpr int64_t m = 0x001FFFFFFFFFFFFF; // ((1<<53) - 1). + wuffs_base__result_i64 r = wuffs_base__parse_number_i64(tsr.src_data); + if (!r.status.is_ok()) { + return Result(r.status.message()); + } else if ((r.value < -m) || (+m < r.value)) { + return Result(wuffs_base__error__out_of_bounds); + } + JsonThing jt; + jt.kind = Kind::Int64; + jt.value.i = r.value; + return Result("", jt); + } else if (vbd & WUFFS_BASE__TOKEN__VBD__NUMBER__CONTENT_FLOATING_POINT) { + wuffs_base__result_f64 r = wuffs_base__parse_number_f64(tsr.src_data); + if (!r.status.is_ok()) { + return Result(r.status.message()); + } + JsonThing jt; + jt.kind = Kind::Float64; + jt.value.f = r.value; + return Result("", jt); + } + } + return Result("main: internal error: unexpected number"); +} + +JsonThing::Result // +JsonThing::parse_object(TokenStream& ts) { + JsonThing jt; + jt.kind = Kind::Object; + + std::string key; + bool have_key = false; + + while (true) { + TokenStream::Result tsr = ts.peek(); + if (!tsr.status_msg.empty()) { + return Result(std::move(tsr.status_msg)); + } + uint64_t vbc = tsr.token.value_base_category(); + uint64_t vbd = tsr.token.value_base_detail(); + if (vbc == WUFFS_BASE__TOKEN__VBC__FILLER) { + ts.next(); + continue; + } else if ((vbc == WUFFS_BASE__TOKEN__VBC__STRUCTURE) && + (vbd & WUFFS_BASE__TOKEN__VBD__STRUCTURE__POP)) { + ts.next(); + break; + } + + JsonThing::Result jtr = JsonThing::parse(ts); + if (!jtr.status_msg.empty()) { + return Result(std::move(jtr.status_msg)); + } + + if (have_key) { + have_key = false; + auto iter = jt.value.o.find(key); + if (iter == jt.value.o.end()) { + jt.value.o.insert( + iter, Map::value_type(std::move(key), std::move(jtr.thing))); + } else { + return Result("main: duplicate key: " + key); + } + } else if (jtr.thing.kind == Kind::String) { + have_key = true; + key = std::move(jtr.thing.value.s); + } else { + return Result("main: internal error: unexpected non-string key"); + } + } + return Result("", jt); +} + +JsonThing::Result // +JsonThing::parse_string(TokenStream& ts, TokenStream::Result tsr) { + JsonThing jt; + jt.kind = Kind::String; + while (true) { + uint64_t vbc = tsr.token.value_base_category(); + uint64_t vbd = tsr.token.value_base_detail(); + + switch (vbc) { + case WUFFS_BASE__TOKEN__VBC__STRING: { + if (vbd & WUFFS_BASE__TOKEN__VBD__STRING__CONVERT_0_DST_1_SRC_DROP) { + // No-op. + } else if (vbd & + WUFFS_BASE__TOKEN__VBD__STRING__CONVERT_1_DST_1_SRC_COPY) { + const char* ptr = // Convert from (uint8_t*). + static_cast(static_cast(tsr.src_data.ptr)); + jt.value.s.append(ptr, tsr.src_data.len); + } else { + return Result( + "main: internal error: unexpected string-token conversion", + JsonThing()); + } + break; + } + + case WUFFS_BASE__TOKEN__VBC__UNICODE_CODE_POINT: { + uint8_t u[WUFFS_BASE__UTF_8__BYTE_LENGTH__MAX_INCL]; + size_t n = wuffs_base__utf_8__encode( + wuffs_base__make_slice_u8(&u[0], + WUFFS_BASE__UTF_8__BYTE_LENGTH__MAX_INCL), + vbd); + const char* ptr = // Convert from (uint8_t*). + static_cast(static_cast(&u[0])); + jt.value.s.append(ptr, n); + break; + } + + default: + return Result("main: internal error: unexpected token"); + } + + if (!tsr.token.link_next()) { + break; + } + tsr = ts.next(); + if (!tsr.status_msg.empty()) { + return Result(std::move(tsr.status_msg)); + } + } + return Result("", jt); +} + +// ---- + +std::string // +escape(std::string s) { + for (char& c : s) { + if ((c == '~') || (c == '/') || (c == '\n') || (c == '\r')) { + goto escape_needed; + } + } + return s; + +escape_needed: + std::string e; + e.reserve(8 + s.length()); + for (char& c : s) { + switch (c) { + case '~': + e += "~0"; + break; + case '/': + e += "~1"; + break; + case '\n': + if (flags.strict_json_pointer_syntax) { + return ""; + } + e += "~n"; + break; + case '\r': + if (flags.strict_json_pointer_syntax) { + return ""; + } + e += "~r"; + break; + default: + e += c; + break; + } + } + return e; +} + +std::string // +print_json_pointers(JsonThing& jt, std::string s, uint32_t depth) { + std::cout << s << std::endl; + if (depth++ >= flags.max_output_depth) { + return ""; + } + + switch (jt.kind) { + case JsonThing::Kind::Array: + s += "/"; + for (size_t i = 0; i < jt.value.a.size(); i++) { + TRY(print_json_pointers(jt.value.a[i], s + std::to_string(i), depth)); + } + break; + case JsonThing::Kind::Object: + s += "/"; + for (auto& kv : jt.value.o) { + std::string e = escape(kv.first); + if (e.empty() && !kv.first.empty()) { + return "main: unsupported \"\\u000A\" or \"\\u000D\" in object key"; + } + TRY(print_json_pointers(kv.second, s + e, depth)); + } + break; + } + return ""; +} + +std::string // +main1(int argc, char** argv) { + TRY(parse_flags(argc, argv)); + + int input_file_descriptor = 0; // A 0 default means stdin. + if (flags.remaining_argc > 1) { + return usage; + } else if (flags.remaining_argc == 1) { + const char* arg = flags.remaining_argv[0]; + input_file_descriptor = open(arg, O_RDONLY); + if (input_file_descriptor < 0) { + return std::string("main: cannot read ") + arg + ": " + strerror(errno); + } + } + + TokenStream ts(input_file_descriptor); + JsonThing::Result jtr = JsonThing::parse(ts); + if (!jtr.status_msg.empty()) { + return jtr.status_msg; + } + return print_json_pointers(jtr.thing, "", 0); +} + +// ---- + +int // +compute_exit_code(std::string status_msg) { + if (status_msg.empty()) { + return 0; + } + std::cerr << status_msg << std::endl; + // Return an exit code of 1 for regular (forseen) errors, e.g. badly + // formatted or unsupported input. + // + // Return an exit code of 2 for internal (exceptional) errors, e.g. defensive + // run-time checks found that an internal invariant did not hold. + // + // Automated testing, including badly formatted inputs, can therefore + // discriminate between expected failure (exit code 1) and unexpected failure + // (other non-zero exit codes). Specifically, exit code 2 for internal + // invariant violation, exit code 139 (which is 128 + SIGSEGV on x86_64 + // linux) for a segmentation fault (e.g. null pointer dereference). + return (status_msg.find("internal error:") != std::string::npos) ? 2 : 1; +} + +int // +main(int argc, char** argv) { + std::string z = main1(argc, argv); + int exit_code = compute_exit_code(z); + return exit_code; +} diff --git a/example/jsonptr/jsonptr.cc b/example/jsonptr/jsonptr.cc index 54801398..11f4201f 100644 --- a/example/jsonptr/jsonptr.cc +++ b/example/jsonptr/jsonptr.cc @@ -57,6 +57,35 @@ After modifying this program, run "build-example.sh example/jsonptr/" and then ---- +This program uses Wuffs' JSON decoder at a relatively low level, processing the +decoder's token-stream output individually. The core loop, in pseudo-code, is +"for_each_token { handle_token(etc); }", where the handle_token function +changes global state (e.g. the `depth` and `context` variables) and prints +output text based on that state and the token's source text. Notably, +handle_token is not recursive, even though JSON values can nest. + +This approach is centered around JSON tokens. Each JSON 'thing' (e.g. number, +string, object) comprises one or more JSON tokens. + +An alternative, higher-level approach is in the sibling example/jsonfindptrs +program. Neither approach is better or worse per se, but when studying this +program, be aware that there are multiple ways to use Wuffs' JSON decoder. + +The two programs, jsonfindptrs and jsonptr, also demonstrate different +trade-offs with regard to JSON object duplicate keys. The JSON spec permits +different implementations to allow or reject duplicate keys. It is not always +clear which approach is safer. Rejecting them is certainly unambiguous, and +security bugs can lurk in ambiguous corners of a file format, if two different +implementations both silently accept a file but differ on how to interpret it. +On the other hand, in the worst case, detecting duplicate keys requires O(N) +memory, where N is the size of the (potentially untrusted) input. + +This program (jsonptr) allows duplicate keys and requires only O(1) memory. As +mentioned above, it doesn't dynamically allocate memory at all, and on Linux, +it runs in a SECCOMP_MODE_STRICT sandbox. + +---- + This example program differs from most other example Wuffs programs in that it is written in C++, not C. @@ -152,9 +181,9 @@ static const char* usage = "\n" "An absent query is equivalent to the empty query, which identifies the\n" "entire input (the root value). Unlike a file system, the \"/\" query\n" - "does not identify the root. Instead, it identifies the child (the value\n" - "in a key-value pair) of the root whose key is the empty string.\n" - "Similarly, \"/foo\" and \"/foo/\" identify two different nodes.\n" + "does not identify the root. Instead, \"\" is the root and \"/\" is the\n" + "child (the value in a key-value pair) of the root whose key is the empty\n" + "string. Similarly, \"/xyz\" and \"/xyz/\" are two different nodes.\n" "\n" "If the query found a valid JSON value, this program will return a zero\n" "exit code even if the rest of the input isn't valid JSON. If the query\n" @@ -982,6 +1011,7 @@ main1(int argc, char** argv) { } curr_token_end_src_index += n; + // Skip filler tokens (e.g. whitespace). if (t.value() == 0) { continue; }