ghostty-rs/docs/rewrite/notes/final-bytes-refactor.md

# Raw-byte config parsing refactor

Scope: Ghostty 1.3.1, the nine remaining groups in `review-rv8-codex.md`, and
the config/CLI-to-consumer encoding sweep. No version change or commit.

## Design

`ArgIter` yields original byte vectors. `LineIterator` trims and splits bytes;
it never escape-encodes values. Process arguments use `collect_bytes`, including
arguments after `-e`. Replay and conditional replay retain byte vectors.

`CliTarget::parse_into_field`, generated `parse_field_bytes`, and
`ConfigField::parse_bytes` take `Option<&[u8]>`. Each field chooses its contract:

- Text/numeric/enum grammars validate UTF-8 at the field boundary and return
  `InvalidValue` for invalid input. Selection delimiters decode codepoints from
  original bytes and validate each unescaped UTF-8 sequence. Clipboard string
  replacements are validated UTF-8, stored as ordinary Rust strings.
- Byte strings, paths, environment entries, commands, font family/style names,
  binding payloads, key-table names, themes, readable input, and command-palette
  labels retain their bytes in `EncodedOs`. Font variation identifiers retain
  their four original bytes.
- Tagged unions, comma splitting, and automatic structs split ASCII syntax
  without interpreting payload bytes. Quoted automatic-struct values use the
  raw Zig string-literal decoder. `config/string.zig` parsing independently
  copies unescaped bytes and UTF-8-encodes escaped codepoints, as upstream does.
- `parse_cli(&str)` and `parse_field(..., &str)` remain convenience APIs for
  **plain UTF-8**. They forward `str::as_bytes`; they never decode OS escapes.

`EncodedOs` remains a distinct storage/OS-plumbing type. `from_bytes` and
`from_plain_text` are input constructors; `from_encoded` is reserved for
already encoded internal data. Runtime font, title, key-table, and command
consumers now carry the typed values to their final boundary. GTK display
conversion is explicit and separate from C callbacks and byte output.

The existing `fmt::Write` config formatter uses an internal encoded string.
Plain text is encoded exactly once when inserted; `+show-config` decodes once
to bytes at the writer. This preserves normal output and arbitrary byte-valued
fields. Action strings retain Zig's `std.zig.stringEscape` formatting, now
applied to the original bytes. Upstream escaping/reset/append semantics are
preserved; this does not promise that every formatted action or repeatable list
is an identity operation when loaded over existing defaults.

## RV8 finding → fix → regression coverage

| # | Fix | Tests |
|---|---|---|
| 1 | `EncodedOs::to_path` is fallible. Windows paths validate WTF-8 and report `BadPathName`; config files, themes, renderer assets and other path consumers propagate/handle the error. | `osbytes::tests::windows_paths_fail_with_bad_path_name_instead_of_replacement` |
| 2 | Windows application, cwd, argv, configured environment, and inherited environment use strict UTF-8 conversion before UTF-16/process creation. Surrogate encodings fail. | `windows_command_inputs_require_strict_utf8`; `windows_args::tests::inherited_environment_rejects_non_utf8`; Windows `env_overrides_keep_literal_fdd0_and_reject_surrogates` |
| 3 | WTF-8 validation accepts separately encoded surrogate pairs without requiring canonical re-encoding equality. The Kitty path conversion uses this corrected decoder. | `wtf8_accepts_separately_encoded_surrogate_pairs_like_zig` |
| 4 | Binding triggers parse original UTF-8, so literal U+FDD0 is one codepoint. Only action payloads and table names become typed byte storage. | `raw_bytes::literal_fdd0_is_a_single_unicode_trigger`, through both file and CLI readers |
| 5 | Selection delimiter codepoint iteration rejects invalid original UTF-8 with `InvalidValue`. | `raw_bytes::selection_word_chars_reject_invalid_utf8` |
| 6 | Clipboard replacements validate UTF-8 before storage; the consumer no longer decodes an escape representation or substitutes U+FFFD. | `raw_bytes::clipboard_replacements_require_utf8_and_keep_literal_fdd0`; clipboard plain-text consumer test |
| 7 | Font styles and codepoint-map families remain typed bytes through descriptors. Fontconfig receives original bytes; CoreText validates UTF-8 when constructing its CF strings. | `raw_bytes::font_style_and_codepoint_family_keep_original_bytes`; `discovery::tests::configured_font_names_reach_fontconfig_as_original_bytes` |
| 8 | Configured, dynamic and binding-action titles remain bytes through runtime actions and C string construction. | `c_title_actions_and_key_table_names_forward_original_bytes`; embedded integration `configured_and_binding_titles_reach_c_as_original_bytes` |
| 9 | New-window preserves valid original argument strings; incoming D-Bus text is plain UTF-8. Invalid UTF-8/interior-NUL arguments abort as Zig/GIO does, without untrusted-GVariant empty-string substitution. Flatpak environment construction reproduces GLib's whole-value `[Invalid UTF-8]` substitution. | `arguments_are_plain_dbus_strings_not_escape_encoded`; subprocess `invalid_dbus_strings_fail_without_substitution`; `glib_env_replaces_the_whole_invalid_value` |

## Additional sweep

Reviewed uses of `from_encoded`, `as_encoded`, `config_str_to_bytes`,
`config_str_to_text`, lossy UTF-8 conversion, config field implementations,
formatters, and downstream consumers across the workspace.

Additional fixes from this sweep:

- Raw command parsing, environment maps, initial input, and auto-struct quoted
  payloads no longer see encoding artifacts.
- Key-table names, command-palette titles/descriptions and C name lengths
  retain original bytes; command-palette sorting compares original bytes.
- Font feature tags pass original bytes to their backends. CoreText's ASCII
  conversion is checked. Font variation tags accept any four bytes, and their
  formatter no longer uses lossy UTF-8.
- Selection, clipboard, trigger and composed keybind formatting distinguish
  plain text from internal encoded output, preventing double escaping.
- GTK diagnostics decode the formatted bytes before insertion and mirror
  GTK's rejection of invalid UTF-8. The working-directory GObject property
  renders plain text from typed byte storage, including in tab tooltips and
  window subtitles. Theme preview display/search and theme-name sorting use
  decoded text/original bytes. Logs render decoded text; Windows edit-config
  output writes the original path bytes.
- Link regex strings are plain text and are no longer mistakenly decoded as
  OS escapes. Clipboard replacement consumers likewise use validated text.
- C binding-action input and embedded initial input enter parsing as raw
  bytes. Text/CSI/ESC output retains original payload bytes and byte lengths.
- Theme lookup carries typed paths without encode/decode detours or an empty
  path fallback. Windows filesystem and process conversion contracts are
  deliberately separate.
- Windows home expansion returns the original byte value, as Zig does. PATH
  lookup and shell-integration resource paths use checked filesystem conversion.
  Bash's extracted rcfile argument validates UTF-8 before entering a Windows
  environment string. Windows-specific regressions cover these three paths.
- Byte-based trimming matches each Zig parser: space/tab for config syntax;
  all Zig ASCII whitespace, including vertical tab, for working directories,
  environment entries, and new-window options.

Remaining encoding uses are storage/output, diagnostics, existing command
execution plumbing, and explicitly tagged GTK clipboard transport.
They do not feed escape-encoded values to config field parsers. Lossy conversion
is confined to display APIs or legacy OS helpers; Windows path and process
operations use checked conversions.

A direct GLib/GIO C probe compared trusted and untrusted serialized strings:
untrusted invalid UTF-8 became an empty D-Bus string; Zig's trusted construction
aborted in `g_dbus_message_to_blob`. The Rust IPC path explicitly aborts for
these malformed strings before creating a variant, preserving that failure
without violating the Rust GLib binding's trusted-data safety contract.

Additional regressions cover raw byte fields/auto structs, whitespace,
four-byte variation tags, `+show-config` output/reload for both byte and text
fields, and action escaping from original bytes. Existing byte-path, shell,
clipboard, key-text and preedit tests remain in the workspace suite.

## Validation

All required gates passed on the completed refactor:

| Gate | Result |
|---|---|
| `timeout 2400 cargo test --workspace --no-fail-fast` | Exit 0; 3,428 passed, 0 failed, 7 ignored across reported test summaries |
| `cargo clippy --workspace --all-targets -- -D warnings` | Exit 0 |
| `cargo +nightly fmt --all` | Exit 0 |
| `just check-macos` | Exit 0 |
| `just check-macos x86_64-apple-darwin` | Exit 0 |
| `just check-windows` | Exit 0 |
| `just check-wasm` | Exit 0, including Node execution |
| `cargo build --bin ghostty` | Exit 0 |
| `git diff --check` | Exit 0 |

The exact required comparison,
`diff <(target/debug/ghostty +show-config --default --docs) <(ghostty +show-config --default --docs)`,
reports only line 3837: `gtk-opengl-debug = true` in Rust versus `false` in the
installed Ghostty binary. Its exit code is 1 for this single expected difference.

Eight differential `+show-config` probes against `/usr/bin/ghostty` match byte
for byte, each exiting 0: ordinary config; raw byte strings/font names;
literal U+FDD0 delimiters/replacements/triggers; rejected invalid UTF-8;
raw title/text/search actions; quoted automatic structs; environment ASCII
whitespace; and four-byte font variation tags. These probes explicitly set
`auto-update-channel = stable`: without this setting the pre-existing loaded
config default differs (`stable` in the port versus `tip` in the installed
reference). The required `--default --docs` comparison is unnormalized.

The macOS and Windows checks compile/lint platform-specific paths; they are
not native hardware execution. The WASM check also executes its Node host
smoke test. Windows conversion regressions execute as platform-independent
unit tests on Linux, with additional Windows-only tests cross-compiled. No new Zig tests were skipped
for these fixes; this refactor adapts existing ports and adds regressions.