ghostty-rs/docs/rewrite/notes/ghostty-terminal-pagelist.md

# ghostty-terminal page memory model — partial port

This is **not a completed Ghostty PageList parity port**. The core structures and
many operations are implemented, but the remaining PageList cases below must be
ported and used to drive corrections before Screen/Terminal relies on exact Zig
reflow, allocation, and viewport behavior. In particular, passing the current
crate tests does not establish equivalence of page boundaries under reflow.

No version bump or commit was made. Only `ghostty-vt` is required; no unicode,
datastruct, or config dependency was introduced. Reference files were read only.

## Module and test inventory

| Zig source | Rust module | Source cases represented | Extra Rust tests | Unported / skipped |
|---|---|---:|---:|---:|
| size.zig | size | 1 / 5 | 0 | 4 pointer-only tests skipped |
| point.zig | point | 0 / 0 | 0 | 0 |
| style.zig | style | 37 / 37 | 0 | 0 |
| hyperlink.zig | hyperlink | 0 / 0 | 0 | 0 |
| ref_counted_set.zig | ref_counted_set | 0 / 0 | 2 | 0 |
| page.zig | page | 46 / 46 | 0 | 0 |
| PageList.zig | page_list | 151 / 202 | 2 | 50 unported, 1 skipped |

Total: **239 passing Rust tests**, consisting of 235 source-named regression
cases and four added regressions. Some PageList cases use smaller equivalent
fixtures and consolidate their assertions; matching a test name is not a claim
that all incidental assertions or allocation paths in that Zig case were copied.
Style formatter fixtures retain their reference expected strings. All 17
semantic-highlight cases retain the reference grid setup and expected endpoints.

Intentional skips:

- `Offset ptr u8`, `Offset ptr structural`, `getOffset bytes`, `getOffset structs`:
  raw-address/base-pointer machinery was replaced with owned storage and logical
  cell indices. `Offset<T>` is a typed u32 logical offset; it does not dereference
  page-relative pointers.
- `PageList init error`: Zig tripwire/allocator failure injection has no Rust
  counterpart here. Fallible grid reservations return errors, but allocation
  failure coverage is incomplete.
- `bitmap_allocator.zig` (21 tests) and `hash_map.zig` (22 tests) are not ported
  as modules, as requested. Their logical accounting is implemented in page.rs.

## Implemented structures and behavior

- Cell and Row are transparent u64 wrappers, Copy + Clone, exactly eight bytes.
  Cell preserves the Zig bit offsets: tag 0, payload 2, style ID 26, wide 42,
  protected 44, hyperlink 45, semantic content 46. Row stores a cell index in its
  low 32 bits and uses the same flag positions as Zig. Row cell indices are not
  byte offsets; rotating rows moves metadata and preserves auxiliary-map keys.
- Page owns row/cell Vecs, a deduplicated style set, grapheme data and allocation
  maps, a hyperlink map/set, and logical string/grapheme chunk allocators.
- `Page::layout` preserves the reference 64-bit Linux layout calculation,
  including the unusual bitmap allocator formula `aligned_cap * chunk_size`,
  bitmap rounding to 64 chunks, map headers even at zero capacity, set 13/16
  load factor and reserved ID zero, and 4096-byte final rounding. A standalone
  Zig ABI probe confirmed Style item size/alignment 36/4, hyperlink item 48/8,
  and offset-slice 16/8. The layout does not report actual Rust heap overhead.
- Logical allocation tracks contiguous chunks, including temporary allocation
  of a larger grapheme before freeing its old chunks. Grapheme and hyperlink
  maps enforce separate entry limits. Dead hyperlink entries retain their
  string chunks until their set slot is reclaimed.
- Page provides styles, graphemes, hyperlinks, copy/clone, partial-row copying,
  moves, swaps, clearing, exact row capacity, and debug/test verification.
  Unmanaged partial-row copies use a slice copy without per-cell maps or
  allocation. Default styles bypass set lookup.
- PageList uses `Vec<Option<Node>>`, a free-index list, generation-bearing
  NodeId, and a separate reusable standard-page pool. It accounts substandard
  pages as standard pool allocations. Growing prunes whole history pages while
  protecting the active rows. Reused node indices receive new generations.
- PageList has list-owned tracked handles, viewport scrolling, byte/row totals,
  splitting, capacity growth, compaction, erasure, reset, history clearing,
  cloning, row/column resize, reflow, prompt iteration/highlighting, and forward
  or reverse page/row/cell iterators. Iterators also implement DoubleEndedIterator
  where appropriate. PromptIterator follows continuation handling from Zig.

## Important deviations / remaining work

1. **Reflow and wider no-reflow resize rebuild a snapshot of rows into new pages.**
   They do not yet implement Zig's incremental ReflowCursor and page backfill
   algorithm. New page auxiliary capacities are sized from encountered content,
   instead of following the exact sequence of capacity doubling and splitting.
   Page boundaries, intermediate allocations, pruning, and maximum-size behavior
   can differ. This is the most important unfinished work.
2. Snapshot reflow is linear in all cells and allocates temporary row vectors,
   copies managed values, and scans tracked locations while copying. It is not
   ready for the intended terminal hot-path performance requirements. Point
   conversion and viewport offsets traverse the list; the Zig active/viewport
   caches have not been reproduced.
3. Cursor preservation during combined row/column resize is incomplete. Basic
   tracked cursor positions, blank-cell clamping, trailing blank trimming, and
   several wide/spacer cases are tested, but cross-page cursors, old/new wrap
   counts, viewport remapping under reflow, and all combinations need the
   remaining reference cases.
4. The HashMap-backed RefCountedSet preserves zero reservation, bounded live
   slots, sequential IDs, trailing dead-slot trimming, `add_with_id`, checked
   reference counts, and OutOfMemory/NeedsRehash errors. It does **not** reproduce
   Robin Hood probe-dependent dead-ID reuse or the 32-probe failure threshold.
   Thus exact reused IDs and the point at which NeedsRehash occurs can differ.
   Hash values are Rust implementation details; Zig Style/Wyhash output is not
   reproduced. Style implements Hash/Eq for the Rust deduplication map.
5. Direct Page clone uses Rust Clone. Some auxiliary temporary allocations and
   owned-value clones still use infallible standard collection operations.
   Actual process allocator exhaustion is not completely surfaced as Result;
   logical capacity exhaustion is. Allocation-failure atomicity has not been
   fully audited. There is no unsafe code or todo!/unimplemented! path.
6. The row-range eraser supports arbitrary start ranges, while Zig's tested
   eraseRows use is primarily prefix/history/active erasure. Exact destination
   pins for arbitrary whole-page removal in the middle need further parity work.
7. `style::BoldColor` is a small local policy enum because terminal must not
   depend on config. TODO(integration): convert the configuration type at the
   Screen boundary. Rgb/Palette/Underline are reused from ghostty-vt.
8. Logical layout is for 64-bit Linux with a 4096-byte minimum page size. Other
   ABIs are not implemented. Pool reuse reuses Pages with matching capacities;
   it does not share a mmap allocator between independent PageLists.

## API overview for Screen / Terminal agents

### Coordinate and lifetime model

`Point::{Active,Viewport,Screen,History}(Coordinate { x: u16, y: u32 })`
selects the coordinate frame. `Point::new(Tag, Coordinate)` is useful for dynamic
frames. `pin(point)` and `point_from_pin(tag, pin)` return Option for bounds errors.
`get_top_left` and `get_bottom_right` return None for an empty region such as
history before the first scroll.

`Pin { node: NodeId, y: u16, x: u16, garbage: bool }` is Copy and contains no
references. Pass the owning list explicitly to its access and movement methods:
`p.row_and_cell(&list)`, `p.style(&list)`, `p.grapheme(&list)`,
`p.cells(&list, CellSubset::All)`, `p.up(&list, n)`, `p.right_wrap(&list, n)`, etc.
`CellSubset::Left` and `Right` both include the pin column. Vertical overflow
methods return `Overflow::{Offset,Overflow { end, remaining }}`. `diagonal` uses
signed deltas and clamps columns. `before`/`is_between` use row-major order.

Untracked pins must be reacquired after content-moving mutations. For a cursor,
selection endpoint, or saved location, call `track_pin(pin) -> Result<TrackedPin>`.
Read its current Copy value using `tracked_pin(handle)`; change its position with
`set_tracked_pin(handle, pin)`. Release with `untrack_pin(handle)`. Handles are
specific to the owning list. Recycled handles have a new generation, so releasing
an old handle does not release its successor. Pruning/reset mark external pins
as garbage; ordinary erasure remaps them without setting garbage. The list owns
one internal tracked viewport handle, included in `count_tracked_pins()`.

### Reading and writing cells

`PageList::get_cell(point)` returns a Copy Cell, and `get_cell_mut(point)` returns
a direct mutable reference. `node(id)` / `node_mut(id)` expose the page.
`Page::get_row(y)`, `get_cells(y)`, `get_cells_mut(y)`, and `cell_index(x,y)` use
logical row coordinates and preserve O(1) row access even after row rotation.
Auxiliary lookup methods take a **cell index** returned by cell_index, not a
column or pointer. The vectors include unused capacity; iteration should respect
`page.size`, not the vector lengths.

Construct plain text with `Cell::init(codepoint)`. Set flags with `set_wide`,
`set_protected`, `set_semantic_content`, etc. `set_codepoint` changes only the
payload; use `set_content_tag` when changing between text and background colors.
For managed cells, clear the old data with page operations before assigning a
new raw Cell. Direct writes must maintain row flags and dirty tracking.

- `set_style(x,y,Style)` handles style add/release and marks the row styled.
  Default style is ID 0; `page.style(cell)` resolves it without map access.
- `append_grapheme(x,y,cp)` and `set_graphemes(x,y,&extra)` store only codepoints
  beyond the cell's first codepoint. Set the first nonzero codepoint beforehand.
- `insert_hyperlink(Hyperlink)` returns an already-retained ID. Attach it with
  `set_hyperlink(x,y,id)`, which consumes that reference without incrementing it.
  For another cell, call `page.hyperlink_set.r#use(id)` first. If attachment fails,
  release the retained ID. `clear_hyperlink(index)` releases the cell reference.
- `clear_cells(y,start,end,protect)` uses half-open columns. `protect=true` is a
  Rust convenience that skips protected cells. Move and swap helpers transfer
  auxiliary ownership and avoid changing reference counts for moved values.

`row.set_dirty(true)` / `pin.mark_dirty(&mut list)` support direct writes;
`list.clear_dirty()` resets page and row flags. `is_dirty` checks both levels.
Conservative managed-row flags may be true even after their last value is cleared.
Use the explicit `update_row_*_flag` methods when a precise flag is needed.

### Errors and capacity recovery

`page::Error` includes separate style set, grapheme allocator/map, hyperlink
set/map, and string exhaustion variants. `ref_counted_set::AddError` distinguishes
OutOfMemory from NeedsRehash, plus invalid IDs/counts for safe API misuse handling.
`RefCountedSet::add_with_id(value,id)` returns None when the requested ID was used,
otherwise Some(actual_id). `get` and `lookup` do not retain values.

`increase_capacity(node, Some(IncreaseCapacity::{Styles,GraphemeBytes,
HyperlinkBytes,StringBytes}))` replaces a page with increased limits and remaps
tracked pins; None rebuilds the same capacity to reclaim dead IDs.
`adjust_capacity` is an alias. OutOfSpace means doubling/page-size limits prevent
further growth; `split(pin)` can divide at a row boundary. Splitting row zero is
otherwise a no-op, and a single-row page returns OutOfSpace. `compact(node)`
returns Some(replacement) only when an oversized page can shrink.
Always use the returned NodeId, or resolve a tracked pin again after replacement.

### List mutations and iteration

`grow()` adds one row, returning Some(new_node) if another page was needed.
`scroll(Scroll::{Active,Top,Row,DeltaRow,DeltaPrompt,Pin})` only changes viewport.
`scrollbar()` reports row counts and hides physical history at max_size zero.
`scroll_clear()` moves the nonempty active rows into history.

`erase_rows(top, optional_bottom)` removes inclusive rows and ensures active height.
`erase_row(point)` shifts the remainder of the screen while retaining page sizes.
`erase_row_bounded(point,limit)` shifts at most limit following rows.
`erase_cells_bounded(start_pin,end_pin,left,right)` is an extra rectangular helper.
`reset()` clears content and invalidates tracked contents; `clear_history()`
preserves the active area.

`resize(Resize { cols, rows, reflow, cursor })` takes optional new dimensions and
an optional active `Cursor { x,y }`. Reflow defaults to true. Cursor is input
context; track the actual cursor pin separately to read back its new position.
Consult the limitations above before relying on complex resize behavior.

`clone_range(top,bottom)` clones inclusive **whole rows**, ignoring x, pads to the
active height, and returns `(PageList, HashMap<old_handle,new_handle>)`.
`try_clone()` clones the whole screen without retaining external tracked pins.
Clone viewport defaults to Active; dirty information is retained by rows.
Page itself implements Clone for an independent full copy; its inherent
`clone_from(source,start,end)` copies a half-open row range into existing capacity.
`clone_partial_row_from` accepts source/destination row and column plus length;
`clone_cells_within` handles same-page copies without conflicting borrows.

`page_iterator`, `row_iterator`, `cell_iterator` take Direction, start Point,
and optional inclusive end Point. End None uses the frame's bottom-right.
Reverse direction traverses the same bounds backwards. PageIterator yields
`Chunk { node,start,end }` with half-open page-local rows. Iterators borrow the
list immutably; collect Copy pins/chunks or end iteration before mutating it.
`prompt_iterator` finds semantic prompt starts; `highlight_semantic_content`
returns an optional inclusive `(start_pin,end_pin)` for a prompt's Prompt, Input,
or Output region.

`Page::verify_integrity` and `PageList::verify_integrity` are available in debug
and test builds, as are assert_integrity helpers. Nestable pause methods support
batched low-level updates. Verification is explicit; raw mutable access can
break invariants until the caller completes its update.

## Remaining PageList source tests

The following 50 cases are **unported**, rather than justified skips. They include
important production behavior and should drive the next implementation pass.
- `PageList grow allows exceeding max size for active area`
- `PageList grow prune required with a single page`
- `PageList scroll delta row back without space preserves active`
- `PageList: jump zero prompts`
- `Screen: jump back one prompt`
- `Screen: jump forward prompt skips multiline continuation`
- `PageList eraseRows invalidates viewport offset cache`
- `PageList eraseRow invalidates viewport offset cache`
- `PageList eraseRowBounded invalidates viewport offset cache`
- `PageList eraseRowBounded multi-page invalidates viewport offset cache`
- `PageList eraseRowBounded full page shift invalidates viewport offset cache`
- `PageList eraseRowBounded exhausts pages invalidates viewport offset cache`
- `PageList promptIterator right_down continuation at start`
- `PageList promptIterator right_down with prompt before continuation`
- `PageList promptIterator right_down limit inclusive`
- `PageList promptIterator left_up limit inclusive`
- `PageList erase resets viewport to active if moves within active`
- `PageList erase resets viewport if inside erased page but not active`
- `PageList erase resets viewport to active if top is inside active`
- `PageList resize (no reflow) less rows cursor on bottom`
- `PageList resize (no reflow) less rows cursor in scrollback`
- `PageList resize (no reflow) more rows contains viewport`
- `PageList resize (no reflow) more cols with spacer head`
- `PageList resize (no reflow) grow cols fast path with spacer head`
- `PageList resize (no reflow) more cols forces less rows per page`
- `PageList resize more rows and cols doesn't fit in single std page`
- `PageList resize (no reflow) more cols forces smaller cap`
- `PageList resize (no reflow) more rows adds blank rows if cursor at bottom`
- `PageList resize reflow invalidates viewport offset cache`
- `PageList resize reflow more cols creates multiple pages`
- `PageList resize reflow more cols wrap across page boundary`
- `PageList resize reflow more cols wrap across page boundary cursor in second page`
- `PageList resize reflow less cols wrap across page boundary cursor in second page`
- `PageList resize reflow more cols cursor in wrapped row`
- `PageList resize reflow more cols cursor in not wrapped row`
- `PageList resize reflow more cols cursor in wrapped row that isn't unwrapped`
- `PageList resize reflow exceeds hyperlink memory forcing capacity increase`
- `PageList resize reflow exceeds grapheme memory forcing capacity increase`
- `PageList resize reflow exceeds style memory forcing capacity increase`
- `PageList resize reflow more cols unwrap wide spacer head`
- `PageList resize reflow more cols unwrap wide spacer head across two rows`
- `PageList resize reflow less cols cursor goes to scrollback`
- `PageList resize reflow less cols copy kitty placeholder`
- `PageList resize reflow more cols clears kitty placeholder`
- `PageList resize reflow wrap moves kitty placeholder`
- `PageList resize reflow grapheme map capacity exceeded`
- `PageList resize grow cols with unwrap fixes viewport pin`
- `PageList grow reuses non-standard page without leak`
- `PageList grow non-standard page prune protection`
- `PageList resize (no reflow) more cols remaps pins in backfill path`

## Quality gate

Final gate (2026-10-11):

- `cargo fmt -p ghostty-terminal`: passed; only the documented nightly-option
  warnings from workspace rustfmt configuration.
- `cargo build -p ghostty-terminal`: passed.
- `cargo test -p ghostty-terminal --quiet`: 239 passed, 0 failed, 0 ignored.
- `cargo clippy -p ghostty-terminal --all-targets -- -D warnings`: passed.
- `git diff --check -- crates/ghostty-terminal docs/rewrite/notes/ghostty-terminal-pagelist.md`:
  passed for tracked changes.

The early missing `ghostty-vt::stream` dependency file was resolved by concurrent
workspace work; no change to that crate was needed here. The terminal Cargo.toml
already had local changes at the beginning of this task and was not edited here.