feat(terminal): port kitty graphics protocol storage and execution

Ports ZIG/src/terminal/kitty/graphics_*.zig:
- image loading for direct, file, temp-file and shared-memory
  transmissions (rustix::shm, no unsafe), chunking, zlib and PNG decode
- image storage with placements (incl. virtual) tracked by PageList pins,
  all delete modes, byte-limit eviction and dirty tracking
- unicode placeholder decoding and placement iteration
- command execution, generic over an ExecTerminal trait

Deviations: storage methods take the PageList explicitly; evicted or
replaced placements now release their tracked pins (Zig leaks them);
file transmissions with S= truncate per the kitty spec. Zig quirks kept
for parity: delete-range matches all placements, eviction stops only
after freeing strictly more than requested, set_limit(0) resets.

Not yet wired: Screen still holds the KittyImages placeholder, so
Terminal does not implement ExecTerminal; integration steps are in
docs/rewrite/notes/ghostty-terminal-kitty.md.

Test Plan:
- cargo test -p ghostty-terminal (66 kitty tests; all Zig tests ported)
This commit is contained in:
ddidderr committed 2026-10-11 22:52:07 +02:00
1 parent 8e9be42da8
commit d47408f31c
12 files changed
+4176

No files matched your search

@@ -0,0 +1,644 @@
//! Kitty graphics command execution (`ZIG/src/terminal/kitty/graphics_exec.zig`).
//!
//! # Integration
//!
//! Zig's `execute` takes a `*Terminal` and reaches
//! `terminal.screens.active.kitty_images` (the [`ImageStorage`]),
//! `terminal.screens.active.pages`, the cursor, `terminal.index()` and
//! `terminal.setCursorPos()`. In Rust the storage lives inside the `Screen`
//! next to the `PageList`, so everything `execute` needs is exposed through
//! the small [`ExecTerminal`] trait (notably a split borrow of storage +
//! page list). `Terminal` implements it by delegating to its active screen;
//! see `docs/rewrite/notes/ghostty-terminal-kitty.md` for the exact impl.
use ghostty_vt::kitty::graphics_command::{Command, Control, Display, Quiet, Response};
use super::graphics_image::{Error as ImageError, LoadingImage};
use super::graphics_storage::{
Error as StorageError, ImageStorage, Location, Placement, TerminalInfo,
};
use crate::page_list::{PageList, Pin};
/// The terminal operations the Kitty graphics executor needs.
pub trait ExecTerminal {
/// The active screen's image storage and page list (a split borrow of
/// `screens.active().kitty_images` and `screens.active().pages`).
fn kitty_parts(&mut self) -> (&mut ImageStorage, &mut PageList);
/// Snapshot of terminal geometry and cursor position.
fn exec_info(&self) -> TerminalInfo;
/// The active screen cursor's current pin (`cursor.page_pin.*`).
fn exec_cursor_pin(&self) -> Pin;
/// `Terminal.index`. Errors are logged and stop cursor movement.
fn exec_index(&mut self) -> Result<(), crate::page::Error>;
/// `Terminal.setCursorPos` (1-based row/col).
fn exec_set_cursor_pos(&mut self, row: usize, col: usize);
}
/// Execute a Kitty graphics command against the given terminal. This
/// will never fail, but the response may indicate an error and the
/// terminal state may not be updated to reflect the command. This will
/// never put the terminal in an unrecoverable state, however.
pub fn execute<T: ExecTerminal>(terminal: &mut T, cmd: &Command) -> Option<Response> {
// If storage is disabled then we disable the full protocol. This means
// we don't even respond to queries so the terminal completely acts as
// if this feature is not supported.
if !terminal.kitty_parts().0.enabled() {
log::debug!("kitty graphics requested but disabled");
return None;
}
log::debug!(
"executing kitty graphics command: quiet={:?} control={:?}",
cmd.quiet,
cmd.control
);
// The quiet settings used to control the response. We have to make this
// a var because in certain special cases (namely chunked transmissions)
// this can change.
let mut quiet = cmd.quiet;
let resp_: Option<Response> = match &cmd.control {
Control::Query(_) => Some(query(cmd)),
Control::Display(_) => Some(display(terminal, cmd)),
Control::Delete(_) => Some(delete(terminal, cmd)),
Control::Transmit(_) | Control::TransmitAndDisplay { .. } => {
// If we're transmitting, then our `q` setting value is complicated.
// The `q` setting inherits the value from the starting command
// unless `q` is set >= 1 on this command. If it is, then we save
// that as the new `q` setting.
let storage = terminal.kitty_parts().0;
if let Some(loading) = storage.loading.as_mut() {
match cmd.quiet {
// q=0 we use whatever the start command value is
Quiet::No => quiet = loading.quiet,
// q>=1 we use the new value, but we should already be set to it
Quiet::Ok | Quiet::Failures => {
quiet = cmd.quiet;
loading.quiet = cmd.quiet;
}
}
}
Some(transmit(terminal, cmd))
}
Control::TransmitAnimationFrame(_)
| Control::ControlAnimation(_)
| Control::ComposeAnimation(_) => Some(Response {
message: "ERROR: unimplemented action".to_owned(),
..Response::default()
}),
};
// Handle the quiet settings
if let Some(resp) = resp_ {
if !resp.ok() {
log::warn!("erroneous kitty graphics response: {}", resp.message);
}
return match quiet {
Quiet::No => {
if resp.empty() {
None
} else {
Some(resp)
}
}
Quiet::Ok => {
if resp.ok() {
None
} else {
Some(resp)
}
}
Quiet::Failures => None,
};
}
None
}
fn err_response(message: &str) -> Response {
Response {
message: message.to_owned(),
..Response::default()
}
}
/// Execute a "query" command.
///
/// This command is used to attempt to load an image and respond with
/// success/error but does not persist any of the command to the terminal
/// state.
fn query(cmd: &Command) -> Response {
let Control::Query(t) = &cmd.control else {
return err_response("EINVAL: internal error");
};
// Query requires image ID. We can't actually send a response without
// an image ID either but we return an error and this will be logged
// downstream.
if t.image_id == 0 {
return err_response("EINVAL: image ID required");
}
// Build a partial response to start
let mut result = Response {
id: t.image_id,
image_number: t.image_number,
placement_id: t.placement_id,
..Response::default()
};
// Attempt to load the image. If we cannot, then set an appropriate error.
if let Err(err) = LoadingImage::init(cmd) {
encode_error(&mut result, err);
}
result
}
/// Transmit image data.
///
/// This loads the image, validates it, and puts it into the terminal
/// screen storage. It does not display the image.
fn transmit<T: ExecTerminal>(terminal: &mut T, cmd: &Command) -> Response {
let Some(t) = cmd.transmission() else {
return err_response("EINVAL: internal error");
};
let mut result = Response {
id: t.image_id,
image_number: t.image_number,
placement_id: t.placement_id,
..Response::default()
};
if t.image_id > 0 && t.image_number > 0 {
return err_response("EINVAL: image ID and number are mutually exclusive");
}
let load = match load_and_add_image(terminal, cmd) {
Ok(l) => l,
Err(err) => {
encode_error(&mut result, err);
return result;
}
};
// If we're also displaying, then do that now. This function does
// both transmit and transmit and display. The display might also be
// deferred if it is multi-chunk.
if let Some(d) = load.display {
debug_assert!(!load.more);
let mut d_copy = d;
d_copy.image_id = load.image_id;
result = display(
terminal,
&Command {
control: Control::Display(d_copy),
quiet: cmd.quiet,
data: Vec::new(),
},
);
}
// If there are more chunks expected we do not respond.
if load.more {
return Response::default();
}
// If the loaded image was assigned its ID automatically, not based
// on a number or explicitly specified ID, then we don't respond.
if load.implicit_id {
return Response::default();
}
// After the image is added, set the ID in case it changed.
// The resulting image number and placement ID never change.
result.id = load.image_id;
result
}
/// Display a previously transmitted image.
fn display<T: ExecTerminal>(terminal: &mut T, cmd: &Command) -> Response {
let Some(d) = cmd.display() else {
return err_response("EINVAL: internal error");
};
// Display requires image ID or number.
if d.image_id == 0 && d.image_number == 0 {
return err_response("EINVAL: image ID or number required");
}
// Build up our response
let mut result = Response {
id: d.image_id,
image_number: d.image_number,
placement_id: d.placement_id,
..Response::default()
};
let info = terminal.exec_info();
let cursor_pin = terminal.exec_cursor_pin();
// Verify the requested image exists if we have an ID
let (storage, pages) = terminal.kitty_parts();
let img_id = {
let img = if d.image_id != 0 {
storage.image_by_id(d.image_id)
} else {
storage.image_by_number(d.image_number)
};
let Some(img) = img else {
"ENOENT: image not found".clone_into(&mut result.message);
return result;
};
img.id
};
// Make sure our response has the image id in case we looked up by number
result.id = img_id;
// Location where the placement will go.
let location = if d.virtual_placement {
// Virtual placements are not tracked
if d.parent_id > 0 {
"EINVAL: virtual placement cannot refer to a parent".clone_into(&mut result.message);
return result;
}
Location::Virtual
} else {
// Track a new pin for our cursor. The cursor is always tracked but we
// don't want this one to move with the cursor.
match pages.track_pin(cursor_pin) {
Ok(pin) => Location::Pin(pin),
Err(err) => {
log::warn!("failed to create pin for Kitty graphics err={err:?}");
"EINVAL: failed to prepare terminal state".clone_into(&mut result.message);
return result;
}
}
};
// Add the placement
let p = Placement {
location,
x_offset: d.x_offset,
y_offset: d.y_offset,
source_x: d.x,
source_y: d.y,
source_width: d.width,
source_height: d.height,
columns: d.columns,
rows: d.rows,
z: d.z,
};
if let Err(err) = storage.add_placement(pages, img_id, result.placement_id, p) {
p.deinit(pages);
encode_error(&mut result, err.into());
return result;
}
// Apply cursor movement setting. This only applies to pin placements.
match p.location {
Location::Virtual => {}
Location::Pin(_) => match d.cursor_movement {
ghostty_vt::kitty::graphics_command::CursorMovement::None => {}
ghostty_vt::kitty::graphics_command::CursorMovement::After => {
// We use terminal.index to properly handle scroll regions.
let size = {
let (storage, _) = terminal.kitty_parts();
match storage.image_by_id(img_id) {
Some(img) => p.grid_size(img, &info),
None => return result,
}
};
for _ in 0..size.rows {
if let Err(err) = terminal.exec_index() {
log::warn!("failed to move cursor: {err:?}");
break;
}
}
let y = terminal.exec_info().cursor_y;
terminal.exec_set_cursor_pos(
usize::from(y),
usize::from(cursor_pin.x) + size.cols as usize + 1,
);
}
},
}
result
}
/// Delete placements/images.
fn delete<T: ExecTerminal>(terminal: &mut T, cmd: &Command) -> Response {
let Control::Delete(d) = &cmd.control else {
return err_response("EINVAL: internal error");
};
let info = terminal.exec_info();
let (storage, pages) = terminal.kitty_parts();
storage.delete(pages, &info, *d);
// Delete never responds on success
Response::default()
}
struct Loaded {
image_id: u32,
implicit_id: bool,
more: bool,
display: Option<Display>,
}
fn load_and_add_image<T: ExecTerminal>(
terminal: &mut T,
cmd: &Command,
) -> Result<Loaded, ImageError> {
let t = cmd.transmission().ok_or(ImageError::InternalError)?;
let (storage, pages) = terminal.kitty_parts();
// Determine our image. This also handles chunking and early exit.
let mut loading: LoadingImage = if let Some(loading) = storage.loading.as_mut() {
// Note: we do NOT want to take the command's data here because
// we're _copying_ the data. We want the command data to be freed.
loading.add_data(&cmd.data)?;
// If we have more then we're done
if t.more_chunks {
return Ok(Loaded {
image_id: loading.image.id,
implicit_id: loading.image.implicit_id,
more: true,
display: None,
});
}
// We have no more chunks. We're going to be completing the
// image so we want to destroy the pointer to the loading
// image and copy it out.
match storage.loading.take() {
Some(l) => *l,
None => return Err(ImageError::InternalError),
}
} else {
LoadingImage::init(cmd)?
};
// If the image has no ID, we assign one
if loading.image.id == 0 {
loading.image.id = storage.next_image_id;
storage.next_image_id = storage.next_image_id.wrapping_add(1);
// If the image also has no number then its auto-ID is "implicit".
// See the doc comment on the Image.implicit_id field for more detail.
if loading.image.number == 0 {
loading.image.implicit_id = true;
}
}
// If this is chunked, this is the beginning of a new chunked transmission.
// (We checked for an in-progress chunk above.)
if t.more_chunks {
// We allocate the pointer on the heap because its rare and we
// don't want to always pay the memory cost to keep it around.
let result = Loaded {
image_id: loading.image.id,
implicit_id: loading.image.implicit_id,
more: true,
display: None,
};
storage.loading = Some(Box::new(loading));
return Ok(result);
}
// Validate and store our image
let img = loading.complete()?;
let image_id = img.id;
let implicit_id = img.implicit_id;
storage.add_image(pages, img)?;
// Get our display settings
let display_ = loading.display;
Ok(Loaded {
image_id,
implicit_id,
more: false,
display: display_,
})
}
impl From<StorageError> for ImageError {
fn from(e: StorageError) -> Self {
match e {
StorageError::OutOfMemory => ImageError::OutOfMemory,
}
}
}
/// Encode an error code into a message for a response.
fn encode_error(r: &mut Response, err: ImageError) {
let msg = match err {
ImageError::OutOfMemory => "ENOMEM: out of memory",
ImageError::InternalError => "EINVAL: internal error",
ImageError::InvalidData => "EINVAL: invalid data",
ImageError::DecompressionFailed => "EINVAL: decompression failed",
ImageError::FilePathTooLong => "EINVAL: file path too long",
ImageError::TemporaryFileNotInTempDir => "EINVAL: temporary file not in temp dir",
ImageError::TemporaryFileNotNamedCorrectly => "EINVAL: temporary file not named correctly",
ImageError::UnsupportedFormat => "EINVAL: unsupported format",
ImageError::UnsupportedMedium => "EINVAL: unsupported medium",
ImageError::UnsupportedDepth => "EINVAL: unsupported pixel depth",
ImageError::DimensionsRequired => "EINVAL: dimensions required",
ImageError::DimensionsTooLarge => "EINVAL: dimensions too large",
};
msg.clone_into(&mut r.message);
}
#[cfg(test)]
mod tests {
use super::*;
use crate::terminal::{Options, Terminal};
use ghostty_vt::kitty::graphics_command::{Parser, TransmissionFormat};
/// A `Terminal` plus the image storage that, once integrated, lives in
/// `Screen.kitty_images`.
// TODO(integration): replace with `Terminal` once `Terminal` implements
// `ExecTerminal` via `screen.kitty_images: ImageStorage`.
struct Harness {
t: Terminal,
storage: ImageStorage,
}
impl Harness {
fn new(cols: u16, rows: u16) -> Self {
Harness {
t: Terminal::new(Options::new(cols, rows)).unwrap(),
storage: ImageStorage::default(),
}
}
}
impl ExecTerminal for Harness {
fn kitty_parts(&mut self) -> (&mut ImageStorage, &mut PageList) {
(&mut self.storage, &mut self.t.screens.active_mut().pages)
}
fn exec_info(&self) -> TerminalInfo {
TerminalInfo::from_terminal(&self.t)
}
fn exec_cursor_pin(&self) -> Pin {
self.t.screens.active().cursor_pin()
}
fn exec_index(&mut self) -> Result<(), crate::page::Error> {
self.t.index()
}
fn exec_set_cursor_pos(&mut self, row: usize, col: usize) {
self.t.set_cursor_pos(row, col);
}
}
fn run(h: &mut Harness, s: &str) -> Option<Response> {
let cmd = Parser::parse_string(s.as_bytes()).unwrap();
execute(h, &cmd)
}
#[test]
fn kittygfx_more_chunks_with_q_1() {
let mut t = Harness::new(5, 5);
// Initial chunk has q=1
let resp = run(&mut t, "a=T,f=24,t=d,i=1,s=1,v=2,c=10,r=1,m=1,q=1;////");
assert!(resp.is_none());
// Subsequent chunk has no q but should respect initial
let resp = run(&mut t, "m=0;////");
assert!(resp.is_none());
}
#[test]
fn kittygfx_more_chunks_with_q_0() {
let mut t = Harness::new(5, 5);
// Initial chunk has q=0
let resp = run(&mut t, "a=t,f=24,t=d,s=1,v=2,c=10,r=1,m=1,i=1,q=0;////");
assert!(resp.is_none());
// Subsequent chunk has no q so should respond OK
let resp = run(&mut t, "m=0;////").unwrap();
assert!(resp.ok());
}
#[test]
fn kittygfx_more_chunks_with_chunk_increasing_q() {
let mut t = Harness::new(5, 5);
// Initial chunk has q=0
let resp = run(&mut t, "a=t,f=24,t=d,s=1,v=2,c=10,r=1,m=1,i=1,q=0;////");
assert!(resp.is_none());
// Subsequent chunk sets q=1 so should not respond
let resp = run(&mut t, "m=0,q=1;////");
assert!(resp.is_none());
}
#[test]
fn kittygfx_default_format_is_rgba() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=t,t=d,i=1,s=1,v=2,c=10,r=1;///////////").unwrap();
assert!(resp.ok());
let img = t.storage.image_by_id(1).unwrap();
assert_eq!(img.format, TransmissionFormat::Rgba);
}
#[test]
fn kittygfx_test_valid_u32_expect_invalid_image_id() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=p,i=4294967295").unwrap();
assert!(!resp.ok());
assert_eq!(resp.message, "ENOENT: image not found");
}
#[test]
fn kittygfx_test_valid_i32_expect_invalid_image_id() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=p,i=1,z=-2147483648").unwrap();
assert!(!resp.ok());
assert_eq!(resp.message, "ENOENT: image not found");
}
#[test]
fn kittygfx_no_response_with_no_image_id_or_number() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=t,f=24,t=d,s=1,v=2,c=10,r=1,i=0,I=0;////////");
assert!(resp.is_none());
}
#[test]
fn kittygfx_no_response_with_no_image_id_or_number_load_and_display() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=T,f=24,t=d,s=1,v=2,c=10,r=1,i=0,I=0;////////");
assert!(resp.is_none());
}
#[test]
fn kittygfx_disabled_storage_never_responds() {
let mut t = Harness::new(5, 5);
t.storage.total_limit = 0;
assert!(run(&mut t, "a=q,i=1,s=1,v=1,f=24;AAAA").is_none());
}
#[test]
fn kittygfx_query_ok_and_error() {
let mut t = Harness::new(5, 5);
let resp = run(&mut t, "a=q,i=31,s=1,v=1,f=24;AAAA").unwrap();
assert!(resp.ok());
assert_eq!(resp.id, 31);
// Nothing is stored by a query.
assert!(t.storage.images.is_empty());
// Without an ID there is nothing to respond to (error is only logged).
assert!(run(&mut t, "a=q,s=1,v=1,f=24;AAAA").is_none());
}
#[test]
fn kittygfx_transmit_and_display_moves_cursor() {
let mut t = Harness::new(10, 10);
t.t.width_px = 100;
t.t.height_px = 100;
// 1x2 pixel image shown in 3 columns x 2 rows.
let resp = run(&mut t, "a=T,f=24,t=d,i=5,s=1,v=2,c=3,r=2;////////").unwrap();
assert!(resp.ok());
assert_eq!(resp.id, 5);
assert_eq!(t.storage.placements.len(), 1);
// The cursor moves to the last row of the image, right of its last column.
let cursor = &t.t.screens.active().cursor;
assert_eq!((cursor.x, cursor.y), (3, 1));
}
#[test]
fn kittygfx_delete_all() {
let mut t = Harness::new(10, 10);
run(&mut t, "a=T,f=24,t=d,i=5,s=1,v=2,C=1;////////").unwrap();
assert_eq!(t.storage.placements.len(), 1);
let resp = run(&mut t, "a=d,d=A");
assert!(resp.is_none());
assert!(t.storage.placements.is_empty());
assert!(t.storage.images.is_empty());
}
}
@@ -0,0 +1,930 @@
//! Kitty graphics image loading (`ZIG/src/terminal/kitty/graphics_image.zig`).
//!
//! [`LoadingImage`] is an image that is still being transmitted (possibly in
//! chunks). [`Image`] is a fully loaded and validated image.
use std::io::{Cursor, Read, Seek, SeekFrom};
use std::os::unix::ffi::OsStrExt;
use std::os::unix::fs::FileExt;
use std::path::Path;
use std::time::Instant;
use ghostty_vt::kitty::graphics_command::{
Command, Display, Quiet, TransmissionCompression, TransmissionFormat, TransmissionMedium,
};
use crate::page_list::Pin;
/// Maximum width or height of an image. Taken directly from Kitty.
pub const MAX_DIMENSION: u32 = 10000;
/// Maximum size in bytes, taken from Kitty.
pub const MAX_SIZE: usize = 400 * 1024 * 1024; // 400MB
/// Errors that can occur while loading an image (`Image.Error` plus
/// `OutOfMemory` in Zig).
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("out of memory")]
OutOfMemory,
#[error("internal error")]
InternalError,
#[error("invalid data")]
InvalidData,
#[error("decompression failed")]
DecompressionFailed,
#[error("dimensions required")]
DimensionsRequired,
#[error("dimensions too large")]
DimensionsTooLarge,
#[error("file path too long")]
FilePathTooLong,
#[error("temporary file not in temp dir")]
TemporaryFileNotInTempDir,
#[error("temporary file not named correctly")]
TemporaryFileNotNamedCorrectly,
#[error("unsupported format")]
UnsupportedFormat,
#[error("unsupported medium")]
UnsupportedMedium,
#[error("unsupported pixel depth")]
UnsupportedDepth,
}
/// An image that is still being loaded. The image should be initialized
/// using [`LoadingImage::init`] on the first chunk and then `add_data` for
/// each subsequent chunk. Once all chunks have been added, `complete`
/// should be called to finalize the image.
#[derive(Debug)]
pub struct LoadingImage {
/// The in-progress image. The first chunk must have all the metadata
/// so this comes from that initially.
pub image: Image,
/// The data that is being built up.
pub data: Vec<u8>,
/// This is non-null when a transmit and display command is given
/// so that we display the image after it is fully loaded.
pub display: Option<Display>,
/// Quiet is the quiet settings for the initial load command. This is
/// used if q isn't set on subsequent chunks.
pub quiet: Quiet,
}
impl LoadingImage {
/// Initialize a chunked image from the first image transmission.
/// If this is a multi-chunk image, this should only be the FIRST
/// chunk.
pub fn init(cmd: &Command) -> Result<LoadingImage, Error> {
// Build our initial image from the properties sent via the control.
// These can be overwritten by the data loading process. For example,
// PNG loading sets the width/height from the data.
let t = cmd.transmission().ok_or(Error::InternalError)?;
let mut result = LoadingImage {
image: Image {
id: t.image_id,
number: t.image_number,
width: t.width,
height: t.height,
compression: t.compression,
format: t.format,
..Image::default()
},
data: Vec::new(),
display: cmd.display(),
quiet: cmd.quiet,
};
// Special case for the direct medium, we just add the chunk directly.
if t.medium == TransmissionMedium::Direct {
result.add_data(&cmd.data)?;
return Ok(result);
}
// Otherwise, the payload data is guaranteed to be a path.
if cmd.data.contains(&0) {
log::warn!("failed to get absolute path: BadPathName");
return Err(Error::InvalidData);
}
match t.medium {
TransmissionMedium::Direct => unreachable!("handled above"),
TransmissionMedium::File | TransmissionMedium::TemporaryFile => {
let path = match std::fs::canonicalize(Path::new(std::ffi::OsStr::from_bytes(
&cmd.data,
))) {
Ok(p) => p,
Err(err) => {
log::warn!("failed to get absolute path: {err}");
return Err(Error::InvalidData);
}
};
result.read_file(
t.medium == TransmissionMedium::TemporaryFile,
t.offset,
t.size,
path.as_os_str().as_bytes(),
)?;
}
TransmissionMedium::SharedMemory => {
result.read_shared_memory(t.offset, t.size, &cmd.data)?;
}
}
Ok(result)
}
/// Reads the data from a shared memory segment.
fn read_shared_memory(&mut self, offset: u32, size: u32, path: &[u8]) -> Result<(), Error> {
use rustix::fs::Mode;
use rustix::shm;
let path_str = String::from_utf8_lossy(path);
let fd = match shm::open(path, shm::OFlags::RDONLY, Mode::empty()) {
Ok(fd) => fd,
Err(err) => {
log::warn!("unable to open shared memory {path_str}: {err}");
return Err(Error::InvalidData);
}
};
// The segment is always unlinked once we're done (even on error).
let result = self.read_shm_fd(fd, offset, size, &path_str);
let _ = shm::unlink(path);
result
}
fn read_shm_fd(
&mut self,
fd: rustix::fd::OwnedFd,
offset: u32,
size: u32,
path: &str,
) -> Result<(), Error> {
let file = std::fs::File::from(fd);
// The size from stat on may be larger than our expected size because
// shared memory has to be a multiple of the page size.
let stat_size: usize = match file.metadata() {
Ok(m) => {
if m.len() == 0 {
return Err(Error::InvalidData);
}
m.len() as usize
}
Err(err) => {
log::warn!("unable to fstat shared memory {path}: {err}");
return Err(Error::InvalidData);
}
};
let expected_size: usize = match self.image.format {
// Png we decode the full data size because later decoding will
// get the proper dimensions and assert validity.
TransmissionFormat::Png => stat_size,
// For these formats we have a size we must have.
f => {
let bpp = usize::from(f.bpp());
(self.image.width as usize)
.saturating_mul(self.image.height as usize)
.saturating_mul(bpp)
}
};
// Our stat size must be at least the expected size otherwise
// the shared memory data is invalid.
if stat_size < expected_size {
log::warn!("shared memory size too small expected={expected_size} actual={stat_size}");
return Err(Error::InvalidData);
}
// Our end size always uses the expected size so we cut off the
// padding for mmap alignment.
let start = offset as usize;
let end = if size > 0 {
(start + size as usize).min(expected_size)
} else {
expected_size
};
if start > end {
return Err(Error::InvalidData);
}
debug_assert_eq!(self.data.len(), 0);
let mut buf = vec![0u8; end - start];
if let Err(err) = file.read_exact_at(&mut buf, start as u64) {
log::warn!("unable to read shared memory {path}: {err}");
return Err(Error::InvalidData);
}
self.data = buf;
Ok(())
}
/// Reads the data from a (temporary) file.
///
/// This will also delete the temporary file if it is in a safe location.
fn read_file(
&mut self,
temporary: bool,
offset: u32,
size: u32,
path: &[u8],
) -> Result<(), Error> {
// Verify file seems "safe". This is logic copied directly from Kitty,
// mostly. This is really rough but it will catch obvious bad actors.
if path.starts_with(b"/proc/")
|| path.starts_with(b"/sys/")
|| (path.starts_with(b"/dev/") && !path.starts_with(b"/dev/shm/"))
{
return Err(Error::InvalidData);
}
// Temporary file logic
if temporary {
if !is_path_in_temp_dir(path) {
return Err(Error::TemporaryFileNotInTempDir);
}
if !contains_subslice(path, b"tty-graphics-protocol") {
return Err(Error::TemporaryFileNotNamedCorrectly);
}
}
let os_path = Path::new(std::ffi::OsStr::from_bytes(path));
let result = self.read_file_inner(os_path, offset, size);
if temporary && let Err(err) = std::fs::remove_file(os_path) {
log::warn!("failed to delete temporary file: {err}");
}
result
}
fn read_file_inner(&mut self, path: &Path, offset: u32, size: u32) -> Result<(), Error> {
let mut file = match std::fs::File::open(path) {
Ok(f) => f,
Err(err) => {
log::warn!("failed to open temporary file: {err}");
return Err(Error::InvalidData);
}
};
// File must be a regular file
match file.metadata() {
Ok(m) => {
if !m.is_file() {
log::warn!("file is not a regular file");
return Err(Error::InvalidData);
}
}
Err(err) => {
log::warn!("failed to stat file: {err}");
return Err(Error::InvalidData);
}
}
if offset > 0
&& let Err(err) = file.seek(SeekFrom::Start(u64::from(offset)))
{
log::warn!("failed to seek to offset {offset}: {err}");
return Err(Error::InvalidData);
}
// Read the file
let limit = if size > 0 {
(size as usize).min(MAX_SIZE)
} else {
MAX_SIZE
};
let mut buf = Vec::new();
if let Err(err) = file.take(limit as u64).read_to_end(&mut buf) {
log::warn!("failed to read temporary file: {err}");
return Err(Error::InvalidData);
}
debug_assert_eq!(self.data.len(), 0);
self.data = buf;
Ok(())
}
/// Adds a chunk of data to the image. Use this if the image
/// is coming in chunks (the "m" parameter in the protocol).
pub fn add_data(&mut self, data: &[u8]) -> Result<(), Error> {
// If no data, skip
if data.is_empty() {
return Ok(());
}
// If our data would get too big, return an error
if self.data.len() + data.len() > MAX_SIZE {
log::warn!("image data too large max_size={MAX_SIZE}");
return Err(Error::InvalidData);
}
self.data.extend_from_slice(data);
Ok(())
}
/// Complete the chunked image, returning a completed image.
pub fn complete(&mut self) -> Result<Image, Error> {
// Decompress the data if it is compressed.
self.decompress()?;
// Decode the png if we have to
if self.image.format == TransmissionFormat::Png {
self.decode_png()?;
}
let img = &self.image;
// Validate our dimensions.
if img.width == 0 || img.height == 0 {
return Err(Error::DimensionsRequired);
}
if img.width > MAX_DIMENSION || img.height > MAX_DIMENSION {
return Err(Error::DimensionsTooLarge);
}
// Data length must be what we expect
let bpp = usize::from(img.format.bpp());
let expected_len = img.width as usize * img.height as usize * bpp;
let actual_len = self.data.len();
if actual_len != expected_len {
log::warn!(
"unexpected length image id={} width={} height={} bpp={} expected_len={} actual_len={}",
img.id,
img.width,
img.height,
bpp,
expected_len,
actual_len
);
return Err(Error::InvalidData);
}
// Set our time
self.image.transmit_time = Instant::now();
// Everything looks good, copy the image data over.
let mut result = std::mem::take(&mut self.image);
result.data = std::mem::take(&mut self.data);
Ok(result)
}
/// Decompress the data in-place.
fn decompress(&mut self) -> Result<(), Error> {
match self.image.compression {
TransmissionCompression::None => Ok(()),
TransmissionCompression::ZlibDeflate => self.decompress_zlib(),
}
}
fn decompress_zlib(&mut self) -> Result<(), Error> {
let decoder = flate2::read::ZlibDecoder::new(self.data.as_slice());
let mut list = Vec::new();
// Read one byte past the limit so that an overlong stream is an
// error (Zig: `.limited(max_size)` + StreamTooLong).
if let Err(err) = decoder.take(MAX_SIZE as u64 + 1).read_to_end(&mut list) {
log::warn!("failed to read decompressed data: {err}");
return Err(Error::DecompressionFailed);
}
if list.len() > MAX_SIZE {
log::warn!("failed to read decompressed data: StreamTooLong");
return Err(Error::DecompressionFailed);
}
self.data = list;
// Make sure we note that our image is no longer compressed
self.image.compression = TransmissionCompression::None;
Ok(())
}
/// Decode the data as PNG. This will also update the image dimensions.
fn decode_png(&mut self) -> Result<(), Error> {
debug_assert_eq!(self.image.format, TransmissionFormat::Png);
let (width, height, data) = decode_png_rgba(&self.data)?;
if data.len() > MAX_SIZE {
log::warn!(
"png image too large size={} max_size={MAX_SIZE}",
data.len()
);
return Err(Error::InvalidData);
}
// Replace our data
self.data = data;
// Store updated image dimensions
self.image.width = width;
self.image.height = height;
self.image.format = TransmissionFormat::Rgba;
Ok(())
}
}
/// Decode a PNG into 8-bit non-premultiplied RGBA (what wuffs produces).
fn decode_png_rgba(data: &[u8]) -> Result<(u32, u32, Vec<u8>), Error> {
let mut decoder = png::Decoder::new(Cursor::new(data));
decoder.set_limits(png::Limits { bytes: MAX_SIZE });
decoder.set_transformations(png::Transformations::EXPAND | png::Transformations::STRIP_16);
let mut reader = decoder.read_info().map_err(|_| Error::InvalidData)?;
let size = reader.output_buffer_size().ok_or(Error::InvalidData)?;
if size > MAX_SIZE {
return Err(Error::InvalidData);
}
let mut buf = vec![0u8; size];
let info = reader
.next_frame(&mut buf)
.map_err(|_| Error::InvalidData)?;
if info.bit_depth != png::BitDepth::Eight {
return Err(Error::InvalidData);
}
let px = (info.width as usize)
.checked_mul(info.height as usize)
.ok_or(Error::InvalidData)?;
let src = &buf[..info.buffer_size()];
let mut out = Vec::with_capacity(px * 4);
match info.color_type {
png::ColorType::Rgba => out.extend_from_slice(src),
png::ColorType::Rgb => {
for c in src.as_chunks::<3>().0 {
out.extend_from_slice(&[c[0], c[1], c[2], 255]);
}
}
png::ColorType::Grayscale => {
for &g in src {
out.extend_from_slice(&[g, g, g, 255]);
}
}
png::ColorType::GrayscaleAlpha => {
for c in src.as_chunks::<2>().0 {
out.extend_from_slice(&[c[0], c[0], c[0], c[1]]);
}
}
png::ColorType::Indexed => return Err(Error::InvalidData),
}
if out.len() != px * 4 {
return Err(Error::InvalidData);
}
Ok((info.width, info.height, out))
}
fn contains_subslice(hay: &[u8], needle: &[u8]) -> bool {
hay.windows(needle.len()).any(|w| w == needle)
}
/// Returns true if path appears to be in a temporary directory.
/// Copies logic from Kitty.
fn is_path_in_temp_dir(path: &[u8]) -> bool {
if path.starts_with(b"/tmp") {
return true;
}
if path.starts_with(b"/dev/shm") {
return true;
}
{
let dir = alloc_tmp_dir();
if path.starts_with(dir.as_bytes()) {
return true;
}
// The temporary dir is sometimes a symlink. On macOS for
// example /tmp is /private/var/...
if let Ok(real_dir) = std::fs::canonicalize(&dir)
&& path.starts_with(real_dir.as_os_str().as_bytes())
{
return true;
}
}
false
}
/// `os/file.zig allocTmpDir` on POSIX (TMPDIR, TMP, then /tmp).
fn alloc_tmp_dir() -> String {
for key in ["TMPDIR", "TMP"] {
if let Some(v) = std::env::var_os(key) {
return v.to_string_lossy().into_owned();
}
}
"/tmp".to_owned()
}
/// Image represents a single fully loaded image.
#[derive(Debug, Clone)]
pub struct Image {
pub id: u32,
pub number: u32,
pub width: u32,
pub height: u32,
pub format: TransmissionFormat,
pub compression: TransmissionCompression,
pub data: Vec<u8>,
pub transmit_time: Instant,
/// Set this to true if this image was loaded by a command that
/// doesn't specify an ID or number, since such commands should
/// not be responded to, even though we do currently give them
/// IDs in the public range (which is bad!).
pub implicit_id: bool,
}
impl Default for Image {
fn default() -> Self {
Image {
id: 0,
number: 0,
width: 0,
height: 0,
format: TransmissionFormat::Rgb,
compression: TransmissionCompression::None,
data: Vec::new(),
transmit_time: Instant::now(),
implicit_id: false,
}
}
}
impl Image {
/// Mostly for logging
pub fn without_data(&self) -> Image {
let mut copy = self.clone();
copy.data = Vec::new();
copy
}
}
/// The rect taken up by some image placement, in grid cells. This will
/// be rounded up to the nearest grid cell since we can't place images
/// in partial grid cells.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Rect {
pub top_left: Pin,
pub bottom_right: Pin,
}
#[cfg(test)]
mod tests {
use super::*;
use ghostty_vt::kitty::graphics_command::{Control, Transmission};
const RGB_NONE_20X15: &[u8] =
include_bytes!("testdata/image-rgb-none-20x15-2147483647-raw.data");
const RGB_ZLIB_128X96: &[u8] =
include_bytes!("testdata/image-rgb-zlib_deflate-128x96-2147483647-raw.data");
const PNG_50X76: &[u8] = include_bytes!("testdata/image-png-none-50x76-2147483647-raw.data");
fn cmd(t: Transmission, data: &[u8]) -> Command {
Command {
control: Control::Transmit(t),
quiet: Quiet::No,
data: data.to_vec(),
}
}
/// Unique temporary directory removed on drop.
struct TmpDir(std::path::PathBuf);
impl TmpDir {
fn new(name: &str) -> TmpDir {
let p =
std::env::temp_dir().join(format!("ghostty-kitty-{}-{name}", std::process::id()));
std::fs::create_dir_all(&p).unwrap();
TmpDir(p)
}
fn write(&self, name: &str, data: &[u8]) -> String {
let p = self.0.join(name);
std::fs::write(&p, data).unwrap();
std::fs::canonicalize(p)
.unwrap()
.to_str()
.unwrap()
.to_owned()
}
}
impl Drop for TmpDir {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.0);
}
}
// This specifically tests we ALLOW invalid RGB data because Kitty
// documents that this should work.
#[test]
fn image_load_with_invalid_rgb_data() {
// <ESC>_Gi=31,s=1,v=1,a=q,t=d,f=24;AAAA<ESC>\
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
width: 1,
height: 1,
image_id: 31,
..Transmission::default()
},
b"AAAA",
);
let _loading = LoadingImage::init(&c).unwrap();
}
#[test]
fn image_load_with_image_too_wide() {
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
width: MAX_DIMENSION + 1,
height: 1,
image_id: 31,
..Transmission::default()
},
b"AAAA",
);
let mut loading = LoadingImage::init(&c).unwrap();
assert_eq!(loading.complete().unwrap_err(), Error::DimensionsTooLarge);
}
#[test]
fn image_load_with_image_too_tall() {
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
height: MAX_DIMENSION + 1,
width: 1,
image_id: 31,
..Transmission::default()
},
b"AAAA",
);
let mut loading = LoadingImage::init(&c).unwrap();
assert_eq!(loading.complete().unwrap_err(), Error::DimensionsTooLarge);
}
#[test]
fn image_load_rgb_zlib_compressed_direct() {
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::Direct,
compression: TransmissionCompression::ZlibDeflate,
height: 96,
width: 128,
image_id: 31,
..Transmission::default()
},
RGB_ZLIB_128X96,
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
// should be decompressed
assert_eq!(img.compression, TransmissionCompression::None);
assert_eq!(img.data.len(), 128 * 96 * 3);
}
#[test]
fn image_load_rgb_not_compressed_direct() {
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::Direct,
compression: TransmissionCompression::None,
width: 20,
height: 15,
image_id: 31,
..Transmission::default()
},
RGB_NONE_20X15,
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
}
#[test]
fn image_load_rgb_zlib_compressed_direct_chunked() {
let data = RGB_ZLIB_128X96;
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::Direct,
compression: TransmissionCompression::ZlibDeflate,
height: 96,
width: 128,
image_id: 31,
more_chunks: true,
..Transmission::default()
},
&data[0..1024],
);
let mut loading = LoadingImage::init(&c).unwrap();
// Read our remaining chunks
for chunk in data[1024..].chunks(1024) {
loading.add_data(chunk).unwrap();
}
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
}
#[test]
fn image_load_rgb_zlib_compressed_direct_chunked_with_zero_initial_chunk() {
let data = RGB_ZLIB_128X96;
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::Direct,
compression: TransmissionCompression::ZlibDeflate,
height: 96,
width: 128,
image_id: 31,
more_chunks: true,
..Transmission::default()
},
b"",
);
let mut loading = LoadingImage::init(&c).unwrap();
for chunk in data.chunks(1024) {
loading.add_data(chunk).unwrap();
}
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
}
#[test]
fn image_load_temporary_file_without_correct_path() {
let tmp = TmpDir::new("nocorrectpath");
let path = tmp.write("image.data", RGB_NONE_20X15);
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::TemporaryFile,
compression: TransmissionCompression::None,
width: 20,
height: 15,
image_id: 31,
..Transmission::default()
},
path.as_bytes(),
);
assert_eq!(
LoadingImage::init(&c).unwrap_err(),
Error::TemporaryFileNotNamedCorrectly
);
// Temporary file should still be there
assert!(Path::new(&path).exists());
}
#[test]
fn image_load_rgb_not_compressed_temporary_file() {
let tmp = TmpDir::new("tempfile");
let path = tmp.write("tty-graphics-protocol-image.data", RGB_NONE_20X15);
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::TemporaryFile,
compression: TransmissionCompression::None,
width: 20,
height: 15,
image_id: 31,
..Transmission::default()
},
path.as_bytes(),
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
// Temporary file should be gone
assert!(!Path::new(&path).exists());
}
#[test]
fn image_load_rgb_not_compressed_regular_file() {
let tmp = TmpDir::new("regularfile");
let path = tmp.write("image.data", RGB_NONE_20X15);
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::File,
compression: TransmissionCompression::None,
width: 20,
height: 15,
image_id: 31,
..Transmission::default()
},
path.as_bytes(),
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
assert!(Path::new(&path).exists());
}
#[test]
fn image_load_png_not_compressed_regular_file() {
let tmp = TmpDir::new("pngfile");
let path = tmp.write("tty-graphics-protocol-image.data", PNG_50X76);
let c = cmd(
Transmission {
format: TransmissionFormat::Png,
medium: TransmissionMedium::File,
compression: TransmissionCompression::None,
width: 0,
height: 0,
image_id: 31,
..Transmission::default()
},
path.as_bytes(),
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
assert_eq!(img.compression, TransmissionCompression::None);
assert_eq!(img.format, TransmissionFormat::Rgba);
assert_eq!((img.width, img.height), (50, 76));
assert_eq!(img.data.len(), 50 * 76 * 4);
assert!(Path::new(&path).exists());
}
#[test]
fn image_load_rejects_proc_and_nul_paths() {
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::File,
width: 1,
height: 1,
..Transmission::default()
},
b"/proc/self/status",
);
assert_eq!(LoadingImage::init(&c).unwrap_err(), Error::InvalidData);
let c = cmd(
Transmission {
medium: TransmissionMedium::File,
..Transmission::default()
},
b"/tmp/a\0b",
);
assert_eq!(LoadingImage::init(&c).unwrap_err(), Error::InvalidData);
}
#[test]
fn image_load_file_offset_and_size() {
let tmp = TmpDir::new("offsetsize");
let mut data = vec![0xAAu8; 5];
data.extend_from_slice(&[1, 2, 3, 4, 5, 6]);
data.extend_from_slice(&[0xBB; 7]);
let path = tmp.write("image.data", &data);
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::File,
width: 2,
height: 1,
offset: 5,
size: 6,
..Transmission::default()
},
path.as_bytes(),
);
let mut loading = LoadingImage::init(&c).unwrap();
assert_eq!(loading.data, vec![1, 2, 3, 4, 5, 6]);
assert!(loading.complete().is_ok());
}
#[test]
fn image_load_shared_memory() {
use rustix::fs::Mode;
use rustix::shm;
use std::io::Write;
let name = format!("/ghostty-kitty-test-{}", std::process::id());
let fd = shm::open(
name.as_str(),
shm::OFlags::CREATE | shm::OFlags::EXCL | shm::OFlags::RDWR,
Mode::RUSR | Mode::WUSR,
)
.unwrap();
let mut f = std::fs::File::from(fd);
f.write_all(&[7u8; 20 * 15 * 3]).unwrap();
drop(f);
let c = cmd(
Transmission {
format: TransmissionFormat::Rgb,
medium: TransmissionMedium::SharedMemory,
width: 20,
height: 15,
..Transmission::default()
},
name.as_bytes(),
);
let mut loading = LoadingImage::init(&c).unwrap();
let img = loading.complete().unwrap();
assert_eq!(img.data.len(), 900);
assert!(img.data.iter().all(|&b| b == 7));
// The segment is unlinked after loading.
assert!(shm::open(name.as_str(), shm::OFlags::RDONLY, Mode::empty()).is_err());
}
}
@@ -0,0 +1,45 @@
//! Kitty graphics render placement (`ZIG/src/terminal/kitty/graphics_render.zig`).
use crate::page_list::Pin;
/// A render placement is a way to position a Kitty graphics image onto
/// the screen. It is broken down into the fields that make it easier to
/// position the image using a renderer.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Placement {
/// The top-left corner of the image in grid coordinates.
pub top_left: Pin,
/// The offset in pixels from the top-left corner of the grid cell.
pub offset_x: u32,
pub offset_y: u32,
/// The source rectangle of the image to render. This doesn't have to
/// match the size the destination size and the renderer is expected
/// to scale the image to fit the destination size.
pub source_x: u32,
pub source_y: u32,
pub source_width: u32,
pub source_height: u32,
/// The final width/height of the image in pixels.
pub dest_width: u32,
pub dest_height: u32,
}
impl Placement {
/// A placement at `top_left` with all other fields zero.
pub fn new(top_left: Pin) -> Self {
Placement {
top_left,
offset_x: 0,
offset_y: 0,
source_x: 0,
source_y: 0,
source_width: 0,
source_height: 0,
dest_width: 0,
dest_height: 0,
}
}
}
@@ -0,0 +1,1374 @@
//! Kitty graphics image + placement storage
//! (`ZIG/src/terminal/kitty/graphics_storage.zig`).
//!
//! # Deviations from Zig
//!
//! * Zig's `ImageStorage` methods take a `*Screen`/`*Terminal`. In Rust the
//! storage lives *inside* the `Screen` (`screen.kitty_images`), so taking
//! `&mut Screen` would alias. The methods instead take the
//! [`PageList`] (`&mut screen.pages`, disjoint field borrow) and, where the
//! Zig code reads terminal state (`t.cols`, `t.width_px`, cursor, ...), a
//! small [`TerminalInfo`] value.
//! * `add_image`, `add_placement` and `set_limit` also take the `PageList`
//! so that placements that are evicted / replaced have their tracked pin
//! released (the Zig code leaks those tracked pins).
//! * `Image` lookups return references instead of copies.
//! * Placements are stored in a `BTreeMap` so iteration order is
//! deterministic.
use std::collections::{BTreeMap, HashMap};
use ghostty_vt::kitty::graphics_command::Delete;
use super::graphics_image::{Image, LoadingImage, Rect};
use crate::page_list::{Overflow, PageList, TrackedPin};
use crate::point::{Coordinate, Point};
use crate::size::CellCountInt;
/// The subset of terminal state that the storage needs to read.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TerminalInfo {
/// `terminal.cols` / `terminal.rows`.
pub cols: CellCountInt,
pub rows: CellCountInt,
/// `terminal.width_px` / `terminal.height_px`.
pub width_px: u32,
pub height_px: u32,
/// `terminal.screens.active.cursor.x` / `.y`.
pub cursor_x: CellCountInt,
pub cursor_y: CellCountInt,
}
impl TerminalInfo {
/// Snapshot the relevant state of a terminal.
pub fn from_terminal(t: &crate::terminal::Terminal) -> Self {
let cursor = &t.screens.active().cursor;
TerminalInfo {
cols: t.cols,
rows: t.rows,
width_px: t.width_px,
height_px: t.height_px,
cursor_x: cursor.x,
cursor_y: cursor.y,
}
}
}
/// Errors from storage operations.
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("out of memory")]
OutOfMemory,
}
/// The tag of a placement ID.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub enum PlacementIdTag {
Internal,
External,
}
/// A placement ID with its tag (`packed struct { tag, id }` in Zig).
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PlacementId {
pub tag: PlacementIdTag,
pub id: u32,
}
/// Every placement is uniquely identified by the image ID and the
/// placement ID. If an image ID isn't specified it is assumed to be 0.
/// Likewise, if a placement ID isn't specified it is assumed to be 0.
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct PlacementKey {
pub image_id: u32,
pub placement_id: PlacementId,
}
impl PlacementKey {
pub fn external(image_id: u32, id: u32) -> Self {
PlacementKey {
image_id,
placement_id: PlacementId {
tag: PlacementIdTag::External,
id,
},
}
}
pub fn internal(image_id: u32, id: u32) -> Self {
PlacementKey {
image_id,
placement_id: PlacementId {
tag: PlacementIdTag::Internal,
id,
},
}
}
}
/// Where a placement lives.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Location {
/// Exactly placed on a screen pin (tracked in the screen's PageList).
Pin(TrackedPin),
/// Virtual placement (U=1) for unicode placeholders.
Virtual,
}
/// The pixel size of a placement.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct CalculatedSize {
pub width: u32,
pub height: u32,
}
/// The size of a placement in grid cells.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct GridSize {
pub cols: u32,
pub rows: u32,
}
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Placement {
/// The location where this placement should be drawn.
pub location: Location,
/// Offset of the x/y from the top-left of the cell.
pub x_offset: u32,
pub y_offset: u32,
/// Source rectangle for the image to pull from
pub source_x: u32,
pub source_y: u32,
pub source_width: u32,
pub source_height: u32,
/// The columns/rows this image occupies.
pub columns: u32,
pub rows: u32,
/// The z-index for this placement.
pub z: i32,
}
impl Placement {
pub fn new(location: Location) -> Self {
Placement {
location,
x_offset: 0,
y_offset: 0,
source_x: 0,
source_y: 0,
source_width: 0,
source_height: 0,
columns: 0,
rows: 0,
z: 0,
}
}
/// Release resources (the tracked pin) held by this placement.
pub fn deinit(&self, pages: &mut PageList) {
match self.location {
Location::Pin(p) => pages.untrack_pin(p),
Location::Virtual => {}
}
}
/// Calculates the size of this placement's image in pixels,
/// taking in to account the specified rows and columns.
pub fn calculated_size(&self, image: &Image, t: &TerminalInfo) -> CalculatedSize {
// Height / width of the image in px.
let width = if self.source_width > 0 {
self.source_width
} else {
image.width
};
let height = if self.source_height > 0 {
self.source_height
} else {
image.height
};
// If we don't have any specified cols or rows then the placement
// should be the native size of the image, and doesn't need to be
// re-scaled.
if self.columns == 0 && self.rows == 0 {
return CalculatedSize { width, height };
}
// We calculate the size of a cell so that we can multiply
// it by the specified cols/rows to get the correct px size.
//
// We assume that the width is divided evenly by the column
// count and the height by the row count, because it should be.
let cell_width: u32 = t.width_px / u32::from(t.cols);
let cell_height: u32 = t.height_px / u32::from(t.rows);
let width_f64 = f64::from(width);
let height_f64 = f64::from(height);
// If we have a specified cols AND rows then we calculate
// the width and height from them directly, we don't need
// to adjust for aspect ratio.
if self.columns > 0 && self.rows > 0 {
return CalculatedSize {
width: cell_width * self.columns,
height: cell_height * self.rows,
};
}
// Either the columns or the rows were specified, but not both,
// so we need to calculate the other one based on the aspect ratio.
// If only the columns were specified, we determine
// the height of the image based on the aspect ratio.
if self.columns > 0 {
let aspect = height_f64 / width_f64;
let calc_width: u32 = cell_width * self.columns;
let calc_height = (f64::from(calc_width) * aspect).round() as u32;
return CalculatedSize {
width: calc_width,
height: calc_height,
};
}
// Otherwise, only the rows were specified, so we
// determine the width based on the aspect ratio.
let aspect = width_f64 / height_f64;
let calc_height: u32 = cell_height * self.rows;
let calc_width = (f64::from(calc_height) * aspect).round() as u32;
CalculatedSize {
width: calc_width,
height: calc_height,
}
}
/// Returns the size in grid cells that this placement takes up.
pub fn grid_size(&self, image: &Image, t: &TerminalInfo) -> GridSize {
// If we have a specified columns and rows then this is trivial.
if self.columns > 0 && self.rows > 0 {
return GridSize {
cols: self.columns,
rows: self.rows,
};
}
// Otherwise we calculate the pixel size, divide by
// cell size, and round up to the nearest integer.
let calc_size = self.calculated_size(image, t);
// NOTE: `div_ceil` can only fail if the cell size is 0, in such a
// case it seems safe to return 0 for this.
GridSize {
cols: div_ceil(
calc_size.width + self.x_offset,
t.width_px / u32::from(t.cols),
),
rows: div_ceil(
calc_size.height + self.y_offset,
t.height_px / u32::from(t.rows),
),
}
}
/// Returns a selection of the entire rectangle this placement
/// occupies within the screen. This can return None if the placement
/// doesn't have an associated rect (i.e. a virtual placement).
pub fn rect(&self, image: &Image, t: &TerminalInfo, pages: &PageList) -> Option<Rect> {
let grid_size = self.grid_size(image, t);
let handle = match self.location {
Location::Pin(p) => p,
Location::Virtual => return None,
};
let pin = pages.tracked_pin(handle)?;
// NOTE: Zig does `grid_size.rows - 1` which would underflow for a
// zero-row grid; we saturate instead.
let mut br = match pin.down_overflow(pages, grid_size.rows.saturating_sub(1) as usize)? {
Overflow::Offset(v) => v,
Overflow::Overflow { end, .. } => end,
};
br.x = std::cmp::min(
// We need to sub one here because the x value is
// one width already. So if the image is width "1"
// then we add zero to X because X itself is width 1.
u32::from(pin.x) + grid_size.cols.saturating_sub(1),
u32::from(t.cols) - 1,
) as CellCountInt;
Some(Rect {
top_left: pin,
bottom_right: br,
})
}
}
fn div_ceil(a: u32, b: u32) -> u32 {
if b == 0 { 0 } else { a.div_ceil(b) }
}
/// An image storage is associated with a terminal screen (i.e. main
/// screen, alt screen) and contains all the transmitted images and
/// placements.
#[derive(Debug)]
pub struct ImageStorage {
/// Dirty is set to true if placements or images change. This is
/// purely informational for the renderer and doesn't affect the
/// correctness of the program. The renderer must set this to false
/// if it cares about this value.
pub dirty: bool,
/// This is the next automatically assigned image ID. We start mid-way
/// through the u32 range to avoid collisions with buggy programs.
/// TODO: This isn't good enough, it's perfectly legal for programs
/// to use IDs in the latter half of the range and collisions
/// are not gracefully handled.
pub next_image_id: u32,
/// This is the next automatically assigned placement ID. This is never
/// user-facing so we can start at 0. This is 32-bits because we use
/// the same space for external placement IDs. We can start at zero
/// because any number is valid.
pub next_internal_placement_id: u32,
/// The set of images that are currently known.
pub images: HashMap<u32, Image>,
/// The set of placements for loaded images.
pub placements: BTreeMap<PlacementKey, Placement>,
/// Non-null if there is an in-progress loading image.
pub loading: Option<Box<LoadingImage>>,
/// The total bytes of image data that have been loaded and the limit.
/// If the limit is reached, the oldest images will be evicted to make
/// space. Unused images take priority.
pub total_bytes: usize,
pub total_limit: usize,
}
impl Default for ImageStorage {
fn default() -> Self {
ImageStorage {
dirty: false,
next_image_id: 2147483647,
next_internal_placement_id: 0,
images: HashMap::new(),
placements: BTreeMap::new(),
loading: None,
total_bytes: 0,
total_limit: 320 * 1000 * 1000, // 320MB
}
}
}
impl ImageStorage {
/// Release all images/placements. Placements' tracked pins are
/// released in `pages` (the screen's page list).
pub fn deinit(&mut self, pages: &mut PageList) {
self.loading = None;
self.images.clear();
self.clear_placements(pages);
}
/// Kitty image protocol is enabled if we have a non-zero limit.
pub fn enabled(&self) -> bool {
self.total_limit != 0
}
/// Sets the limit in bytes for the total amount of image data that
/// can be loaded. If this limit is lower, this will do an eviction
/// if necessary. If the value is zero, then Kitty image protocol will
/// be disabled.
pub fn set_limit(&mut self, pages: &mut PageList, limit: usize) -> Result<(), Error> {
// Special case disabling by quickly deleting all
if limit == 0 {
self.deinit(pages);
*self = ImageStorage::default();
}
// If we re lowering our limit, check if we need to evict.
if limit < self.total_bytes {
let req_bytes = self.total_bytes - limit;
log::info!("evicting images to lower limit, evicting={req_bytes}");
if !self.evict_image(pages, req_bytes) {
log::warn!("failed to evict enough images for required bytes");
}
}
self.total_limit = limit;
Ok(())
}
/// Add an already-loaded image to the storage. This will automatically
/// free any existing image with the same ID.
pub fn add_image(&mut self, pages: &mut PageList, img: Image) -> Result<(), Error> {
// If the image itself is over the limit, then error immediately
if img.data.len() > self.total_limit {
return Err(Error::OutOfMemory);
}
// If this would put us over the limit, then evict.
let total_bytes = self.total_bytes + img.data.len();
if total_bytes > self.total_limit {
let req_bytes = total_bytes - self.total_limit;
log::info!("evicting images to make space for {req_bytes} bytes");
if !self.evict_image(pages, req_bytes) {
log::warn!("failed to evict enough images for required bytes");
return Err(Error::OutOfMemory);
}
}
log::debug!("addImage image={:?}", img.without_data());
// Write our new image
let len = img.data.len();
if let Some(old) = self.images.insert(img.id, img) {
self.total_bytes -= old.data.len();
}
self.total_bytes += len;
self.dirty = true;
Ok(())
}
/// Add a placement for a given image. The caller must verify in advance
/// the image exists.
pub fn add_placement(
&mut self,
pages: &mut PageList,
image_id: u32,
placement_id: u32,
p: Placement,
) -> Result<(), Error> {
debug_assert!(self.images.contains_key(&image_id));
log::debug!("placement image_id={image_id} placement_id={placement_id} placement={p:?}");
// The important piece here is that the placement ID needs to
// be marked internal if it is zero. This allows multiple placements
// to be added for the same image. If it is non-zero, then it is
// an external placement ID and we can only have one placement
// per (image id, placement id) pair.
let key = if placement_id == 0 {
let id = self.next_internal_placement_id;
self.next_internal_placement_id = id.wrapping_add(1);
PlacementKey::internal(image_id, id)
} else {
PlacementKey::external(image_id, placement_id)
};
// (Zig leaks the old tracked pin if the key already exists.)
if let Some(old) = self.placements.insert(key, p) {
old.deinit(pages);
}
self.dirty = true;
Ok(())
}
fn clear_placements(&mut self, pages: &mut PageList) {
for p in self.placements.values() {
p.deinit(pages);
}
self.placements.clear();
}
/// Get an image by its ID. If the image doesn't exist, None is returned.
pub fn image_by_id(&self, image_id: u32) -> Option<&Image> {
self.images.get(&image_id)
}
/// Get an image by its number. If the image doesn't exist, return None.
pub fn image_by_number(&self, image_number: u32) -> Option<&Image> {
let mut newest: Option<&Image> = None;
for img in self.images.values() {
if img.number == image_number
&& newest.is_none_or(|n| img.transmit_time > n.transmit_time)
{
newest = Some(img);
}
}
newest
}
/// Remove (and release) the placement with the given key.
fn remove_placement(&mut self, pages: &mut PageList, key: &PlacementKey) {
if let Some(p) = self.placements.remove(key) {
p.deinit(pages);
}
}
/// Delete placements, images.
pub fn delete(&mut self, pages: &mut PageList, t: &TerminalInfo, cmd: Delete) {
match cmd {
Delete::All(delete_images) => {
let keys: Vec<PlacementKey> = self
.placements
.iter()
.filter(|(_, p)| matches!(p.location, Location::Pin(_)))
.map(|(k, _)| *k)
.collect();
for key in keys {
self.remove_placement(pages, &key);
if delete_images {
self.delete_if_unused(key.image_id);
}
}
if delete_images {
let ids: Vec<u32> = self.images.keys().copied().collect();
for id in ids {
self.delete_if_unused(id);
}
}
self.dirty = true;
}
Delete::Id {
delete,
image_id,
placement_id,
} => self.delete_by_id(pages, image_id, placement_id, delete),
Delete::Newest {
delete,
image_number,
placement_id,
} => {
let Some(img_id) = self.image_by_number(image_number).map(|i| i.id) else {
return;
};
self.delete_by_id(pages, img_id, placement_id, delete);
}
Delete::IntersectCursor(delete_images) => {
self.delete_intersecting(
pages,
t,
Point::Active(Coordinate {
x: t.cursor_x,
y: u32::from(t.cursor_y),
}),
delete_images,
None,
);
}
Delete::IntersectCell { delete, x, y } => {
if x == 0 || y == 0 {
log::warn!("delete intersect cell coords must be at least 1");
return;
}
let (Some(x), Some(y)) = (cast_cell(x - 1), cast_cell(y - 1)) else {
return;
};
self.delete_intersecting(
pages,
t,
Point::Active(Coordinate { x, y: u32::from(y) }),
delete,
None,
);
}
Delete::IntersectCellZ { delete, x, y, z } => {
if x == 0 || y == 0 {
log::warn!("delete intersect cell coords must be at least 1");
return;
}
let (Some(x), Some(y)) = (cast_cell(x - 1), cast_cell(y - 1)) else {
return;
};
self.delete_intersecting(
pages,
t,
Point::Active(Coordinate { x, y: u32::from(y) }),
delete,
Some(z),
);
}
Delete::Column { delete, x } => {
if x == 0 {
log::warn!("delete column must be greater than zero");
return;
}
let x = x - 1;
let keys: Vec<PlacementKey> = self.placements.keys().copied().collect();
for key in keys {
let Some(img) = self.images.get(&key.image_id) else {
continue;
};
let Some(rect) = self.placements[&key].rect(img, t, pages) else {
continue;
};
if u32::from(rect.top_left.x) <= x && u32::from(rect.bottom_right.x) >= x {
let img_id = img.id;
self.remove_placement(pages, &key);
if delete {
self.delete_if_unused(img_id);
}
}
}
// Mark dirty to force redraw
self.dirty = true;
}
Delete::Row { delete, y } => {
if y == 0 {
log::warn!("delete row must be greater than zero");
return;
}
// v.y is in active coords so we want to convert it to a pin
// so we can compare by page offsets.
let Some(y) = cast_cell(y - 1) else {
return;
};
let Some(target_pin) = pages.pin(Point::Active(Coordinate {
x: 0,
y: u32::from(y),
})) else {
return;
};
let keys: Vec<PlacementKey> = self.placements.keys().copied().collect();
for key in keys {
let Some(img) = self.images.get(&key.image_id) else {
continue;
};
let Some(rect) = self.placements[&key].rect(img, t, pages) else {
continue;
};
// We need to copy our pin to ensure we are at least at
// the top-left x.
let mut target_pin_copy = target_pin;
target_pin_copy.x = rect.top_left.x;
if target_pin_copy.is_between(pages, rect.top_left, rect.bottom_right) {
let img_id = img.id;
self.remove_placement(pages, &key);
if delete {
self.delete_if_unused(img_id);
}
}
}
// Mark dirty to force redraw
self.dirty = true;
}
Delete::Z { delete, z } => {
let keys: Vec<PlacementKey> = self
.placements
.iter()
// Virtual placeholders cannot delete by z according
// to the spec.
.filter(|(_, p)| matches!(p.location, Location::Pin(_)) && p.z == z)
.map(|(k, _)| *k)
.collect();
for key in keys {
self.remove_placement(pages, &key);
if delete {
self.delete_if_unused(key.image_id);
}
}
// Mark dirty to force redraw
self.dirty = true;
}
Delete::Range {
delete,
first,
last,
} => {
if first == 0 || last == 0 {
log::warn!("delete range values must be greater than zero");
return;
}
if first > last {
log::warn!(
"delete range 'x' ({first}) must be less than or equal to 'y' ({last})"
);
return;
}
let keys: Vec<PlacementKey> = self.placements.keys().copied().collect();
for key in keys {
// NOTE: this is `or` (not `and`) in the Zig source, which
// makes it match every placement. Preserved faithfully.
if key.image_id >= first || key.image_id <= last {
self.remove_placement(pages, &key);
if delete {
self.delete_if_unused(key.image_id);
}
}
}
// Mark dirty to force redraw
self.dirty = true;
}
// We don't support animation frames yet so they are successfully
// deleted!
Delete::AnimationFrames(_) => {}
}
}
fn delete_by_id(
&mut self,
pages: &mut PageList,
image_id: u32,
placement_id: u32,
delete_unused: bool,
) {
// If no placement, we delete all placements with the ID
if placement_id == 0 {
let keys: Vec<PlacementKey> = self
.placements
.keys()
.filter(|k| k.image_id == image_id)
.copied()
.collect();
for key in keys {
self.remove_placement(pages, &key);
}
} else {
self.remove_placement(pages, &PlacementKey::external(image_id, placement_id));
}
// If this is specified, then we also delete the image
// if it is no longer in use.
if delete_unused {
self.delete_if_unused(image_id);
}
// Mark dirty to force redraw
self.dirty = true;
}
/// Delete an image if it is unused.
fn delete_if_unused(&mut self, image_id: u32) {
if self.placements.keys().any(|k| k.image_id == image_id) {
return;
}
// If we get here, we can delete the image.
if let Some(img) = self.images.remove(&image_id) {
self.total_bytes -= img.data.len();
}
}
/// Deletes all placements intersecting a screen point. `filter_z`
/// restricts deletion to placements with that z-index.
fn delete_intersecting(
&mut self,
pages: &mut PageList,
t: &TerminalInfo,
p: Point,
delete_unused: bool,
filter_z: Option<i32>,
) {
// Convert our target point to a pin for comparison.
let Some(target_pin) = pages.pin(p) else {
return;
};
let keys: Vec<PlacementKey> = self.placements.keys().copied().collect();
for key in keys {
let Some(img) = self.images.get(&key.image_id) else {
continue;
};
let placement = self.placements[&key];
let Some(rect) = placement.rect(img, t, pages) else {
continue;
};
if target_pin.is_between(pages, rect.top_left, rect.bottom_right) {
if let Some(z) = filter_z
&& placement.z != z
{
continue;
}
let img_id = img.id;
self.remove_placement(pages, &key);
if delete_unused {
self.delete_if_unused(img_id);
}
}
}
// Mark dirty to force redraw
self.dirty = true;
}
/// Evict image to make space. This will evict the oldest image,
/// prioritizing unused images first, as recommended by the published
/// Kitty spec.
///
/// This will evict as many images as necessary to make space for
/// req bytes.
fn evict_image(&mut self, pages: &mut PageList, req: usize) -> bool {
debug_assert!(req <= self.total_limit);
let mut candidates: Vec<Candidate> = self
.images
.values()
.map(|img| Candidate {
id: img.id,
time: img.transmit_time,
used: self.placements.keys().any(|k| k.image_id == img.id),
})
.collect();
// Sort: unused first, then oldest first, then lowest id.
candidates.sort_unstable_by(|lhs, rhs| {
lhs.used
.cmp(&rhs.used)
.then(lhs.time.cmp(&rhs.time))
.then(lhs.id.cmp(&rhs.id))
});
// They're in order of best to evict.
let mut evicted: usize = 0;
for c in candidates {
// Delete all the placements for this image and the image.
let keys: Vec<PlacementKey> = self
.placements
.keys()
.filter(|k| k.image_id == c.id)
.copied()
.collect();
for key in keys {
self.remove_placement(pages, &key);
}
if let Some(img) = self.images.remove(&c.id) {
log::info!("evicting image id={} bytes={}", c.id, img.data.len());
evicted += img.data.len();
self.total_bytes -= img.data.len();
if evicted > req {
return true;
}
}
}
false
}
}
struct Candidate {
id: u32,
time: std::time::Instant,
used: bool,
}
fn cast_cell(v: u32) -> Option<CellCountInt> {
CellCountInt::try_from(v).ok()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::terminal::{Options, Terminal};
fn term(cols: u16, rows: u16) -> Terminal {
Terminal::new(Options::new(cols, rows)).unwrap()
}
fn pages(t: &mut Terminal) -> &mut PageList {
&mut t.screens.active_mut().pages
}
fn tracked(t: &Terminal) -> usize {
t.screens.active().pages.count_tracked_pins()
}
fn track_pin(t: &mut Terminal, pt: Coordinate) -> Location {
let pages = pages(t);
let p = pages.pin(Point::Active(pt)).unwrap();
Location::Pin(pages.track_pin(p).unwrap())
}
fn img(id: u32, w: u32, h: u32) -> Image {
Image {
id,
width: w,
height: h,
..Image::default()
}
}
fn coord(x: u16, y: u32) -> Coordinate {
Coordinate { x, y }
}
fn add_p(s: &mut ImageStorage, t: &mut Terminal, image: u32, pid: u32, c: Coordinate) {
let loc = track_pin(t, c);
s.add_placement(pages(t), image, pid, Placement::new(loc))
.unwrap();
}
fn add_i(s: &mut ImageStorage, t: &mut Terminal, i: Image) {
s.add_image(pages(t), i).unwrap();
}
fn del(s: &mut ImageStorage, t: &mut Terminal, cmd: Delete) {
let info = TerminalInfo::from_terminal(t);
s.delete(pages(t), &info, cmd);
}
#[test]
fn storage_add_placement_with_zero_placement_id() {
let mut t = term(100, 100);
t.width_px = 100;
t.height_px = 100;
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 50, 50));
add_i(&mut s, &mut t, img(2, 25, 25));
add_p(&mut s, &mut t, 1, 0, coord(25, 25));
add_p(&mut s, &mut t, 1, 0, coord(25, 25));
assert_eq!(s.placements.len(), 2);
assert_eq!(s.images.len(), 2);
// verify the placement is what we expect
assert!(s.placements.contains_key(&PlacementKey::internal(1, 0)));
assert!(s.placements.contains_key(&PlacementKey::internal(1, 1)));
s.deinit(pages(&mut t));
}
fn setup3() -> (Terminal, ImageStorage, usize) {
let mut t = term(3, 3);
let tr = tracked(&t);
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 0, 0));
add_i(&mut s, &mut t, img(2, 0, 0));
add_i(&mut s, &mut t, img(3, 0, 0));
add_p(&mut s, &mut t, 1, 1, coord(1, 1));
add_p(&mut s, &mut t, 2, 1, coord(1, 1));
(t, s, tr)
}
#[test]
fn storage_delete_all_placements_and_images() {
let (mut t, mut s, tr) = setup3();
s.dirty = false;
del(&mut s, &mut t, Delete::All(true));
assert!(s.dirty);
assert_eq!(s.images.len(), 0);
assert_eq!(s.placements.len(), 0);
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_delete_all_placements_and_images_preserves_limit() {
let mut t = term(3, 3);
let tr = tracked(&t);
let mut s = ImageStorage {
total_limit: 5000,
..ImageStorage::default()
};
add_i(&mut s, &mut t, img(1, 0, 0));
add_i(&mut s, &mut t, img(2, 0, 0));
add_i(&mut s, &mut t, img(3, 0, 0));
add_p(&mut s, &mut t, 1, 1, coord(1, 1));
add_p(&mut s, &mut t, 2, 1, coord(1, 1));
s.dirty = false;
del(&mut s, &mut t, Delete::All(true));
assert!(s.dirty);
assert_eq!(s.images.len(), 0);
assert_eq!(s.placements.len(), 0);
assert_eq!(s.total_limit, 5000);
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_delete_all_placements() {
let (mut t, mut s, tr) = setup3();
s.dirty = false;
del(&mut s, &mut t, Delete::All(false));
assert!(s.dirty);
assert_eq!(s.placements.len(), 0);
assert_eq!(s.images.len(), 3);
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_delete_all_placements_by_image_id() {
let (mut t, mut s, tr) = setup3();
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Id {
delete: false,
image_id: 2,
placement_id: 0,
},
);
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 3);
assert_eq!(tr + 1, tracked(&t));
}
#[test]
fn storage_delete_all_placements_by_image_id_and_unused_images() {
let (mut t, mut s, tr) = setup3();
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Id {
delete: true,
image_id: 2,
placement_id: 0,
},
);
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 2);
assert_eq!(tr + 1, tracked(&t));
}
#[test]
fn storage_delete_placement_by_specific_id() {
let mut t = term(3, 3);
let tr = tracked(&t);
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 0, 0));
add_i(&mut s, &mut t, img(2, 0, 0));
add_i(&mut s, &mut t, img(3, 0, 0));
add_p(&mut s, &mut t, 1, 1, coord(1, 1));
add_p(&mut s, &mut t, 1, 2, coord(1, 1));
add_p(&mut s, &mut t, 2, 1, coord(1, 1));
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Id {
delete: true,
image_id: 1,
placement_id: 2,
},
);
assert!(s.dirty);
assert_eq!(s.placements.len(), 2);
assert_eq!(s.images.len(), 3);
assert_eq!(tr + 2, tracked(&t));
}
fn setup100() -> (Terminal, ImageStorage, usize) {
let mut t = term(100, 100);
t.width_px = 100;
t.height_px = 100;
let tr = tracked(&t);
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 50, 50));
add_i(&mut s, &mut t, img(2, 25, 25));
add_p(&mut s, &mut t, 1, 1, coord(0, 0));
add_p(&mut s, &mut t, 1, 2, coord(25, 25));
(t, s, tr)
}
#[test]
fn storage_delete_intersecting_cursor() {
let (mut t, mut s, tr) = setup100();
t.screens.active_mut().cursor_absolute(12, 12);
s.dirty = false;
del(&mut s, &mut t, Delete::IntersectCursor(false));
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 2);
assert_eq!(tr + 1, tracked(&t));
assert!(s.placements.contains_key(&PlacementKey::external(1, 2)));
}
#[test]
fn storage_delete_intersecting_cursor_plus_unused() {
let (mut t, mut s, tr) = setup100();
t.screens.active_mut().cursor_absolute(12, 12);
s.dirty = false;
del(&mut s, &mut t, Delete::IntersectCursor(true));
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 2);
assert_eq!(tr + 1, tracked(&t));
assert!(s.placements.contains_key(&PlacementKey::external(1, 2)));
}
#[test]
fn storage_delete_intersecting_cursor_hits_multiple() {
let (mut t, mut s, tr) = setup100();
t.screens.active_mut().cursor_absolute(26, 26);
s.dirty = false;
del(&mut s, &mut t, Delete::IntersectCursor(true));
assert!(s.dirty);
assert_eq!(s.placements.len(), 0);
assert_eq!(s.images.len(), 1);
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_delete_by_column() {
let (mut t, mut s, tr) = setup100();
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Column {
delete: false,
x: 60,
},
);
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 2);
assert_eq!(tr + 1, tracked(&t));
assert!(s.placements.contains_key(&PlacementKey::external(1, 1)));
}
#[test]
fn storage_delete_by_column_1x1() {
let mut t = term(100, 100);
t.width_px = 100;
t.height_px = 100;
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 1, 1));
add_p(&mut s, &mut t, 1, 1, coord(0, 0));
add_p(&mut s, &mut t, 1, 2, coord(1, 0));
add_p(&mut s, &mut t, 1, 3, coord(2, 0));
del(
&mut s,
&mut t,
Delete::Column {
delete: false,
x: 2,
},
);
assert_eq!(s.placements.len(), 2);
assert_eq!(s.images.len(), 1);
assert!(s.placements.contains_key(&PlacementKey::external(1, 1)));
assert!(s.placements.contains_key(&PlacementKey::external(1, 3)));
}
#[test]
fn storage_delete_by_row() {
let (mut t, mut s, tr) = setup100();
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Row {
delete: false,
y: 60,
},
);
assert!(s.dirty);
assert_eq!(s.placements.len(), 1);
assert_eq!(s.images.len(), 2);
assert_eq!(tr + 1, tracked(&t));
assert!(s.placements.contains_key(&PlacementKey::external(1, 1)));
}
#[test]
fn storage_delete_by_row_1x1() {
let mut t = term(100, 100);
t.width_px = 100;
t.height_px = 100;
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 1, 1));
add_p(&mut s, &mut t, 1, 1, coord(0, 0));
add_p(&mut s, &mut t, 1, 2, coord(0, 1));
add_p(&mut s, &mut t, 1, 3, coord(0, 2));
del(
&mut s,
&mut t,
Delete::Row {
delete: false,
y: 2,
},
);
assert_eq!(s.placements.len(), 2);
assert_eq!(s.images.len(), 1);
assert!(s.placements.contains_key(&PlacementKey::external(1, 1)));
assert!(s.placements.contains_key(&PlacementKey::external(1, 3)));
}
fn range_test(delete: bool, first: u32, last: u32, images_left: usize) {
let (mut t, mut s, tr) = setup3();
assert_eq!(s.images.len(), 3);
assert_eq!(s.placements.len(), 2);
s.dirty = false;
del(
&mut s,
&mut t,
Delete::Range {
delete,
first,
last,
},
);
assert!(s.dirty);
assert_eq!(s.images.len(), images_left);
assert_eq!(s.placements.len(), 0);
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_delete_images_by_range_1() {
range_test(false, 1, 2, 3);
}
#[test]
fn storage_delete_images_by_range_2() {
range_test(true, 1, 2, 1);
}
#[test]
fn storage_delete_images_by_range_3() {
range_test(false, 1, 1, 3);
}
#[test]
fn storage_delete_images_by_range_4() {
range_test(true, 1, 1, 1);
}
#[test]
fn storage_aspect_ratio_calculation_when_only_columns_or_rows_specified() {
let mut t = term(100, 100);
t.width_px = 1000; // 10 px per col
t.height_px = 2000; // 20 px per row
let info = TerminalInfo::from_terminal(&t);
// Case 1: Only columns specified
{
let image = img(1, 16, 9);
let mut placement = Placement::new(Location::Virtual);
placement.columns = 10;
placement.rows = 0;
// Image is 16x9, set to a width of 10 columns, at 10px per column
// that's 100px width. 100px * (9 / 16) = 56.25, which should round
// to a height of 56px.
let calc = placement.calculated_size(&image, &info);
assert_eq!(calc.width, 100);
assert_eq!(calc.height, 56);
}
// Case 2: Only rows specified
{
let image = img(2, 16, 9);
let mut placement = Placement::new(Location::Virtual);
placement.columns = 0;
placement.rows = 5;
// Image is 16x9, set to a height of 5 rows, at 20px per row that's
// 100px height. 100px * (16 / 9) = 177.77..., which should round to
// a width of 178px.
let calc = placement.calculated_size(&image, &info);
assert_eq!(calc.width, 178);
assert_eq!(calc.height, 100);
}
}
#[test]
fn storage_add_placement_replaces_and_releases_pin() {
let mut t = term(3, 3);
let tr = tracked(&t);
let mut s = ImageStorage::default();
add_i(&mut s, &mut t, img(1, 0, 0));
add_p(&mut s, &mut t, 1, 5, coord(0, 0));
add_p(&mut s, &mut t, 1, 5, coord(1, 1));
assert_eq!(s.placements.len(), 1);
assert_eq!(tr + 1, tracked(&t));
s.deinit(pages(&mut t));
assert_eq!(tr, tracked(&t));
}
#[test]
fn storage_eviction_prefers_unused_then_oldest() {
let mut t = term(3, 3);
let tr = tracked(&t);
let mut s = ImageStorage {
total_limit: 30,
..ImageStorage::default()
};
let mk = |id: u32, n: usize| Image {
id,
data: vec![0; n],
..Image::default()
};
add_i(&mut s, &mut t, mk(1, 10));
add_i(&mut s, &mut t, mk(2, 10));
add_i(&mut s, &mut t, mk(3, 10));
// Image 1 is oldest but in use; image 2 is the oldest unused.
add_p(&mut s, &mut t, 1, 1, coord(0, 0));
assert_eq!(s.total_bytes, 30);
add_i(&mut s, &mut t, mk(4, 10));
assert!(s.images.contains_key(&1));
assert!(!s.images.contains_key(&2));
// Eviction continues until strictly more than the requested bytes
// were freed (`evicted > req` in Zig), so image 3 goes too.
assert!(!s.images.contains_key(&3));
assert!(s.images.contains_key(&4));
assert_eq!(s.total_bytes, 20);
// Too large for the limit outright.
assert_eq!(
s.add_image(pages(&mut t), mk(5, 31)),
Err(Error::OutOfMemory)
);
// Lowering the limit evicts (the used image goes last) and releases
// the evicted placement's pin.
s.set_limit(pages(&mut t), 5).unwrap();
assert_eq!(s.images.len(), 0);
assert_eq!(s.placements.len(), 0);
assert_eq!(tr, tracked(&t));
// Zero disables.
s.set_limit(pages(&mut t), 0).unwrap();
assert!(!s.enabled());
}
#[test]
fn storage_image_by_number_newest() {
let mut t = term(3, 3);
let mut s = ImageStorage::default();
let a = Image {
id: 1,
number: 7,
..Image::default()
};
std::thread::sleep(std::time::Duration::from_millis(2));
let b = Image {
id: 2,
number: 7,
..Image::default()
};
add_i(&mut s, &mut t, a);
add_i(&mut s, &mut t, b);
assert_eq!(s.image_by_number(7).unwrap().id, 2);
assert!(s.image_by_number(8).is_none());
}
}
@@ -0,0 +1,1086 @@
//! This file contains various logic and data for working with the
//! Kitty graphics protocol unicode placeholder, virtual placement feature
//! (`ZIG/src/terminal/kitty/graphics_unicode.zig`).
use super::graphics_image::Image;
use super::graphics_render::Placement as RenderPlacement;
use super::graphics_storage::{ImageStorage, Location, PlacementKey};
use crate::page::Cell;
use crate::page_list::{CellSubset, Direction, PageList, Pin, RowIterator};
use crate::point::Tag;
use crate::size::CellCountInt;
use crate::style::Color;
/// Codepoint for the unicode placeholder character.
pub const PLACEHOLDER: u32 = 0x10EEEE;
/// Returns an iterator that iterates over all of the virtual placements
/// in the given pin. If `limit` is provided, the iterator will stop
/// when it reaches that pin (inclusive). If `limit` is not provided,
/// the iterator will continue until the end of the page list.
pub fn placement_iterator(list: &PageList, pin: Pin, limit: Option<Pin>) -> PlacementIterator<'_> {
let start = list
.point_from_pin(Tag::Screen, pin)
.expect("pin must be in the page list");
let end = limit.map(|l| {
list.point_from_pin(Tag::Screen, l)
.expect("limit pin must be in the page list")
});
let mut row_it = list.row_iterator(Direction::RightDown, start, end);
let row = row_it.next();
PlacementIterator { list, row_it, row }
}
/// Iterator over unicode virtual placements.
pub struct PlacementIterator<'a> {
list: &'a PageList,
row_it: RowIterator<'a>,
row: Option<Pin>,
}
impl Iterator for PlacementIterator<'_> {
type Item = Placement;
fn next(&mut self) -> Option<Placement> {
while let Some(row_pin) = self.row {
// This row flag is set on rows that have the virtual placeholder
let (row, _) = row_pin.row_and_cell(self.list)?;
if !row.kitty_virtual_placeholder() {
self.row = self.row_it.next();
continue;
}
// Our current run. A run is always only a single row. This
// assumption is built-in to our logic so if we want to change
// this later we have to redo the logic; tests should cover;
let mut run: Option<IncompletePlacement> = None;
// Iterate over our remaining cells and find one with a placeholder.
let cells = row_pin.cells(self.list, CellSubset::Right)?;
for (i, cell) in cells.iter().enumerate() {
// "row" now points to the top-left pin of the placement.
// We need this temporary state to build our incomplete
// placement.
let pin = Pin {
x: row_pin.x + i as CellCountInt,
..row_pin
};
self.row = Some(pin);
// If this cell doesn't have the placeholder, then we
// complete the run if we have it otherwise we just move
// on and keep searching.
if cell.codepoint() != PLACEHOLDER {
if let Some(prev) = run {
return Some(prev.complete());
}
continue;
}
// If we don't have a previous run, then we save this
// incomplete one, start a run, and move on.
let curr = IncompletePlacement::init(self.list, pin, *cell);
if let Some(prev) = run.as_mut() {
// If we can't append, then we complete the previous
// run and return it.
if !prev.append(&curr) {
// Note: self.row is already updated due to the
// row pin above. It points back at this same
// cell so we can continue the new placements from
// here.
return Some(prev.complete());
}
// append is mutating so if we reached this point
// then prev has been updated.
} else {
// For appending, we need to set our initial values.
let mut prev = curr;
if prev.row.is_none() {
prev.row = Some(0);
}
if prev.col.is_none() {
prev.col = Some(0);
}
run = Some(prev);
}
}
// We move to the next row no matter what
self.row = self.row_it.next();
// If we have a run, we complete it here.
if let Some(prev) = run {
return Some(prev.complete());
}
}
None
}
}
/// Errors from [`Placement::render_placement`].
#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)]
pub enum Error {
#[error("placement grid out of bounds")]
PlacementGridOutOfBounds,
#[error("placement missing placement")]
PlacementMissingPlacement,
}
/// The grid size for a virtual placement.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct Grid {
rows: u32,
columns: u32,
}
#[derive(Default)]
struct PScale {
/// The offsets are pixels from the top-left of the placement-sized
/// image in order to center the image as necessary.
x_offset: f64,
y_offset: f64,
/// The multipliers to apply to the width/height of the original
/// image size in order to reach the placement size.
x_scale: f64,
y_scale: f64,
}
struct ImgScaled {
x_offset: f64,
y_offset: f64,
width: f64,
height: f64,
}
struct Source {
x: f64,
y: f64,
width: f64,
height: f64,
}
struct Dest {
x_offset: f64,
y_offset: f64,
width: f64,
height: f64,
}
/// A virtual placement in the terminal. This can represent more than
/// one cell if the cells combine to be a run.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Placement {
/// The top-left pin of the placement. This can be used to get the
/// screen x/y.
pub pin: Pin,
/// The image ID and placement ID for this virtual placement. The
/// image ID is encoded in the fg color (plus optional a 8-bit high
/// value in the 3rd diacritic). The placement ID is encoded in the
/// underline color (optionally).
pub image_id: u32,
pub placement_id: u32,
/// Starting row/col index for the image itself. This is the "fragment"
/// of the image we want to show in this placement. This is 0-indexed.
pub col: u32,
pub row: u32,
/// The width/height in cells of this placement.
pub width: u32,
pub height: u32,
}
impl Placement {
/// Take this virtual placement and convert it to a render placement.
pub fn render_placement(
&self,
storage: &ImageStorage,
img: &Image,
cell_width: u32,
cell_height: u32,
) -> Result<RenderPlacement, Error> {
// In this function, there is a variable naming convention to try
// to make it slightly less confusing. The prefix will tell you what
// coordinate/size space a variable lives in:
// - img_* is for the original image
// - p_* is for the final placement
// - vp_* is for the virtual placement
// Determine the grid size that this virtual placement fits into.
let p_grid = self.grid(storage, img, cell_width, cell_height)?;
// From here on out we switch to floating point math. These are
// constants that we'll reference repeatedly.
let img_width_f64 = f64::from(img.width);
let img_height_f64 = f64::from(img.height);
// Next we have to fit the source image into the grid size while preserving
// aspect ratio. We will center the image horizontally/vertically if
// necessary.
let p_scale = {
let p_rows_px = f64::from(p_grid.rows * cell_height);
let p_cols_px = f64::from(p_grid.columns * cell_width);
if img_width_f64 * p_rows_px > img_height_f64 * p_cols_px {
// Image is wider than the grid, fit width and center height
let x_scale = p_cols_px / img_width_f64.max(1.0);
let y_scale = x_scale;
let y_offset = (p_rows_px - img_height_f64 * y_scale) / 2.0;
PScale {
y_offset,
x_scale,
y_scale,
..PScale::default()
}
} else {
// Image is taller than the grid, fit height and center width
let y_scale = p_rows_px / img_height_f64.max(1.0);
let x_scale = y_scale;
let x_offset = (p_cols_px - img_width_f64 * x_scale) / 2.0;
PScale {
x_offset,
x_scale,
y_scale,
..PScale::default()
}
}
};
// Scale our original image according to the aspect ratio
// and padding calculated for p_scale.
let img_scaled = {
let x_offset = p_scale.x_offset / p_scale.x_scale;
let y_offset = p_scale.y_offset / p_scale.y_scale;
let width = img_width_f64 + (x_offset * 2.0);
let height = img_height_f64 + (y_offset * 2.0);
ImgScaled {
x_offset,
y_offset,
width,
height,
}
};
// Calculate the source rectangle for the scaled image. These
// coordinates are in the scaled image space.
let mut img_scale_source = {
// Float-converted values we already have
let vp_width = f64::from(self.width);
let vp_height = f64::from(self.height);
let vp_col = f64::from(self.col);
let vp_row = f64::from(self.row);
let p_grid_cols = f64::from(p_grid.columns);
let p_grid_rows = f64::from(p_grid.rows);
// Calculate the scaled source rectangle for the image, undoing
// the aspect ratio scaling as necessary.
let width = img_scaled.width * (vp_width / p_grid_cols);
let height = img_scaled.height * (vp_height / p_grid_rows);
let x = img_scaled.width * (vp_col / p_grid_cols);
let y = img_scaled.height * (vp_row / p_grid_rows);
Source {
x,
y,
width,
height,
}
};
// The destination rectangle. The x/y is specified by offsets from
// the top-left since that's how our RenderPlacement works.
let p_dest = {
let mut x_offset = 0.0f64;
let mut y_offset = 0.0f64;
let mut width = f64::from(self.width * cell_width);
let mut height = f64::from(self.height * cell_height);
if img_scale_source.y < img_scaled.y_offset {
// If our source rect y is within the offset area, we need to
// adjust our source rect and destination since the source texture
// doesn't actually have the offset area blank.
let offset = img_scaled.y_offset - img_scale_source.y;
img_scale_source.height -= offset;
y_offset = offset;
height -= offset * p_scale.y_scale;
img_scale_source.y = 0.0;
// If our height is greater than our original height,
// bring it back down. This addresses the case where the top
// and bottom offsets are both used.
if img_scale_source.height > img_height_f64 {
img_scale_source.height = img_height_f64;
height = img_height_f64 * p_scale.y_scale;
}
} else if img_scale_source.y + img_scale_source.height
> img_scaled.height - img_scaled.y_offset
{
// if our y is in our bottom offset area, we need to shorten the
// source to fit in the cell.
img_scale_source.y -= img_scaled.y_offset;
img_scale_source.height =
img_scaled.height - img_scaled.y_offset - img_scale_source.y;
img_scale_source.height -= img_scaled.y_offset;
height = img_scale_source.height * p_scale.y_scale;
} else {
img_scale_source.y -= img_scaled.y_offset;
}
if img_scale_source.x < img_scaled.x_offset {
// If our source rect x is within the offset area, we need to
// adjust our source rect and destination since the source texture
// doesn't actually have the offset area blank.
let offset = img_scaled.x_offset - img_scale_source.x;
img_scale_source.width -= offset;
x_offset = offset;
width -= offset * p_scale.x_scale;
img_scale_source.x = 0.0;
// If our width is greater than our original width,
// bring it back down. This addresses the case where the left
// and right offsets are both used.
if img_scale_source.width > img_width_f64 {
img_scale_source.width = img_width_f64;
width = img_width_f64 * p_scale.x_scale;
}
} else if img_scale_source.x + img_scale_source.width
> img_scaled.width - img_scaled.x_offset
{
// if our x is in our right offset area, we need to shorten the
// source to fit in the cell.
img_scale_source.x -= img_scaled.x_offset;
img_scale_source.width =
img_scaled.width - img_scaled.x_offset - img_scale_source.x;
img_scale_source.width -= img_scaled.x_offset;
width = img_scale_source.width * p_scale.x_scale;
} else {
img_scale_source.x -= img_scaled.x_offset;
}
// If our modified source width/height is less than zero then
// we render nothing because it means we're rendering outside
// of the visible image.
if img_scale_source.width <= 0.0 || img_scale_source.height <= 0.0 {
return Ok(RenderPlacement::new(self.pin));
}
Dest {
x_offset: x_offset * p_scale.x_scale,
y_offset: y_offset * p_scale.y_scale,
width,
height,
}
};
Ok(RenderPlacement {
top_left: self.pin,
offset_x: p_dest.x_offset.round() as u32,
offset_y: p_dest.y_offset.round() as u32,
source_x: img_scale_source.x.round() as u32,
source_y: img_scale_source.y.round() as u32,
source_width: img_scale_source.width.round() as u32,
source_height: img_scale_source.height.round() as u32,
dest_width: p_dest.width.round() as u32,
dest_height: p_dest.height.round() as u32,
})
}
// Calculate the grid size for the placement. For virtual placements,
// we use the requested row/cols. If either isn't specified, we choose
// the best size based on the image size to fit the entire image in its
// original size.
//
// This part of the code does NOT do preserve any aspect ratios. Its
// dumbly fitting the image into the grid size -- possibly user specified.
fn grid(
&self,
storage: &ImageStorage,
image: &Image,
cell_width: u32,
cell_height: u32,
) -> Result<Grid, Error> {
// Get the placement. If an ID is specified we look for the exact one.
// If no ID, then we find the first virtual placement for this image.
let placement = if self.placement_id > 0 {
storage
.placements
.get(&PlacementKey::external(self.image_id, self.placement_id))
.ok_or(Error::PlacementMissingPlacement)?
} else {
storage
.placements
.iter()
.find(|(k, p)| k.image_id == self.image_id && p.location == Location::Virtual)
.map(|(_, p)| p)
.ok_or(Error::PlacementMissingPlacement)?
};
// Use requested rows/columns if specified
// For unspecified rows/columns, calculate based on the image size.
let mut rows = placement.rows;
let mut columns = placement.columns;
if rows == 0 {
rows = image.height.div_ceil(cell_height);
}
if columns == 0 {
columns = image.width.div_ceil(cell_width);
}
Ok(Grid {
rows: CellCountInt::try_from(rows)
.map_err(|_| Error::PlacementGridOutOfBounds)?
.into(),
columns: CellCountInt::try_from(columns)
.map_err(|_| Error::PlacementGridOutOfBounds)?
.into(),
})
}
}
/// IncompletePlacement is the placement information present in a single
/// cell. It is "incomplete" because the specification allows for missing
/// diacritics and so on that continue from previous valid placements.
#[derive(Debug, Clone, Copy)]
struct IncompletePlacement {
/// The pin of the cell that created this incomplete placement.
pin: Pin,
/// Lower 24 bits of the image ID. This is specified in the fg color
/// and is always required.
image_id_low: u32,
/// Higher 8 bits of the image ID specified using the 3rd diacritic.
/// This is optional.
image_id_high: Option<u8>,
/// Placement ID is optionally specified in the underline color.
placement_id: Option<u32>,
/// The row/col index for the image. These are 0-indexed. These
/// are specified using diacritics. The row is first and the col
/// is second. Both are optional. If not specified, they can continue
/// a previous placement under certain conditions.
row: Option<u32>,
col: Option<u32>,
/// The run width so far in cells.
width: u32,
}
impl IncompletePlacement {
/// Parse the incomplete placement information from a row (pin at the
/// cell) and cell.
///
/// The cell could be derived from the pin but in our usage we already
/// have the cell and we don't want to waste cycles recomputing it.
fn init(list: &PageList, pin: Pin, cell: Cell) -> IncompletePlacement {
debug_assert_eq!(cell.codepoint(), PLACEHOLDER);
let style = pin.style(list).unwrap_or_default();
let mut result = IncompletePlacement {
pin,
image_id_low: color_to_id(style.fg_color),
placement_id: {
let id = color_to_id(style.underline_color);
if id != 0 { Some(id) } else { None }
},
image_id_high: None,
row: None,
col: None,
width: 1,
};
// Try to decode all our diacritics. Any invalid diacritics are
// treated as if they don't exist. This isn't explicitly specified
// at the time of writing this but it appears to be how Kitty behaves.
let cps: &[u32] = pin.grapheme(list).unwrap_or(&[]);
if !cps.is_empty() {
result.row = get_index(cps[0]).or_else(|| {
log::warn!(
"virtual placement with invalid row diacritic cp={:X}",
cps[0]
);
None
});
if cps.len() > 1 {
result.col = get_index(cps[1]).or_else(|| {
log::warn!(
"virtual placement with invalid col diacritic cp={:X}",
cps[1]
);
None
});
if cps.len() > 2 {
let high_ = get_index(cps[2]).or_else(|| {
log::warn!(
"virtual placement with invalid high diacritic cp={:X}",
cps[2]
);
None
});
if let Some(high) = high_ {
result.image_id_high = u8::try_from(high).ok().or_else(|| {
log::warn!(
"virtual placement with invalid high diacritic cp={:X} value={}",
cps[2],
high
);
None
});
}
// Any additional diacritics are ignored.
}
}
}
result
}
/// Append this incomplete placement to an existing placement to
/// create a run. This returns true if the placements are compatible
/// and were combined. If this returns false, the other placement is
/// unchanged.
fn append(&mut self, other: &IncompletePlacement) -> bool {
if !self.can_append(other) {
return false;
}
self.width += 1;
true
}
fn can_append(&self, other: &IncompletePlacement) -> bool {
// Converted from Kitty's logic, don't @ me.
self.image_id_low == other.image_id_low
&& self.placement_id == other.placement_id
&& (other.row.is_none() || other.row == self.row)
&& (other.col.is_none() || other.col == self.col.map(|c| c + self.width))
&& (other.image_id_high.is_none() || other.image_id_high == self.image_id_high)
}
/// Complete the incomplete placement to create a full placement.
/// This creates a new placement that isn't continuous with any previous
/// placements.
///
/// The pin is the pin of the cell that created this incomplete placement.
fn complete(&self) -> Placement {
Placement {
pin: self.pin,
image_id: {
let low = self.image_id_low;
let high = u32::from(self.image_id_high.unwrap_or(0));
low | (high << 24)
},
placement_id: self.placement_id.unwrap_or(0),
col: self.col.unwrap_or(0),
row: self.row.unwrap_or(0),
width: self.width,
height: 1,
}
}
}
/// Convert a style color to a Kitty image protocol ID. This works by
/// taking the 24 most significant bits of the color, which lets it work
/// for both palette and rgb-based colors.
fn color_to_id(c: Color) -> u32 {
match c {
Color::None => 0,
Color::Palette(v) => u32::from(v),
Color::Rgb(rgb) => (u32::from(rgb.r) << 16) | (u32::from(rgb.g) << 8) | u32::from(rgb.b),
}
}
/// Get the row/col index for a diacritic codepoint. These are 0-indexed.
fn get_index(cp: u32) -> Option<u32> {
DIACRITICS.binary_search(&cp).ok().map(|i| i as u32)
}
/// These are the diacritics used with the Kitty graphics protocol
/// Unicode placement feature to specify the row/column for placement.
/// The index into the array determines the value.
///
/// This is derived from:
/// https://sw.kovidgoyal.net/kitty/_downloads/f0a0de9ec8d9ff4456206db8e0814937/rowcolumn-diacritics.txt
const DIACRITICS: &[u32] = &[
0x0305, 0x030D, 0x030E, 0x0310, 0x0312, 0x033D, 0x033E, 0x033F, 0x0346, 0x034A, 0x034B, 0x034C,
0x0350, 0x0351, 0x0352, 0x0357, 0x035B, 0x0363, 0x0364, 0x0365, 0x0366, 0x0367, 0x0368, 0x0369,
0x036A, 0x036B, 0x036C, 0x036D, 0x036E, 0x036F, 0x0483, 0x0484, 0x0485, 0x0486, 0x0487, 0x0592,
0x0593, 0x0594, 0x0595, 0x0597, 0x0598, 0x0599, 0x059C, 0x059D, 0x059E, 0x059F, 0x05A0, 0x05A1,
0x05A8, 0x05A9, 0x05AB, 0x05AC, 0x05AF, 0x05C4, 0x0610, 0x0611, 0x0612, 0x0613, 0x0614, 0x0615,
0x0616, 0x0617, 0x0657, 0x0658, 0x0659, 0x065A, 0x065B, 0x065D, 0x065E, 0x06D6, 0x06D7, 0x06D8,
0x06D9, 0x06DA, 0x06DB, 0x06DC, 0x06DF, 0x06E0, 0x06E1, 0x06E2, 0x06E4, 0x06E7, 0x06E8, 0x06EB,
0x06EC, 0x0730, 0x0732, 0x0733, 0x0735, 0x0736, 0x073A, 0x073D, 0x073F, 0x0740, 0x0741, 0x0743,
0x0745, 0x0747, 0x0749, 0x074A, 0x07EB, 0x07EC, 0x07ED, 0x07EE, 0x07EF, 0x07F0, 0x07F1, 0x07F3,
0x0816, 0x0817, 0x0818, 0x0819, 0x081B, 0x081C, 0x081D, 0x081E, 0x081F, 0x0820, 0x0821, 0x0822,
0x0823, 0x0825, 0x0826, 0x0827, 0x0829, 0x082A, 0x082B, 0x082C, 0x082D, 0x0951, 0x0953, 0x0954,
0x0F82, 0x0F83, 0x0F86, 0x0F87, 0x135D, 0x135E, 0x135F, 0x17DD, 0x193A, 0x1A17, 0x1A75, 0x1A76,
0x1A77, 0x1A78, 0x1A79, 0x1A7A, 0x1A7B, 0x1A7C, 0x1B6B, 0x1B6D, 0x1B6E, 0x1B6F, 0x1B70, 0x1B71,
0x1B72, 0x1B73, 0x1CD0, 0x1CD1, 0x1CD2, 0x1CDA, 0x1CDB, 0x1CE0, 0x1DC0, 0x1DC1, 0x1DC3, 0x1DC4,
0x1DC5, 0x1DC6, 0x1DC7, 0x1DC8, 0x1DC9, 0x1DCB, 0x1DCC, 0x1DD1, 0x1DD2, 0x1DD3, 0x1DD4, 0x1DD5,
0x1DD6, 0x1DD7, 0x1DD8, 0x1DD9, 0x1DDA, 0x1DDB, 0x1DDC, 0x1DDD, 0x1DDE, 0x1DDF, 0x1DE0, 0x1DE1,
0x1DE2, 0x1DE3, 0x1DE4, 0x1DE5, 0x1DE6, 0x1DFE, 0x20D0, 0x20D1, 0x20D4, 0x20D5, 0x20D6, 0x20D7,
0x20DB, 0x20DC, 0x20E1, 0x20E7, 0x20E9, 0x20F0, 0x2CEF, 0x2CF0, 0x2CF1, 0x2DE0, 0x2DE1, 0x2DE2,
0x2DE3, 0x2DE4, 0x2DE5, 0x2DE6, 0x2DE7, 0x2DE8, 0x2DE9, 0x2DEA, 0x2DEB, 0x2DEC, 0x2DED, 0x2DEE,
0x2DEF, 0x2DF0, 0x2DF1, 0x2DF2, 0x2DF3, 0x2DF4, 0x2DF5, 0x2DF6, 0x2DF7, 0x2DF8, 0x2DF9, 0x2DFA,
0x2DFB, 0x2DFC, 0x2DFD, 0x2DFE, 0x2DFF, 0xA66F, 0xA67C, 0xA67D, 0xA6F0, 0xA6F1, 0xA8E0, 0xA8E1,
0xA8E2, 0xA8E3, 0xA8E4, 0xA8E5, 0xA8E6, 0xA8E7, 0xA8E8, 0xA8E9, 0xA8EA, 0xA8EB, 0xA8EC, 0xA8ED,
0xA8EE, 0xA8EF, 0xA8F0, 0xA8F1, 0xAAB0, 0xAAB2, 0xAAB3, 0xAAB7, 0xAAB8, 0xAABE, 0xAABF, 0xAAC1,
0xFE20, 0xFE21, 0xFE22, 0xFE23, 0xFE24, 0xFE25, 0xFE26, 0x10A0F, 0x10A38, 0x1D185, 0x1D186,
0x1D187, 0x1D188, 0x1D189, 0x1D1AA, 0x1D1AB, 0x1D1AC, 0x1D1AD, 0x1D242, 0x1D243, 0x1D244,
];
#[cfg(test)]
mod tests {
use super::*;
use crate::kitty::graphics_storage::Placement as StoragePlacement;
use crate::terminal::{Options, Terminal};
use ghostty_vt::modes::Mode;
use ghostty_vt::sgr::Attribute;
fn term(cols: u16, rows: u16) -> Terminal {
let mut t = Terminal::new(Options::new(cols, rows)).unwrap();
t.modes.set(Mode::GraphemeCluster, true);
t
}
fn top_left(t: &Terminal) -> Pin {
t.screens
.active()
.pages
.get_top_left(Tag::Viewport)
.unwrap()
}
#[test]
fn unicode_diacritic_sorted() {
// diacritics must be sorted since we use a binary search.
assert!(DIACRITICS.windows(2).all(|w| w[0] < w[1]));
}
#[test]
fn unicode_diacritic() {
// Some spot checks based on Kitty behavior
assert_eq!(30, get_index(0x483).unwrap());
assert_eq!(294, get_index(0x1d242).unwrap());
}
#[test]
fn unicode_placement_none() {
let mut t = term(5, 5);
// Single cell
t.print_string("hello\nworld\n1\n2").unwrap();
// No placements
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_single_row_col() {
let mut t = term(5, 5);
// Single cell
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
let pin = top_left(&t);
// Should have exactly one placement
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_continuation_break() {
let mut t = term(10, 5);
// Two runs because it jumps cols
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{030E}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(1, p.width);
}
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(2, p.col);
assert_eq!(1, p.width);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_continuation_with_diacritics_set() {
let mut t = term(10, 5);
// Three cells. They'll continue even though they're explicit
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{030D}").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{030E}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(3, p.width);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_continuation_with_no_col() {
let mut t = term(10, 5);
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(3, p.width);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_continuation_with_no_diacritics() {
let mut t = term(10, 5);
t.print_string("\u{10EEEE}").unwrap();
t.print_string("\u{10EEEE}").unwrap();
t.print_string("\u{10EEEE}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(3, p.width);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_run_ending() {
let mut t = term(10, 5);
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{030D}").unwrap();
t.print_string("ABC").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(2, p.width);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_run_starting_in_the_middle() {
let mut t = term(10, 5);
t.print_string("ABC").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{030D}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(0, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
assert_eq!(2, p.width);
assert_eq!(3, p.pin.x);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_specifying_image_id_as_palette() {
let mut t = term(5, 5);
t.set_attribute(Attribute::Fg256(42)).unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(42, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_specifying_image_id_with_high_bits() {
let mut t = term(5, 5);
t.set_attribute(Attribute::Fg256(42)).unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{0305}\u{030E}")
.unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(33554474, p.image_id);
assert_eq!(0, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_specifying_placement_id_as_palette() {
let mut t = term(5, 5);
t.set_attribute(Attribute::Fg256(42)).unwrap();
t.set_attribute(Attribute::UnderlineColor256(21)).unwrap();
t.print_string("\u{10EEEE}\u{0305}\u{0305}").unwrap();
let pin = top_left(&t);
let mut it = placement_iterator(&t.screens.active().pages, pin, None);
{
let p = it.next().unwrap();
assert_eq!(42, p.image_id);
assert_eq!(21, p.placement_id);
assert_eq!(0, p.row);
assert_eq!(0, p.col);
}
assert!(it.next().is_none());
}
#[test]
fn unicode_placement_multiple_rows_with_limit() {
let mut t = term(5, 5);
t.print_string("\u{10EEEE}\u{0305}\u{0305}\r\n").unwrap();
t.print_string("\u{10EEEE}\u{030D}\u{0305}\r\n").unwrap();
t.print_string("\u{10EEEE}\u{030E}\u{0305}").unwrap();
let pages = &t.screens.active().pages;
let pin = top_left(&t);
let rows: Vec<u32> = placement_iterator(pages, pin, None)
.map(|p| p.row)
.collect();
assert_eq!(rows, vec![0, 1, 2]);
// Limit is inclusive.
let limit = pages.pin(crate::point::Point::Active(crate::point::Coordinate {
x: 0,
y: 1,
}));
let rows: Vec<u32> = placement_iterator(pages, pin, limit)
.map(|p| p.row)
.collect();
assert_eq!(rows, vec![0, 1]);
}
fn dog_storage(t: &mut Terminal, columns: u32, rows: u32) -> (ImageStorage, Image) {
let mut s = ImageStorage::default();
let image = Image {
id: 1,
width: 500,
height: 306,
..Image::default()
};
let pages = &mut t.screens.active_mut().pages;
s.add_image(pages, image.clone()).unwrap();
let mut p = StoragePlacement::new(Location::Virtual);
p.columns = columns;
p.rows = rows;
s.add_placement(pages, 1, 0, p).unwrap();
(s, image)
}
fn vp(t: &Terminal, row: u32, width: u32) -> Placement {
Placement {
pin: t.screens.active().cursor_pin(),
image_id: 1,
placement_id: 0,
col: 0,
row,
width,
height: 1,
}
}
// Fish:
// printf "\033_Gf=100,i=1,t=f,q=2;$(printf dog.png | base64)\033\\"
// printf "\e[38;5;1m\U10EEEE\U0305\U0305\U10EEEE\U0305\U030D\U10EEEE\U0305\U030E\U10EEEE\U0305\U0310\e[39m\n"
// printf "\e[38;5;1m\U10EEEE\U030D\U0305\U10EEEE\U030D\U030D\U10EEEE\U030D\U030E\U10EEEE\U030D\U0310\e[39m\n"
// printf "\033_Ga=p,i=1,U=1,q=2,c=4,r=2\033\\"
#[test]
fn unicode_render_placement_dog_4x2() {
let (cell_width, cell_height) = (36, 80);
let mut t = Terminal::new(Options::new(100, 100)).unwrap();
let (s, image) = dog_storage(&mut t, 4, 2);
// Row 1
{
let p = vp(&t, 0, 4);
let rp = p
.render_placement(&s, &image, cell_width, cell_height)
.unwrap();
assert_eq!(0, rp.offset_x);
assert_eq!(36, rp.offset_y);
assert_eq!(0, rp.source_x);
assert_eq!(0, rp.source_y);
assert_eq!(500, rp.source_width);
assert_eq!(153, rp.source_height);
assert_eq!(144, rp.dest_width);
assert_eq!(44, rp.dest_height);
}
// Row 2
{
let p = vp(&t, 1, 4);
let rp = p
.render_placement(&s, &image, cell_width, cell_height)
.unwrap();
assert_eq!(0, rp.offset_x);
assert_eq!(0, rp.offset_y);
assert_eq!(0, rp.source_x);
assert_eq!(153, rp.source_y);
assert_eq!(500, rp.source_width);
assert_eq!(153, rp.source_height);
assert_eq!(144, rp.dest_width);
assert_eq!(44, rp.dest_height);
}
}
// printf "\033_Ga=p,i=1,U=1,q=2,c=2,r=2\033\\"
#[test]
fn unicode_render_placement_dog_2x2_with_blank_cells() {
let (cell_width, cell_height) = (36, 80);
let mut t = Terminal::new(Options::new(100, 100)).unwrap();
let (s, image) = dog_storage(&mut t, 2, 2);
// Row 1
{
let p = vp(&t, 0, 4);
let rp = p
.render_placement(&s, &image, cell_width, cell_height)
.unwrap();
assert_eq!(0, rp.offset_x);
assert_eq!(58, rp.offset_y);
assert_eq!(0, rp.source_x);
assert_eq!(0, rp.source_y);
assert_eq!(500, rp.source_width);
assert_eq!(153, rp.source_height);
assert_eq!(72, rp.dest_width);
assert_eq!(22, rp.dest_height);
}
// Row 2
{
let p = vp(&t, 1, 4);
let rp = p
.render_placement(&s, &image, cell_width, cell_height)
.unwrap();
assert_eq!(0, rp.offset_x);
assert_eq!(0, rp.offset_y);
assert_eq!(0, rp.source_x);
assert_eq!(153, rp.source_y);
assert_eq!(500, rp.source_width);
assert_eq!(153, rp.source_height);
assert_eq!(72, rp.dest_width);
assert_eq!(22, rp.dest_height);
}
}
// printf "\033_Ga=p,i=1,U=1,q=2,c=1,r=1\033\\"
#[test]
fn unicode_render_placement_dog_1x1() {
let (cell_width, cell_height) = (36, 80);
let mut t = Terminal::new(Options::new(100, 100)).unwrap();
let (s, image) = dog_storage(&mut t, 1, 1);
let p = vp(&t, 0, 4);
let rp = p
.render_placement(&s, &image, cell_width, cell_height)
.unwrap();
assert_eq!(0, rp.offset_x);
assert_eq!(29, rp.offset_y);
assert_eq!(0, rp.source_x);
assert_eq!(0, rp.source_y);
assert_eq!(500, rp.source_width);
assert_eq!(306, rp.source_height);
assert_eq!(36, rp.dest_width);
assert_eq!(22, rp.dest_height);
}
#[test]
fn unicode_render_placement_missing_placement() {
let mut t = Terminal::new(Options::new(10, 10)).unwrap();
let s = ImageStorage::default();
let image = Image {
id: 1,
width: 5,
height: 5,
..Image::default()
};
let p = vp(&t, 0, 1);
assert_eq!(
p.render_placement(&s, &image, 10, 10).unwrap_err(),
Error::PlacementMissingPlacement
);
let _ = &mut t;
}
}
+26
View File
@@ -0,0 +1,26 @@
//! Kitty graphics protocol support (`ZIG/src/terminal/kitty.zig` and
//! `ZIG/src/terminal/kitty/graphics.zig`).
//!
//! Documentation:
//! <https://sw.kovidgoyal.net/kitty/graphics-protocol>
//!
//! Unimplemented features that are still todo (same as Ghostty):
//! - animation
//!
//! The command parser lives in `ghostty_vt::kitty::graphics_command`; the
//! types most callers need are re-exported here.
pub mod graphics_exec;
pub mod graphics_image;
pub mod graphics_render;
pub mod graphics_storage;
pub mod graphics_unicode;
pub use ghostty_vt::kitty::{color, graphics_command, key};
pub use graphics_command::{Command, Parser as CommandParser, Response};
pub use graphics_exec::{ExecTerminal, execute};
pub use graphics_image::Image;
pub use graphics_render::Placement as RenderPlacement;
pub use graphics_storage::ImageStorage;
pub use graphics_unicode as unicode;
Binary file not shown.

After

Width:  |  Height:  |  Size: 232 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 86 B

+1
View File
@@ -4,6 +4,7 @@ pub mod cursor;
pub mod formatter;
pub mod highlight;
pub mod hyperlink;
pub mod kitty;
pub mod page;
pub mod page_list;
pub mod pin_iter;
@@ -0,0 +1,70 @@
# ghostty-terminal: kitty graphics (agent K1)
## Ported (Zig -> Rust, `crates/ghostty-terminal/src/kitty/`)
| Zig | Rust | tests |
|---|---|---|
| graphics_image.zig | `graphics_image.rs` (`LoadingImage`, `Image`, `Rect`, `Error`) | 11/11 Zig + 3 extra (nul/`/proc` path rejection, file offset+size, shared memory) |
| graphics_storage.zig | `graphics_storage.rs` (`ImageStorage`, `Placement`, `PlacementKey`, `TerminalInfo`) | 19/19 Zig + 3 extra (placement replace releases pin, eviction order + limit, image_by_number newest) |
| graphics_unicode.zig | `graphics_unicode.rs` (`placement_iterator`, `Placement`, `render_placement`) | 16/16 Zig + 2 extra |
| graphics_render.zig | `graphics_render.rs` (`Placement`) | no tests in Zig |
| graphics_exec.zig | `graphics_exec.rs` (`execute`, `ExecTerminal` trait) | 8/8 Zig + 4 extra (run against a harness, see below) |
| kitty.zig / graphics.zig | `kitty/mod.rs` (re-exports; the command parser stays in `ghostty_vt::kitty::graphics_command`) | - |
`pub mod kitty;` added to `lib.rs`. Test data copied to `kitty/testdata/`.
Deps added to `ghostty-terminal`: `flate2`, `png`, `rustix` (features `shm`, `fs`), `thiserror`.
Zig tests skipped: none. (`debugDump` not ported: debug-only helper.)
## Deviations
- **Borrowing**: Zig passes `*Screen`/`*Terminal` to storage methods. Rust's `ImageStorage` is meant to live in
`Screen.kitty_images`, so methods take `&mut PageList` (disjoint field borrow `&mut screen.pages`) plus a
`TerminalInfo { cols, rows, width_px, height_px, cursor_x, cursor_y }` snapshot (`TerminalInfo::from_terminal(&Terminal)`).
`deinit(pages)`, `delete(pages, &info, cmd)`, `set_limit(pages, limit)`, `add_image(pages, img)`, `add_placement(pages, image_id, placement_id, p)`.
- `add_image`/`add_placement`/`set_limit` take the PageList only so that placements evicted or replaced release their tracked pin
(Zig leaks them in `evictImage` and when an external placement id is re-used).
- Placement location is `Location::Pin(TrackedPin)` (handle into the PageList) instead of `*Pin`; `Placement::rect(&image, &info, &pages)`.
- `imageById`/`imageByNumber` return `Option<&Image>` (no copy). Placements are a `BTreeMap` (deterministic order); images a `HashMap`.
- zlib via `flate2`, PNG via `png` crate (EXPAND|STRIP_16, then converted to 8-bit RGBA like wuffs' RGBA_NONPREMUL); shm via
`rustix::shm::open` + `File::read_exact_at` (no mmap, so no unsafe). The segment is unlinked after reading, as in Zig.
- File medium `S=` (size): data is truncated to `min(S, 400MB)`. (Zig's `appendRemaining(.limited(n))` may error when the file is longer; I chose
the Kitty-spec behavior. Zig tests only use S=0.)
- `Placement::rect` uses `saturating_sub(1)` on rows/cols (Zig would underflow for a 0-sized grid).
- Preserved Zig quirks: `delete range` uses `||` (`id >= first || id <= last`, matches everything); eviction stops only when
`evicted > req` (strictly); `set_limit(0)` resets storage to defaults (limit then set to 0); screen reset `.{ .dirty = true }` also resets total_limit to default.
- `execute(terminal, &cmd)` has no allocator; `Command` stays borrowed (the loading image copies the payload).
## graphics_exec and Terminal (TODO(integration), needs T3)
`Screen.kitty_images` is still the placeholder `KittyImages { dirty }`, so `Terminal` cannot implement `ExecTerminal` yet.
`execute` is generic over the `ExecTerminal` trait; the exec tests run against a `Harness { t: Terminal, storage: ImageStorage }`
(real `Terminal::index`/`set_cursor_pos`/pages; storage held beside it). No tests are ignored.
### Integration steps
1. `screen.rs`: replace `KittyImages` with `pub kitty_images: crate::kitty::graphics_storage::ImageStorage` (`Default` = Zig `.{}`).
- `Screen::new`: after constructing, `result.kitty_images.set_limit(&mut result.pages, opts.kitty_image_storage_limit).unwrap()`
(infallible there; `set_limit` returns `Result`). Opts default is already 320_000_000.
- `Screen::reset` (Zig `self.kitty_images.deinit(self); self.kitty_images = .{ .dirty = true }`):
`self.kitty_images.deinit(&mut self.pages); self.kitty_images = ImageStorage { dirty: true, ..Default::default() };`
- `Drop`/explicit teardown: `kitty_images.deinit(&mut pages)` must run before pages are dropped (tracked pins). `clone_screen` does not clone images.
- the existing `self.kitty_images.dirty = true` sites keep working unchanged.
2. `terminal.rs`:
- `erase_display` `.complete` and `.scrollback`... (Zig lines ~2495/2552): `let info = TerminalInfo::from_terminal(self); let s = self.screens.active_mut(); s.kitty_images.delete(&mut s.pages, &info, Delete::All(true));`
- `switch_screen_mode`/`set_alternate`: new-screen `kitty_image_storage_limit` = primary `kitty_images.total_limit`; then `new.kitty_images.dirty = true`.
- add `Terminal::kitty_graphics_command(&mut self, cmd: &Command) -> Option<Response> { kitty::graphics_exec::execute(self, cmd) }`
(Zig `Terminal.kittyGraphics`).
- `impl ExecTerminal for Terminal`:
```rust
fn kitty_parts(&mut self) -> (&mut ImageStorage, &mut PageList) { let s = self.screens.active_mut(); (&mut s.kitty_images, &mut s.pages) }
fn exec_info(&self) -> TerminalInfo { TerminalInfo::from_terminal(self) }
fn exec_cursor_pin(&self) -> Pin { self.screens.active().cursor_pin() }
fn exec_index(&mut self) -> Result<(), page::Error> { self.index() }
fn exec_set_cursor_pos(&mut self, row: usize, col: usize) { self.set_cursor_pos(row, col) }
```
- `print`: Zig sets row `kitty_virtual_placeholder` when printing `kitty::graphics_unicode::PLACEHOLDER` (already present in Rust, uses its own constant; can switch to `crate::kitty::unicode::PLACEHOLDER`).
3. `stream_readonly`/termio: APC `G` -> `ghostty_vt::apc` already yields a parsed `Command`; termio calls `terminal.kitty_graphics_command(&cmd)` and, if `Some(resp)`, writes `resp.encode(..)` to the pty.
Termio config: `screen.kitty_images.set_limit(&mut screen.pages, limit)` (Termio.zig 265/469); clear-screen binding: `Delete::All(true)` as in step 2 (Termio.zig 615).
4. Renderer (`renderer/image.zig` equivalent): iterate `storage.placements` / `storage.images`, `Placement::rect`/`grid_size`/`calculated_size`,
`graphics_unicode::placement_iterator(&pages, top, Some(bot))` + `Placement::render_placement(storage, img, cell_w, cell_h)`; reset `storage.dirty = false` after upload.
Font shaper uses `kitty::unicode::PLACEHOLDER`.
5. Once Terminal implements the trait, the exec tests' `Harness` can be replaced by `Terminal` directly.
## Quality gate (2026-10-11)
`cargo build/test/clippy -D warnings -p ghostty-terminal` pass (1243 tests incl. 66 kitty: 54 ported + 12 extra); rustfmt applied to `kitty/*.rs`.