lanspread/PLAN.md

Plan: Transactional Local Installs, Intent-Logged Recovery, Event-Gated Scans

Summary

• local/ being a directory is the install signal. The contents of local/ are the user's responsibility, not ours.
• version.ini is the download completion sentinel: a game is downloaded iff version.ini exists at the game-root level. version.ini is the last file written by every successful download, via atomic rename.
• Each game root carries a per-game JSON intent log for install operations, updated atomically before and after every state-changing FS operation. Startup recovery is a (recorded_intent × fs_state) lookup.
• Installs/updates/uninstalls run as transactions over each game root using fixed-name staging and backup directories, marked Lanspread-owned to prevent collisions with user data.
• Downloads are transactional via a single sentinel: old version.ini is renamed aside at the start, new version.ini is written atomically at the end. No intent log entry needed for downloads — the FS state is authoritative.
• Filesystem watcher replaces 15s polling. Events are gated by per-game operation locks and a per-game rescan-pending flag — no time-based debounce. A 5-minute fallback scan reconciles missed events.
• Install transaction logic moves out of src-tauri into a new lanspread-peer::install module so the peer owns the state machine end-to-end. Unpacker is injected at peer startup so the peer crate stays Tauri-free.
• A game is transferable to peers only when its ID is in the ETI catalog, version.ini is present, and no operation is active. Non-catalog game roots are ignored for peer announcements — we currently have no system to determine title/thumbnail for non-catalog games, so they're future scope.

Wire protocol

Unchanged. Existing library_rev / LibraryDelta / manifest_hash machinery already handles announcement on any state change; uninstall is just another library change. version.ini remains part of the per-game manifest.

Layout

Game-dir root

<game_dir>/
  .lanspread/                 -- peer-wide cache (ignored by watcher and scans)
  <game_id_1>/...
  <game_id_2>/...
  ...

Per game root

<game_dir>/<game_id>/
  <game_id>.eti               -- main archive
  *.eti                       -- additional archives (multi-archive games are valid)
  version.ini                 -- download sentinel. Presence = downloaded.
  local/                      -- committed install. Is-a-directory ⇒ installed.

  .version.ini.tmp            -- transient: version.ini being staged for atomic rename
  .version.ini.discarded      -- transient: previous version.ini parked aside during re-download
  .local.installing/          -- extraction staging dir (Lanspread-owned)
  .local.backup/              -- previous install retained for rollback (Lanspread-owned)
  .lanspread.json             -- per-game install intent log (atomic writes)

Fixed names; no timestamps. The intent log disambiguates "fresh" vs "stale" install-staging/backup dirs; for downloads the FS state alone is authoritative.

Ownership marker

When the peer creates .local.installing/ or .local.backup/, it drops a zero-byte .lanspread_owned file inside as soon as the directory exists.

Two ownership rules apply at recovery time:

• Intent says None: the marker is the only signal. Only .local.* dirs carrying .lanspread_owned may be touched; markerless .local.* dirs are user data — log and leave alone.
• Intent says Installing / Updating / Uninstalling: the corresponding .local.installing/ and .local.backup/ are unconditionally Lanspread-owned regardless of marker. This covers the inevitable crash window between rename(local → .local.backup) and the marker drop, and between mkdir(.local.installing) and the marker drop. The intent log already proves ownership; the marker is belt-and-suspenders for the None-intent case.

Download-related transient files (.version.ini.tmp, .version.ini.discarded) use fixed reserved names — no marker is needed because the names are reserved by Lanspread and any conflict would already be a name collision with our own scheme.

State model

downloaded = version.ini exists (regular file, at game-root level)
installed  = local/ is a directory

The two predicates are independent.

• installed && downloaded → Ready, fully shareable.
• installed && !downloaded → installed locally; peer announces in deltas but availability = LocalOnly (refuses to serve archives).
• !installed && downloaded → ready to install; UI shows pending; eti_version is surfaced from version.ini.
• !installed && !downloaded → blank slate.

is_committed_install (the predicate) requires local/ to be a directory, not merely "a path that exists." This handles symlinks, leftover regular files named local, etc.

Intent log (install operations only)

Per-game JSON at <game_root>/.lanspread.json:

{
  "schema_version": 1,
  "id": "...",
  "recorded_at": <unix_ts>,
  "state": "None" | "Installing" | "Updating" | "Uninstalling",
  "eti_version": "20251028"?,
  "manifest_hash": <u64>?
}

state: "None" means no install-side operation in flight; FS is the truth. The intent log does not track download state — that's read directly from version.ini / .version.ini.* FS signals.

Atomic write

1. Write .lanspread.json.tmp.
2. file.sync_all() (fsync the data).
3. rename(.lanspread.json.tmp, .lanspread.json).
4. fsync(parent_dir) on Unix; on Windows skip (no portable directory-fsync equivalent).

Intent transitions

• Begin operation: write intent with new state before any FS mutation.
• Commit operation: write intent with None immediately after the commit rename, before emitting events or starting cleanup.
• Cleanup (e.g., deleting .local.backup after a successful update): best-effort, after intent flush and event emit. Failure logs but doesn't fail the operation.

Startup recovery

Install recovery matrix

For each game root, read intent + inspect FS, then act:

recorded     | local/ | .local.installing | .local.backup | action
-------------|--------|-------------------|---------------|----------------------------------------
None         | -      | -                 | -             | trust FS; leave markerless .local.* alone (user data); sweep only markered .local.* orphans
Installing   | yes    | -                 | -             | commit landed, intent crash; write None
Installing   | -      | yes               | -             | sweep staging; write None
Installing   | -      | -                 | -             | crash before any staging; write None
Updating     | no     | yes               | yes           | sweep staging, restore backup; write None (old ver)
Updating     | yes    | yes (stale)       | yes (stale)   | sweep both; write None (new ver — commit landed before intent flush)
Updating     | yes    | -                 | yes           | commit landed, intent crash before cleanup; sweep backup, write None
Updating     | yes    | -                 | -             | commit + cleanup landed, intent crash; write None
Uninstalling | no     | -                 | yes           | finish: delete backup; write None
Uninstalling | no     | -                 | -             | write None
Uninstalling | yes    | -                 | -             | intent crash before any FS work; redo uninstall from step 1

Default rule: for any (intent, fs) combination not listed, write intent matching FS observation; do not mutate FS. Conservative bias — never delete unless we have positive proof of ownership (intent is Installing / Updating / Uninstalling for that ID, or the .local.* dir carries the marker).

Sideloading (user drops a game dir in via file manager): no intent log, FS-only observation. The "None" row handles it. JSON corruption or accidental deletion follows the same path — treat missing/unparseable intent as None.

Download recovery

Independent of the install intent log; pure FS sweep:

• .version.ini.tmp present → partial commit; delete.
• .version.ini.discarded present → interrupted download; delete (do not restore; .eti bytes may be partially overwritten).
• After sweeping, the version.ini presence/absence is authoritative.

Transaction primitives

All in the new lanspread-peer::install module (install) plus changes to download.rs (download). The same per-peer operation table holds Downloading | Installing | Updating | Uninstalling entries.

Download transaction

Driven by download_game_files.

1. Operation table: insert Downloading for game ID.
2. If version.ini exists in the game root, rename(version.ini, .version.ini.discarded).
3. prepare_game_storage runs over file descs filtered to exclude is_version_ini() entries — non-version.ini files are pre-allocated and overwritten in place by chunk writes as today.
4. Run parallel chunk downloads as today.
5. Version.ini chunks (small file, typically a few bytes) are routed to an in-memory buffer instead of disk during chunk receipt.
6. After all non-version.ini downloads complete and the version.ini buffer is full:
   • Write buffer to .version.ini.tmp.
   • file.sync_all().
   • rename(.version.ini.tmp, version.ini).
7. Sweep .version.ini.discarded (best-effort).
8. Operation table: remove Downloading.
9. Emit DownloadGameFilesFinished.
10. Peer auto-fires InstallGame { id } (see install transaction).

Rollback (any failure or cancel):

• Sweep .version.ini.tmp if any.
• Sweep .version.ini.discarded.
• Do not restore old version.ini. .eti bytes may be partially overwritten and would no longer match the old state.
• Operation table: remove Downloading.
• Emit DownloadGameFilesFailed.

A failed download is destructive to the previous downloaded state but leaves local/ untouched. The user can re-trigger download. This is acceptable because local/ is what the user plays from, and download is fundamentally about replacing the on-disk archive.

Install (no prior local/)

1. Write intent Installing.
2. Create .local.installing/, drop .lanspread_owned marker.
3. Extract archives into .local.installing/. Non-zero unrar exit ⇒ error ⇒ rollback.
4. rename(.local.installing, local). ← commit point.
5. Write intent None.
6. Emit success event; trigger rescan.

Rollback (any step before commit): rm -rf .local.installing; write intent None; emit failure.

Update (prior local/ present)

1. Write intent Updating.
2. rename(local, .local.backup); drop .lanspread_owned marker inside.
3. Create .local.installing/, drop marker.
4. Extract new archive into .local.installing/. Non-zero unrar exit ⇒ error ⇒ rollback.
5. rename(.local.installing, local). ← commit point.
6. Write intent None.
7. Emit success event; trigger rescan.
8. Best-effort rm -rf .local.backup; failure logged, not fatal.

Rollback: rm -rf .local.installing (best-effort); rename(.local.backup, local); write intent None; emit failure.

Uninstall

1. Write intent Uninstalling.
2. rename(local, .local.backup); drop .lanspread_owned marker.
3. rm -rf .local.backup.
4. Write intent None.
5. Emit success event; trigger rescan.

Rollback (step 3 fails): attempt rename(.local.backup, local); surface error.

unrar correctness

Non-zero exit status from unrar is an error. The None intent and the user-facing "unpack-finished" event fire only after both extraction and the promotion rename succeed.

Peer crate changes

New module crates/lanspread-peer/src/install/

mod.rs              -- public surface: install(), update(), uninstall(), recover_on_startup()
transaction.rs      -- staging/backup/commit/rollback primitives over a game root
intent.rs           -- atomic read/write of per-game intent JSON
unpack.rs           -- unrar invocation, via the injected Unpacker trait

Unpacker injection

Object-safe boxed-future trait — no async_trait dep, no async fn in trait:

use std::{future::Future, path::Path, pin::Pin};

pub type UnpackFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>;

pub trait Unpacker: Send + Sync {
    fn unpack<'a>(&'a self, archive: &'a Path, dest: &'a Path) -> UnpackFuture<'a>;
}

start_peer gains a 4th argument:

pub fn start_peer(
    game_dir: impl Into<PathBuf>,
    tx_notify_ui: UnboundedSender<PeerEvent>,
    peer_game_db: Arc<RwLock<PeerGameDB>>,
    unpacker: Arc<dyn Unpacker>,
    catalog: Arc<RwLock<HashSet<String>>>,
) -> eyre::Result<PeerRuntimeHandle>

Tauri provides the impl that wraps app_handle.shell().sidecar("unrar") and passes it in. The peer crate stays Tauri-free.

Catalog injection

The peer must know which game IDs the ETI catalog covers — only those IDs are announced to peers, only those flow through the install state machine, and only those participate in download/serve paths. The catalog set is owned by Tauri (loaded from the bundled game.db in load_bundled_game_db) and shared with the peer as Arc<RwLock<HashSet<String>>>. The RwLock leaves room for a future "catalog refresh without peer restart" path; today the set is populated once at startup before start_peer is called.

Catalog enforcement points inside the peer:

• scan_local_library and the optimized rescan: a game root whose ID is not in the catalog is skipped — no GameSummary emitted, no entry in the library index, no contribution to library_rev. The intent log and recovery still operate on it (so we don't corrupt on-disk state for an unknown ID), but it is invisible to peers and the UI.
• local_download_available and the file-stream handlers (see "Operation table" below): refuse all serve requests for non-catalog IDs.
• InstallGame / UpdateGame / UninstallGame commands for non-catalog IDs are rejected with an error; the peer logs and emits no events.

This is also why the watcher reacting to a new game root cannot announce it without consulting the catalog — events for non-catalog IDs flow through gating but never become deltas.

Download changes (crates/lanspread-peer/src/download.rs)

• prepare_game_storage: filter file_descs to exclude desc.is_version_ini() entries. Non-version.ini files are pre-allocated as today.
• download_game_files:
  • Partition game_file_descs into version.ini and non-version.ini.
  • Insert Downloading into the operation table; rename existing version.ini to .version.ini.discarded if present.
  • Run parallel chunked downloads for non-version.ini files (existing path).
  • Route version.ini chunk writes to an in-memory buffer keyed by relative path.
  • After all chunks succeed, commit each version.ini buffer atomically via .version.ini.tmp → rename.
  • Sweep .version.ini.discarded.
  • Remove Downloading from the operation table.
• download_chunk for version.ini paths: writes to memory rather than disk.
• New PeerCommand variant: InstallGame { id } — runs the install transaction over an already-downloaded game (auto-fired by the peer when a download commits; can also be fired by Tauri for explicit re-install). Plus UpdateGame { id } and UninstallGame { id }.

Operation table

A single per-peer table holds the in-progress operation for each game ID (Downloading | Installing | Updating | Uninstalling). Consulted by:

• local_download_available (manifest serve path, used by handle_get_game) — refuse GetGame for a game during any active operation, when version.ini is absent, or when the ID is not in the catalog.
• File-stream handlers (handle_file_data_request, handle_file_chunk_request in services/stream.rs) — refuse GetGameFileData and GetGameFileChunk requests when the ID has an active operation, when version.ini is absent at the game root, or when the ID is not in the catalog. A remote peer may already hold a manifest from before our operation began; these handlers must re-verify per request rather than rely on local_download_available having gated the original manifest fetch. Additional rule: local/-relative paths are rejected unconditionally — manifests never contain local/ entries, but defense-in-depth against a malicious or stale request.
• Watcher event handler — drop events for IDs with active operations.
• Scanner — preserve last-known summary for IDs with active operations; do not emit transient deltas.
• Library announcement — do not announce in LibraryDelta while an operation is active.

Aggregation and candidate selection (peer_db.rs)

The Availability flag must propagate through peer-side aggregation; today it does not, which would make LocalOnly peers count as downloadable.

• get_all_games: when aggregating GameSummary values across peers, Game.peer_count counts only peers whose advertised availability == Ready. LocalOnly peers do not contribute to peer_count and do not flip the aggregated downloaded flag.
• peers_with_game and peers_with_latest_version: filter to peers with availability == Ready before returning addresses. A peer advertising LocalOnly must never be selected as a download source — the download will fail when the remote refuses GetGame.
• get_latest_version_for_game: ignore LocalOnly entries when computing latest version. A version we can't actually transfer should not influence "is an update available?" decisions.

State model implementation

• local_games.rs switches downloaded from <id>.eti exists to version.ini exists (exact <game_root>/version.ini, regular file).
• local_download_available gates on version.ini + no active operation + catalog-known ID.
• build_game_summary reads eti_version from version.ini whenever downloaded, not only when installed. The current "Game is not downloaded" error path is removed: a game with local/ present but no version.ini returns a GameSummary with downloaded = false, installed = true, availability = LocalOnly.
• update_index_for_game no longer short-circuits on missing .eti. The "no game here" decision is now: skip iff version.ini is absent and local/ is not a directory and no *.eti is present at root level. Otherwise build a summary.
• Manifest building (scan_game_descriptions / should_skip_root_entry) continues to skip local/; local/ contents are user-owned and never enter manifests, fingerprints, or transfer paths.
• Fingerprint still tracks version.ini mtime (already there); .eti mtime is retained as a cache invalidation hint. local/ is-a-directory remains part of the fingerprint.

is_version_ini predicate

GameFileDescription::is_version_ini is tightened to an exact root-level match: the predicate returns true only when relative_path == "<game_id>/version.ini". The current ends_with("/version.ini") would misclassify nested foo/bar/version.ini as the sentinel; under the new sentinel rule that would break both download partitioning and the install state machine.

If a remote manifest contains zero or more than one root-level version.ini entry, download_game_files fails fast with a clear error — the sentinel is mandatory and must be unique.

Scanner + watcher

Workspace dependency

Add notify = "7" (or the current 7.x) to the top-level Cargo.toml [workspace.dependencies] and pull it into lanspread-peer/Cargo.toml. The crate is not currently in the dependency graph.

Watcher topology

• notify-based, non-recursive watches on game_dir root and on each game root.
• Never watch inside local/, .local.installing/, .local.backup/, or the root .lanspread/.
• Lifecycle: the watcher reconciles its watched set whenever a game root appears or disappears, and tears down + rebuilds all watches on SetGameDir.

No time-based debounce

Per-game-ID gating:

• Operation-lock present → drop event.
• Rescan running for ID → set "rescan-pending" flag; the running task re-runs once on completion.
• Else → spawn rescan for that ID.

Optimized rescan (single ID)

1. Stat version.ini (existence + mtime), check local/ is-a-directory, stat any *.eti files at root level.
2. Compare to cached fingerprint.
3. Recursive walk only on fingerprint change.

Fallback scan

Every 300s, list top-level game-root dirs and run the optimized rescan per ID.

Ignore list

In manifests, scans, and event handling: .local.*, .version.ini.*, .lanspread, .lanspread.json, .sync, .softlan_game_installed (latter two already skipped).

Watcher init failure

Non-fatal. Log a warning; the fallback scan is sufficient on its own.

Tauri / UI changes

src-tauri/src/lib.rs

• Remove direct unpack logic: unpack_game, do_unrar. Remove the unpack-spawn inside handle_download_finished — unpack now happens inside the peer's install transaction. handle_download_finished becomes a plain game-download-finished event relay (no spawn, no sidecar invocation, no backup cleanup).
• Remove the entire ___TO_BE_DELETE___{id} backup-the-whole-game-dir scheme: backup_game_folder, restore_game_folder, cleanup_backup_folder, cleanup_failed_download (download rollback now lives in the peer). The whole-game-folder backup approach is incompatible with the new model (downloads no longer touch local/, so there's nothing to back up).
• Remove update_game_installation_state's "downloaded required for installed" coupling: installed is now local/ is-a-directory, independent of downloaded.
• Provide the Unpacker impl that shells out to the unrar sidecar; pass it to start_peer. Populate the catalog set from load_bundled_game_db and pass it to start_peer too.
• Rename games_in_download: HashSet<String> to a generic in-progress operation set mirroring the peer's operation table (HashMap<String, OpKind> or similar — the variant tracks Downloading / Installing / Updating / Uninstalling so the UI can pick the right spinner).
• install_game(id) is rewritten to route by local state rather than always sending GetGame:
  • version.ini absent → send PeerCommand::GetGame { id } (fetch first; peer auto-fires InstallGame on download commit).
  • version.ini present, local/ absent → send PeerCommand::InstallGame { id } directly (no re-download).
  • version.ini present, local/ present → no-op (already installed); the UI's "Open" affordance handles launch.
• update_game(id) becomes a thin wrapper that sends PeerCommand::GetGame { id }. The fresh download triggers the peer's auto-install, which does a transactional update (rename local → .local.backup, extract, commit-rename). The old Tauri-side whole-folder backup is gone.

New Tauri command

uninstall_game(id) → forwards to PeerCommand::UninstallGame.

Frontend (src/App.tsx)

• New InstallStatus.Installing (extraction in flight) and InstallStatus.Uninstalling.
• New install lifecycle events (peer-driven, separate from download and unpack):
  • game-install-begin — intent flipped to Installing / Updating, before extraction.
  • game-install-failed — extraction or commit-rename failed, rollback complete.
  • game-uninstall-begin, game-uninstall-finished, game-uninstall-failed.
• Existing game-download-finished fires when the version.ini rename commits (which happens inside download_game_files, before the auto-fired InstallGame).
• Existing game-unpack-finished fires only on true transactional install success (peer-driven). It does NOT fire on extract failure or commit-rename failure — those route to game-install-failed.
• Uninstall affordance: secondary action (icon button or row-context menu), not a mode of the main button. Main button stays Install / Open.
• Surface eti_version for games that are downloaded but not installed.
• Render the LocalOnly availability badge for installed-but-not-downloaded games: the user can play them locally but the row should communicate that this peer can't serve archives.

ARCHITECTURE.md

Update crates/lanspread-peer/ARCHITECTURE.md:

• Replace the "short debounce (1-2 seconds)" wording with the per-ID operation-lock + rescan-pending model.
• Replace the fingerprint-cache description with the per-game intent log.
• Document the .local.installing / .local.backup / .lanspread.json / .lanspread_owned layout and the ownership-marker rule.
• Document the version.ini-as-sentinel rule and .version.ini.* reserved names.

Test plan

Unit — install transactions

• Install success, update success, update rollback on extract failure, update rollback on commit-rename failure, uninstall success, uninstall delete-failure restore.

Unit — download transaction

• Fresh download writes version.ini last via atomic rename.
• Re-download renames old version.ini to .version.ini.discarded, commits new one, sweeps discarded.
• Download interrupted between phases 1 and 5 leaves no version.ini; subsequent scan sees game as not downloaded.
• prepare_game_storage skips version.ini entries.
• Version.ini chunk routing to memory: buffer accumulates correctly across out-of-order chunk arrivals.
• Multi-.eti games: all archives downloaded, none of them used as the sentinel.

Unit — recovery

• Each row of the install matrix, including the default-rule cases.
• Download recovery: .version.ini.tmp and .version.ini.discarded swept unconditionally on startup.
• JSON missing / corrupted / wrong schema_version → fall back to FS observation.
• .local.* dir without .lanspread_owned marker → ignored, not deleted.

Unit — atomic intent write

• Simulated crash between tmp-write and rename leaves the previous intent intact.
• Schema version mismatch handled as "treat as missing."

Unit — scanner gating

• Events for an ID under an operation lock are dropped.
• Rescan-pending flag collapses bursts to ≤2 scans per ID.
• Sideloaded dir is picked up by the fallback scan and ingested with correct state.
• Non-catalog game root is observed by the scanner but produces no GameSummary, no library-index entry, and no LibraryDelta change.

Unit — serve gating

• GetGame is refused when an operation is active for the ID.
• GetGameFileData and GetGameFileChunk are refused mid-operation even when the requesting peer holds a pre-operation manifest.
• All three handlers refuse non-catalog IDs.
• All three handlers refuse when version.ini is absent (covers manually-deleted-sentinel case).
• File-chunk handler rejects any relative_path that resolves under local/.

Unit — aggregation

• get_all_games counts only Ready peers toward peer_count; a single LocalOnly peer reports peer_count = 0 for a game no other peer carries.
• peers_with_game and peers_with_latest_version exclude LocalOnly peers.
• get_latest_version_for_game ignores LocalOnly entries.

Unit — installed-only state

• A game root with local/ present and version.ini absent produces a summary with downloaded = false, installed = true, availability = LocalOnly.
• Same root, after the user copies in version.ini, flips to Ready on the next rescan.

Manual

• New game appears via file-manager mv (atomic) and cp (piecemeal).
• Game disappears via rm -rf.
• Install / update / uninstall while watcher is active.
• Failed unpack restores old install.
• 100-game fallback scan stays fast.
• Pull plug mid-download; verify on restart: game shown as not downloaded, no stale partials lingering.
• Pull plug mid-update; verify install recovery restores old version.
• Delete version.ini manually while game is installed; verify LocalOnly availability and that UI still shows installed.

Verify

• just fmt
• just clippy
• just build

Assumptions

• Uninstall removes only local/; the archives and version.ini stay so the game can be reinstalled without re-download.
• All .local.*, .version.ini.*, and .lanspread* paths are reserved namespaces. Existing user dirs with .local.* names are not touched unless they carry the .lanspread_owned marker. .version.ini.* names are reserved by convention.
• The peer process owns all install/update/uninstall and download FS mutations; users editing inside local/ is explicitly unsupported.
• A failed download is destructive to previous download state: old version.ini is not restored because .eti bytes may already be partially overwritten. local/ is preserved (downloads never touch it). The user retries the download.
• Watcher failure is non-fatal; the 5-minute fallback scan is sufficient on its own.
• Sideload recommendation: users move complete game directories into game_dir (atomic rename), rather than copying file-by-file. For cp, the fallback scan reconciles once version.ini lands.
• Non-catalog game roots are ignored because the app has no system yet to determine title and thumbnail for them. Supporting arbitrary non-ETI games is future scope.