157 lines
6.0 KiB
Markdown
157 lines
6.0 KiB
Markdown
# lanspread-peer proposed protocol and architecture
|
|
|
|
This document proposes a tighter, more fault-tolerant protocol while keeping
|
|
the current idea: mDNS discovery, QUIC transport, on-demand metadata, and
|
|
chunked file transfers.
|
|
|
|
## Goals (unchanged)
|
|
|
|
- Local LAN discovery via mDNS.
|
|
- QUIC + JSON messages for control, raw streams for file data.
|
|
- UI drives operations through `PeerCommand`, peers remain headless.
|
|
- Peers can appear/disappear at any time without data loss.
|
|
|
|
## Peer lifecycle and message flow
|
|
|
|
### 1) Startup and advertise
|
|
|
|
- Start QUIC server.
|
|
- Advertise via mDNS with TXT records:
|
|
- `peer_id` (stable ID, not tied to IP)
|
|
- `proto_ver`
|
|
- `library_rev` (monotonic local library revision)
|
|
- optional `hostname`
|
|
|
|
### 2) Discovery and handshake
|
|
|
|
When a peer is discovered:
|
|
|
|
1. Connect and send `Hello { peer_id, proto_ver, library_rev, library_digest, features }`.
|
|
2. Receive `HelloAck { peer_id, proto_ver, library_rev, library_digest, features }`.
|
|
3. If the remote `peer_id` is already known but the address changed, update it.
|
|
4. If protocol versions are incompatible, drop the peer (and keep mDNS watching).
|
|
5. If library digests match, do nothing else.
|
|
6. If digests differ:
|
|
- If we have a known `library_rev` for that peer, request `LibraryDelta`.
|
|
- Otherwise request `LibrarySnapshot`.
|
|
|
|
### 3) Steady state
|
|
|
|
- Any message updates `last_seen`.
|
|
- Pings run only when idle (or on a longer interval), not every 5 seconds.
|
|
- Library updates are pushed as deltas, debounced and coalesced.
|
|
|
|
### 4) Shutdown
|
|
|
|
- Optional `Goodbye { peer_id }` lets others remove the peer quickly.
|
|
- If a peer vanishes without goodbye, stale timeout + ping removal handle it.
|
|
- Goodbye is a hint, never required for correctness.
|
|
|
|
## Library sync protocol
|
|
|
|
### Summary and snapshot
|
|
|
|
- `LibrarySummary { library_rev, library_digest, game_count }`
|
|
- `LibrarySnapshot { library_rev, games: Vec<GameSummary> }`
|
|
|
|
### Delta updates
|
|
|
|
- `LibraryDelta { from_rev, to_rev, added, updated, removed }`
|
|
- `removed` is a list of game IDs.
|
|
- Deltas are idempotent; ignore if `to_rev` <= known rev.
|
|
|
|
### GameSummary (concept)
|
|
|
|
- `id`, `name`, `eti_version`, `size`, `downloaded`, `installed`
|
|
- `manifest_hash` (hash of file list + sizes)
|
|
- `availability` (e.g., `ready`, `downloading`, `local_only`)
|
|
|
|
## When peers broadcast their game list
|
|
|
|
- Only on changes, not on a timer.
|
|
- Use a short debounce (1-2 seconds) to coalesce bursts of filesystem events.
|
|
- Send `LibraryDelta` to known peers; send `LibrarySummary` on new connections.
|
|
|
|
## Local game scanning: fast and low cost
|
|
|
|
### Strategy
|
|
|
|
1. Maintain a persistent on-disk index (per game):
|
|
- `manifest_hash`, total size, file list (optional), and a fingerprint
|
|
(version.ini mtime, .eti mtime/size, local install dir presence).
|
|
2. Use filesystem watchers to update only changed games.
|
|
3. Keep a fallback periodic scan with a long interval (minutes) to recover from
|
|
missed events.
|
|
|
|
### Fast-path scanning
|
|
|
|
- On startup, list only top-level game directories.
|
|
- For each game, read a cheap fingerprint:
|
|
- `.eti` size + mtime
|
|
- `version.ini` mtime (if installed)
|
|
- presence of `local/` content
|
|
- If fingerprint unchanged, reuse cached size and manifest hash.
|
|
- Only run a recursive scan for new or changed games.
|
|
|
|
### Result
|
|
|
|
Most scans become O(number of game dirs), with full recursion only when needed.
|
|
|
|
## File manifests and downloads
|
|
|
|
- Keep `GetGame`/manifest requests, but keyed by `manifest_hash` so repeated
|
|
calls can be skipped when unchanged.
|
|
- Downloads remain chunked QUIC streams with the existing integrity checks.
|
|
|
|
## Fault tolerance rules
|
|
|
|
- Every peer is keyed by `peer_id`, not by IP address.
|
|
- `library_rev` is monotonic and guards against out-of-order updates.
|
|
- Any mismatch or missing delta falls back to `LibrarySnapshot`.
|
|
- Loss of goodbye is harmless; stale timeout is authoritative.
|
|
|
|
## Roadmap from current design to this one
|
|
|
|
1. Protocol updates in `lanspread-proto`:
|
|
- Define `Hello`, `HelloAck`, `LibrarySummary`, `LibrarySnapshot`,
|
|
`LibraryDelta`, and optional `Goodbye` messages.
|
|
- Thread `peer_id`, `library_rev`, and `manifest_hash` through all
|
|
library and manifest-bearing types.
|
|
- Make `HelloAck` carry the remote `library_rev` and `manifest_hash`
|
|
so the client can immediately select `LibraryDelta` vs `LibrarySnapshot`.
|
|
2. Peer identity:
|
|
- Persist a stable `peer_id` (UUID) in the peer config and inject it into
|
|
`PeerInfo` and `PeerGameDB` at startup.
|
|
- Track `peer_id -> SocketAddr` in the discovery table and update the
|
|
address on any incoming handshake or mDNS refresh.
|
|
3. Discovery handshake:
|
|
- Publish `peer_id` and `library_rev` in mDNS TXT records to avoid
|
|
immediate TCP/QUIC roundtrips when nothing changed.
|
|
- Add a lightweight handshake in `run_peer_discovery` that exchanges
|
|
`Hello`/`HelloAck` before any library sync.
|
|
- Keep a fallback path that uses `ListGames` when `Hello` is unsupported.
|
|
4. Library revisioning:
|
|
- Store a monotonic `library_rev` locally and increment only after a
|
|
successful index refresh completes.
|
|
- Apply `LibraryDelta` when `library_rev` matches; reject stale or future
|
|
revisions and request `LibrarySnapshot` instead.
|
|
- Cache the last accepted `manifest_hash` per peer to short-circuit
|
|
manifest requests when unchanged.
|
|
5. Local index + scan optimizations:
|
|
- Introduce a cached index file (e.g., `.lanspread/index.json`) that stores
|
|
per-root fingerprints and computed manifests.
|
|
- Use filesystem watchers with a debounce window to collect changes and
|
|
incrementally update the cache.
|
|
- Schedule a low-frequency full scan to reconcile missed watcher events.
|
|
6. Announce updates:
|
|
- Replace `AnnounceGames` with `LibraryDelta` broadcasts keyed by
|
|
`library_rev`.
|
|
- Send `LibrarySummary` on new connections to seed the delta flow.
|
|
7. File manifest caching:
|
|
- Store per-game `manifest_hash` and only fetch details when changed.
|
|
8. Liveness:
|
|
- Reduce ping frequency; update `last_seen` on any message.
|
|
- Add optional `Goodbye` on shutdown paths.
|
|
9. Tests:
|
|
- Delta apply/merge, rev ordering, manifest hashing, and scan cache behavior.
|