codex review and fixes

This commit is contained in:
2025-11-11 22:02:07 +01:00
parent 4f3c720e33
commit f046fac303
2 changed files with 108 additions and 189 deletions
+93 -184
View File
@@ -1,204 +1,113 @@
# lanspread-peer # lanspread-peer
A peer-to-peer networking component for the Lanspread system that enables distributed game sharing and discovery across local networks using QUIC protocol and mDNS service discovery. `lanspread-peer` is the networking runtime that lets Lanspread nodes find each
other on the local network, exchange library metadata, and transfer game files.
It is designed to run headless other crates (most notably
`lanspread-tauri-deno-ts`) embed it and drive it through a channel-based API.
## Overview ## Runtime Overview
The `lanspread-peer` crate implements a peer-to-peer networking layer that allows multiple instances of Lanspread to discover each other on the local network, share game libraries, and distribute game files efficiently. It operates as both a server (providing local games to other peers) and a client (discovering and downloading games from other peers). - `start_peer(game_dir, tx_events)` boots the asynchronous runtime in the
background and returns an `UnboundedSender<PeerCommand>` that the caller uses
for control. The function immediately forwards the supplied game directory via
`PeerCommand::SetGameDir`.
- `PeerCommand` represents the small control surface exposed to the UI layer:
`ListGames`, `GetGame`, `DownloadGameFiles`, and `SetGameDir`.
- `PeerEvent` enumerates everything the peer runtime reports back to the UI:
library snapshots, download lifecycle updates, and peer membership changes.
- `PeerGameDB` collects remote peer metadata. It aggregates discovered peers
`Game` definitions, tracks the latest ETI version per title, and keeps the
last seen list of `GameFileDescription` entries for each peer.
## Core Architecture Internally the peer runtime owns three long-lived tasks that run for the
lifetime of the process:
### Main Components 1. **Server component** (`run_server_component`) listens for QUIC connections,
advertises via mDNS, and serves `Request::ListGames`, `Request::GetGame`,
`Request::GetGameFileData`, and `Request::GetGameFileChunk` by reading from
the local game directory.
2. **Discovery loop** (`run_peer_discovery`) uses the `lanspread-mdns`
helper to discover other peers. The blocking mDNS work is executed on a
dedicated thread via `tokio::task::spawn_blocking` so that the Tokio runtime
remains responsive.
3. **Ping service** (`run_ping_service`) periodically issues QUIC ping requests
to keep peer liveness up to date and prunes stale entries from `PeerGameDB`.
1. **Peer System (`run_peer`)**: The main orchestrator that manages all peer operations `load_local_game_db` scans the configured game directory (looking for folders
2. **Server Component (`run_server_component`)**: Handles incoming connections from other peers with a `version.ini`) and hydrates a `GameDB`. That database is used to respond
3. **Peer Discovery (`run_peer_discovery`)**: Continuously discovers new peers via mDNS to incoming metadata requests (`Request::ListGames` / `Request::GetGame`).
4. **Ping Service (`run_ping_service`)**: Maintains peer connectivity through health checks
5. **Download Manager**: Handles parallel file downloads from multiple peers
### Key Data Structures ## Networking and File Transfer
- **`PeerGameDB`**: Central database managing all discovered peers and their game collections - Transport is handled by [`s2n-quic`](https://github.com/aws/s2n-quic); TLS
- **`PeerInfo`**: Contains information about a single peer including address, last seen time, and available games cert/key material is compiled in from the repository root.
- **`PeerEvent`**: Events sent to the UI layer (peer discovery, connection status, download progress) - Protocol messages are JSON-encoded structures defined in
- **`PeerCommand`**: Commands received from the UI layer (list games, download, etc.) `lanspread-proto::{Request, Response}`.
- File transfers stream raw bytes over dedicated bidirectional QUIC streams.
`peer::send_game_file_data` sends entire files, while
`peer::send_game_file_chunk` services ranged requests.
## Network Protocol ### Download Pipeline
### Communication Layer When the UI asks to download a game:
The crate uses **QUIC** (via `s2n-quic`) for all peer-to-peer communication with TLS encryption: 1. The UI first issues `PeerCommand::GetGame`. Each peer that still reports the
game is queried via `request_game_details_from_peer`, and their file
manifests are merged inside `PeerGameDB`.
2. Once the UI receives `PeerEvent::GotGameFiles`, it forwards the selected file
list back with `PeerCommand::DownloadGameFiles`.
3. `download_game_files` prepares the filesystem (creating directories and
pre-sizing files where possible), emits `PeerEvent::DownloadGameFilesBegin`,
and builds a per-peer plan (`build_peer_plans`) that round-robins file chunks
across the available peers that advertise the latest version.
4. Each plan is executed in its own task (`download_from_peer`). Chunk requests
use per-chunk QUIC streams and write into pre-created files. The chunk writer
keeps existing data intact and only truncates when we intentionally fall back
to a full file transfer, which prevents corruption when multiple peers fill
different regions of the same file.
5. Failures are accumulated and retried (up to `MAX_RETRY_COUNT`) via
`retry_failed_chunks`. If everything succeeds,
`PeerEvent::DownloadGameFilesFinished` is emitted; otherwise the UI receives
`PeerEvent::DownloadGameFilesFailed`.
- **Certificate-based security**: Uses embedded TLS certificates for secure communication ## Integration with `lanspread-tauri-deno-ts`
- **Bidirectional streams**: Each peer interaction uses separate QUIC streams
- **Connection management**: Built-in connection pooling and keep-alive mechanisms
### Message Protocol The Tauri application embeds this crate in
`crates/lanspread-tauri-deno-ts/src-tauri/src/lib.rs`:
Messages are serialized using JSON and follow the request/response pattern defined in `lanspread-proto`: - `LanSpreadState` holds onto the peer control channel, the latest aggregated
`GameDB`, per-game download state, and the user-selected game directory.
- The Tauri commands (`request_games`, `install_game`, `update_game`, and
`update_game_directory`) translate UI actions into `PeerCommand`s. In
particular, `update_game_directory` records the filesystem path, kicks off the
peer runtime on first use, and mirrors the installed/uninstalled state into
the UI-facing database.
- A background task consumes `PeerEvent`s and fans them out to the front-end via
Tauri publish/subscribe events (`games-list-updated`, `game-download-*`,
`peer-*`). Successful downloads trigger an `unrar` sidecar to unpack ETI
archives and clean up the temporary backup folders that are created when
updates begin.
- When downloads fail the Tauri layer restores the on-disk backup, keeping the
previous installation consistent even after partial transfers.
**Request Types:** ## Security & Operational Notes
- `Ping`: Health check
- `ListGames`: Request available games from a peer
- `GetGame`: Request detailed file list for a specific game
- `GetGameFileData`: Request complete file download
- `GetGameFileChunk`: Request specific file chunk for parallel downloads
**Response Types:** - All QUIC connections are TLS encrypted; the shipped certificates are suitable
- `Pong`: Health check response for local-network trust but should be rotated for production deployments.
- `ListGames(Vec<Game>)`: List of available games - Peer discovery is restricted to the local link via mDNS.
- `GetGame`: Game file descriptions - Long-running blocking mDNS calls are isolated on dedicated threads which keeps
- `GameNotFound`: Game not available the async runtime responsive even when discovery takes a long time.
- Error responses for various failure conditions - File writes are chunk-safe: partial chunk downloads now open files without
truncating existing data, avoiding the corruption that occurred previously
when multiple peers collectively filled a file.
## Peer Discovery ## Known Limitations
### mDNS Integration - `PeerGameDB` currently models the latest metadata that other peers advertise.
If the UI needs to surface titles that only exist locally, additional merging
with the locally scanned `GameDB` will be required.
- The download planner uses a simple round-robin and does not yet take per-peer
throughput or failures into account when distributing work.
The crate uses mDNS (multicast DNS) for automatic peer discovery: Refer to the source (particularly `src/lib.rs`) for the exact message shapes and
state machines.
- **Service Type**: `_lanspread._udp.local.`
- **Instance Naming**: `{hostname}-{uuid}` format to ensure uniqueness
- **Continuous Discovery**: Scans every 10 seconds for new peers
- **Automatic Advertising**: Each peer advertises its availability via mDNS
### Discovery Process
1. **mDNS Browser**: Continuously scans for `_lanspread._udp.local.` services
2. **Peer Registration**: New peers are added to `PeerGameDB`
3. **Game Synchronization**: Automatically requests game lists from newly discovered peers
4. **UI Notification**: Notifies the UI layer about peer discovery events
## Game Distribution System
### File Management
The system supports efficient game file distribution with these features:
**File Description:**
- `GameFileDescription`: Metadata for each file including path, size, and directory structure
- **Version Tracking**: Reads `version.ini` files for ETI game version management
- **Directory Structure**: Preserves complete game directory hierarchies
**Download Strategies:**
1. **Chunked Downloads**: Large files are split into 512KB chunks for parallel processing
2. **Multi-peer Downloads**: Chunks are distributed across available peers for maximum throughput
3. **Retry Mechanism**: Failed chunks are retried up to 3 times with different peers
4. **Integrity Verification**: File integrity is verified after download completion
### Download Process
```rust
// Simplified download flow
1. Prepare local storage (create directories and pre-allocate files)
2. Build download plan (distribute chunks across available peers)
3. Execute parallel downloads from multiple peers
4. Retry failed chunks with alternative peers
5. Verify file integrity and notify completion
```
## Concurrency Model
### Async Architecture
The system uses Tokio's async runtime with these concurrent tasks:
1. **Main Peer Loop**: Handles UI commands and orchestrates operations
2. **Server Task**: Accepts incoming peer connections
3. **Discovery Task**: Continuously scans for new peers
4. **Ping Service**: Maintains peer health monitoring
5. **Connection Handlers**: Spawns per-connection tasks for request handling
6. **Download Tasks**: Parallel download workers for file transfers
### Thread Safety
- **Arc<RwLock<T>>**: Used for shared data structures to enable concurrent reads
- **Message Passing**: Uses Tokio channels (`UnboundedSender/Receiver`) for task communication
- **Connection Isolation**: Each peer connection is handled in an independent task
## Integration Points
### UI Interface
The crate communicates with the UI layer through two channels:
**Events to UI (`PeerEvent`):**
- Game list updates
- Peer discovery/connection events
- Download progress and completion
- Error notifications
**Commands from UI (`PeerCommand`):**
- Set game directory
- List available games
- Request game details
- Initiate game downloads
### Dependency Integration
- **`lanspread-db`**: Game metadata and file description structures
- **`lanspread-proto`**: Message serialization and protocol definitions
- **`lanspread-mdns`**: Service discovery functionality
- **`lanspread-utils`**: Common utilities and macros
## Configuration Constants
```rust
const CHUNK_SIZE: u64 = 512 * 1024; // 512KB file chunks
const MAX_RETRY_COUNT: usize = 3; // Maximum download retries
const PING_INTERVAL: Duration = 10s; // Peer health check interval
const DISCOVERY_INTERVAL: Duration = 10s; // mDNS scan interval
const STALE_TIMEOUT: Duration = 30s; // Peer inactivity timeout
```
## Usage Example
```rust
// Start the peer system
let (tx_notify_ui, rx_notify_ui) = tokio::sync::mpsc::unbounded_channel();
let tx_control = lanspread_peer::start_peer(
"/path/to/games".to_string(),
tx_notify_ui,
)?;
// Send commands to the peer
tx_control.send(PeerCommand::ListGames)?;
tx_control.send(PeerCommand::DownloadGameFiles {
id: "game_id".to_string(),
file_descriptions: vec![/* file descriptions */],
})?;
// Receive events from the peer
while let Some(event) = rx_notify_ui.recv().await {
match event {
PeerEvent::ListGames(games) => println!("Available games: {:?}", games),
PeerEvent::DownloadGameFilesFinished { id } => println!("Downloaded: {}", id),
// Handle other events...
}
}
```
## Error Handling
The crate implements comprehensive error handling:
- **Connection Errors**: Automatic reconnection and peer removal
- **Download Failures**: Retry with alternative peers and fallback strategies
- **Protocol Errors**: Graceful degradation and error reporting
- **File System Errors**: Proper cleanup and rollback on failures
## Security Considerations
- **TLS Encryption**: All peer communication is encrypted using QUIC with TLS
- **Certificate Validation**: Uses embedded certificates for peer authentication
- **Network Isolation**: Operates only on local network via mDNS discovery
- **File Access**: Restricted to configured game directory boundaries
## Performance Optimizations
- **Parallel Downloads**: Maximizes bandwidth by downloading from multiple peers simultaneously
- **Chunked Transfer**: Large files are split for parallel processing and resume capability
- **Connection Reuse**: Maintains persistent connections to reduce handshake overhead
- **Incremental Discovery**: Only synchronizes changed game metadata
This crate provides the foundation for Lanspread's distributed game sharing capabilities, enabling efficient peer-to-peer game distribution across local networks.
+15 -5
View File
@@ -453,10 +453,14 @@ async fn download_chunk(
let path = base_dir.join(&chunk.relative_path); let path = base_dir.join(&chunk.relative_path);
let mut file = OpenOptions::new() let mut file = OpenOptions::new()
.create(true) .create(true)
.truncate(true)
.write(true) .write(true)
.truncate(false)
.open(&path) .open(&path)
.await?; .await?;
if chunk.length == 0 && chunk.offset == 0 {
// fallback-to-whole-file path replaces any existing partial data
file.set_len(0).await?;
}
file.seek(std::io::SeekFrom::Start(chunk.offset)).await?; file.seek(std::io::SeekFrom::Start(chunk.offset)).await?;
let mut remaining = chunk.length; let mut remaining = chunk.length;
@@ -1252,8 +1256,11 @@ async fn run_peer_discovery(
log::info!("Starting peer discovery task"); log::info!("Starting peer discovery task");
loop { loop {
match discover_service(LANSPREAD_SERVICE_TYPE) { let discovery_result =
Ok(peer_addr) => { tokio::task::spawn_blocking(|| discover_service(LANSPREAD_SERVICE_TYPE)).await;
match discovery_result {
Ok(Ok(peer_addr)) => {
log::info!("Discovered peer at: {peer_addr}"); log::info!("Discovered peer at: {peer_addr}");
// Add peer to database // Add peer to database
@@ -1290,12 +1297,15 @@ async fn run_peer_discovery(
}); });
} }
} }
Err(e) => { Ok(Err(e)) => {
log::debug!("Peer discovery error: {e}"); log::debug!("Peer discovery error: {e}");
tokio::time::sleep(Duration::from_secs(5)).await; tokio::time::sleep(Duration::from_secs(5)).await;
} }
Err(e) => {
log::error!("Peer discovery join error: {e}");
tokio::time::sleep(Duration::from_secs(5)).await;
}
} }
// Wait before next discovery cycle // Wait before next discovery cycle
tokio::time::sleep(Duration::from_secs(10)).await; tokio::time::sleep(Duration::from_secs(10)).await;
} }