codex review and fixes
This commit is contained in:
+93
-184
@@ -1,204 +1,113 @@
|
|||||||
# lanspread-peer
|
# lanspread-peer
|
||||||
|
|
||||||
A peer-to-peer networking component for the Lanspread system that enables distributed game sharing and discovery across local networks using QUIC protocol and mDNS service discovery.
|
`lanspread-peer` is the networking runtime that lets Lanspread nodes find each
|
||||||
|
other on the local network, exchange library metadata, and transfer game files.
|
||||||
|
It is designed to run headless – other crates (most notably
|
||||||
|
`lanspread-tauri-deno-ts`) embed it and drive it through a channel-based API.
|
||||||
|
|
||||||
## Overview
|
## Runtime Overview
|
||||||
|
|
||||||
The `lanspread-peer` crate implements a peer-to-peer networking layer that allows multiple instances of Lanspread to discover each other on the local network, share game libraries, and distribute game files efficiently. It operates as both a server (providing local games to other peers) and a client (discovering and downloading games from other peers).
|
- `start_peer(game_dir, tx_events)` boots the asynchronous runtime in the
|
||||||
|
background and returns an `UnboundedSender<PeerCommand>` that the caller uses
|
||||||
|
for control. The function immediately forwards the supplied game directory via
|
||||||
|
`PeerCommand::SetGameDir`.
|
||||||
|
- `PeerCommand` represents the small control surface exposed to the UI layer:
|
||||||
|
`ListGames`, `GetGame`, `DownloadGameFiles`, and `SetGameDir`.
|
||||||
|
- `PeerEvent` enumerates everything the peer runtime reports back to the UI:
|
||||||
|
library snapshots, download lifecycle updates, and peer membership changes.
|
||||||
|
- `PeerGameDB` collects remote peer metadata. It aggregates discovered peers’
|
||||||
|
`Game` definitions, tracks the latest ETI version per title, and keeps the
|
||||||
|
last seen list of `GameFileDescription` entries for each peer.
|
||||||
|
|
||||||
## Core Architecture
|
Internally the peer runtime owns three long-lived tasks that run for the
|
||||||
|
lifetime of the process:
|
||||||
|
|
||||||
### Main Components
|
1. **Server component** (`run_server_component`) – listens for QUIC connections,
|
||||||
|
advertises via mDNS, and serves `Request::ListGames`, `Request::GetGame`,
|
||||||
|
`Request::GetGameFileData`, and `Request::GetGameFileChunk` by reading from
|
||||||
|
the local game directory.
|
||||||
|
2. **Discovery loop** (`run_peer_discovery`) – uses the `lanspread-mdns`
|
||||||
|
helper to discover other peers. The blocking mDNS work is executed on a
|
||||||
|
dedicated thread via `tokio::task::spawn_blocking` so that the Tokio runtime
|
||||||
|
remains responsive.
|
||||||
|
3. **Ping service** (`run_ping_service`) – periodically issues QUIC ping requests
|
||||||
|
to keep peer liveness up to date and prunes stale entries from `PeerGameDB`.
|
||||||
|
|
||||||
1. **Peer System (`run_peer`)**: The main orchestrator that manages all peer operations
|
`load_local_game_db` scans the configured game directory (looking for folders
|
||||||
2. **Server Component (`run_server_component`)**: Handles incoming connections from other peers
|
with a `version.ini`) and hydrates a `GameDB`. That database is used to respond
|
||||||
3. **Peer Discovery (`run_peer_discovery`)**: Continuously discovers new peers via mDNS
|
to incoming metadata requests (`Request::ListGames` / `Request::GetGame`).
|
||||||
4. **Ping Service (`run_ping_service`)**: Maintains peer connectivity through health checks
|
|
||||||
5. **Download Manager**: Handles parallel file downloads from multiple peers
|
|
||||||
|
|
||||||
### Key Data Structures
|
## Networking and File Transfer
|
||||||
|
|
||||||
- **`PeerGameDB`**: Central database managing all discovered peers and their game collections
|
- Transport is handled by [`s2n-quic`](https://github.com/aws/s2n-quic); TLS
|
||||||
- **`PeerInfo`**: Contains information about a single peer including address, last seen time, and available games
|
cert/key material is compiled in from the repository root.
|
||||||
- **`PeerEvent`**: Events sent to the UI layer (peer discovery, connection status, download progress)
|
- Protocol messages are JSON-encoded structures defined in
|
||||||
- **`PeerCommand`**: Commands received from the UI layer (list games, download, etc.)
|
`lanspread-proto::{Request, Response}`.
|
||||||
|
- File transfers stream raw bytes over dedicated bidirectional QUIC streams.
|
||||||
|
`peer::send_game_file_data` sends entire files, while
|
||||||
|
`peer::send_game_file_chunk` services ranged requests.
|
||||||
|
|
||||||
## Network Protocol
|
### Download Pipeline
|
||||||
|
|
||||||
### Communication Layer
|
When the UI asks to download a game:
|
||||||
|
|
||||||
The crate uses **QUIC** (via `s2n-quic`) for all peer-to-peer communication with TLS encryption:
|
1. The UI first issues `PeerCommand::GetGame`. Each peer that still reports the
|
||||||
|
game is queried via `request_game_details_from_peer`, and their file
|
||||||
|
manifests are merged inside `PeerGameDB`.
|
||||||
|
2. Once the UI receives `PeerEvent::GotGameFiles`, it forwards the selected file
|
||||||
|
list back with `PeerCommand::DownloadGameFiles`.
|
||||||
|
3. `download_game_files` prepares the filesystem (creating directories and
|
||||||
|
pre-sizing files where possible), emits `PeerEvent::DownloadGameFilesBegin`,
|
||||||
|
and builds a per-peer plan (`build_peer_plans`) that round-robins file chunks
|
||||||
|
across the available peers that advertise the latest version.
|
||||||
|
4. Each plan is executed in its own task (`download_from_peer`). Chunk requests
|
||||||
|
use per-chunk QUIC streams and write into pre-created files. The chunk writer
|
||||||
|
keeps existing data intact and only truncates when we intentionally fall back
|
||||||
|
to a full file transfer, which prevents corruption when multiple peers fill
|
||||||
|
different regions of the same file.
|
||||||
|
5. Failures are accumulated and retried (up to `MAX_RETRY_COUNT`) via
|
||||||
|
`retry_failed_chunks`. If everything succeeds,
|
||||||
|
`PeerEvent::DownloadGameFilesFinished` is emitted; otherwise the UI receives
|
||||||
|
`PeerEvent::DownloadGameFilesFailed`.
|
||||||
|
|
||||||
- **Certificate-based security**: Uses embedded TLS certificates for secure communication
|
## Integration with `lanspread-tauri-deno-ts`
|
||||||
- **Bidirectional streams**: Each peer interaction uses separate QUIC streams
|
|
||||||
- **Connection management**: Built-in connection pooling and keep-alive mechanisms
|
|
||||||
|
|
||||||
### Message Protocol
|
The Tauri application embeds this crate in
|
||||||
|
`crates/lanspread-tauri-deno-ts/src-tauri/src/lib.rs`:
|
||||||
|
|
||||||
Messages are serialized using JSON and follow the request/response pattern defined in `lanspread-proto`:
|
- `LanSpreadState` holds onto the peer control channel, the latest aggregated
|
||||||
|
`GameDB`, per-game download state, and the user-selected game directory.
|
||||||
|
- The Tauri commands (`request_games`, `install_game`, `update_game`, and
|
||||||
|
`update_game_directory`) translate UI actions into `PeerCommand`s. In
|
||||||
|
particular, `update_game_directory` records the filesystem path, kicks off the
|
||||||
|
peer runtime on first use, and mirrors the installed/uninstalled state into
|
||||||
|
the UI-facing database.
|
||||||
|
- A background task consumes `PeerEvent`s and fans them out to the front-end via
|
||||||
|
Tauri publish/subscribe events (`games-list-updated`, `game-download-*`,
|
||||||
|
`peer-*`). Successful downloads trigger an `unrar` sidecar to unpack ETI
|
||||||
|
archives and clean up the temporary backup folders that are created when
|
||||||
|
updates begin.
|
||||||
|
- When downloads fail the Tauri layer restores the on-disk backup, keeping the
|
||||||
|
previous installation consistent even after partial transfers.
|
||||||
|
|
||||||
**Request Types:**
|
## Security & Operational Notes
|
||||||
- `Ping`: Health check
|
|
||||||
- `ListGames`: Request available games from a peer
|
|
||||||
- `GetGame`: Request detailed file list for a specific game
|
|
||||||
- `GetGameFileData`: Request complete file download
|
|
||||||
- `GetGameFileChunk`: Request specific file chunk for parallel downloads
|
|
||||||
|
|
||||||
**Response Types:**
|
- All QUIC connections are TLS encrypted; the shipped certificates are suitable
|
||||||
- `Pong`: Health check response
|
for local-network trust but should be rotated for production deployments.
|
||||||
- `ListGames(Vec<Game>)`: List of available games
|
- Peer discovery is restricted to the local link via mDNS.
|
||||||
- `GetGame`: Game file descriptions
|
- Long-running blocking mDNS calls are isolated on dedicated threads which keeps
|
||||||
- `GameNotFound`: Game not available
|
the async runtime responsive even when discovery takes a long time.
|
||||||
- Error responses for various failure conditions
|
- File writes are chunk-safe: partial chunk downloads now open files without
|
||||||
|
truncating existing data, avoiding the corruption that occurred previously
|
||||||
|
when multiple peers collectively filled a file.
|
||||||
|
|
||||||
## Peer Discovery
|
## Known Limitations
|
||||||
|
|
||||||
### mDNS Integration
|
- `PeerGameDB` currently models the latest metadata that other peers advertise.
|
||||||
|
If the UI needs to surface titles that only exist locally, additional merging
|
||||||
|
with the locally scanned `GameDB` will be required.
|
||||||
|
- The download planner uses a simple round-robin and does not yet take per-peer
|
||||||
|
throughput or failures into account when distributing work.
|
||||||
|
|
||||||
The crate uses mDNS (multicast DNS) for automatic peer discovery:
|
Refer to the source (particularly `src/lib.rs`) for the exact message shapes and
|
||||||
|
state machines.
|
||||||
- **Service Type**: `_lanspread._udp.local.`
|
|
||||||
- **Instance Naming**: `{hostname}-{uuid}` format to ensure uniqueness
|
|
||||||
- **Continuous Discovery**: Scans every 10 seconds for new peers
|
|
||||||
- **Automatic Advertising**: Each peer advertises its availability via mDNS
|
|
||||||
|
|
||||||
### Discovery Process
|
|
||||||
|
|
||||||
1. **mDNS Browser**: Continuously scans for `_lanspread._udp.local.` services
|
|
||||||
2. **Peer Registration**: New peers are added to `PeerGameDB`
|
|
||||||
3. **Game Synchronization**: Automatically requests game lists from newly discovered peers
|
|
||||||
4. **UI Notification**: Notifies the UI layer about peer discovery events
|
|
||||||
|
|
||||||
## Game Distribution System
|
|
||||||
|
|
||||||
### File Management
|
|
||||||
|
|
||||||
The system supports efficient game file distribution with these features:
|
|
||||||
|
|
||||||
**File Description:**
|
|
||||||
- `GameFileDescription`: Metadata for each file including path, size, and directory structure
|
|
||||||
- **Version Tracking**: Reads `version.ini` files for ETI game version management
|
|
||||||
- **Directory Structure**: Preserves complete game directory hierarchies
|
|
||||||
|
|
||||||
**Download Strategies:**
|
|
||||||
1. **Chunked Downloads**: Large files are split into 512KB chunks for parallel processing
|
|
||||||
2. **Multi-peer Downloads**: Chunks are distributed across available peers for maximum throughput
|
|
||||||
3. **Retry Mechanism**: Failed chunks are retried up to 3 times with different peers
|
|
||||||
4. **Integrity Verification**: File integrity is verified after download completion
|
|
||||||
|
|
||||||
### Download Process
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Simplified download flow
|
|
||||||
1. Prepare local storage (create directories and pre-allocate files)
|
|
||||||
2. Build download plan (distribute chunks across available peers)
|
|
||||||
3. Execute parallel downloads from multiple peers
|
|
||||||
4. Retry failed chunks with alternative peers
|
|
||||||
5. Verify file integrity and notify completion
|
|
||||||
```
|
|
||||||
|
|
||||||
## Concurrency Model
|
|
||||||
|
|
||||||
### Async Architecture
|
|
||||||
|
|
||||||
The system uses Tokio's async runtime with these concurrent tasks:
|
|
||||||
|
|
||||||
1. **Main Peer Loop**: Handles UI commands and orchestrates operations
|
|
||||||
2. **Server Task**: Accepts incoming peer connections
|
|
||||||
3. **Discovery Task**: Continuously scans for new peers
|
|
||||||
4. **Ping Service**: Maintains peer health monitoring
|
|
||||||
5. **Connection Handlers**: Spawns per-connection tasks for request handling
|
|
||||||
6. **Download Tasks**: Parallel download workers for file transfers
|
|
||||||
|
|
||||||
### Thread Safety
|
|
||||||
|
|
||||||
- **Arc<RwLock<T>>**: Used for shared data structures to enable concurrent reads
|
|
||||||
- **Message Passing**: Uses Tokio channels (`UnboundedSender/Receiver`) for task communication
|
|
||||||
- **Connection Isolation**: Each peer connection is handled in an independent task
|
|
||||||
|
|
||||||
## Integration Points
|
|
||||||
|
|
||||||
### UI Interface
|
|
||||||
|
|
||||||
The crate communicates with the UI layer through two channels:
|
|
||||||
|
|
||||||
**Events to UI (`PeerEvent`):**
|
|
||||||
- Game list updates
|
|
||||||
- Peer discovery/connection events
|
|
||||||
- Download progress and completion
|
|
||||||
- Error notifications
|
|
||||||
|
|
||||||
**Commands from UI (`PeerCommand`):**
|
|
||||||
- Set game directory
|
|
||||||
- List available games
|
|
||||||
- Request game details
|
|
||||||
- Initiate game downloads
|
|
||||||
|
|
||||||
### Dependency Integration
|
|
||||||
|
|
||||||
- **`lanspread-db`**: Game metadata and file description structures
|
|
||||||
- **`lanspread-proto`**: Message serialization and protocol definitions
|
|
||||||
- **`lanspread-mdns`**: Service discovery functionality
|
|
||||||
- **`lanspread-utils`**: Common utilities and macros
|
|
||||||
|
|
||||||
## Configuration Constants
|
|
||||||
|
|
||||||
```rust
|
|
||||||
const CHUNK_SIZE: u64 = 512 * 1024; // 512KB file chunks
|
|
||||||
const MAX_RETRY_COUNT: usize = 3; // Maximum download retries
|
|
||||||
const PING_INTERVAL: Duration = 10s; // Peer health check interval
|
|
||||||
const DISCOVERY_INTERVAL: Duration = 10s; // mDNS scan interval
|
|
||||||
const STALE_TIMEOUT: Duration = 30s; // Peer inactivity timeout
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage Example
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// Start the peer system
|
|
||||||
let (tx_notify_ui, rx_notify_ui) = tokio::sync::mpsc::unbounded_channel();
|
|
||||||
let tx_control = lanspread_peer::start_peer(
|
|
||||||
"/path/to/games".to_string(),
|
|
||||||
tx_notify_ui,
|
|
||||||
)?;
|
|
||||||
|
|
||||||
// Send commands to the peer
|
|
||||||
tx_control.send(PeerCommand::ListGames)?;
|
|
||||||
tx_control.send(PeerCommand::DownloadGameFiles {
|
|
||||||
id: "game_id".to_string(),
|
|
||||||
file_descriptions: vec![/* file descriptions */],
|
|
||||||
})?;
|
|
||||||
|
|
||||||
// Receive events from the peer
|
|
||||||
while let Some(event) = rx_notify_ui.recv().await {
|
|
||||||
match event {
|
|
||||||
PeerEvent::ListGames(games) => println!("Available games: {:?}", games),
|
|
||||||
PeerEvent::DownloadGameFilesFinished { id } => println!("Downloaded: {}", id),
|
|
||||||
// Handle other events...
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
The crate implements comprehensive error handling:
|
|
||||||
|
|
||||||
- **Connection Errors**: Automatic reconnection and peer removal
|
|
||||||
- **Download Failures**: Retry with alternative peers and fallback strategies
|
|
||||||
- **Protocol Errors**: Graceful degradation and error reporting
|
|
||||||
- **File System Errors**: Proper cleanup and rollback on failures
|
|
||||||
|
|
||||||
## Security Considerations
|
|
||||||
|
|
||||||
- **TLS Encryption**: All peer communication is encrypted using QUIC with TLS
|
|
||||||
- **Certificate Validation**: Uses embedded certificates for peer authentication
|
|
||||||
- **Network Isolation**: Operates only on local network via mDNS discovery
|
|
||||||
- **File Access**: Restricted to configured game directory boundaries
|
|
||||||
|
|
||||||
## Performance Optimizations
|
|
||||||
|
|
||||||
- **Parallel Downloads**: Maximizes bandwidth by downloading from multiple peers simultaneously
|
|
||||||
- **Chunked Transfer**: Large files are split for parallel processing and resume capability
|
|
||||||
- **Connection Reuse**: Maintains persistent connections to reduce handshake overhead
|
|
||||||
- **Incremental Discovery**: Only synchronizes changed game metadata
|
|
||||||
|
|
||||||
This crate provides the foundation for Lanspread's distributed game sharing capabilities, enabling efficient peer-to-peer game distribution across local networks.
|
|
||||||
|
|||||||
@@ -453,10 +453,14 @@ async fn download_chunk(
|
|||||||
let path = base_dir.join(&chunk.relative_path);
|
let path = base_dir.join(&chunk.relative_path);
|
||||||
let mut file = OpenOptions::new()
|
let mut file = OpenOptions::new()
|
||||||
.create(true)
|
.create(true)
|
||||||
.truncate(true)
|
|
||||||
.write(true)
|
.write(true)
|
||||||
|
.truncate(false)
|
||||||
.open(&path)
|
.open(&path)
|
||||||
.await?;
|
.await?;
|
||||||
|
if chunk.length == 0 && chunk.offset == 0 {
|
||||||
|
// fallback-to-whole-file path replaces any existing partial data
|
||||||
|
file.set_len(0).await?;
|
||||||
|
}
|
||||||
file.seek(std::io::SeekFrom::Start(chunk.offset)).await?;
|
file.seek(std::io::SeekFrom::Start(chunk.offset)).await?;
|
||||||
|
|
||||||
let mut remaining = chunk.length;
|
let mut remaining = chunk.length;
|
||||||
@@ -1252,8 +1256,11 @@ async fn run_peer_discovery(
|
|||||||
log::info!("Starting peer discovery task");
|
log::info!("Starting peer discovery task");
|
||||||
|
|
||||||
loop {
|
loop {
|
||||||
match discover_service(LANSPREAD_SERVICE_TYPE) {
|
let discovery_result =
|
||||||
Ok(peer_addr) => {
|
tokio::task::spawn_blocking(|| discover_service(LANSPREAD_SERVICE_TYPE)).await;
|
||||||
|
|
||||||
|
match discovery_result {
|
||||||
|
Ok(Ok(peer_addr)) => {
|
||||||
log::info!("Discovered peer at: {peer_addr}");
|
log::info!("Discovered peer at: {peer_addr}");
|
||||||
|
|
||||||
// Add peer to database
|
// Add peer to database
|
||||||
@@ -1290,12 +1297,15 @@ async fn run_peer_discovery(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Err(e) => {
|
Ok(Err(e)) => {
|
||||||
log::debug!("Peer discovery error: {e}");
|
log::debug!("Peer discovery error: {e}");
|
||||||
tokio::time::sleep(Duration::from_secs(5)).await;
|
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||||
}
|
}
|
||||||
|
Err(e) => {
|
||||||
|
log::error!("Peer discovery join error: {e}");
|
||||||
|
tokio::time::sleep(Duration::from_secs(5)).await;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Wait before next discovery cycle
|
// Wait before next discovery cycle
|
||||||
tokio::time::sleep(Duration::from_secs(10)).await;
|
tokio::time::sleep(Duration::from_secs(10)).await;
|
||||||
}
|
}
|
||||||
|
|||||||
Reference in New Issue
Block a user