refactor(peer): name streamed integrity boundary

NEXT_STEPS item 4 needed the streamed-install integrity model to be a
conscious decision. Keep the current runtime behavior, but name it as
sender archive integrity: the receiver verifies streamed file size and
RAR CRC32 from the sender's archive metadata before committing the
install transaction.

This protects against truncation, transport corruption, and stream
provider bugs. It deliberately does not claim malicious-peer protection,
because the sender controls both the streamed bytes and the RAR metadata.
The docs now say that trusted content requires a future catalog schema
with catalog-owned archive or extracted-file SHA-256 hashes.

Test Plan:
- just fmt
- just test
- just clippy
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 4
This commit is contained in:
2026-06-07 22:05:03 +02:00
parent 0e970dcec7
commit bb7497c0ff
4 changed files with 112 additions and 31 deletions
+7 -5
View File
@@ -29,12 +29,14 @@ product-ready.
peer-cli flow, including local-only final state, absent root archive/sentinel, peer-cli flow, including local-only final state, absent root archive/sentinel,
byte count, and extracted payload SHA-256 hashes. byte count, and extracted payload SHA-256 hashes.
4. **Decide the integrity model** 4. **Done — Decide the integrity model**
Current prototype verifies streamed bytes against RAR CRC32 from the Streamed installs intentionally verify against sender archive metadata for
senders archive headers. That catches corruption and provider bugs. It does now: each file must match the RAR-advertised size and CRC32. That catches
not protect against a malicious peer lying. If you care about that, the next transport corruption, truncation, and provider bugs, but does not claim
step is catalog-side trusted hashes for archive or extracted files. malicious-peer protection. Trusted content remains a separate catalog schema
step: add catalog-owned archive or extracted-file SHA-256 hashes, then verify
those at the receiver before commit.
5. **Upgrade retry/resume semantics** 5. **Upgrade retry/resume semantics**
+4
View File
@@ -122,6 +122,10 @@ Use S39-S41 to pin down low-disk streamed installs:
- Non-solid and solid archives both install into `local/` without committing a - Non-solid and solid archives both install into `local/` without committing a
root archive or root `version.ini`, so the receiver is installed but not a root archive or root `version.ini`, so the receiver is installed but not a
downloadable source. downloadable source.
- Streamed install integrity is currently sender archive integrity: size and
RAR CRC32 must match the sender's archive metadata. The SHA-256 checks in the
scenarios prove the Docker/provider path matches the source fixture; they are
not catalog-owned trust anchors.
- S41 verifies the fixture is actually solid inside the source container, so - S41 verifies the fixture is actually solid inside the source container, so
solid handling stays covered by the same Docker harness as the existing solid handling stays covered by the same Docker harness as the existing
streamed-install scenarios. streamed-install scenarios.
+12
View File
@@ -166,6 +166,18 @@ Most scans become O(number of game dirs), with full recursion only when needed.
scratch sentinel files. `local/` and install transaction metadata are scratch sentinel files. `local/` and install transaction metadata are
preserved, so a cancelled update of an installed game settles as local-only. preserved, so a cancelled update of an installed game settles as local-only.
### Streamed install integrity
- Low-disk streamed installs request archive-derived file bytes from one peer
and write them directly into the install transaction staging directory.
- The receiver verifies every streamed file against the sender archive's file
size and RAR CRC32 before the transaction may commit. This catches truncated
streams, transport corruption, and provider bugs.
- This is not malicious-peer protection: the peer controls both the archive
metadata and the streamed bytes. A trusted-content model needs catalog-owned
hashes, either for the root archives or for extracted files, and receiver-side
SHA-256 verification against those catalog values before commit.
## Fault tolerance rules ## Fault tolerance rules
- Every peer is keyed by `peer_id`, not by IP address. - Every peer is keyed by `peer_id`, not by IP address.
+89 -26
View File
@@ -37,6 +37,44 @@ const FRAME_CHANNEL_DEPTH: usize = 16;
const STREAM_INSTALL_PROGRESS_UPDATE_INTERVAL: Duration = Duration::from_millis(500); const STREAM_INSTALL_PROGRESS_UPDATE_INTERVAL: Duration = Duration::from_millis(500);
const STREAM_CHUNK_SIZE: usize = 256 * 1024; const STREAM_CHUNK_SIZE: usize = 256 * 1024;
/// Integrity metadata advertised by the sender's RAR archive.
///
/// This catches transport corruption, truncation, and provider bugs. It is not
/// a trusted-content guarantee because a malicious peer controls both the bytes
/// and the archive metadata. Trusted content would need catalog-owned hashes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct SenderArchiveIntegrity {
expected_size: u64,
expected_crc32: u32,
}
impl SenderArchiveIntegrity {
fn new(expected_size: u64, expected_crc32: u32) -> Self {
Self {
expected_size,
expected_crc32,
}
}
fn verify(self, relative_path: &str, received: u64, actual_crc32: u32) -> eyre::Result<()> {
if received != self.expected_size {
eyre::bail!(
"streamed file {relative_path} size mismatch: got {received}, expected {}",
self.expected_size
);
}
if actual_crc32 != self.expected_crc32 {
eyre::bail!(
"streamed file {relative_path} sender RAR CRC32 mismatch: got {actual_crc32:08X}, expected {:08X}",
self.expected_crc32
);
}
Ok(())
}
}
pub type StreamInstallFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>; pub type StreamInstallFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>;
pub trait StreamInstallProvider: Send + Sync { pub trait StreamInstallProvider: Send + Sync {
@@ -685,10 +723,9 @@ fn bytes_per_second(bytes: u64, elapsed: Duration) -> u64 {
struct IncomingFile { struct IncomingFile {
relative_path: String, relative_path: String,
path: PathBuf, path: PathBuf,
expected_size: u64, integrity: SenderArchiveIntegrity,
expected_crc32: u32,
received: u64, received: u64,
hasher: Hasher, crc32: Hasher,
file: File, file: File,
} }
@@ -703,10 +740,9 @@ impl IncomingFile {
Self { Self {
relative_path, relative_path,
path, path,
expected_size, integrity: SenderArchiveIntegrity::new(expected_size, expected_crc32),
expected_crc32,
received: 0, received: 0,
hasher: Hasher::new(), crc32: Hasher::new(),
file, file,
} }
} }
@@ -720,15 +756,15 @@ impl IncomingFile {
) -> eyre::Result<u64> { ) -> eyre::Result<u64> {
let offset = self.received; let offset = self.received;
let length = u64::try_from(bytes.len())?; let length = u64::try_from(bytes.len())?;
if offset.saturating_add(length) > self.expected_size { if offset.saturating_add(length) > self.integrity.expected_size {
eyre::bail!( eyre::bail!(
"streamed file {} exceeded expected size {}", "streamed file {} exceeded expected size {}",
self.relative_path, self.relative_path,
self.expected_size self.integrity.expected_size
); );
} }
self.file.write_all(&bytes).await?; self.file.write_all(&bytes).await?;
self.hasher.update(&bytes); self.crc32.update(&bytes);
self.received = self.received.saturating_add(length); self.received = self.received.saturating_add(length);
let _ = tx_notify_ui.send(PeerEvent::DownloadGameFileChunkFinished { let _ = tx_notify_ui.send(PeerEvent::DownloadGameFileChunkFinished {
@@ -750,23 +786,9 @@ impl IncomingFile {
} }
self.file.flush().await?; self.file.flush().await?;
if self.received != self.expected_size { let actual_crc32 = self.crc32.finalize();
eyre::bail!( self.integrity
"streamed file {} size mismatch: got {}, expected {}", .verify(&self.relative_path, self.received, actual_crc32)?;
self.relative_path,
self.received,
self.expected_size
);
}
let actual = self.hasher.finalize();
if actual != self.expected_crc32 {
eyre::bail!(
"streamed file {} CRC32 mismatch: got {actual:08X}, expected {:08X}",
self.relative_path,
self.expected_crc32
);
}
log::debug!( log::debug!(
"Received streamed file {} -> {}", "Received streamed file {} -> {}",
@@ -853,4 +875,45 @@ Details: RAR 5
assert!(err.to_string().contains("has no CRC32")); assert!(err.to_string().contains("has no CRC32"));
} }
#[test]
fn sender_archive_integrity_accepts_matching_size_and_crc32() {
let bytes = b"payload";
let integrity =
SenderArchiveIntegrity::new(u64::try_from(bytes.len()).unwrap(), crc32_of(bytes));
integrity
.verify(
"bin/payload.bin",
u64::try_from(bytes.len()).unwrap(),
crc32_of(bytes),
)
.expect("matching sender archive metadata should verify");
}
#[test]
fn sender_archive_integrity_rejects_size_mismatch() {
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
let err = integrity
.verify("bin/payload.bin", 6, crc32_of(b"payload"))
.expect_err("truncated file should fail sender archive integrity");
assert!(err.to_string().contains("size mismatch"));
}
#[test]
fn sender_archive_integrity_rejects_crc32_mismatch() {
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
let err = integrity
.verify("bin/payload.bin", 7, crc32_of(b"paylord"))
.expect_err("mutated file should fail sender archive integrity");
assert!(err.to_string().contains("sender RAR CRC32 mismatch"));
}
fn crc32_of(bytes: &[u8]) -> u32 {
let mut hasher = Hasher::new();
hasher.update(bytes);
hasher.finalize()
}
} }