refactor(peer): name streamed integrity boundary
NEXT_STEPS item 4 needed the streamed-install integrity model to be a conscious decision. Keep the current runtime behavior, but name it as sender archive integrity: the receiver verifies streamed file size and RAR CRC32 from the sender's archive metadata before committing the install transaction. This protects against truncation, transport corruption, and stream provider bugs. It deliberately does not claim malicious-peer protection, because the sender controls both the streamed bytes and the RAR metadata. The docs now say that trusted content requires a future catalog schema with catalog-owned archive or extracted-file SHA-256 hashes. Test Plan: - just fmt - just test - just clippy - python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image - git diff --check - git diff --cached --check Refs: NEXT_STEPS.md item 4
This commit is contained in:
+7
-5
@@ -29,12 +29,14 @@ product-ready.
|
||||
peer-cli flow, including local-only final state, absent root archive/sentinel,
|
||||
byte count, and extracted payload SHA-256 hashes.
|
||||
|
||||
4. **Decide the integrity model**
|
||||
4. **Done — Decide the integrity model**
|
||||
|
||||
Current prototype verifies streamed bytes against RAR CRC32 from the
|
||||
sender’s archive headers. That catches corruption and provider bugs. It does
|
||||
not protect against a malicious peer lying. If you care about that, the next
|
||||
step is catalog-side trusted hashes for archive or extracted files.
|
||||
Streamed installs intentionally verify against sender archive metadata for
|
||||
now: each file must match the RAR-advertised size and CRC32. That catches
|
||||
transport corruption, truncation, and provider bugs, but does not claim
|
||||
malicious-peer protection. Trusted content remains a separate catalog schema
|
||||
step: add catalog-owned archive or extracted-file SHA-256 hashes, then verify
|
||||
those at the receiver before commit.
|
||||
|
||||
5. **Upgrade retry/resume semantics**
|
||||
|
||||
|
||||
@@ -122,6 +122,10 @@ Use S39-S41 to pin down low-disk streamed installs:
|
||||
- Non-solid and solid archives both install into `local/` without committing a
|
||||
root archive or root `version.ini`, so the receiver is installed but not a
|
||||
downloadable source.
|
||||
- Streamed install integrity is currently sender archive integrity: size and
|
||||
RAR CRC32 must match the sender's archive metadata. The SHA-256 checks in the
|
||||
scenarios prove the Docker/provider path matches the source fixture; they are
|
||||
not catalog-owned trust anchors.
|
||||
- S41 verifies the fixture is actually solid inside the source container, so
|
||||
solid handling stays covered by the same Docker harness as the existing
|
||||
streamed-install scenarios.
|
||||
|
||||
@@ -166,6 +166,18 @@ Most scans become O(number of game dirs), with full recursion only when needed.
|
||||
scratch sentinel files. `local/` and install transaction metadata are
|
||||
preserved, so a cancelled update of an installed game settles as local-only.
|
||||
|
||||
### Streamed install integrity
|
||||
|
||||
- Low-disk streamed installs request archive-derived file bytes from one peer
|
||||
and write them directly into the install transaction staging directory.
|
||||
- The receiver verifies every streamed file against the sender archive's file
|
||||
size and RAR CRC32 before the transaction may commit. This catches truncated
|
||||
streams, transport corruption, and provider bugs.
|
||||
- This is not malicious-peer protection: the peer controls both the archive
|
||||
metadata and the streamed bytes. A trusted-content model needs catalog-owned
|
||||
hashes, either for the root archives or for extracted files, and receiver-side
|
||||
SHA-256 verification against those catalog values before commit.
|
||||
|
||||
## Fault tolerance rules
|
||||
|
||||
- Every peer is keyed by `peer_id`, not by IP address.
|
||||
|
||||
@@ -37,6 +37,44 @@ const FRAME_CHANNEL_DEPTH: usize = 16;
|
||||
const STREAM_INSTALL_PROGRESS_UPDATE_INTERVAL: Duration = Duration::from_millis(500);
|
||||
const STREAM_CHUNK_SIZE: usize = 256 * 1024;
|
||||
|
||||
/// Integrity metadata advertised by the sender's RAR archive.
|
||||
///
|
||||
/// This catches transport corruption, truncation, and provider bugs. It is not
|
||||
/// a trusted-content guarantee because a malicious peer controls both the bytes
|
||||
/// and the archive metadata. Trusted content would need catalog-owned hashes.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
struct SenderArchiveIntegrity {
|
||||
expected_size: u64,
|
||||
expected_crc32: u32,
|
||||
}
|
||||
|
||||
impl SenderArchiveIntegrity {
|
||||
fn new(expected_size: u64, expected_crc32: u32) -> Self {
|
||||
Self {
|
||||
expected_size,
|
||||
expected_crc32,
|
||||
}
|
||||
}
|
||||
|
||||
fn verify(self, relative_path: &str, received: u64, actual_crc32: u32) -> eyre::Result<()> {
|
||||
if received != self.expected_size {
|
||||
eyre::bail!(
|
||||
"streamed file {relative_path} size mismatch: got {received}, expected {}",
|
||||
self.expected_size
|
||||
);
|
||||
}
|
||||
|
||||
if actual_crc32 != self.expected_crc32 {
|
||||
eyre::bail!(
|
||||
"streamed file {relative_path} sender RAR CRC32 mismatch: got {actual_crc32:08X}, expected {:08X}",
|
||||
self.expected_crc32
|
||||
);
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
pub type StreamInstallFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>;
|
||||
|
||||
pub trait StreamInstallProvider: Send + Sync {
|
||||
@@ -685,10 +723,9 @@ fn bytes_per_second(bytes: u64, elapsed: Duration) -> u64 {
|
||||
struct IncomingFile {
|
||||
relative_path: String,
|
||||
path: PathBuf,
|
||||
expected_size: u64,
|
||||
expected_crc32: u32,
|
||||
integrity: SenderArchiveIntegrity,
|
||||
received: u64,
|
||||
hasher: Hasher,
|
||||
crc32: Hasher,
|
||||
file: File,
|
||||
}
|
||||
|
||||
@@ -703,10 +740,9 @@ impl IncomingFile {
|
||||
Self {
|
||||
relative_path,
|
||||
path,
|
||||
expected_size,
|
||||
expected_crc32,
|
||||
integrity: SenderArchiveIntegrity::new(expected_size, expected_crc32),
|
||||
received: 0,
|
||||
hasher: Hasher::new(),
|
||||
crc32: Hasher::new(),
|
||||
file,
|
||||
}
|
||||
}
|
||||
@@ -720,15 +756,15 @@ impl IncomingFile {
|
||||
) -> eyre::Result<u64> {
|
||||
let offset = self.received;
|
||||
let length = u64::try_from(bytes.len())?;
|
||||
if offset.saturating_add(length) > self.expected_size {
|
||||
if offset.saturating_add(length) > self.integrity.expected_size {
|
||||
eyre::bail!(
|
||||
"streamed file {} exceeded expected size {}",
|
||||
self.relative_path,
|
||||
self.expected_size
|
||||
self.integrity.expected_size
|
||||
);
|
||||
}
|
||||
self.file.write_all(&bytes).await?;
|
||||
self.hasher.update(&bytes);
|
||||
self.crc32.update(&bytes);
|
||||
self.received = self.received.saturating_add(length);
|
||||
|
||||
let _ = tx_notify_ui.send(PeerEvent::DownloadGameFileChunkFinished {
|
||||
@@ -750,23 +786,9 @@ impl IncomingFile {
|
||||
}
|
||||
self.file.flush().await?;
|
||||
|
||||
if self.received != self.expected_size {
|
||||
eyre::bail!(
|
||||
"streamed file {} size mismatch: got {}, expected {}",
|
||||
self.relative_path,
|
||||
self.received,
|
||||
self.expected_size
|
||||
);
|
||||
}
|
||||
|
||||
let actual = self.hasher.finalize();
|
||||
if actual != self.expected_crc32 {
|
||||
eyre::bail!(
|
||||
"streamed file {} CRC32 mismatch: got {actual:08X}, expected {:08X}",
|
||||
self.relative_path,
|
||||
self.expected_crc32
|
||||
);
|
||||
}
|
||||
let actual_crc32 = self.crc32.finalize();
|
||||
self.integrity
|
||||
.verify(&self.relative_path, self.received, actual_crc32)?;
|
||||
|
||||
log::debug!(
|
||||
"Received streamed file {} -> {}",
|
||||
@@ -853,4 +875,45 @@ Details: RAR 5
|
||||
|
||||
assert!(err.to_string().contains("has no CRC32"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sender_archive_integrity_accepts_matching_size_and_crc32() {
|
||||
let bytes = b"payload";
|
||||
let integrity =
|
||||
SenderArchiveIntegrity::new(u64::try_from(bytes.len()).unwrap(), crc32_of(bytes));
|
||||
|
||||
integrity
|
||||
.verify(
|
||||
"bin/payload.bin",
|
||||
u64::try_from(bytes.len()).unwrap(),
|
||||
crc32_of(bytes),
|
||||
)
|
||||
.expect("matching sender archive metadata should verify");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sender_archive_integrity_rejects_size_mismatch() {
|
||||
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
|
||||
let err = integrity
|
||||
.verify("bin/payload.bin", 6, crc32_of(b"payload"))
|
||||
.expect_err("truncated file should fail sender archive integrity");
|
||||
|
||||
assert!(err.to_string().contains("size mismatch"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sender_archive_integrity_rejects_crc32_mismatch() {
|
||||
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
|
||||
let err = integrity
|
||||
.verify("bin/payload.bin", 7, crc32_of(b"paylord"))
|
||||
.expect_err("mutated file should fail sender archive integrity");
|
||||
|
||||
assert!(err.to_string().contains("sender RAR CRC32 mismatch"));
|
||||
}
|
||||
|
||||
fn crc32_of(bytes: &[u8]) -> u32 {
|
||||
let mut hasher = Hasher::new();
|
||||
hasher.update(bytes);
|
||||
hasher.finalize()
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user