Compare commits

..

8 Commits

Author SHA1 Message Date
ddidderr ea709b6277 test(peer-cli): align scenarios with catalog versions
Remote aggregation now filters to catalog-version roots, but the checked-in
peer-cli fixtures and skew scenarios still stamped synthetic future versions.
That hid fixture rows in S3 and left scenario docs asserting latest-version
behavior.

Teach the harness the catalog versions for fixture game IDs, stamp generated
fixtures with catalog versions by default, and update skew, mesh, propagation,
and throughput scenarios to expect only catalog-version peers. Also wire S38
into the executable matrix so the documented first-play launch-setting scenario
is covered by the same full run as S1-S47.

This keeps stale peers as negative coverage: they are absent from list-games and
cannot provide descriptors, votes, or chunks. The fixture version.ini updates
are checked in so alpha, bravo, charlie, and persona roots advertise
downloadable catalog games again.

Test Plan:
- python3 -m py_compile
  crates/lanspread-peer-cli/scripts/run_extended_scenarios.py
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py \
  S3 S8 S14 S15 S16 S17 S21 S22 S23 S24 S29 S30 S31 S34 S36 S37 \
  S39 S40 S41 S42 S43 S44 S45 S46 S47 --build-image
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S38
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py
- git diff --check
- git diff --cached --check

Docs: PEER_CLI_SCENARIOS.md
2026-06-07 23:14:08 +02:00
ddidderr f62515451b feat(ui): label streamed installs as not shareable
NEXT_STEPS item 7 needed the installed-but-not-downloaded state to be
clear to users. Keep streamed installs in the installed visual state so
sorting, filters, and the primary Play action stay unchanged, but make the
sharing limitation visible in the UI.

Cards now label that state as `Not shareable`, while the detail modal
status says `Installed, not shareable`. Downloaded-and-installed games
keep the normal `Installed` wording.

Test Plan:
- just frontend-test
- just build
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 7
2026-06-07 22:29:26 +02:00
ddidderr 9288fda037 test(peer-cli): expand streamed install edge coverage
NEXT_STEPS item 6 called for the remaining streamed-install edge cases to
be covered in the peer-cli matrix. Add S43-S47 for already-installed
rejection, corrupt archive rollback, sender disconnect, receiver cancel,
and sorted multi-archive streaming.

The receiver-cancel scenario needs the harness to drive the same runtime
path as the GUI, so `lanspread-peer-cli` now accepts a narrow
`cancel-download` command that forwards to `PeerCommand::CancelDownload`.
A parser test covers the new JSONL command shape.

Add `fixture-multi/cnctw`, a tiny two-archive RAR fixture. S47 uses it to
prove streamed installs process root `.eti` archives in sorted order and
commit only extracted `local/` payloads, not the root archives or
`version.ini` sentinel.

Test Plan:
- just fmt
- python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S43 S44 S45 S46 S47 --build-image
- just test
- just clippy
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 6
2026-06-07 22:26:49 +02:00
ddidderr 88bfaeb04a test(peer-cli): cover streamed retry fallback
NEXT_STEPS item 5 needs streamed installs to have an explicit retry
policy. The handler already retries whole-stream attempts across the
majority-validated peer set, so add S42 to prove that behavior with the
Docker harness instead of leaving it implicit.

S42 starts two catalog-version-matching `cnctw` sources. The first source
sorts first in retry order but has `--unrar /missing-unrar`, so its stream
attempt fails before sending chunks. The second source then completes a
fresh whole-stream attempt. The scenario asserts local-only installed
state, no root archive or sentinel, no `.local.installing` staging
leftover, chunk events only from the good source, matching streamed byte
count, and SHA-256 payload equality against the good source's `unrar p`.

This pins the current policy: retry the entire stream from another
validated peer, do not preserve partial files across attempts, and do not
promise byte-offset resume.

Test Plan:
- python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S42
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 5
2026-06-07 22:14:41 +02:00
ddidderr bb7497c0ff refactor(peer): name streamed integrity boundary
NEXT_STEPS item 4 needed the streamed-install integrity model to be a
conscious decision. Keep the current runtime behavior, but name it as
sender archive integrity: the receiver verifies streamed file size and
RAR CRC32 from the sender's archive metadata before committing the
install transaction.

This protects against truncation, transport corruption, and stream
provider bugs. It deliberately does not claim malicious-peer protection,
because the sender controls both the streamed bytes and the RAR metadata.
The docs now say that trusted content requires a future catalog schema
with catalog-owned archive or extracted-file SHA-256 hashes.

Test Plan:
- just fmt
- just test
- just clippy
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 4
2026-06-07 22:05:03 +02:00
ddidderr 0e970dcec7 test(peer-cli): cover solid streamed installs
NEXT_STEPS item 3 needed solid archive handling to be a deliberate
contract instead of an incidental RAR header attribute. Add a tiny real
solid RAR fixture and S41 to the extended peer-cli scenarios so the
Docker harness proves this path end to end.

The scenario verifies the source archive with container-bundled
`unrar lt`, streams the install with the injected provider, and then
asserts the receiver is installed local-only without a root archive or
root `version.ini`. It also compares local payload SHA-256 hashes against
`unrar p` output and checks the streamed byte count matches the extracted
entries. This keeps the existing one metadata pass plus one sequential
payload pass contract covered for solid archives.

Test Plan:
- just fmt
- just test
- python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image
- python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41
- git diff --check
- git diff --cached --check

Refs: NEXT_STEPS.md item 3
2026-06-07 22:00:21 +02:00
ddidderr c313f7c9ae docs(stream-install): mark one-pass provider complete
NEXT_STEPS item 2 still described the old per-file unrar provider shape even
though the current shared provider now performs one technical listing pass and
one sequential unrar payload pass per archive. Update the roadmap so the next
implementation slice starts at the remaining solid-archive policy work instead
of chasing an already-replaced extraction loop.

The item 3 wording now keeps the solid/non-solid archive fork explicit without
suggesting the current provider still needs to be swapped merely to avoid
per-file extraction.

Test Plan:
- git diff --check

Refs: NEXT_STEPS.md item 2
2026-06-07 21:40:48 +02:00
ddidderr 40697a73e5 feat(tauri): add low-disk streamed install action
NEXT_STEPS item 1 called out that streamed install was still CLI-only
because the Tauri app started the peer with no stream provider. Users can now
choose an explicit "Low disk install" action from the game detail modal for
remote-only games instead of taking the default archive-preserving download
path.

The GUI command queues a normal peer detail fetch first so the peer database
has the file metadata needed for source validation. A small pending handoff in
Tauri routes the resulting GotGameFiles event into StreamInstallGame instead
of DownloadGameFiles, and clears that pending state on no-peer or download
failure events. This keeps the existing download continuation untouched for
the default action.

The external unrar stream provider moved from the CLI harness into
lanspread-peer so CLI and Tauri use the same implementation. Tauri resolves
the bundled unrar sidecar path and injects that provider at peer startup;
falling back to the noop provider keeps peer startup alive if the sidecar
cannot be resolved, while the streamed install operation still fails safely.

Test Plan:
- just fmt
- just test
- just frontend-test
- just clippy
- just build
- git diff --check

Refs: NEXT_STEPS.md item 1
2026-06-07 21:39:02 +02:00
34 changed files with 1602 additions and 604 deletions
Generated
-3
View File
@@ -2060,16 +2060,13 @@ dependencies = [
name = "lanspread-peer-cli" name = "lanspread-peer-cli"
version = "0.1.0" version = "0.1.0"
dependencies = [ dependencies = [
"bytes",
"eyre", "eyre",
"lanspread-compat", "lanspread-compat",
"lanspread-db", "lanspread-db",
"lanspread-peer", "lanspread-peer",
"lanspread-proto",
"serde", "serde",
"serde_json", "serde_json",
"tokio", "tokio",
"tokio-util",
] ]
[[package]] [[package]]
+42 -41
View File
@@ -5,60 +5,61 @@ archive-derived install bytes into `local/` without making the receiver a
source?” Yes. Next Id harden the pieces that decide whether this is source?” Yes. Next Id harden the pieces that decide whether this is
product-ready. product-ready.
1. **Move from CLI-only to real app integration** 1. **Done — Move from CLI-only to real app integration**
Add a GUI command/control path for “stream install / low disk mode”, The GUI now has an explicit “Low disk install” action in the game detail
probably behind an explicit option. The Tauri crate currently opts out with modal for remote-only games. The Tauri backend queues that path through
`stream_install_provider: None`, so the GUI cannot use it yet. `stream_install_game`, injects the shared external `unrar` stream provider,
and hands fetched file details to `StreamInstallGame` instead of the normal
download command.
2. **Replace per-file `unrar p` with a final archive provider** 2. **Done — Replace per-file `unrar p` with a final archive provider**
The prototype provider is intentionally simple: `unrar lt`, then `unrar p` The shared external `unrar` stream provider now runs `unrar lt` once for the
per file. Good for non-solid archives, bad for solid archives. Final shape archive metadata and one sequential `unrar p` pass per archive for payload
should be a one-pass provider with real entry boundaries, likely via libunrar bytes. It frames directories, file starts, file chunks, and file ends from
or a purpose-built wrapper. the technical listing, so CLI and GUI callers use one purpose-built provider
instead of a per-file extraction loop.
3. **Handle solid archives deliberately** 3. **Done — Handle solid archives deliberately**
Add archive inspection that decides: The provider exposes the RAR `solid` flag in `ArchiveBegin` and always uses
one sequential payload pass per archive, which is the safe path for solid
archives. S41 now verifies a real solid RAR fixture through the Docker
peer-cli flow, including local-only final state, absent root archive/sentinel,
byte count, and extracted payload SHA-256 hashes.
- non-solid: per-file streaming is fine 4. **Done — Decide the integrity model**
- solid: one sequential archive pass only
This is the big architectural fork we discussed, and the prototypes Streamed installs intentionally verify against sender archive metadata for
provider is the thing to swap. now: each file must match the RAR-advertised size and CRC32. That catches
transport corruption, truncation, and provider bugs, but does not claim
malicious-peer protection. Trusted content remains a separate catalog schema
step: add catalog-owned archive or extracted-file SHA-256 hashes, then verify
those at the receiver before commit.
4. **Decide the integrity model** 5. **Done — Upgrade retry/resume semantics**
Current prototype verifies streamed bytes against RAR CRC32 from the Streamed install attempts now use the same majority-validated peer set as
senders archive headers. That catches corruption and provider bugs. It does normal downloads, and each failed attempt rolls back its staging transaction
not protect against a malicious peer lying. If you care about that, the next before trying the next peer. S42 pins the policy: retry the whole stream from
step is catalog-side trusted hashes for archive or extracted files. another validated peer, keep no partial files across attempts, and do not add
byte-offset resume until there is a strong reason.
5. **Upgrade retry/resume semantics** 6. **Done — Expand scenario coverage**
Right now, failed stream means failed operation and rollback. Next useful S43-S47 cover the remaining streamed-install edges: already-installed
step: rejection, corrupt archive rollback, sender disconnect mid-stream, receiver
cancel mid-stream, and multi-archive `.eti` roots streamed in sorted order.
The peer-cli harness now exposes `cancel-download` so cancellation scenarios
exercise the same runtime path as the GUI.
- retry whole stream from another trusted peer 7. **Done — Clean product semantics**
- later, maybe keep completed files and restart only the interrupted file
- avoid byte-offset resume until theres a strong reason
6. **Expand scenario coverage** The UI now keeps streamed installs in the installed visual state while making
the sharing limitation explicit: cards show `Not shareable`, and the detail
Id add cases for: modal status shows `Installed, not shareable`. Downloaded-and-installed games
keep the normal `Installed` label.
- sender disconnect mid-stream
- receiver cancel mid-stream
- corrupted/truncated stream fails and leaves no `local/`
- already-installed game rejects streamed install
- multi-archive `.eti` roots stream in sorted order
7. **Clean product semantics**
Decide how the UI labels this state. It is installed but not downloaded, so
“Local only” is technically correct, but users may need a clear affordance
like “Installed, not shareable”.
My recommended next slice: make the provider abstraction final-ish, then My recommended next slice: make the provider abstraction final-ish, then
implement a real one-pass provider. Everything else builds cleanly on that. implement a real one-pass provider. Everything else builds cleanly on that.
+140 -24
View File
@@ -22,51 +22,57 @@ for deterministic local runs; mDNS/macvlan remains an environment smoke path.
| S12 | Transfer serving gates | A peer has a non-catalog, missing-sentinel, active-operation, or `local/` path request. | The serving peer declines metadata/data; covered by unit tests where timing is too small for a stable CLI race test. | | S12 | Transfer serving gates | A peer has a non-catalog, missing-sentinel, active-operation, or `local/` path request. | The serving peer declines metadata/data; covered by unit tests where timing is too small for a stable CLI race test. |
| S13 | Exact transferred-file equality | Repeat small and large downloads, then compare every transferred regular file against its source with SHA-256 manifests. | Source and receiver manifests match exactly for each transferred file; no extra or missing files appear in the downloaded game root. | | S13 | Exact transferred-file equality | Repeat small and large downloads, then compare every transferred regular file against its source with SHA-256 manifests. | Source and receiver manifests match exactly for each transferred file; no extra or missing files appear in the downloaded game root. |
| S14 | Large multi-peer chunked download | `fixture-alpha/alienswarm` contains a renamed RAR `.eti` larger than 100 MB. A second peer downloads it, then a third peer downloads `alienswarm` from both peers. | The third peer's downloaded files match the source by SHA-256; `download-chunk-finished` events show the large `.eti` chunks coming from both peers with byte counts balanced within one chunk. | | S14 | Large multi-peer chunked download | `fixture-alpha/alienswarm` contains a renamed RAR `.eti` larger than 100 MB. A second peer downloads it, then a third peer downloads `alienswarm` from both peers. | The third peer's downloaded files match the source by SHA-256; `download-chunk-finished` events show the large `.eti` chunks coming from both peers with byte counts balanced within one chunk. |
| S15 | Three-way version skew | Three peers advertise the same catalog game ID. Peer A has `version.ini=20250101`, peer B has `version.ini=20250201`, and peer C has `version.ini=20250301`; each version has distinguishable file contents. An empty client connects to all three and downloads the game with `install=false`. | `list-games` shows one row for the game with `peer_count=3` and `eti_game_version=20250301`. The `got-game-files` descriptor set and transfer source are peer C's newest version only; no chunks come from A or B. The receiver's `version.ini` and SHA-256 manifest match C exactly. | | S15 | Catalog-version skew | Three peers advertise the same catalog game ID. Peers A and B have stale `version.ini` values; peer C has the catalog's expected version. An empty client connects to all three and downloads the game with `install=false`. | `list-games` shows one row for the game with `peer_count=1` and the catalog `eti_game_version`. The `got-game-files` descriptor set and transfer source are peer C only; no chunks come from A or B. The receiver's `version.ini` and SHA-256 manifest match C exactly. |
| S16 | Latest-version fanout with stale peers present | Peer A has an older version of a game. Peers B and C both advertise the same newest version with matching file manifests; use a large file when proving chunk split. | The aggregated row still counts all ready peers, but eligible transfer peers are only B and C. Large-file chunks may split between B and C; peer A contributes no manifest majority vote and no file chunks. | | S16 | Catalog-version fanout with stale peers present | Peer A has a stale version of a game. Peers B and C both advertise the catalog version with matching file manifests; use a large file when proving chunk split. | The aggregated row counts only catalog-version ready peers. Large-file chunks may split between B and C; peer A is not listed as downloadable and contributes no manifest vote or file chunks. |
| S17 | Latest-version conflict rejection | Peer A has an older version. Peers B and C both advertise the newest version, but their latest-version file sizes conflict. | Validation considers only the latest-version peers, so A cannot rescue the majority. The download fails with `download-failed`, and no committed target `version.ini` remains. | | S17 | Catalog-version conflict rejection | Peer A has a stale version. Peers B and C both advertise the catalog version, but their file sizes conflict. | Validation considers only the catalog-version peers, so A cannot rescue the majority. The download fails with `download-failed`, and no committed target `version.ini` remains. |
| S18 | Mid-download source drop with redundancy | Client downloads a large shared game from two ready peers, then one source is killed after the download has begun. | Failed chunks are retried against the surviving source; the download finishes, no `download-failed` is emitted, and the receiver's files match the source by diff or SHA-256. | | S18 | Mid-download source drop with redundancy | Client downloads a large shared game from two ready peers, then one source is killed after the download has begun. | Failed chunks are retried against the surviving source; the download finishes, no `download-failed` is emitted, and the receiver's files match the source by diff or SHA-256. |
| S19 | Mid-download sole-source drop | Client downloads a large game from one source, then that source is killed after the download has begun. | The download emits `download-failed`; no committed target `version.ini` remains; any partial payload is not advertised as ready; active operation state clears so a retry is possible. | | S19 | Mid-download sole-source drop | Client downloads a large game from one source, then that source is killed after the download has begun. | The download emits `download-failed`; no committed target `version.ini` remains; any partial payload is not advertised as ready; active operation state clears so a retry is possible. |
| S20 | Receiver write failure | Client downloads a large game into a constrained `/games` filesystem. | The download fails deterministically, no committed `version.ini` is advertised, and active operation state clears so the peer can retry later. | | S20 | Receiver write failure | Client downloads a large game into a constrained `/games` filesystem. | The download fails deterministically, no committed `version.ini` is advertised, and active operation state clears so the peer can retry later. |
| S21 | Add-game propagation | Two connected peers are running; one peer gains a new catalog game root through a completed download or an external drop. | The other peer receives a library update without reconnecting, and `list-games` shows the new remote game under the existing peer. | | S21 | Add-game propagation | Two connected peers are running; one peer gains a new catalog game root through a completed download or an external drop. | The other peer receives a library update without reconnecting, and `list-games` shows the new remote game under the existing peer. |
| S22 | Remove-game propagation | Two connected peers are running; one peer loses a previously advertised game root. | The other peer receives a library update without dropping the peer, and `list-games` no longer shows that remote game. | | S22 | Remove-game propagation | Two connected peers are running; one peer loses a previously advertised game root. | The other peer receives a library update without dropping the peer, and `list-games` no longer shows that remote game. |
| S23 | Version bump propagation | Two connected peers are running; one peer's ready game root gets a newer `version.ini`. | The other peer receives a library update without reconnecting, and the aggregated row reflects the newer `eti_game_version`. | | S23 | Version bump propagation | Two connected peers are running; one peer's ready game root starts with a stale `version.ini`, then changes to the catalog version. | The other peer receives a library update without reconnecting; the stale row is absent before the change, then the catalog-version game appears as downloadable. |
| S24 | Two clients pull from one source | Two empty clients connect to the same source and download the same large game concurrently. | Both downloads finish, both receivers match the source by diff or SHA-256, and the source remains responsive. | | S24 | Two clients pull from one source | Two empty clients connect to the same source and download the same large game concurrently. | Both downloads finish, both receivers match the source by diff or SHA-256, and the source remains responsive. |
| S25 | One client downloads two games concurrently | One client connected to a source issues two different `download` commands without waiting for the first to finish. | Both operations may run in parallel; both eventually finish, each game reaches the requested install state, and each transferred root matches its source. | | S25 | One client downloads two games concurrently | One client connected to a source issues two different `download` commands without waiting for the first to finish. | Both operations may run in parallel; both eventually finish, each game reaches the requested install state, and each transferred root matches its source. |
| S26 | Same-game duplicate download rejection | A client starts downloading a game, then issues a second `download` command for the same game while the first operation is active. | The second request is rejected deterministically as an operation-in-progress condition; the first download is not corrupted and still reaches its documented final state. | | S26 | Same-game duplicate download rejection | A client starts downloading a game, then issues a second `download` command for the same game while the first operation is active. | The second request is rejected deterministically as an operation-in-progress condition; the first download is not corrupted and still reaches its documented final state. |
| S27 | Self-connect rejection | A peer sends `connect` to its own advertised listener address. | The command fails cleanly, no self-peer entry is created, and the peer remains responsive. | | S27 | Self-connect rejection | A peer sends `connect` to its own advertised listener address. | The command fails cleanly, no self-peer entry is created, and the peer remains responsive. |
| S28 | Address change without identity change | A known peer is rediscovered with the same peer ID and a different listener address while its library is still known. | The peer record updates in place to the new address, the existing library stays attached to that peer ID, and no duplicate peer entry appears. This is covered with a deterministic unit-level check until the CLI can rebind a live listener without restart. | | S28 | Address change without identity change | A known peer is rediscovered with the same peer ID and a different listener address while its library is still known. | The peer record updates in place to the new address, the existing library stays attached to that peer ID, and no duplicate peer entry appears. This is covered with a deterministic unit-level check until the CLI can rebind a live listener without restart. |
| S29 | Empty-library peer participates | A peer with no games connects into the mesh. | Other peers list it as a peer with zero games; it can receive a download, advertise the new game without restart, and become a source. | | S29 | Empty-library peer participates | A peer with no games connects into the mesh. | Other peers list it as a peer with zero games; it can receive a download, advertise the new game without restart, and become a source. |
| S30 | 5+ peer mesh aggregation | Five peers advertise partially overlapping catalog games with a mix of unique games, shared games, and differing versions; a sixth client connects to all five. | The client shows one row per game ID, correct ready-source `peer_count`, latest `eti_game_version`, no duplicates, and no self entries. | | S30 | 5+ peer mesh aggregation | Five peers advertise partially overlapping catalog games with a mix of unique and shared catalog-version games; a sixth client connects to all five. | The client shows one row per game ID, correct catalog-version ready-source `peer_count`, catalog `eti_game_version`, no duplicates, and no self entries. |
| S31 | Bootstrapped peer becomes source in same session | An empty client downloads a game from a source, the original source shuts down, then a fresh third peer downloads the same game from the bootstrapped client. | The third peer's files match the original source by diff or SHA-256, proving downloaded files become servable without restart. | | S31 | Bootstrapped peer becomes source in same session | An empty client downloads a game from a source, the original source shuts down, then a fresh third peer downloads the same game from the bootstrapped client. | The third peer's files match the original source by diff or SHA-256, proving downloaded files become servable without restart. |
| S32 | Reinstall after uninstall | A downloaded game is installed, uninstalled, then installed again without another download. | `local/` is recreated from preserved root files, no transfer events occur during reinstall, and the game returns to `installed=true`. | | S32 | Reinstall after uninstall | A downloaded game is installed, uninstalled, then installed again without another download. | `local/` is recreated from preserved root files, no transfer events occur during reinstall, and the game returns to `installed=true`. |
| S33 | Install after external root mutation | A downloaded game root is externally mutated before `install` is issued. | The CLI fixture installer installs from the current root bytes. The resulting `local/fixture-payload.txt` must match the mutated archive bytes exactly. | | S33 | Install after external root mutation | A downloaded game root is externally mutated before `install` is issued. | The CLI fixture installer installs from the current root bytes. The resulting `local/fixture-payload.txt` must match the mutated archive bytes exactly. |
| S34 | Many-small-files game without `.eti` | A catalog game root contains `version.ini` plus many small regular files and no archive. | Download with `install=false` transfers every file, chunk events are coherent for small files, and source/receiver manifests match exactly. | | S34 | Many-small-files game without `.eti` | A catalog game root contains `version.ini` plus many small regular files and no archive. | Download with `install=false` transfers every file, chunk events are coherent for small files, and source/receiver manifests match exactly. |
| S35 | Unknown game ID from remote peer | A remote peer advertises a game ID that is not in the receiver's catalog. | The receiver does not list the unknown game as downloadable, download attempts fail deterministically, and no local files are created. | | S35 | Unknown game ID from remote peer | A remote peer advertises a game ID that is not in the receiver's catalog. | The receiver does not list the unknown game as downloadable, download attempts fail deterministically, and no local files are created. |
| S36 | Latest singleton beats stale majority | Five peers advertise one game; one peer has `20260501`, four peers have `20250101`. | `list-games` reports `eti_game_version=20260501`; all descriptors and chunks come from the singleton latest peer; stale peers contribute zero bytes. | | S36 | Catalog singleton beats stale majority | Five peers advertise one game; one peer has the catalog version and four peers have stale versions. | `list-games` reports `peer_count=1` and the catalog `eti_game_version`; all descriptors and chunks come from the singleton catalog-version peer, while stale peers remain hidden and contribute zero bytes. |
| S37 | Single-source download throughput | A source peer advertises a temporary catalog game with one sparse `2 GiB` `.eti`; an empty client downloads it with `install=false`. | The client emits `download-finished` with throughput measurements (`bytes`, `duration_ms`, `mib_per_s`, `mbit_per_s`), and the downloaded archive size matches the source. | | S37 | Single-source download throughput | A source peer advertises a temporary catalog game with one sparse `2 GiB` `.eti`; an empty client downloads it with `install=false`. | The client emits `download-finished` with throughput measurements (`bytes`, `duration_ms`, `mib_per_s`, `mbit_per_s`), and the downloaded archive size matches the source. |
| S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. | | S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. |
| S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. | | S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. |
| S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. | | S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. |
| S41 | Solid archive streamed install | Empty client connects to a peer serving `fixture-solid/cnctw`, whose `.eti` is a real solid RAR archive. The receiver uses the container-bundled `unrar` stream provider. | The fixture is verified as solid with `unrar lt`; streamed install finishes with `downloaded=false`, `installed=true`, `availability=LocalOnly`; root archive and `version.ini` are absent; streamed byte count equals the extracted solid entries; local payload SHA-256 hashes match `unrar p` output. |
| S42 | Streamed install whole-stream retry | Empty client connects to two peers serving the same catalog-version `cnctw`: one broken source whose `--unrar` path is missing, followed by one good source. | The broken source sorts before the good source in retry order, contributes zero chunks, and the good source completes a fresh whole-stream attempt. The final state is local-only installed, no root archive/sentinel, no `.local.installing`, byte count matches the extracted entries, and payload hashes match the good source. |
| S43 | Already-installed streamed install rejection | A client first stream-installs `cnctw`, then attempts `stream-install cnctw` again. | The second request emits `download-failed`, does not emit a new success event, leaves the existing local-only install intact, and clears active operations. |
| S44 | Corrupt archive streamed install rollback | A source advertises catalog-version `cnctw`, but its root `.eti` is replaced with invalid bytes before the client requests `stream-install cnctw`. | The stream emits `download-failed`, does not emit download/install success, clears active operations, and leaves no `local/`, `.local.installing`, root archive, or root `version.ini` on the receiver. |
| S45 | Sender disconnect during streamed install | A source serves large catalog-version `alienswarm`; after the client receives the first streamed chunk, the source container is killed. | The operation reaches a terminal failure/peers-gone event, emits no download/install success, clears active operations, and rolls back local/staging state. |
| S46 | Receiver cancel during streamed install | A client starts streaming large catalog-version `alienswarm`, receives the first chunk, then sends `cancel-download alienswarm`. | The receiver cancels without emitting download/install success or a user-visible download failure, clears active operations, and rolls back local/staging state. |
| S47 | Multi-archive streamed install order | A source serves `fixture-multi/cnctw` with two root `.eti` archives named to require sorted processing. | Streamed chunk paths arrive in root archive sort order, both payloads install under `local/`, the receiver is local-only installed, and no root archives or sentinel are committed. |
## Version-Skew Contract ## Version-Skew Contract
Use S15-S17 to pin down what "newer" means when several peers have the same Use S15-S17 to pin down what happens when several peers have the same game ID
game ID: but only some match the local catalog version:
- Version comparison uses the eight-digit `version.ini` string, so use sortable - The receiver's catalog is authoritative. A remote root whose `version.ini`
`YYYYMMDD` values in manual fixtures. does not match the catalog's expected version for that game ID is not
downloadable.
- `list-games` aggregates by game ID. The game appears once; `peer_count` - `list-games` aggregates by game ID. The game appears once; `peer_count`
counts all ready peers with that ID, including peers that only have older counts only ready peers with that ID and the catalog version.
versions. - The aggregated `eti_game_version` must be the catalog version.
- The aggregated `eti_game_version` must be the newest ready version.
- The descriptor set emitted to the download path, file-size validation, and - The descriptor set emitted to the download path, file-size validation, and
transfer planning are latest-only. Older-version peers may be queried by a transfer planning are catalog-version-only. Stale peers must not supply
generic detail request, but their descriptors must not supply download download descriptors, majority votes, or chunks.
descriptors, majority votes, or chunks once a newer version exists. - If exactly one peer has the catalog version, that peer is the only transfer
- If exactly one peer has the latest version, that peer is the only transfer source. If several peers match the catalog version, validation and chunk
source. If several peers tie on the latest version, validation and chunk fanout happen among that catalog-version set only.
fanout happen among that latest-version set only.
- Capture proof with the `list-games` row, `got-game-files` descriptors, - Capture proof with the `list-games` row, `got-game-files` descriptors,
`download-chunk-finished` source addresses, and source/receiver SHA-256 `download-chunk-finished` source addresses, and source/receiver SHA-256
manifests. manifests.
@@ -80,7 +86,7 @@ GUI:
payload files may remain, but they must not be advertised as a ready local payload files may remain, but they must not be advertised as a ready local
game and must not leave an active operation stuck. game and must not leave an active operation stuck.
- Source failure during a redundant download should retry failed chunks against - Source failure during a redundant download should retry failed chunks against
another validated source for the same latest-version file. another validated source for the same catalog-version file.
- Live local library changes are observable by connected peers through library - Live local library changes are observable by connected peers through library
deltas; reconnect is not required for add, remove, or version-bump cases. deltas; reconnect is not required for add, remove, or version-bump cases.
- Same-game operations are single-flight. A duplicate download request while a - Same-game operations are single-flight. A duplicate download request while a
@@ -89,10 +95,10 @@ GUI:
are not downloadable. are not downloadable.
For a manual run, prefer a catalog game ID already served by the fixture lab, For a manual run, prefer a catalog game ID already served by the fixture lab,
such as `cnc4`, then create temporary `just peer-cli-run` game roots with such as `cnc4`, then create temporary `just peer-cli-run` game roots where some
different `version.ini` contents. The existing alpha/bravo/charlie fixtures peers match the catalog version and others deliberately use stale
cover duplicate-source and shared-game cases, but not the three-version skew `version.ini` contents. The existing alpha/bravo/charlie fixtures cover
until a dedicated fixture or temporary games root is prepared. duplicate-source and shared-game cases; S15-S17 add the focused skew cases.
## First-Play Launch-Setting Contract ## First-Play Launch-Setting Contract
@@ -112,8 +118,118 @@ Use S38 to pin down how launcher settings are stamped into an installed game:
crate's `launch_settings` unit tests cover the rewrite, line-ending, and crate's `launch_settings` unit tests cover the rewrite, line-ending, and
marker logic deterministically. marker logic deterministically.
## Streamed Install Archive Contract
Use S39-S41 to pin down low-disk streamed installs:
- The stream provider performs one archive metadata pass and one payload pass
per `.eti`, then frames entry boundaries for the receiver.
- Non-solid and solid archives both install into `local/` without committing a
root archive or root `version.ini`, so the receiver is installed but not a
downloadable source.
- Streamed install integrity is currently sender archive integrity: size and
RAR CRC32 must match the sender's archive metadata. The SHA-256 checks in the
scenarios prove the Docker/provider path matches the source fixture; they are
not catalog-owned trust anchors.
- S41 verifies the fixture is actually solid inside the source container, so
solid handling stays covered by the same Docker harness as the existing
streamed-install scenarios.
- S42 verifies retry/resume semantics: failed streamed attempts roll back their
staging directory and retry the whole stream from another validated peer.
There is no byte-offset resume contract.
- S43-S47 cover the remaining streamed-install failure and archive-shape edges:
already-installed rejection, corrupt archive rollback, sender disconnect,
receiver cancel, and multi-archive root sorting.
## Run Log ## Run Log
### 2026-06-07 - Catalog-Version Matrix Alignment (S1-S47)
- Code under test aligned checked-in fixture `version.ini` sentinels with the
catalog, made `run_extended_scenarios.py` stamp generated fixture games with
catalog versions by default, updated S15-S17/S23/S30/S36/S37 to assert
catalog-authoritative aggregation, and wired S38 into the executable matrix.
- Gates before Docker: `python3 -m py_compile
crates/lanspread-peer-cli/scripts/run_extended_scenarios.py` passed.
- Targeted rebuilt-image runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S3 S8 S14 S15 S16 S17 S21 S22 S23 S24 S29 S30 S31 S34 S36 S37 S39 S40 S41 S42 S43 S44 S45 S46 S47 --build-image`
passed.
- S38 standalone runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S38`
passed, proving the real-RAR `css` fixture installs with the container
`/usr/local/bin/unrar` sidecar and stamps launch settings only once.
- Full matrix runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py`
passed for S1-S47 against the rebuilt `lanspread-peer-cli:dev` image.
- The final full-run highlights included S3 aggregation, S15-S17
catalog-version skew/fanout/conflict, S23 stale-to-catalog propagation, S30
mesh aggregation, S36 catalog singleton over stale majority, S37 throughput,
S38 first-play stamping, and S39-S47 streamed-install coverage.
### 2026-06-07 - Streamed Install Edge Coverage (S43-S47)
- Code under test added `cancel-download` to `lanspread-peer-cli`, added the
tiny `fixture-multi/cnctw` two-archive fixture, and added S43-S47 in
`run_extended_scenarios.py`.
- Gates before Docker: `just fmt` and `python3 -m py_compile
crates/lanspread-peer-cli/scripts/run_extended_scenarios.py` passed.
- Runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S43 S44 S45 S46 S47 --build-image`
passed against the rebuilt `lanspread-peer-cli:dev` image.
- S43 stream-installed `cnctw`, retried `stream-install cnctw`, observed
`download-failed`, and verified the existing local-only install stayed intact.
- S44 replaced the source `cnctw.eti` with invalid bytes. The receiver emitted
`download-failed`, cleared active operations, and left no `local/`,
`.local.installing`, root archive, or root `version.ini`.
- S45 killed the sole `alienswarm` source after the first streamed chunk. The
receiver ended with `download-failed`, emitted no success, cleared active
operations, and rolled back local/staging state.
- S46 cancelled `alienswarm` on the receiver after the first streamed chunk.
The receiver emitted no success and no user-visible `download-failed`, cleared
active operations, and rolled back local/staging state.
- S47 streamed `fixture-multi/cnctw` and observed chunk paths in sorted root
archive order: `cnctw/.local.installing/order/first.txt`, then
`cnctw/.local.installing/order/second.txt`.
### 2026-06-07 - Streamed Install Whole-Stream Retry (S42)
- Code under test added S42 in `run_extended_scenarios.py`.
- Gates before Docker: `python3 -m py_compile
crates/lanspread-peer-cli/scripts/run_extended_scenarios.py` passed.
- Runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S42`
passed against the current `lanspread-peer-cli:dev` image.
- S42 started a broken source with `--unrar /missing-unrar` and a good source
with the same catalog-version `cnctw` metadata. The broken source sorted first
(`10.66.0.2:32897`) and the good source second (`10.66.0.3:34092`).
- The broken source contributed zero chunks; the good source completed the fresh
whole-stream attempt with `3145728` streamed file bytes.
- The final client state was `downloaded=false`, `installed=true`,
`availability=LocalOnly`, with no root `version.ini`, no root `cnctw.eti`,
and no `.local.installing` staging directory. Payload SHA-256 hashes matched
the good source's `unrar p` output.
### 2026-06-07 - Solid Streamed Install Coverage (S41)
- Code under test added `fixture-solid/cnctw`, a real solid RAR `.eti`, plus
S41 in `run_extended_scenarios.py`.
- Gates before Docker: `just fmt`, `git diff --check`, and
`python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py`
passed.
- Runner:
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image`
passed against the rebuilt `lanspread-peer-cli:dev` image.
- S41 verified the source archive with `unrar lt -cfg-` inside the source
container; the archive reported `Details: RAR 5, solid`.
- The streamed install finished with `downloaded=false`, `installed=true`,
`availability=LocalOnly`, no root `version.ini`, and no root `cnctw.eti`.
- The client received `118` streamed file bytes, matching the extracted solid
entries. Payload SHA-256 hashes matched `unrar p` output:
`88764c9a6c9b5b846b4323cf7725cb7fd70766ddd7fba4168332804a839fa193`
(`bin/cnctw-solid-payload.bin`) and
`44afc308269b2381b7c707a056dd8d9d393274108ac4d880237fa6772c861d7a`
(`data/cnctw-solid-assets.dat`).
### 2026-06-07 - Streamed Install Prototype (S39-S40) ### 2026-06-07 - Streamed Install Prototype (S39-S40)
- Code under test added `stream-install` to `lanspread-peer-cli`, a peer - Code under test added `stream-install` to `lanspread-peer-cli`, a peer
-3
View File
@@ -14,14 +14,11 @@ path = "src/main.rs"
lanspread-compat = { path = "../lanspread-compat" } lanspread-compat = { path = "../lanspread-compat" }
lanspread-db = { path = "../lanspread-db" } lanspread-db = { path = "../lanspread-db" }
lanspread-peer = { path = "../lanspread-peer" } lanspread-peer = { path = "../lanspread-peer" }
lanspread-proto = { path = "../lanspread-proto" }
bytes = { workspace = true }
eyre = { workspace = true } eyre = { workspace = true }
serde = { workspace = true } serde = { workspace = true }
serde_json = { workspace = true } serde_json = { workspace = true }
tokio = { workspace = true } tokio = { workspace = true }
tokio-util = { workspace = true }
[lints.clippy] [lints.clippy]
needless_pass_by_value = "allow" needless_pass_by_value = "allow"
@@ -1 +1 @@
20250101 20190317
@@ -1 +1 @@
20250103 20160130
@@ -1 +1 @@
20250102 20200721
@@ -1 +1 @@
20250201 20210416
@@ -1 +1 @@
20250202 20170204
@@ -1 +1 @@
20250203 20160128
@@ -1 +1 @@
20250102 20200721
@@ -1 +1 @@
20250202 20170204
@@ -1 +1 @@
20250301 20160920
@@ -1 +1 @@
20250302 20200315
@@ -1 +1 @@
20250303 20200907
@@ -0,0 +1 @@
20160128
@@ -1 +1 @@
20250101 20240623
@@ -0,0 +1 @@
20160128
@@ -1,10 +1,11 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Run the peer-cli scenarios S1-S40 through Docker.""" """Run the peer-cli scenarios S1-S47 through Docker."""
from __future__ import annotations from __future__ import annotations
import argparse import argparse
import hashlib import hashlib
import ipaddress
import json import json
import os import os
import queue import queue
@@ -27,7 +28,20 @@ CONTAINER_PREFIX = "lanspread-peer-cli-ext"
CATALOG_DB = "/app/game.db" CATALOG_DB = "/app/game.db"
FIXTURES = REPO / "crates" / "lanspread-peer-cli" / "fixtures" FIXTURES = REPO / "crates" / "lanspread-peer-cli" / "fixtures"
CHUNK_SIZE = 128 * 1024 * 1024 CHUNK_SIZE = 128 * 1024 * 1024
CATALOG_VERSIONS = {
"alienswarm": "20190317",
"bf1942": "20160130",
"bfbc2": "20210416",
"cnc4": "20170204",
"cnctw": "20160128",
"cod5": "20160920",
"cod6": "20200315",
"coh": "20200907",
"css": "20240623",
"ggoo": "20200721",
}
PERF_GAME_ID = "bf1942" PERF_GAME_ID = "bf1942"
PERF_GAME_VERSION = CATALOG_VERSIONS[PERF_GAME_ID]
PERF_GAME_SIZE = 2 * 1024 * 1024 * 1024 PERF_GAME_SIZE = 2 * 1024 * 1024 * 1024
IGNORED_DIFF_NAMES = {".lanspread", ".lanspread.json", "local"} IGNORED_DIFF_NAMES = {".lanspread", ".lanspread.json", "local"}
@@ -304,8 +318,8 @@ class Runner:
("S13", self.s13_exact_transfer_equality), ("S13", self.s13_exact_transfer_equality),
("S14", self.s14_large_multi_peer_chunking), ("S14", self.s14_large_multi_peer_chunking),
("S15", self.s15_three_way_version_skew), ("S15", self.s15_three_way_version_skew),
("S16", self.s16_latest_fanout_with_stale), ("S16", self.s16_catalog_fanout_with_stale),
("S17", self.s17_latest_conflict_rejection), ("S17", self.s17_catalog_conflict_rejection),
("S18", self.s18_redundant_source_drop), ("S18", self.s18_redundant_source_drop),
("S19", self.s19_sole_source_drop), ("S19", self.s19_sole_source_drop),
("S20", self.s20_receiver_write_failure), ("S20", self.s20_receiver_write_failure),
@@ -324,10 +338,18 @@ class Runner:
("S33", self.s33_install_after_mutation), ("S33", self.s33_install_after_mutation),
("S34", self.s34_many_small_files), ("S34", self.s34_many_small_files),
("S35", self.s35_unknown_game_filtered), ("S35", self.s35_unknown_game_filtered),
("S36", self.s36_latest_singleton), ("S36", self.s36_catalog_singleton),
("S37", self.s37_single_source_download_throughput), ("S37", self.s37_single_source_download_throughput),
("S38", self.s38_first_play_launch_settings),
("S39", self.s39_streamed_install_local_only), ("S39", self.s39_streamed_install_local_only),
("S40", self.s40_streamed_receiver_not_source), ("S40", self.s40_streamed_receiver_not_source),
("S41", self.s41_solid_archive_streamed_install),
("S42", self.s42_streamed_install_retries_next_source),
("S43", self.s43_streamed_install_rejects_installed_game),
("S44", self.s44_corrupt_stream_rolls_back),
("S45", self.s45_sender_disconnect_mid_stream),
("S46", self.s46_receiver_cancel_mid_stream),
("S47", self.s47_multi_archive_streams_in_sorted_order),
] ]
for scenario_id, scenario in scenarios: for scenario_id, scenario in scenarios:
@@ -512,20 +534,20 @@ class Runner:
def s8_ambiguous_metadata_rejection(self) -> str: def s8_ambiguous_metadata_rejection(self) -> str:
dir_a = self.fixture_root / "s8-a" dir_a = self.fixture_root / "s8-a"
dir_b = self.fixture_root / "s8-b" dir_b = self.fixture_root / "s8-b"
copy_game("ggoo", dir_a, version="20260101") copy_game("ggoo", dir_a)
copy_game("ggoo", dir_b, version="20260101") copy_game("ggoo", dir_b)
with (dir_b / "ggoo" / "ggoo.eti").open("ab") as handle: with (dir_b / "ggoo" / "ggoo.eti").open("ab") as handle:
handle.write(b"conflict") handle.write(b"conflict")
peer_a = self.peer("s8-a", games_dir=dir_a) peer_a = self.peer("s8-a", games_dir=dir_a)
peer_b = self.peer("s8-b", games_dir=dir_b) peer_b = self.peer("s8-b", games_dir=dir_b)
client = self.peer("s8-client") client = self.peer("s8-client")
connect_many(client, [peer_a, peer_b]) connect_many(client, [peer_a, peer_b])
wait_remote_game(client, "ggoo", peer_count=2, version="20260101") wait_remote_game(client, "ggoo", peer_count=2, version=CATALOG_VERSIONS["ggoo"])
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": "ggoo", "install": False}) client.send({"cmd": "download", "game_id": "ggoo", "install": False})
client.wait_for(event_is("download-failed", "ggoo"), timeout=30, description="ggoo failed", waiter=waiter) client.wait_for(event_is("download-failed", "ggoo"), timeout=30, description="ggoo failed", waiter=waiter)
assert_not_exists(client.host_games_dir / "ggoo" / "version.ini") assert_not_exists(client.host_games_dir / "ggoo" / "version.ini")
return "conflicting latest ggoo file sizes emitted download-failed and left no version.ini" return "conflicting catalog-version ggoo file sizes emitted download-failed and left no version.ini"
def s9_missing_game(self) -> str: def s9_missing_game(self) -> str:
client = self.peer("s9-client") client = self.peer("s9-client")
@@ -607,7 +629,7 @@ class Runner:
diff_game_dirs(source_dir / game_id, stage.host_games_dir / game_id) diff_game_dirs(source_dir / game_id, stage.host_games_dir / game_id)
client = self.peer("s14-client") client = self.peer("s14-client")
connect_many(client, [alpha, stage]) connect_many(client, [alpha, stage])
wait_remote_game(client, game_id, peer_count=2, version="20260520") wait_remote_game(client, game_id, peer_count=2, version=PERF_GAME_VERSION)
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": game_id, "install": False}) client.send({"cmd": "download", "game_id": game_id, "install": False})
client.wait_for(event_is("download-finished", game_id), timeout=90, description="client finish", waiter=waiter) client.wait_for(event_is("download-finished", game_id), timeout=90, description="client finish", waiter=waiter)
@@ -621,7 +643,11 @@ class Runner:
return f"{game_id} downloaded from two sources, diff matched, chunk totals={totals}" return f"{game_id} downloaded from two sources, diff matched, chunk totals={totals}"
def s15_three_way_version_skew(self) -> str: def s15_three_way_version_skew(self) -> str:
specs = [("s15-a", "20250101"), ("s15-b", "20250201"), ("s15-c", "20250301")] specs = [
("s15-a", "20150101"),
("s15-b", "20160101"),
("s15-c", CATALOG_VERSIONS["cnc4"]),
]
peers = [] peers = []
for name, version in specs: for name, version in specs:
game_dir = self.fixture_root / name game_dir = self.fixture_root / name
@@ -629,19 +655,19 @@ class Runner:
peers.append(self.peer(name, games_dir=game_dir)) peers.append(self.peer(name, games_dir=game_dir))
client = self.peer("s15-client") client = self.peer("s15-client")
connect_many(client, peers) connect_many(client, peers)
wait_remote_game(client, "cnc4", peer_count=3, version="20250301") wait_remote_game(client, "cnc4", peer_count=1, version=CATALOG_VERSIONS["cnc4"])
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": "cnc4", "install": False}) client.send({"cmd": "download", "game_id": "cnc4", "install": False})
client.wait_for(event_is("download-finished", "cnc4"), timeout=60, description="cnc4 finish", waiter=waiter) client.wait_for(event_is("download-finished", "cnc4"), timeout=60, description="cnc4 finish", waiter=waiter)
assert_only_chunk_sources(client, "cnc4", {peers[2].ready_addr}) assert_only_chunk_sources(client, "cnc4", {peers[2].ready_addr})
diff_game_dirs(peers[2].host_games_dir / "cnc4", client.host_games_dir / "cnc4") diff_game_dirs(peers[2].host_games_dir / "cnc4", client.host_games_dir / "cnc4")
return "three-way skew selected only 20250301 peer and receiver diffed cleanly" return "three-way skew exposed only the catalog-version peer and receiver diffed cleanly"
def s16_latest_fanout_with_stale(self) -> str: def s16_catalog_fanout_with_stale(self) -> str:
specs = [ specs = [
("s16-a", "20250101"), ("s16-a", "20180101"),
("s16-b", "20250301"), ("s16-b", CATALOG_VERSIONS["alienswarm"]),
("s16-c", "20250301"), ("s16-c", CATALOG_VERSIONS["alienswarm"]),
] ]
peers = [] peers = []
for name, version in specs: for name, version in specs:
@@ -650,7 +676,7 @@ class Runner:
peers.append(self.peer(name, games_dir=game_dir)) peers.append(self.peer(name, games_dir=game_dir))
client = self.peer("s16-client") client = self.peer("s16-client")
connect_many(client, peers) connect_many(client, peers)
wait_remote_game(client, "alienswarm", peer_count=3, version="20250301") wait_remote_game(client, "alienswarm", peer_count=2, version=CATALOG_VERSIONS["alienswarm"])
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": "alienswarm", "install": False}) client.send({"cmd": "download", "game_id": "alienswarm", "install": False})
client.wait_for(event_is("download-finished", "alienswarm"), timeout=90, description="alienswarm finish", waiter=waiter) client.wait_for(event_is("download-finished", "alienswarm"), timeout=90, description="alienswarm finish", waiter=waiter)
@@ -659,13 +685,13 @@ class Runner:
if peers[0].ready_addr in totals: if peers[0].ready_addr in totals:
raise ScenarioError(f"stale peer contributed chunks: {totals}") raise ScenarioError(f"stale peer contributed chunks: {totals}")
diff_game_dirs(peers[1].host_games_dir / "alienswarm", client.host_games_dir / "alienswarm") diff_game_dirs(peers[1].host_games_dir / "alienswarm", client.host_games_dir / "alienswarm")
return f"latest B/C peers served alienswarm while stale A contributed zero; totals={totals}" return f"catalog-version B/C peers served alienswarm while stale A contributed zero; totals={totals}"
def s17_latest_conflict_rejection(self) -> str: def s17_catalog_conflict_rejection(self) -> str:
specs = [ specs = [
("s17-a", "20250101", False), ("s17-a", "20150101", False),
("s17-b", "20250301", False), ("s17-b", CATALOG_VERSIONS["cnc4"], False),
("s17-c", "20250301", True), ("s17-c", CATALOG_VERSIONS["cnc4"], True),
] ]
peers = [] peers = []
for name, version, conflict in specs: for name, version, conflict in specs:
@@ -677,12 +703,12 @@ class Runner:
peers.append(self.peer(name, games_dir=game_dir)) peers.append(self.peer(name, games_dir=game_dir))
client = self.peer("s17-client") client = self.peer("s17-client")
connect_many(client, peers) connect_many(client, peers)
wait_remote_game(client, "cnc4", peer_count=3, version="20250301") wait_remote_game(client, "cnc4", peer_count=2, version=CATALOG_VERSIONS["cnc4"])
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": "cnc4", "install": False}) client.send({"cmd": "download", "game_id": "cnc4", "install": False})
client.wait_for(event_is("download-failed", "cnc4"), timeout=30, description="cnc4 failed", waiter=waiter) client.wait_for(event_is("download-failed", "cnc4"), timeout=30, description="cnc4 failed", waiter=waiter)
assert_not_exists(client.host_games_dir / "cnc4" / "version.ini") assert_not_exists(client.host_games_dir / "cnc4" / "version.ini")
return "latest-version file conflict failed download and left no committed version.ini" return "catalog-version file conflict failed download and left no committed version.ini"
def s18_redundant_source_drop(self) -> str: def s18_redundant_source_drop(self) -> str:
source_a_dir = self.fixture_root / "s18-a" source_a_dir = self.fixture_root / "s18-a"
@@ -765,13 +791,13 @@ class Runner:
def s23_version_bump_propagation(self) -> str: def s23_version_bump_propagation(self) -> str:
alpha = self.peer("s23-alpha") alpha = self.peer("s23-alpha")
bravo_dir = self.fixture_root / "s23-bravo" bravo_dir = self.fixture_root / "s23-bravo"
copy_game("cnc4", bravo_dir, version="20250101") copy_game("cnc4", bravo_dir, version="20160101")
bravo = self.peer("s23-bravo", games_dir=bravo_dir) bravo = self.peer("s23-bravo", games_dir=bravo_dir)
connect_many(alpha, [bravo]) connect_many(alpha, [bravo])
wait_remote_game(alpha, "cnc4", peer_count=1, version="20250101") wait_remote_absent(alpha, "cnc4", timeout=5)
(bravo_dir / "cnc4" / "version.ini").write_text("20260501", encoding="utf-8") (bravo_dir / "cnc4" / "version.ini").write_text(CATALOG_VERSIONS["cnc4"], encoding="utf-8")
wait_remote_game(alpha, "cnc4", peer_count=1, version="20260501") wait_remote_game(alpha, "cnc4", peer_count=1, version=CATALOG_VERSIONS["cnc4"])
return "alpha observed cnc4 eti_game_version change 20250101 -> 20260501 without reconnect" return "alpha observed stale cnc4 become catalog-version downloadable without reconnect"
def s24_two_clients_one_source(self) -> str: def s24_two_clients_one_source(self) -> str:
source = self.peer("s24-alpha", games_dir=FIXTURES / "fixture-alpha", readonly_games=True) source = self.peer("s24-alpha", games_dir=FIXTURES / "fixture-alpha", readonly_games=True)
@@ -868,11 +894,11 @@ class Runner:
def s30_mesh_aggregation(self) -> str: def s30_mesh_aggregation(self) -> str:
dirs = [] dirs = []
specs = [ specs = [
("s30-a", [("ggoo", "20250101"), ("bf1942", "20250101")]), ("s30-a", [("ggoo", CATALOG_VERSIONS["ggoo"]), ("bf1942", CATALOG_VERSIONS["bf1942"])]),
("s30-b", [("ggoo", "20250101"), ("cnc4", "20250101")]), ("s30-b", [("ggoo", CATALOG_VERSIONS["ggoo"]), ("cnc4", CATALOG_VERSIONS["cnc4"])]),
("s30-c", [("cnc4", "20250301"), ("cod5", "20250101")]), ("s30-c", [("cnc4", CATALOG_VERSIONS["cnc4"]), ("cod5", CATALOG_VERSIONS["cod5"])]),
("s30-d", [("cnctw", "20250101"), ("coh", "20250101")]), ("s30-d", [("cnctw", CATALOG_VERSIONS["cnctw"]), ("coh", CATALOG_VERSIONS["coh"])]),
("s30-e", [("cnctw", "20250201"), ("bf1942", "20250201")]), ("s30-e", [("cnctw", CATALOG_VERSIONS["cnctw"]), ("bf1942", CATALOG_VERSIONS["bf1942"])]),
] ]
peers = [] peers = []
for name, games in specs: for name, games in specs:
@@ -884,12 +910,12 @@ class Runner:
client = self.peer("s30-client") client = self.peer("s30-client")
connect_many(client, peers) connect_many(client, peers)
expected = { expected = {
"ggoo": (2, "20250101"), "ggoo": (2, CATALOG_VERSIONS["ggoo"]),
"bf1942": (2, "20250201"), "bf1942": (2, CATALOG_VERSIONS["bf1942"]),
"cnc4": (2, "20250301"), "cnc4": (2, CATALOG_VERSIONS["cnc4"]),
"cod5": (1, "20250101"), "cod5": (1, CATALOG_VERSIONS["cod5"]),
"cnctw": (2, "20250201"), "cnctw": (2, CATALOG_VERSIONS["cnctw"]),
"coh": (1, "20250101"), "coh": (1, CATALOG_VERSIONS["coh"]),
} }
for game_id, (peer_count, version) in expected.items(): for game_id, (peer_count, version) in expected.items():
wait_remote_game(client, game_id, peer_count=peer_count, version=version) wait_remote_game(client, game_id, peer_count=peer_count, version=version)
@@ -899,7 +925,7 @@ class Runner:
raise ScenarioError(f"duplicate game rows: {ids}") raise ScenarioError(f"duplicate game rows: {ids}")
if any(peer["peer_id"] == client.peer_id for peer in client.list_peers()): if any(peer["peer_id"] == client.peer_id for peer in client.list_peers()):
raise ScenarioError("client listed itself as a peer") raise ScenarioError("client listed itself as a peer")
return f"client aggregated {len(expected)} IDs from 5 peers with expected peer_count/latest versions" return f"client aggregated {len(expected)} IDs from 5 peers with expected peer_count/catalog versions"
def s31_bootstrapped_peer_source(self) -> str: def s31_bootstrapped_peer_source(self) -> str:
source = self.peer("s31-alpha", games_dir=FIXTURES / "fixture-alpha", readonly_games=True) source = self.peer("s31-alpha", games_dir=FIXTURES / "fixture-alpha", readonly_games=True)
@@ -995,34 +1021,34 @@ class Runner:
assert_not_exists(client.host_games_dir / "mystery-game") assert_not_exists(client.host_games_dir / "mystery-game")
return f"unknown game absent from list-games; download errored '{err['error']}'; no local files" return f"unknown game absent from list-games; download errored '{err['error']}'; no local files"
def s36_latest_singleton(self) -> str: def s36_catalog_singleton(self) -> str:
peers = [] peers = []
for index in range(5): for index in range(5):
game_dir = self.fixture_root / f"s36-{index}" game_dir = self.fixture_root / f"s36-{index}"
version = "20260501" if index == 0 else "20250101" version = CATALOG_VERSIONS["cnc4"] if index == 0 else "20160101"
copy_game("cnc4", game_dir, version=version) copy_game("cnc4", game_dir, version=version)
peers.append(self.peer(f"s36-{index}", games_dir=game_dir)) peers.append(self.peer(f"s36-{index}", games_dir=game_dir))
client = self.peer("s36-client") client = self.peer("s36-client")
connect_many(client, peers) connect_many(client, peers)
wait_remote_game(client, "cnc4", peer_count=5, version="20260501") wait_remote_game(client, "cnc4", peer_count=1, version=CATALOG_VERSIONS["cnc4"])
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": "cnc4", "install": False}) client.send({"cmd": "download", "game_id": "cnc4", "install": False})
got = client.wait_for(event_is("got-game-files", "cnc4"), timeout=20, description="got game files", waiter=waiter) got = client.wait_for(event_is("got-game-files", "cnc4"), timeout=20, description="got game files", waiter=waiter)
client.wait_for(event_is("download-finished", "cnc4"), timeout=60, description="download finish", waiter=waiter) client.wait_for(event_is("download-finished", "cnc4"), timeout=60, description="download finish", waiter=waiter)
latest_addr = peers[0].ready_addr catalog_addr = peers[0].ready_addr
if latest_addr is None: if catalog_addr is None:
raise ScenarioError("latest peer had no ready addr") raise ScenarioError("catalog-version peer had no ready addr")
for item in client.output: for item in client.output:
if item.get("type") != "event" or item.get("event") != "download-chunk-finished": if item.get("type") != "event" or item.get("event") != "download-chunk-finished":
continue continue
data = item["data"] data = item["data"]
if data.get("game_id") == "cnc4" and data.get("peer_addr") != latest_addr: if data.get("game_id") == "cnc4" and data.get("peer_addr") != catalog_addr:
raise ScenarioError(f"stale peer contributed chunk: {data}") raise ScenarioError(f"stale peer contributed chunk: {data}")
diff_game_dirs(peers[0].host_games_dir / "cnc4", client.host_games_dir / "cnc4") diff_game_dirs(peers[0].host_games_dir / "cnc4", client.host_games_dir / "cnc4")
descs = got["data"]["file_descriptions"] descs = got["data"]["file_descriptions"]
if not descs: if not descs:
raise ScenarioError("got-game-files had no descriptors") raise ScenarioError("got-game-files had no descriptors")
return "client reported latest 20260501 with peer_count=5; only singleton latest peer sent chunks; diff matched" return "client reported singleton catalog-version peer; stale peers stayed hidden and sent no chunks; diff matched"
def s37_single_source_download_throughput(self) -> str: def s37_single_source_download_throughput(self) -> str:
source_dir = self.fixture_root / "s37-source" source_dir = self.fixture_root / "s37-source"
@@ -1030,7 +1056,7 @@ class Runner:
source = self.peer("s37-source", games_dir=source_dir) source = self.peer("s37-source", games_dir=source_dir)
client = self.peer("s37-client") client = self.peer("s37-client")
connect_many(client, [source]) connect_many(client, [source])
wait_remote_game(client, PERF_GAME_ID, peer_count=1, version="20260520") wait_remote_game(client, PERF_GAME_ID, peer_count=1, version=PERF_GAME_VERSION)
waiter = LineWaiter(len(client.output)) waiter = LineWaiter(len(client.output))
client.send({"cmd": "download", "game_id": PERF_GAME_ID, "install": False}) client.send({"cmd": "download", "game_id": PERF_GAME_ID, "install": False})
@@ -1049,7 +1075,7 @@ class Runner:
throughput = finished.get("data", {}).get("throughput") throughput = finished.get("data", {}).get("throughput")
if not throughput: if not throughput:
raise ScenarioError(f"download-finished did not include throughput: {finished}") raise ScenarioError(f"download-finished did not include throughput: {finished}")
expected_bytes = PERF_GAME_SIZE + len("20260520") expected_bytes = PERF_GAME_SIZE + len(PERF_GAME_VERSION)
if int(throughput["bytes"]) != expected_bytes: if int(throughput["bytes"]) != expected_bytes:
raise ScenarioError( raise ScenarioError(
f"throughput byte count mismatch: {throughput['bytes']} != {expected_bytes}" f"throughput byte count mismatch: {throughput['bytes']} != {expected_bytes}"
@@ -1063,6 +1089,116 @@ class Runner:
f"{throughput['chunks']} chunks" f"{throughput['chunks']} chunks"
) )
def s38_first_play_launch_settings(self) -> str:
client_dir = self.fixture_root / "s38-client"
copy_game("css", client_dir)
client = self.peer(
"s38-client",
games_dir=client_dir,
extra_args=["--unrar", "/usr/local/bin/unrar"],
)
waiter = LineWaiter(len(client.output))
client.send({"cmd": "install", "game_id": "css"})
client.wait_for(
event_is("install-finished", "css"),
timeout=30,
description="css install",
waiter=waiter,
)
wait_local_game(client, "css", downloaded=True, installed=True)
marker = client.host_state_dir / "games" / "css" / "launch_settings_applied"
if marker.exists():
raise ScenarioError("launch settings marker existed before first play")
local_root = client.host_games_dir / "css" / "local"
account_file = local_root / "profiles" / "local" / "account_name.txt"
language_file = local_root / "profiles" / "local" / "language.txt"
ini_file = (
local_root
/ "engine"
/ "bin"
/ "win64"
/ "steam_settings"
/ "SmartSteamEmu.ini"
)
for path in [account_file, language_file, ini_file]:
if not path.is_file():
raise ScenarioError(f"expected installed launch settings file: {path}")
if b"PersonaName = stubplayer\r\n" not in ini_file.read_bytes():
raise ScenarioError("installed SmartSteamEmu.ini did not preserve CRLF stub PersonaName")
first = client.send(
{
"cmd": "play",
"game_id": "css",
"username": "Lan Hero",
"language": "german",
}
)["data"]["outcome"]
expected_first = {
"already_applied": False,
"account_name_written": True,
"language_written": True,
"persona_name_written": True,
}
if first != expected_first:
raise ScenarioError(f"unexpected first play outcome: {first}")
if not marker.is_file():
raise ScenarioError("launch settings marker was not written after first play")
if account_file.read_text(encoding="utf-8") != "Lan Hero":
raise ScenarioError("account_name.txt was not stamped with username")
if language_file.read_text(encoding="utf-8") != "german":
raise ScenarioError("language.txt was not stamped with language")
stamped_ini = ini_file.read_bytes()
if b"PersonaName = Lan Hero\r\n" not in stamped_ini:
raise ScenarioError("PersonaName was not stamped with CRLF preserved")
if b"AppId = 240\r\n" not in stamped_ini or b"Language = english\r\n" not in stamped_ini:
raise ScenarioError("SmartSteamEmu.ini sibling lines were not preserved")
client.docker_exec(
"sh",
"-c",
"printf resetaccount > /games/css/local/profiles/local/account_name.txt",
)
client.docker_exec(
"sh",
"-c",
"printf resetlang > /games/css/local/profiles/local/language.txt",
)
client.docker_exec(
"sh",
"-c",
"printf '[Settings]\\r\\nAppId = 240\\r\\n"
"PersonaName = resetplayer\\r\\nLanguage = english\\r\\n' > "
"/games/css/local/engine/bin/win64/steam_settings/SmartSteamEmu.ini",
)
second = client.send(
{
"cmd": "play",
"game_id": "css",
"username": "Second User",
"language": "french",
}
)["data"]["outcome"]
expected_second = {
"already_applied": True,
"account_name_written": False,
"language_written": False,
"persona_name_written": False,
}
if second != expected_second:
raise ScenarioError(f"unexpected second play outcome: {second}")
if account_file.read_text(encoding="utf-8") != "resetaccount":
raise ScenarioError("second play rewrote account_name.txt despite marker")
if language_file.read_text(encoding="utf-8") != "resetlang":
raise ScenarioError("second play rewrote language.txt despite marker")
if b"PersonaName = resetplayer\r\n" not in ini_file.read_bytes():
raise ScenarioError("second play rewrote PersonaName despite marker")
return "css first play stamped launch settings once; second play respected the marker"
def stream_install_cnctw(self, prefix: str) -> tuple[Peer, Peer]: def stream_install_cnctw(self, prefix: str) -> tuple[Peer, Peer]:
source_dir = self.fixture_root / f"{prefix}-bravo" source_dir = self.fixture_root / f"{prefix}-bravo"
copy_game("cnctw", source_dir, version="20160128") copy_game("cnctw", source_dir, version="20160128")
@@ -1171,6 +1307,333 @@ class Runner:
f"and download errored '{err['error']}'" f"and download errored '{err['error']}'"
) )
def s41_solid_archive_streamed_install(self) -> str:
source_dir = self.fixture_root / "s41-solid-source"
source_game = source_dir / "cnctw"
shutil.copytree(FIXTURES / "fixture-solid" / "cnctw", source_game)
source = self.peer("s41-solid-source", games_dir=source_dir)
assert_peer_rar_archive_solid(source, "cnctw")
client = self.peer("s41-solid-client")
connect_many(client, [source])
wait_remote_game(client, "cnctw", peer_count=1, version="20160128")
waiter = LineWaiter(len(client.output))
client.send({"cmd": "stream-install", "game_id": "cnctw"})
client.wait_for(
event_is("got-game-files", "cnctw"),
timeout=20,
description="got solid cnctw files",
waiter=waiter,
)
client.wait_for(
event_is("download-finished", "cnctw"),
timeout=60,
description="solid stream finish cnctw",
waiter=waiter,
)
client.wait_for(
event_is("install-finished", "cnctw"),
timeout=30,
description="solid stream install cnctw",
waiter=waiter,
)
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
assert_game_state(
game,
downloaded=False,
installed=True,
availability="LocalOnly",
)
game_root = client.host_games_dir / "cnctw"
assert_not_exists(game_root / "version.ini")
assert_not_exists(game_root / "cnctw.eti")
expected = {
"bin/cnctw-solid-payload.bin": unrar_entry_sha256(
source, "cnctw", "bin/cnctw-solid-payload.bin"
),
"data/cnctw-solid-assets.dat": unrar_entry_sha256(
source, "cnctw", "data/cnctw-solid-assets.dat"
),
}
actual = {
rel: sha256_file(game_root / "local" / rel)
for rel in expected
}
if actual != expected:
raise ScenarioError(
f"solid streamed payload hashes mismatched: {actual} != {expected}"
)
streamed_bytes = sum(
int(item.get("data", {}).get("length", 0))
for item in client.output
if item.get("type") == "event"
and item.get("event") == "download-chunk-finished"
and item.get("data", {}).get("game_id") == "cnctw"
)
expected_bytes = sum((game_root / "local" / rel).stat().st_size for rel in expected)
if streamed_bytes != expected_bytes:
raise ScenarioError(
f"solid streamed byte count mismatch: {streamed_bytes} != {expected_bytes}"
)
return (
"solid cnctw archive streamed through one local-only install; "
f"payload hashes={actual}, bytes={streamed_bytes}"
)
def s42_streamed_install_retries_next_source(self) -> str:
bad_dir = self.fixture_root / "s42-bad-source"
good_dir = self.fixture_root / "s42-good-source"
copy_game("cnctw", bad_dir, version="20160128")
copy_game("cnctw", good_dir, version="20160128")
bad = self.peer(
"s42-bad-source",
games_dir=bad_dir,
extra_args=["--unrar", "/missing-unrar"],
)
good = self.peer("s42-good-source", games_dir=good_dir)
if socket_addr_sort_key(bad.ready_addr) > socket_addr_sort_key(good.ready_addr):
raise ScenarioError(
"S42 requires the broken source to sort before the good source; "
f"bad={bad.ready_addr}, good={good.ready_addr}"
)
client = self.peer("s42-client")
connect_many(client, [bad, good])
wait_remote_game(client, "cnctw", peer_count=2, version="20160128")
waiter = LineWaiter(len(client.output))
client.send({"cmd": "stream-install", "game_id": "cnctw"})
client.wait_for(
event_is("got-game-files", "cnctw"),
timeout=20,
description="got retry cnctw files",
waiter=waiter,
)
client.wait_for(
event_is("download-finished", "cnctw"),
timeout=60,
description="retry stream finish cnctw",
waiter=waiter,
)
client.wait_for(
event_is("install-finished", "cnctw"),
timeout=30,
description="retry stream install cnctw",
waiter=waiter,
)
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
assert_game_state(
game,
downloaded=False,
installed=True,
availability="LocalOnly",
)
game_root = client.host_games_dir / "cnctw"
assert_not_exists(game_root / ".local.installing")
assert_not_exists(game_root / "version.ini")
assert_not_exists(game_root / "cnctw.eti")
assert_only_chunk_sources(client, "cnctw", {good.ready_addr})
expected = {
"bin/cnctw-payload.bin": unrar_entry_sha256(
good, "cnctw", "bin/cnctw-payload.bin"
),
"data/cnctw-assets.dat": unrar_entry_sha256(
good, "cnctw", "data/cnctw-assets.dat"
),
}
actual = {
rel: sha256_file(game_root / "local" / rel)
for rel in expected
}
if actual != expected:
raise ScenarioError(f"retry streamed payload hashes mismatched: {actual} != {expected}")
streamed_bytes = sum(
int(item.get("data", {}).get("length", 0))
for item in client.output
if item.get("type") == "event"
and item.get("event") == "download-chunk-finished"
and item.get("data", {}).get("game_id") == "cnctw"
)
expected_bytes = 3 * 1024 * 1024
if streamed_bytes != expected_bytes:
raise ScenarioError(
f"retry streamed byte count mismatch: {streamed_bytes} != {expected_bytes}"
)
return (
"broken first source failed without chunks, next source completed whole stream; "
f"good={good.ready_addr}, bad={bad.ready_addr}, bytes={streamed_bytes}"
)
def s43_streamed_install_rejects_installed_game(self) -> str:
_source, client = self.stream_install_cnctw("s43")
start = len(client.output)
waiter = LineWaiter(start)
client.send({"cmd": "stream-install", "game_id": "cnctw"})
client.wait_for(
event_is("download-failed", "cnctw"),
timeout=20,
description="already-installed stream rejection",
waiter=waiter,
)
assert_no_event_since(client, start, "install-finished", "cnctw")
assert_no_event_since(client, start, "download-finished", "cnctw")
wait_no_active(client, "cnctw")
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
assert_game_state(
game,
downloaded=False,
installed=True,
availability="LocalOnly",
)
return "already-installed cnctw rejected a second streamed install without state drift"
def s44_corrupt_stream_rolls_back(self) -> str:
source_dir = self.fixture_root / "s44-corrupt-source"
copy_game("cnctw", source_dir, version="20160128")
(source_dir / "cnctw" / "cnctw.eti").write_bytes(b"not a rar archive")
source = self.peer("s44-corrupt-source", games_dir=source_dir)
client = self.peer("s44-client")
connect_many(client, [source])
wait_remote_game(client, "cnctw", peer_count=1, version="20160128")
start = len(client.output)
waiter = LineWaiter(start)
client.send({"cmd": "stream-install", "game_id": "cnctw"})
client.wait_for(
event_is("download-failed", "cnctw"),
timeout=30,
description="corrupt stream failed",
waiter=waiter,
)
assert_no_event_since(client, start, "download-finished", "cnctw")
assert_no_event_since(client, start, "install-finished", "cnctw")
wait_no_active(client, "cnctw")
assert_failed_stream_left_no_local(client, "cnctw")
return "corrupt cnctw archive emitted download-failed and left no local install"
def s45_sender_disconnect_mid_stream(self) -> str:
source_dir = self.fixture_root / "s45-source"
copy_game("alienswarm", source_dir, version="20190317")
source = self.peer("s45-source", games_dir=source_dir)
client = self.peer("s45-client")
connect_many(client, [source])
wait_remote_game(client, "alienswarm", peer_count=1, version="20190317")
start = len(client.output)
waiter = LineWaiter(start)
client.send({"cmd": "stream-install", "game_id": "alienswarm"})
client.wait_for(
event_is("download-chunk-finished", "alienswarm"),
timeout=30,
description="first alienswarm stream chunk before source drop",
waiter=waiter,
)
source.kill()
terminal = client.wait_for(
event_name_in({"download-failed", "download-peers-gone"}, "alienswarm"),
timeout=60,
description="sender disconnect terminal event",
waiter=waiter,
)
assert_no_event_since(client, start, "download-finished", "alienswarm")
assert_no_event_since(client, start, "install-finished", "alienswarm")
wait_no_active(client, "alienswarm")
assert_failed_stream_left_no_local(client, "alienswarm")
return (
"sender disconnect after first alienswarm chunk rolled back stream; "
f"terminal={terminal['event']}"
)
def s46_receiver_cancel_mid_stream(self) -> str:
source_dir = self.fixture_root / "s46-source"
copy_game("alienswarm", source_dir, version="20190317")
source = self.peer("s46-source", games_dir=source_dir)
client = self.peer("s46-client")
connect_many(client, [source])
wait_remote_game(client, "alienswarm", peer_count=1, version="20190317")
start = len(client.output)
waiter = LineWaiter(start)
client.send({"cmd": "stream-install", "game_id": "alienswarm"})
client.wait_for(
event_is("download-chunk-finished", "alienswarm"),
timeout=30,
description="first alienswarm stream chunk before receiver cancel",
waiter=waiter,
)
client.send({"cmd": "cancel-download", "game_id": "alienswarm"})
wait_no_active(client, "alienswarm", timeout=60)
assert_no_event_since(client, start, "download-finished", "alienswarm")
assert_no_event_since(client, start, "download-failed", "alienswarm")
assert_no_event_since(client, start, "install-finished", "alienswarm")
assert_failed_stream_left_no_local(client, "alienswarm")
return "receiver cancel after first alienswarm chunk rolled back without failed event"
def s47_multi_archive_streams_in_sorted_order(self) -> str:
source_dir = self.fixture_root / "s47-source"
source_game = source_dir / "cnctw"
shutil.copytree(FIXTURES / "fixture-multi" / "cnctw", source_game)
source = self.peer("s47-source", games_dir=source_dir)
client = self.peer("s47-client")
connect_many(client, [source])
wait_remote_game(client, "cnctw", peer_count=1, version="20160128")
waiter = LineWaiter(len(client.output))
client.send({"cmd": "stream-install", "game_id": "cnctw"})
client.wait_for(
event_is("download-finished", "cnctw"),
timeout=30,
description="multi-archive stream finish",
waiter=waiter,
)
client.wait_for(
event_is("install-finished", "cnctw"),
timeout=30,
description="multi-archive stream install",
waiter=waiter,
)
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
assert_game_state(
game,
downloaded=False,
installed=True,
availability="LocalOnly",
)
game_root = client.host_games_dir / "cnctw"
assert_not_exists(game_root / "version.ini")
assert_not_exists(game_root / "a-first.eti")
assert_not_exists(game_root / "z-second.eti")
chunk_paths = streamed_chunk_paths(client, "cnctw")
expected_paths = [
"cnctw/.local.installing/order/first.txt",
"cnctw/.local.installing/order/second.txt",
]
if chunk_paths != expected_paths:
raise ScenarioError(f"multi-archive stream order mismatch: {chunk_paths}")
first = (game_root / "local" / "order" / "first.txt").read_text(encoding="utf-8")
second = (game_root / "local" / "order" / "second.txt").read_text(encoding="utf-8")
if first != "first archive payload\n" or second != "second archive payload\n":
raise ScenarioError(f"multi-archive payload mismatch: {first!r}, {second!r}")
return f"multi-archive cnctw streamed in sorted order: {chunk_paths}"
def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]: def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]:
result = subprocess.run( result = subprocess.run(
@@ -1239,6 +1702,7 @@ def copy_game(game_id: str, destination_games_dir: Path, *, version: str | None
shutil.rmtree(destination) shutil.rmtree(destination)
destination.parent.mkdir(parents=True, exist_ok=True) destination.parent.mkdir(parents=True, exist_ok=True)
shutil.copytree(source, destination) shutil.copytree(source, destination)
version = version if version is not None else CATALOG_VERSIONS.get(game_id)
if version is not None: if version is not None:
(destination / "version.ini").write_text(version, encoding="utf-8") (destination / "version.ini").write_text(version, encoding="utf-8")
@@ -1275,14 +1739,14 @@ def create_many_small_game(root: Path) -> None:
for index in range(20): for index in range(20):
child = root / f"file-{index:02}.bin" child = root / f"file-{index:02}.bin"
child.write_bytes(hashlib.sha256(f"small-{index}".encode()).digest() * 8) child.write_bytes(hashlib.sha256(f"small-{index}".encode()).digest() * 8)
(root / "version.ini").write_text("20250101", encoding="utf-8") (root / "version.ini").write_text(CATALOG_VERSIONS.get(root.name, "20250101"), encoding="utf-8")
def create_large_sparse_game(root: Path, *, size: int) -> None: def create_large_sparse_game(root: Path, *, size: int) -> None:
if root.exists(): if root.exists():
shutil.rmtree(root) shutil.rmtree(root)
root.mkdir(parents=True) root.mkdir(parents=True)
(root / "version.ini").write_text("20260520", encoding="utf-8") (root / "version.ini").write_text(PERF_GAME_VERSION, encoding="utf-8")
archive = root / f"{root.name}.eti" archive = root / f"{root.name}.eti"
with archive.open("wb") as handle: with archive.open("wb") as handle:
handle.truncate(size) handle.truncate(size)
@@ -1307,6 +1771,30 @@ def unrar_entry_sha256(peer: Peer, game_id: str, relative_path: str) -> str:
return output.split()[0] return output.split()[0]
def assert_peer_rar_archive_solid(peer: Peer, game_id: str) -> None:
output = peer.docker_exec(
"unrar",
"lt",
"-cfg-",
f"/games/{game_id}/{game_id}.eti",
).stdout
for line in output.splitlines():
stripped = line.strip()
if stripped.startswith("Details:"):
if "solid" in stripped.lower():
return
raise ScenarioError(f"RAR archive is not solid: {game_id}")
raise ScenarioError(f"RAR archive details were not reported: {game_id}")
def socket_addr_sort_key(addr: str | None) -> tuple[int, int]:
if addr is None:
raise ScenarioError("cannot sort missing peer address")
host, port = addr.rsplit(":", 1)
host = host.removeprefix("[").removesuffix("]")
return (int(ipaddress.ip_address(host)), int(port))
def format_bytes(size: int) -> str: def format_bytes(size: int) -> str:
return f"{size / 1024 / 1024 / 1024:.2f} GiB" return f"{size / 1024 / 1024 / 1024:.2f} GiB"
@@ -1383,6 +1871,18 @@ def wait_local_game(
) )
def wait_no_active(peer: Peer, game_id: str, timeout: float = 20) -> None:
deadline = time.monotonic() + timeout
last_active: list[dict[str, Any]] = []
while time.monotonic() < deadline:
active = peer.status()["active_operations"]
last_active = active
if all(item["game_id"] != game_id for item in active):
return
time.sleep(0.4)
raise ScenarioError(f"{peer.name} still has active operation for {game_id}: {last_active}")
def assert_game_state( def assert_game_state(
game: dict[str, Any], game: dict[str, Any],
*, *,
@@ -1429,7 +1929,10 @@ def wait_peer_has_game(
def assert_local_absent(peer: Peer, game_id: str) -> None: def assert_local_absent(peer: Peer, game_id: str) -> None:
rows = peer.list_games()["local"] rows = peer.list_games()["local"]
if any(row["id"] == game_id and row.get("downloaded") for row in rows): if any(
row["id"] == game_id and (row.get("downloaded") or row.get("installed"))
for row in rows
):
raise ScenarioError(f"{peer.name} advertises failed local {game_id}: {rows}") raise ScenarioError(f"{peer.name} advertises failed local {game_id}: {rows}")
@@ -1445,6 +1948,15 @@ def assert_not_exists(path: Path) -> None:
raise ScenarioError(f"expected path to be absent: {path}") raise ScenarioError(f"expected path to be absent: {path}")
def assert_failed_stream_left_no_local(peer: Peer, game_id: str) -> None:
game_root = peer.host_games_dir / game_id
assert_local_absent(peer, game_id)
assert_not_exists(game_root / "local")
assert_not_exists(game_root / ".local.installing")
assert_not_exists(game_root / "version.ini")
assert_not_exists(game_root / f"{game_id}.eti")
def event_is(event: str, game_id: str | None = None) -> Callable[[dict[str, Any]], bool]: def event_is(event: str, game_id: str | None = None) -> Callable[[dict[str, Any]], bool]:
def predicate(item: dict[str, Any]) -> bool: def predicate(item: dict[str, Any]) -> bool:
if item.get("type") != "event" or item.get("event") != event: if item.get("type") != "event" or item.get("event") != event:
@@ -1456,6 +1968,17 @@ def event_is(event: str, game_id: str | None = None) -> Callable[[dict[str, Any]
return predicate return predicate
def event_name_in(events: set[str], game_id: str | None = None) -> Callable[[dict[str, Any]], bool]:
def predicate(item: dict[str, Any]) -> bool:
if item.get("type") != "event" or item.get("event") not in events:
return False
if game_id is None:
return True
return item.get("data", {}).get("game_id") == game_id
return predicate
def assert_no_event(peer: Peer, waiter: LineWaiter, event: str, game_id: str) -> None: def assert_no_event(peer: Peer, waiter: LineWaiter, event: str, game_id: str) -> None:
for item in peer.output[waiter.seen :]: for item in peer.output[waiter.seen :]:
if item.get("type") == "event" and item.get("event") == event: if item.get("type") == "event" and item.get("event") == event:
@@ -1463,6 +1986,13 @@ def assert_no_event(peer: Peer, waiter: LineWaiter, event: str, game_id: str) ->
raise ScenarioError(f"unexpected {event} for {game_id}: {item}") raise ScenarioError(f"unexpected {event} for {game_id}: {item}")
def assert_no_event_since(peer: Peer, start: int, event: str, game_id: str) -> None:
for item in peer.output[start:]:
if item.get("type") == "event" and item.get("event") == event:
if item.get("data", {}).get("game_id") == game_id:
raise ScenarioError(f"unexpected {event} for {game_id}: {item}")
def assert_only_chunk_sources( def assert_only_chunk_sources(
peer: Peer, peer: Peer,
game_id: str, game_id: str,
@@ -1488,6 +2018,16 @@ def assert_only_chunk_sources(
raise ScenarioError(f"no chunk events recorded for {game_id}") raise ScenarioError(f"no chunk events recorded for {game_id}")
def streamed_chunk_paths(peer: Peer, game_id: str) -> list[str]:
return [
item["data"]["relative_path"]
for item in peer.output
if item.get("type") == "event"
and item.get("event") == "download-chunk-finished"
and item.get("data", {}).get("game_id") == game_id
]
def chunk_totals(peer: Peer, game_id: str, relative_path: str) -> dict[str, int]: def chunk_totals(peer: Peer, game_id: str, relative_path: str) -> dict[str, int]:
totals: dict[str, int] = {} totals: dict[str, int] = {}
for item in peer.output: for item in peer.output:
+15 -409
View File
@@ -5,24 +5,15 @@
use std::{ use std::{
net::SocketAddr, net::SocketAddr,
path::{Path, PathBuf}, path::{Path, PathBuf},
process::Stdio,
time::Duration, time::Duration,
}; };
use bytes::Bytes;
use eyre::{Context, OptionExt}; use eyre::{Context, OptionExt};
use lanspread_peer::{StreamInstallFuture, StreamInstallProvider, UnpackFuture, Unpacker}; use lanspread_peer::{UnpackFuture, Unpacker};
use lanspread_proto::StreamInstallFrame;
use serde::Serialize; use serde::Serialize;
use serde_json::{Value, json}; use serde_json::{Value, json};
use tokio::{
io::{AsyncRead, AsyncReadExt},
sync::mpsc,
};
use tokio_util::sync::CancellationToken;
pub const DEFAULT_FIXTURE_VERSION: &str = "20250101"; pub const DEFAULT_FIXTURE_VERSION: &str = "20250101";
const STREAM_CHUNK_SIZE: usize = 256 * 1024;
#[derive(Debug, Clone, PartialEq, Eq)] #[derive(Debug, Clone, PartialEq, Eq)]
pub struct CommandEnvelope { pub struct CommandEnvelope {
@@ -45,6 +36,9 @@ pub enum CliCommand {
StreamInstall { StreamInstall {
game_id: String, game_id: String,
}, },
CancelDownload {
game_id: String,
},
Install { Install {
game_id: String, game_id: String,
}, },
@@ -76,6 +70,7 @@ impl CliCommand {
Self::SetGameDir { .. } => "set-game-dir", Self::SetGameDir { .. } => "set-game-dir",
Self::Download { .. } => "download", Self::Download { .. } => "download",
Self::StreamInstall { .. } => "stream-install", Self::StreamInstall { .. } => "stream-install",
Self::CancelDownload { .. } => "cancel-download",
Self::Install { .. } => "install", Self::Install { .. } => "install",
Self::Uninstall { .. } => "uninstall", Self::Uninstall { .. } => "uninstall",
Self::Play { .. } => "play", Self::Play { .. } => "play",
@@ -117,6 +112,9 @@ pub fn parse_command_value(value: &Value) -> eyre::Result<CommandEnvelope> {
"stream-install" => CliCommand::StreamInstall { "stream-install" => CliCommand::StreamInstall {
game_id: game_id(object)?, game_id: game_id(object)?,
}, },
"cancel-download" => CliCommand::CancelDownload {
game_id: game_id(object)?,
},
"install" => CliCommand::Install { "install" => CliCommand::Install {
game_id: game_id(object)?, game_id: game_id(object)?,
}, },
@@ -270,356 +268,6 @@ impl Unpacker for ExternalUnrarUnpacker {
} }
} }
pub struct ExternalUnrarStreamProvider {
program: PathBuf,
}
impl ExternalUnrarStreamProvider {
#[must_use]
pub fn new(program: PathBuf) -> Self {
Self { program }
}
}
impl StreamInstallProvider for ExternalUnrarStreamProvider {
fn stream_archive<'a>(
&'a self,
archive: &'a Path,
frames: mpsc::Sender<StreamInstallFrame>,
cancel_token: CancellationToken,
) -> StreamInstallFuture<'a> {
Box::pin(async move {
let listing = unrar_listing(&self.program, archive).await?;
let archive_name = archive
.file_name()
.and_then(|name| name.to_str())
.unwrap_or("archive.eti")
.to_string();
send_stream_frame(
&frames,
StreamInstallFrame::ArchiveBegin {
archive_name: archive_name.clone(),
solid: listing.solid,
unpacked_size: listing.unpacked_size(),
},
)
.await?;
stream_unrar_entries(
&self.program,
archive,
&listing.entries,
&frames,
cancel_token.clone(),
)
.await?;
send_stream_frame(&frames, StreamInstallFrame::ArchiveEnd { archive_name }).await
})
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct RarListing {
solid: bool,
entries: Vec<RarEntry>,
}
impl RarListing {
fn unpacked_size(&self) -> u64 {
self.entries
.iter()
.filter(|entry| entry.kind == RarEntryKind::File)
.map(|entry| entry.size)
.sum()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct RarEntry {
relative_path: String,
kind: RarEntryKind,
size: u64,
crc32: Option<u32>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum RarEntryKind {
File,
Directory,
}
#[derive(Default)]
struct RarEntryDraft {
relative_path: Option<String>,
kind: Option<RarEntryKind>,
size: Option<u64>,
crc32: Option<u32>,
}
async fn unrar_listing(program: &Path, archive: &Path) -> eyre::Result<RarListing> {
let output = tokio::process::Command::new(program)
.arg("lt")
.arg("-cfg-")
.arg(archive)
.output()
.await?;
if !output.status.success() {
eyre::bail!(
"unrar lt failed for {} with status {}: {}",
archive.display(),
output.status,
String::from_utf8_lossy(&output.stderr)
);
}
parse_unrar_listing(&String::from_utf8_lossy(&output.stdout))
}
fn parse_unrar_listing(output: &str) -> eyre::Result<RarListing> {
let mut solid = false;
let mut entries = Vec::new();
let mut current = RarEntryDraft::default();
for line in output.lines() {
let trimmed = line.trim();
if let Some(details) = trimmed.strip_prefix("Details:") {
solid = details.to_ascii_lowercase().contains("solid");
continue;
}
if let Some(name) = trimmed.strip_prefix("Name:") {
push_rar_entry(&mut entries, std::mem::take(&mut current))?;
current.relative_path = Some(name.trim().to_string());
continue;
}
if let Some(kind) = trimmed.strip_prefix("Type:") {
current.kind = match kind.trim() {
"File" => Some(RarEntryKind::File),
"Directory" => Some(RarEntryKind::Directory),
_ => None,
};
continue;
}
if let Some(size) = trimmed.strip_prefix("Size:") {
current.size = Some(size.trim().parse()?);
continue;
}
if let Some(crc) = trimmed.strip_prefix("CRC32:") {
current.crc32 = Some(u32::from_str_radix(crc.trim(), 16)?);
}
}
push_rar_entry(&mut entries, current)?;
Ok(RarListing { solid, entries })
}
fn push_rar_entry(entries: &mut Vec<RarEntry>, draft: RarEntryDraft) -> eyre::Result<()> {
let Some(relative_path) = draft.relative_path else {
return Ok(());
};
let Some(kind) = draft.kind else {
return Ok(());
};
let (size, crc32) = match kind {
RarEntryKind::File => {
let size = draft
.size
.ok_or_else(|| eyre::eyre!("RAR file entry {relative_path} has no Size"))?;
let crc32 = draft
.crc32
.ok_or_else(|| eyre::eyre!("RAR file entry {relative_path} has no CRC32"))?;
(size, Some(crc32))
}
RarEntryKind::Directory => (0, None),
};
entries.push(RarEntry {
relative_path,
kind,
size,
crc32,
});
Ok(())
}
async fn stream_unrar_entries(
program: &Path,
archive: &Path,
entries: &[RarEntry],
frames: &mpsc::Sender<StreamInstallFrame>,
cancel_token: CancellationToken,
) -> eyre::Result<()> {
let mut child = tokio::process::Command::new(program)
.arg("p")
.arg("-inul")
.arg("-cfg-")
.arg(archive)
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()?;
let result = async {
let mut stdout = child
.stdout
.take()
.ok_or_eyre("unrar stdout was not captured")?;
let mut buffer = vec![0_u8; STREAM_CHUNK_SIZE];
for entry in entries {
if cancel_token.is_cancelled() {
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
match entry.kind {
RarEntryKind::Directory => {
send_stream_frame(
frames,
StreamInstallFrame::Directory {
relative_path: entry.relative_path.clone(),
},
)
.await?;
}
RarEntryKind::File => {
let Some(crc32) = entry.crc32 else {
eyre::bail!("RAR file entry {} has no CRC32", entry.relative_path);
};
send_stream_frame(
frames,
StreamInstallFrame::FileBegin {
relative_path: entry.relative_path.clone(),
size: entry.size,
crc32,
},
)
.await?;
stream_unrar_file_from_stdout(
&mut stdout,
archive,
entry,
frames,
&mut buffer,
&cancel_token,
)
.await?;
send_stream_frame(
frames,
StreamInstallFrame::FileEnd {
relative_path: entry.relative_path.clone(),
},
)
.await?;
}
}
}
let extra =
read_unrar_stdout(&mut stdout, &mut buffer[..1], &cancel_token, archive).await?;
if extra != 0 {
eyre::bail!(
"unrar produced bytes after listed entries for {}",
archive.display()
);
}
let status = wait_unrar_child(&mut child, &cancel_token, archive).await?;
if !status.success() {
eyre::bail!(
"unrar p failed for {} with status {status}",
archive.display()
);
}
Ok(())
}
.await;
if result.is_err() {
let _ = child.kill().await;
}
result
}
async fn stream_unrar_file_from_stdout(
stdout: &mut (impl AsyncRead + Unpin),
archive: &Path,
entry: &RarEntry,
frames: &mpsc::Sender<StreamInstallFrame>,
buffer: &mut [u8],
cancel_token: &CancellationToken,
) -> eyre::Result<()> {
let mut remaining = entry.size;
while remaining > 0 {
let read_len = usize::try_from(remaining.min(buffer.len() as u64))?;
let read =
read_unrar_stdout(stdout, &mut buffer[..read_len], cancel_token, archive).await?;
if read == 0 {
eyre::bail!(
"unrar ended while streaming {} from {}; {remaining} bytes missing",
entry.relative_path,
archive.display()
);
}
send_stream_frame(
frames,
StreamInstallFrame::FileChunk {
bytes: Bytes::copy_from_slice(&buffer[..read]),
},
)
.await?;
remaining = remaining.saturating_sub(u64::try_from(read)?);
}
Ok(())
}
async fn read_unrar_stdout(
stdout: &mut (impl AsyncRead + Unpin),
buffer: &mut [u8],
cancel_token: &CancellationToken,
archive: &Path,
) -> eyre::Result<usize> {
tokio::select! {
() = cancel_token.cancelled() => {
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
read = stdout.read(buffer) => Ok(read?),
}
}
async fn wait_unrar_child(
child: &mut tokio::process::Child,
cancel_token: &CancellationToken,
archive: &Path,
) -> eyre::Result<std::process::ExitStatus> {
tokio::select! {
() = cancel_token.cancelled() => {
let _ = child.kill().await;
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
status = child.wait() => Ok(status?),
}
}
async fn send_stream_frame(
frames: &mpsc::Sender<StreamInstallFrame>,
frame: StreamInstallFrame,
) -> eyre::Result<()> {
frames
.send(frame)
.await
.map_err(|_| eyre::eyre!("streamed install frame receiver closed"))
}
pub fn result_line(id: &Option<Value>, command: &str, data: Value) -> eyre::Result<String> { pub fn result_line(id: &Option<Value>, command: &str, data: Value) -> eyre::Result<String> {
output_line(json!({ output_line(json!({
"type": "result", "type": "result",
@@ -724,60 +372,18 @@ mod tests {
} }
#[test] #[test]
fn parses_unrar_technical_listing() { fn parses_cancel_download_command() {
let listing = parse_unrar_listing( let parsed = parse_command_line(r#"{"cmd":"cancel-download","game_id":"cnctw"}"#)
r#" .expect("command should parse");
Archive: game.eti
Details: RAR 5, solid
Name: bin/payload.bin
Type: File
Size: 123
CRC32: 38B488A7
Name: bin
Type: Directory
"#,
)
.expect("listing should parse");
assert!(listing.solid);
assert_eq!( assert_eq!(
listing.entries, parsed.command,
vec![ CliCommand::CancelDownload {
RarEntry { game_id: "cnctw".to_string(),
relative_path: "bin/payload.bin".to_string(), }
kind: RarEntryKind::File,
size: 123,
crc32: Some(0x38B4_88A7),
},
RarEntry {
relative_path: "bin".to_string(),
kind: RarEntryKind::Directory,
size: 0,
crc32: None,
},
]
); );
} }
#[test]
fn rejects_unrar_file_entries_without_crc32() {
let err = parse_unrar_listing(
r#"
Archive: game.eti
Details: RAR 5
Name: bin/payload.bin
Type: File
Size: 123
"#,
)
.expect_err("file entries without CRC32 should be rejected");
assert!(err.to_string().contains("has no CRC32"));
}
#[tokio::test] #[tokio::test]
async fn fixture_unpacker_creates_install_payload() { async fn fixture_unpacker_creates_install_payload() {
let temp = TempDir::new("lanspread-peer-cli-fixture"); let temp = TempDir::new("lanspread-peer-cli-fixture");
+8 -1
View File
@@ -16,6 +16,7 @@ use lanspread_db::db::{Game, GameCatalog, GameFileDescription};
use lanspread_peer::{ use lanspread_peer::{
ActiveOperation, ActiveOperation,
ActiveOperationKind, ActiveOperationKind,
ExternalUnrarStreamProvider,
InstallOperation, InstallOperation,
NoopStreamInstallProvider, NoopStreamInstallProvider,
PeerCommand, PeerCommand,
@@ -33,7 +34,6 @@ use lanspread_peer_cli::{
CliCommand, CliCommand,
CommandEnvelope, CommandEnvelope,
DEFAULT_FIXTURE_VERSION, DEFAULT_FIXTURE_VERSION,
ExternalUnrarStreamProvider,
ExternalUnrarUnpacker, ExternalUnrarUnpacker,
FixtureSeed, FixtureSeed,
FixtureUnpacker, FixtureUnpacker,
@@ -267,6 +267,13 @@ async fn handle_command(
})?; })?;
Ok(json!({"queued": true, "game_id": game_id})) Ok(json!({"queued": true, "game_id": game_id}))
} }
CliCommand::CancelDownload { game_id } => {
ensure_catalog_game(shared, game_id).await?;
sender.send(PeerCommand::CancelDownload {
id: game_id.clone(),
})?;
Ok(json!({"queued": true, "game_id": game_id}))
}
CliCommand::Install { game_id } => { CliCommand::Install { game_id } => {
ensure_catalog_game(shared, game_id).await?; ensure_catalog_game(shared, game_id).await?;
ensure_no_active_operation(shared, game_id).await?; ensure_no_active_operation(shared, game_id).await?;
+12
View File
@@ -166,6 +166,18 @@ Most scans become O(number of game dirs), with full recursion only when needed.
scratch sentinel files. `local/` and install transaction metadata are scratch sentinel files. `local/` and install transaction metadata are
preserved, so a cancelled update of an installed game settles as local-only. preserved, so a cancelled update of an installed game settles as local-only.
### Streamed install integrity
- Low-disk streamed installs request archive-derived file bytes from one peer
and write them directly into the install transaction staging directory.
- The receiver verifies every streamed file against the sender archive's file
size and RAR CRC32 before the transaction may commit. This catches truncated
streams, transport corruption, and provider bugs.
- This is not malicious-peer protection: the peer controls both the archive
metadata and the streamed bytes. A trusted-content model needs catalog-owned
hashes, either for the root archives or for extracted files, and receiver-side
SHA-256 verification against those catalog values before commit.
## Fault tolerance rules ## Fault tolerance rules
- Every peer is keyed by `peer_id`, not by IP address. - Every peer is keyed by `peer_id`, not by IP address.
+6 -1
View File
@@ -83,7 +83,12 @@ pub use crate::{
launch_settings::{LaunchSettingsOutcome, apply_launch_settings_once}, launch_settings::{LaunchSettingsOutcome, apply_launch_settings_once},
startup::PeerRuntimeHandle, startup::PeerRuntimeHandle,
state_paths::{launch_settings_applied_path, setup_done_path}, state_paths::{launch_settings_applied_path, setup_done_path},
stream_install::{NoopStreamInstallProvider, StreamInstallFuture, StreamInstallProvider}, stream_install::{
ExternalUnrarStreamProvider,
NoopStreamInstallProvider,
StreamInstallFuture,
StreamInstallProvider,
},
}; };
// ============================================================================= // =============================================================================
+499 -27
View File
@@ -3,6 +3,7 @@ use std::{
net::SocketAddr, net::SocketAddr,
path::{Path, PathBuf}, path::{Path, PathBuf},
pin::Pin, pin::Pin,
process::Stdio,
sync::Arc, sync::Arc,
time::{Duration, Instant}, time::{Duration, Instant},
}; };
@@ -14,7 +15,8 @@ use lanspread_proto::{Message, Request, StreamInstallFrame};
use s2n_quic::stream::SendStream; use s2n_quic::stream::SendStream;
use tokio::{ use tokio::{
fs::File, fs::File,
io::AsyncWriteExt, io::{AsyncRead, AsyncReadExt, AsyncWriteExt},
process::Command,
sync::{mpsc, mpsc::UnboundedSender}, sync::{mpsc, mpsc::UnboundedSender},
time::{self, MissedTickBehavior}, time::{self, MissedTickBehavior},
}; };
@@ -33,6 +35,45 @@ use crate::{
const FRAME_CHANNEL_DEPTH: usize = 16; const FRAME_CHANNEL_DEPTH: usize = 16;
const STREAM_INSTALL_PROGRESS_UPDATE_INTERVAL: Duration = Duration::from_millis(500); const STREAM_INSTALL_PROGRESS_UPDATE_INTERVAL: Duration = Duration::from_millis(500);
const STREAM_CHUNK_SIZE: usize = 256 * 1024;
/// Integrity metadata advertised by the sender's RAR archive.
///
/// This catches transport corruption, truncation, and provider bugs. It is not
/// a trusted-content guarantee because a malicious peer controls both the bytes
/// and the archive metadata. Trusted content would need catalog-owned hashes.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
struct SenderArchiveIntegrity {
expected_size: u64,
expected_crc32: u32,
}
impl SenderArchiveIntegrity {
fn new(expected_size: u64, expected_crc32: u32) -> Self {
Self {
expected_size,
expected_crc32,
}
}
fn verify(self, relative_path: &str, received: u64, actual_crc32: u32) -> eyre::Result<()> {
if received != self.expected_size {
eyre::bail!(
"streamed file {relative_path} size mismatch: got {received}, expected {}",
self.expected_size
);
}
if actual_crc32 != self.expected_crc32 {
eyre::bail!(
"streamed file {relative_path} sender RAR CRC32 mismatch: got {actual_crc32:08X}, expected {:08X}",
self.expected_crc32
);
}
Ok(())
}
}
pub type StreamInstallFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>; pub type StreamInstallFuture<'a> = Pin<Box<dyn Future<Output = eyre::Result<()>> + Send + 'a>>;
@@ -64,6 +105,357 @@ impl StreamInstallProvider for NoopStreamInstallProvider {
} }
} }
#[derive(Debug)]
pub struct ExternalUnrarStreamProvider {
program: PathBuf,
}
impl ExternalUnrarStreamProvider {
#[must_use]
pub fn new(program: PathBuf) -> Self {
Self { program }
}
}
impl StreamInstallProvider for ExternalUnrarStreamProvider {
fn stream_archive<'a>(
&'a self,
archive: &'a Path,
frames: mpsc::Sender<StreamInstallFrame>,
cancel_token: CancellationToken,
) -> StreamInstallFuture<'a> {
Box::pin(async move {
let listing = unrar_listing(&self.program, archive).await?;
let archive_name = archive
.file_name()
.and_then(|name| name.to_str())
.unwrap_or("archive.eti")
.to_string();
send_stream_frame(
&frames,
StreamInstallFrame::ArchiveBegin {
archive_name: archive_name.clone(),
solid: listing.solid,
unpacked_size: listing.unpacked_size(),
},
)
.await?;
stream_unrar_entries(
&self.program,
archive,
&listing.entries,
&frames,
cancel_token.clone(),
)
.await?;
send_stream_frame(&frames, StreamInstallFrame::ArchiveEnd { archive_name }).await
})
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct RarListing {
solid: bool,
entries: Vec<RarEntry>,
}
impl RarListing {
fn unpacked_size(&self) -> u64 {
self.entries
.iter()
.filter(|entry| entry.kind == RarEntryKind::File)
.map(|entry| entry.size)
.sum()
}
}
#[derive(Debug, Clone, PartialEq, Eq)]
struct RarEntry {
relative_path: String,
kind: RarEntryKind,
size: u64,
crc32: Option<u32>,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
enum RarEntryKind {
File,
Directory,
}
#[derive(Default)]
struct RarEntryDraft {
relative_path: Option<String>,
kind: Option<RarEntryKind>,
size: Option<u64>,
crc32: Option<u32>,
}
async fn unrar_listing(program: &Path, archive: &Path) -> eyre::Result<RarListing> {
let output = Command::new(program)
.arg("lt")
.arg("-cfg-")
.arg(archive)
.output()
.await?;
if !output.status.success() {
eyre::bail!(
"unrar lt failed for {} with status {}: {}",
archive.display(),
output.status,
String::from_utf8_lossy(&output.stderr)
);
}
parse_unrar_listing(&String::from_utf8_lossy(&output.stdout))
}
fn parse_unrar_listing(output: &str) -> eyre::Result<RarListing> {
let mut solid = false;
let mut entries = Vec::new();
let mut current = RarEntryDraft::default();
for line in output.lines() {
let trimmed = line.trim();
if let Some(details) = trimmed.strip_prefix("Details:") {
solid = details.to_ascii_lowercase().contains("solid");
continue;
}
if let Some(name) = trimmed.strip_prefix("Name:") {
push_rar_entry(&mut entries, std::mem::take(&mut current))?;
current.relative_path = Some(name.trim().to_string());
continue;
}
if let Some(kind) = trimmed.strip_prefix("Type:") {
current.kind = match kind.trim() {
"File" => Some(RarEntryKind::File),
"Directory" => Some(RarEntryKind::Directory),
_ => None,
};
continue;
}
if let Some(size) = trimmed.strip_prefix("Size:") {
current.size = Some(size.trim().parse()?);
continue;
}
if let Some(crc) = trimmed.strip_prefix("CRC32:") {
current.crc32 = Some(u32::from_str_radix(crc.trim(), 16)?);
}
}
push_rar_entry(&mut entries, current)?;
Ok(RarListing { solid, entries })
}
fn push_rar_entry(entries: &mut Vec<RarEntry>, draft: RarEntryDraft) -> eyre::Result<()> {
let Some(relative_path) = draft.relative_path else {
return Ok(());
};
let Some(kind) = draft.kind else {
return Ok(());
};
let (size, crc32) = match kind {
RarEntryKind::File => {
let size = draft
.size
.ok_or_else(|| eyre::eyre!("RAR file entry {relative_path} has no Size"))?;
let crc32 = draft
.crc32
.ok_or_else(|| eyre::eyre!("RAR file entry {relative_path} has no CRC32"))?;
(size, Some(crc32))
}
RarEntryKind::Directory => (0, None),
};
entries.push(RarEntry {
relative_path,
kind,
size,
crc32,
});
Ok(())
}
async fn stream_unrar_entries(
program: &Path,
archive: &Path,
entries: &[RarEntry],
frames: &mpsc::Sender<StreamInstallFrame>,
cancel_token: CancellationToken,
) -> eyre::Result<()> {
let mut child = Command::new(program)
.arg("p")
.arg("-inul")
.arg("-cfg-")
.arg(archive)
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()?;
let result = async {
let mut stdout = child
.stdout
.take()
.ok_or_else(|| eyre::eyre!("unrar stdout was not captured"))?;
let mut buffer = vec![0_u8; STREAM_CHUNK_SIZE];
for entry in entries {
if cancel_token.is_cancelled() {
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
match entry.kind {
RarEntryKind::Directory => {
send_stream_frame(
frames,
StreamInstallFrame::Directory {
relative_path: entry.relative_path.clone(),
},
)
.await?;
}
RarEntryKind::File => {
let Some(crc32) = entry.crc32 else {
eyre::bail!("RAR file entry {} has no CRC32", entry.relative_path);
};
send_stream_frame(
frames,
StreamInstallFrame::FileBegin {
relative_path: entry.relative_path.clone(),
size: entry.size,
crc32,
},
)
.await?;
stream_unrar_file_from_stdout(
&mut stdout,
archive,
entry,
frames,
&mut buffer,
&cancel_token,
)
.await?;
send_stream_frame(
frames,
StreamInstallFrame::FileEnd {
relative_path: entry.relative_path.clone(),
},
)
.await?;
}
}
}
let extra =
read_unrar_stdout(&mut stdout, &mut buffer[..1], &cancel_token, archive).await?;
if extra != 0 {
eyre::bail!(
"unrar produced bytes after listed entries for {}",
archive.display()
);
}
let status = wait_unrar_child(&mut child, &cancel_token, archive).await?;
if !status.success() {
eyre::bail!(
"unrar p failed for {} with status {status}",
archive.display()
);
}
Ok(())
}
.await;
if result.is_err() {
let _ = child.kill().await;
}
result
}
async fn stream_unrar_file_from_stdout(
stdout: &mut (impl AsyncRead + Unpin),
archive: &Path,
entry: &RarEntry,
frames: &mpsc::Sender<StreamInstallFrame>,
buffer: &mut [u8],
cancel_token: &CancellationToken,
) -> eyre::Result<()> {
let mut remaining = entry.size;
while remaining > 0 {
let read_len = usize::try_from(remaining.min(u64::try_from(buffer.len())?))?;
let read =
read_unrar_stdout(stdout, &mut buffer[..read_len], cancel_token, archive).await?;
if read == 0 {
eyre::bail!(
"unrar ended while streaming {} from {}; {remaining} bytes missing",
entry.relative_path,
archive.display()
);
}
send_stream_frame(
frames,
StreamInstallFrame::FileChunk {
bytes: Bytes::copy_from_slice(&buffer[..read]),
},
)
.await?;
remaining = remaining.saturating_sub(u64::try_from(read)?);
}
Ok(())
}
async fn read_unrar_stdout(
stdout: &mut (impl AsyncRead + Unpin),
buffer: &mut [u8],
cancel_token: &CancellationToken,
archive: &Path,
) -> eyre::Result<usize> {
tokio::select! {
() = cancel_token.cancelled() => {
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
read = stdout.read(buffer) => Ok(read?),
}
}
async fn wait_unrar_child(
child: &mut tokio::process::Child,
cancel_token: &CancellationToken,
archive: &Path,
) -> eyre::Result<std::process::ExitStatus> {
tokio::select! {
() = cancel_token.cancelled() => {
let _ = child.kill().await;
eyre::bail!("streamed archive {} was cancelled", archive.display());
}
status = child.wait() => Ok(status?),
}
}
async fn send_stream_frame(
frames: &mpsc::Sender<StreamInstallFrame>,
frame: StreamInstallFrame,
) -> eyre::Result<()> {
frames
.send(frame)
.await
.map_err(|_| eyre::eyre!("streamed install frame receiver closed"))
}
pub(crate) async fn send_stream_install_error( pub(crate) async fn send_stream_install_error(
tx: SendStream, tx: SendStream,
message: impl Into<String>, message: impl Into<String>,
@@ -331,10 +723,9 @@ fn bytes_per_second(bytes: u64, elapsed: Duration) -> u64 {
struct IncomingFile { struct IncomingFile {
relative_path: String, relative_path: String,
path: PathBuf, path: PathBuf,
expected_size: u64, integrity: SenderArchiveIntegrity,
expected_crc32: u32,
received: u64, received: u64,
hasher: Hasher, crc32: Hasher,
file: File, file: File,
} }
@@ -349,10 +740,9 @@ impl IncomingFile {
Self { Self {
relative_path, relative_path,
path, path,
expected_size, integrity: SenderArchiveIntegrity::new(expected_size, expected_crc32),
expected_crc32,
received: 0, received: 0,
hasher: Hasher::new(), crc32: Hasher::new(),
file, file,
} }
} }
@@ -366,15 +756,15 @@ impl IncomingFile {
) -> eyre::Result<u64> { ) -> eyre::Result<u64> {
let offset = self.received; let offset = self.received;
let length = u64::try_from(bytes.len())?; let length = u64::try_from(bytes.len())?;
if offset.saturating_add(length) > self.expected_size { if offset.saturating_add(length) > self.integrity.expected_size {
eyre::bail!( eyre::bail!(
"streamed file {} exceeded expected size {}", "streamed file {} exceeded expected size {}",
self.relative_path, self.relative_path,
self.expected_size self.integrity.expected_size
); );
} }
self.file.write_all(&bytes).await?; self.file.write_all(&bytes).await?;
self.hasher.update(&bytes); self.crc32.update(&bytes);
self.received = self.received.saturating_add(length); self.received = self.received.saturating_add(length);
let _ = tx_notify_ui.send(PeerEvent::DownloadGameFileChunkFinished { let _ = tx_notify_ui.send(PeerEvent::DownloadGameFileChunkFinished {
@@ -396,23 +786,9 @@ impl IncomingFile {
} }
self.file.flush().await?; self.file.flush().await?;
if self.received != self.expected_size { let actual_crc32 = self.crc32.finalize();
eyre::bail!( self.integrity
"streamed file {} size mismatch: got {}, expected {}", .verify(&self.relative_path, self.received, actual_crc32)?;
self.relative_path,
self.received,
self.expected_size
);
}
let actual = self.hasher.finalize();
if actual != self.expected_crc32 {
eyre::bail!(
"streamed file {} CRC32 mismatch: got {actual:08X}, expected {:08X}",
self.relative_path,
self.expected_crc32
);
}
log::debug!( log::debug!(
"Received streamed file {} -> {}", "Received streamed file {} -> {}",
@@ -444,4 +820,100 @@ mod tests {
assert!(resolve_stream_path(&staging, "/absolute").is_err()); assert!(resolve_stream_path(&staging, "/absolute").is_err());
assert!(resolve_stream_path(&staging, "C:/windows").is_err()); assert!(resolve_stream_path(&staging, "C:/windows").is_err());
} }
#[test]
fn parses_unrar_technical_listing() {
let listing = parse_unrar_listing(
r#"
Archive: game.eti
Details: RAR 5, solid
Name: bin/payload.bin
Type: File
Size: 123
CRC32: 38B488A7
Name: bin
Type: Directory
"#,
)
.expect("listing should parse");
assert!(listing.solid);
assert_eq!(
listing.entries,
vec![
RarEntry {
relative_path: "bin/payload.bin".to_string(),
kind: RarEntryKind::File,
size: 123,
crc32: Some(0x38B4_88A7),
},
RarEntry {
relative_path: "bin".to_string(),
kind: RarEntryKind::Directory,
size: 0,
crc32: None,
},
]
);
}
#[test]
fn rejects_unrar_file_entries_without_crc32() {
let err = parse_unrar_listing(
r#"
Archive: game.eti
Details: RAR 5
Name: bin/payload.bin
Type: File
Size: 123
"#,
)
.expect_err("file entries without CRC32 should be rejected");
assert!(err.to_string().contains("has no CRC32"));
}
#[test]
fn sender_archive_integrity_accepts_matching_size_and_crc32() {
let bytes = b"payload";
let integrity =
SenderArchiveIntegrity::new(u64::try_from(bytes.len()).unwrap(), crc32_of(bytes));
integrity
.verify(
"bin/payload.bin",
u64::try_from(bytes.len()).unwrap(),
crc32_of(bytes),
)
.expect("matching sender archive metadata should verify");
}
#[test]
fn sender_archive_integrity_rejects_size_mismatch() {
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
let err = integrity
.verify("bin/payload.bin", 6, crc32_of(b"payload"))
.expect_err("truncated file should fail sender archive integrity");
assert!(err.to_string().contains("size mismatch"));
}
#[test]
fn sender_archive_integrity_rejects_crc32_mismatch() {
let integrity = SenderArchiveIntegrity::new(7, crc32_of(b"payload"));
let err = integrity
.verify("bin/payload.bin", 7, crc32_of(b"paylord"))
.expect_err("mutated file should fail sender archive integrity");
assert!(err.to_string().contains("sender RAR CRC32 mismatch"));
}
fn crc32_of(bytes: &[u8]) -> u32 {
let mut hasher = Hasher::new();
hasher.update(bytes);
hasher.finalize()
}
} }
@@ -14,11 +14,14 @@ use lanspread_db::db::{Availability, Game, GameCatalog, GameDB, GameFileDescript
use lanspread_peer::{ use lanspread_peer::{
ActiveOperation, ActiveOperation,
ActiveOperationKind, ActiveOperationKind,
ExternalUnrarStreamProvider,
NoopStreamInstallProvider,
PeerCommand, PeerCommand,
PeerEvent, PeerEvent,
PeerGameDB, PeerGameDB,
PeerRuntimeHandle, PeerRuntimeHandle,
PeerStartOptions, PeerStartOptions,
StreamInstallProvider,
UnpackFuture, UnpackFuture,
Unpacker, Unpacker,
migrate_legacy_state, migrate_legacy_state,
@@ -82,6 +85,7 @@ struct LanSpreadState {
peer_runtime: Arc<RwLock<Option<PeerRuntimeHandle>>>, peer_runtime: Arc<RwLock<Option<PeerRuntimeHandle>>>,
games: Arc<RwLock<GameDB>>, games: Arc<RwLock<GameDB>>,
active_operations: Arc<RwLock<HashMap<String, UiOperationKind>>>, active_operations: Arc<RwLock<HashMap<String, UiOperationKind>>>,
pending_stream_installs: Arc<RwLock<HashSet<String>>>,
games_folder: Arc<RwLock<String>>, games_folder: Arc<RwLock<String>>,
peer_game_db: Arc<RwLock<PeerGameDB>>, peer_game_db: Arc<RwLock<PeerGameDB>>,
catalog: Arc<RwLock<GameCatalog>>, catalog: Arc<RwLock<GameCatalog>>,
@@ -255,6 +259,16 @@ async fn install_game(
log::warn!("Game already has an active operation: {id}"); log::warn!("Game already has an active operation: {id}");
return Ok(false); return Ok(false);
} }
if state
.inner()
.pending_stream_installs
.read()
.await
.contains(&id)
{
log::warn!("Game already has a pending streamed install: {id}");
return Ok(false);
}
let peer_ctrl_arc = state.inner().peer_ctrl.clone(); let peer_ctrl_arc = state.inner().peer_ctrl.clone();
let peer_ctrl = peer_ctrl_arc.read().await.clone(); let peer_ctrl = peer_ctrl_arc.read().await.clone();
@@ -294,6 +308,77 @@ async fn install_game(
Ok(handled) Ok(handled)
} }
#[tauri::command]
async fn stream_install_game(
id: String,
state: tauri::State<'_, LanSpreadState>,
) -> tauri::Result<bool> {
if state
.inner()
.active_operations
.read()
.await
.contains_key(&id)
{
log::warn!("Game already has an active operation: {id}");
return Ok(false);
}
if state
.inner()
.pending_stream_installs
.read()
.await
.contains(&id)
{
log::warn!("Game already has a pending streamed install: {id}");
return Ok(false);
}
let Some((downloaded, installed, peer_count)) = state
.inner()
.games
.read()
.await
.get_game_by_id(&id)
.map(|game| (game.downloaded, game.installed, game.peer_count))
else {
log::warn!("Ignoring streamed install request for unknown game: {id}");
return Ok(false);
};
if downloaded || installed || peer_count == 0 {
log::warn!(
"Ignoring streamed install request for {id}: downloaded={downloaded}, \
installed={installed}, peer_count={peer_count}"
);
return Ok(false);
}
let peer_ctrl_arc = state.inner().peer_ctrl.clone();
let peer_ctrl = peer_ctrl_arc.read().await.clone();
let Some(peer_ctrl) = peer_ctrl else {
log::warn!("Peer system not initialized yet");
return Ok(false);
};
{
let mut pending = state.inner().pending_stream_installs.write().await;
pending.insert(id.clone());
}
if let Err(e) = peer_ctrl.send(PeerCommand::GetGame(id.clone())) {
log::error!("Failed to send PeerCommand::GetGame for streamed install: {e:?}");
state
.inner()
.pending_stream_installs
.write()
.await
.remove(&id);
return Ok(false);
}
Ok(true)
}
#[tauri::command] #[tauri::command]
async fn update_game( async fn update_game(
id: String, id: String,
@@ -1867,6 +1952,7 @@ async fn ensure_peer_started(app_handle: &AppHandle, games_folder: &Path) {
let unpacker = Arc::new(SidecarUnpacker { let unpacker = Arc::new(SidecarUnpacker {
app_handle: app_handle.clone(), app_handle: app_handle.clone(),
}); });
let stream_install_provider = stream_install_provider_for_app(app_handle);
match start_peer_with_options( match start_peer_with_options(
games_folder.to_path_buf(), games_folder.to_path_buf(),
tx_peer_event, tx_peer_event,
@@ -1876,7 +1962,7 @@ async fn ensure_peer_started(app_handle: &AppHandle, games_folder: &Path) {
PeerStartOptions { PeerStartOptions {
state_dir: Some(state_dir), state_dir: Some(state_dir),
active_outbound_transfers: Some(state.active_outbound_transfers.clone()), active_outbound_transfers: Some(state.active_outbound_transfers.clone()),
stream_install_provider: None, stream_install_provider: Some(stream_install_provider),
}, },
) { ) {
Ok(handle) => { Ok(handle) => {
@@ -1894,6 +1980,22 @@ async fn ensure_peer_started(app_handle: &AppHandle, games_folder: &Path) {
} }
} }
fn stream_install_provider_for_app(app_handle: &AppHandle) -> Arc<dyn StreamInstallProvider> {
match resolve_unrar_sidecar_program(app_handle) {
Ok(program) => Arc::new(ExternalUnrarStreamProvider::new(program)),
Err(err) => {
log::error!("Failed to resolve streamed-install unrar sidecar: {err}");
Arc::new(NoopStreamInstallProvider)
}
}
}
fn resolve_unrar_sidecar_program(app_handle: &AppHandle) -> eyre::Result<PathBuf> {
let sidecar = app_handle.shell().sidecar("unrar")?;
let command: std::process::Command = sidecar.into();
Ok(PathBuf::from(command.get_program()))
}
fn emit_game_id_event(app_handle: &AppHandle, event: &str, id: &str, label: &str) { fn emit_game_id_event(app_handle: &AppHandle, event: &str, id: &str, label: &str) {
if let Err(e) = app_handle.emit(event, Some(id.to_owned())) { if let Err(e) = app_handle.emit(event, Some(id.to_owned())) {
log::error!("{label}: Failed to emit {event} event: {e}"); log::error!("{label}: Failed to emit {event} event: {e}");
@@ -1990,6 +2092,7 @@ async fn handle_peer_event(app_handle: &AppHandle, event: PeerEvent) {
} }
PeerEvent::NoPeersHaveGame { id } => { PeerEvent::NoPeersHaveGame { id } => {
log::warn!("PeerEvent::NoPeersHaveGame received for {id}"); log::warn!("PeerEvent::NoPeersHaveGame received for {id}");
clear_pending_stream_install(app_handle, &id).await;
emit_game_id_event( emit_game_id_event(
app_handle, app_handle,
"game-no-peers", "game-no-peers",
@@ -2028,6 +2131,7 @@ async fn handle_peer_event(app_handle: &AppHandle, event: PeerEvent) {
} }
PeerEvent::DownloadGameFilesFailed { id } => { PeerEvent::DownloadGameFilesFailed { id } => {
log::warn!("PeerEvent::DownloadGameFilesFailed received"); log::warn!("PeerEvent::DownloadGameFilesFailed received");
clear_pending_stream_install(app_handle, &id).await;
emit_game_id_event( emit_game_id_event(
app_handle, app_handle,
"game-download-failed", "game-download-failed",
@@ -2037,6 +2141,7 @@ async fn handle_peer_event(app_handle: &AppHandle, event: PeerEvent) {
} }
PeerEvent::DownloadGameFilesAllPeersGone { id } => { PeerEvent::DownloadGameFilesAllPeersGone { id } => {
log::warn!("PeerEvent::DownloadGameFilesAllPeersGone received for {id}"); log::warn!("PeerEvent::DownloadGameFilesAllPeersGone received for {id}");
clear_pending_stream_install(app_handle, &id).await;
emit_game_id_event( emit_game_id_event(
app_handle, app_handle,
"game-download-peers-gone", "game-download-peers-gone",
@@ -2175,17 +2280,27 @@ async fn handle_got_game_files(
); );
let state = app_handle.state::<LanSpreadState>(); let state = app_handle.state::<LanSpreadState>();
let stream_install = state.pending_stream_installs.write().await.remove(&id);
let peer_ctrl = state.peer_ctrl.read().await.clone(); let peer_ctrl = state.peer_ctrl.read().await.clone();
if let Some(peer_ctrl) = peer_ctrl if let Some(peer_ctrl) = peer_ctrl
&& let Err(e) = peer_ctrl.send(PeerCommand::DownloadGameFiles { && let Err(e) = if stream_install {
id, peer_ctrl.send(PeerCommand::StreamInstallGame { id })
file_descriptions, } else {
}) peer_ctrl.send(PeerCommand::DownloadGameFiles {
id,
file_descriptions,
})
}
{ {
log::error!("Failed to send PeerCommand::DownloadGameFiles: {e}"); log::error!("Failed to continue queued game transfer: {e}");
} }
} }
async fn clear_pending_stream_install(app_handle: &AppHandle, id: &str) {
let state = app_handle.state::<LanSpreadState>();
state.pending_stream_installs.write().await.remove(id);
}
fn handle_download_finished(app_handle: &AppHandle, id: String) { fn handle_download_finished(app_handle: &AppHandle, id: String) {
log::info!("PeerEvent::DownloadGameFilesFinished received"); log::info!("PeerEvent::DownloadGameFilesFinished received");
emit_game_id_event( emit_game_id_event(
@@ -2679,6 +2794,7 @@ pub fn run() {
.invoke_handler(tauri::generate_handler![ .invoke_handler(tauri::generate_handler![
request_games, request_games,
install_game, install_game,
stream_install_game,
run_game, run_game,
start_server, start_server,
game_directory_exists, game_directory_exists,
@@ -1,13 +1,5 @@
import { Game } from '../lib/types'; import { Game } from '../lib/types';
import { deriveState } from '../lib/gameState'; import { deriveState, stateChipLabel } from '../lib/gameState';
const LABELS: Record<string, string> = {
installed: 'Installed',
local: 'Local',
downloading: 'Downloading',
busy: 'Working',
none: '',
};
interface Props { interface Props {
game: Game; game: Game;
@@ -17,7 +9,7 @@ interface Props {
export const StateChip = ({ game, showNone = false }: Props) => { export const StateChip = ({ game, showNone = false }: Props) => {
const state = deriveState(game); const state = deriveState(game);
const label = LABELS[state] ?? ''; const label = stateChipLabel(game);
if (!label && !showNone) return null; if (!label && !showNone) return null;
return ( return (
<div className="state-chip" data-state={state}> <div className="state-chip" data-state={state}>
@@ -5,7 +5,7 @@ import { StateChip } from '../StateChip';
import { ActionButton } from '../ActionButton'; import { ActionButton } from '../ActionButton';
import { Game, InstallStatus } from '../../lib/types'; import { Game, InstallStatus } from '../../lib/types';
import { deriveState, hasNewerLocalVersion, isInProgress } from '../../lib/gameState'; import { canStreamInstall, gameStatusLabel, hasNewerLocalVersion, isInProgress } from '../../lib/gameState';
import { formatBytes, formatEtiVersion, formatPlayers } from '../../lib/format'; import { formatBytes, formatEtiVersion, formatPlayers } from '../../lib/format';
interface Props { interface Props {
@@ -13,6 +13,7 @@ interface Props {
thumbnailUrl: string | null; thumbnailUrl: string | null;
onClose: () => void; onClose: () => void;
onPrimary: (game: Game) => void; onPrimary: (game: Game) => void;
onStreamInstall: (game: Game) => void;
onUninstall: (game: Game) => void; onUninstall: (game: Game) => void;
onRemoveDownload: (game: Game) => void; onRemoveDownload: (game: Game) => void;
onCancelDownload: (game: Game) => void; onCancelDownload: (game: Game) => void;
@@ -28,21 +29,12 @@ const tagsFromGame = (game: Game): string[] => {
return tags; return tags;
}; };
const statusLabelFor = (game: Game): string => {
switch (deriveState(game)) {
case 'installed': return 'Installed';
case 'local': return 'Downloaded';
case 'downloading': return 'Downloading';
case 'busy': return 'Working…';
case 'none': return 'Not downloaded';
}
};
export const GameDetailModal = ({ export const GameDetailModal = ({
game, game,
thumbnailUrl, thumbnailUrl,
onClose, onClose,
onPrimary, onPrimary,
onStreamInstall,
onUninstall, onUninstall,
onRemoveDownload, onRemoveDownload,
onCancelDownload, onCancelDownload,
@@ -55,6 +47,7 @@ export const GameDetailModal = ({
const canRemoveDownload = game.downloaded const canRemoveDownload = game.downloaded
&& !game.installed && !game.installed
&& !isInProgress(game.install_status); && !isInProgress(game.install_status);
const showStreamInstall = canStreamInstall(game);
const canViewFiles = game.downloaded const canViewFiles = game.downloaded
|| game.installed || game.installed
|| game.install_status === InstallStatus.Downloading || game.install_status === InstallStatus.Downloading
@@ -112,7 +105,7 @@ export const GameDetailModal = ({
</div> </div>
<div className="meta-cell"> <div className="meta-cell">
<div className="meta-label">Status</div> <div className="meta-label">Status</div>
<div className="meta-value">{statusLabelFor(game)}</div> <div className="meta-value">{gameStatusLabel(game)}</div>
</div> </div>
</div> </div>
@@ -133,6 +126,17 @@ export const GameDetailModal = ({
onClick={() => onPrimary(game)} onClick={() => onPrimary(game)}
onCancelDownload={onCancelDownload} onCancelDownload={onCancelDownload}
/> />
{showStreamInstall && (
<button
type="button"
className="ghost-btn"
title="Install without keeping archive files"
onClick={() => onStreamInstall(game)}
>
<Icon.install />
<span>Low disk install</span>
</button>
)}
{game.installed && game.can_host_server === true && ( {game.installed && game.can_host_server === true && (
<button <button
type="button" type="button"
@@ -9,6 +9,7 @@ export interface GameActions {
play: (id: string) => Promise<void>; play: (id: string) => Promise<void>;
startServer: (id: string) => Promise<void>; startServer: (id: string) => Promise<void>;
install: (id: string) => Promise<void>; install: (id: string) => Promise<void>;
streamInstall: (id: string) => Promise<void>;
update: (id: string) => Promise<void>; update: (id: string) => Promise<void>;
uninstall: (id: string) => Promise<void>; uninstall: (id: string) => Promise<void>;
removeDownload: (id: string) => Promise<void>; removeDownload: (id: string) => Promise<void>;
@@ -68,6 +69,15 @@ export const useGameActions = (
} }
}, [games, settings.language, settings.username]); }, [games, settings.language, settings.username]);
const streamInstall = useCallback(async (id: string) => {
try {
const success = await invoke<boolean>('stream_install_game', { id });
if (success) games.markChecking(id);
} catch (err) {
console.error('stream_install_game failed:', err);
}
}, [games]);
const update = useCallback(async (id: string) => { const update = useCallback(async (id: string) => {
try { try {
const game = games.games.find(item => item.id === id); const game = games.games.find(item => item.id === id);
@@ -129,5 +139,15 @@ export const useGameActions = (
} }
}, []); }, []);
return { play, startServer, install, update, uninstall, removeDownload, cancelDownload, viewFiles }; return {
play,
startServer,
install,
streamInstall,
update,
uninstall,
removeDownload,
cancelDownload,
viewFiles,
};
}; };
@@ -82,6 +82,35 @@ export const deriveState = (game: Game): DerivedState => {
return 'none'; return 'none';
}; };
export const isInstalledNotShareable = (game: Game): boolean =>
game.installed && !game.downloaded;
export const stateChipLabel = (game: Game): string => {
const state = deriveState(game);
if (state === 'installed' && isInstalledNotShareable(game)) return 'Not shareable';
switch (state) {
case 'installed': return 'Installed';
case 'local': return 'Local';
case 'downloading': return 'Downloading';
case 'busy': return 'Working';
case 'none': return '';
}
};
export const gameStatusLabel = (game: Game): string => {
const state = deriveState(game);
if (state === 'installed' && isInstalledNotShareable(game)) {
return 'Installed, not shareable';
}
switch (state) {
case 'installed': return 'Installed';
case 'local': return 'Downloaded';
case 'downloading': return 'Downloading';
case 'busy': return 'Working…';
case 'none': return 'Not downloaded';
}
};
export const isUnavailable = (game: Game): boolean => export const isUnavailable = (game: Game): boolean =>
!game.installed !game.installed
&& !game.downloaded && !game.downloaded
@@ -114,6 +143,12 @@ export const needsUpdate = (game: Game): boolean => {
return (compareVersionStamps(game.eti_game_version, game.local_version) ?? 0) > 0; return (compareVersionStamps(game.eti_game_version, game.local_version) ?? 0) > 0;
}; };
export const canStreamInstall = (game: Game): boolean =>
!game.downloaded
&& !game.installed
&& game.peer_count > 0
&& !isInProgress(game.install_status);
/** What pressing the card's main action button should do, given the state. */ /** What pressing the card's main action button should do, given the state. */
export type PrimaryAction = 'play' | 'install' | 'update' | 'download' | 'busy' | 'disabled'; export type PrimaryAction = 'play' | 'install' | 'update' | 'download' | 'busy' | 'disabled';
@@ -192,6 +192,7 @@ export const MainWindow = () => {
thumbnailUrl={thumbnails.get(openGame.id)} thumbnailUrl={thumbnails.get(openGame.id)}
onClose={() => setOpenGameId(null)} onClose={() => setOpenGameId(null)}
onPrimary={handlePrimary} onPrimary={handlePrimary}
onStreamInstall={(g) => actions.streamInstall(g.id)}
onUninstall={handleUninstall} onUninstall={handleUninstall}
onRemoveDownload={handleRemoveDownload} onRemoveDownload={handleRemoveDownload}
onCancelDownload={(g) => actions.cancelDownload(g.id)} onCancelDownload={(g) => actions.cancelDownload(g.id)}
@@ -2,6 +2,7 @@ import {
actionLabel, actionLabel,
activeStatusById, activeStatusById,
applyFilterAndSort, applyFilterAndSort,
canStreamInstall,
countByFilter, countByFilter,
deriveState, deriveState,
downloadProgressPercent, downloadProgressPercent,
@@ -10,7 +11,9 @@ import {
formatDownloadEta, formatDownloadEta,
formatDownloadSpeed, formatDownloadSpeed,
formatDownloadSpeedShort, formatDownloadSpeedShort,
gameStatusLabel,
mergeGameUpdate, mergeGameUpdate,
stateChipLabel,
} from '../src/lib/gameState.ts'; } from '../src/lib/gameState.ts';
import { import {
ActiveOperationKind, ActiveOperationKind,
@@ -209,3 +212,75 @@ Deno.test('download progress formatting matches the progress-bar layouts', () =>
); );
assertEquals(formatDownloadEta(485), '8 min', 'eta format should stay compact'); assertEquals(formatDownloadEta(485), '8 min', 'eta format should stay compact');
}); });
Deno.test('stream install is available only for idle remote games', () => {
assertEquals(
canStreamInstall(game({ downloaded: false, installed: false, peer_count: 1 })),
true,
'remote-only idle games should allow streamed install',
);
assertEquals(
canStreamInstall(game({ downloaded: true, installed: false, peer_count: 1 })),
false,
'downloaded games should install from local archives',
);
assertEquals(
canStreamInstall(game({ downloaded: false, installed: true, peer_count: 1 })),
false,
'installed games should not expose streamed install',
);
assertEquals(
canStreamInstall(game({ downloaded: false, installed: false, peer_count: 0 })),
false,
'games without peers should not expose streamed install',
);
assertEquals(
canStreamInstall(game({
downloaded: false,
installed: false,
peer_count: 1,
install_status: InstallStatus.CheckingPeers,
})),
false,
'busy games should not expose streamed install',
);
});
Deno.test('streamed local installs are labeled installed but not shareable', () => {
const streamed = game({
downloaded: false,
installed: true,
install_status: InstallStatus.Installed,
});
const downloadedInstall = game({
downloaded: true,
installed: true,
install_status: InstallStatus.Installed,
});
assertEquals(
deriveState(streamed),
'installed',
'streamed local installs should keep installed visual state',
);
assertEquals(
stateChipLabel(streamed),
'Not shareable',
'card chip should make the non-shareable state visible',
);
assertEquals(
gameStatusLabel(streamed),
'Installed, not shareable',
'detail status should spell out installed plus non-shareable',
);
assertEquals(
stateChipLabel(downloadedInstall),
'Installed',
'normal downloaded installs should keep the installed chip label',
);
assertEquals(
gameStatusLabel(downloadedInstall),
'Installed',
'normal downloaded installs should keep the installed detail label',
);
});