diff --git a/NEXT_STEPS.md b/NEXT_STEPS.md index c882b29..621dd64 100644 --- a/NEXT_STEPS.md +++ b/NEXT_STEPS.md @@ -21,17 +21,13 @@ product-ready. the technical listing, so CLI and GUI callers use one purpose-built provider instead of a per-file extraction loop. -3. **Handle solid archives deliberately** +3. **Done — Handle solid archives deliberately** - The provider exposes the RAR `solid` flag in `ArchiveBegin`; the remaining - work is to make that flag a tested policy decision instead of an incidental - stream attribute. Add archive inspection that decides: - - - non-solid: current one-pass streaming is fine - - solid: prove and enforce one sequential archive pass only - - This is the big architectural fork we discussed: keep non-solid and solid - archive handling explicit in the provider contract and scenario coverage. + The provider exposes the RAR `solid` flag in `ArchiveBegin` and always uses + one sequential payload pass per archive, which is the safe path for solid + archives. S41 now verifies a real solid RAR fixture through the Docker + peer-cli flow, including local-only final state, absent root archive/sentinel, + byte count, and extracted payload SHA-256 hashes. 4. **Decide the integrity model** diff --git a/PEER_CLI_SCENARIOS.md b/PEER_CLI_SCENARIOS.md index 1cb8cbf..996efed 100644 --- a/PEER_CLI_SCENARIOS.md +++ b/PEER_CLI_SCENARIOS.md @@ -48,6 +48,7 @@ for deterministic local runs; mDNS/macvlan remains an environment smoke path. | S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. | | S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. | | S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. | +| S41 | Solid archive streamed install | Empty client connects to a peer serving `fixture-solid/cnctw`, whose `.eti` is a real solid RAR archive. The receiver uses the container-bundled `unrar` stream provider. | The fixture is verified as solid with `unrar lt`; streamed install finishes with `downloaded=false`, `installed=true`, `availability=LocalOnly`; root archive and `version.ini` are absent; streamed byte count equals the extracted solid entries; local payload SHA-256 hashes match `unrar p` output. | ## Version-Skew Contract @@ -112,8 +113,42 @@ Use S38 to pin down how launcher settings are stamped into an installed game: crate's `launch_settings` unit tests cover the rewrite, line-ending, and marker logic deterministically. +## Streamed Install Archive Contract + +Use S39-S41 to pin down low-disk streamed installs: + +- The stream provider performs one archive metadata pass and one payload pass + per `.eti`, then frames entry boundaries for the receiver. +- Non-solid and solid archives both install into `local/` without committing a + root archive or root `version.ini`, so the receiver is installed but not a + downloadable source. +- S41 verifies the fixture is actually solid inside the source container, so + solid handling stays covered by the same Docker harness as the existing + streamed-install scenarios. + ## Run Log +### 2026-06-07 - Solid Streamed Install Coverage (S41) + +- Code under test added `fixture-solid/cnctw`, a real solid RAR `.eti`, plus + S41 in `run_extended_scenarios.py`. +- Gates before Docker: `just fmt`, `git diff --check`, and + `python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py` + passed. +- Runner: + `python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image` + passed against the rebuilt `lanspread-peer-cli:dev` image. +- S41 verified the source archive with `unrar lt -cfg-` inside the source + container; the archive reported `Details: RAR 5, solid`. +- The streamed install finished with `downloaded=false`, `installed=true`, + `availability=LocalOnly`, no root `version.ini`, and no root `cnctw.eti`. +- The client received `118` streamed file bytes, matching the extracted solid + entries. Payload SHA-256 hashes matched `unrar p` output: + `88764c9a6c9b5b846b4323cf7725cb7fd70766ddd7fba4168332804a839fa193` + (`bin/cnctw-solid-payload.bin`) and + `44afc308269b2381b7c707a056dd8d9d393274108ac4d880237fa6772c861d7a` + (`data/cnctw-solid-assets.dat`). + ### 2026-06-07 - Streamed Install Prototype (S39-S40) - Code under test added `stream-install` to `lanspread-peer-cli`, a peer diff --git a/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/cnctw.eti b/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/cnctw.eti new file mode 100644 index 0000000..ac98efd Binary files /dev/null and b/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/cnctw.eti differ diff --git a/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/version.ini b/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/version.ini new file mode 100644 index 0000000..f71f081 --- /dev/null +++ b/crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw/version.ini @@ -0,0 +1 @@ +20160128 \ No newline at end of file diff --git a/crates/lanspread-peer-cli/scripts/run_extended_scenarios.py b/crates/lanspread-peer-cli/scripts/run_extended_scenarios.py index 85874fa..0552fe4 100644 --- a/crates/lanspread-peer-cli/scripts/run_extended_scenarios.py +++ b/crates/lanspread-peer-cli/scripts/run_extended_scenarios.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Run the peer-cli scenarios S1-S40 through Docker.""" +"""Run the peer-cli scenarios S1-S41 through Docker.""" from __future__ import annotations @@ -328,6 +328,7 @@ class Runner: ("S37", self.s37_single_source_download_throughput), ("S39", self.s39_streamed_install_local_only), ("S40", self.s40_streamed_receiver_not_source), + ("S41", self.s41_solid_archive_streamed_install), ] for scenario_id, scenario in scenarios: @@ -1171,6 +1172,84 @@ class Runner: f"and download errored '{err['error']}'" ) + def s41_solid_archive_streamed_install(self) -> str: + source_dir = self.fixture_root / "s41-solid-source" + source_game = source_dir / "cnctw" + shutil.copytree(FIXTURES / "fixture-solid" / "cnctw", source_game) + + source = self.peer("s41-solid-source", games_dir=source_dir) + assert_peer_rar_archive_solid(source, "cnctw") + client = self.peer("s41-solid-client") + connect_many(client, [source]) + wait_remote_game(client, "cnctw", peer_count=1, version="20160128") + + waiter = LineWaiter(len(client.output)) + client.send({"cmd": "stream-install", "game_id": "cnctw"}) + client.wait_for( + event_is("got-game-files", "cnctw"), + timeout=20, + description="got solid cnctw files", + waiter=waiter, + ) + client.wait_for( + event_is("download-finished", "cnctw"), + timeout=60, + description="solid stream finish cnctw", + waiter=waiter, + ) + client.wait_for( + event_is("install-finished", "cnctw"), + timeout=30, + description="solid stream install cnctw", + waiter=waiter, + ) + + game = wait_local_game(client, "cnctw", downloaded=False, installed=True) + assert_game_state( + game, + downloaded=False, + installed=True, + availability="LocalOnly", + ) + game_root = client.host_games_dir / "cnctw" + assert_not_exists(game_root / "version.ini") + assert_not_exists(game_root / "cnctw.eti") + + expected = { + "bin/cnctw-solid-payload.bin": unrar_entry_sha256( + source, "cnctw", "bin/cnctw-solid-payload.bin" + ), + "data/cnctw-solid-assets.dat": unrar_entry_sha256( + source, "cnctw", "data/cnctw-solid-assets.dat" + ), + } + actual = { + rel: sha256_file(game_root / "local" / rel) + for rel in expected + } + if actual != expected: + raise ScenarioError( + f"solid streamed payload hashes mismatched: {actual} != {expected}" + ) + + streamed_bytes = sum( + int(item.get("data", {}).get("length", 0)) + for item in client.output + if item.get("type") == "event" + and item.get("event") == "download-chunk-finished" + and item.get("data", {}).get("game_id") == "cnctw" + ) + expected_bytes = sum((game_root / "local" / rel).stat().st_size for rel in expected) + if streamed_bytes != expected_bytes: + raise ScenarioError( + f"solid streamed byte count mismatch: {streamed_bytes} != {expected_bytes}" + ) + + return ( + "solid cnctw archive streamed through one local-only install; " + f"payload hashes={actual}, bytes={streamed_bytes}" + ) + def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]: result = subprocess.run( @@ -1307,6 +1386,22 @@ def unrar_entry_sha256(peer: Peer, game_id: str, relative_path: str) -> str: return output.split()[0] +def assert_peer_rar_archive_solid(peer: Peer, game_id: str) -> None: + output = peer.docker_exec( + "unrar", + "lt", + "-cfg-", + f"/games/{game_id}/{game_id}.eti", + ).stdout + for line in output.splitlines(): + stripped = line.strip() + if stripped.startswith("Details:"): + if "solid" in stripped.lower(): + return + raise ScenarioError(f"RAR archive is not solid: {game_id}") + raise ScenarioError(f"RAR archive details were not reported: {game_id}") + + def format_bytes(size: int) -> str: return f"{size / 1024 / 1024 / 1024:.2f} GiB"