Feature/streamed install prototype #27
+6
-10
@@ -21,17 +21,13 @@ product-ready.
|
||||
the technical listing, so CLI and GUI callers use one purpose-built provider
|
||||
instead of a per-file extraction loop.
|
||||
|
||||
3. **Handle solid archives deliberately**
|
||||
3. **Done — Handle solid archives deliberately**
|
||||
|
||||
The provider exposes the RAR `solid` flag in `ArchiveBegin`; the remaining
|
||||
work is to make that flag a tested policy decision instead of an incidental
|
||||
stream attribute. Add archive inspection that decides:
|
||||
|
||||
- non-solid: current one-pass streaming is fine
|
||||
- solid: prove and enforce one sequential archive pass only
|
||||
|
||||
This is the big architectural fork we discussed: keep non-solid and solid
|
||||
archive handling explicit in the provider contract and scenario coverage.
|
||||
The provider exposes the RAR `solid` flag in `ArchiveBegin` and always uses
|
||||
one sequential payload pass per archive, which is the safe path for solid
|
||||
archives. S41 now verifies a real solid RAR fixture through the Docker
|
||||
peer-cli flow, including local-only final state, absent root archive/sentinel,
|
||||
byte count, and extracted payload SHA-256 hashes.
|
||||
|
||||
4. **Decide the integrity model**
|
||||
|
||||
|
||||
@@ -48,6 +48,7 @@ for deterministic local runs; mDNS/macvlan remains an environment smoke path.
|
||||
| S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. |
|
||||
| S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. |
|
||||
| S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. |
|
||||
| S41 | Solid archive streamed install | Empty client connects to a peer serving `fixture-solid/cnctw`, whose `.eti` is a real solid RAR archive. The receiver uses the container-bundled `unrar` stream provider. | The fixture is verified as solid with `unrar lt`; streamed install finishes with `downloaded=false`, `installed=true`, `availability=LocalOnly`; root archive and `version.ini` are absent; streamed byte count equals the extracted solid entries; local payload SHA-256 hashes match `unrar p` output. |
|
||||
|
||||
## Version-Skew Contract
|
||||
|
||||
@@ -112,8 +113,42 @@ Use S38 to pin down how launcher settings are stamped into an installed game:
|
||||
crate's `launch_settings` unit tests cover the rewrite, line-ending, and
|
||||
marker logic deterministically.
|
||||
|
||||
## Streamed Install Archive Contract
|
||||
|
||||
Use S39-S41 to pin down low-disk streamed installs:
|
||||
|
||||
- The stream provider performs one archive metadata pass and one payload pass
|
||||
per `.eti`, then frames entry boundaries for the receiver.
|
||||
- Non-solid and solid archives both install into `local/` without committing a
|
||||
root archive or root `version.ini`, so the receiver is installed but not a
|
||||
downloadable source.
|
||||
- S41 verifies the fixture is actually solid inside the source container, so
|
||||
solid handling stays covered by the same Docker harness as the existing
|
||||
streamed-install scenarios.
|
||||
|
||||
## Run Log
|
||||
|
||||
### 2026-06-07 - Solid Streamed Install Coverage (S41)
|
||||
|
||||
- Code under test added `fixture-solid/cnctw`, a real solid RAR `.eti`, plus
|
||||
S41 in `run_extended_scenarios.py`.
|
||||
- Gates before Docker: `just fmt`, `git diff --check`, and
|
||||
`python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py`
|
||||
passed.
|
||||
- Runner:
|
||||
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image`
|
||||
passed against the rebuilt `lanspread-peer-cli:dev` image.
|
||||
- S41 verified the source archive with `unrar lt -cfg-` inside the source
|
||||
container; the archive reported `Details: RAR 5, solid`.
|
||||
- The streamed install finished with `downloaded=false`, `installed=true`,
|
||||
`availability=LocalOnly`, no root `version.ini`, and no root `cnctw.eti`.
|
||||
- The client received `118` streamed file bytes, matching the extracted solid
|
||||
entries. Payload SHA-256 hashes matched `unrar p` output:
|
||||
`88764c9a6c9b5b846b4323cf7725cb7fd70766ddd7fba4168332804a839fa193`
|
||||
(`bin/cnctw-solid-payload.bin`) and
|
||||
`44afc308269b2381b7c707a056dd8d9d393274108ac4d880237fa6772c861d7a`
|
||||
(`data/cnctw-solid-assets.dat`).
|
||||
|
||||
### 2026-06-07 - Streamed Install Prototype (S39-S40)
|
||||
|
||||
- Code under test added `stream-install` to `lanspread-peer-cli`, a peer
|
||||
|
||||
Binary file not shown.
@@ -0,0 +1 @@
|
||||
20160128
|
||||
@@ -1,5 +1,5 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the peer-cli scenarios S1-S40 through Docker."""
|
||||
"""Run the peer-cli scenarios S1-S41 through Docker."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -328,6 +328,7 @@ class Runner:
|
||||
("S37", self.s37_single_source_download_throughput),
|
||||
("S39", self.s39_streamed_install_local_only),
|
||||
("S40", self.s40_streamed_receiver_not_source),
|
||||
("S41", self.s41_solid_archive_streamed_install),
|
||||
]
|
||||
|
||||
for scenario_id, scenario in scenarios:
|
||||
@@ -1171,6 +1172,84 @@ class Runner:
|
||||
f"and download errored '{err['error']}'"
|
||||
)
|
||||
|
||||
def s41_solid_archive_streamed_install(self) -> str:
|
||||
source_dir = self.fixture_root / "s41-solid-source"
|
||||
source_game = source_dir / "cnctw"
|
||||
shutil.copytree(FIXTURES / "fixture-solid" / "cnctw", source_game)
|
||||
|
||||
source = self.peer("s41-solid-source", games_dir=source_dir)
|
||||
assert_peer_rar_archive_solid(source, "cnctw")
|
||||
client = self.peer("s41-solid-client")
|
||||
connect_many(client, [source])
|
||||
wait_remote_game(client, "cnctw", peer_count=1, version="20160128")
|
||||
|
||||
waiter = LineWaiter(len(client.output))
|
||||
client.send({"cmd": "stream-install", "game_id": "cnctw"})
|
||||
client.wait_for(
|
||||
event_is("got-game-files", "cnctw"),
|
||||
timeout=20,
|
||||
description="got solid cnctw files",
|
||||
waiter=waiter,
|
||||
)
|
||||
client.wait_for(
|
||||
event_is("download-finished", "cnctw"),
|
||||
timeout=60,
|
||||
description="solid stream finish cnctw",
|
||||
waiter=waiter,
|
||||
)
|
||||
client.wait_for(
|
||||
event_is("install-finished", "cnctw"),
|
||||
timeout=30,
|
||||
description="solid stream install cnctw",
|
||||
waiter=waiter,
|
||||
)
|
||||
|
||||
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
|
||||
assert_game_state(
|
||||
game,
|
||||
downloaded=False,
|
||||
installed=True,
|
||||
availability="LocalOnly",
|
||||
)
|
||||
game_root = client.host_games_dir / "cnctw"
|
||||
assert_not_exists(game_root / "version.ini")
|
||||
assert_not_exists(game_root / "cnctw.eti")
|
||||
|
||||
expected = {
|
||||
"bin/cnctw-solid-payload.bin": unrar_entry_sha256(
|
||||
source, "cnctw", "bin/cnctw-solid-payload.bin"
|
||||
),
|
||||
"data/cnctw-solid-assets.dat": unrar_entry_sha256(
|
||||
source, "cnctw", "data/cnctw-solid-assets.dat"
|
||||
),
|
||||
}
|
||||
actual = {
|
||||
rel: sha256_file(game_root / "local" / rel)
|
||||
for rel in expected
|
||||
}
|
||||
if actual != expected:
|
||||
raise ScenarioError(
|
||||
f"solid streamed payload hashes mismatched: {actual} != {expected}"
|
||||
)
|
||||
|
||||
streamed_bytes = sum(
|
||||
int(item.get("data", {}).get("length", 0))
|
||||
for item in client.output
|
||||
if item.get("type") == "event"
|
||||
and item.get("event") == "download-chunk-finished"
|
||||
and item.get("data", {}).get("game_id") == "cnctw"
|
||||
)
|
||||
expected_bytes = sum((game_root / "local" / rel).stat().st_size for rel in expected)
|
||||
if streamed_bytes != expected_bytes:
|
||||
raise ScenarioError(
|
||||
f"solid streamed byte count mismatch: {streamed_bytes} != {expected_bytes}"
|
||||
)
|
||||
|
||||
return (
|
||||
"solid cnctw archive streamed through one local-only install; "
|
||||
f"payload hashes={actual}, bytes={streamed_bytes}"
|
||||
)
|
||||
|
||||
|
||||
def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]:
|
||||
result = subprocess.run(
|
||||
@@ -1307,6 +1386,22 @@ def unrar_entry_sha256(peer: Peer, game_id: str, relative_path: str) -> str:
|
||||
return output.split()[0]
|
||||
|
||||
|
||||
def assert_peer_rar_archive_solid(peer: Peer, game_id: str) -> None:
|
||||
output = peer.docker_exec(
|
||||
"unrar",
|
||||
"lt",
|
||||
"-cfg-",
|
||||
f"/games/{game_id}/{game_id}.eti",
|
||||
).stdout
|
||||
for line in output.splitlines():
|
||||
stripped = line.strip()
|
||||
if stripped.startswith("Details:"):
|
||||
if "solid" in stripped.lower():
|
||||
return
|
||||
raise ScenarioError(f"RAR archive is not solid: {game_id}")
|
||||
raise ScenarioError(f"RAR archive details were not reported: {game_id}")
|
||||
|
||||
|
||||
def format_bytes(size: int) -> str:
|
||||
return f"{size / 1024 / 1024 / 1024:.2f} GiB"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user