Feature/streamed install prototype #27
+6
-10
@@ -21,17 +21,13 @@ product-ready.
|
|||||||
the technical listing, so CLI and GUI callers use one purpose-built provider
|
the technical listing, so CLI and GUI callers use one purpose-built provider
|
||||||
instead of a per-file extraction loop.
|
instead of a per-file extraction loop.
|
||||||
|
|
||||||
3. **Handle solid archives deliberately**
|
3. **Done — Handle solid archives deliberately**
|
||||||
|
|
||||||
The provider exposes the RAR `solid` flag in `ArchiveBegin`; the remaining
|
The provider exposes the RAR `solid` flag in `ArchiveBegin` and always uses
|
||||||
work is to make that flag a tested policy decision instead of an incidental
|
one sequential payload pass per archive, which is the safe path for solid
|
||||||
stream attribute. Add archive inspection that decides:
|
archives. S41 now verifies a real solid RAR fixture through the Docker
|
||||||
|
peer-cli flow, including local-only final state, absent root archive/sentinel,
|
||||||
- non-solid: current one-pass streaming is fine
|
byte count, and extracted payload SHA-256 hashes.
|
||||||
- solid: prove and enforce one sequential archive pass only
|
|
||||||
|
|
||||||
This is the big architectural fork we discussed: keep non-solid and solid
|
|
||||||
archive handling explicit in the provider contract and scenario coverage.
|
|
||||||
|
|
||||||
4. **Decide the integrity model**
|
4. **Decide the integrity model**
|
||||||
|
|
||||||
|
|||||||
@@ -48,6 +48,7 @@ for deterministic local runs; mDNS/macvlan remains an environment smoke path.
|
|||||||
| S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. |
|
| S38 | First-play launch-setting stamping | `fixture-persona/css` ships a real RAR `.eti` whose tree buries a CRLF `SmartSteamEmu.ini` with a stub `PersonaName` line under `engine/bin/win64/steam_settings/`, plus a stub `account_name.txt` and `language.txt` under `profiles/local/`. A peer installs `css` (with `--unrar`), then sends `play css` with a username and language, then `play css` again. | After install the marker `games/css/launch_settings_applied` is absent and the stub files are intact under `local/`. The first `play` returns `already_applied=false` with `account_name_written`, `language_written`, and `persona_name_written` all true; the deep `SmartSteamEmu.ini` `PersonaName` value becomes the username with its `\r\n` ending and sibling lines preserved, `account_name.txt` becomes the username, `language.txt` becomes the passed language, and the marker now exists. A second `play` returns `already_applied=true`, rewrites nothing, and leaves the files untouched even if their values were reset externally. |
|
||||||
| S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. |
|
| S39 | Streamed install without keeping archive payload | Empty client connects to `fixture-bravo`, then sends `stream-install cnctw`. The source has real RAR `.eti` payload entries under `bin/` and `data/`; the receiver uses the container-bundled `unrar` stream provider. | Client emits `got-game-files`, `download-begin`, streamed `download-chunk-finished`, `download-finished`, `install-begin`, and `install-finished`. Local `cnctw` is `downloaded=false`, `installed=true`, `availability=LocalOnly`; root `version.ini` and `.eti` are absent; `local/bin/cnctw-payload.bin` and `local/data/cnctw-assets.dat` match `unrar p` output by SHA-256. |
|
||||||
| S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. |
|
| S40 | Streamed install receiver is not a peer source | After S39, a third peer connects only to the streamed-install receiver. | The third peer may see the receiver's local-only summary in peer snapshots, but `list-games` remote aggregation does not expose `cnctw` as downloadable, `peer_count` remains zero/absent, and attempting `download cnctw` fails with no local files created. |
|
||||||
|
| S41 | Solid archive streamed install | Empty client connects to a peer serving `fixture-solid/cnctw`, whose `.eti` is a real solid RAR archive. The receiver uses the container-bundled `unrar` stream provider. | The fixture is verified as solid with `unrar lt`; streamed install finishes with `downloaded=false`, `installed=true`, `availability=LocalOnly`; root archive and `version.ini` are absent; streamed byte count equals the extracted solid entries; local payload SHA-256 hashes match `unrar p` output. |
|
||||||
|
|
||||||
## Version-Skew Contract
|
## Version-Skew Contract
|
||||||
|
|
||||||
@@ -112,8 +113,42 @@ Use S38 to pin down how launcher settings are stamped into an installed game:
|
|||||||
crate's `launch_settings` unit tests cover the rewrite, line-ending, and
|
crate's `launch_settings` unit tests cover the rewrite, line-ending, and
|
||||||
marker logic deterministically.
|
marker logic deterministically.
|
||||||
|
|
||||||
|
## Streamed Install Archive Contract
|
||||||
|
|
||||||
|
Use S39-S41 to pin down low-disk streamed installs:
|
||||||
|
|
||||||
|
- The stream provider performs one archive metadata pass and one payload pass
|
||||||
|
per `.eti`, then frames entry boundaries for the receiver.
|
||||||
|
- Non-solid and solid archives both install into `local/` without committing a
|
||||||
|
root archive or root `version.ini`, so the receiver is installed but not a
|
||||||
|
downloadable source.
|
||||||
|
- S41 verifies the fixture is actually solid inside the source container, so
|
||||||
|
solid handling stays covered by the same Docker harness as the existing
|
||||||
|
streamed-install scenarios.
|
||||||
|
|
||||||
## Run Log
|
## Run Log
|
||||||
|
|
||||||
|
### 2026-06-07 - Solid Streamed Install Coverage (S41)
|
||||||
|
|
||||||
|
- Code under test added `fixture-solid/cnctw`, a real solid RAR `.eti`, plus
|
||||||
|
S41 in `run_extended_scenarios.py`.
|
||||||
|
- Gates before Docker: `just fmt`, `git diff --check`, and
|
||||||
|
`python3 -m py_compile crates/lanspread-peer-cli/scripts/run_extended_scenarios.py`
|
||||||
|
passed.
|
||||||
|
- Runner:
|
||||||
|
`python3 crates/lanspread-peer-cli/scripts/run_extended_scenarios.py S41 --build-image`
|
||||||
|
passed against the rebuilt `lanspread-peer-cli:dev` image.
|
||||||
|
- S41 verified the source archive with `unrar lt -cfg-` inside the source
|
||||||
|
container; the archive reported `Details: RAR 5, solid`.
|
||||||
|
- The streamed install finished with `downloaded=false`, `installed=true`,
|
||||||
|
`availability=LocalOnly`, no root `version.ini`, and no root `cnctw.eti`.
|
||||||
|
- The client received `118` streamed file bytes, matching the extracted solid
|
||||||
|
entries. Payload SHA-256 hashes matched `unrar p` output:
|
||||||
|
`88764c9a6c9b5b846b4323cf7725cb7fd70766ddd7fba4168332804a839fa193`
|
||||||
|
(`bin/cnctw-solid-payload.bin`) and
|
||||||
|
`44afc308269b2381b7c707a056dd8d9d393274108ac4d880237fa6772c861d7a`
|
||||||
|
(`data/cnctw-solid-assets.dat`).
|
||||||
|
|
||||||
### 2026-06-07 - Streamed Install Prototype (S39-S40)
|
### 2026-06-07 - Streamed Install Prototype (S39-S40)
|
||||||
|
|
||||||
- Code under test added `stream-install` to `lanspread-peer-cli`, a peer
|
- Code under test added `stream-install` to `lanspread-peer-cli`, a peer
|
||||||
|
|||||||
Binary file not shown.
@@ -0,0 +1 @@
|
|||||||
|
20160128
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
#!/usr/bin/env python3
|
#!/usr/bin/env python3
|
||||||
"""Run the peer-cli scenarios S1-S40 through Docker."""
|
"""Run the peer-cli scenarios S1-S41 through Docker."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -328,6 +328,7 @@ class Runner:
|
|||||||
("S37", self.s37_single_source_download_throughput),
|
("S37", self.s37_single_source_download_throughput),
|
||||||
("S39", self.s39_streamed_install_local_only),
|
("S39", self.s39_streamed_install_local_only),
|
||||||
("S40", self.s40_streamed_receiver_not_source),
|
("S40", self.s40_streamed_receiver_not_source),
|
||||||
|
("S41", self.s41_solid_archive_streamed_install),
|
||||||
]
|
]
|
||||||
|
|
||||||
for scenario_id, scenario in scenarios:
|
for scenario_id, scenario in scenarios:
|
||||||
@@ -1171,6 +1172,84 @@ class Runner:
|
|||||||
f"and download errored '{err['error']}'"
|
f"and download errored '{err['error']}'"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def s41_solid_archive_streamed_install(self) -> str:
|
||||||
|
source_dir = self.fixture_root / "s41-solid-source"
|
||||||
|
source_game = source_dir / "cnctw"
|
||||||
|
shutil.copytree(FIXTURES / "fixture-solid" / "cnctw", source_game)
|
||||||
|
|
||||||
|
source = self.peer("s41-solid-source", games_dir=source_dir)
|
||||||
|
assert_peer_rar_archive_solid(source, "cnctw")
|
||||||
|
client = self.peer("s41-solid-client")
|
||||||
|
connect_many(client, [source])
|
||||||
|
wait_remote_game(client, "cnctw", peer_count=1, version="20160128")
|
||||||
|
|
||||||
|
waiter = LineWaiter(len(client.output))
|
||||||
|
client.send({"cmd": "stream-install", "game_id": "cnctw"})
|
||||||
|
client.wait_for(
|
||||||
|
event_is("got-game-files", "cnctw"),
|
||||||
|
timeout=20,
|
||||||
|
description="got solid cnctw files",
|
||||||
|
waiter=waiter,
|
||||||
|
)
|
||||||
|
client.wait_for(
|
||||||
|
event_is("download-finished", "cnctw"),
|
||||||
|
timeout=60,
|
||||||
|
description="solid stream finish cnctw",
|
||||||
|
waiter=waiter,
|
||||||
|
)
|
||||||
|
client.wait_for(
|
||||||
|
event_is("install-finished", "cnctw"),
|
||||||
|
timeout=30,
|
||||||
|
description="solid stream install cnctw",
|
||||||
|
waiter=waiter,
|
||||||
|
)
|
||||||
|
|
||||||
|
game = wait_local_game(client, "cnctw", downloaded=False, installed=True)
|
||||||
|
assert_game_state(
|
||||||
|
game,
|
||||||
|
downloaded=False,
|
||||||
|
installed=True,
|
||||||
|
availability="LocalOnly",
|
||||||
|
)
|
||||||
|
game_root = client.host_games_dir / "cnctw"
|
||||||
|
assert_not_exists(game_root / "version.ini")
|
||||||
|
assert_not_exists(game_root / "cnctw.eti")
|
||||||
|
|
||||||
|
expected = {
|
||||||
|
"bin/cnctw-solid-payload.bin": unrar_entry_sha256(
|
||||||
|
source, "cnctw", "bin/cnctw-solid-payload.bin"
|
||||||
|
),
|
||||||
|
"data/cnctw-solid-assets.dat": unrar_entry_sha256(
|
||||||
|
source, "cnctw", "data/cnctw-solid-assets.dat"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
actual = {
|
||||||
|
rel: sha256_file(game_root / "local" / rel)
|
||||||
|
for rel in expected
|
||||||
|
}
|
||||||
|
if actual != expected:
|
||||||
|
raise ScenarioError(
|
||||||
|
f"solid streamed payload hashes mismatched: {actual} != {expected}"
|
||||||
|
)
|
||||||
|
|
||||||
|
streamed_bytes = sum(
|
||||||
|
int(item.get("data", {}).get("length", 0))
|
||||||
|
for item in client.output
|
||||||
|
if item.get("type") == "event"
|
||||||
|
and item.get("event") == "download-chunk-finished"
|
||||||
|
and item.get("data", {}).get("game_id") == "cnctw"
|
||||||
|
)
|
||||||
|
expected_bytes = sum((game_root / "local" / rel).stat().st_size for rel in expected)
|
||||||
|
if streamed_bytes != expected_bytes:
|
||||||
|
raise ScenarioError(
|
||||||
|
f"solid streamed byte count mismatch: {streamed_bytes} != {expected_bytes}"
|
||||||
|
)
|
||||||
|
|
||||||
|
return (
|
||||||
|
"solid cnctw archive streamed through one local-only install; "
|
||||||
|
f"payload hashes={actual}, bytes={streamed_bytes}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]:
|
def run(command: list[str], description: str) -> subprocess.CompletedProcess[str]:
|
||||||
result = subprocess.run(
|
result = subprocess.run(
|
||||||
@@ -1307,6 +1386,22 @@ def unrar_entry_sha256(peer: Peer, game_id: str, relative_path: str) -> str:
|
|||||||
return output.split()[0]
|
return output.split()[0]
|
||||||
|
|
||||||
|
|
||||||
|
def assert_peer_rar_archive_solid(peer: Peer, game_id: str) -> None:
|
||||||
|
output = peer.docker_exec(
|
||||||
|
"unrar",
|
||||||
|
"lt",
|
||||||
|
"-cfg-",
|
||||||
|
f"/games/{game_id}/{game_id}.eti",
|
||||||
|
).stdout
|
||||||
|
for line in output.splitlines():
|
||||||
|
stripped = line.strip()
|
||||||
|
if stripped.startswith("Details:"):
|
||||||
|
if "solid" in stripped.lower():
|
||||||
|
return
|
||||||
|
raise ScenarioError(f"RAR archive is not solid: {game_id}")
|
||||||
|
raise ScenarioError(f"RAR archive details were not reported: {game_id}")
|
||||||
|
|
||||||
|
|
||||||
def format_bytes(size: int) -> str:
|
def format_bytes(size: int) -> str:
|
||||||
return f"{size / 1024 / 1024 / 1024:.2f} GiB"
|
return f"{size / 1024 / 1024 / 1024:.2f} GiB"
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user