ddidderr/lanspread
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

build(catalog): expose production publishing workflow

Browse Source 
Add small Justfile entry points for generating the complete production
catalog authority or safely regenerating one game. Keep fixture validation
separate and allow operators to select a trusted unrar executable.

Document the complete package layout, BLAKE3 and ContentId model,
full/incremental publication procedure, verification and bundle gates, and
peer-CLI workflows. Record the fresh final local acceptance run without
presenting Docker throughput as physical-LAN evidence.

Test Plan:
- just fixture-catalogs-check
- just test
- just clippy
- just frontend-test
- just build
- LANSPREAD_S37_MIN_MIB_PER_S=100 just peer-cli-tests
- just catalog-check-production (expected fail-closed: production manifests absent)
- deno fmt --check README.md organize/testing/PEER_CLI_SCENARIOS.md
- rumdl check --flavor commonmark README.md organize/testing/PEER_CLI_SCENARIOS.md
- just --fmt --check
- git diff --check
This commit is contained in:
ddidderr
2026-08-12 23:27:40 +02:00
parent cf3a11c732
commit 3bdd8d06da
3 changed files with 220 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+174 -23
View File
@@ -16,7 +16,7 @@ responder-owned, revisioned snapshots over identity-pinned endpoints; file and
streamed-install requests name the catalog's exact `ContentId`. There are no
legacy wire fallbacks or repository-shared identity keys.
## Build / install
## First-time setup
Install Rust, Deno, and `just` first, then bootstrap the project:
@@ -24,10 +24,14 @@ Install Rust, Deno, and `just` first, then bootstrap the project:
just setup
```
That installs the Tauri CLI with `cargo install tauri-cli` and installs the
Deno/npm dependencies from `crates/lanspread-tauri-deno-ts`.
That installs the Tauri CLI and the locked frontend dependencies. All routine
repository work should go through `just`; the recipes carry the catalog mode and
Tauri resource configuration needed to keep fixture authority out of production
builds.
Run the desktop app in development mode:
## Daily development
Run the desktop app with the checked-in development catalog:
```bash
just run
@@ -39,30 +43,177 @@ Build without bundling:
just build
```
Create production bundles:
Run the standard checks before committing:
```bash
just fmt
just clippy
just test
just frontend-test
```
`just run` and `just build` first check the default development catalog. The
workspace Rust commands explicitly opt into that exact fixture resource map so
Tauri's build script does not mistake test data for production authority.
## Catalog checksums and production manifests
Lanspread does not trust file descriptions received from peers. The release
catalog contains the expected bytes for each game version. The catalog publisher
derives that authority from the canonical game packages and writes:
- one `<game_id>.json` manifest containing every ordinary file's size, full-file
BLAKE3, and BLAKE3 for each 128 MiB transfer chunk;
- BLAKE3 and size for every file obtained by extracting the root-level `.eti`
archives, which authorizes Stream Install output; and
- `catalog-content-index-v1.jsonl`, which binds every catalog game/version to
its complete manifest `ContentId` and Stream Install capability.
The `ContentId` is a digest of the canonical manifest transcript, not merely a
hash of one archive. Never calculate or edit these values by hand.
### Prepare the package tree
Place the canonical packages under one directory, with each direct child named
exactly like the `game_id` in the production `game.db`:
```text
/srv/lanspread/packages/
├── alienswarm/
│ ├── version.ini
│ ├── alienswarm.eti
│ └── ...
├── bf1942/
│ ├── version.ini
│ ├── bf1942.eti
│ └── ...
└── ...
```
Each `version.ini` must match that game's `game_version` in `game.db`. Package
entries must be regular files/directories with canonical portable names; links,
reparse points, special files, aliases, unsafe archive output, or changing bytes
make generation fail closed.
The default publisher uses the checked-in Linux `unrar` sidecar. Select another
trusted executable when generating on a different supported host:
```bash
LANSPREAD_UNRAR=/absolute/path/to/unrar \
just catalog-generate-production /srv/lanspread/packages
```
### Generate and verify all checksums
Generate the complete production manifest set:
```bash
just catalog-generate-production /srv/lanspread/packages
```
The publisher independently reads and hashes each selected package twice before
publication. Under a durable publication marker, it atomically replaces each
manifest and writes the complete content index last. Interruption leaves the
marker in place so later checks and builds fail closed instead of accepting a
mixed generation.
Verify the published database/index/manifest set without rereading the package
corpus:
```bash
just catalog-check-production
```
This is a packaging integrity check, not a substitute for retaining the
canonical packages and the successful generation log as provenance. The output
prints the `game_id`, `game_version`, and `content_id` for every checked game.
### Regenerate one changed game
After a complete manifest set exists, update one game without rehashing every
package:
```bash
just catalog-generate-production-game /srv/lanspread/packages alienswarm
just catalog-check-production
```
Incremental generation refuses to bootstrap a partial catalog. Under its durable
publication marker it revalidates every unselected manifest/index identity,
rebuilds the complete index, and then publishes the selected game. Use the full
generation command for a new catalog or any intentionally broad catalog change.
### Build a production bundle
Only after the production check succeeds, create the installer/bundle:
```bash
just bundle
```
Development commands use the checked-in peer-CLI fixture catalog only after its
database and manifests pass the catalog checker. Production bundles never use
that fixture authority: `just bundle` validates the production `game.db` and
requires a generated manifest for every catalog game before Tauri packaging can
start. This checkout does not include the canonical production package corpus,
so maintainers must generate and independently check those manifests beside the
production packages before a bundle can succeed.
`just bundle` repeats the production catalog gate before Tauri packaging. It
never falls back to fixture manifests. This checkout does not include the
canonical 186-game package corpus, so the command intentionally fails until that
external corpus has been processed into
`crates/lanspread-tauri-deno-ts/src-tauri/manifests/`.
## Fixture catalogs and peer-CLI acceptance
The committed fixture profiles are test authority only:
```bash
just fixture-catalog-check # default GUI/CLI profile
just fixture-catalogs-check # default + solid + multi + unknown profiles
just fixture-catalogs # explicitly regenerate all committed profiles
```
The solid, multi-archive, and unknown-game profiles intentionally describe
different package bytes, so the peer-CLI matrix checks all of them before
building its Docker image. Normal GUI development needs only the default
profile.
Build or run the JSONL peer harness:
```bash
just peer-cli-build
just peer-cli-image
just peer-cli-tests S1 S2 S3
just peer-cli-run alpha
```
Run the entire S1-S49 acceptance matrix by omitting scenario arguments:
```bash
just peer-cli-tests
```
For an interactive three-peer session, start these in separate terminals:
```bash
just peer-cli-alpha
just peer-cli-bravo
just peer-cli-charlie
```
## Important just commands
- `just setup` - install the Tauri CLI and frontend dependencies.
- `just run` - run the Tauri app in dev mode.
- `just build` - build the app without bundling.
- `just bundle` - validate the complete production catalog and create bundles.
- `just fmt` - format Rust, TOML, and the justfile.
- `just clippy` - lint the Rust workspace.
- `just test` - run workspace tests.
- `just frontend-test` - run frontend tests.
- `just peer-cli-build` - build the JSONL peer test harness.
- `just peer-cli-image` - build the peer harness Docker image.
- `just peer-cli-run NAME` - run one peer harness container.
| Command | Purpose |
| --------------------------------------------------- | ------------------------------------------------------------------------ |
| `just setup` | Install the Tauri CLI and frontend dependencies. |
| `just run` | Check the default fixture catalog and run the GUI. |
| `just build` | Check the default fixture catalog and build without bundling. |
| `just bundle` | Check production authority and build production bundles. |
| `just catalog-generate-production PACKAGES_DIR` | Hash and publish the complete production package corpus. |
| `just catalog-generate-production-game DIR GAME_ID` | Safely regenerate one game in an already complete catalog. |
| `just catalog-check-production` | Validate all production database/index/manifest artifacts. |
| `just fixture-catalog-check` | Validate the default development fixture catalog. |
| `just fixture-catalogs-check` | Validate every peer-CLI fixture profile. |
| `just fixture-catalogs` | Regenerate all committed fixture profiles from their packages. |
| `just fmt` | Format Rust, TOML, Markdown, and the Justfile. |
| `just clippy` | Lint every Rust workspace target with warnings denied. |
| `just test` | Run every Rust workspace test target. |
| `just frontend-test` | Run the frontend reducer/lifecycle tests. |
| `just peer-cli-build` | Build the JSONL peer harness. |
| `just peer-cli-image` | Check fixture authority and build the harness Docker image. |
| `just peer-cli-tests [SCENARIO ...]` | Run selected scenarios, or the complete matrix when no IDs are supplied. |
| `just peer-cli-run NAME` | Start one persistent-state interactive harness peer. |
justfile
+30 -8
View File
@@ -6,7 +6,9 @@ default: run
FIXTURE_CATALOG_SOURCE := "crates/lanspread-tauri-deno-ts/src-tauri/game.db"
FIXTURE_CATALOG_ROOT := "crates/lanspread-peer-cli/catalogs"
FIXTURE_UNRAR := "crates/lanspread-tauri-deno-ts/src-tauri/binaries/unrar-x86_64-unknown-linux-gnu"
PRODUCTION_CATALOG_DB := "crates/lanspread-tauri-deno-ts/src-tauri/game.db"
PRODUCTION_MANIFEST_ROOT := "crates/lanspread-tauri-deno-ts/src-tauri/manifests"
CATALOG_UNRAR := env_var_or_default("LANSPREAD_UNRAR", "crates/lanspread-tauri-deno-ts/src-tauri/binaries/unrar-x86_64-unknown-linux-gnu")
TAURI_DEV_CONFIG := '{"bundle":{"resources":{"../../lanspread-peer-cli/catalogs/default/game.db":"game.db","../../lanspread-peer-cli/catalogs/default/manifests/":"manifests/","assets/*":"assets/"}}}'
TAURI_FIXTURE_ENV := "LANSPREAD_USE_FIXTURE_CATALOG=1"
@@ -47,7 +49,7 @@ fixture-catalogs:
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir {{ FIXTURE_CATALOG_ROOT }}/default \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root crates/lanspread-peer-cli/fixtures/fixture-alpha/alienswarm \
--game-root crates/lanspread-peer-cli/fixtures/fixture-alpha/bf1942 \
--game-root crates/lanspread-peer-cli/fixtures/fixture-alpha/ggoo \
@@ -61,31 +63,31 @@ fixture-catalogs:
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir {{ FIXTURE_CATALOG_ROOT }}/solid \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root crates/lanspread-peer-cli/fixtures/fixture-solid/cnctw
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir {{ FIXTURE_CATALOG_ROOT }}/multi \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root crates/lanspread-peer-cli/fixtures/fixture-multi/cnctw
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir {{ FIXTURE_CATALOG_ROOT }}/unknown \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root crates/lanspread-peer-cli/fixtures/fixture-unknown/cod2
fixture-catalog OUTPUT GAME_ROOT:
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir "{{ OUTPUT }}" \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root "{{ GAME_ROOT }}"
fixture-download-only-catalog OUTPUT GAME_ID GAME_ROOT:
cargo run -p lanspread-compat --bin lanspread-fixture-catalog -- \
--source-catalog-db {{ FIXTURE_CATALOG_SOURCE }} \
--output-dir "{{ OUTPUT }}" \
--unrar {{ FIXTURE_UNRAR }} \
--unrar {{ CATALOG_UNRAR }} \
--game-root "{{ GAME_ROOT }}" \
--no-stream-install "{{ GAME_ID }}"
@@ -104,7 +106,27 @@ fixture-catalogs-check: fixture-catalog-check
catalog-check-production:
cargo run -p lanspread-compat --bin lanspread-catalog-publisher -- check \
--catalog-db crates/lanspread-tauri-deno-ts/src-tauri/game.db --all
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--all
# Generate the complete production authority from PACKAGES_DIR/<game_id>/.
catalog-generate-production PACKAGES_DIR:
cargo run -p lanspread-compat --bin lanspread-catalog-publisher -- generate \
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--packages-dir "{{ PACKAGES_DIR }}" \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--unrar "{{ CATALOG_UNRAR }}" \
--all
# Regenerate one game in an already complete production authority.
catalog-generate-production-game PACKAGES_DIR GAME_ID:
cargo run -p lanspread-compat --bin lanspread-catalog-publisher -- generate \
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--packages-dir "{{ PACKAGES_DIR }}" \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--unrar "{{ CATALOG_UNRAR }}" \
--game-id "{{ GAME_ID }}"
frontend-test:
cd crates/lanspread-tauri-deno-ts && deno test --unstable-sloppy-imports tests
organize/testing/PEER_CLI_SCENARIOS.md
+16
View File
@@ -149,6 +149,22 @@ Use S39-S47 to pin down low-disk streamed installs:
## Run Log
### 2026-08-12 - Final local acceptance refresh
- A fresh-image `LANSPREAD_S37_MIN_MIB_PER_S=100 just peer-cli-tests` run passed
every scenario from S1 through S49. S37 transferred and verified the exact
`2,147,483,656` bytes in 17 chunks over `3.487s`, reporting `587.33 MiB/s` and
`4926.92 Mbit/s` against the active 100 MiB/s floor. These are Docker
acceptance measurements on this host and storage, not representative
physical-LAN performance evidence.
- On the same source tree, `just test` passed all 708 workspace tests (including
480 peer and 56 Tauri tests), `just clippy` passed with warnings denied,
`just frontend-test` passed 91/91, `just fixture-catalogs-check` passed, and
`just build` passed the default fixture gate plus the Deno/Vite and
release-mode Tauri build. `just catalog-check-production` separately failed
closed because the external production manifest corpus is absent; no fixture
authority was accepted as a production substitute.
### 2026-08-10 - Phase 5 Unfiltered Acceptance
- A fresh-image `LANSPREAD_S37_MIN_MIB_PER_S=100 just peer-cli-tests` run passed