feat(catalog): separate generated production and local test authority
The launcher previously defaulted to fixtures and production generation used the source database directly. Generate a separate database and manifest set from package directories, validating staged output with the application loader before installation. Keep strict selection by default, with independent opt-ins for missing games and package version overrides in the copied database. Share database filtering and staging with the fixture publisher, and include source metadata and generation modes in the publication cache. Make normal runs consume existing production authority. Add an explicit local test recipe with separate resources, app settings, and a compiled startup game directory. Build-time gates exclude local authority from production and require Tauri development mode. Document the generation and launch workflows. Clear generated catalog copies before Tauri copies the selected resource tree so mode switches and reduced catalogs cannot retain stale manifests. Watch the copied files to repair deletion and preserve prior output mtimes only when the bytes are unchanged, allowing subsequent builds to become fresh. Test Plan: - `just fmt` -- passed. - `just clippy` -- passed with warnings denied. - `just test` -- workspace tests passed using fixture authority. - `just frontend-test` -- 94 passed. - `python3 -m unittest discover -s tools -p 'test_catalog_source_cache.py'` -- 4 passed. - `git diff --cached --check` -- passed. - Interactive GUI launches and production bundles were not exercised.
This commit is contained in:
ddidderr committed
1 parent
ab0c95b80f
commit
6b65a66465
21 files changed
+1777
-273
No files matched your search
@@ -31,12 +31,30 @@ builds.
|
||||
|
||||
## Daily development
|
||||
|
||||
Run the desktop app with the checked-in test catalog:
|
||||
Run the desktop app with the already-generated production catalog:
|
||||
|
||||
```bash
|
||||
just run
|
||||
```
|
||||
|
||||
Plain `just` (or the `j` shell alias) does the same. It checks the existing
|
||||
catalog and starts the launcher; no source package directory or environment
|
||||
variables are needed. Select the folder containing your games in the launcher.
|
||||
|
||||
To test directly with a local game folder and a separate catalog:
|
||||
|
||||
```bash
|
||||
just run-local /srv/lanspread/test-games
|
||||
```
|
||||
|
||||
This selects that game folder and generates or reuses test authority under
|
||||
`crates/lanspread-tauri-deno-ts/src-tauri/local-catalog/`, with its own cache in
|
||||
`.lanspread/catalog-cache/local.json`. Missing catalog games and differing
|
||||
`version.ini` values are allowed; directory names must still match known game
|
||||
IDs. The console identifies this as local test mode. Its opt-in is checked at
|
||||
build time and cannot enable local authority in a production build. The local
|
||||
launcher uses separate test settings and a window title marked `LOCAL TEST`.
|
||||
|
||||
Build a launcher from the already-published production catalog without creating
|
||||
an installer:
|
||||
|
||||
@@ -51,14 +69,10 @@ the same no-bundle launcher:
|
||||
just build-production /srv/lanspread/packages
|
||||
```
|
||||
|
||||
To make the default `just` command use a real package tree, set the directory
|
||||
before invoking it. The command generates or reuses the production catalog and
|
||||
then runs the launcher with the production resource map:
|
||||
To regenerate or reuse production authority from a package tree and then run:
|
||||
|
||||
```bash
|
||||
LANSPREAD_GAMES_DIR=/srv/lanspread/packages just
|
||||
# Equivalent Just-variable form:
|
||||
just --set GAMES_DIR /srv/lanspread/packages
|
||||
just run-production /srv/lanspread/packages
|
||||
```
|
||||
|
||||
The fixture-only paths remain explicit:
|
||||
@@ -77,16 +91,21 @@ just test
|
||||
just frontend-test
|
||||
```
|
||||
|
||||
`just run` uses the fixture path only when `LANSPREAD_GAMES_DIR` is unset.
|
||||
`just build` always uses the production resource map and production profile; use
|
||||
`just build-fixture` when a fixture build is intended. Tauri's build script
|
||||
still rejects fixture resources unless the explicit fixture opt-in is present.
|
||||
rejects fixture and local resources unless their explicit development opt-in is
|
||||
present, and the production profile rejects both modes. Before Tauri copies
|
||||
catalog resources, the build script clears the generated `game.db` and
|
||||
`manifests/` in the active Cargo profile directory. This keeps
|
||||
fixture/production switches and reduced production catalogs free of stale
|
||||
manifests. Cargo also watches these generated copies, so deleting them causes
|
||||
the next build to recreate them even when the source catalog is unchanged.
|
||||
|
||||
## Catalog checksums and production manifests
|
||||
|
||||
Lanspread does not trust file descriptions received from peers. The release
|
||||
catalog contains the expected bytes for each game version. The catalog publisher
|
||||
derives that authority from the canonical game packages and writes:
|
||||
catalog contains the expected bytes for each included game version. The catalog
|
||||
publisher derives that authority from the canonical game packages and writes:
|
||||
|
||||
- one `<game_id>.json` manifest containing every ordinary file's size, full-file
|
||||
BLAKE3, and BLAKE3 for each 128 MiB transfer chunk;
|
||||
@@ -129,32 +148,73 @@ LANSPREAD_UNRAR=/absolute/path/to/unrar \
|
||||
just catalog-generate-production /srv/lanspread/packages
|
||||
```
|
||||
|
||||
### Generate and verify all checksums
|
||||
### Generate and verify checksums
|
||||
|
||||
Generate the complete production manifest set:
|
||||
Generate the complete production catalog:
|
||||
|
||||
```bash
|
||||
just catalog-generate-production /srv/lanspread/packages
|
||||
```
|
||||
|
||||
The `--all` recipe keeps a local metadata-only stamp in
|
||||
The default remains strict: every game in the checked-in source `game.db` must
|
||||
have a package directory. To build a reduced catalog from only the package
|
||||
directories currently available, pass the explicit optional mode:
|
||||
|
||||
```bash
|
||||
just catalog-generate-production /srv/lanspread/packages allow-missing-games
|
||||
```
|
||||
|
||||
`allow-missing-games` omits absent package directories and reports their count;
|
||||
it still rejects a package whose `version.ini` differs from the source catalog.
|
||||
Version mismatches have their own independent opt-in:
|
||||
|
||||
```bash
|
||||
just catalog-generate-production /srv/lanspread/packages allow-version-mismatch
|
||||
```
|
||||
|
||||
That mode reports mismatched game IDs and uses each package's actual
|
||||
`version.ini` value in the generated database copy, while still requiring every
|
||||
package directory. Combine both options for an older, incomplete package corpus:
|
||||
|
||||
```bash
|
||||
just catalog-generate-production /srv/lanspread/packages \
|
||||
allow-missing-games allow-version-mismatch
|
||||
```
|
||||
|
||||
Invalid directories and unreadable or non-UTF-8 version files always fail. The
|
||||
generator copies and filters `game.db` to exactly the included games, then
|
||||
generates and validates their complete manifest/index set. At least one matching
|
||||
source-catalog game must be present. The checked-in full database is never
|
||||
modified; the coherent generated output is installed under
|
||||
`crates/lanspread-tauri-deno-ts/src-tauri/production-catalog/`.
|
||||
|
||||
The mode can also be used while running or building real data:
|
||||
|
||||
```bash
|
||||
just run-production /srv/lanspread/packages \
|
||||
allow-missing-games allow-version-mismatch
|
||||
just build-production /srv/lanspread/packages \
|
||||
allow-missing-games allow-version-mismatch
|
||||
```
|
||||
|
||||
The recipe keeps a local metadata-only stamp in
|
||||
`.lanspread/catalog-cache/production.json`. It compares the package tree's
|
||||
paths, entry types, sizes, and nanosecond mtimes, plus the catalog database and
|
||||
`unrar` metadata. If those values and the published output still match, the
|
||||
expensive package hashing and extraction step is skipped. This is only a
|
||||
performance cache: `catalog-check-production` and every production build still
|
||||
validate the catalog authority. Force a refresh when needed:
|
||||
paths, entry types, sizes, and nanosecond mtimes, plus the source and generated
|
||||
catalog databases, generation mode, and `unrar` metadata. If those values and
|
||||
the published output still match, the expensive package hashing and extraction
|
||||
step is skipped. This is only a performance cache: `catalog-check-production`
|
||||
and every production build still validate the catalog authority. Force a refresh
|
||||
when needed:
|
||||
|
||||
```bash
|
||||
LANSPREAD_CATALOG_FORCE=1 just catalog-generate-production /srv/lanspread/packages
|
||||
```
|
||||
|
||||
The publisher first checks every selected package's `version.ini` against
|
||||
`game.db`, then independently reads and hashes each selected package twice
|
||||
before publication. Under a durable publication marker, it atomically replaces
|
||||
each manifest and writes the complete content index last. Interruption leaves
|
||||
the marker in place so later checks and builds fail closed instead of accepting
|
||||
a mixed generation.
|
||||
The publisher first checks every included package's `version.ini` against the
|
||||
generated `game.db`, then independently reads and hashes each package twice
|
||||
before publication. It creates the complete database, manifest, and index set in
|
||||
staging and validates it with the application loader before atomically replacing
|
||||
the previous production catalog directory.
|
||||
|
||||
Verify the published database/index/manifest set without rereading the package
|
||||
corpus:
|
||||
@@ -177,10 +237,11 @@ just catalog-generate-production-game /srv/lanspread/packages alienswarm
|
||||
just catalog-check-production
|
||||
```
|
||||
|
||||
Incremental generation refuses to bootstrap a partial catalog. Under its durable
|
||||
Incremental generation refuses to bootstrap a catalog. Under its durable
|
||||
publication marker it revalidates every unselected manifest/index identity,
|
||||
rebuilds the complete index, and then publishes the selected game. Use the full
|
||||
generation command for a new catalog or any intentionally broad catalog change.
|
||||
rebuilds the complete generated index, and then publishes the selected game. Use
|
||||
the main generation command for a new catalog or any intentionally broad catalog
|
||||
change.
|
||||
|
||||
### Build a production bundle
|
||||
|
||||
@@ -191,25 +252,24 @@ just bundle
|
||||
```
|
||||
|
||||
`just bundle` repeats the production catalog gate before Tauri packaging. It
|
||||
never falls back to fixture manifests. This checkout does not include the
|
||||
canonical 186-game package corpus, so the command intentionally fails until that
|
||||
external corpus has been processed into
|
||||
`crates/lanspread-tauri-deno-ts/src-tauri/manifests/`.
|
||||
never falls back to fixture manifests. This checkout does not include generated
|
||||
production authority, so the command intentionally fails until an external
|
||||
package corpus has been processed into
|
||||
`crates/lanspread-tauri-deno-ts/src-tauri/production-catalog/`.
|
||||
|
||||
## Fixture catalogs and peer-CLI acceptance
|
||||
|
||||
The committed fixture profiles are test authority only:
|
||||
|
||||
```bash
|
||||
just fixture-catalog-check # default GUI/CLI profile
|
||||
just fixture-catalog-check # explicit fixture GUI/CLI profile
|
||||
just fixture-catalogs-check # default + solid + multi + unknown profiles
|
||||
just fixture-catalogs # explicitly regenerate all committed profiles
|
||||
```
|
||||
|
||||
The solid, multi-archive, and unknown-game profiles intentionally describe
|
||||
different package bytes, so the peer-CLI matrix checks all of them before
|
||||
building its Docker image. Normal GUI development needs only the default
|
||||
profile.
|
||||
building its Docker image. `just run-fixture` uses the default fixture profile.
|
||||
|
||||
Build or run the JSONL peer harness:
|
||||
|
||||
@@ -239,13 +299,15 @@ just peer-cli-charlie
|
||||
| Command | Purpose |
|
||||
| --------------------------------------------------- | ------------------------------------------------------------------------ |
|
||||
| `just setup` | Install the Tauri CLI and frontend dependencies. |
|
||||
| `just` / `just run` | Run fixtures, or real data when `LANSPREAD_GAMES_DIR` is set. |
|
||||
| `just` / `just run` | Check the existing production catalog and run the GUI. |
|
||||
| `just run-local GAMES_DIR` | Generate/reuse separate local test authority and run with that folder. |
|
||||
| `just run-production PACKAGES_DIR [MODE...]` | Generate/reuse production authority and run the GUI. |
|
||||
| `just run-fixture` | Check the default fixture catalog and run the GUI. |
|
||||
| `just build` | Validate production authority and build without bundling. |
|
||||
| `just build-production PACKAGES_DIR` | Generate/reuse production authority and build without bundling. |
|
||||
| `just build-production PACKAGES_DIR [MODE]` | Generate/reuse production authority and build without bundling. |
|
||||
| `just build-fixture` | Check the default fixture catalog and build without bundling. |
|
||||
| `just bundle` | Check production authority and build production bundles. |
|
||||
| `just catalog-generate-production PACKAGES_DIR` | Hash, cache, and publish the complete production package corpus. |
|
||||
| `just catalog-generate-production DIR [MODE...]` | Publish packages with explicit missing/version mismatch opt-ins. |
|
||||
| `just catalog-generate-production-game DIR GAME_ID` | Safely regenerate one game in an already complete catalog. |
|
||||
| `just catalog-check-production` | Validate all production database/index/manifest artifacts. |
|
||||
| `just fixture-catalog-check` | Validate the default development fixture catalog. |
|
||||
|
||||
Reference in new issue
Block a user