feat(catalog): separate generated production and local test authority

The launcher previously defaulted to fixtures and production generation used
the source database directly. Generate a separate database and manifest set
from package directories, validating staged output with the application loader
before installation. Keep strict selection by default, with independent
opt-ins for missing games and package version overrides in the copied database.
Share database filtering and staging with the fixture publisher, and include
source metadata and generation modes in the publication cache.

Make normal runs consume existing production authority. Add an explicit local
test recipe with separate resources, app settings, and a compiled startup game
directory. Build-time gates exclude local authority from production and require
Tauri development mode. Document the generation and launch workflows.

Clear generated catalog copies before Tauri copies the selected resource tree
so mode switches and reduced catalogs cannot retain stale manifests. Watch the
copied files to repair deletion and preserve prior output mtimes only when the
bytes are unchanged, allowing subsequent builds to become fresh.

Test Plan:
- `just fmt` -- passed.
- `just clippy` -- passed with warnings denied.
- `just test` -- workspace tests passed using fixture authority.
- `just frontend-test` -- 94 passed.
- `python3 -m unittest discover -s tools -p 'test_catalog_source_cache.py'`
  -- 4 passed.
- `git diff --cached --check` -- passed.
- Interactive GUI launches and production bundles were not exercised.
This commit is contained in:
2026-09-12 19:41:35 +02:00
parent ab0c95b80f
commit 6b65a66465
21 changed files with 1777 additions and 273 deletions
+5
View File
@@ -1,6 +1,11 @@
/target
/.lanspread-peer-cli/
/.lanspread/catalog-cache/
/crates/lanspread-tauri-deno-ts/src-tauri/production-catalog/
/crates/lanspread-tauri-deno-ts/src-tauri/.production-catalog.old-*/
/crates/lanspread-tauri-deno-ts/src-tauri/.production-catalog.tmp-*/
/game.db
/thumbnails/
__pycache__/
# Generated authority for explicit local development runs.
/crates/lanspread-tauri-deno-ts/src-tauri/local-catalog/
+4 -2
View File
@@ -31,8 +31,10 @@ Top-level `Cargo.toml` pins workspace dependency versions; per-crate
Never use normal cargo ... commands, use the just ... commands instead.
- `just run` — run the fixture GUI, or real data when `LANSPREAD_GAMES_DIR` is
set.
- `just run` — run the GUI with the already-published production catalogue.
- `just run-local GAMES_DIR` — create/reuse a separate local test catalogue from
the supplied game files and start a development-only launcher with that
folder.
- `just build` — validate production authority and build the launcher without
bundling.
- `just build-production PACKAGES_DIR` — generate/reuse production authority and
+100 -38
View File
@@ -31,12 +31,30 @@ builds.
## Daily development
Run the desktop app with the checked-in test catalog:
Run the desktop app with the already-generated production catalog:
```bash
just run
```
Plain `just` (or the `j` shell alias) does the same. It checks the existing
catalog and starts the launcher; no source package directory or environment
variables are needed. Select the folder containing your games in the launcher.
To test directly with a local game folder and a separate catalog:
```bash
just run-local /srv/lanspread/test-games
```
This selects that game folder and generates or reuses test authority under
`crates/lanspread-tauri-deno-ts/src-tauri/local-catalog/`, with its own cache in
`.lanspread/catalog-cache/local.json`. Missing catalog games and differing
`version.ini` values are allowed; directory names must still match known game
IDs. The console identifies this as local test mode. Its opt-in is checked at
build time and cannot enable local authority in a production build. The local
launcher uses separate test settings and a window title marked `LOCAL TEST`.
Build a launcher from the already-published production catalog without creating
an installer:
@@ -51,14 +69,10 @@ the same no-bundle launcher:
just build-production /srv/lanspread/packages
```
To make the default `just` command use a real package tree, set the directory
before invoking it. The command generates or reuses the production catalog and
then runs the launcher with the production resource map:
To regenerate or reuse production authority from a package tree and then run:
```bash
LANSPREAD_GAMES_DIR=/srv/lanspread/packages just
# Equivalent Just-variable form:
just --set GAMES_DIR /srv/lanspread/packages
just run-production /srv/lanspread/packages
```
The fixture-only paths remain explicit:
@@ -77,16 +91,21 @@ just test
just frontend-test
```
`just run` uses the fixture path only when `LANSPREAD_GAMES_DIR` is unset.
`just build` always uses the production resource map and production profile; use
`just build-fixture` when a fixture build is intended. Tauri's build script
still rejects fixture resources unless the explicit fixture opt-in is present.
rejects fixture and local resources unless their explicit development opt-in is
present, and the production profile rejects both modes. Before Tauri copies
catalog resources, the build script clears the generated `game.db` and
`manifests/` in the active Cargo profile directory. This keeps
fixture/production switches and reduced production catalogs free of stale
manifests. Cargo also watches these generated copies, so deleting them causes
the next build to recreate them even when the source catalog is unchanged.
## Catalog checksums and production manifests
Lanspread does not trust file descriptions received from peers. The release
catalog contains the expected bytes for each game version. The catalog publisher
derives that authority from the canonical game packages and writes:
catalog contains the expected bytes for each included game version. The catalog
publisher derives that authority from the canonical game packages and writes:
- one `<game_id>.json` manifest containing every ordinary file's size, full-file
BLAKE3, and BLAKE3 for each 128 MiB transfer chunk;
@@ -129,32 +148,73 @@ LANSPREAD_UNRAR=/absolute/path/to/unrar \
just catalog-generate-production /srv/lanspread/packages
```
### Generate and verify all checksums
### Generate and verify checksums
Generate the complete production manifest set:
Generate the complete production catalog:
```bash
just catalog-generate-production /srv/lanspread/packages
```
The `--all` recipe keeps a local metadata-only stamp in
The default remains strict: every game in the checked-in source `game.db` must
have a package directory. To build a reduced catalog from only the package
directories currently available, pass the explicit optional mode:
```bash
just catalog-generate-production /srv/lanspread/packages allow-missing-games
```
`allow-missing-games` omits absent package directories and reports their count;
it still rejects a package whose `version.ini` differs from the source catalog.
Version mismatches have their own independent opt-in:
```bash
just catalog-generate-production /srv/lanspread/packages allow-version-mismatch
```
That mode reports mismatched game IDs and uses each package's actual
`version.ini` value in the generated database copy, while still requiring every
package directory. Combine both options for an older, incomplete package corpus:
```bash
just catalog-generate-production /srv/lanspread/packages \
allow-missing-games allow-version-mismatch
```
Invalid directories and unreadable or non-UTF-8 version files always fail. The
generator copies and filters `game.db` to exactly the included games, then
generates and validates their complete manifest/index set. At least one matching
source-catalog game must be present. The checked-in full database is never
modified; the coherent generated output is installed under
`crates/lanspread-tauri-deno-ts/src-tauri/production-catalog/`.
The mode can also be used while running or building real data:
```bash
just run-production /srv/lanspread/packages \
allow-missing-games allow-version-mismatch
just build-production /srv/lanspread/packages \
allow-missing-games allow-version-mismatch
```
The recipe keeps a local metadata-only stamp in
`.lanspread/catalog-cache/production.json`. It compares the package tree's
paths, entry types, sizes, and nanosecond mtimes, plus the catalog database and
`unrar` metadata. If those values and the published output still match, the
expensive package hashing and extraction step is skipped. This is only a
performance cache: `catalog-check-production` and every production build still
validate the catalog authority. Force a refresh when needed:
paths, entry types, sizes, and nanosecond mtimes, plus the source and generated
catalog databases, generation mode, and `unrar` metadata. If those values and
the published output still match, the expensive package hashing and extraction
step is skipped. This is only a performance cache: `catalog-check-production`
and every production build still validate the catalog authority. Force a refresh
when needed:
```bash
LANSPREAD_CATALOG_FORCE=1 just catalog-generate-production /srv/lanspread/packages
```
The publisher first checks every selected package's `version.ini` against
`game.db`, then independently reads and hashes each selected package twice
before publication. Under a durable publication marker, it atomically replaces
each manifest and writes the complete content index last. Interruption leaves
the marker in place so later checks and builds fail closed instead of accepting
a mixed generation.
The publisher first checks every included package's `version.ini` against the
generated `game.db`, then independently reads and hashes each package twice
before publication. It creates the complete database, manifest, and index set in
staging and validates it with the application loader before atomically replacing
the previous production catalog directory.
Verify the published database/index/manifest set without rereading the package
corpus:
@@ -177,10 +237,11 @@ just catalog-generate-production-game /srv/lanspread/packages alienswarm
just catalog-check-production
```
Incremental generation refuses to bootstrap a partial catalog. Under its durable
Incremental generation refuses to bootstrap a catalog. Under its durable
publication marker it revalidates every unselected manifest/index identity,
rebuilds the complete index, and then publishes the selected game. Use the full
generation command for a new catalog or any intentionally broad catalog change.
rebuilds the complete generated index, and then publishes the selected game. Use
the main generation command for a new catalog or any intentionally broad catalog
change.
### Build a production bundle
@@ -191,25 +252,24 @@ just bundle
```
`just bundle` repeats the production catalog gate before Tauri packaging. It
never falls back to fixture manifests. This checkout does not include the
canonical 186-game package corpus, so the command intentionally fails until that
external corpus has been processed into
`crates/lanspread-tauri-deno-ts/src-tauri/manifests/`.
never falls back to fixture manifests. This checkout does not include generated
production authority, so the command intentionally fails until an external
package corpus has been processed into
`crates/lanspread-tauri-deno-ts/src-tauri/production-catalog/`.
## Fixture catalogs and peer-CLI acceptance
The committed fixture profiles are test authority only:
```bash
just fixture-catalog-check # default GUI/CLI profile
just fixture-catalog-check # explicit fixture GUI/CLI profile
just fixture-catalogs-check # default + solid + multi + unknown profiles
just fixture-catalogs # explicitly regenerate all committed profiles
```
The solid, multi-archive, and unknown-game profiles intentionally describe
different package bytes, so the peer-CLI matrix checks all of them before
building its Docker image. Normal GUI development needs only the default
profile.
building its Docker image. `just run-fixture` uses the default fixture profile.
Build or run the JSONL peer harness:
@@ -239,13 +299,15 @@ just peer-cli-charlie
| Command | Purpose |
| --------------------------------------------------- | ------------------------------------------------------------------------ |
| `just setup` | Install the Tauri CLI and frontend dependencies. |
| `just` / `just run` | Run fixtures, or real data when `LANSPREAD_GAMES_DIR` is set. |
| `just` / `just run` | Check the existing production catalog and run the GUI. |
| `just run-local GAMES_DIR` | Generate/reuse separate local test authority and run with that folder. |
| `just run-production PACKAGES_DIR [MODE...]` | Generate/reuse production authority and run the GUI. |
| `just run-fixture` | Check the default fixture catalog and run the GUI. |
| `just build` | Validate production authority and build without bundling. |
| `just build-production PACKAGES_DIR` | Generate/reuse production authority and build without bundling. |
| `just build-production PACKAGES_DIR [MODE]` | Generate/reuse production authority and build without bundling. |
| `just build-fixture` | Check the default fixture catalog and build without bundling. |
| `just bundle` | Check production authority and build production bundles. |
| `just catalog-generate-production PACKAGES_DIR` | Hash, cache, and publish the complete production package corpus. |
| `just catalog-generate-production DIR [MODE...]` | Publish packages with explicit missing/version mismatch opt-ins. |
| `just catalog-generate-production-game DIR GAME_ID` | Safely regenerate one game in an already complete catalog. |
| `just catalog-check-production` | Validate all production database/index/manifest artifacts. |
| `just fixture-catalog-check` | Validate the default development fixture catalog. |
@@ -0,0 +1,202 @@
use std::{
ffi::OsString,
io::{self, Write},
path::PathBuf,
};
use lanspread_compat::catalog_publisher::production::{
ProductionCatalogOptions,
generate_production_catalog,
};
const HELP: &str = "\
Usage:
lanspread-production-catalog --source-catalog-db PATH --packages-dir PATH --output-dir PATH --unrar PATH [--allow-missing-games] [--allow-version-mismatch]
Missing games and version mismatches each require their corresponding explicit option.";
enum ParseOutcome {
Help,
Options(ProductionCatalogOptions),
}
#[tokio::main]
async fn main() {
if let Err(error) = run().await {
eprintln!("error: {error:#}");
std::process::exit(1);
}
}
async fn run() -> eyre::Result<()> {
match parse_args(std::env::args_os().skip(1))? {
ParseOutcome::Help => println!("{HELP}"),
ParseOutcome::Options(options) => {
let report = generate_production_catalog(&options).await?;
let stdout = io::stdout();
let mut stdout = stdout.lock();
for manifest in &report.manifests {
writeln!(
stdout,
"generated game_id={} game_version={} content_id={}",
manifest.game_id, manifest.game_version, manifest.content_id
)?;
}
for game_id in &report.version_mismatches {
writeln!(stdout, "included game_id={game_id} source=package-version")?;
}
writeln!(stdout, "generated total={}", report.manifests.len())?;
writeln!(
stdout,
"skipped missing_games={} allowed_version_mismatches={}",
report.missing_games,
report.version_mismatches.len()
)?;
}
}
Ok(())
}
fn parse_args(args: impl IntoIterator<Item = OsString>) -> eyre::Result<ParseOutcome> {
let mut source_catalog_db = None;
let mut packages_dir = None;
let mut output_dir = None;
let mut unrar = None;
let mut allow_missing_games = false;
let mut allow_version_mismatch = false;
let mut args = args.into_iter();
while let Some(argument) = args.next() {
match argument.to_str() {
Some("--help" | "-h") => return Ok(ParseOutcome::Help),
Some("--source-catalog-db") => set_once(
&mut source_catalog_db,
next_path(&mut args, "--source-catalog-db")?,
"--source-catalog-db",
)?,
Some("--packages-dir") => set_once(
&mut packages_dir,
next_path(&mut args, "--packages-dir")?,
"--packages-dir",
)?,
Some("--output-dir") => set_once(
&mut output_dir,
next_path(&mut args, "--output-dir")?,
"--output-dir",
)?,
Some("--unrar") => {
set_once(&mut unrar, next_path(&mut args, "--unrar")?, "--unrar")?;
}
Some("--allow-missing-games") => {
if allow_missing_games {
eyre::bail!("--allow-missing-games may be specified only once");
}
allow_missing_games = true;
}
Some("--allow-version-mismatch") => {
if allow_version_mismatch {
eyre::bail!("--allow-version-mismatch may be specified only once");
}
allow_version_mismatch = true;
}
Some(other) => eyre::bail!("unknown argument: {other}"),
None => eyre::bail!("argument is not valid UTF-8: {argument:?}"),
}
}
Ok(ParseOutcome::Options(ProductionCatalogOptions {
source_catalog_db: required(source_catalog_db, "--source-catalog-db")?,
packages_dir: required(packages_dir, "--packages-dir")?,
output_dir: required(output_dir, "--output-dir")?,
unrar: required(unrar, "--unrar")?,
allow_missing_games,
allow_version_mismatch,
}))
}
fn next_path(args: &mut impl Iterator<Item = OsString>, option: &str) -> eyre::Result<PathBuf> {
args.next()
.map(PathBuf::from)
.ok_or_else(|| eyre::eyre!("{option} requires a value"))
}
fn set_once<T>(slot: &mut Option<T>, value: T, option: &str) -> eyre::Result<()> {
if slot.replace(value).is_some() {
eyre::bail!("{option} may be specified only once");
}
Ok(())
}
fn required<T>(value: Option<T>, option: &str) -> eyre::Result<T> {
value.ok_or_else(|| eyre::eyre!("{option} is required"))
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn missing_and_version_mismatch_options_are_independent() {
let strict = parse_args(
[
"--source-catalog-db",
"source.db",
"--packages-dir",
"packages",
"--output-dir",
"output",
"--unrar",
"unrar",
]
.map(OsString::from),
)
.expect("strict arguments should parse");
let ParseOutcome::Options(strict) = strict else {
panic!("arguments should produce options");
};
assert!(!strict.allow_missing_games);
assert!(!strict.allow_version_mismatch);
let missing_only = parse_args(
[
"--source-catalog-db",
"source.db",
"--packages-dir",
"packages",
"--output-dir",
"output",
"--unrar",
"unrar",
"--allow-missing-games",
]
.map(OsString::from),
)
.expect("missing-game opt-in should parse");
let ParseOutcome::Options(missing_only) = missing_only else {
panic!("arguments should produce options");
};
assert!(missing_only.allow_missing_games);
assert!(!missing_only.allow_version_mismatch);
let version_only = parse_args(
[
"--source-catalog-db",
"source.db",
"--packages-dir",
"packages",
"--output-dir",
"output",
"--unrar",
"unrar",
"--allow-version-mismatch",
]
.map(OsString::from),
)
.expect("version-mismatch opt-in should parse");
let ParseOutcome::Options(version_only) = version_only else {
panic!("arguments should produce options");
};
assert!(!version_only.allow_missing_games);
assert!(version_only.allow_version_mismatch);
}
}
@@ -8,8 +8,7 @@
use std::{
collections::{BTreeMap, BTreeSet},
fs,
path::{Path, PathBuf},
sync::atomic::{AtomicU64, Ordering},
path::PathBuf,
};
use eyre::WrapErr;
@@ -19,21 +18,19 @@ use lanspread_db::content_manifest::{
write_canonical_content_index_atomic,
write_canonical_manifest_atomic,
};
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
use super::{
CatalogGame,
CatalogSelection,
CheckOptions,
ManifestReport,
catalog::{load_catalog_games, validate_regular_directory},
catalog::load_catalog_games,
check_catalog_manifests,
package::{StreamedInstallPolicy, build_manifest_from_package_with_policy},
staging::{CatalogStagingDirectory, copy_filtered_catalog},
};
use crate::catalog_bundle::load_catalog_bundle;
static TEMP_SEQUENCE: AtomicU64 = AtomicU64::new(0);
/// One canonical package selected for a generated acceptance-test catalog.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct FixturePackage {
@@ -69,7 +66,7 @@ pub async fn generate_fixture_catalog(
) -> eyre::Result<Vec<ManifestReport>> {
let source_catalog = load_catalog_games(&options.source_catalog_db).await?;
let packages = validate_packages(&source_catalog, &options.packages)?;
let staging = StagingDirectory::create(&options.output_dir)?;
let staging = CatalogStagingDirectory::create(&options.output_dir)?;
let staged_db = staging.path().join("game.db");
let staged_manifests = staging.path().join("manifests");
@@ -77,6 +74,7 @@ pub async fn generate_fixture_catalog(
&options.source_catalog_db,
&staged_db,
packages.keys().map(String::as_str),
&BTreeMap::new(),
)
.await?;
fs::create_dir(&staged_manifests).wrap_err_with(|| {
@@ -178,152 +176,6 @@ fn validate_packages<'a>(
Ok(selected)
}
async fn copy_filtered_catalog<'a>(
source: &Path,
destination: &Path,
selected_ids: impl Iterator<Item = &'a str>,
) -> eyre::Result<()> {
fs::copy(source, destination).wrap_err_with(|| {
format!(
"failed to copy source catalog {} to {}",
source.display(),
destination.display()
)
})?;
let selected_ids = selected_ids.map(str::to_owned).collect::<BTreeSet<_>>();
let all_games = load_catalog_games(destination).await?;
let options = SqliteConnectOptions::new().filename(destination);
let pool = SqlitePoolOptions::new()
.max_connections(1)
.connect_with(options)
.await
.wrap_err_with(|| format!("failed to open fixture catalog {}", destination.display()))?;
let mutation_result = async {
let mut transaction = pool.begin().await?;
for game_id in all_games.keys() {
if !selected_ids.contains(game_id) {
sqlx::query("DELETE FROM games WHERE game_id = ?")
.bind(game_id)
.execute(&mut *transaction)
.await?;
}
}
sqlx::query(
"DELETE FROM genre
WHERE genre_id NOT IN (SELECT DISTINCT genre_id FROM games)",
)
.execute(&mut *transaction)
.await?;
transaction.commit().await?;
sqlx::query("VACUUM").execute(&pool).await?;
Ok::<(), sqlx::Error>(())
}
.await;
pool.close().await;
mutation_result.wrap_err("failed to filter fixture catalog database")?;
Ok(())
}
struct StagingDirectory {
path: Option<PathBuf>,
}
impl StagingDirectory {
fn create(output: &Path) -> eyre::Result<Self> {
let parent = output
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or_else(|| Path::new("."));
fs::create_dir_all(parent)?;
validate_regular_directory(parent)?;
if let Ok(metadata) = fs::symlink_metadata(output)
&& (!metadata.is_dir() || is_link_or_reparse(&metadata))
{
eyre::bail!(
"fixture catalog output is not a regular non-link directory: {}",
output.display()
);
}
let stem = output
.file_name()
.and_then(|name| name.to_str())
.ok_or_else(|| eyre::eyre!("fixture catalog output needs a UTF-8 directory name"))?;
for _ in 0..100 {
let sequence = TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let candidate = parent.join(format!(".{stem}.tmp-{}-{sequence}", std::process::id()));
match fs::create_dir(&candidate) {
Ok(()) => {
return Ok(Self {
path: Some(candidate),
});
}
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => {}
Err(error) => return Err(error.into()),
}
}
eyre::bail!("could not allocate a fixture catalog staging directory")
}
fn path(&self) -> &Path {
match self.path.as_deref() {
Some(path) => path,
None => panic!("staging directory is present until installation"),
}
}
fn install(mut self, output: &Path) -> eyre::Result<()> {
let staging = self.path().to_path_buf();
if !output.exists() {
fs::rename(&staging, output)?;
self.path = None;
return Ok(());
}
validate_regular_directory(output)?;
let parent = output
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or_else(|| Path::new("."));
let stem = output
.file_name()
.and_then(|name| name.to_str())
.ok_or_else(|| eyre::eyre!("fixture catalog output needs a UTF-8 directory name"))?;
let sequence = TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let backup = parent.join(format!(".{stem}.old-{}-{sequence}", std::process::id()));
fs::rename(output, &backup)?;
if let Err(error) = fs::rename(&staging, output) {
let restore = fs::rename(&backup, output);
return match restore {
Ok(()) => Err(error.into()),
Err(restore_error) => Err(eyre::eyre!(
"failed to install fixture catalog: {error}; failed to restore previous output: {restore_error}"
)),
};
}
self.path = None;
fs::remove_dir_all(&backup).wrap_err_with(|| {
format!(
"installed fixture catalog but failed to remove backup {}",
backup.display()
)
})?;
Ok(())
}
}
impl Drop for StagingDirectory {
fn drop(&mut self) {
if let Some(path) = self.path.take() {
let _ = fs::remove_dir_all(path);
}
}
}
#[cfg(unix)]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
metadata.file_type().is_symlink()
}
#[cfg(test)]
mod tests {
use std::{
@@ -331,6 +183,8 @@ mod tests {
time::{SystemTime, UNIX_EPOCH},
};
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
use super::*;
struct TestDirectory(PathBuf);
@@ -488,17 +342,3 @@ mod tests {
.expect("installed replacement profile should be coherent");
}
}
#[cfg(windows)]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
use std::os::windows::fs::MetadataExt;
const FILE_ATTRIBUTE_REPARSE_POINT: u32 = 0x400;
metadata.file_type().is_symlink()
|| metadata.file_attributes() & FILE_ATTRIBUTE_REPARSE_POINT != 0
}
#[cfg(not(any(unix, windows)))]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
metadata.file_type().is_symlink()
}
@@ -8,6 +8,8 @@ mod catalog;
pub mod cli;
pub mod fixture;
mod package;
pub mod production;
mod staging;
mod unrar;
#[cfg(unix)]
@@ -128,15 +128,23 @@ pub(super) fn preflight_package_version(package_root: &Path, expected: &str) ->
}
pub(super) fn validate_package_version(package_root: &Path, expected: &str) -> eyre::Result<()> {
if !package_version_matches(package_root, expected)? {
eyre::bail!("package version mismatch: game.db expects {expected}");
}
Ok(())
}
pub(super) fn package_version_matches(package_root: &Path, expected: &str) -> eyre::Result<bool> {
Ok(package_version(package_root)? == expected)
}
pub(super) fn package_version(package_root: &Path) -> eyre::Result<String> {
let path = package_root.join("version.ini");
let bytes = read_bounded_regular_file(&path, MAX_VERSION_INI_BYTES)?;
let version = std::str::from_utf8(&bytes)
.wrap_err("version.ini is not valid UTF-8")?
.trim();
if version != expected {
eyre::bail!("package version mismatch: game.db expects {expected}");
}
Ok(())
Ok(version.to_owned())
}
fn scan_ordinary_package(root: &Path) -> eyre::Result<Vec<CatalogFileEntry>> {
@@ -0,0 +1,361 @@
//! Coherent production catalog generation from a complete or reduced package tree.
use std::{
collections::{BTreeMap, BTreeSet},
fs,
path::PathBuf,
};
use eyre::WrapErr;
use super::{
CatalogSelection,
GenerateOptions,
ManifestReport,
catalog::{load_catalog_games, validate_regular_directory},
generate_catalog_manifests,
package::package_version,
staging::{CatalogStagingDirectory, copy_filtered_catalog},
};
use crate::catalog_bundle::load_catalog_bundle;
/// Inputs for atomically generating a production `game.db` and manifest set.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ProductionCatalogOptions {
pub source_catalog_db: PathBuf,
pub packages_dir: PathBuf,
pub output_dir: PathBuf,
pub unrar: PathBuf,
pub allow_missing_games: bool,
pub allow_version_mismatch: bool,
}
/// Result of generating a complete or reduced production catalog.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct ProductionCatalogReport {
pub manifests: Vec<ManifestReport>,
pub missing_games: usize,
pub version_mismatches: Vec<String>,
}
struct PackageSelection {
selected_ids: BTreeSet<String>,
missing_games: usize,
version_mismatches: Vec<String>,
version_overrides: BTreeMap<String, String>,
}
/// Builds one coherent production catalog in `output_dir`.
///
/// The checked-in source database is copied into staging and filtered to the
/// package directories that exist only when `allow_missing_games` is enabled.
/// The generated database, manifests, and compact index are fully validated
/// before they atomically replace the previous output directory.
///
/// # Errors
///
/// Returns an error when the source catalog, package tree, generated manifests,
/// filtered database, or output directory is invalid.
pub async fn generate_production_catalog(
options: &ProductionCatalogOptions,
) -> eyre::Result<ProductionCatalogReport> {
let source_catalog = load_catalog_games(&options.source_catalog_db).await?;
let selected = select_package_games(
&source_catalog,
&options.packages_dir,
options.allow_missing_games,
options.allow_version_mismatch,
)?;
let staging = CatalogStagingDirectory::create(&options.output_dir)?;
let staged_db = staging.path().join("game.db");
let staged_manifests = staging.path().join("manifests");
copy_filtered_catalog(
&options.source_catalog_db,
&staged_db,
selected.selected_ids.iter().map(String::as_str),
&selected.version_overrides,
)
.await?;
let reports = generate_catalog_manifests(&GenerateOptions {
catalog_db: staged_db.clone(),
packages_dir: options.packages_dir.clone(),
manifests_dir: staged_manifests.clone(),
unrar: options.unrar.clone(),
selection: CatalogSelection::All,
})
.await?;
let loaded = load_catalog_bundle(&staged_db, &staged_manifests)
.await
.wrap_err("generated production catalog failed application loader validation")?;
loaded
.bundle()
.validate_all()
.wrap_err("generated production catalog failed complete manifest validation")?;
staging.install(&options.output_dir)?;
Ok(ProductionCatalogReport {
manifests: reports,
missing_games: selected.missing_games,
version_mismatches: selected.version_mismatches,
})
}
fn select_package_games(
catalog: &std::collections::BTreeMap<String, super::CatalogGame>,
packages_dir: &std::path::Path,
allow_missing_games: bool,
allow_version_mismatch: bool,
) -> eyre::Result<PackageSelection> {
validate_regular_directory(packages_dir)?;
if !allow_missing_games && !allow_version_mismatch {
return Ok(PackageSelection {
selected_ids: catalog.keys().cloned().collect(),
missing_games: 0,
version_mismatches: Vec::new(),
version_overrides: BTreeMap::new(),
});
}
let mut selected_ids = BTreeSet::new();
let mut missing_games = 0;
let mut version_mismatches = Vec::new();
let mut version_overrides = BTreeMap::new();
for (game_id, game) in catalog {
let package_root = packages_dir.join(game_id);
match fs::symlink_metadata(&package_root) {
Ok(_) => {
validate_regular_directory(&package_root).wrap_err_with(|| {
format!("invalid package directory for catalog game {game_id}")
})?;
let observed_version = package_version(&package_root)
.wrap_err_with(|| format!("failed to inspect package version for {game_id}"))?;
let version_matches = observed_version == game.game_version;
if version_matches || !allow_version_mismatch {
selected_ids.insert(game_id.clone());
} else {
selected_ids.insert(game_id.clone());
version_mismatches.push(game_id.clone());
version_overrides.insert(game_id.clone(), observed_version);
}
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
if allow_missing_games {
missing_games += 1;
} else {
selected_ids.insert(game_id.clone());
}
}
Err(error) => {
return Err(error).wrap_err_with(|| {
format!(
"failed to inspect package directory {}",
package_root.display()
)
});
}
}
}
if selected_ids.is_empty() {
eyre::bail!("package tree contains no games matching the source catalog IDs and versions");
}
Ok(PackageSelection {
selected_ids,
missing_games,
version_mismatches,
version_overrides,
})
}
#[cfg(test)]
mod tests {
use std::{
fs,
path::{Path, PathBuf},
sync::atomic::{AtomicU64, Ordering},
time::{SystemTime, UNIX_EPOCH},
};
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
use super::*;
static TEST_SEQUENCE: AtomicU64 = AtomicU64::new(0);
struct TestDirectory(PathBuf);
impl TestDirectory {
fn new() -> Self {
let sequence = TEST_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("system clock should follow the epoch")
.as_nanos();
let path = std::env::temp_dir().join(format!(
"lanspread-production-catalog-{}-{nanos}-{sequence}",
std::process::id()
));
fs::create_dir(&path).expect("test directory should be created");
Self(path)
}
fn path(&self) -> &Path {
&self.0
}
}
impl Drop for TestDirectory {
fn drop(&mut self) {
let _ = fs::remove_dir_all(&self.0);
}
}
async fn create_source_catalog(path: &Path) {
let options = SqliteConnectOptions::new()
.filename(path)
.create_if_missing(true);
let pool = SqlitePoolOptions::new()
.max_connections(1)
.connect_with(options)
.await
.expect("source catalog should open");
sqlx::query(
"CREATE TABLE games (
game_id TEXT, db_id INTEGER PRIMARY KEY, game_title TEXT,
game_key TEXT, game_release TEXT, game_publisher TEXT,
game_size NUMERIC, game_readme_de TEXT, game_readme_en TEXT,
game_readme_fr TEXT, game_maxplayers INTEGER,
game_master_req INTEGER, genre_id INTEGER, game_version TEXT
)",
)
.execute(&pool)
.await
.expect("games table should be created");
sqlx::query(
"CREATE TABLE genre (
genre_id INTEGER PRIMARY KEY, genre_de TEXT,
genre_en TEXT, genre_fr TEXT
)",
)
.execute(&pool)
.await
.expect("genre table should be created");
sqlx::query("INSERT INTO genre VALUES (1, 'Games', 'Games', 'Games')")
.execute(&pool)
.await
.expect("genre should be inserted");
sqlx::query(
"INSERT INTO games VALUES
('keep', 1, 'Keep', '', '', '', 1, '', '', '', 1, 0, 1, '20250101'),
('outdated', 2, 'Outdated', '', '', '', 1, '', '', '', 1, 0, 1, '20250102'),
('absent', 3, 'Absent', '', '', '', 1, '', '', '', 1, 0, 1, '20250103')",
)
.execute(&pool)
.await
.expect("games should be inserted");
pool.close().await;
}
fn create_package(root: &Path) {
let package = root.join("packages/keep");
fs::create_dir_all(&package).expect("package directory should be created");
fs::write(package.join("version.ini"), "20250101")
.expect("package version should be written");
fs::write(package.join("payload.bin"), b"payload")
.expect("package payload should be written");
let outdated = root.join("packages/outdated");
fs::create_dir(&outdated).expect("outdated package directory should be created");
fs::write(outdated.join("version.ini"), "20240101")
.expect("outdated package version should be written");
}
#[tokio::test]
async fn combined_options_generate_a_complete_reduced_catalog_with_package_versions() {
let root = TestDirectory::new();
let source_catalog_db = root.path().join("source.db");
create_source_catalog(&source_catalog_db).await;
create_package(root.path());
let output_dir = root.path().join("output");
let reports = generate_production_catalog(&ProductionCatalogOptions {
source_catalog_db,
packages_dir: root.path().join("packages"),
output_dir: output_dir.clone(),
unrar: root.path().join("unused-unrar"),
allow_missing_games: true,
allow_version_mismatch: true,
})
.await
.expect("reduced production catalog should generate");
assert_eq!(reports.manifests.len(), 2);
assert_eq!(reports.manifests[0].game_id, "keep");
assert_eq!(reports.manifests[1].game_id, "outdated");
assert_eq!(reports.manifests[1].game_version, "20240101");
assert_eq!(reports.missing_games, 1);
assert_eq!(reports.version_mismatches, ["outdated"]);
let games = load_catalog_games(&output_dir.join("game.db"))
.await
.expect("generated database should load");
assert_eq!(
games.keys().map(String::as_str).collect::<Vec<_>>(),
["keep", "outdated"]
);
assert_eq!(games["outdated"].game_version, "20240101");
load_catalog_bundle(&output_dir.join("game.db"), &output_dir.join("manifests"))
.await
.expect("generated catalog should pass the application loader");
}
#[tokio::test]
async fn strict_generation_still_rejects_a_missing_game() {
let root = TestDirectory::new();
let source_catalog_db = root.path().join("source.db");
create_source_catalog(&source_catalog_db).await;
create_package(root.path());
let output_dir = root.path().join("output");
let error = generate_production_catalog(&ProductionCatalogOptions {
source_catalog_db,
packages_dir: root.path().join("packages"),
output_dir: output_dir.clone(),
unrar: root.path().join("unused-unrar"),
allow_missing_games: false,
allow_version_mismatch: false,
})
.await
.expect_err("strict generation must require every source-catalog game");
assert!(
error
.to_string()
.contains("failed to preflight package version")
);
assert!(!output_dir.exists());
}
#[tokio::test]
async fn allow_missing_games_alone_still_rejects_a_version_mismatch() {
let root = TestDirectory::new();
let source_catalog_db = root.path().join("source.db");
create_source_catalog(&source_catalog_db).await;
create_package(root.path());
let output_dir = root.path().join("output");
let error = generate_production_catalog(&ProductionCatalogOptions {
source_catalog_db,
packages_dir: root.path().join("packages"),
output_dir: output_dir.clone(),
unrar: root.path().join("unused-unrar"),
allow_missing_games: true,
allow_version_mismatch: false,
})
.await
.expect_err("missing-game opt-in must not allow version mismatches");
assert!(format!("{error:#}").contains("package version mismatch"));
assert!(!output_dir.exists());
}
}
@@ -0,0 +1,181 @@
use std::{
collections::{BTreeMap, BTreeSet},
fs,
path::{Path, PathBuf},
sync::atomic::{AtomicU64, Ordering},
};
use eyre::WrapErr;
use sqlx::sqlite::{SqliteConnectOptions, SqlitePoolOptions};
use super::catalog::{load_catalog_games, validate_regular_directory};
static TEMP_SEQUENCE: AtomicU64 = AtomicU64::new(0);
pub(super) async fn copy_filtered_catalog<'a>(
source: &Path,
destination: &Path,
selected_ids: impl Iterator<Item = &'a str>,
version_overrides: &BTreeMap<String, String>,
) -> eyre::Result<()> {
fs::copy(source, destination).wrap_err_with(|| {
format!(
"failed to copy source catalog {} to {}",
source.display(),
destination.display()
)
})?;
let selected_ids = selected_ids.map(str::to_owned).collect::<BTreeSet<_>>();
let all_games = load_catalog_games(destination).await?;
let options = SqliteConnectOptions::new().filename(destination);
let pool = SqlitePoolOptions::new()
.max_connections(1)
.connect_with(options)
.await
.wrap_err_with(|| format!("failed to open catalog {}", destination.display()))?;
let mutation_result = async {
let mut transaction = pool.begin().await?;
for game_id in all_games.keys() {
if !selected_ids.contains(game_id) {
sqlx::query("DELETE FROM games WHERE game_id = ?")
.bind(game_id)
.execute(&mut *transaction)
.await?;
}
}
for (game_id, game_version) in version_overrides {
sqlx::query("UPDATE games SET game_version = ? WHERE game_id = ?")
.bind(game_version)
.bind(game_id)
.execute(&mut *transaction)
.await?;
}
sqlx::query(
"DELETE FROM genre
WHERE genre_id NOT IN (SELECT DISTINCT genre_id FROM games)",
)
.execute(&mut *transaction)
.await?;
transaction.commit().await?;
sqlx::query("VACUUM").execute(&pool).await?;
Ok::<(), sqlx::Error>(())
}
.await;
pool.close().await;
mutation_result.wrap_err("failed to filter catalog database")?;
Ok(())
}
pub(super) struct CatalogStagingDirectory {
path: Option<PathBuf>,
}
impl CatalogStagingDirectory {
pub(super) fn create(output: &Path) -> eyre::Result<Self> {
let parent = output
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or_else(|| Path::new("."));
fs::create_dir_all(parent)?;
validate_regular_directory(parent)?;
if let Ok(metadata) = fs::symlink_metadata(output)
&& (!metadata.is_dir() || is_link_or_reparse(&metadata))
{
eyre::bail!(
"catalog output is not a regular non-link directory: {}",
output.display()
);
}
let stem = output
.file_name()
.and_then(|name| name.to_str())
.ok_or_else(|| eyre::eyre!("catalog output needs a UTF-8 directory name"))?;
for _ in 0..100 {
let sequence = TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let candidate = parent.join(format!(".{stem}.tmp-{}-{sequence}", std::process::id()));
match fs::create_dir(&candidate) {
Ok(()) => {
return Ok(Self {
path: Some(candidate),
});
}
Err(error) if error.kind() == std::io::ErrorKind::AlreadyExists => {}
Err(error) => return Err(error.into()),
}
}
eyre::bail!("could not allocate a catalog staging directory")
}
pub(super) fn path(&self) -> &Path {
match self.path.as_deref() {
Some(path) => path,
None => panic!("staging directory is present until installation"),
}
}
pub(super) fn install(mut self, output: &Path) -> eyre::Result<()> {
let staging = self.path().to_path_buf();
if !output.exists() {
fs::rename(&staging, output)?;
self.path = None;
return Ok(());
}
validate_regular_directory(output)?;
let parent = output
.parent()
.filter(|parent| !parent.as_os_str().is_empty())
.unwrap_or_else(|| Path::new("."));
let stem = output
.file_name()
.and_then(|name| name.to_str())
.ok_or_else(|| eyre::eyre!("catalog output needs a UTF-8 directory name"))?;
let sequence = TEMP_SEQUENCE.fetch_add(1, Ordering::Relaxed);
let backup = parent.join(format!(".{stem}.old-{}-{sequence}", std::process::id()));
fs::rename(output, &backup)?;
if let Err(error) = fs::rename(&staging, output) {
let restore = fs::rename(&backup, output);
return match restore {
Ok(()) => Err(error.into()),
Err(restore_error) => Err(eyre::eyre!(
"failed to install catalog: {error}; failed to restore previous output: {restore_error}"
)),
};
}
self.path = None;
fs::remove_dir_all(&backup).wrap_err_with(|| {
format!(
"installed catalog but failed to remove backup {}",
backup.display()
)
})?;
Ok(())
}
}
impl Drop for CatalogStagingDirectory {
fn drop(&mut self) {
if let Some(path) = self.path.take() {
let _ = fs::remove_dir_all(path);
}
}
}
#[cfg(unix)]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
metadata.file_type().is_symlink()
}
#[cfg(windows)]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
use std::os::windows::fs::MetadataExt;
const FILE_ATTRIBUTE_REPARSE_POINT: u32 = 0x400;
metadata.file_type().is_symlink()
|| metadata.file_attributes() & FILE_ATTRIBUTE_REPARSE_POINT != 0
}
#[cfg(not(any(unix, windows)))]
fn is_link_or_reparse(metadata: &fs::Metadata) -> bool {
metadata.file_type().is_symlink()
}
+24 -12
View File
@@ -408,14 +408,19 @@ Most scans become O(number of game dirs), with full recursion only when needed.
## Catalog publication and packaging
- `lanspread-catalog-publisher` generates production manifests and their
complete compact identity index beside the canonical packages, independently
rebuilds/verifies each selected artifact, and uses a durable corpus marker so
an interrupted body/index publication fails closed. Incremental generation
requires an existing exact indexed corpus and atomically republishes the full
index after the selected bodies. After it owns the marker, it freshly
validates every unselected body/index pair before deriving that mixed index;
no pre-marker snapshot can roll another completed publication backward.
- `lanspread-production-catalog` creates a production `game.db`, manifests, and
complete compact identity index in staging, validates that authority with the
application loader, and atomically replaces the generated catalog directory.
Its default selection requires every game and exact version from the
checked-in source database. Independent explicit modes may omit missing
package directories or accept package versions that differ by updating the
copied database. The generated manifests and database still form one exact
set, and the source database is never modified.
- `lanspread-catalog-publisher` incrementally regenerates selected manifests in
an existing exact indexed corpus. After it owns the durable publication
marker, it freshly validates every unselected body/index pair before deriving
and atomically publishing the complete mixed index; no pre-marker snapshot can
roll another completed publication backward.
- `lanspread-fixture-catalog` is a separate test-only generator. It derives
reduced peer-CLI `game.db` files and manifests from explicitly selected
fixtures; those outputs are development and acceptance-test authority only.
@@ -423,10 +428,17 @@ Most scans become O(number of game dirs), with full recursion only when needed.
and `LANSPREAD_USE_FIXTURE_CATALOG=1`. Every other build mode defaults to the
production resource map, and the custom production profile cannot be
downgraded to fixtures.
- Production packaging requires the complete production `game.db`/manifest
corpus to pass `check --all`. The canonical production packages and their 186
generated manifests are not present in this checkout, so that gate remains
intentionally blocked; fixture success is not production completion.
- `just run` uses the existing production catalog. `just run-local GAMES_DIR`
creates a separate cached catalog from the supplied files, allowing missing
packages and package versions that differ from the metadata database. Local
builds require a distinct compile-time opt-in and resource map, are rejected
by the production profile and bundled builds, and use separate app settings.
Their startup game directory is compiled in; production embeds no override.
- Production packaging requires the exact generated production
`game.db`/manifest corpus to pass `check --all`. The external production
packages and generated authority are not present in this checkout, so that
gate remains intentionally blocked; fixture success is not production
completion.
## Fault tolerance rules
@@ -14,39 +14,101 @@ use build_support::catalog_gate::{
mod build_support {
#[path = "catalog_gate.rs"]
pub(crate) mod catalog_gate;
#[path = "catalog_resources.rs"]
pub(crate) mod catalog_resources;
}
const FIXTURE_DEVELOPMENT_ENV: &str = "LANSPREAD_USE_FIXTURE_CATALOG";
const LOCAL_DEVELOPMENT_ENV: &str = "LANSPREAD_USE_LOCAL_CATALOG";
const LOCAL_GAMES_DIR_ENV: &str = "LANSPREAD_LOCAL_GAMES_DIR";
const CATALOG_CONTENT_INDEX_NAME: &str = "catalog-content-index-v1.jsonl";
const EMBEDDED_CATALOG_INDEX_NAME: &str = "embedded-catalog-content-index-v1.jsonl";
const FIXTURE_MANIFEST_ROOT: &str = "../../lanspread-peer-cli/catalogs/default/manifests";
const FIXTURE_CATALOG_ROOT: &str = "../../lanspread-peer-cli/catalogs/default";
const PRODUCTION_CATALOG_ROOT: &str = "production-catalog";
fn main() {
println!("cargo:rerun-if-env-changed=TAURI_CONFIG");
println!("cargo:rerun-if-env-changed={FIXTURE_DEVELOPMENT_ENV}");
println!("cargo:rerun-if-env-changed={LOCAL_DEVELOPMENT_ENV}");
println!("cargo:rerun-if-env-changed={LOCAL_GAMES_DIR_ENV}");
println!("cargo:rerun-if-changed=tauri.conf.json");
println!("cargo:rerun-if-changed=game.db");
println!("cargo:rerun-if-changed=manifests");
println!("cargo:rerun-if-changed={PRODUCTION_CATALOG_ROOT}");
let mode = catalog_build_mode().unwrap_or_else(|error| {
panic!("catalog packaging policy failed: {error}");
});
if mode == CatalogBuildMode::Production
&& let Err(error) = validate_production_catalog("game.db", "manifests")
assert!(
mode != CatalogBuildMode::LocalDevelopment || tauri_build::is_dev(),
"local catalog testing is only available with tauri dev"
);
if mode != CatalogBuildMode::FixtureDevelopment
&& let Err(error) = validate_production_catalog(
catalog_root(mode).join("game.db"),
catalog_root(mode).join("manifests"),
)
{
panic!("production catalog authority gate failed: {error}");
}
embed_catalog_content_index(mode).unwrap_or_else(|error| {
panic!("failed to embed catalog launch authority: {error}");
});
embed_startup_game_directory(mode).unwrap_or_else(|error| {
panic!("failed to embed local test game directory: {error}");
});
let out_dir = env::var_os("OUT_DIR").expect("OUT_DIR is not set");
let resources = build_support::catalog_resources::CatalogResources::prepare(
Path::new(&out_dir),
catalog_root(mode),
)
.unwrap_or_else(|error| panic!("failed to reset generated catalog resources: {error}"));
tauri_build::build();
// Tauri watches source resources only. Watching the generated database and
// manifest tree also repairs deleted copies and notices another catalog
// mode overwriting this profile's shared resource destination.
for resource in resources
.finish()
.unwrap_or_else(|error| panic!("failed to track generated catalog resources: {error}"))
{
println!("cargo:rerun-if-changed={}", resource.display());
}
}
fn catalog_root(mode: CatalogBuildMode) -> &'static Path {
Path::new(match mode {
CatalogBuildMode::FixtureDevelopment => FIXTURE_CATALOG_ROOT,
CatalogBuildMode::LocalDevelopment => "local-catalog",
CatalogBuildMode::Production => PRODUCTION_CATALOG_ROOT,
})
}
fn embed_startup_game_directory(mode: CatalogBuildMode) -> Result<(), Box<dyn std::error::Error>> {
let directory = if mode == CatalogBuildMode::LocalDevelopment {
let path = env::var_os(LOCAL_GAMES_DIR_ENV)
.ok_or_else(|| format!("{LOCAL_GAMES_DIR_ENV} is required for local test builds"))?;
let path = PathBuf::from(path).canonicalize()?;
if !path.is_dir() {
return Err("local test games path must be a directory".into());
}
Some(
path.into_os_string()
.into_string()
.map_err(|_| "local test games path must be UTF-8")?,
)
} else {
None
};
let out_dir = env::var_os("OUT_DIR").ok_or("OUT_DIR is not set")?;
// Rust string escaping keeps arbitrary path characters out of Cargo's
// line-based build-script protocol. Production compiles a literal None.
fs::write(
PathBuf::from(out_dir).join("startup-game-directory.rs"),
format!("{directory:?}"),
)?;
Ok(())
}
fn embed_catalog_content_index(mode: CatalogBuildMode) -> Result<(), Box<dyn std::error::Error>> {
let manifest_root = match mode {
CatalogBuildMode::FixtureDevelopment => Path::new(FIXTURE_MANIFEST_ROOT),
CatalogBuildMode::Production => Path::new("manifests"),
};
let manifest_root = catalog_root(mode).join("manifests");
let source = manifest_root.join(CATALOG_CONTENT_INDEX_NAME);
println!("cargo:rerun-if-changed={}", source.display());
let bytes = fs::read(&source)?;
@@ -74,12 +136,20 @@ fn catalog_build_mode() -> Result<CatalogBuildMode, Box<dyn std::error::Error>>
}
};
let cargo_profile = env::var("PROFILE").ok();
let local_development_opt_in = match env::var_os(LOCAL_DEVELOPMENT_ENV) {
None => false,
Some(value) if value == "1" => true,
Some(_) => {
return Err(format!("{LOCAL_DEVELOPMENT_ENV} must be exactly 1 when set").into());
}
};
let out_dir = env::var_os("OUT_DIR").map(std::path::PathBuf::from);
select_catalog_build_mode(CatalogGateInput {
base_config: &base_config,
config_override: config_override.as_deref(),
fixture_development_opt_in,
local_development_opt_in,
cargo_profile: cargo_profile.as_deref(),
out_dir: out_dir.as_deref(),
})
@@ -5,7 +5,17 @@ use std::{
use lanspread_compat::catalog_bundle::load_catalog_bundle;
const PRODUCTION_RESOURCES: [&str; 3] = ["assets/*", "game.db", "manifests/*"];
const PRODUCTION_RESOURCES: [(&str, &str); 3] = [
("assets/*", "assets/"),
("production-catalog/game.db", "game.db"),
("production-catalog/manifests/", "manifests/"),
];
const SOURCE_PRODUCTION_RESOURCES: [&str; 3] = ["assets/*", "game.db", "manifests/*"];
const LOCAL_RESOURCES: [(&str, &str); 3] = [
("assets/*", "assets/"),
("local-catalog/game.db", "game.db"),
("local-catalog/manifests/", "manifests/"),
];
const DEVELOPMENT_RESOURCES: [(&str, &str); 3] = [
(
"../../lanspread-peer-cli/catalogs/default/game.db",
@@ -21,6 +31,7 @@ const DEVELOPMENT_RESOURCES: [(&str, &str); 3] = [
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub(crate) enum CatalogBuildMode {
FixtureDevelopment,
LocalDevelopment,
Production,
}
@@ -29,6 +40,7 @@ pub(crate) struct CatalogGateInput<'a> {
pub(crate) base_config: &'a str,
pub(crate) config_override: Option<&'a str>,
pub(crate) fixture_development_opt_in: bool,
pub(crate) local_development_opt_in: bool,
pub(crate) cargo_profile: Option<&'a str>,
pub(crate) out_dir: Option<&'a Path>,
}
@@ -51,6 +63,21 @@ pub(crate) fn select_catalog_build_mode(
.any(|component| component.as_os_str() == "production")
});
if input.local_development_opt_in {
if input.fixture_development_opt_in || forced_production {
return Err(
"local catalog testing cannot be combined with fixture or production builds"
.to_owned(),
);
}
if resources != ResourceAuthority::LocalDevelopment {
return Err(
"local catalog opt-in requires the exact local development resource map".to_owned(),
);
}
return Ok(CatalogBuildMode::LocalDevelopment);
}
if input.fixture_development_opt_in && !forced_production {
if resources != ResourceAuthority::FixtureDevelopment {
return Err(
@@ -102,6 +129,7 @@ pub(crate) fn validate_production_catalog(
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum ResourceAuthority {
FixtureDevelopment,
LocalDevelopment,
Production,
}
@@ -137,8 +165,10 @@ fn classify_resources(resources: &serde_json::Value) -> Result<ResourceAuthority
})
.collect::<Result<Vec<_>, _>>()?;
let unique = actual.iter().copied().collect::<BTreeSet<_>>();
let expected = PRODUCTION_RESOURCES.into_iter().collect::<BTreeSet<_>>();
if actual.len() == PRODUCTION_RESOURCES.len() && unique == expected {
let expected = SOURCE_PRODUCTION_RESOURCES
.into_iter()
.collect::<BTreeSet<_>>();
if actual.len() == SOURCE_PRODUCTION_RESOURCES.len() && unique == expected {
return Ok(ResourceAuthority::Production);
}
return Err(format!(
@@ -156,18 +186,26 @@ fn classify_resources(resources: &serde_json::Value) -> Result<ResourceAuthority
.ok_or_else(|| "development resource destinations must be strings".to_owned())
})
.collect::<Result<BTreeMap<_, _>, _>>()?;
let expected = DEVELOPMENT_RESOURCES
let production = PRODUCTION_RESOURCES.into_iter().collect::<BTreeMap<_, _>>();
if actual == production {
return Ok(ResourceAuthority::Production);
}
let local = LOCAL_RESOURCES.into_iter().collect::<BTreeMap<_, _>>();
if actual == local {
return Ok(ResourceAuthority::LocalDevelopment);
}
let development = DEVELOPMENT_RESOURCES
.into_iter()
.collect::<BTreeMap<_, _>>();
if actual == expected {
if actual == development {
return Ok(ResourceAuthority::FixtureDevelopment);
}
return Err(format!(
"development resource map must be exactly {expected:?}, got {actual:?}"
"resource map must be exactly production {production:?}, fixture {development:?}, or local {local:?}, got {actual:?}"
));
}
Err("bundle.resources must be an exact production list or development map".to_owned())
Err("bundle.resources must be an exact production list or catalog resource map".to_owned())
}
#[cfg(test)]
@@ -186,7 +224,11 @@ mod tests {
static TEST_SEQUENCE: AtomicU64 = AtomicU64::new(0);
const PRODUCTION: &str = r#"{
"bundle": {"resources": ["game.db", "manifests/*", "assets/*"]}
"bundle": {"resources": {
"production-catalog/game.db": "game.db",
"production-catalog/manifests/": "manifests/",
"assets/*": "assets/"
}}
}"#;
const DEVELOPMENT: &str = r#"{
"bundle": {"resources": {
@@ -204,6 +246,7 @@ mod tests {
base_config: PRODUCTION,
config_override,
fixture_development_opt_in,
local_development_opt_in: false,
cargo_profile: Some("release"),
out_dir: Some(Path::new("target/release/build/app/out")),
}
@@ -219,6 +262,13 @@ mod tests {
select_catalog_build_mode(input(Some(PRODUCTION), false)),
Ok(CatalogBuildMode::Production)
);
assert_eq!(
select_catalog_build_mode(input(
Some(r#"{"bundle":{"resources":["game.db","manifests/*","assets/*"]}}"#),
false
)),
Ok(CatalogBuildMode::Production)
);
assert_eq!(
select_catalog_build_mode(input(Some(r#"{"build": {}}"#), false)),
Ok(CatalogBuildMode::Production)
@@ -235,6 +285,48 @@ mod tests {
assert!(select_catalog_build_mode(input(None, true)).is_err());
}
#[test]
fn local_catalog_is_an_explicit_build_time_mode_excluded_from_production() {
let local = include_str!("../tauri.local.conf.json");
let mut local_input = input(Some(local), false);
assert!(select_catalog_build_mode(local_input).is_err());
local_input.local_development_opt_in = true;
assert_eq!(
select_catalog_build_mode(local_input),
Ok(CatalogBuildMode::LocalDevelopment)
);
for config in [PRODUCTION, DEVELOPMENT] {
assert!(
select_catalog_build_mode(CatalogGateInput {
config_override: Some(config),
..local_input
})
.is_err()
);
}
assert!(
select_catalog_build_mode(CatalogGateInput {
fixture_development_opt_in: true,
..local_input
})
.is_err()
);
assert!(
select_catalog_build_mode(CatalogGateInput {
cargo_profile: Some("production"),
..local_input
})
.is_err()
);
assert!(
select_catalog_build_mode(CatalogGateInput {
out_dir: Some(Path::new("target/production/build/app/out")),
..local_input
})
.is_err()
);
}
#[test]
fn production_profile_cannot_be_downgraded_to_fixture_authority() {
for mut input in [
@@ -260,7 +352,7 @@ mod tests {
fn incomplete_duplicated_or_unknown_resource_shapes_fail_closed() {
for config in [
r#"{"bundle":{"resources":["game.db","manifests/*"]}}"#,
r#"{"bundle":{"resources":["game.db","manifests/*","assets/*","assets/*"]}}"#,
r#"{"bundle":{"resources":{"production-catalog/game.db":"game.db"}}}"#,
r#"{"bundle":{"resources":{"fixture.db":"game.db"}}}"#,
r#"{"bundle":{"resources":true}}"#,
"not JSON",
@@ -0,0 +1,118 @@
use std::{
fs,
io,
path::{Path, PathBuf},
time::SystemTime,
};
pub(crate) struct CatalogResources {
files: Vec<(PathBuf, Option<SystemTime>)>,
}
impl CatalogResources {
pub(crate) fn prepare(out_dir: &Path, source: &Path) -> io::Result<Self> {
let profile_dir = profile_directory(out_dir)?;
let mut relative_files = vec![PathBuf::from("game.db")];
for entry in fs::read_dir(source.join("manifests"))? {
relative_files.push(Path::new("manifests").join(entry?.file_name()));
}
relative_files.sort();
let files = relative_files
.into_iter()
.map(|relative| {
let destination = profile_dir.join(&relative);
let previous_mtime = unchanged_mtime(&source.join(relative), &destination)?;
Ok((destination, previous_mtime))
})
.collect::<io::Result<Vec<_>>>()?;
reset_catalog_resources(out_dir)?;
Ok(Self { files })
}
pub(crate) fn finish(self) -> io::Result<Vec<PathBuf>> {
self.files
.into_iter()
.map(|(path, previous_mtime)| {
// Cargo compares watched files to the build-script start time.
// Tauri unconditionally copies, so retain the PREVIOUS OUTPUT
// mtime for identical bytes to let the next build become fresh.
// Changed bytes keep their new mtime: restoring source mtimes
// would hide another feature variant overwriting shared outputs.
if let Some(modified) = previous_mtime {
fs::OpenOptions::new()
.write(true)
.open(&path)?
.set_modified(modified)?;
}
Ok(path)
})
.collect()
}
}
fn unchanged_mtime(source: &Path, destination: &Path) -> io::Result<Option<SystemTime>> {
let metadata = match fs::symlink_metadata(destination) {
Ok(metadata) if metadata.is_file() && !metadata.is_symlink() => metadata,
Ok(_) => return Ok(None),
Err(error) if error.kind() == io::ErrorKind::NotFound => return Ok(None),
Err(error) => return Err(error),
};
if metadata.len() == fs::metadata(source)?.len() && fs::read(source)? == fs::read(destination)?
{
Ok(Some(metadata.modified()?))
} else {
Ok(None)
}
}
/// Removes only Tauri's generated catalog copies before it copies the selected
/// authority. Tauri merges resource trees, which otherwise retains manifests
/// from a previous catalog mode or a larger previous production catalog.
pub(crate) fn reset_catalog_resources(out_dir: &Path) -> io::Result<[PathBuf; 2]> {
let profile_dir = profile_directory(out_dir)?;
let resources = [profile_dir.join("game.db"), profile_dir.join("manifests")];
for path in &resources {
match fs::symlink_metadata(path) {
Ok(metadata) if metadata.is_symlink() => {
return Err(io::Error::other(format!(
"generated catalog resource is a symlink: {}",
path.display()
)));
}
Ok(metadata) if metadata.is_dir() && path == &resources[1] => {
fs::remove_dir_all(path)?;
}
Ok(metadata) if metadata.is_file() => fs::remove_file(path)?,
Ok(_) => {
return Err(io::Error::other(format!(
"unexpected generated catalog resource type: {}",
path.display()
)));
}
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => return Err(error),
}
}
Ok(resources)
}
fn profile_directory(out_dir: &Path) -> io::Result<&Path> {
// Match tauri-build's OUT_DIR -> profile directory calculation, including
// custom target directories, cross-compilation targets and Cargo profiles.
// Require Cargo's layout before deriving any removal target.
if !out_dir.is_absolute()
|| out_dir.file_name().is_none_or(|name| name != "out")
|| out_dir
.ancestors()
.nth(2)
.and_then(Path::file_name)
.is_none_or(|name| name != "build")
{
return Err(io::Error::other("unexpected Cargo OUT_DIR layout"));
}
out_dir
.ancestors()
.nth(3)
.ok_or_else(|| io::Error::other("Cargo OUT_DIR has no profile directory"))
}
@@ -3010,6 +3010,11 @@ async fn set_local_network_sharing(
}
}
#[tauri::command]
fn get_startup_game_directory() -> Option<&'static str> {
include!(concat!(env!("OUT_DIR"), "/startup-game-directory.rs"))
}
#[tauri::command]
async fn update_game_directory(
app_handle: tauri::AppHandle,
@@ -4747,6 +4752,7 @@ pub fn run() {
run_game,
start_server,
game_directory_exists,
get_startup_game_directory,
update_game_directory,
update_game,
uninstall_game,
@@ -0,0 +1,22 @@
{
"$schema": "https://schema.tauri.app/config/2",
"productName": "softlan-launcher-local-test",
"identifier": "berlin.softlan.launcher.local-test",
"app": {
"windows": [
{
"title": "softlan-launcher — LOCAL TEST",
"width": 1526,
"height": 1120
}
]
},
"bundle": {
"active": false,
"resources": {
"local-catalog/game.db": "game.db",
"local-catalog/manifests/": "manifests/",
"assets/*": "assets/"
}
}
}
@@ -1,10 +1,10 @@
{
"$schema": "https://schema.tauri.app/config/2",
"bundle": {
"resources": [
"game.db",
"manifests/*",
"assets/*"
]
"resources": {
"production-catalog/game.db": "game.db",
"production-catalog/manifests/": "manifests/",
"assets/*": "assets/"
}
}
}
@@ -0,0 +1,372 @@
#[path = "../build_support/catalog_resources.rs"]
mod catalog_resources;
use std::{
fs,
path::{Path, PathBuf},
sync::atomic::{AtomicU64, Ordering},
time::{Duration, SystemTime},
};
use catalog_resources::{CatalogResources, reset_catalog_resources};
static NEXT_DIRECTORY: AtomicU64 = AtomicU64::new(0);
struct TestDirectory(PathBuf);
impl TestDirectory {
fn new() -> Self {
let path = std::env::temp_dir().join(format!(
"lanspread-catalog-resources-{}-{}",
std::process::id(),
NEXT_DIRECTORY.fetch_add(1, Ordering::Relaxed),
));
fs::create_dir(&path).expect("test directory should be unique");
Self(path)
}
}
impl Drop for TestDirectory {
fn drop(&mut self) {
fs::remove_dir_all(&self.0).expect("test directory should be removed");
}
}
#[test]
fn resets_catalog_copies_for_native_and_cross_target_profiles() {
let root = TestDirectory::new();
for profile in ["target/release", "custom/x86_64-pc-windows-msvc/production"] {
let profile = root.0.join(profile);
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
fs::create_dir(profile.join("manifests"))
.expect("fixture filesystem operation should succeed");
fs::write(profile.join("manifests/fixture.json"), b"old fixture")
.expect("fixture filesystem operation should succeed");
fs::write(
profile.join("manifests/removed-game.json"),
b"old production",
)
.expect("fixture filesystem operation should succeed");
fs::write(profile.join("game.db"), b"old database")
.expect("fixture filesystem operation should succeed");
fs::write(profile.join("launcher"), b"binary")
.expect("fixture filesystem operation should succeed");
fs::write(out_dir.join("embedded-index"), b"new authority")
.expect("fixture filesystem operation should succeed");
let resources =
reset_catalog_resources(&out_dir).expect("fixture filesystem operation should succeed");
assert_eq!(
resources,
[profile.join("game.db"), profile.join("manifests")]
);
assert!(resources.iter().all(|path| !path.exists()));
assert_eq!(
fs::read(profile.join("launcher"))
.expect("fixture filesystem operation should succeed"),
b"binary"
);
assert_eq!(
fs::read(out_dir.join("embedded-index"))
.expect("fixture filesystem operation should succeed"),
b"new authority"
);
// Missing copies are a normal input after manual cleanup or a clean build.
assert_eq!(
reset_catalog_resources(&out_dir).expect("fixture filesystem operation should succeed"),
resources
);
}
}
#[test]
fn rejects_non_cargo_layout_before_removing_anything() {
let root = TestDirectory::new();
fs::write(root.0.join("game.db"), b"keep")
.expect("fixture filesystem operation should succeed");
assert!(reset_catalog_resources(&root.0.join("unexpected/launcher/out")).is_err());
assert!(reset_catalog_resources(Path::new("out")).is_err());
assert!(reset_catalog_resources(Path::new("target/release/build/launcher/out")).is_err());
assert_eq!(
fs::read(root.0.join("game.db")).expect("fixture filesystem operation should succeed"),
b"keep"
);
}
#[cfg(unix)]
#[test]
fn rejects_linked_manifest_destination_without_traversing_it() {
let root = TestDirectory::new();
let profile = root.0.join("target/debug");
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
let source = root.0.join("source-manifests");
fs::create_dir(&source).expect("fixture filesystem operation should succeed");
fs::write(source.join("keep.json"), b"authority")
.expect("fixture filesystem operation should succeed");
std::os::unix::fs::symlink(&source, profile.join("manifests"))
.expect("fixture filesystem operation should succeed");
assert!(reset_catalog_resources(&out_dir).is_err());
assert_eq!(
fs::read(source.join("keep.json")).expect("fixture filesystem operation should succeed"),
b"authority"
);
}
fn timestamp(offset: u64) -> SystemTime {
SystemTime::UNIX_EPOCH + Duration::from_secs(1_600_000_000 + offset)
}
fn set_modified(path: &Path, modified: SystemTime) {
fs::OpenOptions::new()
.write(true)
.open(path)
.expect("fixture filesystem operation should succeed")
.set_modified(modified)
.expect("fixture filesystem operation should succeed");
}
fn write_catalog(root: &Path, database: &str, manifests: &[(&str, &str)]) {
fs::create_dir_all(root.join("manifests"))
.expect("fixture filesystem operation should succeed");
fs::write(root.join("game.db"), database).expect("fixture filesystem operation should succeed");
for (name, body) in manifests {
fs::write(root.join("manifests").join(name), body)
.expect("fixture filesystem operation should succeed");
}
for path in catalog_files(root) {
// Sources can be older than every build using them. Their timestamps
// must never replace the timestamps of changed destination bytes.
set_modified(&path, timestamp(0));
}
}
fn catalog_files(root: &Path) -> Vec<PathBuf> {
let mut paths = vec![root.join("game.db")];
paths.extend(
fs::read_dir(root.join("manifests"))
.expect("fixture filesystem operation should succeed")
.map(|entry| {
entry
.expect("manifest directory entry should be readable")
.path()
}),
);
paths.sort();
paths
}
fn simulate_tauri_copy(source: &Path, destination: &Path, modified: SystemTime) {
fs::create_dir_all(destination.join("manifests"))
.expect("fixture filesystem operation should succeed");
for path in catalog_files(source) {
let destination = destination.join(
path.strip_prefix(source)
.expect("catalog path should be below its source root"),
);
fs::copy(path, &destination).expect("fixture filesystem operation should succeed");
set_modified(&destination, modified);
}
}
fn assert_catalog_matches(source: &Path, destination: &Path, watched: &[PathBuf]) {
let expected = catalog_files(source)
.iter()
.map(|path| {
destination.join(
path.strip_prefix(source)
.expect("catalog path should be below its source root"),
)
})
.collect::<Vec<_>>();
assert_eq!(watched, expected);
assert_eq!(catalog_files(destination), expected);
for path in watched {
let source = source.join(
path.strip_prefix(destination)
.expect("copied path should be below its destination root"),
);
assert_eq!(
fs::read(path).expect("fixture filesystem operation should succeed"),
fs::read(source).expect("fixture filesystem operation should succeed")
);
}
}
#[test]
fn unchanged_copies_keep_previous_output_timestamps() {
let root = TestDirectory::new();
let source = root.0.join("source");
let profile = root.0.join("target/release");
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
write_catalog(&source, "database", &[("game.json", "manifest")]);
simulate_tauri_copy(&source, &profile, timestamp(10));
let resources =
CatalogResources::prepare(&out_dir, &source).expect("catalog resources should prepare");
simulate_tauri_copy(&source, &profile, timestamp(20));
let watched = resources.finish().expect("copied resources should finish");
assert_catalog_matches(&source, &profile, &watched);
for path in watched {
assert_eq!(
fs::metadata(path)
.expect("fixture filesystem operation should succeed")
.modified()
.expect("fixture filesystem operation should succeed"),
timestamp(10)
);
}
}
#[test]
fn changed_copies_keep_new_timestamps_then_stabilize_on_the_next_build() {
let root = TestDirectory::new();
let source = root.0.join("source");
let profile = root.0.join("target/release");
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
write_catalog(
&source,
"old database",
&[("game.json", "old manifest"), ("shared.json", "unchanged")],
);
simulate_tauri_copy(&source, &profile, timestamp(10));
// Equal lengths ensure that changed bytes, rather than just file sizes,
// distinguish a different catalog occupying the shared profile directory.
write_catalog(
&source,
"new database",
&[("game.json", "new manifest"), ("shared.json", "unchanged")],
);
for copied_at in [timestamp(20), timestamp(30)] {
let resources =
CatalogResources::prepare(&out_dir, &source).expect("catalog resources should prepare");
simulate_tauri_copy(&source, &profile, copied_at);
let watched = resources.finish().expect("copied resources should finish");
assert_catalog_matches(&source, &profile, &watched);
for path in watched {
let expected = if path.ends_with("shared.json") {
timestamp(10)
} else {
timestamp(20)
};
assert_eq!(
fs::metadata(path)
.expect("fixture filesystem operation should succeed")
.modified()
.expect("fixture filesystem operation should succeed"),
expected
);
}
}
}
#[test]
fn missing_database_manifest_or_manifest_directory_is_repaired_and_watched() {
let root = TestDirectory::new();
let source = root.0.join("source");
let profile = root.0.join("target/release");
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
write_catalog(&source, "database", &[("game.json", "manifest")]);
for missing in ["game.db", "manifests/game.json", "manifests"] {
simulate_tauri_copy(&source, &profile, timestamp(10));
let missing_path = profile.join(missing);
if missing_path.is_dir() {
fs::remove_dir_all(&missing_path).expect("fixture filesystem operation should succeed");
} else {
fs::remove_file(&missing_path).expect("fixture filesystem operation should succeed");
}
let resources =
CatalogResources::prepare(&out_dir, &source).expect("catalog resources should prepare");
simulate_tauri_copy(&source, &profile, timestamp(20));
let watched = resources.finish().expect("copied resources should finish");
assert_catalog_matches(&source, &profile, &watched);
for path in watched {
let expected = if path.starts_with(&missing_path) {
timestamp(20)
} else {
timestamp(10)
};
assert_eq!(
fs::metadata(path)
.expect("fixture filesystem operation should succeed")
.modified()
.expect("fixture filesystem operation should succeed"),
expected
);
}
}
}
#[test]
fn mode_switch_and_catalog_shrink_remove_previous_manifests() {
let root = TestDirectory::new();
let fixture = root.0.join("fixture");
let production = root.0.join("production");
let profile = root.0.join("target/release");
let out_dir = profile.join("build/launcher-test/out");
fs::create_dir_all(&out_dir).expect("fixture filesystem operation should succeed");
write_catalog(
&fixture,
"fixture database",
&[
("fixture.json", "fixture only"),
("shared.json", "fixture game"),
],
);
write_catalog(
&production,
"production database",
&[
("production.json", "production only"),
("shared.json", "real game"),
],
);
simulate_tauri_copy(&fixture, &profile, timestamp(10));
let resources = CatalogResources::prepare(&out_dir, &production)
.expect("production resources should prepare");
simulate_tauri_copy(&production, &profile, timestamp(20));
assert_catalog_matches(
&production,
&profile,
&resources.finish().expect("copied resources should finish"),
);
assert!(!profile.join("manifests/fixture.json").exists());
fs::remove_file(production.join("manifests/production.json"))
.expect("fixture filesystem operation should succeed");
write_catalog(
&production,
"smaller database",
&[("shared.json", "real game")],
);
let resources = CatalogResources::prepare(&out_dir, &production)
.expect("production resources should prepare");
simulate_tauri_copy(&production, &profile, timestamp(30));
assert_catalog_matches(
&production,
&profile,
&resources.finish().expect("copied resources should finish"),
);
assert!(!profile.join("manifests/production.json").exists());
let resources =
CatalogResources::prepare(&out_dir, &fixture).expect("fixture resources should prepare");
simulate_tauri_copy(&fixture, &profile, timestamp(40));
assert_catalog_matches(
&fixture,
&profile,
&resources.finish().expect("copied resources should finish"),
);
}
@@ -100,6 +100,9 @@ export const useGameDirectory = (backendPolicyReady = true) => {
const initialSelectionVersion = selectionVersionRef.current;
const hydration = hydrateGameDirectory({
loadSavedPath: async () => {
const startupPath = await invoke<string | null>('get_startup_game_directory');
if (startupPath !== null) return startupPath;
const store = await load(SETTINGS_FILE, SETTINGS_FILE_OPTIONS);
return await store.get<string>(GAME_DIR_KEY);
},
+72 -22
View File
@@ -8,12 +8,20 @@ default: run
FIXTURE_CATALOG_SOURCE := "crates/lanspread-tauri-deno-ts/src-tauri/game.db"
FIXTURE_CATALOG_ROOT := "crates/lanspread-peer-cli/catalogs"
PRODUCTION_CATALOG_DB := "crates/lanspread-tauri-deno-ts/src-tauri/game.db"
PRODUCTION_MANIFEST_ROOT := "crates/lanspread-tauri-deno-ts/src-tauri/manifests"
PRODUCTION_CATALOG_SOURCE := "crates/lanspread-tauri-deno-ts/src-tauri/game.db"
PRODUCTION_CATALOG_ROOT := "crates/lanspread-tauri-deno-ts/src-tauri/production-catalog"
PRODUCTION_CATALOG_DB := PRODUCTION_CATALOG_ROOT / "game.db"
PRODUCTION_MANIFEST_ROOT := PRODUCTION_CATALOG_ROOT / "manifests"
CATALOG_CACHE_STAMP := ".lanspread/catalog-cache/production.json"
LOCAL_CATALOG_ROOT := "crates/lanspread-tauri-deno-ts/src-tauri/local-catalog"
LOCAL_CATALOG_DB := LOCAL_CATALOG_ROOT / "game.db"
LOCAL_MANIFEST_ROOT := LOCAL_CATALOG_ROOT / "manifests"
LOCAL_CATALOG_CACHE_STAMP := ".lanspread/catalog-cache/local.json"
CATALOG_UNRAR := env_var_or_default("LANSPREAD_UNRAR", "crates/lanspread-tauri-deno-ts/src-tauri/binaries/unrar-x86_64-unknown-linux-gnu")
GAMES_DIR := env_var_or_default("LANSPREAD_GAMES_DIR", "")
CATALOG_FORCE := env_var_or_default("LANSPREAD_CATALOG_FORCE", "0")
CATALOG_MISSING_MODE := env_var_or_default("LANSPREAD_CATALOG_MISSING_MODE", "")
CATALOG_VERSION_MODE := env_var_or_default("LANSPREAD_CATALOG_VERSION_MODE", "")
TAURI_DEV_CONFIG := '{"bundle":{"resources":{"../../lanspread-peer-cli/catalogs/default/game.db":"game.db","../../lanspread-peer-cli/catalogs/default/manifests/":"manifests/","assets/*":"assets/"}}}'
TAURI_FIXTURE_ENV := "LANSPREAD_USE_FIXTURE_CATALOG=1"
@@ -21,30 +29,34 @@ setup:
cargo install tauri-cli
cd crates/lanspread-tauri-deno-ts && deno install --frozen=true
run:
if [ -n "$GAMES_DIR" ]; then \
just run-production "$GAMES_DIR"; \
else \
just run-fixture; \
fi
run: catalog-check-production
cargo tauri dev --config crates/lanspread-tauri-deno-ts/src-tauri/tauri.production.conf.json --release
run-fixture: fixture-catalog-check
{{ TAURI_FIXTURE_ENV }} cargo tauri dev --config crates/lanspread-tauri-deno-ts/src-tauri/tauri.dev.conf.json --release
run-production GAMES_DIR:
just catalog-generate-production "$GAMES_DIR"
run-production GAMES_DIR MODE_1="" MODE_2="":
just catalog-generate-production "$GAMES_DIR" "$MODE_1" "$MODE_2"
just catalog-check-production
cargo tauri dev --config crates/lanspread-tauri-deno-ts/src-tauri/tauri.production.conf.json --release
# Build a separate test authority from these local games and select their folder.
run-local GAMES_DIR:
@local_games_dir="$(cd -- "$GAMES_DIR" && pwd -P)" && \
echo "LOCAL TEST MODE: $local_games_dir (separate local catalog; development build only)" && \
just catalog-generate-local "$local_games_dir" && \
LANSPREAD_USE_LOCAL_CATALOG=1 LANSPREAD_LOCAL_GAMES_DIR="$local_games_dir" \
cargo tauri dev --config crates/lanspread-tauri-deno-ts/src-tauri/tauri.local.conf.json --release
build:
just build-production "$GAMES_DIR"
just build-production "$GAMES_DIR" "$CATALOG_MISSING_MODE" "$CATALOG_VERSION_MODE"
build-fixture: fixture-catalog-check
{{ TAURI_FIXTURE_ENV }} cargo tauri build --config crates/lanspread-tauri-deno-ts/src-tauri/tauri.dev.conf.json --no-bundle #-- --profile dev
build-production GAMES_DIR="":
build-production GAMES_DIR="" MODE_1="" MODE_2="":
if [ -n "$GAMES_DIR" ]; then \
just catalog-generate-production "$GAMES_DIR"; \
just catalog-generate-production "$GAMES_DIR" "$MODE_1" "$MODE_2"; \
else \
just catalog-check-production; \
fi
@@ -148,30 +160,68 @@ catalog-check-production:
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--all
# Generate the complete production authority from PACKAGES_DIR/<game_id>/.
catalog-generate-production PACKAGES_DIR:
@if [ "$CATALOG_FORCE" = "1" ] || ! python3 tools/catalog_source_cache.py check \
# Generate production authority from PACKAGES_DIR/<game_id>/.
# Pass allow-missing-games and/or allow-version-mismatch as explicit modes.
catalog-generate-production PACKAGES_DIR MODE_1="" MODE_2="":
@set --; \
for mode in "$MODE_1" "$MODE_2"; do \
case "$mode" in \
"") ;; \
allow-missing-games) set -- "$@" --allow-missing-games ;; \
allow-version-mismatch) set -- "$@" --allow-version-mismatch ;; \
*) echo "mode must be allow-missing-games or allow-version-mismatch" >&2; exit 2 ;; \
esac; \
done; \
if [ "$CATALOG_FORCE" = "1" ] || ! python3 tools/catalog_source_cache.py check \
--stamp {{ CATALOG_CACHE_STAMP }} \
--packages-dir "$PACKAGES_DIR" \
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--unrar "$CATALOG_UNRAR"; then \
cargo run --release -p lanspread-compat --bin lanspread-catalog-publisher -- generate \
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--unrar "$CATALOG_UNRAR" "$@"; then \
cargo run --release -p lanspread-compat --bin lanspread-production-catalog -- \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--packages-dir "$PACKAGES_DIR" \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--output-dir {{ PRODUCTION_CATALOG_ROOT }} \
--unrar "$CATALOG_UNRAR" \
--all && \
"$@" && \
python3 tools/catalog_source_cache.py record \
--stamp {{ CATALOG_CACHE_STAMP }} \
--packages-dir "$PACKAGES_DIR" \
--catalog-db {{ PRODUCTION_CATALOG_DB }} \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--manifests-dir {{ PRODUCTION_MANIFEST_ROOT }} \
--unrar "$CATALOG_UNRAR"; \
--unrar "$CATALOG_UNRAR" "$@"; \
else \
echo "catalog generation skipped: source metadata matches {{ CATALOG_CACHE_STAMP }}"; \
fi
# Local test authority never overwrites the generated production catalog.
catalog-generate-local PACKAGES_DIR:
@if [ "$CATALOG_FORCE" = "1" ] || ! python3 tools/catalog_source_cache.py check \
--stamp {{ LOCAL_CATALOG_CACHE_STAMP }} \
--packages-dir "$PACKAGES_DIR" \
--catalog-db {{ LOCAL_CATALOG_DB }} \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--manifests-dir {{ LOCAL_MANIFEST_ROOT }} \
--unrar "$CATALOG_UNRAR" --allow-missing-games --allow-version-mismatch; then \
cargo run --release -p lanspread-compat --bin lanspread-production-catalog -- \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--packages-dir "$PACKAGES_DIR" \
--output-dir {{ LOCAL_CATALOG_ROOT }} \
--unrar "$CATALOG_UNRAR" \
--allow-missing-games --allow-version-mismatch && \
python3 tools/catalog_source_cache.py record \
--stamp {{ LOCAL_CATALOG_CACHE_STAMP }} \
--packages-dir "$PACKAGES_DIR" \
--catalog-db {{ LOCAL_CATALOG_DB }} \
--source-catalog-db {{ PRODUCTION_CATALOG_SOURCE }} \
--manifests-dir {{ LOCAL_MANIFEST_ROOT }} \
--unrar "$CATALOG_UNRAR" --allow-missing-games --allow-version-mismatch; \
else \
echo "local catalog generation skipped: source metadata matches {{ LOCAL_CATALOG_CACHE_STAMP }}"; \
fi
# Regenerate one game in an already complete production authority.
catalog-generate-production-game PACKAGES_DIR GAME_ID:
cargo run -p lanspread-compat --bin lanspread-catalog-publisher -- generate \
+44 -4
View File
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Cheap source metadata cache for complete catalog publication."""
"""Cheap source metadata cache for production catalog publication."""
from __future__ import annotations
@@ -14,7 +14,7 @@ from pathlib import Path
from typing import Iterable
CACHE_SCHEMA = 1
CACHE_SCHEMA = 3
CATALOG_CONTENT_INDEX = "catalog-content-index-v1.jsonl"
CATALOG_PUBLICATION_MARKER = ".lanspread-catalog-publication-in-progress"
@@ -74,8 +74,11 @@ def _path_metadata(path: Path) -> dict[str, int | str]:
def _cache_payload(
packages_dir: Path,
catalog_db: Path,
source_catalog_db: Path,
manifests_dir: Path,
unrar: Path,
allow_missing_games: bool,
allow_version_mismatch: bool,
) -> dict[str, object]:
return {
"schema": CACHE_SCHEMA,
@@ -84,11 +87,17 @@ def _cache_payload(
"path": _absolute(catalog_db),
**_path_metadata(catalog_db),
},
"source_catalog_db": {
"path": _absolute(source_catalog_db),
**_path_metadata(source_catalog_db),
},
"manifests_dir": _absolute(manifests_dir),
"unrar": {
"path": _absolute(unrar),
**_path_metadata(unrar),
},
"allow_missing_games": allow_missing_games,
"allow_version_mismatch": allow_version_mismatch,
"packages_fingerprint": _tree_fingerprint(packages_dir),
}
@@ -110,8 +119,11 @@ def cache_is_current(
stamp: Path,
packages_dir: Path,
catalog_db: Path,
source_catalog_db: Path,
manifests_dir: Path,
unrar: Path,
allow_missing_games: bool = False,
allow_version_mismatch: bool = False,
) -> bool:
if not _output_is_ready(catalog_db, manifests_dir):
return False
@@ -119,21 +131,40 @@ def cache_is_current(
cached = json.loads(stamp.read_text(encoding="utf-8"))
except (FileNotFoundError, OSError, UnicodeDecodeError, json.JSONDecodeError):
return False
return cached == _cache_payload(packages_dir, catalog_db, manifests_dir, unrar)
return cached == _cache_payload(
packages_dir,
catalog_db,
source_catalog_db,
manifests_dir,
unrar,
allow_missing_games,
allow_version_mismatch,
)
def record_cache(
stamp: Path,
packages_dir: Path,
catalog_db: Path,
source_catalog_db: Path,
manifests_dir: Path,
unrar: Path,
allow_missing_games: bool = False,
allow_version_mismatch: bool = False,
) -> None:
if not _output_is_ready(catalog_db, manifests_dir):
raise RuntimeError("cannot record a catalog cache before publication is complete")
stamp.parent.mkdir(parents=True, exist_ok=True)
payload = json.dumps(
_cache_payload(packages_dir, catalog_db, manifests_dir, unrar),
_cache_payload(
packages_dir,
catalog_db,
source_catalog_db,
manifests_dir,
unrar,
allow_missing_games,
allow_version_mismatch,
),
ensure_ascii=True,
indent=2,
sort_keys=True,
@@ -162,8 +193,11 @@ def _parser() -> argparse.ArgumentParser:
subparser.add_argument("--stamp", type=Path, required=True)
subparser.add_argument("--packages-dir", type=Path, required=True)
subparser.add_argument("--catalog-db", type=Path, required=True)
subparser.add_argument("--source-catalog-db", type=Path, required=True)
subparser.add_argument("--manifests-dir", type=Path, required=True)
subparser.add_argument("--unrar", type=Path, required=True)
subparser.add_argument("--allow-missing-games", action="store_true")
subparser.add_argument("--allow-version-mismatch", action="store_true")
return parser
@@ -173,8 +207,11 @@ def main(arguments: list[str] | None = None) -> int:
options.stamp,
options.packages_dir,
options.catalog_db,
options.source_catalog_db,
options.manifests_dir,
options.unrar,
options.allow_missing_games,
options.allow_version_mismatch,
)
if options.command == "check":
if current:
@@ -187,8 +224,11 @@ def main(arguments: list[str] | None = None) -> int:
options.stamp,
options.packages_dir,
options.catalog_db,
options.source_catalog_db,
options.manifests_dir,
options.unrar,
options.allow_missing_games,
options.allow_version_mismatch,
)
print(f"catalog cache recorded: {options.stamp}")
return 0
+57 -1
View File
@@ -20,6 +20,8 @@ class CatalogSourceCacheTests(unittest.TestCase):
(self.package / "game.eti").write_bytes(b"eti")
self.catalog_db = self.root / "game.db"
self.catalog_db.write_bytes(b"catalog")
self.source_catalog_db = self.root / "source-game.db"
self.source_catalog_db.write_bytes(b"source catalog")
self.unrar = self.root / "unrar"
self.unrar.write_bytes(b"unrar")
self.manifests = self.root / "manifests"
@@ -38,6 +40,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -46,6 +49,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -54,6 +58,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -64,6 +69,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -74,6 +80,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -83,6 +90,7 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
@@ -93,15 +101,63 @@ class CatalogSourceCacheTests(unittest.TestCase):
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
stamp = json.loads(self.stamp.read_text(encoding="utf-8"))
self.assertEqual(stamp["schema"], 1)
self.assertEqual(stamp["schema"], 3)
self.assertEqual(stamp["catalog_db"]["size"], len(b"catalog"))
self.assertEqual(
stamp["source_catalog_db"]["size"], len(b"source catalog")
)
self.assertEqual(stamp["unrar"]["size"], len(b"unrar"))
self.assertEqual(stamp["packages_dir"], str(self.packages.absolute()))
def test_source_catalog_and_generation_modes_are_cache_inputs(self) -> None:
catalog_source_cache.record_cache(
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
self.source_catalog_db.write_bytes(b"changed source catalog")
self.assertFalse(
catalog_source_cache.cache_is_current(
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
)
)
self.source_catalog_db.write_bytes(b"source catalog")
self.assertFalse(
catalog_source_cache.cache_is_current(
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
allow_missing_games=True,
)
)
self.assertFalse(
catalog_source_cache.cache_is_current(
self.stamp,
self.packages,
self.catalog_db,
self.source_catalog_db,
self.manifests,
self.unrar,
allow_version_mismatch=True,
)
)
if __name__ == "__main__":
unittest.main()