Greatest: serve catalog covers from a local copy (BL-0053)

The page hotlinked ~100 AniList CDN covers per load and some failed at random.
The reader now downloads each catalog cover once (sequential, ~1/s, stops on
401/403/429, images up to 2 MB) into data/greatest-covers/ and serves it from
/api/greatest-cover with a year-long browser cache; until a copy exists the
route redirects to the catalog URL. The page also accepts the server's
relative cover URLs, so owned titles show their library cover again.
GREATEST_COVER_WARM=0 turns the downloads off (tests, offline installs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MaashkinoozheandClaude Opus 5.5 committed 2026-10-26 00:38:55 +01:00
1 parent cd6a30a76b
commit e9da27e688
16 files changed
+277 -11

No files matched your search

+1
View File
@@ -207,6 +207,7 @@ All requests and responses are JSON. POSTs require `Content-Type: application/js
| `POST /api/admin/user/role` `{name, admin}` | admin | Grant or revoke admin rights; keeps the password and sessions. Revoking your own rights is refused, so one admin always remains |
| `POST /api/admin/user/delete` `{name}` | admin | Delete a user and their data |
| `GET /api/admin/backup` · `POST /api/admin/backup` `{command:"settings"\|"run"}` | admin | Built-in backup status, schedule/retention, "back up now" ([backup.md](backup.md)) |
| `GET /api/greatest-cover?id=<entry>&v=<version>` | user | The Greatest entry's cover from `data/greatest-covers/` (`greatest/covers.js`), cached by the browser for a year; `v` changes with the catalog URL. A `302` to the catalog URL until the copy exists. `/api/greatest` lists this URL as `cover` for titles not in the library |
| `GET /api/discovery` | user | The caller's discovery feed, settings, feedback and `refresh: { state: idle\|queued\|running, position?, nextAllowedAt?, blockedReason? }` (§6.6) |
| `POST /api/discovery/feedback` `{candidateId, value}` | user | The caller's choice for a candidate; requires opt-in |
| `POST /api/discovery/refresh` | user | Queue a per-user AniList harvest for the caller (opt-in required, else `403`). `{ harvestId, position }`; `409` while one is queued or running for the caller; `429 { nextAllowedAt }` within an hour of the last accepted refresh; `503 { blockedReason }` after AniList pushed back, until a nightly or admin harvest |
+2 -1
View File
@@ -21,7 +21,7 @@ Data on the host, all bind mounts (no named volumes):
| Path (`.env`) | Contents | Back up? |
|---|---|---|
| `LIBRARY_PATH` | The manga: `Manga/Book/Chapter/images` ([library-format.md](library-format.md)) | Yes, it is your collection |
| `DATA_PATH` | `db.json` (users, progress, votes), catalog, discovery feeds, library index cache | Yes (built-in backup, section 8). `discovery-cache/` inside it is safe to delete and may be left out of backups: the next harvest rebuilds it |
| `DATA_PATH` | `db.json` (users, progress, votes), catalog, discovery feeds, library index cache | Yes (built-in backup, section 8). `discovery-cache/` inside it is safe to delete and may be left out of backups: the next harvest rebuilds it. The same holds for `greatest-covers/` (local copies of the Greatest catalog's cover images, fetched again after the next start) |
| `SCRAPER_DATA_PATH` | Queue, settings, source ledger, metadata cache | Included in the built-in backup |
| `BACKUP_PATH` | Built-in backup archives | Keep it on another disk or NAS |
| `SCRAPER_TOKEN_PATH` | Shared secret between reader and scraper | Regenerate if lost |
@@ -71,6 +71,7 @@ demo manga in an empty library after the build in section 5:
| `READING_WINDOWS_JSON`, `READING_TIMEZONE`, `READING_QUIZ_*` | off | Optional server-enforced reading hours ([reading-limiter.md](reading-limiter.md)) |
| `DISCOVERY_HARVEST_AT` | `03:30` | Daily time (`HH:MM`) of the nightly Discover harvest from AniList, in the container's local time zone (`TZ`). Only users who opted in are harvested; the admin can change the time or switch it off on the Discover page ([discovery.md](discovery.md)) |
| `DISCOVERY_ANILIST_URL` | AniList's public API | Tests only: points the discovery harvest at a fixture server. Leave unset in production |
| `GREATEST_COVER_WARM` | `1` | `0` stops the reader from downloading the Greatest catalog's covers (about 100 images, one per second, 10 s after start and once a day for missing ones). Tests and offline installs set `0`; the page then loads covers from the catalog's own URLs |
| `APP_IMAGE`, `VCS_REF` | `mangacircus:local`, `development` | Image to run; set a pinned registry image to deploy prebuilt releases (section 7) |
| `COMPOSE_FILE` | `docker-compose.yml` | Extra Compose files, `:`-separated (e.g. `docker-compose.yml:deploy/compose.https.yml`) |
| `MANGACIRCUS_DOMAIN` | – | Host name for `deploy/compose.https.yml` |
+1 -1
View File
@@ -23,7 +23,7 @@ Within each cohort, **Rank by my vote** sorts flame → good → neutral → sku
The one-time seed is curated from the research document. All 106 catalog entries now have a cover URL. Most were matched to public AniList manga metadata; the five initial gaps were checked individually. Phoenix (1954) and Nausicaä use matching AniList records; Kitarō uses Drawn & Quarterly's Kitaro edition, Someone Hertz uses Shonen Jump's first-volume cover, and Hitoner uses Shueisha's 2026 first-volume cover. Hitoner's AniList 2024 entry represents an earlier work, so it was **not** used for the 2026 series. The five added URLs returned image content on 2026-10-09, but remote hosts can change them; the UI still shows a title-letter fallback if an image fails.
`node tools/enrich-greatest-catalog.js` is an **optional manual** public-AniList metadata refresh, paced at one request every 1.5 seconds. It only accepts matching manga title and **original serialization year**, leaving uncertain results untouched rather than using a sequel or remake cover. The dashboard makes **no metadata calls when loading**; only an admin pressing **Search MangaDex** triggers a bounded lookup. Browser covers are lazy-loaded from their listed source; local library covers take precedence.
`node tools/enrich-greatest-catalog.js` is an **optional manual** public-AniList metadata refresh, paced at one request every 1.5 seconds. It only accepts matching manga title and **original serialization year**, leaving uncertain results untouched rather than using a sequel or remake cover. The dashboard makes **no metadata calls when loading**; only an admin pressing **Search MangaDex** triggers a bounded lookup. Local library covers take precedence. For the rest, the reader keeps its own copy of each catalog cover in `data/greatest-covers/` (`greatest/covers.js`): it downloads missing ones one at a time, about one a second, 10 seconds after start and once a day, stops on HTTP 401/403/429, and accepts only JPEG/PNG/WebP/GIF/AVIF images up to 2 MB from the catalog's own HTTPS URLs. The page loads covers from `/api/greatest-cover`, which the browser caches for a year; until a copy exists that route redirects to the catalog URL. Hotlinking about 100 covers per page load made the image CDN drop some of them at random. `GREATEST_COVER_WARM=0` switches the downloads off.
Public catalog chapter counts are often absent; the UI says “Chapters unknown” rather than inventing one. For owned titles, the local library count wins. Status comes from AniList's releasing/finished flag or local library metadata, and may be unknown. Official source links may be publisher editions, chapter services, or an award/info lead; they do not imply all chapters are available in every country. No unverified hoster coverage is claimed.