Greatest: serve catalog covers from a local copy (BL-0053)

The page hotlinked ~100 AniList CDN covers per load and some failed at random.
The reader now downloads each catalog cover once (sequential, ~1/s, stops on
401/403/429, images up to 2 MB) into data/greatest-covers/ and serves it from
/api/greatest-cover with a year-long browser cache; until a copy exists the
route redirects to the catalog URL. The page also accepts the server's
relative cover URLs, so owned titles show their library cover again.
GREATEST_COVER_WARM=0 turns the downloads off (tests, offline installs).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

.dockerignore

@@ -5,7 +5,9 @@
 !library.js
 !reading-direction.js
 !chapter-identity.js
+!chapter-dates.js
 !catalog.js
+!backup.js
 !reading-analytics.js
 !reading-access.js
 !public/
@@ -16,8 +18,6 @@
 !discovery/**
 !runtime/
 !runtime/**
-!sentinel/
-!sentinel/**
 !scraper/
 !scraper/**
 !peering/

.env.example

@@ -7,15 +7,22 @@ READER_PORT=8080
 LIBRARY_PATH=./library
 DATA_PATH=./data
 SCRAPER_DATA_PATH=./data/scraper
-SENTINEL_DATA_PATH=./data/sentinel
 SCRAPER_TOKEN_PATH=./.secrets/scraper_token
-TZ=Europe/Berlin
+# Built-in backups of DATA_PATH (docs/backup.md): an existing folder, ideally on another
+# disk/NAS, writable by uid 1000. Unset = backups off (./backup is then mounted unused).
+# BACKUP_PATH=./backup
+TZ=UTC
 COOKIE_SECURE=0
 # Optional server-enforced reading windows. Day keys: 0=Sunday through 6=Saturday.
 # Ranges may cross midnight. Multiple ranges per day are supported.
 # READING_WINDOWS_JSON='{"0":[["08:00","01:00"]],"1":[["07:00","24:00"]],"2":[["07:00","24:00"]],"3":[["07:00","24:00"]],"4":[["07:00","24:00"]],"5":[["07:00","24:00"]],"6":[["08:00","01:00"]]}'
-READING_TIMEZONE=Europe/Berlin
+READING_TIMEZONE=UTC
 READING_QUIZ_UNLOCK_MINUTES=60
 # Optional deployment-managed JSON file under DATA_PATH, e.g. /data/reading-quiz.json.
 # READING_QUIZ_CATALOG_PATH=/data/reading-quiz.json
 PROXY_NETWORK=mangacircus-edge
+# Extra Compose files, ':'-separated (docs/deployment.md), e.g. HTTPS with the bundled Caddy:
+# COMPOSE_FILE=docker-compose.yml:deploy/compose.https.yml
+# MANGACIRCUS_DOMAIN=manga.example.org
+# PEER_DATA_PATH=./data/peering
+# PEER_USAGE_PATH=./data/peering-usage

.gitattributes

@@ -0,0 +1,4 @@
+# Shell scripts and config used inside Linux containers keep LF on every platform
+*.sh text eol=lf
+deploy/Caddyfile text eol=lf
+*.yml text eol=lf

.gitignore

@@ -3,6 +3,8 @@
 /data/
 # the manga library: demo content is generated by `node tools/make-dummy.js`, a real one is yours alone
 /library/
+# built-in backups (BACKUP_PATH default)
+/backup/
 
 # --- Node ---
 node_modules/
@@ -19,7 +21,6 @@ pnpm-debug.log*
 !.env.example
 !.env.*.example
 .secrets/
-/ops/proxmox/environment.json
 *.pem
 *.key
 
@@ -50,3 +51,10 @@ $RECYCLE.BIN/
 *.swp
 *.swo
 *~
+
+# --- deploy/circus.sh state ---
+.queue-was-paused
+
+# --- personal agent coordination (synced by a private tool; never published) ---
+CLAUDE.local.md
+.claude/skills/manga-org/

AGENTS.md

@@ -0,0 +1,54 @@
+# Working on MangaCircus
+
+Guide for contributors and coding agents. Users who only want to run it: [docs/deployment.md](docs/deployment.md).
+
+## Layout
+
+| Path | What |
+|---|---|
+| `server.js`, `library.js` | Reader: HTTP server, accounts, progress, JSON storage; folder scanner and image-size reader |
+| `chapter-identity.js`, `chapter-dates.js`, `reading-*.js`, `catalog.js`, `backup.js` | Shared modules (chapter keys, chapter dates, reading clock/limits, collections, built-in backup) |
+| `public/` | The web pages: plain HTML, CSS and JavaScript, no build step |
+| `scraper/` | Download queue daemon (`service.js`, `daemon.js`), source adapters and parsers, the publisher that writes the library, metadata lookups |
+| `peering/` | Optional server-to-server sharing (gateway, client, adapter) |
+| `runtime/` | In-container supervisor and configuration |
+| `discovery/`, `greatest/` | Discovery Scout and the Greatest of All Time dashboard |
+| `tools/` | Demo library, health checks, maintenance tools (dry run by default), UI and Compose test drivers |
+| `deploy/` | `circus.sh` operations helper, HTTPS with Caddy |
+| `docs/`, `vision/` | Contracts and guides; design notes for planned features |
+
+## Rules
+
+- **No runtime dependencies.** Node.js built-ins only. Playwright is an optional dev tool for UI tests.
+- **One writer per store.** The reader alone writes `DATA` (users, progress, catalog). The scraper alone
+  writes the library and its queue. Tools that edit `chapter.json` files are dry runs by default and need
+  `--apply --writers-stopped`.
+- **Library writes go through the publisher** (`scraper/lib/publisher.js`): staging folder, validated
+  images, one atomic rename. The folder contract is [docs/scraper-guide.md](docs/scraper-guide.md).
+- **Sources are polite.** Sequential requests, persisted per-host delays, stop on 401/403/429, never bypass
+  access controls ([scraper/AGENT_RUNBOOK.md](scraper/AGENT_RUNBOOK.md)).
+- **No secrets or personal data in the repository**: no `.env`, `.secrets/`, `data/`, libraries, host
+  names, IP addresses or local paths. Tests use disposable fixtures and never a real library or user
+  database.
+- Keep the docs current in the same change: a new file format field goes into
+  [docs/library-format.md](docs/library-format.md), a new setting into the README or
+  [docs/deployment.md](docs/deployment.md), an API change into [docs/architecture.md](docs/architecture.md).
+- Match the surrounding style: small functions, short comments that say why, no framework.
+
+## Tests
+
+```bash
+npm test                         # reader, scraper, runtime and peering suites (node --test)
+```
+
+Without Node on the machine, run them in Docker (the peering test needs `openssl`, present in `node:24`):
+
+```bash
+docker run --rm -v "$PWD:/app" -w /app node:24 npm test
+```
+
+Browser and Compose acceptance tests (Playwright, disposable data, never the live stack):
+`npm run test:ui`, `npm run test:discovery-ui`, `npm run test:direction-ui`, `npm run test:peer-ab`.
+
+Try changes against demo data: `node tools/make-dummy.js library && npm run start:all`, or
+`docker compose -f docker-compose.yml -f compose.dev.yml up -d --build` (mounts the editable source).

CLAUDE.md

@@ -0,0 +1,3 @@
+# MangaCircus
+
+Read and follow @AGENTS.md (layout, rules, tests). Deployment: docs/deployment.md.

Dockerfile

@@ -3,17 +3,16 @@ RUN apk add --no-cache tzdata openssl
 WORKDIR /app
 ARG VCS_REF=development
 LABEL org.opencontainers.image.title="MangaCircus" org.opencontainers.image.revision=$VCS_REF
-COPY package.json server.js library.js reading-direction.js chapter-identity.js reading-analytics.js reading-access.js catalog.js ./
+COPY package.json server.js library.js reading-direction.js chapter-identity.js chapter-dates.js reading-analytics.js reading-access.js catalog.js backup.js ./
 COPY public ./public
 COPY greatest ./greatest
 COPY discovery ./discovery
 COPY tools ./tools
 COPY scraper ./scraper
 COPY peering ./peering
-COPY sentinel ./sentinel
 COPY runtime ./runtime
 RUN mkdir -p /library /data && chown node:node /data /library
-ENV PORT=8080 LIBRARY=/library DATA=/data TZ=Europe/Berlin
+ENV PORT=8080 LIBRARY=/library DATA=/data TZ=UTC
 EXPOSE 8080
 USER node
 HEALTHCHECK --interval=30s --timeout=5s --start-period=60s CMD node tools/healthcheck.js reader

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 maashkinoozhe
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -8,59 +8,53 @@ A no-frills, self-hosted manga / comic / webtoon reader.
 - Reader with three view modes (**Flow**, **Page**, **Book**), a page zoom that leaves the controls alone, and tap zones for mobile.
 - A separate **Greatest of All Time** manga-history dashboard, with five-year periods, per-user wish ratings, community ranking, and local-library badges. Its curated titles do not enter the download queue.
 
-## Run it
+## Run it with Docker
+
+The supported way to run MangaCircus is Docker Compose on a Linux host: one image, a **reader**
+service (the web app) and a **scraper** service (the managed download queue). Short version:
 
 ```bash
-node server.js
+git clone <repository-url> mangacircus && cd mangacircus
+mkdir -p library data/scraper data/peering data/peering-usage backup .secrets
+openssl rand -hex 32 > .secrets/scraper_token && chmod 600 .secrets/scraper_token
+cp .env.example .env                      # set LIBRARY_PATH etc.; see the guide
+sudo chown -R 1000:1000 library data backup
+docker compose build
+bash deploy/circus.sh up                  # reader first, then the scraper
 ```
 
-Open http://localhost:8080. The first account you create becomes the admin. Try it with demo content:
+Open http://127.0.0.1:8080. The first account you create becomes the admin. The full guide covers
+configuration, LAN and HTTPS access (a ready Caddy setup is included), adding manga, updates, backups,
+peering and troubleshooting: **[docs/deployment.md](docs/deployment.md)**.
+
+## Try it without Docker
+
+Needs Node.js 18 or newer, nothing else:
 
 ```bash
-node tools/make-dummy.js library
+node tools/make-dummy.js library      # five demo manga
+npm run start:all                     # reader on :8080 and the scraper daemon on 127.0.0.1:8091
 ```
 
-To run the reader, managed scraper and Sentinel together, use `npm run start:all`. Admins can
-then manage the persistent download queue, progress, pauses, retries, and rate limits
-under **Settings → Admin → Scraper**. The reader continues working if the scraper is stopped.
-**Settings → Admin → Collections & metadata** manages grouped tiles, reading order and relationships;
-**Settings → Admin → Service health** shows recovery incidents. See [collections](docs/collections.md)
-and [resilience](docs/resilience.md) for the ownership and agent-maintenance contracts.
-The current Windows installation uses Docker Compose; its old `MangaCircus Supervisor`
-Scheduled Task is disabled. Do not run a native duplicate writer. See
-[docs/local-deployment.md](docs/local-deployment.md) for its observed layout, health
-checks, restart procedure, and recovery point.
-Deployment-controlled reading hours and the optional progress-aware quiz are
-documented in [reading limiter](docs/reading-limiter.md). They are disabled
-unless configured on the reader container.
-See [Greatest of All Time](docs/greatest.md) for catalog ownership, voting, and metadata refresh.
-
-### Docker
-
-```bash
-node tools/prepare-compose.js
-docker compose config -q
-docker compose up -d --build
-```
-
-Stop existing native writers before using their data in containers. `docker-compose.yml`
-runs reader, scraper and Sentinel with explicit persistent bind mounts (`library/`,
-`data/`), file-backed secrets and only a loopback reader port. No named data volume is
-used. See [Compose deployment](docs/compose-deployment.md) for local development, VM
-recommendations, cutover, image releases and backup/rollback. The local Docker stack
-is verified; VM deployment is still pending.
+`npm start` runs the reader alone. Admins manage the persistent download queue, pauses, retries and rate
+limits under **Settings → Admin → Scraper**. The reader keeps working when the scraper is stopped.
+**Settings → Admin → Collections & metadata** manages grouped tiles, reading order and relationships, and
+**Settings → Admin → Service health** shows recovery incidents and backups ([collections](docs/collections.md),
+[resilience](docs/resilience.md)). Deployment-controlled reading hours and the optional progress-aware quiz
+are described in [reading limiter](docs/reading-limiter.md); they are off unless configured.
+See [Greatest of All Time](docs/greatest.md) for that dashboard's catalog and voting.
 
 ### Settings workspace
 
-Settings has **User** and **Admin** menu groups. User sections cover Reading,
-Reading time & privacy, Notifications and Account. Admin sections cover Scraper,
-Server & library, Users & access, Collections & metadata and Service health.
-Scraper has local Queue, Automatic updates and Hoster settings tabs. Only admins
-see administration; personal settings do not poll scraper services. See
-[settings navigation](docs/settings-navigation.md) for routes and verification.
+Settings has **User** and **Admin** menu groups. User sections cover Reading, Reading time & privacy,
+Notifications and Account. Admin sections cover Scraper, Server & library, Users & access, Collections &
+metadata and Service health. Only admins see administration ([settings navigation](docs/settings-navigation.md)).
 
 ### Settings (environment variables)
 
+These configure the reader process. In Docker most are set for you by `docker-compose.yml`; the variables
+you put in `.env` are listed in [docs/deployment.md](docs/deployment.md#4-configure-env).
+
 | Variable        | Default     | Meaning |
 |-----------------|-------------|---------|
 | `PORT`          | `8080`      | HTTP port |
@@ -69,13 +63,24 @@ see administration; personal settings do not poll scraper services. See
 | `SIGNUP`        | `closed`    | `open` lets anyone create an account. Otherwise the admin adds users under Settings |
 | `SCAN_INTERVAL` | `10`        | Minutes between shallow library change checks (`0` disables this timer) |
 | `FULL_SCAN_INTERVAL` | `1440` | Minutes between full library reconciliations (`0` disables this timer) |
+| `SCAN_CONCURRENCY` | `8` | Chapter folders and images probed at once by an uncached scan (1–64); higher helps on network storage |
 | `COOKIE_SECURE` | –           | `1` marks the login cookie `Secure` (use it when you serve over HTTPS) |
 | `SCRAPER_URL`   | `http://127.0.0.1:8091` | Private scraper daemon URL |
 | `SCRAPER_TOKEN` | empty       | Shared daemon token; required when it listens beyond loopback |
 | `SCRAPER_TOKEN_FILE` | empty | Alternative secret file (used in Compose) |
-| `SENTINEL_URL` | `http://127.0.0.1:8092` | Private observation service URL |
+| `BACKUP`        | –           | Existing folder for built-in backups of `DATA`; unset = off. Compose: set `BACKUP_PATH`. Schedule and retention under Settings → Service health ([backup](docs/backup.md)) |
 
-Serve it behind a reverse proxy with HTTPS if it is reachable from the internet. All pages use relative URLs, so a sub-path like `/manga/` works.
+Serve it behind a reverse proxy with HTTPS if it is reachable from the internet. All pages use relative
+URLs, so a sub-path like `/manga/` works.
+
+### Sources and copyright
+
+The scraper can download from MangaDex (on by default; official API) and from WeebCentral and MangaPill.
+The last two are unofficial sites that mostly host unlicensed copies, so they are **off by default**. An admin
+can switch each one on under **Settings → Admin → Scraper → Hoster settings**. Peering can copy chapters
+between servers. You are responsible for only downloading and sharing what you are allowed to copy where you live,
+and for respecting each site's terms. The scraper is deliberately slow and stops on access errors; it never
+bypasses CAPTCHAs, logins or other access controls ([scraper/AGENT_RUNBOOK.md](scraper/AGENT_RUNBOOK.md)).
 
 ---
 
@@ -100,22 +105,27 @@ library/
 
 ## Documentation
 
-Unnumbered MangaDex pilots/one-shots use stable `special-mangadex-<chapter-uuid>`
-chapter keys and human labels, not fabricated numbers. Existing numeric IDs remain
-unchanged. In Settings → Admin → Scraper, **Inspect & queue** shows the source edition
-before **Confirm download**. “Job finished” refers to its selected scope, not verified
-whole-series completeness. See the scraper service and library format contracts below.
-
 | Document | For |
 |----------|-----|
-| [docs/library-format.md](docs/library-format.md) | The folder format: naming rules, page tags, metadata fields, versions, extras, what survives a rename |
-| [docs/import.md](docs/import.md) | How the library is scanned and indexed, caching, change detection and notifications, troubleshooting |
-| [docs/architecture.md](docs/architecture.md) | Components, data model, API, security, the reader's layout engine, deployment, design decisions |
-| [docs/local-deployment.md](docs/local-deployment.md) | Current local Compose deployment, Tailscale access, persistent state, checks, restart, and recovery |
-| [docs/settings-navigation.md](docs/settings-navigation.md) | User/Admin settings sections, scraper tabs, routes and UI checks |
-| [docs/reading-insights-concept.md](docs/reading-insights-concept.md) | Active-time presentation, private summaries, deletion/export controls, and the future journal design |
+| [docs/deployment.md](docs/deployment.md) | Docker deployment: setup, configuration, HTTPS, operation (`deploy/circus.sh`), updates, backups, troubleshooting |
+| [docs/library-format.md](docs/library-format.md) | The folder format: naming rules, page tags, metadata fields, chapter dates, versions, extras, what survives a rename |
+| [docs/import.md](docs/import.md) | How the library is scanned and indexed, caching, change detection and notifications |
+| [docs/architecture.md](docs/architecture.md) | Components, data model, API, security, the reader's layout engine, design decisions |
+| [docs/scraper-service.md](docs/scraper-service.md) | The managed download queue, private API and adapter contract |
 | [docs/scraper-guide.md](docs/scraper-guide.md) | Rule book (agent-readable) for scrapers and other tools that write into the library |
-| [scraper/AGENT_RUNBOOK.md](scraper/AGENT_RUNBOOK.md) | Conservative scraper workflow and host-specific procedures for automated agents |
+| [scraper/AGENT_RUNBOOK.md](scraper/AGENT_RUNBOOK.md) | Conservative scraper workflow, rate limits and stop conditions for automated agents |
+| [docs/source-modules.md](docs/source-modules.md) | Adding a new source to the scraper |
+| [docs/peering.md](docs/peering.md) | Optional server-to-server sharing over mutual TLS |
+| [docs/backup.md](docs/backup.md) | Built-in scheduled backup of `data/`, what it contains, retention and restore |
+| [docs/settings-navigation.md](docs/settings-navigation.md) | User/Admin settings sections, scraper tabs and routes |
+| [docs/reading-insights-concept.md](docs/reading-insights-concept.md) | Active-time presentation, private summaries, export and deletion |
+| [docs/resilience.md](docs/resilience.md) | Supervisors, automatic recovery limits and incidents |
+| [AGENTS.md](AGENTS.md) | Working on the code: layout, tests, rules for contributors and coding agents |
+
+Unnumbered MangaDex pilots and one-shots use stable `special-mangadex-<chapter-uuid>` chapter keys and
+human labels, not invented numbers. In Settings → Admin → Scraper, **Inspect & queue** shows the source
+edition before **Confirm download**. "Job finished" refers to its selected scope, not verified whole-series
+completeness.
 
 ---
 
@@ -154,5 +164,12 @@ whole-series completeness. See the scraper service and library format contracts
 server.js         HTTP server, accounts, progress, votes, notifications (JSON file storage)
 library.js        folder scanner + image size reader
 public/           the web pages (library, manga, reader, settings, login)
-tools/            demo library generator (pure Node, draws PNGs itself)
+scraper/          download queue daemon, source adapters, publisher
+peering/          optional server-to-server sharing
+deploy/           Docker operations helper (circus.sh) and HTTPS with Caddy
+tools/            demo library generator, maintenance and test tools
 ```
+
+## License
+
+MIT, see [LICENSE](LICENSE). Manga you read or download with it are not covered by this license and belong to their rights holders.

backup.js

@@ -0,0 +1,402 @@
+'use strict';
+/*
+ * Built-in backup of the reader's DATA tree into BACKUP (see docs/backup.md).
+ *
+ * One run: pause the scraper queue through its control API, wait (bounded) until no
+ * job is active, write a gzip'd tar of DATA into BACKUP, resume the queue only if this
+ * run paused it, then apply retention. No container stops. db.json comes from the
+ * reader's in-memory state; scraper/queue.json is stored with `paused: true` so a
+ * restored copy never starts downloading by itself. Runtime/lock/pid/temp files are
+ * left out. Retention only ever deletes `circus-YYYYMMDD-HHMMSS.tar.gz` files directly
+ * inside BACKUP. No dependencies: the tar writer/reader below is plain ustar + PAX.
+ */
+const fs = require('fs'), fsp = fs.promises, path = require('path'), zlib = require('zlib');
+const { Readable } = require('stream'), { pipeline } = require('stream/promises');
+
+const ARCHIVE = /^circus-(\d{4})(\d{2})(\d{2})-(\d{2})(\d{2})(\d{2})\.tar\.gz$/;
+const PART = /^circus-\d{8}-\d{6}\.tar\.gz\.part$/;
+const EXCLUDE = [/^runtime.*\.json$/, /\.lock$/, /\.pid$/, /\.tmp$/];
+const DEFAULT_SETTINGS = { enabled: false, time: '03:30', retention: { mode: 'count', value: 14 } };
+const RETENTION_LIMITS = { count: [1, 365], days: [1, 3650] };
+const CATCH_UP_MS = 24 * 3600e3;
+
+class BackupError extends Error { constructor(code, msg) { super(msg); this.code = code; } }
+const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
+const pad = n => String(n).padStart(2, '0');
+const stampOf = d => `${d.getFullYear()}${pad(d.getMonth() + 1)}${pad(d.getDate())}-${pad(d.getHours())}${pad(d.getMinutes())}${pad(d.getSeconds())}`;
+const archiveTime = name => { const m = ARCHIVE.exec(name); return m ? new Date(+m[1], m[2] - 1, +m[3], +m[4], +m[5], +m[6]).getTime() : NaN; };
+const message = e => String((e && e.message) || e).slice(0, 500);
+
+function validateSettings(input, current = DEFAULT_SETTINGS) {
+  const next = { ...current, retention: { ...current.retention } };
+  if (input.enabled !== undefined) {
+    if (typeof input.enabled !== 'boolean') throw new BackupError(400, 'enabled must be true or false');
+    next.enabled = input.enabled;
+  }
+  if (input.time !== undefined) {
+    if (!/^([01]\d|2[0-3]):[0-5]\d$/.test(String(input.time))) throw new BackupError(400, 'The backup time must be HH:MM');
+    next.time = String(input.time);
+  }
+  if (input.retention !== undefined) {
+    const mode = input.retention && input.retention.mode, value = Number(input.retention && input.retention.value);
+    const limits = RETENTION_LIMITS[mode];
+    if (!limits) throw new BackupError(400, 'Retention is "count" or "days"');
+    if (!Number.isInteger(value) || value < limits[0] || value > limits[1]) throw new BackupError(400, `Keep ${limits[0]}–${limits[1]} ${mode === 'count' ? 'backups' : 'days'}`);
+    next.retention = { mode, value };
+  }
+  return next;
+}
+
+// Next local occurrence of HH:MM strictly after `now` (process TZ).
+function nextOccurrence(time, now) {
+  const [hh, mm] = time.split(':').map(Number), d = new Date(now);
+  const at = new Date(d.getFullYear(), d.getMonth(), d.getDate(), hh, mm, 0, 0);
+  if (at.getTime() <= now) at.setDate(at.getDate() + 1);
+  return at.getTime();
+}
+
+// ---------- tar (ustar + PAX path records), enough for `tar -xzf` everywhere ----------
+function octal(buf, offset, length, value) { buf.write(value.toString(8).padStart(length - 1, '0') + '\0', offset, length, 'ascii'); }
+function rawHeader(name, { size = 0, mode = 0o644, uid = 0, gid = 0, mtime = 0, type = '0' }) {
+  const b = Buffer.alloc(512);
+  b.write(name, 0, 100, 'utf8');
+  octal(b, 100, 8, mode & 0o7777); octal(b, 108, 8, uid & 0o7777777); octal(b, 116, 8, gid & 0o7777777);
+  octal(b, 124, 12, size); octal(b, 136, 12, Math.max(0, Math.floor(mtime / 1000)));
+  b.write('        ', 148, 8, 'ascii');
+  b.write(type, 156, 1, 'ascii');
+  b.write('ustar\0' + '00', 257, 8, 'ascii');
+  let sum = 0; for (const byte of b) sum += byte;
+  b.write(sum.toString(8).padStart(6, '0') + '\0 ', 148, 8, 'ascii');
+  return b;
+}
+const padding = size => Buffer.alloc((512 - (size % 512)) % 512);
+function paxRecord(key, value) {
+  const body = ` ${key}=${value}\n`, bytes = Buffer.byteLength(body);
+  let length = bytes + String(bytes).length;
+  if (String(length).length !== String(bytes).length) length = bytes + String(length).length;
+  return `${length}${body}`;
+}
+function tarEntry(name, data, meta) {
+  const parts = [];
+  // Names longer than the ustar field or non-ASCII go into a PAX header first.
+  if (Buffer.byteLength(name) > 100 || /[^\x20-\x7e]/.test(name)) {
+    const pax = Buffer.from(paxRecord('path', name), 'utf8');
+    parts.push(rawHeader('PaxHeader/' + name.replace(/[^\x20-\x7e]/g, '_').slice(-80), { size: pax.length, type: 'x', mtime: meta.mtime }), pax, padding(pax.length));
+    name = name.replace(/[^\x20-\x7e]/g, '_').slice(-100);
+  }
+  parts.push(rawHeader(name, { ...meta, size: data ? data.length : 0 }));
+  if (data && data.length) parts.push(data, padding(data.length));
+  return parts;
+}
+
+// Streaming reader for verification and tests: [{ name, type, size, mode, mtime, data? }].
+async function readArchive(file, { contents = false } = {}) {
+  const entries = []; let pending = Buffer.alloc(0), current = null, paxPath = null, ended = false;
+  const consume = chunk => {
+    pending = pending.length ? Buffer.concat([pending, chunk]) : chunk;
+    for (;;) {
+      if (ended) return;
+      if (current) {
+        if (pending.length < current.need) return;
+        const body = pending.subarray(0, current.size);
+        if (current.entry.type === 'x') paxPath = /(?:^|\n)\d+ path=([^\n]*)\n/.exec(body.toString('utf8'))?.[1] ?? null;
+        else { if (contents) current.entry.data = Buffer.from(body); entries.push(current.entry); }
+        pending = pending.subarray(current.need); current = null; continue;
+      }
+      if (pending.length < 512) return;
+      const h = pending.subarray(0, 512); pending = pending.subarray(512);
+      if (h.every(byte => byte === 0)) { ended = true; return; }
+      let sum = 0; for (let i = 0; i < 512; i++) sum += i >= 148 && i < 156 ? 32 : h[i];
+      if (sum !== parseInt(h.toString('ascii', 148, 156), 8)) throw new Error('Corrupt tar header');
+      const field = (o, l) => h.toString('utf8', o, o + l).replace(/\0.*$/s, '');
+      const size = parseInt(field(124, 12), 8) || 0, type = field(156, 1) || '0';
+      const prefix = field(345, 155), name = prefix ? prefix + '/' + field(0, 100) : field(0, 100);
+      const entry = { name: type === 'x' ? name : (paxPath ?? name), type, size, mode: parseInt(field(100, 8), 8), mtime: parseInt(field(136, 12), 8) * 1000 };
+      if (type !== 'x') paxPath = null;
+      current = { entry, size, need: size + padding(size).length };
+    }
+  };
+  await pipeline(fs.createReadStream(file), zlib.createGunzip(), async function* (source) { for await (const chunk of source) consume(chunk); });
+  if (!ended) throw new Error('Archive ends without the tar end marker');
+  return entries;
+}
+
+class BackupService {
+  /**
+   * dataRoot     DATA (read)            backupRoot  BACKUP (rw) or '' = feature off
+   * stateFile    settings + history     snapshotDb  () => db.json text from memory
+   * scraper      { state(), control(action) } or null
+   */
+  constructor({ dataRoot, backupRoot, stateFile, snapshotDb, scraper = null, now = Date.now, log = console,
+    idleTimeoutMs = 15 * 60e3, pollMs = 2000, resumeRetryMs = 5000 }) {
+    this.dataRoot = path.resolve(dataRoot);
+    this.backupRoot = backupRoot ? path.resolve(backupRoot) : '';
+    this.stateFile = stateFile || path.join(this.dataRoot, 'backup.json');
+    this.snapshotDb = snapshotDb; this.scraper = scraper; this.now = now; this.log = log;
+    this.idleTimeoutMs = idleTimeoutMs; this.pollMs = pollMs; this.resumeRetryMs = resumeRetryMs;
+    this.state = { version: 1, settings: { ...DEFAULT_SETTINGS }, nextAt: null, last: null, history: [], pending: null };
+    this.running = null; this.archives = []; this.targetError = null; this.timer = null;
+  }
+
+  load() {
+    try { Object.assign(this.state, JSON.parse(fs.readFileSync(this.stateFile, 'utf8'))); }
+    catch (e) { if (e.code !== 'ENOENT') this.log.warn(`[backup] ignoring unreadable ${this.stateFile}: ${e.message}`); }
+    this.state.settings = validateSettings(this.state.settings || {}, DEFAULT_SETTINGS);
+    this.state.history = Array.isArray(this.state.history) ? this.state.history.slice(-10) : [];
+    return this;
+  }
+
+  persist() {
+    fs.writeFileSync(this.stateFile + '.tmp', JSON.stringify(this.state, null, 2) + '\n');
+    fs.renameSync(this.stateFile + '.tmp', this.stateFile);
+  }
+
+  // A run interrupted by a restart may have left the queue paused on its behalf.
+  async recoverInterrupted() {
+    const pending = this.state.pending;
+    if (!pending) return;
+    this.state.pending = null;
+    const result = { at: pending.startedAt, trigger: pending.trigger, error: 'Interrupted by a reader restart', warnings: [] };
+    if (pending.pausedByUs) {
+      try { await this.resume(); result.warnings.push('Queue resumed after the interrupted backup'); }
+      catch (e) { result.warnings.push(`Queue left paused: ${message(e)}`); }
+    }
+    this.record(result);
+  }
+
+  async refreshTarget() {
+    if (!this.backupRoot) { this.targetError = null; this.archives = []; return; }
+    try {
+      const st = await fsp.stat(this.backupRoot);
+      if (!st.isDirectory()) throw new Error('not a directory');
+      // Probe with a real write: access(W_OK) is unreliable on NFS with squashed ids (Synology
+      // "map all users to admin" denies it although writes succeed).
+      const probe = path.join(this.backupRoot, `.circus-backup-probe-${process.pid}`);
+      await fsp.writeFile(probe, '');
+      await fsp.rm(probe, { force: true });
+      const names = (await fsp.readdir(this.backupRoot)).filter(name => ARCHIVE.test(name)).sort();
+      this.archives = await Promise.all(names.map(async name => ({ name, size: (await fsp.stat(path.join(this.backupRoot, name))).size, at: archiveTime(name) })));
+      this.targetError = null;
+    } catch (e) { this.targetError = `Backup folder ${this.backupRoot} is not usable: ${e.code || message(e)}`; this.archives = []; }
+  }
+
+  publicState() {
+    const s = this.state;
+    return {
+      configured: !!this.backupRoot, path: this.backupRoot || null, targetError: this.targetError,
+      settings: s.settings, nextAt: s.settings.enabled && this.backupRoot ? s.nextAt : null,
+      running: this.running && { ...this.running }, last: s.last, history: s.history.slice().reverse(),
+      archives: { count: this.archives.length, bytes: this.archives.reduce((n, a) => n + a.size, 0),
+        newest: this.archives.length ? this.archives[this.archives.length - 1].name : null },
+    };
+  }
+
+  updateSettings(input) {
+    this.state.settings = validateSettings(input || {}, this.state.settings);
+    this.state.nextAt = this.state.settings.enabled ? nextOccurrence(this.state.settings.time, this.now()) : null;
+    this.persist();
+    return this.publicState();
+  }
+
+  start(intervalMs = 30000) {
+    this.load();
+    this.refreshTarget().then(() => this.recoverInterrupted()).catch(e => this.log.error('[backup]', message(e)));
+    this.timer = setInterval(() => this.tick(), intervalMs);
+    this.timer.unref?.();
+    return this;
+  }
+
+  stop() { clearInterval(this.timer); }
+
+  // Scheduled runs: catch up a missed slot from the last day, never older ones
+  // (e.g. right after restoring an old backup.json).
+  async tick() {
+    const s = this.state.settings, now = this.now();
+    if (!s.enabled || !this.backupRoot || this.running) return null;
+    if (!this.state.nextAt) { this.state.nextAt = nextOccurrence(s.time, now); this.persist(); return null; }
+    if (now < this.state.nextAt) return null;
+    const due = now - this.state.nextAt < CATCH_UP_MS;
+    this.state.nextAt = nextOccurrence(s.time, now); this.persist();
+    return due ? this.run('schedule').catch(() => null) : null;
+  }
+
+  /** Starts a run and resolves with its result; rejects at once when no run is possible. */
+  run(trigger = 'manual') {
+    if (!this.backupRoot) return Promise.reject(new BackupError(409, 'Backups are off: set BACKUP to a mounted folder'));
+    if (this.running) return Promise.reject(new BackupError(409, 'A backup is already running'));
+    this.running = { startedAt: this.now(), trigger, phase: 'starting' };
+    return this.perform(trigger).finally(() => { this.running = null; });
+  }
+
+  async perform(trigger) {
+    const started = this.now(), result = { at: started, trigger, warnings: [] };
+    let part = null, pausedByUs = false;
+    try {
+      await this.refreshTarget();
+      if (this.targetError) throw new Error(this.targetError);
+      await this.removeStaleParts();
+      // 1. Pause the queue (unless already paused) and wait for the active chapter.
+      let scraperState = null;
+      if (this.scraper) {
+        try { scraperState = await this.scraper.state(); }
+        catch (e) { result.warnings.push(`Scraper unreachable, queue not paused: ${message(e)}`); }
+      }
+      if (scraperState) {
+        result.queueWasPaused = !!scraperState.paused;
+        if (!scraperState.paused) {
+          this.state.pending = { startedAt: started, trigger, pausedByUs: true }; this.persist();
+          await this.scraper.control('pause'); pausedByUs = true;
+        }
+        this.running.phase = 'waiting for the scraper to finish its current step';
+        await this.waitIdle(scraperState);
+      }
+      // 2. Snapshot DATA into BACKUP/<name>.part, then rename.
+      this.running.phase = 'writing archive';
+      let stamp = started, name, file;
+      do { name = `circus-${stampOf(new Date(stamp))}.tar.gz`; file = path.join(this.backupRoot, name); stamp += 1000; }
+      while (fs.existsSync(file)); // two runs within one second must not overwrite each other
+      part = file + '.part';
+      const stats = await this.writeArchive(part);
+      this.running.phase = 'verifying archive';
+      const entries = await readArchive(part);
+      const files = entries.filter(e => e.type === '0').length;
+      if (files !== stats.files) throw new Error(`Verification read ${files} files, expected ${stats.files}`);
+      await fsp.rename(part, file); part = null;
+      Object.assign(result, { file: name, size: (await fsp.stat(file)).size, files, dataBytes: stats.bytes });
+    } catch (e) {
+      result.error = message(e);
+      if (part) await fsp.rm(part, { force: true }).catch(() => {});
+    } finally {
+      // 3. Resume only what this run paused.
+      if (pausedByUs) {
+        this.running.phase = 'resuming queue';
+        try { await this.resume(); result.queueResumed = true; }
+        catch (e) { result.warnings.push(`Queue could not be resumed, resume it in Settings → Scraper: ${message(e)}`); }
+      }
+      if (this.state.pending) this.state.pending = null;
+    }
+    // 4. Retention only after a successful run, so failures never thin out old copies.
+    if (!result.error) {
+      this.running.phase = 'applying retention';
+      try { result.deleted = await this.applyRetention(result.file); }
+      catch (e) { result.warnings.push(`Retention failed: ${message(e)}`); }
+    }
+    result.durationMs = this.now() - started;
+    await this.refreshTarget();
+    this.record(result);
+    if (result.error) this.log.error(`[backup] failed: ${result.error}`);
+    else this.log.log(`[backup] ${result.file} ${result.size} bytes in ${result.durationMs} ms`);
+    return result;
+  }
+
+  record(result) {
+    this.state.last = result;
+    this.state.history = [...this.state.history, result].slice(-10);
+    this.persist();
+  }
+
+  async waitIdle(initial) {
+    const deadline = this.now() + this.idleTimeoutMs;
+    let state = initial, lastError = null;
+    for (;;) {
+      if (state && !(state.active || []).length && state.status !== 'pausing') return;
+      if (this.now() >= deadline) {
+        throw new Error(`The scraper did not become idle within ${Math.round(this.idleTimeoutMs / 60000)} min${lastError ? ` (${lastError})` : ''}; no backup written`);
+      }
+      await delay(this.pollMs);
+      try { state = await this.scraper.state(); lastError = null; }
+      catch (e) { state = null; lastError = message(e); }
+    }
+  }
+
+  async resume() {
+    let error;
+    for (let attempt = 0; attempt < 3; attempt++) {
+      try { await this.scraper.control('resume'); return; }
+      catch (e) { error = e; if (attempt < 2) await delay(this.resumeRetryMs); }
+    }
+    throw error;
+  }
+
+  // Files of DATA in a stable order, skipping excluded names, symlinks and a nested BACKUP.
+  async collect(dir = this.dataRoot, rel = '') {
+    const out = [];
+    const entries = (await fsp.readdir(dir, { withFileTypes: true })).sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
+    for (const entry of entries) {
+      const full = path.join(dir, entry.name), relative = rel ? rel + '/' + entry.name : entry.name;
+      if (entry.isDirectory()) {
+        if (full === this.backupRoot) continue;
+        out.push({ full, relative, dir: true });
+        out.push(...await this.collect(full, relative));
+      } else if (entry.isFile() && !EXCLUDE.some(re => re.test(entry.name))) out.push({ full, relative, dir: false });
+    }
+    return out;
+  }
+
+  // Contents of one file as it goes into the archive.
+  async contentOf(item) {
+    if (item.relative === 'db.json' && this.snapshotDb) return Buffer.from(this.snapshotDb());
+    if (item.relative === path.relative(this.dataRoot, this.stateFile).split(path.sep).join('/')) {
+      const { pending, ...rest } = this.state; void pending;
+      return Buffer.from(JSON.stringify(rest, null, 2) + '\n');
+    }
+    const data = await fsp.readFile(item.full);
+    if (item.relative === 'scraper/queue.json') {
+      const queue = JSON.parse(data.toString('utf8'));
+      queue.paused = true; queue.status = 'paused';
+      return Buffer.from(JSON.stringify(queue, null, 2) + '\n');
+    }
+    return data;
+  }
+
+  async writeArchive(part) {
+    const items = await this.collect(), stats = { files: 0, bytes: 0 };
+    // The in-memory db is archived even before its first debounced write.
+    if (this.snapshotDb && !items.some(item => item.relative === 'db.json')) items.unshift({ full: path.join(this.dataRoot, 'db.json'), relative: 'db.json', dir: false, memory: true });
+    const manifest = { kind: 'mangacircus-data-backup', version: 1, created: new Date(this.now()).toISOString(),
+      excluded: 'runtime*.json, *.lock, *.pid, *.tmp', queuePaused: true };
+    const self = this;
+    async function* tar() {
+      yield* tarEntry('.circus-backup.json', Buffer.from(JSON.stringify(manifest, null, 2) + '\n'), { mtime: self.now() });
+      for (const item of items) {
+        let st;
+        try { st = item.memory ? { mode: 0o644, uid: process.getuid?.() ?? 0, gid: process.getgid?.() ?? 0, mtimeMs: self.now() } : await fsp.stat(item.full); }
+        catch (e) { if (e.code === 'ENOENT') continue; throw e; }
+        const meta = { mode: st.mode, uid: st.uid, gid: st.gid, mtime: st.mtimeMs };
+        if (item.dir) { yield* tarEntry(item.relative + '/', null, { ...meta, type: '5' }); continue; }
+        let data;
+        try { data = await self.contentOf(item); } catch (e) { if (e.code === 'ENOENT') continue; throw new Error(`${item.relative}: ${message(e)}`); }
+        stats.files++; stats.bytes += data.length;
+        yield* tarEntry(item.relative, data, meta);
+      }
+      yield Buffer.alloc(1024);
+    }
+    await pipeline(Readable.from(tar()), zlib.createGzip(), fs.createWriteStream(part, { flags: 'wx' }));
+    const handle = await fsp.open(part, 'r+'); // fsync before the rename makes it visible
+    try { await handle.sync(); } finally { await handle.close(); }
+    stats.files++; // the manifest
+    return stats;
+  }
+
+  async removeStaleParts() {
+    for (const name of await fsp.readdir(this.backupRoot)) if (PART.test(name)) await fsp.rm(path.join(this.backupRoot, name), { force: true });
+  }
+
+  // Deletes only our own archive names directly inside BACKUP; never the newest one.
+  async applyRetention(keepName) {
+    const { mode, value } = this.state.settings.retention;
+    const names = (await fsp.readdir(this.backupRoot)).filter(name => ARCHIVE.test(name)).sort();
+    let doomed;
+    if (mode === 'count') doomed = names.slice(0, Math.max(0, names.length - value));
+    else { const cutoff = this.now() - value * 864e5; doomed = names.filter(name => archiveTime(name) < cutoff); }
+    doomed = doomed.filter(name => name !== keepName && name !== names[names.length - 1]);
+    for (const name of doomed) {
+      const target = path.join(this.backupRoot, name);
+      if (path.dirname(target) !== this.backupRoot) continue;
+      await fsp.rm(target);
+    }
+    return doomed.length;
+  }
+}
+
+module.exports = { BackupService, BackupError, readArchive, validateSettings, nextOccurrence, DEFAULT_SETTINGS, ARCHIVE };

chapter-dates.js

@@ -0,0 +1,109 @@
+'use strict';
+// Chapter dates (BL-0047). chapter.json keeps three independent dates:
+//   added           when the chapter entered this library (kept on replace)
+//   x-dates.source  when the source site uploaded it; batch=true marks a bulk import, not a release
+//   x-dates.release the official public release (filled by a metadata job; precision "day")
+//   x-dates.ecosystem  when it first entered any Manga Circus library; travels over peering, earliest wins
+// The reader shows one release date per chapter by tier: official > source (not batch) > ecosystem > library.
+
+const BATCH_SIZE = 3;          // this many uploads ...
+const BATCH_GAP_MS = 10 * 60e3; // ... each within 10 minutes of the next form a bulk import
+
+const iso = value => {
+  const time = typeof value === 'number' ? value : Date.parse(value);
+  return Number.isFinite(time) && time > 0 ? new Date(time).toISOString() : null;
+};
+const text = (value, max = 120) => (typeof value === 'string' ? value.trim().slice(0, max) : '');
+
+// Returns one flag per item: true when its upload time belongs to a bulk import, null without a time.
+function batchFlags(times) {
+  const order = times.map((value, index) => ({ index, time: Date.parse(value) }))
+    .filter(item => Number.isFinite(item.time)).sort((a, b) => a.time - b.time);
+  const flags = times.map(() => null);
+  for (let start = 0; start < order.length;) {
+    let end = start + 1;
+    while (end < order.length && order[end].time - order[end - 1].time <= BATCH_GAP_MS) end++;
+    for (let i = start; i < end; i++) flags[order[i].index] = end - start >= BATCH_SIZE;
+    start = end;
+  }
+  return flags;
+}
+
+function sourceDate(at, name, batch) {
+  const value = iso(at);
+  return value ? { at: value, name: text(name), batch: !!batch } : null;
+}
+
+function cleanSource(value) {
+  return value && typeof value === 'object' ? sourceDate(value.at, value.name, value.batch) : null;
+}
+function cleanRelease(value) {
+  if (!value || typeof value !== 'object' || !iso(value.at)) return null;
+  const precision = ['day', 'month', 'year'].includes(value.precision) ? value.precision : 'day';
+  return { at: String(value.at).slice(0, 30), precision, source: text(value.source) };
+}
+function cleanEcosystem(value) {
+  if (!value || typeof value !== 'object' || !iso(value.at)) return null;
+  return { at: iso(value.at), via: text(value.via) || 'this server' };
+}
+
+// chapter.json fields for a (re)published chapter. A prior "added" and an earlier ecosystem
+// date always survive; a peer's ecosystem claim may lower the date but never raise it.
+function publishedDates(priorMeta = {}, incoming = {}, now = Date.now()) {
+  const prior = priorMeta['x-dates'] && typeof priorMeta['x-dates'] === 'object' ? priorMeta['x-dates'] : {};
+  const added = iso(priorMeta.added) || iso(incoming.added) || iso(now);
+  const source = cleanSource(incoming.source) || cleanSource(prior.source);
+  const release = cleanRelease(incoming.release) || cleanRelease(prior.release);
+  let claim = cleanEcosystem(incoming.ecosystem);
+  if (claim) {
+    // Nobody can have had the chapter before the source uploaded it, or in the future.
+    const floor = source ? Date.parse(source.at) : 0, time = Math.min(Date.parse(claim.at), now);
+    claim = { ...claim, at: iso(Math.max(time, floor)) };
+  }
+  const candidates = [cleanEcosystem(prior.ecosystem), claim, { at: added, via: 'this server' }].filter(Boolean);
+  const ecosystem = candidates.reduce((best, item) => (Date.parse(item.at) < Date.parse(best.at) ? item : best));
+  return { added, 'x-dates': { version: 1, ...(source ? { source } : {}), ...(release ? { release } : {}), ecosystem } };
+}
+
+// Dates of one library chapter from its versions ({ added: ms, meta: chapter.json }).
+function chapterDates(versions) {
+  const known = {};
+  const keep = (slot, time, value) => {
+    if (Number.isFinite(time) && (!known[slot] || time < known[slot].time)) known[slot] = { time, ...value };
+  };
+  for (const version of versions) {
+    const dates = (version.meta && version.meta['x-dates']) || {};
+    const release = cleanRelease(dates.release), source = cleanSource(dates.source), ecosystem = cleanEcosystem(dates.ecosystem);
+    if (release) keep('official', Date.parse(release.at), { at: release.at, precision: release.precision, name: release.source });
+    if (source) keep(source.batch ? 'batch' : 'source', Date.parse(source.at), { at: source.at, name: source.name });
+    if (ecosystem) keep('ecosystem', Date.parse(ecosystem.at), { at: ecosystem.at, name: ecosystem.via });
+    if (Number.isFinite(version.added)) keep('library', version.added, { at: iso(version.added) });
+  }
+  const tier = ['official', 'source', 'ecosystem', 'library'].find(slot => known[slot]) || null;
+  const out = {};
+  for (const [slot, { time, ...value }] of Object.entries(known)) out[slot] = value;
+  return { tier, release: tier ? out[tier] : null, known: out };
+}
+
+// Ecosystem dates offered to peers: the earliest per chapter number, falling back to "added"
+// for chapters published before x-dates existed (this server was then the one that had it).
+function ecosystemOf(meta, fallbackMs) {
+  const dates = (meta && meta['x-dates']) || {};
+  return cleanEcosystem(dates.ecosystem)?.at || iso(meta && meta.added) || iso(fallbackMs);
+}
+// What a gateway sends with a chapter: only cleaned, non-identifying date fields.
+function peerDates(meta, fallbackMs) {
+  const dates = (meta && meta['x-dates']) || {};
+  const source = cleanSource(dates.source), release = cleanRelease(dates.release), ecosystem = ecosystemOf(meta, fallbackMs);
+  return { ...(ecosystem ? { ecosystem } : {}), ...(source ? { source } : {}), ...(release ? { release } : {}) };
+}
+// Incoming peer dates in publishedDates() shape.
+function fromPeer(value, alias) {
+  if (!value || typeof value !== 'object') return {};
+  const ecosystem = iso(value.ecosystem);
+  return { ...(ecosystem ? { ecosystem: { at: ecosystem, via: `peer ${text(alias, 40)}`.trim() } } : {}),
+    ...(cleanSource(value.source) ? { source: cleanSource(value.source) } : {}),
+    ...(cleanRelease(value.release) ? { release: cleanRelease(value.release) } : {}) };
+}
+
+module.exports = { BATCH_SIZE, BATCH_GAP_MS, batchFlags, sourceDate, publishedDates, chapterDates, ecosystemOf, peerDates, fromPeer };

compose.dev.yml

@@ -11,6 +11,3 @@ services:
   scraper:
     volumes:
       - ./scraper:/app/scraper:ro
-  sentinel:
-    volumes:
-      - ./sentinel:/app/sentinel:ro

deploy/Caddyfile

@@ -0,0 +1,5 @@
+# Used by deploy/compose.https.yml. Caddy obtains and renews the certificate on its own.
+{$MANGACIRCUS_DOMAIN} {
+	encode zstd gzip
+	reverse_proxy reader:8080
+}

deploy/circus.sh

@@ -0,0 +1,142 @@
+#!/bin/bash
+# Operate a MangaCircus Docker Compose deployment. Run it from anywhere; it works in the repository
+# folder that contains it. Needs Docker with Compose, nothing else on the host (no Node).
+# Extra Compose files (e.g. compose.lan.yml) come from COMPOSE_FILE in .env, like plain `docker compose`.
+#
+#   status                 services, reader health, scraper queue
+#   up                     reader first (waits for health), then the scraper; resumes the queue if `down` paused it
+#   down                   pause the queue, wait for the running chapter (<= 15 min), stop all services
+#   restart [service]      down + up, or restart one service
+#   logs [service] [lines] recent logs (default: all services, 100 lines)
+#   pause | resume         pause or resume the scraper queue
+#   update                 git pull --ff-only, rebuild the image, recreate safely (queue paused meanwhile)
+#   rescan                 make the reader re-read every chapter.json (full reconciliation) without an admin login
+#   restore-test           restore the newest built-in backup into a throwaway reader and check it starts
+#
+# Safety: never `docker compose down -v`. Only the scraper writes the library; only the reader writes users.
+set -euo pipefail
+cd "$(dirname "$(readlink -f "$0")")/.."
+log() { echo "[circus] $*"; }
+[ -f .env ] || { log "no .env here; see docs/deployment.md" >&2; exit 1; }
+env_value() { sed -n "s/^$1=//p" .env | tail -n 1 | sed -e 's/^["'\'']//' -e 's/["'\'']$//'; }
+running() { [ -n "$(docker compose ps -q --status running "$1" 2>/dev/null)" ]; }
+
+# Scraper control through the reader container: the token never leaves it and nothing secret is printed.
+control() {
+  docker compose exec -T -e ACTION="$1" reader node -e '
+const {secret}=require("./runtime/config");
+(async()=>{
+  const a=process.env.ACTION, state=a==="state";
+  const r=await fetch(process.env.SCRAPER_URL+(state?"/api/state":"/api/control"),{method:state?"GET":"POST",
+    signal:AbortSignal.timeout(20000),headers:{authorization:"Bearer "+secret("SCRAPER_TOKEN"),"content-type":"application/json"},
+    ...(state?{}:{body:JSON.stringify({action:a})})});
+  if(!r.ok)throw new Error("scraper HTTP "+r.status);
+  const s=await r.json();
+  console.log(JSON.stringify({status:s.status,paused:s.paused,sourcePaused:s.sourcePaused,active:(s.active||[]).length,
+    queue:(s.queue||[]).length,nextDiscoveryAt:s.nextDiscoveryAt,library:s.libraryStatus&&{state:s.libraryStatus.state,error:s.libraryStatus.error}}));
+})().catch(e=>{console.error(e.message);process.exitCode=1;});'
+}
+field() { sed -n "s/.*\"$1\":\([^,}]*\).*/\1/p"; }
+health() { docker compose exec -T reader node -e 'fetch("http://127.0.0.1:8080/health").then(r=>r.text()).then(t=>console.log(t))' 2>/dev/null; }
+wait_reader() {
+  for _ in $(seq 1 60); do health | grep -q '"ok":true' && return 0; sleep 5; done
+  log "reader not healthy after 5 minutes" >&2; return 1
+}
+# Pause the queue and wait for the running chapter; prints whether the queue was paused before.
+pause_wait() {
+  local was=true
+  if running scraper && running reader; then
+    was=$(control state | field paused)
+    [ "$was" = true ] || control pause >/dev/null
+    for _ in $(seq 1 90); do [ "$(control state | field active)" = 0 ] && break; sleep 10; done
+  fi
+  echo "$was"
+}
+resume_if() {
+  [ "$1" = false ] || { log "queue stays paused"; return 0; }
+  for _ in $(seq 1 30); do control resume >/dev/null 2>&1 && { log "queue resumed"; return 0; }; sleep 5; done
+  log "could not resume the queue" >&2; return 1
+}
+start_all() {
+  docker compose up -d reader
+  wait_reader
+  docker compose up -d --remove-orphans
+  sleep 15
+}
+
+cmd=${1:-status}; shift || true
+case "$cmd" in
+  status)
+    docker compose ps --format 'table {{.Service}}\t{{.Status}}\t{{.Ports}}'
+    running reader && { health || echo "reader: health check failed"; } || echo "reader: not running"
+    running scraper && running reader && control state || echo "scraper: not running"
+    ;;
+  up)
+    start_all
+    # Only a queue that `down` paused is resumed; without that marker the queue is left as it is.
+    if [ -f .queue-was-paused ]; then was=$(cat .queue-was-paused); rm -f .queue-was-paused; resume_if "$was"; fi
+    docker compose ps --format 'table {{.Service}}\t{{.Status}}'
+    ;;
+  down)
+    was=$(pause_wait); echo "$was" > .queue-was-paused
+    docker compose stop
+    log "stopped (queue was paused before: $was)"
+    ;;
+  restart)
+    if [ -n "${1:-}" ]; then docker compose restart "$1"; [ "$1" != reader ] || wait_reader
+    else "$0" down && "$0" up; fi
+    ;;
+  logs)
+    docker compose logs --no-color --tail "${2:-100}" ${1:+"$1"}
+    ;;
+  pause|resume)
+    control "$cmd"
+    ;;
+  update)
+    git pull --ff-only
+    VCS_REF=$(git rev-parse --short HEAD) docker compose build
+    was=$(pause_wait)
+    start_all            # recreates the containers whose image or configuration changed
+    resume_if "$was"
+    log "now running $(git log -1 --format='%h %s')"
+    ;;
+  rescan)
+    # The reader re-reads in-place chapter.json edits only on a full reconciliation. Marking the saved
+    # index as overdue makes it run one within a minute while it keeps serving the current index.
+    data=$(env_value DATA_PATH); data=${data:-./data}
+    image=$(docker compose images -q reader | head -n 1)
+    [ -f "$data/library-index.json" ] || { log "no saved index yet; the reader scans on its own" >&2; exit 1; }
+    docker compose stop reader
+    docker run --rm --user 1000:1000 -v "$(readlink -f "$data")":/data "$image" node -e '
+const fs=require("fs"),f="/data/library-index.json",j=JSON.parse(fs.readFileSync(f,"utf8"));
+j.fullAt=1; fs.writeFileSync(f+".tmp",JSON.stringify(j)); fs.renameSync(f+".tmp",f);'
+    docker compose up -d reader
+    wait_reader
+    log "full reconciliation scheduled; it logs \"[scan] full: …\" when done (docker compose logs reader)"
+    ;;
+  restore-test)
+    backup=$(env_value BACKUP_PATH); library=$(env_value LIBRARY_PATH); library=${library:-./library}
+    [ -n "$backup" ] || { log "BACKUP_PATH is not set in .env" >&2; exit 1; }
+    archive=$(ls -1t "$backup"/circus-*.tar.gz 2>/dev/null | head -n 1)
+    [ -n "$archive" ] || { log "no circus-*.tar.gz in $backup" >&2; exit 1; }
+    image=$(docker compose images -q reader | head -n 1)
+    tmp=$(mktemp -d); mkdir "$tmp/data"
+    trap 'docker rm -f circus-restore-test >/dev/null 2>&1 || true; rm -rf "$tmp"' EXIT
+    tar -C "$tmp/data" -xzf "$archive"
+    chmod -R a+rwX "$tmp/data"
+    log "archive: $archive"
+    docker run -d --name circus-restore-test --network none --user 1000:1000 --read-only --tmpfs /tmp \
+      -v "$(readlink -f "$library")":/library:ro -v "$tmp/data":/data -e DATA=/data -e LIBRARY=/library \
+      -e SIGNUP=closed -e SCAN_INTERVAL=0 "$image" node server.js >/dev/null
+    out=""
+    for _ in $(seq 1 30); do
+      out=$(docker exec circus-restore-test node -e 'fetch("http://127.0.0.1:8080/api/auth").then(r=>r.text()).then(console.log)' 2>/dev/null || true)
+      [ -n "$out" ] && break; sleep 2
+    done
+    echo "auth: $out"
+    echo "$out" | grep -q '"firstRun":false' && echo "RESTORE TEST OK" || { echo "RESTORE TEST FAILED" >&2; exit 1; }
+    ;;
+  *)
+    sed -n '2,17p' "$0"; exit 2
+    ;;
+esac

deploy/compose.https.yml

@@ -0,0 +1,27 @@
+# HTTPS in front of the reader with Caddy and automatic Let's Encrypt certificates.
+# Needs a DNS name for this host (MANGACIRCUS_DOMAIN in .env) and ports 80 and 443 reachable
+# from the internet. Use it with the main file, e.g. in .env:
+#   COMPOSE_FILE=docker-compose.yml:deploy/compose.https.yml
+# Already running a reverse proxy? Skip this file and see docs/deployment.md, "Your own proxy".
+services:
+  reader:
+    ports: !reset []
+    environment:
+      COOKIE_SECURE: "1"
+
+  caddy:
+    image: caddy:2.8.4-alpine
+    restart: unless-stopped
+    ports: ["80:80", "443:443", "443:443/udp"]
+    environment:
+      MANGACIRCUS_DOMAIN: ${MANGACIRCUS_DOMAIN:?Set MANGACIRCUS_DOMAIN in .env}
+    volumes:
+      - ./deploy/Caddyfile:/etc/caddy/Caddyfile:ro
+      - caddy-data:/data
+      - caddy-config:/config
+    networks: [egress]
+    depends_on: [reader]
+
+volumes:
+  caddy-data: {}
+  caddy-config: {}

discovery/anchors.js

@@ -0,0 +1,65 @@
+'use strict';
+
+// Anchors are the titles a user has shown taste for; their tags steer the wider AniList pool.
+const ANCHOR_WEIGHTS = { flame: 5, good: 3, interested: 4, maybe: 2, read: 3, owned: 1 };
+const MAX_ANCHORS = 15;
+const MIN_TAG_RANK = 60;
+const MAX_TOP_TAGS = 3;
+
+function anchorsFor({ user, db, library, greatest }) {
+  if (!user?.settings?.discoveryEnabled) return [];
+  const byId = new Map();
+  const add = (anilistId, source, title) => {
+    if (!Number.isSafeInteger(anilistId) || anilistId <= 0) return;
+    const weight = ANCHOR_WEIGHTS[source];
+    const prior = byId.get(anilistId);
+    if (!prior || weight > prior.weight) byId.set(anilistId, { anilistId, weight, source, title: title ?? null });
+  };
+
+  const ratings = db?.greatestRatings?.[user.name] || {};
+  for (const entry of greatest?.entries || []) {
+    const vote = ratings[entry.id];
+    if (vote === 'flame' || vote === 'good') add(Number(entry.metadata?.id), vote, entry.title);
+  }
+
+  const feedback = db?.discoveryFeedback?.[user.name] || {};
+  for (const [key, value] of Object.entries(feedback)) {
+    const match = /^anilist:(\d+)$/.exec(key);
+    if (match && (value === 'interested' || value === 'maybe')) add(Number(match[1]), value, null);
+  }
+
+  const progress = db?.progress?.[user.name] || {};
+  const reading = Boolean(user.settings.discoveryReadingActivity);
+  for (const manga of library || []) {
+    const id = Number(manga.meta?.['x-anilist']?.id);
+    if (!id) continue;
+    const chapters = progress[manga.id];
+    const done = reading && chapters && Object.values(chapters).some(chapter => chapter?.done);
+    add(id, done ? 'read' : 'owned', manga.title);
+  }
+
+  return [...byId.values()]
+    .sort((a, b) => b.weight - a.weight || a.anilistId - b.anilistId)
+    .slice(0, MAX_ANCHORS);
+}
+
+function topTagsFor(anchors, recordOf, { mature = false } = {}) {
+  const scores = new Map();
+  for (const anchor of anchors) {
+    const record = recordOf(anchor.anilistId);
+    for (const tag of Array.isArray(record?.tags) ? record.tags : []) {
+      // A tag without a rank is no evidence of taste, so it must not pass the rank filter.
+      if (!tag || typeof tag !== 'object' || typeof tag.name !== 'string') continue;
+      if (!(tag.rank >= MIN_TAG_RANK) || (tag.isAdult && !mature)) continue;
+      const entry = scores.get(tag.name) || { name: tag.name, kind: tag.kind, score: 0 };
+      entry.score += anchor.weight;
+      scores.set(tag.name, entry);
+    }
+  }
+  return [...scores.values()]
+    .sort((a, b) => b.score - a.score || (a.name < b.name ? -1 : a.name > b.name ? 1 : 0))
+    .slice(0, MAX_TOP_TAGS)
+    .map(({ name, kind }) => ({ name, kind }));
+}
+
+module.exports = { ANCHOR_WEIGHTS, MAX_ANCHORS, anchorsFor, topTagsFor };

discovery/anilist.js

@@ -0,0 +1,165 @@
+'use strict';
+
+// Polite AniList GraphQL client for discovery seeds. Sequential, rate limited, stops on access controls.
+
+const STAFF_ROLE = /^(?:original story|story(?: & art)?|art|illustration)$/i;
+const USER_AGENT = 'MangaCircus-Discovery/1.0';
+
+// Stop: the whole run should end. Skip: this one request is lost, carry on.
+class AniListStop extends Error {
+  constructor(reason, message) { super(message || `AniList stop: ${reason}`); this.name = 'AniListStop'; this.reason = reason; }
+}
+class AniListSkip extends Error {
+  constructor(reason, message) { super(message || `AniList skip: ${reason}`); this.name = 'AniListSkip'; this.reason = reason; }
+}
+
+function createLimiter({ perMinute = 25, now = Date.now, sleep = ms => new Promise(resolve => setTimeout(resolve, ms)) } = {}) {
+  const interval = 60000 / perMinute;
+  let nextAllowed = 0;
+  let tail = Promise.resolve();
+  // Chained so concurrent callers queue up instead of all reading the same clock.
+  function wait() {
+    const turn = tail.then(async () => {
+      await sleep(Math.max(0, nextAllowed - now()));
+      nextAllowed = now() + interval;
+    });
+    tail = turn.catch(() => {});
+    return turn;
+  }
+  return { wait };
+}
+
+const invalid = message => new AniListSkip('invalid', `AniList response is malformed: ${message}`);
+const isObject = value => !!value && typeof value === 'object' && !Array.isArray(value);
+const optionalArray = (value, what) => { if (value != null && !Array.isArray(value)) throw invalid(what); };
+
+// Malformed input is a Skip('invalid'), never a plain TypeError from deep inside the mapping.
+function assertShape(media) {
+  if (media.title != null && !isObject(media.title)) throw invalid('title');
+  for (const key of ['synonyms', 'genres', 'tags']) optionalArray(media[key], key);
+  if (media.staff != null && !isObject(media.staff)) throw invalid('staff');
+  optionalArray(media.staff?.edges, 'staff edges');
+  if ((media.tags || []).some(tag => !isObject(tag))) throw invalid('tags');
+  if ((media.staff?.edges || []).some(edge => !isObject(edge) || (edge.node != null && !isObject(edge.node)))) throw invalid('staff edges');
+}
+
+function normalizeMedia(media) {
+  if (!isObject(media) || !Number.isInteger(media.id)) throw invalid('media without integer id');
+  if (media.format !== 'MANGA' && media.format !== 'ONE_SHOT') return null;
+  assertShape(media);
+  const title = media.title || {};
+  const names = [title.english || title.romaji || title.native, title.romaji, title.native, ...(media.synonyms || [])].filter(Boolean);
+  return {
+    anilistId: media.id,
+    title: names[0] || '',
+    altTitles: [...new Set(names.slice(1))].filter(name => name !== names[0]),
+    format: media.format,
+    isAdult: !!media.isAdult,
+    status: media.status === 'RELEASING' ? 'ongoing' : media.status === 'FINISHED' ? 'completed' : 'unknown',
+    year: media.startDate?.year ?? null,
+    averageScore: media.averageScore ?? null,
+    popularity: media.popularity ?? null,
+    trending: media.trending ?? null,
+    tags: [
+      ...(media.genres || []).map(name => ({ name, rank: 100, kind: 'genre', isAdult: false })),
+      ...(media.tags || []).map(tag => ({ name: tag.name, rank: tag.rank, kind: 'tag', isAdult: !!tag.isAdult }))],
+    authors: (media.staff?.edges || [])
+      // An author without an integer id could never match a creator seed.
+      .filter(edge => STAFF_ROLE.test(edge?.role || '') && edge.node?.name?.full && Number.isInteger(edge.node.id))
+      .map(edge => ({ staffId: edge.node.id, name: edge.node.name.full, role: edge.role })),
+    coverUrl: media.coverImage?.extraLarge || null,
+    siteUrl: media.siteUrl || null,
+  };
+}
+
+const MEDIA_FIELDS = `fragment MediaFields on Media {
+  id format isAdult title { romaji english native } synonyms status startDate { year }
+  averageScore popularity trending genres tags { name rank isAdult }
+  staff(perPage: 20) { edges { role node { id name { full } } } }
+  coverImage { extraLarge } siteUrl
+}`;
+
+const QUERIES = {
+  title: `query ($id: Int) { Media(id: $id, type: MANGA) { ...MediaFields } } ${MEDIA_FIELDS}`,
+  recs: `query ($id: Int) { Media(id: $id, type: MANGA) {
+    recommendations(perPage: 25, sort: RATING_DESC) { nodes { rating mediaRecommendation { ...MediaFields } } } } } ${MEDIA_FIELDS}`,
+  creator: `query ($id: Int) { Staff(id: $id) {
+    staffMedia(type: MANGA, perPage: 25, sort: POPULARITY_DESC) { edges { staffRole node { ...MediaFields } } } } } ${MEDIA_FIELDS}`,
+  tagTopGenre: `query ($genre_in: [String]) { Page(perPage: 25) {
+    media(type: MANGA, genre_in: $genre_in, popularity_greater: 1000, sort: SCORE_DESC) { ...MediaFields } } } ${MEDIA_FIELDS}`,
+  tagTopTag: `query ($tag_in: [String]) { Page(perPage: 25) {
+    media(type: MANGA, tag_in: $tag_in, popularity_greater: 1000, sort: SCORE_DESC) { ...MediaFields } } } ${MEDIA_FIELDS}`,
+  fresh: `query ($startDate_greater: FuzzyDateInt, $perPage: Int) { Page(perPage: $perPage) {
+    media(type: MANGA, status: RELEASING, startDate_greater: $startDate_greater, sort: TRENDING_DESC) { ...MediaFields } } } ${MEDIA_FIELDS}`,
+};
+
+const isTimeout = error => error && (error.name === 'TimeoutError' || error.name === 'AbortError');
+const need = (value, check = Array.isArray) => { if (!check(value)) throw new AniListSkip('invalid', 'AniList response is missing data'); return value; };
+
+function createClient({ endpoint = process.env.DISCOVERY_ANILIST_URL || 'https://graphql.anilist.co', limiter = createLimiter(),
+  fetchImpl = fetch, timeoutMs = 30000, today = () => new Date() } = {}) {
+  async function post(query, variables) {
+    await limiter.wait();
+    let response, text;
+    try {
+      response = await fetchImpl(endpoint, {
+        method: 'POST', headers: { 'content-type': 'application/json', 'user-agent': USER_AGENT },
+        body: JSON.stringify({ query, variables }), signal: AbortSignal.timeout(timeoutMs),
+        redirect: 'error' }); // never follow a redirect to another host
+      if (response.status === 429) throw new AniListStop('rate-limited');
+      if (response.status === 401 || response.status === 403) throw new AniListStop('forbidden');
+      text = await response.text();
+    } catch (error) {
+      if (error instanceof AniListStop || error instanceof AniListSkip) throw error;
+      if (isTimeout(error)) throw new AniListSkip('timeout');
+      throw new AniListSkip('upstream', `AniList request failed: ${error.message}`);
+    }
+    // An HTML or non-JSON page means a bot-check in front of the API, even on a 5xx; never work around it.
+    if (/text\/html/i.test(response.headers.get('content-type') || '')) throw new AniListStop('challenge');
+    let body;
+    try { body = JSON.parse(text); } catch { throw new AniListStop('challenge'); }
+    if (response.status >= 500) throw new AniListSkip('upstream', `AniList HTTP ${response.status}`);
+    if (!response.ok || !body || typeof body !== 'object') throw new AniListSkip('invalid', `AniList HTTP ${response.status}`);
+    return body.data || {};
+  }
+
+  // Every item is normalized (and so validated) before any is dropped: one malformed item invalidates the response.
+  const mediaOf = data => need(data.Media, isObject);
+  const entries = (list, pick) => need(list).map(item => {
+    if (!isObject(item)) throw invalid('list item');
+    const media = pick(item);
+    return { item, media, record: normalizeMedia(media) };
+  }).filter(entry => entry.record);
+
+  return {
+    async title(id) {
+      return normalizeMedia(mediaOf(await post(QUERIES.title, { id })));
+    },
+    async recs(id) {
+      const nodes = mediaOf(await post(QUERIES.recs, { id })).recommendations?.nodes;
+      // AniList returns a null mediaRecommendation when the recommended media was removed: valid, just drop it.
+      const present = need(nodes).filter(node => !isObject(node) || node.mediaRecommendation !== null);
+      return entries(present, node => node.mediaRecommendation).filter(({ item }) => item.rating > 0)
+        .map(({ item, record }) => ({ anilistId: record.anilistId, rating: item.rating, record }));
+    },
+    async creator(staffId) {
+      const data = await post(QUERIES.creator, { id: staffId });
+      return entries(need(data.Staff, isObject).staffMedia?.edges, edge => edge.node)
+        .map(({ item, record }) => ({ anilistId: record.anilistId, role: item.staffRole, record }));
+    },
+    async tagTop({ name, kind }) {
+      const genre = kind === 'genre';
+      const data = await post(genre ? QUERIES.tagTopGenre : QUERIES.tagTopTag, genre ? { genre_in: [name] } : { tag_in: [name] });
+      return entries(data.Page?.media, media => media).map(({ record }) => ({ anilistId: record.anilistId, record }));
+    },
+    async fresh() {
+      const date = today();
+      // AniList FuzzyDateInt: YYYYMMDD, two years back.
+      const startDate = (date.getUTCFullYear() - 2) * 10000 + (date.getUTCMonth() + 1) * 100 + date.getUTCDate();
+      const data = await post(QUERIES.fresh, { startDate_greater: startDate, perPage: 50 });
+      return entries(data.Page?.media, media => media).map(({ record }) => ({ anilistId: record.anilistId, record }));
+    },
+  };
+}
+
+module.exports = { createLimiter, createClient, normalizeMedia, AniListStop, AniListSkip };

discovery/cache.js

@@ -0,0 +1,93 @@
+'use strict';
+
+const fs = require('node:fs');
+const path = require('node:path');
+const crypto = require('node:crypto');
+
+const DAY = 86400000;
+const KINDS = ['title', 'recs', 'creator', 'tagTop', 'fresh'];
+const TTL_MS = { title: 30 * DAY, recs: 30 * DAY, creator: 30 * DAY, tagTop: 7 * DAY, fresh: 7 * DAY };
+
+function checkKind(kind) {
+  if (!KINDS.includes(kind)) throw new TypeError(`Unknown discovery cache kind: ${kind}`);
+}
+
+class DiscoveryCache {
+  constructor(dir, { maxFiles = 5000, now = Date.now } = {}) {
+    this.dir = path.resolve(dir);
+    this.maxFiles = maxFiles;
+    this.now = now;
+  }
+  // base64url keeps any key (slashes, dots, spaces) inside the kind folder.
+  fileFor(kind, key) {
+    checkKind(kind);
+    return path.join(this.dir, kind, `${Buffer.from(String(key)).toString('base64url')}.json`);
+  }
+  readFile(file, kind) {
+    let entry;
+    try { entry = JSON.parse(fs.readFileSync(file, 'utf8')); }
+    catch (error) {
+      if (error.code === 'ENOENT') return null;
+      entry = null;
+    }
+    const valid = entry && entry.schema === 1 && entry.kind === kind && typeof entry.fetchedAt === 'string'
+      && Number.isFinite(Date.parse(entry.fetchedAt)) && 'data' in entry;
+    if (valid) return entry;
+    // A bad file would otherwise be re-read on every lookup; drop it so the caller refetches.
+    try { fs.rmSync(file, { force: true }); } catch {}
+    return null;
+  }
+  get(kind, key) { return this.readFile(this.fileFor(kind, key), kind); }
+  isFresh(kind, key) {
+    const entry = this.get(kind, key);
+    return !!entry && this.now() - Date.parse(entry.fetchedAt) < TTL_MS[kind];
+  }
+  put(kind, key, data) {
+    const file = this.fileFor(kind, key);
+    const entry = { schema: 1, kind, key, fetchedAt: new Date(this.now()).toISOString(), data };
+    fs.mkdirSync(path.dirname(file), { recursive: true });
+    const temp = `${file}.${crypto.randomBytes(6).toString('hex')}.tmp`;
+    try {
+      fs.writeFileSync(temp, JSON.stringify(entry), { mode: 0o600 });
+      fs.renameSync(temp, file);
+    } finally { try { fs.rmSync(temp, { force: true }); } catch {} }
+    this.prune();
+    return entry;
+  }
+  files(kind) {
+    try {
+      return fs.readdirSync(path.join(this.dir, kind)).filter((name) => name.endsWith('.json'))
+        .map((name) => path.join(this.dir, kind, name));
+    } catch { return []; }
+  }
+  list(kind) {
+    checkKind(kind);
+    return this.files(kind).map((file) => this.readFile(file, kind)).filter(Boolean);
+  }
+  stats() {
+    let files = 0, bytes = 0;
+    for (const kind of KINDS) for (const file of this.files(kind)) {
+      try { bytes += fs.statSync(file).size; files++; } catch {}
+    }
+    return { files, bytes };
+  }
+  // Oldest entries go first so the cache never grows past maxFiles. Every put() calls this, so
+  // under the cap it only counts directory entries; files are read only when something must go.
+  prune() {
+    const byKind = KINDS.map(kind => [kind, this.files(kind)]);
+    if (byKind.reduce((sum, [, files]) => sum + files.length, 0) <= this.maxFiles) return;
+    const all = [];
+    for (const [kind, files] of byKind) for (const file of files) {
+      // readFile drops corrupt and schema-mismatched files itself.
+      const entry = this.readFile(file, kind);
+      if (entry) all.push({ file, at: Date.parse(entry.fetchedAt) });
+    }
+    if (all.length <= this.maxFiles) return;
+    all.sort((a, b) => a.at - b.at);
+    for (const { file } of all.slice(0, all.length - this.maxFiles)) {
+      try { fs.rmSync(file, { force: true }); } catch {}
+    }
+  }
+}
+
+module.exports = { KINDS, TTL_MS, DiscoveryCache };

discovery/cli.js

@@ -5,6 +5,7 @@ const fs = require('node:fs');
 const path = require('node:path');
 const { loadCatalog } = require('../greatest');
 const { generate } = require('./engine');
+const { DiscoveryCache } = require('./cache');
 
 if (!process.argv.includes('--dry-run')) {
   console.error('Usage: node discovery/cli.js run --dry-run [--data DIR] [--catalog FILE] [--show]');
@@ -17,8 +18,12 @@ let library = [];
 try { library = JSON.parse(fs.readFileSync(path.join(data, 'library-index.json'), 'utf8')).mangas || []; }
 catch (error) { if (error.code !== 'ENOENT') throw error; }
 const greatest = loadCatalog(arg('--catalog', path.join(__dirname, '..', 'greatest', 'catalog.json')));
-const result = generate({ db, library, greatest, cacheDir: path.join(data, 'scraper', 'metadata', 'anilist') });
-const summary = { examined: result.examined, candidates: Object.keys(result.candidates).length,
+// The harvest cache is only read here (get/list, never put); note get/list still drop a corrupt entry file.
+const harvestDir = path.join(data, 'discovery-cache');
+const harvestCache = fs.existsSync(harvestDir) ? new DiscoveryCache(harvestDir) : null;
+const result = generate({ db, library, greatest, cacheDir: path.join(data, 'scraper', 'metadata', 'anilist'), harvestCache });
+// candidates: the ones some feed shows (what a run would publish); examined/clipped: the whole pool.
+const summary = { examined: result.examined, clipped: result.clipped, candidates: Object.keys(result.candidates).length,
   optedInUsers: Object.values(db.users || {}).filter(u => u.settings?.discoveryEnabled).length,
   feedSizes: Object.values(result.feeds).filter(f => !f.disabled).map(f => f.items.length) };
 if (process.argv.includes('--show')) summary.titles = Object.values(result.feeds).find(f => !f.disabled)?.items

discovery/engine.js

@@ -3,12 +3,18 @@
 const fs = require('node:fs');
 const path = require('node:path');
 const { normalize } = require('../greatest');
+const { anchorsFor } = require('./anchors');
+const { MAX_FEED, SLOTS, round2, seedIndexFor, seedNotes, groupFor, partRank, mixSlots } = require('./scoring');
 
-const MAX_CANDIDATES = 50;
-const MAX_FEED = 20;
+const MAX_CANDIDATES = 2000;
+const SEED_KINDS = ['recs', 'creator', 'tagTop', 'fresh'];
+const MATURE_TAG_RANK = 60;
 const VALID_FORMATS = new Set(['MANGA', 'ONE_SHOT']);
 const safeUrl = value => /^https:\/\/[^\s]+$/i.test(String(value || '')) ? String(value) : null;
 const unique = values => [...new Set(values.filter(Boolean))];
+// manga.json and cached metadata are hand-editable files: a field that is not a list counts as empty.
+const listOf = value => Array.isArray(value) ? value : [];
+const stringsOf = value => unique(listOf(value).filter(item => typeof item === 'string'));
 
 function cachedMetadata(dir, id) {
   if (!Number.isSafeInteger(Number(id)) || Number(id) < 1) return null;
@@ -26,7 +32,7 @@ function candidateFromGreatest(entry, cacheDir) {
     type: cached?.['x-anilist']?.format || 'MANGA', status: cached?.status || entry.status || 'unknown',
     cover: safeUrl(cached?.['x-anilist']?.coverUrl) || safeUrl(entry.cover),
     metadataUrl: safeUrl(cached?.['x-anilist']?.url) || safeUrl(entry.metadata?.url) || null,
-    tags: unique(cached?.tags || []), authors: unique(cached?.authors || []),
+    tags: stringsOf(cached?.tags), authors: stringsOf(cached?.authors),
     period: entry.period, editorialRank: entry.rank, greatestId: entry.id,
     evidence: [{ id: `greatest:${entry.id}`, kind: 'curation', label: 'Greatest of All Time selection',
       note: String(entry.note || '').slice(0, 600), url: safeUrl(entry.source?.url) || safeUrl(entry.metadata?.url) }],
@@ -34,7 +40,8 @@ function candidateFromGreatest(entry, cacheDir) {
   };
 }
 function addRelations(map, anchor, cacheDir) {
-  for (const relation of anchor.relations || []) {
+  for (const relation of listOf(anchor.relations)) {
+    if (!relation || typeof relation !== 'object') continue;
     if (!VALID_FORMATS.has(relation.format) || !Number.isSafeInteger(Number(relation.id))) continue;
     const id = `anilist:${Number(relation.id)}`;
     if (map.has(id)) continue;
@@ -44,16 +51,51 @@ function addRelations(map, anchor, cacheDir) {
       year: cached?.year || null, type: relation.format, status: cached?.status || 'unknown',
       cover: safeUrl(cached?.['x-anilist']?.coverUrl),
       metadataUrl: safeUrl(cached?.['x-anilist']?.url) || safeUrl(relation.url) || `https://anilist.co/manga/${relation.id}`,
-      tags: unique(cached?.tags || []), authors: unique(cached?.authors || []), period: null,
+      tags: stringsOf(cached?.tags), authors: stringsOf(cached?.authors), period: null,
       editorialRank: null, greatestId: null,
       evidence: [{ id: `relation:${anchor.id}:${id}`, kind: 'relation', label: `${relation.type || 'RELATED'} to ${anchor.title}`,
         note: '', url: safeUrl(relation.url) || `https://anilist.co/manga/${relation.id}` }],
-      relatedTo: { id: anchor.id, title: anchor.title, type: relation.type || 'RELATED' },
+      relatedTo: { id: anchor.id, title: anchor.title, type: String(relation.type || 'RELATED') },
       relations: cached?.['x-anilist']?.relations || [],
     });
   }
 }
-function buildCandidates({ greatest, library, cacheDir }) {
+const textList = values => unique(Array.isArray(values) ? values.map(value => value?.name).filter(name => typeof name === 'string') : []);
+const recordList = values => Array.isArray(values) ? values.filter(value => value && typeof value === 'object') : [];
+function candidateFromRecord(record, id, seed, fetchedAt) {
+  return {
+    id: `anilist:${id}`, title: String(record.title || `AniList ${id}`), year: Number(record.year) || null,
+    type: record.format || 'MANGA', status: record.status || 'unknown', cover: safeUrl(record.coverUrl),
+    metadataUrl: safeUrl(record.siteUrl) || `https://anilist.co/manga/${id}`,
+    tags: textList(record.tags), authors: textList(record.authors), period: null, editorialRank: null, greatestId: null,
+    evidence: [], relations: [], isAdult: record.isAdult === true,
+    averageScore: Number.isFinite(record.averageScore) ? record.averageScore : null,
+    tagRecords: recordList(record.tags), authorRecords: recordList(record.authors), fetchedAt, seed,
+  };
+}
+// Greatest and relation candidates keep their own fields; AniList only fills what they lack.
+function fillFromRecord(candidate, record, fetchedAt) {
+  if (candidate.isAdult === undefined) candidate.isAdult = record.isAdult === true;
+  if (!candidate.tagRecords) candidate.tagRecords = recordList(record.tags);
+  if (!candidate.authorRecords) candidate.authorRecords = recordList(record.authors);
+  if (candidate.averageScore == null && Number.isFinite(record.averageScore)) candidate.averageScore = record.averageScore;
+  if (!candidate.fetchedAt) candidate.fetchedAt = fetchedAt;
+}
+function addHarvested(map, harvestCache) {
+  for (const kind of SEED_KINDS) {
+    const entries = harvestCache.list(kind).sort((a, b) => String(a.key).localeCompare(String(b.key)));
+    for (const entry of entries) for (const item of Array.isArray(entry.data) ? entry.data : []) {
+      const record = item?.record;
+      const id = Number(item?.anilistId);
+      if (!record || typeof record !== 'object' || !Number.isSafeInteger(id) || id < 1) continue;
+      if (record.format && !VALID_FORMATS.has(record.format)) continue;
+      const existing = map.get(`anilist:${id}`);
+      if (existing) fillFromRecord(existing, record, entry.fetchedAt);
+      else map.set(`anilist:${id}`, candidateFromRecord(record, id, kind, entry.fetchedAt));
+    }
+  }
+}
+function buildCandidates({ greatest, library, cacheDir, harvestCache = null }) {
   const map = new Map();
   for (const entry of greatest.entries) {
     const candidate = candidateFromGreatest(entry, cacheDir);
@@ -65,8 +107,15 @@ function buildCandidates({ greatest, library, cacheDir }) {
     addRelations(map, { id: `anilist:${identity.id}`, title: manga.title, relations: identity.relations }, cacheDir);
   }
   for (const candidate of [...map.values()]) if (candidate.greatestId) addRelations(map, candidate, cacheDir);
+  if (harvestCache) addHarvested(map, harvestCache);
   return [...map.values()];
 }
+// Title-level isAdult or a strong adult tag; the Phase A regex only where AniList gave no isAdult.
+function isMature(candidate) {
+  if (candidate.isAdult === undefined) return candidate.tags.some(tag => /adult|hentai|erotica/i.test(tag));
+  return candidate.isAdult === true
+    || (candidate.tagRecords || []).some(tag => tag?.isAdult && Number(tag.rank) >= MATURE_TAG_RANK);
+}
 function ownedKeySet(library) {
   const ids = new Set(), names = new Set();
   for (const manga of library) {
@@ -77,9 +126,10 @@ function ownedKeySet(library) {
   return { ids, names };
 }
 function labelForVote(greatest, id) { return greatest.entries.find(entry => entry.id === id)?.title || null; }
-function feedForUser({ user, db, candidates, greatest, library, queuedTitles = [] }) {
+function feedForUser({ user, db, candidates, greatest, library, queuedTitles = [], harvestCache = null, now = () => new Date() }) {
   const settings = user.settings || {};
-  if (!settings.discoveryEnabled) return { generatedAt: new Date().toISOString(), items: [], disabled: true };
+  const today = now();
+  if (!settings.discoveryEnabled) return { generatedAt: today.toISOString(), items: [], disabled: true };
   const votes = db.greatestRatings?.[user.name] || {};
   const feedback = db.discoveryFeedback?.[user.name] || {};
   const owned = ownedKeySet(library);
@@ -93,20 +143,21 @@ function feedForUser({ user, db, candidates, greatest, library, queuedTitles = [
     for (const tag of candidate?.tags || []) likedTags.set(tag.toLowerCase(), anchor);
   }
   const ownedTags = new Map();
-  for (const manga of library) for (const tag of manga.meta?.tags || []) if (typeof tag === 'string') ownedTags.set(tag.toLowerCase(), manga.title);
+  for (const manga of library) for (const tag of stringsOf(manga.meta?.tags)) ownedTags.set(tag.toLowerCase(), manga.title);
   const readTags = new Map();
   if (settings.discoveryReadingActivity) for (const manga of library) {
     const progress = db.progress?.[user.name]?.[manga.id] || {};
     if (!Object.values(progress).some(chapter => chapter?.done)) continue;
-    for (const tag of manga.meta?.tags || []) if (typeof tag === 'string') readTags.set(tag.toLowerCase(), manga.title);
+    for (const tag of stringsOf(manga.meta?.tags)) readTags.set(tag.toLowerCase(), manga.title);
   }
+  const seeds = seedIndexFor({ anchors: anchorsFor({ user, db, library, greatest }), harvestCache, mature: !!settings.discoveryMature });
   const rows = [];
   for (const candidate of candidates) {
     if (candidate.greatestId && votes[candidate.greatestId]) continue;
     if (['dismissed', 'hidden'].includes(feedback[candidate.id])) continue;
     if (owned.ids.has(candidate.id) || owned.names.has(`${normalize(candidate.title)}:${candidate.year || '?'}`)) continue;
     if (queued.has(normalize(candidate.title))) continue;
-    if (!settings.discoveryMature && candidate.tags.some(tag => /adult|hentai|erotica/i.test(tag))) continue;
+    if (!settings.discoveryMature && isMature(candidate)) continue;
     const matchingLikedTag = candidate.tags.find(tag => likedTags.has(tag.toLowerCase()));
     const matchingOwnedTag = candidate.tags.find(tag => ownedTags.has(tag.toLowerCase()));
     const matchingReadTag = candidate.tags.find(tag => readTags.has(tag.toLowerCase()));
@@ -115,47 +166,50 @@ function feedForUser({ user, db, candidates, greatest, library, queuedTitles = [
     const scoreParts = { editorial: candidate.greatestId ? Math.max(0, 6 - Math.min(6, Number(candidate.editorialRank) || 6)) : 0,
       relation: relation ? 3 : 0, sharedTaste: matchingLikedTag ? 4 : 0, sharedLibrary: matchingOwnedTag ? 1 : 0,
       era: era ? 2 : 0, reading: matchingReadTag ? 2 : 0,
-      interest: feedback[candidate.id] === 'interested' ? 5 : feedback[candidate.id] === 'maybe' ? 2 : 0 };
-    let reason, personalEvidence = null;
-    if (relation) reason = `${candidate.title} is listed as ${relation.type.toLowerCase().replaceAll('_', ' ')} to ${relation.title}. Check the edition and reading order first.`;
-    else if (matchingLikedTag) {
+      interest: feedback[candidate.id] === 'interested' ? 5 : feedback[candidate.id] === 'maybe' ? 2 : 0,
+      communityRec: 0, sameCreator: 0, tagTop: 0, fresh: 0 };
+    // Every applicable part brings its reason and, for taste parts, its personal evidence.
+    const notes = [];
+    if (scoreParts.editorial && candidate.evidence[0]?.note) notes.push({ part: 'editorial', text: candidate.evidence[0].note });
+    if (relation) notes.push({ part: 'relation', text: `${candidate.title} is listed as ${relation.type.toLowerCase().replaceAll('_', ' ')} to ${relation.title}. Check the edition and reading order first.` });
+    if (matchingLikedTag) {
       const anchor = likedTags.get(matchingLikedTag.toLowerCase());
-      reason = `You liked ${anchor.title}; this also carries the ${matchingLikedTag} tag.`;
-      personalEvidence = { id: `taste:greatest:${anchor.id}`, kind: 'personal', label: `Your Greatest vote for ${anchor.title}`, note: `Shared tag: ${matchingLikedTag}`, url: null };
-    } else if (era) {
-      reason = `You liked ${era.title}; this is another curated work from ${candidate.period}.`;
-      personalEvidence = { id: `taste:greatest:${era.id}`, kind: 'personal', label: `Your Greatest vote for ${era.title}`, note: `Shared period: ${candidate.period}`, url: null };
-    } else if (matchingReadTag) {
-      const title = readTags.get(matchingReadTag.toLowerCase());
-      reason = `You have read ${title}; this also carries the ${matchingReadTag} tag.`;
-      personalEvidence = { id: `taste:read:${normalize(title)}`, kind: 'personal', label: `Your completed reading in ${title}`, note: `Shared tag: ${matchingReadTag}`, url: null };
-    } else if (matchingOwnedTag) {
-      const title = ownedTags.get(matchingOwnedTag.toLowerCase());
-      reason = `This shares the ${matchingOwnedTag} tag with ${title} in your library.`;
-      personalEvidence = { id: `taste:library:${normalize(title)}`, kind: 'personal', label: `${title} in your library`, note: `Shared tag: ${matchingOwnedTag}`, url: null };
+      notes.push({ part: 'sharedTaste', text: `You liked ${anchor.title}; this also carries the ${matchingLikedTag} tag.`,
+        evidence: { id: `taste:greatest:${anchor.id}`, kind: 'personal', label: `Your Greatest vote for ${anchor.title}`, note: `Shared tag: ${matchingLikedTag}`, url: null } });
     }
-    else reason = candidate.evidence[0]?.note || 'A curated manga worth a closer look.';
-    reason = reason.slice(0, 280);
-    const evidence = [...candidate.evidence, ...(personalEvidence ? [personalEvidence] : [])];
-    rows.push({ candidateId: candidate.id, reason, evidence, evidenceIds: evidence.map(e => e.id), scoreParts,
-      score: Object.values(scoreParts).reduce((a, b) => a + b, 0), period: candidate.period || 'Related works' });
+    if (era) notes.push({ part: 'era', text: `You liked ${era.title}; this is another curated work from ${candidate.period}.`,
+      evidence: { id: `taste:greatest:${era.id}`, kind: 'personal', label: `Your Greatest vote for ${era.title}`, note: `Shared period: ${candidate.period}`, url: null } });
+    if (matchingReadTag) {
+      const title = readTags.get(matchingReadTag.toLowerCase());
+      notes.push({ part: 'reading', text: `You have read ${title}; this also carries the ${matchingReadTag} tag.`,
+        evidence: { id: `taste:read:${normalize(title)}`, kind: 'personal', label: `Your completed reading in ${title}`, note: `Shared tag: ${matchingReadTag}`, url: null } });
+    }
+    if (matchingOwnedTag) {
+      const title = ownedTags.get(matchingOwnedTag.toLowerCase());
+      notes.push({ part: 'sharedLibrary', text: `This shares the ${matchingOwnedTag} tag with ${title} in your library.`,
+        evidence: { id: `taste:library:${normalize(title)}`, kind: 'personal', label: `${title} in your library`, note: `Shared tag: ${matchingOwnedTag}`, url: null } });
+    }
+    for (const note of seedNotes(candidate, seeds, today.getFullYear())) { scoreParts[note.part] = note.value; notes.push(note); }
+    notes.sort((a, b) => scoreParts[b.part] - scoreParts[a.part] || partRank(a.part) - partRank(b.part));
+    const reasons = notes.map(note => String(note.text).slice(0, 280));
+    const fallback = candidate.evidence[0]?.note || (candidate.greatestId || relation ? 'A curated manga worth a closer look.' : 'A manga from AniList worth a closer look.');
+    const evidence = [...candidate.evidence];
+    for (const note of notes) if (note.evidence && !evidence.some(e => e.id === note.evidence.id)) evidence.push(note.evidence);
+    rows.push({ candidateId: candidate.id, reason: (reasons[0] || fallback).slice(0, 280), reasons, evidence,
+      evidenceIds: evidence.map(e => e.id), scoreParts, score: round2(Object.values(scoreParts).reduce((a, b) => a + b, 0)),
+      period: candidate.period || 'Related works', group: groupFor(scoreParts, candidate.period), tags: candidate.tags });
   }
-  rows.sort((a, b) => b.score - a.score || a.candidateId.localeCompare(b.candidateId));
-  const picked = [], periodCounts = new Map();
-  for (const row of rows) {
-    if (picked.length >= MAX_FEED) break;
-    if ((periodCounts.get(row.period) || 0) >= 4) continue;
-    picked.push(row); periodCounts.set(row.period, (periodCounts.get(row.period) || 0) + 1);
-  }
-  return { generatedAt: new Date().toISOString(), items: picked.map((row, index) => ({ ...row, rank: index + 1 })) };
+  // Tags only steer the slot mix; the published row does not repeat them.
+  const items = mixSlots(rows, { topTags: seeds.topTags }).map(({ tags, ...row }) => row);
+  return { generatedAt: today.toISOString(), items };
 }
-function generate({ db, library, greatest, cacheDir, queuedTitles = [] }) {
-  const all = buildCandidates({ greatest, library, cacheDir });
-  // Sample across eras before the work cap; insertion order in the curated catalog
-  // must not make recent manga invisible.
+// Sample across eras (and seed kinds) before the work cap; insertion order in the curated
+// catalog must not make recent manga invisible.
+function candidatePool({ greatest, library, cacheDir, harvestCache }) {
+  const all = buildCandidates({ greatest, library, cacheDir, harvestCache });
   const buckets = new Map();
   for (const candidate of all) {
-    const key = candidate.period || 'Related works';
+    const key = candidate.period || candidate.seed || 'Related works';
     if (!buckets.has(key)) buckets.set(key, []);
     buckets.get(key).push(candidate);
   }
@@ -165,10 +219,50 @@ function generate({ db, library, greatest, cacheDir, queuedTitles = [] }) {
       if (values.length && candidates.length < MAX_CANDIDATES) candidates.push(values.shift());
     }
   }
-  const feeds = {};
-  for (const user of Object.values(db.users || {})) feeds[user.name] = feedForUser({ user, db, candidates, greatest, library, queuedTitles });
-  return { candidates: Object.fromEntries(candidates.map(c => [c.id, c])), feeds,
+  return { all, candidates };
+}
+// Only candidates a feed shows are published; the rest of the pool (up to 2000, with AniList
+// records) would bloat discovery.json for nothing. examined/clipped still describe the whole pool.
+function runResult(all, candidates, feeds) {
+  const shown = new Set();
+  for (const feed of Object.values(feeds)) for (const item of feed?.items || []) shown.add(item.candidateId);
+  return { candidates: Object.fromEntries(candidates.filter(c => shown.has(c.id)).map(c => [c.id, c])), feeds,
     examined: all.length, clipped: Math.max(0, all.length - candidates.length) };
 }
+function generate({ db, library, greatest, cacheDir, harvestCache = null, queuedTitles = [], now }) {
+  const { all, candidates } = candidatePool({ greatest, library, cacheDir, harvestCache });
+  const feeds = {};
+  for (const user of Object.values(db.users || {}))
+    feeds[user.name] = feedForUser({ user, db, candidates, greatest, library, queuedTitles, harvestCache, now });
+  return runResult(all, candidates, feeds);
+}
+// Same result as generate(), but yields to the event loop between users so the reader stays responsive.
+async function generateAsync({ db, library, greatest, cacheDir, harvestCache = null, queuedTitles = [], onlyUser, now }) {
+  const { all, candidates } = candidatePool({ greatest, library, cacheDir, harvestCache });
+  const feeds = {};
+  for (const user of Object.values(db.users || {})) {
+    if (onlyUser !== undefined && user.name !== onlyUser) continue;
+    await new Promise(setImmediate);
+    feeds[user.name] = feedForUser({ user, db, candidates, greatest, library, queuedTitles, harvestCache, now });
+  }
+  return runResult(all, candidates, feeds);
+}
+// A per-user rescore replaces one feed; the others and the candidates they show are carried over.
+function mergeUserFeed(prior, result, userName) {
+  const feeds = { ...(prior?.feeds || {}) };
+  if (result.feeds?.[userName]) feeds[userName] = result.feeds[userName];
+  else delete feeds[userName];
+  const candidates = {};
+  for (const [name, feed] of Object.entries(feeds)) {
+    if (name === userName) continue;
+    for (const item of feed?.items || []) {
+      const candidate = prior?.candidates?.[item.candidateId];
+      if (candidate) candidates[item.candidateId] = candidate;
+    }
+  }
+  Object.assign(candidates, result.candidates || {});
+  return { candidates, feeds };
+}
 
-module.exports = { generate, buildCandidates, feedForUser, cachedMetadata, MAX_CANDIDATES, MAX_FEED };
+module.exports = { generate, generateAsync, buildCandidates, candidatePool, feedForUser, mixSlots, mergeUserFeed, cachedMetadata,
+  isMature, MAX_CANDIDATES, MAX_FEED, SLOTS };

discovery/harvest.js

@@ -0,0 +1,174 @@
+'use strict';
+
+const crypto = require('node:crypto');
+const { AniListStop, AniListSkip } = require('./anilist');
+const { topTagsFor } = require('./anchors');
+
+const BUDGETS = { nightly: 300, admin: 300, user: 60 };
+const REFRESH_COOLDOWN_MS = 3600000;
+const MAX_CONSECUTIVE_FAILURES = 5;
+const CREATOR_MIN_WEIGHT = 3;
+
+// Thrown out of a run to unwind every stage at once.
+class StopRun extends Error {
+  constructor(status, stopReason) { super(stopReason); this.status = status; this.stopReason = stopReason; }
+}
+
+async function runHarvest({ kind, targets, cache, client, budget }) {
+  let requests = 0, failures = 0;
+  const seen = new Set();
+  // One request: skipped when already done this run or still fresh in the cache, otherwise awaited in turn.
+  const fetchInto = async (cacheKind, key, call) => {
+    const id = `${cacheKind}\n${key}`;
+    if (seen.has(id)) return;
+    seen.add(id);
+    if (cache.isFresh(cacheKind, key)) return;
+    if (requests >= budget) throw new StopRun('success', 'budget');
+    requests++;
+    let data;
+    try { data = await call(); }
+    catch (error) {
+      if (error instanceof AniListStop) throw new StopRun('stopped', error.reason);
+      if (!(error instanceof AniListSkip)) throw error;
+      if (++failures >= MAX_CONSECUTIVE_FAILURES) throw new StopRun('stopped', 'upstream-errors');
+      return;
+    }
+    failures = 0;
+    // A null title is not a manga format: nothing worth keeping.
+    if (data !== null) cache.put(cacheKind, key, data);
+  };
+
+  const anchors = targets.flatMap(target => target.anchors);
+  const recordOf = id => cache.get('title', id)?.data || null;
+  try {
+    for (const { anilistId } of anchors) await fetchInto('title', anilistId, () => client.title(anilistId));
+    for (const { anilistId } of anchors) await fetchInto('recs', anilistId, () => client.recs(anilistId));
+    for (const anchor of anchors) {
+      if (anchor.weight < CREATOR_MIN_WEIGHT) continue;
+      for (const { staffId } of recordOf(anchor.anilistId)?.authors || [])
+        await fetchInto('creator', staffId, () => client.creator(staffId));
+    }
+    for (const target of targets) {
+      for (const tag of topTagsFor(target.anchors, recordOf, { mature: target.mature }))
+        await fetchInto('tagTop', `${tag.kind}:${tag.name}`, () => client.tagTop(tag));
+    }
+    // With nobody to harvest for, even the global list is not worth a request.
+    if (targets.length) await fetchInto('fresh', 'global', () => client.fresh());
+  } catch (error) {
+    // The caller records how many requests a local fault had already spent.
+    if (!(error instanceof StopRun)) throw Object.assign(error, { requests });
+    return { requests, budget, status: error.status, stopReason: error.stopReason };
+  }
+  return { requests, budget, status: 'success', stopReason: null };
+}
+
+const localDate = date => [date.getFullYear(), String(date.getMonth() + 1).padStart(2, '0'),
+  String(date.getDate()).padStart(2, '0')].join('-');
+
+function shouldRunNightly({ now, harvestAt, lastNightlyDate, enabled }) {
+  if (!enabled) return false;
+  const match = /^(\d{2}):(\d{2})$/.exec(harvestAt || '');
+  if (!match) return false;
+  if (now.getHours() * 60 + now.getMinutes() < Number(match[1]) * 60 + Number(match[2])) return false;
+  return localDate(now) !== lastNightlyDate;
+}
+
+const httpError = (status, message, extra = {}) => Object.assign(new Error(message), { status }, extra);
+
+// These reasons mean AniList pushed back: no further calls until a nightly or admin run.
+const BLOCKING_REASONS = ['rate-limited', 'forbidden', 'challenge'];
+const blockingReason = record => record?.status === 'stopped' && BLOCKING_REASONS.includes(record.stopReason)
+  ? record.stopReason : null;
+
+class Harvester {
+  constructor({ cache, client, store, targetsFor, onDone, now = Date.now }) {
+    Object.assign(this, { cache, client, store, targetsFor, onDone, now });
+    this.queue = [];
+    this.active = null;
+    this.lastAccepted = new Map();
+    // The pause outlives a restart through the persisted harvest record.
+    try { this.blockedReason = blockingReason(store.load?.()?.harvest); }
+    catch { this.blockedReason = null; }
+  }
+  nextAllowedAt(user) {
+    const last = this.lastAccepted.get(user);
+    if (last === undefined || this.now() - last >= REFRESH_COOLDOWN_MS) return null;
+    return new Date(last + REFRESH_COOLDOWN_MS).toISOString();
+  }
+  jobs() { return [this.active, ...this.queue].filter(Boolean); }
+  request({ kind, user }) {
+    if (!Object.hasOwn(BUDGETS, kind)) throw httpError(400, `Unknown harvest kind: ${kind}`);
+    if (kind === 'user') {
+      if (!user) throw httpError(400, 'A user refresh needs a user');
+      if (this.jobs().some(job => job.kind === 'user' && job.user === user))
+        throw httpError(409, 'A refresh for this user is already queued or running');
+      if (this.blockedReason)
+        throw httpError(503, `AniList paused the harvest: ${this.blockedReason}`, { blockedReason: this.blockedReason });
+      const nextAllowedAt = this.nextAllowedAt(user);
+      if (nextAllowedAt) throw httpError(429, 'Refresh is cooling down', { nextAllowedAt });
+      this.lastAccepted.set(user, this.now());
+    } else if (this.jobs().length) {
+      throw httpError(409, 'A harvest is already queued or running');
+    } else {
+      // An accepted nightly or admin run is the one allowed retry after a pause.
+      this.blockedReason = null;
+    }
+    const job = { id: crypto.randomBytes(8).toString('hex'), kind, ...(kind === 'user' ? { user } : {}) };
+    const position = this.jobs().length;
+    this.queue.push(job);
+    if (!this.active) this.drain();
+    return { harvestId: job.id, position };
+  }
+  state(userName) {
+    const nextAllowedAt = this.nextAllowedAt(userName);
+    const extra = { ...(nextAllowedAt ? { nextAllowedAt } : {}), ...(this.blockedReason ? { blockedReason: this.blockedReason } : {}) };
+    const index = this.jobs().findIndex(job => job.kind === 'user' && job.user === userName);
+    if (index === 0 && this.active) return { state: 'running', ...extra };
+    if (index >= 0) return { state: 'queued', position: index, ...extra };
+    return { state: 'idle', ...extra };
+  }
+  current() {
+    if (!this.active) return null;
+    const { id, kind, user, startedAt } = this.active;
+    return { id, kind, ...(user ? { user } : {}), startedAt };
+  }
+  // Queued refreshes are dropped unrun; their users get their cooldown back for after the pause.
+  pause(reason) {
+    this.blockedReason = reason;
+    for (const job of this.queue) if (job.kind === 'user') this.lastAccepted.delete(job.user);
+    this.queue = this.queue.filter(job => job.kind !== 'user');
+  }
+  // Runs synchronously up to the first await, so a job is active before request() returns.
+  async drain() {
+    while (this.queue.length) {
+      const job = this.queue.shift();
+      job.startedAt = new Date(this.now()).toISOString();
+      this.active = job;
+      try { await this.run(job); }
+      catch (error) { console.error(`Discovery harvest ${job.id} failed:`, error); }
+      finally { this.active = null; }
+    }
+  }
+  async run(job) {
+    const budget = BUDGETS[job.kind];
+    const base = { id: job.id, kind: job.kind, user: job.user ?? null };
+    await this.store.patch({ harvest: { ...base, status: 'running', startedAt: job.startedAt, requests: 0, budget } });
+    let result;
+    try {
+      const targets = await this.targetsFor(job.kind, job.user);
+      result = await runHarvest({ kind: job.kind, targets, cache: this.cache, client: this.client, budget });
+    } catch (error) {
+      // A local fault (cache write, bad target) must not leave the record "running" until restart.
+      console.error(`Discovery harvest ${job.id} failed:`, error);
+      result = { requests: error?.requests ?? 0, budget, status: 'stopped', stopReason: 'error' };
+    }
+    const blocked = blockingReason(result);
+    if (blocked) this.pause(blocked);
+    await this.store.patch({ harvest: { ...base, status: result.status, startedAt: job.startedAt,
+      finishedAt: new Date(this.now()).toISOString(), requests: result.requests, budget, stopReason: result.stopReason } });
+    try { await this.onDone(job, result); }
+    catch (error) { console.error(`Discovery harvest ${job.id} follow-up failed:`, error); }
+  }
+}
+
+module.exports = { BUDGETS, REFRESH_COOLDOWN_MS, MAX_CONSECUTIVE_FAILURES, runHarvest, shouldRunNightly, Harvester };

discovery/scoring.js

@@ -0,0 +1,158 @@
+'use strict';
+
+// Pure scoring helpers for the AniList seed parts and the feed slot mix (BL-0051).
+const { topTagsFor } = require('./anchors');
+
+const MAX_FEED = 20;
+const SLOTS = { perKind: 8, perPeriod: 4, fresh: 4, outside: 4 };
+const CREATOR_MIN_WEIGHT = 3;
+const TAG_TOP_MIN_SCORE = 75;
+const FRESH_YEARS = 2;
+// Part name -> group (seed kind) for the four AniList seed parts.
+const SEED_PART_GROUP = { communityRec: 'recs', sameCreator: 'creator', tagTop: 'tagTop', fresh: 'fresh' };
+const SEED_GROUPS = new Set(Object.values(SEED_PART_GROUP));
+// Tie-break when two parts are equal: seed parts first, then Phase A order.
+const PART_ORDER = ['communityRec', 'sameCreator', 'tagTop', 'fresh',
+  'interest', 'editorial', 'sharedTaste', 'relation', 'era', 'reading', 'sharedLibrary'];
+
+const round2 = value => Math.round(value * 100) / 100;
+const listOf = value => Array.isArray(value) ? value : [];
+// Cached lists come from disk; anything that is not an object is skipped, never dereferenced.
+const objectsOf = value => listOf(value).filter(item => item && typeof item === 'object');
+const partRank = part => { const index = PART_ORDER.indexOf(part); return index < 0 ? PART_ORDER.length : index; };
+
+function communityRecValue(rating, weight) {
+  const safe = Math.max(0, Number(rating) || 0);
+  return Math.min(5, 1 + Math.log2(1 + safe) / 2) * weight / 5;
+}
+const sameCreatorValue = weight => 4 * weight / 5;
+
+// What the harvest cache says about this user's anchors: one lookup map per seed kind.
+function seedIndexFor({ anchors, harvestCache, mature = false }) {
+  const index = { recs: new Map(), creators: new Map(), tagTop: new Map(), fresh: new Map(), topTags: [] };
+  if (!harvestCache) return index;
+  const recordOf = id => harvestCache.get('title', id)?.data || null;
+  const nameOf = anchor => anchor.title || recordOf(anchor.anilistId)?.title || `AniList ${anchor.anilistId}`;
+  for (const anchor of anchors) {
+    const name = nameOf(anchor);
+    const entry = harvestCache.get('recs', anchor.anilistId);
+    for (const item of objectsOf(entry?.data)) {
+      const id = `anilist:${Number(item.anilistId)}`;
+      if (!index.recs.has(id)) index.recs.set(id, []);
+      index.recs.get(id).push({ anchor, name, rating: Number(item.rating) || 0, record: item.record || null, fetchedAt: entry.fetchedAt });
+    }
+    if (anchor.weight < CREATOR_MIN_WEIGHT) continue;
+    for (const author of objectsOf(recordOf(anchor.anilistId)?.authors)) {
+      const prior = index.creators.get(author.staffId);
+      if (prior && prior.anchor.weight >= anchor.weight) continue;
+      index.creators.set(author.staffId, { anchor, name, author,
+        fetchedAt: harvestCache.get('creator', author.staffId)?.fetchedAt || null });
+    }
+  }
+  index.topTags = topTagsFor(anchors, recordOf, { mature });
+  index.topTags.forEach((tag, place) => {
+    const entry = harvestCache.get('tagTop', `${tag.kind}:${tag.name}`);
+    for (const item of objectsOf(entry?.data)) {
+      const id = `anilist:${Number(item.anilistId)}`;
+      // Tags come best-first, so the first hit is the strongest.
+      if (!index.tagTop.has(id)) index.tagTop.set(id, { tag, value: 3 - place, record: item.record || null, fetchedAt: entry.fetchedAt });
+    }
+  });
+  const fresh = harvestCache.get('fresh', 'global');
+  for (const item of objectsOf(fresh?.data)) {
+    index.fresh.set(`anilist:${Number(item.anilistId)}`, { record: item.record || null, fetchedAt: fresh.fetchedAt });
+  }
+  return index;
+}
+
+// Seed parts for one candidate with the reason and evidence each one brings.
+function seedNotes(candidate, seeds, thisYear) {
+  const notes = [];
+  const urlFor = record => record?.siteUrl && /^https:\/\/[^\s]+$/i.test(String(record.siteUrl)) ? String(record.siteUrl) : candidate.metadataUrl || null;
+
+  let bestRec = null;
+  for (const hit of seeds.recs.get(candidate.id) || []) {
+    const value = communityRecValue(hit.rating, hit.anchor.weight);
+    if (!bestRec || value > bestRec.value) bestRec = { ...hit, value };
+  }
+  if (bestRec) notes.push({ part: 'communityRec', value: round2(bestRec.value),
+    text: `AniList readers who liked ${bestRec.name} recommend this (${bestRec.rating}).`,
+    evidence: { id: `rec:${bestRec.anchor.anilistId}:${candidate.id}`, kind: 'recommendation',
+      label: `AniList recommendation from readers of ${bestRec.name}`, note: `Rating ${bestRec.rating}`,
+      url: urlFor(bestRec.record), fetchedAt: bestRec.fetchedAt } });
+
+  let bestCreator = null;
+  for (const author of objectsOf(candidate.authorRecords)) {
+    const hit = seeds.creators.get(author.staffId);
+    // An anchor is never its own "also by" suggestion.
+    if (!hit || `anilist:${hit.anchor.anilistId}` === candidate.id) continue;
+    if (!bestCreator || hit.anchor.weight > bestCreator.anchor.weight) bestCreator = { ...hit, candidateAuthor: author };
+  }
+  if (bestCreator) {
+    const author = bestCreator.candidateAuthor.name || bestCreator.author.name;
+    const verb = bestCreator.anchor.source === 'read' ? 'read' : 'liked';
+    notes.push({ part: 'sameCreator', value: round2(sameCreatorValue(bestCreator.anchor.weight)),
+      text: `Also by ${author} — you ${verb} ${bestCreator.name}.`,
+      evidence: { id: `creator:${bestCreator.author.staffId}:${candidate.id}`, kind: 'creator',
+        label: `Also by ${author}`, note: `Shared creator with ${bestCreator.name}`,
+        url: urlFor(null), fetchedAt: bestCreator.fetchedAt || candidate.fetchedAt || null } });
+  }
+
+  const top = seeds.tagTop.get(candidate.id);
+  const averageScore = Number(candidate.averageScore ?? top?.record?.averageScore);
+  if (top && averageScore >= TAG_TOP_MIN_SCORE) notes.push({ part: 'tagTop', value: top.value,
+    text: `Top-rated in ${top.tag.name}, one of your most-liked tags.`,
+    evidence: { id: `tagtop:${top.tag.kind}:${top.tag.name}:${candidate.id}`, kind: 'tag-top',
+      label: `Top-rated in ${top.tag.name} on AniList`, note: `Average score ${averageScore}`,
+      url: urlFor(top.record), fetchedAt: top.fetchedAt } });
+
+  const fresh = seeds.fresh.get(candidate.id);
+  if (fresh) {
+    const status = fresh.record?.status ?? candidate.status;
+    const year = Number(fresh.record?.year ?? candidate.year);
+    const isNew = status === 'ongoing' && year >= thisYear - FRESH_YEARS;
+    notes.push({ part: 'fresh', value: isNew ? 2 : 1,
+      // The exact template claims a start year and "ongoing", which only the value-2 case proves.
+      text: isNew ? `New: started ${year}, ongoing and trending.` : 'Trending on AniList right now.',
+      evidence: { id: `fresh:${candidate.id}`, kind: 'fresh', label: 'Trending new manga on AniList', note: '',
+        url: urlFor(fresh.record), fetchedAt: fresh.fetchedAt } });
+  }
+  return notes;
+}
+
+function largestPart(scoreParts) {
+  let best = null;
+  for (const [part, value] of Object.entries(scoreParts)) {
+    if (!best || value > best.value || (value === best.value && partRank(part) < partRank(best.part))) best = { part, value };
+  }
+  return best;
+}
+function groupFor(scoreParts, period) {
+  const top = largestPart(scoreParts);
+  return top && top.value > 0 && SEED_PART_GROUP[top.part] ? SEED_PART_GROUP[top.part] : period || 'Related works';
+}
+const byScore = (a, b) => b.score - a.score || (a.candidateId < b.candidateId ? -1 : a.candidateId > b.candidateId ? 1 : 0);
+
+// Reserve slots for fresh and outside-the-top-tags rows, then fill greedily; every step respects the group caps.
+function mixSlots(rows, { topTags = [], max = MAX_FEED } = {}) {
+  const sorted = [...rows].sort(byScore);
+  const picked = new Set(), counts = new Map();
+  const capOf = group => SEED_GROUPS.has(group) ? SLOTS.perKind : SLOTS.perPeriod;
+  const take = row => {
+    if (picked.has(row) || picked.size >= max || (counts.get(row.group) || 0) >= capOf(row.group)) return false;
+    picked.add(row); counts.set(row.group, (counts.get(row.group) || 0) + 1);
+    return true;
+  };
+  const reserve = (limit, wanted) => {
+    let taken = 0;
+    for (const row of sorted) if (taken < limit && wanted(row) && take(row)) taken++;
+  };
+  const top = new Set(topTags.map(tag => String(tag.name).toLowerCase()));
+  reserve(SLOTS.fresh, row => row.scoreParts?.fresh > 0);
+  reserve(SLOTS.outside, row => !listOf(row.tags).some(tag => top.has(String(tag).toLowerCase())));
+  for (const row of sorted) take(row);
+  return [...picked].sort(byScore).map((row, index) => ({ ...row, rank: index + 1 }));
+}
+
+module.exports = { MAX_FEED, SLOTS, PART_ORDER, round2, communityRecValue, sameCreatorValue,
+  seedIndexFor, seedNotes, largestPart, groupFor, partRank, mixSlots };

discovery/store.js

@@ -4,6 +4,8 @@ const fs = require('node:fs');
 const path = require('node:path');
 const crypto = require('node:crypto');
 
+const META_KEYS = ['harvest', 'settings', 'nightly'];
+
 class DiscoveryStore {
   constructor(dataDir) { this.file = path.join(path.resolve(dataDir), 'discovery.json'); }
   load() {
@@ -11,9 +13,14 @@ class DiscoveryStore {
       const data = JSON.parse(fs.readFileSync(this.file, 'utf8'));
       if (data.schema !== 1 || !Number.isSafeInteger(data.revision) || !data.candidates || !data.feeds || !data.lastRun)
         throw new Error('Invalid discovery store schema');
+      // Phase A files lack the metadata fields; default them.
+      if (data.harvest === undefined) data.harvest = null;
+      if (data.settings === undefined) data.settings = null;
+      if (!data.nightly) data.nightly = { lastDate: null };
       return data;
     } catch (error) {
-      if (error.code === 'ENOENT') return { schema: 1, revision: 0, candidates: {}, feeds: {}, lastRun: { status: 'never' } };
+      if (error.code === 'ENOENT') return { schema: 1, revision: 0, candidates: {}, feeds: {}, lastRun: { status: 'never' },
+        harvest: null, settings: null, nightly: { lastDate: null } };
       throw new Error(`Discovery store unavailable; refusing to overwrite it: ${error.message}`);
     }
   }
@@ -21,7 +28,17 @@ class DiscoveryStore {
     const prior = this.load();
     const data = { schema: 1, revision: prior.revision + 1, candidates: value.candidates, feeds: value.feeds,
       lastRun: { status: 'success', startedAt: value.startedAt, finishedAt: new Date().toISOString(),
-        candidateCount: Object.keys(value.candidates).length, examined: value.examined, clipped: value.clipped } };
+        candidateCount: Object.keys(value.candidates).length, examined: value.examined, clipped: value.clipped },
+      harvest: prior.harvest, settings: prior.settings, nightly: prior.nightly };
+    return this.write(data);
+  }
+  // Metadata only: each given key replaces that field; revision stays.
+  patch(fields) {
+    const data = this.load();
+    for (const key of META_KEYS) if (key in fields) data[key] = fields[key];
+    return this.write(data);
+  }
+  write(data) {
     fs.mkdirSync(path.dirname(this.file), { recursive: true });
     const temp = `${this.file}.${crypto.randomBytes(6).toString('hex')}.tmp`;
     try {

docker-compose.yml

@@ -36,21 +36,31 @@ services:
         source: ${DATA_PATH:-./data}
         target: /data
         bind: {create_host_path: false}
+      # Built-in backups (docs/backup.md). The folder must exist and be writable by
+      # uid 1000; backups stay off unless BACKUP_PATH is set.
+      - type: bind
+        source: ${BACKUP_PATH:-./backup}
+        target: /backup
+        bind: {create_host_path: false}
     environment:
       PORT: "8080"
       DATA: /data
       LIBRARY: /library
-      TZ: ${TZ:-Europe/Berlin}
+      BACKUP: ${BACKUP_PATH:+/backup}
+      TZ: ${TZ:-UTC}
       SIGNUP: closed
       SCAN_INTERVAL: "10"
+      SCAN_CONCURRENCY: ${SCAN_CONCURRENCY:-8}
       COOKIE_SECURE: ${COOKIE_SECURE:-0}
       READING_WINDOWS_JSON: ${READING_WINDOWS_JSON:-}
-      READING_TIMEZONE: ${READING_TIMEZONE:-Europe/Berlin}
+      READING_TIMEZONE: ${READING_TIMEZONE:-UTC}
       READING_QUIZ_UNLOCK_MINUTES: ${READING_QUIZ_UNLOCK_MINUTES:-60}
       READING_QUIZ_CATALOG_PATH: ${READING_QUIZ_CATALOG_PATH:-}
       SCRAPER_URL: http://scraper:8091
-      SENTINEL_URL: http://sentinel:8092
       SCRAPER_TOKEN_FILE: /run/secrets/scraper_token
+      DISCOVERY_HARVEST_AT: ${DISCOVERY_HARVEST_AT:-03:30}
+      DISCOVERY_ANILIST_URL: ${DISCOVERY_ANILIST_URL:-}
+      GREATEST_COVER_WARM: ${GREATEST_COVER_WARM:-1}
     networks: [backend, egress]
     healthcheck:
       test: [CMD, node, tools/healthcheck.js, reader]
@@ -80,7 +90,7 @@ services:
       LIBRARY: /library
       SCRAPER_DATA: /data/scraper
       METADATA_CACHE: /data/scraper/metadata/anilist
-      TZ: ${TZ:-Europe/Berlin}
+      TZ: ${TZ:-UTC}
       SCRAPER_HOST: 0.0.0.0
       SCRAPER_PORT: "8091"
       SCRAPER_TOKEN_FILE: /run/secrets/scraper_token
@@ -93,31 +103,6 @@ services:
       start_period: 60s
       retries: 3
 
-  sentinel:
-    <<: *app
-    command: [node, runtime/launch.js, sentinel]
-    volumes:
-      - type: bind
-        source: ${SENTINEL_DATA_PATH:-./data/sentinel}
-        target: /data/sentinel
-        bind: {create_host_path: false}
-    environment:
-      DATA: /data
-      SENTINEL_DATA: /data/sentinel
-      SENTINEL_HOST: 0.0.0.0
-      SENTINEL_PORT: "8092"
-      READER_URL: http://reader:8080
-      SCRAPER_URL: http://scraper:8091
-      SCRAPER_TOKEN_FILE: /run/secrets/scraper_token
-      TZ: ${TZ:-Europe/Berlin}
-    networks: [backend]
-    healthcheck:
-      test: [CMD, node, tools/healthcheck.js, sentinel]
-      interval: 30s
-      timeout: 5s
-      start_period: 60s
-      retries: 3
-
   peer-gateway:
     <<: *app
     profiles: [peering]
@@ -142,7 +127,7 @@ services:
       PEER_USAGE_FILE: /data/peering-usage/usage.json
       PEER_HOST: 0.0.0.0
       PEER_PORT: "8093"
-    # Docker Desktop does not publish host ports from an internal-only network.
+    # Docker (Desktop) does not publish host ports from an internal-only network.
     networks: [backend, egress]
     healthcheck:
       test: [CMD, node, tools/healthcheck-peer.js]

docs/architecture.md

@@ -2,8 +2,8 @@
 
 MangaCircus is a self-hosted reader for manga, comics and webtoons. The reader owns
 accounts/progress and a separate collection catalog. The scraper owns the persistent
-queue and chapter publication. Sentinel observes health/progress and requests bounded
-checkpoint recovery. Separate supervisors isolate service failures. All content remains
+queue and chapter publication, and its built-in stall watchdog requeues a stuck job
+once at a checkpoint. Separate supervisors isolate service failures. All content remains
 a plain folder tree; there is no database server or embedded autonomous LLM.
 
 Related documents:
@@ -13,11 +13,12 @@ Related documents:
 - [scraper-service.md](scraper-service.md): managed queue, private API, and adapter contract.
 - [collections.md](collections.md): relations, groups, curator UI and agent research.
 - [resilience.md](resilience.md): supervision, recovery limits and incidents.
-- [compose-deployment.md](compose-deployment.md): local/VM deployment and release workflow.
+- [deployment.md](deployment.md): Docker deployment, HTTPS, operation, updates and backups.
 - [settings-navigation.md](settings-navigation.md): User/Admin workspace, scraper tabs and deep links.
 - [peering.md](peering.md): optional private server-to-server catalog and transfer.
 - [discovery.md](discovery.md): reader-owned, opt-in deterministic discovery feed and its limits.
 - [source-modules.md](source-modules.md): adding and maintaining first-party hosters inside the single scraper process.
+- [backup.md](backup.md): built-in scheduled backup of `data/` and the restore procedure.
 
 ## 1. Principles
 
@@ -47,12 +48,12 @@ Related documents:
 
 ## 3. Components
 
-Runtime topology (2026-10-05): native `tools/start-all.js` launches reader, scraper and
-Sentinel independently. In Compose, each container runs `runtime/launch.js <service>`
+Runtime topology (2026-10-22): native `tools/start-all.js` launches reader and scraper
+independently. In Compose, each container runs `runtime/launch.js <service>`
 and exactly one service child. A failure does not stop unrelated services. Exactly one
 reader and queue-owning scraper may write each shared state tree; do not scale replicas.
-The catalog and relation store is reader-owned `data/catalog.json`. Sentinel writes only
-`data/sentinel/`; it has no Docker socket or arbitrary code-repair authority.
+The catalog and relation store is reader-owned `data/catalog.json`. No service has a
+Docker socket or arbitrary code-repair authority.
 
 The admin browser never talks to the scraper daemon directly. Admin-only reader
 routes proxy commands to the daemon's private API, so its shared token stays on the
@@ -67,6 +68,7 @@ survives restarts, so restarting does not itself force another complete walk.
 |------|------:|----------------|
 | `server.js` | ~350 | HTTP server and router, auth and sessions, JSON storage, API handlers, scan scheduling, notifications and webhooks |
 | `library.js` | ~170 | Folder scanner, name parsing, image-size reader, scan cache |
+| `backup.js` | ~330 | Built-in `DATA` backup: queue pause/resume, tar.gz writer and verifier, schedule, retention |
 | `public/common.js` | ~85 | Browser helpers: `h()` element builder, `api()` fetch wrapper, top navigation with the notification bell |
 | `public/index.html` + `library.js` | ~60 | Library overview (cover grid, filter, sort) |
 | `public/manga.html` + `manga.js` | ~110 | Manga page: info, follow, chapter list with book separators, versions and votes, read/unread toggle |
@@ -157,7 +159,11 @@ This keeps ids human-readable and derivable without a database. The price: renam
 - `data/library-index.json` holds a disposable, versioned index snapshot and shallow
   directory stamps. The library folders remain authoritative; a missing or invalid
   snapshot triggers a full scan. Rollback-compatible `scan-cache.json` is retained.
-- **Backups:** copy `data/db.json`. The atomic writes make a copy at any time consistent.
+- `data/backup.json` holds the built-in backup schedule, retention and recent results.
+- **Backups:** the reader's built-in backup ([backup.md](backup.md)) archives the whole
+  `data/` tree into `BACKUP` on a schedule, with `db.json` taken from memory and the
+  queue paused around the copy. A plain copy of `data/db.json` also works: atomic
+  writes make it consistent at any time.
 
 ## 6. Server
 
@@ -165,7 +171,7 @@ This keeps ids human-readable and derivable without a database. The price: renam
 
 | Path | Handling |
 |------|----------|
-| `/api/*` | `routes["<METHOD> <path>"]`. Handlers get `{ u, b, q, req, res }` and return an object, which is sent as JSON. Throwing `HttpError` sends `{ error }` with that status |
+| `/api/*` | `routes["<METHOD> <path>"]`. Handlers get `{ u, b, q, req, res }` and return an object, which is sent as JSON. Throwing `HttpError(code, message, extra)` sends `{ error, ...extra }` with that status (for example `nextAllowedAt` on a discovery `429`) |
 | `/lib/*` | Image files from `LIBRARY`. Requires a session, allows image extensions only, and the path must resolve inside `LIBRARY`. `ETag` + `private, max-age=86400` |
 | everything else | Static files from `public/` (`/` → `index.html`). `ETag` + `no-cache` |
 
@@ -198,7 +204,17 @@ All requests and responses are JSON. POSTs require `Content-Type: application/js
 | `GET /api/admin` | admin | Users, last scan result, library path |
 | `POST /api/admin/rescan` `{full}` | admin | Scan now, and wait for the result |
 | `POST /api/admin/user` `{name, password, admin}` | admin | Create a user or reset a password |
+| `POST /api/admin/user/role` `{name, admin}` | admin | Grant or revoke admin rights; keeps the password and sessions. Revoking your own rights is refused, so one admin always remains |
 | `POST /api/admin/user/delete` `{name}` | admin | Delete a user and their data |
+| `GET /api/admin/backup` · `POST /api/admin/backup` `{command:"settings"\|"run"}` | admin | Built-in backup status, schedule/retention, "back up now" ([backup.md](backup.md)) |
+| `GET /api/greatest-cover?id=<entry>&v=<version>` | user | The Greatest entry's cover from `data/greatest-covers/` (`greatest/covers.js`), cached by the browser for a year; `v` changes with the catalog URL. A `302` to the catalog URL until the copy exists. `/api/greatest` lists this URL as `cover` for titles not in the library |
+| `GET /api/discovery` | user | The caller's discovery feed, settings, feedback and `refresh: { state: idle\|queued\|running, position?, nextAllowedAt?, blockedReason? }` (§6.6) |
+| `POST /api/discovery/feedback` `{candidateId, value}` | user | The caller's choice for a candidate; requires opt-in |
+| `POST /api/discovery/refresh` | user | Queue a per-user AniList harvest for the caller (opt-in required, else `403`). `{ harvestId, position }`; `409` while one is queued or running for the caller; `429 { nextAllowedAt }` within an hour of the last accepted refresh; `503 { blockedReason }` after AniList pushed back, until a nightly or admin harvest |
+| `GET /api/admin/discovery` | admin | Last scoring run, `lastError` (a failed admin run or a failed re-score after a harvest), opted-in users, candidate count, plus `harvest` (last run record), `currentHarvest`, `cache: { files, bytes }`, `settings: { harvestEnabled, harvestAt }` and `nextNightlyAt` (`null` when off) |
+| `POST /api/admin/discovery/run` | admin | Re-score every user from local data; `409` while a run is active |
+| `POST /api/admin/discovery/harvest` | admin | Start a full harvest for all opted-in users: `{ harvestId, position }`, `409` while any harvest is queued or running |
+| `POST /api/admin/discovery/settings` `{harvestEnabled, harvestAt}` | admin | Exactly a boolean and an `HH:MM` time, anything else is `400`; stored in `discovery.json` `settings` |
 
 ### 6.3 Authentication and security
 
@@ -221,6 +237,28 @@ At request time: the sum of all users' votes, then `chapter.json.priority`, then
 
 In-app notifications and optional webhooks are produced by `announce()` after each scan, from the append-only `known` list. See [import.md §5](import.md#5-change-detection-and-notifications).
 
+### 6.6 Discovery harvest
+
+The reader alone talks to AniList for discovery ([discovery.md](discovery.md)). A single-flight harvester
+(`discovery/harvest.js`) fetches seeds for the opted-in users' anchors into `data/discovery-cache/`; scoring
+(`discovery/engine.js`) is local and publishes `data/discovery.json` atomically.
+
+- **Kinds and budgets:** `nightly` and `admin` (300 requests, every opted-in user), `user` (60 requests, the
+  caller only). After a nightly or admin harvest every user is re-scored; after a per-user refresh only that
+  user, and the other feeds are carried over unchanged into the same publish.
+- **Schedule:** checked every minute and once 30 s after startup. The time is `discovery.json` `settings`, else
+  `DISCOVERY_HARVEST_AT` (`HH:MM`), else `03:30`; `nightly.lastDate` is written only when the run is accepted.
+- **Run record** `discovery.json` `harvest`: `{ id, kind, user, status: running|success|stopped|interrupted,
+  startedAt, finishedAt, requests, budget, stopReason }`. `stopReason` is `budget`, `rate-limited`, `forbidden`,
+  `challenge`, `upstream-errors` or `error` (a local fault). A run cut off by a restart is marked `interrupted`
+  at startup; queued refreshes are not persisted.
+- **Published candidates:** only those at least one feed shows. `lastRun.candidateCount` (the admin
+  candidate count) counts that set; `lastRun.examined` and `clipped` still describe the whole pool.
+- **Published fields:** candidates found through AniList carry `tagRecords`, `authorRecords`, `seed`
+  (`recs|creator|tagTop|fresh`) and `fetchedAt`; feed items carry `group` (seed kind or period) and `reasons`
+  (all reasons, strongest first) next to `reason`; seed evidence carries `fetchedAt`.
+- `DISCOVERY_ANILIST_URL` overrides the AniList endpoint (tests point it at a local fixture server).
+
 ## 7. Frontend
 
 ### 7.1 Pages
@@ -317,20 +355,11 @@ mode       = adapt && intended ∉ compatible ? compatible[0] : intended
 
 ## 8. Deployment
 
-### Current Windows deployment
-
-The observed local installation uses the Windows Scheduled Task
-`\MangaCircus Supervisor`. At logon it runs `tools/run-supervisor.ps1`, which appends
-logs under `data/scraper/` and starts one `tools/start-all.js` supervisor. That parent
-owns exactly one reader (`server.js`, port 8080) and one scraper daemon
-(`scraper/daemon.js`, private port 8091). Starting `npm run start:all` manually at the
-same time would create conflicting writers and is not supported.
-
-The scraper's authoritative `data/scraper/queue.json` is schema version 3. It supports
-legacy archive checkpoints and paused conversion of completed legacy work while adding
-source-aware lanes, scoped queue jobs, incremental chapter publication, and nightly catalog discovery. See
-[local-deployment.md](local-deployment.md) for operational checks and restart/recovery,
-and [scraper-service.md](scraper-service.md) for the queue/API contract.
+The step-by-step guide is [deployment.md](deployment.md). The scraper's authoritative
+`data/scraper/queue.json` is schema version 3. It supports legacy archive checkpoints
+and paused conversion of completed legacy work while adding source-aware lanes, scoped
+queue jobs, incremental chapter publication, and nightly catalog discovery. See
+[scraper-service.md](scraper-service.md) for the queue/API contract.
 
 ### Docker or manual deployment
 
@@ -342,8 +371,9 @@ docker compose up -d --build
 - **Volumes:**
   - `/library`: read-only in the reader and writable in the scraper. Files must be readable—and scraper targets writable—by uid 1000.
   - `/data`: reader state must be writable; the scraper also needs its writable `/data/scraper` state.
-- **Configuration:** process wiring uses environment variables (`PORT`, `LIBRARY`, `DATA`, `SIGNUP`, `SCAN_INTERVAL`, `FULL_SCAN_INTERVAL`, `COOKIE_SECURE`, `SCRAPER_URL`, and `SCRAPER_TOKEN`). Queue pacing and automatic-update schedule are persisted in `data/scraper/queue.json` and edited through the authenticated admin UI.
-- **Without Docker:** `npm run start:all` with Node ≥ 18, or supervise `npm start` and `npm run start:scraper` independently. On the current Windows machine that supervision is already owned by `\MangaCircus Supervisor`.
+  - `/backup`: reader only, writable target of the built-in backup (`BACKUP_PATH`; off when unset).
+- **Configuration:** process wiring uses environment variables (`PORT`, `LIBRARY`, `DATA`, `BACKUP`, `SIGNUP`, `SCAN_INTERVAL`, `FULL_SCAN_INTERVAL`, `COOKIE_SECURE`, `SCRAPER_URL`, and `SCRAPER_TOKEN`). Queue pacing and automatic-update schedule are persisted in `data/scraper/queue.json` and edited through the authenticated admin UI.
+- **Without Docker:** `npm run start:all` with Node ≥ 18 starts one supervisor that owns exactly one reader (`server.js`, port 8080) and one scraper daemon (`scraper/daemon.js`, private port 8091), or supervise `npm start` and `npm run start:scraper` independently. On Windows, `tools/run-supervisor.ps1` runs `start:all` with logs under `data/scraper/`, e.g. from a scheduled task at logon. Never start a second supervisor or a container against the same data.
 - **Behind a reverse proxy:** terminate TLS, set `COOKIE_SECURE=1`. A sub-path works.
 - **Upgrades:** replace the code and restart. Reader state has no explicit migration;
   unknown fields in `db.json` are kept. The scraper normalizes old queue entries into
@@ -355,7 +385,7 @@ docker compose up -d --build
 | Aspect | Behaviour |
 |--------|-----------|
 | Library size | The index keeps a small object per page (file name, size, tags), in the order of a few hundred bytes. Hundreds of thousands of pages fit in a modest container. Unchanged chapters cost one `stat` + one small read per scan |
-| First scan | Reads ≤ 256 KB per image once. Later scans use the cache |
+| First scan | Reads 32 KB per image once (≤ 256 KB, or the whole file for JPEGs with huge EXIF/ICC blocks), `SCAN_CONCURRENCY` (8) at a time. Later scans use the cache |
 | User data | `db.json` is rewritten as a whole, at most about twice per second while people read. Comfortable up to a few MB (a household or small group, thousands of chapters read) |
 | Concurrency | One process, one writer. No clustering or horizontal scaling by design |
 | Images | Served as stored. No resizing or thumbnails, so large covers make the library page heavy |
@@ -366,6 +396,7 @@ docker compose up -d --build
 |----------|--------------|-----|
 | Index the folder tree in place | Import into a DB / managed storage | Stays readable by any viewer and writable by any tool. Nothing to keep in sync |
 | No dependencies, no framework, no build | Express, React, bundler | Minimal code, maximum longevity, runs anywhere Node runs |
+| Built-in backup in the reader, queue paused, no container stop | Host job that stops the stack; backup in a separate service | Only the reader sees all of `/data` and the in-memory `db.json`; the scraper's pause API gives a consistent point without downtime |
 | JSON file for user data | SQLite (`node:sqlite` is still experimental in Node 22/24), Postgres | Zero dependencies, trivial backup, enough for the target size. Easy to swap: all access goes through the `db` object |
 | Page roles as file-name tags | Page lists in JSON | Visible in any file browser, survives copying, no second source of truth. Unknown tags are harmless |
 | Image sizes read on the server | Measure in the browser after load | Layout before load: exact scroll restore, zoom anchoring, auto-detected double pages, no layout jumps |

docs/backup.md

@@ -0,0 +1,116 @@
+# Built-in backup
+
+The reader can back up its own `DATA` folder on a schedule set in
+**Settings → Admin → Service health → Backups**, without stopping any container.
+The library is not part of it (it is plain files; back it up with the storage that
+holds it). Implementation: [`backup.js`](../backup.js); tests: `test/backup.test.js`.
+
+## Turning it on
+
+| Where | Setting |
+|---|---|
+| Native | `BACKUP=/path/to/backups` for `server.js`. Unset = feature off |
+| Compose | `BACKUP_PATH=/host/folder` in `.env`. The reader mounts it rw at `/backup` and gets `BACKUP=/backup`; without `BACKUP_PATH` the variable stays empty and backups are off |
+
+The folder must already exist and be writable by uid 1000 (`create_host_path: false`:
+a missing folder makes `docker compose up` fail instead of silently creating an empty
+one on the wrong disk). Use another disk or a NAS share, never a folder inside `DATA`. Writability is checked by
+creating and removing a probe file (`.circus-backup-probe-<pid>`), not with `access()`, which
+NFS shares with squashed ids (e.g. Synology "map all users to admin") answer wrongly.
+Without `BACKUP_PATH`, Compose still mounts the unused default `./backup`, so that
+folder must exist too (`node tools/prepare-compose.js` creates it). Upgrading an
+existing deployment therefore needs either `BACKUP_PATH` in `.env` or an empty
+`backup/` beside `docker-compose.yml`.
+
+Archives contain password hashes, sessions and the peering private key: keep the
+target private.
+
+### Why the reader owns it
+
+The reader is the only service that mounts all of `/data` (the scraper sees only
+`/data/scraper`), it holds `db.json` in memory and owns its atomic writer, it already
+talks to the scraper's private control API with the shared token, and the admin
+settings UI lives there. Running it in the scraper would need a full read-write `/data`
+mount there (see [architecture.md §3](architecture.md#3-components)).
+
+## Settings and status
+
+Admin only, under Service health (`GET/POST /api/admin/backup`, also included in
+`GET /api/admin/health` as `backup`):
+
+- **Back up daily at HH:MM** (container time zone `TZ`) and an enable switch.
+- **Keep the newest** N backups (`count`, 1–365) or N days (`days`, 1–3650).
+- **Back up now** starts a run immediately; the page polls every 2 s while it runs.
+- Status: target folder, number/size of archives, the running phase, last backup
+  (time, trigger, size, duration, file), its error and warnings, next scheduled run.
+
+Settings and the last 10 results persist atomically in `data/backup.json`. A slot
+missed while the reader was down runs once at the next start if it is less than a
+day old; older slots (for example after restoring an old `backup.json`) are skipped.
+
+```text
+POST /api/admin/backup {command:"settings", settings:{enabled, time:"HH:MM", retention:{mode:"count"|"days", value}}}
+POST /api/admin/backup {command:"run"}       -> 409 when off or already running
+```
+
+## What one run does
+
+1. Ask the scraper for its state (`GET /api/state`). If the queue runs, pause it
+   (`POST /api/control {action:"pause"}`) and remember that this run paused it.
+2. Wait until no job is active (the worker stops at its next page/chapter boundary),
+   polling every 2 s for at most 15 minutes. On timeout the run fails, writes nothing
+   and resumes the queue if it paused it. An unreachable scraper does not block the
+   backup; that is recorded as a warning.
+3. Write `BACKUP/circus-YYYYMMDD-HHMMSS.tar.gz.part`: a gzip'd tar of `DATA` with
+   paths relative to `DATA`, fsync it, read it back to verify, rename it to `.tar.gz`.
+   - `db.json` is serialized from the reader's in-memory state at that moment.
+   - `scraper/queue.json` is stored with `paused: true` (status `paused`), so a
+     restored copy never starts downloading by itself. The live file is not touched.
+   - `scan-cache.json`, `library-index.json`, `catalog.json`, peering, logs and
+     everything else under `DATA` is copied as found; all of them are written by
+     atomic rename, so each file is complete. A leftover `sentinel/` folder from
+     before 2026-10-22 is copied too; it is unused and can be deleted.
+   - Left out: `runtime*.json`, `*.lock` (`daemon.lock`, `supervisor.lock`), `*.pid`,
+     `*.tmp`, symlinks, and the `BACKUP` folder itself if it lies inside `DATA`.
+   - `.circus-backup.json` at the archive root records when and how it was made.
+4. Resume the queue only if this run paused it (three attempts; a failure is shown
+   as a warning: resume it under Settings → Scraper).
+5. After a successful run only, apply retention. It deletes only files named
+   `circus-YYYYMMDD-HHMMSS.tar.gz` directly inside `BACKUP` (age from the name),
+   never the newest one, never other files or subfolders. Stale
+   `circus-*.tar.gz.part` files from an interrupted run are removed at the next run.
+
+If the reader restarts during a run, `backup.json` still carries the run's
+"queue paused by backup" marker; the next start resumes the queue and records the
+run as interrupted. The marker is never part of an archive.
+
+Measured on a copy of a production data folder (54 MB, 1000 files): 5.8 MB archive in
+about 5 s including verification. Pauses last until the current chapter is done
+plus those seconds.
+
+## Restore
+
+Restoring replaces the user data with the state of the archive. Reading progress
+and queue changes made after that backup are lost.
+
+1. Stop the stack (or at least reader and scraper):
+   `docker compose stop reader scraper`.
+2. Move the current data folder aside instead of deleting it:
+   `mv /srv/mangacircus/data /srv/mangacircus/data.before-restore`.
+3. Unpack the archive into an empty data folder owned by uid 1000:
+   ```bash
+   mkdir /srv/mangacircus/data
+   tar -xzf /path/to/circus-YYYYMMDD-HHMMSS.tar.gz -C /srv/mangacircus/data
+   chown -R 1000:1000 /srv/mangacircus/data
+   ```
+   (`--same-owner` is the default for root; the archive keeps the original uid/gid.)
+4. Start the reader first: `docker compose up -d --no-build reader`. Check `/health`,
+   log in, check progress and library. The reader rebuilds a missing or stale library
+   index by itself.
+5. Start `scraper`. The restored queue is paused: check it under
+   Settings → Scraper and resume it when it looks right.
+
+The scraper token is not in `DATA`; Compose keeps it under `.secrets/` (or the
+deployment's `SCRAPER_TOKEN_PATH`). Restore tests should unpack into a separate
+folder and run an isolated reader (no second writer against the live library), as
+`deploy/circus.sh restore-test` does.

docs/collections.md

@@ -26,7 +26,7 @@ selection and must use **Validate & save JSON**, not an ordinary collection save
 Metadata imports/research are blocked while drafts exist so they cannot silently
 replace them. Relationship edits save immediately when there is no collection draft.
 
-Creation works on HTTPS, localhost and ordinary HTTP LAN/Tailscale addresses. The
+Creation works on HTTPS, localhost and ordinary HTTP LAN/VPN addresses. The
 browser uses secure random bytes when its secure-context-only `randomUUID` API is
 unavailable. This does not change networking or the server's revision/validation rules.
 
@@ -65,7 +65,7 @@ membership edits. It should preserve rejected decisions and report uncertainty.
 It must not turn a shared-universe link into an automatic collection or an unquestioned
 reading order. No embedded LLM, API credentials, automatic research loop, or autonomous
 catalog approval is enabled in this release. That optional capability belongs behind
-this same evidence/review boundary, not inside adapters or Sentinel.
+this same evidence/review boundary, not inside adapters or the scraper's stall watchdog.
 
 ## Persistence and API contract
 
@@ -115,9 +115,9 @@ Provider contract: [AniList relationship connections](https://docs.anilist.co/gu
 
 ## Repeatable functional acceptance
 
-Run `npm.cmd test`, build `docker compose build reader`, then:
+Run `npm test`, build `docker compose build reader`, then:
 
-```powershell
+```bash
 # Playwright is optional developer tooling; not a deployed application dependency.
 # Set NODE_PATH to its installed package root if it is not on normal resolution paths.
 node tools/test-compose.js --collections-ui --settings-ui

docs/deployment.md

@@ -0,0 +1,201 @@
+# Deploying MangaCircus with Docker
+
+This guide sets up MangaCircus on one Linux host with Docker Compose: the reader (web app), the managed
+scraper, and optionally HTTPS and server peering. Every step is a command with what you should see. It is
+written so that a person or a coding agent can follow it without other context.
+
+## 1. What runs
+
+| Service | Port | Role |
+|---|---|---|
+| `reader` | 8080 (published on `READER_BIND`) | Web app: library, reader, accounts, settings. The only writer of users, progress and the catalog (`DATA_PATH`). Reads the library read-only |
+| `scraper` | 8091, internal only | Download queue and source adapters. The only writer of the library (`LIBRARY_PATH`) and of its own state (`SCRAPER_DATA_PATH`). Never publish or proxy this port |
+| `peer-gateway` | 8093, optional (`--profile peering`) | Shares chosen manga with other MangaCircus servers over mutual TLS ([peering.md](peering.md)) |
+
+All services run from one image (`node:24-alpine`, no runtime dependencies) as uid/gid 1000. Each container
+has a read-only root file system, no capabilities, rotated logs, a health check and an in-container
+supervisor that restarts a crashed process. The reader works when the scraper is stopped.
+
+Data on the host, all bind mounts (no named volumes):
+
+| Path (`.env`) | Contents | Back up? |
+|---|---|---|
+| `LIBRARY_PATH` | The manga: `Manga/Book/Chapter/images` ([library-format.md](library-format.md)) | Yes, it is your collection |
+| `DATA_PATH` | `db.json` (users, progress, votes), catalog, discovery feeds, library index cache | Yes (built-in backup, section 8). `discovery-cache/` inside it is safe to delete and may be left out of backups: the next harvest rebuilds it. The same holds for `greatest-covers/` (local copies of the Greatest catalog's cover images, fetched again after the next start) |
+| `SCRAPER_DATA_PATH` | Queue, settings, source ledger, metadata cache | Included in the built-in backup |
+| `BACKUP_PATH` | Built-in backup archives | Keep it on another disk or NAS |
+| `SCRAPER_TOKEN_PATH` | Shared secret between reader and scraper | Regenerate if lost |
+
+## 2. Requirements
+
+- Linux x86_64 or arm64 with Docker Engine 24+ and the Compose plugin 2.24.4+ (`docker compose version`).
+  Docker Desktop on Windows or macOS works for trying it out. Bind mounts are slow there for large libraries.
+- 1–2 CPU cores and 1 GB RAM are enough for a few readers. The first scan of a large library is
+  I/O-bound (about 14 minutes for 300,000 pages on a NAS over NFS). Later starts reuse the index cache.
+- Disk: the library plus room for downloads in progress. The image is about 60 MB.
+- Git to fetch the code. Node is **not** needed on the host.
+
+## 3. Get the code and prepare folders
+
+```bash
+git clone <repository-url> mangacircus && cd mangacircus
+mkdir -p library data/scraper data/peering data/peering-usage backup .secrets
+openssl rand -hex 32 > .secrets/scraper_token && chmod 600 .secrets/scraper_token
+cp .env.example .env
+sudo chown -R 1000:1000 library data backup
+```
+
+The containers run as uid 1000: the reader must be able to read the library and write `data/`, and the
+scraper must be able to write the library. On an NFS share with squashed ids, map all users to one account
+that owns the files. Don't use `chmod 777`. Bind mounts never create missing folders: a typo in a path
+fails the start instead of showing an empty library.
+
+Already have a library? Point `LIBRARY_PATH` at it (folder format in [library-format.md](library-format.md)).
+Starting empty? Fill it from the web UI's scraper queue (section 6). To try the reader first, create five
+demo manga in an empty library after the build in section 5:
+`docker compose run --rm --no-deps scraper node tools/make-dummy.js /library`.
+
+## 4. Configure `.env`
+
+| Variable | Default | Meaning |
+|---|---|---|
+| `LIBRARY_PATH` | `./library` | Library folder (absolute path recommended) |
+| `DATA_PATH` | `./data` | Reader data |
+| `SCRAPER_DATA_PATH` | `./data/scraper` | Scraper state |
+| `PEER_DATA_PATH`, `PEER_USAGE_PATH` | `./data/peering`, `./data/peering-usage` | Peering identity and transfer counters (folders must exist even when peering is off) |
+| `SCRAPER_TOKEN_PATH` | `./.secrets/scraper_token` | The secret file from section 3 |
+| `BACKUP_PATH` | unset = backups off | Folder for built-in backups (section 8) |
+| `READER_BIND` / `READER_PORT` | `127.0.0.1` / `8080` | Where the reader listens on the host. `0.0.0.0` = every interface (plain HTTP, LAN only) |
+| `TZ` | `UTC` | Time zone of the containers; affects the nightly update time and backup schedule |
+| `COOKIE_SECURE` | `0` | `1` when readers reach the site over HTTPS |
+| `READING_WINDOWS_JSON`, `READING_TIMEZONE`, `READING_QUIZ_*` | off | Optional server-enforced reading hours ([reading-limiter.md](reading-limiter.md)) |
+| `DISCOVERY_HARVEST_AT` | `03:30` | Daily time (`HH:MM`) of the nightly Discover harvest from AniList, in the container's local time zone (`TZ`). Only users who opted in are harvested; the admin can change the time or switch it off on the Discover page ([discovery.md](discovery.md)) |
+| `DISCOVERY_ANILIST_URL` | AniList's public API | Tests only: points the discovery harvest at a fixture server. Leave unset in production |
+| `GREATEST_COVER_WARM` | `1` | `0` stops the reader from downloading the Greatest catalog's covers (about 100 images, one per second, 10 s after start and once a day for missing ones). Tests and offline installs set `0`; the page then loads covers from the catalog's own URLs |
+| `APP_IMAGE`, `VCS_REF` | `mangacircus:local`, `development` | Image to run; set a pinned registry image to deploy prebuilt releases (section 7) |
+| `COMPOSE_FILE` | `docker-compose.yml` | Extra Compose files, `:`-separated (e.g. `docker-compose.yml:deploy/compose.https.yml`) |
+| `MANGACIRCUS_DOMAIN` | – | Host name for `deploy/compose.https.yml` |
+
+Sign-up is closed: the first account becomes the admin, who then adds the other users. Reader settings
+not listed here (scan intervals, scan concurrency) are in the README's environment table.
+
+## 5. Build and start
+
+```bash
+docker compose config -q            # no output = the configuration is valid
+docker compose build
+bash deploy/circus.sh up            # reader first, waits for its health check, then the scraper
+bash deploy/circus.sh status
+```
+
+Expected:
+
+```
+SERVICE   STATUS                    PORTS
+reader    Up 1 minute (healthy)     127.0.0.1:8080->8080/tcp
+scraper   Up 40 seconds (healthy)   8080/tcp
+{"ok":true,"scan":{"mangas":…,"stale":false,"error":null}, …}
+{"status":"idle","paused":false,"active":0,"queue":0, …}
+```
+
+Open `http://127.0.0.1:8080` (or an SSH tunnel: `ssh -L 8080:127.0.0.1:8080 <host>`) and create the admin
+account. The library page fills as soon as the first scan finishes.
+
+## 6. Access and adding manga
+
+- **On the LAN:** set `READER_BIND=0.0.0.0` (or the host's LAN address), or use `compose.lan.yml` with
+  `READER_LAN_BIND`. This is plain HTTP: logins and cookies travel unencrypted, so never forward it to the
+  internet.
+- **HTTPS with the bundled Caddy:** point a DNS name at the host, open ports 80 and 443, set
+  `MANGACIRCUS_DOMAIN=manga.example.org` and `COMPOSE_FILE=docker-compose.yml:deploy/compose.https.yml`,
+  then `bash deploy/circus.sh up`. Caddy fetches the certificate. The reader gets no host port and secure
+  cookies.
+- **Your own proxy:** keep `READER_BIND=127.0.0.1` and proxy `http://127.0.0.1:8080`. Alternatively, use
+  `compose.vm.yml`, which joins the reader to an external Docker network (`PROXY_NETWORK`, default
+  `mangacircus-edge`) and removes the host port. Set `COOKIE_SECURE=1`. All pages use relative URLs, so a
+  sub-path such as `/manga/` works.
+- **Private access over a VPN** (WireGuard, Tailscale, …): bind the reader to the VPN address, or keep it on
+  loopback and forward from the VPN client.
+- **Adding manga:** admins use **Settings → Admin → Scraper → Inspect & queue** with a series URL from a
+  supported source, or copy folders into the library by hand. MangaDex is on by default. WeebCentral and
+  MangaPill are unofficial sites that mostly host unlicensed copies, so they start switched off. Switch them
+  on per source under **Hoster settings** if you may copy what you download there. While a source is off,
+  its queued jobs wait and nightly update checks skip it. Upgraded installations keep the sources their
+  queue already used. The reader
+  notices new folders within `SCAN_INTERVAL` minutes. Rules for tools that write into the library:
+  [scraper-guide.md](scraper-guide.md). Only download what you are allowed to copy; the scraper keeps
+  conservative per-host rate limits and stops on 401/403/429 ([scraper/AGENT_RUNBOOK.md](../scraper/AGENT_RUNBOOK.md)).
+
+## 7. Operating it
+
+`deploy/circus.sh` wraps the safe sequences (run it from any folder; it finds the repository itself):
+
+| Task | Command | Notes |
+|---|---|---|
+| Status | `bash deploy/circus.sh status` | services, reader health, queue |
+| Stop / start | `… down` / `… up` | `down` pauses the queue and waits for the running chapter (up to 15 minutes); `up` resumes only what `down` paused |
+| Restart | `… restart` or `… restart reader` | |
+| Logs | `… logs scraper 200` | |
+| Pause / resume downloads | `… pause` / `… resume` | Same as the buttons in Settings → Admin → Scraper |
+| Update to the newest code | `… update` | `git pull --ff-only`, rebuild, pause, recreate, resume |
+| Re-read edited `chapter.json` files | `… rescan` | Full reconciliation without an admin login; admins can also use Settings → Admin → Rescan |
+| Check the newest backup restores | `… restore-test` | Throwaway reader without network; prints `RESTORE TEST OK` |
+
+**Deploying prebuilt images** (several hosts, or no building on the server): build once, push to your
+registry and pin the digest:
+
+```bash
+docker build --build-arg VCS_REF=$(git rev-parse --short HEAD) -t registry.example.org/mangacircus:$(git rev-parse --short HEAD) .
+docker push registry.example.org/mangacircus:<commit>          # note the sha256 digest it prints
+# on the server, in .env: APP_IMAGE=registry.example.org/mangacircus@sha256:<digest>
+docker compose pull && bash deploy/circus.sh restart
+```
+
+Pinned digests make rollbacks exact: put the previous `APP_IMAGE` back and restart. Data formats only grow
+(new fields are ignored by older releases), but a rollback does not undo queue or catalog changes made by
+the newer release.
+
+**Rules for operators and agents:**
+- Never run two scrapers, or a container and a native `node` process, against the same library or data
+  folder.
+- Never run `docker compose down -v`. Stop with `deploy/circus.sh down`.
+- Never publish port 8091, and never print or commit `.secrets/` or `.env`.
+- One-off maintenance tools that write `chapter.json` files are dry runs by default. Run them with
+  `--apply --writers-stopped` only after `docker compose stop scraper`, then `deploy/circus.sh rescan`.
+  Examples: `tools/backfill-chapter-dates.js` and `tools/release-dates.js`, run inside the image with
+  `docker compose run --rm --no-deps scraper node tools/<tool>.js /library …`.
+
+## 8. Backups
+
+- **Reader and scraper data:** set `BACKUP_PATH` to an existing folder on another disk or NAS (owned by uid
+  1000), restart, and choose the schedule and retention under **Settings → Service health → Backups**. A
+  run pauses the queue, waits for the current chapter, archives the data folder without stopping anything
+  and resumes. Details, archive contents and the restore procedure: [backup.md](backup.md). Check it now
+  and then with `deploy/circus.sh restore-test`.
+- **The library:** back it up with your usual file backup (it is plain folders and images), including
+  hidden `.mcscraper-*` folders if downloads are in progress.
+- **Restore:** `deploy/circus.sh down`, move the data folder aside, unpack the archive into a fresh data
+  folder (`tar -xzf circus-….tar.gz -C data`, owned by uid 1000), `deploy/circus.sh up`. Restored archives
+  have the queue paused; resume it after checking.
+
+## 9. Optional: server peering
+
+Share selected manga with friends' MangaCircus servers: enable the `peering` profile
+(`docker compose --profile peering up -d peer-gateway`) and follow [peering.md](peering.md). The gateway
+needs its own reachable port (`PEER_BIND`, `PEER_PORT`) and uses mutual TLS with pinned certificates.
+
+## 10. Troubleshooting
+
+| Symptom | Cause / fix |
+|---|---|
+| `docker compose up` fails with "bind source path does not exist" | A folder from `.env` is missing; create it (section 3) |
+| Reader healthy but the library is empty | First scan still running (`docker compose logs reader` shows `[scan] …` when done), or `LIBRARY_PATH` points at the wrong folder |
+| Scraper jobs fail with `EACCES` | The library or `SCRAPER_DATA_PATH` is not writable by uid 1000 |
+| Backups fail with `EACCES` on a NAS | Map the share's users to the uid that owns the folder; the backup probes with a real write |
+| Settings → Scraper says the service is unavailable | Scraper container stopped or unhealthy: `deploy/circus.sh status`, `logs scraper` |
+| Edited `chapter.json` changes don't show | Run `deploy/circus.sh rescan` (in-place edits are picked up by full reconciliations, every 24 hours by default) |
+| Logins fail behind HTTPS | `COOKIE_SECURE=1` needs HTTPS end to end at the browser; with plain HTTP keep it `0` |
+
+More background: [architecture.md](architecture.md) (components, API, security model),
+[import.md](import.md) (scanning and caching), [scraper-service.md](scraper-service.md) (queue, adapters,
+recovery) and [resilience.md](resilience.md) (supervisors and incidents).

docs/discovery.md

@@ -1,23 +1,28 @@
-# Discovery Scout (deterministic first release)
+# Discovery Scout
 
-The reader now has a separate **Discover** page. This first slice deliberately uses
-the curated Greatest catalog, already-cached AniList identities/relations, library
-metadata, each user's Greatest votes, and explicit Discover feedback. It does not
-call an LLM, fetch new metadata, inspect chapter/image hosters, edit a collection,
+The reader has a separate **Discover** page. Suggestions come from the curated
+Greatest catalog, cached AniList relations of Greatest and library titles, and a
+wider pool that the reader harvests from AniList's public API for the users who
+opted in (design: [the wider-pool spec](superpowers/specs/2026-10-25-discovery-wider-pool-design.md)).
+Discovery does not call an LLM, inspect chapter/image hosters, edit a collection,
 or enqueue downloads. The reader remains the only writer of account and discovery
-state; no fourth Compose service, credential, or database is required.
+state; no further Compose service, credential, or database is required.
 
 ## Use
 
 1. Open `discover.html`. Each account independently enables **Personalized
    discovery** (off by default). Reading activity is a separate opt-in. The
-   mature-title switch only applies to candidates with a known mature tag; it is
-   not a content-safety guarantee.
-2. An admin presses **Run discovery now** on the same page. Existing suggestions
-   remain visible until the new feed is published. Runs are bounded to 50
-   candidates and 20 suggestions per opted-in user, with no more than four from
-   one period. A second concurrent run is rejected.
-3. A reader can open **Why this is here · evidence** and record Interested, Maybe,
+   mature-title switch only affects titles AniList marks as adult or with a known
+   mature tag; it is not a content-safety guarantee.
+2. **Refresh my suggestions** queues a harvest for your own account (at most once an
+   hour). The button shows the queue position, `Fetching…` while it runs, and when
+   the next refresh is possible. Your existing suggestions stay visible until the
+   new feed is published.
+3. An admin can press **Run discovery now** (re-score from local data, no network)
+   or **Harvest now** (fetch from AniList for every opted-in user, then re-score).
+   A second concurrent run or harvest is rejected. A nightly harvest runs on its
+   own; the admin panel can switch it off or change its time.
+4. A reader can open **Why this is here · evidence** and record Interested, Maybe,
    No thanks, or Hide. Feedback is private to the account. Dismissed/hidden
    titles disappear at once; other feedback influences the next run. The user
    can undo a choice by clicking its active button again.
@@ -28,17 +33,72 @@ the existing source inspection and confirmation process to download anything.
 The title/year/format matching is intentionally conservative but not a verified
 identity merge. Missing year, cover, or status remains missing.
 
+## Seeds from AniList
+
+A harvest starts from each opted-in user's **anchors**: titles they voted
+`flame`/`good` in Greatest, marked Interested/Maybe in Discover, read (only with
+reading activity opted in), or own with an AniList ID. The 15 strongest anchors
+count. It then asks AniList for four seed kinds:
+
+| Seed | What | Reason shown on the card |
+|---|---|---|
+| Community recommendations | "readers who liked X recommend Y" for each anchor | AniList readers who liked {anchor} recommend this ({rating}). |
+| Same creator | other works by the story/art authors of strong anchors | Also by {author} — you read/liked {anchor}. |
+| Top in your tags | well-rated titles in the user's three strongest tags | Top-rated in {tag}, one of your most-liked tags. |
+| Fresh | recently started, releasing and trending titles | New: started {year}, ongoing and trending. — or, when the cached record does not show an ongoing title started in the last two years, Trending on AniList right now. |
+
+A title found by several seeds keeps every reason and evidence item; the card
+shows the strongest, the details list all of them, each with the date it was
+fetched. A feed holds at most 20 suggestions, at most 8 from one seed kind, at most
+4 from one Greatest period (or from related works), up to 4 fresh titles, and up to 4
+from outside your strongest tags.
+
+- **Budgets:** a nightly or admin harvest makes at most 300 requests, a
+  per-user refresh 60. Requests are sequential and limited to 25 a minute. When
+  the budget is spent the harvest stops; what was fetched is still used.
+- **Cache:** `data/discovery-cache/`, one file per response. Titles,
+  recommendations and creators stay fresh for 30 days, tag lists and the fresh
+  list for 7 days; at most 5,000 files, oldest pruned first. Scoring reads only
+  this cache and never the network, so a failed harvest means older candidates,
+  never an empty feed.
+- **Nightly time:** `03:30` by default, or `DISCOVERY_HARVEST_AT`, in the
+  container's local time zone (`TZ`). The admin panel overrides both and can turn
+  the nightly harvest off. A night missed while the reader was down runs once
+  after startup.
+- **Privacy:** for opted-in users only, the server sends AniList the IDs of
+  their anchor titles (liked, read or owned), the AniList staff IDs of those titles'
+  authors and the names of their top tags, without login, cookies or user names. Feeds and feedback stay private to this server; cache entries are
+  not tied to a user.
+
+### Stop reasons
+
+The admin panel shows the last harvest as `status · requests/budget · stop
+reason · cache files`.
+
+| Stop reason | Meaning | What happens next |
+|---|---|---|
+| `budget` | The request budget ran out | Normal; the next harvest continues where the cache is stale |
+| `rate-limited` | AniList answered 429 | Harvesting pauses; refresh buttons show "Paused by AniList" until the next nightly or admin harvest |
+| `forbidden` | AniList answered 401/403 | Same pause |
+| `challenge` | AniList answered with an HTML or challenge page | Same pause |
+| `upstream-errors` | Five timeouts or server errors in a row | The harvest ends; nothing is paused |
+| `error` | A local fault (for example a cache write) | See the reader log |
+
+Discovery never retries with a different identity, user agent or host. A
+re-scoring failure after a harvest is shown as the admin panel's error line.
+
 ## Data and API
 
-- `data/discovery.json`: atomic, reader-owned candidate and per-user feed
-  snapshots. A previous valid version is retained as `discovery.json.prev`.
-  A corrupt file makes reads/runs fail closed rather than overwrite it.
+- `data/discovery.json`: atomic, reader-owned per-user feed snapshots and the
+  candidates those feeds show (the rest of the pool is only counted:
+  `examined`, `clipped`), plus the last harvest record and the admin's harvest settings. A
+  previous valid version is retained as `discovery.json.prev`. A corrupt file
+  makes reads/runs/harvests fail closed rather than overwrite it.
+- `data/discovery-cache/`: harvested AniList responses; safe to delete.
 - `data/db.json`: consent in user settings and explicit feedback under
   `discoveryFeedback`. The account-deletion route removes that feedback.
-- `GET /api/discovery`: caller's own feed, settings, evidence, and feedback only.
-- `POST /api/discovery/feedback`: caller's candidate choice; requires opt-in.
-- `GET /api/admin/discovery` and `POST /api/admin/discovery/run`: admin-only
-  status and manual generation. A run keeps the last good feed on failure.
+- Routes: [architecture.md](architecture.md) (`/api/discovery*`,
+  `/api/admin/discovery*`) and §6.6 there for the harvester.
 
 The job asks the private scraper state API for queued titles to avoid recommending
 them. If that API is unavailable, the run still uses only cached sources; the
@@ -49,24 +109,24 @@ No scraper queue edit occurs.
 
 From the project directory:
 
-```powershell
+```bash
 node discovery/cli.js run --dry-run --data data --show
 npm run test:reader
 docker build -t mangacircus:discovery-test .
-$env:APP_IMAGE='mangacircus:discovery-test'; npm run test:discovery-ui
+APP_IMAGE=mangacircus:discovery-test npm run test:discovery-ui
 ```
 
 The dry run reads `db.json`, a saved library index if present, Greatest, and
-cached AniList metadata. It writes nothing and does not contact providers. The
-Compose/browser suite uses a disposable data tree and queue, not the live one.
+cached AniList metadata. It does not contact AniList or any other provider and does
+not publish `discovery.json`; reading the harvest cache may still delete a corrupt
+cache file, as the reader would. The
+Compose/browser suite uses a disposable data tree and queue, not the live one,
+and points `DISCOVERY_ANILIST_URL` at a closed local port so no harvest reaches
+AniList.
 
-## Deliberate limits and next stage
+## Limits
 
-This slice has no scheduled run, external metadata discovery, LLM, admin seed
-form, edition-finding action, or explicit purge control for old feed snapshots.
-Disabling consent stops feed display and future personalized generation, but
-existing snapshot/feedback is retained until account deletion or a later data
-management feature. Candidate quality is limited by the curated list and local
-metadata cache. Add external public metadata feeds and optional LLM only behind
-the existing bounded candidate/evidence contract after this deterministic path
-has been used and reviewed.
+There is no admin seed form, edition-finding action, LLM rationale, or explicit
+purge control for old feed snapshots. Disabling consent stops feed display,
+harvesting and future personalized generation for that account, but existing
+snapshot/feedback is retained until account deletion.

docs/greatest.md

@@ -23,12 +23,12 @@ Within each cohort, **Rank by my vote** sorts flame → good → neutral → sku
 
 The one-time seed is curated from the research document. All 106 catalog entries now have a cover URL. Most were matched to public AniList manga metadata; the five initial gaps were checked individually. Phoenix (1954) and Nausicaä use matching AniList records; Kitarō uses Drawn & Quarterly's Kitaro edition, Someone Hertz uses Shonen Jump's first-volume cover, and Hitoner uses Shueisha's 2026 first-volume cover. Hitoner's AniList 2024 entry represents an earlier work, so it was **not** used for the 2026 series. The five added URLs returned image content on 2026-10-09, but remote hosts can change them; the UI still shows a title-letter fallback if an image fails.
 
-`node tools/enrich-greatest-catalog.js` is an **optional manual** public-AniList metadata refresh, paced at one request every 1.5 seconds. It only accepts matching manga title and **original serialization year**, leaving uncertain results untouched rather than using a sequel or remake cover. The dashboard makes **no metadata calls when loading**; only an admin pressing **Search MangaDex** triggers a bounded lookup. Browser covers are lazy-loaded from their listed source; local library covers take precedence.
+`node tools/enrich-greatest-catalog.js` is an **optional manual** public-AniList metadata refresh, paced at one request every 1.5 seconds. It only accepts matching manga title and **original serialization year**, leaving uncertain results untouched rather than using a sequel or remake cover. The dashboard makes **no metadata calls when loading**; only an admin pressing **Search MangaDex** triggers a bounded lookup. Local library covers take precedence. For the rest, the reader keeps its own copy of each catalog cover in `data/greatest-covers/` (`greatest/covers.js`): it downloads missing ones one at a time, about one a second, 10 seconds after start and once a day, stops on HTTP 401/403/429, and accepts only JPEG/PNG/WebP/GIF/AVIF images up to 2 MB from the catalog's own HTTPS URLs. The page loads covers from `/api/greatest-cover`, which the browser caches for a year; until a copy exists that route redirects to the catalog URL. Hotlinking about 100 covers per page load made the image CDN drop some of them at random. `GREATEST_COVER_WARM=0` switches the downloads off.
 
 Public catalog chapter counts are often absent; the UI says “Chapters unknown” rather than inventing one. For owned titles, the local library count wins. Status comes from AniList's releasing/finished flag or local library metadata, and may be unknown. Official source links may be publisher editions, chapter services, or an award/info lead; they do not imply all chapters are available in every country. No unverified hoster coverage is claimed.
 
 ## Deployment and QA
 
-The Docker image now copies `greatest/` as app code; no new Compose service, volume, port, secret, or migration is needed. Rebuild/recreate **reader only** to pick up catalog/UI changes. The scraper and sentinel do not need restarting. Existing `data/db.json` accepts the new `greatestRatings` object without migration.
+The Docker image now copies `greatest/` as app code; no new Compose service, volume, port, secret, or migration is needed. Rebuild/recreate **reader only** to pick up catalog/UI changes. The scraper does not need restarting. Existing `data/db.json` accepts the new `greatestRatings` object without migration.
 
 Run `npm run test:reader` for catalog/API tests and `node tools/test-greatest-ui.js` for disposable headless-browser QA. The UI fixture blocks remote covers and never contacts the scraper. Manual catalog edits should preserve stable IDs once anyone has voted; changing a title's ID strands its votes.

docs/import.md

@@ -148,7 +148,9 @@ Everything is plain data, and the whole index lives in memory.
 
 ## 3. Image sizes
 
-The reader lays pages out before they load, so it needs every image's size. Instead of decoding images, the scanner reads the **first 256 KB** and parses the header:
+The reader lays pages out before they load, so it needs every image's size. Instead of decoding images, the scanner reads the **first 32 KB** and parses the header; only if the size isn't there yet does it read on to 256 KB (and, for JPEG, the whole file). Parsing a prefix either finds nothing or the same size as the whole file, so the small first read never changes a result.
+
+Uncached chapter folders and their images are probed **`SCAN_CONCURRENCY` at a time** (default 8, 1–64; chapters of a book in parallel, then assembled in folder order). A first scan over a NAS is bound by per-file latency, not bandwidth: on a VM reading a Synology NAS over NFS the old sequential 256 KB probe took about 20 ms per image; see §3.1 for the measurements. `SCAN_CONCURRENCY=1` restores the sequential order.
 
 | Format | Where the size is |
 |--------|-------------------|
@@ -156,9 +158,25 @@ The reader lays pages out before they load, so it needs every image's size. Inst
 | GIF | logical screen (bytes 6–9) |
 | BMP | DIB header (bytes 18–25) |
 | WebP | `VP8 ` / `VP8L` / `VP8X` chunk |
-| JPEG | first SOFn marker (the scan skips APPn/EXIF/ICC segments). If it isn't within 256 KB, the whole file is read once |
+| JPEG | first SOFn marker (the scan skips APPn/EXIF/ICC segments). If it isn't within 32 KB, then 256 KB, the whole file is read once |
 | AVIF / HEIF | first `ispe` box |
 
+### 3.1 Measured first-scan speed
+
+A Debian VM (4 vCPU) reading a Synology NAS over NFS 4.0, page cache dropped before each run, production
+running alongside. Each variant scanned a different manga of ~5,000 pages (the NAS caches what it has just read).
+
+| Variant | ms per image (round 1 / round 2) |
+|---|---|
+| sequential, 256 KB (before) | 7.4 / 9.7 |
+| 32 KB first read, `SCAN_CONCURRENCY=1` | 3.6 / 6.0 |
+| 32 KB first read, `SCAN_CONCURRENCY=8` (default) | 2.9 / 2.1 |
+| 32 KB first read, `SCAN_CONCURRENCY=16` | 2.4 / 3.3 |
+
+Whole library without cache (56 manga, 10,766 chapters, 299,467 pages) with the default: **825 s (13.8 min)**,
+against an estimated 37–48 min before (about 4 h while the NAS was busy with a large copy). The page lists and
+sizes of all 10,775 cached chapter folders were identical to those of the previous code.
+
 - If the size can't be read, it is stored as `0×0`. The reader then assumes 1000×1414 (2000×1414 for `double`) and corrects the layout when the image arrives.
 - EXIF orientation is **not** applied. Use `rot…` tags instead.
 

docs/library-format.md

@@ -208,11 +208,53 @@ An AniList refresh with a different ID is stopped for review before chapter publ
 |-------|------|---------|
 | `title` | string | Chapter title without "Chapter N" (default: title part of the folder name) |
 | `source` | `{name, url}` or string | Where this version came from (default: `[source]` in the folder name) |
-| `added` | ISO date/time string | Release (or download) date, used for "recently updated" (default: the folder's modification time) |
+| `added` | ISO date/time string | When this version entered **this** library, used for "recently updated" (default: the folder's modification time). The scraper keeps it when it replaces a version |
+| `x-dates` | object | Other chapter dates, written by the scraper (see below) |
 | `extra` | boolean | Marks the chapter as extra without moving it |
 | `priority` | number | Tie-breaker between versions with equal votes. Higher wins (default `0`) |
 | `direction` | as above | Overrides book/manga direction, e.g. a colour webtoon special inside a printed series |
 
+#### Chapter dates (`x-dates`)
+
+```json
+"added": "2026-10-17T15:04:03.424Z",
+"x-dates": {
+  "version": 1,
+  "source": { "at": "2026-10-18T15:06:05.348Z", "name": "WeebCentral", "batch": false },
+  "release": { "at": "2023-09-17", "precision": "day", "source": "MangaUpdates · Viz" },
+  "ecosystem": { "at": "2026-10-17T15:04:03.424Z", "via": "this server" }
+}
+```
+
+- `source`: when the source site uploaded this version (WeebCentral's full chapter list,
+  MangaDex `publishAt`). `batch: true` means it arrived with 3 or more other uploads
+  less than 10 minutes apart, which is a site import, not a release.
+- `release`: the official public release (`precision` `day`, `month` or `year`). It is
+  filled from MangaUpdates releases by publishers such as Viz, MANGA Plus or Webtoon
+  (`scraper/metadata/release-dates.js`). Those lists include English re-releases of old
+  chapters, so a date is kept only when all of these hold:
+  - it lies within the work's original run on AniList;
+  - no scanlation or source upload shows the chapter more than 4 days earlier;
+  - it fits the release order: walking down from the newest chapter, a lower chapter
+    can't be dated more than 2 days after a higher one.
+
+  Series whose chapter numbers restart in a later book are skipped.
+  - The scraper refreshes ongoing series nightly before its update checks, but only
+    while no download is running.
+  - `node tools/release-dates.js <library>` covers a whole library: a dry run by
+    default, then `--apply --writers-stopped`.
+  - A date that this job wrote and no longer accepts is removed. Release dates from any
+    other source stay.
+- `ecosystem`: when the chapter first entered any Manga Circus library. Peer transfers
+  carry it, and the earliest known date always wins. A peer's claim can't be earlier than
+  the source upload or later than now.
+
+The chapter list shows one release date per chapter, choosing the first tier that has
+a date: `release` → `source` unless `batch` (shown as ≈) → `ecosystem` (◎) → `added` (↓).
+Each tier takes the earliest value across the chapter's versions. Existing libraries get
+these fields from `node tools/backfill-chapter-dates.js <library>`: a dry run by default,
+then `--apply --writers-stopped` and a deep Rescan.
+
 Manual directions have priority at **chapter → book → manga**. A `directionOverride`
 or a plain `direction` without matching generated provenance is manual. Otherwise,
 the scanner checks the version's existing image dimensions for coherent tall strips

docs/peering.md

@@ -41,24 +41,24 @@ separate lifetimes.
 The optional `peer-gateway` Compose profile publishes port 8093 on **127.0.0.1** by
 default. It uses the same image as reader/scraper, a read-only library mount, a
 read-only peering policy/identity mount, and its own writable usage-counter mount.
-The gateway needs an external-facing Docker network for its host port to work on
-Docker Desktop; it also joins the private backend. It has no reader account database,
+The gateway needs an external-facing Docker network for its host port to be
+published; it also joins the private backend. It has no reader account database,
 scraper queue, Docker socket, or writable library mount. It must never be scaled
 past one instance for the same usage-counter directory.
 
 1. Build the image and start the normal stack as documented in
-   [`compose-deployment.md`](compose-deployment.md). Set the local server name and
+   [`deployment.md`](deployment.md). Set the local server name and
    generate its certificate in **Settings → Admin → Peering**. Keep
    `/data/peering/identity.key` private and backed up.
 2. Start the optional service with `docker compose --profile peering up -d
    peer-gateway`. Verify `docker compose --profile peering ps` shows healthy and
    `https://127.0.0.1:8093/health` responds. `/health` is deliberately unauthenticated
    and contains only `{ "ok": true }`.
-3. Give the other server a reachable HTTPS endpoint for port 8093. A private
-   Tailscale TCP forward is recommended for the pilot; the default loopback binding
-   is not remotely reachable. Do not expose port 8093 on the public internet merely
-   to make pairing work. Use a stable tailnet DNS name, not a changing IP. Keep
-   scraper (8091) and Sentinel (8092) private.
+3. Give the other server a reachable HTTPS endpoint for port 8093. A private VPN
+   (WireGuard, Tailscale or similar) with a TCP forward is recommended; the default
+   loopback binding is not remotely reachable. Do not expose port 8093 on the public
+   internet merely to make pairing work. Use a stable DNS name, not a changing IP.
+   Keep the scraper (8091) private.
 
 The `PEER_BIND`, `PEER_PORT`, `PEER_DATA_PATH`, and `PEER_USAGE_PATH` Compose variables
 control the bind and persistent paths. The gateway's internal port remains 8093.
@@ -110,8 +110,18 @@ an admin can distinguish pairing from permission.
 | `GET /peer/v1/access` | Pairing state, certificate expiry and current grant status/scopes; no catalog |
 | `GET /peer/v1/catalog?cursor=N` | Up to 50 approved opaque title IDs, safe metadata, a stable work identity and cover availability |
 | `GET /peer/v1/covers/{mangaId}` | Approved cover image only; bounded to 5 MiB and charged to peer byte quota |
-| `GET /peer/v1/manga/{id}?cursor=N` | Up to 100 complete chapter IDs, public metadata and the same stable work identity |
-| `GET /peer/v1/manga/{id}/chapters/{id}` | Ordered page IDs, MIME, byte size, SHA-256 and ETag |
+| `GET /peer/v1/manga/{id}?cursor=N` | Up to 100 complete chapter IDs, public metadata, chapter `dates` and the same stable work identity |
+| `GET /peer/v1/manga/{id}/chapters/{id}` | Ordered page IDs, MIME, byte size, SHA-256 and ETag, plus chapter `dates` |
+
+Chapter `dates` are `{ ecosystem, source?, release? }`. `ecosystem` is the earliest
+date at which any version of that chapter number entered a Manga Circus library known
+to the sender. Older chapters fall back to their `added` date. The receiver stores the
+date as `x-dates.ecosystem` (`via: "peer <alias>"`) and only lets it lower an existing
+date, never raise it. See `docs/library-format.md`. Older peers omit `dates`, and the
+receiver then uses its own arrival time. A sync also lowers the ecosystem date of
+chapters this library already has, including ones it scraped itself, when the peer
+knows an earlier date for the same book and chapter number. The job reports these as
+`ecosystemLowered`. The reader shows the new date after its next full reconciliation.
 | `GET /peer/v1/blobs/{id}` | One image, optional `Range: bytes=N-` and `If-Match`; 206 on resume |
 
 Unknown/unshared IDs return 404. Revoked serving returns 403; bad/missing grants
@@ -122,9 +132,10 @@ not shared. Catalog and chapter lists are short-lived in-memory caches. Turning
 serving blocks subsequent requests immediately, though an already streaming image
 may finish its current chunk/response.
 
-The initial implementation sends original image bytes; it does not transcode them.
-See [`vision/server-peering-compression-test.md`](../vision/server-peering-compression-test.md)
-for the measured reason. The transport accepts a small amount of controlled page
+The initial implementation sends original image bytes; it does not transcode them:
+lossless compression of already encoded pages saved only about 6% of the transfer size
+in a measurement over 287 pages (webtoon strips about 11%, PNG chapters under 1%);
+`node tools/benchmark-peer-transfer.js` repeats it on your own library. The transport accepts a small amount of controlled page
 parallelism, not unbounded connections.
 
 ## Tests and safe operation
@@ -139,11 +150,8 @@ checks the queue remains paused, and tears both stacks down. The UI suite checks
 guided setup, approvals, and non-admin boundary. An isolated single-stack Compose
 acceptance is:
 
-```powershell
-$env:APP_IMAGE = 'mangacircus:local' # replace with the image just built
-$env:QA_PORT = '8188'
-$env:QA_PEER_PORT = '8193'
-node tools/test-compose.js --peer-gateway --peer-ui
+```bash
+APP_IMAGE=mangacircus:local QA_PORT=8188 QA_PEER_PORT=8193 node tools/test-compose.js --peer-gateway --peer-ui
 ```
 
 It makes disposable library/data, four containers, a certificate and a browser

docs/reading-limiter.md

@@ -16,7 +16,7 @@ hours**; that page is read-only.
 Set these deployment variables in the Compose `.env` file next to
 `docker-compose.yml`, then recreate **only the reader**. The `reader` service
 passes them through to the container. `READER_IMAGE` selects a reader-only
-release while `APP_IMAGE` continues to select the scraper and Sentinel image.
+release while `APP_IMAGE` continues to select the scraper image.
 Leaving `READING_WINDOWS_JSON` empty disables the limiter. Day keys
 are `0` Sunday through `6` Saturday. Each day has any number of `[start,end]`
 intervals; an end earlier than its start carries into the next day. `24:00` is

docs/resilience.md

@@ -4,19 +4,18 @@
 
 ```text
 Browser → Reader ──private API──→ Scraper → library (atomic chapter publication)
-             │                       ↑
-             └──health──→ Sentinel ───┘ bounded checkpoint recovery only
+                                     └─ stall watchdog: bounded checkpoint recovery only
 
-Supervisor(reader)   Supervisor(scraper)   Supervisor(sentinel)
+Supervisor(reader)   Supervisor(scraper)
                          ↑
            Coding agent: inspect incidents → reproduce → patch → test → release
 ```
 
 The reader is the only writer of users/progress and the catalog. The scraper is the
-only queue owner and content publisher. Sentinel owns only observations and incidents.
+only queue owner and content publisher, and it records its own failures and incidents.
 Each service's supervisor owns only that child process. The coding agent is an
 **escalation/maintenance operator**, not a necessary part of normal downloading.
-The recurring Codex check is an additional backstop, not the runtime watchdog.
+A periodic check by an operator or coding agent is an additional backstop, not the runtime watchdog.
 
 ## Automatic recovery implemented
 
@@ -28,18 +27,18 @@ The recurring Codex check is an additional backstop, not the runtime watchdog.
 | Parser/local adapter failure | Quarantine job, keep partial data, continue other titles | Two different affected jobs in an hour pause source as systemic failure |
 | Storage/permission failure | Pause all downloads and report | No automatic retry |
 | HTTP 401/403/429 or access challenge | Pause affected source lane | No automatic retry/bypass |
-| Active job with no progress for 15 minutes | Sentinel requests cooperative checkpoint-safe retry | One stall recovery; next stall quarantined |
+| Active job with no progress for 15 minutes | Scraper stall watchdog stops it at its next checkpoint and requeues it | One stall recovery; next stall quarantined |
 
-Sentinel only treats an active job as stalled when its checkpoint can be read.
-The scraper resolves source-specific work folders (including MangaPill's
-`mangapill-` prefix) before reporting progress; a missing checkpoint is not
-evidence that downloads stopped. An operator retry resets the job's stall
-recovery count without deleting its partial download.
-
-The daemon remains authoritative: a recovery request includes the observed progress
-fingerprint and is rejected if the job has moved, paused, or is no longer stale.
-Sentinel also observes completed/page counts; a deliberately paused queue is not a
-stall. Timeouts bound source fetches to 60 seconds and metadata calls to 30 seconds.
+The stall watchdog runs inside the scraper (`ScraperService.checkStalls`, every 30 s
+from the work loop). A job is stalled when its checkpoint fingerprint (completed
+chapters/pages) has not moved for 15 minutes, or for one chapter delay plus three
+minutes if that is longer. Time spent paused does not count, and a deliberately
+paused queue is not a stall. It only treats a job as stalled when its checkpoint can
+be read: the scraper resolves source-specific work folders (including MangaPill's
+`mangapill-` prefix), and a missing checkpoint is not evidence that downloads stopped.
+Each stall records an incident with `failure.kind: 'stall'` and an `events.log` entry.
+An operator retry resets the job's stall recovery count without deleting its partial
+download. Timeouts bound source fetches to 60 seconds and metadata calls to 30 seconds.
 Publication remains one complete chapter at a time, on the same filesystem.
 No recovery action raises pacing, creates another worker for the same source,
 changes identities or deletes partial images.
@@ -62,14 +61,14 @@ use an explicit operator retry after fixing their cause.
 
 ## Observability
 
-**Settings → Service health** shows Sentinel status, active checkpoints, failed jobs
-and incident history. **Settings → Scraper** retains queue, pacing and discovery controls.
+**Settings → Service health** shows scraper reachability ("Scraper: unreachable" when
+the reader cannot reach it), active checkpoints, failed jobs and the scraper's incident
+history. Process crashes are handled by the supervisor and Docker restart policy, not
+recorded as incidents. **Settings → Scraper** retains queue, pacing and discovery controls.
 Service errors are classified rather than buried in a single historical `lastError`.
-`npm run check` performs read-only checks. Sentinel's private, token-protected `/metrics`
-exports reader-up, scraper-up and open-incident gauges for a future Prometheus collector.
-It is not a public endpoint.
+`npm run check` performs read-only checks of reader and scraper.
 
-Scraper status is independent of expensive library scans. Concurrent admin/Sentinel
+Scraper status is independent of expensive library scans. Concurrent admin/health
 probes share one background inventory refresh and receive the previous summary
 immediately. The dashboard summary refreshes at most once per minute unless a
 publication or metadata change invalidates it; duplicate-sensitive decisions still
@@ -82,22 +81,23 @@ may temporarily show no library targets, but queue and rate controls remain usab
 Storage:
 
 - `data/catalog.json`, `data/collection-covers/`: reader catalog and covers.
-- `data/scraper/queue.json`, `events.log`: queue, circuits, failures and incidents.
-- `data/sentinel/state.json`: Sentinel incidents and last observation.
-- `data/runtime-reader.json`, `data/scraper/runtime.json`, `data/sentinel/runtime.json`:
-  supervisor status and persisted crash budget.
+- `data/scraper/queue.json`, `events.log`: queue, circuits, failures and incidents
+  (including stalls).
+- `data/runtime-reader.json`, `data/scraper/runtime.json`: supervisor status and
+  persisted crash budget.
+- `data/sentinel/`: only on installs from before 2026-10-22; unused, safe to delete.
 - `data/scraper/supervisor.lock`, `daemon.lock`: native process ownership; transient.
 
-Scraper/Sentinel/supervisor emit structured logs. Queue and Sentinel histories retain
-200 incidents; the UI shows a bounded subset. Docker logs are rotated. Native Windows
-supervisor logs remain append-only: archive them during stopped maintenance if needed.
+Scraper and supervisor emit structured logs. The queue history retains
+200 incidents; the UI shows a bounded subset. Docker logs are rotated. Native supervisor logs
+(`tools/run-supervisor.ps1`) remain append-only: archive them during stopped maintenance if needed.
 Never export user databases, tokens, cookies, signed query strings or unredacted environments.
 
 ## What self-repair intentionally does not mean
 
-Sentinel cannot execute a shell, patch adapters, query image sources, access a Docker
-socket, or issue unrestricted restart commands. It cannot repair broken markup or
-solve source challenges. The agent handles those escalations in the repository:
+The stall watchdog cannot execute a shell, patch adapters, access a Docker socket,
+or restart processes; it only requeues or quarantines a job. Nothing at runtime
+repairs broken markup or solves source challenges. The agent handles those escalations in the repository:
 
 1. Inspect one bounded incident and relevant checkpoint; preserve the active lane.
 2. Add a fixture reproducing the failure, patch the smallest adapter/logic surface.

docs/scraper-service.md

@@ -31,11 +31,9 @@ npm start               # reader only
 npm run start:scraper   # daemon only
 ```
 
-These commands are for native development only. The current Windows installation
-runs the three-service Docker Compose stack; the old Scheduled Task is disabled.
-Never start a native supervisor or daemon against its mounted state. Its observed
-ports, logs, persisted settings, restart procedure, and recovery snapshot are recorded in
-[`local-deployment.md`](local-deployment.md).
+These commands are for native development. Deployments run the Docker Compose stack
+([`deployment.md`](deployment.md)); never start a native supervisor or daemon against
+the state a container has mounted.
 
 | Variable | Default | Purpose |
 |---|---|---|
@@ -207,7 +205,7 @@ All responses are JSON. When `SCRAPER_TOKEN` is set, send
 | `POST` | `/api/queue/clear-completed` | `{source?}`; remove inactive successful download job rows only; preserve library, history, checkpoints and other statuses |
 | `POST` | `/api/queue/reorder` | `{ids}` containing every queue id exactly once |
 | `POST` | `/api/control` | `{action:"pause"|"resume"|"retry"|"inspect"|"confirm"|"skip",id?}`; `pause-source`/`resume-source` use `id` as the source name and do not change the global pause |
-| `POST` | `/api/settings` | `sourceRates.{adapter}.{minMs,maxMs,chapterMinMs,chapterMaxMs}` plus `autoUpdateEnabled` and local `autoUpdateTime` (`HH:MM`) |
+| `POST` | `/api/settings` | `sourceRates.{adapter}.{minMs,maxMs,chapterMinMs,chapterMaxMs}`, `sourcesEnabled.{weebcentral,mangadex,mangapill}` (booleans; WeebCentral and MangaPill default to off, except on installations whose queue already used them; a switched-off source refuses new jobs, keeps queued ones waiting and is skipped by update checks), plus `autoUpdateEnabled` and local `autoUpdateTime` (`HH:MM`) |
 | `POST` | `/api/discovery/run` | Queue an immediate update check for every enabled manga |
 | `POST` | `/api/library/auto-update` | `{id,enabled}` updates `manga.json` atomically |
 | `POST` | `/api/legacy/publish-ready` | Paused-only validation/publication of completed legacy work; converts its checkpoint to incremental mode |

docs/settings-navigation.md

@@ -59,6 +59,10 @@ download times by manga/source, and a collapsed recent-check log. Successful
 no-change nightly checks update this summary and leave the queue. Sources owns
 the per-hoster request and chapter delay settings.
 
+Service health also holds the **Backups** card: built-in backup status (last run,
+size, duration, error), **Back up now**, and the daily schedule and retention form,
+whose draft survives the page's polling. See [backup.md](backup.md).
+
 Reading-time collection controls, export and deletion live under Reading time &
 privacy; reader layout controls stay under Reading. Rescan controls and user
 administration are separate. Server & library describes existing maintenance,
@@ -69,8 +73,8 @@ not an editor for Docker environment variables or an unrestricted restart consol
 `npm test` covers route defaults, deep links, ownership and existing service/API
 behavior. `node tools/test-compose.js --settings-ui` adds optional Playwright
 headless browser QA against the disposable Compose fixture, never the real user
-database or queue. Make Playwright available on the development machine; Windows
-uses installed headless Edge by default, overridable with `QA_BROWSER_CHANNEL`.
+database or queue. Make Playwright available on the development machine; on Windows it
+uses the installed headless Edge by default, overridable with `QA_BROWSER_CHANNEL`.
 The application itself remains dependency-free.
 
 The UI check verifies desktop and 390×844 mobile navigation, form saves, draft

docs/superpowers/plans/2026-10-25-discovery-wider-pool.md

@@ -0,0 +1,337 @@
+# Discovery wider pool Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Widen Discover's candidate pool with four AniList-sourced seed kinds (community recommendations, same creator, top in tags, fresh), harvested server-side nightly and per user, scored locally.
+
+**Architecture:** New reader-side modules: a file cache, a rate-limited AniList client, anchor selection, and a single-flight harvester. The existing engine scores from Greatest + relations + harvest cache; `server.js` wires routes, the nightly scheduler and rescoring. Harvest (network) and scoring (local) stay separate, so a failed harvest never empties a feed.
+
+**Tech Stack:** Node.js built-ins only (`node:test`, `fetch`, `fs`), plain browser JS, Playwright for the Compose UI test.
+
+**Spec:** [`docs/superpowers/specs/2026-10-25-discovery-wider-pool-design.md`](../specs/2026-10-25-discovery-wider-pool-design.md)
+
+## Global Constraints
+
+- No runtime dependencies; Node built-ins only (`AGENTS.md`).
+- One writer: only the reader writes `data/discovery.json` and `data/discovery-cache/`; never write under `data/scraper/`.
+- All fetching runs in the reader container, never in the browser.
+- AniList: at most 25 requests per minute, one at a time; stop the whole harvest on 429, 403 or a challenge; never retry with a changed identity, user agent or host.
+- Budgets: nightly and admin harvests 300 requests, per-user refresh 60. Per-user cooldown 1 hour. At most 15 anchors per user.
+- Cache TTL: `title`, `recs`, `creator` 30 days; `tagTop`, `fresh` 7 days. Cap 5,000 files, oldest `fetchedAt` pruned first.
+- Feed: 20 items; at most 8 per seed kind, at most 4 per Greatest period/`Related works`; up to 4 slots reserved for `fresh`, up to 4 for candidates outside the user's top-3 tags. Candidate cap 2,000.
+- Personalized discovery and reading activity remain separate opt-ins, both off by default.
+- Discovery never edits the scraper queue, library or collections.
+- Tests never call live AniList; fixtures in `test/fixtures/anilist/` are handcrafted and contain no personal data.
+- Public repo: no host names, IPs, local paths or personal seeds anywhere.
+- Commits end with `Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>`; push `feature/discovery-wider-pool` after each task.
+
+## Review Focus
+
+1. **A fresh install with an empty cache and no network** must behave exactly like Phase A: the feed comes from Greatest and relations, and the harvest record shows a stop reason. Test in Task 6 (`engine: empty harvest cache equals Phase A pool`) and Task 4 (`harvest: network error stops with upstream-errors after 5`).
+2. **A user with no anchors** (opted in, no votes, empty library) gets no harvest requests except the global `fresh` list, and still a feed. Test in Task 4 (`plan for anchorless user only fetches fresh`).
+3. **Old `discovery.json` files without `harvest`, `settings` or `nightly` fields** must load without the fail-closed error. Test in Task 5 (`store loads a Phase A file without new fields`).
+4. **A per-user rescore** must keep every other user's feed and the candidates those feeds reference. Test in Task 6 (`mergeUserFeed keeps other users' referenced candidates`).
+5. **AniList text containing HTML or script** (titles, tag names) is shown as text, never as markup. Test in Task 8 (UI test renders a fixture title `<img src=x onerror=alert(1)>` as text).
+
+---
+
+### Task 1: Discovery cache
+
+**Files:**
+- Create: `discovery/cache.js`
+- Test: `test/discovery-cache.test.js`
+
+**Interfaces:**
+- Produces:
+  - `KINDS = ['title', 'recs', 'creator', 'tagTop', 'fresh']`
+  - `TTL_MS = { title: 30*86400000, recs: 30*86400000, creator: 30*86400000, tagTop: 7*86400000, fresh: 7*86400000 }`
+  - `class DiscoveryCache { constructor(dir: string, { maxFiles = 5000, now = Date.now } = {}) }`
+  - `get(kind, key) -> { schema: 1, kind, key, fetchedAt: ISO string, data } | null`. A corrupt or schema-mismatched file is deleted and returns `null`.
+  - `isFresh(kind, key) -> boolean`: true when an entry exists and `now() - fetchedAt < TTL_MS[kind]`.
+  - `put(kind, key, data) -> entry`: atomic write (temp file + rename), then pruning.
+  - `list(kind) -> entry[]`, including expired entries.
+  - `stats() -> { files: number, bytes: number }`
+- File path: `<dir>/<kind>/<base64url(String(key))>.json`. An unknown kind throws `TypeError`.
+
+- [ ] **Step 1: Write the failing tests** in `test/discovery-cache.test.js`, each using an `fs.mkdtempSync` dir and a fake `now`:
+  - `cache round-trips an entry`: `put('title', 42, {a:1})`, then `get('title', 42).data` deep-equals `{a:1}` and `fetchedAt` equals `new Date(now()).toISOString()`.
+  - `freshness follows TTL per kind`: with `now` advanced by 8 days, `isFresh('fresh','global')` is `false` and `isFresh('title',1)` is `true`. At 31 days `isFresh('title',1)` is `false`, while `get` still returns the entry.
+  - `prune keeps the newest maxFiles`: with `maxFiles: 3` and 5 puts at increasing `now`, `stats().files === 3` and the first two keys return `null`.
+  - `corrupt entry is deleted`: write `{broken` to the entry file; `get` returns `null` and the file no longer exists.
+  - `odd keys are file-safe`: `put('tagTop', 'genre:Slice of Life/../x', …)` lands inside `<dir>/tagTop/`.
+  - `unknown kind throws`.
+- [ ] **Step 2: Run** `node --test test/discovery-cache.test.js`. Expected: FAIL, `Cannot find module '../discovery/cache'`.
+- [ ] **Step 3: Implement `discovery/cache.js`** per the Interfaces. Prune by reading each entry's `fetchedAt` across all kinds, sorting oldest first, and deleting until `files <= maxFiles`. Follow `discovery/store.js` style.
+- [ ] **Step 4: Run** `node --test test/discovery-cache.test.js`. Expected: PASS.
+- [ ] **Step 5: Commit** `Discovery cache with per-kind TTL and size cap (BL-0051)` and push.
+
+### Task 2: AniList client
+
+**Files:**
+- Create: `discovery/anilist.js`, `test/fixtures/anilist/{title,recs,creator,tagtop,fresh,novel}.json`
+- Test: `test/discovery-anilist.test.js`
+
+**Interfaces:**
+- Produces:
+  - `createLimiter({ perMinute = 25, now = Date.now, sleep = ms => new Promise(r => setTimeout(r, ms)) }) -> { wait(): Promise<void> }`. Calls are serialized and spaced at least `60000 / perMinute` ms apart.
+  - `class AniListStop extends Error { reason: 'rate-limited' | 'forbidden' | 'challenge' }`
+  - `class AniListSkip extends Error { reason: 'timeout' | 'upstream' | 'invalid' }`
+  - `normalizeMedia(media) -> Record | null`. Returns `null` unless `format` is `MANGA` or `ONE_SHOT`.
+  - `Record = { anilistId, title, altTitles: string[], format, isAdult: boolean, status: 'ongoing'|'completed'|'unknown', year: number|null, averageScore: number|null, popularity: number|null, trending: number|null, tags: [{ name, rank, kind: 'genre'|'tag', isAdult: boolean }], authors: [{ staffId, name, role }], coverUrl: string|null, siteUrl: string|null }`.
+    - Genres map to `{ rank: 100, kind: 'genre', isAdult: false }`.
+    - Status: `RELEASING` → `ongoing`, `FINISHED` → `completed`, anything else → `unknown`.
+    - `authors` keeps the staff roles matched by the regex in `scraper/metadata/anilist.js` (`original story|story(?: & art)?|art|illustration`).
+  - `createClient({ endpoint = process.env.DISCOVERY_ANILIST_URL || 'https://graphql.anilist.co', limiter = createLimiter(), fetchImpl = fetch, timeoutMs = 30000, today = () => new Date() })` returns:
+    - `title(id) -> Record`
+    - `recs(id) -> [{ anilistId, rating, record }]`, keeping only `rating > 0`
+    - `creator(staffId) -> [{ anilistId, role, record }]`
+    - `tagTop({ name, kind }) -> [{ anilistId, record }]`
+    - `fresh() -> [{ anilistId, record }]`
+    - Entries whose `normalizeMedia` returns `null` are dropped.
+- Queries share one `MEDIA_FIELDS` fragment: `id format isAdult title{romaji english native} synonyms status startDate{year} averageScore popularity trending genres tags{name rank isAdult} staff(perPage:20){edges{role node{id name{full}}}} coverImage{extraLarge} siteUrl`. Page sizes and sort orders come from spec §2.1:
+  - `tagTop` uses `popularity_greater: 1000`, with `genre_in` or `tag_in` by `kind`.
+  - `fresh` uses `status: RELEASING, startDate_greater: <(today.year − 2) * 10000 + month * 100 + day>, sort: TRENDING_DESC, perPage: 50`.
+  - All list queries use the full fragment, so no follow-up `title` fetches are needed. This replaces the spec's follow-up stage on purpose.
+- Error mapping, applied after `await limiter.wait()`:
+  - HTTP 429 → `AniListStop('rate-limited')`
+  - HTTP 401 or 403 → `AniListStop('forbidden')`
+  - non-JSON body or `content-type` text/html → `AniListStop('challenge')`
+  - abort or timeout → `AniListSkip('timeout')`
+  - HTTP 5xx → `AniListSkip('upstream')`
+  - missing `data.Media` / `data.Page.media` / `data.Staff`, or a media without an integer `id` → `AniListSkip('invalid')`
+  - Requests send `content-type: application/json` and the fixed header `user-agent: MangaCircus-Discovery/1.0`. Do not reuse `SlowHttp` identity rotation.
+
+- [ ] **Step 1: Write fixtures** with minimal handcrafted responses:
+  - `recs.json` has 3 recommendations, one with `rating: -2` and one with `format: NOVEL`.
+  - `title.json` has `isAdult: false`, 2 genres and 2 tags, one of them `isAdult: true`.
+- [ ] **Step 2: Write the failing tests** in `test/discovery-anilist.test.js`. Use a stub `fetchImpl` returning `new Response(JSON.stringify(fixture), { status, headers })` and a fake limiter `{ wait: async () => {} }`:
+  - `normalizeMedia maps fields and drops novels`: asserts the `Record` shape on `title.json`; `normalizeMedia({ ...media, format: 'NOVEL' }) === null`.
+  - `recs keeps positive ratings and manga formats`: exactly 1 entry with its rating.
+  - `fresh query sends startDate two years back`: with `today = new Date('2026-10-25')`, the request body's variables contain `startDate_greater: 20241025`.
+  - `tagTop uses genre_in for genres and tag_in for tags`.
+  - `429 stops`, `403 stops`, `html body is a challenge`: each throws `AniListStop` with its reason.
+  - `5xx and timeout skip`, `missing data is invalid`: each throws `AniListSkip` with its reason.
+  - `limiter spaces calls 2400 ms apart`: the fake `sleep` records each duration and advances the fake `now` by it. Three `wait()` calls issued at the same instant record `[0, 2400, 2400]`.
+- [ ] **Step 3: Run** `node --test test/discovery-anilist.test.js`. Expected: FAIL (module missing).
+- [ ] **Step 4: Implement `discovery/anilist.js`** per the Interfaces.
+- [ ] **Step 5: Run** the same command. Expected: PASS.
+- [ ] **Step 6: Commit** `Rate-limited AniList client for discovery seeds (BL-0051)` and push.
+
+### Task 3: Anchors and top tags
+
+**Files:**
+- Create: `discovery/anchors.js`
+- Test: `test/discovery-anchors.test.js`
+
+**Interfaces:**
+- Consumes: `Record` from Task 2.
+- Produces:
+  - `ANCHOR_WEIGHTS = { flame: 5, good: 3, interested: 4, maybe: 2, read: 3, owned: 1 }`
+  - `MAX_ANCHORS = 15`
+  - `anchorsFor({ user, db, library, greatest }) -> [{ anilistId: number, weight, source: keyof ANCHOR_WEIGHTS, title }]`. Returns `[]` unless `user.settings.discoveryEnabled`. Sources:
+    - Greatest votes: `db.greatestRatings[user.name][entryId]` is `flame` or `good`, and `entry.metadata.id` is set.
+    - Feedback: `db.discoveryFeedback[user.name]['anilist:N']` is `interested` or `maybe`.
+    - `read`: only with `settings.discoveryReadingActivity`; a library manga with `x-anilist.id` and any chapter in `db.progress[user.name][manga.id]` that has `done`.
+    - `owned`: a library manga with `meta['x-anilist'].id`.
+    - Dedupe by id, keeping the highest weight. Sort by weight desc, then `anilistId` asc. Cap at 15.
+  - `topTagsFor(anchors, recordOf: id => Record|null, { mature = false }) -> [{ name, kind }]`, at most 3. Sums anchor weight per tag with `rank >= 60`. Skips `isAdult` tags unless `mature`. Ties are broken by name asc.
+
+- [ ] **Step 1: Write the failing tests**:
+  - `anchors are empty without opt-in`
+  - `weights, dedupe and cap`: a title voted `good` that is also owned yields one anchor with weight 3. Twenty owned titles yield 15 anchors.
+  - `reading anchors need the reading opt-in`
+  - `feedback keys parse anilist ids and ignore others`: `greatest:x` and `anilist:abc` are ignored.
+  - `top tags weigh anchors, skip low rank and adult tags`: an adult tag is included with `{ mature: true }`.
+- [ ] **Step 2: Run** `node --test test/discovery-anchors.test.js`. Expected: FAIL.
+- [ ] **Step 3: Implement `discovery/anchors.js`.**
+- [ ] **Step 4: Run** the same command. Expected: PASS.
+- [ ] **Step 5: Commit** `Discovery anchors and top tags (BL-0051)` and push.
+
+### Task 4: Harvester
+
+**Files:**
+- Create: `discovery/harvest.js`
+- Test: `test/discovery-harvest.test.js`
+
+**Interfaces:**
+- Consumes: `DiscoveryCache` (Task 1), the client and its errors (Task 2), `anchorsFor`/`topTagsFor` (Task 3). It also consumes the store methods `load()` and `patch({ harvest })` from Task 5; tests pass a stub `{ patch(){} }`, so Task 5 can land later.
+- Produces:
+  - `BUDGETS = { nightly: 300, admin: 300, user: 60 }`
+  - `REFRESH_COOLDOWN_MS = 3600000`
+  - `MAX_CONSECUTIVE_FAILURES = 5`
+  - `async runHarvest({ kind, targets: [{ user, anchors, mature }], cache, client, budget }) -> { requests, budget, status: 'success'|'stopped', stopReason: null|'budget'|'rate-limited'|'forbidden'|'challenge'|'upstream-errors' }`. Stages, in order, each skipping requests whose `cache.isFresh(kind, key)`:
+    1. `title` for every anchor.
+    2. `recs` per anchor.
+    3. `creator` per author `staffId` of anchors with `weight >= 3`, read from the cached `title` record.
+    4. `tagTop` per `topTagsFor(...)` tag, keyed `${kind}:${name}`.
+    5. One `fresh`, key `global`.
+
+    Requests are deduped across targets. Each success is `cache.put`. An `AniListSkip` increments a consecutive-failure counter that resets on success; at 5 the run returns `stopped`/`upstream-errors`. An `AniListStop` returns `stopped` with its reason immediately. Reaching `budget` returns `success`/`budget`.
+  - `shouldRunNightly({ now: Date, harvestAt: 'HH:MM', lastNightlyDate: 'YYYY-MM-DD'|null, enabled }) -> boolean`: true when enabled, local `now` is at or after `harvestAt`, and the local date differs from `lastNightlyDate`.
+  - `class Harvester`:
+    - `constructor({ cache, client, store, targetsFor: (kind, userName?) => targets, onDone: async (job, result) => void, now = Date.now })`
+    - `request({ kind: 'nightly'|'admin'|'user', user?: string }) -> { harvestId, position }`. Throws `Object.assign(new Error(msg), { status, nextAllowedAt? })` with:
+      - `409` for an admin or nightly harvest while one is running or queued, and for a user who already has one queued or running.
+      - `429` within `REFRESH_COOLDOWN_MS` of that user's last accepted request; the error carries `nextAllowedAt` as an ISO string.
+    - `state(userName) -> { state: 'idle'|'queued'|'running', position?: number, nextAllowedAt?: string }`
+    - `current() -> { id, kind, user?, startedAt } | null`
+    - Jobs run one at a time, FIFO. Each run calls `store.patch({ harvest: { id, kind, user, status: 'running', startedAt, requests: 0, budget } })`, then the final record (`status`, `finishedAt`, `requests`, `stopReason`), then `onDone(job, result)`. Errors thrown in `onDone` are logged and do not break the queue.
+
+- [ ] **Step 1: Write the failing tests** using a fake client (methods resolve from fixture maps, count calls, and can throw scripted errors) and a real `DiscoveryCache` in a temp dir:
+  - `plan order and dedupe`: two targets sharing an anchor. The call log starts with `title` calls, then `recs`, `creator`, `tagTop`, `fresh`; the shared anchor is fetched once.
+  - `fresh cache entries are skipped`
+  - `budget stops with reason budget`: budget 3 → `requests === 3`, `stopReason: 'budget'`.
+  - `429 stops at once`: the call count does not grow after the throw.
+  - `network error stops with upstream-errors after 5`: five consecutive `AniListSkip`; a success in between resets the counter.
+  - `plan for anchorless user only fetches fresh`
+  - `shouldRunNightly`: 03:29 → false; 03:30 → true; same date → false; disabled → false.
+  - `harvester single-flight, positions, 409 and 429 cooldown`: with fake `now`, a second user request inside one hour throws `429` with `nextAllowedAt`. An admin request while one is running throws `409`. A queued user's `state()` returns `queued` with `position` 1.
+  - `onDone failure does not stall the queue`
+- [ ] **Step 2: Run** `node --test test/discovery-harvest.test.js`. Expected: FAIL.
+- [ ] **Step 3: Implement `discovery/harvest.js`.**
+- [ ] **Step 4: Run** the same command. Expected: PASS.
+- [ ] **Step 5: Commit** `Single-flight discovery harvester with budgets and stop reasons (BL-0051)` and push.
+
+### Task 5: Store metadata
+
+**Files:**
+- Modify: `discovery/store.js`
+- Test: `test/discovery.test.js` (add tests)
+
+**Interfaces:**
+- Produces:
+  - `load()` defaults `harvest: null`, `settings: null`, `nightly: { lastDate: null }` when the fields are absent. The validation of existing fields is unchanged.
+  - `publish(value)` keeps the prior `harvest`, `settings` and `nightly`.
+  - `patch({ harvest?, settings?, nightly? }) -> data` writes atomically through the same temp+`.prev`+rename path, leaves `revision` unchanged, and refuses on a corrupt file like `publish`.
+  - Extract a private `write(data)` shared by `publish` and `patch`.
+
+- [ ] **Step 1: Write the failing tests**:
+  - `store loads a Phase A file without new fields`: write a Phase A-shaped JSON; `load().nightly.lastDate === null`.
+  - `patch keeps revision and publish keeps harvest`: `patch({ harvest: { status: 'success' } })`, then `publish(...)`; `load().harvest.status === 'success'` and the revision went up by exactly 1.
+  - `patch refuses a corrupt store`
+- [ ] **Step 2: Run** `node --test test/discovery.test.js`. Expected: the new tests FAIL.
+- [ ] **Step 3: Implement** the store changes.
+- [ ] **Step 4: Run** `npm run test:reader`. Expected: PASS.
+- [ ] **Step 5: Commit** `Discovery store keeps harvest, settings and nightly state (BL-0051)` and push.
+
+### Task 6: Engine — wider pool, new scores, slot mix
+
+**Files:**
+- Modify: `discovery/engine.js`, `discovery/cli.js` (pass `harvestCache` when `data/discovery-cache` exists)
+- Test: `test/discovery.test.js` (update the cap assertion; add tests), `test/fixtures/anilist/` reused through a temp `DiscoveryCache`
+
+**Interfaces:**
+- Consumes: `DiscoveryCache` (Task 1), `Record` (Task 2), `anchorsFor`/`topTagsFor` (Task 3).
+- Produces:
+  - `MAX_CANDIDATES = 2000`, `MAX_FEED = 20`
+  - `SLOTS = { perKind: 8, perPeriod: 4, fresh: 4, outside: 4 }`
+  - `buildCandidates({ greatest, library, cacheDir, harvestCache? })`: adds one candidate per distinct `anilistId` found in `recs`, `creator`, `tagTop` and `fresh` entries. New candidate fields: `isAdult`, `averageScore`, `tagRecords: Record.tags`, `authorRecords: Record.authors`, `fetchedAt`. `tags` and `authors` stay name arrays. Greatest candidates keep their fields; when a harvested record has the same id, it fills `isAdult` and `tagRecords`. Era bucketing stays, and harvested candidates bucket by their seed kind instead of `Related works`.
+  - `feedForUser({ user, db, candidates, greatest, library, queuedTitles, harvestCache? })` adds these parts, with values from spec §4, rounded to 2 decimals:
+    - `communityRec = min(5, 1 + log2(1 + rating) / 2) * anchorWeight / 5`, using the best (anchor, rating) pair among the user's anchors whose `recs` list contains the candidate.
+    - `sameCreator = 4 * anchorWeight / 5`, when the candidate's `authorRecords` shares a `staffId` with an anchor of `weight >= 3`.
+    - `tagTop = 3 | 2 | 1` for the user's 1st, 2nd or 3rd top tag, only if `averageScore >= 75` and the candidate is in that tag's `tagTop` list.
+    - `fresh = 2` if `status === 'ongoing'` and `year >= today.year - 2`; `1` if only present in the `fresh` list.
+  - Mature filter: exclude unless `settings.discoveryMature` when `isAdult === true` or any `tagRecords` entry has `isAdult && rank >= 60`. Use the Phase A regex only when `isAdult` is `undefined`.
+  - Each row gets `group`. If the row's largest part is one of the four new parts, `group` is its kind (`recs`, `creator`, `tagTop`, `fresh`); otherwise it is `period || 'Related works'`.
+  - `reason` comes from the largest part. `reasons: string[]` lists every applicable reason in descending part order. Exact templates:
+    - recs: `AniList readers who liked ${anchor} recommend this (${rating}).`
+    - creator: `Also by ${author} — you ${read|liked} ${anchor}.` Use `read` when the anchor source is `read`, otherwise `liked`.
+    - tagTop: `Top-rated in ${tag}, one of your most-liked tags.`
+    - fresh: `New: started ${year}, ongoing and trending.`
+  - Evidence ids: `rec:<anchorId>:<candidateId>`, `creator:<staffId>:<candidateId>`, `tagtop:<kind>:<name>:<candidateId>`, `fresh:<candidateId>`. Kinds: `recommendation|creator|tag-top|fresh`. Each carries `url: record.siteUrl` and `fetchedAt`.
+  - `mixSlots(rows, { topTags }) -> rows` (at most 20, ranked):
+    1. Sort by score desc, then `candidateId`.
+    2. Take up to 4 rows with `scoreParts.fresh > 0`.
+    3. Take up to 4 rows whose `tags` intersect none of the `topTags` names.
+    4. Fill greedily.
+    - Steps 2–4 all respect the caps: `perKind` 8 for the four kinds and `perPeriod` 4 for other groups.
+    - Never pad.
+    - The final order is by score desc, then `rank = index + 1`.
+  - `async generateAsync({ db, library, greatest, cacheDir, harvestCache, queuedTitles, onlyUser? }) -> same shape as generate()`. Awaits `new Promise(setImmediate)` between users; with `onlyUser`, scores only that user. `generate()` stays synchronous for `cli.js` and the existing tests.
+  - `mergeUserFeed(prior: { candidates, feeds }, result, userName) -> { candidates, feeds }`: replaces `feeds[userName]`, keeps every other feed, and keeps exactly the candidates referenced by any kept feed plus `result.candidates`.
+
+- [ ] **Step 1: Update and write the failing tests**:
+  - Change `assert.equal(Object.keys(result.candidates).length, 50)` to `Math.min(2000, examined)`; keep the other Phase A assertions.
+  - `engine: empty harvest cache equals Phase A pool`: with and without an empty `harvestCache`, the candidate ids are equal.
+  - `community rec, creator, tagTop and fresh parts and reasons`: fixture cache plus a user with a `flame` anchor. Asserts exact part values, for example: rating 42, weight 5 → `min(5, 1 + log2(43)/2) = 3.71`. Also asserts the exact reason strings and evidence kinds with `fetchedAt`.
+  - `isAdult filter`: an `isAdult` candidate is absent without the mature opt-in and present with it.
+  - `slot mix`: 30 `recs`-group rows → at most 8 from that kind. Fresh rows are present even when low-scored. Rows outside the top tags are present. Fewer than 20 eligible rows → no padding.
+  - `multiple seeds merge reasons`: one candidate found by recs and creator has both reasons and both evidence entries.
+  - `owned, queued, voted, dismissed still excluded` for harvested candidates.
+  - `mergeUserFeed keeps other users' referenced candidates`
+  - `scoring one user stays under 200 ms on 2000 candidates`: synthetic records; measure `feedForUser` with `performance.now()`.
+- [ ] **Step 2: Run** `node --test test/discovery.test.js`. Expected: the new tests FAIL.
+- [ ] **Step 3: Implement** the engine changes and the `cli.js` pass-through.
+- [ ] **Step 4: Run** `npm run test:reader`. Expected: PASS.
+- [ ] **Step 5: Commit** `Discovery scores AniList seeds with slot mix and isAdult filter (BL-0051)` and push.
+
+### Task 7: Server wiring
+
+**Files:**
+- Modify: `server.js` (around the discovery block at lines 240–267, the routes at 490–517, `HttpError` at 81, the error response at 974, and the timers near 980–1012), `docker-compose.yml` (reader `environment`: `DISCOVERY_HARVEST_AT: ${DISCOVERY_HARVEST_AT:-03:30}`, `DISCOVERY_ANILIST_URL: ${DISCOVERY_ANILIST_URL:-}`), `docs/architecture.md`
+- Test: `test/discovery-server.test.js` (spawns `server.js` on a free port with a temp `DATA`/`LIBRARY` and `DISCOVERY_ANILIST_URL` pointing to an in-test `http.createServer` serving fixtures). Copy the helper pattern from an existing server-spawning test in `test/` if one exists; otherwise write a minimal spawn/wait-for-port helper inside this file.
+
+**Interfaces:**
+- Consumes: Tasks 1–6.
+- Produces:
+  - `HttpError(code, msg, extra = {})`; the error response becomes `{ error, ...extra }`.
+  - `DISCOVERY_HARVEST_CACHE = path.join(DATA, 'discovery-cache')`
+  - `harvestAt` default: `process.env.DISCOVERY_HARVEST_AT` if it matches `/^([01]\d|2[0-3]):[0-5]\d$/`, else `'03:30'`. `store.settings` overrides it.
+  - `async rescore({ onlyUser })`: refactors the body of `startDiscoveryRun`. With `onlyUser` it calls `generateAsync({ onlyUser })` then `mergeUserFeed` before `publish`; otherwise it runs a full `generateAsync`. `startDiscoveryRun` keeps its route behavior.
+  - `targetsFor(kind, userName)`: all opted-in users, or only `userName` for `kind === 'user'`. Each target is `{ user, anchors: anchorsFor(...), mature: !!settings.discoveryMature }`.
+  - Startup: if `store.load().harvest?.status === 'running'`, `patch` it to `interrupted` with `finishedAt`. Queued jobs are not persisted.
+  - Scheduler: `setInterval(checkNightly, 60000).unref()`, plus one call 30 s after startup. When `shouldRunNightly` is true and no harvest is active, call `patch({ nightly: { lastDate } })` and then `harvester.request({ kind: 'nightly' })`.
+  - Routes:
+    - `POST /api/discovery/refresh`: caller only; `403` without opt-in. Returns `{ harvestId, position }`, `409`, or `429` with `{ nextAllowedAt }`.
+    - `GET /api/discovery`: adds `refresh: harvester.state(u.name)`.
+    - `POST /api/admin/discovery/harvest`: admin only; returns `{ harvestId, position }` or `409`.
+    - `GET /api/admin/discovery`: adds `harvest`, `currentHarvest`, `cache: harvestCache.stats()`, `settings: { harvestEnabled, harvestAt }`, `nextNightlyAt`.
+    - `POST /api/admin/discovery/settings`: admin only. Accepts exactly `{ harvestEnabled: boolean, harvestAt: 'HH:MM' }`; anything else → `400`. Saved via `patch({ settings })`.
+
+- [ ] **Step 1: Write the failing integration tests**:
+  - `refresh requires opt-in, queues, runs and rescores only the caller`: two opted-in users. Alice refreshes, and her feed afterwards contains a fixture rec candidate. Bob's `generatedAt` is unchanged.
+  - `second refresh within an hour is 429 with nextAllowedAt`
+  - `admin harvest is admin-only and single-flight`: `403` for a non-admin; `409` on a second call while one is running (the fixture server delays its responses by 500 ms).
+  - `settings validation`: `harvestAt: '25:00'` → `400`; an extra key → `400`; a valid body round-trips through `GET /api/admin/discovery`.
+  - `interrupted harvest is marked on startup`: pre-write a `discovery.json` with `harvest.status: 'running'`, start the server, and get `interrupted`.
+  - `harvest never touches scraper queue or library`: no files change under the temp `LIBRARY` or `DATA/scraper`. Snapshot the directory listings and mtimes before and after.
+- [ ] **Step 2: Run** `node --test test/discovery-server.test.js`. Expected: FAIL.
+- [ ] **Step 3: Implement** the server changes, the compose env lines, and the new routes in `docs/architecture.md`.
+- [ ] **Step 4: Run** `npm test`. Expected: all suites PASS.
+- [ ] **Step 5: Commit** `Discovery harvest routes, nightly schedule and per-user refresh (BL-0051)` and push.
+
+### Task 8: Discover UI, browser test and docs
+
+**Files:**
+- Modify: `public/discover.js`, `public/discover.css` if present (else the stylesheet `discover.html` links), `tools/test-discovery-ui.js`, `tools/test-compose.js` (pass `DISCOVERY_ANILIST_URL=http://127.0.0.1:9/` for `--discovery-ui`, so a harvest fails fast without reaching AniList), `docs/discovery.md`, `docs/deployment.md`, `vision/discovery-scout.md`
+
+**Interfaces:**
+- Consumes: Task 7 routes and response fields.
+- UI, using the existing `h()`/`button()` helpers. Never use `innerHTML`; text goes in through `h()` children.
+  - **Refresh my suggestions** button, shown when `state.enabled`. While `refresh.state` is `queued` it reads `Queued (position N)`, while `running` it reads `Fetching…`, and both disable it. During a cooldown it shows `Next refresh possible at <local HH:MM>` (from `nextAllowedAt`) and is disabled. It polls every 1.5 s while queued or running, like `run()`.
+  - Card: show `item.reason`; the details list shows all `item.reasons` and evidence, plus ` · fetched <local date>` when `entry.fetchedAt` is set.
+  - Settings hint for **Personalized discovery**, exact text: `Off by default. When on, the server sends AniList the IDs of titles you liked, without login, to find related work. Your feed and feedback stay private to this server.`
+  - The mature hint becomes `Only affects titles AniList marks as adult or with a known mature tag.`
+  - Admin panel:
+    - adds a line `Last harvest: <status> · <requests>/<budget> requests · <stopReason or —> · cache <files> files`, plus the next nightly time
+    - adds a **Harvest now** button
+    - adds an **Automatic harvest** checkbox and a time `<input type="time">` that save through the settings route
+    - replaces the old "This first version…" paragraph with: `Harvests ask AniList for public metadata only. They never ask image hosters, call an LLM, or edit the scraper queue.`
+- Docs:
+  - `docs/discovery.md`: replace the deliberate-limits section with the seed kinds, budgets, cache, nightly time, privacy note and stop reasons.
+  - `docs/deployment.md`: `data/discovery-cache/` is safe to delete and can be left out of backups; document `DISCOVERY_HARVEST_AT` and `DISCOVERY_ANILIST_URL` (test only).
+  - `vision/discovery-scout.md` §4: mark the candidate input as implemented, with a link to the spec.
+
+- [ ] **Step 1: Extend `tools/test-discovery-ui.js`**, after the existing Maybe click:
+  - Click **Refresh my suggestions**; wait until the button text matches `/Next refresh possible at \d{1,2}:\d{2}/`.
+  - The admin panel shows `Last harvest: stopped` and `upstream-errors`.
+  - Turn on **Automatic harvest**, set the time to `04:15`, reload, and see `04:15`.
+  - **Harvest now** while one is running shows the 409 message in the status note.
+  - Review Focus 5: before the stack starts, seed the disposable data tree with a `discovery.json` whose candidate title is `<img src=x onerror=alert(1)>` and which is in the QA admin's feed. Assert that the card's `h2` `textContent` equals that string and that `h2 img` count is 0.
+  - The existing non-admin checks additionally assert `Harvest now` count 0 and `POST /api/admin/discovery/harvest` → 403.
+  - Mobile viewport (375×812): the refresh button is visible without horizontal scroll (`document.documentElement.scrollWidth <= 375`).
+- [ ] **Step 2: Run** `docker build -t mangacircus:discovery-test .` then `APP_IMAGE=mangacircus:discovery-test npm run test:discovery-ui`. Expected: FAIL on the missing button.
+- [ ] **Step 3: Implement** the UI and docs.
+- [ ] **Step 4: Run** the same two commands, then `npm test`. Expected: PASS.
+- [ ] **Step 5: Commit** `Discover page: refresh, harvest admin and seed reasons; docs (BL-0051)` and push.
+- [ ] **Step 6: Update the project's coordination backlog item and release the claim.** Merging into `main` waits for the user.

docs/superpowers/specs/2026-10-25-discovery-wider-pool-design.md

@@ -0,0 +1,306 @@
+# Discovery: wider candidate pool — design
+
+Status: implemented on branch feature/discovery-wider-pool. Date: 2026-10-25. Builds on the
+deterministic Discovery Scout ([`docs/discovery.md`](../../discovery.md),
+[`vision/discovery-scout.md`](../../../vision/discovery-scout.md)).
+
+## 1. Goal
+
+Today Discover can only suggest titles from the curated Greatest catalog plus
+AniList *relations* (sequels, spin-offs) of Greatest and library titles, and only
+when those are already in the local metadata cache. A run makes no network call.
+Users run out of new suggestions quickly.
+
+This change widens the candidate pool with four seed kinds, fetched server-side
+from AniList:
+
+1. **Community recommendations** — "readers who liked X recommend Y" for the
+   user's anchor titles.
+2. **Same creator** — other works by the story/art authors of anchor titles.
+3. **Top in your tags** — well-rated titles in the user's strongest genres/tags.
+4. **Fresh** — recently started or currently releasing, trending titles.
+
+Success: suggestions the user did not already know, each with a traceable
+reason and evidence link, without exceeding AniList's rate limits or making the
+reader noticeably slower.
+
+### Unchanged rules from Phase A
+
+- Personalized discovery and reading activity stay separate opt-ins, both off
+  by default; feedback stays private to the account.
+- Discovery never edits the scraper queue, library or collections and never
+  downloads anything.
+- Every suggestion carries evidence; missing data stays missing.
+- The reader is the only writer of discovery state; no new Compose service, no
+  database, no LLM.
+- All fetching runs in the reader container, never in the browser.
+- The repository is public: no host names, paths or personal seeds in code,
+  docs or fixtures.
+
+### Non-goals
+
+- No other metadata provider (AniList's API covers all four seed kinds).
+- No LLM rationale or rerank (vision Phase C).
+- No edition finding or queueing from a suggestion (vision Phase B).
+- No worker container; it is only a fallback if section 7's latency check fails.
+
+## 2. Components
+
+| Unit | Responsibility | Depends on |
+|---|---|---|
+| `discovery/anilist.js` (new) | GraphQL queries for the four seed kinds and for a title by ID; one shared rate limiter; schema check and normalization of responses | `fetch`, `discovery/cache.js` |
+| `discovery/cache.js` (new) | `data/discovery-cache/`: atomic writes, TTL per entry kind, size cap with oldest-first pruning | filesystem |
+| `discovery/harvest.js` (new) | Anchor selection per user, request plan, budgets, single-flight run state, stop reasons | `anilist.js`, `cache.js`, `db` (read-only) |
+| `discovery/engine.js` (changed) | Builds candidates from Greatest, relations **and the harvest cache**; new score parts, evidence kinds, slot mix, `isAdult` filter | `cache.js` (read-only) |
+| `server.js` (changed) | Nightly scheduler, admin "Harvest now", per-user "Refresh my suggestions", admin status fields; route handlers stay thin | `harvest.js`, `engine.js`, `discovery/store.js` |
+| `public/discover.*` (changed) | Refresh button states, new reason labels, fetch date in evidence, admin harvest panel | reader API |
+
+The existing `discovery/store.js` (atomic `discovery.json`, `.prev` recovery,
+fail-closed on corruption) is reused unchanged. The reader keeps reading the
+scraper's AniList cache read-only for relations; the harvester never writes into
+`data/scraper/`.
+
+### 2.1 `discovery/anilist.js`
+
+Queries (all `type: MANGA`, unauthenticated, `https://graphql.anilist.co`):
+
+| Kind | Query | Page size |
+|---|---|---|
+| `title` | `Media(id)` with the fields of `scraper/metadata/anilist.js` plus `isAdult averageScore popularity trending tags { name rank isAdult } recommendations staff` | 1 |
+| `recs` | `Media(id).recommendations(sort: RATING_DESC)` → `mediaRecommendation` + `rating` | 25 |
+| `creator` | `Staff(id).staffMedia(type: MANGA, sort: POPULARITY_DESC)` | 25 |
+| `tagTop` | `Page.media(tag_in or genre_in: [one tag], sort: SCORE_DESC, popularity_greater: 1000)` | 25 |
+| `fresh` | `Page.media(status: RELEASING, startDate_greater: <today − 2 years>, sort: TRENDING_DESC)` | 50 |
+
+- **Rate limiter:** at most 25 requests per minute, one at a time, shared by all
+  harvests in the process. AniList allows more, so this stays below its limit
+  with headroom.
+- **Normalized record:** `{ anilistId, title, altTitles, format, isAdult,
+  status, year, averageScore, popularity, trending, tags[{name, rank}],
+  authors[{staffId, name, role}], coverUrl, siteUrl, fetchedAt }`.
+  Each list response becomes a list of `{ anilistId, rating? }` plus one record
+  per entry it contains.
+- **Formats:** only `MANGA` and `ONE_SHOT` are kept; novels and anime are dropped.
+- **Response check:** responses are checked against a schema before they are
+  cached; an invalid one counts as a failed request. Titles, descriptions and
+  tags are treated as untrusted text: stored as data, escaped when shown.
+
+### 2.2 `discovery/cache.js`
+
+- **Location:** `data/discovery-cache/<kind>/<key>.json`. Every entry has
+  `{ schema: 1, kind, key, fetchedAt, data }`.
+- **Lifetime:** `title` and `recs` entries live 30 days, `creator` 30 days,
+  `tagTop` and `fresh` 7 days. An expired entry is still used for scoring until
+  it is refreshed. Only the harvest checks the TTL.
+- **Size cap:** 5,000 files. On write, the oldest `fetchedAt` entries are pruned
+  first.
+- **Writes:** temporary file then rename, as in the existing stores. A corrupt
+  entry is ignored and deleted.
+
+### 2.3 `discovery/harvest.js`
+
+**Anchors per opted-in user.** Each anchor has a weight; the list is
+deduplicated by AniList ID and capped at the 15 strongest:
+
+| Source | Weight |
+|---|---|
+| Greatest vote `flame` | 5 |
+| Greatest vote `good` | 3 |
+| Discover feedback `interested` | 4 |
+| Discover feedback `maybe` | 2 |
+| Completed chapter in a library title (only with reading activity opted in) | 3 |
+| Owned library title with an AniList ID | 1 |
+
+**User's top tags.** The 3 tags with the highest summed anchor weight, using
+the anchors' cached `title` records (tag rank ≥ 60, no adult tags unless the
+user enabled mature titles).
+
+**Request plan, in this order:**
+
+1. `title` for anchors that are not cached.
+2. `recs` for each anchor.
+3. `creator` for the story/art staff of anchors weighted 3 or more.
+4. `tagTop` for each top tag.
+5. One global `fresh` list.
+
+Requests whose cache entry is still within its TTL are skipped. Then follow-up
+`title` requests run for new candidates that are missing tags or `isAdult`, up
+to the remaining budget.
+
+**Budgets:** nightly harvest 300 requests, per-user refresh 60 requests. When
+the budget runs out, the plan stops; whatever was cached so far is still used.
+
+**Run state:** one harvest at a time (single-flight).
+
+- A queued per-user refresh waits behind a running harvest; a second refresh
+  for the same user while one is queued returns `409`.
+- The run record is persisted in `discovery.json` `harvest`:
+  `{ id, kind: nightly|admin|user, user?, status, startedAt, finishedAt,
+  requests, budget, stopReason }`.
+
+## 3. Data flow
+
+```
+anchors ─▶ request plan ─▶ AniList (rate-limited) ─▶ discovery-cache
+                                                          │
+Greatest + scraper AniList cache (relations) ─────────────┤
+                                                          ▼
+                              engine.generate() ─▶ discovery.json (atomic publish)
+```
+
+Harvest and scoring are separate steps. Scoring is local and never touches the
+network, so a failed harvest means older candidates, never an empty or broken
+feed.
+
+- **After a nightly or admin harvest:** re-score every opted-in user.
+- **After a per-user refresh:** re-score only that user. The other users' feed
+  snapshots are carried over unchanged into the same publish.
+
+## 4. Scoring
+
+`engine.generate()` builds the candidate map from Greatest, relations, and every
+record referenced by the harvest cache, limited to 2,000 candidates; the
+replaced Phase A cap was 50. Exclusions stay as in Phase A:
+
+- titles the user already voted on in Greatest
+- titles dismissed or hidden in Discover
+- owned titles, matched by ID or by normalized title + year
+- queued titles
+
+The mature filter now uses `isAdult` (title-level, or an adult tag with
+rank ≥ 60). The Phase A tag regex is kept only for candidates with no `isAdult`
+data.
+
+New score parts, all bounded:
+
+| Part | Value | Rule |
+|---|---|---|
+| `communityRec` | 0–5 | `min(5, 1 + log2(1 + rating) / 2) × anchorWeight / 5`; strongest anchor counts |
+| `sameCreator` | 0–4 | `4 × anchorWeight / 5` for a shared story/art author with an anchor weighted ≥ 3 |
+| `tagTop` | 0–3 | `3` for the user's top tag, `2` second, `1` third, only if `averageScore ≥ 75` |
+| `fresh` | 0–2 | `2` if releasing and started within 2 years, `1` if only trending |
+
+Existing parts (`editorial`, `relation`, `sharedTaste`, `era`, `reading`,
+`interest`) and their weights are unchanged.
+
+**Slot mix** for the 20 feed slots, applied greedily by descending score:
+
+- At most 8 items whose strongest part comes from the same seed kind.
+- Up to 4 slots reserved for `fresh` candidates when such candidates exist.
+- About 20% (4 slots) taken from candidates outside the user's strongest tag
+  cluster, as in the vision doc.
+- If a rule cannot be met, the slot falls back to the next best candidate; the
+  feed is never padded.
+
+**Reasons.** A candidate found by several seeds collects every reason and
+evidence item. The tile shows the reason of the strongest part; the detail
+panel lists all of them. Reason templates:
+
+| Kind | Template |
+|---|---|
+| `recs` | "AniList readers who liked {anchor} recommend this ({rating})." |
+| `creator` | "Also by {author} — you {read/liked} {anchor}." |
+| `tagTop` | "Top-rated in {tag}, one of your most-liked tags." |
+| `fresh` | "New: started {year}, ongoing and trending." |
+
+Evidence entries use `kind: recommendation|creator|tag-top|fresh`, link to the
+AniList record, and carry `fetchedAt`.
+
+## 5. API and UI
+
+Routes:
+
+- `POST /api/discovery/refresh` — caller only, requires opt-in. Returns
+  `{ harvestId, position }`. Returns `429` with `nextAllowedAt` within 1 hour of
+  the caller's last refresh, and `409` if one is already queued for the caller.
+- `GET /api/discovery` — adds
+  `refresh: { state: idle|queued|running, position?, nextAllowedAt? }` and
+  `fetchedAt` on evidence.
+- `POST /api/admin/discovery/harvest` — admin only; starts a full harvest.
+  Returns `409` while one runs.
+- `GET /api/admin/discovery` — adds the last harvest record, cache file count
+  and size, and the next nightly time.
+- `POST /api/admin/discovery/settings` (new) — `{ harvestEnabled: boolean,
+  harvestAt: "HH:MM" }`. Defaults: `true` and `03:30`, overridable by
+  `DISCOVERY_HARVEST_AT`. The value is validated; nothing else is accepted.
+
+UI on `discover.html`:
+
+- **Refresh my suggestions** button with three states: idle, queued/fetching
+  (with queue position), and cooldown ("next refresh possible at 14:20").
+- Reason labels per section 4; evidence shows "fetched {date}".
+- The settings text for **Personalized discovery** says that the server sends
+  AniList the IDs of titles you liked, without login, to find related work.
+- Admin panel: last harvest (time, requests / budget, stop reason), cache size,
+  next nightly run, **Harvest now**, enable switch and time.
+
+## 6. Failure handling
+
+| Condition | Behaviour |
+|---|---|
+| HTTP 429, 403, or a challenge page | Stop the harvest at once with `stopReason`, shown in the admin panel. No retry until the next nightly slot or admin action; never change identity, user agent or host. |
+| Timeout or 5xx | Skip that request and count it. Five in a row abort the run (`stopReason: upstream-errors`). A timeout never counts as an empty result. |
+| Invalid response | Dropped, counted as a failed request, not cached. |
+| Restart during a harvest | On startup the run is marked `interrupted`. If the nightly slot was missed, one harvest runs after startup; there is never a catch-up storm. A queued per-user refresh is dropped and the user can press the button again. |
+| Corrupt `discovery.json` | Unchanged from Phase A: reads and runs fail closed; the file is never overwritten. |
+| No network | Harvest fails with a stop reason; scoring uses whatever is cached. A fresh install behaves exactly like Phase A. |
+| Account deletion | Unchanged (feedback removed). Cache entries are not user-specific and stay. |
+
+## 7. Performance
+
+- Harvesting is I/O-bound and single-flight. At 25 requests per minute, a full
+  nightly harvest takes about 12 minutes.
+- Scoring 2,000 candidates for each user must keep the reader event loop
+  responsive. Each user is scored in its own `setImmediate` turn, and a test
+  asserts that one user's scoring stays below 200 ms on the fixture set.
+- If production shows reader stalls, moving the job to a worker container is
+  recorded as a backlog item; it is not part of this change.
+
+## 8. Testing
+
+All network tests use recorded, anonymized AniList fixtures; no test calls the
+live API.
+
+**Unit:**
+
+- query builders and response normalization
+- schema rejection of malformed responses
+- dropping non-manga formats
+- the rate limiter, with a fake clock
+- stop on 429/403/challenge, and the consecutive-error abort
+- cache TTL, size pruning, and corrupt-entry handling
+- anchor weights, cap and dedup; reading activity used only with opt-in
+- top-tag selection
+- each new score part
+- slot-mix limits and fallback
+- merging reasons from several seeds
+- `isAdult` filter
+- exclusion of owned, queued, voted and dismissed titles
+
+**Integration:**
+
+- per-user refresh: queue position, `429` cooldown, `409` duplicate
+- admin harvest single-flight
+- re-scoring after a per-user refresh leaves other users' feeds unchanged
+- interrupted-run recovery
+- one missed-night run after startup
+- no harvest or score ever changes the scraper queue or library
+- settings validation
+
+**Browser** (existing disposable Compose suite):
+
+- refresh button states
+- reason labels for each seed kind
+- evidence fetch date
+- admin harvest panel
+- desktop and mobile layout
+
+## 9. Documentation
+
+- `docs/discovery.md`: seed kinds, budgets, cache, nightly time, privacy note,
+  stop reasons.
+- `docs/deployment.md`: `data/discovery-cache/` (safe to delete, rebuilt by the
+  next harvest; may be left out of backups) and `DISCOVERY_HARVEST_AT`.
+- `docs/architecture.md`: the new routes.
+- `vision/discovery-scout.md`: mark the candidate-input part of section 4 as
+  implemented.

greatest/covers.js

@@ -0,0 +1,73 @@
+'use strict';
+
+// Local copies of the Greatest catalog's cover images. Hotlinking ~100 covers per page
+// load makes the image CDN drop some of them at random; one polite download each fixes that.
+const fs = require('node:fs');
+const path = require('node:path');
+const crypto = require('node:crypto');
+
+const TYPES = { 'image/jpeg': '.jpg', 'image/png': '.png', 'image/webp': '.webp', 'image/gif': '.gif', 'image/avif': '.avif' };
+const EXTENSIONS = new Set(Object.values(TYPES));
+const USER_AGENT = 'MangaCircus-Greatest/1.0';
+
+class GreatestCovers {
+  constructor(dir, { fetchImpl = fetch, sleep = ms => new Promise(resolve => setTimeout(resolve, ms)),
+    delayMs = 1000, maxBytes = 2 * 1024 * 1024, timeoutMs = 20000 } = {}) {
+    this.dir = path.resolve(dir);
+    Object.assign(this, { fetchImpl, sleep, delayMs, maxBytes, timeoutMs });
+    this.files = new Map(); // key -> file name
+    try {
+      for (const name of fs.readdirSync(this.dir)) {
+        const ext = path.extname(name);
+        if (EXTENSIONS.has(ext)) this.files.set(path.basename(name, ext), name);
+      }
+    } catch (error) { if (error.code !== 'ENOENT') throw error; }
+  }
+  // A hash of the URL names the file, so a changed catalog URL is fetched again.
+  version(url) { return crypto.createHash('sha256').update(String(url)).digest('hex').slice(0, 16); }
+  file(url) {
+    const name = url ? this.files.get(this.version(url)) : null;
+    return name ? path.join(this.dir, name) : null;
+  }
+  // Sequential, paced, and stops on 403/429 like every other outbound client here.
+  async warm(urls) {
+    const result = { fetched: 0, skipped: 0, failed: 0, stopped: null };
+    let requested = false;
+    for (const url of new Set(urls.filter(url => /^https:\/\/\S+$/i.test(String(url || ''))))) {
+      if (this.file(url)) { result.skipped++; continue; }
+      if (requested) await this.sleep(this.delayMs);
+      requested = true;
+      try {
+        await this.download(url);
+        result.fetched++;
+      } catch (error) {
+        if (error.stop) { result.stopped = error.message; break; }
+        result.failed++;
+      }
+    }
+    // Duplicates and repeats of stored covers count as skipped.
+    result.skipped += urls.filter(Boolean).length - new Set(urls.filter(Boolean)).size;
+    return result;
+  }
+  async download(url) {
+    const response = await this.fetchImpl(url, { headers: { 'user-agent': USER_AGENT },
+      signal: AbortSignal.timeout(this.timeoutMs) });
+    if ([401, 403, 429].includes(response.status)) throw Object.assign(new Error(`HTTP ${response.status}`), { stop: true });
+    if (!response.ok) throw new Error(`HTTP ${response.status}`);
+    const ext = TYPES[String(response.headers.get('content-type') || '').split(';')[0].trim().toLowerCase()];
+    if (!ext) throw new Error('Not an image');
+    if (Number(response.headers.get('content-length')) > this.maxBytes) throw new Error('Image too large');
+    const body = Buffer.from(await response.arrayBuffer());
+    if (!body.length || body.length > this.maxBytes) throw new Error('Image too large or empty');
+    const key = this.version(url), name = key + ext;
+    fs.mkdirSync(this.dir, { recursive: true });
+    const temp = path.join(this.dir, `.${key}.${crypto.randomBytes(4).toString('hex')}.tmp`);
+    try {
+      fs.writeFileSync(temp, body);
+      fs.renameSync(temp, path.join(this.dir, name));
+    } finally { fs.rmSync(temp, { force: true }); }
+    this.files.set(key, name);
+  }
+}
+
+module.exports = { GreatestCovers };

library.js

@@ -5,6 +5,8 @@
  *
  * Only folder names, small JSON files and the first bytes of each image (for its size) are read.
  * Image sizes are cached per chapter folder and only re-read when the folder's mtime changes.
+ * Uncached chapters and their images are probed SCAN_CONCURRENCY (default 8) at a time: on a NAS a
+ * first scan is bound by per-file latency, not bandwidth, so a few requests in flight roughly halve it.
  */
 const fsp = require('fs').promises, path = require('path');
 const { parseSpecialFolder, parseKey } = require('./chapter-identity');
@@ -15,6 +17,25 @@ const COVER = /^cover\.(jpe?g|png|webp|gif|avif|bmp)$/i;
 const cmp = new Intl.Collator('en', { numeric: true, sensitivity: 'base' }).compare;
 const str = x => (typeof x === 'string' ? x.trim() : '');
 const order = (a, b) => Number(!!a.special) - Number(!!b.special) || a.x - b.x || a.num - b.num || cmp(a.key, b.key);
+const SCAN_CONCURRENCY = Math.min(64, Math.max(1, Math.floor(+process.env.SCAN_CONCURRENCY) || 8));
+
+// Runs at most n of the given async tasks at once, in submission order.
+function limiter(n) {
+  let active = 0;
+  const waiting = [];
+  const next = () => {
+    if (active >= n || !waiting.length) return;
+    active++;
+    waiting.shift()();
+  };
+  return task => new Promise((resolve, reject) => {
+    waiting.push(() => Promise.resolve().then(task).then(resolve, reject).finally(() => { active--; next(); }));
+    next();
+  });
+}
+// One pair of limits per scan: chapters (stat, chapter.json, listing) and image probes.
+// Chapter tasks wait for image slots but never hold one, so the two cannot deadlock.
+const scanLimits = (concurrency = SCAN_CONCURRENCY) => ({ chapters: limiter(concurrency), images: limiter(concurrency) });
 
 async function readJson(file) {
   try { return JSON.parse((await fsp.readFile(file, 'utf8')).replace(/^/, '')); }
@@ -60,16 +81,24 @@ function sizeFromBuffer(b) {
   return null;
 }
 
+// Most headers fit in the first few KB, so read 32 KB first and only then up to 256 KB.
+// sizeFromBuffer() on a prefix is either null or the same answer as on the whole file,
+// so the smaller first read never changes a result.
+const HEAD_SMALL = 32 * 1024, HEAD_LARGE = 256 * 1024;
 async function imageSize(file) {
   const fh = await fsp.open(file, 'r');
   let head;
   try {
-    const buf = Buffer.alloc(256 * 1024);
-    const { bytesRead } = await fh.read(buf, 0, buf.length, 0);
-    head = buf.subarray(0, bytesRead);
+    const buf = Buffer.alloc(HEAD_LARGE);
+    let n = (await fh.read(buf, 0, HEAD_SMALL, 0)).bytesRead;
+    head = buf.subarray(0, n);
+    const size = sizeFromBuffer(head);
+    if (size || n < HEAD_SMALL) return size;
+    n += (await fh.read(buf, n, HEAD_LARGE - n, n)).bytesRead;
+    head = buf.subarray(0, n);
   } finally { await fh.close(); }
   // JPEGs with big EXIF/ICC blocks can have their size marker further in.
-  return sizeFromBuffer(head) || (head.length === 256 * 1024 ? sizeFromBuffer(await fsp.readFile(file)) : null);
+  return sizeFromBuffer(head) || (head.length === HEAD_LARGE ? sizeFromBuffer(await fsp.readFile(file)) : null);
 }
 
 function pageInfo([f, w, h]) {
@@ -78,17 +107,16 @@ function pageInfo([f, w, h]) {
   return { f, w, h, t, r: rot ? +rot.slice(3) : 0 };
 }
 
-async function scanVersion(root, rel, ck, oldCache, cache) {
+async function scanVersion(root, rel, ck, oldCache, cache, limits) {
   const dir = path.join(root, rel), st = await fsp.stat(dir);
   const meta = await readJson(path.join(dir, 'chapter.json'));
   let c = oldCache[rel];
   if (!c || c.m !== st.mtimeMs) {
-    const p = [];
-    for (const f of (await list(dir)).files) {
-      if (!IMG.test(f)) continue;
+    const files = (await list(dir)).files.filter(f => IMG.test(f));
+    const p = await Promise.all(files.map(f => limits.images(async () => {
       const s = await imageSize(path.join(dir, f)).catch(() => null);
-      p.push([f, s ? s.w : 0, s ? s.h : 0]);
-    }
+      return [f, s ? s.w : 0, s ? s.h : 0];
+    })));
     c = { m: st.mtimeMs, p };
   }
   cache[rel] = c;
@@ -104,7 +132,7 @@ async function scanVersion(root, rel, ck, oldCache, cache) {
 const thumbOf = v => { const p = v.pages.find(p => p.t.includes('cover')) || v.pages[0]; return v.rel + '/' + p.f; };
 const coverIn = (files, rel) => { const f = files.find(f => COVER.test(f)); return f ? rel + '/' + f : null; };
 
-async function scanManga(root, name, oldCache, cache) {
+async function scanManga(root, name, oldCache, cache, limits = scanLimits()) {
   const { dirs, files } = await list(path.join(root, name));
   const meta = await readJson(path.join(root, name, 'manga.json'));
   const m = { id: name, title: str(meta.title) || name, meta, direction: resolveReadingDirection([meta]).value, cover: coverIn(files, name), books: [], chapters: [], chapterById: new Map(), updated: 0 };
@@ -120,11 +148,17 @@ async function scanManga(root, name, oldCache, cache) {
       key: bk.key, num: bk.num, x: bk.x, extra: bk.x || bMeta.extra === true,
       label: (bk.x ? 'Extra book ' : 'Book ') + bk.num, title: str(bMeta.title) || bk.title, cover: coverIn(bl.files, bRel), chapters: [],
     };
-    const byKey = new Map();
+    const byKey = new Map(), folders = [];
     for (const cName of bl.dirs) {
       const ck = parseSpecialFolder(cName) || parseKey(cName);
       if (!ck) { console.warn(`[scan] ignoring "${bRel}/${cName}": chapter folders must be named <letters><number>[ title], e.g. "c0001 The Seed"`); continue; }
-      const v = await scanVersion(root, bRel + '/' + cName, ck, oldCache, cache);
+      folders.push({ cName, ck });
+    }
+    // Probe the book's chapters concurrently, then assemble them in folder order as before.
+    const versions = await Promise.all(folders.map(({ cName, ck }) =>
+      limits.chapters(() => scanVersion(root, bRel + '/' + cName, ck, oldCache, cache, limits))));
+    for (const [i, { ck }] of folders.entries()) {
+      const v = versions[i];
       if (!v.pages.length) continue;
       v.directionInfo = resolveReadingDirection([v.meta, bMeta, meta], v.pages);
       v.direction = v.directionInfo.value;
@@ -151,11 +185,12 @@ async function scanManga(root, name, oldCache, cache) {
 }
 
 // Returns { index, cache }. Pass the previous cache to skip re-reading unchanged chapter folders.
-async function scanLibrary(root, oldCache = {}) {
-  const cache = {}, mangas = [];
+// options.concurrency overrides SCAN_CONCURRENCY (tests, benchmarks).
+async function scanLibrary(root, oldCache = {}, options = {}) {
+  const cache = {}, mangas = [], limits = scanLimits(options.concurrency);
   for (const name of (await list(root)).dirs) {
     try {
-      const m = await scanManga(root, name, oldCache, cache);
+      const m = await scanManga(root, name, oldCache, cache, limits);
       if (m.chapters.length) mangas.push(m);
       else console.warn(`[scan] "${name}" has no chapters (expected <manga>/<book>/<chapter>/<images>)`);
     } catch (e) { console.warn(`[scan] skipped "${name}": ${e.message}`); }
@@ -185,8 +220,9 @@ async function libraryStamps(root) {
   return stamps;
 }
 
-async function scanChangedLibrary(root, prior, oldCache, oldStamps, stamps) {
+async function scanChangedLibrary(root, prior, oldCache, oldStamps, stamps, options = {}) {
   stamps = stamps || await libraryStamps(root);
+  const limits = scanLimits(options.concurrency);
   const names = new Set([...Object.keys(oldStamps || {}), ...Object.keys(stamps)]);
   const changed = [...names].filter(name => JSON.stringify(oldStamps?.[name]) !== JSON.stringify(stamps[name]));
   if (!changed.length) return { index: prior, cache: oldCache, stamps, changed };
@@ -196,7 +232,7 @@ async function scanChangedLibrary(root, prior, oldCache, oldStamps, stamps) {
     byId.delete(name);
     for (const key of Object.keys(cache)) if (key.startsWith(name + '/')) delete cache[key];
     if (!Object.hasOwn(stamps, name)) continue;
-    const m = await scanManga(root, name, oldCache, cache);
+    const m = await scanManga(root, name, oldCache, cache, limits);
     if (m.chapters.length) byId.set(name, m);
   }
   const mangas = [...byId.values()].sort((a, b) => cmp(a.title, b.title));
@@ -232,4 +268,4 @@ function restoreIndex(mangas) {
 }
 
 module.exports = { scanLibrary, scanChangedLibrary, libraryStamps, snapshotIndex, restoreIndex,
-  parseKey, sizeFromBuffer, imageSize };
+  parseKey, sizeFromBuffer, imageSize, limiter };

package.json

@@ -2,6 +2,7 @@
   "name": "mangacircus",
   "version": "1.0.0",
   "private": true,
+  "license": "MIT",
   "description": "No-frills self-hosted manga reader with no runtime dependencies.",
   "main": "server.js",
   "scripts": {
@@ -9,11 +10,10 @@
     "start:scraper": "node scraper/daemon.js",
     "start:peer": "node peering/gateway.js",
     "start:all": "node tools/start-all.js",
-    "start:sentinel": "node sentinel/daemon.js",
     "check": "node tools/check.js",
     "check:compose": "node tools/compose-control.js check",
-    "test:runtime": "node --test sentinel/test/*.test.js",
-    "test:compose": "node tools/test-compose.js",
+    "test:runtime": "node --test runtime/test/*.test.js",
+    "test:compose":"node tools/test-compose.js",
     "test:ui": "node tools/test-compose.js --settings-ui --queue-ui --collections-ui --reading-ui",
     "test:direction-ui": "node tools/test-compose.js --direction-ui",
     "test:discovery-ui": "node tools/test-compose.js --discovery-ui",

peering/adapter.js

@@ -12,6 +12,8 @@ const { publishManifest } = require('../scraper/lib/publisher');
 const { writeJsonAtomic } = require('../scraper/lib/json-file');
 const { sleep } = require('../scraper/lib/delay');
 const { normalizeWorkIdentity, sameWorkIdentity, resolveWorkIdentity } = require('../scraper/lib/work-identity');
+const { fromPeer, publishedDates } = require('../chapter-dates');
+const { parseKey, parseSpecialFolder } = require('../chapter-identity');
 
 const dataRoot = () => path.resolve(process.env.PEER_DATA || path.join(process.env.DATA || 'data', 'peering'));
 const store = () => new PeeringState(dataRoot());
@@ -27,7 +29,37 @@ const infoFor = value => {
 };
 function refsOf(result) {
   return result.chapters.map(ref => ({ id: ref.id, number: ref.number, volume: ref.volume,
-    book: { number: Number(ref.volume) || 1 }, title: ref.title, pages: ref.pageCount }));
+    book: { number: Number(ref.volume) || 1 }, title: ref.title, pages: ref.pageCount,
+    ...(ref.dates && typeof ref.dates === 'object' ? { dates: ref.dates } : {}) }));
+}
+// Chapters this library already has adopt a peer's earlier ecosystem date (BL-0049): the
+// chapter entered Manga Circus there first. Later claims never raise a date.
+async function lowerEcosystemDates(mangaPath, refs, alias) {
+  const claims = new Map();
+  for (const ref of refs) {
+    const claim = fromPeer(ref.dates, alias).ecosystem;
+    const key = `${ref.volume}/${ref.number}`;
+    if (claim && (!claims.has(key) || Date.parse(claim.at) < Date.parse(claims.get(key).at))) claims.set(key, claim);
+  }
+  let lowered = 0;
+  if (!claims.size) return lowered;
+  for (const book of await fs.readdir(mangaPath, { withFileTypes: true })) {
+    const bookKey = book.isDirectory() && parseKey(book.name)?.key;
+    if (!bookKey) continue;
+    for (const chapter of await fs.readdir(path.join(mangaPath, book.name), { withFileTypes: true })) {
+      const parsed = chapter.isDirectory() && (parseSpecialFolder(chapter.name) || parseKey(chapter.name));
+      const claim = parsed && claims.get(`${bookKey}/${parsed.key}`);
+      if (!claim) continue;
+      const file = path.join(mangaPath, book.name, chapter.name, 'chapter.json');
+      let meta;
+      try { meta = JSON.parse(await fs.readFile(file, 'utf8')); } catch { continue; }
+      const next = publishedDates(meta, { ecosystem: claim });
+      if (next['x-dates'].ecosystem.at === meta['x-dates']?.ecosystem?.at) continue;
+      await writeJsonAtomic(file, { ...meta, ...next });
+      lowered++;
+    }
+  }
+  return lowered;
 }
 function sameTitle(left, right) {
   return String(left || '').normalize('NFKC').trim().replace(/\s+/g, ' ').toLowerCase()
@@ -101,6 +133,7 @@ async function run(info, item, options) {
     await writeJsonAtomic(path.join(manga.path, 'manga.json'), manga.meta);
   }
   const selected = selectByScope(catalog.chapters.filter(ref => shouldDownload(manga, ref, source, item.existingPolicy)), item.scope);
+  const ecosystemLowered = manga ? await lowerEcosystemDates(manga.path, catalog.chapters, peer.alias) : 0;
   job.total = selected.length; job.jobCompleted = 0; job.current = null; await record();
   let publishedCount = 0;
   const identityToPersist = localIdentity || workIdentity;
@@ -132,6 +165,8 @@ async function run(info, item, options) {
     }
     const publication = await publishManifest({ version: 1, series, chapters: [{
       number: ref.number, title: ref.title, book: { number: Number(ref.volume) || 1 },
+      // The peer's ecosystem date travels along: the chapter entered Circus there, not here.
+      dates: fromPeer(chapter.dates || ref.dates, peer.alias),
       source: { name: `Peer ${peer.alias} ${peer.id.slice(1, 7)}`, url: sourceUrl },
       pages: pages.map(page => ({ path: page.path })),
     }] }, { libraryRoot: options.libraryRoot, ...(manga ? { mangaDir: manga.path } : {}) });
@@ -147,7 +182,7 @@ async function run(info, item, options) {
     if (index < selected.length - 1) { stop(); await sleep(Math.max(500, Number(options.settings.chapterMinMs || 500))); }
   }
   job.status = 'complete'; job.current = null; await record();
-  return { finalPath: manga?.path || null, publishedCount, selectedCount: selected.length };
+  return { finalPath: manga?.path || null, publishedCount, selectedCount: selected.length, ecosystemLowered };
 }
 function adapterForPeer(value) {
   const info = infoFor(value);

peering/gateway.js

@@ -9,6 +9,7 @@ const crypto = require('node:crypto');
 const { once } = require('node:events');
 const { PeeringState, fp, hash, requirePeer, grantStatus } = require('./state');
 const { parseKey, parseSpecialFolder } = require('../chapter-identity');
+const { peerDates } = require('../chapter-dates');
 const { resolveWorkIdentity } = require('../scraper/lib/work-identity');
 
 const DATA = path.resolve(process.env.PEER_DATA || path.join(process.env.DATA || 'data', 'peering'));
@@ -116,12 +117,23 @@ async function mangaIndex(peer, share) {
       if (!pages.length) continue;
       const id = `c${opaque(`${share.id}/${bookName}/${chapterName}`)}`;
       const chapterMeta = await jsonFile(path.join(chapterDir, 'chapter.json'));
+      const dirStat = await fsp.stat(chapterDir);
       const item = { id, number: parsed.key, volume: book.key, title: String(chapterMeta.title || parsed.title || '').slice(0, 200),
-        pageCount: pages.length, chapterDir, pages };
+        pageCount: pages.length, chapterDir, pages, dates: peerDates(chapterMeta, dirStat.mtimeMs) };
       chapters.push(item); byId.set(id, item);
       if (chapters.length > 10000) fail(413, 'Shared manga chapter limit exceeded');
     }
   }
+  // Every version of a chapter number offers the earliest ecosystem date known for it.
+  const earliest = new Map();
+  for (const item of chapters) {
+    const key = `${item.volume}/${item.number}`, at = item.dates.ecosystem;
+    if (at && (!earliest.has(key) || Date.parse(at) < Date.parse(earliest.get(key)))) earliest.set(key, at);
+  }
+  for (const item of chapters) {
+    const at = earliest.get(`${item.volume}/${item.number}`);
+    if (at) item.dates = { ...item.dates, ecosystem: at };
+  }
   const result = { at: Date.now(), mangaDir, meta, chapters, byId };
   indexCache.set(key, result);
   return result;
@@ -147,7 +159,7 @@ async function chapterManifest(peer, share, chapterId) {
       size: stat.size, mtimeMs: stat.mtimeMs, at: Date.now() });
     if (pages.length > 500) fail(413, 'Chapter page limit exceeded');
   }
-  return { id: chapter.id, number: chapter.number, volume: chapter.volume, title: chapter.title, pages };
+  return { id: chapter.id, number: chapter.number, volume: chapter.volume, title: chapter.title, dates: chapter.dates, pages };
 }
 async function streamBlob(req, res, peer, id) {
   const item = blobCache.get(`${peer.id}:${id}`);
@@ -248,7 +260,7 @@ const server = https.createServer({ key: identity.key, cert: identity.cert, requ
       return json(res, 200, { id: share.id, title: String(index.meta.title || share.mangaId).slice(0, 200),
         workIdentity: resolveWorkIdentity(index.meta, share.mangaId, identity.id),
         ...publicMetadata(index.meta), coverAvailable: !!coverFile(index.mangaDir),
-        chapters: index.chapters.slice(cursor, cursor + 100).map(({ id, number, volume, title, pageCount }) => ({ id, number, volume, title, pageCount })),
+        chapters: index.chapters.slice(cursor, cursor + 100).map(({ id, number, volume, title, pageCount, dates }) => ({ id, number, volume, title, pageCount, dates })),
         nextCursor: cursor + 100 < index.chapters.length ? String(cursor + 100) : null });
     }
     const chapterMatch = /^\/peer\/v1\/manga\/(m[0-9a-f]{24})\/chapters\/(c[0-9a-f]{32})$/.exec(url.pathname);

peering/test/peering.test.js

@@ -68,7 +68,8 @@ test('mTLS pairing, opt-in catalog, peer queue publication, and revocation', { t
   await fsp.mkdir(chapterDir, { recursive: true });
   await fsp.writeFile(path.join(mangaDir, 'manga.json'), JSON.stringify({ title: 'Sample Manga', direction: 'ltr' }));
   await fsp.writeFile(path.join(mangaDir, 'cover.png'), PNG);
-  await fsp.writeFile(path.join(chapterDir, 'chapter.json'), JSON.stringify({ title: 'The first chapter', source: { name: 'Local' } }));
+  await fsp.writeFile(path.join(chapterDir, 'chapter.json'), JSON.stringify({ title: 'The first chapter', source: { name: 'Local' },
+    added: '2026-10-01T08:00:00.000Z', 'x-dates': { version: 1, source: { at: '2026-09-30T15:00:00.000Z', name: 'WeebCentral', batch: false } } }));
   await fsp.writeFile(path.join(chapterDir, '0001.png'), PNG);
   const ga = await gateway(dataA, libA, portA, path.join(root, 'usage-a.json'));
   let gb = await gateway(dataB, libB, portB, path.join(root, 'usage-b.json'));
@@ -134,10 +135,22 @@ test('mTLS pairing, opt-in catalog, peer queue publication, and revocation', { t
   assert.equal(result.publishedCount, 1);
   assert.deepEqual(JSON.parse(await fsp.readFile(path.join(libB, 'Sample Manga', 'manga.json'), 'utf8'))['x-work-identity'], remote.workIdentity,
     'The stable work identity is stored with the imported peer edition');
-  assert.equal(fs.existsSync(path.join(libB, 'Sample Manga', 'v01', 'c0001 The first chapter [Peer ServerA ' + inB.id.slice(1, 7) + ']', '0001.png')), true);
+  const receivedDir = path.join(libB, 'Sample Manga', 'v01', 'c0001 The first chapter [Peer ServerA ' + inB.id.slice(1, 7) + ']');
+  assert.equal(fs.existsSync(path.join(receivedDir, '0001.png')), true);
+  const received = JSON.parse(await fsp.readFile(path.join(receivedDir, 'chapter.json'), 'utf8'));
+  assert.deepEqual(received['x-dates'].ecosystem, { at: '2026-10-01T08:00:00.000Z', via: 'peer ServerA' },
+    'The chapter keeps the date it first entered Manga Circus on A, not its arrival on B');
+  assert.equal(received['x-dates'].source.at, '2026-09-30T15:00:00.000Z');
+  assert.notEqual(received.added, '2026-10-01T08:00:00.000Z', '"added" is the arrival in B\'s own library');
+  // B claims it got the chapter later on its own: the next sync lowers it to A's date again.
+  await fsp.writeFile(path.join(receivedDir, 'chapter.json'), JSON.stringify({ ...received,
+    'x-dates': { ...received['x-dates'], ecosystem: { at: '2026-10-20T00:00:00.000Z', via: 'this server' } } }));
   const second = await run(info, { scope: { mode: 'missing' }, existingPolicy: 'skip-any' },
     { libraryRoot: libB, settings: { chapterMinMs: 500 }, shouldStop: () => false });
   assert.equal(second.publishedCount, 0);
+  assert.equal(second.ecosystemLowered, 1, 'An existing chapter adopts the peer\'s earlier ecosystem date');
+  assert.deepEqual(JSON.parse(await fsp.readFile(path.join(receivedDir, 'chapter.json'), 'utf8'))['x-dates'].ecosystem,
+    { at: '2026-10-01T08:00:00.000Z', via: 'peer ServerA' });
 
   // Simulate B adding one chapter to its imported edition, then sharing it back.
   // A must identify its original folder by the carried origin identity and fetch

public/app.css

@@ -180,6 +180,11 @@ main { max-width: 1200px; margin: 0 auto; padding: 16px; }
 .ch { display: flex; width: 100%; max-width: 100%; align-items: center; gap: 6px; border-bottom: 1px solid var(--line); white-space: nowrap; }
 .ch-main { flex: 1 1 auto; min-width: 0; display: grid; grid-template-columns: 76px 52px minmax(70px, 1fr) max-content; gap: 8px; align-items: baseline; padding: 6px 4px; text-decoration: none; }
 .ch-main.has-time { grid-template-columns: 76px 52px 72px minmax(70px, 1fr) max-content; }
+.ch-main.has-date { grid-template-columns: 76px 52px 104px minmax(70px, 1fr) max-content; }
+.ch-main.has-time.has-date { grid-template-columns: 76px 52px 104px 72px minmax(70px, 1fr) max-content; }
+.ch-date { min-width: 0; overflow: hidden; color: var(--muted); font-size: 13px; font-variant-numeric: tabular-nums; text-overflow: ellipsis; white-space: nowrap; }
+.ch-date.tier-official { color: inherit; }
+.ch-date.tier-ecosystem, .ch-date.tier-library { opacity: .75; }
 .ch-main:hover .ch-title { text-decoration: underline; }
 .ch-label { min-width: 0; overflow: hidden; text-overflow: ellipsis; font-weight: 650; font-variant-numeric: tabular-nums; }
 .ch-pages { min-width: 0; overflow: hidden; color: var(--muted); font-size: 13px; font-variant-numeric: tabular-nums; text-align: left; white-space: nowrap; }
@@ -205,6 +210,10 @@ main { max-width: 1200px; margin: 0 auto; padding: 16px; }
   .ch { flex-wrap: nowrap; gap: 4px; overflow: hidden; }
   .ch-main { grid-template-columns: 64px 44px minmax(0, 1fr) max-content; gap: 6px; }
   .ch-main.has-time { grid-template-columns: 64px 44px 62px minmax(0, 1fr) max-content; }
+  /* Phones: the date takes the page-count column. */
+  .ch-main.has-date { grid-template-columns: 64px 64px minmax(0, 1fr) max-content; }
+  .ch-main.has-time.has-date { grid-template-columns: 64px 64px 62px minmax(0, 1fr) max-content; }
+  .ch-main.has-date .ch-pages { display: none; }
   .ch > .btn.small { min-width: 0; padding-left: 5px; padding-right: 5px; }
   .ch > .state { flex: none; }
 }
@@ -309,6 +318,10 @@ main { max-width: 1200px; margin: 0 auto; padding: 16px; }
 .discovery-card-body details { margin: 10px 0; }
 .discovery-card-body ul { padding-left: 18px; }
 .discovery-actions { margin: 10px 0; }
+.discovery-refresh-row { margin: 0 0 14px; }
+/* The paused and cooldown labels are long; let them wrap on phones instead of widening the page. */
+.discovery-refresh { max-width: 100%; white-space: normal; text-align: center; }
+.discovery-harvest-settings { align-items: center; gap: 12px; margin: 10px 0; }
 .catalog-workspace { min-width: 0; margin: 0; padding: 0; border: 0; }
 .catalog-workspace > .row { margin-bottom: 16px; }
 .catalog-workspace input, .catalog-workspace select, .catalog-workspace textarea { min-width: 0; max-width: 100%; }

public/discover.js

@@ -5,13 +5,25 @@
   topNav(me);
   const main = $('main'), note = h('p', { role: 'status', 'aria-live': 'polite', class: 'muted small' });
   const button = (label, action, className = '') => h('button', { type: 'button', class: `btn ${className}`, onclick: action }, label);
+  const clock = iso => new Date(iso).toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' });
+  const fail = error => { note.textContent = error.message; note.className = 'error'; };
   async function refresh() {
     try { state = await api('api/discovery'); if (me.admin) adminState = await api('api/admin/discovery'); render(); }
-    catch (error) { note.textContent = error.message; note.className = 'error'; }
+    catch (error) { fail(error); }
+  }
+  // Something is still working server-side: a scoring run, a harvest, or this user's refresh.
+  const busy = () => adminState?.currentRun?.status === 'running' || !!adminState?.currentHarvest
+    || ['queued', 'running'].includes(state.refresh?.state);
+  function watch() {
+    if (active || !busy()) return;
+    active = setInterval(async () => {
+      await refresh();
+      if (!busy()) { clearInterval(active); active = null; }
+    }, 1500);
   }
   async function preference(key, checked) {
     try { await api('api/settings', { [key]: checked }); await refresh(); }
-    catch (error) { note.textContent = error.message; note.className = 'error'; }
+    catch (error) { fail(error); }
   }
   async function feedback(candidateId, value) {
     try {
@@ -19,19 +31,27 @@
       state.feedback[candidateId] = value;
       note.textContent = value ? 'Preference saved. The next run will use it.' : 'Preference removed.';
       render();
-    } catch (error) { note.textContent = error.message; note.className = 'error'; }
+    } catch (error) { fail(error); }
   }
-  async function run() {
+  // Start a server job, then poll until it finishes; on refusal re-read the state (cooldown, pause).
+  async function start(endpoint, message) {
     try {
-      await api('api/admin/discovery/run', {});
-      note.textContent = 'Discovery running. Existing suggestions remain visible until it finishes.';
-      await refresh();
-      if (active) clearInterval(active);
-      active = setInterval(async () => {
-        await refresh();
-        if (adminState?.currentRun?.status !== 'running') { clearInterval(active); active = null; }
-      }, 1500);
-    } catch (error) { note.textContent = error.message; note.className = 'error'; }
+      await api(endpoint, {});
+      note.textContent = message; note.className = 'muted small';
+    } catch (error) { fail(error); }
+    await refresh();
+    watch();
+  }
+  const run = () => start('api/admin/discovery/run', 'Discovery running. Existing suggestions remain visible until it finishes.');
+  const refreshMine = () => start('api/discovery/refresh', 'Refresh queued. Your current suggestions stay visible until it finishes.');
+  const harvestNow = () => start('api/admin/discovery/harvest', 'Harvest started. Suggestions are re-scored when it finishes.');
+  function refreshButton() {
+    const refreshState = state.refresh || { state: 'idle' };
+    const label = refreshState.state === 'queued' ? `Queued (position ${refreshState.position})`
+      : refreshState.state === 'running' ? 'Fetching…'
+        : refreshState.blockedReason ? `Paused by AniList (${refreshState.blockedReason}) until the next nightly or admin harvest`
+          : refreshState.nextAllowedAt ? `Next refresh possible at ${clock(refreshState.nextAllowedAt)}` : null;
+    return h('button', { type: 'button', class: 'btn discovery-refresh', disabled: !!label, onclick: refreshMine }, label || 'Refresh my suggestions');
   }
   function toggle(label, key, hint) {
     return h('label', { class: 'check' }, h('input', { type: 'checkbox', checked: !!state.settings[key],
@@ -40,18 +60,37 @@
   }
   function preferences() {
     return h('section', { class: 'card discovery-settings' }, h('h2', {}, 'Your discovery preferences'),
-      toggle('Personalized discovery', 'discoveryEnabled', 'Off by default. Your feed and feedback stay private to this server.'),
+      toggle('Personalized discovery', 'discoveryEnabled', 'Off by default. When on, the server sends AniList the IDs of titles you liked, read or own, their authors\' IDs and your top tags, without login, to find related work. Your feed and feedback stay private to this server.'),
       toggle('Use reading activity', 'discoveryReadingActivity', 'Separately opt in to using completed chapters as a small taste signal.'),
-      toggle('Include mature titles', 'discoveryMature', 'Only affects titles with a known mature tag.'));
+      toggle('Include mature titles', 'discoveryMature', 'Only affects titles AniList marks as adult or with a known mature tag.'));
+  }
+  function harvestSettings() {
+    const settings = adminState?.settings || { harvestEnabled: false, harvestAt: '03:30' };
+    const enabled = h('input', { type: 'checkbox', checked: settings.harvestEnabled, onchange: save });
+    const time = h('input', { type: 'time', value: settings.harvestAt, 'aria-label': 'Harvest time', required: true, onchange: save });
+    async function save() {
+      if (!time.value) return;
+      try {
+        const saved = await api('api/admin/discovery/settings', { harvestEnabled: enabled.checked, harvestAt: time.value });
+        note.textContent = `Harvest settings saved (${saved.settings.harvestEnabled ? 'on' : 'off'}, ${saved.settings.harvestAt})`;
+        note.className = 'muted small';
+        await refresh();
+      } catch (error) { fail(error); }
+    }
+    return h('div', { class: 'row discovery-harvest-settings' }, h('label', { class: 'check' }, enabled, h('span', {}, 'Automatic harvest')), time);
   }
   function adminPanel() {
     if (!me.admin) return null;
     const runState = adminState?.currentRun?.status || adminState?.lastRun?.status || 'never';
+    const harvest = adminState?.harvest, cache = adminState?.cache;
     return h('section', { class: 'card discovery-admin' }, h('h2', {}, 'Discovery job'),
       h('p', { class: 'muted small' }, `Status: ${runState} · ${adminState?.candidateCount || 0} candidates · ${adminState?.optedInUsers || 0} opted-in users`),
+      h('p', { class: 'muted small' }, `Last harvest: ${harvest?.status || 'never'} · ${harvest?.requests ?? 0}/${harvest?.budget ?? 0} requests · ${harvest?.stopReason || '—'} · cache ${cache?.files ?? 0} files`),
+      h('p', { class: 'muted small' }, adminState?.nextNightlyAt ? `Next nightly harvest: ${new Date(adminState.nextNightlyAt).toLocaleString()}` : 'Automatic harvest is off.'),
       adminState?.lastError ? h('p', { class: 'error' }, adminState.lastError) : null,
-      button('Run discovery now', run, 'primary'),
-      h('p', { class: 'muted small' }, 'This first version uses cached public metadata and the curated catalog. It does not ask image hosters, call an LLM, or edit the scraper queue.'));
+      h('div', { class: 'row' }, button('Run discovery now', run, 'primary'), button('Harvest now', harvestNow)),
+      harvestSettings(),
+      h('p', { class: 'muted small' }, 'Harvests ask AniList for public metadata only. They never ask image hosters, call an LLM, or edit the scraper queue.'));
   }
   function card(item) {
     const candidate = item.candidate, vote = state.feedback[candidate.id] || null;
@@ -64,8 +103,10 @@
         h('p', { class: 'muted small' }, [candidate.year || 'Year unknown', candidate.status || 'Status unknown', candidate.period].filter(Boolean).join(' · ')),
         h('p', {}, item.reason),
         h('details', {}, h('summary', {}, 'Why this is here · evidence'),
-          h('ul', {}, evidence.map(entry => h('li', {}, entry.url ? h('a', { href: entry.url, target: '_blank', rel: 'noopener noreferrer' }, entry.label) : entry.label,
-            entry.note ? h('span', { class: 'muted' }, ` — ${entry.note}`) : null))),
+          h('ul', {}, (item.reasons || []).map(reason => h('li', {}, reason)),
+            evidence.map(entry => h('li', {}, entry.url ? h('a', { href: entry.url, target: '_blank', rel: 'noopener noreferrer' }, entry.label) : entry.label,
+              entry.note ? h('span', { class: 'muted' }, ` — ${entry.note}`) : null,
+              entry.fetchedAt ? h('span', { class: 'muted' }, ` · fetched ${new Date(entry.fetchedAt).toLocaleDateString()}`) : null))),
           h('p', { class: 'muted small' }, 'A recommendation does not verify a particular edition or English chapter availability.')),
         h('div', { class: 'row discovery-actions' }, votes.map(([value, label]) => button(label, () => feedback(candidate.id, vote === value ? null : value), vote === value ? 'on' : ''))),
         candidate.metadataUrl ? h('a', { class: 'small', href: candidate.metadataUrl, target: '_blank', rel: 'noopener noreferrer' }, 'Public metadata ↗') : null));
@@ -75,10 +116,12 @@
     main.append(h('h1', {}, 'Discover'), h('p', { class: 'muted' }, 'A small, evidence-backed shelf of manga you might like. Suggestions are not downloads.'), note,
       preferences(), adminPanel());
     if (!state.enabled) return main.append(h('p', { class: 'muted' }, 'Turn on personalized discovery to see your suggestions.'));
+    main.append(h('div', { class: 'row discovery-refresh-row' }, refreshButton()));
     if (!state.items.length) return main.append(h('p', { class: 'muted' }, state.lastRun?.status === 'never'
       ? 'No discovery run yet. An admin can run one above.' : 'No current suggestions passed the filters. Your previous feedback is retained.'));
     main.append(h('p', { class: 'muted small' }, `Last built ${new Date(state.generatedAt).toLocaleString()} · ${state.items.length} suggestions`),
       h('div', { class: 'discovery-grid' }, state.items.filter(item => !['hidden', 'dismissed'].includes(state.feedback[item.candidateId])).map(card)));
   }
   await refresh();
+  watch();
 })();

public/greatest.js

@@ -65,8 +65,10 @@
 
   function cover(item) {
     const box = h('span', { class: 'greatest-cover' }, h('span', { class: 'greatest-cover-fallback', 'aria-hidden': 'true' }, item.title.slice(0, 1)));
-    if (item.cover && safeHref(item.cover)) {
-      const image = h('img', { src: item.cover, alt: '', loading: 'lazy' });
+    // Covers come from the server (cached catalog covers, owned library covers) as relative URLs.
+    const src = /^(api\/greatest-cover\?|lib\/)/.test(item.cover || '') ? item.cover : safeHref(item.cover);
+    if (src) {
+      const image = h('img', { src, alt: '', loading: 'lazy' });
       image.onerror = () => image.remove();
       box.append(image);
     }

public/health.js

@@ -4,14 +4,16 @@
   try { me = await api('api/me'); } catch (e) { return showError(e); }
   topNav(me);
   if (!me.admin) return showError(new Error('Administrator access required.'));
-  const main = settingsShell(me, 'admin/health').content;
+  const shell = settingsShell(me, 'admin/health').content;
+  const main = h('div'), backup = backupCard();
+  shell.append(main, backup.card);
   async function load() {
     try {
       const [health, scraper] = await Promise.all([api('api/admin/health'), api('api/admin/scraper')]);
+      backup.draw(health.backup);
       main.textContent = '';
       main.append(h('section', { class: 'card' },
-        h('p', {}, `Reader: ${health.reader.ok ? 'healthy' : 'scan error'} · Sentinel: ${health.sentinel.status} · Scraper: ${scraper.status || 'unavailable'}`),
-        h('p', { class: 'muted small' }, health.sentinel.checkedAt ? `Last independent check: ${new Date(health.sentinel.checkedAt).toLocaleString()}` : 'Sentinel has not reported yet.'),
+        h('p', {}, `Reader: ${health.reader.ok ? 'healthy' : 'scan error'} · Scraper: ${scraper.status || 'unreachable'}`),
         h('a', { class: 'btn', href: 'settings.html#admin/scraper' }, 'Queue & hoster settings')));
       for (const entry of scraper.active || []) main.append(h('section', { class: 'card' }, h('h2', {}, entry.source),
         h('p', {}, `${entry.current.title} · ${entry.job?.completed || 0}/${entry.job?.total || '?'} chapters`),
@@ -24,11 +26,60 @@
           if (item.failure?.kind === 'access' && !confirm('Has the source access problem been resolved? This manually authorizes one retry.')) return;
           await api('api/admin/scraper', { command: 'control', action: item.status === 'unavailable' ? 'inspect' : 'retry', id: item.id }); load();
         } }, item.status === 'unavailable' ? 'Inspect source' : 'Retry'), h('button', { class: 'btn', onclick: async () => { await api('api/admin/scraper', { command: 'control', action: 'skip', id: item.id }); load(); } }, 'Skip job')))) : h('p', { class: 'muted' }, 'No failed jobs.')));
-      const incidents = [...(health.sentinel.incidents || []), ...(scraper.incidents || [])];
+      const incidents = scraper.incidents || [];
       main.append(h('details', { class: 'card', open: incidents.some(i => i.state === 'open') }, h('summary', {}, 'Incident & recovery history'), incidents.slice(-50).map(i =>
         h('p', {}, `${i.at} · ${i.state} · ${i.message || i.failure?.message || ''}${i.resolution ? ' · ' + i.resolution : ''}`))));
     } catch (e) { main.textContent = ''; main.append(h('p', { class: 'error', role: 'alert' }, e.message)); }
-    setTimeout(load, document.hidden ? 30000 : 10000);
+    setTimeout(load, document.hidden ? 30000 : backup.busy() ? 2000 : 10000);
   }
   load();
+
+  // Built-in backup: status redraws on every poll; the schedule form keeps its draft.
+  function backupCard() {
+    const mb = n => (n / 1048576).toFixed(n < 10485760 ? 1 : 0) + ' MB';
+    const secs = ms => ms < 60000 ? Math.round(ms / 1000) + ' s' : Math.round(ms / 60000) + ' min';
+    const status = h('div'), msg = h('p', { class: 'small', role: 'status' });
+    const enabled = h('input', { type: 'checkbox' }), time = h('input', { type: 'time', required: true });
+    const mode = h('select', {}, h('option', { value: 'count' }, 'backups'), h('option', { value: 'days' }, 'days'));
+    const keep = h('input', { type: 'number', min: '1', step: '1', style: 'width:6em' });
+    let state = null, filled = false;
+    const fill = s => { enabled.checked = s.enabled; time.value = s.time; mode.value = s.retention.mode; keep.value = s.retention.value; filled = true; };
+    const command = async (body, button) => {
+      button.disabled = true; msg.textContent = ''; msg.className = 'small';
+      try { const next = await api('api/admin/backup', body); if (body.command === 'settings') { fill(next.settings); msg.textContent = 'Saved.'; } draw(next); }
+      catch (e) { msg.textContent = e.message; msg.className = 'small error'; }
+      finally { button.disabled = false; }
+    };
+    const runButton = h('button', { class: 'btn primary', onclick: e => command({ command: 'run' }, e.target) }, 'Back up now');
+    const saveButton = h('button', { class: 'btn', onclick: e => command({ command: 'settings', settings: {
+      enabled: enabled.checked, time: time.value, retention: { mode: mode.value, value: Number(keep.value) } } }, e.target) }, 'Save schedule');
+    const form = h('div', { class: 'field' }, h('span', {}, 'Schedule'),
+      h('div', { class: 'row' }, h('label', { class: 'row' }, enabled, 'Back up daily at'), time),
+      h('div', { class: 'row', style: 'margin-top:8px' }, h('span', {}, 'Keep the newest'), keep, mode),
+      h('div', { class: 'row', style: 'margin-top:8px' }, saveButton, runButton), msg);
+    const card = h('section', { class: 'card', id: 'backup' }, h('h2', {}, 'Backups'), status, form);
+    function draw(next) {
+      if (!next) return;
+      state = next;
+      if (!filled) fill(next.settings);
+      const last = next.last, lines = [];
+      if (!next.configured) lines.push(h('p', { class: 'muted' }, 'Backups are off: mount a folder and set BACKUP_PATH (Compose) or BACKUP (native). See docs/backup.md.'));
+      else {
+        lines.push(h('p', {}, `Target: ${next.path} · ${next.archives.count} archive${next.archives.count === 1 ? '' : 's'}, ${mb(next.archives.bytes)}`));
+        if (next.targetError) lines.push(h('p', { class: 'error' }, next.targetError));
+      }
+      if (next.running) lines.push(h('p', { class: 'tag' }, `Running (${next.running.trigger}): ${next.running.phase}…`));
+      if (last) {
+        lines.push(h('p', {}, `Last backup: ${new Date(last.at).toLocaleString()} (${last.trigger})`,
+          last.error ? '' : ` · ${mb(last.size)} · ${secs(last.durationMs || 0)} · ${last.file}`));
+        if (last.error) lines.push(h('p', { class: 'error', role: 'alert' }, `Failed: ${last.error}`));
+        for (const warning of last.warnings || []) lines.push(h('p', { class: 'small error' }, warning));
+      } else if (next.configured) lines.push(h('p', { class: 'muted' }, 'No backup yet.'));
+      lines.push(h('p', { class: 'muted small' }, next.nextAt ? `Next scheduled backup: ${new Date(next.nextAt).toLocaleString()}` : 'No scheduled backup.'));
+      status.textContent = ''; status.append(...lines);
+      runButton.disabled = !next.configured || !!next.running;
+      form.hidden = !next.configured && !next.settings.enabled;
+    }
+    return { card, draw, busy: () => !!state?.running };
+  }
 })();

public/manga.js

@@ -213,14 +213,16 @@
     const panel = h('div', { class: 'versions', hidden: true });
     function draw() {
       const p = ch.progress, done = p && p.done;
-      const state = h('button', { class: 'state' + (done ? ' done' : ''), title: done ? 'Mark as unread' : 'Mark as read' },
-        done ? '✓ read' : p ? `page ${p.p + 1} / ${p.n}` : 'unread');
+      const readAt = done ? readDate(ch.readAt) : null;
+      const state = h('button', { class: 'state' + (done ? ' done' : ''), title: (readAt ? readAt.title + ' · ' : '') + (done ? 'Mark as unread' : 'Mark as read') },
+        done ? (readAt ? '✓ ' + readAt.text : '✓ read') : p ? `page ${p.p + 1} / ${p.n}` : 'unread');
       state.onclick = async () => {
-        if (done) { await api('api/unread', { m: M.id, c: ch.id }); ch.progress = null; }
+        if (done) { await api('api/unread', { m: M.id, c: ch.id }); ch.progress = null; ch.readAt = null; }
         else {
           const version = ch.versions[0];
           await api('api/progress', { m: M.id, c: ch.id, v: version.id, p: version.pages - 1, n: version.pages, done: true });
           ch.progress = { v: version.id, p: version.pages - 1, n: version.pages, done: true };
+          ch.readAt = { at: Date.now(), approx: false };
         }
         draw();
       };
@@ -228,12 +230,14 @@
         ? h('button', { class: 'btn small', onclick: () => { panel.hidden = !panel.hidden; } }, ch.versions.length + ' versions')
         : null;
       const pages = ch.versions[0] ? ch.versions[0].pages : 0;
-      const timing = chapterTiming(ch);
+      const timing = chapterTiming(ch), released = chapterRelease(ch.dates);
       wrap.textContent = '';
       wrap.append(h('div', { class: 'ch' + (done ? ' read' : '') },
-        h('a', { class: 'ch-main' + (timing ? ' has-time' : ''), href: readUrl(M.id, ch.id) },
+        h('a', { class: 'ch-main' + (timing ? ' has-time' : '') + (M.chapterDates?.release ? ' has-date' : ''), href: readUrl(M.id, ch.id) },
           h('span', { class: 'ch-label' }, ch.label),
           h('span', { class: 'ch-pages' }, `${pages} p.`),
+          M.chapterDates?.release ? h('span', { class: 'ch-date' + (released ? ' tier-' + released.tier : ''), title: released ? released.title : 'No date known', 'aria-label': released ? released.title : 'No date known' },
+            released ? released.text : '–') : null,
           timing ? h('span', { class: 'ch-time', title: timing.title, 'aria-label': timing.title }, timing.text) : null,
           h('span', { class: 'ch-title', title: ch.title || '' }, ch.title || 'Untitled'),
           ch.extra ? h('span', { class: 'tag' }, 'extra') : null),
@@ -261,6 +265,42 @@
     return wrap;
   }
 
+  // Phones (the chapter-row breakpoint in app.css) get numeric dates to leave room for titles.
+  const compact = matchMedia('(max-width: 640px)').matches;
+
+  // Read date: the day and the hour (local time) the chapter was first completed.
+  function readDate(value) {
+    const mode = M.chapterDates?.read;
+    if (!value || !value.at || mode === 'off') return null;
+    const when = new Date(value.at), thisYear = when.getFullYear() === new Date().getFullYear();
+    const day = when.toLocaleDateString(undefined, compact ? { day: '2-digit', month: '2-digit', ...(thisYear ? {} : { year: '2-digit' }) }
+      : { day: 'numeric', month: 'short', ...(thisYear ? {} : { year: 'numeric' }) });
+    const text = (value.approx ? '≤ ' : '') + day + (mode === 'hour' ? `, ${String(when.getHours()).padStart(2, '0')}h` : '');
+    const full = when.toLocaleDateString(undefined, { dateStyle: 'long' }) + (mode === 'hour' ? `, ${String(when.getHours()).padStart(2, '0')}:00–${String((when.getHours() + 1) % 24).padStart(2, '0')}:00` : '');
+    return { text, title: (value.approx ? 'Read by ' : 'Read on ') + full + (value.approx ? ' (earlier reads were not recorded)' : '') };
+  }
+
+  // Release date by tier: official > source upload > Manga Circus ecosystem > this library.
+  const TIER = { official: { mark: '' }, source: { mark: '≈' }, ecosystem: { mark: '◎' }, library: { mark: '↓' } };
+  const dayText = (at, precision, long) => {
+    // Day-precision dates are calendar dates, not instants: show them in UTC so they do not shift.
+    const dayOnly = !String(at).includes('T');
+    const opts = precision === 'year' ? { year: 'numeric' } : precision === 'month' ? { year: 'numeric', month: 'short' }
+      : long ? { dateStyle: 'long' } : compact ? { day: '2-digit', month: '2-digit', year: '2-digit' } : { day: 'numeric', month: 'short', year: 'numeric' };
+    return new Date(at).toLocaleDateString(undefined, { ...opts, ...(dayOnly ? { timeZone: 'UTC' } : {}) });
+  };
+  function chapterRelease(dates) {
+    if (!dates || !dates.release) return null;
+    const known = dates.known || {}, lines = [];
+    if (known.official) lines.push(`Released ${dayText(known.official.at, known.official.precision, true)}${known.official.name ? ' (' + known.official.name + ')' : ''}`);
+    if (known.source) lines.push(`Uploaded at ${known.source.name || 'the source'} ${dayText(known.source.at, 'day', true)}`);
+    if (known.batch) lines.push(`Bulk upload at ${known.batch.name || 'the source'} ${dayText(known.batch.at, 'day', true)} (not a release date)`);
+    if (known.ecosystem) lines.push(`First in Manga Circus ${dayText(known.ecosystem.at, 'day', true)}${known.ecosystem.name && known.ecosystem.name !== 'this server' ? ' (via ' + known.ecosystem.name + ')' : ''}`);
+    if (known.library) lines.push(`Added to this library ${dayText(known.library.at, 'day', true)}`);
+    const tier = TIER[dates.tier] || TIER.library;
+    return { tier: dates.tier, text: tier.mark + dayText(dates.release.at, dates.release.precision), title: lines.join('\n') };
+  }
+
   function chapterTiming(chapter) {
     if (!M.reading) return null;
     const value = chapter.reading;

public/settings.js

@@ -38,6 +38,11 @@
     h('option', { value: 'dim' }, 'Dim (70% brightness)'),
     h('option', { value: 'extra-dim' }, 'Extra dim (45% brightness)'));
   tone.value = S.pageTone;
+  const readDate = h('select', { onchange: e => save({ readDate: e.target.value }) },
+    h('option', { value: 'hour' }, 'Date and hour (e.g. 23 Oct, 21h)'),
+    h('option', { value: 'date' }, 'Date only'),
+    h('option', { value: 'off' }, 'Do not show'));
+  readDate.value = S.readDate;
   const LABEL = { flow: 'Flow', page: 'Page', book: 'Book', strip: 'Webtoon flow' };
   const zooms = () => Object.keys(LABEL).map(k => `${LABEL[k]} ${Math.round((S.zoom[k] || 1) * 100)}%`).join(' · ');
   const zoomText = h('span', { class: 'muted' }, zooms());
@@ -85,6 +90,13 @@
       h('button', { class: 'btn small', onclick: async () => { await save({ zoom: {} }); zoomText.textContent = zooms(); } }, 'Reset to fit'),
       h('div', { class: 'muted small' }, '100% = the page fits the screen. Change it in the reader with Ctrl + mouse wheel, a pinch, or the + / - keys.'))));
 
+  section('user/reading').append(h('section', { class: 'card' },
+    h('h2', {}, 'Chapter list dates'),
+    h('label', { class: 'field' }, h('span', {}, 'When I read a chapter'), readDate,
+      h('div', { class: 'muted small' }, 'Shown next to ✓ in the chapter list: when you first finished the chapter, in your local time. Marking it unread clears it.')),
+    check('releaseDate', 'Show when each chapter was released',
+      'Best known date first: the official release, else the upload at the source (≈), else when it first reached Manga Circus (◎), else when it was added here (↓). Hover a date for all known dates.')));
+
   const readingHoursBody = h('div', {}, h('p', { class: 'muted' }, 'Loading server reading hours…'));
   section('user/reading-hours').append(h('section', { class: 'card' },
     h('h2', {}, 'Server reading hours'), readingHoursBody,
@@ -194,7 +206,8 @@
         return;
       }
       const activeEntries = state.active?.length ? state.active : state.current ? [{ source: state.current.source, current: state.current, job: state.job }] : [];
-      const url = h('input', { type: 'url', placeholder: 'WeebCentral or MangaDex title URL', class: 'grow' });
+      const usable = (state.adapters || []).filter(adapter => state.settings?.sourcesEnabled?.[adapter.id] !== false).map(adapter => adapter.label);
+      const url = h('input', { type: 'url', placeholder: `${usable.join(', ') || 'Source'} title URL`, class: 'grow' });
       const title = h('input', { placeholder: 'Title override (optional)' });
       const scope = h('select', {},
         h('option', { value: 'missing' }, 'All missing chapters'),
@@ -217,8 +230,16 @@
         const max = h('input', { type: 'number', min: '.5', step: '.5', value: (sourceRate.maxMs ?? 4000) / 1000, title: 'Maximum seconds between image and API requests' });
         const chapterMin = h('input', { type: 'number', min: '.5', step: '1', value: (sourceRate.chapterMinMs ?? 5000) / 1000, title: 'Minimum seconds between chapters' });
         const chapterMax = h('input', { type: 'number', min: '.5', step: '1', value: (sourceRate.chapterMaxMs ?? 15000) / 1000, title: 'Maximum seconds between chapters' });
+        // Sources with a switch (sourcesEnabled) start switched off on new installations.
+        const switchable = Object.hasOwn(rate.sourcesEnabled || {}, adapter.id);
+        const enabled = switchable ? h('input', { type: 'checkbox', checked: rate.sourcesEnabled[adapter.id] === true,
+          onchange: e => command({ command: 'settings', settings: { sourcesEnabled: { [adapter.id]: e.target.checked } } }) }) : null;
         return h('div', { class: 'source-rate' },
           h('b', {}, adapter.label),
+          switchable ? h('label', { class: 'check' }, enabled, h('span', {}, `Use ${adapter.label}`,
+            h('div', { class: 'muted small' }, adapter.id === 'mangadex'
+              ? 'Official API; titles removed at a publisher\'s request are not offered.'
+              : 'Unofficial site that mostly hosts unlicensed copies. Switch it on only for what you may copy. While it is off, its queued jobs wait and update checks skip it.'))) : null,
           h('div', { class: 'row rate-row' },
             h('label', {}, h('small', {}, 'Requests min (s)'), min), h('label', {}, h('small', {}, 'Requests max (s)'), max),
             h('label', {}, h('small', {}, 'Chapters min (s)'), chapterMin), h('label', {}, h('small', {}, 'Chapters max (s)'), chapterMax),
@@ -236,7 +257,8 @@
       const completed = visible(state.queue.filter(item => item.status === 'complete' && item.kind === 'download'));
       const when = value => value && !Number.isNaN(Date.parse(value)) ? new Date(value).toLocaleString() : 'Never';
       const chips = h('div', { class: 'row scraper-filters', role: 'group', 'aria-label': 'Filter by source' },
-        [{ id: '', label: 'All sources' }, ...(state.adapters || []).map(adapter => ({ id: adapter.id, label: adapter.label }))].map(option =>
+        [{ id: '', label: 'All sources' }, ...(state.adapters || []).map(adapter => ({ id: adapter.id,
+          label: adapter.label + (state.settings?.sourcesEnabled?.[adapter.id] === false ? ' (off)' : '') }))].map(option =>
           h('button', { class: 'btn small', type: 'button', 'aria-pressed': sourceFilter === option.id ? 'true' : 'false',
             onclick: () => { sourceFilter = option.id; draw(lastState); } }, option.label)));
       card.append(chips);
@@ -473,10 +495,16 @@
         h('div', { class: 'field' }, h('span', {}, 'Users'),
           h('div', { class: 'users' }, data.users.map(u => h('div', {},
             h('span', {}, u.name, u.admin ? h('span', { class: 'tag', style: 'margin-left:8px' }, 'admin') : null),
-            u.name === me.name ? h('span', { class: 'muted small' }, 'you') : h('button', { class: 'btn small', onclick: async () => {
-              if (!confirm(`Delete ${u.name} and their reading progress?`)) return;
-              await api('api/admin/user/delete', { name: u.name }); load();
-            } }, 'Delete'))))),
+            u.name === me.name ? h('span', { class: 'muted small' }, 'you') : h('span', { class: 'row' },
+              h('button', { class: 'btn small', onclick: async () => {
+                if (!confirm(u.admin ? `Remove admin rights from ${u.name}?` : `Make ${u.name} an admin? Admins can manage users, the library and the scraper.`)) return;
+                try { await api('api/admin/user/role', { name: u.name, admin: !u.admin }); load(); }
+                catch (e) { msg.textContent = e.message; msg.className = 'small error'; }
+              } }, u.admin ? 'Remove admin' : 'Make admin'),
+              h('button', { class: 'btn small', onclick: async () => {
+                if (!confirm(`Delete ${u.name} and their reading progress?`)) return;
+                await api('api/admin/user/delete', { name: u.name }); load();
+              } }, 'Delete')))))),
         h('div', { class: 'field' }, h('span', {}, 'Add a user or reset a password'),
           h('div', { class: 'row' }, name, pw, h('label', { class: 'row' }, isAdmin, 'admin'),
             h('button', { class: 'btn', onclick: async () => {

reading-analytics.js

@@ -115,6 +115,17 @@ function chapterInsight(record) {
   };
 }
 
+// Earliest completion time the clock still remembers (only the last few visits are kept).
+// approx: older completed passes were already dropped, so the first read may be earlier.
+function earliestCompletion(record) {
+  if (!record || typeof record !== 'object') return null;
+  const passes = [record.currentPass, ...(record.recentSessions || []).map(item => item && item.pass)];
+  const times = passes.map(pass => pass && whole(pass.completedAt)).filter(time => time > 0);
+  if (!times.length) return null;
+  const counted = new Set(times).size;
+  return { at: Math.min(...times), approx: whole(record.completedPasses) > counted };
+}
+
 function median(values) {
   const sorted = values.filter(value => Number.isFinite(value) && value > 0).sort((a, b) => a - b);
   if (!sorted.length) return null;
@@ -295,6 +306,6 @@ function applyPulse(record, pulse, { version, pageCount, eligible, now = Date.no
 }
 
 module.exports = {
-  INPUT_TYPES, applyPulse, applyVisit, chapterInsight, compactRanges, eligiblePages, emptyRecord,
+  INPUT_TYPES, applyPulse, applyVisit, chapterInsight, compactRanges, earliestCompletion, eligiblePages, emptyRecord,
   libraryInsight, mangaInsight, publicSummary, rangeSet,
 };

runtime/supervisor.js

@@ -3,7 +3,6 @@ const fs = require('fs'), path = require('path'), { spawn } = require('child_pro
 const SPECS = {
   reader: { script: 'server.js', port: () => process.env.PORT || 8080, file: () => path.join(process.env.DATA || 'data', 'runtime-reader.json') },
   scraper: { script: 'scraper/daemon.js', port: () => process.env.SCRAPER_PORT || 8091, file: () => path.join(process.env.SCRAPER_DATA || path.join(process.env.DATA || 'data', 'scraper'), 'runtime.json') },
-  sentinel: { script: 'sentinel/daemon.js', port: () => process.env.SENTINEL_PORT || 8092, file: () => path.join(process.env.SENTINEL_DATA || path.join(process.env.DATA || 'data', 'sentinel'), 'runtime.json') },
 };
 class Supervisor {
   constructor(name, options = {}) {

runtime/test/supervisor.test.js

@@ -1,7 +1,7 @@
 'use strict';
 const test = require('node:test'), assert = require('node:assert/strict');
 const fs = require('fs'), os = require('os'), path = require('path'), { EventEmitter } = require('events');
-const { Supervisor } = require('../../runtime/supervisor');
+const { Supervisor } = require('../supervisor');
 const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
 test('supervisor isolates a child crash and persists its bounded restart budget', async () => {
   const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-supervisor-')), file = path.join(dir, 'runtime.json');

scraper/AGENT_RUNBOOK.md

@@ -22,7 +22,7 @@ Runtime recovery is documented in `../docs/resilience.md`. Request-level 5xx ret
 remain limited to 30/60 seconds; after those are exhausted the daemon can resume the
 same checkpoint once after a five-minute source cooldown. No automatic access-error
 retry is permitted. Unavailable titles and parser failures are job-scoped; repeated
-independent parser failures stop the source lane. Sentinel never changes adapters.
+independent parser failures stop the source lane. The stall watchdog never changes adapters.
 
 ```text
 public metadata -> cached series metadata

scraper/README.md

@@ -4,9 +4,8 @@ The scraper uses small host adapters to create a common manifest, followed by on
 
 Start with [AGENT_RUNBOOK.md](AGENT_RUNBOOK.md). It contains the host-specific procedures, safety limits, and stop conditions. `manifest.example.json` documents the interchange format.
 
-The historical human-readable wishlist is [WISHLIST.md](WISHLIST.md). Once the
-managed daemon has started, `data/scraper/queue.json` is the authoritative queue and
-is edited through the admin UI.
+Once the managed daemon has started, `data/scraper/queue.json` is the authoritative
+queue and is edited through the admin UI.
 
 ## Managed service
 
@@ -32,9 +31,8 @@ Queue schema version 3 supports scoped jobs, multiple jobs for the same upstream
 series, one sequential worker per source, incremental chapter publication, library
 duplicate/version checks, and configurable nightly update discovery. See
 [`docs/scraper-service.md`](../docs/scraper-service.md) for the service contract and
-[`docs/local-deployment.md`](../docs/local-deployment.md) for the currently running
-Windows Scheduled Task deployment. On that machine the scheduled task owns the single
-supervisor; do not run `npm run start:all` beside it.
+[`docs/deployment.md`](../docs/deployment.md) for running it in Docker. Run only one
+supervisor or container against the same library and data.
 
 ```bash
 npm run test:scraper

scraper/adapters/mangadex.js

@@ -3,6 +3,7 @@
 const API = 'https://api.mangadex.org';
 const { parse } = require('../plugins/registry');
 const { localizedTitle, chapterNumber, selectChapterReleases, availabilitySummary, pageUrls } = require('../plugins/mangadex');
+const { sourceDate } = require('../../chapter-dates');
 
 function mangaDexInfo(value) {
   let url;
@@ -71,8 +72,9 @@ async function discoverMangaDexChapter(ref, http) {
   const atHome = await jsonGet(http, `${API}/at-home/server/${ref.id}?forcePort443=true`);
   const groupName = ref.group?.name ? ` · ${ref.group.name}` : '';
   const parsed = parse('mangadex', 'atHome', { data: { response: atHome, chapterUrl: ref.url, expectedPages: ref.pages }, url: `${API}/at-home/server/${ref.id}` });
+  const source = sourceDate(ref.uploadedAt, `MangaDex${groupName}`, ref.uploadBatch);
   return {
-    number: ref.number, title: ref.title || '', added: ref.added,
+    number: ref.number, title: ref.title || '', ...(source ? { dates: { source } } : {}),
     ...(ref.special ? { special: true, label: ref.label, sourceId: ref.id, sourceNumber: ref.sourceNumber ?? null } : {}),
     source: { name: `MangaDex${groupName}`, url: ref.url },
     pages: parsed.pages,

scraper/adapters/weebcentral.js

@@ -4,6 +4,7 @@ const { SlowHttp } = require('../lib/http');
 const { fallbackDirection } = require('../../reading-direction');
 const parser = require('../plugins/weebcentral');
 const { parse } = require('../plugins/registry');
+const { sourceDate } = require('../../chapter-dates');
 
 function weebCentralInfo(value) {
   let url;
@@ -36,7 +37,8 @@ async function discoverWeebCatalog(seriesUrl, http) {
 
 async function discoverWeebChapter(chapter, http) {
   const pageHtml = await (await http.get(chapter.url, { accept: 'text/html' })).text();
-  const { manifestUrl, added } = parse('weebcentral', 'chapter', { html: pageHtml, url: chapter.url });
+  const { manifestUrl } = parse('weebcentral', 'chapter', { html: pageHtml, url: chapter.url });
+  const source = sourceDate(chapter.uploadedAt, 'WeebCentral', chapter.uploadBatch);
   const fragment = await (await http.get(`${manifestUrl}&reading_style=long_strip&current_page=1`, { accept: 'text/html' })).text();
   const parsed = parse('weebcentral', 'pages', { html: fragment, url: manifestUrl });
   const pages = parsed.pages.map(url => ({ url, headers: { referer: chapter.url } }));
@@ -46,7 +48,7 @@ async function discoverWeebChapter(chapter, http) {
     ...(chapter.special ? { special: true, label: chapter.label, sourceId: chapter.id,
       sourceNumber: chapter.volume, releaseKind: chapter.releaseKind, volume: chapter.volume } : {}),
     ...(chapter.book ? { book: chapter.book } : {}),
-    added,
+    ...(source ? { dates: { source } } : {}),
     source: { name: 'WeebCentral', url: chapter.url }, pages,
   };
 }

scraper/daemon.js

@@ -42,7 +42,6 @@ async function body(req) {
       if (url.pathname === '/api/queue/clear-completed') { const result = await service.clearCompleted(String(input.source || '')); return json(res, 200, { ...result, state: await service.publicState() }); }
       if (url.pathname === '/api/queue/reorder') { await service.reorder(input.ids); return json(res, 200, await service.publicState()); }
       if (url.pathname === '/api/control') { await service.control(String(input.action || ''), input.id); return json(res, 200, await service.publicState()); }
-      if (url.pathname === '/api/recovery') return json(res, 200, await service.recover(input));
       if (url.pathname === '/api/settings') { await service.updateSettings(input); return json(res, 200, await service.publicState()); }
       if (url.pathname === '/api/discovery/run') { await service.enqueueDiscoveryChecks('manual'); return json(res, 200, await service.publicState()); }
       if (url.pathname === '/api/library/auto-update') { await service.setMangaAutoUpdate(String(input.id || ''), !!input.enabled); return json(res, 200, await service.publicState()); }

scraper/lib/publisher.js

@@ -12,6 +12,7 @@ const { directionFields } = require('../../reading-direction');
 const { mergeImportedMetadata } = require('./metadata-merge');
 const { normalizeWorkIdentity } = require('./work-identity');
 const { parseKey } = require('../../chapter-identity');
+const { publishedDates } = require('../../chapter-dates');
 
 const token = () => crypto.randomBytes(5).toString('hex');
 // Staging is a recovery asset, not disposable garbage. Legacy stages without an
@@ -180,7 +181,8 @@ async function stageChapter(bookDir, chapter, http, onPage, priorMeta = {}) {
     await fsp.writeFile(path.join(stage, 'chapter.json'), JSON.stringify({
       ...priorMeta,
       title: chapter.title || '', source: chapter.source,
-      added: chapter.added || new Date().toISOString(),
+      // "added" stays the first arrival in this library; x-dates keeps source/release/ecosystem.
+      ...publishedDates(priorMeta, { ...(chapter.dates || {}), added: chapter.added }),
       priority: Number(chapter.priority || 0), extra: !!chapter.extra,
       ...(chapter.special ? { special: true, label: chapter.label || 'Special', sourceId: chapter.sourceId,
         sourceNumber: chapter.sourceNumber ?? null,

scraper/metadata/release-dates.js

@@ -0,0 +1,180 @@
+'use strict';
+// Official chapter release dates (BL-0048) from MangaUpdates publisher releases.
+// MangaUpdates also lists English digital re-releases of old chapters as "official"
+// (Naruto ch. 700 shows 2020; it came out in Japan in 2014). A date is accepted only when
+//   1. it lies inside the work's original run (AniList start - 7 days ... end + 30 days), and
+//   2. nothing shows the chapter earlier: no scanlation release in MangaUpdates and no
+//      source upload (bulk or not) more than EARLY_TOLERANCE_DAYS before it, and
+//   3. it fits the release order (see officialDates).
+// Simulpub releases pass all three; back-catalog re-releases fail at least one.
+const fs = require('node:fs').promises;
+const path = require('node:path');
+const crypto = require('node:crypto');
+const { parseKey } = require('../../chapter-identity');
+const { writeJsonAtomic } = require('../lib/json-file');
+
+const DAY = 864e5;
+const EARLY_TOLERANCE_DAYS = 4; // early scans and source uploads of weekly series arrive 1-3 days before the official date
+const ORDER_TOLERANCE_DAYS = 2; // double releases and day-granular dates may put a lower chapter a day or two later
+const OFFICIAL = new Set(['viz', 'viz media', 'manga plus', 'mangaplus', 'shueisha', 'kodansha', 'kodansha usa', 'k manga',
+  'yen press', 'seven seas', 'square enix', 'square enix manga', 'manga up!', 'comikey', 'azuki', 'tappytoon', 'webtoon',
+  'line webtoon', 'webtoons', 'tapas', 'pocket comics', 'manta', 'lezhin', 'lezhin comics', 'j-novel club', 'dark horse',
+  'vertical', 'denpa', 'tokyopop', 'ize press', 'one peace books', 'inky pen', 'kakaopage', 'kakao webtoon', 'naver webtoon']);
+const UA = 'MangaCircus/1.0 (self-hosted manga reader; chapter release dates)';
+const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
+const norm = value => String(value || '').toLowerCase().normalize('NFKD').replace(/[^a-z0-9]+/g, '');
+// "001", "1.50" and "1" are the same chapter; ranges, volumes and letters are not single chapters.
+const chapterKey = value => (/^\s*\d+(?:\.\d+)?\s*$/.test(String(value ?? '')) ? String(Number(value)) : null);
+const isOfficial = groups => groups.some(name => OFFICIAL.has(String(name).toLowerCase().trim()));
+
+async function cached(cacheDir, key, maxAgeMs, load) {
+  const file = cacheDir && path.join(cacheDir, crypto.createHash('sha256').update(key).digest('hex').slice(0, 32) + '.json');
+  if (file) {
+    try {
+      const stat = await fs.stat(file);
+      if (Date.now() - stat.mtimeMs < maxAgeMs) return JSON.parse(await fs.readFile(file, 'utf8'));
+    } catch { /* not cached */ }
+  }
+  const value = await load();
+  if (file) { await fs.mkdir(cacheDir, { recursive: true }); await writeJsonAtomic(file, value); }
+  return value;
+}
+
+class ReleaseDateSources {
+  constructor(options = {}) {
+    this.cacheDir = options.cacheDir || null;
+    this.minGapMs = options.minGapMs ?? 1100;
+    this.fetch = options.fetch || globalThis.fetch;
+    this.last = 0;
+  }
+  async request(url, body) {
+    const wait = this.last + this.minGapMs - Date.now();
+    if (wait > 0) await sleep(wait);
+    this.last = Date.now();
+    const response = await this.fetch(url, { method: 'POST', headers: { 'content-type': 'application/json', 'user-agent': UA },
+      body: JSON.stringify(body), signal: AbortSignal.timeout(30000) });
+    if (!response.ok) throw new Error(`${new URL(url).hostname} HTTP ${response.status}`);
+    return response.json();
+  }
+  // MangaUpdates series whose matched name equals one of the manga's titles and whose year fits.
+  async findSeries(meta) {
+    const titles = [meta.title, ...(Array.isArray(meta.altTitles) ? meta.altTitles : [])].map(norm).filter(Boolean);
+    const year = Number(meta.year) || null;
+    const result = await cached(this.cacheDir, `mu-search:${meta.title}`, 30 * DAY,
+      () => this.request('https://api.mangaupdates.com/v1/series/search', { search: String(meta.title || ''), perpage: 10 }));
+    for (const hit of result.results || []) {
+      const record = hit.record || {}, hitYear = Number(record.year) || null;
+      if (year && hitYear && Math.abs(hitYear - year) > 1) continue;
+      if ([record.title, hit.hit_title].map(norm).some(name => name && titles.includes(name))) {
+        return { id: String(record.series_id), title: record.title, url: record.url || null };
+      }
+    }
+    return null;
+  }
+  async releases(seriesId, { pages = Infinity, maxAgeMs = 7 * DAY } = {}) {
+    return cached(this.cacheDir, `mu-releases:${seriesId}:${pages}`, maxAgeMs, async () => {
+      const rows = [];
+      for (let page = 1; page <= pages; page++) {
+        const result = await this.request('https://api.mangaupdates.com/v1/releases/search',
+          { search: String(seriesId), search_type: 'series', perpage: 100, page, orderby: 'date' });
+        for (const { record } of result.results || []) {
+          if (record?.release_date) rows.push({ chapter: record.chapter, date: record.release_date, groups: (record.groups || []).map(g => g.name) });
+        }
+        if (!result.results?.length || page * 100 >= (result.total_hits || 0)) break;
+      }
+      return rows;
+    });
+  }
+  async originalRun(anilistId) {
+    if (!anilistId) return null;
+    const result = await cached(this.cacheDir, `anilist-run:${anilistId}`, 30 * DAY, () => this.request('https://graphql.anilist.co', {
+      query: 'query ($id: Int) { Media(id: $id, type: MANGA) { status startDate { year month day } endDate { year month day } } }',
+      variables: { id: Number(anilistId) },
+    }));
+    const media = result.data?.Media;
+    if (!media) return null;
+    const date = (d, end) => (d?.year ? Date.UTC(d.year, (d.month || (end ? 12 : 1)) - 1, d.day || (end ? 28 : 1)) : null);
+    return { status: media.status, start: date(media.startDate, false), end: media.status === 'RELEASING' ? null : date(media.endDate, true) };
+  }
+}
+
+// Pure decision: official dates per chapter key, with the reasons others were rejected.
+// evidence: Map chapterKey -> earliest other time (ms) the library knows for that chapter.
+function officialDates(releases, run, evidence = new Map()) {
+  const official = new Map(), scans = new Map();
+  for (const row of releases) {
+    const key = chapterKey(row.chapter), time = Date.parse(row.date);
+    if (!key || !Number.isFinite(time)) continue;
+    const target = isOfficial(row.groups) ? official : scans;
+    const prior = target.get(key);
+    if (!prior || time < prior.time) target.set(key, { time, date: String(row.date).slice(0, 10), groups: row.groups });
+  }
+  const accepted = new Map(), rejected = { outsideRun: 0, earlier: 0, outOfOrder: 0, noRun: 0 };
+  // 3. Walking down from the newest chapter, release dates never rise again: a lower chapter
+  //    dated after a higher one is a back-catalog re-release (One Piece ch. 8 on 2019-02-06).
+  let ceiling = Infinity;
+  for (const [key, item] of [...official].sort((a, b) => Number(b[0]) - Number(a[0]))) {
+    if (!run || !run.start) { rejected.noRun++; continue; }
+    if (item.time < run.start - 7 * DAY || (run.end && item.time > run.end + 30 * DAY)) { rejected.outsideRun++; continue; }
+    const earliest = Math.min(scans.get(key)?.time ?? Infinity, evidence.get(key) ?? Infinity);
+    if (earliest < item.time - EARLY_TOLERANCE_DAYS * DAY) { rejected.earlier++; continue; }
+    if (item.time > ceiling + ORDER_TOLERANCE_DAYS * DAY) { rejected.outOfOrder++; continue; }
+    ceiling = Math.min(ceiling, item.time);
+    accepted.set(key, { at: item.date, precision: 'day', source: `MangaUpdates · ${item.groups.join(', ')}` });
+  }
+  return { accepted, rejected, officialCount: official.size };
+}
+
+// Chapter version folders of one manga with their numeric chapter key.
+async function chapterVersions(mangaDir) {
+  const out = [];
+  const dirs = async dir => (await fs.readdir(dir, { withFileTypes: true })).filter(e => e.isDirectory() && !e.name.startsWith('.')).map(e => e.name);
+  for (const book of await dirs(mangaDir)) {
+    if (!parseKey(book)) continue;
+    for (const chapter of await dirs(path.join(mangaDir, book))) {
+      const parsed = parseKey(chapter);
+      if (!parsed || parsed.x) continue; // extras and specials have no chapter number to match
+      const file = path.join(mangaDir, book, chapter, 'chapter.json');
+      let meta;
+      try { meta = JSON.parse(await fs.readFile(file, 'utf8')); } catch { continue; }
+      out.push({ key: chapterKey(parsed.num), book, file, meta });
+    }
+  }
+  return out;
+}
+
+// Look up, decide and (with apply) write x-dates.release for one manga folder.
+async function refreshMangaReleaseDates(mangaDir, sources, { apply = false, pages = Infinity, maxAgeMs } = {}) {
+  const meta = JSON.parse(await fs.readFile(path.join(mangaDir, 'manga.json'), 'utf8').catch(() => '{}'));
+  const versions = await chapterVersions(mangaDir);
+  const summary = { manga: path.basename(mangaDir), series: null, chapters: new Set(versions.map(v => v.key)).size, official: 0, accepted: 0, written: 0, cleared: 0, rejected: null, ambiguous: false };
+  // Seasons that restart at chapter 1 cannot be matched to MangaUpdates' running numbers.
+  const books = new Map();
+  for (const version of versions) books.set(version.key, (books.get(version.key) || new Set()).add(version.book));
+  if ([...books.values()].some(set => set.size > 1)) { summary.ambiguous = true; return summary; }
+  const series = await sources.findSeries(meta);
+  if (!series) return summary;
+  summary.series = series;
+  const run = await sources.originalRun(meta['x-anilist']?.id || meta['x-scraper']?.ids?.anilist);
+  const evidence = new Map();
+  for (const version of versions) {
+    const time = Date.parse(version.meta['x-dates']?.source?.at);
+    if (version.key && Number.isFinite(time) && time < (evidence.get(version.key) ?? Infinity)) evidence.set(version.key, time);
+  }
+  const { accepted, rejected, officialCount } = officialDates(await sources.releases(series.id, { pages, ...(maxAgeMs ? { maxAgeMs } : {}) }), run, evidence);
+  summary.official = officialCount; summary.rejected = rejected;
+  const have = new Set(versions.map(v => v.key));
+  summary.accepted = [...accepted.keys()].filter(key => have.has(key)).length;
+  for (const version of versions) {
+    const dates = version.meta['x-dates'] || { version: 1 }, next = accepted.get(version.key);
+    const prior = dates.release;
+    let changed = false;
+    if (next && JSON.stringify(prior) !== JSON.stringify(next)) { dates.release = next; changed = true; summary.written++; }
+    // A date this job wrote earlier and no longer accepts is withdrawn; other release dates stay.
+    else if (!next && prior && /^MangaUpdates\b/.test(prior.source || '') && pages === Infinity) { delete dates.release; changed = true; summary.cleared++; }
+    if (changed && apply) await writeJsonAtomic(version.file, { ...version.meta, 'x-dates': dates });
+  }
+  return summary;
+}
+
+module.exports = { ReleaseDateSources, officialDates, refreshMangaReleaseDates, chapterKey, isOfficial, EARLY_TOLERANCE_DAYS };

scraper/plugins/mangadex.js

@@ -1,6 +1,7 @@
 'use strict';
 
 const { specialKey } = require('../../chapter-identity');
+const { batchFlags } = require('../../chapter-dates');
 
 function localizedTitle(attributes = {}) {
   const titles = attributes.title || {};
@@ -41,14 +42,18 @@ function selectChapterReleases(resources = [], specialLabel = 'Special') {
     const prior = selected.get(number);
     if (!prior || compareRank(resource, prior) < 0) selected.set(number, resource);
   }
-  return [...selected.entries()].map(([number, resource]) => {
+  const chosen = [...selected.entries()];
+  const uploaded = chosen.map(([, resource]) => resource.attributes?.publishAt || resource.attributes?.readableAt || resource.attributes?.createdAt);
+  const batches = batchFlags(uploaded);
+  return chosen.map(([number, resource], index) => {
     const group = groupFor(resource);
     return {
       id: resource.id, number, title: resource.attributes?.title || '',
       ...(!chapterNumber(resource.attributes?.chapter) ? { special: true, label: specialLabel, sourceId: resource.id, sourceNumber: resource.attributes?.chapter ?? null } : {}),
       ...(resource.attributes?.volume ? { volume: chapterNumber(resource.attributes.volume) || String(resource.attributes.volume) } : {}),
       pages: Number(resource.attributes?.pages || 0),
-      added: resource.attributes?.publishAt || resource.attributes?.readableAt || resource.attributes?.createdAt,
+      // A group's upload time, not an official release; bulk uploads are flagged.
+      ...(uploaded[index] ? { uploadedAt: uploaded[index], uploadBatch: !!batches[index] } : {}),
       group,
       url: `https://mangadex.org/chapter/${resource.id}`,
     };

scraper/plugins/weebcentral.js

@@ -1,5 +1,6 @@
 'use strict';
 const { volumeKey } = require('../../chapter-identity');
+const { batchFlags } = require('../../chapter-dates');
 
 // First-party, pure parsing functions. This module never fetches remote content.
 const decode = value => value.replace(/&amp;/g, '&').replace(/&#39;/g, "'").replace(/&quot;/g, '"').replace(/<[^>]+>/g, '').trim();
@@ -21,19 +22,26 @@ function catalog({ html, url }) {
     const volume = !regular ? text.match(/^Volume\s+([0-9]+(?:\.[0-9]+)?)(?!\d)/i)?.[1] : null;
     const prologue = text.match(/Prologue(?:\s+([0-9]+))?/i)?.[1];
     const number = regular || (prologue ? `0.${prologue.padStart(2, '0')}` : /Prologue/i.test(text) ? '0' : null);
+    // The full chapter list carries each upload time; the chapter page itself has none.
+    const uploaded = match[3].match(/datetime="([^"]+)"/i)?.[1];
+    const uploadedAt = uploaded && Number.isFinite(Date.parse(uploaded)) ? uploaded : undefined;
     if (number) found.push({
       id: match[2], number, title: regular ? '' : `Prologue ${prologue || ''}`.trim(),
       ...(season ? { book: { number: season, title: `Season ${season}` } } : {}),
-      url: absolute(url, match[1]), raw: text,
+      url: absolute(url, match[1]), raw: text, ...(uploadedAt ? { uploadedAt } : {}),
     });
     else if (volume) found.push({
       id: match[2], number: volumeKey(match[2]), title: '', special: true,
       label: `Volume ${volume}`, releaseKind: 'volume', volume,
       book: { number: Number(volume), title: `Volume ${volume}` },
-      url: absolute(url, match[1]), raw: text,
+      url: absolute(url, match[1]), raw: text, ...(uploadedAt ? { uploadedAt } : {}),
     });
   }
   const chapters = [...new Map(found.map(chapter => [chapter.id, chapter])).values()];
+  // Back-catalog chapters were imported together; their upload time is not a release date.
+  batchFlags(chapters.map(chapter => chapter.uploadedAt)).forEach((batch, index) => {
+    if (batch !== null) chapters[index].uploadBatch = batch;
+  });
   const chapterLinks = [...html.matchAll(/<a\b[^>]*href="[^"]*\/chapters\/[A-Z0-9]+[^"]*"/gi)].length;
   return { chapters, chapterLinks };
 }
@@ -42,7 +50,6 @@ function chapter({ html, url }) {
   return {
     manifestUrl: html.match(/['"](https?:\/\/weebcentral\.com\/chapters\/[^'"]+\/images\?is_prev=[^'"]+)['"]/i)?.[1]
       || `${url}/images?is_prev=False`,
-    added: html.match(/datetime="([^"]+)"/i)?.[1],
   };
 }
 

scraper/service.js

@@ -13,6 +13,7 @@ const { scanLibrary, findManga, shouldDownload, isNewerThanLibrary, summarizeInv
 const { inspectLegacyWork, promoteLegacyWork } = require('./lib/legacy-publication');
 const { classifyFailure, safeMessage, progressKey } = require('./lib/failures');
 const { writeJsonAtomic } = require('./lib/json-file');
+const { ReleaseDateSources, refreshMangaReleaseDates } = require('./metadata/release-dates');
 
 const RATE_KEYS = ['minMs', 'maxMs', 'chapterMinMs', 'chapterMaxMs'];
 const FAST_RATE_SETTINGS = {
@@ -23,9 +24,13 @@ const DEFAULT_SOURCE_RATES = {
   mangadex: { ...FAST_RATE_SETTINGS },
   mangapill: { ...FAST_RATE_SETTINGS },
 };
+// Sources an admin must switch on first. WeebCentral and MangaPill mostly host unlicensed
+// copies; MangaDex has an official API and handles takedowns. Peers are always available.
+const DEFAULT_SOURCES_ENABLED = { weebcentral: false, mangadex: true, mangapill: false };
 const DEFAULT_SETTINGS = {
   ...FAST_RATE_SETTINGS,
   sourceRates: Object.fromEntries(Object.entries(DEFAULT_SOURCE_RATES).map(([source, rate]) => [source, { ...rate }])),
+  sourcesEnabled: { ...DEFAULT_SOURCES_ENABLED },
   autoUpdateEnabled: true, autoUpdateTime: '03:00',
 };
 
@@ -72,6 +77,15 @@ function validateSettings(input = {}, prior = DEFAULT_SETTINGS) {
     };
     next.sourceRates[source] = validateRateSettings(patch || {}, base, `sourceRates.${source}.`);
   }
+  if (input.sourcesEnabled !== undefined && (!input.sourcesEnabled || typeof input.sourcesEnabled !== 'object' || Array.isArray(input.sourcesEnabled))) {
+    throw new Error('sourcesEnabled must be an object');
+  }
+  next.sourcesEnabled = { ...DEFAULT_SOURCES_ENABLED, ...(prior.sourcesEnabled || {}) };
+  for (const [source, enabled] of Object.entries(input.sourcesEnabled || {})) {
+    if (!Object.hasOwn(DEFAULT_SOURCES_ENABLED, source)) throw new Error(`Unknown source: ${source}`);
+    if (typeof enabled !== 'boolean') throw new Error(`sourcesEnabled.${source} must be true or false`);
+    next.sourcesEnabled[source] = enabled;
+  }
   if (input.autoUpdateEnabled !== undefined) next.autoUpdateEnabled = !!input.autoUpdateEnabled;
   if (input.autoUpdateTime !== undefined) {
     const value = String(input.autoUpdateTime).trim();
@@ -85,6 +99,13 @@ function settingsForSource(settings, source) {
   return { ...settings, ...((settings.sourceRates || {})[source] || {}) };
 }
 
+// Installations from before the switch keep every source they already use: off-by-default
+// applies to new installations, it must not silently stop a working queue.
+function inheritedSourcesEnabled(saved) {
+  const used = new Set([...(saved.queue || []), ...(saved.history || [])].map(item => item && item.source));
+  return Object.fromEntries(Object.entries(DEFAULT_SOURCES_ENABLED).map(([source, enabled]) => [source, enabled || used.has(source)]));
+}
+
 function existingPolicy(value) {
   const policy = String(value || 'skip-any');
   if (!['skip-any', 'add-source'].includes(policy)) throw new Error('existingPolicy must be skip-any or add-source');
@@ -124,6 +145,9 @@ class ScraperService {
     this.resolveAdapter = options.adapterFor || adapterFor;
     this.adapterList = options.adapters || adapters;
     this.retryDelayMs = options.retryDelayMs ?? 300000;
+    this.stallMs = options.stallMs ?? 900000;
+    this.stallObservations = new Map();
+    this.stallCheckedAt = 0;
     this.runtimeHeartbeat = Date.now();
   }
 
@@ -213,11 +237,13 @@ class ScraperService {
   async loadState() {
     if (await exists(this.stateFile)) {
       const saved = JSON.parse(await fs.readFile(this.stateFile, 'utf8'));
+      const savedSettings = saved.settings || {};
+      if (!savedSettings.sourcesEnabled) savedSettings.sourcesEnabled = inheritedSourcesEnabled(saved);
       return {
         version: 3, paused: false, status: 'idle', queue: [], history: [], publishedRevision: 0,
         currentIds: {}, sourcePaused: {},
         settings: DEFAULT_SETTINGS, ...saved,
-        settings: validateSettings(saved.settings || {}, DEFAULT_SETTINGS),
+        settings: validateSettings(savedSettings, DEFAULT_SETTINGS),
       };
     }
     const state = {
@@ -235,7 +261,7 @@ class ScraperService {
         status: completed.has(item.id) ? 'complete' : 'queued', order: index,
         added: legacy.created || new Date().toISOString(),
       }));
-      state.settings = validateSettings(legacy.settings || {}, DEFAULT_SETTINGS);
+      state.settings = validateSettings({ ...(legacy.settings || {}), sourcesEnabled: inheritedSourcesEnabled(state) }, DEFAULT_SETTINGS);
       state.migratedFrom = this.legacyStateFile;
     }
     return state;
@@ -350,12 +376,36 @@ class ScraperService {
     const delay = Math.max(1000, next.getTime() - Date.now());
     this.discoveryTimer = setTimeout(() => {
       this.discoveryTimer = null;
-      this.enqueueDiscoveryChecks('schedule').catch(error => this.event('error', `Nightly discovery failed: ${error.message}`))
+      this.refreshReleaseDates('schedule').catch(error => this.event('error', `Nightly release dates failed: ${error.message}`))
+        .then(() => this.enqueueDiscoveryChecks('schedule'))
+        .catch(error => this.event('error', `Nightly discovery failed: ${error.message}`))
         .finally(() => { if (!this.stopping) this.scheduleDiscovery(); });
     }, delay);
     this.discoveryTimer.unref();
   }
 
+  // Nightly, before the update checks: official release dates of ongoing series from their
+  // newest MangaUpdates releases (BL-0048). Runs only while no download is active, so the
+  // scraper stays the single writer of chapter.json.
+  async refreshReleaseDates(reason = 'manual') {
+    if (this.active.size) { await this.event('info', 'Release dates skipped: a download is running', { reason }); return null; }
+    const inventory = await this.libraryInventory(true);
+    const sources = new ReleaseDateSources({ cacheDir: path.join(this.dataRoot, 'release-dates-cache') });
+    let checked = 0, written = 0;
+    for (const manga of inventory.mangas) {
+      if (this.stopping || this.active.size) break;
+      if (manga.meta?.status !== 'ongoing') continue;
+      try {
+        const result = await refreshMangaReleaseDates(manga.path, sources, { apply: true, pages: 1, maxAgeMs: 20 * 3600e3 });
+        checked++; written += result.written;
+      } catch (error) { await this.event('warning', `Release dates for ${manga.title} failed: ${error.message}`, { mangaId: manga.id }); }
+    }
+    // The reader reconciles fully when the revision rises without a visible folder change.
+    if (written) { this.state.publishedRevision = Number(this.state.publishedRevision || 0) + 1; await this.persist(); }
+    await this.event('info', `Release dates: ${written} chapter${written === 1 ? '' : 's'} dated in ${checked} ongoing series`, { reason, checked, written });
+    return { checked, written };
+  }
+
   pendingSeries(source, seriesId) {
     return this.state.queue.some(item => item.source === source && String(item.seriesId) === String(seriesId)
       && ['queued', 'downloading', 'discovering', 'ready', 'error', 'retry_wait', 'quarantined', 'unavailable'].includes(item.status));
@@ -373,7 +423,8 @@ class ScraperService {
     let queued = 0, skipped = 0;
     for (const manga of inventory.mangas) {
       if (!manga.autoUpdate) continue;
-      const supported = manga.sources.map(source => ({ source, found: this.resolveAdapter(source.url) })).filter(item => item.found);
+      const supported = manga.sources.map(source => ({ source, found: this.resolveAdapter(source.url) }))
+        .filter(item => item.found && this.sourceEnabled(item.found.adapter.id));
       if (!supported.length) { skipped++; continue; }
       const preferred = manga.meta?.['x-scraper']?.preferredSource;
       const selected = supported.find(item => item.found.adapter.id === preferred) || supported[0];
@@ -570,9 +621,16 @@ class ScraperService {
     };
   }
 
+  sourceEnabled(source) {
+    return !Object.hasOwn(DEFAULT_SOURCES_ENABLED, source) || this.state.settings.sourcesEnabled?.[source] === true;
+  }
+
   async enqueue(input) {
     const found = this.resolveAdapter(input.url);
     if (!found) throw new Error('No installed scraper adapter accepts this URL');
+    if (!this.sourceEnabled(found.adapter.id)) {
+      throw new Error(`${found.adapter.label || found.adapter.id} is switched off. An admin can switch it on under Settings → Admin → Scraper → Hoster settings.`);
+    }
     const workIdentity = normalizeWorkIdentity(input.workIdentity);
     if (input.workIdentity != null && !workIdentity) throw new Error('Invalid manga work identity');
     const scope = normalizeScope(input.scope || {});
@@ -738,22 +796,32 @@ class ScraperService {
     throw new Error('Unknown control action');
   }
 
-  async recover(input) {
-    const item = this.state.queue.find(entry => entry.id === input.id);
-    if (!item) throw new Error('Job not found');
-    // Compare observations so a stale watcher cannot disturb a progressing worker.
-    if (Object.values(this.state.currentIds).includes(item.id)) {
+  // Stall watchdog: an active job whose checkpoint has not moved for stallMs (or one
+  // chapter delay + 3 min, if longer) is stopped at its next checkpoint and requeued
+  // once; a second stall quarantines it. A missing checkpoint proves nothing, and time
+  // spent paused does not count.
+  async checkStalls(now = Date.now()) {
+    const activeIds = new Set(Object.values(this.state.currentIds || {}));
+    for (const id of this.stallObservations.keys()) if (!activeIds.has(id)) this.stallObservations.delete(id);
+    for (const item of this.state.queue) {
+      if (!activeIds.has(item.id)) continue;
       const job = await this.jobState(item);
-      if (!job) return { changed: false };
-      if (progressKey(job) !== input.progressKey || this.state.paused || item.stopReason) return { changed: false };
-      if (Date.now() - Date.parse(job?.updated || item.started) < 900000) return { changed: false };
-      if ((item.stallRecoveries || 0) >= 1) {
-        item.stopReason = 'quarantined';
-      } else { item.stallRecoveries = (item.stallRecoveries || 0) + 1; item.stopReason = 'recovering'; }
-      await this.event('warning', `Sentinel requested recovery at the next checkpoint for ${item.title}`, { id: item.id, source: item.source });
-      return { changed: true };
+      if (!job) { this.stallObservations.delete(item.id); continue; }
+      const key = progressKey(job), prior = this.stallObservations.get(item.id);
+      if (!prior || prior.key !== key) { this.stallObservations.set(item.id, { key, at: now }); continue; }
+      if (this.state.paused || this.state.sourcePaused[item.source] || item.stopReason) { prior.at = now; continue; }
+      if (now - prior.at < Math.max(this.stallMs, Number(job.rate?.chapterMaxMs || 0) + 180000)) continue;
+      this.stallObservations.delete(item.id);
+      const minutes = Math.round((now - prior.at) / 60000), quarantine = (item.stallRecoveries || 0) >= 1;
+      const failure = { kind: 'stall', scope: 'job', retryable: !quarantine, message: quarantine
+        ? `No progress for ${minutes} minutes after a restart; quarantined`
+        : `No progress for ${minutes} minutes; restarting from the last checkpoint` };
+      if (quarantine) { item.stopReason = 'quarantined'; item.failure = failure; item.error = failure.message; }
+      else { item.stallRecoveries = (item.stallRecoveries || 0) + 1; item.stopReason = 'recovering'; }
+      this.state.incidents.push({ id: crypto.randomUUID(), at: new Date(now).toISOString(), jobId: item.id, source: item.source, title: item.title, failure, state: 'open' });
+      this.state.incidents = this.state.incidents.slice(-200);
+      await this.event('warning', `${item.title}: ${failure.message}`, { id: item.id, title: item.title, source: item.source });
     }
-    return { changed: false };
   }
 
   recomputeStatus() {
@@ -864,7 +932,8 @@ class ScraperService {
             continue;
           }
           const source = found.adapter.id;
-          if (this.active.has(source) || this.state.sourcePaused[source]) continue;
+          // A switched-off source keeps its queued jobs; they wait until it is switched on again.
+          if (this.active.has(source) || this.state.sourcePaused[source] || !this.sourceEnabled(source)) continue;
           if (this.state.circuits[source]?.state === 'open') continue;
           if (item.libraryMangaId && [...this.active.values()].some(entry => {
             const activeItem = this.state.queue.find(candidate => candidate.id === entry.id);
@@ -876,6 +945,10 @@ class ScraperService {
       const prior = this.state.status;
       this.recomputeStatus();
       if (prior !== this.state.status) await this.persist();
+      if (Date.now() - this.stallCheckedAt >= 30000) {
+        this.stallCheckedAt = Date.now();
+        await this.checkStalls().catch(error => console.error('[scraper] stall check failed:', safeMessage(error)));
+      }
       await sleep(500);
     }
   }

scraper/test/checkpoint-location.test.js

@@ -29,12 +29,45 @@ test('status and watchdog read the checkpoint for each source layout', async t =
   }
 });
 
-test('daemon refuses recovery when the checkpoint cannot be read', async () => {
-  const service = Object.create(ScraperService.prototype);
-  service.state = { paused: false, currentIds: { mangapill: 'job' },
-    queue: [{ id: 'job', source: 'mangapill', status: 'downloading' }] };
-  service.jobState = async () => null;
-  const result = await service.recover({ id: 'job', progressKey: '[0,0,"",0,""]' });
-  assert.deepEqual(result, { changed: false });
+function stallService(job) {
+  const service = new ScraperService({ stallMs: 100 });
+  service.state = { paused: false, sourcePaused: {}, currentIds: { a: 'job' }, incidents: [],
+    queue: [{ id: 'job', source: 'a', title: 'Manga', status: 'downloading' }] };
+  service.jobState = async () => job.value;
+  service.event = async () => {};
+  return service;
+}
+
+test('stall check leaves progressing and deliberately paused jobs alone', async () => {
+  const job = { value: { completed: 0, current: { downloadedPages: 1 } } };
+  const service = stallService(job), item = service.state.queue[0];
+  await service.checkStalls(0);
+  job.value = { completed: 0, current: { downloadedPages: 2 } };
+  await service.checkStalls(500000);
+  service.state.paused = true;
+  await service.checkStalls(1000000);
+  assert.equal(item.stopReason, undefined);
+  // Paused time does not count: the clock restarts on resume.
+  service.state.paused = false;
+  await service.checkStalls(1000050);
+  assert.equal(item.stopReason, undefined);
+  await service.checkStalls(1190000);
+  assert.equal(item.stopReason, 'recovering');
+  assert.equal(item.stallRecoveries, 1);
+  assert.equal(service.state.incidents[0].failure.kind, 'stall');
+});
+
+test('a second stall quarantines the job', async () => {
+  const service = stallService({ value: { completed: 2, total: 5 } }), item = service.state.queue[0];
+  item.stallRecoveries = 1;
+  await service.checkStalls(0); await service.checkStalls(1000000);
+  assert.equal(item.stopReason, 'quarantined');
+  assert.equal(item.failure.kind, 'stall');
+});
+
+test('missing checkpoint does not trigger a false stall recovery', async () => {
+  const service = stallService({ value: null });
+  for (const now of [0, 1000000, 2000000]) await service.checkStalls(now);
   assert.equal(service.state.queue[0].stopReason, undefined);
+  assert.equal(service.state.incidents.length, 0);
 });

scraper/test/release-dates.test.js

@@ -0,0 +1,93 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs').promises;
+const os = require('node:os');
+const path = require('node:path');
+const { officialDates, refreshMangaReleaseDates, chapterKey } = require('../metadata/release-dates');
+
+const DAY = 864e5;
+const run = (start, end = null) => ({ start: Date.parse(start), end: end ? Date.parse(end) : null });
+const viz = (chapter, date) => ({ chapter, date, groups: ['Viz', 'MANGA Plus'] });
+const scan = (chapter, date) => ({ chapter, date, groups: ['Some Scans'] });
+
+test('chapter keys only match single numbered chapters', () => {
+  assert.equal(chapterKey('001'), '1');
+  assert.equal(chapterKey('10.50'), '10.5');
+  assert.equal(chapterKey('1-3'), null);
+  assert.equal(chapterKey('7a'), null);
+});
+
+test('simulpub dates are accepted, re-releases after the run are not', () => {
+  const ongoing = officialDates([viz('1', '2023-09-17'), viz('2', '2023-09-24'), scan('2', '2023-09-23')], run('2023-09-18'));
+  assert.deepEqual([...ongoing.accepted.keys()].sort(), ['1', '2'], 'scans a day early do not disqualify');
+  assert.equal(ongoing.accepted.get('1').at, '2023-09-17');
+  assert.equal(ongoing.accepted.get('1').source, 'MangaUpdates · Viz, MANGA Plus');
+  const ended = officialDates([viz('700', '2020-12-20')], run('1999-09-21', '2014-11-10'));
+  assert.equal(ended.accepted.size, 0);
+  assert.equal(ended.rejected.outsideRun, 1);
+  assert.equal(officialDates([viz('1', '2020-01-01')], null).rejected.noRun, 1);
+});
+
+test('an earlier scanlation or source upload means the official date is a re-release', () => {
+  const scanned = officialDates([viz('5', '2021-06-01'), scan('5', '2019-01-01')], run('2018-01-01'));
+  assert.equal(scanned.rejected.earlier, 1);
+  const uploaded = officialDates([viz('5', '2021-06-01')], run('2018-01-01'), new Map([['5', Date.parse('2021-05-01')]]));
+  assert.equal(uploaded.rejected.earlier, 1);
+  const leak = officialDates([viz('5', '2021-06-04')], run('2018-01-01'), new Map([['5', Date.parse('2021-06-04') - 2 * DAY]]));
+  assert.equal(leak.accepted.size, 1, 'an upload two days early is a leak, not an older release');
+});
+
+test('back-catalog chapters dated after newer chapters are out of order', () => {
+  const result = officialDates([
+    viz('8', '2019-02-06'), viz('33', '2019-07-31'), // weekly back-catalog re-release
+    viz('928', '2018-12-21'), viz('930', '2019-01-20'), viz('931', '2019-02-03'), viz('1196', '2026-10-18'),
+    viz('929', '2019-01-21'), // a day after 930: a double release is still in order
+  ], run('1997-07-22'));
+  assert.deepEqual([...result.accepted.keys()].sort((a, b) => a - b), ['928', '929', '930', '931', '1196']);
+  assert.equal(result.rejected.outOfOrder, 2);
+});
+
+async function library(chapters) {
+  const root = await fs.mkdtemp(path.join(os.tmpdir(), 'release-dates-'));
+  const manga = path.join(root, 'Kagurabachi');
+  await fs.mkdir(manga, { recursive: true });
+  await fs.writeFile(path.join(manga, 'manga.json'), JSON.stringify({ title: 'Kagurabachi', year: 2023, 'x-anilist': { id: 169355 } }));
+  for (const [folder, meta] of Object.entries(chapters)) {
+    await fs.mkdir(path.join(manga, folder), { recursive: true });
+    await fs.writeFile(path.join(manga, folder, 'chapter.json'), JSON.stringify(meta));
+  }
+  return manga;
+}
+const fakeSources = releases => ({
+  async findSeries(meta) { return meta.title === 'Kagurabachi' ? { id: '60441951323', title: 'Kagurabachi' } : null; },
+  async originalRun() { return run('2023-09-18'); },
+  async releases() { return releases; },
+});
+
+test('refresh writes x-dates.release, keeps other dates and withdraws its own stale dates', async () => {
+  const manga = await library({
+    'v01/c0001 [WeebCentral]': { title: '', added: '2026-10-01T00:00:00Z', 'x-dates': { version: 1, source: { at: '2024-09-07T10:00:00Z', name: 'WeebCentral', batch: true } } },
+    'v01/c0002 [WeebCentral]': { title: '', 'x-dates': { version: 1, release: { at: '2020-01-01', precision: 'day', source: 'MangaUpdates · Viz' } } },
+    'v01/c0003 [WeebCentral]': { title: '', 'x-dates': { version: 1, release: { at: '2023-10-01', precision: 'day', source: 'Hand-entered' } } },
+  });
+  const dry = await refreshMangaReleaseDates(manga, fakeSources([viz('1', '2023-09-17')]));
+  assert.equal(dry.written, 1);
+  assert.equal(JSON.parse(await fs.readFile(path.join(manga, 'v01/c0001 [WeebCentral]/chapter.json'), 'utf8'))['x-dates'].release, undefined, 'dry run writes nothing');
+  const applied = await refreshMangaReleaseDates(manga, fakeSources([viz('1', '2023-09-17')]), { apply: true });
+  assert.deepEqual([applied.written, applied.cleared], [1, 1]);
+  const one = JSON.parse(await fs.readFile(path.join(manga, 'v01/c0001 [WeebCentral]/chapter.json'), 'utf8'));
+  assert.deepEqual(one['x-dates'].release, { at: '2023-09-17', precision: 'day', source: 'MangaUpdates · Viz, MANGA Plus' });
+  assert.equal(one['x-dates'].source.batch, true);
+  assert.equal(one.added, '2026-10-01T00:00:00Z');
+  assert.equal(JSON.parse(await fs.readFile(path.join(manga, 'v01/c0002 [WeebCentral]/chapter.json'), 'utf8'))['x-dates'].release, undefined);
+  assert.equal(JSON.parse(await fs.readFile(path.join(manga, 'v01/c0003 [WeebCentral]/chapter.json'), 'utf8'))['x-dates'].release.source, 'Hand-entered');
+});
+
+test('seasons that restart their numbering are skipped', async () => {
+  const manga = await library({ 'v01 Season 1/c0001': { title: '' }, 'v02 Season 2/c0001': { title: '' } });
+  const result = await refreshMangaReleaseDates(manga, fakeSources([viz('1', '2023-09-17')]), { apply: true });
+  assert.equal(result.ambiguous, true);
+  assert.equal(result.written, 0);
+});

scraper/test/scraper.test.js

@@ -426,6 +426,9 @@ test('daemon queue state is durable while paused', async t => {
   }));
   const service = await new ScraperService({ libraryRoot: path.join(temp, 'library'), dataRoot }).init();
   t.after(async () => { await service.close(); await fs.rm(temp, { recursive: true, force: true }); });
+  // WeebCentral is off on a fresh installation until an admin switches it on.
+  await assert.rejects(() => service.enqueue({ url: 'https://weebcentral.com/series/01ABCDEF/My-Series' }), /switched off/);
+  await service.updateSettings({ sourcesEnabled: { weebcentral: true } });
   const item = await service.enqueue({ url: 'https://weebcentral.com/series/01ABCDEF/My-Series' });
   assert.equal(item.status, 'queued');
   const scoped = await service.enqueue({ url: 'https://weebcentral.com/series/01ABCDEF/My-Series', scope: { mode: 'latest' } });
@@ -439,6 +442,30 @@ test('daemon queue state is durable while paused', async t => {
   assert.equal((await service.publicState()).queue.length, 0);
 });
 
+test('sources are off by default but existing installations keep the ones they use', async t => {
+  assert.deepEqual(validateSettings({}).sourcesEnabled, { weebcentral: false, mangadex: true, mangapill: false });
+  assert.throws(() => validateSettings({ sourcesEnabled: { weebcentral: 'yes' } }), /true or false/);
+  assert.throws(() => validateSettings({ sourcesEnabled: { other: true } }), /Unknown source/);
+  const temp = await fs.mkdtemp(path.join(os.tmpdir(), 'mcscraper-sources-'));
+  const dataRoot = path.join(temp, 'data');
+  await fs.mkdir(dataRoot, { recursive: true });
+  const { sourcesEnabled, ...older } = validateSettings({});
+  await fs.writeFile(path.join(dataRoot, 'queue.json'), JSON.stringify({
+    version: 3, paused: true, status: 'paused', settings: older, publishedRevision: 0,
+    queue: [{ id: 'A', title: 'A', url: 'https://weebcentral.com/series/AAA/A', source: 'weebcentral', status: 'queued', order: 0 }],
+    history: [],
+  }));
+  const service = await new ScraperService({ libraryRoot: path.join(temp, 'library'), dataRoot }).init();
+  t.after(async () => { await service.close(); await fs.rm(temp, { recursive: true, force: true }); });
+  assert.deepEqual(service.state.settings.sourcesEnabled, { weebcentral: true, mangadex: true, mangapill: false });
+  await assert.rejects(() => service.enqueue({ url: 'https://mangapill.com/manga/2/one-piece' }), /MangaPill is switched off/);
+  // Switching a source off keeps its jobs queued instead of failing them.
+  await service.updateSettings({ sourcesEnabled: { weebcentral: false } });
+  assert.equal(service.sourceEnabled('weebcentral'), false);
+  assert.equal(service.sourceEnabled('peer-abc'), true, 'peers have no switch');
+  assert.equal(service.state.queue[0].status, 'queued');
+});
+
 test('retry recovers stale downloading entries before work resumes', async t => {
   const temp = await fs.mkdtemp(path.join(os.tmpdir(), 'mcscraper-retry-'));
   const dataRoot = path.join(temp, 'data');

scraper/test/source-plugin.test.js

@@ -10,8 +10,8 @@ const { adapters, adapterFor } = require('../adapters');
 const seriesUrl = 'https://weebcentral.com/series/ABC/title';
 const chapterUrl = 'https://weebcentral.com/chapters/XYZ';
 const seriesHtml = '<h1>Test Manga</h1><div hx-get="/series/ABC/full-chapter-list"></div>';
-const listHtml = '<a href="/chapters/XYZ">Chapter 1</a><a href="/chapters/LMN">Log. 2</a>';
-const chapterHtml = '<time datetime="2026-01-01"></time>';
+const listHtml = '<a href="/chapters/XYZ"><span>Chapter 1</span><time datetime="2026-01-01T15:06:05.348Z">x</time></a><a href="/chapters/LMN">Log. 2</a>';
+const chapterHtml = '<div>no dates on the chapter page</div>';
 const imagesHtml = '<img src="https://img.example/1.jpg"><img data-src="https://img.example/2.jpg">';
 
 test('first-party registry rejects unknown plugins and malformed input', () => {
@@ -45,7 +45,9 @@ test('in-process parser preserves WeebCentral catalog and page discovery', async
   assert.deepEqual(catalog.chapters.map(chapter => chapter.number), ['1', '2']);
   const chapter = await discoverWeebChapter(catalog.chapters[0], http);
   assert.equal(chapter.pages.length, 2);
-  assert.equal(chapter.added, '2026-01-01');
+  assert.equal(chapter.added, undefined);
+  assert.deepEqual(chapter.dates, { source: { at: '2026-01-01T15:06:05.348Z', name: 'WeebCentral', batch: false } });
+  assert.equal(catalog.chapters[1].uploadedAt, undefined);
   assert.equal(requested.length, 4);
 });
 

sentinel/daemon.js

@@ -1,51 +0,0 @@
-#!/usr/bin/env node
-'use strict';
-const fs = require('fs'), path = require('path'), http = require('http');
-const { secret } = require('../runtime/config');
-const { Monitor } = require('./monitor');
-const root = path.resolve(process.env.SENTINEL_DATA || path.join(process.env.DATA || 'data', 'sentinel'));
-fs.mkdirSync(root, { recursive: true });
-const token = secret('SCRAPER_TOKEN');
-const host = process.env.SENTINEL_HOST || '127.0.0.1', port = +(process.env.SENTINEL_PORT || 8092);
-if (!['127.0.0.1', 'localhost', '::1'].includes(host) && !token) throw new Error('SCRAPER_TOKEN is required for non-loopback Sentinel');
-const reader = process.env.READER_URL || 'http://127.0.0.1:8080';
-const scraper = process.env.SCRAPER_URL || 'http://127.0.0.1:8091';
-const file = path.join(root, 'state.json');
-let saved = {};
-if (fs.existsSync(file)) saved = JSON.parse(fs.readFileSync(file, 'utf8'));
-const monitor = new Monitor({ incidents: saved.incidents || [] });
-const request = async (base, endpoint, body) => {
-  const response = await fetch(base + endpoint, { headers: { ...(token ? { authorization: `Bearer ${token}` } : {}), 'content-type': 'application/json' },
-    signal: AbortSignal.timeout(8000), ...(body ? { method: 'POST', body: JSON.stringify(body) } : {}) });
-  if (!response.ok) throw new Error(`Service HTTP ${response.status}`);
-  return response.json();
-};
-let stopped = false, timer, lastTick = Date.now();
-async function tick() {
-  try {
-    const results = await Promise.allSettled([request(reader, '/health'), request(scraper, '/api/state')]);
-    const state = await monitor.observe(results[0].status === 'fulfilled' ? results[0].value : null,
-      results[1].status === 'fulfilled' ? results[1].value : null, input => request(scraper, '/api/recovery', input));
-    fs.writeFileSync(file + '.tmp', JSON.stringify(state, null, 2) + '\n'); fs.renameSync(file + '.tmp', file);
-    lastTick = Date.now();
-    for (const incident of state.incidents.filter(i => i.state === 'open' && !i.logged)) {
-      console.log(JSON.stringify({ ...incident, targetService: incident.service, service: 'sentinel', level: 'warning' })); incident.logged = true;
-    }
-  } catch (error) { console.error('[sentinel]', error.message); }
-  if (!stopped) timer = setTimeout(tick, 30000);
-}
-const server = http.createServer((req, res) => {
-  const endpoint = new URL(req.url, 'http://localhost').pathname;
-  let code = 200, value;
-  if (endpoint === '/health') { code = Date.now() - lastTick < 120000 ? 200 : 503; value = { ok: code === 200, service: 'sentinel', checkedAt: monitor.state.checkedAt }; }
-  else if (token && req.headers.authorization !== `Bearer ${token}`) { code = 401; value = { error: 'Unauthorized' }; }
-  else if (endpoint === '/api/state' && req.method === 'GET') value = monitor.state;
-  else if (endpoint === '/metrics' && req.method === 'GET') {
-    res.writeHead(200, { 'content-type': 'text/plain; version=0.0.4' });
-    return res.end(`mangacircus_reader_up ${Number(!!monitor.state.services.reader)}\nmangacircus_scraper_up ${Number(!!monitor.state.services.scraper)}\nmangacircus_open_incidents ${monitor.incidents.filter(i => i.state === 'open').length}\n`);
-  } else { code = 404; value = { error: 'Not found' }; }
-  res.writeHead(code, { 'content-type': 'application/json', 'cache-control': 'no-store' }); res.end(JSON.stringify(value));
-});
-server.listen(port, host, () => console.log(JSON.stringify({ service: 'sentinel', event: 'started', port })));
-tick();
-for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => { stopped = true; clearTimeout(timer); server.close(() => process.exit(0)); });

sentinel/monitor.js

@@ -1,65 +0,0 @@
-'use strict';
-const { progressKey } = require('../scraper/lib/failures');
-
-// No source requests, filesystem checkpoint edits, shell execution or Docker socket.
-class Monitor {
-  constructor(options = {}) {
-    this.now = options.now || Date.now;
-    this.stallMs = options.stallMs || 900000;
-    this.observations = new Map();
-    this.incidents = options.incidents || [];
-    this.state = { status: 'starting', services: {}, incidents: this.incidents };
-  }
-  incident(key, message, extra = {}) {
-    let incident = this.incidents.find(i => i.key === key && i.state === 'open');
-    if (!incident) {
-      incident = { key, message, state: 'open', at: new Date(this.now()).toISOString(), ...extra };
-      this.incidents.push(incident);
-      if (this.incidents.length > 200) this.incidents.splice(0, this.incidents.length - 200);
-    }
-    return incident;
-  }
-  resolve(key) {
-    for (const i of this.incidents) if (i.key === key && i.state === 'open') { i.state = 'resolved'; i.resolvedAt = new Date(this.now()).toISOString(); }
-  }
-  async observe(reader, scraper, recover) {
-    const now = this.now();
-    this.state.checkedAt = new Date(now).toISOString();
-    this.state.services = { reader: !!reader?.ok, scraper: !!scraper };
-    if (reader?.ready === false) this.incident('reader:scan', 'Reader is alive but its library scan failed; inspect storage and scan diagnostics', { service: 'reader' });
-    else if (reader?.ok) this.resolve('reader:scan');
-    for (const [service, healthy] of Object.entries(this.state.services)) {
-      if (healthy) this.resolve(`service:${service}`);
-      else this.incident(`service:${service}`, `${service} is unreachable or unhealthy; inspect its supervisor`, { service });
-    }
-    if (!scraper) { this.state.status = 'degraded'; return this.state; }
-    const ids = new Set();
-    for (const entry of scraper.active || []) {
-      const id = entry.current.id; ids.add(id);
-      // Without a checkpoint, a null progress key stays constant forever and
-      // cannot prove a stall. Do not quarantine a live worker on that basis.
-      if (!entry.job) { this.observations.delete(id); this.resolve(`stall:${id}`); continue; }
-      const key = progressKey(entry.job), prior = this.observations.get(id);
-      if (!prior || prior.key !== key) { this.observations.set(id, { key, at: now }); this.resolve(`stall:${id}`); continue; }
-      if (scraper.paused || scraper.sourcePaused?.[entry.source]) continue;
-      const allowance = Math.max(this.stallMs, Number(entry.job?.rate?.chapterMaxMs || 0) + 180000);
-      if (now - prior.at < allowance) continue;
-      const incident = this.incident(`stall:${id}`, 'Job progress has stopped; recovery is limited to one checkpoint-safe attempt', { jobId: id, source: entry.source, title: entry.current.title });
-      if (!incident.attempted && recover) {
-        // Persist attempted in the daemon via stopReason/stallRecoveries; monitor records audit too.
-        incident.attempted = true;
-        try { incident.recovery = await recover({ id, progressKey: key }); }
-        catch { incident.recovery = { changed: false, error: 'Recovery API unavailable' }; }
-      }
-    }
-    for (const id of this.observations.keys()) if (!ids.has(id)) { this.observations.delete(id); this.resolve(`stall:${id}`); }
-    const failures = (scraper.queue || []).filter(item => ['error', 'quarantined'].includes(item.status));
-    for (const item of failures) this.incident(`job:${item.id}`, item.failure?.message || item.error || 'Job needs attention', { jobId: item.id, source: item.source, kind: item.failure?.kind || 'unknown' });
-    for (const i of this.incidents) if (i.key.startsWith('job:') && !failures.some(item => i.key === `job:${item.id}`)) this.resolve(i.key);
-    this.state.sourcePaused = scraper.sourcePaused || {};
-    this.state.active = (scraper.active || []).map(e => ({ source: e.source, id: e.current.id, title: e.current.title, completed: e.job?.completed, total: e.job?.total, updated: e.job?.updated }));
-    this.state.status = this.incidents.some(i => i.state === 'open') ? 'attention' : scraper.paused ? 'paused' : 'healthy';
-    return this.state;
-  }
-}
-module.exports = { Monitor };

sentinel/test/monitor.test.js

@@ -1,38 +0,0 @@
-'use strict';
-const test = require('node:test'), assert = require('node:assert/strict');
-const { Monitor } = require('../monitor');
-test('Sentinel does not disturb progressing or deliberately paused jobs', async () => {
-  let now = 0, recoveries = 0;
-  const monitor = new Monitor({ now: () => now, stallMs: 100 });
-  const state = { paused: false, active: [{ source: 'a', current: { id: 'job', title: 'Manga' }, job: { completed: 0, current: { downloadedPages: 1 } } }], queue: [] };
-  const recover = async () => { recoveries++; return { changed: true }; };
-  await monitor.observe({ ok: true }, state, recover);
-  now = 500000; state.active[0].job.current.downloadedPages = 2;
-  await monitor.observe({ ok: true }, state, recover);
-  now = 1000000; state.paused = true;
-  await monitor.observe({ ok: true }, state, recover);
-  assert.equal(recoveries, 0);
-  state.paused = false; await monitor.observe({ ok: true }, state, recover); await monitor.observe({ ok: true }, state, recover);
-  assert.equal(recoveries, 1);
-  state.active[0].job.completed = 1; await monitor.observe({ ok: true }, state, recover);
-  assert.equal(monitor.incidents.find(i => i.key === 'stall:job').state, 'resolved');
-});
-test('Sentinel reports service failures without restart loops and resolves recovery', async () => {
-  const monitor = new Monitor();
-  await monitor.observe(null, null); await monitor.observe(null, null);
-  assert.equal(monitor.incidents.length, 2);
-  await monitor.observe({ ok: true }, { active: [], queue: [], paused: false });
-  assert.equal(monitor.state.status, 'healthy'); assert.ok(monitor.incidents.every(i => i.state === 'resolved'));
-});
-
-test('missing checkpoint does not trigger a false stall recovery', async () => {
-  let now = 0, recoveries = 0;
-  const monitor = new Monitor({ now: () => now, stallMs: 100 });
-  const state = { paused: false, active: [{ source: 'mangapill', current: { id: 'job', title: 'Manga' }, job: null }], queue: [] };
-  for (const advance of [0, 1000000, 2000000]) {
-    now = advance;
-    await monitor.observe({ ok: true }, state, async () => { recoveries++; return { changed: true }; });
-  }
-  assert.equal(recoveries, 0);
-  assert.equal(monitor.incidents.some(item => item.key === 'stall:job'), false);
-});

server.js

@@ -7,9 +7,11 @@
  *   SIGNUP=closed|open   (the very first account can always be created and becomes admin)
  *   SCAN_INTERVAL=10     minutes between shallow change checks, 0 = off
  *   FULL_SCAN_INTERVAL=1440  minutes between full reconciliations, 0 = off
+ *   SCAN_CONCURRENCY=8   chapter folders / images probed at once by an uncached scan (library.js)
  *   COOKIE_SECURE=1      mark the session cookie Secure (when served over HTTPS)
  *   SCRAPER_URL=http://127.0.0.1:8091   scraper daemon API
  *   SCRAPER_TOKEN=...                    shared daemon token (optional on loopback)
+ *   BACKUP=/backup       folder for built-in DATA backups (unset = backups off)
  */
 const http = require('http'), fs = require('fs'), path = require('path'), crypto = require('crypto');
 const { scanLibrary, scanChangedLibrary, libraryStamps, snapshotIndex, restoreIndex } = require('./library');
@@ -17,15 +19,22 @@ const { Catalog, CatalogError, collectionViews, TYPES, ROLES } = require('./cata
 const { RATINGS: GREATEST_RATINGS, loadCatalog: loadGreatestCatalog, findOwned: findGreatestOwned,
   countsFor: greatestCounts, scoreFor: greatestScore } = require('./greatest');
 const { hosterLinks } = require('./greatest/hosters');
+const { GreatestCovers } = require('./greatest/covers');
 const { secret } = require('./runtime/config');
 const { PeeringState, requirePeer } = require('./peering/state');
 const peerClient = require('./peering/client');
 const { DiscoveryStore } = require('./discovery/store');
-const { generate: generateDiscovery } = require('./discovery/engine');
+const { generateAsync: generateDiscovery, mergeUserFeed } = require('./discovery/engine');
+const { DiscoveryCache } = require('./discovery/cache');
+const { createClient: createAniListClient } = require('./discovery/anilist');
+const { anchorsFor } = require('./discovery/anchors');
+const { shouldRunNightly, Harvester } = require('./discovery/harvest');
 const {
-  applyPulse, applyVisit, chapterInsight, eligiblePages, emptyRecord, libraryInsight, mangaInsight, publicSummary,
+  applyPulse, applyVisit, chapterInsight, earliestCompletion, eligiblePages, emptyRecord, libraryInsight, mangaInsight, publicSummary,
 } = require('./reading-analytics');
+const { chapterDates } = require('./chapter-dates');
 const { parseSchedule, publicSchedule, loadQuestionBank, createReadingAccess } = require('./reading-access');
+const { BackupService, BackupError } = require('./backup');
 
 const PORT = +process.env.PORT || 8080;
 const LIB = path.resolve(process.env.LIBRARY || 'library');
@@ -36,11 +45,13 @@ const FULL_SCAN_MINUTES = +(process.env.FULL_SCAN_INTERVAL || 1440);
 const SECURE_COOKIE = process.env.COOKIE_SECURE === '1';
 const SCRAPER_URL = String(process.env.SCRAPER_URL || 'http://127.0.0.1:8091').replace(/\/$/, '');
 const SCRAPER_TOKEN = secret('SCRAPER_TOKEN');
-const SENTINEL_URL = String(process.env.SENTINEL_URL || 'http://127.0.0.1:8092').replace(/\/$/, '');
 const PEER_DATA = path.resolve(process.env.PEER_DATA || path.join(DATA, 'peering'));
 const peerStore = new PeeringState(PEER_DATA);
 const discoveryStore = new DiscoveryStore(DATA);
 const DISCOVERY_CACHE = path.join(DATA, 'scraper', 'metadata', 'anilist');
+const DISCOVERY_HARVEST_CACHE = path.join(DATA, 'discovery-cache');
+const HARVEST_TIME = /^([01]\d|2[0-3]):[0-5]\d$/;
+const DISCOVERY_HARVEST_AT = HARVEST_TIME.test(process.env.DISCOVERY_HARVEST_AT || '') ? process.env.DISCOVERY_HARVEST_AT : '03:30';
 let discoveryRun = null, discoveryError = null;
 const PUB = path.join(__dirname, 'public');
 const IMG = /\.(jpe?g|png|webp|gif|avif|bmp)$/i;
@@ -55,6 +66,10 @@ fs.mkdirSync(DATA, { recursive: true });
 const catalog = new Catalog(DATA);
 const greatestCatalog = loadGreatestCatalog();
 const greatestById = new Map(greatestCatalog.entries.map(entry => [entry.id, entry]));
+const greatestCovers = new GreatestCovers(path.join(DATA, 'greatest-covers'));
+// The versioned URL lets browsers keep a cover for a year; a new catalog URL changes `v`.
+const greatestCoverUrl = entry => entry.cover
+  ? `api/greatest-cover?id=${encodeURIComponent(entry.id)}&v=${greatestCovers.version(entry.cover)}` : null;
 const DB_FILE = path.join(DATA, 'db.json'), CACHE_FILE = path.join(DATA, 'scan-cache.json'), INDEX_FILE = path.join(DATA, 'library-index.json');
 const db = { users: {}, sessions: {}, progress: {}, reading: {}, readingAccess: {}, follows: {}, notes: {}, votes: {}, greatestRatings: {}, discoveryFeedback: {}, known: {} };
 // A corrupt db.json stops the server instead of silently starting empty and overwriting it.
@@ -65,6 +80,9 @@ function save() { saveTimer = saveTimer || setTimeout(flush, 500); }
 function flush() { clearTimeout(saveTimer); saveTimer = null; writeAtomic(DB_FILE, JSON.stringify(db)); }
 for (const sig of ['SIGINT', 'SIGTERM']) process.on(sig, () => { flush(); process.exit(0); });
 db.readingAccess ||= {};
+// The reader owns backups: it alone mounts all of DATA and holds db.json in memory.
+const backup = new BackupService({ dataRoot: DATA, backupRoot: process.env.BACKUP || '', snapshotDb: () => JSON.stringify(db),
+  scraper: { state: () => scraperRequest('/api/state'), control: action => scraperRequest('/api/control', { action }) } }).start();
 const readingSchedule = parseSchedule(process.env.READING_WINDOWS_JSON || '', process.env.READING_TIMEZONE || process.env.TZ || 'Europe/Berlin');
 const readingAccess = createReadingAccess({ schedule: readingSchedule,
   bank: loadQuestionBank(process.env.READING_QUIZ_CATALOG_PATH || ''),
@@ -72,8 +90,9 @@ const readingAccess = createReadingAccess({ schedule: readingSchedule,
   getProgress: (user, mangaId) => db.progress[user]?.[mangaId] || {},
   getMangaTitle: mangaId => lib.byId.get(mangaId)?.title || null });
 
-class HttpError extends Error { constructor(code, msg) { super(msg); this.code = code; } }
-function bad(code, msg) { throw new HttpError(code, msg); }
+// `extra` fields are sent next to `error`, e.g. `nextAllowedAt` on a 429.
+class HttpError extends Error { constructor(code, msg, extra = {}) { super(msg); this.code = code; this.extra = extra; } }
+function bad(code, msg, extra) { throw new HttpError(code, msg, extra); }
 function peerForRoute(id) {
   try { return requirePeer(peerStore.load(), id); }
   catch { bad(404, 'Peer not found'); }
@@ -92,7 +111,7 @@ function checkPw(pw, stored) {
 }
 const DUMMY_HASH = hashPw('x');
 const DEFAULTS = { startMode: 'last', lastMode: 'page', adapt: true, skipExtras: false, skipCovers: false, flowPageGap: true, pageTone: 'original', readingClock: false, readingStory: true,
-  discoveryEnabled: false, discoveryReadingActivity: false, discoveryMature: false, zoom: {}, webhook: '' };
+  readDate: 'hour', releaseDate: true, discoveryEnabled: false, discoveryReadingActivity: false, discoveryMature: false, zoom: {}, webhook: '' };
 const settingsOf = u => ({ ...DEFAULTS, ...u.settings, zoom: { ...(u.settings && u.settings.zoom) } });
 
 function createUser(name, pw, admin, replace) {
@@ -235,6 +254,26 @@ function readDiscovery() {
   try { return discoveryStore.load(); }
   catch (error) { bad(503, error.message); }
 }
+// Scoring is local: it reads the harvest cache and never calls AniList. With `onlyUser` only that
+// feed is rebuilt and the other users' feeds are carried over into the same publish.
+async function rescoreNow(onlyUser) {
+  const startedAt = new Date().toISOString();
+  let queuedTitles = [], queueAvailable = true;
+  try { queuedTitles = (await scraperRequest('/api/state')).queue?.filter(item => !['complete', 'removed'].includes(item.status)).map(item => item.title) || []; }
+  catch { queueAvailable = false; }
+  const result = await generateDiscovery({ db, library: lib.mangas, greatest: greatestCatalog,
+    cacheDir: DISCOVERY_CACHE, harvestCache, queuedTitles, onlyUser });
+  // Read the prior store only now, right before the synchronous publish, so no other publish slips between.
+  const value = onlyUser === undefined ? result : { ...result, ...mergeUserFeed(discoveryStore.load(), result, onlyUser) };
+  return { published: discoveryStore.publish({ ...value, startedAt }), queueAvailable };
+}
+// One rescore at a time: an admin run and a harvest follow-up must not publish over each other.
+let rescoring = Promise.resolve();
+function rescore({ onlyUser } = {}) {
+  const run = rescoring.then(() => rescoreNow(onlyUser));
+  rescoring = run.catch(() => {});
+  return run;
+}
 function startDiscoveryRun() {
   if (discoveryRun?.status === 'running') bad(409, 'A discovery run is already active');
   readDiscovery(); // Refuse a new run if the on-disk store is corrupt.
@@ -243,12 +282,7 @@ function startDiscoveryRun() {
   discoveryError = null;
   setImmediate(async () => {
     try {
-      let queuedTitles = [], queueAvailable = true;
-      try { queuedTitles = (await scraperRequest('/api/state')).queue?.filter(item => !['complete', 'removed'].includes(item.status)).map(item => item.title) || []; }
-      catch { queueAvailable = false; }
-      const result = generateDiscovery({ db, library: lib.mangas, greatest: greatestCatalog,
-        cacheDir: DISCOVERY_CACHE, queuedTitles });
-      const published = discoveryStore.publish({ ...result, startedAt });
+      const { published, queueAvailable } = await rescore();
       discoveryRun = { id, status: 'success', startedAt, finishedAt: published.lastRun.finishedAt,
         candidates: published.lastRun.candidateCount, queueAvailable };
     } catch (error) {
@@ -260,6 +294,62 @@ function startDiscoveryRun() {
   return discoveryRun;
 }
 
+// ---------- discovery harvest (AniList seeds, BL-0051) ----------
+const harvestCache = new DiscoveryCache(DISCOVERY_HARVEST_CACHE);
+const discoverySettings = state => ({ harvestEnabled: state.settings?.harvestEnabled ?? true,
+  harvestAt: state.settings?.harvestAt ?? DISCOVERY_HARVEST_AT });
+const localDate = date => [date.getFullYear(), String(date.getMonth() + 1).padStart(2, '0'), String(date.getDate()).padStart(2, '0')].join('-');
+function nextNightlyAt(state, now = new Date()) {
+  const { harvestEnabled, harvestAt } = discoverySettings(state);
+  if (!harvestEnabled) return null;
+  const [hours, minutes] = harvestAt.split(':').map(Number);
+  const slot = new Date(now);
+  slot.setHours(hours, minutes, 0, 0);
+  if (state.nightly?.lastDate === localDate(now)) slot.setDate(slot.getDate() + 1);
+  else if (slot < now) return now.toISOString(); // due: the next minute's check starts it
+  return slot.toISOString();
+}
+function harvestTargets(kind, userName) {
+  const users = kind === 'user' ? [db.users[userName]].filter(Boolean) : Object.values(db.users);
+  return users.filter(user => settingsOf(user).discoveryEnabled).map(user => ({ user,
+    anchors: anchorsFor({ user, db, library: lib.mangas, greatest: greatestCatalog }), mature: !!settingsOf(user).discoveryMature }));
+}
+// Queued jobs live in memory only; a run cut off by a restart is recorded as interrupted.
+try {
+  const { harvest } = discoveryStore.load();
+  if (harvest?.status === 'running') discoveryStore.patch({ harvest: { ...harvest, status: 'interrupted', finishedAt: new Date().toISOString() } });
+} catch (error) { console.error('[discovery]', error.message); }
+const harvester = new Harvester({ cache: harvestCache, client: createAniListClient(), store: discoveryStore,
+  targetsFor: harvestTargets, onDone: async job => {
+    // A failed follow-up rescore must reach the admin panel (lastError), not only the log.
+    try {
+      await rescore(job.kind === 'user' ? { onlyUser: job.user } : {});
+      discoveryError = null; // a later success must not leave an old failure on the admin panel
+    } catch (error) { discoveryError = `Re-scoring after harvest ${job.id} failed: ${String(error.message || error)}`.slice(0, 300); throw error; }
+  } });
+function requestHarvest(args) {
+  readDiscovery(); // A corrupt store fails closed before any AniList call.
+  try { return harvester.request(args); }
+  catch (error) {
+    if (!error.status) throw error;
+    const extra = {};
+    for (const key of ['nextAllowedAt', 'blockedReason']) if (error[key]) extra[key] = error[key];
+    bad(error.status, error.message, extra);
+  }
+}
+function checkNightly() {
+  let state;
+  try { state = discoveryStore.load(); } catch { return; }
+  const now = new Date(), { harvestEnabled, harvestAt } = discoverySettings(state);
+  if (!shouldRunNightly({ now, harvestAt, lastNightlyDate: state.nightly?.lastDate, enabled: harvestEnabled })) return;
+  if (harvester.current()) return;
+  // The date is written only once the run was accepted; a 409 (refresh in flight) retries next minute.
+  try { harvester.request({ kind: 'nightly' }); }
+  catch (error) { if (error.status !== 409) console.error('[discovery] nightly harvest:', error.message); return; }
+  try { discoveryStore.patch({ nightly: { lastDate: localDate(now) } }); }
+  catch (error) { console.error('[discovery]', error.message); }
+}
+
 const mangaDexSearchCache = new Map();
 let lastMangaDexSearch = 0;
 async function searchGreatestMangaDex(entry) {
@@ -317,6 +407,13 @@ function firstReleaseOf(meta) {
   }
   return { label: label || '', time: Number.isFinite(time) ? time : 0 };
 }
+// When a done chapter was completed: the progress record, or for reads saved before doneAt
+// existed, the earliest completion the reading clock still remembers.
+function readAtOf(progress, record) {
+  if (!progress || !progress.done) return null;
+  if (progress.doneAt) return { at: progress.doneAt, approx: false };
+  return earliestCompletion(record);
+}
 function continueOf(m, prog, s) {
   const ok = c => !(s.skipExtras && c.extra);
   let last = null;
@@ -375,9 +472,19 @@ const routes = {
   },
   'GET /api/admin/health': async ({ u }) => {
     admin(u);
-    const sentinel = await fetch(SENTINEL_URL + '/api/state', { headers: SCRAPER_TOKEN ? { authorization: `Bearer ${SCRAPER_TOKEN}` } : {}, signal: AbortSignal.timeout(4000) })
-      .then(async r => { if (!r.ok) throw new Error(); return r.json(); }).catch(() => ({ status: 'unreachable' }));
-    return { reader: { ok: !scanState.error, uptime: Math.round(process.uptime()), scan: scanState }, sentinel };
+    return { reader: { ok: !scanState.error, uptime: Math.round(process.uptime()), scan: scanState }, backup: backup.publicState() };
+  },
+  'GET /api/admin/backup': ({ u }) => { admin(u); return backup.publicState(); },
+  'POST /api/admin/backup': ({ u, b }) => {
+    admin(u);
+    try {
+      if (b.command === 'settings') return backup.updateSettings(b.settings);
+      if (b.command !== 'run') bad(400, 'Unknown backup command');
+      if (!backup.backupRoot) bad(409, 'Backups are off: set BACKUP to a mounted folder');
+      if (backup.running) bad(409, 'A backup is already running');
+      backup.run('manual').catch(() => {}); // the result lands in last/history
+      return backup.publicState();
+    } catch (e) { if (e instanceof BackupError) bad(e.code, e.message); throw e; }
   },
   'GET /api/auth': () => ({ firstRun: !Object.keys(db.users).length, signup: OPEN_SIGNUP }),
 
@@ -422,7 +529,8 @@ const routes = {
     if (['last', 'flow', 'page', 'book'].includes(b.startMode)) s.startMode = b.startMode;
     if (['flow', 'page', 'book'].includes(b.lastMode)) s.lastMode = b.lastMode;
     if (['original', 'inverted', 'sepia', 'dim', 'extra-dim'].includes(b.pageTone)) s.pageTone = b.pageTone;
-    for (const k of ['adapt', 'skipExtras', 'skipCovers', 'flowPageGap', 'readingClock', 'readingStory',
+    if (['off', 'date', 'hour'].includes(b.readDate)) s.readDate = b.readDate;
+    for (const k of ['adapt', 'skipExtras', 'skipCovers', 'flowPageGap', 'readingClock', 'readingStory', 'releaseDate',
       'discoveryEnabled', 'discoveryReadingActivity', 'discoveryMature']) if (typeof b[k] === 'boolean') s[k] = b[k];
     if (b.zoom && typeof b.zoom === 'object') {
       s.zoom = {};
@@ -470,6 +578,7 @@ const routes = {
       discoveryReadingActivity: !!settingsOf(u).discoveryReadingActivity,
       discoveryMature: !!settingsOf(u).discoveryMature },
     generatedAt: feed?.generatedAt || null, revision: state.revision, lastRun: state.lastRun,
+    refresh: harvester.state(u.name),
     feedback: db.discoveryFeedback?.[u.name] || {},
     items: (feed?.items || []).map(item => ({ ...item, candidate: state.candidates[item.candidateId] || null })).filter(item => item.candidate) };
   },
@@ -483,14 +592,30 @@ const routes = {
     save();
     return { candidateId: id, value };
   },
+  'POST /api/discovery/refresh': ({ u }) => {
+    if (!settingsOf(u).discoveryEnabled) bad(403, 'Enable personalized discovery first');
+    return requestHarvest({ kind: 'user', user: u.name });
+  },
   'GET /api/admin/discovery': ({ u }) => {
     admin(u);
     const state = readDiscovery();
     return { revision: state.revision, lastRun: state.lastRun, currentRun: discoveryRun,
       lastError: discoveryError, optedInUsers: Object.values(db.users).filter(user => user.settings?.discoveryEnabled).length,
-      candidateCount: Object.keys(state.candidates).length };
+      candidateCount: Object.keys(state.candidates).length,
+      harvest: state.harvest, currentHarvest: harvester.current(), cache: harvestCache.stats(),
+      settings: discoverySettings(state), nextNightlyAt: nextNightlyAt(state) };
   },
   'POST /api/admin/discovery/run': ({ u }) => { admin(u); return startDiscoveryRun(); },
+  'POST /api/admin/discovery/harvest': ({ u }) => { admin(u); return requestHarvest({ kind: 'admin' }); },
+  'POST /api/admin/discovery/settings': ({ u, b }) => {
+    admin(u);
+    const keys = b && typeof b === 'object' && !Array.isArray(b) ? Object.keys(b).sort().join() : '';
+    if (keys !== 'harvestAt,harvestEnabled' || typeof b.harvestEnabled !== 'boolean' || typeof b.harvestAt !== 'string' || !HARVEST_TIME.test(b.harvestAt))
+      bad(400, 'Send exactly { harvestEnabled: true|false, harvestAt: "HH:MM" }');
+    readDiscovery();
+    const state = discoveryStore.patch({ settings: { harvestEnabled: b.harvestEnabled, harvestAt: b.harvestAt } });
+    return { settings: discoverySettings(state), nextNightlyAt: nextNightlyAt(state) };
+  },
 
   'GET /api/greatest': ({ u }) => ({
     title: greatestCatalog.title, updatedAt: greatestCatalog.updatedAt, periods: greatestCatalog.periods,
@@ -500,7 +625,7 @@ const routes = {
       const counts = greatestCounts(entry.id, db.greatestRatings || {}, db.users);
       return {
         ...entry,
-        cover: owned?.cover ? url(owned.cover) : entry.cover,
+        cover: owned?.cover ? url(owned.cover) : greatestCoverUrl(entry),
         chapters: owned ? owned.chapters.length : entry.chapters,
         status: owned?.meta?.status || entry.status,
         library: owned ? { id: owned.id, chapters: owned.chapters.length } : null,
@@ -542,9 +667,10 @@ const routes = {
   },
 
   'GET /api/manga': ({ u, q }) => {
-    const m = mangaOf(q.get('m')), prog = progressOf(u, m), showStory = settingsOf(u).readingStory;
+    const m = mangaOf(q.get('m')), prog = progressOf(u, m), s = settingsOf(u), showStory = s.readingStory;
     const records = mangaReadingOf(u, m);
     return {
+      chapterDates: { read: s.readDate, release: s.releaseDate },
       id: m.id, title: m.title, meta: m.meta, cover: url(m.cover), direction: m.direction,
       collections: catalog.state.collections.filter(c => c.members.some(member => member.mangaId === m.id)).map(c => ({ id: c.id, title: c.title })),
       relations: catalog.state.relations.filter(r => r.state === 'confirmed' && (r.from === m.id || r.to === m.id)).map(r => ({ ...r,
@@ -554,6 +680,8 @@ const routes = {
       chapters: m.chapters.map(ch => ({
         id: ch.id, book: ch.book, label: ch.label, title: ch.title, extra: ch.extra, progress: prog[ch.id] || null, versions: versionsOf(m, ch, u),
         reading: showStory ? chapterInsight(records[ch.id]) : null,
+        readAt: s.readDate === 'off' ? null : readAtOf(prog[ch.id], records[ch.id]),
+        dates: s.releaseDate ? chapterDates(ch.versions) : null,
       })),
       reading: showStory ? mangaInsight(records, m.chapters.map(ch => ch.id)) : null,
       continue: continueOf(m, prog, settingsOf(u)),
@@ -622,8 +750,11 @@ const routes = {
   'POST /api/progress': ({ u, b }) => {
     const m = mangaOf(b.m), ch = chapterOf(m, b.c);
     const all = (db.progress[u.name] = db.progress[u.name] || {}), mp = (all[m.id] = all[m.id] || {}), old = mp[ch.id] || {};
+    const now = Date.now(), done = !!(old.done || b.done);
+    // doneAt: when the chapter was first completed; kept until it is marked unread.
+    const doneAt = old.done ? old.doneAt : done ? now : undefined;
     mp[ch.id] = {
-      v: String(b.v || ''), p: Math.max(0, b.p | 0), n: Math.max(0, b.n | 0), t: Date.now(), done: !!(old.done || b.done),
+      v: String(b.v || ''), p: Math.max(0, b.p | 0), n: Math.max(0, b.n | 0), t: now, done, ...(doneAt ? { doneAt } : {}),
       shift: Array.isArray(b.shift) ? b.shift.filter(Number.isInteger).slice(0, 1000) : old.shift || [],
     };
     save();
@@ -718,6 +849,21 @@ const routes = {
     createUser(name, b.password, name === u.name || !!b.admin, true);
     return { ok: true };
   },
+  // Grant or revoke admin rights without touching the password. Admins only (checked here, not
+  // just hidden in the UI); nobody can revoke their own rights, so at least one admin always remains.
+  'POST /api/admin/user/role': ({ u, b }) => {
+    admin(u);
+    const name = String(b.name || '').trim().toLowerCase();
+    const target = db.users[name] || bad(404, 'No such user.');
+    if (typeof b.admin !== 'boolean') bad(400, 'admin must be true or false.');
+    if (name === u.name && !b.admin) bad(400, "You can't remove your own admin rights.");
+    if (!!target.admin !== b.admin) {
+      target.admin = b.admin;
+      save();
+      console.log(`[users] ${u.name} ${b.admin ? 'made' : 'removed'} ${name} ${b.admin ? 'an admin' : 'as admin'}`);
+    }
+    return { ok: true, name, admin: target.admin };
+  },
   'POST /api/admin/user/delete': ({ u, b }) => {
     admin(u);
     const name = String(b.name || '');
@@ -893,6 +1039,16 @@ http.createServer(async (req, res) => {
       if (!c?.customCover) bad(404, 'Cover not found');
       return serveFile(req, res, path.join(DATA, 'collection-covers'), c.customCover, 'private, no-cache', IMG);
     }
+    if (pathname === '/api/greatest-cover' && req.method === 'GET') {
+      if (!u) bad(401, 'Please log in');
+      const entry = greatestById.get(searchParams.get('id'));
+      if (!entry?.cover) bad(404, 'Cover not found');
+      const file = greatestCovers.file(entry.cover);
+      if (file) return serveFile(req, res, greatestCovers.dir, path.basename(file), 'private, max-age=31536000, immutable', IMG);
+      // Not downloaded yet: the browser fetches the catalog's own URL this once.
+      res.writeHead(302, { Location: entry.cover, 'Cache-Control': 'no-store' });
+      return res.end();
+    }
     if (pathname === '/api/peering/cover' && req.method === 'GET') {
       if (!u) bad(401, 'Please log in');
       admin(u);
@@ -926,13 +1082,25 @@ http.createServer(async (req, res) => {
   } catch (e) {
     const expected = e instanceof HttpError || e instanceof CatalogError;
     if (!expected) console.error(e);
-    if (!res.headersSent) sendJson(res, expected ? e.code : 500, { error: expected ? e.message : 'Internal error' });
+    if (!res.headersSent) sendJson(res, expected ? e.code : 500, expected ? { error: e.message, ...e.extra } : { error: 'Internal error' });
   }
 }).listen(PORT, () => console.log(`MangaCircus on http://localhost:${PORT}  (library: ${LIB}, data: ${DATA})`));
 
 if (lib.mangas.length) refreshChanged();
 else rescan();
 if (SCAN_MINUTES > 0) setInterval(() => refreshChanged(), SCAN_MINUTES * 60000).unref();
+// Fetch missing Greatest covers shortly after start and retry failures once a day;
+// covers already on disk cost nothing.
+async function warmGreatestCovers() {
+  try {
+    const result = await greatestCovers.warm(greatestCatalog.entries.map(entry => entry.cover));
+    if (result.fetched || result.failed || result.stopped) console.log('[greatest] covers', JSON.stringify(result));
+  } catch (error) { console.error('[greatest] covers', error.message); }
+}
+if (process.env.GREATEST_COVER_WARM !== '0') {
+  setTimeout(warmGreatestCovers, 10000).unref();
+  setInterval(warmGreatestCovers, 86400000).unref();
+}
 // Persist the last reconciliation time. Restarting a healthy reader should not
 // trigger another full tree walk; overdue reconciliations run within a minute.
 if (FULL_SCAN_MINUTES > 0) setInterval(() => {
@@ -965,3 +1133,7 @@ async function pollScraper() {
 }
 pollScraper();
 setInterval(pollScraper, 15000).unref();
+
+// Nightly discovery harvest; the delayed first check also covers a slot missed while the reader was down.
+setInterval(checkNightly, 60000).unref();
+setTimeout(checkNightly, 30000).unref();

test/admin-role.test.js

@@ -0,0 +1,88 @@
+'use strict';
+// BL-0045: admins grant or revoke admin rights in Settings -> Users & access.
+const test = require('node:test'), assert = require('node:assert/strict');
+const fs = require('fs'), os = require('os'), path = require('path'), net = require('net');
+const { spawn } = require('child_process');
+
+async function freePort() {
+  const listener = net.createServer();
+  await new Promise(resolve => listener.listen(0, '127.0.0.1', resolve));
+  const port = listener.address().port;
+  await new Promise(resolve => listener.close(resolve));
+  return port;
+}
+
+test('only admins can promote or demote, nobody can demote themselves, and changes apply at once', { timeout: 20000 }, async () => {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-admin-role-'));
+  let child;
+  try {
+    const library = path.join(root, 'library'), data = path.join(root, 'data');
+    fs.mkdirSync(library); fs.mkdirSync(data);
+    const port = await freePort(), output = [];
+    child = spawn(process.execPath, ['server.js'], { cwd: path.resolve(__dirname, '..'), windowsHide: true,
+      env: { ...process.env, LIBRARY: library, DATA: data, PORT: String(port), SCAN_INTERVAL: '0', FULL_SCAN_INTERVAL: '0',
+        SCRAPER_URL: 'http://127.0.0.1:9', SCRAPER_TOKEN: 'fixture-token', SCRAPER_TOKEN_FILE: '', BACKUP: '',
+        GREATEST_COVER_WARM: '0' } });
+    child.stdout.on('data', d => output.push(String(d)));
+    child.stderr.on('data', d => output.push(String(d)));
+    const base = `http://127.0.0.1:${port}`;
+    let ready = false;
+    for (let n = 0; n < 100 && !ready; n++) {
+      ready = await fetch(base + '/health').then(r => r.ok).catch(() => false);
+      if (!ready) await new Promise(resolve => setTimeout(resolve, 50));
+    }
+    assert.ok(ready, output.join(''));
+
+    const post = (cookie, url, body) => fetch(base + url, { method: 'POST',
+      headers: { 'content-type': 'application/json', ...(cookie ? { cookie } : {}) }, body: JSON.stringify(body) });
+    const get = (cookie, url) => fetch(base + url, { headers: { cookie } });
+    const session = r => r.headers.get('set-cookie').split(';')[0];
+
+    const first = await post(null, '/api/signup', { name: 'boss', password: 'boss-password' });
+    assert.equal(first.status, 200);
+    const boss = session(first);
+    assert.equal((await post(boss, '/api/admin/user', { name: 'reader', password: 'reader-password' })).status, 200);
+    const login = await post(null, '/api/login', { name: 'reader', password: 'reader-password' });
+    assert.equal(login.status, 200);
+    const reader = session(login);
+
+    // a normal user can neither promote themselves nor reach admin pages
+    assert.equal((await post(reader, '/api/admin/user/role', { name: 'reader', admin: true })).status, 403);
+    assert.equal((await get(reader, '/api/admin')).status, 403);
+    assert.equal((await post(null, '/api/admin/user/role', { name: 'reader', admin: true })).status, 401);
+
+    // input checks
+    assert.equal((await post(boss, '/api/admin/user/role', { name: 'nobody', admin: true })).status, 404);
+    assert.equal((await post(boss, '/api/admin/user/role', { name: 'reader', admin: 'yes' })).status, 400);
+    assert.equal((await post(boss, '/api/admin/user/role', { name: 'boss', admin: false })).status, 400);
+
+    // promotion keeps the password and applies to the existing session immediately
+    const promoted = await post(boss, '/api/admin/user/role', { name: 'Reader', admin: true });
+    assert.equal(promoted.status, 200);
+    assert.deepEqual(await promoted.json(), { ok: true, name: 'reader', admin: true });
+    assert.equal((await (await get(reader, '/api/me')).json()).admin, true);
+    assert.equal((await get(reader, '/api/admin')).status, 200);
+    assert.equal((await post(null, '/api/login', { name: 'reader', password: 'reader-password' })).status, 200);
+    const users = (await (await get(boss, '/api/admin')).json()).users;
+    assert.deepEqual(users.map(x => [x.name, x.admin]).sort(), [['boss', true], ['reader', true]]);
+
+    // another admin may revoke; the revoked admin loses access at once
+    assert.equal((await post(reader, '/api/admin/user/role', { name: 'boss', admin: false })).status, 200);
+    assert.equal((await get(boss, '/api/admin')).status, 403);
+    assert.equal((await post(boss, '/api/admin/user/role', { name: 'boss', admin: true })).status, 403);
+    assert.match(output.join(''), /\[users\] reader removed boss as admin/);
+
+    // db.json is written shortly after a change (batched writes)
+    let saved = null;
+    for (let n = 0; n < 60; n++) {
+      try { saved = JSON.parse(fs.readFileSync(path.join(data, 'db.json'), 'utf8')); } catch { saved = null; }
+      if (saved && saved.users.boss && saved.users.boss.admin === false) break;
+      await new Promise(resolve => setTimeout(resolve, 100));
+    }
+    assert.equal(saved.users.boss.admin, false);
+    assert.equal(saved.users.reader.admin, true);
+  } finally {
+    if (child) await new Promise(resolve => { if (child.exitCode !== null) return resolve(); child.once('exit', resolve); child.kill(); });
+    fs.rmSync(root, { recursive: true, force: true });
+  }
+});

test/backup.test.js

@@ -0,0 +1,286 @@
+'use strict';
+const test = require('node:test'), assert = require('node:assert/strict');
+const fs = require('fs'), os = require('os'), path = require('path'), net = require('net');
+const { spawn, spawnSync } = require('child_process');
+const { BackupService, readArchive, validateSettings, nextOccurrence } = require('../backup');
+const root = path.resolve(__dirname, '..');
+const delay = ms => new Promise(resolve => setTimeout(resolve, ms));
+const quiet = { log() {}, warn() {}, error() {} };
+const LONG = 'collection-covers/' + 'a-very-long-folder-name-that-keeps-going-'.repeat(3) + '/cover-with-a-long-name.png';
+
+function fixture() {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-backup-'));
+  const data = path.join(dir, 'data'), target = path.join(dir, 'backup');
+  const write = (rel, text) => { fs.mkdirSync(path.dirname(path.join(data, rel)), { recursive: true }); fs.writeFileSync(path.join(data, rel), text); };
+  write('db.json', '{"stale":true}');
+  write('scan-cache.json', '{"cache":1}');
+  write('library-index.json', '{"version":1}');
+  write('catalog.json', '{"collections":[]}');
+  write('scraper/queue.json', JSON.stringify({ version: 3, paused: false, status: 'idle', queue: [{ id: 'a' }] }));
+  write('scraper/events.log', '{"level":"info"}\n');
+  write('sentinel/state.json', '{}');
+  write(LONG, 'png-bytes');
+  // Must not be archived.
+  write('runtime-reader.json', '{}'); write('scraper/runtime.json', '{}'); write('sentinel/runtime.json', '{}');
+  write('scraper/daemon.lock', '1'); write('scraper/supervisor.lock', '1'); write('reader.pid', '1'); write('db.json.tmp', '{}');
+  fs.mkdirSync(target);
+  return { dir, data, target, cleanup: () => fs.rmSync(dir, { recursive: true, force: true }) };
+}
+
+function fakeScraper({ paused = false, busyPolls = 0, failResume = false } = {}) {
+  const calls = [];
+  let polls = 0;
+  return {
+    calls,
+    get paused() { return paused; },
+    async state() { polls++; return { paused, status: paused && polls <= busyPolls ? 'pausing' : paused ? 'paused' : 'downloading', active: polls <= busyPolls ? [{ source: 'x' }] : [] }; },
+    async control(action) {
+      calls.push(action);
+      if (action === 'resume' && failResume) throw new Error('scraper gone');
+      paused = action === 'pause';
+    },
+  };
+}
+
+const service = (f, options = {}) => new BackupService({ dataRoot: f.data, backupRoot: f.target, log: quiet, pollMs: 5, resumeRetryMs: 1,
+  snapshotDb: () => JSON.stringify({ users: { admin: {} }, live: true }), ...options }).load();
+
+test('settings validation and schedule computation', () => {
+  assert.deepEqual(validateSettings({ enabled: true, time: '04:15', retention: { mode: 'days', value: 30 } }),
+    { enabled: true, time: '04:15', retention: { mode: 'days', value: 30 } });
+  assert.throws(() => validateSettings({ time: '24:00' }), /HH:MM/);
+  assert.throws(() => validateSettings({ retention: { mode: 'count', value: 0 } }), /Keep/);
+  assert.throws(() => validateSettings({ retention: { mode: 'weeks', value: 2 } }), /count/);
+  assert.throws(() => validateSettings({ enabled: 'yes' }), /enabled/);
+  const now = new Date(2026, 9, 22, 3, 0).getTime();
+  assert.equal(nextOccurrence('04:00', now), new Date(2026, 9, 22, 4, 0).getTime());
+  assert.equal(nextOccurrence('03:00', now), new Date(2026, 9, 23, 3, 0).getTime());
+});
+
+test('a backup pauses a running queue, waits for idle, archives a consistent DATA snapshot and resumes', async () => {
+  const f = fixture();
+  try {
+    const scraper = fakeScraper({ busyPolls: 3 });
+    const backup = service(f, { scraper });
+    const result = await backup.run('manual');
+    assert.equal(result.error, undefined, result.error);
+    assert.deepEqual(scraper.calls, ['pause', 'resume']);
+    assert.equal(result.queueWasPaused, false); assert.equal(result.queueResumed, true);
+    assert.match(result.file, /^circus-\d{8}-\d{6}\.tar\.gz$/);
+    assert.deepEqual(fs.readdirSync(f.target), [result.file]);
+    const entries = await readArchive(path.join(f.target, result.file), { contents: true });
+    const byName = new Map(entries.map(e => [e.name, e]));
+    const text = name => byName.get(name).data.toString('utf8');
+    // db.json comes from memory, not from the stale file on disk.
+    assert.deepEqual(JSON.parse(text('db.json')), { users: { admin: {} }, live: true });
+    // The archived queue is paused; the live one is untouched by the snapshot.
+    const queue = JSON.parse(text('scraper/queue.json'));
+    assert.equal(queue.paused, true); assert.deepEqual(queue.queue, [{ id: 'a' }]);
+    assert.equal(JSON.parse(fs.readFileSync(path.join(f.data, 'scraper/queue.json'), 'utf8')).paused, false);
+    for (const name of ['scan-cache.json', 'library-index.json', 'catalog.json', 'scraper/events.log', 'sentinel/state.json', LONG, '.circus-backup.json'])
+      assert.ok(byName.has(name), `${name} missing`);
+    assert.equal(text(LONG), 'png-bytes');
+    for (const name of ['runtime-reader.json', 'scraper/runtime.json', 'sentinel/runtime.json', 'scraper/daemon.lock', 'scraper/supervisor.lock', 'reader.pid', 'db.json.tmp'])
+      assert.ok(!byName.has(name), `${name} must be excluded`);
+    assert.ok(byName.get('scraper/').type === '5');
+    // The persisted state records the result but never a pending marker inside the archive.
+    assert.equal(JSON.parse(text('backup.json')).pending, undefined);
+    assert.equal(backup.publicState().last.file, result.file);
+    assert.equal(backup.publicState().archives.count, 1);
+  } finally { f.cleanup(); }
+});
+
+test('an already paused queue stays paused and is not resumed', async () => {
+  const f = fixture();
+  try {
+    const scraper = fakeScraper({ paused: true });
+    const result = await service(f, { scraper }).run('schedule');
+    assert.equal(result.error, undefined);
+    assert.deepEqual(scraper.calls, []);
+    assert.equal(result.queueWasPaused, true); assert.equal(scraper.paused, true);
+  } finally { f.cleanup(); }
+});
+
+test('a scraper that never becomes idle aborts the backup, leaves no archive and resumes the queue', async () => {
+  const f = fixture();
+  try {
+    const scraper = fakeScraper({ busyPolls: Infinity });
+    const result = await service(f, { scraper, idleTimeoutMs: 40 }).run('manual');
+    assert.match(result.error, /did not become idle/);
+    assert.deepEqual(scraper.calls, ['pause', 'resume']);
+    assert.deepEqual(fs.readdirSync(f.target), []);
+  } finally { f.cleanup(); }
+});
+
+test('an unreachable scraper does not block the backup and is reported', async () => {
+  const f = fixture();
+  try {
+    const scraper = { async state() { throw new Error('connect ECONNREFUSED'); }, async control() { throw new Error('unused'); } };
+    const result = await service(f, { scraper }).run('manual');
+    assert.equal(result.error, undefined);
+    assert.match(result.warnings[0], /unreachable/);
+    const queue = (await readArchive(path.join(f.target, result.file), { contents: true })).find(e => e.name === 'scraper/queue.json');
+    assert.equal(JSON.parse(queue.data).paused, true);
+  } finally { f.cleanup(); }
+});
+
+test('a failed resume is surfaced as a warning', async () => {
+  const f = fixture();
+  try {
+    const result = await service(f, { scraper: fakeScraper({ failResume: true }) }).run('manual');
+    assert.equal(result.error, undefined);
+    assert.match(result.warnings.join(' '), /could not be resumed/);
+  } finally { f.cleanup(); }
+});
+
+test('retention keeps the newest archives and never touches other files', async () => {
+  const f = fixture();
+  try {
+    const old = ['circus-20260101-030000.tar.gz', 'circus-20260102-030000.tar.gz', 'circus-20260103-030000.tar.gz'];
+    const foreign = ['circus-20250101-0300.tar.zst', 'notes.txt', 'circus-20250101-030000.tar.gz.bak'];
+    for (const name of [...old, ...foreign]) fs.writeFileSync(path.join(f.target, name), 'x');
+    fs.mkdirSync(path.join(f.target, 'circus-20250101-030000.tar.gz-dir'));
+    fs.writeFileSync(path.join(f.target, 'circus-20250101-030000.tar.gz.part'), 'stale');
+    const backup = service(f);
+    backup.updateSettings({ retention: { mode: 'count', value: 2 } });
+    const result = await backup.run('manual');
+    assert.equal(result.error, undefined); assert.equal(result.deleted, 2);
+    assert.deepEqual(fs.readdirSync(f.target).sort(), [...foreign, 'circus-20250101-030000.tar.gz-dir', old[2], result.file].sort());
+    // Days mode works from the timestamp in the name.
+    backup.updateSettings({ retention: { mode: 'days', value: 1 } });
+    const second = await backup.run('manual');
+    assert.ok(!fs.existsSync(path.join(f.target, old[2])));
+    assert.ok(fs.existsSync(path.join(f.target, second.file)));
+    for (const name of foreign) assert.ok(fs.existsSync(path.join(f.target, name)));
+  } finally { f.cleanup(); }
+});
+
+test('backups are off without BACKUP, and a missing folder is reported, never created', async () => {
+  const f = fixture();
+  try {
+    const off = service(f, { backupRoot: '' });
+    await assert.rejects(off.run('manual'), /Backups are off/);
+    assert.equal(off.publicState().configured, false);
+    const missing = path.join(f.dir, 'not-mounted');
+    const result = await service(f, { backupRoot: missing }).run('manual');
+    assert.match(result.error, /not usable/);
+    assert.ok(!fs.existsSync(missing));
+  } finally { f.cleanup(); }
+});
+
+test('the writability probe leaves nothing behind in the backup folder', async () => {
+  const f = fixture();
+  try {
+    const backup = service(f);
+    await backup.refreshTarget();
+    assert.equal(backup.publicState().targetError, null);
+    assert.deepEqual(fs.readdirSync(f.target).filter(name => name.startsWith('.circus-backup-probe')), []);
+  } finally { f.cleanup(); }
+});
+
+test('only one run at a time', async () => {
+  const f = fixture();
+  try {
+    const backup = service(f, { scraper: fakeScraper({ busyPolls: 5 }) });
+    const first = backup.run('manual');
+    await assert.rejects(backup.run('manual'), /already running/);
+    assert.equal((await first).error, undefined);
+  } finally { f.cleanup(); }
+});
+
+test('the schedule runs once per slot, catches up a recent miss and skips stale slots', async () => {
+  const f = fixture();
+  try {
+    let now = new Date(2026, 9, 22, 2, 0).getTime();
+    const backup = service(f, { now: () => now });
+    backup.updateSettings({ enabled: true, time: '03:30' });
+    assert.equal(backup.state.nextAt, new Date(2026, 9, 22, 3, 30).getTime());
+    assert.equal(await backup.tick(), null);
+    now = new Date(2026, 9, 22, 9, 0).getTime(); // reader was down at 03:30
+    const result = await backup.tick();
+    assert.equal(result.trigger, 'schedule'); assert.equal(result.error, undefined);
+    assert.equal(backup.state.nextAt, new Date(2026, 9, 23, 3, 30).getTime());
+    assert.equal(await backup.tick(), null);
+    now = new Date(2026, 9, 30, 9, 0).getTime(); // e.g. an old backup.json was restored
+    assert.equal(await backup.tick(), null);
+    assert.equal(backup.state.nextAt, new Date(2026, 9, 31, 3, 30).getTime());
+    assert.equal(fs.readdirSync(f.target).length, 1);
+  } finally { f.cleanup(); }
+});
+
+test('a run interrupted by a restart resumes the queue it paused', async () => {
+  const f = fixture();
+  try {
+    fs.writeFileSync(path.join(f.data, 'backup.json'), JSON.stringify({ version: 1, pending: { startedAt: 1, trigger: 'schedule', pausedByUs: true } }));
+    const scraper = fakeScraper({ paused: true });
+    const backup = service(f, { scraper });
+    await backup.recoverInterrupted();
+    assert.deepEqual(scraper.calls, ['resume']);
+    assert.match(backup.state.last.error, /Interrupted/);
+    assert.equal(JSON.parse(fs.readFileSync(path.join(f.data, 'backup.json'), 'utf8')).pending, null);
+  } finally { f.cleanup(); }
+});
+
+test('system tar extracts the archive', { skip: spawnSync('tar', ['--version']).error ? 'no tar binary' : false }, async () => {
+  const f = fixture();
+  try {
+    const result = await service(f).run('manual');
+    const out = path.join(f.dir, 'restored'); fs.mkdirSync(out);
+    const tar = spawnSync('tar', ['-xzf', path.join(f.target, result.file), '-C', out], { encoding: 'utf8' });
+    assert.equal(tar.status, 0, tar.stderr);
+    assert.equal(fs.readFileSync(path.join(out, LONG), 'utf8'), 'png-bytes');
+    assert.equal(JSON.parse(fs.readFileSync(path.join(out, 'scraper/queue.json'), 'utf8')).paused, true);
+    assert.ok(!fs.existsSync(path.join(out, 'scraper/daemon.lock')));
+  } finally { f.cleanup(); }
+});
+
+async function port() {
+  const server = net.createServer();
+  await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
+  const result = server.address().port; await new Promise(resolve => server.close(resolve)); return result;
+}
+
+test('reader and scraper: admin backup API pauses and resumes the real queue', { timeout: 30000 }, async () => {
+  const f = fixture(), children = [], output = [];
+  try {
+    const library = path.join(f.dir, 'library'); fs.mkdirSync(library);
+    fs.writeFileSync(path.join(f.data, 'scraper', 'queue.json'), JSON.stringify({ paused: false, queue: [], settings: { autoUpdateEnabled: false } }));
+    for (const name of ['runtime.json', 'daemon.lock', 'supervisor.lock']) fs.rmSync(path.join(f.data, 'scraper', name), { force: true });
+    for (const name of ['db.json', 'catalog.json', 'library-index.json', 'scan-cache.json']) fs.rmSync(path.join(f.data, name));
+    const [rp, sp] = await Promise.all([port(), port()]);
+    const base = `http://127.0.0.1:${rp}`, token = 'backup-test-only';
+    const env = { ...process.env, DATA: f.data, LIBRARY: library, SCRAPER_DATA: path.join(f.data, 'scraper'), BACKUP: f.target,
+      PORT: String(rp), SCRAPER_PORT: String(sp), SCRAPER_HOST: '127.0.0.1', SCRAPER_URL: `http://127.0.0.1:${sp}`, SENTINEL_URL: 'http://127.0.0.1:9',
+      SCRAPER_TOKEN: token, SCRAPER_TOKEN_FILE: '', SCAN_INTERVAL: '0', GREATEST_COVER_WARM: '0' };
+    for (const script of ['server.js', 'scraper/daemon.js']) {
+      const child = spawn(process.execPath, [script], { cwd: root, env, windowsHide: true }); children.push(child);
+      for (const stream of [child.stdout, child.stderr]) stream.on('data', chunk => output.push(chunk.toString()));
+    }
+    for (const url of [base, `http://127.0.0.1:${sp}`]) {
+      let ready = false;
+      for (let n = 0; n < 80 && !ready; n++) { ready = await fetch(url + '/health').then(r => r.ok).catch(() => false); if (!ready) await delay(100); }
+      assert.ok(ready, output.join(''));
+    }
+    const signup = await fetch(base + '/api/signup', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ name: 'admin', password: 'local-test-password' }) });
+    const cookie = signup.headers.get('set-cookie').split(';')[0];
+    const request = (endpoint, body) => fetch(base + endpoint, { headers: { cookie, 'content-type': 'application/json' }, ...(body ? { method: 'POST', body: JSON.stringify(body) } : {}) });
+    assert.equal((await request('/api/admin/backup', { command: 'settings', settings: { time: '25:00' } })).status, 400);
+    const saved = await (await request('/api/admin/backup', { command: 'settings', settings: { enabled: true, time: '04:00', retention: { mode: 'count', value: 3 } } })).json();
+    assert.equal(saved.settings.retention.value, 3); assert.ok(saved.nextAt > Date.now());
+    assert.equal((await request('/api/admin/backup', { command: 'run' })).status, 200);
+    let state;
+    for (let n = 0; n < 100; n++) { state = await (await request('/api/admin/backup')).json(); if (state.last && !state.running) break; await delay(100); }
+    assert.equal(state.last.error, undefined, state.last.error);
+    assert.equal(state.last.queueWasPaused, false); assert.equal(state.last.queueResumed, true);
+    const scraperState = await (await request('/api/admin/scraper')).json();
+    assert.equal(scraperState.paused, false);
+    const entries = await readArchive(path.join(f.target, state.last.file), { contents: true });
+    const db = JSON.parse(entries.find(e => e.name === 'db.json').data);
+    assert.ok(db.users.admin);
+    assert.equal(JSON.parse(entries.find(e => e.name === 'scraper/queue.json').data).paused, true);
+    assert.ok((await (await request('/api/admin/health')).json()).backup.last.file);
+  } finally {
+    await Promise.all(children.map(child => new Promise(resolve => { if (child.exitCode !== null) return resolve(); child.once('exit', resolve); child.kill(); })));
+    f.cleanup();
+  }
+});

test/chapter-dates.test.js

@@ -0,0 +1,81 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const { batchFlags, publishedDates, chapterDates, peerDates, fromPeer } = require('../chapter-dates');
+const { earliestCompletion } = require('../reading-analytics');
+
+test('bulk imports are flagged, weekly uploads are not', () => {
+  const flags = batchFlags([
+    '2024-09-07T10:00:00Z', '2024-09-07T10:00:03Z', '2024-09-07T10:04:00Z', // one import
+    '2026-10-11T15:05:58Z', '2026-10-18T15:06:05Z', // weekly releases
+    '2026-10-20T09:00:00Z', '2026-10-20T09:01:00Z', // a double release stays a release
+    undefined,
+  ]);
+  assert.deepEqual(flags, [true, true, true, false, false, false, false, null]);
+});
+
+test('publishing keeps the first arrival and the earliest ecosystem date', () => {
+  const now = Date.parse('2026-10-24T12:00:00Z');
+  const fresh = publishedDates({}, { source: { at: '2026-10-18T15:06:05Z', name: 'WeebCentral', batch: false } }, now);
+  assert.equal(fresh.added, '2026-10-24T12:00:00.000Z');
+  assert.deepEqual(fresh['x-dates'].ecosystem, { at: '2026-10-24T12:00:00.000Z', via: 'this server' });
+  assert.equal(fresh['x-dates'].source.name, 'WeebCentral');
+
+  // Replacing a version later: "added" and the ecosystem date survive.
+  const replaced = publishedDates(fresh, {}, now + 864e5);
+  assert.equal(replaced.added, fresh.added);
+  assert.deepEqual(replaced['x-dates'].ecosystem, fresh['x-dates'].ecosystem);
+  assert.equal(replaced['x-dates'].source.at, '2026-10-18T15:06:05.000Z');
+
+  // A peer that had it earlier lowers the date; a later claim never raises it.
+  const earlier = publishedDates(fresh, { ecosystem: { at: '2026-10-19T08:00:00Z', via: 'peer bob' } }, now);
+  assert.deepEqual(earlier['x-dates'].ecosystem, { at: '2026-10-19T08:00:00.000Z', via: 'peer bob' });
+  const later = publishedDates(earlier, { ecosystem: { at: '2026-10-23T00:00:00Z', via: 'peer eve' } }, now);
+  assert.equal(later['x-dates'].ecosystem.via, 'peer bob');
+});
+
+test('peer ecosystem claims cannot predate the source upload or lie in the future', () => {
+  const now = Date.parse('2026-10-24T12:00:00Z');
+  const source = { at: '2026-10-18T15:06:05Z', name: 'WeebCentral', batch: false };
+  const early = publishedDates({}, { source, ecosystem: { at: '2001-01-01T00:00:00Z', via: 'peer x' } }, now);
+  assert.equal(early['x-dates'].ecosystem.at, '2026-10-18T15:06:05.000Z');
+  const future = publishedDates({}, { ecosystem: { at: '2099-01-01T00:00:00Z', via: 'peer x' } }, now);
+  assert.equal(future['x-dates'].ecosystem.at, '2026-10-24T12:00:00.000Z');
+});
+
+test('release date tiers: official, source upload, ecosystem, library', () => {
+  const added = Date.parse('2026-10-17T15:04:03Z');
+  const version = dates => ({ added, meta: { 'x-dates': dates } });
+  const ecosystem = { at: '2026-10-10T00:00:00Z', via: 'peer bob' };
+  assert.equal(chapterDates([version({})]).tier, 'library');
+  assert.equal(chapterDates([version({ ecosystem })]).release.at, '2026-10-10T00:00:00.000Z');
+  const batch = chapterDates([version({ ecosystem, source: { at: '2024-09-07T10:00:00Z', name: 'WeebCentral', batch: true } })]);
+  assert.equal(batch.tier, 'ecosystem', 'a bulk upload is no release date');
+  assert.equal(batch.known.batch.at, '2024-09-07T10:00:00.000Z');
+  const source = chapterDates([version({ ecosystem }), version({ source: { at: '2026-10-04T15:06:05Z', name: 'MangaDex', batch: false } })]);
+  assert.equal(source.tier, 'source');
+  assert.equal(source.known.ecosystem.at, '2026-10-10T00:00:00.000Z', 'earliest across versions');
+  const official = chapterDates([version({ release: { at: '2023-09-17', precision: 'day', source: 'MangaUpdates · Viz' }, ecosystem })]);
+  assert.deepEqual(official.release, { at: '2023-09-17', precision: 'day', name: 'MangaUpdates · Viz' });
+  assert.equal(official.tier, 'official');
+});
+
+test('peers exchange cleaned dates and the receiver records the peer', () => {
+  const meta = { added: '2026-10-17T15:04:03Z', secret: 'x',
+    'x-dates': { source: { at: '2026-10-18T15:06:05Z', name: 'WeebCentral', batch: false, extra: 'drop' } } };
+  const sent = peerDates(meta, 0);
+  assert.deepEqual(sent, { ecosystem: '2026-10-17T15:04:03.000Z', source: { at: '2026-10-18T15:06:05.000Z', name: 'WeebCentral', batch: false } });
+  const received = fromPeer(sent, 'bob');
+  assert.deepEqual(received.ecosystem, { at: '2026-10-17T15:04:03.000Z', via: 'peer bob' });
+  assert.deepEqual(fromPeer('nonsense', 'bob'), {});
+  assert.deepEqual(fromPeer({ ecosystem: 'not a date' }, 'bob'), {});
+});
+
+test('legacy reads get the earliest completion the reading clock remembers', () => {
+  assert.equal(earliestCompletion(null), null);
+  const record = { completedPasses: 3, currentPass: { completedAt: 5000 },
+    recentSessions: [{ pass: { completedAt: 3000 } }, { pass: { completedAt: null } }, { pass: { completedAt: 5000 } }] };
+  assert.deepEqual(earliestCompletion(record), { at: 3000, approx: true });
+  assert.deepEqual(earliestCompletion({ ...record, completedPasses: 2 }), { at: 3000, approx: false });
+});

test/discovery-anchors.test.js

@@ -0,0 +1,73 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const { anchorsFor, topTagsFor, ANCHOR_WEIGHTS, MAX_ANCHORS } = require('../discovery/anchors');
+
+const user = (settings = { discoveryEnabled: true }) => ({ name: 'ann', settings });
+const manga = (id, anilist, title = `Manga ${id}`) => ({ id, title, meta: anilist ? { 'x-anilist': { id: anilist } } : {} });
+
+test('anchors are empty without opt-in', () => {
+  const library = [manga('a', 1)];
+  assert.deepEqual(anchorsFor({ user: user({}), db: {}, library, greatest: { entries: [] } }), []);
+  assert.deepEqual(anchorsFor({ user: user({ discoveryEnabled: false }), db: {}, library, greatest: { entries: [] } }), []);
+});
+
+test('weights, dedupe and cap', () => {
+  const greatest = { entries: [{ id: 'g1', title: 'Great One', metadata: { id: 7 } }, { id: 'g2', title: 'No Meta' }] };
+  const db = { greatestRatings: { ann: { g1: 'good', g2: 'flame' } } };
+  const anchors = anchorsFor({ user: user(), db, library: [manga('m7', 7, 'Owned Seven')], greatest });
+  assert.deepEqual(anchors, [{ anilistId: 7, weight: 3, source: 'good', title: 'Great One' }]);
+  assert.equal(ANCHOR_WEIGHTS.flame, 5);
+  const many = Array.from({ length: 20 }, (_, i) => manga(`m${i}`, i + 1));
+  const capped = anchorsFor({ user: user(), db: {}, library: many, greatest: { entries: [] } });
+  assert.equal(capped.length, MAX_ANCHORS);
+  assert.equal(capped.length, 15);
+  assert.deepEqual(capped.map(a => a.anilistId), Array.from({ length: 15 }, (_, i) => i + 1));
+  assert.equal(capped[0].source, 'owned');
+});
+
+test('reading anchors need the reading opt-in', () => {
+  const library = [manga('m1', 1), manga('m2', 2)];
+  const db = { progress: { ann: { m1: { c1: { done: true } }, m2: { c1: { done: false } } } } };
+  const off = anchorsFor({ user: user(), db, library, greatest: { entries: [] } });
+  assert.deepEqual(off.map(a => [a.anilistId, a.source]), [[1, 'owned'], [2, 'owned']]);
+  const on = anchorsFor({ user: user({ discoveryEnabled: true, discoveryReadingActivity: true }), db, library, greatest: { entries: [] } });
+  assert.deepEqual(on.map(a => [a.anilistId, a.source, a.weight, a.title]), [[1, 'read', 3, 'Manga m1'], [2, 'owned', 1, 'Manga m2']]);
+});
+
+test('feedback keys parse anilist ids and ignore others', () => {
+  const db = { discoveryFeedback: { ann: {
+    'anilist:10': 'interested', 'anilist:11': 'maybe', 'anilist:12': 'dismissed', 'anilist:13': 'hidden',
+    'greatest:x': 'interested', 'anilist:abc': 'interested' } } };
+  const anchors = anchorsFor({ user: user(), db, library: [], greatest: { entries: [] } });
+  assert.deepEqual(anchors, [
+    { anilistId: 10, weight: 4, source: 'interested', title: null },
+    { anilistId: 11, weight: 2, source: 'maybe', title: null }]);
+});
+
+test('top tags weigh anchors, skip low rank and adult tags', () => {
+  const records = {
+    1: { tags: [{ name: 'Action', rank: 100, kind: 'genre', isAdult: false }, { name: 'Ecchi', rank: 90, kind: 'tag', isAdult: true },
+      { name: 'Faint', rank: 59, kind: 'tag', isAdult: false }] },
+    2: { tags: [{ name: 'Action', rank: 100, kind: 'genre', isAdult: false }, { name: 'Zeta', rank: 60, kind: 'tag', isAdult: false },
+      { name: 'Alpha', rank: 80, kind: 'tag', isAdult: false }] }
+  };
+  const anchors = [{ anilistId: 1, weight: 5 }, { anilistId: 2, weight: 3 }, { anilistId: 3, weight: 5 }];
+  const recordOf = id => records[id] || null;
+  assert.deepEqual(topTagsFor(anchors, recordOf), [
+    { name: 'Action', kind: 'genre' }, { name: 'Alpha', kind: 'tag' }, { name: 'Zeta', kind: 'tag' }]);
+  assert.deepEqual(topTagsFor(anchors, recordOf, { mature: true }), [
+    { name: 'Action', kind: 'genre' }, { name: 'Ecchi', kind: 'tag' }, { name: 'Alpha', kind: 'tag' }]);
+});
+
+test('top tags drop tags without a rank and survive malformed records', () => {
+  const records = {
+    1: { tags: [{ name: 'NoRank', kind: 'tag', isAdult: false }, { name: 'NullRank', rank: null, kind: 'tag', isAdult: false },
+      null, 7, { name: 'Kept', rank: 60, kind: 'tag', isAdult: false }] },
+    2: { tags: 5 },
+    3: { tags: { name: 'Object', rank: 90 } },
+  };
+  const anchors = [{ anilistId: 1, weight: 5 }, { anilistId: 2, weight: 3 }, { anilistId: 3, weight: 3 }];
+  assert.deepEqual(topTagsFor(anchors, id => records[id]), [{ name: 'Kept', kind: 'tag' }]);
+});

test/discovery-anilist.test.js

@@ -0,0 +1,175 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const { createLimiter, createClient, normalizeMedia, AniListStop, AniListSkip } = require('../discovery/anilist');
+
+const fixture = name => JSON.parse(fs.readFileSync(path.join(__dirname, 'fixtures', 'anilist', `${name}.json`), 'utf8'));
+const json = (body, status = 200) => new Response(JSON.stringify(body), { status, headers: { 'content-type': 'application/json' } });
+const noWait = { wait: async () => {} };
+
+function stub(respond) {
+  const calls = [];
+  const fetchImpl = async (url, init) => { calls.push({ url, init, body: JSON.parse(init.body) }); return respond(); };
+  return { calls, client: createClient({ endpoint: 'http://localhost/graphql', limiter: noWait, fetchImpl, today: () => new Date('2026-10-25') }) };
+}
+const clientFor = respond => stub(respond).client;
+
+test('normalizeMedia maps fields and drops novels', () => {
+  const media = fixture('title').data.Media;
+  assert.deepEqual(normalizeMedia(media), {
+    anilistId: 101, title: 'Moon Traveller', altTitles: ['Tsuki no Tabibito', '月の旅人', 'Lunar Wanderer'],
+    format: 'MANGA', isAdult: false, status: 'ongoing', year: 2019, averageScore: 81, popularity: 5400, trending: 37,
+    tags: [
+      { name: 'Adventure', rank: 100, kind: 'genre', isAdult: false },
+      { name: 'Fantasy', rank: 100, kind: 'genre', isAdult: false },
+      { name: 'Travel', rank: 88, kind: 'tag', isAdult: false },
+      { name: 'Nudity', rank: 40, kind: 'tag', isAdult: true }],
+    authors: [{ staffId: 900, name: 'Aoi Example', role: 'Story & Art' }],
+    coverUrl: 'https://example.invalid/cover/101.jpg', siteUrl: 'https://anilist.co/manga/101' });
+  assert.equal(normalizeMedia({ ...media, format: 'NOVEL' }), null);
+  assert.equal(normalizeMedia(fixture('novel').data.Media), null);
+  assert.equal(normalizeMedia({ ...media, status: 'HIATUS' }).status, 'unknown');
+});
+
+test('normalizeMedia drops authors without an integer staff id', () => {
+  const media = fixture('title').data.Media;
+  const edge = (id, full) => ({ role: 'Story', node: { id, name: { full } } });
+  const record = normalizeMedia({ ...media, staff: { edges: [edge(5, 'Has Id'), edge(undefined, 'No Id'),
+    edge('6', 'String Id'), edge(1.5, 'Float Id'), edge(null, 'Null Id')] } });
+  assert.deepEqual(record.authors, [{ staffId: 5, name: 'Has Id', role: 'Story' }]);
+});
+
+test('title returns a record and sends the fixed headers', async () => {
+  const { client, calls } = stub(() => json(fixture('title')));
+  const record = await client.title(101);
+  assert.equal(record.anilistId, 101);
+  assert.equal(calls[0].body.variables.id, 101);
+  assert.equal(calls[0].init.headers['user-agent'], 'MangaCircus-Discovery/1.0');
+  assert.equal(calls[0].init.headers['content-type'], 'application/json');
+});
+
+test('recs keeps positive ratings and manga formats', async () => {
+  const out = await clientFor(() => json(fixture('recs'))).recs(101);
+  assert.equal(out.length, 1);
+  assert.equal(out[0].anilistId, 201);
+  assert.equal(out[0].rating, 12);
+  assert.equal(out[0].record.title, 'Sky Town');
+});
+
+test('recs query follows spec sort and page size', async () => {
+  const { client, calls } = stub(() => json(fixture('recs')));
+  await client.recs(101);
+  assert.match(calls[0].body.query, /recommendations\(perPage: 25, sort: RATING_DESC\)/);
+});
+
+test('creator query follows spec sort and page size', async () => {
+  const { client, calls } = stub(() => json(fixture('creator')));
+  await client.creator(900);
+  assert.match(calls[0].body.query, /staffMedia\(type: MANGA, perPage: 25, sort: POPULARITY_DESC\)/);
+});
+
+test('title on a novel resolves to null', async () => {
+  assert.equal(await clientFor(() => json(fixture('novel'))).title(777), null);
+});
+
+test('requests refuse redirects', async () => {
+  const { client, calls } = stub(() => json(fixture('title')));
+  await client.title(101);
+  assert.equal(calls[0].init.redirect, 'error');
+  const redirected = () => { throw new TypeError('fetch failed: unexpected redirect'); };
+  await assert.rejects(clientFor(redirected).title(1), e => e instanceof AniListSkip && e.reason === 'upstream');
+});
+
+test('recs drops recommendations whose media was removed', async () => {
+  const recs = fixture('recs');
+  recs.data.Media.recommendations.nodes = [recs.data.Media.recommendations.nodes[0], { rating: 10, mediaRecommendation: null }];
+  const out = await clientFor(() => json(recs)).recs(101);
+  assert.deepEqual(out.map(e => e.anilistId), [201]);
+});
+
+test('one malformed item invalidates a list response', async () => {
+  const invalid = e => e instanceof AniListSkip && e.reason === 'invalid';
+  const recs = fixture('recs');
+  recs.data.Media.recommendations.nodes.push(null);
+  await assert.rejects(clientFor(() => json(recs)).recs(1), invalid);
+  const tagtop = fixture('tagtop');
+  tagtop.data.Page.media[0].id = 'x';
+  await assert.rejects(clientFor(() => json(tagtop)).tagTop({ name: 'Mystery', kind: 'genre' }), invalid);
+  for (const patch of [{ genres: 'Drama' }, { tags: [null] }, { title: 'x' }, { staff: { edges: {} } }, { staff: { edges: [null] } }]) {
+    const fresh = fixture('fresh');
+    Object.assign(fresh.data.Page.media[0], patch);
+    await assert.rejects(clientFor(() => json(fresh)).fresh(), invalid, JSON.stringify(patch));
+  }
+  await assert.rejects(async () => normalizeMedia({ id: 1, format: 'MANGA', title: 5 }), invalid);
+});
+
+test('creator returns roles and drops novels', async () => {
+  const out = await clientFor(() => json(fixture('creator'))).creator(900);
+  assert.deepEqual(out.map(e => [e.anilistId, e.role]), [[301, 'Story & Art']]);
+});
+
+test('fresh query sends startDate two years back', async () => {
+  const { client, calls } = stub(() => json(fixture('fresh')));
+  const out = await client.fresh();
+  assert.equal(calls[0].body.variables.startDate_greater, 20241025);
+  assert.equal(calls[0].body.variables.perPage, 50);
+  assert.match(calls[0].body.query, /TRENDING_DESC/);
+  assert.deepEqual(out.map(e => e.anilistId), [501]);
+});
+
+test('tagTop uses genre_in for genres and tag_in for tags', async () => {
+  const g = stub(() => json(fixture('tagtop')));
+  const out = await g.client.tagTop({ name: 'Mystery', kind: 'genre' });
+  assert.deepEqual(g.calls[0].body.variables.genre_in, ['Mystery']);
+  assert.equal(g.calls[0].body.variables.tag_in, undefined);
+  assert.match(g.calls[0].body.query, /genre_in: \$genre_in, popularity_greater: 1000, sort: SCORE_DESC/);
+  assert.match(g.calls[0].body.query, /Page\(perPage: 25\)/);
+  assert.deepEqual(out.map(e => e.anilistId), [401]);
+  const t = stub(() => json(fixture('tagtop')));
+  await t.client.tagTop({ name: 'Detective', kind: 'tag' });
+  assert.match(t.calls[0].body.query, /tag_in: \$tag_in, popularity_greater: 1000, sort: SCORE_DESC/);
+  assert.deepEqual(t.calls[0].body.variables.tag_in, ['Detective']);
+  assert.equal(t.calls[0].body.variables.genre_in, undefined);
+});
+
+test('429 stops, 403 stops, html body is a challenge', async () => {
+  await assert.rejects(clientFor(() => json({}, 429)).title(1), e => e instanceof AniListStop && e.reason === 'rate-limited');
+  await assert.rejects(clientFor(() => json({}, 403)).title(1), e => e instanceof AniListStop && e.reason === 'forbidden');
+  await assert.rejects(clientFor(() => json({}, 401)).title(1), e => e instanceof AniListStop && e.reason === 'forbidden');
+  const html = () => new Response('<html>checking your browser</html>', { status: 200, headers: { 'content-type': 'text/html' } });
+  await assert.rejects(clientFor(html).title(1), e => e instanceof AniListStop && e.reason === 'challenge');
+  const garbage = () => new Response('not json', { status: 200 });
+  await assert.rejects(clientFor(garbage).title(1), e => e instanceof AniListStop && e.reason === 'challenge');
+});
+
+test('5xx and timeout skip', async () => {
+  await assert.rejects(clientFor(() => json({}, 503)).title(1), e => e instanceof AniListSkip && e.reason === 'upstream');
+  // A challenge page served with a 5xx is still a challenge, not an outage.
+  const html503 = () => new Response('<html>just a moment</html>', { status: 503, headers: { 'content-type': 'text/html' } });
+  await assert.rejects(clientFor(html503).title(1), e => e instanceof AniListStop && e.reason === 'challenge');
+  const text502 = () => new Response('Bad Gateway', { status: 502 });
+  await assert.rejects(clientFor(text502).title(1), e => e instanceof AniListStop && e.reason === 'challenge');
+  const timeout = () => { throw Object.assign(new Error('timed out'), { name: 'TimeoutError' }); };
+  await assert.rejects(clientFor(timeout).title(1), e => e instanceof AniListSkip && e.reason === 'timeout');
+  const abort = () => { throw Object.assign(new Error('aborted'), { name: 'AbortError' }); };
+  await assert.rejects(clientFor(abort).title(1), e => e instanceof AniListSkip && e.reason === 'timeout');
+});
+
+test('missing data is invalid', async () => {
+  const invalid = e => e instanceof AniListSkip && e.reason === 'invalid';
+  await assert.rejects(clientFor(() => json({ data: {} })).title(1), invalid);
+  await assert.rejects(clientFor(() => json({ data: { Page: {} } })).fresh(), invalid);
+  await assert.rejects(clientFor(() => json({ data: {} })).creator(1), invalid);
+  await assert.rejects(clientFor(() => json({ data: { Media: { ...fixture('title').data.Media, id: 'x' } } })).title(1), invalid);
+});
+
+test('limiter spaces calls 2400 ms apart', async () => {
+  let clock = 1000;
+  const sleeps = [];
+  const limiter = createLimiter({ perMinute: 25, now: () => clock, sleep: async ms => { sleeps.push(ms); clock += ms; } });
+  await Promise.all([limiter.wait(), limiter.wait(), limiter.wait()]);
+  assert.deepEqual(sleeps, [0, 2400, 2400]);
+});

test/discovery-cache.test.js

@@ -0,0 +1,116 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { DiscoveryCache } = require('../discovery/cache');
+
+const DAY = 86400000;
+
+function setup(t, options = {}) {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'circus-dcache-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  const clock = { t: Date.UTC(2026, 0, 1) };
+  const cache = new DiscoveryCache(dir, { ...options, now: () => clock.t });
+  return { dir, clock, cache };
+}
+
+test('cache round-trips an entry', (t) => {
+  const { clock, cache } = setup(t);
+  cache.put('title', 42, { a: 1 });
+  const entry = cache.get('title', 42);
+  assert.deepEqual(entry.data, { a: 1 });
+  assert.equal(entry.fetchedAt, new Date(clock.t).toISOString());
+  assert.equal(entry.schema, 1);
+  assert.equal(entry.kind, 'title');
+});
+
+test('freshness follows TTL per kind', (t) => {
+  const { clock, cache } = setup(t);
+  cache.put('fresh', 'global', []);
+  cache.put('title', 1, {});
+  clock.t += 8 * DAY;
+  assert.equal(cache.isFresh('fresh', 'global'), false);
+  assert.equal(cache.isFresh('title', 1), true);
+  clock.t += 23 * DAY;
+  assert.equal(cache.isFresh('title', 1), false);
+  assert.notEqual(cache.get('title', 1), null);
+  assert.equal(cache.list('title').length, 1);
+});
+
+test('prune keeps the newest maxFiles', (t) => {
+  const { clock, cache } = setup(t, { maxFiles: 3 });
+  for (let i = 1; i <= 5; i++) { clock.t += 1000; cache.put('title', i, { i }); }
+  assert.equal(cache.stats().files, 3);
+  assert.equal(cache.get('title', 1), null);
+  assert.equal(cache.get('title', 2), null);
+  assert.notEqual(cache.get('title', 5), null);
+});
+
+test('prune removes the oldest entries across kinds', (t) => {
+  const { clock, cache } = setup(t, { maxFiles: 3 });
+  const kinds = ['title', 'recs', 'creator', 'fresh', 'tagTop'];
+  kinds.forEach((kind, i) => { clock.t += 1000; cache.put(kind, i, { i }); });
+  assert.equal(cache.stats().files, 3);
+  assert.equal(cache.get('title', 0), null);
+  assert.equal(cache.get('recs', 1), null);
+  for (const [i, kind] of kinds.entries()) if (i >= 2) assert.notEqual(cache.get(kind, i), null, kind);
+});
+
+test('a write under the cap does not read other entries', (t) => {
+  const { dir, cache } = setup(t, { maxFiles: 10 });
+  cache.put('recs', 7, {});
+  const file = path.join(dir, 'recs', fs.readdirSync(path.join(dir, 'recs'))[0]);
+  fs.writeFileSync(file, '{broken');
+  cache.put('title', 1, {});
+  assert.equal(fs.readFileSync(file, 'utf8'), '{broken');
+});
+
+test('over the cap, corrupt and schema-mismatch entries go before valid ones', (t) => {
+  const { dir, clock, cache } = setup(t, { maxFiles: 2 });
+  cache.put('recs', 1, {});
+  cache.put('recs', 2, {});
+  const [first, second] = fs.readdirSync(path.join(dir, 'recs')).map(name => path.join(dir, 'recs', name));
+  fs.writeFileSync(first, '{broken');
+  fs.writeFileSync(second, JSON.stringify({ schema: 2, kind: 'recs', key: 2, fetchedAt: new Date(clock.t).toISOString(), data: {} }));
+  clock.t += 1000;
+  cache.put('title', 3, {});
+  assert.equal(fs.existsSync(first), false);
+  assert.equal(fs.existsSync(second), false);
+  assert.notEqual(cache.get('title', 3), null);
+});
+
+test('schema-mismatch entry is deleted like a corrupt one', (t) => {
+  const { dir, clock, cache } = setup(t);
+  cache.put('recs', 7, {});
+  const file = path.join(dir, 'recs', fs.readdirSync(path.join(dir, 'recs'))[0]);
+  fs.writeFileSync(file, JSON.stringify({ schema: 2, kind: 'recs', key: 7, fetchedAt: new Date(clock.t).toISOString(), data: {} }));
+  assert.equal(cache.get('recs', 7), null);
+  assert.equal(fs.existsSync(file), false);
+});
+
+test('corrupt entry is deleted', (t) => {
+  const { dir, cache } = setup(t);
+  cache.put('recs', 7, {});
+  const file = path.join(dir, 'recs', fs.readdirSync(path.join(dir, 'recs'))[0]);
+  fs.writeFileSync(file, '{broken');
+  assert.equal(cache.get('recs', 7), null);
+  assert.equal(fs.existsSync(file), false);
+});
+
+test('odd keys are file-safe', (t) => {
+  const { dir, cache } = setup(t);
+  cache.put('tagTop', 'genre:Slice of Life/../x', [1]);
+  const files = fs.readdirSync(path.join(dir, 'tagTop'));
+  assert.equal(files.length, 1);
+  assert.deepEqual(fs.readdirSync(dir), ['tagTop']);
+  assert.deepEqual(cache.get('tagTop', 'genre:Slice of Life/../x').data, [1]);
+});
+
+test('unknown kind throws', (t) => {
+  const { cache } = setup(t);
+  assert.throws(() => cache.put('nope', 1, {}), TypeError);
+  assert.throws(() => cache.get('nope', 1), TypeError);
+});

test/discovery-harvest.test.js

@@ -0,0 +1,342 @@
+'use strict';
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { DiscoveryCache } = require('../discovery/cache');
+const { AniListStop, AniListSkip } = require('../discovery/anilist');
+const {
+  BUDGETS, REFRESH_COOLDOWN_MS, MAX_CONSECUTIVE_FAILURES, runHarvest, shouldRunNightly, Harvester,
+} = require('../discovery/harvest');
+
+const tempCache = (t) => {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'discovery-harvest-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  return new DiscoveryCache(dir);
+};
+
+const record = (id, { tags = [], authors = [] } = {}) => ({ anilistId: id, title: `T${id}`, tags, authors });
+
+// Fake client: answers from fixture maps, logs every call, and throws scripted errors by call number.
+function fakeClient({ titles = {}, errors = {} } = {}) {
+  const calls = [];
+  const answer = (name, key, value) => {
+    calls.push(`${name}:${key}`);
+    const error = errors[calls.length];
+    if (error) throw error;
+    return value;
+  };
+  return {
+    calls,
+    async title(id) { return answer('title', id, id in titles ? titles[id] : record(id)); },
+    async recs(id) { return answer('recs', id, [{ anilistId: id * 100 }]); },
+    async creator(staffId) { return answer('creator', staffId, [{ anilistId: staffId * 10 }]); },
+    async tagTop({ name, kind }) { return answer('tagTop', `${kind}:${name}`, [{ anilistId: 1 }]); },
+    async fresh() { return answer('fresh', 'global', [{ anilistId: 2 }]); },
+  };
+}
+
+const anchor = (anilistId, weight = 1) => ({ anilistId, weight, source: 'owned', title: null });
+
+test('constants', () => {
+  assert.deepEqual(BUDGETS, { nightly: 300, admin: 300, user: 60 });
+  assert.equal(REFRESH_COOLDOWN_MS, 3600000);
+  assert.equal(MAX_CONSECUTIVE_FAILURES, 5);
+});
+
+test('plan order and dedupe', async (t) => {
+  const cache = tempCache(t);
+  const titles = {
+    1: record(1, { tags: [{ name: 'Drama', kind: 'genre', rank: 100, isAdult: false }], authors: [{ staffId: 9, name: 'A', role: 'Story' }] }),
+    2: record(2, { tags: [{ name: 'Drama', kind: 'genre', rank: 100, isAdult: false }] }),
+    3: record(3, { tags: [{ name: 'Gore', kind: 'tag', rank: 80, isAdult: false }], authors: [{ staffId: 8, name: 'B', role: 'Art' }] }),
+  };
+  const client = fakeClient({ titles });
+  const targets = [
+    { user: 'ann', anchors: [anchor(1, 5), anchor(2, 1)], mature: false },
+    { user: 'bob', anchors: [anchor(1, 3), anchor(3, 1)], mature: false },
+  ];
+  const result = await runHarvest({ kind: 'admin', targets, cache, client, budget: 300 });
+  assert.deepEqual(client.calls, [
+    'title:1', 'title:2', 'title:3',
+    'recs:1', 'recs:2', 'recs:3',
+    'creator:9',
+    'tagTop:genre:Drama', 'tagTop:tag:Gore',
+    'fresh:global',
+  ]);
+  assert.deepEqual(result, { requests: 10, budget: 300, status: 'success', stopReason: null });
+  assert.equal(cache.get('title', 1).data.anilistId, 1);
+  assert.deepEqual(cache.get('recs', 2).data, [{ anilistId: 200 }]);
+  assert.deepEqual(cache.get('creator', 9).data, [{ anilistId: 90 }]);
+  assert.deepEqual(cache.get('tagTop', 'genre:Drama').data, [{ anilistId: 1 }]);
+  assert.deepEqual(cache.get('fresh', 'global').data, [{ anilistId: 2 }]);
+});
+
+test('non-manga titles are counted but not cached', async (t) => {
+  const cache = tempCache(t);
+  const client = fakeClient({ titles: { 4: null } });
+  const result = await runHarvest({ kind: 'user', targets: [{ user: 'ann', anchors: [anchor(4, 5)], mature: false }], cache, client, budget: 60 });
+  assert.equal(cache.get('title', 4), null);
+  assert.deepEqual(client.calls, ['title:4', 'recs:4', 'fresh:global']);
+  assert.equal(result.requests, 3);
+});
+
+test('fresh cache entries are skipped', async (t) => {
+  const cache = tempCache(t);
+  cache.put('title', 1, record(1));
+  cache.put('recs', 1, []);
+  cache.put('fresh', 'global', []);
+  const client = fakeClient();
+  const result = await runHarvest({ kind: 'user', targets: [{ user: 'ann', anchors: [anchor(1), anchor(2)], mature: false }], cache, client, budget: 60 });
+  assert.deepEqual(client.calls, ['title:2', 'recs:2']);
+  assert.equal(result.requests, 2);
+  assert.equal(result.stopReason, null);
+});
+
+test('budget stops with reason budget', async (t) => {
+  const cache = tempCache(t);
+  const client = fakeClient();
+  const targets = [{ user: 'ann', anchors: [anchor(1), anchor(2), anchor(3)], mature: false }];
+  const result = await runHarvest({ kind: 'user', targets, cache, client, budget: 3 });
+  assert.deepEqual(result, { requests: 3, budget: 3, status: 'success', stopReason: 'budget' });
+  assert.equal(client.calls.length, 3);
+});
+
+test('429 stops at once', async (t) => {
+  const cache = tempCache(t);
+  const client = fakeClient({ errors: { 2: new AniListStop('rate-limited') } });
+  const targets = [{ user: 'ann', anchors: [anchor(1), anchor(2), anchor(3)], mature: false }];
+  const result = await runHarvest({ kind: 'admin', targets, cache, client, budget: 300 });
+  assert.deepEqual(result, { requests: 2, budget: 300, status: 'stopped', stopReason: 'rate-limited' });
+  assert.equal(client.calls.length, 2);
+  assert.equal(cache.get('title', 2), null);
+});
+
+test('network error stops with upstream-errors after 5', async (t) => {
+  const skip = () => new AniListSkip('timeout');
+  const targets = [{ user: 'ann', anchors: [1, 2, 3, 4, 5, 6, 7, 8].map((id) => anchor(id)), mature: false }];
+
+  const client = fakeClient({ errors: { 1: skip(), 2: skip(), 3: skip(), 4: skip(), 5: skip() } });
+  const result = await runHarvest({ kind: 'admin', targets, cache: tempCache(t), client, budget: 300 });
+  assert.deepEqual(result, { requests: 5, budget: 300, status: 'stopped', stopReason: 'upstream-errors' });
+  assert.equal(client.calls.length, 5);
+
+  // A success in the middle resets the counter, so the run gets past five failures in total.
+  const mixed = fakeClient({ errors: { 1: skip(), 2: skip(), 3: skip(), 4: skip(), 6: skip(), 7: skip(), 8: skip() } });
+  const reset = await runHarvest({ kind: 'admin', targets, cache: tempCache(t), client: mixed, budget: 300 });
+  assert.equal(reset.status, 'success');
+  assert.equal(reset.stopReason, null);
+  assert.ok(mixed.calls.length > 8);
+});
+
+test('plan for anchorless user only fetches fresh', async (t) => {
+  const client = fakeClient();
+  const result = await runHarvest({ kind: 'user', targets: [{ user: 'ann', anchors: [], mature: false }], cache: tempCache(t), client, budget: 60 });
+  assert.deepEqual(client.calls, ['fresh:global']);
+  assert.deepEqual(result, { requests: 1, budget: 60, status: 'success', stopReason: null });
+});
+
+test('shouldRunNightly', () => {
+  const at = (h, m) => new Date(2026, 9, 25, h, m);
+  assert.equal(shouldRunNightly({ now: at(3, 29), harvestAt: '03:30', lastNightlyDate: null, enabled: true }), false);
+  assert.equal(shouldRunNightly({ now: at(3, 30), harvestAt: '03:30', lastNightlyDate: null, enabled: true }), true);
+  assert.equal(shouldRunNightly({ now: at(23, 0), harvestAt: '03:30', lastNightlyDate: '2026-10-24', enabled: true }), true);
+  assert.equal(shouldRunNightly({ now: at(3, 30), harvestAt: '03:30', lastNightlyDate: '2026-10-25', enabled: true }), false);
+  assert.equal(shouldRunNightly({ now: at(3, 30), harvestAt: '03:30', lastNightlyDate: null, enabled: false }), false);
+});
+
+// A run that waits until the test releases it, so queue state can be observed.
+function gatedHarvester({ onDone = async () => {}, now, load }) {
+  const gates = [];
+  const patches = [];
+  const done = [];
+  const client = {
+    calls: [],
+    async fresh() {
+      this.calls.push('fresh');
+      await new Promise((resolve, reject) => gates.push({ resolve, reject }));
+      return [];
+    },
+  };
+  // Nothing stays cached, so every job makes its one fresh call and waits at the gate.
+  const cache = { isFresh: () => false, get: () => null, put() {} };
+  const harvester = new Harvester({
+    cache,
+    client,
+    store: { patch(value) { patches.push(structuredClone(value)); }, ...(load ? { load } : {}) },
+    targetsFor: (kind, user) => [{ user: user || 'all', anchors: [], mature: false }],
+    onDone: async (job, result) => { done.push({ job, result }); await onDone(job, result); },
+    now,
+  });
+  // Lets the waiting request finish, or fail with `error`.
+  const release = async (error) => {
+    for (let i = 0; !gates.length; i++) {
+      if (i > 1000) throw new Error('no harvest request is waiting');
+      await new Promise(setImmediate);
+    }
+    const gate = gates.shift();
+    if (error) gate.reject(error); else gate.resolve();
+    await new Promise(setImmediate);
+  };
+  return { harvester, patches, done, release, client };
+}
+
+const settle = async () => { for (let i = 0; i < 20; i++) await new Promise(setImmediate); };
+
+test('harvester single-flight, positions, 409 and 429 cooldown', async (t) => {
+  let clock = Date.parse('2026-10-25T10:00:00Z');
+  const h = gatedHarvester({ now: () => clock });
+  const { harvester } = h;
+
+  assert.deepEqual(harvester.state('ann'), { state: 'idle' });
+  assert.equal(harvester.current(), null);
+
+  const first = harvester.request({ kind: 'user', user: 'ann' });
+  assert.equal(first.position, 0);
+  assert.equal(typeof first.harvestId, 'string');
+  assert.equal(harvester.state('ann').state, 'running');
+  assert.deepEqual(harvester.current(), { id: first.harvestId, kind: 'user', user: 'ann', startedAt: '2026-10-25T10:00:00.000Z' });
+
+  assert.throws(() => harvester.request({ kind: 'admin' }), (error) => error.status === 409);
+  assert.throws(() => harvester.request({ kind: 'nightly' }), (error) => error.status === 409);
+  assert.throws(() => harvester.request({ kind: 'user', user: 'ann' }), (error) => error.status === 409);
+
+  const second = harvester.request({ kind: 'user', user: 'bob' });
+  assert.equal(second.position, 1);
+  assert.deepEqual(harvester.state('bob'), { state: 'queued', position: 1, nextAllowedAt: '2026-10-25T11:00:00.000Z' });
+  assert.throws(() => harvester.request({ kind: 'user', user: 'bob' }), (error) => error.status === 409);
+
+  await h.release();
+  await settle();
+  assert.equal(harvester.state('bob').state, 'running');
+  assert.equal(harvester.state('ann').state, 'idle');
+
+  clock += 30 * 60000;
+  assert.throws(() => harvester.request({ kind: 'user', user: 'ann' }), (error) => {
+    assert.equal(error.status, 429);
+    assert.equal(error.nextAllowedAt, '2026-10-25T11:00:00.000Z');
+    return true;
+  });
+  assert.deepEqual(harvester.state('ann'), { state: 'idle', nextAllowedAt: '2026-10-25T11:00:00.000Z' });
+
+  await h.release();
+  await settle();
+  assert.equal(harvester.current(), null);
+
+  // The persisted records: running first, then the final one, for each job in order.
+  assert.deepEqual(h.patches.map((p) => [p.harvest.user, p.harvest.status]),
+    [['ann', 'running'], ['ann', 'success'], ['bob', 'running'], ['bob', 'success']]);
+  assert.deepEqual(h.patches[0].harvest, { id: first.harvestId, kind: 'user', user: 'ann', status: 'running',
+    startedAt: '2026-10-25T10:00:00.000Z', requests: 0, budget: 60 });
+  assert.equal(h.patches[1].harvest.requests, 1);
+  assert.equal(h.patches[1].harvest.stopReason, null);
+  assert.equal(typeof h.patches[1].harvest.finishedAt, 'string');
+  assert.deepEqual(h.done.map((d) => d.job.user), ['ann', 'bob']);
+
+  clock += 30 * 60000;
+  const admin = harvester.request({ kind: 'admin' });
+  assert.equal(admin.position, 0);
+  const ann = harvester.request({ kind: 'user', user: 'ann' });
+  assert.equal(ann.position, 1);
+  await h.release();
+  await h.release();
+  await settle();
+  assert.equal(h.patches.at(-4).harvest.budget, 300);
+});
+
+test('onDone failure does not stall the queue', async (t) => {
+  const errors = [];
+  const original = console.error;
+  console.error = (...args) => errors.push(args);
+  t.after(() => { console.error = original; });
+  const h = gatedHarvester({ now: () => Date.parse('2026-10-25T10:00:00Z'), onDone: async () => { throw new Error('score failed'); } });
+  h.harvester.request({ kind: 'nightly' });
+  h.harvester.request({ kind: 'user', user: 'ann' });
+  await h.release();
+  await h.release();
+  await settle();
+  assert.deepEqual(h.done.map((d) => d.job.kind), ['nightly', 'user']);
+  assert.equal(h.harvester.current(), null);
+  assert.ok(errors.length >= 2);
+});
+
+test('AniList stop pauses user refreshes until an admin or nightly run', async () => {
+  let clock = Date.parse('2026-10-25T10:00:00Z');
+  const h = gatedHarvester({ now: () => clock });
+  const { harvester, client } = h;
+  harvester.request({ kind: 'user', user: 'ann' });
+  harvester.request({ kind: 'user', user: 'bob' });
+  assert.equal(harvester.state('bob').state, 'queued');
+
+  // (a) 429 while ann runs: bob's queued refresh is dropped without touching AniList.
+  await h.release(new AniListStop('rate-limited'));
+  await settle();
+  assert.equal(client.calls.length, 1);
+  assert.equal(harvester.current(), null);
+  assert.deepEqual(harvester.state('bob'), { state: 'idle', blockedReason: 'rate-limited' });
+  assert.deepEqual(h.patches.map((p) => [p.harvest.user, p.harvest.status, p.harvest.stopReason ?? null]),
+    [['ann', 'running', null], ['ann', 'stopped', 'rate-limited']]);
+  assert.deepEqual(h.done.map((d) => d.job.user), ['ann']);
+
+  // (b) a new refresh is refused while paused, even for a user outside any cooldown.
+  clock += 2 * REFRESH_COOLDOWN_MS;
+  assert.throws(() => harvester.request({ kind: 'user', user: 'cid' }), (error) => {
+    assert.equal(error.status, 503);
+    assert.equal(error.blockedReason, 'rate-limited');
+    assert.match(error.message, /AniList paused the harvest: rate-limited/);
+    return true;
+  });
+  assert.equal(client.calls.length, 1);
+
+  // (c) an accepted admin run is the allowed retry and lifts the pause.
+  harvester.request({ kind: 'admin' });
+  assert.equal(harvester.state('cid').blockedReason, undefined);
+  await h.release();
+  await settle();
+  assert.equal(client.calls.length, 2);
+  assert.equal(harvester.request({ kind: 'user', user: 'cid' }).position, 0);
+  await h.release();
+  await settle();
+});
+
+test('pause survives a restart through the stored harvest record', () => {
+  const load = () => ({ harvest: { status: 'stopped', stopReason: 'forbidden' } });
+  const { harvester } = gatedHarvester({ now: () => 0, load });
+  assert.throws(() => harvester.request({ kind: 'user', user: 'ann' }),
+    (error) => error.status === 503 && error.blockedReason === 'forbidden');
+  assert.equal(harvester.state('ann').blockedReason, 'forbidden');
+
+  const budgetStop = gatedHarvester({ now: () => 0, load: () => ({ harvest: { status: 'stopped', stopReason: 'budget' } }) });
+  assert.deepEqual(budgetStop.harvester.state('ann'), { state: 'idle' });
+  const broken = gatedHarvester({ now: () => 0, load: () => { throw new Error('corrupt'); } });
+  assert.deepEqual(broken.harvester.state('ann'), { state: 'idle' });
+});
+
+test('unknown kinds are refused', () => {
+  const { harvester } = gatedHarvester({ now: () => 0 });
+  for (const kind of ['constructor', 'toString', 'other'])
+    assert.throws(() => harvester.request({ kind }), (error) => error.status === 400);
+});
+
+test('local fault keeps the real request count and async targets work', async (t) => {
+  const original = console.error;
+  console.error = () => {};
+  t.after(() => { console.error = original; });
+  let puts = 0;
+  const cache = { isFresh: () => false, get: () => null, put() { if (++puts === 2) throw new Error('disk full'); } };
+  const patches = [];
+  const done = [];
+  const harvester = new Harvester({
+    cache, client: fakeClient(), store: { patch(value) { patches.push(value); } },
+    targetsFor: async () => [{ user: 'ann', anchors: [anchor(1)], mature: false }],
+    onDone: async (job, result) => { done.push(result); },
+  });
+  harvester.request({ kind: 'admin' });
+  await settle();
+  assert.deepEqual(patches.at(-1).harvest.status, 'stopped');
+  assert.equal(patches.at(-1).harvest.stopReason, 'error');
+  assert.equal(patches.at(-1).harvest.requests, 2);
+  assert.equal(done.length, 1);
+});

test/discovery-server.test.js

@@ -0,0 +1,318 @@
+'use strict';
+// BL-0051: discovery harvest routes, per-user refresh and nightly state, against a fixture AniList server.
+const test = require('node:test'), assert = require('node:assert/strict');
+const fs = require('fs'), os = require('os'), path = require('path'), net = require('net'), http = require('http'), crypto = require('crypto');
+const { spawn } = require('child_process');
+const { Canvas } = require('../tools/png');
+
+const FIXTURES = path.join(__dirname, 'fixtures', 'anilist');
+const fixture = name => JSON.parse(fs.readFileSync(path.join(FIXTURES, `${name}.json`), 'utf8'));
+const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
+
+// The anchor title carries no tags or staff, so a run stays at title + recs + fresh (the client allows 25 a minute).
+function fixtureBodies() {
+  const title = fixture('title');
+  Object.assign(title.data.Media, { genres: [], tags: [], staff: { edges: [] } });
+  const recs = fixture('recs');
+  recs.data.Media.recommendations.nodes[0].rating = 500;
+  return { title, recs, creator: fixture('creator'), tagtop: fixture('tagtop'), fresh: fixture('fresh') };
+}
+
+async function startAniList({ delayMs = 0 } = {}) {
+  const bodies = fixtureBodies(), seen = [];
+  const server = http.createServer((req, res) => {
+    let raw = '';
+    req.on('data', chunk => { raw += chunk; });
+    req.on('end', async () => {
+      const query = String(JSON.parse(raw || '{}').query || '');
+      const kind = query.includes('recommendations') ? 'recs' : query.includes('Staff(') ? 'creator'
+        : /genre_in|tag_in/.test(query) ? 'tagtop' : query.includes('startDate_greater') ? 'fresh' : 'title';
+      seen.push(kind);
+      if (delayMs) await sleep(delayMs);
+      res.writeHead(200, { 'content-type': 'application/json' });
+      res.end(JSON.stringify(bodies[kind]));
+    });
+  });
+  await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
+  return { url: `http://127.0.0.1:${server.address().port}/`, seen, close: () => new Promise(resolve => server.close(resolve)) };
+}
+
+async function freePort() {
+  const listener = net.createServer();
+  await new Promise(resolve => listener.listen(0, '127.0.0.1', resolve));
+  const port = listener.address().port;
+  await new Promise(resolve => listener.close(resolve));
+  return port;
+}
+
+const hashPw = pw => { const salt = crypto.randomBytes(16).toString('hex'); return salt + ':' + crypto.scryptSync(pw, salt, 32).toString('hex'); };
+const sha = s => crypto.createHash('sha256').update(s).digest('hex');
+const USERS = { boss: { admin: true, discovery: false }, alice: { discovery: true }, bob: { discovery: true }, carol: { discovery: false } };
+
+// A disposable reader with ready-made accounts and sessions; nightly harvests are off unless the store says otherwise.
+async function startReader({ anilist, discovery = {}, mangaMeta = null, scraperUrl = 'http://127.0.0.1:9' } = {}) {
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-discovery-server-'));
+  const library = path.join(root, 'library'), data = path.join(root, 'data');
+  fs.mkdirSync(path.join(library, 'Example Manga'), { recursive: true });
+  fs.writeFileSync(path.join(library, 'Example Manga', 'notes.txt'), 'not a chapter');
+  if (mangaMeta) {
+    const chapter = path.join(library, mangaMeta.title, 'v01', 'c0001');
+    fs.mkdirSync(chapter, { recursive: true });
+    fs.writeFileSync(path.join(chapter, '0001.png'), new Canvas(20, 30, [60, 90, 140]).png());
+    fs.writeFileSync(path.join(library, mangaMeta.title, 'manga.json'), JSON.stringify(mangaMeta));
+  }
+  fs.mkdirSync(path.join(data, 'scraper'), { recursive: true });
+  fs.writeFileSync(path.join(data, 'scraper', 'queue.json'), JSON.stringify({ queue: [] }));
+  const users = {}, sessions = {}, cookies = {};
+  for (const [name, spec] of Object.entries(USERS)) {
+    users[name] = { name, hash: hashPw(`${name}-password`), admin: !!spec.admin, created: Date.now(),
+      settings: { discoveryEnabled: spec.discovery } };
+    const token = crypto.randomBytes(32).toString('base64url');
+    sessions[sha(token)] = { u: name, t: Date.now() };
+    cookies[name] = `sid=${token}`;
+  }
+  fs.writeFileSync(path.join(data, 'db.json'), JSON.stringify({ users, sessions,
+    discoveryFeedback: { alice: { 'anilist:101': 'interested' } } }));
+  fs.writeFileSync(path.join(data, 'discovery.json'), JSON.stringify({ schema: 1, revision: 0, candidates: {}, feeds: {},
+    lastRun: { status: 'never' }, harvest: null, settings: { harvestEnabled: false, harvestAt: '03:30' },
+    nightly: { lastDate: null }, ...discovery }));
+
+  const port = await freePort(), output = [];
+  const child = spawn(process.execPath, ['server.js'], { cwd: path.resolve(__dirname, '..'), windowsHide: true,
+    env: { ...process.env, LIBRARY: library, DATA: data, PORT: String(port), SCAN_INTERVAL: '0', FULL_SCAN_INTERVAL: '0',
+      SCRAPER_URL: scraperUrl, SCRAPER_TOKEN: 'fixture-token', SCRAPER_TOKEN_FILE: '', BACKUP: '',
+      DISCOVERY_ANILIST_URL: anilist ? anilist.url : 'http://127.0.0.1:9/', DISCOVERY_HARVEST_AT: '', GREATEST_COVER_WARM: '0' } });
+  child.stdout.on('data', d => output.push(String(d)));
+  child.stderr.on('data', d => output.push(String(d)));
+  const base = `http://127.0.0.1:${port}`;
+  let ready = false;
+  for (let n = 0; n < 100 && !ready; n++) {
+    ready = await fetch(base + '/health').then(r => r.ok).catch(() => false);
+    if (!ready) await sleep(50);
+  }
+  assert.ok(ready, output.join(''));
+  const call = async (who, method, url, body) => {
+    const response = await fetch(base + url, { method, headers: { cookie: cookies[who], ...(body ? { 'content-type': 'application/json' } : {}) },
+      body: body === undefined ? undefined : JSON.stringify(body) });
+    return { status: response.status, body: await response.json().catch(() => null) };
+  };
+  return {
+    root, library, data, output,
+    get: (who, url) => call(who, 'GET', url),
+    post: (who, url, body = {}) => call(who, 'POST', url, body),
+    async stop() {
+      await new Promise(resolve => { if (child.exitCode !== null) return resolve(); child.once('exit', resolve); child.kill(); });
+      fs.rmSync(root, { recursive: true, force: true });
+    },
+  };
+}
+
+async function until(check, what, timeoutMs = 20000) {
+  const deadline = Date.now() + timeoutMs;
+  let last;
+  while (Date.now() < deadline) {
+    last = await check();
+    if (last) return last;
+    await sleep(100);
+  }
+  assert.fail(`Timed out waiting for ${what}`);
+}
+const idle = (reader, who) => until(async () => {
+  const { body } = await reader.get(who, '/api/discovery');
+  return body.refresh.state === 'idle' ? body : null;
+}, `${who}'s refresh to finish`);
+const harvestDone = reader => until(async () => {
+  const { body } = await reader.get('boss', '/api/admin/discovery');
+  return !body.currentHarvest && body.harvest && body.harvest.status !== 'running' ? body : null;
+}, 'the harvest to finish');
+
+function snapshot(dir) {
+  const out = {};
+  const walk = rel => {
+    for (const entry of fs.readdirSync(path.join(dir, rel), { withFileTypes: true })) {
+      const child = path.join(rel, entry.name), stat = fs.statSync(path.join(dir, child));
+      out[child] = entry.isDirectory() ? 'dir' : `${stat.size}:${stat.mtimeMs}`;
+      if (entry.isDirectory()) walk(child);
+    }
+  };
+  walk('');
+  return out;
+}
+
+test('refresh requires opt-in, queues, runs and rescores only the caller', { timeout: 60000 }, async () => {
+  const anilist = await startAniList();
+  const reader = await startReader({ anilist });
+  try {
+    assert.equal((await reader.post('boss', '/api/admin/discovery/run')).status, 200);
+    await until(async () => (await reader.get('boss', '/api/admin/discovery')).body.currentRun?.status === 'success', 'the first run');
+    const bobBefore = (await reader.get('bob', '/api/discovery')).body;
+    assert.ok(bobBefore.generatedAt);
+    assert.deepEqual(bobBefore.refresh, { state: 'idle' });
+
+    const refused = await reader.post('carol', '/api/discovery/refresh');
+    assert.equal(refused.status, 403);
+
+    const accepted = await reader.post('alice', '/api/discovery/refresh');
+    assert.equal(accepted.status, 200, JSON.stringify(accepted.body));
+    assert.match(accepted.body.harvestId, /^[0-9a-f]{16}$/);
+    assert.equal(accepted.body.position, 0);
+
+    const alice = await idle(reader, 'alice');
+    assert.ok(alice.items.some(item => item.candidateId === 'anilist:201'), JSON.stringify(alice.items.map(i => i.candidateId)));
+    const rec = alice.items.find(item => item.candidateId === 'anilist:201');
+    assert.ok(rec.reasons.some(reason => /recommend/.test(reason)));
+    assert.ok(rec.candidate.fetchedAt);
+    assert.deepEqual(anilist.seen.sort(), ['fresh', 'recs', 'title']);
+
+    const bobAfter = (await reader.get('bob', '/api/discovery')).body;
+    assert.equal(bobAfter.generatedAt, bobBefore.generatedAt);
+    assert.deepEqual(bobAfter.items.map(i => i.candidateId), bobBefore.items.map(i => i.candidateId));
+
+    const admin = (await reader.get('boss', '/api/admin/discovery')).body;
+    assert.equal(admin.harvest.kind, 'user');
+    assert.equal(admin.harvest.user, 'alice');
+    assert.equal(admin.harvest.status, 'success');
+    assert.equal(admin.harvest.requests, 3);
+    assert.equal(admin.harvest.budget, 60);
+    assert.ok(admin.cache.files >= 3 && admin.cache.bytes > 0, JSON.stringify(admin.cache));
+    assert.ok(fs.existsSync(path.join(reader.data, 'discovery-cache')));
+  } finally { await reader.stop(); await anilist.close(); }
+});
+
+test('second refresh within an hour is 429 with nextAllowedAt', { timeout: 60000 }, async () => {
+  const anilist = await startAniList({ delayMs: 400 });
+  const reader = await startReader({ anilist });
+  try {
+    assert.equal((await reader.post('bob', '/api/discovery/refresh')).status, 200);
+    const duplicate = await reader.post('bob', '/api/discovery/refresh');
+    assert.equal(duplicate.status, 409);
+    const state = await idle(reader, 'bob');
+    assert.ok(Date.parse(state.refresh.nextAllowedAt) > Date.now() + 50 * 60000);
+
+    const cooling = await reader.post('bob', '/api/discovery/refresh');
+    assert.equal(cooling.status, 429);
+    assert.equal(cooling.body.nextAllowedAt, state.refresh.nextAllowedAt);
+    assert.equal(typeof cooling.body.error, 'string');
+  } finally { await reader.stop(); await anilist.close(); }
+});
+
+test('admin harvest is admin-only and single-flight', { timeout: 60000 }, async () => {
+  const anilist = await startAniList({ delayMs: 500 });
+  const reader = await startReader({ anilist });
+  try {
+    assert.equal((await reader.post('alice', '/api/admin/discovery/harvest')).status, 403);
+    const first = await reader.post('boss', '/api/admin/discovery/harvest');
+    assert.equal(first.status, 200, JSON.stringify(first.body));
+    assert.equal(first.body.position, 0);
+    assert.equal((await reader.post('boss', '/api/admin/discovery/harvest')).status, 409);
+    const running = (await reader.get('boss', '/api/admin/discovery')).body;
+    assert.equal(running.currentHarvest.id, first.body.harvestId);
+    assert.equal(running.currentHarvest.kind, 'admin');
+    const done = await harvestDone(reader);
+    assert.equal(done.harvest.kind, 'admin');
+    assert.equal(done.harvest.budget, 300);
+    // Afterwards every opted-in user was rescored.
+    for (const who of ['alice', 'bob']) assert.ok((await reader.get(who, '/api/discovery')).body.generatedAt, who);
+  } finally { await reader.stop(); await anilist.close(); }
+});
+
+// A scraper stand-in. The rescore asks it for the queue after the harvest's final store patch and
+// right before publishing, so onState can break discovery.json at exactly that point.
+async function startScraper(onState) {
+  const server = http.createServer((req, res) => {
+    if (req.url.startsWith('/api/state')) onState();
+    res.writeHead(200, { 'content-type': 'application/json' });
+    res.end(JSON.stringify({ queue: [] }));
+  });
+  await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
+  return { url: `http://127.0.0.1:${server.address().port}`, close: () => new Promise(resolve => server.close(resolve)) };
+}
+
+test('a failed rescore after a harvest reaches the admin as lastError', { timeout: 60000 }, async () => {
+  const anilist = await startAniList();
+  let reader, armed = false, saved = null;
+  const scraper = await startScraper(() => {
+    if (!armed) return;
+    armed = false;
+    const file = path.join(reader.data, 'discovery.json');
+    saved = fs.readFileSync(file, 'utf8');
+    fs.writeFileSync(file, '{broken'); // the publish now refuses to overwrite it
+  });
+  // A malformed manga.json (tags not a list, relations an object) must not be what breaks scoring.
+  reader = await startReader({ anilist, scraperUrl: scraper.url,
+    mangaMeta: { title: 'Broken Tags', tags: 5, 'x-anilist': { id: 101, relations: {} } } });
+  try {
+    await until(async () => (await reader.get('boss', '/api/library')).body?.mangas?.length === 1, 'the library scan');
+    armed = true;
+    const first = await reader.post('boss', '/api/admin/discovery/harvest');
+    assert.equal(first.status, 200);
+    await until(async () => saved && reader.output.join('').includes(`${first.body.harvestId} follow-up failed`), 'the failed rescore');
+    fs.writeFileSync(path.join(reader.data, 'discovery.json'), saved);
+    const admin = await until(async () => {
+      const { body } = await reader.get('boss', '/api/admin/discovery');
+      return body.lastError ? body : null;
+    }, 'the rescore error');
+    assert.match(admin.lastError, /^Re-scoring after harvest [0-9a-f]{16} failed: Discovery store unavailable/);
+    assert.equal(admin.currentRun, null);
+    assert.equal(admin.harvest.kind, 'admin');
+
+    // The next harvest's successful rescore clears the stale error.
+    const second = await reader.post('boss', '/api/admin/discovery/harvest');
+    assert.equal(second.status, 200, JSON.stringify(second.body));
+    const cleared = await until(async () => {
+      const { body } = await reader.get('boss', '/api/admin/discovery');
+      return body.harvest?.id === second.body.harvestId && !body.currentHarvest && body.lastError === null ? body : null;
+    }, 'the error to clear');
+    assert.ok(cleared.candidateCount > 0);
+  } finally { await reader.stop(); await anilist.close(); await scraper.close(); }
+});
+
+test('settings validation', { timeout: 30000 }, async () => {
+  const reader = await startReader();
+  try {
+    const before = (await reader.get('boss', '/api/admin/discovery')).body;
+    assert.deepEqual(before.settings, { harvestEnabled: false, harvestAt: '03:30' });
+    assert.equal(before.nextNightlyAt, null);
+    assert.equal((await reader.post('alice', '/api/admin/discovery/settings', { harvestEnabled: true, harvestAt: '04:00' })).status, 403);
+    for (const body of [{ harvestEnabled: true, harvestAt: '25:00' }, { harvestEnabled: true, harvestAt: '4:00' },
+      { harvestEnabled: 'yes', harvestAt: '04:00' }, { harvestEnabled: true }, { harvestEnabled: true, harvestAt: '04:00', extra: 1 }]) {
+      const response = await reader.post('boss', '/api/admin/discovery/settings', body);
+      assert.equal(response.status, 400, JSON.stringify(body));
+    }
+    const saved = await reader.post('boss', '/api/admin/discovery/settings', { harvestEnabled: true, harvestAt: '23:59' });
+    assert.equal(saved.status, 200);
+    const after = (await reader.get('boss', '/api/admin/discovery')).body;
+    assert.deepEqual(after.settings, { harvestEnabled: true, harvestAt: '23:59' });
+    const next = new Date(after.nextNightlyAt);
+    // The next slot today at 23:59, or "due now" when the test happens to run inside that minute.
+    const atSlot = next.getHours() === 23 && next.getMinutes() === 59 && next.getTime() > Date.now() - 120000;
+    assert.ok(atSlot || Math.abs(next.getTime() - Date.now()) < 120000, after.nextNightlyAt);
+  } finally { await reader.stop(); }
+});
+
+test('interrupted harvest is marked on startup', { timeout: 30000 }, async () => {
+  const reader = await startReader({ discovery: { harvest: { id: 'abc', kind: 'nightly', user: null, status: 'running',
+    startedAt: '2026-10-24T03:30:00.000Z', requests: 12, budget: 300 } } });
+  try {
+    const { body } = await reader.get('boss', '/api/admin/discovery');
+    assert.equal(body.harvest.status, 'interrupted');
+    assert.equal(body.harvest.id, 'abc');
+    assert.equal(body.harvest.requests, 12);
+    assert.ok(Date.parse(body.harvest.finishedAt));
+    assert.equal(body.currentHarvest, null);
+  } finally { await reader.stop(); }
+});
+
+test('harvest never touches scraper queue or library', { timeout: 60000 }, async () => {
+  const anilist = await startAniList();
+  const reader = await startReader({ anilist });
+  try {
+    const libraryBefore = snapshot(reader.library), scraperBefore = snapshot(path.join(reader.data, 'scraper'));
+    assert.equal((await reader.post('alice', '/api/discovery/refresh')).status, 200);
+    await idle(reader, 'alice');
+    assert.equal((await reader.post('boss', '/api/admin/discovery/harvest')).status, 200);
+    await harvestDone(reader);
+    assert.deepEqual(snapshot(reader.library), libraryBefore);
+    assert.deepEqual(snapshot(path.join(reader.data, 'scraper')), scraperBefore);
+  } finally { await reader.stop(); await anilist.close(); }
+});

test/discovery.test.js

@@ -4,8 +4,9 @@ const assert = require('node:assert/strict');
 const fs = require('node:fs');
 const os = require('node:os');
 const path = require('node:path');
-const { generate } = require('../discovery/engine');
+const { generate, generateAsync, buildCandidates, candidatePool, feedForUser, mixSlots, mergeUserFeed, MAX_CANDIDATES, SLOTS } = require('../discovery/engine');
 const { DiscoveryStore } = require('../discovery/store');
+const { DiscoveryCache } = require('../discovery/cache');
 
 test('discovery is bounded, diversified, opt-in, evidence-linked and user-specific', () => {
   const { loadCatalog } = require('../greatest');
@@ -15,12 +16,17 @@ test('discovery is bounded, diversified, opt-in, evidence-linked and user-specif
     off: { name: 'off', settings: {} } };
   const db = { users, greatestRatings: { alice: { [greatest.entries[0].id]: 'flame' },
     bob: { [greatest.entries[20].id]: 'flame' } }, discoveryFeedback: { alice: {} }, progress: {} };
-  const result = generate({ db, library: [], greatest, cacheDir: path.join(os.tmpdir(), 'nonexistent-discovery-cache') });
-  assert.equal(Object.keys(result.candidates).length, 50);
+  const cacheDir = path.join(os.tmpdir(), 'nonexistent-discovery-cache');
+  const result = generate({ db, library: [], greatest, cacheDir });
+  // Only candidates a feed shows are published; the pool itself stays bounded at 2000.
+  const shown = new Set(Object.values(result.feeds).flatMap(feed => (feed.items || []).map(item => item.candidateId)));
+  assert.deepEqual(Object.keys(result.candidates).sort(), [...shown].sort());
+  const pool = candidatePool({ greatest, library: [], cacheDir });
+  assert.equal(pool.candidates.length, Math.min(2000, result.examined));
   assert.ok(result.feeds.alice.items.length > 0 && result.feeds.alice.items.length <= 20);
   assert.ok(result.feeds.bob.items.length > 0);
   assert.equal(result.feeds.off.disabled, true);
-  assert.ok(new Set(Object.values(result.candidates).map(c => c.period)).size >= 10, 'An early catalog page must not crowd out later eras');
+  assert.ok(new Set(pool.candidates.map(c => c.period)).size >= 10, 'An early catalog page must not crowd out later eras');
   assert.ok(result.feeds.alice.items.every(item => item.evidenceIds.every(id => item.evidence.some(e => e.id === id))));
   assert.ok(result.feeds.alice.items.some(item => item.evidence.some(e => e.kind === 'personal')));
   assert.ok(!result.feeds.bob.items.some(item => item.evidence.some(e => e.label?.includes(greatest.entries[0].title) && e.kind === 'personal')));
@@ -46,3 +52,321 @@ test('discovery store publishes atomically and refuses to overwrite corrupt data
   assert.throws(() => store.publish({ candidates: {}, feeds: {} }), /refusing to overwrite/);
   assert.equal(fs.readFileSync(store.file, 'utf8'), '{broken');
 });
+
+function tempStore(t) {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-discovery-meta-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  return new DiscoveryStore(dir);
+}
+const emptyRun = () => ({ candidates: {}, feeds: {}, startedAt: new Date().toISOString(), examined: 0, clipped: 0 });
+
+test('store loads a Phase A file without new fields', t => {
+  const store = tempStore(t);
+  fs.writeFileSync(store.file, JSON.stringify({ schema: 1, revision: 3, candidates: {}, feeds: {}, lastRun: { status: 'success' } }));
+  const data = store.load();
+  assert.equal(data.revision, 3);
+  assert.equal(data.harvest, null);
+  assert.equal(data.settings, null);
+  assert.deepEqual(data.nightly, { lastDate: null });
+});
+
+test('patch keeps revision and publish keeps harvest', t => {
+  const store = tempStore(t);
+  store.publish(emptyRun());
+  store.patch({ harvest: { status: 'success' } });
+  assert.equal(store.load().revision, 1);
+  store.patch({ settings: { a: 1 } });
+  store.patch({ nightly: { lastDate: '2026-10-25' } });
+  store.publish(emptyRun());
+  const data = store.load();
+  assert.equal(data.harvest.status, 'success');
+  assert.deepEqual(data.settings, { a: 1 });
+  assert.equal(data.nightly.lastDate, '2026-10-25');
+  assert.equal(data.revision, 2);
+});
+
+test('patch refuses a corrupt store', t => {
+  const store = tempStore(t);
+  fs.writeFileSync(store.file, '{broken');
+  assert.throws(() => store.patch({ harvest: {} }), /refusing to overwrite/);
+  assert.equal(fs.readFileSync(store.file, 'utf8'), '{broken');
+});
+
+// ---- Wider pool (BL-0051): AniList seeds, new score parts, slot mix ----
+const NOW = () => new Date('2026-10-25T12:00:00Z');
+const FETCHED_AT = '2026-10-20T00:00:00.000Z';
+const NO_SCRAPER_CACHE = path.join(os.tmpdir(), 'nonexistent-discovery-cache');
+const AUTHOR = { staffId: 7, name: 'Author Seven', role: 'Story' };
+function seedRecord(id, extra = {}) {
+  return { anilistId: id, title: `Seed ${id}`, altTitles: [], format: 'MANGA', isAdult: false, status: 'completed',
+    year: 2010, averageScore: 70, popularity: 1000, trending: 0,
+    tags: [{ name: 'Action', rank: 90, kind: 'genre', isAdult: false }],
+    authors: [{ staffId: 900 + id, name: `Writer ${id}`, role: 'Story' }],
+    coverUrl: `https://img.example/${id}.jpg`, siteUrl: `https://anilist.co/manga/${id}`, ...extra };
+}
+const seed = (id, extra) => ({ anilistId: id, record: seedRecord(id, extra) });
+function harvestFixture(t) {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-discovery-harvest-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  const cache = new DiscoveryCache(dir, { now: () => Date.parse(FETCHED_AT) });
+  cache.put('title', 100, seedRecord(100, { title: 'Anchor One', authors: [AUTHOR],
+    tags: [{ name: 'Drama', rank: 100, kind: 'genre', isAdult: false }, { name: 'Time Skip', rank: 80, kind: 'tag', isAdult: false }] }));
+  cache.put('recs', 100, [
+    { ...seed(201), rating: 42 },
+    { ...seed(209, { authors: [AUTHOR] }), rating: 10 },
+    { ...seed(207, { isAdult: true }), rating: 5 },
+    { ...seed(208, { tags: [{ name: 'Nudity', rank: 65, kind: 'tag', isAdult: true }] }), rating: 5 },
+    { ...seed(212), rating: 3 },
+  ]);
+  cache.put('creator', 7, [{ ...seed(202, { authors: [AUTHOR] }), role: 'Story' }, { ...seed(209, { authors: [AUTHOR] }), role: 'Story' }]);
+  cache.put('tagTop', 'genre:Drama', [seed(203, { averageScore: 80, tags: [{ name: 'Drama', rank: 90, kind: 'genre', isAdult: false }] })]);
+  cache.put('tagTop', 'tag:Time Skip', [seed(204, { averageScore: 70 })]);
+  cache.put('fresh', 'global', [seed(205, { status: 'ongoing', year: 2025 }), seed(206, { status: 'ongoing', year: 2020 }),
+    seed(210, { title: 'Owned Title', year: 2020 }), seed(211, { title: 'Queued Title' }), seed(213, { status: 'ongoing', year: 2026 })]);
+  return cache;
+}
+const fixtureGreatest = { entries: [
+  { id: 'g1', title: 'Anchor One', period: '2000s', rank: 1, note: '', metadata: { id: 100 } },
+  { id: 'g2', title: 'Voted Two', period: '1990s', rank: 2, note: '', metadata: { id: 212 } },
+] };
+function fixtureWorld(t, settings = {}) {
+  const user = { name: 'alice', settings: { discoveryEnabled: true, ...settings } };
+  const db = { users: { alice: user }, greatestRatings: { alice: { g1: 'flame', g2: 'good' } },
+    discoveryFeedback: { alice: { 'anilist:213': 'dismissed' } }, progress: {} };
+  const library = [{ id: 'm1', title: 'Owned Title', meta: { year: 2020, 'x-anilist': { id: 210 } } }];
+  return { user, db, library, greatest: fixtureGreatest, harvestCache: harvestFixture(t), queuedTitles: ['Queued Title'] };
+}
+function fixtureFeed(t, settings) {
+  const world = fixtureWorld(t, settings);
+  const candidates = buildCandidates({ greatest: world.greatest, library: world.library, cacheDir: NO_SCRAPER_CACHE, harvestCache: world.harvestCache });
+  const feed = feedForUser({ ...world, candidates, now: NOW });
+  return { world, candidates, feed, item: id => feed.items.find(item => item.candidateId === `anilist:${id}`) };
+}
+
+test('engine: empty harvest cache equals Phase A pool', t => {
+  const { loadCatalog } = require('../greatest');
+  const greatest = loadCatalog();
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-discovery-empty-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  const db = { users: {}, greatestRatings: {}, discoveryFeedback: {}, progress: {} };
+  const without = candidatePool({ greatest, library: [], cacheDir: NO_SCRAPER_CACHE });
+  const withEmpty = candidatePool({ greatest, library: [], cacheDir: NO_SCRAPER_CACHE, harvestCache: new DiscoveryCache(dir) });
+  assert.equal(MAX_CANDIDATES, 2000);
+  assert.deepEqual(withEmpty.candidates.map(c => c.id).sort(), without.candidates.map(c => c.id).sort());
+  const users = { alice: { name: 'alice', settings: { discoveryEnabled: true } } };
+  const run = harvestCache => generate({ db: { ...db, users }, library: [], greatest, cacheDir: NO_SCRAPER_CACHE, harvestCache });
+  assert.deepEqual(Object.keys(run(new DiscoveryCache(dir)).candidates).sort(), Object.keys(run(null).candidates).sort());
+});
+
+test('published candidates are only those referenced by a feed', async t => {
+  const world = fixtureWorld(t);
+  world.db.users.bob = { name: 'bob', settings: { discoveryEnabled: true } };
+  world.db.users.off = { name: 'off', settings: {} };
+  const args = { db: world.db, library: world.library, greatest: world.greatest, cacheDir: NO_SCRAPER_CACHE,
+    harvestCache: world.harvestCache, queuedTitles: world.queuedTitles, now: NOW };
+  const referenced = result => new Set(Object.values(result.feeds).flatMap(feed => (feed.items || []).map(item => item.candidateId)));
+  for (const result of [generate(args), await generateAsync(args), await generateAsync({ ...args, onlyUser: 'alice' })]) {
+    const shown = referenced(result);
+    assert.ok(shown.size > 0);
+    for (const id of Object.keys(result.candidates)) assert.ok(shown.has(id), `${id} is published but not shown`);
+    for (const id of shown) assert.ok(result.candidates[id], `${id} is shown but not published`);
+    assert.ok(result.examined > Object.keys(result.candidates).length, 'examined still counts the whole pool');
+    assert.equal(result.clipped, 0);
+  }
+});
+
+test('malformed library metadata and cache records score without throwing', t => {
+  const world = fixtureWorld(t, { discoveryReadingActivity: true });
+  world.library.push(
+    { id: 'm2', title: 'Bad Tags', meta: { tags: 5, 'x-anilist': { id: 220, relations: {} } } },
+    { id: 'm3', title: 'Bad Relations', meta: { tags: [7, null, { name: 'Drama' }, 'Drama'],
+      'x-anilist': { id: 221, relations: [null, 3, 'x', { id: 222, format: 'MANGA', type: 'SEQUEL', title: 'Sequel Title' }] } } },
+    { id: 'm4', title: 'Bad Identity', meta: { tags: { a: 1 }, 'x-anilist': 5 } });
+  world.db.progress = { alice: { m2: { c1: { done: true } }, m3: { c1: { done: true } } } };
+  world.harvestCache.put('title', 220, seedRecord(220, { authors: [null, 5, AUTHOR], tags: [null, 'Drama', { name: 'Drama', kind: 'genre' }] }));
+  world.harvestCache.put('recs', 220, [null, 5, { ...seed(230), rating: 7 }]);
+  const args = { db: world.db, library: world.library, greatest: world.greatest, cacheDir: NO_SCRAPER_CACHE,
+    harvestCache: world.harvestCache, queuedTitles: world.queuedTitles, now: NOW };
+  const result = generate(args);
+  const ids = result.feeds.alice.items.map(item => item.candidateId);
+  assert.ok(ids.includes('anilist:222'), JSON.stringify(ids));
+  assert.ok(ids.includes('anilist:230'), JSON.stringify(ids));
+});
+
+test('harvested records join the pool and fill Greatest candidates', t => {
+  const { candidates } = fixtureFeed(t);
+  const byId = new Map(candidates.map(c => [c.id, c]));
+  for (const id of [201, 202, 203, 204, 205, 206, 207, 208, 209]) assert.ok(byId.has(`anilist:${id}`), `anilist:${id}`);
+  const fromRecs = byId.get('anilist:201');
+  assert.equal(fromRecs.isAdult, false);
+  assert.equal(fromRecs.averageScore, 70);
+  assert.deepEqual(fromRecs.tags, ['Action']);
+  assert.deepEqual(fromRecs.authors, ['Writer 201']);
+  assert.equal(fromRecs.tagRecords[0].rank, 90);
+  assert.equal(fromRecs.authorRecords[0].staffId, 1101);
+  assert.equal(fromRecs.fetchedAt, FETCHED_AT);
+  const greatestOne = byId.get('anilist:212');
+  assert.equal(greatestOne.greatestId, 'g2');
+  assert.equal(greatestOne.title, 'Voted Two');
+  assert.equal(greatestOne.isAdult, false);
+  assert.equal(greatestOne.tagRecords.length, 1);
+});
+
+test('community rec, creator, tagTop and fresh parts and reasons', t => {
+  const { item } = fixtureFeed(t);
+  const rec = item(201);
+  assert.equal(rec.scoreParts.communityRec, 3.71);
+  assert.equal(rec.group, 'recs');
+  assert.equal(rec.reason, 'AniList readers who liked Anchor One recommend this (42).');
+  const recEvidence = rec.evidence.find(e => e.id === 'rec:100:anilist:201');
+  assert.equal(recEvidence.kind, 'recommendation');
+  assert.equal(recEvidence.fetchedAt, FETCHED_AT);
+  assert.equal(recEvidence.url, 'https://anilist.co/manga/201');
+  assert.ok(rec.evidenceIds.includes('rec:100:anilist:201'));
+
+  const creator = item(202);
+  assert.equal(creator.scoreParts.sameCreator, 4);
+  assert.equal(creator.group, 'creator');
+  assert.equal(creator.reason, 'Also by Author Seven — you liked Anchor One.');
+  const creatorEvidence = creator.evidence.find(e => e.id === 'creator:7:anilist:202');
+  assert.equal(creatorEvidence.kind, 'creator');
+  assert.equal(creatorEvidence.fetchedAt, FETCHED_AT);
+
+  const top = item(203);
+  assert.equal(top.scoreParts.tagTop, 3);
+  assert.equal(top.group, 'tagTop');
+  assert.equal(top.reason, 'Top-rated in Drama, one of your most-liked tags.');
+  assert.equal(top.evidence.find(e => e.id === 'tagtop:genre:Drama:anilist:203').kind, 'tag-top');
+  assert.equal(item(204).scoreParts.tagTop, 0, 'averageScore below 75 earns no tagTop part');
+
+  const fresh = item(205);
+  assert.equal(fresh.scoreParts.fresh, 2);
+  assert.equal(fresh.group, 'fresh');
+  assert.equal(fresh.reason, 'New: started 2025, ongoing and trending.');
+  const freshEvidence = fresh.evidence.find(e => e.id === 'fresh:anilist:205');
+  assert.equal(freshEvidence.kind, 'fresh');
+  assert.equal(freshEvidence.fetchedAt, FETCHED_AT);
+  assert.equal(item(206).scoreParts.fresh, 1);
+  for (const row of [rec, creator, top, fresh]) assert.ok(row.reason.length <= 280);
+});
+
+test('isAdult filter', t => {
+  const plain = fixtureFeed(t);
+  assert.equal(plain.item(207), undefined);
+  assert.equal(plain.item(208), undefined, 'an adult tag with rank >= 60 counts as mature');
+  const mature = fixtureFeed(t, { discoveryMature: true });
+  assert.ok(mature.item(207));
+  assert.ok(mature.item(208));
+});
+
+test('multiple seeds merge reasons', t => {
+  const { item } = fixtureFeed(t);
+  const both = item(209);
+  assert.equal(both.scoreParts.sameCreator, 4);
+  assert.equal(both.scoreParts.communityRec, 2.73);
+  assert.equal(both.score, 6.73);
+  assert.equal(both.group, 'creator');
+  assert.deepEqual(both.reasons, ['Also by Author Seven — you liked Anchor One.',
+    'AniList readers who liked Anchor One recommend this (10).']);
+  assert.equal(both.reason, both.reasons[0]);
+  assert.ok(both.evidenceIds.includes('rec:100:anilist:209'));
+  assert.ok(both.evidenceIds.includes('creator:7:anilist:209'));
+});
+
+test('owned, queued, voted, dismissed still excluded', t => {
+  const { item, candidates } = fixtureFeed(t);
+  for (const id of [210, 211, 212, 213]) {
+    assert.ok(candidates.some(c => c.id === `anilist:${id}`), `anilist:${id} is in the pool`);
+    assert.equal(item(id), undefined, `anilist:${id} must not be suggested`);
+  }
+});
+
+test('slot mix', () => {
+  const row = (id, score, group, extra = {}) => ({ candidateId: `anilist:${id}`, score, group, tags: ['Drama'],
+    scoreParts: { fresh: 0 }, ...extra });
+  const rows = [];
+  for (let i = 0; i < 30; i++) rows.push(row(1000 + i, 30 - i, 'recs'));
+  rows.push(row(2001, 0.5, 'fresh', { scoreParts: { fresh: 1 } }), row(2002, 0.4, 'fresh', { scoreParts: { fresh: 1 } }));
+  rows.push(row(3001, 0.2, '1990s', { tags: ['Sports'] }), row(3002, 0.1, '1990s', { tags: [] }));
+  const picked = mixSlots(rows, { topTags: [{ name: 'Drama', kind: 'genre' }] });
+  assert.equal(SLOTS.perKind, 8);
+  assert.equal(picked.filter(r => r.group === 'recs').length, 8);
+  assert.ok(picked.some(r => r.candidateId === 'anilist:2001') && picked.some(r => r.candidateId === 'anilist:2002'));
+  assert.ok(picked.some(r => r.candidateId === 'anilist:3001') && picked.some(r => r.candidateId === 'anilist:3002'));
+  assert.equal(picked.length, 12, 'never pads up to 20');
+  assert.deepEqual(picked.map(r => r.rank), picked.map((_, i) => i + 1));
+  for (let i = 1; i < picked.length; i++) assert.ok(picked[i - 1].score >= picked[i].score);
+
+  const many = [];
+  for (let i = 0; i < 40; i++) many.push(row(4000 + i, 100 - i, ['recs', 'creator', 'tagTop', 'fresh', '1980s'][i % 5],
+    { scoreParts: { fresh: i % 5 === 3 ? 1 : 0 } }));
+  const full = mixSlots(many, { topTags: [] });
+  assert.equal(full.length, 20);
+  assert.ok(full.filter(r => r.group === '1980s').length <= SLOTS.perPeriod);
+});
+
+test('slot mix reservations let low-scored fresh and outside rows past a full feed', () => {
+  const row = (id, score, group, extra = {}) => ({ candidateId: `anilist:${id}`, score, group, tags: ['Drama'],
+    scoreParts: { fresh: 0 }, ...extra });
+  // 32 higher-scored rows that the caps alone would allow: 8 per seed kind, 4 per period.
+  const rows = [];
+  const groups = ['recs', 'creator', 'tagTop', '1980s', '1990s'];
+  const caps = { recs: 8, creator: 8, tagTop: 8, '1980s': 4, '1990s': 4 };
+  let id = 5000;
+  for (const group of groups) for (let i = 0; i < caps[group]; i++) rows.push(row(id++, 100 - rows.length, group));
+  assert.ok(rows.length > 20);
+  rows.push(row(2001, 1, 'fresh', { scoreParts: { fresh: 1 } }), row(2002, 0.9, 'fresh', { scoreParts: { fresh: 1 } }));
+  rows.push(row(3001, 0.2, '2000s', { tags: ['Sports'] }), row(3002, 0.1, '2000s', { tags: [] }));
+  const picked = mixSlots(rows, { topTags: [{ name: 'Drama', kind: 'genre' }] });
+  assert.equal(picked.length, 20);
+  for (const low of ['anilist:2001', 'anilist:2002', 'anilist:3001', 'anilist:3002'])
+    assert.ok(picked.some(r => r.candidateId === low), `${low} needs its reserved slot`);
+  for (const group of groups) assert.ok(picked.filter(r => r.group === group).length <= caps[group], group);
+});
+
+test('mergeUserFeed keeps other users\' referenced candidates', () => {
+  const prior = { candidates: { a: { id: 'a' }, b: { id: 'b' }, c: { id: 'c' } },
+    feeds: { alice: { items: [{ candidateId: 'a' }] }, bob: { items: [{ candidateId: 'b' }] }, off: { items: [], disabled: true } } };
+  const result = { candidates: { d: { id: 'd' }, b: { id: 'b', fresh: true } }, feeds: { alice: { items: [{ candidateId: 'd' }] } } };
+  const merged = mergeUserFeed(prior, result, 'alice');
+  assert.deepEqual(Object.keys(merged.feeds).sort(), ['alice', 'bob', 'off']);
+  assert.deepEqual(merged.feeds.alice.items, [{ candidateId: 'd' }]);
+  assert.deepEqual(merged.feeds.bob, prior.feeds.bob);
+  assert.deepEqual(Object.keys(merged.candidates).sort(), ['b', 'd']);
+  assert.equal(merged.candidates.b.fresh, true);
+});
+
+test('generateAsync matches generate and can score one user', async t => {
+  const world = fixtureWorld(t);
+  world.db.users.bob = { name: 'bob', settings: { discoveryEnabled: true } };
+  const args = { db: world.db, library: world.library, greatest: world.greatest, cacheDir: NO_SCRAPER_CACHE,
+    harvestCache: world.harvestCache, queuedTitles: world.queuedTitles, now: NOW };
+  const sync = generate(args);
+  const all = await generateAsync(args);
+  assert.deepEqual(all.feeds.alice.items, sync.feeds.alice.items);
+  assert.deepEqual(Object.keys(all.candidates), Object.keys(sync.candidates));
+  const one = await generateAsync({ ...args, onlyUser: 'alice' });
+  assert.deepEqual(Object.keys(one.feeds), ['alice']);
+  assert.deepEqual(one.feeds.alice.items, sync.feeds.alice.items);
+});
+
+test('scoring one user stays under 200 ms on 2000 candidates', t => {
+  const world = fixtureWorld(t);
+  const tagNames = ['Drama', 'Action', 'Romance', 'Time Skip', 'Comedy', 'Horror'];
+  const candidates = [];
+  for (let i = 0; i < 2000; i++) {
+    const id = 10000 + i;
+    candidates.push({ id: `anilist:${id}`, title: `Synthetic ${id}`, year: 2000 + (i % 27), type: 'MANGA',
+      status: i % 3 ? 'completed' : 'ongoing', cover: null, metadataUrl: `https://anilist.co/manga/${id}`,
+      tags: [tagNames[i % 6], tagNames[(i + 1) % 6]], authors: [`Writer ${i % 50}`], period: null, editorialRank: null,
+      greatestId: null, evidence: [], relations: [], isAdult: i % 97 === 0, averageScore: 60 + (i % 40),
+      tagRecords: [{ name: tagNames[i % 6], rank: 80, kind: 'tag', isAdult: false }],
+      authorRecords: [{ staffId: i % 50, name: `Writer ${i % 50}`, role: 'Story' }], fetchedAt: FETCHED_AT, seed: 'recs' });
+  }
+  feedForUser({ ...world, candidates, now: NOW });
+  const started = performance.now();
+  const feed = feedForUser({ ...world, candidates, now: NOW });
+  const elapsed = performance.now() - started;
+  assert.ok(feed.items.length > 0);
+  assert.ok(elapsed < 200, `took ${elapsed.toFixed(1)} ms`);
+});

test/fixtures/anilist/creator.json

@@ -0,0 +1,10 @@
+{"data":{"Staff":{"staffMedia":{"edges":[
+  {"staffRole":"Story & Art","node":{"id":301,"format":"MANGA","isAdult":false,
+    "title":{"romaji":"Hoshi no Uta","english":"Star Song","native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2018},"averageScore":79,"popularity":2500,"trending":3,
+    "genres":["Fantasy"],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}},
+  {"staffRole":"Original Story","node":{"id":777,"format":"NOVEL","isAdult":false,
+    "title":{"romaji":"Hon no Tabi","english":null,"native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2010},"averageScore":70,"popularity":1200,"trending":1,
+    "genres":[],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}}
+]}}}}

test/fixtures/anilist/fresh.json

@@ -0,0 +1,6 @@
+{"data":{"Page":{"media":[
+  {"id":501,"format":"MANGA","isAdult":false,
+    "title":{"romaji":"Atarashii Kaze","english":"New Wind","native":null},"synonyms":[],
+    "status":"RELEASING","startDate":{"year":2025},"averageScore":72,"popularity":2100,"trending":150,
+    "genres":["Sports"],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}
+]}}}

test/fixtures/anilist/novel.json

@@ -0,0 +1,5 @@
+{"data":{"Media":{"id":777,"format":"NOVEL","isAdult":false,
+  "title":{"romaji":"Hon no Tabi","english":null,"native":null},
+  "synonyms":[],"status":"FINISHED","startDate":{"year":2010},
+  "averageScore":70,"popularity":1200,"trending":1,"genres":[],"tags":[],
+  "staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":"https://anilist.co/manga/777"}}}

test/fixtures/anilist/recs.json

@@ -0,0 +1,14 @@
+{"data":{"Media":{"recommendations":{"nodes":[
+  {"rating":12,"mediaRecommendation":{"id":201,"format":"MANGA","isAdult":false,
+    "title":{"romaji":"Sora no Machi","english":"Sky Town","native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2015},"averageScore":77,"popularity":3000,"trending":5,
+    "genres":["Drama"],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":"https://anilist.co/manga/201"}},
+  {"rating":-2,"mediaRecommendation":{"id":202,"format":"MANGA","isAdult":false,
+    "title":{"romaji":"Kuroi Mori","english":null,"native":null},"synonyms":[],
+    "status":"HIATUS","startDate":{"year":2012},"averageScore":60,"popularity":900,"trending":0,
+    "genres":[],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}},
+  {"rating":9,"mediaRecommendation":{"id":777,"format":"NOVEL","isAdult":false,
+    "title":{"romaji":"Hon no Tabi","english":null,"native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2010},"averageScore":70,"popularity":1200,"trending":1,
+    "genres":[],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}}
+]}}}}

test/fixtures/anilist/tagtop.json

@@ -0,0 +1,11 @@
+{"data":{"Page":{"media":[
+  {"id":401,"format":"ONE_SHOT","isAdult":false,
+    "title":{"romaji":"Ichi Wa","english":"One Tale","native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2020},"averageScore":74,"popularity":1800,"trending":2,
+    "genres":["Mystery"],"tags":[{"name":"Detective","rank":90,"isAdult":false}],"staff":{"edges":[]},
+    "coverImage":{"extraLarge":null},"siteUrl":null},
+  {"id":777,"format":"NOVEL","isAdult":false,
+    "title":{"romaji":"Hon no Tabi","english":null,"native":null},"synonyms":[],
+    "status":"FINISHED","startDate":{"year":2010},"averageScore":70,"popularity":1200,"trending":1,
+    "genres":[],"tags":[],"staff":{"edges":[]},"coverImage":{"extraLarge":null},"siteUrl":null}
+]}}}

test/fixtures/anilist/title.json

@@ -0,0 +1,11 @@
+{"data":{"Media":{"id":101,"format":"MANGA","isAdult":false,
+  "title":{"romaji":"Tsuki no Tabibito","english":"Moon Traveller","native":"月の旅人"},
+  "synonyms":["Lunar Wanderer"],"status":"RELEASING","startDate":{"year":2019},
+  "averageScore":81,"popularity":5400,"trending":37,
+  "genres":["Adventure","Fantasy"],
+  "tags":[{"name":"Travel","rank":88,"isAdult":false},{"name":"Nudity","rank":40,"isAdult":true}],
+  "staff":{"edges":[
+    {"role":"Story & Art","node":{"id":900,"name":{"full":"Aoi Example"}}},
+    {"role":"Translator","node":{"id":901,"name":{"full":"Not An Author"}}}]},
+  "coverImage":{"extraLarge":"https://example.invalid/cover/101.jpg"},
+  "siteUrl":"https://anilist.co/manga/101"}}}

test/greatest-cover-route.test.js

@@ -0,0 +1,75 @@
+'use strict';
+// BL-0053: Greatest covers are served from the reader's own copy, with a redirect until one exists.
+const test = require('node:test'), assert = require('node:assert/strict');
+const fs = require('fs'), os = require('os'), path = require('path'), net = require('net'), crypto = require('crypto');
+const { spawn } = require('child_process');
+const { loadCatalog } = require('../greatest');
+const { GreatestCovers } = require('../greatest/covers');
+
+const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
+const hashPw = pw => { const salt = crypto.randomBytes(16).toString('hex'); return salt + ':' + crypto.scryptSync(pw, salt, 32).toString('hex'); };
+const sha = s => crypto.createHash('sha256').update(s).digest('hex');
+const PNG = Buffer.from('89504e470d0a1a0a0000000d49484452', 'hex');
+
+async function freePort() {
+  const listener = net.createServer();
+  await new Promise(resolve => listener.listen(0, '127.0.0.1', resolve));
+  const port = listener.address().port;
+  await new Promise(resolve => listener.close(resolve));
+  return port;
+}
+
+test('Greatest covers come from the local copy, else redirect to the catalog URL', async t => {
+  const entries = loadCatalog().entries.filter(entry => entry.cover);
+  const [cached, missing] = entries;
+  const root = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-greatest-cover-'));
+  const library = path.join(root, 'library'), data = path.join(root, 'data');
+  fs.mkdirSync(library, { recursive: true });
+  fs.mkdirSync(path.join(data, 'greatest-covers'), { recursive: true });
+  const version = new GreatestCovers(path.join(data, 'greatest-covers')).version(cached.cover);
+  fs.writeFileSync(path.join(data, 'greatest-covers', `${version}.png`), PNG);
+  const token = crypto.randomBytes(32).toString('base64url');
+  fs.writeFileSync(path.join(data, 'db.json'), JSON.stringify({
+    users: { reader: { name: 'reader', hash: hashPw('reader-password'), admin: false, created: Date.now(), settings: {} } },
+    sessions: { [sha(token)]: { u: 'reader', t: Date.now() } } }));
+
+  const port = await freePort(), output = [];
+  const child = spawn(process.execPath, ['server.js'], { cwd: path.resolve(__dirname, '..'), windowsHide: true,
+    env: { ...process.env, LIBRARY: library, DATA: data, PORT: String(port), SCAN_INTERVAL: '0', FULL_SCAN_INTERVAL: '0',
+      SCRAPER_URL: 'http://127.0.0.1:9', SCRAPER_TOKEN: 'fixture-token', SCRAPER_TOKEN_FILE: '', BACKUP: '',
+      GREATEST_COVER_WARM: '0' } });
+  child.stdout.on('data', d => output.push(String(d)));
+  child.stderr.on('data', d => output.push(String(d)));
+  t.after(async () => {
+    await new Promise(resolve => { if (child.exitCode !== null) return resolve(); child.once('exit', resolve); child.kill(); });
+    fs.rmSync(root, { recursive: true, force: true });
+  });
+  const base = `http://127.0.0.1:${port}`;
+  let ready = false;
+  for (let n = 0; n < 100 && !ready; n++) {
+    ready = await fetch(base + '/health').then(r => r.ok).catch(() => false);
+    if (!ready) await sleep(50);
+  }
+  assert.ok(ready, output.join(''));
+  const get = (url, cookie = `sid=${token}`) => fetch(base + '/' + url, { headers: cookie ? { cookie } : {}, redirect: 'manual' });
+
+  const listing = await (await get('api/greatest')).json();
+  const coverOf = id => listing.entries.find(entry => entry.id === id).cover;
+  assert.equal(coverOf(cached.id), `api/greatest-cover?id=${encodeURIComponent(cached.id)}&v=${version}`);
+  assert.match(coverOf(missing.id), /^api\/greatest-cover\?id=.+&v=[0-9a-f]{16}$/);
+
+  const hit = await get(coverOf(cached.id));
+  assert.equal(hit.status, 200);
+  assert.equal(hit.headers.get('content-type'), 'image/png');
+  assert.equal(hit.headers.get('cache-control'), 'private, max-age=31536000, immutable');
+  assert.deepEqual(Buffer.from(await hit.arrayBuffer()), PNG);
+
+  const miss = await get(coverOf(missing.id));
+  assert.equal(miss.status, 302);
+  assert.equal(miss.headers.get('location'), missing.cover);
+  assert.equal(miss.headers.get('cache-control'), 'no-store');
+
+  assert.equal((await get(coverOf(cached.id), null)).status, 401);
+  assert.equal((await get('api/greatest-cover?id=not-in-the-catalog')).status, 404);
+  assert.equal((await get('api/greatest-cover?id=../../db.json')).status, 404);
+});

test/greatest-covers.test.js

@@ -0,0 +1,84 @@
+'use strict';
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const { GreatestCovers } = require('../greatest/covers');
+
+const PNG = Buffer.from('89504e470d0a1a0a0000000d49484452', 'hex');
+const image = (body = PNG, type = 'image/png') => new Response(body, { status: 200, headers: { 'content-type': type } });
+
+function setup(t, responses, options = {}) {
+  const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-greatest-covers-'));
+  t.after(() => fs.rmSync(dir, { recursive: true, force: true }));
+  const calls = [];
+  const fetchImpl = async (url, init) => {
+    calls.push({ url, init });
+    const next = responses[url];
+    if (next instanceof Error) throw next;
+    return typeof next === 'function' ? next() : next || new Response('missing', { status: 404 });
+  };
+  const sleeps = [];
+  const covers = new GreatestCovers(dir, { fetchImpl, sleep: async ms => { sleeps.push(ms); }, ...options });
+  return { dir, calls, sleeps, covers };
+}
+
+test('warm downloads each cover once and serves it from disk afterwards', async t => {
+  const a = 'https://img.example/a.png', b = 'https://img.example/b.jpg';
+  const { dir, calls, sleeps, covers } = setup(t, { [a]: () => image(), [b]: () => image(PNG, 'image/jpeg') });
+  const result = await covers.warm([a, b, a]);
+  assert.deepEqual(result, { fetched: 2, skipped: 1, failed: 0, stopped: null });
+  assert.equal(calls.length, 2);
+  assert.deepEqual(sleeps, [1000], 'one pause between two downloads, none before the first');
+  assert.ok(covers.file(a).endsWith('.png') && fs.existsSync(covers.file(a)));
+  assert.ok(covers.file(b).endsWith('.jpg'));
+  assert.equal(covers.file('https://img.example/other.png'), null);
+
+  // A restarted reader finds the stored files and asks nobody.
+  const reopened = new GreatestCovers(dir, { fetchImpl: async () => { throw new Error('must not fetch'); } });
+  assert.deepEqual(await reopened.warm([a, b]), { fetched: 0, skipped: 2, failed: 0, stopped: null });
+  assert.equal(reopened.file(a), covers.file(a));
+});
+
+test('a changed cover URL is a new file', async t => {
+  const { covers } = setup(t, { 'https://img.example/v1.png': () => image(), 'https://img.example/v2.png': () => image() });
+  await covers.warm(['https://img.example/v1.png', 'https://img.example/v2.png']);
+  assert.notEqual(covers.file('https://img.example/v1.png'), covers.file('https://img.example/v2.png'));
+  assert.notEqual(covers.version('https://img.example/v1.png'), covers.version('https://img.example/v2.png'));
+});
+
+test('non-images, oversized files, plain http and errors are skipped', async t => {
+  const html = 'https://img.example/page', big = 'https://img.example/big.png', declaredBig = 'https://img.example/declared.png';
+  const broken = 'https://img.example/broken.png', gone = 'https://img.example/gone.png';
+  const { covers, calls } = setup(t, {
+    [html]: () => new Response('<html>', { status: 200, headers: { 'content-type': 'text/html' } }),
+    [big]: () => image(Buffer.alloc(64)),
+    [declaredBig]: () => new Response(PNG, { status: 200, headers: { 'content-type': 'image/png', 'content-length': '999999' } }),
+    [broken]: new TypeError('fetch failed'),
+  }, { maxBytes: 32 });
+  const result = await covers.warm([html, big, declaredBig, broken, gone, 'http://img.example/plain.png', '', null]);
+  assert.deepEqual(result, { fetched: 0, skipped: 0, failed: 5, stopped: null });
+  assert.equal(calls.length, 5, 'plain http and empty URLs are never requested');
+  for (const url of [html, big, declaredBig, broken, gone]) assert.equal(covers.file(url), null);
+  assert.deepEqual(fs.readdirSync(covers.dir).filter(name => !name.startsWith('.')), []);
+});
+
+test('429 and 403 stop the run at once', async t => {
+  for (const status of [429, 403]) {
+    const first = 'https://img.example/1.png', second = 'https://img.example/2.png';
+    const { covers, calls } = setup(t, { [first]: () => new Response('', { status }), [second]: () => image() });
+    const result = await covers.warm([first, second]);
+    assert.equal(result.stopped, `HTTP ${status}`);
+    assert.equal(calls.length, 1);
+    assert.equal(covers.file(second), null);
+  }
+});
+
+test('requests carry a timeout and a fixed user agent', async t => {
+  const a = 'https://img.example/a.png';
+  const { covers, calls } = setup(t, { [a]: () => image() });
+  await covers.warm([a]);
+  assert.ok(calls[0].init.signal instanceof AbortSignal);
+  assert.match(calls[0].init.headers['user-agent'], /^MangaCircus/);
+});

test/library-index.test.js

@@ -78,7 +78,7 @@ test('reader serves a saved index immediately after restart and verifies it in b
     const output = [];
     child = spawn(process.execPath, ['server.js'], { cwd: path.resolve(__dirname, '..'), windowsHide: true,
       env: { ...process.env, LIBRARY: library, DATA: data, PORT: String(port), SCAN_INTERVAL: '0', FULL_SCAN_INTERVAL: '0',
-        SCRAPER_URL: 'http://127.0.0.1:9', SCRAPER_TOKEN: 'fixture-token', SCRAPER_TOKEN_FILE: '' } });
+        SCRAPER_URL: 'http://127.0.0.1:9', SCRAPER_TOKEN: 'fixture-token', SCRAPER_TOKEN_FILE: '', GREATEST_COVER_WARM: '0' } });
     child.stdout.on('data', data => output.push(String(data)));
     child.stderr.on('data', data => output.push(String(data)));
     const base = `http://127.0.0.1:${port}`;

test/library-scan.test.js

@@ -0,0 +1,106 @@
+'use strict';
+// First-scan speed (BL-0043): small first reads and bounded concurrency must not change any result.
+const test = require('node:test'), assert = require('node:assert/strict');
+const fs = require('fs'), os = require('os'), path = require('path');
+const { Canvas } = require('../tools/png');
+const { scanLibrary, snapshotIndex, sizeFromBuffer, imageSize, limiter } = require('../library');
+
+const tmp = prefix => fs.mkdtempSync(path.join(os.tmpdir(), prefix));
+const u16be = n => Buffer.from([n >> 8, n & 255]);
+
+// JPEG with `appBytes` of APP1 payload (in 64 KB segments) before the SOF0 marker.
+function jpeg(w, h, appBytes) {
+  const parts = [Buffer.from([0xff, 0xd8])];
+  for (let left = appBytes; left > 0;) {
+    const n = Math.min(left, 65533);
+    parts.push(Buffer.from([0xff, 0xe1]), u16be(n + 2), Buffer.alloc(n, 0x41));
+    left -= n;
+  }
+  parts.push(Buffer.from([0xff, 0xc0, 0x00, 0x11, 0x08]), u16be(h), u16be(w), Buffer.alloc(12));
+  parts.push(Buffer.from([0xff, 0xda]), Buffer.alloc(4096, 0x55), Buffer.from([0xff, 0xd9]));
+  return Buffer.concat(parts);
+}
+function webpVp8x(w, h) {
+  const b = Buffer.alloc(64);
+  b.write('RIFF', 0, 'latin1'); b.writeUInt32LE(56, 4); b.write('WEBP', 8, 'latin1'); b.write('VP8X', 12, 'latin1');
+  b.writeUIntLE(w - 1, 24, 3); b.writeUIntLE(h - 1, 27, 3);
+  return b;
+}
+function gif(w, h) {
+  const b = Buffer.alloc(32);
+  b.write('GIF89a', 0, 'latin1'); b.writeUInt16LE(w, 6); b.writeUInt16LE(h, 8);
+  return b;
+}
+// AVIF-like: ftyp box, then an `ispe` box after `pad` bytes.
+function avif(w, h, pad) {
+  const b = Buffer.alloc(32 + pad + 20);
+  b.writeUInt32BE(24, 0); b.write('ftypavif', 4, 'latin1');
+  const k = 32 + pad;
+  b.write('ispe', k, 'latin1'); b.writeUInt32BE(w, k + 8); b.writeUInt32BE(h, k + 12);
+  return b;
+}
+
+test('the 32 KB first read gives the same size as the whole file, in every read stage', async () => {
+  const dir = tmp('mc-probe-');
+  try {
+    const cases = {
+      'small.png': new Canvas(37, 53, [1, 2, 3]).png(),
+      'plain.jpg': jpeg(800, 1200, 0),
+      'exif-20k.jpg': jpeg(801, 1201, 20 * 1024),      // found in the first 32 KB
+      'exif-100k.jpg': jpeg(802, 1202, 100 * 1024),    // needs the 256 KB read
+      'exif-400k.jpg': jpeg(803, 1203, 400 * 1024),    // needs the whole file
+      'tiny.webp': webpVp8x(640, 960),
+      'tiny.gif': gif(10, 20),
+      'deep.avif': avif(1024, 1536, 50 * 1024),        // ispe past 32 KB
+      'broken.jpg': Buffer.concat([Buffer.from([0xff, 0xd8]), Buffer.alloc(40 * 1024, 0x41)]),
+    };
+    for (const [name, bytes] of Object.entries(cases)) {
+      fs.writeFileSync(path.join(dir, name), bytes);
+      assert.deepEqual(await imageSize(path.join(dir, name)), sizeFromBuffer(bytes), name);
+    }
+    assert.deepEqual(await imageSize(path.join(dir, 'exif-400k.jpg')), { w: 803, h: 1203 });
+    assert.deepEqual(await imageSize(path.join(dir, 'deep.avif')), { w: 1024, h: 1536 });
+    assert.equal(await imageSize(path.join(dir, 'broken.jpg')), null);
+  } finally { fs.rmSync(dir, { recursive: true, force: true }); }
+});
+
+test('concurrent scanning builds the same index and cache as a sequential scan', async () => {
+  const root = tmp('mc-concurrency-');
+  try {
+    let n = 0;
+    for (const manga of ['Alpha', 'Beta']) for (const book of ['v01', 'v02']) for (let c = 1; c <= 6; c++) {
+      const dir = path.join(root, manga, book, `c${String(c).padStart(4, '0')} [Src]`);
+      fs.mkdirSync(dir, { recursive: true });
+      for (let p = 1; p <= 5; p++, n++) {
+        const file = path.join(dir, `${String(p).padStart(4, '0')}${p === 3 ? '_double' : ''}`);
+        if (n % 3 === 0) fs.writeFileSync(file + '.jpg', jpeg(700 + n, 1000 + n, (n % 5) * 30 * 1024));
+        else fs.writeFileSync(file + '.png', new Canvas(10 + n, 20 + n, [n, n, n]).png());
+      }
+      fs.writeFileSync(path.join(dir, 'chapter.json'), JSON.stringify({ title: `${manga} ${book} ${c}` }));
+    }
+    fs.mkdirSync(path.join(root, 'Alpha', 'v01', 'not a chapter'));
+    const one = await scanLibrary(root, {}, { concurrency: 1 });
+    const many = await scanLibrary(root, {}, { concurrency: 8 });
+    assert.deepEqual(snapshotIndex(many.index), snapshotIndex(one.index));
+    const sorted = c => Object.fromEntries(Object.entries(c).sort(([a], [b]) => a.localeCompare(b)));
+    assert.deepEqual(sorted(many.cache), sorted(one.cache));
+    assert.equal(Object.keys(one.cache).length, 24);
+    assert.equal(one.index.byId.get('Beta').chapters.length, 12);
+  } finally { fs.rmSync(root, { recursive: true, force: true }); }
+});
+
+test('limiter runs at most n tasks at once and keeps results and errors per task', async () => {
+  const limit = limiter(3);
+  let active = 0, peak = 0;
+  const task = i => limit(async () => {
+    active++; peak = Math.max(peak, active);
+    await new Promise(r => setTimeout(r, 5 + (i % 4) * 3));
+    active--;
+    if (i === 7) throw new Error('boom');
+    return i * 2;
+  });
+  const results = await Promise.allSettled(Array.from({ length: 20 }, (_, i) => task(i)));
+  assert.equal(peak, 3);
+  assert.equal(results[7].status, 'rejected');
+  assert.deepEqual(results.filter((_, i) => i !== 7).map(r => r.value), Array.from({ length: 20 }, (_, i) => i * 2).filter((_, i) => i !== 7));
+});

test/services.test.js

@@ -24,23 +24,22 @@ test('three services integrate collections, auth, custom covers and private heal
       fs.writeFileSync(path.join(library, name, 'cover.png'), png);
       fs.writeFileSync(path.join(library, name, 'manga.json'), JSON.stringify({ title: name, tags: ['Fantasy'], status: 'ongoing' }));
     }
-    const [rp, sp, wp] = await Promise.all([port(), port(), port()]);
-    const base = `http://127.0.0.1:${rp}`, scraper = `http://127.0.0.1:${sp}`, sentinel = `http://127.0.0.1:${wp}`;
+    const [rp, sp] = await Promise.all([port(), port()]);
+    const base = `http://127.0.0.1:${rp}`, scraper = `http://127.0.0.1:${sp}`;
     const token = 'integration-test-only';
-    const env = { ...process.env, DATA: data, LIBRARY: library, SCRAPER_DATA: path.join(data, 'scraper'), SENTINEL_DATA: path.join(data, 'sentinel'),
-      PORT: String(rp), SCRAPER_PORT: String(sp), SENTINEL_PORT: String(wp), SCRAPER_HOST: '127.0.0.1', SENTINEL_HOST: '127.0.0.1',
-      READER_URL: base, SCRAPER_URL: scraper, SENTINEL_URL: sentinel, SCRAPER_TOKEN: token, SCRAPER_TOKEN_FILE: '', SCAN_INTERVAL: '0', COOKIE_SECURE: '0' };
-    for (const script of ['server.js', 'scraper/daemon.js', 'sentinel/daemon.js']) {
+    const env = { ...process.env, DATA: data, LIBRARY: library, SCRAPER_DATA: path.join(data, 'scraper'),
+      PORT: String(rp), SCRAPER_PORT: String(sp), SCRAPER_HOST: '127.0.0.1',
+      READER_URL: base, SCRAPER_URL: scraper, SCRAPER_TOKEN: token, SCRAPER_TOKEN_FILE: '', SCAN_INTERVAL: '0', COOKIE_SECURE: '0', GREATEST_COVER_WARM: '0' };
+    for (const script of ['server.js', 'scraper/daemon.js']) {
       const child = spawn(process.execPath, [script], { cwd: root, env, windowsHide: true }); children.push(child);
       for (const stream of [child.stdout, child.stderr]) stream.on('data', chunk => output.push(chunk.toString()));
     }
-    for (const url of [base, scraper, sentinel]) {
+    for (const url of [base, scraper]) {
       let ready = false;
       for (let n = 0; n < 80 && !ready; n++) { ready = await fetch(url + '/health').then(r => r.ok).catch(() => false); if (!ready) await delay(100); }
       assert.ok(ready, output.join(''));
     }
     assert.equal((await fetch(scraper + '/api/state')).status, 401);
-    assert.equal((await fetch(sentinel + '/api/state')).status, 401);
     const signup = await fetch(base + '/api/signup', { method: 'POST', headers: { 'content-type': 'application/json' }, body: JSON.stringify({ name: 'tester', password: 'local-test-password' }) });
     assert.equal(signup.status, 200); const cookie = signup.headers.get('set-cookie').split(';')[0];
     const request = (endpoint, body) => fetch(base + endpoint, { headers: { cookie, 'content-type': 'application/json' }, ...(body ? { method: 'POST', body: JSON.stringify(body) } : {}) });
@@ -121,7 +120,7 @@ test('three services integrate collections, auth, custom covers and private heal
       const cover = await (await request('/api/admin/collection-cover', { id: 'demo', revision, base64: png.toString('base64') })).json(); revision = cover.revision;
       const image = await request('/api/collection-cover?id=demo'); assert.equal(image.status, 200); assert.deepEqual(Buffer.from(await image.arrayBuffer()), png);
     }
-    const health = await (await request('/api/admin/health')).json(); assert.ok(health.reader); assert.ok(health.sentinel);
+    const health = await (await request('/api/admin/health')).json(); assert.ok(health.reader); assert.equal(health.sentinel, undefined);
     assert.equal((await request('/api/admin/scraper')).status, 200);
     // Exercise the actual authenticated visit route and cross-chapter lease races.
     assert.equal((await request('/api/settings', { readingClock: true })).status, 200);

test/ui-fixture.js

@@ -11,9 +11,9 @@ for (const [index, name] of ['Moon Garden', 'Moon Garden Before the Rain', 'Iron
   fs.writeFileSync(path.join(folder, '001.png'), png); fs.writeFileSync(path.join(library, name, 'cover.png'), png);
   fs.writeFileSync(path.join(library, name, 'manga.json'), JSON.stringify({ title: name, status: index ? 'completed' : 'ongoing', tags: ['Fantasy', 'Demo'] }));
 }
-const env = { ...process.env, DATA: data, LIBRARY: library, PORT: '8180', SCRAPER_PORT: '8191', SENTINEL_PORT: '8192',
-  READER_URL: 'http://127.0.0.1:8180', SCRAPER_URL: 'http://127.0.0.1:8191', SENTINEL_URL: 'http://127.0.0.1:8192',
-  SCRAPER_DATA: path.join(data, 'scraper'), SENTINEL_DATA: path.join(data, 'sentinel'), COOKIE_SECURE: '0', SCAN_INTERVAL: '0', SCRAPER_TOKEN: 'fixture', SCRAPER_TOKEN_FILE: '' };
+const env = { ...process.env, DATA: data, LIBRARY: library, PORT: '8180', SCRAPER_PORT: '8191',
+  READER_URL: 'http://127.0.0.1:8180', SCRAPER_URL: 'http://127.0.0.1:8191',
+  SCRAPER_DATA: path.join(data, 'scraper'), COOKIE_SECURE: '0', SCAN_INTERVAL: '0', SCRAPER_TOKEN: 'fixture', SCRAPER_TOKEN_FILE: '' };
 const child = spawn(process.execPath, ['tools/start-all.js'], { cwd: path.resolve(__dirname, '..'), env, stdio: 'inherit', windowsHide: true });
 console.log('Isolated UI fixture at http://127.0.0.1:8180 · first account is test admin');
 for (const signal of ['SIGINT', 'SIGTERM']) process.on(signal, () => child.kill('SIGTERM'));

tools/backfill-chapter-dates.js

@@ -0,0 +1,83 @@
+#!/usr/bin/env node
+'use strict';
+// One-time migration to chapter dates (BL-0047). Dry run by default; apply only with the
+// scraper stopped (--apply --writers-stopped), then run a deep "Rescan" in Settings.
+//  - WeebCentral: x-dates.source from the series' full chapter list (one request per series).
+//  - MangaDex: chapters published before x-dates stored the group's publishAt in "added";
+//    it moves to x-dates.source and "added" becomes the folder's arrival time.
+//  - Every chapter: x-dates.ecosystem = "added" unless an earlier one is already known.
+const fs = require('node:fs').promises;
+const path = require('node:path');
+const { batchFlags, sourceDate, publishedDates } = require('../chapter-dates');
+const weebcentral = require('../scraper/plugins/weebcentral');
+const { writeJsonAtomic } = require('../scraper/lib/publisher');
+
+const args = process.argv.slice(2);
+const apply = args.includes('--apply');
+const only = args.find(arg => arg.startsWith('--manga='))?.slice('--manga='.length);
+const offline = args.includes('--offline');
+const root = args.find(arg => !arg.startsWith('--'));
+if (!root || args.some(arg => arg.startsWith('--') && !['--apply', '--writers-stopped', '--offline'].includes(arg) && !arg.startsWith('--manga='))
+  || (apply && !args.includes('--writers-stopped'))) {
+  console.error('Usage: node tools/backfill-chapter-dates.js <library> [--manga=<folder>] [--offline] [--apply --writers-stopped]');
+  process.exit(2);
+}
+const UA = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130 Safari/537.36';
+const sleep = ms => new Promise(resolve => setTimeout(resolve, ms));
+const readJson = async file => { try { return JSON.parse(await fs.readFile(file, 'utf8')); } catch { return null; } };
+const dirs = async dir => (await fs.readdir(dir, { withFileTypes: true })).filter(e => e.isDirectory() && !e.name.startsWith('.')).map(e => e.name).sort();
+
+async function weebCentralUploads(seriesId) {
+  const url = `https://weebcentral.com/series/${seriesId}/full-chapter-list`;
+  const response = await fetch(url, { headers: { 'user-agent': UA, accept: 'text/html' }, signal: AbortSignal.timeout(30000) });
+  if (!response.ok) throw new Error(`WeebCentral HTTP ${response.status}`);
+  const { chapters } = weebcentral.catalog({ html: await response.text(), url });
+  return new Map(chapters.filter(c => c.uploadedAt).map(c => [c.id, sourceDate(c.uploadedAt, 'WeebCentral', c.uploadBatch)]));
+}
+
+async function main() {
+  const totals = { chapters: 0, changed: 0, alreadyDated: 0, weebSource: 0, weebBatch: 0, mangaDexMoved: 0, missing: 0 };
+  for (const manga of await dirs(root)) {
+    if (only && manga !== only) continue;
+    const mangaDir = path.join(root, manga), meta = (await readJson(path.join(mangaDir, 'manga.json'))) || {};
+    const versions = [];
+    for (const book of await dirs(mangaDir)) {
+      for (const chapter of await dirs(path.join(mangaDir, book))) {
+        const dir = path.join(mangaDir, book, chapter), file = path.join(dir, 'chapter.json'), chapterMeta = await readJson(file);
+        if (chapterMeta) versions.push({ dir, file, meta: chapterMeta, mtime: (await fs.stat(dir)).mtimeMs });
+      }
+    }
+    let uploads = new Map();
+    const seriesId = meta['x-scraper']?.ids?.weebcentral;
+    if (seriesId && !offline && versions.some(v => /weebcentral\.com\/chapters\//.test(v.meta.source?.url || '') && !v.meta['x-dates'])) {
+      try { uploads = await weebCentralUploads(seriesId); await sleep(2000); }
+      catch (error) { console.warn(`[backfill] ${manga}: WeebCentral list failed: ${error.message}`); }
+    }
+    // Legacy MangaDex "added" values are group upload times; flag bulk uploads among them.
+    const legacyDex = versions.filter(v => !v.meta['x-dates'] && /^MangaDex\b/.test(v.meta.source?.name || '') && Date.parse(v.meta.added));
+    const dexBatch = new Map(batchFlags(legacyDex.map(v => v.meta.added)).map((flag, i) => [legacyDex[i], flag]));
+    let changed = 0;
+    for (const version of versions) {
+      totals.chapters++;
+      if (version.meta['x-dates']) { totals.alreadyDated++; continue; }
+      const prior = { ...version.meta }, incoming = {};
+      const weebId = /weebcentral\.com\/chapters\/([0-9A-Z]{26})/.exec(prior.source?.url || '')?.[1];
+      if (weebId && uploads.has(weebId)) {
+        incoming.source = uploads.get(weebId);
+        totals[incoming.source.batch ? 'weebBatch' : 'weebSource']++;
+      } else if (dexBatch.has(version)) {
+        incoming.source = sourceDate(prior.added, prior.source.name, dexBatch.get(version));
+        prior.added = new Date(version.mtime).toISOString();
+        totals.mangaDexMoved++;
+      } else totals.missing++;
+      const next = { ...prior, ...publishedDates(prior, incoming, version.mtime) };
+      changed++; totals.changed++;
+      if (apply) await writeJsonAtomic(version.file, next);
+    }
+    if (changed) console.log(`[backfill] ${manga}: ${changed} of ${versions.length} chapter versions ${apply ? 'updated' : 'would change'}`);
+  }
+  console.log(JSON.stringify({ apply, ...totals }));
+  if (apply) console.log('Run a deep "Rescan" in Settings so the reader picks up the edited chapter.json files.');
+}
+
+main().catch(error => { console.error(error); process.exit(1); });

tools/check.js

@@ -4,9 +4,8 @@ const { secret } = require('../runtime/config');
 const token = secret('SCRAPER_TOKEN');
 const root = process.env.READER_URL || 'http://127.0.0.1:8080';
 const scraper = process.env.SCRAPER_URL || 'http://127.0.0.1:8091';
-const sentinel = process.env.SENTINEL_URL || 'http://127.0.0.1:8092';
 (async () => {
-  for (const [name, base, endpoint] of [['reader', root, '/health'], ['scraper', scraper, '/api/state'], ['sentinel', sentinel, '/api/state']]) {
+  for (const [name, base, endpoint] of [['reader', root, '/health'], ['scraper', scraper, '/api/state']]) {
     try {
       const response = await fetch(base + endpoint, { headers: token ? { authorization: `Bearer ${token}` } : {}, signal: AbortSignal.timeout(10000) });
       if (!response.ok) throw new Error(`HTTP ${response.status}`);

tools/healthcheck.js

@@ -1,5 +1,5 @@
 'use strict';
-const ports = { reader: process.env.PORT || 8080, scraper: process.env.SCRAPER_PORT || 8091, sentinel: process.env.SENTINEL_PORT || 8092 };
+const ports = { reader: process.env.PORT || 8080, scraper: process.env.SCRAPER_PORT || 8091 };
 const service = process.argv[2] || 'reader';
 fetch(`http://127.0.0.1:${ports[service]}/health`, { signal: AbortSignal.timeout(4000) })
   .then(async r => { const body = await r.json(); if (!r.ok || !body.ok) process.exitCode = 1; }).catch(() => { process.exitCode = 1; });

tools/prepare-compose.js

@@ -2,7 +2,7 @@
 'use strict';
 const fs = require('fs'), path = require('path'), crypto = require('crypto');
 const root = path.resolve(__dirname, '..');
-for (const relative of ['data', 'data/scraper', 'data/sentinel', 'library', '.secrets']) fs.mkdirSync(path.join(root, relative), { recursive: true });
+for (const relative of ['data', 'data/scraper','library', 'backup', '.secrets']) fs.mkdirSync(path.join(root, relative), { recursive: true });
 const file = path.join(root, '.secrets', 'scraper_token');
 if (!fs.existsSync(file)) fs.writeFileSync(file, crypto.randomBytes(32).toString('hex') + '\n', { flag: 'wx', mode: 0o600 });
 if (!fs.existsSync(path.join(root, '.env'))) fs.copyFileSync(path.join(root, '.env.example'), path.join(root, '.env'));

tools/release-dates.js

@@ -0,0 +1,42 @@
+#!/usr/bin/env node
+'use strict';
+// Official chapter release dates from MangaUpdates (BL-0048) for a whole library or one manga.
+// Dry run by default. Apply only with the scraper stopped (--apply --writers-stopped), then let a
+// full reconciliation (or admin Rescan) pick up the edited chapter.json files.
+const fs = require('node:fs').promises;
+const path = require('node:path');
+const { ReleaseDateSources, refreshMangaReleaseDates } = require('../scraper/metadata/release-dates');
+
+const args = process.argv.slice(2);
+const apply = args.includes('--apply');
+const only = args.find(arg => arg.startsWith('--manga='))?.slice('--manga='.length);
+const cacheDir = args.find(arg => arg.startsWith('--cache='))?.slice('--cache='.length) || path.join(require('node:os').tmpdir(), 'mangacircus-release-dates');
+const root = args.find(arg => !arg.startsWith('--'));
+if (!root || args.some(arg => arg.startsWith('--') && !['--apply', '--writers-stopped'].includes(arg) && !/^--(manga|cache)=/.test(arg))
+  || (apply && !args.includes('--writers-stopped'))) {
+  console.error('Usage: node tools/release-dates.js <library> [--manga=<folder>] [--cache=<dir>] [--apply --writers-stopped]');
+  process.exit(2);
+}
+
+async function main() {
+  const sources = new ReleaseDateSources({ cacheDir });
+  const totals = { manga: 0, matched: 0, chapters: 0, accepted: 0, written: 0, cleared: 0, outsideRun: 0, earlier: 0, outOfOrder: 0, noRun: 0 };
+  const names = (await fs.readdir(root, { withFileTypes: true })).filter(e => e.isDirectory() && !e.name.startsWith('.')).map(e => e.name).sort();
+  for (const name of names) {
+    if (only && name !== only) continue;
+    try {
+      const s = await refreshMangaReleaseDates(path.join(root, name), sources, { apply });
+      totals.manga++; totals.chapters += s.chapters;
+      if (s.ambiguous) { console.log(`${name.slice(0, 44).padEnd(44)} skipped: chapter numbers repeat across books`); continue; }
+      if (!s.series) { console.log(`${name.slice(0, 44).padEnd(44)} no MangaUpdates match`); continue; }
+      totals.matched++; totals.accepted += s.accepted; totals.written += s.written; totals.cleared += s.cleared;
+      for (const k of ['outsideRun', 'earlier', 'outOfOrder', 'noRun']) totals[k] += s.rejected[k];
+      console.log(`${name.slice(0, 44).padEnd(44)} ${String(s.chapters).padStart(5)} ch  official ${String(s.official).padStart(4)}  accepted ${String(s.accepted).padStart(4)}`
+        + `  rejected: outside run ${s.rejected.outsideRun}, earlier evidence ${s.rejected.earlier}, out of order ${s.rejected.outOfOrder}`
+        + `${s.rejected.noRun ? `, no AniList run ${s.rejected.noRun}` : ''}  (${s.series.title})`);
+    } catch (error) { console.warn(`${name}: ${error.message}`); }
+  }
+  console.log(JSON.stringify({ apply, ...totals }));
+}
+
+main().catch(error => { console.error(error); process.exit(1); });

tools/snapshot-stopped.js

@@ -21,7 +21,6 @@ function copy(source, relative) {
 }
 for (const name of ['db.json', 'scan-cache.json', 'catalog.json', 'collection-covers', 'runtime-reader.json']) copy(path.join(data, name), path.join('data', name));
 for (const name of ['queue.json', 'runtime.json', 'daemon.lock', 'supervisor.lock']) copy(path.join(data, 'scraper', name), path.join('data', 'scraper', name));
-for (const name of ['state.json', 'runtime.json']) copy(path.join(data, 'sentinel', name), path.join('data', 'sentinel', name));
 for (const name of fs.readdirSync(library)) {
   copy(path.join(library, name, name.startsWith('.mcscraper-work-') ? 'job.json' : 'manga.json'), path.join('library', name, name.startsWith('.mcscraper-work-') ? 'job.json' : 'manga.json'));
 }

tools/start-all.js

@@ -12,7 +12,7 @@ if (fs.existsSync(lock)) {
   catch (e) { if (e.code !== 'ESRCH') throw e; fs.unlinkSync(lock); }
 }
 fs.writeFileSync(lock, String(process.pid), { flag: 'wx' });
-const supervisors = ['reader', 'scraper', 'sentinel'].map(name => new Supervisor(name, { root }));
+const supervisors = ['reader', 'scraper'].map(name => new Supervisor(name, { root }));
 supervisors.forEach(service => service.start());
 const keepalive = setInterval(() => {}, 60000);
 let closing = false;

tools/test-compose.js

@@ -6,10 +6,10 @@ const assert = require('assert/strict'), { spawn } = require('child_process');
 const { Canvas } = require('./png');
 const root = path.resolve(__dirname, '..'), dir = fs.mkdtempSync(path.join(os.tmpdir(), 'mc-compose-'));
 const project = 'mc-qa-' + crypto.randomBytes(3).toString('hex'), port = process.env.QA_PORT || '8180';
-const library = path.join(dir, 'library'), data = path.join(dir, 'data'), tokenFile = path.join(dir, 'token');
+const library = path.join(dir, 'library'), data = path.join(dir, 'data'), backupDir = path.join(dir, 'backup'), tokenFile = path.join(dir, 'token');
 const directionQa = process.argv.includes('--direction-ui');
 const peerGatewayQa = process.argv.includes('--peer-gateway');
-fs.mkdirSync(path.join(data, 'scraper'), { recursive: true }); fs.mkdirSync(path.join(data, 'sentinel'));
+fs.mkdirSync(path.join(data, 'scraper'), { recursive: true }); fs.mkdirSync(backupDir);
 if (peerGatewayQa) {
   const peerData = path.join(data, 'peering');
   const peerUsage = path.join(data, 'peering-usage');
@@ -18,6 +18,23 @@ if (peerGatewayQa) {
   if (process.platform === 'win32' && !process.env.OPENSSL_BIN) process.env.OPENSSL_BIN = 'C:\\Program Files\\Git\\usr\\bin\\openssl.exe';
   new PeeringState(peerData).generateIdentity('ComposeQA');
 }
+const discoveryQa = process.argv.includes('--discovery-ui');
+// Review focus: an untrusted AniList title in the QA admin's first feed must render as text, never markup.
+const XSS_TITLE = '<img src=x onerror=alert(1)>';
+if (discoveryQa) {
+  const fetchedAt = '2026-10-20T10:00:00.000Z';
+  const evidence = [{ id: 'recommendation:990001', kind: 'recommendation', label: 'AniList recommendation',
+    note: 'Fixture recommendation', url: 'https://anilist.co/manga/990001', fetchedAt }];
+  const reasons = ['AniList readers who liked Moon Garden recommend this (42).', 'New: started 2025, ongoing and trending.'];
+  fs.writeFileSync(path.join(data, 'discovery.json'), JSON.stringify({ schema: 1, revision: 1,
+    candidates: { 'anilist:990001': { id: 'anilist:990001', title: XSS_TITLE, year: 2025, status: 'RELEASING', cover: null,
+      seed: 'recs', fetchedAt, evidence, metadataUrl: 'https://anilist.co/manga/990001' } },
+    feeds: { qa: { generatedAt: fetchedAt, items: [{ candidateId: 'anilist:990001', reason: reasons[0], reasons, evidence,
+      evidenceIds: evidence.map(entry => entry.id), scoreParts: { communityRec: 3, fresh: 2 }, score: 5, period: 'Related works', group: 'recs' }] } },
+    lastRun: { status: 'success', startedAt: fetchedAt, finishedAt: fetchedAt, candidateCount: 1, examined: 1, clipped: 0 },
+    // Nightly off, so only the test's own clicks start harvests.
+    harvest: null, settings: { harvestEnabled: false, harvestAt: '03:30' }, nightly: { lastDate: null } }));
+}
 const pausedConfirmQa = process.argv.includes('--paused-confirm');
 const readingGateQa = process.argv.includes('--reading-limit-ui');
 if (readingGateQa) fs.writeFileSync(path.join(data, 'reading-quiz.json'), JSON.stringify({ version: 1, manga: {
@@ -45,14 +62,19 @@ for (const [index, name] of ['Moon Garden', 'Before the Rain', 'Iron Harbor'].en
   fs.writeFileSync(path.join(library, name, 'manga.json'), JSON.stringify({ title: name,
     direction: directionQa && index === 1 ? 'ltr' : process.argv.includes('--reading-ui') && index === 0 ? 'ttb' : 'rtl',
     ...(directionQa && index === 0 ? { 'x-reading-direction': { version: 1, value: 'rtl', source: 'fallback' } } : {}),
+    // AniList IDs make these owned titles harvest anchors; the unreachable endpoint turns their requests into upstream errors.
+    ...(discoveryQa ? { 'x-anilist': { id: 990101 + index } } : {}),
     status: index ? 'completed' : 'ongoing', tags: ['Fantasy', 'Demo'] }));
 }
-const env = { ...process.env, LIBRARY_PATH: library, DATA_PATH: data, SCRAPER_DATA_PATH: path.join(data, 'scraper'), SENTINEL_DATA_PATH: path.join(data, 'sentinel'),
-  SCRAPER_TOKEN_PATH: tokenFile, READER_PORT: port, READER_BIND: '127.0.0.1', COOKIE_SECURE: '0', APP_IMAGE: process.env.APP_IMAGE || 'mangacircus:local',
+const env = { ...process.env, LIBRARY_PATH: library, DATA_PATH: data, SCRAPER_DATA_PATH: path.join(data, 'scraper'),
+  SCRAPER_TOKEN_PATH: tokenFile, BACKUP_PATH: backupDir, READER_PORT: port, READER_BIND: '127.0.0.1', COOKIE_SECURE: '0', APP_IMAGE: process.env.APP_IMAGE || 'mangacircus:local',
   READER_IMAGE: process.env.APP_IMAGE || 'mangacircus:local',
+  GREATEST_COVER_WARM: '0', // QA stacks never download catalog covers
   ...(readingGateQa ? { READING_WINDOWS_JSON: '{"0":[],"1":[],"2":[],"3":[],"4":[],"5":[],"6":[]}',
     READING_TIMEZONE: 'Europe/Berlin', READING_QUIZ_UNLOCK_MINUTES: '60', READING_QUIZ_CATALOG_PATH: '/data/reading-quiz.json' }
     : { READING_WINDOWS_JSON: '', READING_QUIZ_CATALOG_PATH: '' }),
+  // A closed local port: every harvest request fails fast and AniList is never contacted.
+  ...(discoveryQa ? { DISCOVERY_ANILIST_URL: 'http://127.0.0.1:9/' } : {}),
   ...(peerGatewayQa ? { PEER_DATA_PATH: path.join(data, 'peering'), PEER_USAGE_PATH: path.join(data, 'peering-usage'), PEER_PORT: process.env.QA_PEER_PORT || '8193' } : {}) };
 const args = ['compose', '-p', project, '-f', path.join(root, 'docker-compose.yml'), ...(peerGatewayQa ? ['--profile', 'peering'] : [])];
 const docker = (...command) => new Promise((resolve, reject) => {
@@ -93,6 +115,14 @@ let keep = false;
     let inventory;
     await until(async () => { inventory = await (await api('/api/library')).json(); return inventory.mangas?.length === 3; }, 'Fixture inventory missing');
     const byTitle = title => inventory.mangas.find(m => m.title === title).id;
+    // Built-in backup through the real control API: archive written, paused queue left paused.
+    const queueBefore = JSON.parse(fs.readFileSync(path.join(data, 'scraper', 'queue.json'), 'utf8'));
+    assert.equal((await api('/api/admin/backup', { command: 'run' })).status, 200);
+    let backupState;
+    await until(async () => { backupState = await (await api('/api/admin/backup')).json(); return backupState.last && !backupState.running; }, 'Backup did not finish');
+    assert.equal(backupState.last.error, undefined, backupState.last.error);
+    assert.ok(fs.existsSync(path.join(backupDir, backupState.last.file)), 'Backup archive missing on the host');
+    assert.equal(JSON.parse(fs.readFileSync(path.join(data, 'scraper', 'queue.json'), 'utf8')).paused, queueBefore.paused);
     if (readingGateQa) {
       const manga = byTitle('Moon Garden'), other = byTitle('Before the Rain');
       const chapter = (await (await api('/api/manga?m=' + encodeURIComponent(manga))).json()).chapters[0].id;
@@ -193,12 +223,12 @@ let keep = false;
     }
     const settingsJs = await (await fetch(url + '/settings.js')).text();
     assert.ok(!settingsJs.includes('Start it with: npm run start:scraper'));
-    await until(async () => { const h = await (await api('/api/admin/health')).json(); return h.sentinel.services?.reader && h.sentinel.services?.scraper; }, 'Internal Sentinel networking failed');
+    await until(async () => (await (await api('/api/admin/scraper')).json()).available, 'Internal scraper networking failed');
     const mounts = JSON.parse(await docker('config', '--format', 'json'));
     assert.ok(mounts.services.reader.volumes.find(v => v.target === '/library').read_only);
-    assert.ok(!mounts.services.scraper.ports?.length && !mounts.services.sentinel.ports?.length);
-    const denied = JSON.parse(await docker('exec', '-T', 'reader', 'node', '-e', "Promise.all(['http://scraper:8091/api/state','http://sentinel:8092/api/state'].map(u=>fetch(u).then(r=>r.status))).then(v=>console.log(JSON.stringify(v)))"));
-    assert.deepEqual(denied, [401, 401]);
+    assert.ok(!mounts.services.scraper.ports?.length && !mounts.services.sentinel);
+    const denied = JSON.parse(await docker('exec', '-T', 'reader', 'node', '-e', "fetch('http://scraper:8091/api/state').then(r=>console.log(r.status))"));
+    assert.equal(denied, 401);
     // Deliberate fixture-only child crash: its supervisor must recover just the reader.
     const before = JSON.parse(fs.readFileSync(path.join(data, 'runtime-reader.json'))).pid;
     await docker('exec', '-T', 'reader', 'node', '-e', "const fs=require('fs');process.kill(JSON.parse(fs.readFileSync('/data/runtime-reader.json')).pid,'SIGKILL')");
@@ -207,10 +237,10 @@ let keep = false;
     const image = await api('/lib/' + encodeURIComponent('Moon Garden') + '/cover.png'); assert.equal(image.status, 200);
     state = await (await api('/api/admin/scraper')).json(); assert.equal(state.paused, true);
     if (process.argv.includes('--settings-ui')) await require('./test-settings-ui')(url, cookie);
-    if (process.argv.includes('--discovery-ui')) {
+    if (discoveryQa) {
       const login = await api('/api/login', { name: 'qa', password: 'disposable-local-test' });
       assert.equal(login.status, 200);
-      await require('./test-discovery-ui')(url, login.headers.get('set-cookie').split(';')[0]);
+      await require('./test-discovery-ui')(url, login.headers.get('set-cookie').split(';')[0], { xssTitle: XSS_TITLE });
     }
     if (process.argv.includes('--peer-ui')) {
       const login = await api('/api/login', { name: 'qa', password: 'disposable-local-test' });
@@ -239,11 +269,11 @@ let keep = false;
       assert.equal(login.status, 200);
       await require('./test-direction-ui')(url, login.headers.get('set-cookie').split(';')[0]);
     }
-    console.log('PASS: isolated Compose auth, collections, read-only reader, private APIs, Sentinel networking and child crash recovery');
+    console.log('PASS: isolated Compose auth, collections, read-only reader, private APIs, scraper networking and child crash recovery');
     keep = process.argv.includes('--keep');
     if (keep) {
       const envFile = path.join(dir, 'qa.env');
-      fs.writeFileSync(envFile, ['LIBRARY_PATH', 'DATA_PATH', 'SCRAPER_DATA_PATH', 'SENTINEL_DATA_PATH', 'SCRAPER_TOKEN_PATH', 'READER_PORT', 'READER_BIND', 'COOKIE_SECURE', 'APP_IMAGE',
+      fs.writeFileSync(envFile, ['LIBRARY_PATH', 'DATA_PATH', 'SCRAPER_DATA_PATH', 'SCRAPER_TOKEN_PATH', 'BACKUP_PATH', 'READER_PORT', 'READER_BIND', 'COOKIE_SECURE', 'APP_IMAGE',
         ...(readingGateQa ? ['READING_WINDOWS_JSON', 'READING_TIMEZONE', 'READING_QUIZ_UNLOCK_MINUTES', 'READING_QUIZ_CATALOG_PATH'] : [])]
         .map(k => `${k}=${k === 'READING_WINDOWS_JSON' ? `'${env[k]}'` : env[k].replaceAll('\\', '/')}`).join('\n') + '\n');
       console.log(JSON.stringify({ project, url, envFile, fixture: dir }));

tools/test-discovery-ui.js

@@ -1,7 +1,7 @@
 'use strict';
 const assert = require('node:assert/strict');
 
-module.exports = async function testDiscoveryUI(url, cookie) {
+module.exports = async function testDiscoveryUI(url, cookie, { xssTitle } = {}) {
   const { chromium } = require('playwright');
   const browser = await chromium.launch({ channel: process.env.QA_BROWSER_CHANNEL || 'msedge', headless: true });
   const context = await browser.newContext({ viewport: { width: 1280, height: 900 } });
@@ -9,24 +9,72 @@ module.exports = async function testDiscoveryUI(url, cookie) {
   await context.addCookies([{ name, value: value.join('='), url }]);
   const page = await context.newPage(), errors = [];
   page.on('pageerror', error => errors.push(error.message));
+  // An injected title must never run script; any dialog counts as a failure.
+  page.on('dialog', dialog => { errors.push(`Unexpected dialog: ${dialog.message()}`); dialog.dismiss().catch(() => {}); });
   try {
     await page.goto(url + '/discover.html');
     await page.getByRole('heading', { name: 'Discover', exact: true }).waitFor();
     await page.getByText('Turn on personalized discovery').waitFor();
+    await page.getByText('Off by default. When on, the server sends AniList the IDs of titles you liked, read or own, '
+      + 'their authors\' IDs and your top tags, without login, to find related work. Your feed and feedback stay private to this server.').waitFor();
     await page.getByLabel('Personalized discovery').check();
+    // Review focus 5: the seeded feed is the first one shown, before any run replaces it.
+    // The XSS check must never be skipped silently: the Compose driver always seeds the title.
+    assert.ok(xssTitle, 'The discovery UI test needs the seeded XSS title (run it through tools/test-compose.js --discovery-ui)');
+    const seeded = page.locator('.discovery-card').filter({ hasText: 'AniList readers who liked Moon Garden' });
+    await seeded.waitFor();
+    assert.equal(await seeded.locator('h2').textContent(), xssTitle);
+    assert.equal(await page.locator('.discovery-card h2 img').count(), 0);
+    await seeded.getByText('Why this is here · evidence').click();
+    const details = await seeded.locator('details li').allTextContents();
+    assert.ok(details.some(text => text.includes('New: started 2025')), `All reasons listed: ${details}`);
+    assert.ok(details.some(text => /AniList recommendation.* · fetched \S+/.test(text)), `Evidence fetch date: ${details}`);
     await page.getByRole('button', { name: 'Run discovery now' }).click();
+    // The seeded store already says "success"; wait until the run's own feed replaced the seeded card.
+    await page.waitForFunction(title => /Status: success/.test(document.querySelector('.discovery-admin')?.textContent || '')
+      && ![...document.querySelectorAll('.discovery-card h2')].some(heading => heading.textContent === title), xssTitle, { timeout: 60000 });
     await page.getByText(/suggestions/, { exact: false }).first().waitFor();
     await page.waitForFunction(() => document.querySelectorAll('.discovery-card').length > 0);
     await page.locator('.discovery-card').first().getByText('Why this is here · evidence').click();
     assert.ok(await page.locator('.discovery-card').first().locator('details li').count() > 0);
     await page.locator('.discovery-card').first().getByRole('button', { name: '👍 Maybe' }).click();
     await page.getByText('Preference saved').waitFor();
+
+    // Per-user refresh: the unreachable AniList endpoint ends it with upstream errors, then the cooldown shows.
+    const refresh = page.locator('.discovery-refresh');
+    await page.getByRole('button', { name: 'Refresh my suggestions' }).click();
+    await page.waitForFunction(() => /Next refresh possible at \d{1,2}:\d{2}/.test(document.querySelector('.discovery-refresh')?.textContent || ''),
+      null, { timeout: 90000 });
+    assert.equal(await refresh.isDisabled(), true);
+    await page.locator('.discovery-admin').getByText('Last harvest: stopped').waitFor({ timeout: 30000 });
+    await page.locator('.discovery-admin').getByText('upstream-errors').waitFor();
+
+    // Nightly settings save through the settings route and survive a reload.
+    await page.getByLabel('Automatic harvest').check();
+    await page.getByText('Harvest settings saved').waitFor();
+    await page.getByLabel('Harvest time').fill('04:15');
+    await page.getByText(/Harvest settings saved \(on, 04:15\)/).waitFor();
+    await page.reload();
+    await page.getByRole('heading', { name: 'Discovery job' }).waitFor();
+    assert.equal(await page.getByLabel('Harvest time').inputValue(), '04:15');
+    assert.equal(await page.getByLabel('Automatic harvest').isChecked(), true);
+
+    // Harvest now is single-flight: a second click while one runs shows the server's 409 message.
+    await page.getByRole('button', { name: 'Harvest now' }).click();
+    await page.getByText('Harvest started').waitFor();
+    await page.getByRole('button', { name: 'Harvest now' }).click();
+    await page.getByText('A harvest is already queued or running').waitFor();
     assert.equal(errors.length, 0, `Browser errors: ${errors.join('; ')}`);
-    await page.setViewportSize({ width: 390, height: 844 });
+
+    await page.setViewportSize({ width: 375, height: 812 });
+    await page.getByRole('button', { name: /Refresh my suggestions|Next refresh possible|Queued|Fetching|Paused by AniList/ }).waitFor();
+    assert.ok(await refresh.isVisible());
+    const box = await refresh.boundingBox();
+    assert.ok(box && box.x >= 0 && box.x + box.width <= 375, `Refresh button off screen: ${JSON.stringify(box)}`);
     const overflow = await page.evaluate(() => ({ width: innerWidth, scroll: document.documentElement.scrollWidth,
       elements: [...document.querySelectorAll('body *')].filter(el => el.getBoundingClientRect().right > innerWidth + 1)
         .slice(0, 8).map(el => `${el.tagName}.${el.className}: ${Math.round(el.getBoundingClientRect().right)}`) }));
-    assert.ok(overflow.scroll <= overflow.width, `Discover page overflows on mobile: ${JSON.stringify(overflow)}`);
+    assert.ok(overflow.scroll <= 375, `Discover page overflows on mobile: ${JSON.stringify(overflow)}`);
     const create = await page.evaluate(() => fetch('/api/admin/user', { method: 'POST', headers: { 'content-type': 'application/json' },
       body: JSON.stringify({ name: 'discovery-reader-qa', password: 'disposable-reader-test', admin: false }) }).then(response => response.status));
     assert.equal(create, 200);
@@ -36,7 +84,10 @@ module.exports = async function testDiscoveryUI(url, cookie) {
     await page.reload();
     await page.getByRole('heading', { name: 'Discover', exact: true }).waitFor();
     assert.equal(await page.getByRole('button', { name: 'Run discovery now' }).count(), 0);
+    assert.equal(await page.getByRole('button', { name: 'Harvest now' }).count(), 0);
     assert.equal(await page.evaluate(() => fetch('/api/admin/discovery').then(response => response.status)), 403);
+    assert.equal(await page.evaluate(() => fetch('/api/admin/discovery/harvest', { method: 'POST', headers: { 'content-type': 'application/json' }, body: '{}' })
+      .then(response => response.status)), 403);
     assert.equal(errors.length, 0, `Browser errors: ${errors.join('; ')}`);
   } finally { await browser.close(); }
 };

tools/test-greatest-ui.js

@@ -40,7 +40,7 @@ async function freePort() {
   const scraperPort = fakeScraper.address().port;
   const child = spawn(process.execPath, ['server.js'], { cwd: root, windowsHide: true,
     env: { ...process.env, PORT: String(port), LIBRARY: library, DATA: data, SCAN_INTERVAL: '0',
-      FULL_SCAN_INTERVAL: '0', SCRAPER_URL: `http://127.0.0.1:${scraperPort}`, COOKIE_SECURE: '0' }, stdio: 'ignore' });
+      FULL_SCAN_INTERVAL: '0', SCRAPER_URL: `http://127.0.0.1:${scraperPort}`, COOKIE_SECURE: '0', GREATEST_COVER_WARM: '0' }, stdio: 'ignore' });
   let browser;
   try {
     for (let i = 0; i < 80; i++) {

tools/test-peer-ab.js

@@ -50,9 +50,9 @@ async function until(check, message, attempts = 120) {
 }
 async function setup(label, readerPort, peerPort) {
   const root = path.join(TEST_ROOT, label), library = path.join(root, 'library'), data = path.join(root, 'data');
-  const scraper = path.join(data, 'scraper'), sentinel = path.join(data, 'sentinel'), peerData = path.join(data, 'peering');
+  const scraper = path.join(data, 'scraper'), peerData = path.join(data, 'peering');
   const usage = path.join(data, 'peer-usage'), token = path.join(root, 'scraper-token');
-  for (const dir of [library, scraper, sentinel, peerData, usage]) await fsp.mkdir(dir, { recursive: true });
+  for (const dir of [library, scraper, peerData, usage]) await fsp.mkdir(dir, { recursive: true });
   await fsp.writeFile(path.join(scraper, 'queue.json'), JSON.stringify({ paused: true, queue: [], settings: { autoUpdateEnabled: false } }));
   await fsp.writeFile(token, crypto.randomBytes(32).toString('hex'), { mode: 0o600 });
   const png = new Canvas(180, 270, [55, 105, 150]).png();
@@ -71,7 +71,7 @@ async function setup(label, readerPort, peerPort) {
   }
   new PeeringState(peerData).generateIdentity(label === 'a' ? 'PeerA' : 'PeerB');
   const env = { ...process.env, APP_IMAGE, VCS_REF: 'peer-redesign-test', LIBRARY_PATH: library, DATA_PATH: data,
-    SCRAPER_DATA_PATH: scraper, SENTINEL_DATA_PATH: sentinel, PEER_DATA_PATH: peerData, PEER_USAGE_PATH: usage,
+    SCRAPER_DATA_PATH: scraper, PEER_DATA_PATH: peerData, PEER_USAGE_PATH: usage,
     SCRAPER_TOKEN_PATH: token, READER_PORT: String(readerPort), READER_BIND: '127.0.0.1',
     PEER_PORT: String(peerPort), PEER_BIND: '127.0.0.1', COOKIE_SECURE: '0', TZ: 'UTC' };
   return { label, root, library, data, peerData, manga, readerPort, peerPort, env, url: `http://127.0.0.1:${readerPort}` };
@@ -244,7 +244,7 @@ async function main() {
     assert.deepEqual(peerQueueItems[0].workIdentity, catalog.data.items[0].workIdentity,
       'The resolved work identity is persisted on the queued peer transfer');
     const volumes = await Promise.all(started.map(({ project, env }) => docker(project, env, 'ps', '--format', '{{.Service}}')));
-    for (const services of volumes) for (const name of ['reader', 'scraper', 'sentinel', 'peer-gateway']) assert.ok(services.includes(name), `${name} is running in A and B`);
+    for (const services of volumes) for (const name of ['reader', 'scraper', 'peer-gateway']) assert.ok(services.includes(name), `${name} is running in A and B`);
     console.log('PASS: isolated Compose A↔B pairing, catalog-only access, cover proxy, grant update, browser details/bulk queue, one paused queue writer, and admin-only API/UI boundary.');
     console.log(`Test Compose projects: ${projects.join(', ')}; disposable roots: ${TEST_ROOT}`);
     succeeded = true;

vision/discovery-scout.md

@@ -1,6 +1,6 @@
 # Discovery Scout — personalized manga discovery
 
-Status: **deterministic first slice implemented; later phases remain a vision**. Updated 2026-10-09. See [`docs/discovery.md`](../docs/discovery.md) for the current contract and deliberate omissions.
+Status: **deterministic first slice and the wider AniList candidate pool implemented; later phases remain a vision**. Updated 2026-10-09. See [`docs/discovery.md`](../docs/discovery.md) for the current contract and deliberate omissions.
 
 ## 1. Outcome
 
@@ -12,7 +12,7 @@ This is a discovery and curation feature, not a new image scraper, metadata auth
 
 Current MangaCircus constraints remain: plain Node and browser JavaScript, Docker Compose, a folder-tree library, JSON state rather than a database, one reader owner of account state, one scraper owner of the queue and library publication, and source-specific slow request pacing. See [`docs/architecture.md`](../docs/architecture.md), [`docs/greatest.md`](../docs/greatest.md), and [`docs/scraper-service.md`](../docs/scraper-service.md).
 
-MVP lives in the **reader service** as a bounded asynchronous job and a reader-owned JSON store. Do not add a permanently running fourth service at first. If real LLM or metadata latency later makes reader isolation necessary, move the job behind the same API to a one-off/optional worker container using the existing image; do not change the data contract or add a database merely to do that. Sentinel observes availability but does not judge recommendations, call an LLM, or repair research claims.
+MVP lives in the **reader service** as a bounded asynchronous job and a reader-owned JSON store. Do not add a permanently running third service at first. If real LLM or metadata latency later makes reader isolation necessary, move the job behind the same API to a one-off/optional worker container using the existing image; do not change the data contract or add a database merely to do that. Service health reports availability but does not judge recommendations, call an LLM, or repair research claims.
 
 Explicit non-goals for the first release:
 
@@ -41,6 +41,8 @@ Use a two-stage pipeline. **Code** gathers, identifies, filters, and computes re
 
 ### Candidate input, in priority order
 
+**Implemented (2026-10-25):** items 1 and 2 — the reader harvests AniList recommendations, same-creator works, top titles in the user's tags and fresh titles into a local cache, with budgets, a nightly run and per-user refresh. See the [wider-pool spec](../docs/superpowers/specs/2026-10-25-discovery-wider-pool-design.md) and [`docs/discovery.md`](../docs/discovery.md). Item 3 (admin seed form) is still open.
+
 1. Reuse public AniList metadata/relations already cached by MangaCircus, plus the read-only Greatest catalog, for cold start and sequel/creator leads. Existing *Greatest* votes are stronger preference signals than curator rank.
 2. Add one or two explicit, public metadata feeds only after checking their API terms and stable identity fields. Cache responses with timestamps. Publisher/award lists may seed editorial candidates, but only with source links and dates.
 3. Let an admin seed a candidate by metadata URL or exact provider ID. This is the fallback when public APIs are incomplete. It is **not** a scraper URL and creates no queue item.
@@ -134,7 +136,7 @@ Queueing remains through `POST /api/admin/scraper` with `command:"enqueue"` and
 - One in-process run lock plus a persisted run ID prevents overlap across browser clicks and restarts. A stale interrupted run is marked interrupted on startup; keep the previous successful feed. Publish only after validation completes.
 - Preserve source errors and diagnostics with bounded log size. Do not store sensitive prompts or external response bodies indefinitely. Provider blocks/challenges require user attention, not automated retries or evasions.
 - Nightly scheduling comes after the manual path has passed acceptance. Make local-time schedule configurable, persist last-success time, skip duplicate same-day runs after restart, and allow a manual run without disturbing scraper nightly updates. A missed night can run once after startup, never a catch-up storm.
-- The feature is optional: reader, library, Greatest, scraper, and Sentinel continue normally if metadata or LLM access is absent.
+- The feature is optional: reader, library, Greatest, and scraper continue normally if metadata or LLM access is absent.
 
 ## 8. Phased implementation and acceptance
 
@@ -152,7 +154,7 @@ Add one configurable model adapter, schema validation, evidence-bound output, op
 
 ### Phase D — scheduling and operations
 
-Add nightly schedule only after manual runs are stable. Expose last success/error and a `discovery_run_failed` incident to admin observability, but do not give Sentinel authority to rewrite recommendations or restart the scraper. Add backup/restore instructions for `data/discovery.json` and cache behavior. If runs measurably harm reader latency, move the job into an optional one-off Compose worker with one-writer handoff, not an always-on autonomous service.
+Add nightly schedule only after manual runs are stable. Expose last success/error and a `discovery_run_failed` incident to admin observability, but do not give any watchdog authority to rewrite recommendations or restart the scraper. Add backup/restore instructions for `data/discovery.json` and cache behavior. If runs measurably harm reader latency, move the job into an optional one-off Compose worker with one-writer handoff, not an always-on autonomous service.
 
 ### Suggested implementation map
 
@@ -167,7 +169,7 @@ Keep the first code change small and reviewable:
 | `public/discover.html`, `discover.js`, `discover.css` (new) | Feed, evidence, feedback, empty/error states, admin run control |
 | `public/common.js` | One navigation link; no global discovery polling |
 | `test/discovery*.test.js`, disposable browser test | Identity, consent, isolation, budgets, action permissions and UI behavior |
-| `docs/architecture.md`, `docs/compose-deployment.md` | Record ownership, backup, local release and rollback once implemented |
+| `docs/architecture.md`, `docs/deployment.md` | Record ownership, backup, local release and rollback once implemented |
 
 The exact file split is a proposal, not a requirement to create empty modules. Start with storage and identity tests, then the deterministic pipeline, then API/UI. Do not add model-provider plumbing, a new Compose service, or a nightly timer in the first implementation slice. A candidate must pass schema validation and duplicate/identity checks before it can appear in a published feed.
 

vision/server-peering.md

@@ -74,7 +74,7 @@ Catalog snapshots use cursor/revision pagination and bounded page sizes. A revis
 
 Input from a peer is untrusted even after pairing: cap manifests, chapter/page sizes and total transfer size; validate schemas and image MIME/content; reject traversal paths, archives/executables, and malformed names; stream rather than buffering a whole manga; verify each page hash before publication. Hashes detect corruption but do not prove editorial correctness. Never overwrite an existing chapter silently.
 
-For transport efficiency, start with original image bytes and page-level resume. A [local compression test](server-peering-compression-test.md) found an average ~6% reduction with zstd-3 but a wide 0.7–10.7% spread by chapter, so codec negotiation is an optional later optimization, not a prerequisite for peering. Measure actual chapter-ready time over a real peer link before adding chapter packing or automatic compression.
+For transport efficiency, start with original image bytes and page-level resume. A local compression test (`tools/benchmark-peer-transfer.js`) found an average ~6% reduction with zstd-3 but a wide 0.7–10.7% spread by chapter, so codec negotiation is an optional later optimization, not a prerequisite for peering. Measure actual chapter-ready time over a real peer link before adding chapter packing or automatic compression.
 
 ## 5. Reader and queue experience
 
@@ -107,7 +107,7 @@ Suggested NoDB files (names are design targets, not existing files): reader-owne
 3. **Read-only publisher:** optional gateway, bounded approved-only catalog/manifest/blob endpoints, quota enforcement, read-only mounts. Test that unshared, partially downloaded, deleted, and revoked content cannot be listed or fetched—even by guessed IDs or stale manifests. Verify no user data fields or raw source URLs appear on the wire.
 4. **Consumer browsing:** local-server proxy and reader UI with peer status, search, details, selection preview, and queue permission checks. No browser-to-peer credentials or direct requests.
 5. **Scraper transfer lane:** chapter/page checkpoints, range resume, hashes, local throttles, source-aware deduplication, and existing atomic chapter publication. Test corruption, interrupted transfer, stale revision, out-of-space, repeated queue submission, and preservation of current jobs.
-6. **Observability and operations:** per-peer health, bytes, throttling, grant expiry warnings, queue stalls, and safe retry controls. Sentinel may report a dead gateway or stuck transfer but must not renew trust, broaden a share, or bypass a quota. Document backup/restore and key-rotation runbooks.
+6. **Observability and operations:** per-peer health, bytes, throttling, grant expiry warnings, queue stalls, and safe retry controls. Service health may report a dead gateway or stuck transfer but must not renew trust, broaden a share, or bypass a quota. Document backup/restore and key-rotation runbooks.
 7. **Local then VM pilot:** three local Compose instances with distinct data directories; pair A↔B and A↔C, exercise every direction combination, and transfer a few admin-approved test chapters. Verify disabling B does not affect C, no account/progress exchange, and quota isolation. Only then deploy the optional service on the VM's private network. Leave existing reader/scraper ports and active downloads undisturbed during the pilot.
 
 **Ship gate:** a fresh unpaired server sees nothing; a paired server with serving off or no granted manga sees an empty catalog; an expired/revoked peer cannot list or fetch; invitations cannot be reused; disabling one direction or peer leaves unrelated peers intact; quotas survive restart; a receiver can resume and publish one chapter without duplicates; and its browser never receives the grant token or any private key. The feature remains disabled by default until those gates pass.