Discovery harvest routes, nightly schedule and per-user refresh (BL-0051)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5de5eb6a08
commit
0f87fbe948
4 files changed
+395
-12
No files matched your search
+28
-1
@@ -171,7 +171,7 @@ This keeps ids human-readable and derivable without a database. The price: renam
|
||||
|
||||
| Path | Handling |
|
||||
|------|----------|
|
||||
| `/api/*` | `routes["<METHOD> <path>"]`. Handlers get `{ u, b, q, req, res }` and return an object, which is sent as JSON. Throwing `HttpError` sends `{ error }` with that status |
|
||||
| `/api/*` | `routes["<METHOD> <path>"]`. Handlers get `{ u, b, q, req, res }` and return an object, which is sent as JSON. Throwing `HttpError(code, message, extra)` sends `{ error, ...extra }` with that status (for example `nextAllowedAt` on a discovery `429`) |
|
||||
| `/lib/*` | Image files from `LIBRARY`. Requires a session, allows image extensions only, and the path must resolve inside `LIBRARY`. `ETag` + `private, max-age=86400` |
|
||||
| everything else | Static files from `public/` (`/` → `index.html`). `ETag` + `no-cache` |
|
||||
|
||||
@@ -207,6 +207,13 @@ All requests and responses are JSON. POSTs require `Content-Type: application/js
|
||||
| `POST /api/admin/user/role` `{name, admin}` | admin | Grant or revoke admin rights; keeps the password and sessions. Revoking your own rights is refused, so one admin always remains |
|
||||
| `POST /api/admin/user/delete` `{name}` | admin | Delete a user and their data |
|
||||
| `GET /api/admin/backup` · `POST /api/admin/backup` `{command:"settings"\|"run"}` | admin | Built-in backup status, schedule/retention, "back up now" ([backup.md](backup.md)) |
|
||||
| `GET /api/discovery` | user | The caller's discovery feed, settings, feedback and `refresh: { state: idle\|queued\|running, position?, nextAllowedAt?, blockedReason? }` (§6.6) |
|
||||
| `POST /api/discovery/feedback` `{candidateId, value}` | user | The caller's choice for a candidate; requires opt-in |
|
||||
| `POST /api/discovery/refresh` | user | Queue a per-user AniList harvest for the caller (opt-in required, else `403`). `{ harvestId, position }`; `409` while one is queued or running for the caller; `429 { nextAllowedAt }` within an hour of the last accepted refresh; `503 { blockedReason }` after AniList pushed back, until a nightly or admin harvest |
|
||||
| `GET /api/admin/discovery` | admin | Last scoring run, opted-in users, candidate count, plus `harvest` (last run record), `currentHarvest`, `cache: { files, bytes }`, `settings: { harvestEnabled, harvestAt }` and `nextNightlyAt` (`null` when off) |
|
||||
| `POST /api/admin/discovery/run` | admin | Re-score every user from local data; `409` while a run is active |
|
||||
| `POST /api/admin/discovery/harvest` | admin | Start a full harvest for all opted-in users: `{ harvestId, position }`, `409` while any harvest is queued or running |
|
||||
| `POST /api/admin/discovery/settings` `{harvestEnabled, harvestAt}` | admin | Exactly a boolean and an `HH:MM` time, anything else is `400`; stored in `discovery.json` `settings` |
|
||||
|
||||
### 6.3 Authentication and security
|
||||
|
||||
@@ -229,6 +236,26 @@ At request time: the sum of all users' votes, then `chapter.json.priority`, then
|
||||
|
||||
In-app notifications and optional webhooks are produced by `announce()` after each scan, from the append-only `known` list. See [import.md §5](import.md#5-change-detection-and-notifications).
|
||||
|
||||
### 6.6 Discovery harvest
|
||||
|
||||
The reader alone talks to AniList for discovery ([discovery.md](discovery.md)). A single-flight harvester
|
||||
(`discovery/harvest.js`) fetches seeds for the opted-in users' anchors into `data/discovery-cache/`; scoring
|
||||
(`discovery/engine.js`) is local and publishes `data/discovery.json` atomically.
|
||||
|
||||
- **Kinds and budgets:** `nightly` and `admin` (300 requests, every opted-in user), `user` (60 requests, the
|
||||
caller only). After a nightly or admin harvest every user is re-scored; after a per-user refresh only that
|
||||
user, and the other feeds are carried over unchanged into the same publish.
|
||||
- **Schedule:** checked every minute and once 30 s after startup. The time is `discovery.json` `settings`, else
|
||||
`DISCOVERY_HARVEST_AT` (`HH:MM`), else `03:30`; `nightly.lastDate` is written only when the run is accepted.
|
||||
- **Run record** `discovery.json` `harvest`: `{ id, kind, user, status: running|success|stopped|interrupted,
|
||||
startedAt, finishedAt, requests, budget, stopReason }`. `stopReason` is `budget`, `rate-limited`, `forbidden`,
|
||||
`challenge`, `upstream-errors` or `error` (a local fault). A run cut off by a restart is marked `interrupted`
|
||||
at startup; queued refreshes are not persisted.
|
||||
- **Published fields:** candidates found through AniList carry `tagRecords`, `authorRecords`, `seed`
|
||||
(`recs|creator|tagTop|fresh`) and `fetchedAt`; feed items carry `group` (seed kind or period) and `reasons`
|
||||
(all reasons, strongest first) next to `reason`; seed evidence carries `fetchedAt`.
|
||||
- `DISCOVERY_ANILIST_URL` overrides the AniList endpoint (tests point it at a local fixture server).
|
||||
|
||||
## 7. Frontend
|
||||
|
||||
### 7.1 Pages
|
||||
|
||||
Reference in new issue
Block a user