MangaCircus/docs/discovery.md

Discovery Scout

The reader has a separate Discover page. Suggestions come from the curated Greatest catalog, cached AniList relations of Greatest and library titles, and a wider pool that the reader harvests from AniList's public API for the users who opted in (design: the wider-pool spec). Discovery does not call an LLM, inspect chapter/image hosters, edit a collection, or enqueue downloads. The reader remains the only writer of account and discovery state; no further Compose service, credential, or database is required.

Use

1. Open discover.html. Each account independently enables Personalized discovery (off by default). Reading activity is a separate opt-in. The mature-title switch only affects titles AniList marks as adult or with a known mature tag; it is not a content-safety guarantee.
2. Refresh my suggestions queues a harvest for your own account (at most once an hour). The button shows the queue position, Fetching… while it runs, and when the next refresh is possible. Your existing suggestions stay visible until the new feed is published.
3. An admin can press Run discovery now (re-score from local data, no network) or Harvest now (fetch from AniList for every opted-in user, then re-score). A second concurrent run or harvest is rejected. A nightly harvest runs on its own; the admin panel can switch it off or change its time.
4. A reader can open Why this is here · evidence and record Interested, Maybe, No thanks, or Hide. Feedback is private to the account. Dismissed/hidden titles disappear at once; other feedback influences the next run. The user can undo a choice by clicking its active button again.

The recommendation is an editorial lead, not an availability or edition check. The public metadata link opens the provider record; an admin still uses the existing source inspection and confirmation process to download anything. The title/year/format matching is intentionally conservative but not a verified identity merge. Missing year, cover, or status remains missing.

Seeds from AniList

A harvest starts from each opted-in user's anchors: titles they voted flame/good in Greatest, marked Interested/Maybe in Discover, read (only with reading activity opted in), or own with an AniList ID. The 15 strongest anchors count. It then asks AniList for four seed kinds:

Seed	What	Reason shown on the card
Community recommendations	"readers who liked X recommend Y" for each anchor	AniList readers who liked {anchor} recommend this ({rating}).
Same creator	other works by the story/art authors of strong anchors	Also by {author} — you read/liked {anchor}.
Top in your tags	well-rated titles in the user's three strongest tags	Top-rated in {tag}, one of your most-liked tags.
Fresh	recently started, releasing and trending titles	New: started {year}, ongoing and trending. — or, when the cached record does not show an ongoing title started in the last two years, Trending on AniList right now.

A title found by several seeds keeps every reason and evidence item; the card shows the strongest, the details list all of them, each with the date it was fetched. A feed holds at most 20 suggestions, at most 8 from one seed kind, at most 4 from one Greatest period (or from related works), up to 4 fresh titles, and up to 4 from outside your strongest tags.

• Budgets: a nightly or admin harvest makes at most 300 requests, a per-user refresh 60. Requests are sequential and limited to 25 a minute. When the budget is spent the harvest stops; what was fetched is still used.
• Cache: data/discovery-cache/, one file per response. Titles, recommendations and creators stay fresh for 30 days, tag lists and the fresh list for 7 days; at most 5,000 files, oldest pruned first. Scoring reads only this cache and never the network, so a failed harvest means older candidates, never an empty feed.
• Nightly time: 03:30 by default, or DISCOVERY_HARVEST_AT, in the container's local time zone (TZ). The admin panel overrides both and can turn the nightly harvest off. A night missed while the reader was down runs once after startup.
• Privacy: for opted-in users only, the server sends AniList the IDs of their anchor titles (liked, read or owned), the AniList staff IDs of those titles' authors and the names of their top tags, without login, cookies or user names. Feeds and feedback stay private to this server; cache entries are not tied to a user.

Stop reasons

The admin panel shows the last harvest as status · requests/budget · stop reason · cache files.

Stop reason	Meaning	What happens next
budget	The request budget ran out	Normal; the next harvest continues where the cache is stale
rate-limited	AniList answered 429	Harvesting pauses; refresh buttons show "Paused by AniList" until the next nightly or admin harvest
forbidden	AniList answered 401/403	Same pause
challenge	AniList answered with an HTML or challenge page	Same pause
upstream-errors	Five timeouts or server errors in a row	The harvest ends; nothing is paused
error	A local fault (for example a cache write)	See the reader log

Discovery never retries with a different identity, user agent or host. A re-scoring failure after a harvest is shown as the admin panel's error line.

Data and API

• data/discovery.json: atomic, reader-owned per-user feed snapshots and the candidates those feeds show (the rest of the pool is only counted: examined, clipped), plus the last harvest record and the admin's harvest settings. A previous valid version is retained as discovery.json.prev. A corrupt file makes reads/runs/harvests fail closed rather than overwrite it.
• data/discovery-cache/: harvested AniList responses; safe to delete.
• data/db.json: consent in user settings and explicit feedback under discoveryFeedback. The account-deletion route removes that feedback.
• Routes: architecture.md (/api/discovery*, /api/admin/discovery*) and §6.6 there for the harvester.

The job asks the private scraper state API for queued titles to avoid recommending them. If that API is unavailable, the run still uses only cached sources; the admin result records queueAvailable: false and queue exclusion is incomplete. No scraper queue edit occurs.

Read-only dry run and verification

From the project directory:

node discovery/cli.js run --dry-run --data data --show
npm run test:reader
docker build -t mangacircus:discovery-test .
APP_IMAGE=mangacircus:discovery-test npm run test:discovery-ui

The dry run reads db.json, a saved library index if present, Greatest, and cached AniList metadata. It does not contact AniList or any other provider and does not publish discovery.json; reading the harvest cache may still delete a corrupt cache file, as the reader would. The Compose/browser suite uses a disposable data tree and queue, not the live one, and points DISCOVERY_ANILIST_URL at a closed local port so no harvest reaches AniList.

Limits

There is no admin seed form, edition-finding action, LLM rationale, or explicit purge control for old feed snapshots. Disabling consent stops feed display, harvesting and future personalized generation for that account, but existing snapshot/feedback is retained until account deletion.