Discovery final review fixes: lean publish, robust scoring, cheap prune (BL-0051)
- Cache prune counts directory entries first and only reads files when over the cap; tests for cross-kind prune, schema-mismatch and corrupt entries. - Scoring tolerates malformed manga.json and cached records (non-list tags, object or null relations, non-object items); a tag without a rank no longer passes the top-tag filter; authors without an integer AniList id are dropped. - discovery.json publishes only candidates some feed shows; examined/clipped still describe the pool. Tests that asserted the published count equals the pool size now check the pool via candidatePool() and the published set against the feeds. - The post-harvest rescore failure test now breaks discovery.json right before the publish (via a scraper stand-in) instead of relying on tags: 5, and a following successful harvest clears the stale lastError. - Consent text names everything sent to AniList; paused label mentions admin harvests; the UI test asserts the XSS seed instead of skipping it. - Slot-mix test proves the fresh/outside reservations matter. - Docs: privacy note, fresh fallback reason, per-period cap, dry-run wording, published-candidate rule; plan/spec hygiene and spec status. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
613db96355
commit
cd6a30a76b
18 files changed
+250
-65
No files matched your search
+12
-8
@@ -45,12 +45,13 @@ count. It then asks AniList for four seed kinds:
|
||||
| Community recommendations | "readers who liked X recommend Y" for each anchor | AniList readers who liked {anchor} recommend this ({rating}). |
|
||||
| Same creator | other works by the story/art authors of strong anchors | Also by {author} — you read/liked {anchor}. |
|
||||
| Top in your tags | well-rated titles in the user's three strongest tags | Top-rated in {tag}, one of your most-liked tags. |
|
||||
| Fresh | recently started, releasing and trending titles | New: started {year}, ongoing and trending. |
|
||||
| Fresh | recently started, releasing and trending titles | New: started {year}, ongoing and trending. — or, when the cached record does not show an ongoing title started in the last two years, Trending on AniList right now. |
|
||||
|
||||
A title found by several seeds keeps every reason and evidence item; the card
|
||||
shows the strongest, the details list all of them, each with the date it was
|
||||
fetched. A feed holds at most 20 suggestions, at most 8 from one seed kind, up to
|
||||
4 fresh titles, and some from outside your strongest tags.
|
||||
fetched. A feed holds at most 20 suggestions, at most 8 from one seed kind, at most
|
||||
4 from one Greatest period (or from related works), up to 4 fresh titles, and up to 4
|
||||
from outside your strongest tags.
|
||||
|
||||
- **Budgets:** a nightly or admin harvest makes at most 300 requests, a
|
||||
per-user refresh 60. Requests are sequential and limited to 25 a minute. When
|
||||
@@ -65,8 +66,8 @@ fetched. A feed holds at most 20 suggestions, at most 8 from one seed kind, up t
|
||||
the nightly harvest off. A night missed while the reader was down runs once
|
||||
after startup.
|
||||
- **Privacy:** for opted-in users only, the server sends AniList the IDs of
|
||||
their anchor titles (and the names of their top tags), without login, cookies or
|
||||
user names. Feeds and feedback stay private to this server; cache entries are
|
||||
their anchor titles (liked, read or owned), the AniList staff IDs of those titles'
|
||||
authors and the names of their top tags, without login, cookies or user names. Feeds and feedback stay private to this server; cache entries are
|
||||
not tied to a user.
|
||||
|
||||
### Stop reasons
|
||||
@@ -88,8 +89,9 @@ re-scoring failure after a harvest is shown as the admin panel's error line.
|
||||
|
||||
## Data and API
|
||||
|
||||
- `data/discovery.json`: atomic, reader-owned candidate and per-user feed
|
||||
snapshots plus the last harvest record and the admin's harvest settings. A
|
||||
- `data/discovery.json`: atomic, reader-owned per-user feed snapshots and the
|
||||
candidates those feeds show (the rest of the pool is only counted:
|
||||
`examined`, `clipped`), plus the last harvest record and the admin's harvest settings. A
|
||||
previous valid version is retained as `discovery.json.prev`. A corrupt file
|
||||
makes reads/runs/harvests fail closed rather than overwrite it.
|
||||
- `data/discovery-cache/`: harvested AniList responses; safe to delete.
|
||||
@@ -115,7 +117,9 @@ APP_IMAGE=mangacircus:discovery-test npm run test:discovery-ui
|
||||
```
|
||||
|
||||
The dry run reads `db.json`, a saved library index if present, Greatest, and
|
||||
cached AniList metadata. It writes nothing and does not contact providers. The
|
||||
cached AniList metadata. It does not contact AniList or any other provider and does
|
||||
not publish `discovery.json`; reading the harvest cache may still delete a corrupt
|
||||
cache file, as the reader would. The
|
||||
Compose/browser suite uses a disposable data tree and queue, not the live one,
|
||||
and points `DISCOVERY_ANILIST_URL` at a closed local port so no harvest reaches
|
||||
AniList.
|
||||
|
||||
Reference in new issue
Block a user