Scraper: stop walking the whole library once a minute for status polls

The reader polled /api/state every 15 s; each call rebuilt the scraper's library
summary when it was over 60 s old, reading every chapter.json (on the NAS about
5,300 opens / 16,000 NFS calls and 1.3 s CPU per walk, once a minute, plus 115 KB
per poll). The reader now polls GET /api/revision (falls back to /api/state on an
older scraper), and the summary is rebuilt after a library change or after 15
minutes. Release-date writes invalidate it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MaashkinoozheandClaude Opus 5.5 committed 2026-10-29 23:10:38 +01:00
1 parent b328ad98fa
commit 03767d5704
5 files changed
+40 -7

No files matched your search

+2 -1
View File
@@ -199,7 +199,8 @@ All responses are JSON. When `SCRAPER_TOKEN` is set, send
| Method | Path | Body / result |
|---|---|---|
| `GET` | `/health` | Basic liveness |
| `GET` | `/api/state` | Queue, all active source checkpoints, settings, errors, history, adapters |
| `GET` | `/api/state` | Queue, all active source checkpoints, settings, errors, history, adapters. The library summary in it is rebuilt in the background after a publish, or when it is 15 minutes old (the walk reads every `chapter.json`) |
| `GET` | `/api/revision` | `{ publishedRevision, status, paused }` only; the reader polls this every 15 s |
| `POST` | `/api/queue` | `{url,title?,metadataSearch?,scope?,existingPolicy?,libraryMangaId?,inspect?}` |
| `POST` | `/api/queue/remove` | `{id}`; active downloads cannot be removed |
| `POST` | `/api/queue/clear-completed` | `{source?}`; remove inactive successful download job rows only; preserve library, history, checkpoints and other statuses |