Scraper: stop walking the whole library once a minute for status polls

The reader polled /api/state every 15 s; each call rebuilt the scraper's library
summary when it was over 60 s old, reading every chapter.json (on the NAS about
5,300 opens / 16,000 NFS calls and 1.3 s CPU per walk, once a minute, plus 115 KB
per poll). The reader now polls GET /api/revision (falls back to /api/state on an
older scraper), and the summary is rebuilt after a library change or after 15
minutes. Release-date writes invalidate it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MaashkinoozheandClaude Opus 5.5 committed 2026-10-29 23:10:38 +01:00
1 parent b328ad98fa
commit 03767d5704
5 files changed
+40 -7

No files matched your search

+5 -2
View File
@@ -1218,10 +1218,13 @@ if (FULL_SCAN_MINUTES > 0) setInterval(() => {
// A completed scraper publish increments a durable revision. Polling it here keeps the
// reader index fresh without exposing an unauthenticated rescan endpoint to the daemon.
let scraperRevision = null;
let scraperRevision = null, revisionRoute = true;
async function pollScraper() {
try {
const state = await scraperRequest('/api/state');
// /api/revision is a few bytes; the full state is ~100 KB and makes the scraper summarise the library.
let state;
if (revisionRoute) state = await scraperRequest('/api/revision').catch(error => { if (error.code !== 404) throw error; revisionRoute = false; });
if (!revisionRoute) state = await scraperRequest('/api/state'); // a scraper from before /api/revision
const revision = Number(state.publishedRevision || 0);
if (scraperRevision === null || revision > scraperRevision) {
const checked = await refreshChanged();