MaashkinoozheandClaude Opus 5.5
03767d5704
The reader polled /api/state every 15 s; each call rebuilt the scraper's library summary when it was over 60 s old, reading every chapter.json (on the NAS about 5,300 opens / 16,000 NFS calls and 1.3 s CPU per walk, once a minute, plus 115 KB per poll). The reader now polls GET /api/revision (falls back to /api/state on an older scraper), and the summary is rebuilt after a library change or after 15 minutes. Release-date writes invalidate it too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MangaCircus scraper
The scraper uses small host adapters to create a common manifest, followed by one conservative publisher that writes the MangaCircus library atomically.
Start with AGENT_RUNBOOK.md. It contains the host-specific procedures, safety limits, and stop conditions. manifest.example.json documents the interchange format.
Once the managed daemon has started, data/scraper/queue.json is the authoritative
queue and is edited through the admin UI.
Managed service
Run MangaCircus and the scraper daemon together:
npm run start:all
Or run them as two independently supervised processes:
npm start
npm run start:scraper
The daemon binds to 127.0.0.1:8091 by default, keeps its durable queue in
data/scraper/queue.json, and resumes incomplete archive checkpoints. An admin can
control it from Settings → Scraper. Set SCRAPER_TOKEN on both processes when
the daemon is not restricted to loopback.
Queue schema version 3 supports scoped jobs, multiple jobs for the same upstream
series, one sequential worker per source, incremental chapter publication, library
duplicate/version checks, and configurable nightly update discovery. See
docs/scraper-service.md for the service contract and
docs/deployment.md for running it in Docker. Run only one
supervisor or container against the same library and data.
npm run test:scraper
node scraper/cli.js metadata-anilist "One Piece"
node scraper/cli.js discover-weebcentral "https://weebcentral.com/series/..." --limit 3 --out manifest.json
node scraper/cli.js publish manifest.json --library library --metadata-search "Series title"
node scraper/cli.js archive-weebcentral "https://weebcentral.com/series/..." --library library --metadata-search "Series title"
node scraper/cli.js status-weebcentral "https://weebcentral.com/series/..." --library library
The implementation uses only Node.js built-ins. Browser-only sources must first export observed page assets to local files; the publisher consumes those files and does not automate access-control challenges.