Commit Graph
12 Commits
Author SHA1 Message Date
MaashkinoozheandClaude Opus 5.5 03767d5704 Scraper: stop walking the whole library once a minute for status polls
The reader polled /api/state every 15 s; each call rebuilt the scraper's library
summary when it was over 60 s old, reading every chapter.json (on the NAS about
5,300 opens / 16,000 NFS calls and 1.3 s CPU per walk, once a minute, plus 115 KB
per poll). The reader now polls GET /api/revision (falls back to /api/state on an
older scraper), and the summary is rebuilt after a library change or after 15
minutes. Release-date writes invalidate it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 23:10:38 +01:00
MaashkinoozheandClaude Opus 5.5 57a571d2f0 WeebCentral and MangaPill are off by default, with an admin switch per source
Both are unofficial sites that mostly host unlicensed copies; MangaDex (official
API) stays on. Settings → Admin → Scraper → Hoster settings has a 'Use <source>'
switch. A switched-off source refuses new jobs, keeps queued jobs waiting and is
skipped by nightly update checks. Installations whose queue or history already
used a source keep it switched on, so upgrades don't stop a working queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 07:47:47 +01:00
MaashkinoozheandClaude Opus 5.5 f53b92a7dc Prepare the repository for a public release
- docs/deployment.md: step-by-step Docker deployment for anyone (setup,
  configuration, LAN/HTTPS/VPN access, adding manga, operation, prebuilt
  images, backups, peering, troubleshooting, rules for agents).
- deploy/circus.sh: host helper without Node (status, up/down with queue
  pause, logs, pause/resume, update, rescan, restore-test); deploy/compose.https.yml
  + Caddyfile for automatic HTTPS.
- README rewritten for public readers (Docker first, sources/copyright note,
  docs index); AGENTS.md + CLAUDE.md for contributors and coding agents.
- Removed notes about one particular installation (Windows host, handoffs,
  audits of a private library, benchmarks with local paths, wishlist, reading
  map) and the unused ops/proxmox templates; generalized remaining mentions of
  hosts, VPN and Windows-only commands.
- Defaults: TZ and reading time zone UTC; .gitattributes keeps shell/YAML LF.
- Personal agent coordination files are git-ignored (CLAUDE.local.md,
  .claude/skills/manga-org/).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 11:09:00 +02:00
MaashkinoozheandClaude Opus 5.5 b2bb34174c Fold the sentinel into the scraper and drop the sentinel service (BL-0046)
The stall watchdog now runs inside the scraper (ScraperService.checkStalls,
every 30 s from the work loop): same rule as before, one checkpoint-safe
restart, quarantine on the second stall, missing checkpoint is no evidence,
and paused time no longer counts. Stalls are recorded as scraper incidents
(failure.kind 'stall'). The private /api/recovery endpoint is gone.

Dropped with the service: reader/scraper-unreachable incidents (supervisor
and Docker handle crashes; the health page shows "Scraper: unreachable"),
duplicate job-failure incidents and the unused /metrics endpoint.
The stack is now reader + scraper (+ optional peer-gateway).
The supervisor test moved to runtime/test (npm run test:runtime stays).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 22:58:03 +02:00
Codex b96ab42add Allow queued confirmation during source pause 2026-10-14 09:42:21 +02:00
Codex a22913129a Preserve per-source pacing across queue jobs and add source pause 2026-10-14 09:23:04 +02:00
Codex a08e847932 Add first-party MangaPill source and distinct volume releases 2026-10-14 09:05:34 +02:00
Codex 6d713e0a8c Keep source plugins in single scraper process 2026-10-13 22:42:20 +02:00
Codex 47de081721 Checkpoint peering identity and opt-in discovery 2026-10-13 22:00:54 +02:00
Codex 1c61686c9b Checkpoint opt-in MangaDex parser worker 2026-10-13 21:57:59 +02:00
Codex cc2fd05062 Document and package opt-in source worker 2026-10-13 21:52:22 +02:00
Maashkinoozhe eef4ffa07c Consolidate MangaCircus app, scraper, and Compose baseline 2026-10-07 23:28:20 +02:00