Fold the sentinel into the scraper and drop the sentinel service (BL-0046)

The stall watchdog now runs inside the scraper (ScraperService.checkStalls,
every 30 s from the work loop): same rule as before, one checkpoint-safe
restart, quarantine on the second stall, missing checkpoint is no evidence,
and paused time no longer counts. Stalls are recorded as scraper incidents
(failure.kind 'stall'). The private /api/recovery endpoint is gone.

Dropped with the service: reader/scraper-unreachable incidents (supervisor
and Docker handle crashes; the health page shows "Scraper: unreachable"),
duplicate job-failure incidents and the unused /metrics endpoint.
The stack is now reader + scraper (+ optional peer-gateway).
The supervisor test moved to runtime/test (npm run test:runtime stays).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MaashkinoozheandClaude Opus 5.5 committed 2026-10-22 22:58:03 +02:00
1 parent 88555d4780
commit b2bb34174c
37 files changed
+163 -311

No files matched your search

+3 -3
View File
@@ -12,7 +12,7 @@ This is a discovery and curation feature, not a new image scraper, metadata auth
Current MangaCircus constraints remain: plain Node and browser JavaScript, Docker Compose, a folder-tree library, JSON state rather than a database, one reader owner of account state, one scraper owner of the queue and library publication, and source-specific slow request pacing. See [`docs/architecture.md`](../docs/architecture.md), [`docs/greatest.md`](../docs/greatest.md), and [`docs/scraper-service.md`](../docs/scraper-service.md).
MVP lives in the **reader service** as a bounded asynchronous job and a reader-owned JSON store. Do not add a permanently running fourth service at first. If real LLM or metadata latency later makes reader isolation necessary, move the job behind the same API to a one-off/optional worker container using the existing image; do not change the data contract or add a database merely to do that. Sentinel observes availability but does not judge recommendations, call an LLM, or repair research claims.
MVP lives in the **reader service** as a bounded asynchronous job and a reader-owned JSON store. Do not add a permanently running third service at first. If real LLM or metadata latency later makes reader isolation necessary, move the job behind the same API to a one-off/optional worker container using the existing image; do not change the data contract or add a database merely to do that. Service health reports availability but does not judge recommendations, call an LLM, or repair research claims.
Explicit non-goals for the first release:
@@ -134,7 +134,7 @@ Queueing remains through `POST /api/admin/scraper` with `command:"enqueue"` and
- One in-process run lock plus a persisted run ID prevents overlap across browser clicks and restarts. A stale interrupted run is marked interrupted on startup; keep the previous successful feed. Publish only after validation completes.
- Preserve source errors and diagnostics with bounded log size. Do not store sensitive prompts or external response bodies indefinitely. Provider blocks/challenges require user attention, not automated retries or evasions.
- Nightly scheduling comes after the manual path has passed acceptance. Make local-time schedule configurable, persist last-success time, skip duplicate same-day runs after restart, and allow a manual run without disturbing scraper nightly updates. A missed night can run once after startup, never a catch-up storm.
- The feature is optional: reader, library, Greatest, scraper, and Sentinel continue normally if metadata or LLM access is absent.
- The feature is optional: reader, library, Greatest, and scraper continue normally if metadata or LLM access is absent.
## 8. Phased implementation and acceptance
@@ -152,7 +152,7 @@ Add one configurable model adapter, schema validation, evidence-bound output, op
### Phase D — scheduling and operations
Add nightly schedule only after manual runs are stable. Expose last success/error and a `discovery_run_failed` incident to admin observability, but do not give Sentinel authority to rewrite recommendations or restart the scraper. Add backup/restore instructions for `data/discovery.json` and cache behavior. If runs measurably harm reader latency, move the job into an optional one-off Compose worker with one-writer handoff, not an always-on autonomous service.
Add nightly schedule only after manual runs are stable. Expose last success/error and a `discovery_run_failed` incident to admin observability, but do not give any watchdog authority to rewrite recommendations or restart the scraper. Add backup/restore instructions for `data/discovery.json` and cache behavior. If runs measurably harm reader latency, move the job into an optional one-off Compose worker with one-writer handoff, not an always-on autonomous service.
### Suggested implementation map