Fold the sentinel into the scraper and drop the sentinel service (BL-0046)
The stall watchdog now runs inside the scraper (ScraperService.checkStalls, every 30 s from the work loop): same rule as before, one checkpoint-safe restart, quarantine on the second stall, missing checkpoint is no evidence, and paused time no longer counts. Stalls are recorded as scraper incidents (failure.kind 'stall'). The private /api/recovery endpoint is gone. Dropped with the service: reader/scraper-unreachable incidents (supervisor and Docker handle crashes; the health page shows "Scraper: unreachable"), duplicate job-failure incidents and the unused /metrics endpoint. The stack is now reader + scraper (+ optional peer-gateway). The supervisor test moved to runtime/test (npm run test:runtime stays). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
88555d4780
commit
b2bb34174c
37 files changed
+163
-311
No files matched your search
+11
-11
@@ -28,11 +28,10 @@ target private.
|
||||
### Why the reader owns it
|
||||
|
||||
The reader is the only service that mounts all of `/data` (the scraper sees only
|
||||
`/data/scraper`, Sentinel only `/data/sentinel`), it holds `db.json` in memory and owns
|
||||
its atomic writer, it already talks to the scraper's private control API with the
|
||||
shared token, and the admin settings UI lives there. Running it in Sentinel would need
|
||||
a full read-write `/data` mount there and break its "writes only `data/sentinel/`"
|
||||
contract (see [architecture.md §3](architecture.md#3-components)).
|
||||
`/data/scraper`), it holds `db.json` in memory and owns its atomic writer, it already
|
||||
talks to the scraper's private control API with the shared token, and the admin
|
||||
settings UI lives there. Running it in the scraper would need a full read-write `/data`
|
||||
mount there (see [architecture.md §3](architecture.md#3-components)).
|
||||
|
||||
## Settings and status
|
||||
|
||||
@@ -67,9 +66,10 @@ POST /api/admin/backup {command:"run"} -> 409 when off or already running
|
||||
- `db.json` is serialized from the reader's in-memory state at that moment.
|
||||
- `scraper/queue.json` is stored with `paused: true` (status `paused`), so a
|
||||
restored copy never starts downloading by itself. The live file is not touched.
|
||||
- `scan-cache.json`, `library-index.json`, `catalog.json`, peering, Sentinel state,
|
||||
logs and everything else under `DATA` is copied as found; all of them are
|
||||
written by atomic rename, so each file is complete.
|
||||
- `scan-cache.json`, `library-index.json`, `catalog.json`, peering, logs and
|
||||
everything else under `DATA` is copied as found; all of them are written by
|
||||
atomic rename, so each file is complete. A leftover `sentinel/` folder from
|
||||
before 2026-10-22 is copied too; it is unused and can be deleted.
|
||||
- Left out: `runtime*.json`, `*.lock` (`daemon.lock`, `supervisor.lock`), `*.pid`,
|
||||
`*.tmp`, symlinks, and the `BACKUP` folder itself if it lies inside `DATA`.
|
||||
- `.circus-backup.json` at the archive root records when and how it was made.
|
||||
@@ -93,8 +93,8 @@ plus those seconds.
|
||||
Restoring replaces the user data with the state of the archive. Reading progress
|
||||
and queue changes made after that backup are lost.
|
||||
|
||||
1. Stop the stack (or at least reader, scraper and Sentinel):
|
||||
`docker compose stop reader scraper sentinel`.
|
||||
1. Stop the stack (or at least reader and scraper):
|
||||
`docker compose stop reader scraper`.
|
||||
2. Move the current data folder aside instead of deleting it:
|
||||
`mv /srv/mangacircus/data /srv/mangacircus/data.before-restore`.
|
||||
3. Unpack the archive into an empty data folder owned by uid 1000:
|
||||
@@ -107,7 +107,7 @@ and queue changes made after that backup are lost.
|
||||
4. Start the reader first: `docker compose up -d --no-build reader`. Check `/health`,
|
||||
log in, check progress and library. The reader rebuilds a missing or stale library
|
||||
index by itself.
|
||||
5. Start `scraper sentinel`. The restored queue is paused: check it under
|
||||
5. Start `scraper`. The restored queue is paused: check it under
|
||||
Settings → Scraper and resume it when it looks right.
|
||||
|
||||
The scraper token is not in `DATA`; Compose keeps it under `.secrets/` (or the
|
||||
|
||||
Reference in new issue
Block a user