Commit Graph
20 Commits
Author SHA1 Message Date
MaashkinoozheandClaude Opus 5.5 57a571d2f0 WeebCentral and MangaPill are off by default, with an admin switch per source
Both are unofficial sites that mostly host unlicensed copies; MangaDex (official
API) stays on. Settings → Admin → Scraper → Hoster settings has a 'Use <source>'
switch. A switched-off source refuses new jobs, keeps queued jobs waiting and is
skipped by nightly update checks. Installations whose queue or history already
used a source keep it switched on, so upgrades don't stop a working queue.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-25 07:47:47 +01:00
MaashkinoozheandClaude Opus 5.5 7b52da2f9c Official release dates from MangaUpdates; peers lower ecosystem dates (BL-0048, BL-0049)
- scraper/metadata/release-dates.js fills x-dates.release from MangaUpdates
  publisher releases. MangaUpdates also lists English back-catalog re-releases,
  so a date is kept only inside the AniList original run, when no scanlation or
  source upload is more than 4 days earlier, and when it fits the release order
  (no lower chapter dated after a higher one). Seasons that restart chapter
  numbers are skipped; dates this job wrote and no longer accepts are withdrawn.
- The scraper refreshes ongoing series nightly before its update checks, only
  while no download runs, and bumps the publish revision so the reader reconciles.
- tools/release-dates.js for a whole library (dry run default). Production dry
  run: 728 chapters dated (Kagurabachi, JJK, Chainsaw Man, One Piece from 928).
- Peer sync: chapters this library already has adopt a peer's earlier ecosystem
  date (job result ecosystemLowered); the peer listing now carries dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 08:47:36 +02:00
MaashkinoozheandClaude Opus 5.5 e0dacd81c6 Chapter dates: read date setting and tiered release dates (BL-0047)
- Chapter rows show when you first finished a chapter (setting: date + hour,
  date, off) from a new durable progress doneAt; older reads fall back to the
  reading clock's earliest remembered completion.
- chapter.json keeps added (arrival in this library, now kept on replace) plus
  x-dates.source (source upload, bulk imports flagged), x-dates.release
  (official, for a later metadata job) and x-dates.ecosystem (first entry into
  any Manga Circus library; carried over peering, earliest wins, clamped).
- Rows show one release date by tier: official > source upload > ecosystem >
  library, with all known dates in the tooltip (setting to hide).
- Fixes: WeebCentral dates came from the chapter page, which has none (every
  chapter got the download time); they now come from the full chapter list.
  MangaDex no longer writes a group's publishAt into added. Peering sends dates.
- tools/backfill-chapter-dates.js migrates an existing library (dry run default).
- Coverage dry run on the server: docs/chapter-dates-coverage-2026-10-24.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 01:01:33 +02:00
MaashkinoozheandClaude Opus 5.5 b2bb34174c Fold the sentinel into the scraper and drop the sentinel service (BL-0046)
The stall watchdog now runs inside the scraper (ScraperService.checkStalls,
every 30 s from the work loop): same rule as before, one checkpoint-safe
restart, quarantine on the second stall, missing checkpoint is no evidence,
and paused time no longer counts. Stalls are recorded as scraper incidents
(failure.kind 'stall'). The private /api/recovery endpoint is gone.

Dropped with the service: reader/scraper-unreachable incidents (supervisor
and Docker handle crashes; the health page shows "Scraper: unreachable"),
duplicate job-failure incidents and the unused /metrics endpoint.
The stack is now reader + scraper (+ optional peer-gateway).
The supervisor test moved to runtime/test (npm run test:runtime stays).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-22 22:58:03 +02:00
Codex edbc9c6056 Handle large WeebCentral catalogs and correct MangaPill checkpoint monitoring 2026-10-14 14:01:05 +02:00
Codex b96ab42add Allow queued confirmation during source pause 2026-10-14 09:42:21 +02:00
Codex a22913129a Preserve per-source pacing across queue jobs and add source pause 2026-10-14 09:23:04 +02:00
Codex a08e847932 Add first-party MangaPill source and distinct volume releases 2026-10-14 09:05:34 +02:00
Codex a8df57f165 Recognize additional WeebCentral chapter labels and record source pilot 2026-10-13 23:58:33 +02:00
Codex 6d713e0a8c Keep source plugins in single scraper process 2026-10-13 22:42:20 +02:00
Codex 0fc1a7b52a Mark sparse MangaDex hosted catalogs as partial 2026-10-13 22:01:57 +02:00
Codex 47de081721 Checkpoint peering identity and opt-in discovery 2026-10-13 22:00:54 +02:00
Codex 1c61686c9b Checkpoint opt-in MangaDex parser worker 2026-10-13 21:57:59 +02:00
Codex 93f07e838e Fix pinned lookup for dual-stack HTTPS requests 2026-10-13 21:53:59 +02:00
Codex f4123b32e4 Checkpoint guarded source request broker 2026-10-13 21:51:03 +02:00
Codex 6599d0a244 Validate first-party parser boundaries 2026-10-13 21:47:18 +02:00
Codex c5a92282a9 Checkpoint first-party source parser pilot 2026-10-13 21:45:27 +02:00
Codex 26d6cd59fc Repair metadata title on queue retry 2026-10-11 09:09:37 +02:00
Codex a500ea7c3f Use inspected manga title for metadata lookup 2026-10-11 09:07:42 +02:00
Maashkinoozhe eef4ffa07c Consolidate MangaCircus app, scraper, and Compose baseline 2026-10-07 23:28:20 +02:00