# Adding a source to the single scraper

MangaCircus uses **one scraper process**. A source module is reviewed first-party
code bundled into that process, not a container or a downloaded extension.
`scraper/adapters/index.js` is the explicit registration point. The library,
queue, pacing settings, checkpointing, source-error policy, and publication
remain shared infrastructure. Do not add another queue writer for a new hoster.

## Source boundary

| Concern | Location | New-source responsibility |
| --- | --- | --- |
| URL recognition and adapter contract | `scraper/sources/<hoster>.js` | `id`, `label`, `match`, `discover`, `run` |
| Catalog and chapter discovery | `scraper/adapters/<hoster>.js` | Map source responses to stable chapter IDs, metadata and page URLs |
| Source HTML/API parsing | `scraper/plugins/<hoster>.js` | Pure functions; no network or filesystem writes |
| Parser operation/URL validation | `scraper/plugins/manifest.js`, `registry.js`, `validation.js` | Allow only known operations and expected hosts/links |
| Resumable archive workflow | `scraper/jobs/<hoster>-archive.js` | Use the common publisher, inventory, pacing and checkpoints |
| Registration | `scraper/adapters/index.js` | Add one explicit `require` to `adapters` |

The module contract is shown in [scraper-service.md](scraper-service.md#adapter-contract).
Start from the closest existing hoster. `match(url)` returns `null` for unrelated
URLs and `{ source, id, title, url }` for an exact supported series URL. `discover`
must be read-only and return a catalog. `run` must honor scope, existing-content
policy, `shouldStop`, source-specific delays, and publication callbacks. Stable
source series/chapter IDs are essential for retries and duplicate detection.

## Before enabling a hoster

1. Document its permitted access method, terms and source-specific pacing. Do
   not work around 401, 403, 429, CAPTCHA, challenges or source protections.
2. Add fixture-backed tests for URL matching, catalog, empty/partial catalogs,
   chapter pages, unsafe links, and duplicate release handling. No test should
   contact the live hoster by default.
3. Test pause/resume from a partial chapter and an existing library target with
   the common publisher; verify one source lane and no parallel image workers.
4. Run `npm run test:scraper` and an isolated Compose smoke test. Review and deploy
   the single scraper image with a checkpoint-safe procedure; never start another
   daemon against the live queue.

Removing a hoster from registration stops new matches; it does **not** delete its
library chapters or queued/checkpointed work. Inspect affected queue items and
preserve checkpoints before disabling a source in a release.