MangaCircus/docs/peering.md

Server peering (v1)

Status: implemented, tested in isolated local fixtures, and enabled for a local two-instance pilot on 2026-10-10. It has not been deployed to the VM. See the broader design and future ideas in vision/server-peering.md.

Peering is an opt-in, pull-only transfer between MangaCircus servers. It does not exchange accounts, reading history, ratings, notes, or scraper credentials. The publishing gateway reads a library mount, and only the existing scraper publishes received chapters. No database server or second library writer is introduced.

Roles and direction

The interface is administrator-only and split by task: Settings → Admin → Peering manages server identity, peers, grants and transfer limits; Peer catalogs manages per-peer title approvals and remote catalog browsing. Ordinary users cannot see either page or call their APIs. Each server may have many peers.

• Grant scopes: Catalog permits catalog, covers and metadata. Download includes Catalog and permits chapter transfers. Each direction has its own grant.
• Serving state: Active or Paused. Pausing preserves the grant/token; explicit Revoke invalidates it. Scope changes take effect without exchanging a new token.
• Approved titles: the publisher chooses titles on Peer catalogs, not in the connection settings. Pairing or a grant alone does not share a title.
• Local receiving is a separate switch. Browsing works with Catalog permission even when local receiving is off; downloads require an active Download grant and local receiving enabled.
• Transfer limits cover upload streams, bytes/s and bytes/day, plus receiving pages in flight and bytes/s. Defaults are two in-flight pages and 2 MiB/s each direction per peer. Publisher caps remain authoritative; a gateway-wide four-stream ceiling applies.

The required local server display name is 1–25 ASCII letters/digits. It is a friendly label, not the cryptographic identity; changing it does not rotate the certificate. An invitation expires after 48 hours, the certificate currently lasts one year, and a grant lasts up to 90 days bounded by certificate validity. The UI labels these as separate lifetimes.

Deployment

The optional peer-gateway Compose profile publishes port 8093 on 127.0.0.1 by default. It uses the same image as reader/scraper, a read-only library mount, a read-only peering policy/identity mount, and its own writable usage-counter mount. The gateway needs an external-facing Docker network for its host port to work on Docker Desktop; it also joins the private backend. It has no reader account database, scraper queue, Docker socket, or writable library mount. It must never be scaled past one instance for the same usage-counter directory.

1. Build the image and start the normal stack as documented in compose-deployment.md. Set the local server name and generate its certificate in Settings → Admin → Peering. Keep /data/peering/identity.key private and backed up.
2. Start the optional service with docker compose --profile peering up -d peer-gateway. Verify docker compose --profile peering ps shows healthy and https://127.0.0.1:8093/health responds. /health is deliberately unauthenticated and contains only { "ok": true }.
3. Give the other server a reachable HTTPS endpoint for port 8093. A private Tailscale TCP forward is recommended for the pilot; the default loopback binding is not remotely reachable. Do not expose port 8093 on the public internet merely to make pairing work. Use a stable tailnet DNS name, not a changing IP. Keep the scraper (8091) private.

The PEER_BIND, PEER_PORT, PEER_DATA_PATH, and PEER_USAGE_PATH Compose variables control the bind and persistent paths. The gateway's internal port remains 8093. OpenSSL is bundled in the app image for one-time P-256 self-signed certificate generation. Each certificate is valid for one year. Expired/replaced identities require administrator-supervised re-pairing; there is no automatic key rotation yet.

Pairing and transfer

1. Each admin generates an identity, creates a 48-hour invitation, and exchanges the JSON outside the app. The JSON contains a public certificate, SHA-256 fingerprint, URL, invitation ID and one-use code—not the private key. Compare the fingerprints through a separate trusted channel.
2. Both admins import the other's invitation and press Test & verify pairing. The HTTPS handshake presents client and server certificates, checks their exact pins and dates, and consumes the one-use invitation code. Both sides must probe.
3. A publisher approves titles for that peer on Peer catalogs, chooses Catalog or Download grant scopes, and sends the one-time token privately. The receiving admin imports it and independently enables local receiving for downloads. Reverse direction needs its own approvals and grant. The server stores only a hash of a grant it issued; the receiver stores its imported token server-side.
4. An admin browses the permitted catalog, selects missing/newest/last-five or an exact chapter, and queues it through the normal scraper. The queue carries a validated stable work identity: AniList first, then MangaDex/WeebCentral IDs, then another known hoster ID. If none exists, the owning server supplies an opaque origin ID that follows the work across peers without exposing its folder name. Imports retain that identity, so a round trip resolves to the original library title; sync missing skips chapters already present and fetches only additions. Titles alone are never used to merge works. The peer adapter downloads up to the configured pages in flight, validates size and SHA-256, resumes a partial page with HTTP Range, checkpoints between batches, and atomically publishes each complete chapter into the ordinary library. Existing-chapter policy is applied before download. Partial transfers remain in the hidden scraper work folder.

The publisher sees the receiving server certificate and transfer requests, not the human reader. It can still observe requested titles, times and byte counts.

Peer v1 HTTP interface

Catalog, cover, manifest and blob endpoints require TLS 1.3 client certificate pinning, paired identity, an active grant with the matching scope, and no serving pause. There is no insecure HTTP fallback or path-based file browsing. /access requires the pinned client certificate and reports status without a bearer grant so an admin can distinguish pairing from permission.

Request	Response
GET /peer/v1/hello with one-use invitation headers	Protocol version and server identity fingerprint; no catalog
GET /peer/v1/access	Pairing state, certificate expiry and current grant status/scopes; no catalog
GET /peer/v1/catalog?cursor=N	Up to 50 approved opaque title IDs, safe metadata, a stable work identity and cover availability
GET /peer/v1/covers/{mangaId}	Approved cover image only; bounded to 5 MiB and charged to peer byte quota
GET /peer/v1/manga/{id}?cursor=N	Up to 100 complete chapter IDs, public metadata and the same stable work identity
GET /peer/v1/manga/{id}/chapters/{id}	Ordered page IDs, MIME, byte size, SHA-256 and ETag
GET /peer/v1/blobs/{id}	One image, optional Range: bytes=N- and If-Match; 206 on resume

Unknown/unshared IDs return 404. Revoked serving returns 403; bad/missing grants return 401; caps return 429 and Retry-After. No source-site URL, local filesystem path, or user record is sent in protocol responses. Individual complete chapter manifests are discovered from final chapter.json folders; an unfinished folder is not shared. Catalog and chapter lists are short-lived in-memory caches. Turning off serving blocks subsequent requests immediately, though an already streaming image may finish its current chunk/response.

The initial implementation sends original image bytes; it does not transcode them. See vision/server-peering-compression-test.md for the measured reason. The transport accepts a small amount of controlled page parallelism, not unbounded connections.

Tests and safe operation

npm run test:peer creates disposable A/B identities, gateways and libraries and tests mTLS pairing, no-cert/bad-grant rejection, private catalog, per-title sharing, catalog-only cover access, manifest hashes, a resumed range response, scraper publication, duplicate skip, pause and revocation. npm run test:peer-ab additionally builds two independent Compose projects from synthetic fixtures, exercises real two-way pairing, browsing, cover retrieval, permission updates and sequential queueing, checks the queue remains paused, and tears both stacks down. The UI suite checks the guided setup, approvals, and non-admin boundary. An isolated single-stack Compose acceptance is:

$env:APP_IMAGE = 'mangacircus:local' # replace with the image just built
$env:QA_PORT = '8188'
$env:QA_PEER_PORT = '8193'
node tools/test-compose.js --peer-gateway --peer-ui

It makes disposable library/data, four containers, a certificate and a browser session, then tears them down. It never mounts the live library/queue.

For live rollout, wait for a scraper checkpoint, back up the library and data (including the peering identity, policy, outbound grants, gateway usage and hidden work folders), and deploy reader/scraper/gateway from one immutable image digest. There must remain exactly one queue-owning scraper and one reader database writer. Peering is optional: stop the peer-gateway profile to close incoming service without affecting reading or existing local downloads. Do not delete the identity to recover from a network error; that would break every peer pin.

Current limits: no automatic certificate renewal, invitation cancellation UI, multi-peer performance pilot, VM deployment, or public internet exposure review. The gateway has no automated hoster-interaction logic; source scraping remains subject to the existing source-safety policy.