Add scoped peer access and reversible serving controls
This commit is contained in:
Codex committed
1 parent
6c994113c6
commit
492477f86a
7 files changed
+1081
No files matched your search
+138
@@ -0,0 +1,138 @@
|
||||
# Server peering (v1)
|
||||
|
||||
Status: implemented, tested in isolated local fixtures, and enabled for a local
|
||||
two-instance pilot on 2026-10-10. It has **not** been deployed to the VM. See the broader design and
|
||||
future ideas in [`vision/server-peering.md`](../vision/server-peering.md).
|
||||
|
||||
Peering is an opt-in, pull-only transfer between MangaCircus servers. It does not
|
||||
exchange accounts, reading history, ratings, notes, or scraper credentials. The
|
||||
publishing gateway reads a library mount, and only the existing scraper publishes
|
||||
received chapters. No database server or second library writer is introduced.
|
||||
|
||||
## Roles and direction
|
||||
|
||||
The admin UI is **Settings → Peering**, and all users can open the **Peers** catalog
|
||||
page. Each server may have many peers. For every peer, its admin independently sets:
|
||||
|
||||
- **Serve approved manga**: expose only individually checked library titles to that
|
||||
peer. Off is the default and invalidates the current serving grant.
|
||||
- **Allow downloads**: queue and pull chapters from that peer. Off is the default;
|
||||
a verified pairing and imported grant still allow browsing its approved catalog.
|
||||
- **Let readers queue**: ordinary readers may request transfers; otherwise only an
|
||||
admin may queue. Browsing a permitted catalog does not disclose the local user to
|
||||
the remote server.
|
||||
- Upload streams, bytes/s and bytes/day, and receiving pages in flight and bytes/s.
|
||||
Defaults are two in-flight pages and 2 MiB/s in each direction per peer. The
|
||||
publisher's caps remain authoritative. A gateway-wide four-stream ceiling applies.
|
||||
|
||||
Serving and receiving are separate directions. Pairing by itself exposes no title.
|
||||
|
||||
## Deployment
|
||||
|
||||
The optional `peer-gateway` Compose profile publishes port 8093 on **127.0.0.1** by
|
||||
default. It uses the same image as reader/scraper, a read-only library mount, a
|
||||
read-only peering policy/identity mount, and its own writable usage-counter mount.
|
||||
The gateway needs an external-facing Docker network for its host port to work on
|
||||
Docker Desktop; it also joins the private backend. It has no reader account database,
|
||||
scraper queue, Docker socket, or writable library mount. It must never be scaled
|
||||
past one instance for the same usage-counter directory.
|
||||
|
||||
1. Build the image and start the normal stack as documented in
|
||||
[`compose-deployment.md`](compose-deployment.md). Generate the local certificate
|
||||
in **Settings → Peering**. Keep `/data/peering/identity.key` private and backed up.
|
||||
2. Start the optional service with `docker compose --profile peering up -d
|
||||
peer-gateway`. Verify `docker compose --profile peering ps` shows healthy and
|
||||
`https://127.0.0.1:8093/health` responds. `/health` is deliberately unauthenticated
|
||||
and contains only `{ "ok": true }`.
|
||||
3. Give the other server a reachable HTTPS endpoint for port 8093. A private
|
||||
Tailscale TCP forward is recommended for the pilot; the default loopback binding
|
||||
is not remotely reachable. Do not expose port 8093 on the public internet merely
|
||||
to make pairing work. Use a stable tailnet DNS name, not a changing IP. Keep
|
||||
scraper (8091) and Sentinel (8092) private.
|
||||
|
||||
The `PEER_BIND`, `PEER_PORT`, `PEER_DATA_PATH`, and `PEER_USAGE_PATH` Compose variables
|
||||
control the bind and persistent paths. The gateway's internal port remains 8093.
|
||||
OpenSSL is bundled in the app image for one-time P-256 self-signed certificate
|
||||
generation. Each certificate is valid for one year. Expired/replaced identities
|
||||
require administrator-supervised re-pairing; there is no automatic key rotation yet.
|
||||
|
||||
## Pairing and transfer
|
||||
|
||||
1. Each admin generates an identity, creates a 48-hour invitation, and exchanges the
|
||||
JSON outside the app. The JSON contains a public certificate, SHA-256 fingerprint,
|
||||
URL, invitation ID and one-use code—**not** the private key. Compare the
|
||||
fingerprints through a separate trusted channel.
|
||||
2. Both admins import the other's invitation and press **Test & verify pairing**.
|
||||
The HTTPS handshake presents client and server certificates, checks their exact
|
||||
pins and dates, and consumes the one-use invitation code. Both sides must probe.
|
||||
3. A publisher checks specific titles under that peer, enables **Serve**, issues a
|
||||
90-day random grant, and sends the token privately to the receiving admin. The
|
||||
receiving admin imports the grant and enables **Allow downloads**. Reverse
|
||||
direction needs its own share selection and grant. The server stores only a hash
|
||||
of a grant it issued; the receiver stores its imported token server-side.
|
||||
4. A local user browses the catalog, selects missing/newest/last-five or an exact
|
||||
chapter, and queues it through the normal scraper. The peer adapter downloads up
|
||||
to the configured pages in flight, validates size and SHA-256, resumes a partial
|
||||
page with HTTP Range, checkpoints between batches, and atomically publishes each
|
||||
complete chapter into the ordinary library. Existing-chapter policy is applied
|
||||
before download. Partial transfers remain in the hidden scraper work folder.
|
||||
|
||||
The publisher sees the receiving **server** certificate and transfer requests, not
|
||||
the human reader. It can still observe requested titles, times and byte counts.
|
||||
|
||||
## Peer v1 HTTP interface
|
||||
|
||||
All content endpoints require TLS 1.3 client certificate pinning, a paired/serving
|
||||
peer, and a currently valid certificate-bound bearer grant. There is no insecure
|
||||
HTTP fallback and no path-based file browsing.
|
||||
|
||||
| Request | Response |
|
||||
| --- | --- |
|
||||
| `GET /peer/v1/hello` with one-use invitation headers | Protocol version and server identity fingerprint; no catalog |
|
||||
| `GET /peer/v1/catalog?cursor=N` | Up to 50 approved opaque title IDs, title/year/status, next cursor |
|
||||
| `GET /peer/v1/manga/{id}?cursor=N` | Up to 100 complete chapter IDs and light metadata |
|
||||
| `GET /peer/v1/manga/{id}/chapters/{id}` | Ordered page IDs, MIME, byte size, SHA-256 and ETag |
|
||||
| `GET /peer/v1/blobs/{id}` | One image, optional `Range: bytes=N-` and `If-Match`; 206 on resume |
|
||||
|
||||
Unknown/unshared IDs return 404. Revoked serving returns 403; bad/missing grants
|
||||
return 401; caps return 429 and `Retry-After`. No source-site URL, local filesystem
|
||||
path, or user record is sent in protocol responses. Individual complete chapter
|
||||
manifests are discovered from final `chapter.json` folders; an unfinished folder is
|
||||
not shared. Catalog and chapter lists are short-lived in-memory caches. Turning off
|
||||
serving blocks subsequent requests immediately, though an already streaming image
|
||||
may finish its current chunk/response.
|
||||
|
||||
The initial implementation sends original image bytes; it does not transcode them.
|
||||
See [`vision/server-peering-compression-test.md`](../vision/server-peering-compression-test.md)
|
||||
for the measured reason. The transport accepts a small amount of controlled page
|
||||
parallelism, not unbounded connections.
|
||||
|
||||
## Tests and safe operation
|
||||
|
||||
`npm run test:peer` creates disposable A/B identities, gateways and libraries and
|
||||
tests mTLS pairing, no-cert/bad-grant rejection, private catalog, per-title sharing,
|
||||
manifest hashes, a resumed range response, scraper publication, duplicate skip and
|
||||
revocation. `npm test` includes it. An isolated Compose acceptance is:
|
||||
|
||||
```powershell
|
||||
$env:APP_IMAGE = 'mangacircus:local' # replace with the image just built
|
||||
$env:QA_PORT = '8188'
|
||||
$env:QA_PEER_PORT = '8193'
|
||||
node tools/test-compose.js --peer-gateway --peer-ui
|
||||
```
|
||||
|
||||
It makes disposable library/data, four containers, a certificate and a browser
|
||||
session, then tears them down. It never mounts the live library/queue.
|
||||
|
||||
For live rollout, wait for a scraper checkpoint, back up the library and data
|
||||
(including the peering identity, policy, outbound grants, gateway usage and hidden
|
||||
work folders), and deploy reader/scraper/gateway from one immutable image digest.
|
||||
There must remain exactly one queue-owning scraper and one reader database writer.
|
||||
Peering is optional: stop the `peer-gateway` profile to close incoming service
|
||||
without affecting reading or existing local downloads. Do not delete the identity
|
||||
to recover from a network error; that would break every peer pin.
|
||||
|
||||
Current limits: no automatic certificate renewal, invitation cancellation UI,
|
||||
peer cover-image endpoint, multi-peer performance pilot, VM deployment, or public
|
||||
internet exposure review. The gateway has no automated hoster-interaction logic;
|
||||
source scraping remains subject to the existing source-safety policy.
|
||||
Reference in new issue
Block a user