Maashkinoozhe/MangaCircus
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
MangaCircus/docs/architecture.md
T
MaashkinoozheandClaude Opus 5.5 fe8956087e Reading activity grid in Settings: months, weeks, days, hours with drill-down 
Active reader time is now kept per user in small UTC buckets (6 minutes for
120 days, then hours, dropped after three years), seeded once from the visits
the clock still remembers. Settings -> Reading activity shows them as a grid
whose cells glow with their total and carry a start-to-end strip; month/week
cells open their days, days open their hours. Finished chapters come from
progress and work without the clock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 12:41:44 +01:00

458 lines
40 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# MangaCircus architecture
MangaCircus is a self-hosted reader for manga, comics and webtoons. The reader owns
accounts/progress and a separate collection catalog. The scraper owns the persistent
queue and chapter publication, and its built-in stall watchdog requeues a stuck job
once at a checkpoint. Separate supervisors isolate service failures. All content remains
a plain folder tree; there is no database server or embedded autonomous LLM.
Related documents:
- [library-format.md](library-format.md): the folder format.
- [import.md](import.md): how the folder tree is indexed.
- [scraper-guide.md](scraper-guide.md): rules for tools that write the library.
- [scraper-service.md](scraper-service.md): managed queue, private API, and adapter contract.
- [collections.md](collections.md): relations, groups, curator UI and agent research.
- [resilience.md](resilience.md): supervision, recovery limits and incidents.
- [deployment.md](deployment.md): Docker deployment, HTTPS, operation, updates and backups.
- [settings-navigation.md](settings-navigation.md): User/Admin workspace, scraper tabs and deep links.
- [peering.md](peering.md): optional private server-to-server catalog and transfer.
- [discovery.md](discovery.md): reader-owned, opt-in deterministic discovery feed and its limits.
- [source-modules.md](source-modules.md): adding and maintaining first-party hosters inside the single scraper process.
- [backup.md](backup.md): built-in scheduled backup of `data/` and the restore procedure.
## 1. Principles
| Principle | Consequence |
|-----------|-------------|
| **The file system is the source of truth for content** | No import step, no copies, no content database. The library can be mounted read-only. Anything that writes files can feed it |
| **As dumb as possible** | No dependencies (`package.json` has none), no framework, no build step, no transpiler. Plain ES2019+ JavaScript in the browser, plain Node on the server |
| **Small independent services** | The reader stays dependency-free; the optional scraper daemon can stop or restart without taking reading offline |
| **Readable without the app** | The folder format works in any picture viewer. The app adds progress, layout and convenience, not lock-in |
## 2. System context
```
┌──────────────┐ writes folders (staged + renamed)
scraper / ───▶│ library/ │◀──────────────────────────────────┐
person │ (read-only │ │
│ for the app)│ │
└──────┬───────┘ │
│ readdir / stat / small reads │
┌──────▼─────────────────────────────┐ │
browser ─────▶│ node server.js │ webhook │
(desktop, │ ├ library.js scanner → index │────────────▶ ntfy / Gotify / …
phone, PWA) │ ├ HTTP: / /lib/… /api/… │ (optional)
│ └ data/db.json data/scan-cache.json│
└─────────────────────────────────────┘
```
## 3. Components
Runtime topology (2026-10-22): native `tools/start-all.js` launches reader and scraper
independently. In Compose, each container runs `runtime/launch.js <service>`
and exactly one service child. A failure does not stop unrelated services. Exactly one
reader and queue-owning scraper may write each shared state tree; do not scale replicas.
The catalog and relation store is reader-owned `data/catalog.json`. No service has a
Docker socket or arbitrary code-repair authority.
The admin browser never talks to the scraper daemon directly. Admin-only reader
routes proxy commands to the daemon's private API, so its shared token stays on the
server. The daemon publishes complete folders atomically and increments a durable
revision; the reader polls that revision and refreshes changed manga after publication.
The reader restores a disposable full-index snapshot at startup so browsing need not
wait for a filesystem walk. Shallow directory stamps identify changed manga; a
daily full reconciliation catches in-place manual edits. The last full-scan time
survives restarts, so restarting does not itself force another complete walk.
| Path | Lines | Responsibility |
|------|------:|----------------|
| `server.js` | ~350 | HTTP server and router, auth and sessions, JSON storage, API handlers, scan scheduling, notifications and webhooks |
| `library.js` | ~170 | Folder scanner, name parsing, image-size reader, scan cache |
| `backup.js` | ~330 | Built-in `DATA` backup: queue pause/resume, tar.gz writer and verifier, schedule, retention |
| `public/common.js` | ~85 | Browser helpers: `h()` element builder, `api()` fetch wrapper, top navigation with the notification bell |
| `public/index.html` + `library.js` | ~60 | Library overview (cover grid, filter, sort) |
| `public/manga.html` + `manga.js` | ~110 | Manga page: info, follow, chapter list with book separators, versions and votes, read/unread toggle |
| `public/read.html` + `reader.js` | ~470 | The reader (§7.2) |
| `public/settings.html` + `settings.js` + `settings-nav.js` | — | User/Admin navigation, personal preferences, lazy admin sections and scraper tabs |
| `public/login.html` + `login.js` | ~50 | Sign-in, first-run admin creation, optional sign-up |
| `public/app.css` | ~150 | All styles (dark theme, responsive) |
| `tools/png.js`, `tools/make-dummy.js` | ~310 | Demo library generator: a tiny PNG encoder with a bitmap font, no image libraries |
| `Dockerfile`, `docker-compose.yml` | | `node:24-alpine`, non-root, `/library` + `/data` |
## 4. Runtime model
- **Reader process:** one event loop. All disk I/O in the scanner is asynchronous and sequential, so a scan never blocks requests.
- **Optional scraper process:** one queue worker, one remote request at a time, durable checkpoints, and a private control API.
- **In-memory state:**
- `lib` is the current library index, replaced as a whole after each scan.
- `db` is all user data, loaded from `db.json` at start.
- **Timers:**
- a saved-index restore and shallow check at start (full scan when no snapshot),
- a shallow check every `SCAN_INTERVAL` minutes, full reconciliation every `FULL_SCAN_INTERVAL` minutes,
- a debounced `db.json` write, 500 ms after the last change.
- **Shutdown:** `SIGINT` / `SIGTERM` flush `db.json`; the daemon cooperatively stops at a page boundary and keeps its archive checkpoint.
- **Durability:** every file write goes to `*.tmp` first and is then renamed over the target, so a file is never half-written. A hard kill loses at most the changes of the last ~0.5 s. A corrupt `db.json` makes the server **refuse to start** rather than start empty and overwrite it.
## 5. Data model
### 5.1 Content: the library index
The index is built by `library.js` from the folder tree (see [import.md §2.6](import.md#26-the-result)). It is never written anywhere, except for the scan cache.
### 5.2 Identity
Every reference from user data to content uses ids derived from names on disk:
| Entity | Id | Example | Stable under |
|--------|----|---------|--------------|
| Manga | manga folder name | `Moon Garden` | JSON edits, anything inside the folder |
| Book | normalised book number (`x` prefix for extras) | `1`, `x1` | title changes |
| Chapter | `<book key>/<chapter key>` | `1/2.5` | title changes, adding/removing versions |
| Version | chapter folder name | `c0002 Roots [MangaDex]` | nothing: renaming the folder creates a new version |
This keeps ids human-readable and derivable without a database. The price: renaming a manga folder, renumbering a chapter or moving it to another book creates a new entity (see [library-format.md → Renaming](library-format.md#renaming-what-survives)).
### 5.3 User data: `data/db.json`
```jsonc
{
"users": {
"admin": {
"name": "admin", "admin": true, "created": 1790796794000,
"hash": "<salt hex>:<scrypt hex>",
"settings": {
"startMode": "last", // "last" | "flow" | "page" | "book"
"lastMode": "book", // the user's intended mode; changed only by clicking a mode
"adapt": true, // use a fitting mode for incompatible content (webtoon → flow)
"skipExtras": false, "skipCovers": false,
"flowPageGap": true, // show the 8 px separator between pages in Flow
"readingClock": false, // opt in to credible per-chapter active-time tracking
"readingStory": true, // show private retrospective summaries when history exists
"continueRow": true, // library: "Continue where you left off" row at the top
"rereadDays": 7, // continue row: a later chapter finished this long before the head is left again (0 = off)
"rereadChapters": 5, // ... or one finished before a run of more than this many in-order chapters (0 = off)
"zoom": { "flow": 1, "page": 1.5, "book": 1, "strip": 0.8 }, // page zoom per mode
"webhook": "",
"readerPerSession": true // reader settings per login session (section 6.7)
},
"readerFrom": "3f9c…" // public id of the session the latest reader settings came from
}
},
"sessions": { "<sha256 of token>": { "u": "admin", "t": 1790796794000, "seen": 1790796794000,
"agent": "<User-Agent>", "reader": { …locked reader settings, only after a change… } } },
"progress": { "admin": { "Moon Garden": { "1/2": {
"v": "c0002 Roots [MangaDex]", // version read
"p": 4, "n": 14, // page index in that version (all pages), page count
"t": 1790796800000, // last activity
"done": false, // sticky once true (reset via "mark unread")
"shift": [] // book mode: run starts shown alone
} } } },
"reading": { "admin": { "Moon Garden": { "1/2": {
"schema": 1, "totalActiveMs": 412000, "loadingMs": 7400,
"sessions": 3, "completedPasses": 2,
"inputMs": { "mouse": 12000, "touch": 380000, "keyboard": 20000 },
"currentPass": { "version": "c0002 Roots [Official]", "seenRanges": [[0, 13]], "activeMs": 151000 }
} } } },
"follows": { "admin": { "Moon Garden": true } },
"notes": { "admin": [ { "id": "a1b2c3d4e5f6", "t": 1790797300000, "m": "Moon Garden", "c": "2/6",
"text": "Moon Garden · Ch. 6: Frost", "read": false } ] },
"moments": { "admin": [ { "id": "9f1c2b3a4d5e", "m": "Moon Garden", "c": "1/2", "v": "c0002 Roots [MangaDex]",
"i": [4], "f": ["0005.png"], "sha": ["<sha256 of the file>"], // 1–4 pages in displayed order
"layout": "row", // "row" side by side (Page/Book), "column" stacked (Flow)
"region": { "x": 0.1, "y": 0.5, "w": 0.5, "h": 0.25 }, // fractions of the pages put together; null = all
"cat": "awesome", "note": "", "t": 1790797400000 } ] },
"votes": { "Moon Garden": { "1/2": { "c0002 Roots [Official]": { "admin": 1, "reader": -1 } } } },
"known": { "Moon Garden": ["1/1", "1/2", "1/2.5", "..."] } // every chapter id ever seen (append-only)
}
```
- `data/scan-cache.json` holds `{ "<Manga>/<Book>/<Chapter>": { m: <folder mtimeMs>, p: [[file, w, h], …] } }`. It is disposable.
- `data/library-index.json` holds a disposable, versioned index snapshot and shallow
directory stamps. The library folders remain authoritative; a missing or invalid
snapshot triggers a full scan. Rollback-compatible `scan-cache.json` is retained.
- `data/backup.json` holds the built-in backup schedule, retention and recent results.
- **Backups:** the reader's built-in backup ([backup.md](backup.md)) archives the whole
`data/` tree into `BACKUP` on a schedule, with `db.json` taken from memory and the
queue paused around the copy. A plain copy of `data/db.json` also works: atomic
writes make it consistent at any time.
## 6. Server
### 6.1 Request routing
| Path | Handling |
|------|----------|
| `/api/*` | `routes["<METHOD> <path>"]`. Handlers get `{ u, b, q, req, res, sess, sk }` (`sess` = the login session record, `sk` its key) and return an object, which is sent as JSON. Throwing `HttpError(code, message, extra)` sends `{ error, ...extra }` with that status (for example `nextAllowedAt` on a discovery `429`) |
| `/lib/*` | Image files from `LIBRARY`. Requires a session, allows image extensions only, and the path must resolve inside `LIBRARY`. `ETag` + `private, max-age=86400` |
| everything else | Static files from `public/` (`/` → `index.html`). `ETag` + `no-cache` |
Pages are static. Each page's script calls the API and redirects to `login.html?next=…` on `401`. All URLs are relative, so the app also works under a sub-path behind a reverse proxy.
### 6.2 API
All requests and responses are JSON. POSTs require `Content-Type: application/json`.
| Method & path | Auth | Purpose |
|---------------|------|---------|
| `GET /api/auth` | public | `{ firstRun, signup }` for the login page |
| `POST /api/signup` `{name, password}` | public | First account (becomes admin), or anyone when `SIGNUP=open` |
| `POST /api/login` `{name, password}` | public | Starts a session (cookie `sid`) |
| `POST /api/logout` | user | Ends the session |
| `GET /api/me` | user | Name, admin flag, settings, unread count |
| `POST /api/settings` `{…partial}` | user | Validated update of the settings keys above; reader keys go to the current session (6.7) |
| `GET /api/reader-sessions` | user | `{ enabled, current, latestFrom, latest, sessions: [{ id, device, created, seen, current, locked, reader }] }` |
| `POST /api/reader-sessions` `{id, action}` | user | `follow`: the session drops its own reader settings and follows the latest; `latest`: its settings become the latest |
| `POST /api/password` `{old, new}` | user | Change password |
| `GET /api/library` | user | Manga list with covers, counts, the user's read count, unread notifications, last read, `left` (chapters after the newest saved position that are unfinished, unread, or finished in an earlier pass — over `rereadDays` before it, or before a run of more than `rereadChapters` chapters read in order up to it; 0 = caught up), and optional personal Reading Story aggregates |
| `GET /api/manga?m=` | user | Manga details, books, chapters with ranked versions, progress, optional chapter-time summaries, and `continue` target |
| `GET /api/chapter?m=&c=[&v=]` | user | Pages (`u, w, h, t, r`), direction, versions, prev/next, progress, `latest` (as in `/api/progress/latest`), optional Active Reading Clock summary, plus all books and chapters of the manga (with the user's read state) for the chapter picker. Also marks that chapter's notifications read |
| `POST /api/progress` `{m, c, v, p, n, done, shift[, tab, seen]}` | user | Save the reading position. With `seen`, a save is refused (`{ok:false, stale:true, latest}`) when another tab saved a newer position of this manga since; otherwise `{ok:true, t}` |
| `GET /api/progress/latest?m=` | user | The manga's newest saved position (`latest`: `t, w, c, label, v, p, n, done, continue`) or `null` |
| `POST /api/reading/pulse` `{m, c, v, session, seq, …deltas}` | user | When opted in, idempotently merge a clamped active-time pulse, verified page coverage and input totals under a single-reader lease |
| `GET /api/reading/export` | user | Export the user's private derived reading-time summary without raw input or page-exposure diagnostics |
| `GET /api/reading/activity` | user | The user's reading activity for the Settings grid: active reader time in UTC buckets `[start, size, ms, [[mangaIndex, ms]...]]` (6 minutes for the last 120 days, hours up to three years), chapter completions `[doneAt, mangaIndex, label]`, manga titles and whether the clock is on. The browser groups them into local months, weeks, days and hours (`public/activity-core.js`) |
| `POST /api/reading/delete` `{m}` or `{all:true,password}` | user | Delete one manga's reading history, or all history after password confirmation, without changing normal reading progress |
| `POST /api/unread` `{m, c}` | user | Forget progress for a chapter |
| `POST /api/vote` `{m, c, v, value: 1\|-1\|0}` | user | Vote a version; returns the re-ranked versions |
| `POST /api/follow` `{m, on}` | user | Follow a manga for notifications |
| `GET /api/moments?m=` | user | The caller's moments of that manga in story order: category, note, region, chapter, `order`, page images, `missing` when a page file is gone. `/api/chapter` also lists the caller's moments on the opened version |
| `POST /api/moments` `{m, c, v, pages, layout, region, cat, note}` | user | Save a moment: `pages` are 1–4 distinct indexes in displayed order, `layout` `row` (default) or `column`, `cat` one of `twist hot awesome cute funny feels` (required), `region` fractions of the pages put together, or null |
| `POST /api/moments/update` `{id, cat?, region?, note?}` · `POST /api/moments/delete` `{id}` | user | Edit or delete one of the caller's own moments (another user's id is `404`) |
| `GET /api/notifications` · `POST /api/notifications/read` `{id \| all}` | user | Notification list and read state |
| `POST /api/perf` `{page, events: […]}` | user | Browser timings for the performance trace; ignored with `{on: false}` while the trace is off. `/api/chapter` returns `perf: true` while it is on ([performance.md](performance.md)) |
| `GET /api/admin` | admin | Users, last scan result, library path |
| `POST /api/admin/rescan` `{full}` | admin | Scan now, and wait for the result |
| `POST /api/admin/user` `{name, password, admin}` | admin | Create a user or reset a password |
| `POST /api/admin/user/role` `{name, admin}` | admin | Grant or revoke admin rights; keeps the password and sessions. Revoking your own rights is refused, so one admin always remains |
| `POST /api/admin/user/delete` `{name}` | admin | Delete a user and their data |
| `GET /api/admin/backup` · `POST /api/admin/backup` `{command:"settings"\|"run"}` | admin | Built-in backup status, schedule/retention, "back up now" ([backup.md](backup.md)) |
| `GET /api/greatest-cover?id=<entry>&v=<version>` | user | The Greatest entry's cover from `data/greatest-covers/` (`greatest/covers.js`), cached by the browser for a year; `v` changes with the catalog URL. A `302` to the catalog URL until the copy exists. `/api/greatest` lists this URL as `cover` for titles not in the library |
| `GET /api/discovery` | user | The caller's discovery feed, settings, feedback and `refresh: { state: idle\|queued\|running, position?, nextAllowedAt?, blockedReason? }` (§6.6) |
| `POST /api/discovery/feedback` `{candidateId, value}` | user | The caller's choice for a candidate; requires opt-in |
| `POST /api/discovery/refresh` | user | Queue a per-user AniList harvest for the caller (opt-in required, else `403`). `{ harvestId, position }`; `409` while one is queued or running for the caller; `429 { nextAllowedAt }` within an hour of the last accepted refresh; `503 { blockedReason }` after AniList pushed back, until a nightly or admin harvest |
| `GET /api/admin/discovery` | admin | Last scoring run, `lastError` (a failed admin run or a failed re-score after a harvest), opted-in users, candidate count, plus `harvest` (last run record), `currentHarvest`, `cache: { files, bytes }`, `settings: { harvestEnabled, harvestAt }` and `nextNightlyAt` (`null` when off) |
| `POST /api/admin/discovery/run` | admin | Re-score every user from local data; `409` while a run is active |
| `POST /api/admin/discovery/harvest` | admin | Start a full harvest for all opted-in users: `{ harvestId, position }`, `409` while any harvest is queued or running |
| `POST /api/admin/discovery/settings` `{harvestEnabled, harvestAt}` | admin | Exactly a boolean and an `HH:MM` time, anything else is `400`; stored in `discovery.json` `settings` |
### 6.3 Authentication and security
| Concern | Measure |
|---------|---------|
| Passwords | `scrypt` (32-byte key, 16-byte random salt), constant-time compare. Unknown users are checked against a dummy hash to keep timing similar |
| Sessions | 256-bit random token in cookie `sid` (`HttpOnly`, `SameSite=Lax`, 1 year, `Secure` with `COOKIE_SECURE=1`). Only its SHA-256 is stored. Pruned at start after a year or when the user is gone |
| CSRF | POSTs must be `application/json` (plain cross-site forms can't send that) and must not carry `Sec-Fetch-Site: cross-site` |
| Path traversal | Paths are decoded, resolved and checked against the root directory. `/lib/` serves only image extensions, never the JSON files |
| XSS | The UI builds DOM nodes and inserts all data as text (`h()` helper). The only markup inserted is the app's own constant SVG icons. External links are rendered only for `http(s)` URLs |
| Request size | JSON bodies are limited to 1 MB |
| Authorisation | `admin` flag checked per admin route. Users only ever see their own progress, follows and notifications. Votes are shown as totals plus the user's own |
| Not included | Login rate limiting, a CSP header, TLS. Run behind a reverse proxy if exposed. Webhook URLs are user-chosen, so any user can make the server send a POST to an address they choose. Fine for a household; consider it before `SIGNUP=open` |
### 6.4 Version ranking
At request time: the sum of all users' votes, then `chapter.json.priority`, then folder order. The top version is what "open chapter" shows. An explicit `v=` (version picker, "Continue" of a manually chosen version) overrides it.
### 6.5 Notifications
In-app notifications and optional webhooks are produced by `announce()` after each scan, from the append-only `known` list. See [import.md §5](import.md#5-change-detection-and-notifications).
### 6.6 Discovery harvest
The reader alone talks to AniList for discovery ([discovery.md](discovery.md)). A single-flight harvester
(`discovery/harvest.js`) fetches seeds for the opted-in users' anchors into `data/discovery-cache/`; scoring
(`discovery/engine.js`) is local and publishes `data/discovery.json` atomically.
- **Kinds and budgets:** `nightly` and `admin` (300 requests, every opted-in user), `user` (60 requests, the
caller only). After a nightly or admin harvest every user is re-scored; after a per-user refresh only that
user, and the other feeds are carried over unchanged into the same publish.
- **Schedule:** checked every minute and once 30 s after startup. The time is `discovery.json` `settings`, else
`DISCOVERY_HARVEST_AT` (`HH:MM`), else `03:30`; `nightly.lastDate` is written only when the run is accepted.
- **Run record** `discovery.json` `harvest`: `{ id, kind, user, status: running|success|stopped|interrupted,
startedAt, finishedAt, requests, budget, stopReason }`. `stopReason` is `budget`, `rate-limited`, `forbidden`,
`challenge`, `upstream-errors` or `error` (a local fault). A run cut off by a restart is marked `interrupted`
at startup; queued refreshes are not persisted.
- **Published candidates:** only those at least one feed shows. `lastRun.candidateCount` (the admin
candidate count) counts that set; `lastRun.examined` and `clipped` still describe the whole pool.
- **Published fields:** candidates found through AniList carry `tagRecords`, `authorRecords`, `seed`
(`recs|creator|tagTop|fresh`) and `fetchedAt`; feed items carry `group` (seed kind or period) and `reasons`
(all reasons, strongest first) next to `reason`; seed evidence carries `fetchedAt`.
- `DISCOVERY_ANILIST_URL` overrides the AniList endpoint (tests point it at a local fixture server).
### 6.7 Reader settings per session
The reader keys (`startMode`, `lastMode`, `adapt`, `skipExtras`, `skipCovers`, `flowPageGap`, `pageTone`, `zoom`)
can differ per login session, so a phone and a desktop keep their own mode and zoom (`reader-sessions.js`):
- `user.settings` holds the **latest** reader set. A session without its own set follows it.
- The first reader change in a session copies the latest set into `session.reader` with the change applied: the session
is now **locked** and keeps that set whatever other sessions do. The change also becomes the latest set.
- Loading a page (`GET /api/me`) from a locked session makes its set the latest again, so a new sign-in starts from the
session used last. A new sign-in has no set of its own until it changes something.
- Other settings (Reading Story, dates, discovery, webhook, …) stay per user.
- `readerPerSession: false` (Settings → Reading) brings back one shared set: changes go to `user.settings` and apply to
every session. Locked sets are kept and apply again when the switch is turned back on.
Settings → Reading → *Reader settings per session* lists the user's sessions (device from the User-Agent, sign-in and last
activity, own or following) with "Follow latest again" and "Use for new sessions".
## 7. Frontend
### 7.1 Pages
Every page is a static HTML file plus one script, loaded with `defer`. All pages share `common.js` and `app.css`. There is no client-side router, no state library and no component system. Each script fetches what it needs, builds DOM with `h()`, and re-renders small parts on change.
A web app manifest (`display: standalone`) makes "Add to home screen" open the app without browser bars on phones.
### 7.2 The reader (`reader.js`)
**Inputs:**
- the chapter payload (pages with pixel sizes, tags and rotation; direction; versions; neighbours; progress),
- the user's settings.
**Page model.** Each page's size is known before it loads, so the reader computes the whole layout itself and positions every page absolutely:
```
all pages ──filter (skipExtras / skipCovers)──▶ list
list ──mode──▶ build(): create page elements (flow: all, lazy · page/book: current + preload next)
└──▶ layout(): compute a box {x, y, w, h} per element from the viewport, zoom and page aspect
└─▶ place(): apply the boxes; rotated images are sized swapped and turned with CSS
```
`layout()` repeats up to three times until the viewport size is stable, because scrollbars that appear or disappear change the viewport.
**Layout rules** (z = page zoom of the current mode):
| Mode | Box sizes |
|------|-----------|
| Flow (paged content) | `W = min(vw, vh · refAspect) · z`: a normal single page fits the screen at 100 %. Wide pages keep the single page height, capped at `max(vw, W)`. 8 px gap |
| Flow (webtoon, zoom key `strip`) | `W = min(vw, median image width) · z`, no gaps, so slices join seamlessly |
| Page | The page fits the viewport (`contain`) · z, centred. The canvas grows beyond the viewport when zoomed in |
| Book | The pages of the spread share one height `H = min(vh, vw / Σaspect) · z`. Order is right-to-left for `rtl` |
**Anchoring.** Before any re-layout (zoom, resize, pinch), `anchor(x, y)` records which page is under that point and where inside it, as a fraction. `restore()` scrolls so the same point is under the finger or cursor again. That is why zooming feels pinned to the cursor and a resize keeps the reading position.
**Book pairing:**
- Covers and wide pages stand alone.
- The single pages between them form *runs*, and each run pairs from its start.
- "⇄ Shift" toggles a run to start with a lone page. It is stored as the run's first page index in `progress.shift`.
- One broken spot can therefore never shift the pairing of the whole chapter.
**Mode resolution:**
```
intended = startMode == "last" ? lastMode : startMode
compatible = direction == "ttb" ? [flow] : [flow, page, book]
mode = adapt && intended ∉ compatible ? compatible[0] : intended
```
`lastMode` is written only when the user clicks a mode, never on an automatic switch.
**Input:**
| Input | Action |
|-------|--------|
| Click or tap outside the pages | Toggle the bars (the end card uses the whole screen as zones instead) |
| Click or tap on the page(s), outer 35 % of the visible page area (paged) | Previous/next, following the reading direction (`rtl`: left = next) |
| Click or tap on a page, top/bottom 35 % of the screen (flow) | Scroll to the previous/next page top, at most 90 % of a screen (85 % steps for webtoons). At the very bottom → next chapter |
| Click or tap on a page, middle band | Toggle the bars |
| Horizontal swipe ≥ 50 px within 800 ms (paged, not zoomed in) | Previous/next |
| Ctrl + wheel, touchpad pinch, Safari gesture events, two-finger pinch | Continuous page zoom 20–600 %, snapping to 100 % within ±1 %, anchored at the pointer or finger midpoint. Browser zoom is suppressed only over the pages |
| Mouse drag (moved ≥ 6 px) | Pan; the following click is swallowed. Hand cursor when the content is larger than the screen |
| Keys | Dispatched through `READER_KEYS` in `public/keys.js`, which also feeds Settings → Keyboard shortcuts and the reader's `?` list. Arrows follow the direction; Space/PageDown/PageUp (paged); `+` and `-` zoom in 5-percentage-point steps, `0` fits at 100 %; `m` saves a moment; `i`/Esc bars; `f` fullscreen; `?` shortcuts |
**Bars:**
- Shown on open.
- Hidden on the first page turn or after 100 px of scrolling. The first automatic hide (per device, once for Flow and once for Page/Book) shows a hint on how to get them back.
- **Never shown by scrolling**, only by a tap beside the pages or in the middle of a page (or `i`). Showing them stops a running smooth scroll, so it can't hide them again right away.
- The top bar has the chapter picker (every chapter, grouped by book, ✓ = read).
**Loading:**
- Flow mode loads images through an `IntersectionObserver` on the scroll container, 1.5 screens ahead.
- Page and Book mode keep a small pool (12) of `Image` objects:
- Once the pages on screen have loaded, the next 3 pages (Book: next 2 spreads) and the previous one are fetched at low priority and **decoded** (`img.decode()`).
- A page turn then inserts those same, already decoded elements (`decoding="sync"`), so the old and new page swap in one frame. A fresh `<img>` would be painted empty while it decodes, which is what caused the flicker on the first view of each page.
- A page that is still loading shows a spinner (only after 150 ms, so fast loads never flash it). A page that fails shows a message.
- A page with unknown size is laid out at 1000×1414 and re-laid out when it loads.
**Progress:**
- Saved 700 ms after the position settles, and on `pagehide` (a `fetch` with `keepalive`).
- Stores the page index in the full page list, so skipping extras doesn't corrupt it.
- On open, the saved page is used for the same version, and mapped proportionally for another version. A finished chapter reopens at the start.
- **Stale reading sessions.** A tab left open on one device (or restored by the browser) must not move "Continue" back after the user read on elsewhere. Each browser tab has an id (`sessionStorage`) that its saves store as `w`; reader links carry `at`, the newest progress time the linking page knew (`continue.at`), and the reader keeps `at` current in its address bar so a reload or restored tab still knows it. A link without `at` (bookmark, notification) counts as a deliberate open. When the manga's newest position is later than `at` and came from another tab, the reader saves nothing and asks: jump to the newest position, or keep reading here, which makes this chapter the head again. It checks on open, when the tab becomes visible again, and with every save (the server refuses the stale save). The comparison is by save time, not chapter order; the question names no device.
**Active Reading Clock (opt-in):**
- Runs only while the reader is visible and focused, a sufficiently exposed page is decoded, a server lease is held, and accepted reader input occurred in the last 50 seconds.
- Backgrounding or blurring pauses immediately and requires fresh input to resume. A delayed callback can never add time beyond the 50-second input deadline.
- A page is verified after at least 35% meaningful exposure for 200 ms cumulatively. Exposure is closed and evaluated when the page leaves the viewport, so a fast but plausible webtoon scroll is not lost between samples. A five-pages-per-second token bucket rejects implausible coverage without discarding otherwise credible active time.
- `BroadcastChannel` coordinates tabs on one browser. The authenticated API adds a 15-second per-user lease for other devices, and releases it when the tab pauses or leaves.
- Pulses are sent at most every 10 seconds and on pause/page exit. The server validates the manga/chapter/version and page indices, de-duplicates `(session, seq)`, clamps time to server-observed intervals and stores compact page ranges separately from ordinary progress.
- The bottom toolbar chip shows active time and the current state; its panel adds pass time, verified pages, predominant input and loading wait.
**Moments** (`public/moments.js`): the top bar's bookmark button (or `m`) opens a sheet for the page(s) on screen: in Flow the page at the reading line, in Book the spread. "+ Previous page" / "+ Next page" add neighbours (up to four pages) for a picture that runs over several pages: stacked in Flow (`column`), side by side in Page and Book (`row`, right to left for `rtl`). A rectangle dragged over the preview keeps only that region, and pages it does not touch are left out. Exactly one category is required. The button is filled when the page already has a moment. The manga page shows the user's moments as a strip of crops (sort by story or by date saved, category filters from five moments), with a viewer to step through them, edit, delete or open the reader at `read.html?…&p=<page index>`. Crops position the full page images inside a clipping box; nothing is generated on the server.
**Zoom persistence:** per mode key (`flow`, `page`, `book`, `strip`), saved 800 ms after the last change.
## 8. Deployment
The step-by-step guide is [deployment.md](deployment.md). The scraper's authoritative
`data/scraper/queue.json` is schema version 3. It supports legacy archive checkpoints
and paused conversion of completed legacy work while adding source-aware lanes, scoped
queue jobs, incremental chapter publication, and nightly catalog discovery. See
[scraper-service.md](scraper-service.md) for the queue/API contract.
### Docker or manual deployment
```bash
docker compose up -d --build
```
- **Images/services:** the same `node:24-alpine` image runs a reader service and a scraper service. The reader health check targets `/api/auth`.
- **Volumes:**
- `/library`: read-only in the reader and writable in the scraper. Files must be readable—and scraper targets writable—by uid 1000.
- `/data`: reader state must be writable; the scraper also needs its writable `/data/scraper` state.
- `/backup`: reader only, writable target of the built-in backup (`BACKUP_PATH`; off when unset).
- **Configuration:** process wiring uses environment variables (`PORT`, `LIBRARY`, `DATA`, `BACKUP`, `SIGNUP`, `SCAN_INTERVAL`, `FULL_SCAN_INTERVAL`, `COOKIE_SECURE`, `SCRAPER_URL`, and `SCRAPER_TOKEN`). Queue pacing and automatic-update schedule are persisted in `data/scraper/queue.json` and edited through the authenticated admin UI.
- **Without Docker:** `npm run start:all` with Node ≥ 18 starts one supervisor that owns exactly one reader (`server.js`, port 8080) and one scraper daemon (`scraper/daemon.js`, private port 8091), or supervise `npm start` and `npm run start:scraper` independently. On Windows, `tools/run-supervisor.ps1` runs `start:all` with logs under `data/scraper/`, e.g. from a scheduled task at logon. Never start a second supervisor or a container against the same data.
- **Behind a reverse proxy:** terminate TLS, set `COOKIE_SECURE=1`. A sub-path works.
- **Upgrades:** replace the code and restart. Reader state has no explicit migration;
unknown fields in `db.json` are kept. The scraper normalizes old queue entries into
schema version 3 on startup and preserves legacy job checkpoint behavior. Take a
queue/checkpoint recovery copy before upgrading a live scraper.
## 9. Capacity and limits
| Aspect | Behaviour |
|--------|-----------|
| Library size | The index keeps a small object per page (file name, size, tags), in the order of a few hundred bytes. Hundreds of thousands of pages fit in a modest container. Unchanged chapters cost one `stat` + one small read per scan |
| First scan | Reads 32 KB per image once (≤ 256 KB, or the whole file for JPEGs with huge EXIF/ICC blocks), `SCAN_CONCURRENCY` (8) at a time. Later scans use the cache |
| User data | `db.json` is rewritten as a whole, at most about twice per second while people read. Comfortable up to a few MB (a household or small group, thousands of chapters read) |
| Concurrency | One process, one writer. No clustering or horizontal scaling by design |
| Images | Served as stored. No resizing or thumbnails, so large covers make the library page heavy |
| Slow storage | Opening a chapter reads its pages (then the next chapter's) into the OS cache, two at a time (`READ_AHEAD`). Where the time goes: [performance.md](performance.md) |
## 10. Design decisions
| Decision | Alternatives | Why |
|----------|--------------|-----|
| Index the folder tree in place | Import into a DB / managed storage | Stays readable by any viewer and writable by any tool. Nothing to keep in sync |
| No dependencies, no framework, no build | Express, React, bundler | Minimal code, maximum longevity, runs anywhere Node runs |
| Built-in backup in the reader, queue paused, no container stop | Host job that stops the stack; backup in a separate service | Only the reader sees all of `/data` and the in-memory `db.json`; the scraper's pause API gives a consistent point without downtime |
| JSON file for user data | SQLite (`node:sqlite` is still experimental in Node 22/24), Postgres | Zero dependencies, trivial backup, enough for the target size. Easy to swap: all access goes through the `db` object |
| Page roles as file-name tags | Page lists in JSON | Visible in any file browser, survives copying, no second source of truth. Unknown tags are harmless |
| Image sizes read on the server | Measure in the browser after load | Layout before load: exact scroll restore, zoom anchoring, auto-detected double pages, no layout jumps |
| Absolute layout engine in the reader | CSS flow/flex layout | Deterministic geometry, needed for pointer-anchored zoom and exact position restore |
| Chapter id = book number + chapter number | Random ids stored in JSON | Human-readable and needs no writes into the library. Downside: moving a chapter between books changes its identity |
| Global votes, per-visit version choice | Per-user preferred version | "The better version is shown to everyone", while any user can still pick another one explicitly |
| Bars never reappear on scroll | Show on scroll-up (common pattern) | Explicit requirement: readers scroll up and down constantly |
## 11. Known limitations and extension points
- **Identity is name-based.** An optional stable id in `manga.json` (with a migration of the user data keys) would make manga renames safe.
- **No thumbnails.** Would need an image library (e.g. `sharp`) or pre-generated `cover` files.
- **Notifications:** in-app plus webhook only. Web Push would need VAPID keys and a service worker.
- **No login rate limiting and no CSP header.** Both are small additions in `server.js`.
- **In-place image overwrites** are only detected by a full rescan.
- **Reader layout remains mostly smoke/manual.** Reading analytics has focused Node tests
for its persisted merge rules, while DOM layout, authentication and mobile browser
lifecycle behavior still need browser-level/manual verification.
- **Adding content by API** (instead of files) would be a new route plus the staging and renaming rules from [import.md §7](import.md#7-consistency-writing-into-a-live-library).
Reference in new issue View Git Blame Copy Permalink