MangaCircus/docs/architecture.md

MangaCircus architecture

MangaCircus is a self-hosted reader for manga, comics and webtoons. One Node.js process serves a handful of static pages and a small JSON API. The content is a folder tree on disk, and everything user-related lives in one JSON file.

Related documents:

• library-format.md: the folder format.
• import.md: how the folder tree is indexed.
• scraper-guide.md: rules for tools that write the library.

1. Principles

Principle	Consequence
The file system is the source of truth for content	No import step, no copies, no content database. The library can be mounted read-only. Anything that writes files can feed it
As dumb as possible	No dependencies (package.json has none), no framework, no build step, no transpiler. Plain ES2019+ JavaScript in the browser, plain Node on the server
One process, one data file	In-memory library index, user data in data/db.json. Back up one file, run one container
Readable without the app	The folder format works in any picture viewer. The app adds progress, layout and convenience, not lock-in

2. System context

                ┌──────────────┐    writes folders (staged + renamed)
  scraper / ───▶│   library/   │◀──────────────────────────────────┐
  person        │ (read-only   │                                   │
                │  for the app)│                                   │
                └──────┬───────┘                                   │
                       │ readdir / stat / small reads              │
                ┌──────▼─────────────────────────────┐             │
  browser ─────▶│ node server.js                      │   webhook   │
  (desktop,     │  ├ library.js   scanner → index     │────────────▶ ntfy / Gotify / …
   phone, PWA)  │  ├ HTTP: /  /lib/…  /api/…          │  (optional)
                │  └ data/db.json  data/scan-cache.json│
                └─────────────────────────────────────┘

3. Components

Path	Lines	Responsibility
server.js	~350	HTTP server and router, auth and sessions, JSON storage, API handlers, scan scheduling, notifications and webhooks
library.js	~170	Folder scanner, name parsing, image-size reader, scan cache
public/common.js	~85	Browser helpers: h() element builder, api() fetch wrapper, top navigation with the notification bell
public/index.html + library.js	~60	Library overview (cover grid, filter, sort)
public/manga.html + manga.js	~110	Manga page: info, follow, chapter list with book separators, versions and votes, read/unread toggle
public/read.html + reader.js	~470	The reader (§7.2)
public/settings.html + settings.js	~115	Reading preferences, webhook, password, admin (rescan, users)
public/login.html + login.js	~50	Sign-in, first-run admin creation, optional sign-up
public/app.css	~150	All styles (dark theme, responsive)
tools/png.js, tools/make-dummy.js	~310	Demo library generator: a tiny PNG encoder with a bitmap font, no image libraries
Dockerfile, docker-compose.yml		node:24-alpine, non-root, /library + /data

4. Runtime model

• One process, event loop only. All disk I/O in the scanner is asynchronous and sequential, so a scan never blocks requests.
• In-memory state:
  • lib is the current library index, replaced as a whole after each scan.
  • db is all user data, loaded from db.json at start.
• Timers:
  • a library scan at start and every SCAN_INTERVAL minutes,
  • a debounced db.json write, 500 ms after the last change.
• Shutdown: SIGINT / SIGTERM flush db.json synchronously, then exit. That matters for docker stop.
• Durability: every file write goes to *.tmp first and is then renamed over the target, so a file is never half-written. A hard kill loses at most the changes of the last ~0.5 s. A corrupt db.json makes the server refuse to start rather than start empty and overwrite it.

5. Data model

5.1 Content: the library index

The index is built by library.js from the folder tree (see import.md §2.6). It is never written anywhere, except for the scan cache.

5.2 Identity

Every reference from user data to content uses ids derived from names on disk:

Entity	Id	Example	Stable under
Manga	manga folder name	Moon Garden	JSON edits, anything inside the folder
Book	normalised book number (x prefix for extras)	1, x1	title changes
Chapter	<book key>/<chapter key>	1/2.5	title changes, adding/removing versions
Version	chapter folder name	c0002 Roots [MangaDex]	nothing: renaming the folder creates a new version

This keeps ids human-readable and derivable without a database. The price: renaming a manga folder, renumbering a chapter or moving it to another book creates a new entity (see library-format.md → Renaming).

5.3 User data: data/db.json

{
  "users": {
    "admin": {
      "name": "admin", "admin": true, "created": 1790796794000,
      "hash": "<salt hex>:<scrypt hex>",
      "settings": {
        "startMode": "last",            // "last" | "flow" | "page" | "book"
        "lastMode": "book",             // the user's intended mode; changed only by clicking a mode
        "adapt": true,                  // use a fitting mode for incompatible content (webtoon → flow)
        "skipExtras": false, "skipCovers": false,
        "zoom": { "flow": 1, "page": 1.5, "book": 1, "strip": 0.8 },   // page zoom per mode
        "webhook": ""
      }
    }
  },
  "sessions":  { "<sha256 of token>": { "u": "admin", "t": 1790796794000 } },
  "progress":  { "admin": { "Moon Garden": { "1/2": {
                  "v": "c0002 Roots [MangaDex]",  // version read
                  "p": 4, "n": 14,                // page index in that version (all pages), page count
                  "t": 1790796800000,             // last activity
                  "done": false,                  // sticky once true (reset via "mark unread")
                  "shift": [],                    // book mode: run starts shown alone
                  "top": 4                        // furthest page read in this version (Senpai horizon)
               } } } },
  "follows":   { "admin": { "Moon Garden": true } },
  "notes":     { "admin": [ { "id": "a1b2c3d4e5f6", "t": 1790797300000, "m": "Moon Garden", "c": "2/6",
                              "text": "Moon Garden · Ch. 6: Frost", "read": false } ] },
  "votes":     { "Moon Garden": { "1/2": { "c0002 Roots [Official]": { "admin": 1, "reader": -1 } } } },
  "known":     { "Moon Garden": ["1/1", "1/2", "1/2.5", "..."] }   // every chapter id ever seen (append-only)
}

• data/scan-cache.json holds { "<Manga>/<Book>/<Chapter>": { m: <folder mtimeMs>, p: [[file, w, h], …] } }. It is disposable.
• Backups: copy data/db.json. The atomic writes make a copy at any time consistent.

6. Server

6.1 Request routing

Path	Handling
/api/*	routes["<METHOD> <path>"]. Handlers get { u, b, q, req, res } and return an object, which is sent as JSON. Throwing HttpError sends { error } with that status
/lib/*	Image files from LIBRARY. Requires a session, allows image extensions only, and the path must resolve inside LIBRARY. ETag + private, max-age=86400
everything else	Static files from public/ (/ → index.html). ETag + no-cache

Pages are static. Each page's script calls the API and redirects to login.html?next=… on 401. All URLs are relative, so the app also works under a sub-path behind a reverse proxy.

6.2 API

All requests and responses are JSON. POSTs require Content-Type: application/json.

Method & path	Auth	Purpose
GET /api/auth	public	{ firstRun, signup } for the login page
POST /api/signup {name, password}	public	First account (becomes admin), or anyone when SIGNUP=open
POST /api/login {name, password}	public	Starts a session (cookie sid)
POST /api/logout	user	Ends the session
GET /api/me	user	Name, admin flag, settings, unread count
POST /api/settings {…partial}	user	Validated update of the settings keys above
POST /api/password {old, new}	user	Change password
GET /api/library	user	Manga list with covers, counts, the user's read count, unread notifications, last read
GET /api/manga?m=	user	Manga details, books, chapters with ranked versions and the user's progress, continue target
GET /api/chapter?m=&c=[&v=]	user	Pages (u, w, h, t, r), direction, versions, prev/next, progress, plus all books and chapters of the manga (with the user's read state) for the chapter picker. Also marks that chapter's notifications read
POST /api/progress {m, c, v, p, n, done, shift}	user	Save the reading position
POST /api/unread {m, c}	user	Forget progress for a chapter
POST /api/vote {m, c, v, value: 1|-1|0}	user	Vote a version; returns the re-ranked versions
POST /api/follow {m, on}	user	Follow a manga for notifications
GET /api/notifications · POST /api/notifications/read {id | all}	user	Notification list and read state
GET /api/admin	admin	Users, last scan result, library path
POST /api/admin/rescan {full}	admin	Scan now, and wait for the result
POST /api/admin/user {name, password, admin}	admin	Create a user or reset a password
POST /api/admin/user/delete {name}	admin	Delete a user and their data
GET /api/senpai/v1/{locate,xray,characters/<ref>[/timeline],portraits/<ref>}?m=…	user	Optional proxy to Senpai (SENPAI_URL): adds Authorization: Bearer SENPAI_TOKEN, X-Senpai-User and X-Senpai-Horizon (furthest chapter with progress in this manga, its version and max(p, top), percent-encoded). 404 when SENPAI_URL is unset; other paths are not forwarded

6.3 Authentication and security

Concern	Measure
Passwords	scrypt (32-byte key, 16-byte random salt), constant-time compare. Unknown users are checked against a dummy hash to keep timing similar
Sessions	256-bit random token in cookie sid (HttpOnly, SameSite=Lax, 1 year, Secure with COOKIE_SECURE=1). Only its SHA-256 is stored. Pruned at start after a year or when the user is gone
CSRF	POSTs must be application/json (plain cross-site forms can't send that) and must not carry Sec-Fetch-Site: cross-site
Path traversal	Paths are decoded, resolved and checked against the root directory. /lib/ serves only image extensions, never the JSON files
XSS	The UI builds DOM nodes and inserts all data as text (h() helper). The only markup inserted is the app's own constant SVG icons. External links are rendered only for http(s) URLs
Request size	JSON bodies are limited to 1 MB
Authorisation	admin flag checked per admin route. Users only ever see their own progress, follows and notifications. Votes are shown as totals plus the user's own
Not included	Login rate limiting, a CSP header, TLS. Run behind a reverse proxy if exposed. Webhook URLs are user-chosen, so any user can make the server send a POST to an address they choose. Fine for a household; consider it before SIGNUP=open

6.4 Version ranking

At request time: the sum of all users' votes, then chapter.json.priority, then folder order. The top version is what "open chapter" shows. An explicit v= (version picker, "Continue" of a manually chosen version) overrides it.

6.5 Notifications

In-app notifications and optional webhooks are produced by announce() after each scan, from the append-only known list. See import.md §5.

7. Frontend

7.1 Pages

Every page is a static HTML file plus one script, loaded with defer. All pages share common.js and app.css. There is no client-side router, no state library and no component system. Each script fetches what it needs, builds DOM with h(), and re-renders small parts on change.

A web app manifest (display: standalone) makes "Add to home screen" open the app without browser bars on phones.

7.2 The reader (reader.js)

Inputs:

• the chapter payload (pages with pixel sizes, tags and rotation; direction; versions; neighbours; progress),
• the user's settings.

Page model. Each page's size is known before it loads, so the reader computes the whole layout itself and positions every page absolutely:

all pages ──filter (skipExtras / skipCovers)──▶ list
list ──mode──▶ build(): create page elements (flow: all, lazy · page/book: current + preload next)
          └──▶ layout(): compute a box {x, y, w, h} per element from the viewport, zoom and page aspect
                   └─▶ place(): apply the boxes; rotated images are sized swapped and turned with CSS

layout() repeats up to three times until the viewport size is stable, because scrollbars that appear or disappear change the viewport.

Layout rules (z = page zoom of the current mode):

Mode	Box sizes
Flow (paged content)	W = min(vw, vh · refAspect) · z: a normal single page fits the screen at 100 %. Wide pages keep the single page height, capped at max(vw, W). 8 px gap
Flow (webtoon, zoom key strip)	W = min(vw, median image width) · z, no gaps, so slices join seamlessly
Page	The page fits the viewport (contain) · z, centred. The canvas grows beyond the viewport when zoomed in
Book	The pages of the spread share one height H = min(vh, vw / Σaspect) · z. Order is right-to-left for rtl

Anchoring. Before any re-layout (zoom, resize, pinch), anchor(x, y) records which page is under that point and where inside it, as a fraction. restore() scrolls so the same point is under the finger or cursor again. That is why zooming feels pinned to the cursor and a resize keeps the reading position.

Book pairing:

• Covers and wide pages stand alone.
• The single pages between them form runs, and each run pairs from its start.
• "⇄ Shift" toggles a run to start with a lone page. It is stored as the run's first page index in progress.shift.
• One broken spot can therefore never shift the pairing of the whole chapter.

Mode resolution:

intended   = startMode == "last" ? lastMode : startMode
compatible = direction == "ttb" ? [flow] : [flow, page, book]
mode       = adapt && intended ∉ compatible ? compatible[0] : intended

lastMode is written only when the user clicks a mode, never on an automatic switch.

Input:

Input	Action
Click or tap outside the pages	Toggle the bars (the end card uses the whole screen as zones instead)
Click or tap on the page(s), outer 35 % of the visible page area (paged)	Previous/next, following the reading direction (rtl: left = next)
Click or tap on a page, top/bottom 35 % of the screen (flow)	Scroll to the previous/next page top, at most 90 % of a screen (85 % steps for webtoons). At the very bottom → next chapter
Click or tap on a page, middle band	Toggle the bars
Horizontal swipe ≥ 50 px within 800 ms (paged, not zoomed in)	Previous/next
Ctrl + wheel, touchpad pinch, Safari gesture events, two-finger pinch	Page zoom 20–600 %, snapping to 100 % within ±3 %, anchored at the pointer or finger midpoint. Browser zoom is suppressed only over the pages
Mouse drag (moved ≥ 6 px)	Pan; the following click is swallowed. Hand cursor when the content is larger than the screen
Keys	Arrows follow the direction; Space/PageDown/PageUp (paged); + - 0 zoom; m/Esc bars; f fullscreen; x Who's here (Esc closes it)

Bars:

• Shown on open.
• Hidden on the first page turn or after 100 px of scrolling. The first automatic hide (per device, once for Flow and once for Page/Book) shows a hint on how to get them back.
• Never shown by scrolling, only by a tap beside the pages or in the middle of a page (or m). Showing them stops a running smooth scroll, so it can't hide them again right away.
• The top bar has the chapter picker (every chapter, grouped by book, ✓ = read).

Loading:

• Flow mode loads images through an IntersectionObserver on the scroll container, 1.5 screens ahead.
• Page and Book mode keep a small pool (12) of Image objects:
  • Once the pages on screen have loaded, the next 3 pages (Book: next 2 spreads) and the previous one are fetched at low priority and decoded (img.decode()).
  • A page turn then inserts those same, already decoded elements (decoding="sync"), so the old and new page swap in one frame. A fresh <img> would be painted empty while it decodes, which is what caused the flicker on the first view of each page.
• A page that is still loading shows a spinner (only after 150 ms, so fast loads never flash it). A page that fails shows a message.
• A page with unknown size is laid out at 1000×1414 and re-laid out when it loads.

Progress:

• Saved 700 ms after the position settles, and on pagehide (a fetch with keepalive).
• Stores the page index in the full page list, so skipping extras doesn't corrupt it.
• On open, the saved page is used for the same version, and mapped proportionally for another version. A finished chapter reopens at the start.

Zoom persistence: per mode key (flow, page, book, strip), saved 800 ms after the last change.

8. Deployment

docker compose up -d --build

• Image: node:24-alpine plus the app files. It runs as user node (uid 1000) and has a health check against /api/auth.
• Volumes:
  • /library: read-only is enough. Files must be readable by uid 1000.
  • /data: must be writable.
• Configuration: only environment variables (PORT, LIBRARY, DATA, SIGNUP, SCAN_INTERVAL, COOKIE_SECURE, optional SENPAI_URL + SENPAI_TOKEN).
• With Senpai: the Senpai repository's docker-compose.circus.yml builds this branch next to Senpai and sets SENPAI_URL=http://senpai-api:8780; Senpai is reached only through the proxy, never by the browser.
• Without Docker: node server.js with Node ≥ 18.
• Behind a reverse proxy: terminate TLS, set COOKIE_SECURE=1. A sub-path works.
• Upgrades: replace the code and restart. There are no migrations: unknown fields in db.json are kept, and missing settings fall back to defaults.

9. Capacity and limits

Aspect	Behaviour
Library size	The index keeps a small object per page (file name, size, tags), in the order of a few hundred bytes. Hundreds of thousands of pages fit in a modest container. Unchanged chapters cost one stat + one small read per scan
First scan	Reads ≤ 256 KB per image once. Later scans use the cache
User data	db.json is rewritten as a whole, at most about twice per second while people read. Comfortable up to a few MB (a household or small group, thousands of chapters read)
Concurrency	One process, one writer. No clustering or horizontal scaling by design
Images	Served as stored. No resizing or thumbnails, so large covers make the library page heavy

10. Design decisions

Decision	Alternatives	Why
Index the folder tree in place	Import into a DB / managed storage	Stays readable by any viewer and writable by any tool. Nothing to keep in sync
No dependencies, no framework, no build	Express, React, bundler	Minimal code, maximum longevity, runs anywhere Node runs
JSON file for user data	SQLite (node:sqlite is still experimental in Node 22/24), Postgres	Zero dependencies, trivial backup, enough for the target size. Easy to swap: all access goes through the db object
Page roles as file-name tags	Page lists in JSON	Visible in any file browser, survives copying, no second source of truth. Unknown tags are harmless
Image sizes read on the server	Measure in the browser after load	Layout before load: exact scroll restore, zoom anchoring, auto-detected double pages, no layout jumps
Absolute layout engine in the reader	CSS flow/flex layout	Deterministic geometry, needed for pointer-anchored zoom and exact position restore
Chapter id = book number + chapter number	Random ids stored in JSON	Human-readable and needs no writes into the library. Downside: moving a chapter between books changes its identity
Global votes, per-visit version choice	Per-user preferred version	"The better version is shown to everyone", while any user can still pick another one explicitly
Bars never reappear on scroll	Show on scroll-up (common pattern)	Explicit requirement: readers scroll up and down constantly

11. Known limitations and extension points

• Identity is name-based. An optional stable id in manga.json (with a migration of the user data keys) would make manga renames safe.
• No thumbnails. Would need an image library (e.g. sharp) or pre-generated cover files.
• Notifications: in-app plus webhook only. Web Push would need VAPID keys and a service worker.
• No login rate limiting and no CSP header. Both are small additions in server.js.
• In-place image overwrites are only detected by a full rescan.
• No automated test suite. Verification so far was done with the demo library (tools/make-dummy.js), API smoke scripts and manual browser testing on desktop and mobile viewports. parseKey and sizeFromBuffer are pure functions, and scanLibrary only needs a folder. Together with the book pairing they are the natural first targets for unit tests.
• Adding content by API (instead of files) would be a new route plus the staging and renaming rules from import.md §7.