Maashkinoozhe/MangaCircus
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
MangaCircus/docs/library-format.md
T
MaashkinoozheandClaude Opus 5.5 7b52da2f9c Official release dates from MangaUpdates; peers lower ecosystem dates (BL-0048, BL-0049) 
- scraper/metadata/release-dates.js fills x-dates.release from MangaUpdates
  publisher releases. MangaUpdates also lists English back-catalog re-releases,
  so a date is kept only inside the AniList original run, when no scanlation or
  source upload is more than 4 days earlier, and when it fits the release order
  (no lower chapter dated after a higher one). Seasons that restart chapter
  numbers are skipped; dates this job wrote and no longer accepts are withdrawn.
- The scraper refreshes ongoing series nightly before its update checks, only
  while no download runs, and bumps the publish revision so the reader reconciles.
- tools/release-dates.js for a whole library (dry run default). Production dry
  run: 728 chapters dated (Kagurabachi, JJK, Chainsaw Man, One Piece from 928).
- Peer sync: chapters this library already has adopt a peer's earlier ecosystem
  date (job result ecosystemLowered); the peer listing now carries dates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 08:47:36 +02:00

326 lines
19 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# The file system library
MangaCircus has no import database for content. The **library folder is the source of truth**. The app reads it in place (read-only access is enough), and anything that can write files can add manga: a scraper, a sync tool, or a person with a file browser.
This document is the format specification. [import.md](import.md) describes how the app reads it, and [scraper-guide.md](scraper-guide.md) is a rule book for tools that write it.
## Goals of the format
| Goal | How the format meets it |
|------|-------------------------|
| Readable without the app | Plain image files in plain folders. Zero-padded numbers make alphabetical order equal reading order in every file browser and picture viewer |
| Fast, simple indexing | Everything the app needs is in folder names, file names and a few small JSON files. Image contents are never decoded. Only the first bytes of each image are read once to learn its size |
| Self-describing | Page roles (cover, double page, extra, rotated) are visible in the file name, not hidden in a database |
| Several sources | One manga can list several upstream sources. One chapter can exist in several versions side by side |
| Room for irregular releases | In-between chapters (`10.5`), extra chapters and extra books outside the numbering |
## Layout
Always three folder levels, **Manga / Book / Chapter**, with the pages inside the chapter folder:
```
<library>/
<Manga>/ manga folder; its name is the manga's id
manga.json optional
cover.<ext> optional
<Book>/ e.g. "v01 Spring"
book.json optional
cover.<ext> optional
<Chapter>/ e.g. "c0001 The Seed" (one folder per chapter version)
chapter.json recommended
0001_cover.<ext>
0002.<ext>
...
```
Series without volumes (most webtoons, ongoing magazine series) still need the book level. Use a single `v01`, or seasons (`s01 Season 1`).
Anything else is ignored:
- files at the library root,
- files other than `manga.json` / `cover.*` in a manga folder,
- files other than `book.json` / `cover.*` in a book folder,
- non-image files and sub-folders inside a chapter folder,
- every file or folder whose name starts with `.`.
## Manga folders
- Any name that does not start with `.`. The folder name is the manga's **id**: URLs, reading progress, follows, votes and the "already seen chapters" list all refer to it. **Renaming the folder turns it into a new manga for the app.**
- The displayed title comes from `manga.json` → `title`, and falls back to the folder name.
- Manga without a single readable chapter are left out (and logged).
## Book and chapter folder names
```
<letters>[ ]<number>[<separator><title>][ [<source>]]
```
| Part | Rule | Examples |
|------|------|----------|
| letters | Only letters (any case), optionally followed by **one** space. Free choice and just a hint for humans, **except `x`, which means "extra"** | `v`, `c`, `s`, `vol`, `Ch `, `Chapter `, `x` |
| number | Digits, optionally one decimal part. It decides the order and the identity. Zero-pad it so file browsers sort it right (4 digits for chapters, 2 for books) | `0001`, `12`, `0010.5` |
| separator | The number must be followed by the end of the name, a space, `_`, `-`, `[` or `(`. **Use a space**: `_` sorts after `.`, which breaks the order of decimal chapters in file browsers | ` `, ` - ` |
| title | Anything. Used as the title when the JSON file has none. Leading `-`, `:` or `.` are trimmed | `The Seed` |
| `[source]` | Optional, at the very end. Names the version (scan group, site) when `chapter.json` has no `source` | `[MangaDex]` |
Valid: `v01`, `v01 Spring`, `c0001 The Seed`, `c0002 Roots [MangaDex]`, `c0002.5 Tea Time`, `x01 Character Sheets`, `Vol 3`, `Chapter 12 - Title`, `c0010.5[Official]`.
Ignored, with a warning in the log:
- `c0012a`: a letter right after the number.
- `Ch.12`: a dot is not a letter.
- `Specials`: no number.
- `- c0001`: the name must start with the key.
### The key
The additive unnumbered MangaDex chapter form is
`special-mangadex-<chapter-uuid> Pilot [MangaDex · Group]`. It is accepted only for
chapters, never books. Its logical key is the complete lower-case prefix/UUID;
`chapter.json` supplies `special:true`, `label` (Pilot/One-shot/Special), `sourceId`
and nullable `sourceNumber`. The reader shows that label with no fabricated number.
Such chapters sort after numbered entries and are not inherently skippable extras.
Different source UUIDs remain distinct releases; no content equivalence is guessed.
Numeric IDs and source-version folders are unchanged. Older reader releases ignore
these new folders but retain them on disk; upgrade again to restore visibility.
Whole-volume WeebCentral releases use the analogous
`special-weebcentral-volume-<26-character-release-id>` chapter key. They live in
the matching volume book and carry `releaseKind:"volume"`, `volume`, `sourceId`,
and a `Volume N` label in `chapter.json`. This keeps an entire volume distinct
from a numbered chapter in the same book. The default `skip-any` download policy
also skips a whole-volume release if that book already contains numbered chapters;
`add-source` allows the extra source edition explicitly.
The number is normalised: leading zeros and trailing decimal zeros are dropped (`0010.50` → `10.5`), and an `x` prefix is kept. The result is the **key**:
| Folder | Key |
|--------|-----|
| `v01 Spring` | `1` |
| `c0002.5 Tea Time` | `2.5` |
| `x01 Artbook` | `x1` |
- **Book keys must be unique within a manga.** A second folder with the same book number is ignored and logged.
- **Chapter id = `<book key>/<chapter key>`**, for example `1/2.5` or `x1/x1`. Moving a chapter to another book, or renumbering it, therefore makes it a different chapter: reading progress and votes stay behind, and followers are told about a "new" chapter.
- Because the key is a number, `c0010.1` and `c0010.10` are the **same** key (`10.1`). Avoid trailing zeros in decimals.
### Order
Inside a manga, books sort by (regular before extra, number). Inside a book, chapters sort the same way. The reading order of the whole manga is books in order, each with its chapters in order:
```
v01: c0001, c0002, c0002.5, x01, x02 <- extras of book 1 come after its regular chapters
v02: c0003, c0004
x01: x01 <- extra books come after all regular books
```
### Extras
Something "extra" can be skipped while reading (user setting *Skip extra pages and extra chapters*) and gets an "extra" badge in the chapter list:
- A chapter whose letters are exactly `x` (`x01 Character Sheets`) is extra. It sits outside the continuous numbering and is listed after the regular chapters of its book.
- Every chapter in an extra book (`x01 Specials`) is extra.
- `"extra": true` in `chapter.json` or `book.json` marks something as extra **without moving it**. Use it for an in-order omake such as `c0010.5`.
## Chapter versions
Several chapter folders **in the same book** with the **same chapter key** are versions of one chapter, for example two scan groups or an official release plus a fan translation:
```
c0002 Roots [MangaDex]/
c0002 Roots [Official]/
```
- The **version id is the folder name**. Votes and reading progress refer to it, so don't rename version folders.
- Users vote versions up or down. The chapter opens in the version with the highest total. Ties go to the higher `"priority"` in `chapter.json` (default `0`), then to the first folder name in sort order.
- Users can still open any other version from the chapter list or the reader's version picker.
- Each version has its own `chapter.json`, page count and page tags. When versions have different titles, the chapter title comes from the first version (in folder order) that has one.
- Versions without any image are ignored.
## Pages
A chapter folder contains its pages as plain images: `jpg`, `jpeg`, `png`, `webp`, `gif`, `avif` or `bmp` (extension case doesn't matter).
```
<number>[_<tag>]...<.ext>
```
- **Order** is the natural sort order of the file names. Zero-pad the number (`0001` … `9999`) so every picture viewer agrees. For more than 9999 pages, use 5 digits throughout.
- **Tags** follow the number, each introduced by `_`, and are case-insensitive: `0012_double.jpg`, `0030_extra.png`, `0007_double_rot90.jpg`.
| Tag | Meaning | Effect in the reader |
|-----|---------|----------------------|
| `cover` | Chapter cover / title page. It is also **the chapter's image**: the fallback cover for the book or manga when those have none | Stands alone in Book mode. Can be skipped (*Skip chapter cover pages*) |
| `double` | One image showing two facing pages (a spread) | Stands alone in Book mode. In Flow mode it has the height of a single page, so it is twice as wide, but never wider than the screen |
| `extra` | Extra content: translator notes, credits, recruitment pages, ads | Can be skipped (*Skip extra pages…*) |
| `rot90`, `rot180`, `rot270` | The image is stored turned. The viewer turns it **clockwise** by this angle | Width and height swap for `rot90` / `rot270` |
- A page whose (rotated) image is **wider than tall** counts as a double page even without the tag. Tag it anyway; it documents the intent.
- Unknown tags are ignored, so tools may add their own (`0005_color.jpg`) without breaking anything.
- A file name without a leading number (`page-a.jpg`) is still a page. It is just sorted by name, which you rarely want.
### Covers
| Level | Cover | Fallback |
|-------|-------|----------|
| Manga (library tile, manga page) | `cover.<ext>` in the manga folder | cover of the first book |
| Book | `cover.<ext>` in the book folder | first `cover`-tagged page (else the first page) of the first chapter's first version |
| Chapter | first page tagged `cover` | first page |
Covers are served at their original size. A cover around 600×900 px keeps the library page light.
## Metadata files
UTF-8 JSON (a BOM is tolerated). All fields are optional, and unknown fields are kept and ignored. A broken file is logged and treated as empty, so the folder still works with its name-based defaults.
### `manga.json`
| Field | Type | Meaning |
|-------|------|---------|
| `title` | string | Display title (default: folder name) |
| `altTitles` | string[] | Other titles |
| `authors` | string[] | Authors / artists |
| `status` | string | Free text, e.g. `ongoing`, `completed`, `hiatus` |
| `year` | number | First release year |
| `tags` | string[] | Genres / tags |
| `description` | string | Synopsis (line breaks are kept) |
| `direction` | `"rtl"` \| `"ltr"` \| `"ttb"` | How the pages are read. `rtl` (the default): manga, right to left. `ltr`: comics, printed manhwa. `ttb`: webtoon, one vertical strip cut into slices (aliases `vertical`, `webtoon`) |
| `directionOverride` | same values | Explicit manual choice. Use this even when choosing `rtl`, which is also the fallback. May be set at manga, book, or chapter level |
| `x-reading-direction` | object | Generated provenance: `{ "version": 1, "value": "ttb", "source": "detected" }` or `source: "fallback"`, with optional numeric evidence. The scraper owns this field; it is not a confidence score |
| `sources` | `{name, url}[]` | Upstream sources the manga was scraped from. Several are allowed |
| `x-anilist` | object | AniList work ID and linked metadata. The ID identifies a work, not necessarily a particular pilot or edition |
| `x-metadata.imported` | object | Last imported values for title, alternative titles, authors, status, year, tags and description. The scraper updates a displayed field only when it is empty or still equals its previous imported value; edits in legacy files without this snapshot are preserved |
| `x-scraper.autoUpdate` | boolean | Optional explicit admin choice. When absent, `status: "ongoing"` enables nightly checks and other statuses disable them |
The whole file is sent to logged-in users as part of the manga page. **Don't put secrets in it** (scraper tokens, cookies).
An AniList refresh with a different ID is stopped for review before chapter publication. Existing files that already contain `x-scraper.autoUpdate` keep that choice; the scraper does not guess whether an older value was automatic or manual. New files leave the choice absent so the default can follow status changes.
### `book.json`
| Field | Type | Meaning |
|-------|------|---------|
| `title` | string | Book title (default: title part of the folder name) |
| `direction` | as above | Overrides the manga's direction for this book |
| `extra` | boolean | Marks all its chapters as extra without renaming the folder |
### `chapter.json` (one per version folder, recommended)
| Field | Type | Meaning |
|-------|------|---------|
| `title` | string | Chapter title without "Chapter N" (default: title part of the folder name) |
| `source` | `{name, url}` or string | Where this version came from (default: `[source]` in the folder name) |
| `added` | ISO date/time string | When this version entered **this** library, used for "recently updated" (default: the folder's modification time). The scraper keeps it when it replaces a version |
| `x-dates` | object | Other chapter dates, written by the scraper (see below) |
| `extra` | boolean | Marks the chapter as extra without moving it |
| `priority` | number | Tie-breaker between versions with equal votes. Higher wins (default `0`) |
| `direction` | as above | Overrides book/manga direction, e.g. a colour webtoon special inside a printed series |
#### Chapter dates (`x-dates`)
```json
"added": "2026-10-17T15:04:03.424Z",
"x-dates": {
"version": 1,
"source": { "at": "2026-10-18T15:06:05.348Z", "name": "WeebCentral", "batch": false },
"release": { "at": "2023-09-17", "precision": "day", "source": "MangaUpdates · Viz" },
"ecosystem": { "at": "2026-10-17T15:04:03.424Z", "via": "this server" }
}
```
- `source`: when the source site uploaded this version (WeebCentral's full chapter list,
MangaDex `publishAt`). `batch: true` means it arrived with 3 or more other uploads
less than 10 minutes apart, which is a site import, not a release.
- `release`: the official public release (`precision` `day`, `month` or `year`). It is
filled from MangaUpdates releases by publishers such as Viz, MANGA Plus or Webtoon
(`scraper/metadata/release-dates.js`). Those lists include English re-releases of old
chapters, so a date is kept only when all of these hold:
- it lies within the work's original run on AniList;
- no scanlation or source upload shows the chapter more than 4 days earlier;
- it fits the release order: walking down from the newest chapter, a lower chapter
can't be dated more than 2 days after a higher one.
Series whose chapter numbers restart in a later book are skipped.
- The scraper refreshes ongoing series nightly before its update checks, but only
while no download is running.
- `node tools/release-dates.js <library>` covers a whole library: a dry run by
default, then `--apply --writers-stopped`.
- A date that this job wrote and no longer accepts is removed. Release dates from any
other source stay.
- `ecosystem`: when the chapter first entered any Manga Circus library. Peer transfers
carry it, and the earliest known date always wins. A peer's claim can't be earlier than
the source upload or later than now.
The chapter list shows one release date per chapter, choosing the first tier that has
a date: `release` → `source` unless `batch` (shown as ≈) → `ecosystem` (◎) → `added` (↓).
Each tier takes the earliest value across the chapter's versions. Existing libraries get
these fields from `node tools/backfill-chapter-dates.js <library>`: a dry run by default,
then `--apply --writers-stopped` and a deep Rescan.
Manual directions have priority at **chapter → book → manga**. A `directionOverride`
or a plain `direction` without matching generated provenance is manual. Otherwise,
the scanner checks the version's existing image dimensions for coherent tall strips
and uses `ttb` when they are present. A prior generated vertical series direction
applies to chapters with insufficient shape evidence. The final fallback is `rtl`;
this does **not** classify a work as Japanese or prove its reading order. A direct
edit to a generated `direction` becomes manual when it differs from its provenance.
Existing plain directions are treated as manual because old scraper defaults and
human edits cannot be distinguished safely. Set `directionOverride` to make a
lasting choice, or explicitly review and migrate a legacy entry. Image files and
chapter IDs do not change when direction metadata changes.
## Picture-viewer compatibility
- Alphabetical order of chapter folders and page files equals reading order, as long as numbers are zero-padded and separated from titles by a space.
- In-between chapters sort correctly: `c0010 Title` < `c0010.5 Omake` < `c0011` (a space sorts before `.`).
- Extras (`x…`) sort after the numbered folders. That is also where the app lists them.
- The one thing a plain viewer can't do is apply `rot…` tags.
## Examples
**Manga with everything**
```
Moon Garden/
manga.json cover.png
v01 Spring/
book.json cover.png
c0001 The Seed/ chapter.json 0001_cover.png 0002.png … 0006_double.png … 0012.png
c0002 Roots [MangaDex]/ chapter.json 0001_cover.png … 0014.png
c0002 Roots [Official]/ chapter.json 0001_cover.png … 0011.png <- second version of ch. 2
c0002.5 Tea Time/ chapter.json 0001.png … 0004.png <- in-between chapter
x01 Character Sheets/ chapter.json 0001_extra.png … <- extra chapter
v02 Summer/
c0003 The Tide/ … 0009_rot90.png … 0014_extra.png 0015_extra.png
x01 Specials/
x01 Artbook/ … <- extra book
```
**Webtoon** (`manga.json`: `"direction": "ttb"`)
```
Skyline Tower/
v01 Season 1/
c0001 Episode 1/ 0001.png … 0008.png <- slices of one strip, same width, no gaps
```
**Bare minimum** (no JSON at all: titles and ids come from the folder names)
```
Blank Slate/
v01/
c0001/ 0001.png 0002.png …
```
**One-shot**: a manga with one book and one chapter (`Oneshot Title/v01/c0001/…`), or an extra book inside the related series (`x01 Oneshots/x01 Title/`).
## Renaming: what survives
| Change | Effect |
|--------|--------|
| Title part of a **book** folder (`v01 Spring` → `v01 Early Spring`) | Nothing lost |
| Title part of a **chapter** folder (`c0003 Tide` → `c0003 The Tide`) | Same chapter (the key is unchanged), so progress and notifications are fine. The full folder name is the version id, though, so votes for that version are lost, and saved progress is re-mapped proportionally |
| `manga.json` / `chapter.json` content | Picked up on the next scan. Nothing lost |
| Manga folder name | New manga: progress, follows, votes and seen-chapters are gone |
| Book number, chapter number, or moving a chapter to another book | New chapter id: progress/votes stay with the old id, and followers are notified again |
| Version folder name | Votes for that version are lost. Progress falls back to the best version at a proportional page |
| Page files replaced **in place** (same file names) | Cached image sizes may be stale until an admin runs **Full rescan** |
Reference in new issue View Git Blame Copy Permalink