MangaCircus/docs/backup.md

Built-in backup

The reader can back up its own DATA folder on a schedule set in Settings → Admin → Service health → Backups, without stopping any container. The library is not part of it (it is plain files; back it up with the storage that holds it). Implementation: backup.js; tests: test/backup.test.js.

Turning it on

Where	Setting
Native	BACKUP=/path/to/backups for server.js. Unset = feature off
Compose	BACKUP_PATH=/host/folder in .env. The reader mounts it rw at /backup and gets BACKUP=/backup; without BACKUP_PATH the variable stays empty and backups are off

The folder must already exist and be writable by uid 1000 (create_host_path: false: a missing folder makes docker compose up fail instead of silently creating an empty one on the wrong disk). Use another disk or a NAS share, never a folder inside DATA. Writability is checked by creating and removing a probe file (.circus-backup-probe-<pid>), not with access(), which NFS shares with squashed ids (e.g. Synology "map all users to admin") answer wrongly. Without BACKUP_PATH, Compose still mounts the unused default ./backup, so that folder must exist too (node tools/prepare-compose.js creates it). Upgrading an existing deployment therefore needs either BACKUP_PATH in .env or an empty backup/ beside docker-compose.yml.

Archives contain password hashes, sessions and the peering private key: keep the target private.

Why the reader owns it

The reader is the only service that mounts all of /data (the scraper sees only /data/scraper), it holds db.json in memory and owns its atomic writer, it already talks to the scraper's private control API with the shared token, and the admin settings UI lives there. Running it in the scraper would need a full read-write /data mount there (see architecture.md §3).

Settings and status

Admin only, under Service health (GET/POST /api/admin/backup, also included in GET /api/admin/health as backup):

• Back up daily at HH:MM (container time zone TZ) and an enable switch.
• Keep the newest N backups (count, 1–365) or N days (days, 1–3650).
• Back up now starts a run immediately; the page polls every 2 s while it runs.
• Status: target folder, number/size of archives, the running phase, last backup (time, trigger, size, duration, file), its error and warnings, next scheduled run.

Settings and the last 10 results persist atomically in data/backup.json. A slot missed while the reader was down runs once at the next start if it is less than a day old; older slots (for example after restoring an old backup.json) are skipped.

POST /api/admin/backup {command:"settings", settings:{enabled, time:"HH:MM", retention:{mode:"count"|"days", value}}}
POST /api/admin/backup {command:"run"}       -> 409 when off or already running

What one run does

1. Ask the scraper for its state (GET /api/state). If the queue runs, pause it (POST /api/control {action:"pause"}) and remember that this run paused it.
2. Wait until no job is active (the worker stops at its next page/chapter boundary), polling every 2 s for at most 15 minutes. On timeout the run fails, writes nothing and resumes the queue if it paused it. An unreachable scraper does not block the backup; that is recorded as a warning.
3. Write BACKUP/circus-YYYYMMDD-HHMMSS.tar.gz.part: a gzip'd tar of DATA with paths relative to DATA, fsync it, read it back to verify, rename it to .tar.gz.
   • db.json is serialized from the reader's in-memory state at that moment.
   • scraper/queue.json is stored with paused: true (status paused), so a restored copy never starts downloading by itself. The live file is not touched.
   • scan-cache.json, library-index.json, catalog.json, peering, logs and everything else under DATA is copied as found; all of them are written by atomic rename, so each file is complete. A leftover sentinel/ folder from before 2026-10-22 is copied too; it is unused and can be deleted.
   • Left out: runtime*.json, *.lock (daemon.lock, supervisor.lock), *.pid, *.tmp, symlinks, and the BACKUP folder itself if it lies inside DATA.
   • .circus-backup.json at the archive root records when and how it was made.
4. Resume the queue only if this run paused it (three attempts; a failure is shown as a warning: resume it under Settings → Scraper).
5. After a successful run only, apply retention. It deletes only files named circus-YYYYMMDD-HHMMSS.tar.gz directly inside BACKUP (age from the name), never the newest one, never other files or subfolders. Stale circus-*.tar.gz.part files from an interrupted run are removed at the next run.

If the reader restarts during a run, backup.json still carries the run's "queue paused by backup" marker; the next start resumes the queue and records the run as interrupted. The marker is never part of an archive.

Measured on a copy of a production data folder (54 MB, 1000 files): 5.8 MB archive in about 5 s including verification. Pauses last until the current chapter is done plus those seconds.

Restore

Restoring replaces the user data with the state of the archive. Reading progress and queue changes made after that backup are lost.

1. Stop the stack (or at least reader and scraper): docker compose stop reader scraper.
2. Move the current data folder aside instead of deleting it: mv /srv/mangacircus/data /srv/mangacircus/data.before-restore.
3. Unpack the archive into an empty data folder owned by uid 1000:

   mkdir /srv/mangacircus/data
   tar -xzf /path/to/circus-YYYYMMDD-HHMMSS.tar.gz -C /srv/mangacircus/data
   chown -R 1000:1000 /srv/mangacircus/data
   

   (--same-owner is the default for root; the archive keeps the original uid/gid.)
4. Start the reader first: docker compose up -d --no-build reader. Check /health, log in, check progress and library. The reader rebuilds a missing or stale library index by itself.
5. Start scraper. The restored queue is paused: check it under Settings → Scraper and resume it when it looks right.

The scraper token is not in DATA; Compose keeps it under .secrets/ (or the deployment's SCRAPER_TOKEN_PATH). Restore tests should unpack into a separate folder and run an isolated stack (no second writer against the live library), as a restore-test script does for the host-level archives.