Faster first library scan: bounded concurrency and a 32 KB first read

An uncached scan probed one image at a time and read 256 KB of each. On
a NAS that is bound by per-file latency (the server/Synology: ~20 ms per
image, ~100 min for 298k images). Now chapter folders of a book and
their images are probed SCAN_CONCURRENCY (default 8, 1-64) at a time
and assembled in folder order, and imageSize() reads 32 KB first, then
256 KB, then the whole file: a prefix gives either no size or the same
size, so results do not change. Tests: every format and read stage
equals the whole-file answer; concurrency 1 and 8 build the same index
and cache; the limiter keeps its bound and per-task errors. (BL-0043)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
MaashkinoozheandClaude Opus 5.5 committed 2026-10-22 20:07:29 +02:00
1 parent d6254a8be5
commit 5e14e54286
7 files changed
+169 -22

No files matched your search

+1
View File
@@ -69,6 +69,7 @@ see administration; personal settings do not poll scraper services. See
| `SIGNUP` | `closed` | `open` lets anyone create an account. Otherwise the admin adds users under Settings |
| `SCAN_INTERVAL` | `10` | Minutes between shallow library change checks (`0` disables this timer) |
| `FULL_SCAN_INTERVAL` | `1440` | Minutes between full library reconciliations (`0` disables this timer) |
| `SCAN_CONCURRENCY` | `8` | Chapter folders and images probed at once by an uncached scan (1–64); higher helps on network storage |
| `COOKIE_SECURE` | – | `1` marks the login cookie `Secure` (use it when you serve over HTTPS) |
| `SCRAPER_URL` | `http://127.0.0.1:8091` | Private scraper daemon URL |
| `SCRAPER_TOKEN` | empty | Shared daemon token; required when it listens beyond loopback |