Faster first library scan: bounded concurrency and a 32 KB first read
An uncached scan probed one image at a time and read 256 KB of each. On a NAS that is bound by per-file latency (the server/Synology: ~20 ms per image, ~100 min for 298k images). Now chapter folders of a book and their images are probed SCAN_CONCURRENCY (default 8, 1-64) at a time and assembled in folder order, and imageSize() reads 32 KB first, then 256 KB, then the whole file: a prefix gives either no size or the same size, so results do not change. Tests: every format and read stage equals the whole-file answer; concurrency 1 and 8 build the same index and cache; the limiter keeps its bound and per-task errors. (BL-0043) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d6254a8be5
commit
5e14e54286
7 files changed
+169
-22
No files matched your search
@@ -364,7 +364,7 @@ docker compose up -d --build
|
||||
| Aspect | Behaviour |
|
||||
|--------|-----------|
|
||||
| Library size | The index keeps a small object per page (file name, size, tags), in the order of a few hundred bytes. Hundreds of thousands of pages fit in a modest container. Unchanged chapters cost one `stat` + one small read per scan |
|
||||
| First scan | Reads ≤ 256 KB per image once. Later scans use the cache |
|
||||
| First scan | Reads 32 KB per image once (≤ 256 KB, or the whole file for JPEGs with huge EXIF/ICC blocks), `SCAN_CONCURRENCY` (8) at a time. Later scans use the cache |
|
||||
| User data | `db.json` is rewritten as a whole, at most about twice per second while people read. Comfortable up to a few MB (a household or small group, thousands of chapters read) |
|
||||
| Concurrency | One process, one writer. No clustering or horizontal scaling by design |
|
||||
| Images | Served as stored. No resizing or thumbnails, so large covers make the library page heavy |
|
||||
|
||||
Reference in new issue
Block a user