ddidderr
1923ff2a6f
The browser upload flow was built around one selected file and one global upload state. That made the existing chunk pool useful for a single file, but users could not start several selected files at the same time. Refactor the browser state into per-file upload items. Each selected file now has its own upload record, completed-chunk set, abort controller, retry state, progress row, and saved IndexedDB resume record. The picker accepts multiple files, `Start all` and `Resume all` use a bounded file-level pool, and each file keeps the existing bounded chunk pool. This keeps parallel uploads useful without letting one large selection create unbounded request fan-out. Keep the server API unchanged. Each file still receives a separate server upload id, and server-side progress remains authoritative before any missing chunks are scheduled. Terminal conflicts still stop the affected file without overwriting completed data. Update the user-facing markup, styles, project docs, and test checklist for the multi-file scheduler. Add a server regression test that interleaves two uploads and verifies the completed files contain exactly their own bytes. Test Plan: - just check - git diff --check
97 lines
3.4 KiB
Markdown
97 lines
3.4 KiB
Markdown
# upl
|
|
|
|
`upl` is a small personal resumable upload service. The intended deployment is:
|
|
|
|
```text
|
|
browser -> nginx -> upl Rust server -> local filesystem
|
|
```
|
|
|
|
The server writes upload chunks directly into an inaccessible temp file at
|
|
their final offsets. Once every chunk is present, completion atomically renames
|
|
that temp file into the completed upload directory.
|
|
|
|
## Project Structure
|
|
|
|
```text
|
|
upl
|
|
Rust server
|
|
src/main.rs binary entrypoint and listener setup
|
|
src/app.rs Axum router, shared state, static file service
|
|
src/api.rs HTTP handlers and API error responses
|
|
src/model.rs JSON request, response, and metadata shapes
|
|
src/storage.rs local filesystem layout, offset writes, and final rename
|
|
src/lib.rs library surface used by integration tests
|
|
Browser UI
|
|
static/index.html upload tool markup
|
|
static/styles.css responsive tool styling
|
|
static/app.js multi-file scheduler, retries, and browser resume state
|
|
Deployment
|
|
deploy/nginx/ nginx reverse proxy example
|
|
scripts/ reusable local smoke tests
|
|
Validation
|
|
tests/ integration tests for server behavior
|
|
TESTS.md reusable manual and automated test checklist
|
|
```
|
|
|
|
## Configuration
|
|
|
|
- `--bind` sets the listen address. It overrides `UPL_BIND` and defaults to
|
|
`127.0.0.1:3000`.
|
|
- `--static-dir` sets the static asset directory. It overrides `UPL_STATIC_DIR`
|
|
and defaults to `static/` inside this repository.
|
|
- `--data-dir` sets the completed upload data root. Completed files land under
|
|
its `complete/` subdirectory. It overrides `UPL_DATA_DIR` and defaults to
|
|
`data/` inside this repository.
|
|
- `--temp-dir` sets the directory for upload metadata, completion markers, and
|
|
inaccessible temp upload files. It overrides `UPL_TEMP_DIR` and defaults to
|
|
`<data-dir>/staging`.
|
|
- `upl --help` prints the full argument help text.
|
|
- The server accepts request bodies up to 64 MiB, which leaves room for the
|
|
planned 16 MiB upload chunks and matches the nginx example in `PLAN.md`.
|
|
- Keep `UPL_TEMP_DIR` on the same filesystem as `<data-dir>/complete` so
|
|
completion can promote files with an atomic rename.
|
|
|
|
## Common Commands
|
|
|
|
Use the `justfile` for routine tasks:
|
|
|
|
```sh
|
|
just check
|
|
just run
|
|
```
|
|
|
|
`just check` also syntax-checks the static browser JavaScript with `node`.
|
|
|
|
## Browser Uploads
|
|
|
|
The browser UI accepts multiple selected files. `Start all` runs up to three
|
|
file uploads at the same time, and each file still uploads up to three chunks
|
|
concurrently. Every selected file keeps its own upload id, progress markers,
|
|
abort controller, retry state, and saved IndexedDB resume record.
|
|
|
|
## nginx
|
|
|
|
Run `upl` on localhost and put nginx in front of it for TLS and access control:
|
|
|
|
```sh
|
|
UPL_BIND=127.0.0.1:3000 \
|
|
UPL_DATA_DIR=/srv/upl/data \
|
|
UPL_TEMP_DIR=/srv/upl/data/staging \
|
|
upl
|
|
```
|
|
|
|
Use `deploy/nginx/upl.conf.example` as the starting point for the nginx site.
|
|
Before exposing the service, replace the certificate paths and add a protection
|
|
layer such as HTTP basic auth, an IP allowlist, or VPN-only access. The nginx
|
|
example aliases only `/srv/upl/data/complete`; do not expose `UPL_TEMP_DIR`.
|
|
|
|
For a local Docker-based reverse-proxy smoke test:
|
|
|
|
```sh
|
|
just nginx-smoke
|
|
```
|
|
|
|
The smoke test binds the Rust server to `0.0.0.0` so the nginx container can
|
|
reach it through Docker's host gateway. The production nginx example keeps the
|
|
server bound to localhost.
|