Files

T

ddidderr 1923ff2a6f

feat: support parallel multi-file uploads

The browser upload flow was built around one selected file and one global
upload state. That made the existing chunk pool useful for a single file, but
users could not start several selected files at the same time.

Refactor the browser state into per-file upload items. Each selected file now
has its own upload record, completed-chunk set, abort controller, retry state,
progress row, and saved IndexedDB resume record. The picker accepts multiple
files, `Start all` and `Resume all` use a bounded file-level pool, and each file
keeps the existing bounded chunk pool. This keeps parallel uploads useful
without letting one large selection create unbounded request fan-out.

Keep the server API unchanged. Each file still receives a separate server upload
id, and server-side progress remains authoritative before any missing chunks are
scheduled. Terminal conflicts still stop the affected file without overwriting
completed data.

Update the user-facing markup, styles, project docs, and test checklist for the
multi-file scheduler. Add a server regression test that interleaves two uploads
and verifies the completed files contain exactly their own bytes.

Test Plan:
- just check
- git diff --check

2026-05-30 18:32:29 +02:00

3.4 KiB

Raw Blame History

upl

upl is a small personal resumable upload service. The intended deployment is:

browser -> nginx -> upl Rust server -> local filesystem

The server writes upload chunks directly into an inaccessible temp file at their final offsets. Once every chunk is present, completion atomically renames that temp file into the completed upload directory.

Project Structure

upl
  Rust server
    src/main.rs        binary entrypoint and listener setup
    src/app.rs         Axum router, shared state, static file service
    src/api.rs         HTTP handlers and API error responses
    src/model.rs       JSON request, response, and metadata shapes
    src/storage.rs     local filesystem layout, offset writes, and final rename
    src/lib.rs         library surface used by integration tests
  Browser UI
    static/index.html  upload tool markup
    static/styles.css  responsive tool styling
    static/app.js      multi-file scheduler, retries, and browser resume state
  Deployment
    deploy/nginx/      nginx reverse proxy example
    scripts/           reusable local smoke tests
  Validation
    tests/             integration tests for server behavior
    TESTS.md           reusable manual and automated test checklist

Configuration

--bind sets the listen address. It overrides UPL_BIND and defaults to 127.0.0.1:3000.
--static-dir sets the static asset directory. It overrides UPL_STATIC_DIR and defaults to static/ inside this repository.
--data-dir sets the completed upload data root. Completed files land under its complete/ subdirectory. It overrides UPL_DATA_DIR and defaults to data/ inside this repository.
--temp-dir sets the directory for upload metadata, completion markers, and inaccessible temp upload files. It overrides UPL_TEMP_DIR and defaults to <data-dir>/staging.
upl --help prints the full argument help text.
The server accepts request bodies up to 64 MiB, which leaves room for the planned 16 MiB upload chunks and matches the nginx example in PLAN.md.
Keep UPL_TEMP_DIR on the same filesystem as <data-dir>/complete so completion can promote files with an atomic rename.

Common Commands

Use the justfile for routine tasks:

just check
just run

just check also syntax-checks the static browser JavaScript with node.

Browser Uploads

The browser UI accepts multiple selected files. Start all runs up to three file uploads at the same time, and each file still uploads up to three chunks concurrently. Every selected file keeps its own upload id, progress markers, abort controller, retry state, and saved IndexedDB resume record.

nginx

Run upl on localhost and put nginx in front of it for TLS and access control:

UPL_BIND=127.0.0.1:3000 \
UPL_DATA_DIR=/srv/upl/data \
UPL_TEMP_DIR=/srv/upl/data/staging \
upl

Use deploy/nginx/upl.conf.example as the starting point for the nginx site. Before exposing the service, replace the certificate paths and add a protection layer such as HTTP basic auth, an IP allowlist, or VPN-only access. The nginx example aliases only /srv/upl/data/complete; do not expose UPL_TEMP_DIR.

For a local Docker-based reverse-proxy smoke test:

just nginx-smoke

The smoke test binds the Rust server to 0.0.0.0 so the nginx container can reach it through Docker's host gateway. The production nginx example keeps the server bound to localhost.

3.4 KiB Raw Blame History