Files

T

feat: write chunks directly to temp upload files

Completed uploads used to copy every staged chunk into a second file before
renaming the result into data/complete. That doubled write volume and required
peak disk space for both the chunk set and the final file.

Write each chunk directly into one private temp upload file at its final offset
instead. After a chunk write succeeds, record a tiny durable completion marker
for progress and resume scans. Completion now verifies the temp file length and
all markers, then renames the temp file into the completed upload directory.

Add UPL_TEMP_DIR and --temp-dir so operators can choose where upload metadata,
markers, and temp files live. The default remains data/staging, and docs call
out that the temp directory must be on the same filesystem as data/complete for
atomic promotion. The nginx example now aliases only the completed upload
directory, and the smoke test verifies that final-file alias.

This keeps the existing length-based validation model; it does not add per-chunk
hashing.

Test Plan:
- just check
- just nginx-smoke
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt --all
- cargo clippy && cargo clippy --benches && cargo clippy --tests

Refs: none

2026-05-30 18:10:05 +02:00

2.5 KiB

Raw Blame History

Test Scenarios

Keep this file as the reusable verification checklist while implementing PLAN.md.

Automated

just check
- Runs formatting, all Rust tests, and clippy.
- Current coverage:
  - GET / serves the static browser page.
  - GET /healthz reports ok.
  - POST /api/uploads creates meta.json, a temp upload file, and a completion-marker directory.
  - POST /api/uploads rejects an empty file name.
  - PUT /api/uploads/:id/chunks/:index writes validated chunks into the temp upload file and records completion markers.
  - PUT /api/uploads/:id/chunks/:index rejects wrong-size chunks.
  - PUT /api/uploads/:id/chunks/:index rejects out-of-range indexes.
  - PUT /api/uploads/:id/chunks/:index accepts duplicate chunks.
  - GET /api/uploads/:id reports completed chunks from disk markers.
  - POST /api/uploads/:id/complete renames the verified temp upload file and removes staging data.
  - POST /api/uploads/:id/complete rejects incomplete uploads.
  - POST /api/uploads/:id/complete rejects tampered temp upload files.
  - static/app.js passes node --check.
just nginx-smoke
- Runs upl behind nginx in Docker.
- Uploads a 17 MiB file through nginx.
- Restarts the Rust backend mid-upload, resumes through nginx, completes, and compares SHA-256 hashes.
- Serves the completed file through nginx's final-upload alias.

Manual

These scenarios come from PLAN.md and remain useful for real browser and deployment retests.

Upload a small file in one pass.
Upload a file larger than one chunk.
Kill the browser tab mid-upload and resume.
Restart the Rust server mid-upload and resume.
Interrupt the network and resume.
Pause from the browser controls and resume.
Reload the page and resume from the pending upload list.
In a browser with the File System Access API, resume without reselecting the file after granting read permission.
In a browser without the File System Access API, resume after reselecting the same file.
Retry a duplicate chunk and confirm it is accepted idempotently.
Attempt an invalid chunk index and confirm it is rejected.
Attempt a wrong-size non-final chunk and confirm it is rejected.
Install deploy/nginx/upl.conf.example on the deployment host, add the real TLS certificate and access-control settings, and repeat the resume scenarios through the public nginx URL.
Complete an upload and compare the final file with the source file:

sha256sum source-file data/complete/uploaded-file

2.5 KiB Raw Blame History

Test Scenarios

Automated

Manual

2.5 KiB

Raw Blame History