Completed uploads used to copy every staged chunk into a second file before
renaming the result into data/complete. That doubled write volume and required
peak disk space for both the chunk set and the final file.
Write each chunk directly into one private temp upload file at its final offset
instead. After a chunk write succeeds, record a tiny durable completion marker
for progress and resume scans. Completion now verifies the temp file length and
all markers, then renames the temp file into the completed upload directory.
Add UPL_TEMP_DIR and --temp-dir so operators can choose where upload metadata,
markers, and temp files live. The default remains data/staging, and docs call
out that the temp directory must be on the same filesystem as data/complete for
atomic promotion. The nginx example now aliases only the completed upload
directory, and the smoke test verifies that final-file alias.
This keeps the existing length-based validation model; it does not add per-chunk
hashing.
Test Plan:
- just check
- just nginx-smoke
- cargo clippy && cargo clippy --benches && cargo clippy --tests
- cargo +nightly fmt --all
- cargo clippy && cargo clippy --benches && cargo clippy --tests
Refs: none
Add focused regression coverage for validation rules called out in PLAN.md.
Chunk upload tests now prove that an otherwise valid upload rejects an
out-of-range chunk index through the HTTP API. Completion tests now prove that a
manually corrupted chunk file is not assembled into a final file.
Update TESTS.md so the reusable checklist reflects these automated proofs.
Test Plan:
- just check
- just nginx-smoke
Refs: PLAN.md validation checklist
Add the chunk upload and progress APIs from PLAN.md. PUT
/api/uploads/{id}/chunks/{index} now accepts raw octet-stream bodies, validates
unknown upload ids, out-of-range chunk indexes, and exact chunk lengths, then
writes through a temporary .part.tmp path before renaming the completed chunk
into place. Re-uploading an already-complete chunk is idempotent when the
existing file length matches the expected length.
GET /api/uploads/{id} now reports server-authoritative progress by scanning the
chunk directory and only counting chunk files whose lengths match metadata. The
router also raises Axum's request body limit to 64 MiB so the planned 16 MiB
chunks can reach the handler, matching the nginx deployment guidance.
Document the chunk storage responsibility and extend the reusable test checklist
with the new progress and validation coverage.
Test Plan:
- just check
Refs: PLAN.md milestones 3 and 4
ddidderr
