ddidderr/upl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: ddidderr/upl:97a1f277717ba38bf783cd97af172a1361d2e249
Branches Tags
ddidderr/upl:main
..
compare: ddidderr/upl:547838e285737603e111f85128d0d608fdd1b597
Branches Tags
ddidderr/upl:main

10 Commits
97a1f27771 .. 547838e285

Author SHA1 Message Date
ddidderr 547838e285 fmt: improve formatting 2026-05-30 17:27:50 +02:00
ddidderr c3b47047d4 docs: tidy manual test checklist 
Clarify that the manual scenarios remain useful for real browser and deployment
retests, and keep the nginx deployment retest item separate from the checksum
command example.

Test Plan:
- just check
 2026-05-30 17:22:27 +02:00
ddidderr aeec7a0345 test: cover chunk validation edge cases 
Add focused regression coverage for validation rules called out in PLAN.md.
Chunk upload tests now prove that an otherwise valid upload rejects an
out-of-range chunk index through the HTTP API. Completion tests now prove that a
manually corrupted chunk file is not assembled into a final file.

Update TESTS.md so the reusable checklist reflects these automated proofs.

Test Plan:
- just check
- just nginx-smoke

Refs: PLAN.md validation checklist
 2026-05-30 17:22:27 +02:00
ddidderr 858f4d949c chore: add nginx deployment smoke test 
Add the nginx deployment artifact from PLAN.md. The example config keeps upl
behind nginx, sets client_max_body_size to 64 MiB, disables request buffering for
chunk uploads, forwards standard proxy headers, and leaves explicit placeholders
for TLS certificates and access control before public exposure.

Add just nginx-smoke as a reusable Docker-based verification. The script starts
upl with a temporary data directory, runs nginx as a reverse proxy, uploads a
17 MiB file through nginx, restarts the Rust backend mid-upload, confirms server
progress survives the restart through the proxy, uploads the remaining chunk,
completes the upload, and compares SHA-256 hashes.

Document the production nginx shape, the local Docker smoke-test caveat, and the
manual deployment retest scenario in TESTS.md.

Test Plan:
- bash -n scripts/nginx-smoke.sh
- just check
- just nginx-smoke

Refs: PLAN.md milestone 9
 2026-05-30 17:22:26 +02:00
ddidderr 35cd0657bd feat: add browser resumable upload client 
Replace the placeholder browser script with the PLAN.md upload flow. The static
UI now creates upload records, slices the selected file into fixed-size chunks,
uploads missing chunks with a concurrency pool of three workers, retries failed
chunks with exponential backoff, pauses via AbortController, and completes the
upload once the server has accepted every chunk.

Persist pending upload records in IndexedDB and render them in the page so a
reload can resume from server-authoritative progress. When the File System
Access API is available, the app stores a file handle and asks for read
permission during resume; when it is unavailable or permission is denied, the
same pending record resumes after the user reselects the matching file. Browser
state is helpful but not trusted: every resume starts by querying the server for
completed chunks.

Add a JavaScript syntax check to the justfile, update the static-page test and
documentation, and extend TESTS.md with the manual resume scenarios that still
need real-browser repetition.

Test Plan:
- just check
- UPL_BIND=127.0.0.1:39123 UPL_DATA_DIR=$(mktemp -d) cargo run
- curl -fsS http://127.0.0.1:39123/healthz
- curl -fsS http://127.0.0.1:39123/ | rg "Choose file|Pending uploads|app.js"
- firefox --headless --screenshot /tmp/upl-page.png http://127.0.0.1:39123/

Refs: PLAN.md milestones 5, 6, and 7
 2026-05-30 17:08:02 +02:00
ddidderr 5ca52b5780 feat: assemble completed uploads 
Implement POST /api/uploads/{id}/complete. The storage layer now reloads upload
metadata, verifies that every expected chunk exists with the exact expected
length, concatenates chunks in order into a temporary final file, flushes it,
and renames it into data/complete only after assembly succeeds.

The endpoint preserves staging data after completion, rejects incomplete uploads
with a conflict response, and refuses to overwrite an existing completed file.
This keeps failed or duplicate completion attempts explicit rather than silently
clobbering local files.

Extend the model, router, documentation, and test checklist for completion
responses and add integration coverage for successful assembly, incomplete
uploads, staging preservation, and duplicate completion conflicts.

Test Plan:
- just check

Refs: PLAN.md milestone 8
 2026-05-30 17:02:59 +02:00
ddidderr 1594c65d89 feat: store raw upload chunks 
Add the chunk upload and progress APIs from PLAN.md. PUT
/api/uploads/{id}/chunks/{index} now accepts raw octet-stream bodies, validates
unknown upload ids, out-of-range chunk indexes, and exact chunk lengths, then
writes through a temporary .part.tmp path before renaming the completed chunk
into place. Re-uploading an already-complete chunk is idempotent when the
existing file length matches the expected length.

GET /api/uploads/{id} now reports server-authoritative progress by scanning the
chunk directory and only counting chunk files whose lengths match metadata. The
router also raises Axum's request body limit to 64 MiB so the planned 16 MiB
chunks can reach the handler, matching the nginx deployment guidance.

Document the chunk storage responsibility and extend the reusable test checklist
with the new progress and validation coverage.

Test Plan:
- just check

Refs: PLAN.md milestones 3 and 4
 2026-05-30 17:00:42 +02:00
ddidderr 24ecdbd251 feat: persist upload creation metadata 
Add the first upload API endpoint from PLAN.md. POST /api/uploads now
validates the requested file name, generates a server-owned upload id, creates
the staging and complete directory layout, and writes durable meta.json before
returning chunk scheduling details to the browser.

Keep filesystem layout knowledge in storage.rs so later chunk upload and
completion work can reuse the same boundary. API handlers translate storage
errors into JSON HTTP responses without leaking layout details into the router.

Document the new modules and UPL_DATA_DIR configuration, and extend TESTS.md
with the automated creation coverage.

Test Plan:
- just check

Refs: PLAN.md milestone 2
 2026-05-30 16:57:48 +02:00
ddidderr a3f369f437 feat: serve static upload app from axum 
Introduce the first PLAN.md milestone: replace the hello-world binary with
an Axum server that binds to localhost by default, exposes a health endpoint,
and serves the static browser UI from the repository's static directory. The
router is available through the library crate so integration tests can exercise
server behavior without opening a network listener.

Add a justfile for routine validation and document the initial project shape,
configuration knobs, and reusable test checklist. The rustfmt config now uses
only stable options so the new formatting recipe runs without nightly warnings.

The upload API and resumable chunk behavior are intentionally left for later
milestones; the UI currently handles file selection only.

Test Plan:
- just check

Refs: PLAN.md milestone 1
 2026-05-30 16:54:42 +02:00
ddidderr 4527e23b8b docs: add resumable upload plan 
Document the minimal design for a personal large-file upload app that can
resume after browser, network, or server interruptions. The plan keeps the
first version intentionally small: one Rust server, one static browser UI,
filesystem-backed upload metadata, raw chunk uploads, and no database or
third-party resumable upload protocol.

The deployment notes include nginx as the external TLS and access-control
layer, with the Rust server bound behind it and upload-specific proxy settings
called out.

Test Plan:
- git diff --cached --check

Refs: user request
 2026-05-30 16:46:27 +02:00
22 changed files with 3934 additions and 2 deletions
Expand all files Collapse all files
Cargo.lock
Generated
+1075
View File
@@ -0,0 +1,1075 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "anyhow"
version = "1.0.102"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f202df86484c868dbad7eaa557ef785d5c66295e41b460ef922eca0723b842c"
[[package]]
name = "atomic-waker"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0"
[[package]]
name = "axum"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "31b698c5f9a010f6573133b09e0de5408834d0c82f8d7475a89fc1867a71cd90"
dependencies = [
"axum-core",
"bytes",
"form_urlencoded",
"futures-util",
"http",
"http-body",
"http-body-util",
"hyper",
"hyper-util",
"itoa",
"matchit",
"memchr",
"mime",
"percent-encoding",
"pin-project-lite",
"serde_core",
"serde_json",
"serde_path_to_error",
"serde_urlencoded",
"sync_wrapper",
"tokio",
"tower",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
name = "axum-core"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1"
dependencies = [
"bytes",
"futures-core",
"http",
"http-body",
"http-body-util",
"mime",
"pin-project-lite",
"sync_wrapper",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
name = "bitflags"
version = "2.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4512299f36f043ab09a583e57bceb5a5aab7a73db1805848e8fef3c9e8c78b3"
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "bytes"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e748733b7cbc798e1434b6ac524f0c1ff2ab456fe201501e6497c8417a4fc33"
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "deranged"
version = "0.5.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7cd812cc2bc1d69d4764bd80df88b4317eaef9e773c75226407d9bc0876b211c"
dependencies = [
"powerfmt",
"serde_core",
]
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "fastrand"
version = "2.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f1f227452a390804cdb637b74a86990f2a7d7ba4b7d5693aac9b4dd6defd8d6"
[[package]]
name = "foldhash"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2"
[[package]]
name = "form_urlencoded"
version = "1.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb4cb245038516f5f85277875cdaa4f7d2c9a0fa0468de06ed190163b1581fcf"
dependencies = [
"percent-encoding",
]
[[package]]
name = "futures-channel"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07bbe89c50d7a535e539b8c17bc0b49bdb77747034daa8087407d655f3f7cc1d"
dependencies = [
"futures-core",
]
[[package]]
name = "futures-core"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e3450815272ef58cec6d564423f6e755e25379b217b0bc688e295ba24df6b1d"
[[package]]
name = "futures-sink"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c39754e157331b013978ec91992bde1ac089843443c49cbc7f46150b0fad0893"
[[package]]
name = "futures-task"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "037711b3d59c33004d3856fbdc83b99d4ff37a24768fa1be9ce3538a1cde4393"
[[package]]
name = "futures-util"
version = "0.3.32"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "389ca41296e6190b48053de0321d02a77f32f8a5d2461dd38762c0593805c6d6"
dependencies = [
"futures-core",
"futures-task",
"pin-project-lite",
"slab",
]
[[package]]
name = "getrandom"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0de51e6874e94e7bf76d726fc5d13ba782deca734ff60d5bb2fb2607c7406555"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"wasip2",
"wasip3",
]
[[package]]
name = "hashbrown"
version = "0.15.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1"
dependencies = [
"foldhash",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "http"
version = "1.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8be7462df143984c4598a256ef469b251d7d7f9e271135073e78fc535414f3d0"
dependencies = [
"bytes",
"itoa",
]
[[package]]
name = "http-body"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184"
dependencies = [
"bytes",
"http",
]
[[package]]
name = "http-body-util"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b021d93e26becf5dc7e1b75b1bed1fd93124b374ceb73f43d4d4eafec896a64a"
dependencies = [
"bytes",
"futures-core",
"http",
"http-body",
"pin-project-lite",
]
[[package]]
name = "http-range-header"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9171a2ea8a68358193d15dd5d70c1c10a2afc3e7e4c5bc92bc9f025cebd7359c"
[[package]]
name = "httparse"
version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6dbf3de79e51f3d586ab4cb9d5c3e2c14aa28ed23d180cf89b4df0454a69cc87"
[[package]]
name = "httpdate"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df3b46402a9d5adb4c86a0cf463f42e19994e3ee891101b1841f30a545cb49a9"
[[package]]
name = "hyper"
version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55281c53a1894c864990125767da440a4e630446785086f52523b20033b74498"
dependencies = [
"atomic-waker",
"bytes",
"futures-channel",
"futures-core",
"http",
"http-body",
"httparse",
"httpdate",
"itoa",
"pin-project-lite",
"smallvec",
"tokio",
]
[[package]]
name = "hyper-util"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "96547c2556ec9d12fb1578c4eaf448b04993e7fb79cbaad930a656880a6bdfa0"
dependencies = [
"bytes",
"http",
"http-body",
"hyper",
"pin-project-lite",
"tokio",
"tower-service",
]
[[package]]
name = "id-arena"
version = "2.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3d3067d79b975e8844ca9eb072e16b31c3c1c36928edf9c6789548c524d0d954"
[[package]]
name = "indexmap"
version = "2.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
"serde",
"serde_core",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "js-sys"
version = "0.3.99"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "142bc4740e452c1e57ade0cbc129f139c9093e354346f0872ef985f4f5cf5f11"
dependencies = [
"cfg-if",
"futures-util",
"once_cell",
"wasm-bindgen",
]
[[package]]
name = "leb128fmt"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "09edd9e8b54e49e587e4f6295a7d29c3ea94d469cb40ab8ca70b288248a81db2"
[[package]]
name = "libc"
version = "0.2.186"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66"
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "lock_api"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965"
dependencies = [
"scopeguard",
]
[[package]]
name = "log"
version = "0.4.30"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "616ec5685824bcc94416c6d4a7a446eea774a31efd7062c8480ba6fd06d7a6e5"
[[package]]
name = "matchit"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47e1ffaa40ddd1f3ed91f717a33c8c0ee23fff369e3aa8772b9605cc1d22f4c3"
[[package]]
name = "memchr"
version = "2.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6b947ae49db0d222b1dbc6b113ce7248a3fc3a6ca21b696717bfc000ba4484d8"
[[package]]
name = "mime"
version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6877bb514081ee2a7ff5ef9de3281f14a4dd4bceac4c09388074a6b5df8a139a"
[[package]]
name = "mime_guess"
version = "2.0.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f7c44f8e672c00fe5308fa235f821cb4198414e1c77935c1ab6948d3fd78550e"
dependencies = [
"mime",
"unicase",
]
[[package]]
name = "mio"
version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "02bd0af71c67b473010cbbc60715ee815645a4dc942899111f494b4b737d6fda"
dependencies = [
"libc",
"wasi",
"windows-sys",
]
[[package]]
name = "num-conv"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "521739c6d2bac4aa25192232afe6841231376b2b26d4d9fae5ecf8ca5772e441"
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "parking_lot"
version = "0.12.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93857453250e3077bd71ff98b6a65ea6621a19bb0f559a85248955ac12c45a1a"
dependencies = [
"lock_api",
"parking_lot_core",
]
[[package]]
name = "parking_lot_core"
version = "0.9.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2621685985a2ebf1c516881c026032ac7deafcda1a2c9b7850dc81e3dfcb64c1"
dependencies = [
"cfg-if",
"libc",
"redox_syscall",
"smallvec",
"windows-link",
]
[[package]]
name = "percent-encoding"
version = "2.3.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9b4f627cb1b25917193a259e49bdad08f671f8d9708acfd5fe0a8c1455d87220"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "powerfmt"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "439ee305def115ba05938db6eb1644ff94165c5ab5e9420d1c1bcedbba909391"
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn",
]
[[package]]
name = "proc-macro2"
version = "1.0.106"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "redox_syscall"
version = "0.5.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed2bf2547551a7053d6fdfafda3f938979645c44812fbfcda098faae3f1a362d"
dependencies = [
"bitflags",
]
[[package]]
name = "rustix"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys",
]
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "ryu"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f"
[[package]]
name = "scopeguard"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
[[package]]
name = "semver"
version = "1.0.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd"
[[package]]
name = "serde"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.228"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "serde_json"
version = "1.0.150"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8014e44b4736ed0538adeecded0fce2a272f22dc9578a7eb6b2d9993c74cfb9"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "serde_path_to_error"
version = "0.1.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "10a9ff822e371bb5403e391ecd83e182e0e77ba7f6fe0160b795797109d1b457"
dependencies = [
"itoa",
"serde",
"serde_core",
]
[[package]]
name = "serde_urlencoded"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3491c14715ca2294c4d6a88f15e84739788c1d030eed8c110436aafdaa2f3fd"
dependencies = [
"form_urlencoded",
"itoa",
"ryu",
"serde",
]
[[package]]
name = "signal-hook-registry"
version = "1.4.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b"
dependencies = [
"errno",
"libc",
]
[[package]]
name = "slab"
version = "0.4.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5"
[[package]]
name = "smallvec"
version = "1.15.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03"
[[package]]
name = "socket2"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "52d1cfed4120b4d927bf7c0f86d2087a4a7d6027c906d9f9d525a80573b9be51"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "syn"
version = "2.0.117"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "sync_wrapper"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0bf256ce5efdfa370213c1dabab5935a12e49f2c58d15e9eac2870d3b4f27263"
[[package]]
name = "tempfile"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom",
"once_cell",
"rustix",
"windows-sys",
]
[[package]]
name = "time"
version = "0.3.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "743bd48c283afc0388f9b8827b976905fb217ad9e647fae3a379a9283c4def2c"
dependencies = [
"deranged",
"itoa",
"num-conv",
"powerfmt",
"serde_core",
"time-core",
"time-macros",
]
[[package]]
name = "time-core"
version = "0.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7694e1cfe791f8d31026952abf09c69ca6f6fa4e1a1229e18988f06a04a12dca"
[[package]]
name = "time-macros"
version = "0.2.27"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2e70e4c5a0e0a8a4823ad65dfe1a6930e4f4d756dcd9dd7939022b5e8c501215"
dependencies = [
"num-conv",
"time-core",
]
[[package]]
name = "tokio"
version = "1.52.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fc7f01b389ac15039e4dc9531aa973a135d7a4135281b12d7c1bc79fd57fffe"
dependencies = [
"bytes",
"libc",
"mio",
"parking_lot",
"pin-project-lite",
"signal-hook-registry",
"socket2",
"tokio-macros",
"windows-sys",
]
[[package]]
name = "tokio-macros"
version = "2.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "385a6cb71ab9ab790c5fe8d67f1645e6c450a7ce006a33de03daa956cf70a496"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "tokio-util"
version = "0.7.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ae9cec805b01e8fc3fd2fe289f89149a9b66dd16786abd8b19cfa7b48cb0098"
dependencies = [
"bytes",
"futures-core",
"futures-sink",
"pin-project-lite",
"tokio",
]
[[package]]
name = "tower"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebe5ef63511595f1344e2d5cfa636d973292adc0eec1f0ad45fae9f0851ab1d4"
dependencies = [
"futures-core",
"futures-util",
"pin-project-lite",
"sync_wrapper",
"tokio",
"tower-layer",
"tower-service",
"tracing",
]
[[package]]
name = "tower-http"
version = "0.6.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4cfcf7e2740e6fc6d4d688b4ef00650406bb94adf4731e43c096c3a19fe40840"
dependencies = [
"bitflags",
"bytes",
"futures-core",
"futures-util",
"http",
"http-body",
"http-body-util",
"http-range-header",
"httpdate",
"mime",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"tokio",
"tokio-util",
"tower-layer",
"tower-service",
]
[[package]]
name = "tower-layer"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "121c2a6cda46980bb0fcd1647ffaf6cd3fc79a013de288782836f6df9c48780e"
[[package]]
name = "tower-service"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"log",
"pin-project-lite",
"tracing-core",
]
[[package]]
name = "tracing-core"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
]
[[package]]
name = "unicase"
version = "2.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "dbc4bc3a9f746d862c45cb89d705aa10f187bb96c76001afab07a0d35ce60142"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "unicode-xid"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ebc1c04c71510c7f702b52b7c350734c9ff1295c464a03335b00bb84fc54f853"
[[package]]
name = "upl"
version = "0.1.0"
dependencies = [
"axum",
"http-body-util",
"serde",
"serde_json",
"tempfile",
"time",
"tokio",
"tower",
"tower-http",
"uuid",
]
[[package]]
name = "uuid"
version = "1.23.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d258b83ceec21034727ecee8c382cfa6c3e133699b0742c64571814fb420c9f7"
dependencies = [
"getrandom",
"js-sys",
"serde_core",
"wasm-bindgen",
]
[[package]]
name = "wasi"
version = "0.11.1+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b"
[[package]]
name = "wasip2"
version = "1.0.3+wasi-0.2.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "20064672db26d7cdc89c7798c48a0fdfac8213434a1186e5ef29fd560ae223d6"
dependencies = [
"wit-bindgen 0.57.1",
]
[[package]]
name = "wasip3"
version = "0.4.0+wasi-0.3.0-rc-2026-01-06"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5428f8bf88ea5ddc08faddef2ac4a67e390b88186c703ce6dbd955e1c145aca5"
dependencies = [
"wit-bindgen 0.51.0",
]
[[package]]
name = "wasm-bindgen"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ed04576f974d2b2fba0f38c51dbc5518011e38c36bf1143164be765528fd409"
dependencies = [
"cfg-if",
"once_cell",
"rustversion",
"wasm-bindgen-macro",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-macro"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "916151b09da36bd82f6615cbf3a419e2f0ba23a03c6160e8e92eb6bd4aa1dec6"
dependencies = [
"quote",
"wasm-bindgen-macro-support",
]
[[package]]
name = "wasm-bindgen-macro-support"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "299047362ccbfce148b67ab7e73349f77748e00c8296f9542adfad2ad82c5c5e"
dependencies = [
"bumpalo",
"proc-macro2",
"quote",
"syn",
"wasm-bindgen-shared",
]
[[package]]
name = "wasm-bindgen-shared"
version = "0.2.122"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a929b2c61f11ba3e9bc35b50c1f25cb38e0e892c0c231ae2b8cf78d5dad4437"
dependencies = [
"unicode-ident",
]
[[package]]
name = "wasm-encoder"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "990065f2fe63003fe337b932cfb5e3b80e0b4d0f5ff650e6985b1048f62c8319"
dependencies = [
"leb128fmt",
"wasmparser",
]
[[package]]
name = "wasm-metadata"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb0e353e6a2fbdc176932bbaab493762eb1255a7900fe0fea1a2f96c296cc909"
dependencies = [
"anyhow",
"indexmap",
"wasm-encoder",
"wasmparser",
]
[[package]]
name = "wasmparser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "47b807c72e1bac69382b3a6fb3dbe8ea4c0ed87ff5629b8685ae6b9a611028fe"
dependencies = [
"bitflags",
"hashbrown 0.15.5",
"indexmap",
"semver",
]
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "wit-bindgen"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d7249219f66ced02969388cf2bb044a09756a083d0fab1e566056b04d9fbcaa5"
dependencies = [
"wit-bindgen-rust-macro",
]
[[package]]
name = "wit-bindgen"
version = "0.57.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ebf944e87a7c253233ad6766e082e3cd714b5d03812acc24c318f549614536e"
[[package]]
name = "wit-bindgen-core"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea61de684c3ea68cb082b7a88508a8b27fcc8b797d738bfc99a82facf1d752dc"
dependencies = [
"anyhow",
"heck",
"wit-parser",
]
[[package]]
name = "wit-bindgen-rust"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b7c566e0f4b284dd6561c786d9cb0142da491f46a9fbed79ea69cdad5db17f21"
dependencies = [
"anyhow",
"heck",
"indexmap",
"prettyplease",
"syn",
"wasm-metadata",
"wit-bindgen-core",
"wit-component",
]
[[package]]
name = "wit-bindgen-rust-macro"
version = "0.51.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c0f9bfd77e6a48eccf51359e3ae77140a7f50b1e2ebfe62422d8afdaffab17a"
dependencies = [
"anyhow",
"prettyplease",
"proc-macro2",
"quote",
"syn",
"wit-bindgen-core",
"wit-bindgen-rust",
]
[[package]]
name = "wit-component"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9d66ea20e9553b30172b5e831994e35fbde2d165325bec84fc43dbf6f4eb9cb2"
dependencies = [
"anyhow",
"bitflags",
"indexmap",
"log",
"serde",
"serde_derive",
"serde_json",
"wasm-encoder",
"wasm-metadata",
"wasmparser",
"wit-parser",
]
[[package]]
name = "wit-parser"
version = "0.244.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecc8ac4bc1dc3381b7f59c34f00b67e18f910c2c0f50015669dde7def656a736"
dependencies = [
"anyhow",
"id-arena",
"indexmap",
"log",
"semver",
"serde",
"serde_derive",
"serde_json",
"unicode-xid",
"wasmparser",
]
[[package]]
name = "zmij"
version = "1.0.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8848ee67ecc8aedbaf3e4122217aff892639231befc6a1b58d29fff4c2cabaa"
Cargo.toml
+12
View File
@@ -4,6 +4,18 @@ version = "0.1.0"
edition = "2024" edition = "2024"
[dependencies] [dependencies]
axum = "0.8.9"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.150"
time = { version = "0.3.47", features = ["formatting", "serde"] }
tokio = { version = "1.52.3", features = ["full"] }
tower-http = { version = "0.6.11", features = ["fs"] }
uuid = { version = "1.23.2", features = ["serde", "v4"] }
[dev-dependencies]
http-body-util = "0.1.3"
tempfile = "3.27.0"
tower = { version = "0.5.3", features = ["util"] }
[lints.clippy] [lints.clippy]
pedantic = { level = "warn", priority = -1 } pedantic = { level = "warn", priority = -1 }
PLAN.md
+355
View File
@@ -0,0 +1,355 @@
# Resumable Upload Plan
## Goal
Build a small personal web app for uploading large files without losing
progress when the network drops, the tab closes, or the Rust server restarts.
The final deployment is:
```text
browser -> nginx -> upl Rust server -> local filesystem
```
The program should stay simple:
- one Rust server binary
- one static browser UI
- no database server
- no frontend framework
- no Tus/Uppy/Resumable.js for the first version
- local filesystem metadata as the source of truth
## Top-Level Design
### Browser
The browser owns file selection and chunk scheduling.
- Let the user pick one file.
- Slice it into fixed-size chunks with `Blob.slice()`.
- Upload a few chunks concurrently.
- Retry failed chunks with exponential backoff.
- Persist pending upload state in IndexedDB.
- Use the File System Access API when available so the same local file can be
reopened after a browser restart without making the user browse to it again.
### nginx
nginx owns TLS, external access control, and reverse proxying.
- Bind the Rust server to localhost only.
- Terminate HTTPS in nginx.
- Protect the app because it is a personal upload tool.
- Forward upload API requests to the Rust server without buffering whole request
bodies before they reach Rust.
### Rust Server
The Rust server owns upload identity, chunk validation, progress reporting, and
final assembly.
- Serve the static page.
- Create upload records.
- Accept raw binary chunk bodies.
- Store chunks on disk as they arrive.
- Report which chunks already exist.
- Assemble chunks into the final file once all chunks are present.
## Storage Layout
```text
data/
staging/
<upload_id>/
meta.json
chunks/
000000.part
000001.part
000002.part
complete/
<safe_file_name>
```
`meta.json` is the durable upload record:
```json
{
"id": "random-server-id",
"original_name": "movie.mkv",
"safe_name": "movie.mkv",
"size": 1234567890,
"last_modified": 1760000000000,
"chunk_size": 16777216,
"total_chunks": 74,
"created_at": "2026-05-30T16:00:00Z"
}
```
The server should generate `upload_id`. The browser should not invent the
primary upload identity from file metadata. File name, size, and modified time
are useful for display and sanity checks, but they are not unique enough to be
the durable server identity.
## HTTP API
Keep the API small and boring.
```text
GET /
POST /api/uploads
GET /api/uploads/:id
PUT /api/uploads/:id/chunks/:index
POST /api/uploads/:id/complete
```
### Create Upload
`POST /api/uploads`
Request:
```json
{
"name": "movie.mkv",
"size": 1234567890,
"last_modified": 1760000000000
}
```
Response:
```json
{
"upload_id": "random-server-id",
"chunk_size": 16777216,
"total_chunks": 74,
"completed_chunks": []
}
```
Start with a fixed chunk size of 16 MiB. This keeps request count reasonable
while making failed chunks cheap enough to retry.
### Query Progress
`GET /api/uploads/:id`
Response:
```json
{
"upload_id": "random-server-id",
"name": "movie.mkv",
"size": 1234567890,
"chunk_size": 16777216,
"total_chunks": 74,
"completed_chunks": [0, 1, 2, 5]
}
```
The server can compute `completed_chunks` by scanning the chunk directory and
checking file lengths. This avoids needing a database.
### Upload Chunk
`PUT /api/uploads/:id/chunks/:index`
Use a raw request body:
```http
Content-Type: application/octet-stream
```
Do not use multipart form uploads for chunks in the minimal version. Raw bytes
make the Rust handler simpler and avoid multipart parsing.
Server rules:
- reject unknown upload IDs
- reject out-of-range chunk indexes
- reject chunks with the wrong length
- allow the final chunk to be shorter than `chunk_size`
- write to `000123.part.tmp` first
- rename the temp file to `000123.part` only after the write succeeds
- make duplicate chunk uploads idempotent when the existing chunk has the
expected length
### Complete Upload
`POST /api/uploads/:id/complete`
The server should:
1. Load `meta.json`.
2. Verify every expected chunk exists.
3. Verify every chunk has the expected length.
4. Concatenate chunks in order into a temp final file.
5. Rename the temp final file into `data/complete/`.
6. Return the final file path or download URL.
The server should not delete staging data until assembly succeeds.
## Resume Flow
### First Upload
1. User selects a file.
2. Browser calls `POST /api/uploads`.
3. Browser stores the returned `upload_id` and file handle in IndexedDB.
4. Browser uploads missing chunks with a small concurrency pool.
5. Browser calls `/complete` when all chunks are uploaded.
### After Interruption
1. Browser loads pending upload records from IndexedDB.
2. Browser calls `GET /api/uploads/:id`.
3. Browser asks for read permission on the saved file handle.
4. Browser compares server `completed_chunks` with total chunks.
5. Browser uploads only missing chunks.
6. Browser calls `/complete`.
The server is authoritative. Browser state helps find the file again, but
server state decides what has actually been uploaded.
## Browser State
IndexedDB record:
```json
{
"upload_id": "random-server-id",
"name": "movie.mkv",
"size": 1234567890,
"last_modified": 1760000000000,
"chunk_size": 16777216,
"total_chunks": 74,
"file_handle": "<FileSystemFileHandle>",
"updated_at": "2026-05-30T16:00:00Z"
}
```
If `showOpenFilePicker()` is unavailable, fall back to a normal
`<input type="file">`. That fallback can still resume server-side progress, but
the user must reselect the same file after a page reload.
## Upload Scheduler
Start with these defaults:
```text
chunk size: 16 MiB
concurrency: 3
max retries per chunk: 5
```
The scheduler should support:
- pause with `AbortController`
- resume by rebuilding the missing chunk list
- retry with exponential backoff
- visible progress based on verified completed chunks
Progress should be based on chunks the server has accepted, not bytes merely
sent by the browser.
## nginx Requirements
Example shape:
```nginx
server {
listen 443 ssl;
server_name uploads.example.com;
client_max_body_size 64m;
location / {
proxy_pass http://127.0.0.1:3000;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
}
}
```
Notes:
- `client_max_body_size` only needs to exceed the maximum single chunk size, not
the full file size.
- `proxy_request_buffering off` lets the Rust server receive upload bodies
directly instead of waiting for nginx to buffer the whole chunk first.
- Long timeouts are useful for slow links and large chunks.
- Add HTTP basic auth, an IP allowlist, VPN-only access, or another protection
layer before exposing this publicly.
## Rust Implementation Shape
Suggested crates:
- `axum` for HTTP routing
- `tokio` for async runtime and filesystem operations
- `serde` and `serde_json` for metadata
- `uuid` or `nanoid` for upload IDs
- `tower-http` for static file serving
Suggested modules:
```text
src/
main.rs
api.rs
storage.rs
model.rs
static_files.rs
```
`storage.rs` should be the only module that knows the on-disk layout.
## Validation
Manual checks for the MVP:
- upload a small file in one pass
- upload a file larger than one chunk
- kill the browser tab mid-upload and resume
- restart the Rust server mid-upload and resume
- interrupt the network and resume
- retry a duplicate chunk and confirm it is accepted idempotently
- attempt an invalid chunk index and confirm it is rejected
- attempt a wrong-size non-final chunk and confirm it is rejected
- complete an upload and compare the final file with the source file
Useful checksum command:
```sh
sha256sum source-file data/complete/uploaded-file
```
## Milestones
1. Serve a static page from Rust.
2. Add upload creation and on-disk metadata.
3. Add raw chunk upload and chunk validation.
4. Add progress query from existing chunk files.
5. Add browser chunk slicing and concurrency.
6. Add IndexedDB state.
7. Add File System Access API resume.
8. Add completion assembly.
9. Put the server behind nginx and verify resume still works.
## Explicit Non-Goals For The First Version
- multiple-user accounts
- cloud object storage
- encryption at rest
- background service worker upload
- content-addressed deduplication
- full-file hashing before upload
- Tus protocol compatibility
- drag-and-drop polish
- mobile browser support
These can be added later if they become useful, but they are unnecessary for a
correct personal uploader.
README.md
+77
View File
@@ -0,0 +1,77 @@
# upl
`upl` is a small personal resumable upload service. The intended deployment is:
```text
browser -> nginx -> upl Rust server -> local filesystem
```
The first implementation milestone provides the Rust server shell and static
browser UI. Upload metadata, chunk persistence, resume state, and completion
assembly are tracked in `PLAN.md` and will be added in later coherent slices.
## Project Structure
```text
upl
Rust server
src/main.rs binary entrypoint and listener setup
src/app.rs Axum router, shared state, static file service
src/api.rs HTTP handlers and API error responses
src/model.rs JSON request, response, and metadata shapes
src/storage.rs local filesystem layout, chunks, and assembly
src/lib.rs library surface used by integration tests
Browser UI
static/index.html upload tool markup
static/styles.css responsive tool styling
static/app.js upload scheduler, retries, and browser resume state
Deployment
deploy/nginx/ nginx reverse proxy example
scripts/ reusable local smoke tests
Validation
tests/ integration tests for server behavior
TESTS.md reusable manual and automated test checklist
```
## Configuration
- `UPL_BIND` sets the listen address. It defaults to `127.0.0.1:3000`.
- `UPL_STATIC_DIR` sets the static asset directory. It defaults to `static/`
inside this repository.
- `UPL_DATA_DIR` sets the upload data directory. It defaults to `data/` inside
this repository.
- The server accepts request bodies up to 64 MiB, which leaves room for the
planned 16 MiB upload chunks and matches the nginx example in `PLAN.md`.
## Common Commands
Use the `justfile` for routine tasks:
```sh
just check
just run
```
`just check` also syntax-checks the static browser JavaScript with `node`.
## nginx
Run `upl` on localhost and put nginx in front of it for TLS and access control:
```sh
UPL_BIND=127.0.0.1:3000 UPL_DATA_DIR=/srv/upl/data upl
```
Use `deploy/nginx/upl.conf.example` as the starting point for the nginx site.
Before exposing the service, replace the certificate paths and add a protection
layer such as HTTP basic auth, an IP allowlist, or VPN-only access.
For a local Docker-based reverse-proxy smoke test:
```sh
just nginx-smoke
```
The smoke test binds the Rust server to `0.0.0.0` so the nginx container can
reach it through Docker's host gateway. The production nginx example keeps the
server bound to localhost.
TESTS.md
+56
View File
@@ -0,0 +1,56 @@
# Test Scenarios
Keep this file as the reusable verification checklist while implementing
`PLAN.md`.
## Automated
- `just check`
- Runs formatting, all Rust tests, and clippy.
- Current coverage:
- `GET /` serves the static browser page.
- `GET /healthz` reports `ok`.
- `POST /api/uploads` creates `meta.json` and chunk directories.
- `POST /api/uploads` rejects an empty file name.
- `PUT /api/uploads/:id/chunks/:index` stores validated chunk files.
- `PUT /api/uploads/:id/chunks/:index` rejects wrong-size chunks.
- `PUT /api/uploads/:id/chunks/:index` rejects out-of-range indexes.
- `PUT /api/uploads/:id/chunks/:index` accepts duplicate chunks.
- `GET /api/uploads/:id` reports completed chunks from disk.
- `POST /api/uploads/:id/complete` assembles verified chunks.
- `POST /api/uploads/:id/complete` rejects incomplete uploads.
- `POST /api/uploads/:id/complete` rejects corrupt chunk files.
- `static/app.js` passes `node --check`.
- `just nginx-smoke`
- Runs upl behind nginx in Docker.
- Uploads a 17 MiB file through nginx.
- Restarts the Rust backend mid-upload, resumes through nginx, completes, and
compares SHA-256 hashes.
## Manual
These scenarios come from `PLAN.md` and remain useful for real browser and
deployment retests.
- Upload a small file in one pass.
- Upload a file larger than one chunk.
- Kill the browser tab mid-upload and resume.
- Restart the Rust server mid-upload and resume.
- Interrupt the network and resume.
- Pause from the browser controls and resume.
- Reload the page and resume from the pending upload list.
- In a browser with the File System Access API, resume without reselecting the
file after granting read permission.
- In a browser without the File System Access API, resume after reselecting the
same file.
- Retry a duplicate chunk and confirm it is accepted idempotently.
- Attempt an invalid chunk index and confirm it is rejected.
- Attempt a wrong-size non-final chunk and confirm it is rejected.
- Install `deploy/nginx/upl.conf.example` on the deployment host, add the real
TLS certificate and access-control settings, and repeat the resume scenarios
through the public nginx URL.
- Complete an upload and compare the final file with the source file:
```sh
sha256sum source-file data/complete/uploaded-file
```
deploy/nginx/upl.conf.example
+37
View File
@@ -0,0 +1,37 @@
# Production shape for browser -> nginx -> upl -> local filesystem.
#
# Replace server_name, certificate paths, and access control before exposing
# this app. Keep upl itself bound to 127.0.0.1.
upstream upl_backend {
server 127.0.0.1:3000;
keepalive 16;
}
server {
listen 443 ssl http2;
server_name uploads.example.com;
ssl_certificate /etc/letsencrypt/live/uploads.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/uploads.example.com/privkey.pem;
client_max_body_size 64m;
# Add HTTP basic auth, an IP allowlist, VPN-only access, or another
# protection layer before exposing this personal upload tool publicly.
# auth_basic "upl";
# auth_basic_user_file /etc/nginx/upl.htpasswd;
location / {
proxy_pass http://upl_backend;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Real-IP $remote_addr;
}
}
justfile
+24
View File
@@ -0,0 +1,24 @@
fmt:
cargo +nightly fmt --all
fd -e toml -x tombi format --quiet
test:
cargo test --all-targets
static-check:
node --check static/app.js
clippy:
cargo clippy --all-targets
nginx-smoke:
./scripts/nginx-smoke.sh
check:
just fmt
just test
just static-check
just clippy
run:
cargo run
rustfmt.toml
+1
View File
@@ -1,3 +1,4 @@
edition = "2024"
group_imports = "StdExternalCrate" group_imports = "StdExternalCrate"
imports_granularity = "Crate" imports_granularity = "Crate"
imports_layout = "HorizontalVertical" imports_layout = "HorizontalVertical"
scripts/nginx-smoke.sh
Executable
+152
View File
@@ -0,0 +1,152 @@
#!/usr/bin/env bash
set -euo pipefail
backend_port="${UPL_SMOKE_BACKEND_PORT:-39123}"
proxy_port="${UPL_SMOKE_PROXY_PORT:-39124}"
nginx_image="${NGINX_IMAGE:-nginx:stable-alpine}"
workspace_dir="$(pwd)"
mkdir -p "$workspace_dir/target/nginx-smoke"
tmp_dir="$(mktemp -d "$workspace_dir/target/nginx-smoke/run.XXXXXXXX")"
data_dir="$tmp_dir/data"
nginx_conf_dir="$tmp_dir/nginx-conf.d"
nginx_conf="$nginx_conf_dir/default.conf"
backend_log="$tmp_dir/backend.log"
source_file="$tmp_dir/source.bin"
chunk0="$tmp_dir/chunk0.part"
chunk1="$tmp_dir/chunk1.part"
backend_pid=""
nginx_container="upl-nginx-smoke-$$"
cleanup() {
if [[ -n "$backend_pid" ]] && kill -0 "$backend_pid" 2>/dev/null; then
kill "$backend_pid" 2>/dev/null || true
wait "$backend_pid" 2>/dev/null || true
fi
docker rm -f "$nginx_container" >/dev/null 2>&1 || true
rm -rf "$tmp_dir"
}
trap cleanup EXIT
start_backend() {
UPL_BIND="0.0.0.0:$backend_port" UPL_DATA_DIR="$data_dir" \
cargo run --quiet >"$backend_log" 2>&1 &
backend_pid="$!"
wait_for "http://127.0.0.1:$backend_port/healthz"
}
wait_for() {
local url="$1"
for _ in $(seq 1 80); do
if curl -fsS "$url" >/dev/null 2>&1; then
return 0
fi
sleep 0.1
done
echo "Timed out waiting for $url" >&2
if [[ -f "$backend_log" ]]; then
tail -n 80 "$backend_log" >&2 || true
fi
return 1
}
json_field() {
local field="$1"
node -e '
const field = process.argv[1];
let input = "";
process.stdin.setEncoding("utf8");
process.stdin.on("data", (chunk) => input += chunk);
process.stdin.on("end", () => {
const value = JSON.parse(input)[field];
if (value === undefined) process.exit(2);
process.stdout.write(String(value));
});
' "$field"
}
mkdir -p "$data_dir" "$nginx_conf_dir"
cat >"$nginx_conf" <<EOF
server {
listen $proxy_port;
client_max_body_size 64m;
location / {
proxy_pass http://host.docker.internal:$backend_port;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host \$host;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_set_header X-Real-IP \$remote_addr;
}
}
EOF
start_backend
docker run -d --rm \
--name "$nginx_container" \
--add-host host.docker.internal:host-gateway \
-p "127.0.0.1:$proxy_port:$proxy_port" \
-v "$nginx_conf_dir:/etc/nginx/conf.d:ro" \
"$nginx_image" >/dev/null
wait_for "http://127.0.0.1:$proxy_port/healthz"
dd if=/dev/urandom of="$source_file" bs=1M count=17 status=none
dd if="$source_file" of="$chunk0" bs=1M count=16 status=none
dd if="$source_file" of="$chunk1" bs=1M skip=16 status=none
size="$(wc -c <"$source_file" | tr -d ' ')"
create_response="$(
curl -fsS \
-H "Content-Type: application/json" \
-d "{\"name\":\"source.bin\",\"size\":$size,\"last_modified\":1760000000000}" \
"http://127.0.0.1:$proxy_port/api/uploads"
)"
upload_id="$(printf '%s' "$create_response" | json_field upload_id)"
curl -fsS -X PUT \
-H "Content-Type: application/octet-stream" \
--data-binary "@$chunk0" \
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/0" >/dev/null
progress_before_restart="$(
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
)"
printf '%s' "$progress_before_restart" | grep -q '"completed_chunks":\[0\]'
kill "$backend_pid"
wait "$backend_pid" 2>/dev/null || true
backend_pid=""
start_backend
progress_after_restart="$(
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
)"
printf '%s' "$progress_after_restart" | grep -q '"completed_chunks":\[0\]'
curl -fsS -X PUT \
-H "Content-Type: application/octet-stream" \
--data-binary "@$chunk1" \
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/1" >/dev/null
complete_response="$(
curl -fsS -X POST "http://127.0.0.1:$proxy_port/api/uploads/$upload_id/complete"
)"
complete_path="$(printf '%s' "$complete_response" | json_field file_path)"
source_hash="$(sha256sum "$source_file" | awk '{print $1}')"
complete_hash="$(sha256sum "$complete_path" | awk '{print $1}')"
if [[ "$source_hash" != "$complete_hash" ]]; then
echo "Checksum mismatch after nginx-proxied resume" >&2
exit 1
fi
echo "nginx smoke ok: $upload_id"
src/api.rs
+111
View File
@@ -0,0 +1,111 @@
use axum::{
Json,
body::Bytes,
extract::{Path, State},
http::StatusCode,
response::{IntoResponse, Response},
};
use serde::Serialize;
use crate::{
app::AppState,
model::{CompleteUploadResponse, CreateUploadRequest, CreateUploadResponse},
storage::StorageError,
};
/// Creates an upload record and persists its metadata before returning.
///
/// # Errors
///
/// Returns an API error when request validation fails or metadata cannot be
/// written to storage.
pub async fn create_upload(
State(state): State<AppState>,
Json(request): Json<CreateUploadRequest>,
) -> Result<Json<CreateUploadResponse>, ApiError> {
let meta = state.storage.create_upload(request).await?;
Ok(Json(meta.create_response()))
}
/// Returns server-authoritative upload progress.
///
/// # Errors
///
/// Returns an API error when the upload id is invalid, unknown, or cannot be
/// read from storage.
pub async fn get_upload(
State(state): State<AppState>,
Path(upload_id): Path<String>,
) -> Result<Json<crate::model::UploadProgressResponse>, ApiError> {
Ok(Json(state.storage.progress(&upload_id).await?))
}
/// Stores one raw binary chunk body.
///
/// # Errors
///
/// Returns an API error when the upload id is invalid or unknown, the chunk
/// index is out of range, the body length is wrong, or the write fails.
pub async fn put_chunk(
State(state): State<AppState>,
Path((upload_id, index)): Path<(String, u64)>,
body: Bytes,
) -> Result<StatusCode, ApiError> {
state.storage.store_chunk(&upload_id, index, &body).await?;
Ok(StatusCode::NO_CONTENT)
}
/// Assembles uploaded chunks into the final completed file.
///
/// # Errors
///
/// Returns an API error when the upload is unknown, incomplete, invalid, or
/// cannot be assembled on disk.
pub async fn complete_upload(
State(state): State<AppState>,
Path(upload_id): Path<String>,
) -> Result<Json<CompleteUploadResponse>, ApiError> {
Ok(Json(state.storage.complete_upload(&upload_id).await?))
}
#[derive(Debug)]
pub struct ApiError {
status: StatusCode,
message: String,
}
impl IntoResponse for ApiError {
fn into_response(self) -> Response {
(
self.status,
Json(ErrorResponse {
error: self.message,
}),
)
.into_response()
}
}
impl From<StorageError> for ApiError {
fn from(error: StorageError) -> Self {
let status = if error.is_not_found() {
StatusCode::NOT_FOUND
} else if error.is_conflict() {
StatusCode::CONFLICT
} else if error.is_invalid_input() {
StatusCode::BAD_REQUEST
} else {
StatusCode::INTERNAL_SERVER_ERROR
};
Self {
status,
message: error.to_string(),
}
}
}
#[derive(Serialize)]
struct ErrorResponse {
error: String,
}
src/app.rs
+105
View File
@@ -0,0 +1,105 @@
use std::{
env,
error::Error,
net::SocketAddr,
path::{Path, PathBuf},
};
use axum::{
Router,
extract::DefaultBodyLimit,
routing::{get, post},
};
use tower_http::services::{ServeDir, ServeFile};
use crate::{api, storage::Storage};
const DEFAULT_BIND_ADDR: &str = "127.0.0.1:3000";
const STATIC_DIR_ENV: &str = "UPL_STATIC_DIR";
const DATA_DIR_ENV: &str = "UPL_DATA_DIR";
const BIND_ENV: &str = "UPL_BIND";
const MAX_REQUEST_BODY_BYTES: usize = 64 * 1024 * 1024;
#[derive(Clone, Debug)]
pub struct AppConfig {
pub bind_addr: SocketAddr,
pub static_dir: PathBuf,
pub data_dir: PathBuf,
}
#[derive(Clone, Debug)]
pub struct AppState {
pub storage: Storage,
}
impl AppConfig {
/// Loads bind and static directory settings from environment variables.
///
/// # Errors
///
/// Returns an error when `UPL_BIND` is set but is not a valid socket address.
pub fn from_env() -> Result<Self, Box<dyn Error>> {
let bind_addr = env::var(BIND_ENV)
.unwrap_or_else(|_| DEFAULT_BIND_ADDR.to_owned())
.parse()?;
let static_dir = env::var_os(STATIC_DIR_ENV).map_or_else(default_static_dir, PathBuf::from);
let data_dir = env::var_os(DATA_DIR_ENV).map_or_else(default_data_dir, PathBuf::from);
Ok(Self {
bind_addr,
static_dir,
data_dir,
})
}
#[must_use]
pub fn new(
bind_addr: SocketAddr,
static_dir: impl Into<PathBuf>,
data_dir: impl Into<PathBuf>,
) -> Self {
Self {
bind_addr,
static_dir: static_dir.into(),
data_dir: data_dir.into(),
}
}
}
pub fn build_router(config: &AppConfig) -> Router {
let state = AppState {
storage: Storage::new(&config.data_dir),
};
Router::new()
.route("/healthz", get(healthz))
.route("/api/uploads", post(api::create_upload))
.route("/api/uploads/{upload_id}", get(api::get_upload))
.route(
"/api/uploads/{upload_id}/complete",
post(api::complete_upload),
)
.route(
"/api/uploads/{upload_id}/chunks/{index}",
axum::routing::put(api::put_chunk),
)
.layer(DefaultBodyLimit::max(MAX_REQUEST_BODY_BYTES))
.fallback_service(static_service(&config.static_dir))
.with_state(state)
}
async fn healthz() -> &'static str {
"ok"
}
fn static_service(static_dir: &Path) -> ServeDir<ServeFile> {
ServeDir::new(static_dir).fallback(ServeFile::new(static_dir.join("index.html")))
}
fn default_static_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("static")
}
fn default_data_dir() -> PathBuf {
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("data")
}
src/lib.rs
+4
View File
@@ -0,0 +1,4 @@
pub mod api;
pub mod app;
pub mod model;
pub mod storage;
src/main.rs
+14 -2
View File
@@ -1,3 +1,15 @@
fn main() { use std::error::Error;
println!("Hello, world!");
use upl::app::{AppConfig, build_router};
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
let config = AppConfig::from_env()?;
let listener = tokio::net::TcpListener::bind(config.bind_addr).await?;
println!("upl listening on http://{}", listener.local_addr()?);
axum::serve(listener, build_router(&config)).await?;
Ok(())
} }
src/model.rs
+80
View File
@@ -0,0 +1,80 @@
use serde::{Deserialize, Serialize};
pub const CHUNK_SIZE: u64 = 16 * 1024 * 1024;
#[derive(Debug, Deserialize)]
pub struct CreateUploadRequest {
pub name: String,
pub size: u64,
pub last_modified: i64,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct CreateUploadResponse {
pub upload_id: String,
pub chunk_size: u64,
pub total_chunks: u64,
pub completed_chunks: Vec<u64>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct UploadProgressResponse {
pub upload_id: String,
pub name: String,
pub size: u64,
pub chunk_size: u64,
pub total_chunks: u64,
pub completed_chunks: Vec<u64>,
}
#[derive(Debug, Deserialize, Serialize)]
pub struct CompleteUploadResponse {
pub upload_id: String,
pub name: String,
pub file_path: String,
}
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct UploadMeta {
pub id: String,
pub original_name: String,
pub safe_name: String,
pub size: u64,
pub last_modified: i64,
pub chunk_size: u64,
pub total_chunks: u64,
pub created_at: String,
}
impl UploadMeta {
#[must_use]
pub fn create_response(&self) -> CreateUploadResponse {
CreateUploadResponse {
upload_id: self.id.clone(),
chunk_size: self.chunk_size,
total_chunks: self.total_chunks,
completed_chunks: Vec::new(),
}
}
#[must_use]
pub fn progress_response(&self, completed_chunks: Vec<u64>) -> UploadProgressResponse {
UploadProgressResponse {
upload_id: self.id.clone(),
name: self.original_name.clone(),
size: self.size,
chunk_size: self.chunk_size,
total_chunks: self.total_chunks,
completed_chunks,
}
}
#[must_use]
pub fn complete_response(&self, file_path: String) -> CompleteUploadResponse {
CompleteUploadResponse {
upload_id: self.id.clone(),
name: self.safe_name.clone(),
file_path,
}
}
}
src/storage.rs
+443
View File
@@ -0,0 +1,443 @@
use std::{
error::Error,
fmt::{self, Display},
path::{Path, PathBuf},
};
use time::{OffsetDateTime, format_description::well_known::Rfc3339};
use tokio::{fs, io::AsyncWriteExt};
use uuid::Uuid;
use crate::model::{
CHUNK_SIZE,
CompleteUploadResponse,
CreateUploadRequest,
UploadMeta,
UploadProgressResponse,
};
#[derive(Clone, Debug)]
pub struct Storage {
data_dir: PathBuf,
}
impl Storage {
#[must_use]
pub fn new(data_dir: impl Into<PathBuf>) -> Self {
Self {
data_dir: data_dir.into(),
}
}
/// Creates a durable upload metadata record under `data/staging`.
///
/// # Errors
///
/// Returns an error when directories cannot be created, metadata cannot be
/// serialized, or the metadata file cannot be written atomically.
pub async fn create_upload(
&self,
request: CreateUploadRequest,
) -> Result<UploadMeta, StorageError> {
let original_name = request.name.trim();
if original_name.is_empty() {
return Err(StorageError::InvalidInput("file name cannot be empty"));
}
self.ensure_layout().await?;
let safe_name = safe_file_name(original_name);
let created_at = OffsetDateTime::now_utc().format(&Rfc3339)?;
for _ in 0..8 {
let id = Uuid::new_v4().simple().to_string();
let upload_dir = self.upload_dir(&id);
if fs::try_exists(&upload_dir).await? {
continue;
}
fs::create_dir_all(upload_dir.join("chunks")).await?;
let meta = UploadMeta {
id,
original_name: original_name.to_owned(),
safe_name,
size: request.size,
last_modified: request.last_modified,
chunk_size: CHUNK_SIZE,
total_chunks: total_chunks(request.size, CHUNK_SIZE),
created_at,
};
self.write_meta(&meta).await?;
return Ok(meta);
}
Err(StorageError::IdCollision)
}
/// Loads upload progress by scanning durable chunk files.
///
/// # Errors
///
/// Returns an error when the upload id is invalid, metadata is missing, or
/// the staging directory cannot be scanned.
pub async fn progress(&self, upload_id: &str) -> Result<UploadProgressResponse, StorageError> {
let meta = self.load_meta(upload_id).await?;
let completed_chunks = self.completed_chunks(&meta).await?;
Ok(meta.progress_response(completed_chunks))
}
/// Validates and stores one raw chunk body.
///
/// # Errors
///
/// Returns an error when the upload is unknown, the index is out of range,
/// the body length is not the expected chunk length, or the chunk cannot be
/// written and renamed into place.
pub async fn store_chunk(
&self,
upload_id: &str,
index: u64,
body: &[u8],
) -> Result<(), StorageError> {
let meta = self.load_meta(upload_id).await?;
let expected_len = expected_chunk_len(&meta, index)?;
let body_len = u64::try_from(body.len())
.map_err(|_| StorageError::InvalidInput("chunk body is too large"))?;
if body_len != expected_len {
return Err(StorageError::InvalidInput("chunk has the wrong length"));
}
let part_path = self.chunk_path(upload_id, index);
if let Some(existing_len) = file_len(&part_path).await? {
if existing_len == expected_len {
return Ok(());
}
return Err(StorageError::InvalidInput(
"existing chunk has the wrong length",
));
}
let tmp_path = part_path.with_extension("part.tmp");
fs::write(&tmp_path, body).await?;
fs::rename(&tmp_path, &part_path).await?;
Ok(())
}
/// Assembles a complete upload from verified chunk files.
///
/// # Errors
///
/// Returns an error when the upload is unknown, any expected chunk is
/// missing or has the wrong length, the final file already exists, or the
/// assembled file cannot be written and renamed.
pub async fn complete_upload(
&self,
upload_id: &str,
) -> Result<CompleteUploadResponse, StorageError> {
let meta = self.load_meta(upload_id).await?;
self.verify_all_chunks(&meta).await?;
let final_path = self.complete_dir().join(&meta.safe_name);
if fs::try_exists(&final_path).await? {
return Err(StorageError::Conflict("complete file already exists"));
}
let tmp_path = self
.complete_dir()
.join(format!(".{}.{}.tmp", meta.safe_name, meta.id));
if fs::try_exists(&tmp_path).await? {
fs::remove_file(&tmp_path).await?;
}
let mut output = fs::OpenOptions::new()
.write(true)
.create_new(true)
.open(&tmp_path)
.await?;
for index in 0..meta.total_chunks {
let bytes = fs::read(self.chunk_path(upload_id, index)).await?;
output.write_all(&bytes).await?;
}
output.flush().await?;
drop(output);
fs::rename(&tmp_path, &final_path).await?;
Ok(meta.complete_response(final_path.display().to_string()))
}
#[must_use]
pub fn data_dir(&self) -> &Path {
&self.data_dir
}
fn staging_dir(&self) -> PathBuf {
self.data_dir.join("staging")
}
fn complete_dir(&self) -> PathBuf {
self.data_dir.join("complete")
}
fn upload_dir(&self, upload_id: &str) -> PathBuf {
self.staging_dir().join(upload_id)
}
fn chunk_path(&self, upload_id: &str, index: u64) -> PathBuf {
self.upload_dir(upload_id)
.join("chunks")
.join(format!("{index:06}.part"))
}
async fn ensure_layout(&self) -> Result<(), StorageError> {
fs::create_dir_all(self.staging_dir()).await?;
fs::create_dir_all(self.complete_dir()).await?;
Ok(())
}
async fn write_meta(&self, meta: &UploadMeta) -> Result<(), StorageError> {
let meta_path = self.upload_dir(&meta.id).join("meta.json");
let tmp_path = meta_path.with_extension("json.tmp");
let json = serde_json::to_vec_pretty(meta)?;
fs::write(&tmp_path, json).await?;
fs::rename(&tmp_path, &meta_path).await?;
Ok(())
}
async fn load_meta(&self, upload_id: &str) -> Result<UploadMeta, StorageError> {
validate_upload_id(upload_id)?;
let meta_path = self.upload_dir(upload_id).join("meta.json");
let bytes = match fs::read(meta_path).await {
Ok(bytes) => bytes,
Err(error) if error.kind() == std::io::ErrorKind::NotFound => {
return Err(StorageError::NotFound);
}
Err(error) => return Err(error.into()),
};
Ok(serde_json::from_slice(&bytes)?)
}
async fn completed_chunks(&self, meta: &UploadMeta) -> Result<Vec<u64>, StorageError> {
let mut completed = Vec::new();
for index in 0..meta.total_chunks {
let expected_len = expected_chunk_len(meta, index)?;
if file_len(&self.chunk_path(&meta.id, index)).await? == Some(expected_len) {
completed.push(index);
}
}
Ok(completed)
}
async fn verify_all_chunks(&self, meta: &UploadMeta) -> Result<(), StorageError> {
for index in 0..meta.total_chunks {
let expected_len = expected_chunk_len(meta, index)?;
let actual_len = file_len(&self.chunk_path(&meta.id, index)).await?;
if actual_len != Some(expected_len) {
return Err(StorageError::Conflict(
"upload is missing one or more complete chunks",
));
}
}
Ok(())
}
}
#[derive(Debug)]
pub enum StorageError {
Conflict(&'static str),
Format(time::error::Format),
IdCollision,
InvalidInput(&'static str),
Io(std::io::Error),
Json(serde_json::Error),
NotFound,
}
impl StorageError {
#[must_use]
pub fn is_invalid_input(&self) -> bool {
matches!(self, Self::InvalidInput(_))
}
#[must_use]
pub fn is_not_found(&self) -> bool {
matches!(self, Self::NotFound)
}
#[must_use]
pub fn is_conflict(&self) -> bool {
matches!(self, Self::Conflict(_))
}
}
impl Display for StorageError {
fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Format(error) => write!(formatter, "failed to format timestamp: {error}"),
Self::IdCollision => formatter.write_str("could not allocate a unique upload id"),
Self::Conflict(message) | Self::InvalidInput(message) => formatter.write_str(message),
Self::Io(error) => write!(formatter, "storage I/O error: {error}"),
Self::Json(error) => write!(formatter, "metadata JSON error: {error}"),
Self::NotFound => formatter.write_str("upload not found"),
}
}
}
impl Error for StorageError {
fn source(&self) -> Option<&(dyn Error + 'static)> {
match self {
Self::Format(error) => Some(error),
Self::Io(error) => Some(error),
Self::Json(error) => Some(error),
Self::Conflict(_) | Self::IdCollision | Self::InvalidInput(_) | Self::NotFound => None,
}
}
}
impl From<std::io::Error> for StorageError {
fn from(error: std::io::Error) -> Self {
Self::Io(error)
}
}
impl From<serde_json::Error> for StorageError {
fn from(error: serde_json::Error) -> Self {
Self::Json(error)
}
}
impl From<time::error::Format> for StorageError {
fn from(error: time::error::Format) -> Self {
Self::Format(error)
}
}
fn safe_file_name(name: &str) -> String {
let safe: String = name
.chars()
.map(|character| match character {
'/' | '\\' | ':' | '*' | '?' | '"' | '<' | '>' | '|' => '_',
character if character.is_control() => '_',
character => character,
})
.collect();
let trimmed = safe.trim_matches(['.', ' ', '_']);
if trimmed.is_empty() {
"upload".to_owned()
} else {
trimmed.to_owned()
}
}
fn total_chunks(size: u64, chunk_size: u64) -> u64 {
if size == 0 {
0
} else {
size.div_ceil(chunk_size)
}
}
fn expected_chunk_len(meta: &UploadMeta, index: u64) -> Result<u64, StorageError> {
if index >= meta.total_chunks {
return Err(StorageError::InvalidInput("chunk index is out of range"));
}
if index + 1 == meta.total_chunks {
let bytes_before_final = meta.chunk_size * index;
Ok(meta.size - bytes_before_final)
} else {
Ok(meta.chunk_size)
}
}
async fn file_len(path: &Path) -> Result<Option<u64>, StorageError> {
match fs::metadata(path).await {
Ok(metadata) if metadata.is_file() => Ok(Some(metadata.len())),
Ok(_) => Ok(None),
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(error) => Err(error.into()),
}
}
fn validate_upload_id(upload_id: &str) -> Result<(), StorageError> {
let is_valid = !upload_id.is_empty()
&& upload_id
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || byte == b'-' || byte == b'_');
if is_valid {
Ok(())
} else {
Err(StorageError::InvalidInput("upload id is invalid"))
}
}
#[cfg(test)]
mod tests {
use super::{expected_chunk_len, safe_file_name, total_chunks, validate_upload_id};
use crate::model::UploadMeta;
#[test]
fn computes_total_chunks() {
assert_eq!(total_chunks(0, 16), 0);
assert_eq!(total_chunks(1, 16), 1);
assert_eq!(total_chunks(16, 16), 1);
assert_eq!(total_chunks(17, 16), 2);
}
#[test]
fn sanitizes_file_names() {
assert_eq!(safe_file_name("../movie:mkv"), "movie_mkv");
assert_eq!(safe_file_name(" "), "upload");
}
#[test]
fn computes_expected_chunk_lengths() -> Result<(), Box<dyn std::error::Error>> {
let meta = test_meta(33, 16, 3);
assert_eq!(expected_chunk_len(&meta, 0)?, 16);
assert_eq!(expected_chunk_len(&meta, 1)?, 16);
assert_eq!(expected_chunk_len(&meta, 2)?, 1);
assert!(expected_chunk_len(&meta, 3).is_err());
Ok(())
}
#[test]
fn validates_upload_ids() {
assert!(validate_upload_id("abc123").is_ok());
assert!(validate_upload_id("../abc").is_err());
assert!(validate_upload_id("").is_err());
}
fn test_meta(size: u64, chunk_size: u64, total_chunks: u64) -> UploadMeta {
UploadMeta {
id: "abc123".to_owned(),
original_name: "file.bin".to_owned(),
safe_name: "file.bin".to_owned(),
size,
last_modified: 0,
chunk_size,
total_chunks,
created_at: "2026-05-30T16:00:00Z".to_owned(),
}
}
}
static/app.js
+533
View File
@@ -0,0 +1,533 @@
const DB_NAME = "upl";
const DB_VERSION = 1;
const STORE_NAME = "uploads";
const CONCURRENCY = 3;
const MAX_RETRIES = 5;
const BASE_RETRY_DELAY_MS = 500;
const fileInput = document.querySelector("#file-input");
const pickButton = document.querySelector("#pick-button");
const fileSummary = document.querySelector("#file-summary");
const fileName = document.querySelector("#file-name");
const fileSize = document.querySelector("#file-size");
const startButton = document.querySelector("#start-button");
const pauseButton = document.querySelector("#pause-button");
const resumeButton = document.querySelector("#resume-button");
const eventLog = document.querySelector("#event-log");
const progressBar = document.querySelector("#progress-bar");
const progressMeta = document.querySelector("#progress-meta");
const pendingSection = document.querySelector("#pending-section");
const pendingList = document.querySelector("#pending-list");
const state = {
abortController: null,
completedChunks: new Set(),
file: null,
fileHandle: null,
pendingRecords: [],
record: null,
resumeAfterReselect: null,
running: false,
};
const dbReady = "indexedDB" in window ? openDatabase() : Promise.resolve(null);
let saveChain = Promise.resolve();
function formatBytes(bytes) {
const formatter = new Intl.NumberFormat(undefined, {
maximumFractionDigits: 1,
});
const units = ["B", "KiB", "MiB", "GiB", "TiB"];
let value = bytes;
let unit = units[0];
for (const candidate of units) {
unit = candidate;
if (value < 1024 || candidate === units[units.length - 1]) {
break;
}
value /= 1024;
}
return `${formatter.format(value)} ${unit}`;
}
function log(message) {
const item = document.createElement("li");
item.textContent = message;
eventLog.prepend(item);
while (eventLog.children.length > 8) {
eventLog.lastElementChild.remove();
}
}
function openDatabase() {
return new Promise((resolve, reject) => {
const request = indexedDB.open(DB_NAME, DB_VERSION);
request.addEventListener("upgradeneeded", () => {
const db = request.result;
if (!db.objectStoreNames.contains(STORE_NAME)) {
db.createObjectStore(STORE_NAME, { keyPath: "upload_id" });
}
});
request.addEventListener("success", () => resolve(request.result));
request.addEventListener("error", () => reject(request.error));
}).catch((error) => {
log(`IndexedDB unavailable: ${error.message}`);
return null;
});
}
function requestToPromise(request) {
return new Promise((resolve, reject) => {
request.addEventListener("success", () => resolve(request.result));
request.addEventListener("error", () => reject(request.error));
});
}
async function withStore(mode, callback) {
const db = await dbReady;
if (!db) {
return null;
}
const transaction = db.transaction(STORE_NAME, mode);
return requestToPromise(callback(transaction.objectStore(STORE_NAME)));
}
async function loadRecords() {
const records = (await withStore("readonly", (store) => store.getAll())) ?? [];
records.sort((left, right) => right.updated_at.localeCompare(left.updated_at));
state.pendingRecords = records;
renderPendingRecords();
}
async function saveRecord(record) {
const nextSave = saveChain.catch(() => null).then(() => writeRecord(record));
saveChain = nextSave;
await nextSave;
}
async function writeRecord(record) {
const storedRecord = { ...record, updated_at: new Date().toISOString() };
try {
await withStore("readwrite", (store) => store.put(storedRecord));
} catch (error) {
if (!storedRecord.file_handle) {
throw error;
}
delete storedRecord.file_handle;
await withStore("readwrite", (store) => store.put(storedRecord));
log("Saved resume state without a reusable file handle.");
}
state.record = storedRecord;
await loadRecords();
}
async function deleteRecord(uploadId) {
await withStore("readwrite", (store) => store.delete(uploadId));
if (state.record?.upload_id === uploadId) {
state.record = null;
}
await loadRecords();
}
function renderPendingRecords() {
pendingList.replaceChildren();
pendingSection.hidden = state.pendingRecords.length === 0;
for (const record of state.pendingRecords) {
const item = document.createElement("li");
item.className = "pending-item";
const meta = document.createElement("div");
meta.className = "pending-meta";
const title = document.createElement("strong");
title.textContent = record.name;
const detail = document.createElement("span");
detail.textContent = `${formatBytes(record.size)} - ${record.completed_chunks ?? 0} of ${record.total_chunks} chunks`;
const resume = document.createElement("button");
resume.type = "button";
resume.className = "secondary";
resume.textContent = "Resume";
resume.addEventListener("click", () => {
void resumePendingRecord(record);
});
const remove = document.createElement("button");
remove.type = "button";
remove.className = "danger";
remove.textContent = "Remove";
remove.addEventListener("click", () => {
void deleteRecord(record.upload_id);
});
meta.append(title, detail);
item.append(meta, resume, remove);
pendingList.append(item);
}
}
function renderFile(file) {
if (!file) {
fileSummary.hidden = true;
fileName.textContent = "";
fileSize.textContent = "";
return;
}
fileName.textContent = file.name;
fileSize.textContent = formatBytes(file.size);
fileSummary.hidden = false;
}
function renderButtons() {
startButton.disabled = !state.file || state.running || Boolean(state.record);
pauseButton.disabled = !state.running;
resumeButton.disabled = !state.file || state.running || !state.record;
}
function updateProgress(completedCount, totalChunks) {
const percentage = totalChunks === 0 ? 100 : (completedCount / totalChunks) * 100;
progressBar.style.width = `${percentage}%`;
progressMeta.textContent = `${completedCount} of ${totalChunks} chunks`;
}
function sameFile(record, file) {
return (
record.name === file.name &&
record.size === file.size &&
record.last_modified === file.lastModified
);
}
function findPendingRecord(file) {
return state.pendingRecords.find((record) => sameFile(record, file)) ?? null;
}
async function selectFile(file, fileHandle = null, record = null) {
if (record && !sameFile(record, file)) {
log("Selected file does not match the pending upload.");
return;
}
state.file = file;
state.fileHandle = fileHandle;
state.record = record ?? findPendingRecord(file);
state.completedChunks = new Set();
renderFile(file);
updateProgress(state.record?.completed_chunks ?? 0, state.record?.total_chunks ?? 0);
renderButtons();
log(state.record ? "Ready to resume upload." : "Ready to create an upload record.");
}
async function pickFile() {
if ("showOpenFilePicker" in window) {
try {
const [handle] = await window.showOpenFilePicker({ multiple: false });
const file = await handle.getFile();
await selectFile(file, handle);
return;
} catch (error) {
if (isAbortError(error)) {
return;
}
log(`File picker failed: ${error.message}`);
}
}
fileInput.click();
}
fileInput.addEventListener("change", () => {
const [file] = fileInput.files;
if (!file) {
renderFile(null);
renderButtons();
log("Choose a file to begin.");
return;
}
const record = state.resumeAfterReselect ?? findPendingRecord(file);
state.resumeAfterReselect = null;
void selectFile(file, null, record);
});
pickButton.addEventListener("click", () => {
void pickFile();
});
startButton.addEventListener("click", () => {
void runUpload();
});
pauseButton.addEventListener("click", () => {
if (state.abortController) {
state.abortController.abort();
}
});
resumeButton.addEventListener("click", () => {
void runUpload();
});
async function resumePendingRecord(record) {
if (state.running) {
return;
}
if (record.file_handle) {
const granted = await requestFileHandlePermission(record.file_handle);
if (granted) {
const file = await record.file_handle.getFile();
await selectFile(file, record.file_handle, record);
await runUpload();
return;
}
}
state.resumeAfterReselect = record;
log("Select the same file to resume.");
fileInput.click();
}
async function requestFileHandlePermission(handle) {
if (!("queryPermission" in handle) || !("requestPermission" in handle)) {
return false;
}
const options = { mode: "read" };
if ((await handle.queryPermission(options)) === "granted") {
return true;
}
return (await handle.requestPermission(options)) === "granted";
}
async function runUpload() {
if (!state.file || state.running) {
return;
}
const controller = new AbortController();
state.abortController = controller;
state.running = true;
renderButtons();
try {
if (!state.record) {
await createUploadRecord();
}
const progress = await fetchJson(`/api/uploads/${state.record.upload_id}`, {
signal: controller.signal,
});
state.completedChunks = new Set(progress.completed_chunks);
await saveRecord({
...state.record,
completed_chunks: state.completedChunks.size,
chunk_size: progress.chunk_size,
total_chunks: progress.total_chunks,
});
updateProgress(state.completedChunks.size, progress.total_chunks);
const missingChunks = buildMissingChunkList(progress.total_chunks, state.completedChunks);
log(
missingChunks.length === 0
? "All chunks already uploaded."
: `Uploading ${missingChunks.length} missing chunks.`,
);
await runPool(missingChunks, (index) =>
uploadChunkWithRetry(index, progress.chunk_size, progress.total_chunks, controller.signal),
);
if (controller.signal.aborted) {
return;
}
log("Completing upload.");
const complete = await fetchJson(`/api/uploads/${state.record.upload_id}/complete`, {
method: "POST",
signal: controller.signal,
});
updateProgress(progress.total_chunks, progress.total_chunks);
log(`Complete: ${complete.file_path}`);
await deleteRecord(state.record.upload_id);
state.completedChunks = new Set();
state.record = null;
} catch (error) {
if (controller.signal.aborted || isAbortError(error)) {
log("Upload paused.");
} else {
controller.abort();
log(`Upload failed: ${error.message}`);
}
} finally {
if (state.abortController === controller) {
state.abortController = null;
}
state.running = false;
renderButtons();
}
}
async function createUploadRecord() {
const response = await fetchJson("/api/uploads", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
name: state.file.name,
size: state.file.size,
last_modified: state.file.lastModified,
}),
});
const record = {
upload_id: response.upload_id,
name: state.file.name,
size: state.file.size,
last_modified: state.file.lastModified,
chunk_size: response.chunk_size,
total_chunks: response.total_chunks,
completed_chunks: 0,
file_handle: state.fileHandle,
updated_at: new Date().toISOString(),
};
await saveRecord(record);
updateProgress(0, response.total_chunks);
log("Upload record created.");
}
function buildMissingChunkList(totalChunks, completedChunks) {
const missing = [];
for (let index = 0; index < totalChunks; index += 1) {
if (!completedChunks.has(index)) {
missing.push(index);
}
}
return missing;
}
async function uploadChunkWithRetry(index, chunkSize, totalChunks, signal) {
for (let attempt = 0; attempt <= MAX_RETRIES; attempt += 1) {
throwIfAborted(signal);
try {
await uploadChunk(index, chunkSize, signal);
state.completedChunks.add(index);
updateProgress(state.completedChunks.size, totalChunks);
await saveRecord({
...state.record,
completed_chunks: state.completedChunks.size,
});
return;
} catch (error) {
if (isAbortError(error) || attempt === MAX_RETRIES) {
throw error;
}
const delayMs = BASE_RETRY_DELAY_MS * 2 ** attempt;
log(`Retrying chunk ${index} after ${delayMs} ms.`);
await delay(delayMs, signal);
}
}
}
async function uploadChunk(index, chunkSize, signal) {
const start = index * chunkSize;
const end = Math.min(state.file.size, start + chunkSize);
const body = state.file.slice(start, end);
const response = await fetch(`/api/uploads/${state.record.upload_id}/chunks/${index}`, {
method: "PUT",
headers: { "Content-Type": "application/octet-stream" },
body,
signal,
});
if (!response.ok) {
throw new Error(await responseError(response));
}
}
async function runPool(items, worker) {
let nextIndex = 0;
const workers = Array.from(
{ length: Math.min(CONCURRENCY, items.length) },
async () => {
while (nextIndex < items.length) {
const item = items[nextIndex];
nextIndex += 1;
await worker(item);
}
},
);
await Promise.all(workers);
}
async function fetchJson(url, options = {}) {
const response = await fetch(url, options);
if (!response.ok) {
throw new Error(await responseError(response));
}
return response.json();
}
async function responseError(response) {
const text = await response.text();
if (!text) {
return `${response.status} ${response.statusText}`;
}
try {
return JSON.parse(text).error ?? text;
} catch {
return text;
}
}
function delay(ms, signal) {
return new Promise((resolve, reject) => {
throwIfAborted(signal);
const id = window.setTimeout(resolve, ms);
signal.addEventListener(
"abort",
() => {
window.clearTimeout(id);
reject(new DOMException("Aborted", "AbortError"));
},
{ once: true },
);
});
}
function throwIfAborted(signal) {
if (signal.aborted) {
throw new DOMException("Aborted", "AbortError");
}
}
function isAbortError(error) {
return error?.name === "AbortError";
}
async function initialize() {
await loadRecords();
renderButtons();
}
void initialize();
static/index.html
+53
View File
@@ -0,0 +1,53 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>upl</title>
<link rel="stylesheet" href="/styles.css">
<script src="/app.js" defer></script>
</head>
<body>
<main class="app-shell">
<section class="upload-panel" aria-labelledby="app-title">
<div class="panel-heading">
<div>
<h1 id="app-title">upl</h1>
<p class="subtle">Resumable uploads to this machine.</p>
</div>
<span class="status-pill" id="connection-status">Server online</span>
</div>
<div class="file-picker">
<button id="pick-button" type="button">Choose file</button>
<input id="file-input" type="file">
</div>
<div class="file-summary" id="file-summary" hidden>
<strong id="file-name"></strong>
<span id="file-size"></span>
</div>
<div class="progress-wrap" aria-label="Upload progress">
<div class="progress-bar" id="progress-bar"></div>
</div>
<div class="progress-meta" id="progress-meta">0 of 0 chunks</div>
<div class="actions">
<button id="start-button" type="button" disabled>Start</button>
<button id="pause-button" type="button" disabled>Pause</button>
<button id="resume-button" type="button" disabled>Resume</button>
</div>
<section class="pending-section" id="pending-section" hidden>
<h2>Pending uploads</h2>
<ul class="pending-list" id="pending-list"></ul>
</section>
<ol class="event-log" id="event-log" aria-live="polite">
<li>Choose a file to begin.</li>
</ol>
</section>
</main>
</body>
</html>
static/styles.css
+250
View File
@@ -0,0 +1,250 @@
:root {
color-scheme: light dark;
--bg: #f6f7f9;
--panel: #ffffff;
--text: #1d2430;
--muted: #667085;
--line: #d0d5dd;
--accent: #147a73;
--accent-strong: #0f5f59;
--track: #e4e7ec;
}
* {
box-sizing: border-box;
}
body {
margin: 0;
min-height: 100vh;
background: var(--bg);
color: var(--text);
font-family:
Inter, ui-sans-serif, system-ui, -apple-system, BlinkMacSystemFont, "Segoe UI",
sans-serif;
}
button,
input {
font: inherit;
}
[hidden] {
display: none !important;
}
.app-shell {
width: min(720px, 100%);
margin: 0 auto;
padding: 32px 16px;
}
.upload-panel {
display: grid;
gap: 20px;
padding: 24px;
border: 1px solid var(--line);
border-radius: 8px;
background: var(--panel);
box-shadow: 0 10px 30px rgb(16 24 40 / 8%);
}
.panel-heading {
display: flex;
align-items: flex-start;
justify-content: space-between;
gap: 16px;
}
h1,
p {
margin: 0;
}
h1 {
font-size: 2rem;
line-height: 1.1;
}
.subtle {
margin-top: 6px;
color: var(--muted);
}
.status-pill {
flex: 0 0 auto;
padding: 6px 10px;
border: 1px solid color-mix(in srgb, var(--accent) 34%, transparent);
border-radius: 999px;
color: var(--accent-strong);
background: color-mix(in srgb, var(--accent) 10%, transparent);
font-size: 0.875rem;
font-weight: 700;
}
.file-picker {
display: flex;
align-items: center;
gap: 12px;
padding: 18px;
border: 1px dashed var(--line);
border-radius: 8px;
}
.file-picker input {
position: absolute;
width: 1px;
height: 1px;
overflow: hidden;
clip: rect(0, 0, 0, 0);
}
.file-summary {
display: grid;
gap: 4px;
padding: 14px;
border: 1px solid var(--line);
border-radius: 8px;
}
.file-summary span {
color: var(--muted);
}
.progress-wrap {
height: 14px;
overflow: hidden;
border-radius: 999px;
background: var(--track);
}
.progress-bar {
width: 0%;
height: 100%;
background: var(--accent);
transition: width 180ms ease;
}
.progress-meta {
margin-top: -12px;
color: var(--muted);
font-size: 0.875rem;
}
.actions {
display: flex;
flex-wrap: wrap;
gap: 10px;
}
button {
min-width: 96px;
min-height: 40px;
border: 1px solid transparent;
border-radius: 6px;
color: #ffffff;
background: var(--accent);
font-weight: 700;
cursor: pointer;
}
button.secondary {
border-color: var(--line);
color: var(--text);
background: transparent;
}
button.danger {
border-color: #b42318;
color: #b42318;
background: transparent;
}
button:disabled {
color: var(--muted);
background: var(--track);
cursor: not-allowed;
}
.pending-section {
display: grid;
gap: 12px;
}
h2 {
margin: 0;
font-size: 1rem;
}
.pending-list {
display: grid;
gap: 10px;
margin: 0;
padding: 0;
list-style: none;
}
.pending-item {
display: grid;
grid-template-columns: minmax(0, 1fr) auto auto;
align-items: center;
gap: 10px;
padding: 12px;
border: 1px solid var(--line);
border-radius: 8px;
}
.pending-item strong,
.file-summary strong {
overflow-wrap: anywhere;
}
.pending-meta {
display: grid;
gap: 4px;
}
.pending-meta span {
color: var(--muted);
font-size: 0.875rem;
}
.event-log {
min-height: 80px;
margin: 0;
padding: 14px 14px 14px 32px;
border: 1px solid var(--line);
border-radius: 8px;
color: var(--muted);
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #14161a;
--panel: #1f242b;
--text: #f2f4f7;
--muted: #b8c0cc;
--line: #3d4654;
--accent: #35b8aa;
--accent-strong: #9ce4dc;
--track: #343b46;
}
}
@media (max-width: 520px) {
.panel-heading {
display: grid;
}
.status-pill {
width: max-content;
}
.upload-panel {
padding: 18px;
}
.pending-item {
grid-template-columns: 1fr;
}
}
tests/chunk_upload.rs
+202
View File
@@ -0,0 +1,202 @@
use std::{
net::{Ipv4Addr, SocketAddr},
path::Path,
};
use axum::{
body::Body,
http::{Method, Request, StatusCode, header},
};
use http_body_util::BodyExt;
use serde_json::json;
use tempfile::TempDir;
use tower::ServiceExt;
use upl::{
app::{AppConfig, build_router},
model::{CHUNK_SIZE, CreateUploadResponse, UploadProgressResponse},
};
#[tokio::test]
async fn stores_chunks_and_reports_progress() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, temp_dir.path(), CHUNK_SIZE + 3).await?;
let final_chunk = vec![b'z'; 3];
let response = app
.clone()
.oneshot(chunk_request(&upload.upload_id, 1, final_chunk)?)
.await?;
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let progress = get_progress(&app, &upload.upload_id).await?;
assert_eq!(progress.completed_chunks, vec![1]);
let first_chunk = vec![b'a'; usize::try_from(CHUNK_SIZE)?];
let response = app
.clone()
.oneshot(chunk_request(&upload.upload_id, 0, first_chunk)?)
.await?;
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let progress = get_progress(&app, &upload.upload_id).await?;
assert_eq!(progress.completed_chunks, vec![0, 1]);
let chunk_path = temp_dir
.path()
.join("staging")
.join(&upload.upload_id)
.join("chunks")
.join("000000.part");
assert_eq!(tokio::fs::metadata(chunk_path).await?.len(), CHUNK_SIZE);
Ok(())
}
#[tokio::test]
async fn rejects_wrong_size_non_final_chunk() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, temp_dir.path(), CHUNK_SIZE + 1).await?;
let response = app
.oneshot(chunk_request(&upload.upload_id, 0, b"too short".to_vec())?)
.await?;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
Ok(())
}
#[tokio::test]
async fn rejects_out_of_range_chunk_index() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, temp_dir.path(), 4).await?;
let response = app
.oneshot(chunk_request(&upload.upload_id, 1, b"data".to_vec())?)
.await?;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
Ok(())
}
#[tokio::test]
async fn accepts_duplicate_chunk_when_existing_length_matches()
-> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, temp_dir.path(), 4).await?;
let first = app
.clone()
.oneshot(chunk_request(&upload.upload_id, 0, b"data".to_vec())?)
.await?;
let second = app
.oneshot(chunk_request(&upload.upload_id, 0, b"data".to_vec())?)
.await?;
assert_eq!(first.status(), StatusCode::NO_CONTENT);
assert_eq!(second.status(), StatusCode::NO_CONTENT);
Ok(())
}
#[tokio::test]
async fn rejects_unknown_upload_id() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let response = app
.oneshot(chunk_request("missing", 0, b"data".to_vec())?)
.await?;
assert_eq!(response.status(), StatusCode::NOT_FOUND);
Ok(())
}
async fn create_upload(
app: &axum::Router,
data_dir: &Path,
size: u64,
) -> Result<CreateUploadResponse, Box<dyn std::error::Error>> {
let response = app
.clone()
.oneshot(json_request(
Method::POST,
"/api/uploads",
&json!({
"name": "chunked.bin",
"size": size,
"last_modified": 1_760_000_000_000_i64
}),
)?)
.await?;
assert_eq!(response.status(), StatusCode::OK);
assert!(data_dir.join("staging").is_dir());
decode_body(response).await
}
async fn get_progress(
app: &axum::Router,
upload_id: &str,
) -> Result<UploadProgressResponse, Box<dyn std::error::Error>> {
let response = app
.clone()
.oneshot(
Request::builder()
.method(Method::GET)
.uri(format!("/api/uploads/{upload_id}"))
.body(Body::empty())?,
)
.await?;
assert_eq!(response.status(), StatusCode::OK);
decode_body(response).await
}
fn test_app(data_dir: &Path) -> axum::Router {
build_router(&AppConfig::new(
SocketAddr::from((Ipv4Addr::LOCALHOST, 0)),
concat!(env!("CARGO_MANIFEST_DIR"), "/static"),
data_dir,
))
}
fn json_request(
method: Method,
uri: &str,
body: &serde_json::Value,
) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method(method)
.uri(uri)
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from(serde_json::to_vec(body)?))?)
}
fn chunk_request(
upload_id: &str,
index: u64,
body: Vec<u8>,
) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method(Method::PUT)
.uri(format!("/api/uploads/{upload_id}/chunks/{index}"))
.header(header::CONTENT_TYPE, "application/octet-stream")
.body(Body::from(body))?)
}
async fn decode_body<T>(response: axum::response::Response) -> Result<T, Box<dyn std::error::Error>>
where
T: serde::de::DeserializeOwned,
{
let body = response.into_body().collect().await?.to_bytes();
Ok(serde_json::from_slice(&body)?)
}
tests/completion.rs
+201
View File
@@ -0,0 +1,201 @@
use std::{
net::{Ipv4Addr, SocketAddr},
path::Path,
};
use axum::{
body::Body,
http::{Method, Request, StatusCode, header},
};
use http_body_util::BodyExt;
use serde_json::json;
use tempfile::TempDir;
use tower::ServiceExt;
use upl::{
app::{AppConfig, build_router},
model::{CHUNK_SIZE, CompleteUploadResponse, CreateUploadResponse},
};
#[tokio::test]
async fn assembles_completed_upload() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, "hello.txt", 11).await?;
let response = app
.clone()
.oneshot(chunk_request(
&upload.upload_id,
0,
b"hello world".to_vec(),
)?)
.await?;
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let response = app
.clone()
.oneshot(empty_request(
Method::POST,
&format!("/api/uploads/{}/complete", upload.upload_id),
)?)
.await?;
assert_eq!(response.status(), StatusCode::OK);
let complete: CompleteUploadResponse = decode_body(response).await?;
assert_eq!(complete.name, "hello.txt");
assert_eq!(
tokio::fs::read(temp_dir.path().join("complete").join("hello.txt")).await?,
b"hello world"
);
assert!(
temp_dir
.path()
.join("staging")
.join(&upload.upload_id)
.is_dir()
);
let duplicate = app
.oneshot(empty_request(
Method::POST,
&format!("/api/uploads/{}/complete", upload.upload_id),
)?)
.await?;
assert_eq!(duplicate.status(), StatusCode::CONFLICT);
Ok(())
}
#[tokio::test]
async fn rejects_incomplete_upload() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, "partial.bin", CHUNK_SIZE + 1).await?;
let response = app
.clone()
.oneshot(chunk_request(&upload.upload_id, 1, b"x".to_vec())?)
.await?;
assert_eq!(response.status(), StatusCode::NO_CONTENT);
let response = app
.oneshot(empty_request(
Method::POST,
&format!("/api/uploads/{}/complete", upload.upload_id),
)?)
.await?;
assert_eq!(response.status(), StatusCode::CONFLICT);
assert!(
!temp_dir
.path()
.join("complete")
.join("partial.bin")
.exists()
);
Ok(())
}
#[tokio::test]
async fn rejects_corrupt_chunk_file() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let upload = create_upload(&app, "corrupt.bin", 4).await?;
let chunk_path = temp_dir
.path()
.join("staging")
.join(&upload.upload_id)
.join("chunks")
.join("000000.part");
tokio::fs::write(chunk_path, b"bad").await?;
let response = app
.oneshot(empty_request(
Method::POST,
&format!("/api/uploads/{}/complete", upload.upload_id),
)?)
.await?;
assert_eq!(response.status(), StatusCode::CONFLICT);
assert!(
!temp_dir
.path()
.join("complete")
.join("corrupt.bin")
.exists()
);
Ok(())
}
async fn create_upload(
app: &axum::Router,
name: &str,
size: u64,
) -> Result<CreateUploadResponse, Box<dyn std::error::Error>> {
let response = app
.clone()
.oneshot(json_request(
Method::POST,
"/api/uploads",
&json!({
"name": name,
"size": size,
"last_modified": 1_760_000_000_000_i64
}),
)?)
.await?;
assert_eq!(response.status(), StatusCode::OK);
decode_body(response).await
}
fn test_app(data_dir: &Path) -> axum::Router {
build_router(&AppConfig::new(
SocketAddr::from((Ipv4Addr::LOCALHOST, 0)),
concat!(env!("CARGO_MANIFEST_DIR"), "/static"),
data_dir,
))
}
fn json_request(
method: Method,
uri: &str,
body: &serde_json::Value,
) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method(method)
.uri(uri)
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from(serde_json::to_vec(body)?))?)
}
fn chunk_request(
upload_id: &str,
index: u64,
body: Vec<u8>,
) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method(Method::PUT)
.uri(format!("/api/uploads/{upload_id}/chunks/{index}"))
.header(header::CONTENT_TYPE, "application/octet-stream")
.body(Body::from(body))?)
}
fn empty_request(method: Method, uri: &str) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method(method)
.uri(uri)
.body(Body::empty())?)
}
async fn decode_body<T>(response: axum::response::Response) -> Result<T, Box<dyn std::error::Error>>
where
T: serde::de::DeserializeOwned,
{
let body = response.into_body().collect().await?.to_bytes();
Ok(serde_json::from_slice(&body)?)
}
tests/static_server.rs
+53
View File
@@ -0,0 +1,53 @@
use std::{
net::{Ipv4Addr, SocketAddr},
path::Path,
};
use axum::{body::Body, http::Request};
use http_body_util::BodyExt;
use tower::ServiceExt;
use upl::app::{AppConfig, build_router};
#[tokio::test]
async fn serves_index_page() -> Result<(), Box<dyn std::error::Error>> {
let app = test_app();
let response = app
.oneshot(Request::builder().uri("/").body(Body::empty())?)
.await?;
assert_eq!(response.status(), axum::http::StatusCode::OK);
let body = response.into_body().collect().await?.to_bytes();
let body = String::from_utf8(body.to_vec())?;
assert!(body.contains("<title>upl</title>"));
assert!(body.contains("Choose file"));
assert!(body.contains("Pending uploads"));
Ok(())
}
#[tokio::test]
async fn reports_health() -> Result<(), Box<dyn std::error::Error>> {
let app = test_app();
let response = app
.oneshot(Request::builder().uri("/healthz").body(Body::empty())?)
.await?;
assert_eq!(response.status(), axum::http::StatusCode::OK);
let body = response.into_body().collect().await?.to_bytes();
assert_eq!(body.as_ref(), b"ok");
Ok(())
}
fn test_app() -> axum::Router {
build_router(&AppConfig::new(
SocketAddr::from((Ipv4Addr::LOCALHOST, 0)),
concat!(env!("CARGO_MANIFEST_DIR"), "/static"),
Path::new(env!("CARGO_MANIFEST_DIR")).join("target/test-data/static-server"),
))
}
tests/upload_creation.rs
+96
View File
@@ -0,0 +1,96 @@
use std::{
net::{Ipv4Addr, SocketAddr},
path::Path,
};
use axum::{
body::Body,
http::{Request, StatusCode, header},
};
use http_body_util::BodyExt;
use serde_json::json;
use tempfile::TempDir;
use tower::ServiceExt;
use upl::{
app::{AppConfig, build_router},
model::{CHUNK_SIZE, CreateUploadResponse, UploadMeta},
};
#[tokio::test]
async fn creates_upload_metadata_on_disk() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let response = app
.oneshot(json_request(
"/api/uploads",
&json!({
"name": "movie:mkv",
"size": CHUNK_SIZE + 1,
"last_modified": 1_760_000_000_000_i64
}),
)?)
.await?;
assert_eq!(response.status(), StatusCode::OK);
let response_body = response.into_body().collect().await?.to_bytes();
let response: CreateUploadResponse = serde_json::from_slice(&response_body)?;
assert_eq!(response.chunk_size, CHUNK_SIZE);
assert_eq!(response.total_chunks, 2);
assert!(response.completed_chunks.is_empty());
let upload_dir = temp_dir.path().join("staging").join(&response.upload_id);
let meta_path = upload_dir.join("meta.json");
assert!(upload_dir.join("chunks").is_dir());
assert!(temp_dir.path().join("complete").is_dir());
let meta: UploadMeta = serde_json::from_slice(&tokio::fs::read(meta_path).await?)?;
assert_eq!(meta.id, response.upload_id);
assert_eq!(meta.original_name, "movie:mkv");
assert_eq!(meta.safe_name, "movie_mkv");
assert_eq!(meta.total_chunks, 2);
Ok(())
}
#[tokio::test]
async fn rejects_empty_upload_name() -> Result<(), Box<dyn std::error::Error>> {
let temp_dir = TempDir::new()?;
let app = test_app(temp_dir.path());
let response = app
.oneshot(json_request(
"/api/uploads",
&json!({
"name": " ",
"size": 10,
"last_modified": 1_760_000_000_000_i64
}),
)?)
.await?;
assert_eq!(response.status(), StatusCode::BAD_REQUEST);
Ok(())
}
fn test_app(data_dir: &Path) -> axum::Router {
build_router(&AppConfig::new(
SocketAddr::from((Ipv4Addr::LOCALHOST, 0)),
concat!(env!("CARGO_MANIFEST_DIR"), "/static"),
data_dir,
))
}
fn json_request(
uri: &str,
body: &serde_json::Value,
) -> Result<Request<Body>, Box<dyn std::error::Error>> {
Ok(Request::builder()
.method("POST")
.uri(uri)
.header(header::CONTENT_TYPE, "application/json")
.body(Body::from(serde_json::to_vec(&body)?))?)
}