ddidderr/upl
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: add nginx deployment smoke test

Browse Source 
Add the nginx deployment artifact from PLAN.md. The example config keeps upl
behind nginx, sets client_max_body_size to 64 MiB, disables request buffering for
chunk uploads, forwards standard proxy headers, and leaves explicit placeholders
for TLS certificates and access control before public exposure.

Add just nginx-smoke as a reusable Docker-based verification. The script starts
upl with a temporary data directory, runs nginx as a reverse proxy, uploads a
17 MiB file through nginx, restarts the Rust backend mid-upload, confirms server
progress survives the restart through the proxy, uploads the remaining chunk,
completes the upload, and compares SHA-256 hashes.

Document the production nginx shape, the local Docker smoke-test caveat, and the
manual deployment retest scenario in TESTS.md.

Test Plan:
- bash -n scripts/nginx-smoke.sh
- just check
- just nginx-smoke

Refs: PLAN.md milestone 9
This commit is contained in:
ddidderr
2026-05-30 17:19:13 +02:00
parent 35cd0657bd
commit 858f4d949c
5 changed files with 225 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+25
View File
@@ -25,6 +25,9 @@ upl
static/index.html upload tool markup
static/styles.css responsive tool styling
static/app.js upload scheduler, retries, and browser resume state
Deployment
deploy/nginx/ nginx reverse proxy example
scripts/ reusable local smoke tests
Validation
tests/ integration tests for server behavior
TESTS.md reusable manual and automated test checklist
@@ -50,3 +53,25 @@ just run
```
`just check` also syntax-checks the static browser JavaScript with `node`.
## nginx
Run `upl` on localhost and put nginx in front of it for TLS and access control:
```sh
UPL_BIND=127.0.0.1:3000 UPL_DATA_DIR=/srv/upl/data upl
```
Use `deploy/nginx/upl.conf.example` as the starting point for the nginx site.
Before exposing the service, replace the certificate paths and add a protection
layer such as HTTP basic auth, an IP allowlist, or VPN-only access.
For a local Docker-based reverse-proxy smoke test:
```sh
just nginx-smoke
```
The smoke test binds the Rust server to `0.0.0.0` so the nginx container can
reach it through Docker's host gateway. The production nginx example keeps the
server bound to localhost.
TESTS.md
+8
View File
@@ -19,6 +19,11 @@ Keep this file as the reusable verification checklist while implementing
- `POST /api/uploads/:id/complete` assembles verified chunks.
- `POST /api/uploads/:id/complete` rejects incomplete uploads.
- `static/app.js` passes `node --check`.
- `just nginx-smoke`
- Runs upl behind nginx in Docker.
- Uploads a 17 MiB file through nginx.
- Restarts the Rust backend mid-upload, resumes through nginx, completes, and
compares SHA-256 hashes.
## Manual
@@ -40,6 +45,9 @@ features land.
- Attempt an invalid chunk index and confirm it is rejected.
- Attempt a wrong-size non-final chunk and confirm it is rejected.
- Complete an upload and compare the final file with the source file:
- Install `deploy/nginx/upl.conf.example` on the deployment host, add the real
TLS certificate and access-control settings, and repeat the resume scenarios
through the public nginx URL.
```sh
sha256sum source-file data/complete/uploaded-file
deploy/nginx/upl.conf.example
+37
View File
@@ -0,0 +1,37 @@
# Production shape for browser -> nginx -> upl -> local filesystem.
#
# Replace server_name, certificate paths, and access control before exposing
# this app. Keep upl itself bound to 127.0.0.1.
upstream upl_backend {
server 127.0.0.1:3000;
keepalive 16;
}
server {
listen 443 ssl http2;
server_name uploads.example.com;
ssl_certificate /etc/letsencrypt/live/uploads.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/uploads.example.com/privkey.pem;
client_max_body_size 64m;
# Add HTTP basic auth, an IP allowlist, VPN-only access, or another
# protection layer before exposing this personal upload tool publicly.
# auth_basic "upl";
# auth_basic_user_file /etc/nginx/upl.htpasswd;
location / {
proxy_pass http://upl_backend;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Real-IP $remote_addr;
}
}
justfile
+3
View File
@@ -10,6 +10,9 @@ static-check:
clippy:
cargo clippy --all-targets
nginx-smoke:
./scripts/nginx-smoke.sh
check:
just fmt
just test
scripts/nginx-smoke.sh
Executable
+152
View File
@@ -0,0 +1,152 @@
#!/usr/bin/env bash
set -euo pipefail
backend_port="${UPL_SMOKE_BACKEND_PORT:-39123}"
proxy_port="${UPL_SMOKE_PROXY_PORT:-39124}"
nginx_image="${NGINX_IMAGE:-nginx:stable-alpine}"
workspace_dir="$(pwd)"
mkdir -p "$workspace_dir/target/nginx-smoke"
tmp_dir="$(mktemp -d "$workspace_dir/target/nginx-smoke/run.XXXXXXXX")"
data_dir="$tmp_dir/data"
nginx_conf_dir="$tmp_dir/nginx-conf.d"
nginx_conf="$nginx_conf_dir/default.conf"
backend_log="$tmp_dir/backend.log"
source_file="$tmp_dir/source.bin"
chunk0="$tmp_dir/chunk0.part"
chunk1="$tmp_dir/chunk1.part"
backend_pid=""
nginx_container="upl-nginx-smoke-$$"
cleanup() {
if [[ -n "$backend_pid" ]] && kill -0 "$backend_pid" 2>/dev/null; then
kill "$backend_pid" 2>/dev/null || true
wait "$backend_pid" 2>/dev/null || true
fi
docker rm -f "$nginx_container" >/dev/null 2>&1 || true
rm -rf "$tmp_dir"
}
trap cleanup EXIT
start_backend() {
UPL_BIND="0.0.0.0:$backend_port" UPL_DATA_DIR="$data_dir" \
cargo run --quiet >"$backend_log" 2>&1 &
backend_pid="$!"
wait_for "http://127.0.0.1:$backend_port/healthz"
}
wait_for() {
local url="$1"
for _ in $(seq 1 80); do
if curl -fsS "$url" >/dev/null 2>&1; then
return 0
fi
sleep 0.1
done
echo "Timed out waiting for $url" >&2
if [[ -f "$backend_log" ]]; then
tail -n 80 "$backend_log" >&2 || true
fi
return 1
}
json_field() {
local field="$1"
node -e '
const field = process.argv[1];
let input = "";
process.stdin.setEncoding("utf8");
process.stdin.on("data", (chunk) => input += chunk);
process.stdin.on("end", () => {
const value = JSON.parse(input)[field];
if (value === undefined) process.exit(2);
process.stdout.write(String(value));
});
' "$field"
}
mkdir -p "$data_dir" "$nginx_conf_dir"
cat >"$nginx_conf" <<EOF
server {
listen $proxy_port;
client_max_body_size 64m;
location / {
proxy_pass http://host.docker.internal:$backend_port;
proxy_http_version 1.1;
proxy_request_buffering off;
proxy_buffering off;
proxy_read_timeout 3600s;
proxy_send_timeout 3600s;
proxy_set_header Host \$host;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_set_header X-Real-IP \$remote_addr;
}
}
EOF
start_backend
docker run -d --rm \
--name "$nginx_container" \
--add-host host.docker.internal:host-gateway \
-p "127.0.0.1:$proxy_port:$proxy_port" \
-v "$nginx_conf_dir:/etc/nginx/conf.d:ro" \
"$nginx_image" >/dev/null
wait_for "http://127.0.0.1:$proxy_port/healthz"
dd if=/dev/urandom of="$source_file" bs=1M count=17 status=none
dd if="$source_file" of="$chunk0" bs=1M count=16 status=none
dd if="$source_file" of="$chunk1" bs=1M skip=16 status=none
size="$(wc -c <"$source_file" | tr -d ' ')"
create_response="$(
curl -fsS \
-H "Content-Type: application/json" \
-d "{\"name\":\"source.bin\",\"size\":$size,\"last_modified\":1760000000000}" \
"http://127.0.0.1:$proxy_port/api/uploads"
)"
upload_id="$(printf '%s' "$create_response" | json_field upload_id)"
curl -fsS -X PUT \
-H "Content-Type: application/octet-stream" \
--data-binary "@$chunk0" \
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/0" >/dev/null
progress_before_restart="$(
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
)"
printf '%s' "$progress_before_restart" | grep -q '"completed_chunks":\[0\]'
kill "$backend_pid"
wait "$backend_pid" 2>/dev/null || true
backend_pid=""
start_backend
progress_after_restart="$(
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
)"
printf '%s' "$progress_after_restart" | grep -q '"completed_chunks":\[0\]'
curl -fsS -X PUT \
-H "Content-Type: application/octet-stream" \
--data-binary "@$chunk1" \
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/1" >/dev/null
complete_response="$(
curl -fsS -X POST "http://127.0.0.1:$proxy_port/api/uploads/$upload_id/complete"
)"
complete_path="$(printf '%s' "$complete_response" | json_field file_path)"
source_hash="$(sha256sum "$source_file" | awk '{print $1}')"
complete_hash="$(sha256sum "$complete_path" | awk '{print $1}')"
if [[ "$source_hash" != "$complete_hash" ]]; then
echo "Checksum mismatch after nginx-proxied resume" >&2
exit 1
fi
echo "nginx smoke ok: $upload_id"