chore: add nginx deployment smoke test
Add the nginx deployment artifact from PLAN.md. The example config keeps upl behind nginx, sets client_max_body_size to 64 MiB, disables request buffering for chunk uploads, forwards standard proxy headers, and leaves explicit placeholders for TLS certificates and access control before public exposure. Add just nginx-smoke as a reusable Docker-based verification. The script starts upl with a temporary data directory, runs nginx as a reverse proxy, uploads a 17 MiB file through nginx, restarts the Rust backend mid-upload, confirms server progress survives the restart through the proxy, uploads the remaining chunk, completes the upload, and compares SHA-256 hashes. Document the production nginx shape, the local Docker smoke-test caveat, and the manual deployment retest scenario in TESTS.md. Test Plan: - bash -n scripts/nginx-smoke.sh - just check - just nginx-smoke Refs: PLAN.md milestone 9
This commit is contained in:
@@ -25,6 +25,9 @@ upl
|
|||||||
static/index.html upload tool markup
|
static/index.html upload tool markup
|
||||||
static/styles.css responsive tool styling
|
static/styles.css responsive tool styling
|
||||||
static/app.js upload scheduler, retries, and browser resume state
|
static/app.js upload scheduler, retries, and browser resume state
|
||||||
|
Deployment
|
||||||
|
deploy/nginx/ nginx reverse proxy example
|
||||||
|
scripts/ reusable local smoke tests
|
||||||
Validation
|
Validation
|
||||||
tests/ integration tests for server behavior
|
tests/ integration tests for server behavior
|
||||||
TESTS.md reusable manual and automated test checklist
|
TESTS.md reusable manual and automated test checklist
|
||||||
@@ -50,3 +53,25 @@ just run
|
|||||||
```
|
```
|
||||||
|
|
||||||
`just check` also syntax-checks the static browser JavaScript with `node`.
|
`just check` also syntax-checks the static browser JavaScript with `node`.
|
||||||
|
|
||||||
|
## nginx
|
||||||
|
|
||||||
|
Run `upl` on localhost and put nginx in front of it for TLS and access control:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
UPL_BIND=127.0.0.1:3000 UPL_DATA_DIR=/srv/upl/data upl
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `deploy/nginx/upl.conf.example` as the starting point for the nginx site.
|
||||||
|
Before exposing the service, replace the certificate paths and add a protection
|
||||||
|
layer such as HTTP basic auth, an IP allowlist, or VPN-only access.
|
||||||
|
|
||||||
|
For a local Docker-based reverse-proxy smoke test:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
just nginx-smoke
|
||||||
|
```
|
||||||
|
|
||||||
|
The smoke test binds the Rust server to `0.0.0.0` so the nginx container can
|
||||||
|
reach it through Docker's host gateway. The production nginx example keeps the
|
||||||
|
server bound to localhost.
|
||||||
|
|||||||
@@ -19,6 +19,11 @@ Keep this file as the reusable verification checklist while implementing
|
|||||||
- `POST /api/uploads/:id/complete` assembles verified chunks.
|
- `POST /api/uploads/:id/complete` assembles verified chunks.
|
||||||
- `POST /api/uploads/:id/complete` rejects incomplete uploads.
|
- `POST /api/uploads/:id/complete` rejects incomplete uploads.
|
||||||
- `static/app.js` passes `node --check`.
|
- `static/app.js` passes `node --check`.
|
||||||
|
- `just nginx-smoke`
|
||||||
|
- Runs upl behind nginx in Docker.
|
||||||
|
- Uploads a 17 MiB file through nginx.
|
||||||
|
- Restarts the Rust backend mid-upload, resumes through nginx, completes, and
|
||||||
|
compares SHA-256 hashes.
|
||||||
|
|
||||||
## Manual
|
## Manual
|
||||||
|
|
||||||
@@ -40,6 +45,9 @@ features land.
|
|||||||
- Attempt an invalid chunk index and confirm it is rejected.
|
- Attempt an invalid chunk index and confirm it is rejected.
|
||||||
- Attempt a wrong-size non-final chunk and confirm it is rejected.
|
- Attempt a wrong-size non-final chunk and confirm it is rejected.
|
||||||
- Complete an upload and compare the final file with the source file:
|
- Complete an upload and compare the final file with the source file:
|
||||||
|
- Install `deploy/nginx/upl.conf.example` on the deployment host, add the real
|
||||||
|
TLS certificate and access-control settings, and repeat the resume scenarios
|
||||||
|
through the public nginx URL.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
sha256sum source-file data/complete/uploaded-file
|
sha256sum source-file data/complete/uploaded-file
|
||||||
|
|||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Production shape for browser -> nginx -> upl -> local filesystem.
|
||||||
|
#
|
||||||
|
# Replace server_name, certificate paths, and access control before exposing
|
||||||
|
# this app. Keep upl itself bound to 127.0.0.1.
|
||||||
|
|
||||||
|
upstream upl_backend {
|
||||||
|
server 127.0.0.1:3000;
|
||||||
|
keepalive 16;
|
||||||
|
}
|
||||||
|
|
||||||
|
server {
|
||||||
|
listen 443 ssl http2;
|
||||||
|
server_name uploads.example.com;
|
||||||
|
|
||||||
|
ssl_certificate /etc/letsencrypt/live/uploads.example.com/fullchain.pem;
|
||||||
|
ssl_certificate_key /etc/letsencrypt/live/uploads.example.com/privkey.pem;
|
||||||
|
|
||||||
|
client_max_body_size 64m;
|
||||||
|
|
||||||
|
# Add HTTP basic auth, an IP allowlist, VPN-only access, or another
|
||||||
|
# protection layer before exposing this personal upload tool publicly.
|
||||||
|
# auth_basic "upl";
|
||||||
|
# auth_basic_user_file /etc/nginx/upl.htpasswd;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://upl_backend;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_request_buffering off;
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
proxy_set_header Host $host;
|
||||||
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto https;
|
||||||
|
proxy_set_header X-Real-IP $remote_addr;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -10,6 +10,9 @@ static-check:
|
|||||||
clippy:
|
clippy:
|
||||||
cargo clippy --all-targets
|
cargo clippy --all-targets
|
||||||
|
|
||||||
|
nginx-smoke:
|
||||||
|
./scripts/nginx-smoke.sh
|
||||||
|
|
||||||
check:
|
check:
|
||||||
just fmt
|
just fmt
|
||||||
just test
|
just test
|
||||||
|
|||||||
Executable
+152
@@ -0,0 +1,152 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
backend_port="${UPL_SMOKE_BACKEND_PORT:-39123}"
|
||||||
|
proxy_port="${UPL_SMOKE_PROXY_PORT:-39124}"
|
||||||
|
nginx_image="${NGINX_IMAGE:-nginx:stable-alpine}"
|
||||||
|
|
||||||
|
workspace_dir="$(pwd)"
|
||||||
|
mkdir -p "$workspace_dir/target/nginx-smoke"
|
||||||
|
tmp_dir="$(mktemp -d "$workspace_dir/target/nginx-smoke/run.XXXXXXXX")"
|
||||||
|
data_dir="$tmp_dir/data"
|
||||||
|
nginx_conf_dir="$tmp_dir/nginx-conf.d"
|
||||||
|
nginx_conf="$nginx_conf_dir/default.conf"
|
||||||
|
backend_log="$tmp_dir/backend.log"
|
||||||
|
source_file="$tmp_dir/source.bin"
|
||||||
|
chunk0="$tmp_dir/chunk0.part"
|
||||||
|
chunk1="$tmp_dir/chunk1.part"
|
||||||
|
backend_pid=""
|
||||||
|
nginx_container="upl-nginx-smoke-$$"
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
if [[ -n "$backend_pid" ]] && kill -0 "$backend_pid" 2>/dev/null; then
|
||||||
|
kill "$backend_pid" 2>/dev/null || true
|
||||||
|
wait "$backend_pid" 2>/dev/null || true
|
||||||
|
fi
|
||||||
|
docker rm -f "$nginx_container" >/dev/null 2>&1 || true
|
||||||
|
rm -rf "$tmp_dir"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
start_backend() {
|
||||||
|
UPL_BIND="0.0.0.0:$backend_port" UPL_DATA_DIR="$data_dir" \
|
||||||
|
cargo run --quiet >"$backend_log" 2>&1 &
|
||||||
|
backend_pid="$!"
|
||||||
|
wait_for "http://127.0.0.1:$backend_port/healthz"
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for() {
|
||||||
|
local url="$1"
|
||||||
|
for _ in $(seq 1 80); do
|
||||||
|
if curl -fsS "$url" >/dev/null 2>&1; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 0.1
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Timed out waiting for $url" >&2
|
||||||
|
if [[ -f "$backend_log" ]]; then
|
||||||
|
tail -n 80 "$backend_log" >&2 || true
|
||||||
|
fi
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
json_field() {
|
||||||
|
local field="$1"
|
||||||
|
node -e '
|
||||||
|
const field = process.argv[1];
|
||||||
|
let input = "";
|
||||||
|
process.stdin.setEncoding("utf8");
|
||||||
|
process.stdin.on("data", (chunk) => input += chunk);
|
||||||
|
process.stdin.on("end", () => {
|
||||||
|
const value = JSON.parse(input)[field];
|
||||||
|
if (value === undefined) process.exit(2);
|
||||||
|
process.stdout.write(String(value));
|
||||||
|
});
|
||||||
|
' "$field"
|
||||||
|
}
|
||||||
|
|
||||||
|
mkdir -p "$data_dir" "$nginx_conf_dir"
|
||||||
|
|
||||||
|
cat >"$nginx_conf" <<EOF
|
||||||
|
server {
|
||||||
|
listen $proxy_port;
|
||||||
|
client_max_body_size 64m;
|
||||||
|
|
||||||
|
location / {
|
||||||
|
proxy_pass http://host.docker.internal:$backend_port;
|
||||||
|
proxy_http_version 1.1;
|
||||||
|
proxy_request_buffering off;
|
||||||
|
proxy_buffering off;
|
||||||
|
proxy_read_timeout 3600s;
|
||||||
|
proxy_send_timeout 3600s;
|
||||||
|
proxy_set_header Host \$host;
|
||||||
|
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
|
||||||
|
proxy_set_header X-Forwarded-Proto http;
|
||||||
|
proxy_set_header X-Real-IP \$remote_addr;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
start_backend
|
||||||
|
|
||||||
|
docker run -d --rm \
|
||||||
|
--name "$nginx_container" \
|
||||||
|
--add-host host.docker.internal:host-gateway \
|
||||||
|
-p "127.0.0.1:$proxy_port:$proxy_port" \
|
||||||
|
-v "$nginx_conf_dir:/etc/nginx/conf.d:ro" \
|
||||||
|
"$nginx_image" >/dev/null
|
||||||
|
wait_for "http://127.0.0.1:$proxy_port/healthz"
|
||||||
|
|
||||||
|
dd if=/dev/urandom of="$source_file" bs=1M count=17 status=none
|
||||||
|
dd if="$source_file" of="$chunk0" bs=1M count=16 status=none
|
||||||
|
dd if="$source_file" of="$chunk1" bs=1M skip=16 status=none
|
||||||
|
|
||||||
|
size="$(wc -c <"$source_file" | tr -d ' ')"
|
||||||
|
create_response="$(
|
||||||
|
curl -fsS \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d "{\"name\":\"source.bin\",\"size\":$size,\"last_modified\":1760000000000}" \
|
||||||
|
"http://127.0.0.1:$proxy_port/api/uploads"
|
||||||
|
)"
|
||||||
|
upload_id="$(printf '%s' "$create_response" | json_field upload_id)"
|
||||||
|
|
||||||
|
curl -fsS -X PUT \
|
||||||
|
-H "Content-Type: application/octet-stream" \
|
||||||
|
--data-binary "@$chunk0" \
|
||||||
|
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/0" >/dev/null
|
||||||
|
|
||||||
|
progress_before_restart="$(
|
||||||
|
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
|
||||||
|
)"
|
||||||
|
printf '%s' "$progress_before_restart" | grep -q '"completed_chunks":\[0\]'
|
||||||
|
|
||||||
|
kill "$backend_pid"
|
||||||
|
wait "$backend_pid" 2>/dev/null || true
|
||||||
|
backend_pid=""
|
||||||
|
start_backend
|
||||||
|
|
||||||
|
progress_after_restart="$(
|
||||||
|
curl -fsS "http://127.0.0.1:$proxy_port/api/uploads/$upload_id"
|
||||||
|
)"
|
||||||
|
printf '%s' "$progress_after_restart" | grep -q '"completed_chunks":\[0\]'
|
||||||
|
|
||||||
|
curl -fsS -X PUT \
|
||||||
|
-H "Content-Type: application/octet-stream" \
|
||||||
|
--data-binary "@$chunk1" \
|
||||||
|
"http://127.0.0.1:$proxy_port/api/uploads/$upload_id/chunks/1" >/dev/null
|
||||||
|
|
||||||
|
complete_response="$(
|
||||||
|
curl -fsS -X POST "http://127.0.0.1:$proxy_port/api/uploads/$upload_id/complete"
|
||||||
|
)"
|
||||||
|
complete_path="$(printf '%s' "$complete_response" | json_field file_path)"
|
||||||
|
|
||||||
|
source_hash="$(sha256sum "$source_file" | awk '{print $1}')"
|
||||||
|
complete_hash="$(sha256sum "$complete_path" | awk '{print $1}')"
|
||||||
|
|
||||||
|
if [[ "$source_hash" != "$complete_hash" ]]; then
|
||||||
|
echo "Checksum mismatch after nginx-proxied resume" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "nginx smoke ok: $upload_id"
|
||||||
Reference in New Issue
Block a user