diff --git a/docs/development.md b/docs/development.md index a2e3100..50a45f1 100644 --- a/docs/development.md +++ b/docs/development.md @@ -9,7 +9,7 @@ | `xtask/` | repository tasks run with `cargo xtask` | | `vm-tests/` | the sample NixOS machines, one folder each with its VM check | | `nix/` | the crane builds and the NixOS module | -| `recordings/` | the animations and screenshots the README and the [guide](guide.md) show | +| `recordings/` | the scripted demo GIFs and the sample machine WebP recordings the README and the [guide](guide.md) show | `cargo build` and `cargo test` act on every crate. `cargo run` needs the binary: `--bin bootvine`, `--bin scripted-boot` and so on. @@ -73,9 +73,7 @@ locally from the dev shell with `release-plz update`. | Model and render tests | `cargo test --workspace` | | Scripted demos | `cargo run --bin scripted-boot` (also `scripted-shutdown`, `scripted-luks-unlock`) | | Sample machines in a window | `nix run .#sample-vm-`, see the [guide](guide.md#sample-machines) | -| Boot and shutdown in a VM | `nix build .#checks.x86_64-linux.vm-boot-and-shutdown` | -| Passphrase prompt in a VM | `nix build .#checks.x86_64-linux.vm-luks-passphrase` | -| Failure panel in a VM | `nix build .#checks.x86_64-linux.vm-failures` | +| A sample machine's check | `nix build .#checks.x86_64-linux.vm-` | To watch your own user manager start the sample units: @@ -89,15 +87,18 @@ systemctl --user disable --runtime $(ls $units) ``` Everything a sample machine needs sits in its own folder under `vm-tests/`: -`machine.nix`, and for the machines with a check, `check.nix` and `check.py`. -`vm-tests/shared/` holds the settings every machine shares, the window setup -for `nix run` and the console helpers for the checks. A new machine is a new -folder plus its name in `vm-tests/default.nix`. +`machine.nix`, `check.nix` and `check.py`. `vm-tests/shared/` holds the +settings every machine shares, the window setup for `nix run`, the console +helpers for the checks and the screen recorder. A new machine is a new folder +plus its name in `vm-tests/default.nix`. The VM checks read the console as text, hold the boot at fixed points and wait on marker files instead of sleeping. The failures check starts each failing -unit by hand, so it knows which one is newest in the panel. To step through one by hand, run -`nix run .#checks.x86_64-linux.vm-boot-and-shutdown.driverInteractive` and +unit by hand, so it knows which one is newest in the panel. While a check runs, +`ScreenRecording` takes a frame every half second through a second QEMU +monitor socket, so it never competes with the driver's own monitor; the frames +land in the check's output under `recording/`. To step through a check by hand, run +`nix run .#checks.x86_64-linux.vm-slow-services.driverInteractive` and call `test_script()`. Prefix VM and recording commands with `env -u LD_LIBRARY_PATH` when your shell @@ -106,8 +107,8 @@ exports libraries that clash with [QEMU](https://www.qemu.org) or Chromium. ## Repository tasks - `nix run .#record-screencasts` re-records the animations from the scripted - demos and copies the passphrase and failures screenshots from the VM checks - into `recordings/`, for the README and the [guide](guide.md). + demos and turns the frames each VM check captured into + `recordings/.webp`, for the README and the [guide](guide.md). - `cargo xtask sync-default-config` rewrites the defaults block in [configuration](configuration.md#defaults) from `bootvine defaults`. - `cargo run --bin bootvine -- install --destdir --initrd dracut` stages an install diff --git a/docs/guide.md b/docs/guide.md index 505d67d..e91351b 100644 --- a/docs/guide.md +++ b/docs/guide.md @@ -1,9 +1,9 @@ # Guide What bootvine shows on the console, part by part, with the recordings that go -with each. The animations come from the [scripted demos](#scripted-demos); the -screenshots come from the [VM checks](development.md#testing) on real NixOS -machines. +with each. The GIFs come from the [scripted demos](#scripted-demos); the WebP +animations are real boots of the [sample machines](#sample-machines), recorded +by their [VM checks](development.md#testing). ## Reading the screen @@ -47,8 +47,12 @@ From top to bottom: ## Booting through the initrd -The same recording shows a boot from the start. With a systemd initrd, -bootvine starts about a second after the kernel: +A NixOS machine booting through the initrd into stage 2, handing over to a greetd login, then powering off + +The `slow-services` machine boots with a held hardware probe in the initrd, +slow units in stage 2 and one failing unit, hands over to its greetd login +and powers off. With a systemd initrd, bootvine starts about a second after +the kernel: - **The initrd stage**: udev settling, the root device and its file systems under `initrd.target`. @@ -63,6 +67,22 @@ bootvine starts about a second after the kernel: How bootvine gets in that early and survives the switch is in [architecture](architecture.md#starting-in-the-initrd). +### Without a display manager + +The same boot on a machine without a display manager, ending at a text login + +The `text-console` machine boots the same units without greetd. With no +display manager to wait for, bootvine hands the console back as soon as the +default target is reached, and the text login on `tty1` comes forward. + +### Without a systemd initrd + +The same boot with NixOS's scripted initrd: the kernel and initrd lines scroll past before bootvine starts in stage 2 + +The `scripted-initrd` machine uses NixOS's older scripted initrd, which has no +systemd. The initrd's lines scroll past as usual, and bootvine starts in +stage 2, from `sysinit.target` on. + ## Shutdown A poweroff: units stopping under the targets waiting on them @@ -77,7 +97,11 @@ On a poweroff, reboot or halt the tree turns around: ## Passphrase prompts -A LUKS passphrase prompt floating over the initrd tree +A LUKS passphrase prompt floating over the initrd tree, a wrong answer, then the unlock + +The `encrypted-volume` machine asks for a LUKS passphrase in the initrd; its +recording shows a wrong answer, then the right one. + When a unit asks for a passphrase, such as `systemd-cryptsetup@….service` unlocking a [LUKS](https://gitlab.com/cryptsetup/cryptsetup) volume, the @@ -94,7 +118,7 @@ bootvine started, bootvine waits behind it. ## Failures -The failure panel listing failed units with their reason and last journal lines +Units failing one after another, each landing in the failure panel with its reason and last journal lines A unit that broke or timed out gets an entry in the failure panel, newest first: @@ -114,8 +138,8 @@ first: The footer counts both. Failures from the initrd stay in the panel after the switch to the real root. -The screenshot comes from the `failures` sample machine, with every kind of -failure systemd reports. +The recording is the `failures` machine, where every kind of failure systemd +reports happens one after another. ## Trying it @@ -144,14 +168,14 @@ Each sample machine boots in a [QEMU](https://www.qemu.org) window with [Nix](https://nixos.org/download/), and logs in as root at the end. `poweroff` shows the shutdown. -| Run | Shows | -| --- | --- | -| `nix run .#sample-vm-slow-services` | slow units in the initrd and stage 2, one failing unit, and the handover to a greetd login | -| `nix run .#sample-vm-text-console` | the same units without a display manager, handing over at the default target | -| `nix run .#sample-vm-scripted-initrd` | the same units with NixOS's scripted initrd, so bootvine starts in stage 2 only | -| `nix run .#sample-vm-encrypted-volume` | a LUKS volume unlocked in the initrd; the passphrase is `sample passphrase` | -| `nix run .#sample-vm-failures` | a firmware failure in the initrd, then units that exit with an error, crash, time out, cannot be run, restart too often, report a status, die after starting, and one that fails only through its dependency | +| Run | Shows | Recording | +| --- | --- | --- | +| `nix run .#sample-vm-slow-services` | slow units in the initrd and stage 2, one failing unit, and the handover to a greetd login | [booting through the initrd](#booting-through-the-initrd) | +| `nix run .#sample-vm-text-console` | the same units without a display manager, handing over at the default target | [without a display manager](#without-a-display-manager) | +| `nix run .#sample-vm-scripted-initrd` | the same units with NixOS's scripted initrd, so bootvine starts in stage 2 only | [without a systemd initrd](#without-a-systemd-initrd) | +| `nix run .#sample-vm-encrypted-volume` | a LUKS volume unlocked in the initrd; the passphrase is `sample passphrase` | [passphrase prompts](#passphrase-prompts) | +| `nix run .#sample-vm-failures` | a firmware failure in the initrd, then units that exit with an error, crash, time out, cannot be run, restart too often, report a status, die after starting, and one that fails only through its dependency | [failures](#failures) | Each machine keeps its disk in `~/.cache/bootvine-.qcow2`; delete it to start fresh. The machines live in `vm-tests/`, one folder each, next to the -checks that drive them. +check that drives and records it. diff --git a/flake.nix b/flake.nix index 6d4c639..66e83f5 100644 --- a/flake.nix +++ b/flake.nix @@ -122,18 +122,21 @@ sample-vm-failures = runSampleMachine "failures" "units that fail in every way systemd reports, each with journal messages"; record-screencasts = { type = "app"; - meta.description = "Re-record the screencasts and the passphrase and failures screenshots into recordings/"; + meta.description = "Re-record the scripted demo animations and the sample machine recordings into recordings/"; program = lib.getExe ( pkgs.writeShellApplication { name = "record-screencasts"; runtimeInputs = [ pkgs.vhs + pkgs.libwebp builds.scripted-demos builds.repo-tasks ]; - text = "exec xtask record-screencasts --passphrase-png ${ - self.checks.${system}.vm-luks-passphrase - }/passphrase.png --failures-png ${self.checks.${system}.vm-failures}/failures.png"; + text = "exec xtask record-screencasts --vm-check-outputs ${ + pkgs.linkFarm "vm-check-outputs" ( + lib.mapAttrs (machineName: _: self.checks.${system}."vm-${machineName}") vmTests.machines + ) + }"; } ); }; @@ -172,7 +175,9 @@ bootvine-build = builds.bootvine; repo-tasks-build = builds.repo-tasks; } - // vmTests.checksFor pkgs + // lib.mapAttrs' ( + machineName: check: lib.nameValuePair "vm-${machineName}" check + ) (vmTests.checksFor pkgs) ); devShells = forEachSystem ( diff --git a/recordings/failures.png b/recordings/failures.png deleted file mode 100644 index 99d8607..0000000 Binary files a/recordings/failures.png and /dev/null differ diff --git a/recordings/passphrase.png b/recordings/passphrase.png deleted file mode 100644 index 4bfaa2b..0000000 Binary files a/recordings/passphrase.png and /dev/null differ diff --git a/vm-tests/default.nix b/vm-tests/default.nix index e516868..d546d1a 100644 --- a/vm-tests/default.nix +++ b/vm-tests/default.nix @@ -34,18 +34,10 @@ in ]; }; - checksFor = pkgs: { - vm-boot-and-shutdown = pkgs.testers.runNixOSTest ( - import ./slow-services/check.nix { machine = machines.slow-services; } - ); - vm-luks-passphrase = pkgs.testers.runNixOSTest ( - import ./encrypted-volume/check.nix { - machine = machines.encrypted-volume; - inherit pkgs; - } - ); - vm-failures = pkgs.testers.runNixOSTest ( - import ./failures/check.nix { machine = machines.failures; } - ); - }; + checksFor = + pkgs: + lib.mapAttrs ( + machineName: machine: + pkgs.testers.runNixOSTest (import ./${machineName}/check.nix { inherit machine pkgs; }) + ) machines; } diff --git a/vm-tests/encrypted-volume/check.nix b/vm-tests/encrypted-volume/check.nix index 086467b..b3174e5 100644 --- a/vm-tests/encrypted-volume/check.nix +++ b/vm-tests/encrypted-volume/check.nix @@ -3,16 +3,15 @@ let secretDisk = import ./secret-disk.nix { inherit pkgs; }; in { - name = "bootvine-luks-passphrase"; + name = "bootvine-encrypted-volume"; nodes.machine = { - imports = [ machine ]; + imports = [ + machine + ../shared/recorded-display.nix + ]; testing.initrdBackdoor = true; - virtualisation.qemu.options = [ - "-vga none" - "-device virtio-vga,xres=1280,yres=800" - ]; virtualisation.qemu.drives = [ secretDisk.qemuDrive ]; }; @@ -20,5 +19,6 @@ in PASSPHRASE = "${secretDisk.passphrase}" '' + builtins.readFile ../shared/console.py + + builtins.readFile ../shared/screen_recording.py + builtins.readFile ./check.py; } diff --git a/vm-tests/encrypted-volume/check.py b/vm-tests/encrypted-volume/check.py index 795a697..218cf59 100644 --- a/vm-tests/encrypted-volume/check.py +++ b/vm-tests/encrypted-volume/check.py @@ -7,7 +7,9 @@ def pending_prompt(): return next((name for name in listing if name.startswith("ask.")), "") +recording = ScreenRecording(machine) machine.start() +recording.start() retry(lambda _: pending_prompt() != "", timeout_seconds=60) wait_for_screen("passphrase") @@ -18,7 +20,6 @@ retry(lambda _: answered_prompts() == 1, timeout_seconds=30) retry(lambda _: pending_prompt() not in ("", first_prompt), timeout_seconds=30) wait_for_screen("passphrase") assert_text_visible(font_has_512_glyphs=True) -machine.screenshot("passphrase") machine.send_chars(f"{PASSPHRASE}\n") machine.wait_for_unit("systemd-cryptsetup@secret.service") machine.switch_root() @@ -29,3 +30,4 @@ journal = bootvine_journal() print(journal) assert " in Initrd " in journal, "never ran in the initrd" assert journal.count("answered a password prompt") == 2, "did not answer both attempts itself" +recording.stop() diff --git a/vm-tests/failures/check.nix b/vm-tests/failures/check.nix index ac18a18..41a0ae5 100644 --- a/vm-tests/failures/check.nix +++ b/vm-tests/failures/check.nix @@ -1,4 +1,4 @@ -{ machine }: +{ machine, ... }: { name = "bootvine-failures"; @@ -9,7 +9,10 @@ marker: "${pkgs.bash}/bin/bash -c 'until [ -e /run/${marker} ]; do sleep 0.1; done'"; in { - imports = [ machine ]; + imports = [ + machine + ../shared/recorded-display.nix + ]; bootvineSample.startFailuresByHand = true; testing.initrdBackdoor = true; @@ -36,14 +39,8 @@ }; }; - virtualisation = { - memorySize = 2048; - qemu.options = [ - "-vga none" - "-device virtio-vga,xres=1280,yres=800" - ]; - }; }; - testScript = builtins.readFile ../shared/console.py + builtins.readFile ./check.py; + testScript = builtins.readFile ../shared/console.py + + builtins.readFile ../shared/screen_recording.py + builtins.readFile ./check.py; } diff --git a/vm-tests/failures/check.py b/vm-tests/failures/check.py index c61243c..2c6e4b2 100644 --- a/vm-tests/failures/check.py +++ b/vm-tests/failures/check.py @@ -12,7 +12,9 @@ def failure_panel(): return "\n".join(lines[start:]) +recording = ScreenRecording(machine) machine.start() +recording.start() wait_for_screen("Initrd", "Initrd held for the check") start_failing_unit("initrd-nic-firmware-load") @@ -52,8 +54,8 @@ wait_for_screen("upstream-uplink exited with status 1", "lost connection to ups wait_for_screen("× 9 failed", "connection refused") assert "Started Upstream uplink" not in failure_panel(), f"systemd's own start line reached the panel:\n{screen()}" print(screen()) -machine.screenshot("failures") machine.succeed("touch /run/release-boot-failures-hold") retry(lambda _: "handing the console over" in bootvine_journal(), timeout_seconds=60) print(bootvine_journal()) +recording.stop() diff --git a/vm-tests/scripted-initrd/check.nix b/vm-tests/scripted-initrd/check.nix new file mode 100644 index 0000000..f588fd4 --- /dev/null +++ b/vm-tests/scripted-initrd/check.nix @@ -0,0 +1,25 @@ +{ machine, ... }: +{ + name = "bootvine-scripted-initrd"; + + nodes.machine = + { lib, ... }: + let + waitFor = flag: "/bin/sh -c 'until [ -e /run/${flag} ]; do sleep 0.1; done'"; + in + { + imports = [ + machine + ../shared/recorded-display.nix + ]; + systemd.services = { + sample-database.serviceConfig.ExecStop = lib.mkForce (waitFor "release-sample-database"); + backdoor.before = [ "sample-database.service" ]; + }; + }; + + testScript = + builtins.readFile ../shared/console.py + + builtins.readFile ../shared/screen_recording.py + + builtins.readFile ./check.py; +} diff --git a/vm-tests/scripted-initrd/check.py b/vm-tests/scripted-initrd/check.py new file mode 100644 index 0000000..c4cb06c --- /dev/null +++ b/vm-tests/scripted-initrd/check.py @@ -0,0 +1,28 @@ +def unit_state(unit): + return machine.succeed(f"systemctl is-active {unit} || true").strip() + + +recording = ScreenRecording(machine) +machine.start() +recording.start() + +wait_for_screen("Stage 2", "sample-database") +assert_bootvine_in_front() +assert_text_visible(font_has_512_glyphs=True) + +machine.wait_for_unit("greetd.service") +retry(lambda _: unit_state("bootvine") == "inactive", timeout_seconds=60) +journal = bootvine_journal() +print(journal) +watched = [line for line in journal.splitlines() if line.startswith("bootvine: watching")] +assert watched, "bootvine never started" +assert not any(" in Initrd " in line for line in watched), "bootvine ran in a scripted initrd" +assert "handing the console over" in journal, "never handed the console over" + +machine.succeed("systemctl poweroff --no-block") +retry(lambda _: unit_state("sample-database") == "deactivating", timeout_seconds=60) +wait_for_screen("Shutdown", "sample-database") +assert_bootvine_in_front() +machine.succeed("touch /run/release-sample-database") +machine.wait_for_shutdown() +recording.stop() diff --git a/vm-tests/shared/recorded-display.nix b/vm-tests/shared/recorded-display.nix new file mode 100644 index 0000000..c5968a7 --- /dev/null +++ b/vm-tests/shared/recorded-display.nix @@ -0,0 +1,11 @@ +{ + virtualisation = { + memorySize = 2048; + qemu.options = [ + "-vga none" + "-device virtio-vga,xres=1280,yres=800" + "-chardev socket,id=recording-monitor,path=recording-monitor,server=on,wait=off" + "-mon chardev=recording-monitor,mode=readline" + ]; + }; +} diff --git a/vm-tests/shared/screen_recording.py b/vm-tests/shared/screen_recording.py new file mode 100644 index 0000000..9fc4a1f --- /dev/null +++ b/vm-tests/shared/screen_recording.py @@ -0,0 +1,74 @@ +import os +import socket +import subprocess +import threading +import time + +RECORDING_FRAME_SECONDS = 0.5 +RECORDING_CONNECT_SECONDS = 30 +QEMU_PROMPT = b"(qemu) " + + +class ScreenRecording: + def __init__(self, recorded_machine): + self.recorded_machine = recorded_machine + self.frames_directory = os.path.join(recorded_machine.out_dir, "recording") + self.raw_frame = os.path.join(recorded_machine.state_dir, "recording-frame.ppm") + self.stopping = threading.Event() + self.thread = threading.Thread(target=self.record_frames, daemon=True) + + def start(self): + os.makedirs(self.frames_directory, exist_ok=True) + self.thread.start() + + def stop(self): + self.stopping.set() + self.thread.join() + + def connect_monitor(self): + monitor_path = os.path.join(self.recorded_machine.state_dir, "recording-monitor") + deadline = time.monotonic() + RECORDING_CONNECT_SECONDS + while time.monotonic() < deadline and not self.stopping.is_set(): + monitor = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) + try: + monitor.connect(monitor_path) + return monitor + except OSError: + monitor.close() + time.sleep(0.1) + return None + + def read_until_prompt(self, monitor): + received = b"" + while not received.endswith(QEMU_PROMPT): + chunk = monitor.recv(4096) + if not chunk: + raise ConnectionError("the recording monitor closed") + received += chunk + + def save_frame(self, monitor, frame_number): + monitor.sendall(f"screendump {self.raw_frame}\n".encode()) + self.read_until_prompt(monitor) + frame_path = os.path.join(self.frames_directory, f"frame-{frame_number:05d}.png") + unfinished_path = f"{frame_path}.unfinished" + subprocess.run(f"pnmtopng '{self.raw_frame}' > '{unfinished_path}'", shell=True, check=True) + os.replace(unfinished_path, frame_path) + + def record_frames(self): + monitor = self.connect_monitor() + if monitor is None: + return + try: + self.read_until_prompt(monitor) + frame_number = 0 + while not self.stopping.is_set(): + self.save_frame(monitor, frame_number) + frame_number += 1 + self.stopping.wait(RECORDING_FRAME_SECONDS) + except (OSError, ConnectionError, subprocess.CalledProcessError): + pass + finally: + monitor.close() + for leftover in os.listdir(self.frames_directory): + if leftover.endswith(".unfinished"): + os.remove(os.path.join(self.frames_directory, leftover)) diff --git a/vm-tests/slow-services/check.nix b/vm-tests/slow-services/check.nix index b8bdf04..29c7e01 100644 --- a/vm-tests/slow-services/check.nix +++ b/vm-tests/slow-services/check.nix @@ -1,6 +1,6 @@ -{ machine }: +{ machine, ... }: { - name = "bootvine-boot-and-shutdown"; + name = "bootvine-slow-services"; nodes.machine = { lib, ... }: @@ -8,7 +8,10 @@ waitFor = flag: "/bin/sh -c 'until [ -e /run/${flag} ]; do sleep 0.1; done'"; in { - imports = [ machine ]; + imports = [ + machine + ../shared/recorded-display.nix + ]; testing.initrdBackdoor = true; boot.initrd.systemd.services.initrd-hardware-probe.serviceConfig.ExecStart = lib.mkForce ( waitFor "release-initrd-hardware-probe" @@ -26,14 +29,8 @@ }; }; }; - virtualisation = { - memorySize = 2048; - qemu.options = [ - "-vga none" - "-device virtio-vga,xres=1280,yres=800" - ]; - }; }; - testScript = builtins.readFile ../shared/console.py + builtins.readFile ./check.py; + testScript = builtins.readFile ../shared/console.py + + builtins.readFile ../shared/screen_recording.py + builtins.readFile ./check.py; } diff --git a/vm-tests/slow-services/check.py b/vm-tests/slow-services/check.py index f327ed6..24cecc4 100644 --- a/vm-tests/slow-services/check.py +++ b/vm-tests/slow-services/check.py @@ -6,7 +6,9 @@ def unit_state(unit): return machine.succeed(f"systemctl is-active {unit} || true").strip() +recording = ScreenRecording(machine) machine.start() +recording.start() wait_for_screen("Initrd", "Slow hardware probe") assert_bootvine_in_front() @@ -56,3 +58,4 @@ assert_bootvine_in_front() assert_text_visible(font_has_512_glyphs=True) machine.succeed("touch /run/release-sample-database") machine.wait_for_shutdown() +recording.stop() diff --git a/vm-tests/text-console/check.nix b/vm-tests/text-console/check.nix new file mode 100644 index 0000000..d8e89a9 --- /dev/null +++ b/vm-tests/text-console/check.nix @@ -0,0 +1,29 @@ +{ machine, ... }: +{ + name = "bootvine-text-console"; + + nodes.machine = + { lib, ... }: + let + waitFor = flag: "/bin/sh -c 'until [ -e /run/${flag} ]; do sleep 0.1; done'"; + in + { + imports = [ + machine + ../shared/recorded-display.nix + ]; + testing.initrdBackdoor = true; + boot.initrd.systemd.services.initrd-hardware-probe.serviceConfig.ExecStart = lib.mkForce ( + waitFor "release-initrd-hardware-probe" + ); + systemd.services = { + sample-database.serviceConfig.ExecStop = lib.mkForce (waitFor "release-sample-database"); + backdoor.before = [ "sample-database.service" ]; + }; + }; + + testScript = + builtins.readFile ../shared/console.py + + builtins.readFile ../shared/screen_recording.py + + builtins.readFile ./check.py; +} diff --git a/vm-tests/text-console/check.py b/vm-tests/text-console/check.py new file mode 100644 index 0000000..a754799 --- /dev/null +++ b/vm-tests/text-console/check.py @@ -0,0 +1,31 @@ +def unit_state(unit): + return machine.succeed(f"systemctl is-active {unit} || true").strip() + + +recording = ScreenRecording(machine) +machine.start() +recording.start() + +wait_for_screen("Initrd", "Slow hardware probe") +assert_bootvine_in_front() +machine.succeed("touch /run/release-initrd-hardware-probe") +machine.switch_root() + +wait_for_screen("Stage 2", "slow-warm-up") +assert_bootvine_in_front() +assert_text_visible(font_has_512_glyphs=True) + +retry(lambda _: unit_state("bootvine") == "inactive", timeout_seconds=60) +journal = bootvine_journal() +print(journal) +assert "handing the console over" in journal, "never handed the console over" +assert foreground_terminal() != f"tty{BOOTVINE_TERMINAL}", "bootvine kept the console without a display manager" +machine.wait_until_tty_matches("1", "login: ") + +machine.succeed("systemctl poweroff --no-block") +retry(lambda _: unit_state("sample-database") == "deactivating", timeout_seconds=60) +wait_for_screen("Shutdown", "sample-database") +assert_bootvine_in_front() +machine.succeed("touch /run/release-sample-database") +machine.wait_for_shutdown() +recording.stop() diff --git a/xtask/src/main.rs b/xtask/src/main.rs index f61c226..f627c3d 100644 --- a/xtask/src/main.rs +++ b/xtask/src/main.rs @@ -16,8 +16,8 @@ struct RepoTasks { #[derive(Subcommand)] enum Task { #[command( - about = "Record the screencasts with vhs and copy the passphrase and failures screenshots \ - into recordings/" + about = "Record the screencasts with vhs and convert the passphrase and failures \ + screenshots to WebP in recordings/" )] RecordScreencasts(RecordScreencastsOptions), #[command( @@ -28,19 +28,18 @@ enum Task { #[derive(Args)] struct RecordScreencastsOptions { - #[arg(long, help = "passphrase.png produced by the vm-luks-passphrase check")] - passphrase_png: PathBuf, - #[arg(long, help = "failures.png produced by the vm-failures check")] - failures_png: PathBuf, + #[arg( + long, + help = "Directory with one entry per sample machine, each the output of its vm- \ + check" + )] + vm_check_outputs: PathBuf, } fn main() -> ExitCode { let outcome = match RepoTasks::parse().task { Task::RecordScreencasts(options) => { - record_screencasts::record_screencasts(&record_screencasts::CheckScreenshots { - passphrase_png: &options.passphrase_png, - failures_png: &options.failures_png, - }) + record_screencasts::record_screencasts(&options.vm_check_outputs) } Task::SyncDefaultConfig => sync_default_config::sync_default_config(), }; diff --git a/xtask/src/record_screencasts.rs b/xtask/src/record_screencasts.rs index ec2ea50..d649cc7 100644 --- a/xtask/src/record_screencasts.rs +++ b/xtask/src/record_screencasts.rs @@ -1,7 +1,7 @@ use std::{ - fs::{self, Permissions}, - os::unix::fs::PermissionsExt, - path::Path, + fs::{self, File}, + io::Read, + path::{Path, PathBuf}, }; use xshell::{Shell, cmd}; @@ -12,27 +12,100 @@ const TAPES: [&str; 2] = [ "scripted-demos/tapes/scripted-boot.tape", "scripted-demos/tapes/scripted-shutdown.tape", ]; +const RECORDING_FRAME_MILLISECONDS: &str = "500"; -pub struct CheckScreenshots<'a> { - pub passphrase_png: &'a Path, - pub failures_png: &'a Path, +fn sorted_entries(directory: &Path) -> Result, TaskError> { + let read_failure = |source| TaskError::File { + path: directory.to_path_buf(), + source, + }; + let mut entries = fs::read_dir(directory) + .map_err(read_failure)? + .map(|entry| entry.map(|entry| entry.path())) + .collect::, _>>() + .map_err(read_failure)?; + entries.sort(); + Ok(entries) } -fn copy_screenshot( - screenshot: &Path, - recordings: &Path, - published_name: &str, -) -> Result<(), TaskError> { - let copied_png = recordings.join(published_name); - fs::copy(screenshot, &copied_png) - .and_then(|_| fs::set_permissions(&copied_png, Permissions::from_mode(0o644))) +#[derive(PartialEq, Eq)] +struct FrameSize { + width: u32, + height: u32, +} + +fn png_frame_size(frame: &Path) -> Result { + let mut header = [0_u8; 24]; + File::open(frame) + .and_then(|mut file| file.read_exact(&mut header)) .map_err(|source| TaskError::File { - path: copied_png, + path: frame.to_path_buf(), source, - }) + })?; + let big_endian = |offset: usize| { + u32::from_be_bytes([ + header[offset], + header[offset + 1], + header[offset + 2], + header[offset + 3], + ]) + }; + Ok(FrameSize { + width: big_endian(16), + height: big_endian(20), + }) } -pub fn record_screencasts(screenshots: &CheckScreenshots) -> Result<(), TaskError> { +fn frames_at_final_size(frames: Vec) -> Result, TaskError> { + let Some(final_frame) = frames.last() else { + return Ok(frames); + }; + let final_size = png_frame_size(final_frame)?; + let sizes = frames + .iter() + .map(|frame| png_frame_size(frame)) + .collect::, _>>()?; + Ok(frames + .into_iter() + .zip(sizes) + .filter(|(_, size)| *size == final_size) + .map(|(frame, _)| frame) + .collect()) +} + +fn encode_machine_recording( + shell: &Shell, + check_output: &Path, + recordings: &Path, +) -> Result<(), TaskError> { + let Some(machine_name) = check_output.file_name() else { + return Ok(()); + }; + let frames = frames_at_final_size( + sorted_entries(&check_output.join("recording"))? + .into_iter() + .filter(|frame| { + frame + .extension() + .is_some_and(|extension| extension == "png") + }) + .collect(), + )?; + if frames.is_empty() { + return Err(TaskError::NoRecordingFrames(check_output.to_path_buf())); + } + let animation = recordings.join(machine_name).with_extension("webp"); + let frame_duration = RECORDING_FRAME_MILLISECONDS; + cmd!( + shell, + "img2webp -loop 0 -min_size -mixed -d {frame_duration} {frames...} -o {animation}" + ) + .quiet() + .run()?; + Ok(()) +} + +pub fn record_screencasts(vm_check_outputs: &Path) -> Result<(), TaskError> { let recordings = Path::new("recordings"); if !recordings.is_dir() { return Err(TaskError::NotInRepositoryRoot); @@ -42,6 +115,7 @@ pub fn record_screencasts(screenshots: &CheckScreenshots) -> Result<(), TaskErro let tape_path = Path::new(tape); cmd!(shell, "vhs {tape_path}").run() })?; - copy_screenshot(screenshots.passphrase_png, recordings, "passphrase.png")?; - copy_screenshot(screenshots.failures_png, recordings, "failures.png") + sorted_entries(vm_check_outputs)? + .iter() + .try_for_each(|check_output| encode_machine_recording(&shell, check_output, recordings)) } diff --git a/xtask/src/task_error.rs b/xtask/src/task_error.rs index 88213d8..840fa83 100644 --- a/xtask/src/task_error.rs +++ b/xtask/src/task_error.rs @@ -8,6 +8,8 @@ pub enum TaskError { Shell(#[from] xshell::Error), #[error("{}: {source}", path.display())] File { path: PathBuf, source: io::Error }, + #[error("{} has no recording frames", .0.display())] + NoRecordingFrames(PathBuf), #[error("no recordings directory here; run from the repository root")] NotInRepositoryRoot, #[error("{0} has no ```toml block under its Defaults heading")]