15 nodes (3 pre-stage + 11 stage + Completed) exceeded the 1280px main
container's usable width, producing a horizontal scrollbar under the
pipeline on the run page. Widen main to 1440px, tighten per-node min
widths, drop the scrollbar, and split camelCase labels so multi-word
stages ("WaitingReboot", "SpecValidate", "CPUStress") wrap onto two
lines instead of forcing node width.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a 1s client-side ticker that rewrites .run-duration text from a
data-started-at attribute, so the header timer on /runs/{id}
increments every second while the run is active. When an SSE swap
lands a fresh header the new server-rendered value seamlessly takes
over; when the run goes terminal the template drops the attribute
and the ticker silently skips the node, leaving the final elapsed in
place.
Other templ_*.go churn is cosmetic — regenerator versions differ
between CI and local and only the filename field in templ.Error
callsites changed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Ships all five phases of the deep-profile overhaul together. Runs now
carry a profile (quick/deep/soak); every profile walks the same
11-stage order — Inventory → Firmware → SpecValidate → SMART →
CPUStress → Storage → Network → Burn → GPU → PSU → Reporting —
with only per-stage durations and concurrency scaled.
Phase 1: profiles.ProfileRegistry loaded from vetting.yaml; runs.profile
column + CreateWithProfile; threshold table + evaluator seeded per-run
from the shared vetting.thresholds block; breach flips result at
/sensor + /result.
Phase 2: upgraded CPUStress (stress-ng --cpu-method=all --verify +
EDAC/MCE poll), Storage (fio --verify=md5 + SMART start/end delta),
Network (sustained iperf + /proc/net/dev deltas) with per-profile
knobs from Deps.
Phase 3: Burn super-stage with goroutine fan-out for CPU + memory +
fio + iperf, PSU rails sampled across the Burn window, SensorMux
(2 s flush, 500-sample cap) to absorb backpressure.
Phase 4: Firmware stage + firmware_snapshots table; probes dmidecode
(BIOS), ipmitool (BMC), ethtool -i (NIC), nvme (sysfs + id-ctrl),
lspci (HBA), /proc/cpuinfo (microcode). spec.DiffFirmware folds into
SpecValidate with pin-by-identifier and fan-out-across-component
matching; mismatches park the run in FailedHolding.
Phase 5: profile radio on the host start form, profile chip on the
run header, Firmware section in the HTML report, coverage artifact
uploaded from CI, agent/tests/fakes/ scaffold with Deps.LookPath
seam + stress_ng and dmidecode example fakes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Host page owns host metadata, full runs table with per-row stage strip,
in-flight banner, and empty-state CTA. Run page owns pipeline, active
step, logs, sub-steps, spec diffs, and hold banner with a breadcrumb
back to the host. Dashboard tile reverts to host-only.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reshapes the detail page into a run-view: hybrid horizontal pipeline
+ expanded active-step pane with sub-steps, a per-step log pane with
line-numbered permalinks and client-side search, and a runs-history
sidebar that navigates via ?run=N. Default step is server-picked
(running → failed → Reporting) so the operator lands on the thing
that's moving.
Adds a sub_steps table + SSE topic (substep-{run}-{stage}-{ordinal})
so per-disk and per-pass work (SMART, CPUStress CPU/RAM, Storage,
GPU) is visible in the UI instead of buried in stage summary JSON.
Agent emits sub-step reports from existing per-iteration loops.
Dashboard tiles become a mini run-view with a 9-dot step strip so
the operator reads run health across the whole grid at a glance.
Register page gets the same card shell + button styling.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Detail-page pipeline + log panes weren't updating without a manual
refresh. Root cause: the integrity attribute on htmx-ext-sse@2.2.2
in layout.templ was wrong, so the browser refused to execute the
script (SRI enforcement is silent — no user-visible error unless
you open devtools). htmx core loaded, boosted nav worked, forms
worked — but sse-connect/sse-swap were inert because the extension
never registered, so no EventSource was ever opened.
Replaced the claimed hash (Y4gc0CK6...) with the real one
(fw+eTlCc...) computed via
curl -sL https://unpkg.com/htmx-ext-sse@2.2.2 |
openssl dgst -sha384 -binary | openssl base64 -A
Added sse_e2e_test.go as a regression canary that mounts the real
chi router (RealIP + Recoverer + Logger middleware), opens
GET /events, publishes a tile-update via Runner, and asserts the
event lands on the wire. Server-side unit tests only verified
rendered HTML — this one covers the full publish→wire path, which
is what the next regression in this area will hit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pipeline fragment payload was a bare <div class=pipeline>, but the
sse-swap=pipeline-N wrapper lived only in the page shell. The first
outerHTML swap destroyed the wrapper, so every subsequent pipeline
event had nothing to target — forcing a manual refresh. RenderPipelineString
now emits the full <section id=pipeline-N sse-swap=... hx-swap=outerHTML>
wrapper, used from both the shell and the orchestrator publish path.
Also drop the red-bar styling from the empty DetailHold placeholder:
the wrapper's detail-hold class was painting an unconditional red band
between Pipeline and Actions whenever no hold was active.
The detail page was only partly live: Pipeline + LogTabs subscribed to
SSE, but the summary header, actions row, spec-diffs list and hold-key
block all froze at page-load and required a manual refresh to catch up
with state changes.
Extract each of those four regions into its own named templ component
with a stable id and sse-swap target, add Render*String helpers so the
orchestrator can publish pre-rendered fragments, and register a
HostDetailRenderer alongside the existing Tile/Pipeline renderers.
PublishHostDetail is folded into publishTileUpdate so every call site
that already refreshes a tile now also refreshes the detail page —
keeps the fan-out honest without scattering new publish calls.
The empty-state wrappers for spec-diffs and hold are load-bearing:
without the <section id=... sse-swap=...> present at initial GET, the
first live event after SpecValidate or Hold writes would have no DOM
node to swap into.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Non-destructive pre-declares "don't touch the disks" on Start: the
Storage stage skips wipe-probe, badblocks -w, and write-mode fio,
and reports a read-only summary. Runs a new non_destructive column;
threaded through Claim → agent tests.Deps → Storage stage.
Cancel halts an in-flight run. The orchestrator transitions to a
new StateCancelled via TriggerOperatorCancelled (valid from any
active state); the agent's next heartbeat returns cmd=cancel_stage,
which fires a stored CancelFunc on the per-stage context. Stage
subprocesses spawned with exec.CommandContext die with the context,
the agent posts a cancelled outcome, then powers the host off.
Destructive stages mid-run may leave the host in an intermediate
state — the UI confirm dialog warns the operator; recovery is
manual for now.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Every supported host runs vetting-reporter in-OS and heartbeats every
30s. WoL was never the thing that started vetting — the heartbeat
response's reboot_for_vetting command was. Firing WoL first only
crowded the run log with misleading diagnostics when the real failure
mode is "reporter isn't installed."
- StartRun 409s if the host hasn't heartbeated within 60s, pointing
the operator at /register/quick.sh.
- Dispatcher re-checks LastSeenAt at dispatch time (run may sit in
Queued long enough for the host to go offline); stale hosts mark
the run Failed with failed_stage=dispatch instead of looping.
- New StateWaitingReboot + TriggerRebootCommanded capture the actual
semantics. StateWaitingWoL kept as the hook point for a future
manual-override button.
- Tile disables the Start button with a quick.sh tooltip when the
host is offline, matching the server-side 409.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Pipeline now always renders all 13 nodes (3 pre-stage + 9 stage +
Completed), synthesising ghosts from run state when stage rows
aren't seeded yet. Makes a WaitingWoL host show the full timeline
ahead of it instead of just 4 dots.
Agent tags each log line with its stage; logs.Hub fans out to both
log-{runID} and log-{runID}-{stage} SSE events so the detail page
can show per-stage tabs with a pure-CSS radio-sibling switch. Flat
run log prepends [stage] so grep still works.
Dispatcher writes picked/sent-WoL/heartbeat lines into the per-run
log — the operator opens the detail page, sees WaitingWoL stuck,
and reads exactly what the dispatcher did and why nothing's
progressing, instead of having to tail journalctl on the LXC.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Click a tile to open /hosts/{id} — the canonical control surface per
host. Timeline renders every pre-stage, stage, and terminal node in
order, with the current one pulsing, failed ones flagged, and
downstream ones dimmed as skipped. Detail page shows summary, hold
card (when holding), all action buttons, spec diffs, a full-height
log pane, and a collapsed expected-spec YAML.
Tile slims to name, last-seen, status, and one primary action; a
CSS-overlay <a> makes the whole card clickable while buttons stay
receptive via z-index.
Runner.publishTileUpdate now also emits pipeline-{runID} fragments,
and CompleteStage wraps Stages.CompleteByName so stage completions
advance the timeline live — without this the dots only moved on
state transitions.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
vetting-agent gains a `host` subcommand that runs as a systemd service
installed by the quick-register one-liner, POSTing every 30s to
/api/v1/hosts/{mac}/heartbeat so the dashboard tile shows "online" or
"Nm ago" without waiting on WoL. Ships dormant client code for the
Phase 2 reboot_for_vetting command so the server can flip it on later
without a binary redeploy.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Operator pastes `curl -fsSL $ORCH/register/quick.sh | sudo bash` on the
target host (pre-wipe). The script probes MAC + CPU/RAM/disks/NICs/GPUs,
emits an expected-spec YAML, and POSTs to a new LAN-trusted JSON
endpoint /api/v1/hosts. The register page shows the command prefilled
with the orchestrator URL; the manual form moves into a collapsible
"Register manually" disclosure.
Can't log in from a fresh LXC deploy, and the service is LAN-only by
design. Rip out the whole bcrypt-password / signed-cookie session
layer: internal/auth, login templates, gen-admin-password binary +
Makefile targets, auth config block, login/logout routes and the
RequireSession middleware wrap. Agent bearer-token auth on
/api/v1/runs/{id}/* is untouched.
Operators who want a password can front the service with a reverse
proxy — noted in README and docs/operations.md.
josh