Persist beta, promotion, drift, and rollback payloads until occupancy and restart-capacity checks pass. Preserve rollback snapshots and campaign ordering, and allow held rollback campaigns to resume safely.
Fixes#28
Validation: 805 tests passed, Svelte checks clean, production build passed.
Retain historical server identities, results, and evidence after deletion. Mark archived entries as Deleted and prevent old work from targeting servers that reuse the same name.
Fixes#16.
Rename Deployments to Provisioning with compatible redirects and APIs. Add durable Start, Restart, Stop, and Rebuild operations alongside Create and Destroy; rebuild wipes the instance and recreates it in the same slot and port.
Add lifecycle, migration, recovery, and Komodo command tests. Type checks and production build pass; the full suite retains a telemetry assertion failure reproduced on the unchanged branch.
Refs #39
games_sp.log is not wiped on restart, so old J; lines from prior
sessions were poisoning the anchor. Now the scanner walks the tail
backward to the LATEST InitGame and only considers lines after it,
returning an opaque cycle key (the first J; line) plus a state. The
worker stamps game_started_at/ended_at with Date.now() at observation
time instead of inferring from the log's own timestamps. Same-players
game restarts produce a new key and cleanly re-anchor. When the server
transitions to 0 players, game state is cleared server-side and the
UI also gates the timer display on activeSince.
Adds servers.game_cycle_key column with additive migration.
Fleet cards and the per-server detail hero now show three live-ticking
timers: Active (since players > 0), Inactive (since server emptied), and
Game (since InitGame in games_sp.log, back-computed from the log's own
MM:SS prefix so it stays accurate across restarts). Anchors are
persisted on `servers` and fanned out via the SSE bus; a single 1s
ticker feeds every card. Idle map cycles are filtered out so the game
timer doesn't flicker.
Turns promotion into a first-class deployment (action='promote') so it lands
in the deployments UI with before/after content, per-server restart tracking,
and one-click rollback. Non-beta matching-scope servers get restart_state=
pending so the existing sweep drains and restarts them the same way drift does.
Also adds a per-version state (untested/tested/test-failed) so the upload
flow can visibly signal that a beta must land before promotion, and swaps
the scope-value text box for game/map dropdowns populated from the fleet.
Beta rollback at ready-promote is now labelled Reject and marks the version
test-failed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
GetDeployment returns config, not runtime state, so the health check was
always coming back with 'no container state found in response'. For
deployment-type servers the probe now tries GetDeploymentContainer,
InspectDeploymentContainer, and GetDeploymentStatus first, falling back
to GetDeployment only if those aren't available. The extractor also
recognizes more shapes (Health.Status, container_state, deployment_state,
etc.) and accepts more healthy strings (running, healthy, ok, started,
up).
When nothing matches, the truncated raw response snippet rides along in
the error field so the deployment/beta UI shows exactly which field to
teach the extractor next time.
Currently-stuck deployment_server rows are still 'pending', so the sweep
will retry them on its next 15s tick — no manual reset needed. Once one
of the widened endpoints resolves, those rows advance to 'done'
automatically.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A drift deployment scans each server's sp/ tree for files not in the
server_scripts inventory (untracked scripts left over from before EC was
managing the fleet), deletes them, and records the removed bytes in
deployment_files for audit. The existing sweep then runs the standard
wait-for-empty → Komodo restart cycle on each affected server.
To make that cycle safer, the sweep now has a second phase: after Komodo
restart succeeds we wait for komodoInspectHealthy + RCON to both come
back green before marking the deployment_server row 'done'. If either
stays broken the row stays pending and keeps retrying.
New: /api/drift (GET scan, POST run), drift panel on /deployments.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Scripts become first-class: named, versioned, uploaded via the UI, optionally
scoped to a game or map. Any change that would land on scope-matched servers
(marking-required or new version of a scoped script) must first pass a beta.
The beta runs a per-server state machine — deploy -beta.gsc → wait empty →
Komodo restart → wait healthy → optionally wait for a real match end
(games_sp.log 'ExitLevel: executed') — before it's eligible for promotion.
Rollback is always available and restores from .OLD.
The shared/scripts mirror and its betas/ staging folder are gone. Every
server now has a tracked script inventory (server_scripts table), populated
only by explicit upload flows.
New: /scripts, /scripts/[id], /betas/[id] UI; /api/scripts + /api/betas +
/api/servers/[name]/scripts endpoints; scripts.ts, inventory.ts, beta.ts
(rewritten), beta-runner.ts, match-log.ts, komodoInspectHealthy().
Removed: push.ts, /api/push, BetaPanel, legacy importer.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Player count says 1 but roster shows 0 for t5_ascension_1 — a parser
mismatch (likely Plutonium's status body uses Steam IDs instead of the
16+ char hex GUIDs my roster regex expects). To diagnose these without
guessing, always keep the raw RCON reply around and expose it on the
detail page.
rconStatusResult now includes rawReply on success too (used to only
persist it on bad-reply). Worker + probe-rcon endpoint persist it to
last_reply_snippet on every tick, so the byte-level truth about the
reply survives page loads.
New "RCON reply" panel on the detail page (visible whenever we have a
snippet or an endpoint) shows:
- Probe RCON button (was buried inside the offline-only diagnostic
panel; now accessible when healthy too)
- Parsed values in a grid: endpoint, players, map, gametype, roster
row count, last seen — so mismatches between "what the parser
extracted" and "what the reply actually said" are one glance apart
- Raw hex+ASCII dump of the last reply so the operator can read
literal player rows and see why the roster parser gave 0
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
"unparseable reply (34 bytes)" told us Plutonium answered but hid the
actual bytes, forcing another guess-and-check round. rconStatusResult
now emits a formatRawReply() hex + ASCII dump of the raw UDP payload on
every bad-reply, persisted in a new last_reply_snippet column so it
survives page loads.
Wrong-password detection widens beyond the "bad rcon" phrasing to
"(bad|invalid|wrong)…(rcon|password|pass)"; print-body replies that
don't match get their first non-header line surfaced verbatim so at
least *some* server-supplied text reaches the UI.
Probe RCON button (POST /api/servers/[name]/probe-rcon) fires the same
code path as the periodic sweep on demand, cuts the RCON iteration loop
from 30s per attempt to sub-second, and refreshes the header/roster/
health when it succeeds. Snippet <details> auto-opens on bad-reply so
the bytes are one glance away.
migrateServers() picks up the new column automatically via the
introspection path added last commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
rconGetStatus previously collapsed at least five distinct failure modes
(no password, no port, socket error, timeout, bad reply) into a single
null return, so the UI could only say "rcon unreachable" — useless when
diagnosing a real Alcatraz outage.
Refactored to rconStatusResult() returning a discriminated union with a
RconFailure reason and human-facing detail. The worker persists these as
"<reason>:<detail>" in last_error (no schema change). /api/servers now
returns the derived endpoint (host:port) per server plus a top-level
integrations block (rcon/komodo configured, timeout, default host).
Server detail page gains an RCON diagnostic panel when health is not
online: shows the exact endpoint being dialed, integration status,
timeout, a color-coded reason tag, the detail, and a targeted fix hint.
Wrong-password replies are detected via "bad rcon" text in the print
body.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
josh
