Commit Graph
106 Commits
Author SHA1 Message Date
josh f171889dfb Pace collection startups and clarify deployment queue progress 2026-09-19 20:33:08 -04:00
josh 1d073779fe Fix false stale Fleet reports caused by clock skew 2026-09-19 20:17:43 -04:00
josh 345839fbd4 Use current promoted files for blueprint launches 2026-09-19 19:58:30 -04:00
josh 41891b4209 Add per-server telemetry request diagnostics 2026-09-19 18:59:30 -04:00
josh a182aeb950 Add persistent player leaderboards from game telemetry 2026-09-18 16:43:44 -04:00
josh 5bb49a776e Simplify telemetry to three game events 2026-09-18 15:35:21 -04:00
josh 6d903adb31 Fix fleet stale-status flicker during polling 2026-09-18 15:11:49 -04:00
josh f54cac8c50 Unify UI workflows and streamline fleet management 2026-09-18 14:00:28 -04:00
josh 9cf0bd87f4 Refresh Fleet with discovery, operation progress, and lobby watches 2026-09-18 11:59:14 -04:00
josh ef8ff69d17 Refresh Configs with browser editing and source inspection 2026-09-18 11:02:22 -04:00
josh ec51e38312 Refresh script library with batch uploads and beta handoff 2026-09-18 09:57:57 -04:00
josh b2ec01daf5 Fix scope restoration for promoted scripts 2026-09-18 08:08:17 -04:00
josh 3c49437bf9 Add GitHub plugin release imports and update discovery 2026-09-18 01:29:38 -04:00
josh a718efc9e7 Add publisher release versions to plugins 2026-09-18 01:16:45 -04:00
josh 453821805a Fix server deletion cleanup and allow failed cleanup to resume 2026-09-18 00:49:30 -04:00
josh 274aedb66e Add House Party Protocol for fleet-wide queued destruction 2026-09-18 00:39:19 -04:00
josh 586133c711 Conform blueprint launches to desired fleet counts 2026-09-18 00:28:17 -04:00
josh 989931a0f2 Add multi-server blueprint collections and atomic launches 2026-09-18 00:08:54 -04:00
josh 782c7c0bda Add deployment blueprints and reviewed launch recipes 2026-09-17 23:43:12 -04:00
josh 4237371360 Fix beta preset selection and improve test plan builder UI 2026-09-17 21:51:25 -04:00
josh a50eae9d21 Add configurable beta test plans and staged rollout 2026-09-17 21:35:28 -04:00
josh 5b5266da3a Add operation flows and activity logs for betas and campaigns 2026-09-17 19:34:26 -04:00
josh 9dcf4ac1d3 Add T6 Zombies fleet support with Plutainer configuration 2026-09-17 18:03:12 -04:00
josh 885abbdffe Use database fleet registry and gate RCON polling on container readiness 2026-09-17 17:28:38 -04:00
josh c2ad5cc95a Queue server creation and verify health and selected map before completion 2026-09-17 17:14:36 -04:00
josh a611850649 new destruction flow 2026-09-17 16:57:08 -04:00
josh 6f83700161 Separate campaigns from server deployment lifecycle tracking 2026-09-17 16:42:37 -04:00
josh 60e44636aa Fix initial artifact deployment and drift file replacement 2026-09-17 16:15:47 -04:00
josh e8e7332365 Automatically deploy the lowest available lobby with exact ID preview 2026-09-17 15:59:29 -04:00
josh ae173517b6 Persist Plutonium keys by server ID and add key management 2026-09-17 15:54:10 -04:00
josh e4505a3be8 Set telemetry server ID when creating Komodo servers 2026-09-17 15:28:23 -04:00
josh 858704bb10 adds round telemetry support 2026-09-17 14:35:42 -04:00
josh 8e04a190b3 Roll deployment restarts through a shared fleet-wide recovery limit 2026-09-17 11:25:06 -04:00
josh 3f0d6dd87d Apply selected server maps automatically without config overlays 2026-09-17 11:05:01 -04:00
josh 1cd4eb6f09 Use UpdateResourceMeta to apply Komodo deployment tags 2026-09-17 09:45:18 -04:00
josh 516dfeb4c3 Fix Komodo tag IDs in Mongo Extended JSON responses 2026-09-17 09:41:27 -04:00
joshandClaude Opus 4.7 13a29e5ace Add deploy + destroy server flow; migrate to t5_<mode>_<slug>_<n> naming
Full server lifecycle management from Electric Cherry: pick game/mode/map/pluto
key in the Fleet page, EC provisions the deployment via Komodo, allocates a
port in 28960-28980, creates the 0o777 host dir, and starts the container.
Destroy button tears everything down (Komodo Destroy+Delete, rm -rf host dir,
transactional DB cleanup) and surfaces any partial-failure warnings inline.

Same phase cuts over to the new naming scheme (t5_zombie_cosmodrome_1 etc.).
Komodo entity names now match EC server names 1:1 (no more _/- transform), and
the map catalog carries friendly/slug/canonical for each of the 11 t5 zombie
maps. Server rows gain game_mode, map_slug, host_dir, config_file, game_id,
image, created_by, and destroyed_at columns for provisioning + audit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-13 18:38:42 -04:00
joshandClaude Opus 4.7 8c15cae80d Replace -beta sidecar and .OLD backup with DB snapshots
Beta deploys used to rename foo.gsc -> foo.OLD and write beta bytes to
a foo-beta.gsc sidecar, forcing the mod loader to know the naming
convention. Now the beta content lives at the actual target filename
(same as configs already did) and the pre-beta bytes are captured in
a new beta_file_snapshots table for rollback.

- schema.sql: new beta_file_snapshots(beta_id, server, rel_path,
  prior_content BLOB, prior_sha256, prior_size). Handles scripts,
  plugins, and the composed dedicated_sp.cfg uniformly.
- db/index.ts: assertSnapshotSchemeSafe boot guard refuses to start
  if a legacy running/ready-promote beta is present on first upgrade.
  A settings sentinel (snapshots_v1) makes it one-shot.
- beta.ts openBeta: read prior bytes, insert snapshot, overwrite
  target directly.
- beta.ts rollbackBeta: restore from snapshot if had_original, else
  delete the fresh file. Drop snapshot rows after restore.
- beta.ts promoteBeta: no more rm -beta / rm .OLD calls, pull prior
  content from the snapshot table for the deployment audit, drop
  snapshots after promote.
- files.ts: delete betaFile() and oldFile().
- drift.ts: docstring updated; no special-casing needed since beta
  bytes and inventory now agree on the target rel_path.
- servers/[name]/+page.svelte: drop dead -beta/.OLD row filters.
- tests: plugin write asserts the target filename; unused
  betaFile/oldFile tests removed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-13 17:57:43 -04:00
josh f10f0cefef drift update 2026-09-12 23:15:14 -04:00
josh 6ae9004ed2 Drift now covers all three destinations 2026-09-12 19:16:14 -04:00
josh 8c40bbeff1 config and plugin 2026-09-12 19:11:44 -04:00
josh 85bcb05a7d multi script beta 2026-09-12 18:23:56 -04:00
joshandClaude Opus 4.7 1afdfddc87 Promote as a real deployment; version state and scope dropdowns
Turns promotion into a first-class deployment (action='promote') so it lands
in the deployments UI with before/after content, per-server restart tracking,
and one-click rollback. Non-beta matching-scope servers get restart_state=
pending so the existing sweep drains and restarts them the same way drift does.

Also adds a per-version state (untested/tested/test-failed) so the upload
flow can visibly signal that a beta must land before promotion, and swaps
the scope-value text box for game/map dropdowns populated from the fleet.
Beta rollback at ready-promote is now labelled Reject and marks the version
test-failed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 14:24:35 -04:00
joshandClaude Opus 4.7 edfe4dfc09 Add drift detection and removal as a real deployment
A drift deployment scans each server's sp/ tree for files not in the
server_scripts inventory (untracked scripts left over from before EC was
managing the fleet), deletes them, and records the removed bytes in
deployment_files for audit. The existing sweep then runs the standard
wait-for-empty → Komodo restart cycle on each affected server.

To make that cycle safer, the sweep now has a second phase: after Komodo
restart succeeds we wait for komodoInspectHealthy + RCON to both come
back green before marking the deployment_server row 'done'. If either
stays broken the row stays pending and keeps retrying.

New: /api/drift (GET scan, POST run), drift panel on /deployments.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 13:44:52 -04:00
joshandClaude Opus 4.7 590f803d93 Auto-derive server.game and server.map from the server name
Server folders encode the game and map (e.g. t5_ascension_1 = game t5, map
alias ascension, lobby 1). A T5_MAP_ALIASES table translates the friendly
alias to the canonical mapname RCON reports (ascension → zombie_cosmodrome),
covering every base + Rezurrection map. seedServerRegistry now populates
game and map at insert time and backfills existing NULL rows on every boot;
unknown aliases log a warning so we know when to add them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 13:35:43 -04:00
joshandClaude Opus 4.7 93d5a8e2e5 Rework as scripts-centric, upload-only, gated beta lifecycle
Scripts become first-class: named, versioned, uploaded via the UI, optionally
scoped to a game or map. Any change that would land on scope-matched servers
(marking-required or new version of a scoped script) must first pass a beta.
The beta runs a per-server state machine — deploy -beta.gsc → wait empty →
Komodo restart → wait healthy → optionally wait for a real match end
(games_sp.log 'ExitLevel: executed') — before it's eligible for promotion.
Rollback is always available and restores from .OLD.

The shared/scripts mirror and its betas/ staging folder are gone. Every
server now has a tracked script inventory (server_scripts table), populated
only by explicit upload flows.

New: /scripts, /scripts/[id], /betas/[id] UI; /api/scripts + /api/betas +
/api/servers/[name]/scripts endpoints; scripts.ts, inventory.ts, beta.ts
(rewritten), beta-runner.ts, match-log.ts, komodoInspectHealthy().
Removed: push.ts, /api/push, BetaPanel, legacy importer.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 13:25:06 -04:00
joshandClaude Opus 4.7 6aae72f508 Add per-server log viewer (console.log + games_sp.log)
New Logs section on the server detail page — tabbed viewer with
auto-refresh every 5s (toggleable), sticky-to-bottom scrolling, and a
manual refresh button. Two whitelisted files per server:

  /root/zombies/t5/<name>/logs/console.log
  /root/zombies/t5/<name>/logs/games_sp.log

Backend:
  - src/lib/server/fleet/logs.ts — safe path resolution + tailLog()
    that reads only the last 256 KB of the file, splits on newlines,
    drops the (potentially partial) first line if we didn't start at
    byte 0, and returns the last N lines with size + mtime + truncated.
    Two path-traversal guards: server directory must be a real
    subdirectory of SERVERS_ROOT, and the target must resolve inside
    it.
  - GET /api/servers/[name]/logs/[file]?lines=200 — whitelisted keys
    only (console, games_sp), 400 on unknown keys, ok:true+missing:
    true when the log file hasn't been written yet.

Frontend:
  - LogPanel.svelte with tabs, auto-refresh checkbox, size/mtime meta,
    and a <pre> body that snaps to bottom on refresh unless the user
    has scrolled up.
  - 7 unit tests covering path safety (including .. traversal) and
    tail behavior (missing file → null, small file, oversized file
    → truncated, trailing newline handling).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:50:19 -04:00
joshandClaude Opus 4.7 d21e8187ff Roster parser: header-driven fixed-width, handle Plutonium T5's 9 columns
The raw hex dump showed Plutonium T5's `status` reply is a 9-column
format, not the 5-column shape the parser assumed:

  num score ping guid    name           lastmsg address              qport rate
  --- ----- ---- ------- ---------------- ------- --------------------- ----- -----
    0  3040   36 2741626 JoshWright^7          0 68.235.46.100:28960  -14733 25000

Three bugs in the old regex: it required 16+ hex chars for guid (T5
uses 7-digit decimal IDs), it expected name to run to end-of-line
(4 trailing columns get glued on), and it left Plutonium color codes
(^0..^9) in the rendered name.

Rewrote parsePlayerRoster to detect column boundaries from the header
line word positions instead — column N runs from word N's start to
word N+1's start, matching how CoD's status output actually lays out
(values borrow the padding between adjacent columns). Same code path
handles the old 5-column format and the T5 9-column format cleanly.

stripColorCodes() removes ^\d sequences from names in both dialects.
PlayerRoster component only appends `…` when the guid is actually
long enough to need truncation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:37:25 -04:00
joshandClaude Opus 4.7 57746eef19 Use status for the RCON probe (Plutonium doesn't wrap-recognize getstatus)
The hex dump surfaced Plutonium's actual reply:
  ff ff ff ff 70 72 69 6e 74 0a  ....print.
  55 6e 6b 6e 6f 77 6e 20 63 6f  Unknown co
  6d 6d 61 6e 64 20 22 67 65 74  mmand "get
  73 74 61 74 75 73 22 0a        status".

Auth was fine — Plutonium just doesn't recognize `getstatus` as an rcon
subcommand. In Q3-derived engines, `getstatus` is an anonymous query
(no rcon wrapper); we were sending `rcon <password> getstatus`, which
made Plutonium treat it as an unknown subcommand.

Switch rconStatusResult to use `status`, same command rconGetPlayerCount
has been using successfully all along. The response is a `print` body
with a `key: value` header (`map: mp_kino\n...`) followed by the
scoreboard, not the `\info\` line the statusResponse dialect uses.

New parseStatusHeader extracts key:value dvars from the header, stopping
at the `--- ---` separator (or the first player row) so guids and names
can't leak into the metadata. rconStatusResult now merges header values
with the statusResponse info line so both dialects produce map/gametype
in the RconStatus shape.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:19:26 -04:00
joshandClaude Opus 4.7 8affb11b57 Surface raw RCON reply bytes + Probe RCON button
"unparseable reply (34 bytes)" told us Plutonium answered but hid the
actual bytes, forcing another guess-and-check round. rconStatusResult
now emits a formatRawReply() hex + ASCII dump of the raw UDP payload on
every bad-reply, persisted in a new last_reply_snippet column so it
survives page loads.

Wrong-password detection widens beyond the "bad rcon" phrasing to
"(bad|invalid|wrong)…(rcon|password|pass)"; print-body replies that
don't match get their first non-header line surfaced verbatim so at
least *some* server-supplied text reaches the UI.

Probe RCON button (POST /api/servers/[name]/probe-rcon) fires the same
code path as the periodic sweep on demand, cuts the RCON iteration loop
from 30s per attempt to sub-second, and refreshes the header/roster/
health when it succeeds. Snippet <details> auto-opens on bad-reply so
the bytes are one glance away.

migrateServers() picks up the new column automatically via the
introspection path added last commit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:12:50 -04:00