Commit Graph
11 Commits
Author SHA1 Message Date
josh e5c0fda1ed Add native host management and role-based operator authentication
Introduce an outbound Python host agent, Debian installer, durable remote
operations, immutable game-file libraries, host-aware deployments and ports,
and occupancy-safe campaigns and draining. Native hosts do not require Komodo.

Add administrator, operator and viewer accounts, protected management routes,
secure sessions, password recovery, audit history and separate telemetry tokens.
Document first-admin setup, reporter migration and live-host acceptance checks.

Validation: 1,343 TypeScript tests and 39 agent tests passed, along with type
checking, the production build, compiled SSR/API smoke tests and administrator
recovery. Real Debian/Docker T5/T6 acceptance remains unrun.
2026-10-03 11:42:36 -04:00
josh 65d3a99846 Fix transient health checks and avoid unnecessary recovery 2026-10-02 13:06:18 -04:00
josh 47d6dc07b7 Add Plutonium version checks and scheduled rolling restarts 2026-09-21 18:47:24 -04:00
josh 3a63e6d3ce Add Metrics tab with historical server utilization and map capacity 2026-09-20 21:58:08 -04:00
joshandClaude Opus 4.7 d21e8187ff Roster parser: header-driven fixed-width, handle Plutonium T5's 9 columns
The raw hex dump showed Plutonium T5's `status` reply is a 9-column
format, not the 5-column shape the parser assumed:

  num score ping guid    name           lastmsg address              qport rate
  --- ----- ---- ------- ---------------- ------- --------------------- ----- -----
    0  3040   36 2741626 JoshWright^7          0 68.235.46.100:28960  -14733 25000

Three bugs in the old regex: it required 16+ hex chars for guid (T5
uses 7-digit decimal IDs), it expected name to run to end-of-line
(4 trailing columns get glued on), and it left Plutonium color codes
(^0..^9) in the rendered name.

Rewrote parsePlayerRoster to detect column boundaries from the header
line word positions instead — column N runs from word N's start to
word N+1's start, matching how CoD's status output actually lays out
(values borrow the padding between adjacent columns). Same code path
handles the old 5-column format and the T5 9-column format cleanly.

stripColorCodes() removes ^\d sequences from names in both dialects.
PlayerRoster component only appends `…` when the guid is actually
long enough to need truncation.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:37:25 -04:00
joshandClaude Opus 4.7 d8ed0d2572 Always show RCON reply on server detail — raw dump + parsed values
Player count says 1 but roster shows 0 for t5_ascension_1 — a parser
mismatch (likely Plutonium's status body uses Steam IDs instead of the
16+ char hex GUIDs my roster regex expects). To diagnose these without
guessing, always keep the raw RCON reply around and expose it on the
detail page.

rconStatusResult now includes rawReply on success too (used to only
persist it on bad-reply). Worker + probe-rcon endpoint persist it to
last_reply_snippet on every tick, so the byte-level truth about the
reply survives page loads.

New "RCON reply" panel on the detail page (visible whenever we have a
snippet or an endpoint) shows:
  - Probe RCON button (was buried inside the offline-only diagnostic
    panel; now accessible when healthy too)
  - Parsed values in a grid: endpoint, players, map, gametype, roster
    row count, last seen — so mismatches between "what the parser
    extracted" and "what the reply actually said" are one glance apart
  - Raw hex+ASCII dump of the last reply so the operator can read
    literal player rows and see why the roster parser gave 0

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:29:32 -04:00
joshandClaude Opus 4.7 57746eef19 Use status for the RCON probe (Plutonium doesn't wrap-recognize getstatus)
The hex dump surfaced Plutonium's actual reply:
  ff ff ff ff 70 72 69 6e 74 0a  ....print.
  55 6e 6b 6e 6f 77 6e 20 63 6f  Unknown co
  6d 6d 61 6e 64 20 22 67 65 74  mmand "get
  73 74 61 74 75 73 22 0a        status".

Auth was fine — Plutonium just doesn't recognize `getstatus` as an rcon
subcommand. In Q3-derived engines, `getstatus` is an anonymous query
(no rcon wrapper); we were sending `rcon <password> getstatus`, which
made Plutonium treat it as an unknown subcommand.

Switch rconStatusResult to use `status`, same command rconGetPlayerCount
has been using successfully all along. The response is a `print` body
with a `key: value` header (`map: mp_kino\n...`) followed by the
scoreboard, not the `\info\` line the statusResponse dialect uses.

New parseStatusHeader extracts key:value dvars from the header, stopping
at the `--- ---` separator (or the first player row) so guids and names
can't leak into the metadata. rconStatusResult now merges header values
with the statusResponse info line so both dialects produce map/gametype
in the RconStatus shape.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:19:26 -04:00
joshandClaude Opus 4.7 8affb11b57 Surface raw RCON reply bytes + Probe RCON button
"unparseable reply (34 bytes)" told us Plutonium answered but hid the
actual bytes, forcing another guess-and-check round. rconStatusResult
now emits a formatRawReply() hex + ASCII dump of the raw UDP payload on
every bad-reply, persisted in a new last_reply_snippet column so it
survives page loads.

Wrong-password detection widens beyond the "bad rcon" phrasing to
"(bad|invalid|wrong)…(rcon|password|pass)"; print-body replies that
don't match get their first non-header line surfaced verbatim so at
least *some* server-supplied text reaches the UI.

Probe RCON button (POST /api/servers/[name]/probe-rcon) fires the same
code path as the periodic sweep on demand, cuts the RCON iteration loop
from 30s per attempt to sub-second, and refreshes the header/roster/
health when it succeeds. Snippet <details> auto-opens on bad-reply so
the bytes are one glance away.

migrateServers() picks up the new column automatically via the
introspection path added last commit.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 11:12:50 -04:00
joshandClaude Opus 4.7 7b4e6e0565 Surface structured RCON failure reasons in the fleet UI
rconGetStatus previously collapsed at least five distinct failure modes
(no password, no port, socket error, timeout, bad reply) into a single
null return, so the UI could only say "rcon unreachable" — useless when
diagnosing a real Alcatraz outage.

Refactored to rconStatusResult() returning a discriminated union with a
RconFailure reason and human-facing detail. The worker persists these as
"<reason>:<detail>" in last_error (no schema change). /api/servers now
returns the derived endpoint (host:port) per server plus a top-level
integrations block (rcon/komodo configured, timeout, default host).

Server detail page gains an RCON diagnostic panel when health is not
online: shows the exact endpoint being dialed, integration status,
timeout, a color-coded reason tag, the detail, and a targeted fix hint.
Wrong-password replies are detected via "bad rcon" text in the print
body.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 09:06:27 -04:00
joshandClaude Opus 4.7 9bff47a9a4 Perk-a-cola redesign + surface RCON map/gametype/roster/health
Themes the fleet manager around the Electric Cherry perk-a-cola from Mob of
the Dead — cherry-red art-deco label with electric cyan discharge. New
design tokens (Alfa Slab One + Inter + JetBrains Mono, self-hosted), full
redesign of layout/dashboard/cards/detail page, tokens pass on
deployments/betas/pipeline/beta/diff, cherry+lightning favicon.

Lights up server data that RCON was already fetching but the UI dropped.
parsePlayerRoster() extracts num/score/ping/guid/name from both status
dialects. Worker persists last_map/last_gametype/last_roster/last_seen_at/
last_error (new columns with idempotent boot-time ALTERs) and emits them
over SSE. Fleet cards now show health dot, map · gametype, top-3 roster
preview, and last-seen. Detail page adds a sortable full roster with
distinct empty-lobby vs offline states. New health classifier
(online/stale/offline) drives the visual state.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 08:20:54 -04:00
josh 854f5532a9 Replace IW4MAdmin with direct RCON for player counts
IW4M kept hitting a network wall (container -> host LAN IP -> IW4M
loopback bind) that we couldn't cleanly punch through, and the two-minute
!rt token added enough operational friction that the pause-all-deployments
gate never got a fair test. RCON gives us the same fact — how many
players are on a server right now — direct from each game server over
UDP, with no session, no token, no proxy.

- Add integrations/rcon.ts: dgram UDP client. rconGetPlayerCount() and
  rconGetStatus() parse both `status` (print + scoreboard) and
  `statusResponse` (info line + rows) dialects. Timeout via
  RCON_TIMEOUT_MS.
- Add komodoResolveGamePort() + extractUdpPort(). Reads Komodo's read
  API for a server, walks both docker-compose-style port objects and
  docker-run-style "host:container/udp" strings, caches into
  servers.game_port on first success.
- Replace pipelineReady() guard in sweep with a simple
  integrations-configured check. Same held/failed/manual outcomes,
  but keyed on RCON+Komodo instead of IW4M+auth-state.
- Worker's status refresh fans out per-server over RCON at concurrency=4
  so a 15-server fleet doesn't spike UDP.
- Schema: drop iw4m_match/iw4m_value, add game_port + rcon_host.
- Delete every auth artifact: TokenModal, auth store, auth-state module,
  /api/auth/token, /api/events/auth, iw4m.test.ts. Layout no longer
  mounts the modal, pipeline store no longer tracks a frozen state.
- README + .env.example + docker-compose.yml + komodo/stack.toml swap
  IW4M_* env for RCON_PASSWORD / RCON_HOST / RCON_TIMEOUT_MS. Same
  host.docker.internal:host-gateway extra_hosts entry, repurposed
  for RCON reach.
- Tests: add rcon.test.ts and komodo.test.ts, update sweep.test.ts to
  drop the auth-freeze case. 23/23 passing.
2026-09-12 01:33:06 -04:00