Commit Graph
13 Commits
Author SHA1 Message Date
josh 88a950692a Fix retention query performance and cleanup scheduling 2026-10-06 10:39:32 -04:00
josh 3952c4c938 Add configurable history and storage retention policies 2026-10-05 22:48:04 -04:00
josh 030794635e Coalesce fleet refreshes and suppress unchanged sweep events 2026-10-05 15:29:03 -04:00
josh e6912ceb57 Add configurable retention for successful routine operations 2026-10-05 13:09:44 -04:00
josh 981fb27c1c Preserve deleted server history
Retain historical server identities, results, and evidence after deletion. Mark archived entries as Deleted and prevent old work from targeting servers that reuse the same name.

Fixes #16.
2026-09-27 21:48:55 -04:00
josh 4fe8613678 Move fleet retention and concurrency controls into Settings
Persist settings in SQLite, import legacy environment values once, and apply changes to cleanup and deployment queues without restarting.

Closes #51
2026-09-27 16:11:45 -04:00
josh b9f5902945 feat: add Watchdog recovery and drift rebuild workflows 2026-09-27 15:13:38 -04:00
josh df265d1900 refactor: make provisioning own instance lifecycle operations 2026-09-27 14:45:00 -04:00
josh 8e04a190b3 Roll deployment restarts through a shared fleet-wide recovery limit 2026-09-17 11:25:06 -04:00
joshandClaude Opus 4.7 11e47b2a3f Widen Komodo health probe: try container endpoints, snippet on failure
GetDeployment returns config, not runtime state, so the health check was
always coming back with 'no container state found in response'. For
deployment-type servers the probe now tries GetDeploymentContainer,
InspectDeploymentContainer, and GetDeploymentStatus first, falling back
to GetDeployment only if those aren't available. The extractor also
recognizes more shapes (Health.Status, container_state, deployment_state,
etc.) and accepts more healthy strings (running, healthy, ok, started,
up).

When nothing matches, the truncated raw response snippet rides along in
the error field so the deployment/beta UI shows exactly which field to
teach the extractor next time.

Currently-stuck deployment_server rows are still 'pending', so the sweep
will retry them on its next 15s tick — no manual reset needed. Once one
of the widened endpoints resolves, those rows advance to 'done'
automatically.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 13:51:04 -04:00
joshandClaude Opus 4.7 edfe4dfc09 Add drift detection and removal as a real deployment
A drift deployment scans each server's sp/ tree for files not in the
server_scripts inventory (untracked scripts left over from before EC was
managing the fleet), deletes them, and records the removed bytes in
deployment_files for audit. The existing sweep then runs the standard
wait-for-empty → Komodo restart cycle on each affected server.

To make that cycle safer, the sweep now has a second phase: after Komodo
restart succeeds we wait for komodoInspectHealthy + RCON to both come
back green before marking the deployment_server row 'done'. If either
stays broken the row stays pending and keeps retrying.

New: /api/drift (GET scan, POST run), drift panel on /deployments.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-09-12 13:44:52 -04:00
josh 854f5532a9 Replace IW4MAdmin with direct RCON for player counts
IW4M kept hitting a network wall (container -> host LAN IP -> IW4M
loopback bind) that we couldn't cleanly punch through, and the two-minute
!rt token added enough operational friction that the pause-all-deployments
gate never got a fair test. RCON gives us the same fact — how many
players are on a server right now — direct from each game server over
UDP, with no session, no token, no proxy.

- Add integrations/rcon.ts: dgram UDP client. rconGetPlayerCount() and
  rconGetStatus() parse both `status` (print + scoreboard) and
  `statusResponse` (info line + rows) dialects. Timeout via
  RCON_TIMEOUT_MS.
- Add komodoResolveGamePort() + extractUdpPort(). Reads Komodo's read
  API for a server, walks both docker-compose-style port objects and
  docker-run-style "host:container/udp" strings, caches into
  servers.game_port on first success.
- Replace pipelineReady() guard in sweep with a simple
  integrations-configured check. Same held/failed/manual outcomes,
  but keyed on RCON+Komodo instead of IW4M+auth-state.
- Worker's status refresh fans out per-server over RCON at concurrency=4
  so a 15-server fleet doesn't spike UDP.
- Schema: drop iw4m_match/iw4m_value, add game_port + rcon_host.
- Delete every auth artifact: TokenModal, auth store, auth-state module,
  /api/auth/token, /api/events/auth, iw4m.test.ts. Layout no longer
  mounts the modal, pipeline store no longer tracks a frozen state.
- README + .env.example + docker-compose.yml + komodo/stack.toml swap
  IW4M_* env for RCON_PASSWORD / RCON_HOST / RCON_TIMEOUT_MS. Same
  host.docker.internal:host-gateway extra_hosts entry, repurposed
  for RCON reach.
- Tests: add rcon.test.ts and komodo.test.ts, update sweep.test.ts to
  drop the auth-freeze case. 23/23 passing.
2026-09-12 01:33:06 -04:00
josh d99711563e Initial commit — SvelteKit fleet manager for T5 Plutonium servers
Ports push_scripts.sh (1497 lines of bash) to a SvelteKit + SQLite web app
that manages a fleet of T5 Plutonium servers running on Komodo. Same push
and beta workflow as the shell script, with a live SSE pipeline view, a
!rt-token-gated auth modal that hard-pauses deployments on stale tokens,
per-file audit diffs, and a Komodo-container deployment target.

- Push/beta engines port the exact filter, held-by-beta, conflict, and
  stale-backup rules from the shell script
- SQLite (better-sqlite3) replaces script_metadata.json; a one-shot
  importer moves legacy state over on first boot
- 16 vitest tests cover push add/update/skip, beta open→rollback,
  open→promote, conflict guards, sweep frozen/manual branches, and IW4M
  login/status parsing
- docker/ + komodo/stack.toml deploy as a managed container on the same
  host as the game servers, /root/zombies bind-mounted at the same path
  the shell script used
2026-09-12 00:23:51 -04:00