feat: add Watchdog recovery and drift rebuild workflows
This commit is contained in:
@@ -81,7 +81,7 @@ The schedule and jobs survive app restarts. Missed days become one run, and a
|
||||
server can have only one pending maintenance restart. Disabling the schedule
|
||||
does not cancel work already queued; use **Cancel** for those jobs.
|
||||
|
||||
Maintenance waits for a fresh zero-player RCON response and for existing
|
||||
Maintenance waits for a fresh zero-player RCON or telemetry report and for existing
|
||||
deployment/beta work, then uses the shared rolling restart limit. Unreachable or
|
||||
occupied servers remain queued. After container and RCON recovery, Electric
|
||||
Cherry queries the running revision again. If a fresh published revision was
|
||||
@@ -100,6 +100,13 @@ behavior on your live T5/T6 servers before relying on unattended maintenance.
|
||||
|
||||
## How restarts stay safe
|
||||
|
||||
**Operations → Watchdog** automatically detects unhealthy servers and file drift.
|
||||
Health incidents receive one restart followed by one rebuild if recovery fails;
|
||||
drift queues a rebuild with the promoted setup and a linked Campaign. Incidents,
|
||||
evidence, and attempts survive application restarts. Pause automation, inspect
|
||||
failures, and explicitly retry from Watchdog. Intentional stops and active beta or
|
||||
campaign work are respected. See [Watchdog](docs/watchdog.md) for timing and recovery behavior.
|
||||
|
||||
Betas support configurable gameplay, participation, stability and manual-review
|
||||
requirements, saved presets, coverage rules and staged waves. See
|
||||
[Beta test plans](docs/beta-test-plans.md) for usage and the four-event game telemetry contract.
|
||||
@@ -300,8 +307,10 @@ the full player data, updates Fleet, and applies gameplay progress to active
|
||||
betas automatically. No discovery request or beta-specific fields are needed.
|
||||
|
||||
See [Game reporter protocol](docs/beta-test-plans.md#game-reporter-protocol) for
|
||||
complete payload examples. Telemetry does not replace RCON's empty-server
|
||||
check before an automated restart.
|
||||
complete payload examples. Fresh telemetry can verify player count and game
|
||||
activity alongside RCON during provisioning and Watchdog recovery. Occupied
|
||||
reports prevent interruptions; stale reports cannot authorize them. Komodo still
|
||||
verifies container readiness.
|
||||
|
||||
For a missing scoreboard, open the server's **Diagnostics → Telemetry API requests**.
|
||||
The log shows received requests for that exact server name, including accepted
|
||||
|
||||
Reference in New Issue
Block a user