josh/electric-cherry
1
0
Fork 0
Code Issues 18 Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add Watchdog recovery and drift rebuild workflows

Browse Source
This commit is contained in:
josh
2026-09-27 15:13:38 -04:00
parent a7044f21a1
commit b9f5902945
36 changed files with 2131 additions and 424 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/provisioning.md
+13 -3
View File
@@ -30,12 +30,16 @@ failures before making another request.
Start and Restart share the rolling restart limit with campaigns, betas, and
maintenance. A successful API reply is followed by a recovery delay and actual
container/RCON checks. Failed or ambiguous restart requests retain their recovery
container and game-status checks. Failed or ambiguous restart requests retain their recovery
slot and are not automatically replayed. Stop requires an explicit stopped container state; an
unreachable Komodo API is not evidence that a stop succeeded.
Disruptive operations require a fresh zero-player RCON result, unless Komodo
positively reports a stopped container. Runtime actions wait for conflicting
Disruptive operations require a fresh zero-player result from RCON or accepted
telemetry, unless Komodo positively reports a stopped container. Fresh occupied
telemetry vetoes a conflicting empty RCON reading. Reports must be received within
45 seconds; recovery verification also excludes reports from before the submitted
operation. Telemetry can verify the game's activity and selected map after Komodo
reports healthy. Runtime actions wait for conflicting
campaign, beta, maintenance, or provisioning work. Duplicate pending requests for
the same action return the existing operation; a different action returns a
conflict. Requests and events survive application restarts.
@@ -48,6 +52,12 @@ health verification. The callers retain file deployment, beta evidence, and
Plutonium revision verification. Jobs remain visible while waiting for players,
other work, or integration configuration.
[Watchdog](watchdog.md) also submits recovery operations and links each attempt
to its incident. A failed Watchdog restart can hand its reserved slot to a rebuild
after five minutes; unrelated recovery slots remain held. Drift campaigns follow
their linked rebuild without requesting another restart. Their file evidence and
history remain available after the instance is removed.
Campaign changes queued before submission can share one restart. Changes arriving
after submission require a new operation. A failed or interrupted request keeps
its original ID and never automatically replays. An explicit beta retry starts a