electric-cherry/docs/beta-test-plans.md

Beta test plans

Create a plan in Betas, choose its servers, and deploy. The server's game script reports gameplay to Electric Cherry. The same reports update Fleet and any active beta on that server.

Game reporter protocol

Set EC_TELEMETRY_TOKEN on the Electric Cherry controller to a dedicated random secret of at least 32 characters (for example, generate 64 hexadecimal characters with openssl rand -hex 32). Configure the same secret in each game reporter or its trusted local relay. Use a production HTTPS URL or a reverse proxy that terminates TLS; browser login cookies do not authorize game telemetry.

Send JSON to POST /api/telemetry with Content-Type: application/json and either Authorization: Bearer <EC_TELEMETRY_TOKEN> or X-EC-Telemetry-Token: <EC_TELEMETRY_TOKEN>. If both headers are present, they must contain the same secret. Keep the secret out of URLs, payloads and browser code, and use a different secret from host-agent credentials or account passwords. The JSON event format is unchanged. There are four events. Send them whether or not a beta is active.

Event	Fields
game_start	server, map, players with each player's name and guid
round_change	server, map, round, players with each player's name, guid, kills, headshots, downs, revives, and score
heartbeat	Same fields as round_change; sent periodically during a round to refresh the current round and player stats
game_end	server, map, round, players with each player's name, guid, kills, headshots, downs, revives, and score

server is the exact registered Electric Cherry server name. map is the current map. players is the full player array, including an empty array when there are no players. Names and GUIDs are strings; keep GUIDs as strings to preserve their precision. Round and stat values are nonnegative JSON integers. Each GUID must appear only once within a report's player array.

Game start

Send when the game starts. Round and stats are not required.

{
  "event": "game_start",
  "server": "t6_zombie_town_1",
  "map": "zm_transit",
  "players": [
    { "name": "Josh", "guid": "12345678901234567" },
    { "name": "Alex", "guid": "23456789012345678" }
  ]
}

Round change

Send the new round and each player's current stats.

{
  "event": "round_change",
  "server": "t6_zombie_town_1",
  "map": "zm_transit",
  "round": 25,
  "players": [
    { "name": "Josh", "guid": "12345678901234567", "kills": 420, "headshots": 160, "downs": 1, "revives": 3, "score": 32000 },
    { "name": "Alex", "guid": "23456789012345678", "kills": 380, "headshots": 140, "downs": 3, "revives": 1, "score": 28000 }
  ]
}

Game end

Send the final round and each player's final stats when the game ends.

{
  "event": "game_end",
  "server": "t6_zombie_town_1",
  "map": "zm_transit",
  "round": 26,
  "players": [
    { "name": "Josh", "guid": "12345678901234567", "kills": 445, "headshots": 170, "downs": 2, "revives": 3, "score": 33500 },
    { "name": "Alex", "guid": "23456789012345678", "kills": 401, "headshots": 146, "downs": 4, "revives": 1, "score": 29750 }
  ]
}

Success returns HTTP 200 with { "ok": true, "telemetry": { ... } }. The returned telemetry contains the event and an Electric Cherry receivedAt timestamp. A missing/invalid machine token returns 401; an unset or invalid controller EC_TELEMETRY_TOKEN returns 503. Authentication happens before payload validation. Authenticated invalid JSON or fields return 400, an unknown/destroyed server 404, and a non-JSON content type 415.

The script sends each event once, in gameplay order, directly to Electric Cherry. There is no discovery request, beta token, boot/session ID, sequence number, custom feature event, external helper, or separate beta acknowledgement. Basic payload checks happen in the receiver.

Electric Cherry saves the latest complete report, including player stats, and streams it to connected clients. A new start clears the previous round display; an end closes the match. Receipt order determines the latest report. This small protocol does not attempt to identify delayed events from earlier games or provide exactly-once delivery across games.

Reports also feed the persistent Leaderboard, independently of beta activity. That page documents cumulative stat accounting, round credit, and the limits of identifying matches from this protocol.

The game scripts are uploaded artifacts, not source maintained in this repository. Their reporter should emit the four payloads above and add the authentication header. Existing anonymous reporters must be updated before telemetry is accepted; if a reporter cannot set HTTP headers, configure a trusted host-local relay to add the header. This repository does not bundle that relay or assume a particular GSC HTTP implementation. Rotating the shared secret requires updating the controller environment and all reporters/relays together.

Telemetry diagnostics

Open a server's Diagnostics → Telemetry API requests to inspect incoming POST /api/telemetry calls whose top-level server exactly matches its registered name. Each entry includes the Electric Cherry receipt time, HTTP result, error reason, event, map, round, player count, content type, and submitted JSON. Rejected reports are included, even when the content type is wrong but the body is readable JSON after authentication. Unauthorized requests are rejected before their bodies are read and are not retained in per-server diagnostics. Authentication headers and their secrets are never stored. Processing failures return 500 and record the internal error when diagnostic storage is available; failed database transactions do not update the scoreboard.

The panel refreshes every five seconds while visible and has a manual refresh button. History is stored in SQLite starting with this version, with up to 200 requests per server from the last seven days and a fleet-wide cap of 10,000 requests. Old entries are pruned when new requests arrive, and expired entries are excluded from reads. Payload previews are limited to 16,384 characters and marked when truncated. Request headers other than content type are not stored. GET /api/servers/{name}/telemetry-requests returns the same recent history, newest first, with caching disabled.

• No entries: no attributable requests are retained for this exact name. Check the reporter's destination and server field. Calls that never reach Electric Cherry, malformed JSON, and bodies without a usable server name cannot appear in this server's log. Earlier requests are not backfilled. Missing/invalid telemetry credentials also produce no retained request; inspect the reporter's HTTP result for 401 or an unconfigured-controller 503.
• 400 / 415: inspect the error and payload. One invalid player's stats or a duplicate GUID rejects the entire report. These requests do not update the scoreboard.
• 404: the reported server was not registered or had been destroyed at the time of the request.
• 200: Electric Cherry processed the report. An empty players array produces an empty scoreboard, including when sent with game_end. If a populated accepted report is missing from the scoreboard, investigate the live update/browser path.
• 500: Electric Cherry failed to process the request. Inspect the recorded error and the application log.

RCON's online status and player count are separate from gameplay telemetry, so an occupied server can still have no scoreboard report.

Requirements

Requirements on a qualifying server are ANDed:

Requirement	Evidence
Reach round X	A round change, heartbeat, or game end reports round >= X with the required player count
Complete qualifying matches	Game end at round 2 or later, optionally qualified by a higher round, player count, and duration
Populated minutes	Time between consecutive game reports with enough players at both ends
Player-hours	Populated minutes multiplied by the lower reported player count, divided by 60
Reach player count	Player array length in a game report
Sustained population	Consecutive report intervals above the player threshold within one game
Continuous healthy minutes	Consecutive fresh container/RCON observations
Maximum health outages	Transitions from observed healthy to unhealthy
Successful recovery cycles	Repeat the other requirements after each additional empty-server restart
Tester checklist	Tester name and observation entered on the beta detail page

All gameplay requirements use the script reports. Match counts do not read game logs. The reported round and required player count must occur together in the same game; observations from different games cannot combine to qualify a match. The final game-end report can supply that qualifying round. Consecutive duplicate game ends do not add another completion.

Match duration requires a game start and uses receipt times. Population time advances only when another report arrives in the same game; it uses the lower player count at the interval's endpoints. There is no heartbeat requirement or short interval limit between rounds. These are estimates between reported snapshots, not continuous measurements of player presence.

Health checks and safe restart gates use the existing container/RCON monitoring. Health intervals spanning more than 60 seconds earn no healthy time.

Coverage, waves and outcomes

Coverage can require all selected servers, a quorum, one per selected map, one per selected game, or one per selected game/map combination. Coverage metadata is pinned when the beta opens.

Set wave size to 1 for a canary, a larger number for batches, or 0 for all selected servers. Later waves receive files after the previous waves pass. Health-outage budget violations on any tested server block promotion.

The test window starts when the beta opens or a revision is submitted, including time waiting for empty servers. Missing evidence at the deadline produces inconclusive. Health monitoring continues after a pass until promotion.

For failures, select hold as failed, pause for investigation, or automatic rollback. Rollback queues restoration only for files touched by the beta. Restoration and restart both wait for an empty server and restart capacity. The manual option holds those changes in a rollback campaign until you select Resume when empty. Promotion is always manual.

Confirming promotion creates a campaign without reading the existing files on additional servers. Each server waits for an empty lobby and restart capacity, then captures rollback snapshots immediately before applying its queued files and restarting. Campaign history records snapshot capture, and file comparisons become available once those snapshots are confirmed. Tested servers retain their original pre-beta snapshots; their composed configuration is still checked at promotion to determine whether another activation is needed.

Withdrawing changes that have not activated needs no snapshot and leaves those servers' files untouched. If activation was submitted but its snapshot has not been confirmed, reconcile the provisioning operation before rolling back.

Pause holds evaluation and wave advancement. Resume retains completed progress and the current revision's deadline, but starts new timing intervals. A fresh attempt resets progress and restarts deployed servers when empty. Recovery cycles repeat gameplay milestones and checklist sign-offs while carrying health outages forward.

Plans, progress, and evidence persist in SQLite. The detail page shows the most recent 200 evidence entries. On upgrading from the old reporter protocol, active progress resets. Unsupported checks in existing plans and saved presets become manual review items; they cannot silently pass.

Updating an active beta

Choose Update beta on the beta detail page, upload replacements for existing bundle files, and review the changes and selected servers before choosing Deploy revision. Text files show a diff when both files are small enough; binary and large files show their checksums. Files must actually change.

The beta keeps its ID, servers, scope and test policy. Each revision creates new file versions, clears current test progress and checklist sign-offs, repeats the configured waves, and starts a fresh testing window at submission. Files install and servers restart through the normal empty-server and capacity gates. Promotion requires the new revision to pass. Pause and retry retain the current revision's deadline; they do not create a revision.

Revision history retains downloadable bundle files and previous test results, including the last 200 evidence entries shown at replacement. All underlying attempts and evidence remain in SQLite, separated by revision. Rejecting the beta restores the original pre-beta files, even after multiple revisions or while a new revision is waiting to deploy. Files originally absent are removed.

Updates are available until promotion or rollback, including paused, failed and expired tests. An activation already in progress must settle first; interrupted file writes and deleted participants must be resolved before an update. Pending, unsubmitted restarts are superseded. Adding/removing files, changing targets or changing the policy still requires a new beta.

API and storage

POST /api/betas accepts testPlan. GET /api/betas/:id includes the plan, outcome, per-server progress, and recent evidence under test.

POST /api/betas/:id/test accepts pause, resume, retry, or sign-off (with server, requirementId, and note). GET/POST /api/betas/presets lists/saves reusable plans. Saved presets are copied into a beta; later edits do not change an active plan.

POST /api/betas/:id/revisions accepts multipart expectedRevision, optional notes, and replacement uploads named file:<scriptId>. A stale revision is rejected. GET /api/betas/:id includes the current revision and revisions history; GET /api/betas/:id/revisions/:revision/files/:scriptId downloads an archived file (or the original pinned file before its first update).

Internal progress records support attempts and recovery cycles. The reporter does not know about or supply these records. Legacy betas without a test plan also use game_end at round 2 or later for their match-completion gate.