diff --git a/docs/beta-test-plans.md b/docs/beta-test-plans.md index 13df8db..531d1e7 100644 --- a/docs/beta-test-plans.md +++ b/docs/beta-test-plans.md @@ -79,60 +79,559 @@ recent 200 evidence entries; older entries remain in the database. ## Game reporter protocol -On **game-process startup**, fetch: +This section is a standalone integration guide for the T6 telemetry script. +It describes the **currently implemented API**, including its limitations. +Examples use `t6_zombie_town_1`; replace that with the exact, case-sensitive +Electric Cherry server registry name. The API base URL is the Electric Cherry +application, not Komodo or the game server's RCON address. + +### 1. Two reporting modes: Fleet display and beta evidence + +The reporter must work whether or not anybody is beta testing: + +| Situation | Reporter behavior | +| --- | --- | +| No eligible beta run for this server | Keep sending ordinary `round_change` reports for Fleet. Do not send beta-only events. | +| One eligible run discovered at process startup | Send the beta event stream with that run's `runId`. Scoped `round_change` reports also update Fleet. | +| Several eligible runs discovered at startup | Send a copy of each beta event to each run, with separate acknowledgement/queue state per run. | +| Beta opened while an existing game process is running | Continue its existing reporting mode. The beta waits for the server to empty and restarts it; the new process discovers the new run. | +| Beta has passed and is awaiting promotion | Continue reporting. A passed test still accepts evidence until promotion or another outcome stops it. | +| Beta paused, failed, expired, promoted, rolled back, or run superseded | Stop crediting events to that inactive run; continue ordinary round reporting. See the suspension rules below. | + +**No active beta is normal, not an error.** Fleet telemetry does not depend on a +beta, a run token, or a successful beta discovery request. + +### 2. Startup discovery and an empty result + +On **actual game-process startup**, request: ```http -GET /api/telemetry?server=t5_zombie_theater_1 +GET /api/telemetry?server=t6_zombie_town_1 ``` -The response contains `runs: [{ runId, betaId }]`. Cache that list for the process -lifetime, generate a unique `bootId`, and send each event once to every returned -run. A run ID is an evidence correlation token, not an authentication credential. -Do not refresh it during a match or relabel queued events with a newer run ID. -Each beta-controlled restart creates a new run ID before requesting the restart. -An unexpected process restart fetches the current run ID and uses a new boot ID. +For a registered server without an eligible run, this returns **HTTP 200**: -POST JSON to `/api/telemetry`: +```json +{ + "runs": [] +} +``` + +This can mean there is no beta, the server is not a beta target, its wave has not +started, the beta is waiting to restart it, or the test is not accepting evidence. +Legacy betas without a test plan are not returned either. The response is a list +of eligible **runs**, not a list of every beta in the application. + +With an eligible run, the response is **HTTP 200**: + +```json +{ + "runs": [ + { + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "betaId": 42 + } + ] +} +``` + +The response has `Cache-Control: no-store`. Do not assume it contains exactly one +entry, and do not choose only the first entry. `betaId` is informational; POST +requests use `runId`, not `betaId`, to identify the test attempt. + +A missing or unknown `server` query parameter returns **404**, not an empty +success. Treat that as a configuration issue. The GET handler currently only +checks registry existence; POST additionally rejects a server marked destroyed. +Do not interpret successful discovery as permission to report for another server. + +Cache the startup run list for this process lifetime. Electric Cherry creates a +new run before requesting each beta-controlled restart. That restart is the +boundary that makes gameplay belong to the newly deployed build. **Do not poll +for and adopt new run IDs in an old process, at a map reload, or mid-match.** +Never move queued events to a newer run ID. + +If discovery fails, use bounded retries with backoff while still initializing, +before gameplay starts. If startup discovery cannot finish, run in Fleet-only +mode for that process and log the problem. Do not attach later gameplay to a run +discovered after that startup boundary. A new process/retry is needed for a new +safe beta attachment; the current API has no late-attachment handshake. + +### 3. Reporting without a beta + +Send this on each round change, even when `runs` is empty: + +```http +POST /api/telemetry +Content-Type: application/json +``` ```json { "event": "round_change", - "server": "t5_zombie_theater_1", - "runId": "the-run-id-from-startup", - "eventId": "boot-uuid:session-uuid:12", - "bootId": "boot-uuid", - "sessionId": "session-uuid", - "sequence": 12, + "server": "t6_zombie_town_1", + "round": 17, + "players": 4 +} +``` + +Example success response: + +```json +{ + "ok": true, + "telemetry": { + "server": "t6_zombie_town_1", + "round": 17, + "players": 4, + "receivedAt": "2026-09-18T12:00:00.000Z" + } +} +``` + +Omit `runId` entirely in this mode. `"runId": null` and `"runId": ""` are +invalid; they are not equivalent to omitting the field. The response has no +`test` property when `runId` was omitted. + +Without `runId`, only `round_change` is supported. Do not send unscoped `boot`, +`session_start`, `feature`, or `match_end` requests: they return 400. Do not buffer +feature occurrences from normal operation to submit when a future beta opens. + +Electric Cherry stores only the latest Fleet round report per server and stamps +its receipt time. Lower rounds are valid when a new match begins. These reports +do not change RCON player counts, restart safety decisions, or beta progress. + +### 4. Identifier lifetimes and the T6 process boundary + +| Field | Who creates it | Lifetime and rules | +| --- | --- | --- | +| `server` | Configuration | Exact registered name, unchanged across matches and process restarts. | +| `runId` | Electric Cherry | One server attempt/recovery phase. Use the value discovered at process startup. Treat it as opaque. | +| `bootId` | Reporter | Unique for each actual game-process lifetime. Keep it across matches, map reloads, and reporter reinitialization within that process. | +| `sessionId` | Reporter | Unique for each actual played match. Generate a new value for the next match, even if the players and map are unchanged. | +| `eventId` | Reporter | Unique for each logical event within a run, including across sessions/boots. A delivery retry reuses the exact same ID. | +| `sequence` | Reporter | Start at 0 for `session_start`; increment for every subsequent event in that session, across all event types. Reset only for a new session. | + +Generate collision-resistant IDs, such as UUIDs. A boot ID plus a session ID and +counter is also suitable for event IDs if the combined string fits the length +limit. Example IDs below are illustrative; never hardcode them in the script. +The same event ID can be used for copies sent to different runs because +deduplication is scoped to `(runId, eventId)`. + +**A GSC script's initialization is not necessarily a game-process startup.** +The T6 implementation must establish where its state survives match/map changes. +If script state resets on each map load, preserve process identity, the startup +run list, and delivery state in a process-lifetime facility or a helper that can +reliably identify the game process. A helper's own restart must not invent a new +game boot. Do not assume map-local globals provide process-lifetime storage. +If that distinction is unavailable, leave beta reporting disabled rather than +reporting a fabricated boot; ordinary Fleet round reports still work. + +An unexpected game-process restart may discover the **same** run ID. Send a new +boot ID: the backend counts additional distinct boots in a run as unexpected +restarts. A planned recovery restart receives a **new** run ID and starts a new +evidence phase. Restarting only the Electric Cherry web app changes neither the +game boot nor the reporter's cached IDs. + +### 5. Common request fields + +All requests use JSON objects and `Content-Type: application/json`. Every +beta-scoped event includes these fields unless the table says otherwise: + +| Field | Type | Constraints | +| --- | --- | --- | +| `event` | string | Exactly `boot`, `session_start`, `round_change`, `feature`, or `match_end`. | +| `server` | string | Registered server name, 1-255 characters, case-sensitive. | +| `runId` | string | Required for beta evidence; 1-128 characters from `A-Z a-z 0-9 _ . : -`. | +| `eventId` | string | Same identifier character/length limits. | +| `bootId` | string | Same identifier character/length limits. | +| `sessionId` | string | Same identifier limits; required except for `boot`. Omit for `boot`. | +| `sequence` | number | Nonnegative safe integer. Use 0 for `boot` and `session_start`. | +| `round` | number | Nonnegative safe integer; the actual player-facing round. Use 0 for `boot`. | +| `players` | number | Nonnegative safe integer; actual participating player count at the event. Use 0 for `boot`. | +| `name` | string | Required only for `feature`; 1-120 characters from the identifier character set. | + +Numbers must be JSON numbers, not strings, fractions, `null`, or negative values. +The maximum safe integer is 9007199254740991. Sequence numbers need to increase; +the backend does not require them to be contiguous, but the sender should emit a +single ordered stream. Do not use a separate counter for each event type. + +The examples below show excerpts from one process and match. Earlier round +reports are omitted, hence the sequence jumps to 25. The real script should send +each round change and increment its counter for every event it produces. + +### 6. Full event examples + +#### Process boot + +Send once per discovered run at real game-process startup, before any session +events. A retry is the same request, not another boot with another ID. + +```json +{ + "event": "boot", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:boot", + "bootId": "boot-a81f", + "sequence": 0, + "round": 0, + "players": 0 +} +``` + +#### Start of a played match + +Send when actual gameplay begins with players, not merely when an empty map +loads. Retain this `sessionId` until that match ends or is abandoned. + +```json +{ + "event": "session_start", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:0", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 0, + "round": 1, + "players": 4 +} +``` + +After this is acknowledged, report the initial round with `round_change` too. +The `round` on `session_start` does not itself satisfy a round requirement. + +#### Reaching a round + +Send on entry to each round, with the player count sampled at that event. +There is no separate "round completed" event. + +```json +{ + "event": "round_change", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:25", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 25, "round": 25, "players": 4 } ``` -Send events in order: +This can satisfy a round-25 check requiring four players. It also records the +qualifying round for a later completed-match check. A `match_end` reporting round +25 does not replace a missing `round_change` event for that round. -1. `boot`: once per process, before all session events. `sessionId` can be - omitted. Send `sequence`, `round` and `players` as 0. -2. `session_start`: a new unique session ID, sequence 0, current players/round. - Send when actual gameplay begins, not when an idle map loads. -3. `round_change`: strictly increasing sequence within that session, reported - round and player count. Emit on entry to each round. -4. `feature`: same sequence ordering, current players/round, plus `name` such - as `perk.purchased`. Report one event per occurrence. -5. `match_end`: normal completion only, next sequence, final players/round. - Crashes, forced shutdowns and idle map rotations must not emit this event. +#### Feature: a successful perk purchase -IDs accept letters, numbers, `_`, `.`, `:`, and `-` (up to 128 characters). -Feature names are limited to 120 characters. Numbers must be nonnegative safe -integers. Retries reuse the same event ID and payload. Send sequentially: an -older sequence arriving late is rejected, not retroactively credited. +The top-level event is always `"feature"`. The custom feature identifier goes in +`name`; it is not a new top-level event type. -Responses include `test: { accepted, reason? }`. Duplicates return HTTP 200 with -`accepted: false`; malformed events return 400. Events for retired runs, unknown -boots/sessions, already-ended sessions, or paused/closed tests are not credited. -The existing unauthenticated/trusted-network deployment model is unchanged. +```json +{ + "event": "feature", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:26", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 26, + "round": 25, + "players": 4, + "name": "perk.purchased" +} +``` -Legacy payloads without `runId` continue to support `round_change` for Fleet -display. They do not alter RCON restart safety data or test evidence. +Emit when the purchase actually succeeds, not when a player opens a menu or +attempts an unaffordable purchase. One accepted event adds **one** to each +matching feature requirement whose minimum round/player qualifiers are met. +This event does not require waiting for the match to end. + +For example, configure **Feature event count** with event name `perk.purchased`, +target 10, minimum round 20, and minimum players 4. The example above contributes +one occurrence. A purchase at round 19 or with only three players can still be +accepted into the event stream, but contributes zero to that requirement. + +#### Feature: successful Pack-a-Punch completion + +```json +{ + "event": "feature", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:27", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 27, + "round": 25, + "players": 4, + "name": "weapon.pack_a_punch.completed" +} +``` + +#### Feature: successful execution of a custom mechanic + +```json +{ + "event": "feature", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:28", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 28, + "round": 25, + "players": 4, + "name": "mechanic.teleporter.completed" +} +``` + +These names are **suggested conventions**, not preinstalled T6 hooks. The script +must instrument the corresponding successful actions. Only emit a mechanic's +event on maps/builds where that mechanic actually exists. + +Names are matched exactly and case-sensitively against the UI's **Reported event +name**. A requirement named `perk.purchased` with target 10 counts ten accepted +occurrences of that exact name; `perk.*` is not a wildcard. + +For a specific perk, use a distinct name such as `perk.juggernog.purchased`. +The current API does not filter on a separate `perk`, `weapon`, or `playerId` +field. You may deliberately emit both a generic and a specific marker for one +action, but they must be separate events with separate event IDs and sequences; +each requirement still counts only its exact matching name. + +There is no bulk-count field. `"count": 10`, `value`, `metadata`, `timestamp`, +`duration`, and other extra properties are not consumed or preserved as beta +signal fields. Send one event for each real occurrence. Keep any additional +diagnostics in script logs until the API explicitly supports them. + +#### Normal match completion + +```json +{ + "event": "match_end", + "server": "t6_zombie_town_1", + "runId": "4d0d91f9-39d2-4793-a9a3-4054c2a91c88", + "eventId": "boot-a81f:match-b92c:29", + "bootId": "boot-a81f", + "sessionId": "match-b92c", + "sequence": 29, + "round": 25, + "players": 4 +} +``` + +Send once when a played match ends normally, including a normal game-over. +Capture the actual terminal player count; the example's four players are not a +value to fabricate if players have already left. The backend retains earlier +round/player observations for qualification. + +Do **not** send `match_end` for a crash, forced process shutdown, arbitrary map +rotation, or an idle map cycle. A generic script teardown callback alone is not +proof of normal match completion. There is no `session_abort` event at present; +abandon that session and use a new ID when actual gameplay next begins. Never +replay events for an abandoned session into the new one. + +Accepted `match_end` closes the session. Later events for that session are not +accepted. Round-qualified or multi-player-qualified match requirements use this +stream; unqualified completed-match requirements use game logs instead, so the +same match is not counted twice for one requirement. + +### 7. Responses: transport success is not evidence acceptance + +An accepted `boot`, `session_start`, `feature`, or `match_end` returns HTTP 200: + +```json +{ + "ok": true, + "test": { + "accepted": true + } +} +``` + +An accepted scoped `round_change` additionally updates the Fleet display: + +```json +{ + "ok": true, + "telemetry": { + "server": "t6_zombie_town_1", + "round": 25, + "players": 4, + "receivedAt": "2026-09-18T12:25:00.000Z" + }, + "test": { + "accepted": true + } +} +``` + +Acceptance means the event was recorded in the run's stream. It does not mean +the whole test passed, or even that a feature/round event met a configured +requirement's qualifiers. + +A retry of an already accepted event returns HTTP 200: + +```json +{ + "ok": true, + "test": { + "accepted": false, + "reason": "Duplicate event" + } +} +``` + +Treat that exact duplicate response as delivery already acknowledged. Do not +generate a new ID to force the same action to count twice. + +An inactive run returns HTTP 200: + +```json +{ + "ok": true, + "test": { + "accepted": false, + "reason": "Run is inactive or superseded; obtain the current run at process startup" + } +} +``` + +Invalid ordering or session identity also returns HTTP 200: + +```json +{ + "ok": true, + "test": { + "accepted": false, + "reason": "Duplicate, out-of-order, closed, or unknown session/boot" + } +} +``` + +That last response is **not** an acknowledgement: inspect the boot/session/ +sequence handling. The API currently provides explanatory strings, not distinct +machine-readable error codes for every rejection. Do not treat every +`accepted: false` as a harmless duplicate or assume every inactive run is closed +permanently; paused runs use the same inactive response. + +**Fleet updates are independent of beta acceptance.** A syntactically valid +scoped `round_change` still writes the latest Fleet round even when +`test.accepted` is false, including a duplicate or inactive-run rejection. Its +response contains both `telemetry` and `test`. Do not replay an old round backlog +after newer reports: it can move the Fleet display backwards. There is no +client-timestamp freshness check. + +| HTTP result | Meaning and handling | +| --- | --- | +| 200, no `test` | Ordinary Fleet report accepted. | +| 200, `test.accepted: true` | Beta event accepted; advance that run's queue. | +| 200, exact `Duplicate event` reason | Already accepted earlier; advance that run's queue. | +| 200, inactive-run reason | Suspend beta delivery for that run; continue Fleet reporting. | +| 200, session/order rejection | Log an integration error and stop crediting that broken session. Do not invent a replacement session for an in-progress match. | +| 400 | Invalid JSON, fields, or event type; fix the payload instead of retrying unchanged forever. | +| 404 on POST | Server is unregistered or destroyed; check configuration/lifecycle. | +| 415 | Missing or incorrect JSON content type. | +| Network error, timeout, or 5xx | Delivery is uncertain; retry the same request with backoff within a bounded queue. | + +### 8. Ordering, retries, pauses, and multiple runs + +**Recommended sender behavior** (client implementation guidance, not extra API +features): + +1. Keep delivery asynchronous so a web-service outage never blocks the game. +2. Send `boot` first for each run and wait for acceptance or an acknowledged + duplicate before delivering its `session_start`. +3. Serialize the remaining events per run. Do not allow a later event to overtake + an uncertain request. Use the same session counter for rounds, features, and + match completion. +4. On timeout, retry with exactly the same event ID, sequence, and payload. The + first request may have committed even if its response was lost. +5. Bound queue size, retry count, and event age. For example, back off through + 1, 2, 5, and 10 seconds rather than retrying every frame. These intervals are + a suggested client policy, not server-enforced values. +6. If a long outage makes the queue stale, abandon beta evidence for that session + and keep the current Fleet round visible. Do not replay historical feature or + completion events hours later. Resume beta evidence at a later real match + only if the same startup-pinned run is still eligible and boot delivery is + established. +7. Keep acknowledgement state separately for every discovered run. Failure on + one run does not change another run's IDs or acknowledged sequence. Reuse the + logical event ID across copies, changing only `runId`. + +The server stamps **receipt time**, not game event time. In particular, match +duration is measured between accepted `session_start` and `match_end` receipts. +Delayed replay can distort duration and no client-provided timestamp fixes it. +Keep beta events near real time; do not use them as a historical-import API. + +While scoped round delivery is healthy, no extra unscoped copy is necessary. +If beta delivery is suspended or lagging, ordinary round reports can keep Fleet +current. Retire the affected session queue rather than later sending its old +rounds behind newer unscoped reports. A simple implementation may serialize +delivery across all run copies to avoid cross-run round reordering. + +For paused/inactive runs, the simplest safe policy is Fleet-only reporting until +the next actual process startup. If automatic resume within the same process is +desired, the reporter may recheck **only IDs already pinned at startup**: + +- Intersect discovery results with the original startup list. Never adopt an + unfamiliar run ID returned by a later GET. +- When a cached run becomes eligible again, resume at the **next real match**, + using the existing boot ID and a new session ID. Establish boot delivery first; + reusing the original boot request/event ID is safe if its acknowledgement was + lost. +- Discard events generated during suspension. Do not manufacture a new session + start in the middle of an existing match or replay paused gameplay. +- If startup returned `runs: []`, there are no pinned IDs to resume. Stay + Fleet-only until a real process restart. Similarly, a process started while a + test was paused may require an operator-requested fresh attempt after resume. + +There is no polling interval or resume notification supplied by this API. The +reporter owns its retry/suspension policy. The rule about startup pinning must +remain true regardless of that policy. + +### 9. T6 implementation handoff checklist + +- Configure the Electric Cherry URL and exact server registry name. +- Preserve ordinary round reports when no beta is active or the service is + temporarily unreachable; never block gameplay waiting for HTTP. +- Identify actual game-process boots separately from GSC initialization and + map/match resets. Cache discovery results for the game process lifetime. +- Instrument real session start, round entry, normal game-over, and successful + feature actions. Confirm each chosen T6 callback's semantics; do not infer + normal completion from generic teardown. +- Use one sequence counter per session, unique event IDs, stable retry payloads, + and an ordered bounded queue with separate acknowledgements per run. +- Decide and document the feature names the script emits so beta creators can + enter those exact names. Extra JSON metadata cannot be used as a qualifier. +- Keep beta-only events scoped to startup-pinned runs. Do not backfill pre-beta, + paused, abandoned, or previous-process gameplay into a new run. +- Handle `ok` and `test.accepted` separately. Log rejections with enough context + to diagnose the run, boot, session, event ID, and sequence. + +Suggested verification cases for the telemetry-script session: + +| Case | Expected result | +| --- | --- | +| Startup returns an empty run list | Round reports continue; no beta-only POSTs. | +| Beta opens during an ordinary match | No new token adopted; beta reporting begins after the beta-controlled process restart. | +| Normal new match, same process | Same boot/run IDs; new session ID and sequence starting at 0. | +| Actual unexpected process restart | New boot ID; rediscover runs; no old queued session events replayed. | +| Retry after a lost response | Same event ID and payload; one evidence entry and a duplicate acknowledgement. | +| Round 25 with four players, then normal match end | Round evidence can pass immediately; a qualifying-match count advances only at normal completion. | +| Ten successful `perk.purchased` events | Ten uniquely identified feature events, counted only when each event meets the configured round/player qualifiers. | +| Crash or idle map reset | No fabricated `match_end`. | +| Paused, closed, or superseded run | Beta delivery suspended; Fleet remains current; never switch old queued events to a new token. | +| Two startup run IDs | Each receives the event once, with independent acknowledgements. | +| Reporter script reload, same game process | No new boot identity or fresh adoption of run IDs. | + +The endpoint retains the application's existing trusted-network model. A `runId` +correlates evidence; it is **not** an authentication secret or proof of loaded +artifact bytes. Correct process/session attribution depends on the reporter +following the lifecycle rules above. ## API and storage