From 26bb9e158def68c8d8109d7f60f87a3f52ba394f Mon Sep 17 00:00:00 2001 From: josh Date: Thu, 24 Sep 2026 08:12:25 -0400 Subject: [PATCH] T6: expand telemetry to T5-parity reports and add .join module - Telemetry: async queued game_start/heartbeat/round_change/game_end with sequenced FIFO, watchdog, log-level dvar, and per-player stats snapshot - Join: new tws_t6_join.gsc chat command lets late spectators enter mid-match - Update NOTES/README/check.ps1 to reflect the six-module layout --- t6/NOTES.md | 3 +- t6/README.md | 45 +++-- t6/tools/check.ps1 | 4 +- t6/tws_t6_join.gsc | 116 +++++++++++ t6/tws_t6_telemetry.gsc | 438 ++++++++++++++++++++++++++++++++-------- 5 files changed, 507 insertions(+), 99 deletions(-) create mode 100644 t6/tws_t6_join.gsc diff --git a/t6/NOTES.md b/t6/NOTES.md index 30d0e2d..10c7279 100644 --- a/t6/NOTES.md +++ b/t6/NOTES.md @@ -15,7 +15,8 @@ config have been removed from this project. | Enemy count | Stock `_zm_utility::get_current_zombie_count()` plus queued zombies | Uses the map's zombie team and enemy filters | | Timers | Seed elapsed time for late joiners; stop round watcher/reset on game over | T6 timer behavior; keep frozen timers frozen | | AFK | Maintain control freeze, disable guns/grenades, guard player state and stale grace threads | Requested movement/shooting fix; retain `.afk` / `.back` and two-second grace | -| Telemetry | Global `httpPost(url, options)`; no unnecessary common dependency | Avoid raw-loader `http.gsc` and obsolete common-path lookup errors | +| Telemetry | Global `httpPost(url, options)`; T5-parity queue + heartbeat + game_start/end + watchdog + log-level dvar; per-player `guid` sources from Plutonium's `player.guid` engine property (T5 uses `player getGuid()` method); per-match `kills`/`headshots`/`downs`/`revives` fall back to `0` when `player.stats[]` is unpopulated | T6 has no `getGuid` method but exposes the persistent identity as `.guid` on the player entity; `_zm_stats` writes only into persistent stat storage, so `player.stats[]` is not a stock T6 field. Payload keys match T5 for downstream shape parity | +| Join | Port of T5 `tws_join.gsc`; uses `level.spawnplayer` / `level.custom_spawnplayer` (lowercase) and `maps\mp\zombies\_zm::spectator_respawn`; readiness gated on `self.sessionstate == "spectator"` instead of T5's `self.initialized` | T6 stock spectator respawn hook lives in `_zm.gsc`; `self.initialized` is a T5 SP zombiemode marker that T6 stock does not populate the same way | Common retains the T5 HUD constants, helpers and first-spawn dispatcher. HUD again owns the `hud_game_over` notification fan-out. Lock again waits on HUD diff --git a/t6/README.md b/t6/README.md index ffc8f44..77e6495 100644 --- a/t6/README.md +++ b/t6/README.md @@ -9,11 +9,12 @@ just like T5. There is no separate TWS config file or database setup. | `tws_t6_hud.gsc` | Match/round timers, name, zombie count and health bar | Common | | `tws_t6_afk.gsc` | `.afk` / `.back`, protection, movement and weapon blocking | Common | | `tws_t6_lock.gsc` | Lock at round 20; clear password on game over | Common, HUD | -| `tws_t6_telemetry.gsc` | HTTP JSON round updates, enabled by default | t6-gsc-utils | +| `tws_t6_telemetry.gsc` | Async queued game reports (game_start/heartbeat/round_change/game_end) | t6-gsc-utils | +| `tws_t6_join.gsc` | `.join` chat command lets a spectator newcomer enter mid-match once | Common | ## Install -1. Copy these five `tws_t6_*.gsc` files into `Plutonium/storage/t6/scripts/zm/`. +1. Copy these six `tws_t6_*.gsc` files into `Plutonium/storage/t6/scripts/zm/`. 2. For telemetry, install [t6-gsc-utils v3.1.1](https://github.com/alicealys/t6-gsc-utils/releases/tag/v3.1.1) in `Plutonium/plugins/` and restart the server process. The other four scripts do not require that plugin. @@ -60,21 +61,41 @@ set tws_server_id "t6_zombie_town_1" set tws_telemetry_url "http://192.168.1.199:3000/api/telemetry" ``` -It sends the first occupied positive round and subsequent round changes: +It sends four event types over the life of a match, mirroring the T5 script: -```json -{"event":"round_change","server":"t6_zombie_town_1","round":17,"players":4} -``` +| Event | When | Round in payload | +| --- | --- | --- | +| `game_start` | First playable spawn | omitted | +| `heartbeat` | Every 30s while the match is in progress | current round | +| `round_change` | Every new positive round | new round | +| `game_end` | `end_game` / `game_ended` / `intermission` | final round | -Delivery is asynchronous, with one in-flight request and one replaceable pending -observation. Transport failures retry at most three times (1/2/5 seconds), retiring -snapshots older than 15 seconds or superseded by a newer round. Transport success -does not establish API acceptance. These reports update Fleet; they do not send -beta boot, session, feature or match-completion evidence. +Each event payload includes `event`, `server`, `map`, `round` (except `game_start`) +and a `players` array with `name`, `guid`, `kills`, `headshots`, `downs`, `revives` +and `score` per player. `guid` reads Plutonium's `player.guid` engine property +(T5 uses the `player getGuid()` method — T6 has no such method, but the field is +equivalent for identity purposes). `kills`/`headshots`/`downs`/`revives` read +from `player.stats[]` if a mod populates it, otherwise `0`. + +Delivery is asynchronous with a sequenced FIFO queue and one in-flight request. +The queue survives the game-end signal so the final snapshot drains. There is no +automatic retry: the receiver may already have accepted the event. Transport +success reports the HTTP call finished, not that the API accepted it. Increase +`tws_telemetry_log_level` to `2` for per-event capture/queue/response logging; +set to `0` to silence non-error output. + +## `.join` for newcomers + +Late connectors who never got a spawn can type `.join` in chat to leave spectator +mode before the round ends, once per connection. `.join` is rejected if the player +has already played, if the game is not running, if no active player is alive, or +during intermission. The script hooks the stock `spectator_respawn` path via +`level.custom_spawnplayer`; it does not skip the map-authored spawn callback if +another mod installed one first. ## Validation -All five scripts compile for T6 PC with gsc-tool 1.4.10. To repeat: +All six scripts compile for T6 PC with gsc-tool 1.4.10. To repeat: ```powershell .\tools\check.ps1 -CompilerPath C:\path\to\gsc-tool.exe diff --git a/t6/tools/check.ps1 b/t6/tools/check.ps1 index c08f8a6..67dd161 100644 --- a/t6/tools/check.ps1 +++ b/t6/tools/check.ps1 @@ -6,7 +6,7 @@ $ErrorActionPreference = 'Stop' $compiler = (Resolve-Path -LiteralPath $CompilerPath).Path $repo = Split-Path -Parent $PSScriptRoot $files = @(Get-ChildItem -LiteralPath $repo -Filter 'tws_t6_*.gsc' | Sort-Object Name) -if ($files.Count -ne 5) { throw 'Expected the five TWS modules: common, HUD, AFK, lock, telemetry.' } +if ($files.Count -ne 6) { throw 'Expected the six TWS modules: common, HUD, AFK, lock, telemetry, join.' } $common = Get-Content -LiteralPath (Join-Path $repo 'tws_t6_common.gsc') -Raw $functions = @([regex]::Matches($common, '(?m)^([a-zA-Z_][a-zA-Z_0-9]*)\([^\r\n]*\)\s*\r?\n\{') | ForEach-Object { $_.Groups[1].Value }) foreach ($file in $files) { @@ -31,4 +31,4 @@ foreach ($file in $files) { } Write-Output "PASS $($file.Name)" } -Write-Output 'All five T6 compile checks and common references passed. Runtime linking/gameplay require Plutonium.' +Write-Output 'All six T6 compile checks and common references passed. Runtime linking/gameplay require Plutonium.' diff --git a/t6/tws_t6_join.gsc b/t6/tws_t6_join.gsc new file mode 100644 index 0000000..218e565 --- /dev/null +++ b/t6/tws_t6_join.gsc @@ -0,0 +1,116 @@ +/* tws_t6_join.gsc - .join lets a newcomer leave spectator mode before round end. */ + +#include maps\mp\_utility; +#include common_scripts\utility; + +// Plutonium probes both entry points; runtime setup belongs in init. +main() +{ + println("[tws_t6_join] main reached"); +} + +init() +{ + println("[tws_t6_join] init reached"); + + scripts\zm\tws_t6_common::ensure_config(); + scripts\zm\tws_t6_common::register_spawn_handler(::join_spawn); + + level thread join_chat_listener(); +} + +// Never reset this on death or round change. Spectator spawns do not count. +join_spawn() +{ + self.tws_join_has_played = true; +} + +join_chat_listener() +{ + level endon("end_game"); + level endon("game_ended"); + level endon("intermission"); + + for (;;) + { + level waittill("say", message, player); + + if (!isDefined(player) || !isDefined(message)) + continue; + + if (toLower(message) == ".join") + player thread join_request(); + } +} + +join_request() +{ + self endon("disconnect"); + level endon("end_game"); + level endon("game_ended"); + level endon("intermission"); + + // The spawn handler flag is set on first playable spawn only; spectator + // spawns do not trigger it. Belt-and-suspenders against re-triggering. + if (is_true(self.tws_join_has_played) || is_true(self.tws_join_used)) + { + self iPrintLnBold("^3.join is only for newcomers who have not played yet"); + return; + } + + // Fail closed while the stock connection setup is still in progress. + // T6 stock does not populate self.initialized the way T5 SP zombiemode + // does; use sessionstate as the readiness signal. + if (!isDefined(self.sessionstate) || self.sessionstate != "spectator") + { + self iPrintLnBold("^3.join is only available while waiting to enter the game"); + return; + } + + if (is_true(level.intermission) || is_true(level.gameended) || + !isDefined(level.round_number) || level.round_number < 1 || + !isDefined(level.spawnplayer) || !join_has_active_player()) + { + self iPrintLnBold("^3The game is not ready for .join"); + return; + } + + // Late connections can miss the stock seeding of self.spectator_respawn + // (stock asserts it in spectator_respawn); pick a random map spawn point + // as a fallback so the stock callback can proceed. + if (!isDefined(self.spectator_respawn)) + { + points = getstructarray("initial_spawn_points", "targetname"); + if (!isDefined(points) || points.size == 0) + { + self iPrintLnBold("^3No spawn point available; wait for the next round"); + return; + } + + self.spectator_respawn = points[randomInt(points.size)]; + } + + // Match stock spectators_respawn setup. Preserve any map-specific + // spawn callback if another mod already installed one. + if (!isDefined(level.custom_spawnplayer)) + level.custom_spawnplayer = maps\mp\zombies\_zm::spectator_respawn; + + // Consume BEFORE entering stock code, which may yield. Repeated chat + // commands cannot start overlapping spawns or grant another life. + self.tws_join_used = true; + self [[level.spawnplayer]](); +} + +join_has_active_player() +{ + players = getPlayers(); + for (i = 0; i < players.size; i++) + { + player = players[i]; + if (isDefined(player.sessionstate) && player.sessionstate == "playing" && + isAlive(player) && !(player maps\mp\zombies\_zm_laststand::player_is_in_laststand())) + return true; + } + + return false; +} diff --git a/t6/tws_t6_telemetry.gsc b/t6/tws_t6_telemetry.gsc index 4a6f98e..4e5864c 100644 --- a/t6/tws_t6_telemetry.gsc +++ b/t6/tws_t6_telemetry.gsc @@ -1,4 +1,4 @@ -/* tws_t6_telemetry.gsc - direct HTTP round telemetry via t6-gsc-utils. */ +/* tws_t6_telemetry.gsc - Zombies game reports via t6-gsc-utils v3.1.1. */ #include maps\mp\_utility; #include common_scripts\utility; @@ -12,15 +12,15 @@ main() init() { println("[tws_t6_telemetry] init reached"); - // Loaded only from scripts/zm; no shared HUD configuration is needed. - // Enabled when unset; explicitly set 0 to opt out. + + scripts\zm\tws_t6_common::ensure_config(); + if (GetDvar("tws_telemetry_enabled") == "0") { - println("[tws_t6_telemetry] disabled"); + println("[tws_t6_telemetry] Disabled by tws_telemetry_enabled=0"); return; } - // Repeated init in the same VM must not create competing senders. if (isDefined(level.telemetry_initialized)) return; @@ -31,133 +31,403 @@ init() level.telemetry_server = GetDvar("tws_server_id"); if (level.telemetry_server == "" || level.telemetry_server.size > 255) { - println("[tws_t6_telemetry] Set tws_server_id to the exact registered server name (1-255 characters)"); + println("^1[tws_t6_telemetry] Set tws_server_id to the exact registered server name (1-255 characters)"); return; } level.telemetry_initialized = true; - level.telemetry_pending = undefined; - println("[tws_t6_telemetry] watching for an occupied positive round"); - level thread telemetry_delivery_worker(); + level.telemetry_started = false; + level.telemetry_ended = false; + level.telemetry_last_round = -1; + level.telemetry_queue = []; + level.telemetry_queue_read = 0; + level.telemetry_queue_write = 0; + level.telemetry_sequence = 0; + level.telemetry_completed = 0; + level.telemetry_failed = 0; + level.telemetry_failure_streak = 0; + level.telemetry_last_completed = -1; + level.telemetry_last_capture = getTime(); + level.telemetry_round_tick = getTime(); + level.telemetry_capture_event = "none"; + level.telemetry_sender_state = "starting"; + level.telemetry_inflight = undefined; + + telemetry_log(1, "Initialized; waiting for match start. log_level=" + telemetry_log_level()); + + // Route through the shared dispatcher; a duplicate connected listener + // would fight the T6 common spawn dispatcher. + scripts\zm\tws_t6_common::register_spawn_handler(::telemetry_game_start); + + level thread telemetry_game_end_watcher(); + level thread telemetry_sender(); level thread telemetry_round_watcher(); + level thread telemetry_watchdog(); } -// Independent of HUD installation. Report the first occupied positive round, -// including round 1 when the map pre-seeds it, then each observed transition. -telemetry_round_watcher() +// First playable spawn starts the match, including maps that pre-seed round 1. +// Later spawns and joins must not reopen the match or repeat game_start. +telemetry_game_start() { - level endon("end_game"); - level endon("game_ended"); - level endon("intermission"); + if (level.telemetry_started || level.telemetry_ended) + return; - last_round = -1; + level.telemetry_started = true; + level.telemetry_last_capture = getTime(); + telemetry_log(1, "Match start detected"); + telemetry_enqueue("game_start"); + level thread telemetry_heartbeat_watcher(); +} + +// Start the interval with the match, independently of round changes or joins. +telemetry_heartbeat_watcher() +{ + level endon("telemetry_finished"); for (;;) { - if (isDefined(level.round_number) && level.round_number >= 1 && - level.round_number != last_round) + wait (30); + telemetry_report_round(); + telemetry_enqueue("heartbeat"); + } +} + +// Independent of HUD installation and player count. Poll for a positive round +// with any player present as a fallback in case the dispatcher spawn hook missed +// the first spawn (late-attach or stock ordering quirk). +telemetry_round_watcher() +{ + level endon("telemetry_finished"); + + for (;;) + { + level.telemetry_round_tick = getTime(); + if (!level.telemetry_started && isDefined(level.round_number) && + level.round_number >= 1 && getPlayers().size > 0) { - players = getPlayers(); - if (players.size > 0) - { - last_round = level.round_number; - - // Snapshot values before handing off to the HTTP thread. - data = []; - data["event"] = "round_change"; - data["round"] = int(last_round); - data["server"] = level.telemetry_server; - data["players"] = players.size; - - // No runId field (including null/empty), boot, session or - // feature events in Fleet-only mode. Keep just the newest - // pending observation so an outage cannot build a backlog. - pending = spawnStruct(); - println("[tws_t6_telemetry] serializing round " + last_round); - pending.body = jsonSerialize(data); - pending.observed_at = getTime(); - level.telemetry_pending = pending; - } + telemetry_log(1, "Starting from round_watcher fallback"); + telemetry_game_start(); } + telemetry_report_round(); wait (0.5); } } -telemetry_delivery_worker() +telemetry_report_round() { - level endon("end_game"); - level endon("game_ended"); - level endon("intermission"); + if (!level.telemetry_started || level.telemetry_ended || + !isDefined(level.round_number) || level.round_number < 0) + return; - backoff = []; - backoff[0] = 1; - backoff[1] = 2; - backoff[2] = 5; + round = telemetry_nonnegative_int(level.round_number); + if (round == level.telemetry_last_round) + return; - for (;;) + level.telemetry_last_round = round; + telemetry_enqueue("round_change"); +} + +telemetry_game_end_watcher() +{ + // One waiter consumes the first end signal, even if a map fires all three. + level waittill_any("end_game", "game_ended", "intermission"); + telemetry_log(1, "Game-end signal received; stopping producers and draining queue"); + + // Preserve order even if the match ends before the next polling tick. + telemetry_game_start(); + telemetry_report_round(); + level.telemetry_ended = true; + level notify("telemetry_finished"); + telemetry_enqueue("game_end"); +} + +telemetry_nonnegative_int(value) +{ + if (!isDefined(value) || value < 0) + return 0; + + return int(value); +} + +// T6 stock does not expose per-match kill/headshot/down/revive counters as a +// plain player.stats[] dictionary the way T5 does; _zm_stats writes through +// increment_client_stat / increment_player_stat into persistent stat storage. +// Read player.stats[name] when present so any mod that mirrors T5 into it +// still populates the report; otherwise emit 0 to preserve payload shape. +telemetry_stat(player, name) +{ + if (!isDefined(player.stats) || !isDefined(player.stats[name])) + return 0; + + return telemetry_nonnegative_int(player.stats[name]); +} + +telemetry_players_json(include_stats) +{ + players = getPlayers(); + body = "["; + + for (i = 0; i < players.size; i++) { - if (!isDefined(level.telemetry_pending)) + player = players[i]; + if (!isDefined(player)) { - wait (0.5); + telemetry_log(0, "WARNING: skipping undefined player at roster index " + i); continue; } - pending = level.telemetry_pending; - level.telemetry_pending = undefined; - - // One in-flight request plus one replaceable pending observation. - // A newer observation always retires retries of an older round. - for (attempt = 0; attempt < 4; attempt++) + data = []; + if (isDefined(player.name)) + data["name"] = "" + player.name; + else if (isDefined(player.playername)) + data["name"] = "" + player.playername; + else { - if (isDefined(level.telemetry_pending) || - getTime() - pending.observed_at > 15000) - break; - - if (telemetry_send(pending.body)) - break; - - if (attempt < 3) - wait (backoff[attempt]); + telemetry_log(0, "WARNING: skipping player without a name at roster index " + i); + continue; } + + // T6 exposes the persistent per-player identity as the .guid property + // (Plutonium engine field; not a method call and not from the plugin). + // With a linked Plutonium account it is the account ID; guest/local + // sessions fall back to the client slot number, which is expected. + // Force to string so the receiver never sees a numeric guid. + guid = player.guid; + if (!isDefined(guid) || guid == "" || data["name"] == "") + { + telemetry_log(0, "WARNING: skipping player with incomplete identity at roster index " + i); + continue; + } + data["guid"] = "" + guid; + + if (include_stats) + { + data["kills"] = telemetry_stat(player, "kills"); + data["headshots"] = telemetry_stat(player, "headshots"); + data["downs"] = telemetry_stat(player, "downs"); + data["revives"] = telemetry_stat(player, "revives"); + // Current spendable points, not the lifetime points-earned stat. + data["score"] = telemetry_nonnegative_int(player.score); + } + + if (body != "[") + body += ","; + body += jsonSerialize(data); + } + + // t6-gsc-utils serializes an empty GSC array as null. Build only the + // array delimiters ourselves; the serializer still escapes all strings. + return body + "]"; +} + +telemetry_enqueue(event_name) +{ + level.telemetry_capture_event = event_name; + telemetry_log(2, "Capturing event=" + event_name); + // Snapshot and serialize before yielding, including during game over. + body = "{\"event\":" + jsonSerialize(event_name); + body += ",\"server\":" + jsonSerialize(level.telemetry_server); + body += ",\"map\":" + jsonSerialize(GetDvar("mapname")); + if (event_name != "game_start") + body += ",\"round\":" + telemetry_nonnegative_int(level.round_number); + body += ",\"players\":" + telemetry_players_json(event_name != "game_start") + "}"; + + entry = []; + level.telemetry_sequence++; + entry["id"] = level.telemetry_sequence; + entry["event"] = event_name; + entry["round"] = telemetry_nonnegative_int(level.round_number); + entry["captured"] = getTime(); + entry["body"] = body; + level.telemetry_queue[level.telemetry_queue_write] = entry; + level.telemetry_queue_write++; + level.telemetry_last_capture = getTime(); + level.telemetry_capture_event = "none"; + telemetry_log(2, "Queued " + telemetry_event_label(entry) + " pending=" + telemetry_pending()); + level notify("telemetry_queued"); +} + +// A single worker keeps one request in flight. It survives game-over notifies +// to drain the final snapshot; no retries, discovery, or beta-specific calls. +telemetry_sender() +{ + for (;;) + { + if (level.telemetry_queue_read == level.telemetry_queue_write) + { + level.telemetry_queue = []; + level.telemetry_queue_read = 0; + level.telemetry_queue_write = 0; + if (level.telemetry_ended) + { + level.telemetry_sender_state = "drained"; + telemetry_log(1, "Queue drained after game end; completed=" + level.telemetry_completed + " failed=" + level.telemetry_failed); + level notify("telemetry_drained"); + return; + } + + level.telemetry_sender_state = "idle"; + level waittill("telemetry_queued"); + continue; + } + + entry = level.telemetry_queue[level.telemetry_queue_read]; + level.telemetry_queue[level.telemetry_queue_read] = undefined; + level.telemetry_queue_read++; + telemetry_send(entry); } } -// Returns transport completion only, NEVER beta/API acceptance. The released -// plugin exposes body/success/error but no HTTP status or cancellation. Do not -// start a replacement request on a script timeout: the old one could still -// commit later and overwrite Fleet with a stale round. Gameplay never waits -// on this worker. Malformed/HTML bodies are not passed to throwing jsonParse. -telemetry_send(body) +// v3.1.1 global alias: httpPost(url, options) with options["method"|"headers"|"body"]. +// The plugin exposes no cancellation API and no HTTP status code; treat the +// success flag as transport completion only and never overlap requests. +telemetry_send(entry) { + level.telemetry_inflight = entry; + level.telemetry_request_started = getTime(); + level.telemetry_sender_state = "creating_request"; + label = telemetry_event_label(entry); + telemetry_log(1, "Sending " + label + " queue_ms=" + (getTime() - entry["captured"]) + " pending=" + telemetry_pending()); + headers = []; headers["Content-Type"] = "application/json"; options = []; options["method"] = "POST"; options["headers"] = headers; - options["body"] = body; + options["body"] = entry["body"]; - // v3.1.1 global alias: options array, NOT positional body/headers. - // Avoid the raw loader treating the http namespace as http.gsc. - println("[tws_t6_telemetry] calling httpPost"); request = httpPost(level.telemetry_url, options); - println("[tws_t6_telemetry] HTTP handle returned; waiting for completion"); + if (!isDefined(request)) + { + telemetry_transport_failed(label + " request handle is undefined; skipping event"); + level.telemetry_inflight = undefined; + level.telemetry_sender_state = "idle"; + return; + } + + level.telemetry_sender_state = "waiting_http"; request waittill("done", response, success, error); - println("[tws_t6_telemetry] HTTP completion received"); + elapsed = getTime() - level.telemetry_request_started; + level.telemetry_last_completed = getTime(); // The plugin's success flag reports transport completion, not HTTP status. - // Unscoped round reports replace Fleet state; retrying the same snapshot - // cannot duplicate beta evidence. HTTP errors cannot be classified here, - // so only reported transport failures receive bounded retries. + // No automatic retry: the receiver may already have accepted the event. if (!isDefined(success) || !success) { if (!isDefined(error)) error = "unknown transport error"; - print("^1[tws_t6_telemetry] Round telemetry failed: " + error); - return false; + telemetry_transport_failed(label + " elapsed_ms=" + elapsed + " error=" + telemetry_preview(error)); + } + else + { + level.telemetry_completed++; + if (level.telemetry_failure_streak > 0) + telemetry_log(0, "Transport recovered after " + level.telemetry_failure_streak + " failures; " + label); + level.telemetry_failure_streak = 0; + telemetry_log(1, "Transport completed " + label + " elapsed_ms=" + elapsed + " (API acceptance unverified)"); + if (!isDefined(response) || response == "") + telemetry_log(0, "WARNING: empty API response for " + label); + else + telemetry_log(2, "Response " + label + " body=" + telemetry_preview(response)); } - println("[tws_t6_telemetry] Transport completed; API acceptance is not verified by this plugin"); - return true; + level.telemetry_inflight = undefined; + level.telemetry_sender_state = "idle"; +} + +telemetry_transport_failed(detail) +{ + level.telemetry_failed++; + level.telemetry_failure_streak++; + telemetry_log(0, "ERROR: transport failed " + detail + " consecutive_failures=" + level.telemetry_failure_streak); +} + +telemetry_pending() +{ + return level.telemetry_queue_write - level.telemetry_queue_read; +} + +telemetry_event_label(entry) +{ + return "id=" + entry["id"] + " event=" + entry["event"] + " round=" + entry["round"]; +} + +// Read live so diagnostics can be increased without restarting the map. +telemetry_log_level() +{ + raw = GetDvar("tws_telemetry_log_level"); + if (raw == "0") + return 0; + if (raw == "2") + return 2; + return 1; +} + +telemetry_log(verbosity, message) +{ + if (verbosity > telemetry_log_level()) + return; + + println("[tws_t6_telemetry] server=" + level.telemetry_server + " map=" + GetDvar("mapname") + " time_ms=" + getTime() + " " + message); +} + +// Escape newlines/control characters and bound response/error output. Do not +// jsonParse arbitrary replies: a proxy may return HTML or malformed JSON. +telemetry_preview(value) +{ + if (!isDefined(value)) + return ""; + value = "" + value; + if (value.size > 256) + value = getSubStr(value, 0, 256) + "... [truncated]"; + return jsonSerialize(value); +} + +// Independent of producers and the HTTP wait; survives game over until drain. +// The plugin cannot cancel requests. Warn without opening overlapping POSTs +// that could arrive out of order or exhaust its native worker pool. +telemetry_watchdog() +{ + level endon("telemetry_drained"); + for (;;) + { + wait (30); + now = getTime(); + pending = telemetry_pending(); + completion_age = -1; + if (level.telemetry_last_completed >= 0) + completion_age = now - level.telemetry_last_completed; + + telemetry_log(1, "Health started=" + level.telemetry_started + " ended=" + level.telemetry_ended + + " sender=" + level.telemetry_sender_state + " capturing=" + level.telemetry_capture_event + + " pending=" + pending + " queued_total=" + level.telemetry_sequence + + " completed=" + level.telemetry_completed + " failed=" + level.telemetry_failed + + " last_done_ms_ago=" + completion_age); + + if (isDefined(level.telemetry_inflight)) + { + age = now - level.telemetry_request_started; + if (age >= 30000) + telemetry_log(0, "WARNING: HTTP stalled " + telemetry_event_label(level.telemetry_inflight) + + " elapsed_ms=" + age + " pending=" + pending + " stage=" + level.telemetry_sender_state + + "; waiting for plugin completion (no cancellation API)"); + } + else if (pending > 0) + { + oldest = level.telemetry_queue[level.telemetry_queue_read]; + if (now - oldest["captured"] >= 30000) + telemetry_log(0, "WARNING: sender is not consuming queued reports; " + telemetry_event_label(oldest)); + } + + if (!level.telemetry_ended) + { + if (now - level.telemetry_round_tick > 5000) + telemetry_log(0, "WARNING: round watcher has stopped advancing"); + if (level.telemetry_started && now - level.telemetry_last_capture >= 65000) + telemetry_log(0, "WARNING: no snapshot captured for " + (now - level.telemetry_last_capture) + + "ms; capture_event=" + level.telemetry_capture_event); + } + } }