Make metrics reflect autoscaling policies (Refs #137)

docs/guides/metrics.md

@@ -5,98 +5,137 @@ Choose the past 24 hours, 7 days, or 30 days. The page refreshes every minute.
 
 ## Overview and filters
 
-The overview shows **Time with players** (occupied server-hours divided by known
-server-hours) and **Player-hours**. Known and unknown status time remain visible
-alongside the totals. **Last collected** is the newest recorded batch, including
-unknown checks; **Report refreshed** is when this report was calculated. Collection
-stops when no servers are registered. Earliest retained observations do not prove
-continuous history or the original installation date.
+**Lobbies running** shows the average and peak number of recorded lobbies in the
+selected scope. The average divides captured lobby-hours, including unknown
+status, by captured wall time. Empty spares are deliberate supply for new groups:
+the card shows their hours and share of all captured lobby-hours, rather than
+treating them as failed utilization. Unknown status can include lobbies starting
+or unavailable, so this count does not guarantee every lobby was ready to join.
+
+**Player-hours** integrates reported player counts over time. **No empty lobby**
+shows the share of known map time when no tracked lobby reported zero players.
+Each game/map is measured separately, then its durations are pooled. This is not
+elapsed fleet time or a count of rejected joins. A partly occupied Zombies lobby
+does not count as an empty spare, even if player slots remain.
+
+**Last collected** is the newest recorded batch, including unknown checks;
+**Report refreshed** is when this report was calculated. Collection stops when no
+servers are registered. Earliest retained observations do not prove continuous
+history or the original installation date.
 
 Background refresh keeps the last successful report visible on failure and offers
-Retry. Returning to a visible tab refreshes immediately. The selected range is
-stored in the URL, and refreshes preserve table controls and chart selection.
-
-Game and map filters apply to the whole report, including its comparisons,
-recommendations, and heatmap. Map choices preserve the exact recorded game/map
-pair. Range and scope are shareable through the URL. Table searches remain local.
+Retry. Returning to a visible tab refreshes immediately. Game and map filters apply
+to the whole report, including comparisons, signals, and the heatmap. Map choices
+preserve the exact recorded game/map pair. Range and scope are shareable through
+the URL; refreshes preserve table controls and chart selection.
 
 ## Timeline and tables
 
-The timeline defaults to **Share of captured time**: each stack includes occupied,
-idle, and unknown server time, normalized to 100%. This denominator differs from the
-headline utilization percentage, which excludes unknown time. Switch to **Hours**
-to compare absolute server time. Periods without captured time remain gaps.
-The timeline has labelled axes and marks partial edge periods. Select a period,
-use Previous/Next, or select a bar to inspect its exact interval and values. The
-time table supplies the same figures for comparison and assistive technology.
-Zero captured time is distinct from an observed empty server. Short durations use
-seconds or minutes so a real observation is not rounded to zero.
+The timeline defaults to **Average lobbies running**, split into **With players**,
+**Empty spare**, and **Unknown**. Each bucket divides those lobby-hours by its
+captured wall time, so changing lobby counts remain visible. Collection gaps are
+excluded from that denominator. Switch to the share view to compare proportions,
+or hours to compare absolute lobby time. Periods without captured time remain gaps.
 
-Map and server tables support numeric sorting and local search. Their searches
-apply only to the named table. Known-capacity duration stays next to all-full time.
-Server identities retain the technical name; rows describe history grouped by
-name, which may include different instances if a name was reused. **Open current
-server** appears only when that name is currently registered; its destination
-represents the current registration rather than every instance in the history.
+The timeline has labelled axes and marks partial edge periods. Select a bar or use
+Previous/Next to inspect its exact interval and values. The time table supplies
+the same figures for comparison and assistive technology. Zero captured time is
+distinct from an observed empty lobby. Short durations use seconds or minutes so
+a real observation is not rounded to zero.
 
-## Comparisons and hosting experiments
+Map and instance tables support numeric sorting and local search. The map table's
+**No empty lobby** column uses occupancy alone and retains the known map-time
+denominator in its details. Unknown map availability stays distinct from zero
+time without a spare. Evidence can be limited by short observations or unknown
+occupancy, never by missing player limits.
 
-**Explore this period** adds experimental decision aids below the main timeline:
+Instance rows describe history grouped by server name, which can include different
+instances if a name was reused. **Open current server** appears only when that name
+is currently registered; its destination represents the current registration.
 
-- **Map capacity pressure** pools all-full map-hours divided by known-capacity
-  map-hours across the selected map groups. Overlapping map intervals are counted
-  separately. The score is not elapsed fleet time or a fleet health score; its
-  denominator and contributing map count remain visible.
-- **Previous-period comparisons** use the preceding equally long window with
-  identical filters. Utilization changes are percentage points; player-hours
-  changes are relative percentages. A previous observed zero can show **New**.
-  Missing evidence remains unavailable. Partial retained history is labelled;
-  a 30-day comparison usually has little previous history because default retention is
-  31 days. Both periods' known time is shown. Bounds cannot establish that
-  collection was uninterrupted, and changes are not adjusted for fleet size.
-- **Activity heatmap** groups captured time by weekday and hour in its displayed
-  timezone. Color represents occupied / known server time. Exact cell values
-  include unknown time and contributing dates. No-data cells differ from observed
-  zero utilization; repeated DST hours contribute their actual elapsed time.
-- **Hosting experiments** are generated automatically from the selected map
-  observations. They are advisory and never create, restart, or destroy a server.
+## Comparisons and activity
 
-Hosting Rules v1 deliberately offer experiments even with small samples:
+Previous-period comparisons use the preceding equally long window with identical
+filters. Average lobby counts compare recorded supply, while player-hours changes
+are relative percentages. A previous observed zero can show **New**. Missing
+evidence remains unavailable. Partial retained history is labelled; a 30-day
+comparison usually has little previous history because default retention is
+31 days. Collection gaps are excluded, and comparisons do not prove uninterrupted
+collection or adjust player-hours for fleet size.
 
-| Recorded condition                                                                                     | Suggested experiment                                                                  |
-| ------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------- |
-| At least 20% of known capacity time had every lobby full                                               | Trial one additional lobby during busy hours.                                         |
-| Capacity unavailable and players present for at least 80% of known server time                         | Tentatively trial an extra lobby using occupancy as a proxy.                          |
-| Players present for at most 10% of known time, with under 1% measured pressure or unavailable capacity | Review one idle lobby or a shorter hosting window while preserving access to the map. |
-| Occupancy unavailable                                                                                  | Check telemetry/RCON reporting and watch for known player counts.                     |
-| Other observations                                                                                     | Check missing capacity or keep the lineup and watch peaks.                            |
+The activity heatmap groups captured time by weekday and hour in its displayed
+timezone. Color represents occupied / known lobby time. Exact cell values include
+unknown time and contributing dates. No-data cells differ from observed zero;
+repeated DST hours contribute their actual elapsed time. Occupancy share describes
+how the fleet was used, but can change simply because a policy adds or removes
+spares. Read it alongside player-hours, lobby counts, and empty-lobby availability.
 
-Each suggestion includes evidence and an outcome to watch. Unknown capacity,
-less than one hour of relevant known evidence, or substantial unknown captured
-status makes the suggestion tentative. These thresholds are product hypotheses;
-occupancy does not establish failed joins or a need for additional capacity.
-Suggestions use historical observations. Check the current lineup before trying
-a change; a retained map may no longer have a hosted lobby.
+## Autoscaling signals
+
+Signals compare each map's observations with its **current** applicable policy.
+Game-wide policies apply independently to each supported map variant. Each signal
+links to the policy on **Fleet → Autoscaling**; a map without a policy gets a link
+to add one. Signals are advisory and never change a policy or a lobby.
+
+| Recorded condition                                                                       | Suggested action                                                 |
+| ---------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
+| No empty lobby for at least 2% of known map time, mostly at or above the current maximum | Review raising the policy maximum.                               |
+| No empty lobby for at least 2% of known map time, mostly below the current maximum       | Review the spare target and check creation time or blockers.     |
+| An applicable policy is off or paused                                                    | Review the policy and its pause or activation conditions.        |
+| No applicable policy and known occupancy is available                                    | Consider adding an autoscaling policy to keep spares available.  |
+| Occupancy unavailable                                                                    | Check telemetry/RCON reporting and wait for known player counts. |
+| No shortage signal                                                                       | Keep watching empty-lobby availability.                          |
+
+Evidence includes no-empty map time and how it splits above/below the current
+maximum. Less than one hour of known map availability, less than 80% known map
+availability, or less than 80% known lobby time makes advice tentative. Player limits are not needed. These
+thresholds are advisory rules, not demand forecasts.
+
+Policy settings may have changed during the selected period. Comparing historical
+counts against today's maximum does not establish which settings applied then or
+whether autoscaling was enabled. No reported empty lobby also does not prove
+failed joins or unmet demand. Review provisioning delays, blockers, host resources,
+and game-key availability before changing targets.
 
 ## Definitions and collection
 
-- Utilized means a successful status response reported at least one player.
-- Idle means a successful response reported zero players.
-- Unknown covers failed checks, provisioning, and unavailable containers; it is excluded from utilization percentages.
-- Player-hours integrate the reported player count over observed time. Bots are included if the server counts them as players.
-- All lobbies full measures map wall time when every tracked lobby in that game/map group has a known capacity and is full. Missing capacity or status makes that interval unknown. This is a capacity-pressure indicator, not a count of rejected joins or unmet demand.
+- **With players:** a successful observation reported at least one player.
+- **Empty spare:** an observation reported zero players.
+- **Unknown:** failed checks, provisioning, or otherwise unavailable occupancy.
+- **Player-hours:** reported players × observed time. Bots count if the server includes them.
+- **Empty lobby available:** at least one lobby on the game/map reported zero players.
+- **No empty lobby:** at least one lobby reported occupancy, and none reported zero. Mixed positive and unknown counts mean no empty lobby was reported; an unknown lobby might actually have been empty.
+- **Unknown map availability:** every lobby on the game/map had unknown occupancy. This interval is excluded from the known-map denominator and is never counted as a spare shortage.
 
-Observations are recorded by the existing status worker. Each remains valid until the next collection, up to the 90-second freshness limit. Longer manager outages are excluded. Charts split time at UTC hour/day boundaries; partial periods remain partial. Map variants use the configured variant when it matches the reported map.
+Observations come from the existing status worker. Each complete collection stays
+valid until the next collection, up to the 90-second freshness limit. Longer gaps
+are excluded, not counted as unknown or zero lobbies. Captured wall time includes
+intervals with at least one recorded lobby in the selected scope. Unknown lobby
+time and unknown map time use different units: several unknown lobbies on one map
+contribute several lobby-hours but only one interval of unknown map availability.
 
-Capacity comes from `sv_maxclients` / `sv_maxClients` in the status response. When unavailable, the UI displays unknown rather than assuming a four-player limit. Capacity coverage is the map time for which every tracked lobby reported both occupancy and capacity.
+Charts split time at UTC hour/day boundaries; partial periods remain partial.
+Map variants use the configured variant when it matches the reported map. Player
+capacity may be retained in raw samples, but no metric or signal requires
+`sv_maxclients` or an assumed four-player limit.
 
-Collection begins after this feature is installed; previous occupancy cannot be reconstructed. Samples default to 31 days of retention; Admin → Settings can change this period. Cleanup runs during collection and in the background. They survive removal of a server and contain no player identities. The additive database table is created automatically on startup.
+Collection begins after this feature is installed; earlier occupancy cannot be
+reconstructed. Samples default to 31 days of retention; **Admin → Settings** can
+change this period. Cleanup runs during collection and in the background. Samples
+survive removal of a server and contain no player identities.
 
 ## API
 
-`GET /api/metrics` supplies the same report as the page and disables response
+`GET /api/metrics` supplies the same report as the page and disables HTTP response
 caching. Query parameters are `days` (1, 7, or 30), `game`, `map` (requires game),
 and `timeZone` (an IANA zone). Invalid ranges and zones fall back to the defaults;
 invalid scope syntax is normalized. Fleet collection timestamps remain separate
 from selected-scope evidence timestamps. Collection boundaries are determined
 before scope filtering, so samples from other groups still end a batch interval.
+
+`running` contains `average`, `peak`, and `capturedHours`; timeline points include
+their own `capturedHours`. Map rows expose `noEmpty`, `availabilityObserved`, and
+`availabilityUnknown` durations, plus current `policy` context and
+`noEmptyAtMaximum` / `noEmptyBelowMaximum` durations. `pooledAvailability` combines
+map durations and coverage. Policy edits and global pause changes invalidate
+cached metrics reports.