Automatically place servers using learned resource capacity
This commit is contained in:
+53
-3
@@ -27,8 +27,8 @@ the agent owns local execution and its durable operation journal.
|
||||
progress and errors. Wait for verification before launching. Hosts report
|
||||
connection and readiness separately; deployment also checks the particular
|
||||
game files, image, capacity and ports required for the launch.
|
||||
6. Launch a server or collection. Select the host; the latest game files and
|
||||
default runtime are selected automatically. An advanced runtime override is
|
||||
6. Launch a server or collection. The host, latest game files and default
|
||||
runtime are selected automatically after checking capacity. An advanced runtime override is
|
||||
available. If the latest files change after previewing, refresh the preview
|
||||
before launching.
|
||||
EC selects lobby IDs and reserves UDP/TCP ports atomically.
|
||||
@@ -121,7 +121,57 @@ host-wide busy time across all cores on a 0–100% scale between inventory sampl
|
||||
(normally about 15 seconds). Idle and I/O wait are excluded; stolen CPU time
|
||||
counts as unavailable capacity. The first sample after startup, unavailable
|
||||
counters, and older agents show **Not reported**. Offline hosts retain their
|
||||
last reported value. This is current inventory, not historical metrics.
|
||||
last reported value. The controller retains CPU and memory samples for seven days (missing measurements
|
||||
remain unknown). Scheduling uses the last five minutes as well as the latest
|
||||
heartbeat; the Inventory panel continues to show the latest reading.
|
||||
|
||||
New servers are placed automatically. Preview and submission perform local
|
||||
capacity checks without contacting agents: a connected, active host must have
|
||||
verified game files, its runtime, storage, a free UDP/TCP port, and valid CPU and
|
||||
memory measurements. Hosts with stale or missing measurements cannot accept new
|
||||
servers. Existing API clients may still explicitly constrain a launch to a host.
|
||||
|
||||
Admission reserves a provisional host and port atomically, including collection
|
||||
launches. Agent **1.4.3** samples each managed running container with a bounded
|
||||
[`docker stats`](https://docs.docker.com/reference/cli/docker/container/stats/)
|
||||
batch during inventory collection. CPU is normalized to cores (100% Docker CPU
|
||||
is one core); memory is the Linux working usage reported by Docker, excluding
|
||||
reclaimable cache. Missing or failed samples stay unknown instead of becoming
|
||||
zero usage. Samples must match the host's live instance and generation.
|
||||
|
||||
The controller retains per-instance minute peaks for 24 hours and caches learned
|
||||
CPU/memory allowances. The first five minutes after a container starts count as
|
||||
startup; later samples train the running allowance. Startup uses the observed
|
||||
maximum and running uses the 95th percentile of minute peaks, both with **25%
|
||||
headroom**. Each calculation considers up to the latest 10,000 minute peaks.
|
||||
Host-local evidence is preferred to account for different hardware; new hosts
|
||||
can borrow the fleet's learned allowance. Startup allowances are never lower
|
||||
than running allowances. The host page shows both allowances and their source.
|
||||
|
||||
Lower allowances require at least three startup minute samples spanning two
|
||||
minutes, or twenty running minute samples spanning five minutes. Until then,
|
||||
**0.5 core and 1 GiB** are temporary cold-start allowances, raised immediately
|
||||
if observations exceed them. They are replaced by measured allowances once the
|
||||
evidence threshold is met; a tiny 0.01-core floor prevents zero-CPU admission.
|
||||
Models expire after 24 hours without measurements. Recalculation happens during
|
||||
heartbeat ingestion, at most once per minute per host/fleet phase; new peaks
|
||||
raise the cache immediately. Launch admission reads only the cached model.
|
||||
|
||||
Scheduling still leaves **10% host CPU and 512 MiB RAM** for the OS and bursts.
|
||||
These allowances are scheduling reservations, not Docker limits. Queued and
|
||||
unresolved creations consume the startup allowance; established instances keep
|
||||
the running allowance even while idle or stopped. Observed host load also counts,
|
||||
so other processes and previously unseen load increases reduce available capacity.
|
||||
|
||||
When startup capacity is available, provisioning rechecks all eligible hosts and
|
||||
chooses the lowest projected CPU/memory utilization, using the busier of the
|
||||
latest reading and the five-minute average. It moves the provisional reservation
|
||||
atomically and records the final host in the launch recipe. Host network defaults
|
||||
are resolved again for the final host; explicit overrides are preserved. If no
|
||||
host qualifies, the request waits and is checked again. Once remote staging or
|
||||
creation has been submitted, placement stays fixed to prevent duplicate instances.
|
||||
Cancelling an unsubmitted launch releases its reservation; uncertain remote work
|
||||
retains capacity until cleanup is confirmed.
|
||||
|
||||
When publishing changed agent code, bump its numeric `VERSION` in
|
||||
`agent/ec_agent.py` and rebuild the controller. Packaging derives the manifest
|
||||
|
||||
Reference in New Issue
Block a user