josh/electric-cherry
1
0
Fork 0
Code Issues 18 Pull Requests Actions Packages Projects Releases Wiki Activity

Fix capacity accounting and queued host placement

Browse Source
This commit is contained in:
josh
2026-10-04 21:28:15 -04:00
parent e6cdf9eb7f
commit 8a31de92fa
4 changed files with 160 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/native-hosts.md
+19 -7
View File
@@ -122,8 +122,9 @@ host-wide busy time across all cores on a 0–100% scale between inventory sampl
counts as unavailable capacity. The first sample after startup, unavailable
counters, and older agents show **Not reported**. Offline hosts retain their
last reported value. The controller retains CPU and memory samples for seven days (missing measurements
remain unknown). Scheduling uses the last five minutes as well as the latest
heartbeat; the Inventory panel continues to show the latest reading.
remain unknown). Scheduling checks capacity against the latest heartbeat and uses
the last five minutes to rank eligible hosts; the Inventory panel shows the latest
reading.
New servers are placed automatically. Preview and submission perform local
capacity checks without contacting agents: a connected, active host must have
@@ -160,13 +161,24 @@ raise the cache immediately. Launch admission reads only the cached model.
Scheduling still leaves **10% host CPU and 512 MiB RAM** for the OS and bursts.
These allowances are scheduling reservations, not Docker limits. Queued and
unresolved creations consume the startup allowance; established instances keep
the running allowance even while idle or stopped. Observed host load also counts,
the running allowance even while idle or stopped. Current container measurements
are credited against their reservations because host usage already includes them;
only the unused allowance is added. Missing or mismatched-generation measurements
cannot release an unresolved launch's reservation. Observed host load also counts,
so other processes and previously unseen load increases reduce available capacity.
When startup capacity is available, provisioning rechecks all eligible hosts and
chooses the lowest projected CPU/memory utilization, using the busier of the
latest reading and the five-minute average. It moves the provisional reservation
atomically and records the final host in the launch recipe. Host network defaults
Admission counts all queued reservations. Dispatch evaluates one launch against
existing instances and submitted or uncertain work; other never-submitted queued
launches wait their turn. They retain their port reservations. This lets the queue
make progress even if a changed allowance means the whole backlog no longer fits
at once. Capacity errors report projected usage, usable capacity and reservations.
When startup capacity is available, provisioning checks each eligible host’s
current CPU and available memory against reservations and headroom.
Among hosts that fit, it chooses the lowest projected CPU/memory utilization,
using the busier of the latest reading and the five-minute average for ranking.
A historical resource dip cannot block a host whose current capacity has recovered.
Provisioning moves the provisional reservation atomically and records the final host in the launch recipe. Host network defaults
are resolved again for the final host; explicit overrides are preserved. If no
host qualifies, the request waits and is checked again. Once remote staging or
creation has been submitted, placement stays fixed to prevent duplicate instances.