Refresh agent signing keys from repository (Refs #64)

This commit is contained in:
2026-10-09 21:39:03 -04:00
parent 6f71822d41
commit a93ddb5ef9
14 changed files with 420 additions and 214 deletions
+88 -58
View File
@@ -12,9 +12,9 @@ the agent owns local execution and its durable operation journal.
and roles; agent endpoints authenticate their own host credentials. See
[Users and access](authentication.md) for setup and password recovery.
2. Open **Hosts → Add Host**, choose a name, local game-port range and bind address,
and generate an enrollment. First [provision release trust](#provision-release-trust)
and copy the reviewed installer to `./electric-cherry-install.sh` on the Debian
12/13 amd64 host. Then run the displayed command from that directory.
and generate an enrollment. Run the displayed command on the Debian 12/13
amd64 host. It downloads the installer directly from the public repository;
[release trust](#provision-release-trust) is fetched automatically.
3. The installer validates the platform, installs Python 3, `gpgv` and an absent Docker
installation, verifies the release signature and agent/updater SHA-256 checksums,
enrolls it and enables `electric-cherry-agent.service` and its automatic update
@@ -45,34 +45,44 @@ copying or per-container setup. EC does not supply the game content itself.
### Provision release trust
Obtain the release publisher's public key and reviewed installer through a
trusted channel independent of the controller. Verify the full public-key
fingerprint with the publisher through that independent channel. On a trusted
administrator workstation with GnuPG, inspect the supplied binary public key:
The public Electric Cherry repository is the authority for both the installer
and release-signing public keys. The enrollment command downloads `agent/install.sh`
from that repository over HTTPS, saves it locally, and runs it only after the
download succeeds. No manual key transfer is required.
```sh
gpg --show-keys --with-fingerprint ./release-signing.gpg
During installation and **before every update check**, the host downloads the
binary OpenPGP public keyring directly from this fixed URL:
```text
https://gitea.thewrightserver.net/josh/electric-cherry/raw/branch/main/agent/release-signing.gpg
```
Transfer that verified public key and installer to the host, with the installer
named `electric-cherry-install.sh`. Provision the key before running enrollment:
The URL is built into the trusted installer and updater. Controller configuration,
release metadata and enrollment credentials cannot change it. The host verifies
HTTPS certificates, rejects redirects and incomplete, empty or oversized key
responses, and requests revalidation with `Cache-Control: no-cache`. It uses the
fresh keyring only for that check and deletes its temporary copy afterward.
There is no fallback to a previous host keyring, cached download or controller key.
If the repository is unavailable, new installations stop and automatic updates
wait for the next check; the installed agent and game containers keep running.
Local recovery of an interrupted update does not require the repository.
```sh
sudo install -d -o root -g root -m 0700 /etc/electric-cherry
sudo install -o root -g root -m 0644 ./release-signing.gpg /etc/electric-cherry/release-signing.gpg
```
This deliberately trusts the repository account, its `main` branch and HTTPS
hosting as authorities for root code on game hosts. They must be outside the
controller's write access: a compromised controller must not also have a token,
mount or server access that can change these repository files. A repository
compromise can replace the key or installer. Before initial enrollment, check that
the command downloads from
`https://gitea.thewrightserver.net/josh/electric-cherry/raw/branch/main/agent/install.sh`,
since a compromised controller UI could display a different command. Host provisioning systems can use that known URL directly.
The keyring and every parent directory must be owned by root, must not be
symlinks, and must not be writable by other users. Install only the publisher
keys you intend to trust. The generated enrollment command runs the local
installer with `sudo bash ./electric-cherry-install.sh`; never replace this with
a controller download piped into a root shell. An installer supplied by a
compromised controller could bypass its own verification. The host never obtains
or replaces its trust key through controller APIs.
If you build your own controller releases, act as the publisher using the
[signing workflow below](#publishing-signed-releases). Hosts reject unsigned
releases, including ordinary development builds that have not been signed.
Before deploying the first signed release, the publisher must export and commit
its **public** key as `agent/release-signing.gpg` on `main`, following the
[signing workflow below](#publishing-signed-releases). No production key is
created by packaging. Until the public key and signed release are published,
installation and updates fail closed. Forks must change the fixed repository URLs
in the installer, updater and enrollment command to their own trusted repository;
there is intentionally no controller setting to override them.
### Docker installation
@@ -200,8 +210,9 @@ directory, while file writes reject symlinks and paths outside that directory.
## Automatic agent updates
New installations enable automatic agent updates by default. Each host checks
its controller's release over HTTPS about every five minutes, beginning about
two minutes after boot, with up to 30 seconds of jitter. Updating the controller
the repository public key and then its controller's release over HTTPS about
every five minutes, beginning about two minutes after boot, with up to 30 seconds
of jitter. Updating the controller
therefore makes its compatible, newer signed agent release available to enrolled hosts.
There is no controller command to schedule and no update button to press.
@@ -211,27 +222,30 @@ controller-forged updates until this migration is complete. Publishing a signed
release does not upgrade those bootstraps: they request an installer rerun when
they see protocol 2, but a compromised controller could still serve protocol 1.
First provision release trust as described above. From the host's **Agent
access**, generate a command and run it using the reviewed local installer, with
the same controller and data directory. Preserve a custom root with `--data-dir`;
From the host's **Agent access**, generate and run the command, which fetches
the repository installer and public key automatically. Use the same controller
and data directory. Preserve a custom root with `--data-dir`;
do not use `--reenroll` for this upgrade. To pause old automatic checks during
migration, run `sudo systemctl stop electric-cherry-update.timer`. This does not
interrupt a running update worker; let any active update or journal recovery
finish before installing. The installer preserves host identity, credentials,
game files and running containers, replaces the bootstrap, and enables its timer
on success. Hosts without an updater show **Setup required** until a heartbeat
confirms installation.
confirms installation. Hosts using an earlier protocol 2 bootstrap with a
manually provisioned keyring also need this installer rerun to adopt repository
refresh: the updater does not replace itself automatically. An old local
`/etc/electric-cherry/release-signing.gpg` file is ignored by the new bootstrap.
The **Automatic agent updates** panel reports the running version, target
version, last check, progress and any error. Offline hosts show their last
reported status. Deferred updates report why they are waiting: for example an
active agent operation, an unavailable controller or a newer updater bootstrap.
active agent operation, an unavailable repository/controller or a newer updater bootstrap.
Checks also defer while the agent service is stopped; an update does not start a
service that was stopped outside the update workflow.
The independent updater holds the same execution lock, authenticates the exact
manifest bytes with the pinned keyring before parsing them, verifies the release
checksum, version and supported protocol, replaces agent code, and restarts the
agent service. It never restarts Docker or game containers. It confirms an
manifest bytes with the freshly fetched repository keyring before parsing them,
verifies the release checksum, version and supported protocol, replaces agent
code, and restarts the agent service. It never restarts Docker or game containers. It confirms an
authenticated heartbeat from the new agent process before accepting the update;
failure restores the previous agent code. Its durable journal permits recovery
after a host reboot or interrupted update.
@@ -266,13 +280,21 @@ gpg --homedir "$ec_signing_home" --list-keys --with-fingerprint
Protect the key with a passphrase and keep its backup and revocation certificate
outside release artifacts. Set the full fingerprint of the signing key, then
export only its public key for independent host provisioning:
export only its public key into the reviewed checkout:
```sh
ec_signing_fingerprint=REPLACE_WITH_FULL_SIGNING_KEY_FINGERPRINT
gpg --homedir "$ec_signing_home" --export "$ec_signing_fingerprint" > release-signing.gpg
gpg --homedir "$ec_signing_home" --export "$ec_signing_fingerprint" > agent/release-signing.gpg
```
Inspect the export with `gpg --show-keys --with-fingerprint agent/release-signing.gpg`
and confirm the expected publisher fingerprint. Commit and publish that public
file to the repository's `main` branch before enrolling hosts or releasing code
signed by this key. Never export secret keys into the repository. A YubiKey-held
signing key works with the same workflow: export its public certificate from your
GnuPG home and use the full signing key/subkey fingerprint with the signer. Keep
the YubiKey on the publisher workstation, outside controller and CI access.
From the reviewed checkout on the publisher machine, package and explicitly sign:
```sh
@@ -299,18 +321,22 @@ run the controller, but hosts fail closed and leave their installed code intact.
### Signing-key rotation and removal
Verify new fingerprints independently and replace each host's root-owned keyring
through trusted administration before publishing releases signed by a new key.
For a planned transition, provision both approved public keys temporarily, then
remove the retired key once every host can verify the new signer. The controller
cannot authorize key rotation.
Review new fingerprints on the publisher workstation, then replace
`agent/release-signing.gpg` on the repository's `main` branch. Every host uses the
new keyring on its next update check; no host visits or controller changes are
needed. For a planned transition, export both approved public keys into this file,
publish the combined keyring, then publish releases signed by the new key. Remove
the retired key once the controller serves the new signatures. Checks already in
progress use the keyring they fetched at the start of that check.
Remove compromised or retired keys from every host's keyring. Publishing a
revocation certificate or allowing a key to expire is insufficient: [GnuPG's
For a compromised key, remove it from the repository keyring immediately and
publish releases signed by a replacement key. Hosts with the refreshed keyring
reject the old signer even if the controller still serves its release. Publishing
a revocation certificate or allowing a key to expire is insufficient: [GnuPG's
`gpgv` documentation](https://www.gnupg.org/documentation/manuals/gnupg/gpgv.html)
states that it does not check key expiration or revocation. Host deletion removes
the pinned keyring with `/etc/electric-cherry`; provision trust again before
reenrolling that machine.
states that it does not check key expiration or revocation. Keep the keyring's
contents restricted to currently trusted keys. Removing all keys blocks new
installations and updates; it does not stop or remove already installed agents.
### Inspecting updates
@@ -478,9 +504,9 @@ and polling; Docker has no exposed remote API. Agent endpoints reject HTTP unles
`EC_AGENT_ALLOW_HTTP=true` is explicitly enabled for development. The agent's
insecure development option is limited to loopback. Public release assets contain
no credentials. HTTPS protects transport; an offline publisher signature over the
release manifest authenticates agent and updater code using the independently
provisioned host keyring. Controller credentials do not grant release-signing
authority.
release manifest authenticates agent and updater code using the public keyring
fetched directly from the trusted repository before each check. Controller
credentials do not grant release-signing authority or control the key source.
Browser sessions do not authorize agents. Similarly, host credentials never grant
access to operator pages. Game telemetry uses a separate `EC_TELEMETRY_TOKEN`,
@@ -687,11 +713,11 @@ and invalidates unused installer links. A separate cleanup service removes the
host's downloaded game libraries, instance files, agent journal and configuration,
owned containers, and recorded runtime images unused by other containers. It
stops and removes the agent service and executable, automatic update timer,
update service, updater executable, update state and pinned signing keyring, then
removes its own files.
update service, updater executable and update state (including any legacy local
signing keyring), then removes its own files.
The controller hides the host and revokes its ordinary credentials only after
receiving the final uninstall receipt. Adding the machine again requires a new
host enrollment and independently reprovisioned release trust.
host enrollment; the installer fetches release trust from the repository again.
Game folders uploaded through Electric Cherry belong to the controller. Those
original uploads remain available for other hosts and future deployments; only
@@ -956,7 +982,8 @@ also requires the host's `gpgv` executable. Real signing tests require `gpg` and
Automated tests cover enrollment/authentication, rotation/revocation, host
readiness, scoped ports, immutable library publication, streaming/ranges,
signed manifests, forged checksums, unknown signing keys, interrupted downloads,
signed manifests, forged checksums, unknown signing keys, repository key refresh
and rotation, repository failures, interrupted downloads,
instance generation ownership, local occupancy
holds, controller restart/reconnect, ambiguous results, occupied campaign staging,
activation before images, collections and draining. Host deletion tests cover
@@ -968,15 +995,18 @@ A real Debian/Docker/Plutonium acceptance run is still required before productio
rollout. Automated simulations do not establish successful deployment on real
game hosts. Run:
- Independently provision release trust and run the reviewed installer on Debian
12 and 13 amd64, then rerun without changing identity/data.
- Publish the public key in the repository and run the generated enrollment
command on Debian 12 and 13 amd64 without a local keyring, then rerun without
changing identity/data.
- Migrate a protocol 1 host with the trusted installer, then publish a newer
compatible signed agent release. Verify a timer-driven update, its authenticated
heartbeat and unchanged game containers. Exercise an active-operation hold,
missing/invalid signatures, an unknown key, a forged manifest matching changed
code, bad checksum, failed startup/heartbeat rollback, controller downgrade,
network loss and host reboot during update. Verify updater protocol mismatch
requests an installer rerun and test trusted keyring replacement for rotation.
requests an installer rerun. Replace the repository keyring and confirm the
next check rejects the removed signer and accepts the new signer; block access
to the repository and confirm no cached key is used and local rollback still works.
- T5 and T6 deployments and a mixed collection from a fresh destination host.
- Warm the Plutonium runtime cache, then create and rebuild T5 and T6 lobbies;
verify the running revision and compare first-start downloads and launch times