electric-cherry/README.md

# Electric Cherry

Electric Cherry manages T5 and T6 Plutonium Zombies servers through its own host
agents. Docker and Plutainer run the games; EC handles hosts, game libraries,
collections, configuration, campaigns, maintenance and operation history.
Komodo is not required for native hosts.

## Getting started

Run the controller with Docker Compose behind HTTPS:

```sh
cp .env.example .env
# Set EC_PUBLIC_URL, generate EC_SETUP_TOKEN, and retain EC_DATA_DIR.
docker compose up -d --build
```

Open EC and create the first administrator using `EC_SETUP_TOKEN` (at least 32
characters; generate it with `openssl rand -hex 32`). After setup, remove that
token from the environment. Administrators can add administrators, operators and
viewers from **Settings → Users**. See [Users and access](docs/authentication.md)
for permissions, sessions, password recovery and upgrade instructions.

Operator pages and management APIs require login. Enrolled agents authenticate
independently and connect outbound over HTTPS; the
controller needs no Docker socket, game-server mounts or inbound RCON access.
The default Compose data mount preserves the previous
`/root/zombies/.electric-cherry` database location. For a new installation, set
`EC_DATA_DIR` to your preferred persistent directory. Back up that directory,
including SQLite metadata and published game-file storage.

1. **Library → Game Files**: select T5 or T6 and upload the complete local game
   folder, including its DLC files. Name the library and version, then publish it.
   Uploads work before any host is enrolled.
2. **Hosts → Add Host**: configure a Debian 12/13 amd64 machine and run its
   generated enrollment command. Automatic agent updates are enabled by default.
3. Prepare a Plutainer runtime from the host page and download the published
   game-file version to that host from **Game Files**.
4. Add the exact Plutonium server keys in **Keys**, or supply missing keys when
   launching.
5. Launch a server or collection with an explicit host, game-file version and
   runtime image. EC reserves host-specific ports and verifies startup.

See [Native host management](docs/native-hosts.md) for installation, credentials,
storage, failure recovery, API contracts and the live-host acceptance checklist.

Existing hosts need the current installer once to enable automatic updates.
Generate it under **Agent access** and rerun it on the same host without
`--reenroll`; identity and game data are preserved. Thereafter hosts check their
controller about every five minutes, verify compatible releases and update agent
code between operations. Failed updates roll back; running game containers
continue. The host page shows update status and the last check.

## Fleet and operations

The fleet registry lives in SQLite, independently of host filesystems. Status
combines agent container health, fresh game telemetry and agent-local RCON.
Unreachable hosts report unknown instance state; they do not trigger duplicate
replacements. Running containers continue through controller or agent outages.

Provisioning owns Create, Start, Stop, Restart, Rebuild and Destroy. Commands and
results have durable operation IDs and immutable instance generations. Queues,
capacity reservations and history survive controller restarts. **Rebuild removes
per-instance writable files** and creates a new generation. Shared immutable game
libraries survive rebuilds and destruction.

Disruptive actions wait for a fresh empty lobby or verified stopped container.
The agent rechecks occupancy locally immediately before activation and disruption.
Long player sessions can continue indefinitely. **Drain** prevents new launches
and safely retires existing servers as they become eligible.

Collections save target maps and lobby counts. Fill missing servers adds only the
shortfall; Match collection exactly also queues safe removal of excess servers.
Launch previews select a host, pinned base-game versions and an independent
runtime image. Every new launch uses the latest promoted configuration, scripts
and plugins when execution starts. Saved Plutonium keys are reused by exact
server ID and retained after removal.

## Libraries and campaigns

Game-file versions contain the uploaded T5/T6 folder, with its directory layout
and included DLC preserved. There is no separate DLC declaration. Versions record
the game, label, file manifest, checksums and size. Uploads use bounded chunks,
support pause/resume and verify checksums before publication. Actual game files
use dedicated filesystem storage, never SQLite blobs. Host downloads resume,
report progress and verify checksums before becoming usable.
Multiple instances on one host share read-only base files.

Scripts, plugins, configuration baselines and overlays retain their versioning,
beta, promotion and rollback workflows. Campaigns stage bytes remotely without
stopping games; activation uses the same occupancy and concurrency gates as
lifecycle operations. Agent before images preserve exact rollback bytes and
operation history.

T5 Zombies uses `t5sp` and `dedicated_sp.cfg`; T6 uses `t6zm` and
`dedicated_zm.cfg`. Map selection generates `electric_cherry_map.cfg`, loaded
after the main config. T6 variants retain their distinct catalog selections.
Game-file libraries and Plutonium runtime revisions are tracked independently.

Game telemetry uses its own `EC_TELEMETRY_TOKEN`; configure reporters to send it
in an authorization header. Anonymous reports are rejected before they can
affect occupancy or game history. Existing JSON payloads are unchanged.

## Documentation

- [Native hosts, storage and operation recovery](docs/native-hosts.md)
- [Users, roles and authentication](docs/authentication.md)
- [Provisioning and safety queues](docs/provisioning.md)
- [Collections and promoted launch recipes](docs/deployment-blueprints.md)
- [Maintenance](docs/maintenance.md)
- [Beta test plans and game telemetry](docs/beta-test-plans.md)
- [Plugin releases](docs/plugin-releases.md)
- [Metrics](docs/metrics.md)
- [Leaderboard and match history](docs/leaderboard.md)

Older workflow documents may describe the legacy Komodo backend. Native hosts
use the agent boundary described in the native-host document.

## Development

The controller is SvelteKit/TypeScript with SQLite via better-sqlite3, the Node
adapter and server-sent events. The host agent is Python 3 standard-library code
installed as a systemd service. Hosts do not install Node.js or build EC.

```sh
npm install
npm run agent:package
# Export EC_SETUP_TOKEN in your shell using the secret saved for /setup.
EC_AUTH_ALLOW_HTTP=true npm run dev

npm run check
npm test
npm run test:agent
npm run build
```

Builds regenerate the downloadable agent, standalone updater and checksum manifest. Local controller
metadata defaults to `./data/cherry.db`; use a separate test database when
exercising enrollment. Production agent transport requires HTTPS.

`src/lib/server/hosts` owns enrollment, readiness and command transport;
`src/lib/server/backends` provides instance identity and remote file access;
`src/lib/server/provisioning/native.ts` connects existing lifecycle and campaign
queues to agents; `src/lib/server/game-files` owns immutable libraries and
streaming distribution; `agent/` contains the service, installer and agent tests.

Automated tests use simulated Docker/game hosts. Real Debian installer and
T5/T6 game-server validation remain deployment acceptance work; see the checklist
in the native-host guide.