electric-cherry/docs/development/README.md

# Development

The controller uses SvelteKit, TypeScript and SQLite (`better-sqlite3`), with the
Node adapter and server-sent events. The host agent uses Python 3's standard
library and runs as a systemd service; hosts do not need Node.js.

Agents connect outbound to the controller over HTTPS and manage Docker/Plutainer
and local RCON on each host. The controller owns fleet metadata and queues; it
does not require a Docker socket or direct access to host filesystems.

## Run locally

Use Node.js 22.12+ (the container uses Node.js 22), npm and Python 3.

```sh
npm ci
export EC_SETUP_TOKEN="$(openssl rand -hex 32)"
export BETA_ARCHIVE_DIR=./data/beta-archive
EC_AUTH_ALLOW_HTTP=true npm run dev
```

Use the exported token at `/setup` to create the first administrator, then remove
it from the environment. Local metadata defaults to `./data/cherry.db`; use a
separate database when testing enrollment. HTTP login is for local development;
production agent transport requires HTTPS. See [authentication](../guides/authentication.md)
for access and recovery details.

## Synthetic UI preview

Create a disposable fixture without connecting game hosts:

```sh
fixture_parent=$(mktemp -d "${TMPDIR:-/tmp}/ec-ui.XXXXXX")
npm run seed:fixture -- --db "$fixture_parent/preview/cherry.db" --watch
```

The command prints the generated paths. Leave that terminal open to refresh
synthetic observations every 15 seconds. In a second terminal, load the printed
environment file and start the controller on localhost:

```sh
source /absolute/path/printed/above/preview.env
npm run dev -- --host 127.0.0.1 --port 3000
```

Open `http://127.0.0.1:3000/login`. The private `fixture.json` next to the database
contains the `demo` account's generated password. It is not printed in command
output. To run a fixed snapshot instead of a live feeder, use a fresh directory
and replace `--watch` with `--now 2026-10-07T12:00:00Z`. Domain data is deterministic
for that timestamp; account credentials are always generated separately. Old
snapshots naturally show stale observations when viewed against today's clock.

The fixture contains 3 hosts, 12 canonical Black Ops I/II servers, and 500 match
records: 494 historical and 6 live. It includes solo through four-player squads,
unknown starting counts, enough players and matches to exercise pagination,
rosters, seven days of utilization and capacity samples, and retained Activity
from campaigns, betas, host operations, maintenance, recovery, and autoscaling.
The fleet has 6 live matches, 2 empty lobbies, a preparing server, and 3 problems:
a stopped server, a server not responding, and a server on the offline host.
Script and config downloads contain harmless synthetic text.

`--db` must be an absolute path in a **new directory** whose parent exists. The
seeder refuses existing directories, databases, sidecars and symlinks, and never
uses an inherited `DB_PATH`. It runs the application's database migrations with
isolated paths, without starting the web server or workers. A failed seed removes
only its newly reserved directory. Completed fixtures remain after Ctrl+C.

The preview controller runs its usual local workers. Automatic maintenance and
autoscaling are paused; all seeded operations are terminal. Hostnames, runtime
images, library IDs and credentials are synthetic, so there are no runnable game
files or authenticated agents. Use this environment only for local UI review.
The feeder checks the fixture marker and restores only its known observations,
including the deliberately offline host. Stop both terminals when finished and
remove the generated fixture directory.

## Checks and builds

```sh
npm run check
npm test
npm run test:agent
npm run build
```

The `package.json` overrides keep SvelteKit 2 on patched `devalue` 5.x (5.9.4 or
newer) and `cookie` 0.7.x releases until its dependency ranges include those fixes.
These dependencies ship in the server bundle even though SvelteKit is a
development dependency.
Dependency security checks must include development dependencies: use `npm audit`
without `--omit=dev`.

`npm run dev` and `npm run build` regenerate the downloadable agent, installer,
standalone updater and checksum manifest. Builds also bundle report workers
under `build/reports` and launch preparation under `build/launch`. Development
and test commands compile those workers automatically. Restart `npm run dev`
after changing agent or worker code, or run `npm run agent:package` to refresh
only the downloadable agent assets while the development server is running.

Automated tests simulate Docker and game hosts. Real Debian installation and
T5/T6 server validation use the [host acceptance checklist](../guides/native-hosts.md#development-and-validation).

## Code layout

Paths below are relative to the repository root.

| Path | Responsibility |
| --- | --- |
| `src/routes` | Pages and API endpoints |
| `src/lib/ui` | Shared components, stores and [UI patterns](ui-patterns.md) |
| `src/lib/server/hosts` | Enrollment, readiness and command transport |
| `src/lib/server/backends` | Instance identity and remote file access |
| `src/lib/server/provisioning/native.ts` | Lifecycle and campaign queues connected to agents |
| `src/lib/server/game-files` | Immutable uploads and streaming distribution |
| `src/lib/server/db` | SQLite schema, migrations and shared artifact storage |
| `src/lib/server/reports` | Metrics, leaderboard and capacity workers |
| `agent` | Python service, installer, packaging and tests |

## Performance diagnostics

Set `EC_PERF_LOG=true` to log slow requests, report computation/cache timings and
event-loop delay. Responses expose `Server-Timing`; browser navigation records an
`ec:navigation` performance measure. Report workers use separate read-only SQLite
connections and bounded caches. Metrics snapshots can be reused for up to 30
seconds when their inputs are unchanged.

Benchmark the leaderboard against an isolated synthetic database:

```sh
npm run benchmark:leaderboard
npm run benchmark:leaderboard -- --scenario all --baseline-ref <commit>
```

The comparison checks response parity and warm query timings.

Set `EC_SLOW_SQL_MS=20` to log runtime statements taking at least 20 ms. Logging
omits bound values and string literals and is disabled by default. Capture timings
under the same fleet load before and after database changes. Inspect database
size, large tables/indexes, WAL size and free pages inside the controller with:

```sh
npm run diagnostics:database -- /data/cherry.db
```

This opens an existing database read-only without running migrations. Substitute
`./data/cherry.db` for a default local installation.

## Storage and upgrades

Game-file uploads live in dedicated filesystem storage. Script, plugin and
configuration artifact bodies are stored once by SHA-256 in SQLite; versions,
staging plans, rollback snapshots and launch commands reference those shared
bytes. Startup transactionally backfills older rows. Current agents fetch
assigned artifacts through an authenticated endpoint and verify their hashes;
older agents receive inline content until they update.

Back up the persistent data directory before upgrading. Startup logs show
migration phases, elapsed time and resident memory. Completed migrations are
retained across restarts; interrupted transactions roll back and retry on startup.
The one-time vacuum needs disk space for a temporary database copy. **Do not
remove `cherry.db-wal` to recover from an interrupted upgrade.**

The controller uses a 128 MiB page cache; each reader worker uses 32 MiB.
Main-database migrations use 16 MiB with memory mapping disabled. Temporary
storage stays disk-backed. Containers use `/data/.sqlite-tmp` for scratch files;
override `SQLITE_TMPDIR` only with another writable, disk-backed directory.

For a local data disk, `EC_SQLITE_MMAP_MB=256` enables memory-mapped reads; leave it
at zero for network mounts. Query statistics are optimized at startup and hourly.
WAL reuse is capped at 64 MiB after checkpoints, though active transactions can
temporarily grow it larger.