# Provisioning

Open **Operations → Provisioning** for instance lifecycle work, blueprints, and
persistent activity. Provisioning owns the queues, safety checks, execution, and
recovery for all six actions. Create a server using **Create server**. For an existing
server, choose **Manage server** or open its **Server actions** menu.

| Action | Behavior |
| --- | --- |
| Create | Create a new instance using the latest promoted files and verify container health, RCON, and the selected map. |
| Start | Start the existing container and verify container health and RCON. |
| Restart | Wait until empty, restart the existing container, and verify container health and RCON. |
| Rebuild | Wait until empty, destroy the instance, delete all server files and its registration, then create a fresh instance in the same server slot and port. Verify container health, RCON, and map. |
| Stop | Wait until empty, stop the container, and verify that Komodo reports it stopped. Keep the instance for a later Start. |
| Destroy | Wait until empty, remove the container and Komodo deployment, then delete server files and registration. |

Servers linked to a running or ready-to-promote beta cannot be destroyed or
rebuilt. Promote or roll back the beta first, including paused or failed test
plans. Fleet removal and exact collection launches queue nothing if any server
selected for removal belongs to an active beta. Provisioning checks again before
teardown; already queued removals wait while their server belongs to an active
beta. Opening a beta also requires cancelling or finishing any pending destruction
or rebuild on its selected servers.

Rebuild requires confirmation because **no instance data is preserved**. Its fresh
files come from the latest promoted setup when creation begins, including changes
promoted while waiting. The independently saved key in **Keys** must exist before
teardown. Blueprint definitions, promoted artifacts, saved keys, and historical
activity belong to Electric Cherry, and are not instance files.

Rebuild reserves the original ID and port throughout teardown and creation, even
when a lower lobby is available. It shares the existing creation queue and startup
limit. A failed removal never proceeds to creation. Once Komodo deletion is
confirmed, failed local cleanup offers **Resume rebuild cleanup**; creation cannot
proceed until the old files are fully removed. An interrupted fresh startup can
resume health verification, without repeating teardown or creation. Inspect other
failures before making another request.

Start and Restart share the rolling restart limit with campaigns, betas, and
maintenance. A successful API reply is followed by a recovery delay and actual
container and game-status checks. Failed or ambiguous restart requests retain their recovery
slot and are not automatically replayed. Stop requires an explicit stopped container state; an
unreachable Komodo API is not evidence that a stop succeeded.

Disruptive operations require a fresh zero-player result from RCON or accepted
telemetry, unless Komodo positively reports a stopped container. Fresh occupied
telemetry vetoes a conflicting empty RCON reading. Reports must be received within
45 seconds; recovery verification also excludes reports from before the submitted
operation. Telemetry can verify the game's activity and selected map after Komodo
reports healthy. Runtime actions wait for conflicting
campaign, beta, maintenance, or provisioning work. Duplicate pending requests for
the same action return the existing operation; a different action returns a
conflict. Requests and events survive application restarts.

Campaigns, beta runs, and maintenance submit durable restart requests to
Provisioning and observe the resulting operation by ID. Their pages link to that
operation, and its history links back to the requesting work. Provisioning owns
the empty-server check, conflict handling, restart capacity, remote command, and
health verification. The callers retain file deployment, beta evidence, and
maintenance outcome verification. Jobs remain visible while waiting for players,
other work, or integration configuration.

[Maintenance](maintenance.md) also submits recovery operations and links each attempt
to its incident. A failed Maintenance restart can hand its reserved slot to a rebuild
after five minutes; unrelated recovery slots remain held. Drift campaigns follow
their linked rebuild without requesting another restart. Their file evidence and
history remain available after the instance is removed.

Campaign changes queued before submission can share one restart. Changes arriving
after submission require a new operation. A failed or interrupted request keeps
its original ID and never automatically replays. An explicit beta retry starts a
new attempt, preserving prior history; paused or closed beta work cannot start a
queued restart. Recovery of an already submitted restart continues even if its
requester closes. Upgrades adopt existing restart intent for observation only.

## Routes and compatibility

- `/provisioning`: active operations, blueprints, and history filtered by action.
- `/provisioning/:id`: operation stages, errors, timestamps, and activity.
- `/provisioning/collections/:id`: collection progress.
- `GET /api/provisioning`: lifecycle history (`deployments` response property).
- `POST /api/provisioning`: queue `{ "server": "t5_zombie_theater_1", "action": "start" }`.
  Supported actions are `start`, `restart`, `rebuild`, and `stop`. Returns HTTP 202
  with `deploymentId`. Create continues through `POST /api/servers`; Destroy uses
  `DELETE /api/servers/:name`.
- `/api/provisioning/:id`, `/:id/resume`, `/:id/cancel`, `/preview`, and
  `/house-party` preserve the existing operation, recovery, removal cancellation,
  launch preview, and fleet destruction APIs.

Old `/deployments` pages redirect to their `/provisioning` equivalents, preserving
query parameters. `/api/deployments` remains compatible, including old campaign
detail endpoints. No history or IDs are renamed or discarded.

Start and Stop use Komodo's corresponding runtime commands. Rebuild uses full
destruction and creation, not Komodo's redeploy command. See the
[Komodo execute API](https://docs.rs/komodo_client/latest/komodo_client/api/execute/index.html)
for the underlying container operations.