Move fleet retention and concurrency controls into Settings

Persist settings in SQLite, import legacy environment values once, and apply changes to cleanup and deployment queues without restarting.

Closes #51

.env.example

@@ -9,16 +9,7 @@ BETA_ARCHIVE_DIR=/root/zombies/beta-archive
 # SQLite location
 DB_PATH=/data/cherry.db
 
-# Retention (days) for deployment rows and blob diffs
+# Configure campaign retention and restart/startup concurrency in Settings → Fleet settings.
-DEPLOY_KEEP_DAYS=7
-
-# Fleet-wide rolling restart limit, shared by deployments and betas.
-# A slot stays occupied until Komodo and RCON confirm recovery.
-DEPLOYMENT_RESTART_CONCURRENCY=1
-
-# New-server startup limit, shared by all collections and single-server launches.
-# A slot stays occupied through Komodo, RCON, and selected-map verification.
-DEPLOYMENT_CREATE_CONCURRENCY=1
 
 # Komodo
 KOMODO_ADDRESS=https://komodo.example.internal

README.md

@@ -98,6 +98,30 @@ submitted recovery continue. Each capability retains its own policy. Automatic
 updates remain disabled until explicitly enabled. See [Maintenance](docs/maintenance.md)
 for verification, retry, scheduling, and compatibility details.
 
+## Fleet settings
+
+**Settings → Fleet settings** controls campaign history retention (default 7 days),
+concurrent restarts (default 1), and concurrent server startups (default 1).
+Values are positive whole numbers; retention supports up to 36500 days. Settings
+are saved in SQLite for all users and apply at the next queue or cleanup check
+without restarting the app. Lowering a limit leaves active operations running
+and holds queued work until capacity is available. Collection previews and
+progress pages display the current startup limit.
+
+Retention removes eligible old campaign records and file diffs. Pending, failed,
+held, and health-recovery-linked campaigns remain protected; Provisioning history
+is retained. Shortening retention can remove older history on the next sweep.
+
+On the first startup after upgrading, existing `DEPLOY_KEEP_DAYS`,
+`DEPLOYMENT_RESTART_CONCURRENCY`, and `DEPLOYMENT_CREATE_CONCURRENCY` environment
+values initialize the saved settings. Keep custom values for that first startup;
+afterward, the database values take precedence and those variables can be removed.
+The supplied Compose and Komodo examples use the built-in defaults.
+
+`GET /api/settings` reads the three values (`deployKeepDays`, `restartConcurrency`,
+`createConcurrency`); `PUT /api/settings` saves all three together as JSON.
+Invalid or incomplete requests return HTTP 400 without changing saved settings.
+
 ## How restarts stay safe
 
 Betas support configurable gameplay, participation, stability and manual-review
@@ -107,7 +131,7 @@ requirements, saved presets, coverage rules and staged waves. See
 File changes are staged sequentially. Their activation uses rolling restarts:
 by default only **one server across the fleet** can restart or recover at a
 time. Promotions, rollbacks, drift reconciliation, and beta restarts share this
-limit. Set `DEPLOYMENT_RESTART_CONCURRENCY` to a positive integer to raise it.
+limit. Adjust **Concurrent restarts** in **Settings → Fleet settings** to raise it.
 
 The Provisioning worker runs every 15s. Campaigns, betas, and maintenance submit
 durable restart requests and read their results. Provisioning checks recovering
@@ -164,8 +188,8 @@ launches. A startup slot stays occupied until Komodo is healthy and RCON or
 fresh telemetry reports the selected map; an unready server holds the queue until it recovers
 or is removed. The queue survives application restarts. The collection preview
 shows the startup limit, and its progress page shows active startups and each
-server's place in the shared queue. Set `DEPLOYMENT_CREATE_CONCURRENCY` to a
+server's place in the shared queue. Adjust **Concurrent server startups** in
-positive integer to increase the limit. Each worker pass starts at most the
+**Settings → Fleet settings** to increase the limit. Each worker pass starts at most the
 available number of slots; immediate failures do not drain the remaining queue.
 Existing single-server collections can be expanded with **Edit**. Single-server
 launches and launching again from recorded recipes remain available.

docker-compose.yml

@@ -31,9 +31,6 @@ services:
       REL_DEST: runtime/plutonium/storage/t5/raw/scripts/sp
       BETA_ARCHIVE_DIR: /root/zombies/beta-archive
       DB_PATH: /data/cherry.db
-      DEPLOY_KEEP_DAYS: "7"
-      DEPLOYMENT_RESTART_CONCURRENCY: ${DEPLOYMENT_RESTART_CONCURRENCY:-1}
-      DEPLOYMENT_CREATE_CONCURRENCY: ${DEPLOYMENT_CREATE_CONCURRENCY:-1}
       KOMODO_ADDRESS: ${KOMODO_ADDRESS}
       KOMODO_API_KEY: ${KOMODO_API_KEY}
       KOMODO_API_SECRET: ${KOMODO_API_SECRET}

docs/deployment-blueprints.md

@@ -95,7 +95,7 @@ the current shortfall too.
 All collections and individual launches share one startup queue, ordered by request
 time. By default, one server starts at a time. Its startup slot remains occupied
 until Komodo is healthy, RCON answers, and the reported map matches the selection.
-Only then can the next queued server start. Set `DEPLOYMENT_CREATE_CONCURRENCY` to
+Only then can the next queued server start. Set **Concurrent server startups** in **Settings → Fleet settings** to
 a positive integer to allow more simultaneous startups. Each worker pass admits
 at most the available slots, even if an attempt fails immediately.
 

komodo/stack.toml

@@ -27,8 +27,6 @@ PLUTAINER_T6_GAMEFILES_VOLUME = "t6_game_files"
 REL_DEST         = "runtime/plutonium/storage/t5/raw/scripts/sp"
 BETA_ARCHIVE_DIR = "/root/zombies/beta-archive"
 DB_PATH          = "/data/cherry.db"
-DEPLOY_KEEP_DAYS = "7"
-DEPLOYMENT_RESTART_CONCURRENCY = "1"
 
 # secrets: bind via Komodo's env-file or secret store
 KOMODO_ADDRESS   = "https://komodo.example.internal"

src/lib/fleet-settings.ts

@@ -0,0 +1,34 @@
+export interface FleetSettings {
+    deployKeepDays: number;
+    restartConcurrency: number;
+    createConcurrency: number;
+}
+
+export const defaultFleetSettings: FleetSettings = {
+    deployKeepDays: 7,
+    restartConcurrency: 1,
+    createConcurrency: 1
+};
+
+export const fleetSettingLabels: Record<keyof FleetSettings, string> = {
+    deployKeepDays: 'Campaign history retention',
+    restartConcurrency: 'Concurrent restarts',
+    createConcurrency: 'Concurrent server startups'
+};
+
+export function validateFleetSettings(input: unknown): FleetSettings {
+    if (!input || typeof input !== 'object' || Array.isArray(input)) throw new Error('Provide fleet settings.');
+    const values = input as Record<string, unknown>;
+    for (const key of Object.keys(values)) {
+        if (!Object.hasOwn(defaultFleetSettings, key)) throw new Error(`Unknown setting: ${key}`);
+    }
+    for (const key of Object.keys(defaultFleetSettings) as (keyof FleetSettings)[]) {
+        const value = values[key];
+        if (typeof value !== 'number' || !Number.isSafeInteger(value) || value < 1) {
+            throw new Error(`${fleetSettingLabels[key]} must be a positive whole number.`);
+        }
+        if (key === 'deployKeepDays' && value > 36500) throw new Error('Campaign history retention must be at most 36500 days.');
+    }
+    return { deployKeepDays: values.deployKeepDays as number, restartConcurrency: values.restartConcurrency as number,
+        createConcurrency: values.createConcurrency as number };
+}

src/lib/server/config.ts

@@ -15,12 +15,6 @@ function num(name: string, fallback: number): number {
     return n;
 }
 
-function positiveInt(name: string, fallback: number): number {
-    const value = num(name, fallback);
-    if (!Number.isSafeInteger(value) || value < 1) throw new Error(`env ${name} must be a positive integer`);
-    return value;
-}
-
 export const config = {
     serversRoot: req('SERVERS_ROOT', '/root/zombies/t5'),
     t6ServersRoot: env.T6_SERVERS_ROOT ?? join(dirname(req('SERVERS_ROOT', '/root/zombies/t5')), 't6'),
@@ -38,9 +32,6 @@ export const config = {
     // Cleared per-beta on promote/rollback.
     betaArchiveDir: req('BETA_ARCHIVE_DIR', '/root/zombies/beta-archive'),
     dbPath: req('DB_PATH', './data/cherry.db'),
-    deployKeepDays: num('DEPLOY_KEEP_DAYS', 7),
-    restartConcurrency: positiveInt('DEPLOYMENT_RESTART_CONCURRENCY', 1),
-    createConcurrency: positiveInt('DEPLOYMENT_CREATE_CONCURRENCY', 1),
     httpTimeoutMs: num('HTTP_TIMEOUT_MS', 10_000),
     komodo: {
         address: env.KOMODO_ADDRESS ?? '',

src/lib/server/db/fleet-settings.ts

@@ -0,0 +1,20 @@
+import type Database from 'better-sqlite3';
+import { defaultFleetSettings, validateFleetSettings, type FleetSettings } from '../../fleet-settings.js';
+
+export const FLEET_SETTINGS_KEY = 'fleet_settings';
+
+/** Import legacy environment values once; persisted settings win on every later boot. */
+export function initializeFleetSettings(handle: Database.Database): void {
+    if (handle.prepare('SELECT 1 FROM settings WHERE key = ?').get(FLEET_SETTINGS_KEY)) return;
+    const values = { ...defaultFleetSettings };
+    const legacy: Record<keyof FleetSettings, string> = {
+        deployKeepDays: 'DEPLOY_KEEP_DAYS', restartConcurrency: 'DEPLOYMENT_RESTART_CONCURRENCY',
+        createConcurrency: 'DEPLOYMENT_CREATE_CONCURRENCY'
+    };
+    for (const key of Object.keys(legacy) as (keyof FleetSettings)[]) {
+        const raw = process.env[legacy[key]];
+        if (raw) values[key] = Number(raw);
+    }
+    handle.prepare('INSERT OR IGNORE INTO settings (key, value) VALUES (?, ?)')
+        .run(FLEET_SETTINGS_KEY, JSON.stringify(validateFleetSettings(values)));
+}

src/lib/server/db/index.ts

@@ -6,6 +6,7 @@ import schemaSql from './schema.sql?raw';
 import { requirement, type BetaPlan, type BetaRequirement } from '../../beta-plan.js';
 import { emptyMetrics } from '../fleet/beta-evaluation.js';
 import { backfillLeaderboard } from '../fleet/leaderboard-ingest.js';
+import { initializeFleetSettings } from './fleet-settings.js';
 
 let _db: Database.Database | null = null;
 
@@ -345,6 +346,7 @@ export function db(): Database.Database {
         migrateGameTelemetry(handle);
         migrateTable(handle, 'leaderboard_matches', [{ name: 'start_players', ddl: 'ALTER TABLE leaderboard_matches ADD COLUMN start_players INTEGER' }]);
         backfillLeaderboard(handle);
+        initializeFleetSettings(handle);
     } catch (err) {
         handle.close();
         throw err;

src/lib/server/fleet/collections.ts

@@ -1,3 +1,4 @@
+import { fleetSettings } from '../settings.js';
 import { randomUUID } from 'node:crypto';
 import type { CollectionDetail, CollectionMode, CollectionPreview, CollectionSummary } from '$lib/deployment-plan.js';
 import { db } from '../db/index.js';
@@ -106,7 +107,7 @@ export function previewCollection(blueprintId: unknown, mode: unknown = 'fill'):
         db().prepare('DELETE FROM server_collection_previews WHERE expires_at < ? AND collection_id IS NULL').run(new Date().toISOString());
         db().prepare('INSERT INTO server_collection_previews (id, expires_at, blueprint_name, entries, servers, mode, fleet_snapshot) VALUES (?, ?, ?, ?, ?, ?, ?)')
             .run(id, expiresAt, blueprintName, JSON.stringify(entries), JSON.stringify(servers), mode, mode === 'exact' ? JSON.stringify(fleetSnapshot()) : null);
-        return { id, expiresAt, blueprintName, mode, retainedServers, removals, launchConcurrency: config.createConcurrency, servers, blockers, ready: blockers.length === 0,
+        return { id, expiresAt, blueprintName, mode, retainedServers, removals, launchConcurrency: fleetSettings().createConcurrency, servers, blockers, ready: blockers.length === 0,
             entries: entries.map((entry, index) => ({ count: entry.count, ...counts[index], recipe: summarizeRecipe(entry.recipe) })) };
     })();
 }
@@ -216,6 +217,6 @@ export function getCollection(id: string): CollectionDetail | null {
     const queue = creationQueue();
     const positions = new Map(queue.queued.map((job, index) => [job.id, index + 1]));
     const stored = db().prepare('SELECT retained_servers FROM server_launch_collections WHERE id = ?').get(id) as { retained_servers: string };
-    return { collection, retainedServers: JSON.parse(stored.retained_servers), queue: { concurrency: config.createConcurrency, active: queue.active, queued: queue.queued.length },
+    return { collection, retainedServers: JSON.parse(stored.retained_servers), queue: { concurrency: fleetSettings().createConcurrency, active: queue.active, queued: queue.queued.length },
         servers: servers.map(server => ({ ...server, queuePosition: positions.get(server.id) ?? null })) };
 }

src/lib/server/pipeline/sweep.ts

@@ -1,4 +1,4 @@
-import { config } from '../config.js';
+import { fleetSettings } from '../settings.js';
 import { db } from '../db/index.js';
 import { emit } from './bus.js';
 import { requestRestart } from '../provisioning/index.js';
@@ -44,7 +44,7 @@ async function runSweep(): Promise<SweepResult> {
     db().prepare(`DELETE FROM deployments WHERE started_at < ?
         AND NOT EXISTS (SELECT 1 FROM watchdog_actions WHERE campaign_id = deployments.id) AND NOT EXISTS (
         SELECT 1 FROM deployment_servers WHERE deployment_id = deployments.id AND restart_state IN ('pending','failed','held'))`)
-        .run(new Date(Date.now() - config.deployKeepDays * 86_400_000).toISOString());
+        .run(new Date(Date.now() - fleetSettings().deployKeepDays * 86_400_000).toISOString());
     emit({ type: 'sweep-tick', frozen: false });
     return result;
 }

src/lib/server/provisioning/instances.ts

@@ -1,3 +1,4 @@
+import { fleetSettings } from '../settings.js';
 import { probePlayerCount, probeServerStatus } from '../fleet/status-evidence.js';
 import { beginDeployment, deploymentStage, finishDeployment, recordDeploymentEvent, REMOVING_SERVER, resumeCleanup, resumeVerification, claimRemoval } from './history.js';
 import { randomUUID } from 'node:crypto';
@@ -258,7 +259,7 @@ export async function rebuildServer(id: string): Promise<void> {
     const job = db().prepare(`SELECT j.*, d.stage FROM server_creation_jobs j JOIN server_deployments d ON d.id = j.deployment_id
         WHERE d.id = ? AND d.action = 'rebuild' AND d.state = 'waiting'`).get(id) as (CreationJob & { stage: string }) | undefined;
     if (!job) return;
-    if (creating || destroying.size || creationQueue().active.length >= config.createConcurrency) {
+    if (creating || destroying.size || creationQueue().active.length >= fleetSettings().createConcurrency) {
         // The cleanup checkpoint must survive capacity waits and app restarts.
         if (job.stage !== REMOVING_SERVER) deploymentStage(id, 'Waiting for a server creation slot before rebuilding');
         return;
@@ -775,7 +776,7 @@ async function runCreations(): Promise<CreationOutcomes> {
     // Snapshot the budget once. Fast success/failure must not turn a single
     // worker pass into an unbounded stream of expensive container startups.
     const queue = creationQueue();
-    const available = Math.max(0, config.createConcurrency - queue.active.length);
+    const available = Math.max(0, fleetSettings().createConcurrency - queue.active.length);
     const eligible = queue.queued.filter(({ id }) => {
         const blocker = creationBlocker(id);
         if (blocker) deploymentStage(id, blocker);
@@ -785,7 +786,7 @@ async function runCreations(): Promise<CreationOutcomes> {
         if (creating || destroying.size > 0) break;
         // API calls can cancel a selected job or resume an older startup while
         // a previous provision awaits Komodo. Recheck without expanding this pass.
-        if (creationQueue().active.length >= config.createConcurrency) break;
+        if (creationQueue().active.length >= fleetSettings().createConcurrency) break;
         if (!db().prepare("SELECT 1 FROM server_deployments WHERE id = ? AND action IN ('create', 'rebuild') AND state = 'queued'").get(id)) continue;
         const blocker = creationBlocker(id);
         if (blocker) {

src/lib/server/provisioning/restarts.ts

@@ -1,4 +1,4 @@
-import { config } from '../config.js';
+import { fleetSettings } from '../settings.js';
 import { db } from '../db/index.js';
 import { komodoInspectHealthy, komodoRestart, komodoRuntimeAction } from '../integrations/komodo.js';
 import { probeServerStatus } from '../fleet/status-evidence.js';
@@ -49,7 +49,7 @@ export function rolloutWaitReason(server: string, operationId = ''): string | nu
         return `Waiting for the previous restart of this server to recover${slot.error ? `: ${slot.error}` : ''}`;
     }
     const { n } = db().prepare('SELECT COUNT(*) AS n FROM restart_rollout WHERE healthy_at IS NULL').get() as { n: number };
-    return n - (ownHandoff && slot?.healthy_at === null ? 1 : 0) >= config.restartConcurrency ? 'Queued for a rolling restart; waiting for another server to recover' : null;
+    return n - (ownHandoff && slot?.healthy_at === null ? 1 : 0) >= fleetSettings().restartConcurrency ? 'Queued for a rolling restart; waiting for another server to recover' : null;
 }
 
 export type RolloutRestartResult =

src/lib/server/settings.ts

@@ -0,0 +1,14 @@
+import { validateFleetSettings, type FleetSettings } from '../fleet-settings.js';
+import { db } from './db/index.js';
+import { FLEET_SETTINGS_KEY } from './db/fleet-settings.js';
+
+export function fleetSettings(): FleetSettings {
+    const row = db().prepare('SELECT value FROM settings WHERE key = ?').get(FLEET_SETTINGS_KEY) as { value: string };
+    return JSON.parse(row.value);
+}
+
+export function saveFleetSettings(input: unknown): FleetSettings {
+    const settings = validateFleetSettings(input);
+    db().prepare('UPDATE settings SET value = ? WHERE key = ?').run(JSON.stringify(settings), FLEET_SETTINGS_KEY);
+    return settings;
+}

src/routes/api/settings/+server.ts

@@ -0,0 +1,12 @@
+import { json } from '@sveltejs/kit';
+import { validateFleetSettings } from '$lib/fleet-settings.js';
+import { fleetSettings, saveFleetSettings } from '$lib/server/settings.js';
+
+export function GET() { return json(fleetSettings()); }
+
+export async function PUT({ request }: { request: Request }) {
+    let settings;
+    try { settings = validateFleetSettings(await request.json()); }
+    catch (error) { return json({ error: (error as Error).message }, { status: 400 }); }
+    return json(saveFleetSettings(settings));
+}

src/routes/settings/+page.svelte

@@ -3,14 +3,71 @@
     import { availableTimeZones, DEFAULT_TIME_ZONE } from '$lib/time-zone.js';
     import { onMount } from 'svelte';
     import SectionHeader from '$ui/components/primitives/SectionHeader.svelte';
+    import Button from '$ui/components/primitives/Button.svelte';
+    import type { FleetSettings } from '$lib/fleet-settings.js';
     let exampleDate = $state<Date | null>(null);
+    let settings = $state<FleetSettings | null>(null);
+    let loading = $state(true);
+    let saving = $state(false);
+    let error = $state('');
+    let notice = $state('');
     const zones = $derived([...new Set([...availableTimeZones(), $timeZone])].sort());
-    onMount(() => { exampleDate = new Date(); });
+    onMount(() => { exampleDate = new Date(); void loadSettings(); });
+
+    async function loadSettings() {
+        loading = true; error = '';
+        try {
+            const response = await fetch('/api/settings');
+            if (!response.ok) throw new Error('Could not load fleet settings.');
+            settings = await response.json();
+        } catch (cause) { error = (cause as Error).message; }
+        finally { loading = false; }
+    }
+
+    async function saveSettings(event: SubmitEvent) {
+        event.preventDefault();
+        saving = true; error = ''; notice = '';
+        try {
+            const response = await fetch('/api/settings', { method: 'PUT', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(settings) });
+            const result = await response.json();
+            if (!response.ok) throw new Error(result.error ?? 'Could not save fleet settings.');
+            settings = result;
+            notice = 'Fleet settings saved. Changes apply to the next queue or cleanup check.';
+        } catch (cause) { error = (cause as Error).message; }
+        finally { saving = false; }
+    }
 </script>
 
 <svelte:head><title>Settings · Electric Cherry</title></svelte:head>
 <section class="page">
-    <SectionHeader title="Settings" subtitle="Choose how dates and times appear across Electric Cherry." />
+    <SectionHeader title="Settings" subtitle="Manage fleet limits, history retention, and display preferences." />
+    <section class="guide" aria-labelledby="fleet-heading">
+        <h2 id="fleet-heading">Fleet settings</h2>
+        <p>Saved for everyone using this Electric Cherry instance. Changes apply without restarting the app.</p>
+        {#if error}<p class="ui-error" role="alert">{error}</p>{/if}
+        {#if notice}<p role="status">{notice}</p>{/if}
+        {#if loading}
+            <p role="status">Loading fleet settings…</p>
+        {:else if settings}
+            <form onsubmit={saveSettings}>
+                <fieldset disabled={saving}>
+                    <label for="retention">Campaign history retention (days)</label>
+                    <input id="retention" type="number" min="1" max="36500" step="1" required bind:value={settings.deployKeepDays} aria-describedby="retention-help" />
+                    <p id="retention-help">Default: 7 days. Removes eligible old campaign records and file diffs on the next cleanup check. Pending, failed, held, and health-recovery-linked campaigns are kept. Provisioning history is retained.</p>
+                    <label for="restarts">Concurrent restarts</label>
+                    <input id="restarts" type="number" min="1" step="1" required bind:value={settings.restartConcurrency} aria-describedby="restarts-help" />
+                    <p id="restarts-help">Default: 1 across the fleet. Campaigns, betas, maintenance, and lifecycle actions share recovery slots until container and game health are verified.</p>
+                    <label for="startups">Concurrent server startups</label>
+                    <input id="startups" type="number" min="1" step="1" required bind:value={settings.createConcurrency} aria-describedby="startups-help" />
+                    <p id="startups-help">Default: 1 across collections and individual launches. Each startup holds its slot through container, game health, and selected-map verification.</p>
+                    <p>Lowering a limit lets active operations finish; queued work waits for capacity. Player safety checks still apply.</p>
+                    <Button type="submit" disabled={saving}>{saving ? 'Saving…' : 'Save fleet settings'}</Button>
+                </fieldset>
+            </form>
+        {:else}
+            <Button onclick={loadSettings}>Retry loading</Button>
+        {/if}
+    </section>
     <section class="guide" aria-labelledby="display-heading">
         <h2 id="display-heading">Display preferences</h2>
         <label class="timezone">Timezone
@@ -32,4 +89,7 @@
     p { font-size: var(--text-sm); color: var(--text-secondary); line-height: 1.65; }
     .timezone { display: grid; gap: 8px; margin-top: 16px; font-size: var(--text-sm); max-width: 420px; }
     .timezone select { width: 100%; padding: 10px 12px; font: inherit; color: var(--text-primary); background: var(--surface-sunken); border: 1px solid var(--border-control); border-radius: var(--radius-sm); }
+    fieldset { border: 0; padding: 0; margin: 20px 0 0; min-width: 0; }
+    fieldset label { display: block; margin: 20px 0 8px; font-size: var(--text-sm); }
+    fieldset input { width: 160px; padding: 10px 12px; font: inherit; color: var(--text-primary); background: var(--surface-sunken); border: 1px solid var(--border-control); border-radius: var(--radius-sm); }
 </style>

tests/unit/creation-queue.test.ts

@@ -66,6 +66,44 @@ async function states() {
 }
 
 describe('shared creation queue', () => {
+    it('rechecks a lowered limit after awaiting a startup in the same pass', async () => {
+        await queueServers(3);
+        const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
+        saveFleetSettings({ ...fleetSettings(), createConcurrency: 3 });
+        const { processCreations } = await import('../../src/lib/server/fleet/create.js');
+        const { komodoCreateDeployment } = await import('../../src/lib/server/integrations/komodo.js');
+        vi.mocked(komodoCreateDeployment).mockImplementationOnce(async () => {
+            saveFleetSettings({ ...fleetSettings(), createConcurrency: 1 });
+            return { ok: true };
+        });
+        await processCreations();
+        expect(komodoCreateDeployment).toHaveBeenCalledTimes(1);
+        expect(await states()).toEqual(['awaiting-health', 'queued', 'queued']);
+    });
+
+    it('applies live increases and decreases while preserving active startups', async () => {
+        const jobs = await queueServers(4);
+        const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
+        const { processCreations } = await import('../../src/lib/server/fleet/create.js');
+        const { komodoInspectHealthy, komodoCreateDeployment } = await import('../../src/lib/server/integrations/komodo.js');
+        vi.mocked(komodoInspectHealthy).mockResolvedValue({ ok: false, reason: 'still starting' });
+        await processCreations();
+        saveFleetSettings({ ...fleetSettings(), createConcurrency: 3 });
+        await processCreations();
+        expect(await states()).toEqual(['awaiting-health', 'awaiting-health', 'awaiting-health', 'queued']);
+        saveFleetSettings({ ...fleetSettings(), createConcurrency: 1 });
+        await processCreations();
+        expect(komodoCreateDeployment).toHaveBeenCalledTimes(3);
+        expect(await states()).toEqual(['awaiting-health', 'awaiting-health', 'awaiting-health', 'queued']);
+        vi.mocked(komodoInspectHealthy).mockImplementation(async server => server === jobs[2].name
+            ? { ok: false, reason: 'still starting' } : { ok: true, state: 'running' });
+        await processCreations();
+        expect(await states()).toEqual(['done', 'done', 'awaiting-health', 'queued']);
+        vi.mocked(komodoInspectHealthy).mockResolvedValue({ ok: true, state: 'running' });
+        await processCreations();
+        expect(await states()).toEqual(['done', 'done', 'done', 'awaiting-health']);
+    });
+
     it('starts only one server per pass and holds its slot until a later health check', async () => {
         const jobs = await queueServers(3);
         const { processCreations } = await import('../../src/lib/server/fleet/create.js');
@@ -345,13 +383,14 @@ describe('shared creation queue', () => {
 describe('creation concurrency configuration', () => {
     it('defaults to one startup', async () => {
         vi.stubEnv('DEPLOYMENT_CREATE_CONCURRENCY', undefined);
-        const { config } = await import('../../src/lib/server/config.js');
+        const { fleetSettings } = await import('../../src/lib/server/settings.js');
-        expect(config.createConcurrency).toBe(1);
+        expect(fleetSettings().createConcurrency).toBe(1);
     });
 
     it.each(['0', '-1', '1.5', 'invalid', 'Infinity', '9007199254740992'])('rejects invalid limit %s', async value => {
         vi.stubEnv('DEPLOYMENT_CREATE_CONCURRENCY', value);
-        await expect(import('../../src/lib/server/config.js')).rejects.toThrow(/DEPLOYMENT_CREATE_CONCURRENCY/);
+        const { fleetSettings } = await import('../../src/lib/server/settings.js');
+        expect(() => fleetSettings()).toThrow(/Concurrent server startups/);
         vi.stubEnv('DEPLOYMENT_CREATE_CONCURRENCY', '1');
     });
 });

tests/unit/deployment-collections.test.ts

@@ -51,6 +51,16 @@ function keysFor(preview: CollectionPreview) {
 }
 
 describe('multi-server blueprints', () => {
+    it('shows the saved startup limit in new previews and existing collection progress', async () => {
+        const { blueprint, preview } = await previewLineup();
+        const { queueCollection, getCollection, previewCollection } = await import('../../src/lib/server/fleet/collections.js');
+        const result = queueCollection({ previewId: preview.id, keys: keysFor(preview) });
+        const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
+        saveFleetSettings({ ...fleetSettings(), createConcurrency: 3 });
+        expect(previewCollection(blueprint.id).launchConcurrency).toBe(3);
+        expect(getCollection(result.collectionId!)?.queue.concurrency).toBe(3);
+    });
+
     it('saves a mixed-game lineup and previews unique slots and global ports without reserving them', async () => {
         const { blueprint, preview } = await previewLineup();
         const { db } = await import('../../src/lib/server/db/index.js');

tests/unit/fleet-settings.test.ts

@@ -0,0 +1,81 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { newFixture, type Fixture } from '../helpers/fixture.js';
+
+let fixture: Fixture;
+const defaults = { deployKeepDays: 7, restartConcurrency: 1, createConcurrency: 1 };
+beforeEach(async () => {
+    vi.resetModules();
+    fixture = await newFixture([]);
+    for (const key of ['DEPLOY_KEEP_DAYS', 'DEPLOYMENT_RESTART_CONCURRENCY', 'DEPLOYMENT_CREATE_CONCURRENCY']) vi.stubEnv(key, undefined);
+});
+afterEach(async () => { await fixture.cleanup(); vi.unstubAllEnvs(); });
+
+describe('fleet settings API and persistence', () => {
+    it('returns defaults and saves all settings across a module reload and database reopen', async () => {
+        const { GET, PUT } = await import('../../src/routes/api/settings/+server.js');
+        expect(await GET().json()).toEqual(defaults);
+        const saved = { deployKeepDays: 30, restartConcurrency: 3, createConcurrency: 2 };
+        const response = await PUT({ request: new Request('http://localhost/api/settings', { method: 'PUT', body: JSON.stringify(saved) }) });
+        expect(response.status).toBe(200);
+        expect(await response.json()).toEqual(saved);
+        const { closeDb } = await import('../../src/lib/server/db/index.js');
+        closeDb(); vi.resetModules();
+        // Once saved, even invalid legacy environment values are irrelevant.
+        vi.stubEnv('DEPLOY_KEEP_DAYS', '-1');
+        vi.stubEnv('DEPLOYMENT_RESTART_CONCURRENCY', 'bad');
+        vi.stubEnv('DEPLOYMENT_CREATE_CONCURRENCY', '9');
+        const reopened = await import('../../src/routes/api/settings/+server.js');
+        expect(await reopened.GET().json()).toEqual(saved);
+    });
+
+    it('imports legacy values into an existing database once and preserves unrelated settings', async () => {
+        const { db, closeDb } = await import('../../src/lib/server/db/index.js');
+        db().prepare("DELETE FROM settings WHERE key = 'fleet_settings'").run();
+        db().prepare("INSERT INTO settings (key, value) VALUES ('unrelated', 'true')").run();
+        closeDb();
+        vi.stubEnv('DEPLOY_KEEP_DAYS', '14');
+        vi.stubEnv('DEPLOYMENT_RESTART_CONCURRENCY', '4');
+        vi.stubEnv('DEPLOYMENT_CREATE_CONCURRENCY', '2');
+        const { fleetSettings } = await import('../../src/lib/server/settings.js');
+        expect(fleetSettings()).toEqual({ deployKeepDays: 14, restartConcurrency: 4, createConcurrency: 2 });
+        closeDb();
+        vi.stubEnv('DEPLOY_KEEP_DAYS', '7');
+        expect(fleetSettings().deployKeepDays).toBe(14);
+        expect(db().prepare("SELECT value FROM settings WHERE key = 'unrelated'").get()).toEqual({ value: 'true' });
+    });
+
+    it.each([
+        'null', '[]', '{}', '{',
+        ...['deployKeepDays', 'restartConcurrency', 'createConcurrency'].flatMap(key =>
+            [0, -1, 1.5, '2', true, null, 9007199254740992].map(value => JSON.stringify({ ...defaults, [key]: value }))),
+        JSON.stringify({ ...defaults, deployKeepDays: 36501 }),
+        JSON.stringify({ ...defaults, unknown: 1 })
+    ])('rejects invalid input without partially saving: %s', async body => {
+        const { GET, PUT } = await import('../../src/routes/api/settings/+server.js');
+        const response = await PUT({ request: new Request('http://localhost/api/settings', { method: 'PUT', body }) });
+        expect(response.status).toBe(400);
+        expect(await response.json()).toHaveProperty('error');
+        expect(await GET().json()).toEqual(defaults);
+    });
+});
+
+it('applies saved retention to cleanup while protecting unfinished campaigns and provisioning history', async () => {
+    const { db } = await import('../../src/lib/server/db/index.js');
+    const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
+    const { sweep } = await import('../../src/lib/server/pipeline/sweep.js');
+    const old = new Date(Date.now() - 10 * 86400000).toISOString();
+    for (const id of ['completed', 'pending', 'failed', 'held']) {
+        db().prepare("INSERT INTO deployments (id, action, started_at) VALUES (?, 'test', ?)").run(id, old);
+        db().prepare('INSERT INTO deployment_servers (deployment_id, server, restart_state, files_changed) VALUES (?, ?, ?, 0)')
+            .run(id, 'server-' + id, id === 'completed' ? 'done' : id);
+    }
+    db().prepare("INSERT INTO server_deployments (id, server, action, state, stage, started_at, triggered_by) VALUES ('history', 'removed', 'destroy', 'done', 'Complete', ?, 'ui')").run(old);
+    saveFleetSettings({ ...fleetSettings(), deployKeepDays: 30 });
+    await sweep();
+    expect(db().prepare('SELECT id FROM deployments').all()).toHaveLength(4);
+    saveFleetSettings({ ...fleetSettings(), deployKeepDays: 7 });
+    await sweep();
+    expect(db().prepare('SELECT id FROM deployments ORDER BY id').all()).toEqual([{ id: 'failed' }, { id: 'held' }, { id: 'pending' }]);
+    expect(db().prepare("SELECT server FROM deployment_servers WHERE deployment_id = 'completed'").all()).toEqual([]);
+    expect(db().prepare('SELECT id FROM server_deployments').all()).toEqual([{ id: 'history' }]);
+});

tests/unit/restart-rollout.test.ts

@@ -438,10 +438,8 @@ describe('Provisioning ownership', () => {
 
 describe('rollout configuration', () => {
     it('honors a configured limit larger than one', async () => {
-        const { closeDb } = await import('../../src/lib/server/db/index.js');
+        const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
-        closeDb();
+        saveFleetSettings({ ...fleetSettings(), restartConcurrency: 2 });
-        vi.resetModules();
-        vi.stubEnv('DEPLOYMENT_RESTART_CONCURRENCY', '2');
         await deployment();
         const { sweep } = await import('../helpers/provisioning.js');
         const { komodoRestart } = await import('../../src/lib/server/integrations/komodo.js');
@@ -451,13 +449,20 @@ describe('rollout configuration', () => {
         expect(komodoRestart).toHaveBeenCalledTimes(2);
     });
 
-    it.each(['0', '-1', '1.5', 'abc'])('rejects invalid concurrency %s', async (value) => {
+    it('holds new restarts after lowering a live limit without releasing active slots', async () => {
-        const { closeDb } = await import('../../src/lib/server/db/index.js');
+        const { fleetSettings, saveFleetSettings } = await import('../../src/lib/server/settings.js');
-        closeDb();
+        const { sweep } = await import('../helpers/provisioning.js');
-        vi.resetModules();
+        const { komodoRestart } = await import('../../src/lib/server/integrations/komodo.js');
-        vi.stubEnv('DEPLOYMENT_RESTART_CONCURRENCY', value);
+        saveFleetSettings({ ...fleetSettings(), restartConcurrency: 2 });
-        await expect(import('../../src/lib/server/config.js')).rejects.toThrow(/DEPLOYMENT_RESTART_CONCURRENCY/);
+        await deployment();
-        vi.stubEnv('DEPLOYMENT_RESTART_CONCURRENCY', '1');
+        await sweep();
-        vi.resetModules();
+        saveFleetSettings({ ...fleetSettings(), restartConcurrency: 1 });
+        await sweep();
+        expect(komodoRestart).toHaveBeenCalledTimes(2);
+        const { db } = await import('../../src/lib/server/db/index.js');
+        expect(db().prepare('SELECT server FROM restart_rollout WHERE healthy_at IS NULL').all()).toHaveLength(2);
+        nextTick();
+        await sweep();
+        expect(komodoRestart).toHaveBeenCalledTimes(3);
     });
 });