Support resumable game folder uploads and remove DLC declarations
This commit is contained in:
@@ -29,11 +29,13 @@ The default Compose data mount preserves the previous
|
|||||||
`EC_DATA_DIR` to your preferred persistent directory. Back up that directory,
|
`EC_DATA_DIR` to your preferred persistent directory. Back up that directory,
|
||||||
including SQLite metadata and published game-file storage.
|
including SQLite metadata and published game-file storage.
|
||||||
|
|
||||||
1. **Hosts → Add Host**: configure a Debian 12/13 amd64 machine and run its
|
1. **Library → Game Files**: select T5 or T6 and upload the complete local game
|
||||||
|
folder, including its DLC files. Name the library and version, then publish it.
|
||||||
|
Uploads work before any host is enrolled.
|
||||||
|
2. **Hosts → Add Host**: configure a Debian 12/13 amd64 machine and run its
|
||||||
generated enrollment command.
|
generated enrollment command.
|
||||||
2. Prepare a Plutainer runtime from the host page.
|
3. Prepare a Plutainer runtime from the host page and download the published
|
||||||
3. **Library → Game Files**: import existing T5/T6 files once, then cache a
|
game-file version to that host from **Game Files**.
|
||||||
published version on the destination host.
|
|
||||||
4. Add the exact Plutonium server keys in **Keys**, or supply missing keys when
|
4. Add the exact Plutonium server keys in **Keys**, or supply missing keys when
|
||||||
launching.
|
launching.
|
||||||
5. Launch a server or collection with an explicit host, game-file version and
|
5. Launch a server or collection with an explicit host, game-file version and
|
||||||
@@ -69,9 +71,12 @@ server ID and retained after removal.
|
|||||||
|
|
||||||
## Libraries and campaigns
|
## Libraries and campaigns
|
||||||
|
|
||||||
Game-file versions include game, version, DLC metadata, manifests, checksums and
|
Game-file versions contain the uploaded T5/T6 folder, with its directory layout
|
||||||
size. Actual game files use dedicated filesystem storage, never SQLite blobs.
|
and included DLC preserved. There is no separate DLC declaration. Versions record
|
||||||
Downloads resume, report progress and verify checksums before becoming usable.
|
the game, label, file manifest, checksums and size. Uploads use bounded chunks,
|
||||||
|
support pause/resume and verify checksums before publication. Actual game files
|
||||||
|
use dedicated filesystem storage, never SQLite blobs. Host downloads resume,
|
||||||
|
report progress and verify checksums before becoming usable.
|
||||||
Multiple instances on one host share read-only base files.
|
Multiple instances on one host share read-only base files.
|
||||||
|
|
||||||
Scripts, plugins, configuration baselines and overlays retain their versioning,
|
Scripts, plugins, configuration baselines and overlays retain their versioning,
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ guessable token. An unset setup token holds setup closed.
|
|||||||
| Raw configs, file diffs, logs and telemetry diagnostics | No | Yes | Yes |
|
| Raw configs, file diffs, logs and telemetry diagnostics | No | Yes | Yes |
|
||||||
| Deploy/start/stop/restart/rebuild/remove servers | No | Yes | Yes |
|
| Deploy/start/stop/restart/rebuild/remove servers | No | Yes | Yes |
|
||||||
| Scripts, plugins, configs, campaigns, betas and collections | Read summaries | Manage | Manage |
|
| Scripts, plugins, configs, campaigns, betas and collections | Read summaries | Manage | Manage |
|
||||||
| Host setup, enrollment, credentials, runtime preparation and game-file import/cache | Read status | Read status | Manage |
|
| Host setup, enrollment, credentials, runtime preparation and game-file upload/cache | Read status | Read status | Manage |
|
||||||
| Server keys and global settings/maintenance policies | No | No | Manage |
|
| Server keys and global settings/maintenance policies | No | No | Manage |
|
||||||
| Create users, change roles, disable accounts and reset passwords | No | No | Yes |
|
| Create users, change roles, disable accounts and reset passwords | No | No | Yes |
|
||||||
| Own password and sign out | Yes | Yes | Yes |
|
| Own password and sign out | Yes | Yes | Yes |
|
||||||
|
|||||||
@@ -131,7 +131,7 @@ the typed key so it cannot accidentally be saved against a different lobby.
|
|||||||
|
|
||||||
Komodo connection settings, target host, RCON configuration, config filename,
|
Komodo connection settings, target host, RCON configuration, config filename,
|
||||||
lobby availability, and port availability are checked locally. These checks do
|
lobby availability, and port availability are checked locally. These checks do
|
||||||
not probe the host or verify game/DLC files, images, Docker volumes, firewall
|
not probe the host or verify game files, images, Docker volumes, firewall
|
||||||
rules, or credentials. Container and RCON verification happen after provisioning.
|
rules, or credentials. Container and RCON verification happen after provisioning.
|
||||||
T6 variants retain their exact catalog slug and launch ruleset; RCON health checks
|
T6 variants retain their exact catalog slug and launch ruleset; RCON health checks
|
||||||
verify the engine map, not the variant's start location.
|
verify the engine map, not the variant's start location.
|
||||||
|
|||||||
+38
-8
@@ -17,9 +17,10 @@ the agent owns local execution and its durable operation journal.
|
|||||||
3. The installer validates the platform, installs Python 3 and an absent Docker
|
3. The installer validates the platform, installs Python 3 and an absent Docker
|
||||||
installation, verifies the downloaded agent SHA-256 over HTTPS, enrolls it and
|
installation, verifies the downloaded agent SHA-256 over HTTPS, enrolls it and
|
||||||
enables `electric-cherry-agent.service`. It needs no Git, Node.js or local build.
|
enables `electric-cherry-agent.service`. It needs no Git, Node.js or local build.
|
||||||
4. In **Hosts**, prepare a Plutainer runtime image. In **Game Files**, import your
|
4. In **Hosts**, prepare a Plutainer runtime image. In **Game Files**, select T5 or
|
||||||
existing `t5_game_files` or `t6_game_files` Docker volume on an enrolled source
|
T6 and upload your local game folder. Name the library and version and publish
|
||||||
host. Name the library and version and record its DLC metadata.
|
it after the upload finishes. Include the DLC files in that folder; there is
|
||||||
|
no separate DLC declaration. Uploading also works before enrolling any host.
|
||||||
5. Cache the published version on each destination host. Wait for verification.
|
5. Cache the published version on each destination host. Wait for verification.
|
||||||
Hosts report connection and readiness separately; deployment also checks the
|
Hosts report connection and readiness separately; deployment also checks the
|
||||||
particular library, image, capacity and ports you selected.
|
particular library, image, capacity and ports you selected.
|
||||||
@@ -73,8 +74,15 @@ API outline:
|
|||||||
- `POST /api/agent/heartbeat`: authenticated inventory, current operation and caches.
|
- `POST /api/agent/heartbeat`: authenticated inventory, current operation and caches.
|
||||||
- `GET /api/agent/operations`: deliver/reconcile durable operation IDs.
|
- `GET /api/agent/operations`: deliver/reconcile durable operation IDs.
|
||||||
- `POST /api/agent/operations/:id`: authenticated acceptance, progress or result.
|
- `POST /api/agent/operations/:id`: authenticated acceptance, progress or result.
|
||||||
- `/api/agent/imports/...` and `/api/agent/libraries/...`: host-scoped uploads and
|
- `POST /api/game-files`: administrator creates a folder upload with game, name,
|
||||||
authenticated game-file downloads.
|
version, optional description and `files: [{path, size, lastModified}]`.
|
||||||
|
- `GET /api/game-files/:id/upload`: durable per-file offsets and publication status.
|
||||||
|
- `PUT /api/game-files/:id/upload/files/:path`: administrator uploads a raw chunk
|
||||||
|
with `X-Upload-Offset`, `X-Upload-Length` and `X-Content-SHA256` headers.
|
||||||
|
- `POST /api/game-files/:id/upload/complete`: administrator starts asynchronous
|
||||||
|
verification/publication; poll the upload endpoint for completion.
|
||||||
|
- `/api/agent/libraries/...`: authenticated game-file downloads. Legacy
|
||||||
|
`/api/agent/imports/...` endpoints remain compatible with already queued imports.
|
||||||
|
|
||||||
The operation envelope is independent of polling and can be carried over a
|
The operation envelope is independent of polling and can be carried over a
|
||||||
persistent transport later. No unrestricted shell operation is available.
|
persistent transport later. No unrestricted shell operation is available.
|
||||||
@@ -117,15 +125,37 @@ communication loss stops unrelated running containers.
|
|||||||
|
|
||||||
## Libraries and ports
|
## Libraries and ports
|
||||||
|
|
||||||
Imports scan the source volume and generate per-file SHA-256 manifests. Uploads
|
Choose the actual game folder, not its parent, a ZIP archive or the Plutonium
|
||||||
stream to temporary controller storage, are bounded by their manifest sizes and
|
launcher data folder. The selected folder's name is removed once; nested paths
|
||||||
verified before atomic publication. Published versions cannot be updated;
|
such as `zone/english/...` and `main/...` stay intact. Plutonium's
|
||||||
|
[T5 server guide](https://plutonium.pw/docs/server/t5/setting-up-a-server/) and
|
||||||
|
[T6 server guide](https://plutonium.pw/docs/server/t6/setting-up-a-server/) use a
|
||||||
|
copy of the game directory, separately from the launcher/runtime. The uploader
|
||||||
|
preserves the supplied files rather than attempting to infer installed maps from
|
||||||
|
a manually maintained DLC list.
|
||||||
|
|
||||||
|
Folder uploads use chunks of at most 8 MiB, each checked against its SHA-256 before
|
||||||
|
being appended to temporary controller storage. Memory usage does not grow with
|
||||||
|
the size of an individual game file. Pause/retry uses the controller's saved file
|
||||||
|
offsets. After a page reload, choose **Resume upload** and reselect the original,
|
||||||
|
unchanged folder; relative paths, sizes and modification times must match. Keep
|
||||||
|
the page open while transferring. Once publication starts, the controller hashes
|
||||||
|
every complete file and publishes the directory atomically; closing the browser
|
||||||
|
does not stop that verification. An interrupted controller can retry publication.
|
||||||
|
Published versions cannot be updated;
|
||||||
changing files requires a new version. Hosts download with HTTP Range resumption,
|
changing files requires a new version. Hosts download with HTTP Range resumption,
|
||||||
report progress, verify all checksums and atomically promote the complete cache.
|
report progress, verify all checksums and atomically promote the complete cache.
|
||||||
Corrupt or incomplete content never becomes Ready. Cache inventory is reconciled
|
Corrupt or incomplete content never becomes Ready. Cache inventory is reconciled
|
||||||
on heartbeats, including vanished caches. Older immutable versions remain usable
|
on heartbeats, including vanished caches. Older immutable versions remain usable
|
||||||
when their exact manifest is verified.
|
when their exact manifest is verified.
|
||||||
|
|
||||||
|
The controller and reverse proxy must accept 8 MiB upload requests and folder
|
||||||
|
manifests up to 32 MiB. Compose sets `BODY_SIZE_LIMIT=Infinity`; application routes
|
||||||
|
still enforce these limits, declared file sizes and checksums. Set a proxy body
|
||||||
|
limit of at least 32 MiB (for example `client_max_body_size 32m` in nginx).
|
||||||
|
Existing published versions and cached host libraries remain usable after the
|
||||||
|
upload workflow replaces source-host imports.
|
||||||
|
|
||||||
Reservations include host ID, local bind address, protocol and port. Wildcard
|
Reservations include host ID, local bind address, protocol and port. Wildcard
|
||||||
binds conflict with concrete addresses on the same host. UDP and TCP reservations
|
binds conflict with concrete addresses on the same host. UDP and TCP reservations
|
||||||
for one launch commit together; the same number can be used independently on
|
for one launch commit together; the same number can be used independently on
|
||||||
|
|||||||
@@ -10,10 +10,10 @@ export interface HostLibraryCache {
|
|||||||
}
|
}
|
||||||
export interface GameLibraryVersion {
|
export interface GameLibraryVersion {
|
||||||
id: string; libraryId: string; game: GameLibraryGame; version: string;
|
id: string; libraryId: string; game: GameLibraryGame; version: string;
|
||||||
description: string; dlc: string[]; state: GameLibraryState; sizeBytes: number;
|
description: string; state: GameLibraryState; sizeBytes: number;
|
||||||
fileCount: number; manifestSha256: string | null; files: GameFileEntry[];
|
fileCount: number; manifestSha256: string | null; files: GameFileEntry[];
|
||||||
createdAt: string; publishedAt: string | null; storageLocation: string;
|
createdAt: string; publishedAt: string | null; storageLocation: string;
|
||||||
sourceHostId: string; sourceVolume: string; error: string | null; operationId: string | null;
|
sourceHostId: string | null; sourceVolume: string | null; error: string | null; operationId: string | null;
|
||||||
caches: HostLibraryCache[];
|
caches: HostLibraryCache[];
|
||||||
}
|
}
|
||||||
export interface GameLibrary {
|
export interface GameLibrary {
|
||||||
|
|||||||
@@ -7,7 +7,8 @@ import { assertRegularPath, parseByteRange } from './storage.js';
|
|||||||
|
|
||||||
export function gameFileError(cause: unknown): Response {
|
export function gameFileError(cause: unknown): Response {
|
||||||
const status = typeof cause === 'object' && cause !== null && 'status' in cause && typeof cause.status === 'number' ? cause.status : 400;
|
const status = typeof cause === 'object' && cause !== null && 'status' in cause && typeof cause.status === 'number' ? cause.status : 400;
|
||||||
return json({ error: cause instanceof Error ? cause.message : 'Game file request failed.' }, { status });
|
const receivedBytes = typeof cause === 'object' && cause !== null && 'receivedBytes' in cause && Number.isSafeInteger(cause.receivedBytes) ? cause.receivedBytes : undefined;
|
||||||
|
return json({ error: cause instanceof Error ? cause.message : 'Game file request failed.', ...(receivedBytes === undefined ? {} : { receivedBytes }) }, { status });
|
||||||
}
|
}
|
||||||
export async function readGameJson(request: Request, maxBytes = 64 * 1024): Promise<Record<string, unknown>> {
|
export async function readGameJson(request: Request, maxBytes = 64 * 1024): Promise<Record<string, unknown>> {
|
||||||
if (!request.body) throw new Error('A JSON request body is required.');
|
if (!request.body) throw new Error('A JSON request body is required.');
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import { config } from '../config.js';
|
|||||||
import { getHost, getHostOperation, listHosts, queueHostOperation } from '../hosts/index.js';
|
import { getHost, getHostOperation, listHosts, queueHostOperation } from '../hosts/index.js';
|
||||||
import { ensureGameFileSchema } from './schema.js';
|
import { ensureGameFileSchema } from './schema.js';
|
||||||
import { manifestChecksum, receiveGameFile, validateGamePath, validateManifest, verifyGameFile } from './storage.js';
|
import { manifestChecksum, receiveGameFile, validateGamePath, validateManifest, verifyGameFile } from './storage.js';
|
||||||
|
import { MAX_UPLOAD_CHUNK_BYTES, validateUploadPlan, getUploadFileProgress, receiveUploadChunk, hashUploadedFiles, withUploadLock } from './uploads.js';
|
||||||
import type { GameFileEntry, GameLibrary, GameLibraryGame, GameLibraryVersion, HostLibraryCache, LibraryCacheState } from '../../game-files.js';
|
import type { GameFileEntry, GameLibrary, GameLibraryGame, GameLibraryVersion, HostLibraryCache, LibraryCacheState } from '../../game-files.js';
|
||||||
|
|
||||||
function database() { const handle = db(); ensureGameFileSchema(handle); return handle; }
|
function database() { const handle = db(); ensureGameFileSchema(handle); return handle; }
|
||||||
@@ -17,10 +18,10 @@ const cacheFromRow = (row: CacheRow): HostLibraryCache => ({ hostId: row.host_id
|
|||||||
function versionFromRow(row: VersionRow): GameLibraryVersion {
|
function versionFromRow(row: VersionRow): GameLibraryVersion {
|
||||||
const files = JSON.parse(row.manifest) as GameFileEntry[];
|
const files = JSON.parse(row.manifest) as GameFileEntry[];
|
||||||
return { id: row.id, libraryId: row.library_id, game: row.game, version: row.version,
|
return { id: row.id, libraryId: row.library_id, game: row.game, version: row.version,
|
||||||
description: row.description, dlc: JSON.parse(row.dlc), state: row.state, sizeBytes: row.size_bytes,
|
description: row.description, state: row.state, sizeBytes: row.size_bytes,
|
||||||
files, fileCount: files.length, manifestSha256: row.manifest_sha256, createdAt: row.created_at,
|
files, fileCount: files.length || (database().prepare('SELECT COUNT(*) AS count FROM game_file_upload_entries WHERE version_id=?').get(row.id) as { count: number }).count, manifestSha256: row.manifest_sha256, createdAt: row.created_at,
|
||||||
publishedAt: row.published_at, storageLocation: row.storage_location, sourceHostId: row.source_host_id,
|
publishedAt: row.published_at, storageLocation: row.storage_location, sourceHostId: row.source_host_id || null,
|
||||||
sourceVolume: row.source_volume, error: row.error, operationId: row.operation_id,
|
sourceVolume: row.source_volume || null, error: row.error, operationId: row.operation_id,
|
||||||
caches: (database().prepare('SELECT * FROM host_library_caches WHERE version_id = ?').all(row.id) as CacheRow[]).map(cacheFromRow) };
|
caches: (database().prepare('SELECT * FROM host_library_caches WHERE version_id = ?').all(row.id) as CacheRow[]).map(cacheFromRow) };
|
||||||
}
|
}
|
||||||
export function getLibraryVersion(id: string): GameLibraryVersion | null {
|
export function getLibraryVersion(id: string): GameLibraryVersion | null {
|
||||||
@@ -60,6 +61,112 @@ function textField(input: unknown, label: string, max: number, required = true):
|
|||||||
if (typeof input !== 'string' || input.length > max || (required && !input.trim())) throw new Error(`Provide a valid ${label}.`);
|
if (typeof input !== 'string' || input.length > max || (required && !input.trim())) throw new Error(`Provide a valid ${label}.`);
|
||||||
return input.trim();
|
return input.trim();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** Register a folder on the controller; no host or Docker volume is needed. */
|
||||||
|
export function createLibraryUpload(input: Record<string, unknown>): GameLibraryVersion {
|
||||||
|
const game = input.game;
|
||||||
|
if (game !== 't5' && game !== 't6') throw new Error('Choose T5 or T6 game files.');
|
||||||
|
const name = textField(input.name, 'library name', 160);
|
||||||
|
const description = textField(input.description ?? '', 'description', 4000, false);
|
||||||
|
const version = textField(input.version, 'version label', 120);
|
||||||
|
const files = validateUploadPlan(input.files);
|
||||||
|
const handle = database();
|
||||||
|
const id = randomUUID(); const now = new Date().toISOString();
|
||||||
|
handle.transaction(() => {
|
||||||
|
let library = handle.prepare('SELECT id,game FROM game_file_libraries WHERE name=?').get(name) as { id: string; game: string } | undefined;
|
||||||
|
if (library && library.game !== game) throw new Error('This library name is already used for another game.');
|
||||||
|
if (!library) {
|
||||||
|
library = { id: randomUUID(), game };
|
||||||
|
handle.prepare('INSERT INTO game_file_libraries(id,name,game,description,created_at) VALUES(?,?,?,?,?)').run(library.id, name, game, description, now);
|
||||||
|
}
|
||||||
|
if (handle.prepare('SELECT 1 FROM game_file_versions WHERE library_id=? AND version=?').get(library.id, version)) throw new Error('This version label already exists. Choose a new version.');
|
||||||
|
// Keep legacy non-null columns for existing databases and published versions.
|
||||||
|
handle.prepare(`INSERT INTO game_file_versions(id,library_id,version,description,dlc,state,size_bytes,created_at,storage_location,source_host_id,source_volume)
|
||||||
|
VALUES(?,?,?,?,'[]','importing',?,?,?,'','')`).run(id, library.id, version, description, files.reduce((sum, file) => sum + file.size, 0), now, join(gameFilesRoot(), game, id));
|
||||||
|
const insert = handle.prepare('INSERT INTO game_file_upload_entries(version_id,path,size,last_modified) VALUES(?,?,?,?)');
|
||||||
|
for (const file of files) insert.run(id, file.path, file.size, file.lastModified);
|
||||||
|
})();
|
||||||
|
return getLibraryVersion(id)!;
|
||||||
|
}
|
||||||
|
|
||||||
|
type UploadEntry = { path: string; size: number; lastModified: number };
|
||||||
|
function requireBrowserUpload(id: string): GameLibraryVersion {
|
||||||
|
const version = getLibraryVersion(id);
|
||||||
|
if (!version || version.sourceHostId !== null || !database().prepare('SELECT 1 FROM game_file_upload_entries WHERE version_id=? LIMIT 1').get(id)) {
|
||||||
|
throw Object.assign(new Error('Folder upload not found.'), { status: 404 });
|
||||||
|
}
|
||||||
|
return version;
|
||||||
|
}
|
||||||
|
function uploadEntries(id: string): UploadEntry[] {
|
||||||
|
return database().prepare('SELECT path,size,last_modified AS lastModified FROM game_file_upload_entries WHERE version_id=? ORDER BY path').all(id) as UploadEntry[];
|
||||||
|
}
|
||||||
|
async function uploadStorageRoot(version: GameLibraryVersion): Promise<string> {
|
||||||
|
// Recover publication interrupted after the atomic rename but before its DB commit.
|
||||||
|
try { if ((await stat(version.storageLocation)).isDirectory()) return version.storageLocation; }
|
||||||
|
catch (cause) { if ((cause as NodeJS.ErrnoException).code !== 'ENOENT') throw cause; }
|
||||||
|
return stagingRoot(version.id);
|
||||||
|
}
|
||||||
|
export async function browserUploadStatus(id: string) {
|
||||||
|
const version = requireBrowserUpload(id);
|
||||||
|
const status = version.state === 'ready' ? 'published' as const : publishing.has(id) ? 'verifying' as const : version.state === 'failed' ? 'failed' as const : 'pending' as const;
|
||||||
|
const root = await uploadStorageRoot(version);
|
||||||
|
const entries = uploadEntries(id);
|
||||||
|
const files = [];
|
||||||
|
// Avoid opening thousands of files concurrently for a large folder.
|
||||||
|
for (const file of entries) files.push({ ...file, ...(status === 'published' ? { receivedBytes: file.size, complete: true } : await getUploadFileProgress(root, file)) });
|
||||||
|
const { files: _manifest, ...summary } = version;
|
||||||
|
return { status, chunkSize: MAX_UPLOAD_CHUNK_BYTES, files, version: summary, error: version.error };
|
||||||
|
}
|
||||||
|
export async function uploadBrowserFile(id: string, path: string, chunk: { offset: number; size: number; sha256: string }, body: ReadableStream<Uint8Array> | null) {
|
||||||
|
validateGamePath(path);
|
||||||
|
const checkWritable = () => {
|
||||||
|
const version = requireBrowserUpload(id);
|
||||||
|
if (version.state === 'ready' || version.manifestSha256 || publishing.has(id)) throw Object.assign(new Error('This upload is no longer writable.'), { status: 409 });
|
||||||
|
};
|
||||||
|
checkWritable();
|
||||||
|
const file = database().prepare('SELECT path,size,last_modified AS lastModified FROM game_file_upload_entries WHERE version_id=? AND path=?').get(id, path) as UploadEntry | undefined;
|
||||||
|
if (!file) throw new Error('File is not listed in this upload.');
|
||||||
|
const progress = await receiveUploadChunk(stagingRoot(id), file, chunk, body, checkWritable);
|
||||||
|
database().prepare("UPDATE game_file_versions SET state='importing',error=NULL WHERE id=? AND state IN ('importing','failed')").run(id);
|
||||||
|
return progress;
|
||||||
|
}
|
||||||
|
export async function publishBrowserUpload(id: string): Promise<GameLibraryVersion> {
|
||||||
|
const version = requireBrowserUpload(id);
|
||||||
|
if (version.state === 'ready') return version;
|
||||||
|
const existing = publishing.get(id); if (existing) return existing;
|
||||||
|
const task = withUploadLock(stagingRoot(id), async () => {
|
||||||
|
const root = await uploadStorageRoot(version);
|
||||||
|
const files = await hashUploadedFiles(root, uploadEntries(id));
|
||||||
|
const checksum = manifestChecksum(files);
|
||||||
|
if (version.manifestSha256 && checksum !== version.manifestSha256) throw new Error('Stored files no longer match the verified manifest.');
|
||||||
|
database().transaction(() => {
|
||||||
|
database().prepare("UPDATE game_file_versions SET state='importing',manifest=?,manifest_sha256=?,error=NULL WHERE id=? AND state IN ('importing','failed')").run(JSON.stringify(files), checksum, id);
|
||||||
|
database().prepare('DELETE FROM game_file_entries WHERE version_id=?').run(id);
|
||||||
|
const insert = database().prepare('INSERT INTO game_file_entries(version_id,path,size,sha256) VALUES(?,?,?,?)');
|
||||||
|
for (const file of files) insert.run(id, file.path, file.size, file.sha256);
|
||||||
|
})();
|
||||||
|
if (root !== version.storageLocation) {
|
||||||
|
await mkdir(dirname(version.storageLocation), { recursive: true });
|
||||||
|
await rename(root, version.storageLocation);
|
||||||
|
}
|
||||||
|
database().prepare("UPDATE game_file_versions SET state='ready',published_at=?,error=NULL WHERE id=? AND state='importing'").run(new Date().toISOString(), id);
|
||||||
|
return getLibraryVersion(id)!;
|
||||||
|
});
|
||||||
|
publishing.set(id, task);
|
||||||
|
try { return await task; } finally { publishing.delete(id); }
|
||||||
|
}
|
||||||
|
export async function requestBrowserPublication(id: string) {
|
||||||
|
const version = requireBrowserUpload(id);
|
||||||
|
if (version.state !== 'ready' && !publishing.has(id)) {
|
||||||
|
database().prepare("UPDATE game_file_versions SET state='importing',error=NULL WHERE id=? AND state IN ('importing','failed')").run(id);
|
||||||
|
void publishBrowserUpload(id).catch(cause => {
|
||||||
|
database().prepare("UPDATE game_file_versions SET state='failed',error=? WHERE id=? AND state='importing'")
|
||||||
|
.run((cause instanceof Error ? cause.message : 'Folder publication failed.').slice(0, 2000), id);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return browserUploadStatus(id);
|
||||||
|
}
|
||||||
|
|
||||||
export function createLibraryImport(input: Record<string, unknown>): GameLibraryVersion {
|
export function createLibraryImport(input: Record<string, unknown>): GameLibraryVersion {
|
||||||
const game = input.game;
|
const game = input.game;
|
||||||
if (game !== 't5' && game !== 't6') throw new Error('Choose T5 or T6 game files.');
|
if (game !== 't5' && game !== 't6') throw new Error('Choose T5 or T6 game files.');
|
||||||
@@ -69,9 +176,6 @@ export function createLibraryImport(input: Record<string, unknown>): GameLibrary
|
|||||||
const name = textField(input.name, 'library name', 160);
|
const name = textField(input.name, 'library name', 160);
|
||||||
const description = textField(input.description ?? '', 'description', 4000, false);
|
const description = textField(input.description ?? '', 'description', 4000, false);
|
||||||
const version = textField(input.version, 'version label', 120);
|
const version = textField(input.version, 'version label', 120);
|
||||||
const dlcInput = input.dlc ?? [];
|
|
||||||
if (!Array.isArray(dlcInput) || dlcInput.length > 64) throw new Error('DLC must be a list of content names.');
|
|
||||||
const dlc = dlcInput.map(value => textField(value, 'DLC name', 120));
|
|
||||||
const sourceVolume = game === 't5' ? 't5_game_files' : 't6_game_files';
|
const sourceVolume = game === 't5' ? 't5_game_files' : 't6_game_files';
|
||||||
const handle = database();
|
const handle = database();
|
||||||
let library = handle.prepare('SELECT id,game FROM game_file_libraries WHERE name=?').get(name) as { id: string; game: string } | undefined;
|
let library = handle.prepare('SELECT id,game FROM game_file_libraries WHERE name=?').get(name) as { id: string; game: string } | undefined;
|
||||||
@@ -84,7 +188,7 @@ export function createLibraryImport(input: Record<string, unknown>): GameLibrary
|
|||||||
}
|
}
|
||||||
if (handle.prepare('SELECT 1 FROM game_file_versions WHERE library_id=? AND version=?').get(library.id, version)) throw new Error('This version label already exists. Choose a new version.');
|
if (handle.prepare('SELECT 1 FROM game_file_versions WHERE library_id=? AND version=?').get(library.id, version)) throw new Error('This version label already exists. Choose a new version.');
|
||||||
handle.prepare(`INSERT INTO game_file_versions(id,library_id,version,description,dlc,state,created_at,storage_location,source_host_id,source_volume)
|
handle.prepare(`INSERT INTO game_file_versions(id,library_id,version,description,dlc,state,created_at,storage_location,source_host_id,source_volume)
|
||||||
VALUES(?,?,?,?,?,'importing',?,?,?,?)`).run(id, library.id, version, description, JSON.stringify(dlc), now, join(gameFilesRoot(), game, id), hostId, sourceVolume);
|
VALUES(?,?,?,?,?,'importing',?,?,?,?)`).run(id, library.id, version, description, '[]', now, join(gameFilesRoot(), game, id), hostId, sourceVolume);
|
||||||
})();
|
})();
|
||||||
try {
|
try {
|
||||||
const operation = queueHostOperation(hostId, 'library.import', {
|
const operation = queueHostOperation(hostId, 'library.import', {
|
||||||
|
|||||||
@@ -29,6 +29,11 @@ export function ensureGameFileSchema(handle: Database.Database): void {
|
|||||||
version_id TEXT NOT NULL REFERENCES game_file_versions(id), path TEXT NOT NULL,
|
version_id TEXT NOT NULL REFERENCES game_file_versions(id), path TEXT NOT NULL,
|
||||||
size INTEGER NOT NULL, sha256 TEXT NOT NULL, PRIMARY KEY(version_id,path)
|
size INTEGER NOT NULL, sha256 TEXT NOT NULL, PRIMARY KEY(version_id,path)
|
||||||
);
|
);
|
||||||
|
CREATE TABLE IF NOT EXISTS game_file_upload_entries (
|
||||||
|
version_id TEXT NOT NULL REFERENCES game_file_versions(id), path TEXT NOT NULL,
|
||||||
|
size INTEGER NOT NULL, last_modified INTEGER NOT NULL,
|
||||||
|
PRIMARY KEY(version_id,path)
|
||||||
|
);
|
||||||
INSERT OR IGNORE INTO game_file_entries(version_id,path,size,sha256)
|
INSERT OR IGNORE INTO game_file_entries(version_id,path,size,sha256)
|
||||||
SELECT v.id,json_extract(f.value,'$.path'),json_extract(f.value,'$.size'),json_extract(f.value,'$.sha256')
|
SELECT v.id,json_extract(f.value,'$.path'),json_extract(f.value,'$.size'),json_extract(f.value,'$.sha256')
|
||||||
FROM game_file_versions v,json_each(v.manifest) f;
|
FROM game_file_versions v,json_each(v.manifest) f;
|
||||||
|
|||||||
@@ -63,13 +63,18 @@ export async function assertRegularPath(root: string, path: string, allowMissing
|
|||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
export async function verifyGameFile(root: string, file: GameFileEntry): Promise<void> {
|
/** Compute a file digest without buffering game archives in memory. */
|
||||||
|
export async function hashGameFile(root: string, file: Pick<GameFileEntry, 'path' | 'size'>): Promise<string> {
|
||||||
const path = await assertRegularPath(root, file.path);
|
const path = await assertRegularPath(root, file.path);
|
||||||
const info = await stat(path);
|
const info = await stat(path);
|
||||||
if (info.size !== file.size) throw new Error(`Size verification failed for ${file.path}.`);
|
if (info.size !== file.size) throw new Error(`Size verification failed for ${file.path}.`);
|
||||||
const hash = createHash('sha256');
|
const hash = createHash('sha256');
|
||||||
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
for await (const chunk of createReadStream(path)) hash.update(chunk);
|
||||||
if (hash.digest('hex') !== file.sha256) throw new Error(`Checksum verification failed for ${file.path}.`);
|
return hash.digest('hex');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function verifyGameFile(root: string, file: GameFileEntry): Promise<void> {
|
||||||
|
if (await hashGameFile(root, file) !== file.sha256) throw new Error(`Checksum verification failed for ${file.path}.`);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Stream to a private temporary file; publish only after byte count and SHA256 match. */
|
/** Stream to a private temporary file; publish only after byte count and SHA256 match. */
|
||||||
|
|||||||
@@ -0,0 +1,155 @@
|
|||||||
|
import { createHash, randomUUID } from 'node:crypto';
|
||||||
|
import { constants, createReadStream, createWriteStream } from 'node:fs';
|
||||||
|
import { lstat, mkdir, open, rm } from 'node:fs/promises';
|
||||||
|
import { dirname, resolve } from 'node:path';
|
||||||
|
import { Readable, Transform } from 'node:stream';
|
||||||
|
import { pipeline } from 'node:stream/promises';
|
||||||
|
import type { GameFileEntry } from '../../game-files.js';
|
||||||
|
import { assertRegularPath, hashGameFile, validateManifest } from './storage.js';
|
||||||
|
|
||||||
|
export const MAX_UPLOAD_CHUNK_BYTES = 8 * 1024 * 1024;
|
||||||
|
|
||||||
|
export interface UploadFilePlan {
|
||||||
|
path: string;
|
||||||
|
size: number;
|
||||||
|
lastModified: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface UploadFileProgress {
|
||||||
|
receivedBytes: number;
|
||||||
|
complete: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class UploadOffsetError extends Error {
|
||||||
|
readonly status = 409;
|
||||||
|
readonly receivedBytes: number;
|
||||||
|
constructor(receivedBytes: number) {
|
||||||
|
super(`Upload offset does not match the stored file. Resume at byte ${receivedBytes}.`);
|
||||||
|
this.name = 'UploadOffsetError';
|
||||||
|
this.receivedBytes = receivedBytes;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Apply the same path, case, size and file/directory collision rules as published manifests. */
|
||||||
|
export function validateUploadPlan(input: unknown): UploadFilePlan[] {
|
||||||
|
if (!Array.isArray(input) || !input.length || input.length > 200_000) throw new Error('Provide a folder containing 1–200000 files.');
|
||||||
|
const modified = new Map<string, number>();
|
||||||
|
const manifest = validateManifest(input.map(raw => {
|
||||||
|
if (!raw || typeof raw !== 'object') throw new Error('Invalid upload file.');
|
||||||
|
const lastModified = raw.lastModified === undefined ? 0 : raw.lastModified;
|
||||||
|
if (!Number.isSafeInteger(lastModified) || lastModified < 0) throw new Error('Invalid file modification time.');
|
||||||
|
modified.set(raw.path, lastModified);
|
||||||
|
return { path: raw.path, size: raw.size, sha256: '0'.repeat(64) };
|
||||||
|
}));
|
||||||
|
return manifest.map(file => ({ path: file.path, size: file.size, lastModified: modified.get(file.path)! }));
|
||||||
|
}
|
||||||
|
|
||||||
|
const uploadLocks = new Map<string, Promise<void>>();
|
||||||
|
|
||||||
|
/** Serialize transfers and final publication for one import; a rejected operation never poisons the lock. */
|
||||||
|
export async function withUploadLock<T>(root: string, callback: () => Promise<T> | T): Promise<T> {
|
||||||
|
const key = resolve(root);
|
||||||
|
const previous = uploadLocks.get(key) ?? Promise.resolve();
|
||||||
|
let release!: () => void;
|
||||||
|
const current = new Promise<void>(done => { release = done; });
|
||||||
|
uploadLocks.set(key, current);
|
||||||
|
await previous;
|
||||||
|
try { return await callback(); }
|
||||||
|
finally {
|
||||||
|
release();
|
||||||
|
if (uploadLocks.get(key) === current) uploadLocks.delete(key);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** File size is the resume checkpoint, including after a controller restart during an append. */
|
||||||
|
export async function getUploadFileProgress(root: string, file: Pick<UploadFilePlan, 'path' | 'size'>): Promise<UploadFileProgress> {
|
||||||
|
const path = await assertRegularPath(root, file.path, true);
|
||||||
|
try {
|
||||||
|
const info = await lstat(path);
|
||||||
|
if (!info.isFile() || info.isSymbolicLink()) throw new Error('Game storage contains a non-regular path.');
|
||||||
|
if (info.size > file.size) throw new Error(`Stored upload exceeds the declared size of ${file.path}.`);
|
||||||
|
return { receivedBytes: info.size, complete: info.size === file.size };
|
||||||
|
} catch (cause) {
|
||||||
|
if ((cause as NodeJS.ErrnoException).code === 'ENOENT') return { receivedBytes: 0, complete: false };
|
||||||
|
throw cause;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Verify one bounded request before touching the destination. Only verified bytes may be
|
||||||
|
* appended, so a process interruption during the append can resume at the actual file size.
|
||||||
|
* beforeWrite runs under the lock: callers must recheck that the import is still writable.
|
||||||
|
*/
|
||||||
|
export async function receiveUploadChunk(
|
||||||
|
root: string,
|
||||||
|
file: Pick<UploadFilePlan, 'path' | 'size'>,
|
||||||
|
chunk: { offset: number; size: number; sha256: string },
|
||||||
|
body: ReadableStream<Uint8Array> | null,
|
||||||
|
beforeWrite?: () => Promise<void> | void
|
||||||
|
): Promise<UploadFileProgress> {
|
||||||
|
if (!Number.isSafeInteger(chunk.offset) || chunk.offset < 0 || !Number.isSafeInteger(chunk.size)
|
||||||
|
|| chunk.size < 0 || chunk.size > MAX_UPLOAD_CHUNK_BYTES || chunk.offset + chunk.size > file.size
|
||||||
|
|| !Number.isSafeInteger(chunk.offset + chunk.size) || !/^[a-f0-9]{64}$/.test(chunk.sha256)
|
||||||
|
|| (chunk.size === 0 && file.size !== 0)) {
|
||||||
|
throw new Error('Invalid upload chunk size, offset or SHA256.');
|
||||||
|
}
|
||||||
|
if (!body && chunk.size !== 0) throw new Error('A file chunk body is required.');
|
||||||
|
|
||||||
|
return withUploadLock(root, async () => {
|
||||||
|
await beforeWrite?.();
|
||||||
|
const progress = await getUploadFileProgress(root, file);
|
||||||
|
if (chunk.offset !== progress.receivedBytes) throw new UploadOffsetError(progress.receivedBytes);
|
||||||
|
const destination = await assertRegularPath(root, file.path, true);
|
||||||
|
// Keep transient chunks outside the tree that will become the immutable library.
|
||||||
|
const chunkRoot = `${resolve(root)}.chunks`;
|
||||||
|
const temporary = await assertRegularPath(chunkRoot, randomUUID(), true);
|
||||||
|
await mkdir(dirname(temporary), { recursive: true });
|
||||||
|
let received = 0;
|
||||||
|
const hash = createHash('sha256');
|
||||||
|
const verifier = new Transform({ transform(data, _encoding, done) {
|
||||||
|
received += data.length;
|
||||||
|
if (received > chunk.size) return done(new Error('Upload exceeds the declared chunk size.'));
|
||||||
|
hash.update(data); done(null, data);
|
||||||
|
} });
|
||||||
|
try {
|
||||||
|
await pipeline(body ? Readable.fromWeb(body as never) : Readable.from([]), verifier,
|
||||||
|
createWriteStream(temporary, { flags: 'wx', mode: 0o600, flush: true }));
|
||||||
|
if (received !== chunk.size || hash.digest('hex') !== chunk.sha256) throw new Error('Upload chunk checksum or size mismatch.');
|
||||||
|
|
||||||
|
await mkdir(dirname(destination), { recursive: true });
|
||||||
|
await assertRegularPath(root, file.path, true);
|
||||||
|
const target = await open(destination, constants.O_WRONLY | constants.O_CREAT | constants.O_NOFOLLOW, 0o600);
|
||||||
|
try {
|
||||||
|
const info = await target.stat();
|
||||||
|
if (!info.isFile()) throw new Error('Game storage contains a non-regular path.');
|
||||||
|
if (info.size !== chunk.offset) throw new UploadOffsetError(info.size);
|
||||||
|
let position = chunk.offset;
|
||||||
|
try {
|
||||||
|
for await (const data of createReadStream(temporary)) {
|
||||||
|
const bytes = data as Buffer;
|
||||||
|
let written = 0;
|
||||||
|
while (written < bytes.length) {
|
||||||
|
const result = await target.write(bytes, written, bytes.length - written, position);
|
||||||
|
if (result.bytesWritten === 0) throw new Error('Unable to append the upload chunk.');
|
||||||
|
written += result.bytesWritten;
|
||||||
|
position += result.bytesWritten;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
await target.sync();
|
||||||
|
} catch (cause) {
|
||||||
|
await target.truncate(chunk.offset);
|
||||||
|
await target.sync();
|
||||||
|
throw cause;
|
||||||
|
}
|
||||||
|
} finally { await target.close(); }
|
||||||
|
return { receivedBytes: chunk.offset + chunk.size, complete: chunk.offset + chunk.size === file.size };
|
||||||
|
} finally { await rm(temporary, { force: true }); }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Caller holds withUploadLock through hashing and publication to prevent concurrent writes. */
|
||||||
|
export async function hashUploadedFiles(root: string, files: ReadonlyArray<Pick<UploadFilePlan, 'path' | 'size'>>): Promise<GameFileEntry[]> {
|
||||||
|
const manifest: GameFileEntry[] = [];
|
||||||
|
for (const file of files) manifest.push({ path: file.path, size: file.size, sha256: await hashGameFile(root, file) });
|
||||||
|
return validateManifest(manifest);
|
||||||
|
}
|
||||||
@@ -204,7 +204,7 @@ export function previewCreation(input: { game?: unknown; mapSlug?: unknown; blue
|
|||||||
{ label: 'Lobby', state: name ? 'pass' : 'blocked', detail: name ?? 'All 20 lobbies for this map are occupied or reserved.' },
|
{ label: 'Lobby', state: name ? 'pass' : 'blocked', detail: name ?? 'All 20 lobbies for this map are occupied or reserved.' },
|
||||||
{ label: 'Game port', state: port === null ? 'blocked' : 'pass', detail: port === null ? 'No free game port. Free capacity before launching.' : `${port} / UDP and TCP. Availability is checked again at launch.` },
|
{ label: 'Game port', state: port === null ? 'blocked' : 'pass', detail: port === null ? 'No free game port. Free capacity before launching.' : `${port} / UDP and TCP. Availability is checked again at launch.` },
|
||||||
{ label: 'Promoted configuration', state: 'pass', detail: recipe.configSource },
|
{ label: 'Promoted configuration', state: 'pass', detail: recipe.configSource },
|
||||||
{ label: 'Game files and host readiness', state: 'info', detail: 'The host must have the selected game and DLC files. This preview does not probe volumes, images, credentials, or network access.' }
|
{ label: 'Game files and host readiness', state: 'info', detail: 'The host must have the selected game files. This preview does not probe volumes, images, credentials, or network access.' }
|
||||||
];
|
];
|
||||||
const id = randomUUID();
|
const id = randomUUID();
|
||||||
const expiresAt = new Date(Date.now() + 30 * 60_000).toISOString();
|
const expiresAt = new Date(Date.now() + 30 * 60_000).toISOString();
|
||||||
|
|||||||
@@ -30,7 +30,7 @@
|
|||||||
{#if !loading && !hosts.some(candidate => candidate.readiness === 'ready')}<p>No deployment-ready hosts. <a href="/hosts" target="_blank" rel="noopener">Set up a host</a>, then refresh.</p>{/if}
|
{#if !loading && !hosts.some(candidate => candidate.readiness === 'ready')}<p>No deployment-ready hosts. <a href="/hosts" target="_blank" rel="noopener">Set up a host</a>, then refresh.</p>{/if}
|
||||||
{#if host}<p>{host.hostname} · ports {host.portMin}–{host.portMax} · {host.publicAddress || host.bindAddress}</p>{/if}
|
{#if host}<p>{host.hostname} · ports {host.portMin}–{host.portMax} · {host.publicAddress || host.bindAddress}</p>{/if}
|
||||||
{#each games as game}<label>{game === 't5' ? 'Black Ops I' : 'Black Ops II'} game-file version<select bind:value={selection.libraryIds[game]} disabled={disabled || loading || !host} onchange={() => onchange?.()}><option value="">Choose an immutable library version</option>{#each versions.filter(version => version.game === game) as version}<option value={version.id} disabled={!cacheReady(version.id)}>{version.name} · {version.version} · {cacheReady(version.id) ? 'Ready on host' : 'Not cached on host'}</option>{/each}</select></label>{/each}
|
{#each games as game}<label>{game === 't5' ? 'Black Ops I' : 'Black Ops II'} game-file version<select bind:value={selection.libraryIds[game]} disabled={disabled || loading || !host} onchange={() => onchange?.()}><option value="">Choose an immutable library version</option>{#each versions.filter(version => version.game === game) as version}<option value={version.id} disabled={!cacheReady(version.id)}>{version.name} · {version.version} · {cacheReady(version.id) ? 'Ready on host' : 'Not cached on host'}</option>{/each}</select></label>{/each}
|
||||||
<p><a href="/game-files" target="_blank" rel="noopener">Import or download game files to a host</a>, then refresh. Each launch stays pinned to its selected library version.</p>
|
<p><a href="/game-files" target="_blank" rel="noopener">Upload a game folder or download a version to a host</a>, then refresh. Each launch stays pinned to its selected library version.</p>
|
||||||
<label>Plutainer runtime image<input required bind:value={selection.image} disabled={disabled} onchange={() => onchange?.()} /></label>
|
<label>Plutainer runtime image<input required bind:value={selection.image} disabled={disabled} onchange={() => onchange?.()} /></label>
|
||||||
{#if host && !imageReady}<p class="error">Prepare this runtime image from <a href={`/hosts/${host.id}`} target="_blank" rel="noopener">host setup</a> before launch.</p>{/if}
|
{#if host && !imageReady}<p class="error">Prepare this runtime image from <a href={`/hosts/${host.id}`} target="_blank" rel="noopener">host setup</a> before launch.</p>{/if}
|
||||||
{#if network}<details><summary>Network overrides</summary><label>Local bind address<input bind:value={selection.bindAddress} placeholder={host?.bindAddress ?? 'Host default'} disabled={disabled} onchange={() => onchange?.()} /></label><label>Public / NAT address<input bind:value={selection.publicAddress} placeholder={host?.publicAddress || 'Host default'} disabled={disabled} onchange={() => onchange?.()} /></label><label>Public / NAT port<input type="number" min="1" max="65535" bind:value={selection.publicPort} placeholder="Same as local port" disabled={disabled} onchange={() => onchange?.()} /></label><p>Public mapping does not change the local Docker port reservation.</p></details>{/if}
|
{#if network}<details><summary>Network overrides</summary><label>Local bind address<input bind:value={selection.bindAddress} placeholder={host?.bindAddress ?? 'Host default'} disabled={disabled} onchange={() => onchange?.()} /></label><label>Public / NAT address<input bind:value={selection.publicAddress} placeholder={host?.publicAddress || 'Host default'} disabled={disabled} onchange={() => onchange?.()} /></label><label>Public / NAT port<input type="number" min="1" max="65535" bind:value={selection.publicPort} placeholder="Same as local port" disabled={disabled} onchange={() => onchange?.()} /></label><p>Public mapping does not change the local Docker port reservation.</p></details>{/if}
|
||||||
|
|||||||
@@ -0,0 +1,38 @@
|
|||||||
|
export interface SelectedGameFile { path: string; file: File }
|
||||||
|
export interface SelectedGameFolder { name: string; files: SelectedGameFile[]; size: number }
|
||||||
|
|
||||||
|
/** Directory inputs include the selected folder in every relative path. Remove
|
||||||
|
* that one component and retain the complete game directory below it. */
|
||||||
|
export function selectGameFolder(selection: Iterable<File>): SelectedGameFolder {
|
||||||
|
const files: SelectedGameFile[] = [];
|
||||||
|
const paths = new Set<string>();
|
||||||
|
let name = '';
|
||||||
|
let size = 0;
|
||||||
|
for (const file of selection) {
|
||||||
|
const [root, ...parts] = file.webkitRelativePath.split('/');
|
||||||
|
if (!root || root === '.' || root === '..' || !parts.length || name && root !== name) throw new Error('Choose one game folder with the folder picker.');
|
||||||
|
name = root;
|
||||||
|
const path = parts.join('/');
|
||||||
|
if (path.length > 1024 || /[\\\x00-\x1f\x7f:]/.test(path) || parts.some(part => !part || part === '.' || part === '..' || /[. ]$/.test(part))) {
|
||||||
|
throw new Error(`This file has an unsupported game path: ${file.webkitRelativePath}`);
|
||||||
|
}
|
||||||
|
const key = path.toLowerCase();
|
||||||
|
if (paths.has(key)) throw new Error(`This folder has duplicate game paths: ${path}`);
|
||||||
|
paths.add(key);
|
||||||
|
size += file.size;
|
||||||
|
if (!Number.isSafeInteger(size)) throw new Error('This folder exceeds the supported upload size.');
|
||||||
|
files.push({ path, file });
|
||||||
|
if (files.length > 200_000) throw new Error('Choose a folder with no more than 200,000 files.');
|
||||||
|
}
|
||||||
|
if (!files.length) throw new Error('The selected folder contains no files.');
|
||||||
|
for (const { path } of files) {
|
||||||
|
const parents = path.toLowerCase().split('/');
|
||||||
|
parents.pop();
|
||||||
|
while (parents.length) {
|
||||||
|
if (paths.has(parents.join('/'))) throw new Error(`This folder uses the same game path for a file and a directory: ${path}`);
|
||||||
|
parents.pop();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
files.sort((a, b) => a.path < b.path ? -1 : a.path > b.path ? 1 : 0);
|
||||||
|
return { name, files, size };
|
||||||
|
}
|
||||||
@@ -1,10 +1,10 @@
|
|||||||
import { json } from '@sveltejs/kit';
|
import { json } from '@sveltejs/kit';
|
||||||
import { createLibraryImport, libraryOverview } from '$lib/server/game-files/index.js';
|
import { createLibraryUpload, libraryOverview } from '$lib/server/game-files/index.js';
|
||||||
import { gameFileError, readGameJson } from '$lib/server/game-files/http.js';
|
import { gameFileError, readGameJson } from '$lib/server/game-files/http.js';
|
||||||
import { requireManagementOrigin } from '$lib/server/hosts/http.js';
|
import { requireManagementOrigin } from '$lib/server/hosts/http.js';
|
||||||
|
|
||||||
export function GET() { return json(libraryOverview()); }
|
export function GET() { return json(libraryOverview()); }
|
||||||
export async function POST({ request }: { request: Request }) {
|
export async function POST({ request }: { request: Request }) {
|
||||||
try { requireManagementOrigin(request); return json({ version: createLibraryImport(await readGameJson(request)) }, { status: 202 }); }
|
try { requireManagementOrigin(request); return json({ version: createLibraryUpload(await readGameJson(request, 32 * 1024 * 1024)) }, { status: 202 }); }
|
||||||
catch (cause) { return gameFileError(cause); }
|
catch (cause) { return gameFileError(cause); }
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import { json } from '@sveltejs/kit';
|
||||||
|
import { browserUploadStatus } from '$lib/server/game-files/index.js';
|
||||||
|
import { gameFileError } from '$lib/server/game-files/http.js';
|
||||||
|
|
||||||
|
export async function GET({ params }: { params: { id: string } }) {
|
||||||
|
try { return json(await browserUploadStatus(params.id), { headers: { 'Cache-Control': 'no-store' } }); }
|
||||||
|
catch (cause) { return gameFileError(cause); }
|
||||||
|
}
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
import { json } from '@sveltejs/kit';
|
||||||
|
import { requestBrowserPublication } from '$lib/server/game-files/index.js';
|
||||||
|
import { gameFileError } from '$lib/server/game-files/http.js';
|
||||||
|
import { requireManagementOrigin } from '$lib/server/hosts/http.js';
|
||||||
|
|
||||||
|
export async function POST({ request, params }: { request: Request; params: { id: string } }) {
|
||||||
|
try {
|
||||||
|
requireManagementOrigin(request);
|
||||||
|
return json(await requestBrowserPublication(params.id), { status: 202 });
|
||||||
|
} catch (cause) { return gameFileError(cause); }
|
||||||
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import { json } from '@sveltejs/kit';
|
||||||
|
import { uploadBrowserFile } from '$lib/server/game-files/index.js';
|
||||||
|
import { gameFileError } from '$lib/server/game-files/http.js';
|
||||||
|
import { requireManagementOrigin } from '$lib/server/hosts/http.js';
|
||||||
|
|
||||||
|
function integerHeader(request: Request, name: string): number {
|
||||||
|
const value = request.headers.get(name);
|
||||||
|
if (value === null || !/^\d+$/.test(value) || !Number.isSafeInteger(Number(value))) throw new Error(`Provide a valid ${name} header.`);
|
||||||
|
return Number(value);
|
||||||
|
}
|
||||||
|
export async function PUT({ request, params }: { request: Request; params: { id: string; path: string } }) {
|
||||||
|
try {
|
||||||
|
requireManagementOrigin(request);
|
||||||
|
const chunk = { offset: integerHeader(request, 'X-Upload-Offset'), size: integerHeader(request, 'X-Upload-Length'), sha256: request.headers.get('X-Content-SHA256') ?? '' };
|
||||||
|
return json(await uploadBrowserFile(params.id, params.path, chunk, request.body));
|
||||||
|
} catch (cause) { return gameFileError(cause); }
|
||||||
|
}
|
||||||
@@ -1,9 +1,11 @@
|
|||||||
<script lang="ts">
|
<script lang="ts">
|
||||||
import { onMount } from 'svelte';
|
import { onMount, tick } from 'svelte';
|
||||||
import { page } from '$app/stores';
|
import { page } from '$app/stores';
|
||||||
|
import { beforeNavigate } from '$app/navigation';
|
||||||
import type { GameLibrary, GameLibraryVersion } from '$lib/game-files.js';
|
import type { GameLibrary, GameLibraryVersion } from '$lib/game-files.js';
|
||||||
import { formatGameBytes } from '$lib/game-files.js';
|
import { formatGameBytes } from '$lib/game-files.js';
|
||||||
import type { Host } from '$lib/hosts.js';
|
import type { Host } from '$lib/hosts.js';
|
||||||
|
import { selectGameFolder, type SelectedGameFolder } from '$ui/game-folder.js';
|
||||||
import { pollVisible } from '$ui/poll.js';
|
import { pollVisible } from '$ui/poll.js';
|
||||||
import SectionHeader from '$ui/components/primitives/SectionHeader.svelte';
|
import SectionHeader from '$ui/components/primitives/SectionHeader.svelte';
|
||||||
import Button from '$ui/components/primitives/Button.svelte';
|
import Button from '$ui/components/primitives/Button.svelte';
|
||||||
@@ -11,110 +13,265 @@
|
|||||||
import EmptyState from '$ui/components/primitives/EmptyState.svelte';
|
import EmptyState from '$ui/components/primitives/EmptyState.svelte';
|
||||||
|
|
||||||
type OverviewLibrary = Omit<GameLibrary, 'versions'> & { versions: Omit<GameLibraryVersion, 'files'>[] };
|
type OverviewLibrary = Omit<GameLibrary, 'versions'> & { versions: Omit<GameLibraryVersion, 'files'>[] };
|
||||||
let libraries = $state<OverviewLibrary[]>([]);
|
interface UploadFile { path: string; size: number; lastModified: number; receivedBytes: number; complete: boolean }
|
||||||
let hosts = $state<Host[]>([]);
|
interface UploadStatus { status: 'pending' | 'verifying' | 'published' | 'failed'; chunkSize: number; files: UploadFile[]; error?: string | null; version: { id: string; state: string } }
|
||||||
let loaded = $state(false);
|
type Phase = 'idle' | 'preparing' | 'uploading' | 'paused' | 'verifying' | 'published';
|
||||||
let error = $state(''); let notice = $state(''); let busy = $state(false);
|
let libraries = $state<OverviewLibrary[]>([]); let hosts = $state<Host[]>([]);
|
||||||
let showImport = $state(false); let game = $state<'t5' | 't6'>('t5');
|
let loaded = $state(false); let error = $state(''); let notice = $state(''); let busy = $state(false);
|
||||||
let name = $state(''); let version = $state(''); let description = $state(''); let dlc = $state(''); let hostId = $state('');
|
let showUpload = $state(false); let game = $state<'t5' | 't6'>('t5');
|
||||||
let filterGame = $state('all');
|
let name = $state(''); let version = $state(''); let description = $state('');
|
||||||
let targets = $state<Record<string, string>>({});
|
let filterGame = $state('all'); let targets = $state<Record<string, string>>({});
|
||||||
|
let folder = $state.raw<SelectedGameFolder | null>(null);
|
||||||
|
let uploadStatus = $state.raw<UploadStatus | null>(null);
|
||||||
|
let uploadId = $state(''); let uploadBusy = $state(false); let uploadError = $state('');
|
||||||
|
let phase = $state<Phase>('idle'); let currentPath = $state(''); let pauseRequested = $state(false);
|
||||||
|
let acceptedBytes = $state(0); let sendingBytes = $state(0); let completedFiles = $state(0);
|
||||||
|
let folderInput = $state<HTMLInputElement>();
|
||||||
|
let activeRequest: XMLHttpRequest | null = null; let disposed = false;
|
||||||
const canManage = $derived($page.data.user?.role === 'admin');
|
const canManage = $derived($page.data.user?.role === 'admin');
|
||||||
const onlineHosts = $derived(hosts.filter(host => host.connected && host.credentialActive && host.mode !== 'disabled'));
|
const onlineHosts = $derived(hosts.filter(host => host.connected && host.credentialActive && host.mode !== 'disabled'));
|
||||||
const matching = $derived(libraries.filter(library => filterGame === 'all' || library.game === filterGame));
|
const matching = $derived(libraries.filter(library => filterGame === 'all' || library.game === filterGame));
|
||||||
const hostName = (id: string) => hosts.find(host => host.id === id)?.name ?? id;
|
const totalBytes = $derived(folder?.size ?? uploadStatus?.files.reduce((total, file) => total + file.size, 0) ?? 0);
|
||||||
|
const totalFiles = $derived(folder?.files.length ?? uploadStatus?.files.length ?? 0);
|
||||||
|
const progressBytes = $derived(Math.min(totalBytes, acceptedBytes + sendingBytes));
|
||||||
|
const transferring = $derived(uploadBusy && (phase === 'preparing' || phase === 'uploading'));
|
||||||
|
const hostName = (id: string | null) => id ? hosts.find(host => host.id === id)?.name ?? id : 'Browser upload';
|
||||||
|
|
||||||
|
async function responseBody(response: Response) {
|
||||||
|
let body;
|
||||||
|
try { body = await response.json(); } catch { throw new Error('The controller returned an unexpected response. Sign in again if your session expired, then resume the upload.'); }
|
||||||
|
if (!response.ok) throw new Error(body.error ?? 'The controller could not complete this request.');
|
||||||
|
return body;
|
||||||
|
}
|
||||||
async function load() {
|
async function load() {
|
||||||
try {
|
try {
|
||||||
const response = await fetch('/api/game-files');
|
const body = await responseBody(await fetch('/api/game-files'));
|
||||||
const body = await response.json();
|
|
||||||
if (!response.ok) throw new Error(body.error ?? 'Could not load game libraries.');
|
|
||||||
libraries = body.libraries; hosts = body.hosts; loaded = true;
|
libraries = body.libraries; hosts = body.hosts; loaded = true;
|
||||||
} catch (cause) { error = (cause as Error).message; }
|
} catch (cause) { error = (cause as Error).message; }
|
||||||
}
|
}
|
||||||
async function act(url: string, payload: Record<string, unknown>, message: string) {
|
async function download(versionId: string) {
|
||||||
busy = true; error = ''; notice = '';
|
busy = true; error = ''; notice = '';
|
||||||
try {
|
try {
|
||||||
const response = await fetch(url, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify(payload) });
|
await responseBody(await fetch(`/api/game-files/${versionId}/download`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ hostId: targets[versionId] }) }));
|
||||||
const result = await response.json();
|
notice = 'Library download queued. Progress will update below.'; await load();
|
||||||
if (!response.ok) throw new Error(result.error ?? 'Could not start this operation.');
|
} catch (cause) { error = (cause as Error).message; } finally { busy = false; }
|
||||||
notice = message; await load(); return true;
|
|
||||||
} catch (cause) { error = (cause as Error).message; return false; }
|
|
||||||
finally { busy = false; }
|
|
||||||
}
|
}
|
||||||
async function startImport(event: SubmitEvent) {
|
function resetUpload() {
|
||||||
event.preventDefault();
|
folder = null; uploadStatus = null; uploadId = ''; phase = 'idle'; uploadError = ''; currentPath = '';
|
||||||
if (await act('/api/game-files', { game, name, version, description, dlc: dlc.split(',').map(value => value.trim()).filter(Boolean), hostId }, 'Import queued. The source host will scan and upload files, then verify and publish this version.')) showImport = false;
|
acceptedBytes = 0; sendingBytes = 0; completedFiles = 0; pauseRequested = false;
|
||||||
|
if (folderInput) folderInput.value = '';
|
||||||
}
|
}
|
||||||
function newVersion(library: OverviewLibrary) {
|
async function newUpload(library?: OverviewLibrary) {
|
||||||
showImport = true; game = library.game; name = library.name; description = library.description;
|
resetUpload(); showUpload = true; game = library?.game ?? 't5'; name = library?.name ?? '';
|
||||||
dlc = library.versions[0]?.dlc.join(', ') ?? ''; version = '';
|
description = library?.description ?? ''; version = '';
|
||||||
document.getElementById('library-import')?.scrollIntoView({ behavior: 'smooth' });
|
await tick(); document.getElementById('library-upload')?.scrollIntoView({ behavior: 'smooth' });
|
||||||
}
|
}
|
||||||
onMount(() => pollVisible(load, 5000));
|
function chooseFolder(event: Event) {
|
||||||
|
uploadError = ''; folder = null;
|
||||||
|
const input = event.currentTarget as HTMLInputElement;
|
||||||
|
if (!input.files?.length) return;
|
||||||
|
try {
|
||||||
|
const selected = selectGameFolder(Array.from(input.files));
|
||||||
|
if (uploadStatus) checkSelection(selected, uploadStatus);
|
||||||
|
folder = selected;
|
||||||
|
if (!name && !uploadId) name = game === 't5' ? 'Black Ops I game files' : 'Black Ops II game files';
|
||||||
|
} catch (cause) { uploadError = (cause as Error).message; input.value = ''; }
|
||||||
|
}
|
||||||
|
function checkSelection(selected: SelectedGameFolder, status: UploadStatus) {
|
||||||
|
const originals = new Map(status.files.map(file => [file.path, file]));
|
||||||
|
if (selected.files.length !== originals.size || selected.files.some(({ path, file }) => {
|
||||||
|
const original = originals.get(path); return !original || original.size !== file.size || original.lastModified !== file.lastModified;
|
||||||
|
})) throw new Error('This folder does not match the original upload. Select the same unchanged folder, including its original file names, sizes, and modification dates. Upload changed game files as a new version.');
|
||||||
|
}
|
||||||
|
function applyStatus(status: UploadStatus) {
|
||||||
|
uploadStatus = status;
|
||||||
|
acceptedBytes = status.files.reduce((total, file) => total + file.receivedBytes, 0);
|
||||||
|
completedFiles = status.files.filter(file => file.complete).length; sendingBytes = 0;
|
||||||
|
}
|
||||||
|
async function getStatus(): Promise<UploadStatus> {
|
||||||
|
const status = await responseBody(await fetch(`/api/game-files/${uploadId}/upload`)) as UploadStatus;
|
||||||
|
applyStatus(status); return status;
|
||||||
|
}
|
||||||
|
async function followVerification(initial: UploadStatus) {
|
||||||
|
let status = initial; phase = 'verifying'; currentPath = '';
|
||||||
|
while (!disposed && status.status === 'verifying') {
|
||||||
|
await new Promise(resolve => setTimeout(resolve, 1500));
|
||||||
|
if (disposed) return;
|
||||||
|
status = await getStatus();
|
||||||
|
}
|
||||||
|
if (disposed) return;
|
||||||
|
if (status.status !== 'published') throw new Error(status.error || 'Verification did not finish. Retry this upload to continue.');
|
||||||
|
phase = 'published'; folder = null; notice = `${name || 'Game files'} ${version} published. You can download this version to a host below.`;
|
||||||
|
if (folderInput) folderInput.value = '';
|
||||||
|
await load();
|
||||||
|
}
|
||||||
|
async function resumeUpload(library: OverviewLibrary, item: OverviewLibrary['versions'][number]) {
|
||||||
|
resetUpload(); showUpload = true; uploadId = item.id; game = library.game; name = library.name; version = item.version; description = item.description;
|
||||||
|
uploadBusy = true;
|
||||||
|
try {
|
||||||
|
const status = await getStatus();
|
||||||
|
if (status.status === 'verifying' || status.status === 'published') await followVerification(status);
|
||||||
|
else { phase = 'paused'; uploadError = status.error ?? ''; }
|
||||||
|
} catch (cause) { uploadError = (cause as Error).message; phase = 'paused'; }
|
||||||
|
finally { uploadBusy = false; }
|
||||||
|
await tick(); document.getElementById('library-upload')?.scrollIntoView({ behavior: 'smooth' });
|
||||||
|
}
|
||||||
|
function pauseUpload() { pauseRequested = true; activeRequest?.abort(); }
|
||||||
|
async function sendChunk(path: string, file: File, offset: number, length: number): Promise<number> {
|
||||||
|
const chunk = file.slice(offset, offset + length);
|
||||||
|
const hash = await crypto.subtle.digest('SHA-256', await chunk.arrayBuffer());
|
||||||
|
if (pauseRequested || disposed) throw new Error('Upload paused.');
|
||||||
|
const sha256 = [...new Uint8Array(hash)].map(value => value.toString(16).padStart(2, '0')).join('');
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const request = new XMLHttpRequest(); activeRequest = request;
|
||||||
|
request.open('PUT', `/api/game-files/${uploadId}/upload/files/${path.split('/').map(encodeURIComponent).join('/')}`);
|
||||||
|
request.setRequestHeader('Content-Type', 'application/octet-stream');
|
||||||
|
request.setRequestHeader('X-Upload-Offset', String(offset)); request.setRequestHeader('X-Upload-Length', String(length));
|
||||||
|
request.setRequestHeader('X-Content-SHA256', sha256); request.timeout = 300_000;
|
||||||
|
request.upload.onprogress = event => { sendingBytes = Math.min(length, event.loaded); };
|
||||||
|
request.onload = () => {
|
||||||
|
activeRequest = null;
|
||||||
|
try {
|
||||||
|
const body = JSON.parse(request.responseText);
|
||||||
|
if (request.status < 200 || request.status >= 300) throw new Error(body.error ?? `Upload failed (${request.status}).`);
|
||||||
|
if (body.receivedBytes !== offset + length) throw new Error('The upload position changed. Resume to check the saved progress.');
|
||||||
|
resolve(body.receivedBytes);
|
||||||
|
} catch (cause) { reject(cause); }
|
||||||
|
};
|
||||||
|
request.onerror = () => { activeRequest = null; reject(new Error('The connection was interrupted. Resume to continue from the last saved chunk.')); };
|
||||||
|
request.ontimeout = () => { activeRequest = null; reject(new Error('This chunk timed out. Resume to check saved progress and retry.')); };
|
||||||
|
request.onabort = () => { activeRequest = null; reject(new Error('Upload paused.')); };
|
||||||
|
request.send(chunk);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
async function startUpload(event: SubmitEvent) {
|
||||||
|
event.preventDefault(); if (uploadBusy || !folder) return;
|
||||||
|
if (!globalThis.crypto?.subtle) { uploadError = 'Folder uploads require a secure connection. Open this controller over HTTPS or localhost.'; return; }
|
||||||
|
uploadBusy = true; pauseRequested = false; uploadError = ''; notice = ''; phase = 'preparing';
|
||||||
|
try {
|
||||||
|
if (!uploadId) {
|
||||||
|
const result = await responseBody(await fetch('/api/game-files', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ game, name, version, description, files: folder.files.map(({ path, file }) => ({ path, size: file.size, lastModified: file.lastModified })) }) }));
|
||||||
|
uploadId = result.version.id;
|
||||||
|
}
|
||||||
|
const status = await getStatus();
|
||||||
|
if (status.status === 'verifying' || status.status === 'published') { await followVerification(status); return; }
|
||||||
|
checkSelection(folder, status);
|
||||||
|
const localFiles = new Map(folder.files.map(item => [item.path, item.file]));
|
||||||
|
const chunkSize = Math.min(8_388_608, Math.max(1, status.chunkSize));
|
||||||
|
phase = 'uploading';
|
||||||
|
for (const remote of status.files) {
|
||||||
|
if (pauseRequested || disposed) break;
|
||||||
|
if (remote.complete) continue;
|
||||||
|
currentPath = remote.path;
|
||||||
|
const file = localFiles.get(remote.path)!;
|
||||||
|
do {
|
||||||
|
if (pauseRequested || disposed) break;
|
||||||
|
const length = Math.min(chunkSize, remote.size - remote.receivedBytes);
|
||||||
|
remote.receivedBytes = await sendChunk(remote.path, file, remote.receivedBytes, length);
|
||||||
|
acceptedBytes += length; sendingBytes = 0;
|
||||||
|
if (remote.receivedBytes === remote.size) { remote.complete = true; completedFiles++; }
|
||||||
|
} while (!remote.complete);
|
||||||
|
}
|
||||||
|
if (pauseRequested || disposed) { phase = 'paused'; return; }
|
||||||
|
phase = 'verifying'; currentPath = '';
|
||||||
|
const verification = await responseBody(await fetch(`/api/game-files/${uploadId}/upload/complete`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: '{}' })) as UploadStatus;
|
||||||
|
applyStatus(verification); await followVerification(verification);
|
||||||
|
} catch (cause) {
|
||||||
|
if (!disposed) { uploadError = pauseRequested ? '' : (cause as Error).message; phase = 'paused'; }
|
||||||
|
} finally { uploadBusy = false; sendingBytes = 0; if (!disposed) void load(); }
|
||||||
|
}
|
||||||
|
beforeNavigate(navigation => {
|
||||||
|
if (!transferring) return;
|
||||||
|
if (navigation.type === 'leave' || !window.confirm('This upload is still transferring files. Leave this page? You can resume later by selecting the original unchanged folder.')) navigation.cancel();
|
||||||
|
});
|
||||||
|
onMount(() => {
|
||||||
|
const stop = pollVisible(load, 5000);
|
||||||
|
return () => { disposed = true; pauseRequested = true; activeRequest?.abort(); stop(); };
|
||||||
|
});
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<svelte:head><title>Game Files | Electric Cherry</title></svelte:head>
|
<svelte:head><title>Game Files | Electric Cherry</title></svelte:head>
|
||||||
<div class="page">
|
<div class="page">
|
||||||
<SectionHeader title="Game Files" subtitle="Import immutable base-game versions and distribute verified copies to your hosts.">
|
<SectionHeader title="Game Files" subtitle="Upload your T5 or T6 game folder, then distribute verified versions to your hosts.">
|
||||||
{#snippet actions()}{#if canManage}<Button onclick={() => showImport = !showImport}>{showImport ? 'Close import' : 'Import game files'}</Button>{/if}{/snippet}
|
{#snippet actions()}{#if canManage}<Button disabled={uploadBusy} onclick={() => newUpload()}>Upload game folder</Button>{/if}{/snippet}
|
||||||
</SectionHeader>
|
</SectionHeader>
|
||||||
{#if error}<p class="ui-error" role="alert">{error}</p>{/if}
|
{#if error}<p class="ui-error" role="alert">{error}</p>{/if}
|
||||||
{#if notice}<p role="status">{notice}</p>{/if}
|
{#if notice}<p role="status">{notice}</p>{/if}
|
||||||
{#if !canManage}<p class="muted">Administrators import game files and distribute verified copies to hosts.</p>{/if}
|
{#if !canManage}<p class="muted">Administrators upload game files and distribute verified copies to hosts.</p>{/if}
|
||||||
{#if canManage && showImport}
|
{#if canManage && showUpload}
|
||||||
<section class="card" id="library-import">
|
<section class="card" id="library-upload">
|
||||||
<h2>Import an existing game volume</h2>
|
<div class="heading"><h2>{uploadId ? phase === 'published' ? 'Version published' : 'Continue folder upload' : 'Upload a game folder'}</h2><Button size="sm" variant="ghost" disabled={uploadBusy} onclick={() => showUpload = false}>Close</Button></div>
|
||||||
<p>The selected host scans its {game}_game_files Docker volume, calculates SHA256 checksums, and streams a copy into controller storage. Publication requires every file to pass verification.</p>
|
<p>Select your complete T5 or T6 game folder, including its installed DLC. The files upload directly to this controller; a game host is only needed when you are ready to run servers.</p>
|
||||||
<form onsubmit={startImport}>
|
{#if uploadError}<p class="ui-error" role="alert">{uploadError}</p>{/if}
|
||||||
<div class="fields">
|
<form onsubmit={startUpload}>
|
||||||
|
<fieldset disabled={uploadBusy || !!uploadId} class="fields">
|
||||||
<label>Game<select bind:value={game}><option value="t5">Black Ops I (T5)</option><option value="t6">Black Ops II (T6)</option></select></label>
|
<label>Game<select bind:value={game}><option value="t5">Black Ops I (T5)</option><option value="t6">Black Ops II (T6)</option></select></label>
|
||||||
<label>Source host<select required bind:value={hostId}><option value="">Select an online host</option>{#each onlineHosts as host}<option value={host.id}>{host.name}</option>{/each}</select></label>
|
<label>Library name<input required maxlength="160" bind:value={name} placeholder="Black Ops II game files" /></label>
|
||||||
<label>Library name<input required maxlength="160" bind:value={name} placeholder="Black Ops II — Complete Server Files" /></label>
|
<label>Version label<input required maxlength="120" bind:value={version} placeholder="2026-10-04" /></label>
|
||||||
<label>New version label<input required maxlength="120" bind:value={version} placeholder="2026-10-03" /></label>
|
<label class="wide">Description <span class="muted">(optional)</span><textarea maxlength="4000" bind:value={description} rows="2"></textarea></label>
|
||||||
<label class="wide">Description<textarea maxlength="4000" bind:value={description} rows="2"></textarea></label>
|
</fieldset>
|
||||||
<label class="wide">Content / DLC (comma separated)<input bind:value={dlc} placeholder="Base game, Nuketown, Die Rise, Mob of the Dead, Buried, Origins" /></label>
|
{#if phase !== 'verifying' && phase !== 'published'}
|
||||||
|
<label class="folder-picker">{uploadId ? 'Select the original unchanged folder' : 'Game folder'}<input bind:this={folderInput} type="file" webkitdirectory multiple onchange={chooseFolder} disabled={uploadBusy} aria-describedby="folder-help" /></label>
|
||||||
|
<p class="muted" id="folder-help">{uploadId ? 'To resume, file names, sizes, and modification dates must match the original upload.' : 'Choose the game directory itself. Its top-level folder name is removed once; all files and subfolders inside it keep their paths.'}</p>
|
||||||
|
{/if}
|
||||||
|
{#if folder}<div class="folder-summary"><strong>{folder.name}</strong><span>{folder.files.length.toLocaleString()} files · {formatGameBytes(folder.size)}</span><details><summary>Preview folder paths</summary><ul class="path-list">{#each folder.files.slice(0, 8) as file}<li>{file.path}</li>{/each}</ul>{#if folder.files.length > 8}<p class="muted">And {(folder.files.length - 8).toLocaleString()} more files.</p>{/if}</details></div>{/if}
|
||||||
|
{#if phase !== 'idle'}<div class="upload-progress" aria-live="polite">
|
||||||
|
<div class="heading"><strong>{phase === 'preparing' ? 'Preparing upload…' : phase === 'uploading' ? 'Uploading game files' : phase === 'verifying' ? 'Verifying and publishing…' : phase === 'published' ? 'Published and ready to distribute' : 'Upload paused'}</strong><span>{completedFiles.toLocaleString()} / {totalFiles.toLocaleString()} files</span></div>
|
||||||
|
<progress max={totalBytes || Math.max(1, totalFiles)} value={totalBytes ? progressBytes : completedFiles} aria-label="Game folder upload progress"></progress>
|
||||||
|
<p>{formatGameBytes(progressBytes)} / {formatGameBytes(totalBytes)} saved or transferring</p>
|
||||||
|
{#if currentPath}<p class="current-file">{currentPath}</p>{/if}
|
||||||
|
{#if phase === 'verifying'}<p class="muted">The controller checks every file and publishes this version when verification finishes. Verification continues if you leave this page.</p>{:else if phase === 'paused'}<p class="muted">Saved chunks are kept on the controller. Resume this upload to check the saved progress and retry.</p>{/if}
|
||||||
|
</div>{/if}
|
||||||
|
<div class="upload-actions">
|
||||||
|
{#if transferring}<Button onclick={pauseUpload} disabled={pauseRequested}>{pauseRequested ? 'Pausing…' : 'Pause upload'}</Button>
|
||||||
|
{:else if phase !== 'verifying' && phase !== 'published'}<Button type="submit" variant="primary" disabled={uploadBusy || !folder}>{uploadId ? 'Resume upload' : 'Upload and publish'}</Button>{/if}
|
||||||
|
{#if phase === 'idle' || phase === 'uploading'}<span class="muted">Keep this page open while files transfer. Interrupted uploads can be resumed below.</span>{/if}
|
||||||
</div>
|
</div>
|
||||||
<Button type="submit" disabled={busy || !onlineHosts.length}>Start import</Button>
|
|
||||||
{#if !onlineHosts.length}<p>Enroll an online source host in <a href="/hosts">Hosts</a> before importing.</p>{/if}
|
|
||||||
</form>
|
</form>
|
||||||
</section>
|
</section>
|
||||||
{/if}
|
{/if}
|
||||||
<div class="filters"><label>Game<select bind:value={filterGame}><option value="all">All games</option><option value="t5">Black Ops I</option><option value="t6">Black Ops II</option></select></label><span class="muted">{libraries.length} libraries · Base-game files and runtime images are versioned separately.</span></div>
|
<div class="filters"><label>Game<select bind:value={filterGame}><option value="all">All games</option><option value="t5">Black Ops I</option><option value="t6">Black Ops II</option></select></label><span class="muted">{libraries.length} libraries · Game folders and runtime images are versioned separately.</span></div>
|
||||||
{#each matching as library (library.id)}
|
{#each matching as library (library.id)}
|
||||||
<section class="card">
|
<section class="card">
|
||||||
<div class="heading"><div><h2>{library.name}</h2><p class="muted">{library.game.toUpperCase()} · {library.description}</p></div>{#if canManage}<Button size="sm" onclick={() => newVersion(library)}>Import new version</Button>{/if}</div>
|
<div class="heading"><div><h2>{library.name}</h2><p class="muted">{library.game.toUpperCase()} · {library.description}</p></div>{#if canManage}<Button size="sm" disabled={uploadBusy} onclick={() => newUpload(library)}>Upload new version</Button>{/if}</div>
|
||||||
{#each library.versions as item (item.id)}
|
{#each library.versions as item (item.id)}
|
||||||
|
{@const browserUpload = !item.sourceHostId}
|
||||||
<article class="version">
|
<article class="version">
|
||||||
<div class="heading"><h3>{item.version}</h3><Badge variant={item.state === 'failed' ? 'danger' : item.state === 'ready' ? 'online' : 'info'}>{item.state === 'ready' ? 'Published' : item.state === 'importing' ? 'Importing' : 'Failed'}</Badge></div>
|
<div class="heading"><h3>{item.version}</h3><Badge variant={item.state === 'failed' ? 'danger' : item.state === 'ready' ? 'online' : 'info'}>{item.state === 'ready' ? 'Published' : item.state === 'importing' ? browserUpload ? item.id === uploadId && phase === 'verifying' ? 'Verifying' : 'Uploading' : 'Importing' : 'Failed'}</Badge></div>
|
||||||
<p>{formatGameBytes(item.sizeBytes)} · {item.fileCount.toLocaleString()} files · Created {new Date(item.createdAt).toLocaleString()}</p>
|
<p>{formatGameBytes(item.sizeBytes)} · {item.fileCount.toLocaleString()} files · Created {new Date(item.createdAt).toLocaleString()}</p>
|
||||||
{#if item.dlc.length}<p>Content: {item.dlc.join(', ')}</p>{/if}
|
|
||||||
{#if item.error}<p class="ui-error">{item.error}</p>{/if}
|
{#if item.error}<p class="ui-error">{item.error}</p>{/if}
|
||||||
<details><summary>Version details</summary><dl><dt>Immutable version ID</dt><dd>{item.id}</dd><dt>Manifest SHA256</dt><dd>{item.manifestSha256 ?? 'Waiting for source scan'}</dd><dt>Controller storage</dt><dd>{item.storageLocation}</dd><dt>Imported from</dt><dd>{hostName(item.sourceHostId)} / {item.sourceVolume}</dd></dl><a href={`/api/game-files/${item.id}`} target="_blank" rel="noreferrer">View complete file manifest</a></details>
|
<details><summary>Version details</summary><dl><dt>Immutable version ID</dt><dd>{item.id}</dd><dt>Manifest SHA256</dt><dd>{item.manifestSha256 ?? 'Available after verification'}</dd><dt>Uploaded from</dt><dd>{browserUpload ? 'Game folder in browser' : `${hostName(item.sourceHostId)} / ${item.sourceVolume}`}</dd></dl>{#if item.state === 'ready'}<a href={`/api/game-files/${item.id}`} target="_blank" rel="noreferrer">View complete file manifest</a>{/if}</details>
|
||||||
{#if item.state === 'ready'}
|
{#if item.state === 'ready'}
|
||||||
{#if canManage}<div class="download"><label>Download to host<select bind:value={targets[item.id]}><option value="">Select host</option>{#each onlineHosts as host}<option value={host.id}>{host.name} · {formatGameBytes(Number(host.inventory.storageAvailable ?? 0))} free</option>{/each}</select></label><Button size="sm" disabled={busy || !targets[item.id]} onclick={() => act(`/api/game-files/${item.id}/download`, { hostId: targets[item.id] }, 'Library download queued. Progress will update below.')}>Download / verify</Button></div>{/if}
|
{#if canManage}<div class="download"><label>Download to host<select bind:value={targets[item.id]}><option value="">Select host</option>{#each onlineHosts as host}<option value={host.id}>{host.name} · {formatGameBytes(Number(host.inventory.storageAvailable ?? 0))} free</option>{/each}</select></label><Button size="sm" disabled={busy || !targets[item.id]} onclick={() => download(item.id)}>Download / verify</Button></div>{#if !onlineHosts.length}<p class="muted">This version is stored and ready. <a href="/hosts">Connect a host</a> when you want to deploy game servers.</p>{/if}{/if}
|
||||||
<div class="table-wrap"><table><thead><tr><th>Host</th><th>Cache state</th><th>Progress</th><th>Detail</th></tr></thead><tbody>
|
<div class="table-wrap"><table><thead><tr><th>Host</th><th>Cache state</th><th>Progress</th><th>Detail</th></tr></thead><tbody>
|
||||||
{#each item.caches as cache}<tr><td><a href={`/hosts/${cache.hostId}`}>{hostName(cache.hostId)}</a></td><td>{cache.status}{!hosts.find(host => host.id === cache.hostId)?.connected ? ' · offline' : ''}</td><td>{#if cache.status === 'downloading' || cache.status === 'verifying'}<progress max={cache.bytesTotal || 1} value={cache.bytesDownloaded}></progress><small>{formatGameBytes(cache.bytesDownloaded)} / {formatGameBytes(cache.bytesTotal)}</small>{:else if cache.status === 'ready' || cache.status === 'outdated'}Verified{cache.status === 'outdated' ? ' · older version' : ''}{:else}—{/if}</td><td>{cache.error ?? ''}</td></tr>{:else}<tr><td colspan="4">Enroll a host to cache this version.</td></tr>{/each}
|
{#each item.caches as cache}<tr><td><a href={`/hosts/${cache.hostId}`}>{hostName(cache.hostId)}</a></td><td>{cache.status}{!hosts.find(host => host.id === cache.hostId)?.connected ? ' · offline' : ''}</td><td>{#if cache.status === 'downloading' || cache.status === 'verifying'}<progress max={cache.bytesTotal || 1} value={cache.bytesDownloaded} aria-label="Host download progress"></progress><small>{formatGameBytes(cache.bytesDownloaded)} / {formatGameBytes(cache.bytesTotal)}</small>{:else if cache.status === 'ready' || cache.status === 'outdated'}Verified{cache.status === 'outdated' ? ' · older version' : ''}{:else}—{/if}</td><td>{cache.error ?? ''}</td></tr>{:else}<tr><td colspan="4">No host copies yet.</td></tr>{/each}
|
||||||
</tbody></table></div>
|
</tbody></table></div>
|
||||||
{:else if item.state === 'importing'}<p class="muted">The agent operation continues independently of this page. An interrupted import is reconciled when the host reconnects.</p>{/if}
|
{:else if browserUpload}<p class="muted">Resume with the original unchanged folder. Published versions can be downloaded to any enrolled host.</p>{#if canManage}<div><Button size="sm" disabled={uploadBusy} onclick={() => resumeUpload(library, item)}>Resume upload</Button></div>{/if}
|
||||||
|
{:else if item.state === 'importing'}<p class="muted">The existing host import continues independently. It is reconciled when the host reconnects.</p>{/if}
|
||||||
</article>
|
</article>
|
||||||
{/each}
|
{/each}
|
||||||
</section>
|
</section>
|
||||||
{:else}{#if loaded}<EmptyState title="No game libraries yet" description="Administrators can import T5 or T6 game files from an enrolled host. Each published version can be shared by many instances." />{:else}<p role="status">Loading game libraries…</p>{/if}{/each}
|
{:else}{#if loaded}<EmptyState title="No game libraries yet" description="Upload a complete T5 or T6 game folder to get started. You can add a host after the files are published." />{:else}<p role="status">Loading game libraries…</p>{/if}{/each}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<style>
|
<style>
|
||||||
.page, .card, .version, form { display:grid; gap:20px; }
|
.page, .card, .version, form { display:grid; gap:20px; }
|
||||||
.card { padding: 24px; border: 1px solid var(--border-subtle); background: var(--surface); border-radius: var(--radius-md); }
|
.card { padding:24px; min-width:0; border:1px solid var(--border-subtle); background:var(--surface); border-radius:var(--radius-md); }
|
||||||
.version { padding-top:20px; border-top:1px solid var(--border-subtle); }
|
.version { padding-top:20px; border-top:1px solid var(--border-subtle); }
|
||||||
.heading, .filters, .download { display: flex; gap: 16px; align-items: center; justify-content: space-between; flex-wrap: wrap; }
|
.heading, .filters, .download, .upload-actions { display:flex; gap:16px; align-items:center; justify-content:space-between; flex-wrap:wrap; }
|
||||||
.download { justify-content: flex-start; align-items: end; }
|
.download, .upload-actions { justify-content:flex-start; align-items:end; }
|
||||||
.fields { display: grid; grid-template-columns: repeat(2, minmax(0, 1fr)); gap: 16px; }
|
.fields { display:grid; grid-template-columns:repeat(2,minmax(0,1fr)); gap:16px; border:0; padding:0; margin:0; min-width:0; }
|
||||||
.wide { grid-column:1/-1; } label { display:grid; gap:6px; font-size:var(--text-sm); }
|
.wide { grid-column:1/-1; } label { display:grid; gap:6px; font-size:var(--text-sm); }
|
||||||
input, select, textarea { width:100%; min-width:0; box-sizing:border-box; padding:10px; background:var(--surface); color:inherit; border:1px solid var(--border-strong); border-radius:var(--radius-sm); }
|
input, select, textarea { width:100%; min-width:0; box-sizing:border-box; padding:10px; background:var(--surface); color:inherit; border:1px solid var(--border-strong); border-radius:var(--radius-sm); }
|
||||||
|
input:disabled, select:disabled, textarea:disabled { opacity:.7; } input[type='file'] { padding:16px; cursor:pointer; }
|
||||||
|
.folder-summary,.upload-progress { padding:20px; display:grid; gap:12px; background:var(--surface-sunken); border:1px solid var(--border-subtle); border-radius:var(--radius-md); min-width:0; }
|
||||||
|
.folder-summary>span,.upload-progress span,.upload-actions>span { color:var(--text-secondary); font-size:var(--text-sm); }
|
||||||
|
.path-list { padding-left:20px; font-family:var(--font-mono); font-size:var(--text-xs); overflow-wrap:anywhere; }
|
||||||
|
.current-file { font-family:var(--font-mono); font-size:var(--text-xs); overflow-wrap:anywhere; } progress { width:100%; accent-color:var(--accent-primary); }
|
||||||
h2,h3,p { margin:0; } h2 { font-size:var(--text-md); } h3 { font-size:var(--text-sm); }
|
h2,h3,p { margin:0; } h2 { font-size:var(--text-md); } h3 { font-size:var(--text-sm); }
|
||||||
p,td,th,details { font-size:var(--text-sm); line-height:1.6; } .muted,small { color:var(--text-secondary); }
|
p,td,th,details { font-size:var(--text-sm); line-height:1.6; } .muted,small { color:var(--text-secondary); }
|
||||||
small { display:block; } a { color:var(--accent-primary-hover); } summary { cursor:pointer; }
|
small { display:block; } a { color:var(--accent-primary-hover); } summary { cursor:pointer; }
|
||||||
dl { display:grid; grid-template-columns:auto 1fr; gap:6px 16px; } dd { margin:0; overflow-wrap:anywhere; }
|
dl { display:grid; grid-template-columns:auto 1fr; gap:6px 16px; } dd { margin:0; overflow-wrap:anywhere; }
|
||||||
.table-wrap { overflow-x:auto; } table { width:100%; border-collapse:collapse; text-align:left; } th,td { padding:10px; border-bottom:1px solid var(--border-subtle); }
|
.table-wrap { overflow-x:auto; } table { width:100%; border-collapse:collapse; text-align:left; } th,td { padding:10px; border-bottom:1px solid var(--border-subtle); }
|
||||||
@media(max-width: 640px) { .fields { grid-template-columns: 1fr; } .card { padding: 16px; } dl { grid-template-columns: 1fr; } }
|
@media(max-width:640px) { .fields { grid-template-columns:1fr; } .card,.folder-summary,.upload-progress { padding:16px; } dl { grid-template-columns:1fr; } .heading>div { min-width:0; } }
|
||||||
</style>
|
</style>
|
||||||
|
|||||||
@@ -49,7 +49,8 @@ describe('management route permissions', () => {
|
|||||||
['/api/hosts', 'POST'], ['/api/hosts/[id]', 'PATCH'], ['/api/hosts/[id]/enrollment', 'POST'],
|
['/api/hosts', 'POST'], ['/api/hosts/[id]', 'PATCH'], ['/api/hosts/[id]/enrollment', 'POST'],
|
||||||
['/api/hosts/[id]/credentials', 'DELETE'], ['/api/hosts/[id]/credentials', 'POST'],
|
['/api/hosts/[id]/credentials', 'DELETE'], ['/api/hosts/[id]/credentials', 'POST'],
|
||||||
['/api/hosts/[id]/operations', 'POST'], ['/api/settings', 'PUT'], ['/api/game-files', 'POST'],
|
['/api/hosts/[id]/operations', 'POST'], ['/api/settings', 'PUT'], ['/api/game-files', 'POST'],
|
||||||
['/api/game-files/[id]/download', 'POST'], ['/api/maintenance', 'POST'], ['/api/watchdog', 'POST'], ['/api/plutonium', 'POST']
|
['/api/game-files/[id]/download', 'POST'], ['/api/game-files/[id]/upload/files/[...path]', 'PUT'],
|
||||||
|
['/api/game-files/[id]/upload/complete', 'POST'], ['/api/maintenance', 'POST'], ['/api/watchdog', 'POST'], ['/api/plutonium', 'POST']
|
||||||
])('protects host, game-file, and global policy mutations at %s', (route, method) => {
|
])('protects host, game-file, and global policy mutations at %s', (route, method) => {
|
||||||
expect(routePermission(route, method)).toBe('admin');
|
expect(routePermission(route, method)).toBe('admin');
|
||||||
expect(allows('operator', route, method)).toBe(false);
|
expect(allows('operator', route, method)).toBe(false);
|
||||||
|
|||||||
@@ -0,0 +1,227 @@
|
|||||||
|
import Database from 'better-sqlite3';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import { mkdir, mkdtemp, readFile, rename, rm, stat } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { dirname, join } from 'node:path';
|
||||||
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
|
const state = vi.hoisted(() => ({ handle: null as Database.Database | null }));
|
||||||
|
vi.mock('../../src/lib/server/db/index.js', () => ({ db: () => state.handle }));
|
||||||
|
import {
|
||||||
|
browserUploadStatus, createLibraryImport, createLibraryUpload, getHostLibraryCache,
|
||||||
|
getLibraryVersion, listLibraries, publishBrowserUpload, queueLibraryDownload,
|
||||||
|
requestBrowserPublication, requireReadyLibrary, updateHostLibraryCaches, uploadBrowserFile
|
||||||
|
} from '../../src/lib/server/game-files/index.js';
|
||||||
|
import { createEnrollment, createHost, enrollHost, listHosts } from '../../src/lib/server/hosts/index.js';
|
||||||
|
import { manifestChecksum } from '../../src/lib/server/game-files/storage.js';
|
||||||
|
import { serveGameFile } from '../../src/lib/server/game-files/http.js';
|
||||||
|
import { GET as manifestRoute } from '../../src/routes/api/agent/libraries/[id]/manifest/+server.js';
|
||||||
|
|
||||||
|
let directory: string;
|
||||||
|
const modified = 1_700_000_000_000;
|
||||||
|
const bytes = Buffer.from('verified Plutonium game content');
|
||||||
|
const sha = (value: Uint8Array) => createHash('sha256').update(value).digest('hex');
|
||||||
|
const file = (path = 'zone/base.ff', content = bytes) => ({ path, size: content.length, lastModified: modified });
|
||||||
|
const upload = (id: string, path: string, content: Buffer, offset = 0, checksum = sha(content)) =>
|
||||||
|
uploadBrowserFile(id, path, { offset, size: content.length, sha256: checksum }, new Response(new Uint8Array(content)).body);
|
||||||
|
const begin = (game: 't5' | 't6' = 't6', files = [file()], version = '1') =>
|
||||||
|
createLibraryUpload({ game, name: `Plutonium ${game.toUpperCase()}`, version, description: 'Uploaded game folder', files });
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
directory = await mkdtemp(join(tmpdir(), 'ec-browser-library-'));
|
||||||
|
state.handle = new Database(join(directory, 'controller.sqlite'));
|
||||||
|
state.handle.pragma('foreign_keys=ON');
|
||||||
|
vi.stubEnv('GAME_FILES_ROOT', join(directory, 'libraries'));
|
||||||
|
});
|
||||||
|
afterEach(async () => {
|
||||||
|
state.handle?.close(); state.handle = null;
|
||||||
|
await rm(directory, { recursive: true, force: true });
|
||||||
|
vi.unstubAllEnvs();
|
||||||
|
});
|
||||||
|
|
||||||
|
function host() {
|
||||||
|
const created = createHost({ name: 'Fresh download target' });
|
||||||
|
const { credential } = enrollHost(createEnrollment(created.id).token, {
|
||||||
|
protocolVersion: 1, dockerRunning: true, directoriesWritable: true, storageAvailable: 100 * 1024 ** 3
|
||||||
|
});
|
||||||
|
return { id: created.id, credential };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('browser folder uploads', () => {
|
||||||
|
it.each(['t5', 't6'] as const)('publishes %s from a browser without an enrolled host or DLC declarations', async game => {
|
||||||
|
expect(listHosts()).toEqual([]);
|
||||||
|
const initial = begin(game);
|
||||||
|
expect(initial).toMatchObject({ state: 'importing', operationId: null, sourceHostId: null, sourceVolume: null });
|
||||||
|
expect(initial).not.toHaveProperty('dlc');
|
||||||
|
expect(await browserUploadStatus(initial.id)).toMatchObject({
|
||||||
|
status: 'pending', files: [{ ...file(), receivedBytes: 0, complete: false }]
|
||||||
|
});
|
||||||
|
await upload(initial.id, file().path, bytes);
|
||||||
|
const accepted = await requestBrowserPublication(initial.id);
|
||||||
|
expect(['verifying', 'published']).toContain(accepted.status);
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
expect(ready).toMatchObject({ state: 'ready', sizeBytes: bytes.length, fileCount: 1 });
|
||||||
|
expect(ready.files).toEqual([{ path: file().path, size: bytes.length, sha256: sha(bytes) }]);
|
||||||
|
expect(ready.manifestSha256).toBe(manifestChecksum(ready.files));
|
||||||
|
expect(await readFile(join(ready.storageLocation, file().path))).toEqual(bytes);
|
||||||
|
expect(await browserUploadStatus(initial.id)).toMatchObject({ status: 'published' });
|
||||||
|
expect(listLibraries()[0].versions[0]).not.toHaveProperty('dlc');
|
||||||
|
expect(listHosts()).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('preserves nested relative contents and derives a canonical manifest in path order', async () => {
|
||||||
|
const selected = [file('zone/english/a.ff'), file('main/iw_00.iwd'), file('empty.txt', Buffer.alloc(0))];
|
||||||
|
const initial = begin('t6', selected);
|
||||||
|
for (const item of selected) await upload(initial.id, item.path, item.size ? bytes : Buffer.alloc(0));
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
expect(ready.files.map(item => item.path)).toEqual(['empty.txt', 'main/iw_00.iwd', 'zone/english/a.ff']);
|
||||||
|
expect(ready.files[0].sha256).toBe(sha(Buffer.alloc(0)));
|
||||||
|
expect(ready.manifestSha256).toBe(manifestChecksum(ready.files));
|
||||||
|
expect(await readFile(join(ready.storageLocation, 'zone/english/a.ff'))).toEqual(bytes);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects incomplete publication and allows the remaining files to finish the same version', async () => {
|
||||||
|
const initial = begin('t5', [file('zone/first.ff'), file('zone/second.ff')]);
|
||||||
|
await upload(initial.id, 'zone/first.ff', bytes);
|
||||||
|
await expect(publishBrowserUpload(initial.id)).rejects.toThrow();
|
||||||
|
expect(getLibraryVersion(initial.id)?.state).not.toBe('ready');
|
||||||
|
const hidden = await serveGameFile(new Request('https://ec.test/files'), initial.id, 'zone/first.ff');
|
||||||
|
expect(hidden.status).toBe(404);
|
||||||
|
await upload(initial.id, 'zone/second.ff', bytes);
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
expect(ready.state).toBe('ready');
|
||||||
|
expect(ready.id).toBe(initial.id);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not acknowledge corrupt chunks and permits a verified retry', async () => {
|
||||||
|
const initial = begin();
|
||||||
|
await expect(upload(initial.id, file().path, bytes, 0, '0'.repeat(64))).rejects.toThrow();
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0]).toMatchObject({ receivedBytes: 0, complete: false });
|
||||||
|
await expect(uploadBrowserFile(initial.id, file().path, {
|
||||||
|
offset: 0, size: bytes.length, sha256: sha(bytes)
|
||||||
|
}, new Response(bytes.subarray(0, -1)).body)).rejects.toThrow();
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0].receivedBytes).toBe(0);
|
||||||
|
await upload(initial.id, file().path, bytes);
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0]).toMatchObject({ receivedBytes: bytes.length, complete: true });
|
||||||
|
expect((await publishBrowserUpload(initial.id)).state).toBe('ready');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('surfaces asynchronous incomplete publication as retryable failure', async () => {
|
||||||
|
const initial = begin();
|
||||||
|
await requestBrowserPublication(initial.id);
|
||||||
|
await vi.waitFor(async () => expect((await browserUploadStatus(initial.id)).status).toBe('failed'));
|
||||||
|
expect((await browserUploadStatus(initial.id)).error).toBeTruthy();
|
||||||
|
await upload(initial.id, file().path, bytes);
|
||||||
|
expect(await browserUploadStatus(initial.id)).toMatchObject({ status: 'pending', error: null });
|
||||||
|
await requestBrowserPublication(initial.id);
|
||||||
|
expect((await publishBrowserUpload(initial.id)).state).toBe('ready');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('resumes from the verified chunk boundary and safely retries a lost acknowledgement', async () => {
|
||||||
|
const sizeProbe = begin();
|
||||||
|
const chunkSize = (await browserUploadStatus(sizeProbe.id)).chunkSize;
|
||||||
|
expect(chunkSize).toBeGreaterThan(0);
|
||||||
|
const large = Buffer.alloc(chunkSize + 17, 0x41);
|
||||||
|
const initial = begin('t6', [file('main/archive.iwd', large)], 'large');
|
||||||
|
const first = large.subarray(0, chunkSize);
|
||||||
|
await upload(initial.id, 'main/archive.iwd', first);
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0]).toMatchObject({ receivedBytes: chunkSize, complete: false });
|
||||||
|
state.handle!.close();
|
||||||
|
state.handle = new Database(join(directory, 'controller.sqlite'));
|
||||||
|
state.handle.pragma('foreign_keys=ON');
|
||||||
|
await expect(upload(initial.id, 'main/archive.iwd', first)).rejects.toMatchObject({ receivedBytes: chunkSize });
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0].receivedBytes).toBe(chunkSize);
|
||||||
|
await upload(initial.id, 'main/archive.iwd', large.subarray(chunkSize), chunkSize);
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
expect(ready.files[0].sha256).toBe(sha(large));
|
||||||
|
expect((await readFile(join(ready.storageLocation, 'main/archive.iwd'))).equals(large)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects unknown file paths, traversal, and chunks outside the declared file', async () => {
|
||||||
|
const initial = begin();
|
||||||
|
await expect(upload(initial.id, 'zone/unselected.ff', bytes)).rejects.toThrow();
|
||||||
|
await expect(upload(initial.id, '../escape.ff', bytes)).rejects.toThrow();
|
||||||
|
await expect(upload(initial.id, file().path, bytes, bytes.length + 1)).rejects.toThrow();
|
||||||
|
await expect(upload(initial.id, file().path, Buffer.concat([bytes, bytes]))).rejects.toThrow();
|
||||||
|
expect((await browserUploadStatus(initial.id)).files[0].receivedBytes).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps ready bytes and version metadata immutable and requires a new version label', async () => {
|
||||||
|
const initial = begin(); await upload(initial.id, file().path, bytes);
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
await expect(upload(initial.id, file().path, bytes)).rejects.toThrow();
|
||||||
|
expect(() => state.handle!.prepare('UPDATE game_file_versions SET description=? WHERE id=?').run('changed', ready.id)).toThrow(/immutable/);
|
||||||
|
expect(() => begin()).toThrow(/already exists/);
|
||||||
|
expect((await publishBrowserUpload(initial.id)).manifestSha256).toBe(ready.manifestSha256);
|
||||||
|
expect(begin('t6', [file()], '2').id).not.toBe(ready.id);
|
||||||
|
expect(await readFile(join(ready.storageLocation, file().path))).toEqual(bytes);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('recovers publication after the verified directory was renamed but the ready commit was interrupted', async () => {
|
||||||
|
const initial = begin(); await upload(initial.id, file().path, bytes);
|
||||||
|
const files = [{ path: file().path, size: bytes.length, sha256: sha(bytes) }];
|
||||||
|
const checksum = manifestChecksum(files);
|
||||||
|
state.handle!.transaction(() => {
|
||||||
|
state.handle!.prepare('UPDATE game_file_versions SET manifest=?,manifest_sha256=? WHERE id=?')
|
||||||
|
.run(JSON.stringify(files), checksum, initial.id);
|
||||||
|
state.handle!.prepare('INSERT INTO game_file_entries(version_id,path,size,sha256) VALUES(?,?,?,?)')
|
||||||
|
.run(initial.id, file().path, bytes.length, sha(bytes));
|
||||||
|
})();
|
||||||
|
await mkdir(dirname(initial.storageLocation), { recursive: true });
|
||||||
|
await rename(join(directory, 'libraries', '.imports', initial.id), initial.storageLocation);
|
||||||
|
state.handle!.close(); state.handle = new Database(join(directory, 'controller.sqlite'));
|
||||||
|
state.handle.pragma('foreign_keys=ON');
|
||||||
|
expect(await browserUploadStatus(initial.id)).toMatchObject({ status: 'pending', files: [{ complete: true }] });
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
expect(ready).toMatchObject({ id: initial.id, state: 'ready', manifestSha256: checksum, files });
|
||||||
|
expect(await readFile(join(ready.storageLocation, file().path))).toEqual(bytes);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects chunks racing with final publication without recreating staging files', async () => {
|
||||||
|
const initial = begin(); await upload(initial.id, file().path, bytes);
|
||||||
|
const publishing = publishBrowserUpload(initial.id);
|
||||||
|
await expect(upload(initial.id, file().path, bytes)).rejects.toMatchObject({ status: 409 });
|
||||||
|
const ready = await publishing;
|
||||||
|
expect(ready.state).toBe('ready');
|
||||||
|
expect(await readFile(join(ready.storageLocation, file().path))).toEqual(bytes);
|
||||||
|
await expect(stat(join(directory, 'libraries', '.imports', initial.id))).rejects.toMatchObject({ code: 'ENOENT' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects browser operations against an existing host-sourced import', async () => {
|
||||||
|
const source = host();
|
||||||
|
const legacy = createLibraryImport({ hostId: source.id, game: 't6', name: 'Legacy import', version: '1' });
|
||||||
|
expect(legacy.sourceHostId).toBe(source.id);
|
||||||
|
await expect(Promise.resolve().then(() => browserUploadStatus(legacy.id))).rejects.toThrow();
|
||||||
|
await expect(upload(legacy.id, file().path, bytes)).rejects.toThrow();
|
||||||
|
await expect(publishBrowserUpload(legacy.id)).rejects.toThrow();
|
||||||
|
await expect(Promise.resolve().then(() => requestBrowserPublication(legacy.id))).rejects.toThrow();
|
||||||
|
expect(getLibraryVersion(legacy.id)?.state).toBe('importing');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('distributes uploaded versions through the unchanged agent manifest, ranges, and verified cache contract', async () => {
|
||||||
|
const initial = begin('t5'); await upload(initial.id, file().path, bytes);
|
||||||
|
const ready = await publishBrowserUpload(initial.id);
|
||||||
|
const target = host();
|
||||||
|
const operation = queueLibraryDownload(target.id, ready.id);
|
||||||
|
expect(operation).toMatchObject({ kind: 'library.download', payload: {
|
||||||
|
libraryVersionId: ready.id, game: 't5', manifestSha256: ready.manifestSha256,
|
||||||
|
manifestUrl: `/api/agent/libraries/${ready.id}/manifest`, filesBaseUrl: `/api/agent/libraries/${ready.id}/files`
|
||||||
|
} });
|
||||||
|
const manifest = manifestRoute({ request: new Request(`https://ec.test/api/agent/libraries/${ready.id}/manifest`, {
|
||||||
|
headers: { Authorization: `Bearer ${target.credential}` }
|
||||||
|
}), params: { id: ready.id } });
|
||||||
|
expect(manifest.status).toBe(200);
|
||||||
|
expect(await manifest.json()).toEqual({ id: ready.id, game: 't5', files: ready.files, sizeBytes: bytes.length, manifestSha256: ready.manifestSha256 });
|
||||||
|
const range = await serveGameFile(new Request('https://ec.test/file', { headers: { Range: 'bytes=9-' } }), ready.id, file().path);
|
||||||
|
expect(range.status).toBe(206);
|
||||||
|
expect(Buffer.from(await range.arrayBuffer())).toEqual(bytes.subarray(9));
|
||||||
|
expect(() => requireReadyLibrary(target.id, ready.id)).toThrow(/finish downloading/);
|
||||||
|
updateHostLibraryCaches(target.id, [{ libraryVersionId: ready.id, status: 'ready', manifestSha256: '0'.repeat(64) }]);
|
||||||
|
expect(getHostLibraryCache(target.id, ready.id)?.status).toBe('failed');
|
||||||
|
updateHostLibraryCaches(target.id, [{ libraryVersionId: ready.id, status: 'ready', manifestSha256: ready.manifestSha256 }]);
|
||||||
|
expect(requireReadyLibrary(target.id, ready.id).id).toBe(ready.id);
|
||||||
|
const replacement = begin('t5', [file()], '2'); await upload(replacement.id, file().path, bytes);
|
||||||
|
await publishBrowserUpload(replacement.id);
|
||||||
|
expect(requireReadyLibrary(target.id, ready.id).manifestSha256).toBe(ready.manifestSha256);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import { mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import { join } from 'node:path';
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from 'vitest';
|
||||||
|
import {
|
||||||
|
getUploadFileProgress, hashUploadedFiles, MAX_UPLOAD_CHUNK_BYTES,
|
||||||
|
receiveUploadChunk, UploadOffsetError, validateUploadPlan, withUploadLock
|
||||||
|
} from '../../src/lib/server/game-files/uploads.js';
|
||||||
|
import { manifestChecksum, verifyGameFile } from '../../src/lib/server/game-files/storage.js';
|
||||||
|
|
||||||
|
const sha = (value: string | Uint8Array) => createHash('sha256').update(value).digest('hex');
|
||||||
|
const stream = (value: string) => new Response(value).body!;
|
||||||
|
const file = { path: 'zone/english/map.ff', size: 10, lastModified: 1234 };
|
||||||
|
const chunk = (value: string, offset = 0) => ({ offset, size: Buffer.byteLength(value), sha256: sha(value) });
|
||||||
|
let directory: string;
|
||||||
|
let root: string;
|
||||||
|
|
||||||
|
beforeEach(async () => {
|
||||||
|
directory = await mkdtemp(join(tmpdir(), 'ec-folder-upload-'));
|
||||||
|
root = join(directory, 'import');
|
||||||
|
});
|
||||||
|
afterEach(async () => { await rm(directory, { recursive: true, force: true }); });
|
||||||
|
|
||||||
|
describe('browser folder upload plans', () => {
|
||||||
|
it('preserves folder structure and source identity while sorting canonical paths', () => {
|
||||||
|
expect(validateUploadPlan([{ path: 'zone/map.ff', size: 10, lastModified: 100 }, { path: 'main/data.iwd', size: 0 }])).toEqual([
|
||||||
|
{ path: 'main/data.iwd', size: 0, lastModified: 0 },
|
||||||
|
{ path: 'zone/map.ff', size: 10, lastModified: 100 }
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each([
|
||||||
|
[], [{ path: '../zone.ff', size: 1 }], [{ path: 'zone\\map.ff', size: 1 }],
|
||||||
|
[{ path: 'Zone/a', size: 1 }, { path: 'zone/A', size: 1 }],
|
||||||
|
[{ path: 'ZONE', size: 1 }, { path: 'zone/a', size: 1 }],
|
||||||
|
[{ path: 'zone/a', size: -1 }], [{ path: 'zone/a', size: 1.5 }],
|
||||||
|
[{ path: 'zone/a', size: Number.MAX_SAFE_INTEGER }, { path: 'zone/b', size: 1 }],
|
||||||
|
[{ path: 'zone/a', size: 1, lastModified: -1 }], [{ path: 'zone/a', size: 1, lastModified: 1.5 }],
|
||||||
|
[{ path: 'zone/a', size: 1, lastModified: null }]
|
||||||
|
].map(plan => ({ plan })))('rejects ambiguous or unsafe plan $plan', ({ plan }) => {
|
||||||
|
expect(() => validateUploadPlan(plan)).toThrow();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('bounded resumable file transfer', () => {
|
||||||
|
it('resumes arbitrary chunks by filesystem offset and produces a distribution-compatible manifest', async () => {
|
||||||
|
expect(await getUploadFileProgress(root, file)).toEqual({ receivedBytes: 0, complete: false });
|
||||||
|
expect(await receiveUploadChunk(root, file, chunk('0123'), stream('0123'))).toEqual({ receivedBytes: 4, complete: false });
|
||||||
|
// A new request discovers its checkpoint from disk; no process-local receipt is required.
|
||||||
|
expect(await getUploadFileProgress(root, file)).toEqual({ receivedBytes: 4, complete: false });
|
||||||
|
expect(await receiveUploadChunk(root, file, chunk('456789', 4), stream('456789'))).toEqual({ receivedBytes: 10, complete: true });
|
||||||
|
expect(await readFile(join(root, file.path), 'utf8')).toBe('0123456789');
|
||||||
|
const manifest = await withUploadLock(root, () => hashUploadedFiles(root, [file]));
|
||||||
|
expect(manifest).toEqual([{ path: file.path, size: 10, sha256: sha('0123456789') }]);
|
||||||
|
expect(manifestChecksum(manifest)).toBe(sha(JSON.stringify(manifest)));
|
||||||
|
await verifyGameFile(root, manifest[0]);
|
||||||
|
expect(await readdir(root)).toEqual(['zone']);
|
||||||
|
expect(await readdir(`${root}.chunks`)).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reports committed progress after reply loss instead of appending duplicate bytes', async () => {
|
||||||
|
await receiveUploadChunk(root, file, chunk('0123'), stream('0123'));
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('0123'), stream('0123'))).rejects.toMatchObject({
|
||||||
|
name: 'UploadOffsetError', receivedBytes: 4, status: 409
|
||||||
|
});
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('89', 8), stream('89'))).rejects.toBeInstanceOf(UploadOffsetError);
|
||||||
|
expect(await readFile(join(root, file.path), 'utf8')).toBe('0123');
|
||||||
|
await receiveUploadChunk(root, file, chunk('456789', 4), stream('456789'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('can resume a verified append interrupted by process termination at an unaligned byte', async () => {
|
||||||
|
await mkdir(join(root, 'zone/english'), { recursive: true });
|
||||||
|
await writeFile(join(root, file.path), '0123456');
|
||||||
|
expect(await getUploadFileProgress(root, file)).toEqual({ receivedBytes: 7, complete: false });
|
||||||
|
await receiveUploadChunk(root, file, chunk('789', 7), stream('789'));
|
||||||
|
expect(await readFile(join(root, file.path), 'utf8')).toBe('0123456789');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('distinguishes an absent empty file and creates it using the empty digest', async () => {
|
||||||
|
const empty = { path: 'main/empty.dat', size: 0 };
|
||||||
|
expect(await getUploadFileProgress(root, empty)).toEqual({ receivedBytes: 0, complete: false });
|
||||||
|
await expect(hashUploadedFiles(root, [empty])).rejects.toThrow();
|
||||||
|
expect(await receiveUploadChunk(root, empty, chunk(''), null)).toEqual({ receivedBytes: 0, complete: true });
|
||||||
|
expect(await getUploadFileProgress(root, empty)).toEqual({ receivedBytes: 0, complete: true });
|
||||||
|
expect(await hashUploadedFiles(root, [empty])).toEqual([{ ...empty, sha256: sha('') }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does not advance progress for wrong checksums, truncated requests, oversized streams or disconnects', async () => {
|
||||||
|
await receiveUploadChunk(root, file, chunk('0123'), stream('0123'));
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('456789', 4), stream('xxxxxx'))).rejects.toThrow(/checksum/);
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('456789', 4), stream('45'))).rejects.toThrow(/size mismatch/);
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('456789', 4), stream('4567890'))).rejects.toThrow(/exceeds/);
|
||||||
|
let reads = 0;
|
||||||
|
const disconnect = new ReadableStream<Uint8Array>({ pull(controller) {
|
||||||
|
if (reads++ === 0) controller.enqueue(new TextEncoder().encode('45'));
|
||||||
|
else controller.error(new Error('Browser connection ended.'));
|
||||||
|
} });
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('456789', 4), disconnect)).rejects.toThrow(/connection ended/);
|
||||||
|
expect(await getUploadFileProgress(root, file)).toEqual({ receivedBytes: 4, complete: false });
|
||||||
|
expect(await readFile(join(root, file.path), 'utf8')).toBe('0123');
|
||||||
|
expect(await readdir(`${root}.chunks`)).toEqual([]);
|
||||||
|
await receiveUploadChunk(root, file, chunk('456789', 4), stream('456789'));
|
||||||
|
});
|
||||||
|
|
||||||
|
it('bounds request size before reading any body and rejects invalid metadata', async () => {
|
||||||
|
const tooLarge = MAX_UPLOAD_CHUNK_BYTES + 1;
|
||||||
|
await expect(receiveUploadChunk(root, { path: 'large.iwd', size: tooLarge }, {
|
||||||
|
offset: 0, size: tooLarge, sha256: sha('')
|
||||||
|
}, null)).rejects.toThrow(/Invalid upload chunk/);
|
||||||
|
for (const metadata of [
|
||||||
|
{ ...chunk('a'), offset: -1 }, { ...chunk('a'), offset: 0.5 }, { ...chunk('a'), size: 1.5 },
|
||||||
|
{ ...chunk('a'), sha256: 'wrong' }, chunk('', 0), chunk('01234567890', 0)
|
||||||
|
]) await expect(receiveUploadChunk(root, file, metadata, stream('a'))).rejects.toThrow(/Invalid upload chunk/);
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('a'), null)).rejects.toThrow(/body is required/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('streams a maximum-size chunk supplied in small pieces', async () => {
|
||||||
|
const piece = new Uint8Array(64 * 1024).fill(97);
|
||||||
|
let remaining = MAX_UPLOAD_CHUNK_BYTES;
|
||||||
|
const digest = createHash('sha256');
|
||||||
|
for (let i = 0; i < MAX_UPLOAD_CHUNK_BYTES / piece.length; i++) digest.update(piece);
|
||||||
|
const body = new ReadableStream<Uint8Array>({ pull(controller) {
|
||||||
|
if (remaining === 0) controller.close();
|
||||||
|
else { remaining -= piece.length; controller.enqueue(piece); }
|
||||||
|
} });
|
||||||
|
const large = { path: 'main/large.iwd', size: MAX_UPLOAD_CHUNK_BYTES };
|
||||||
|
const sha256 = digest.digest('hex');
|
||||||
|
await receiveUploadChunk(root, large, { offset: 0, size: large.size, sha256 }, body);
|
||||||
|
expect(await hashUploadedFiles(root, [large])).toEqual([{ ...large, sha256 }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects missing or incorrectly sized files before publication', async () => {
|
||||||
|
await receiveUploadChunk(root, file, chunk('0123'), stream('0123'));
|
||||||
|
await expect(hashUploadedFiles(root, [file])).rejects.toThrow(/Size verification/);
|
||||||
|
await writeFile(join(root, file.path), '01234567890');
|
||||||
|
await expect(getUploadFileProgress(root, file)).rejects.toThrow(/exceeds/);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('refuses traversal and symlinks in destination and temporary storage', async () => {
|
||||||
|
await mkdir(root);
|
||||||
|
const outside = join(directory, 'outside');
|
||||||
|
await mkdir(outside);
|
||||||
|
await expect(receiveUploadChunk(root, { path: '../escape', size: 1 }, chunk('a'), stream('a'))).rejects.toThrow(/relative/);
|
||||||
|
await symlink(outside, join(root, 'zone'));
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('0123'), stream('0123'))).rejects.toThrow(/non-regular/);
|
||||||
|
await rm(join(root, 'zone'));
|
||||||
|
await symlink(outside, `${root}.chunks`);
|
||||||
|
await expect(receiveUploadChunk(root, file, chunk('0123'), stream('0123'))).rejects.toThrow(/non-regular/);
|
||||||
|
expect(await readdir(outside)).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('upload and publication coordination', () => {
|
||||||
|
it('serializes simultaneous writes with the same offset', async () => {
|
||||||
|
const responses = await Promise.allSettled([
|
||||||
|
receiveUploadChunk(root, file, chunk('0123'), stream('0123')),
|
||||||
|
receiveUploadChunk(root, file, chunk('0123'), stream('0123'))
|
||||||
|
]);
|
||||||
|
expect(responses.map(response => response.status)).toEqual(['fulfilled', 'rejected']);
|
||||||
|
expect((responses[1] as PromiseRejectedResult).reason).toMatchObject({ name: 'UploadOffsetError', receivedBytes: 4 });
|
||||||
|
expect(await readFile(join(root, file.path), 'utf8')).toBe('0123');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rechecks writable state inside the lock after queued publication and releases failed locks', async () => {
|
||||||
|
let finishPublication!: () => void;
|
||||||
|
const gate = new Promise<void>(resolve => { finishPublication = resolve; });
|
||||||
|
let published = false;
|
||||||
|
const publication = withUploadLock(root, async () => { await gate; published = true; });
|
||||||
|
const transfer = receiveUploadChunk(root, file, chunk('0123'), stream('0123'), () => {
|
||||||
|
if (published) throw new Error('Library is immutable.');
|
||||||
|
});
|
||||||
|
finishPublication();
|
||||||
|
await publication;
|
||||||
|
await expect(transfer).rejects.toThrow(/immutable/);
|
||||||
|
expect(await getUploadFileProgress(root, file)).toEqual({ receivedBytes: 0, complete: false });
|
||||||
|
await expect(withUploadLock(root, () => { throw new Error('Failed work'); })).rejects.toThrow(/Failed work/);
|
||||||
|
expect(await withUploadLock(root, () => 'next operation')).toBe('next operation');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
import { describe, expect, it, vi } from 'vitest';
|
||||||
|
import { selectGameFolder } from '../../src/lib/ui/game-folder.js';
|
||||||
|
|
||||||
|
function selectedFile(path: string, size = 10) {
|
||||||
|
const read = vi.fn(() => { throw new Error('Folder selection must not read file bodies.'); });
|
||||||
|
const file = {
|
||||||
|
name: path.split('/').at(-1), webkitRelativePath: path, size, lastModified: 1_700_000_000_000,
|
||||||
|
arrayBuffer: read, text: read, stream: read, slice: read
|
||||||
|
} as unknown as File;
|
||||||
|
return { file, read };
|
||||||
|
}
|
||||||
|
|
||||||
|
describe('game folder selection', () => {
|
||||||
|
it('removes the selected root exactly once while preserving nested game paths', () => {
|
||||||
|
const first = selectedFile('T6/T6/zone/english/base.ff', 20);
|
||||||
|
const second = selectedFile('T6/main/iw_00.iwd', 30);
|
||||||
|
const result = selectGameFolder([first.file, second.file]);
|
||||||
|
expect(result.name).toBe('T6');
|
||||||
|
expect(result.size).toBe(50);
|
||||||
|
expect(result.files).toEqual([
|
||||||
|
{ path: 'T6/zone/english/base.ff', file: first.file },
|
||||||
|
{ path: 'main/iw_00.iwd', file: second.file }
|
||||||
|
]);
|
||||||
|
expect(first.read).not.toHaveBeenCalled();
|
||||||
|
expect(second.read).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('prepares multi-gigabyte and empty files using metadata alone', () => {
|
||||||
|
const large = selectedFile('Game/zone/base.ff', 7 * 1024 ** 3);
|
||||||
|
const empty = selectedFile('Game/empty.txt', 0);
|
||||||
|
const result = selectGameFolder([large.file, empty.file]);
|
||||||
|
expect(result.size).toBe(7 * 1024 ** 3);
|
||||||
|
expect(result.files.map(item => item.path)).toEqual(['empty.txt', 'zone/base.ff']);
|
||||||
|
expect(result.files[1].file).toBe(large.file);
|
||||||
|
expect(large.read).not.toHaveBeenCalled();
|
||||||
|
expect(empty.read).not.toHaveBeenCalled();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects case-insensitive duplicate file names', () => {
|
||||||
|
expect(() => selectGameFolder([
|
||||||
|
selectedFile('Game/Zone/base.ff').file,
|
||||||
|
selectedFile('Game/zone/BASE.ff').file
|
||||||
|
])).toThrow('duplicate game paths');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a path used as both a file and a directory regardless of case', () => {
|
||||||
|
expect(() => selectGameFolder([
|
||||||
|
selectedFile('Game/zone/english/base.ff').file,
|
||||||
|
selectedFile('Game/ZONE').file
|
||||||
|
])).toThrow('same game path for a file and a directory');
|
||||||
|
});
|
||||||
|
|
||||||
|
it.each(['../base.ff', './base.ff', '/Game/base.ff', 'base.ff'])(
|
||||||
|
'rejects an absent or parent root in %s', path => {
|
||||||
|
expect(() => selectGameFolder([selectedFile(path).file])).toThrow('one game folder');
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it.each(['Game/../base.ff', 'Game/zone/./base.ff', 'Game/zone//base.ff', 'Game/zone\\base.ff'])(
|
||||||
|
'rejects traversal and non-normalized content paths in %s', path => {
|
||||||
|
expect(() => selectGameFolder([selectedFile(path).file])).toThrow('unsupported game path');
|
||||||
|
}
|
||||||
|
);
|
||||||
|
|
||||||
|
it('rejects files from multiple selected roots', () => {
|
||||||
|
expect(() => selectGameFolder([
|
||||||
|
selectedFile('T5/main/base.ff').file,
|
||||||
|
selectedFile('T6/main/base.ff').file
|
||||||
|
])).toThrow('one game folder');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects empty selections', () => {
|
||||||
|
expect(() => selectGameFolder([])).toThrow('contains no files');
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user