Initial Mist scaffold
admin-web / build (push) Successful in 22s
backend / test (push) Failing after 52s
mistpipe / test (push) Successful in 10s
admin-web / build-and-push (push) Failing after 5s
backend / build-and-push (push) Has been skipped

Successor to the Josh Steam prototypes. Single-VM Docker Compose stack with
the load-bearing core/ logic ported from JoshSteam CDN with bug fixes.

Contents:
- backend/  FastAPI + Celery (same image, two entrypoints)
            core/  hdiff, librsync, chain_replay, manifest, compression,
                   discord, steam, unrealpak, paths
            api/   auth, catalog, admin, builds (skeletons) + downloads (real)
            worker/  Celery factory replacing the missing prototype Tasks/__init__.py
            db/    SQLAlchemy models + Alembic initial migration
- admin-web/  SvelteKit + Tailwind skeleton
- client/    Tauri 2 + Svelte skeleton (Mist placeholder UI)
- mistpipe/  click-based admin CLI with subcommand stubs
- docs/      ARCHITECTURE, DECISIONS (9 ADRs), RUNBOOK
- docker-compose.yml + dev overlay + .github/workflows

Bugs fixed during port:
- Routes/download.py:2 stray backslash on import line
- Utils/celery.py inspect.reserved() missing parens + double active() typo
- Hardcoded OneDrive/Desktop paths replaced with pydantic-settings config
- Discord webhook URL + RabbitMQ password moved to env vars
- Missing Tasks/__init__.py reconstructed as worker/__init__.py

Out of scope for this commit: route bodies, UI screens, mistpipe subcommand
bodies, real image builds.
This commit is contained in:
2026-06-07 19:39:25 -04:00
commit bfd6771a9a
76 changed files with 3890 additions and 0 deletions
+63
View File
@@ -0,0 +1,63 @@
"""Celery task definitions.
Each task is a thin wrapper around `mist.core.*` functions. The patching IP
lives in `core/`; this file is just plumbing.
"""
from __future__ import annotations
from mist.worker import celery_app
@celery_app.task(name="mist.tasks.import_new_game")
def import_new_game(_data: dict) -> dict:
"""Initial import of a game's base version from an uploaded archive.
Expected flow once implemented:
1. Move uploaded archive to NAS as base_version.7z
2. Extract to depot/
3. Generate per-version manifest via core.manifest.generate_manifest
4. Pre-cache the full-game .tar.zst via core.compression.compress_and_save_zstd
5. Insert Game + Version rows; call core.discord.announce_new_game
"""
raise NotImplementedError("import_new_game not implemented yet")
@celery_app.task(name="mist.tasks.push_update")
def push_update(_data: dict) -> dict:
"""Receive a new version's full files, generate per-version manifest +
hdiff direct deltas vs previous version, swap depot.
Replaces the prototype's push_update task. The chain is:
core.manifest.generate_manifest
core.hdiff.generate_delta_patches
(move new files into depot/)
Version row insert
core.discord.announce_update
"""
raise NotImplementedError("push_update not implemented yet")
@celery_app.task(name="mist.tasks.generate_direct_update")
def generate_direct_update(_game_title: str, _from_version: str, _to_version: str) -> dict:
"""Pack the pre-built deltas for a direct (consecutive) update into a .tar.zst."""
raise NotImplementedError("generate_direct_update not implemented yet")
@celery_app.task(name="mist.tasks.generate_indirect_update")
def generate_indirect_update(
_game_title: str, _from_version: str, _to_version: str, _signatures_path: str
) -> dict:
"""Generate librsync .dlt deltas based on client-provided signatures, pack into .tar.zst.
Replaces the prototype's Tasks/generate_indirect_update.py. Calls
core.librsync.generate_delta per file. May call core.chain_replay.prepare_game_version
if `to_version` is not the latest and not cached.
"""
raise NotImplementedError("generate_indirect_update not implemented yet")
@celery_app.task(name="mist.tasks.prepare_full_game_archive")
def prepare_full_game_archive(_game_title: str, _version: str) -> dict:
"""Reconstruct (if needed) and pack `(game_title, version)` as a .tar.zst into cache."""
raise NotImplementedError("prepare_full_game_archive not implemented yet")