# Mist — Architecture

## Purpose

A private Steam-clone for ~5–10 friends. Distributes games and updates with bandwidth-efficient delta patching. Sized to the actual problem — the complexity is in the delta-patching system, not in the deployment.

## Topology

Friends use a regular web/desktop client over the open internet. Public DNS resolves to a border server. Nginx Proxy Manager terminates TLS (Let's Encrypt) and reverse-proxies to the backend VM over Tailscale. The backend VM itself has no public exposure.

```
            ┌──────────────────────────────┐
   Friend   │ Tauri client (Svelte UI      │
   (open    │  inside Rust shell)          │
  internet) └──────────────┬───────────────┘
                           │ HTTPS, public domain
                           │ store.mist.example
                           │ admin.mist.example
                           │ dl.mist.example
                           ▼
            ┌─────────────────────────────────┐
            │ Border server (public IP)       │
            │ Nginx Proxy Manager + Let's     │
            │ Encrypt TLS termination         │
            └────────────────┬────────────────┘
                             │ Tailscale (private backhaul)
                             ▼
       ┌─────────────────────────────────────────────┐
       │ Proxmox VM "mist"                           │
       │   Docker Compose stack:                     │
       │                                             │
       │   ┌──────────────┐    ┌──────────────────┐ │
       │   │ api          │    │ admin-web        │ │
       │   │ FastAPI:     │    │ static Svelte/   │ │
       │   │ /auth        │    │ TS served by     │ │
       │   │ /catalog     │    │ tiny nginx       │ │
       │   │ /admin       │    │ (calls api with  │ │
       │   │ /downloads   │    │  admin JWT)      │ │
       │   │ /builds      │    └──────────────────┘ │
       │   └──────┬───────┘                         │
       │          │                                 │
       │          ▼                                 │
       │   ┌──────────────┐                         │
       │   │ worker       │                         │
       │   │ Celery       │   same image,           │
       │   │ delta-gen,   │   different entrypoint  │
       │   │ archive prep,│                         │
       │   │ notifications│                         │
       │   └──────┬───────┘                         │
       │          │                                 │
       │          ▼                                 │
       │   ┌──────────┐ ┌────────┐ ┌─────────────┐ │
       │   │ postgres │ │ redis  │ │ rabbitmq    │ │
       │   └──────────┘ └────────┘ └─────────────┘ │
       │                                            │
       │   Volumes:                                 │
       │     - hot cache (.tar.zst archives)        │
       │     - postgres data                        │
       │     - redis data                           │
       │     - rabbitmq data                        │
       │     - /mnt/nas → NFS to NAS (games)        │
       └────────────────────────────────────────────┘
```

## Containers

| Container | Stack | Owns |
|---|---|---|
| **api** | FastAPI + SQLAlchemy + Postgres + Redis | Single web app with internally-modular code: `auth/`, `catalog/`, `admin/`, `downloads/`, `builds/`. Issues JWTs. Serves resumable downloads. Receives `mistpipe` uploads. Queues background work into RabbitMQ. |
| **worker** | Celery (same image as `api`, different entrypoint) | Consumes RabbitMQ. Runs the heavy stuff: hdiff delta generation, librsync indirect-delta generation, `chain_replay` cold reconstruction, `.tar.zst` archive packing, Discord notifications. |
| **admin-web** | SvelteKit, built static + tiny nginx | Admin UI. Calls `api/admin/*` with admin JWT. |
| **postgres** | postgres:16 | Catalog, users, build job state. |
| **redis** | redis:7 | Celery result backend, cache, ephemeral session data. |
| **rabbitmq** | rabbitmq:3.13-management | Celery broker, event bus for `notification.*` events. |

## Non-container artifacts

| Artifact | Stack | Notes |
|---|---|---|
| **client** | Tauri 2 (Rust shell + Svelte UI) | Friend-facing app. Distributed as a per-platform installer. Embeds (or spawns) the patch-application logic. |
| **mistpipe** | Python + click | Admin CLI. `login`, `new-game`, `push`, `ls`, `rm`, `resync-steam`. JWT stored in OS keychain. |

## Storage

**NAS** (mounted at `/mnt/nas` inside the VM via NFS) is the **source of truth** for game files:

```
/mnt/nas/mist/games/<Title>/
  base_version.7z              ← immutable original
  depot/                       ← current latest version's files
  manifests/
    <Title>.json               ← ordered linear version list (legacy; will migrate to Postgres)
    <version>.json             ← per-version SHA-256 manifest
  deltas/<version>/
    delta_manifest.json
    new_files/...
    *.patch                    ← hdiff direct patches
```

**Docker volumes** on the VM hold the hot path:

- `cache-vol` — `.tar.zst` archives ready to serve, reconstructed historical versions
- `tmp-vol` — in-flight delta-gen working dirs
- `postgres-vol`, `redis-vol`, `rabbitmq-vol` — service data

## Update modes

Two delta strategies, decided at request time:

- **Direct update** = consecutive versions (`1.0.0.2` → `1.0.0.3`). Deltas were pre-generated by `hdiff` at push time. Serve from cache or zip on demand.
- **Indirect update** = arbitrary version jumps (`1.0.0.0` → `1.0.0.3`). Server tells client which files changed. Client generates `librsync` signatures of its local files, POSTs them. Worker generates `rdiff` deltas against the server's copy and packs them into a `.tar.zst`. Client applies with `rdiff patch`.

For arbitrary historical reconstructions the worker runs `chain_replay` — starts from the base or closest cached version and walks forward applying hdiff patches at each step, caching results for next time.

## Auth

- Username + password, you provision accounts via the admin portal
- Passwords hashed with argon2id
- JWT issued on login, scope claim distinguishes `user` from `admin`
- Admin scope required for `/admin/*` and `/builds/*`
- Per-game `is_private` boolean flag; non-admin users only see public games + games they've been explicitly granted (future)

## Catalog metadata

Steam appdetails pull-through: when an admin adds a game with an `app_id`, the catalog service fetches `https://store.steampowered.com/api/appdetails` and stores `short_description` + `header_image`. Admin can override either via hand-edited fields.

## Out of scope (for MVP)

- Branches (stable/beta/internal)
- Save sync, achievements, friend lists, in-app chat
- Payments
- Multi-tenancy (one store)
- Public self-serve signup
- Per-user entitlements beyond `is_private`
- Client-side delta-gen in `mistpipe` (server does it for MVP)
- High availability — single VM is fine at this scale; backup + restore covers failure

## Why this shape

The original "Josh Steam" prototypes proved out the hard parts: two-mode delta-patching, chain-replay for cold reconstruction, resumable downloads. The 2-year-later rebuild focuses on **finishing the product**, not re-exploring the design space. Microservices and k8s were considered and rejected for the actual scale — see `DECISIONS.md`.