feat(release): version live-image, skip rebuild+redownload when unchanged
Splits the release workflow into three jobs (detect, build-live-image, bundle) so the ~9 min mkosi build only runs when live-image/VERSION bumps. The slim bundle (~30 MB: orchestrator + agent + deploy scripts + a live-image/VERSION pointer) rebuilds every push; the ~300 MB vmlinuz+initrd.img are published separately under the immutable live-image/<version>/ path. install.sh compares the pointer to /var/lib/vetting/live/VERSION and fetches the files only on mismatch, cutting repeat-install wall-clock from ~30 s + 300 MB to ~10 s + 0 MB on the common no-live-image-change release. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
+127
-36
@@ -1,13 +1,20 @@
|
|||||||
name: Release
|
name: Release
|
||||||
|
|
||||||
# Builds the full release tarball (orchestrator + agent + live image +
|
# Publishes two artifact streams to the Gitea generic package registry:
|
||||||
# deploy scripts) and publishes it to the Gitea generic package
|
|
||||||
# registry under two versions:
|
|
||||||
# - sha-<short-sha> immutable, per-commit pin
|
|
||||||
# - latest rolling alias (DELETE+PUT on each run)
|
|
||||||
#
|
#
|
||||||
# The LXC installer (deploy/proxmox-install.sh) curls the "latest"
|
# vetting/latest/vetting-bundle.tar.gz slim bundle (orchestrator +
|
||||||
# version by default; operators can pin via VETTING_VERSION=sha-abc1234.
|
# agent + deploy scripts + a
|
||||||
|
# live-image/VERSION pointer),
|
||||||
|
# DELETE+PUT every release.
|
||||||
|
# live-image/<li_version>/vmlinuz immutable kernel/initrd,
|
||||||
|
# live-image/<li_version>/initrd.img one set per live-image
|
||||||
|
# VERSION bump.
|
||||||
|
#
|
||||||
|
# The slow mkosi build (~9 min) only runs when live-image/VERSION changes;
|
||||||
|
# the slim bundle always rebuilds since its inputs (orchestrator + agent +
|
||||||
|
# deploy scripts) are cheap. On the instance, install.sh compares the
|
||||||
|
# bundle's pointer to /var/lib/vetting/live/VERSION and fetches the
|
||||||
|
# immutable artifacts only when they differ.
|
||||||
|
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
@@ -31,9 +38,48 @@ permissions:
|
|||||||
contents: read
|
contents: read
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
release:
|
detect:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
outputs:
|
||||||
|
li_version: ${{ steps.out.outputs.li_version }}
|
||||||
|
li_changed: ${{ steps.out.outputs.li_changed }}
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 2
|
||||||
|
- id: out
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
v="$(tr -d '[:space:]' < live-image/VERSION)"
|
||||||
|
if [[ ! "${v}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||||
|
echo "::error::live-image/VERSION must match v<major>.<minor>.<patch>, got '${v}'"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
echo "li_version=${v}" >> "$GITHUB_OUTPUT"
|
||||||
|
# li_changed = VERSION file differs between HEAD^ and HEAD. On
|
||||||
|
# the initial commit that introduces the file, HEAD^ is absent
|
||||||
|
# and we fall through to "changed" so the first release
|
||||||
|
# publishes the live image.
|
||||||
|
if git rev-parse --verify HEAD^ >/dev/null 2>&1; then
|
||||||
|
if git diff --name-only HEAD^..HEAD | grep -qx 'live-image/VERSION'; then
|
||||||
|
echo "li_changed=true" >> "$GITHUB_OUTPUT"
|
||||||
|
else
|
||||||
|
echo "li_changed=false" >> "$GITHUB_OUTPUT"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo "li_changed=true" >> "$GITHUB_OUTPUT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
build-live-image:
|
||||||
|
needs: detect
|
||||||
|
if: ${{ needs.detect.outputs.li_changed == 'true' }}
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
timeout-minutes: 45
|
timeout-minutes: 45
|
||||||
|
env:
|
||||||
|
LI_VERSION: ${{ needs.detect.outputs.li_version }}
|
||||||
|
REGISTRY_URL: ${{ vars.REGISTRY_URL }}
|
||||||
|
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||||
|
OWNER: ${{ gitea.repository_owner }}
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
@@ -43,6 +89,9 @@ jobs:
|
|||||||
go-version: "1.26.x"
|
go-version: "1.26.x"
|
||||||
cache: false
|
cache: false
|
||||||
|
|
||||||
|
- name: Build agent (baked into initrd)
|
||||||
|
run: make agent-linux
|
||||||
|
|
||||||
- name: Install live-image build dependencies
|
- name: Install live-image build dependencies
|
||||||
run: |
|
run: |
|
||||||
sudo apt-get update
|
sudo apt-get update
|
||||||
@@ -58,46 +107,88 @@ jobs:
|
|||||||
sudo pip install --break-system-packages \
|
sudo pip install --break-system-packages \
|
||||||
"git+https://github.com/systemd/mkosi.git@v25.3"
|
"git+https://github.com/systemd/mkosi.git@v25.3"
|
||||||
|
|
||||||
|
- name: Build live image
|
||||||
|
run: make live-image
|
||||||
|
|
||||||
|
- name: Publish live-image/${{ env.LI_VERSION }}/
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
# Gitea's generic registry treats each version path as
|
||||||
|
# immutable. A re-upload returns 409. Bumping VERSION is the
|
||||||
|
# only way to publish new artifacts here.
|
||||||
|
for f in vmlinuz initrd.img; do
|
||||||
|
curl -fsSL -H "Authorization: token ${REGISTRY_TOKEN}" \
|
||||||
|
--upload-file "live-image/build/${f}" \
|
||||||
|
"${REGISTRY_URL}/api/packages/${OWNER}/generic/live-image/${LI_VERSION}/${f}"
|
||||||
|
done
|
||||||
|
|
||||||
|
bundle:
|
||||||
|
# Always runs. If build-live-image was skipped, the prior upload at
|
||||||
|
# live-image/<li_version>/ is the one install.sh will fetch. The
|
||||||
|
# bundle itself never carries vmlinuz/initrd — only the VERSION
|
||||||
|
# pointer — so it's cheap to rebuild every push.
|
||||||
|
needs: [detect, build-live-image]
|
||||||
|
if: >-
|
||||||
|
${{ always()
|
||||||
|
&& needs.detect.result == 'success'
|
||||||
|
&& (needs.build-live-image.result == 'success'
|
||||||
|
|| needs.build-live-image.result == 'skipped') }}
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 15
|
||||||
|
env:
|
||||||
|
LI_VERSION: ${{ needs.detect.outputs.li_version }}
|
||||||
|
REGISTRY_URL: ${{ vars.REGISTRY_URL }}
|
||||||
|
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
||||||
|
OWNER: ${{ gitea.repository_owner }}
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
|
||||||
|
- name: Set up Go
|
||||||
|
uses: actions/setup-go@v5
|
||||||
|
with:
|
||||||
|
go-version: "1.26.x"
|
||||||
|
cache: false
|
||||||
|
|
||||||
- name: Install templ
|
- name: Install templ
|
||||||
run: go install github.com/a-h/templ/cmd/templ@v0.3.1001
|
run: go install github.com/a-h/templ/cmd/templ@v0.3.1001
|
||||||
|
|
||||||
- name: Build release bundle
|
- name: Build orchestrator + agent
|
||||||
run: make release
|
run: make orchestrator-linux agent-linux
|
||||||
|
|
||||||
- name: Resolve bundle path + short sha
|
- name: Assemble slim bundle
|
||||||
id: meta
|
|
||||||
run: |
|
run: |
|
||||||
short_sha=$(git rev-parse --short HEAD)
|
set -euo pipefail
|
||||||
echo "short_sha=${short_sha}" >> "$GITHUB_OUTPUT"
|
stamp="vetting-bundle"
|
||||||
echo "bundle=bin/vetting-bundle-${short_sha}.tar.gz" >> "$GITHUB_OUTPUT"
|
rm -rf "build/${stamp}" "bin/${stamp}.tar.gz"
|
||||||
|
mkdir -p "build/${stamp}/bin" "build/${stamp}/live-image"
|
||||||
|
cp bin/vetting-linux-amd64 bin/vetting-agent.linux-amd64 \
|
||||||
|
"build/${stamp}/bin/"
|
||||||
|
cp deploy/install.sh deploy/pxe-setup.sh deploy/vetting.service \
|
||||||
|
deploy/vetting.production.yaml deploy/ipxe-shas.txt \
|
||||||
|
"build/${stamp}/"
|
||||||
|
# Embed only the live-image VERSION pointer. install.sh on
|
||||||
|
# the target will compare this against /var/lib/vetting/live/VERSION
|
||||||
|
# and curl the actual files from live-image/<v>/ in the
|
||||||
|
# registry when they differ.
|
||||||
|
printf '%s\n' "${LI_VERSION}" > "build/${stamp}/live-image/VERSION"
|
||||||
|
# Root-level VERSION keeps the orchestrator's git-short-sha
|
||||||
|
# visible to operators (proxmox-install.sh cats it at the end
|
||||||
|
# of the install).
|
||||||
|
git rev-parse --short HEAD > "build/${stamp}/VERSION"
|
||||||
|
tar -C build -czf "bin/${stamp}.tar.gz" "${stamp}"
|
||||||
|
echo "wrote bin/${stamp}.tar.gz ($(du -h "bin/${stamp}.tar.gz" | cut -f1))"
|
||||||
|
|
||||||
- name: Publish sha-pinned bundle
|
- name: Replace vetting/latest/ bundle
|
||||||
env:
|
|
||||||
REGISTRY_URL: ${{ vars.REGISTRY_URL }}
|
|
||||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
|
||||||
OWNER: ${{ gitea.repository_owner }}
|
|
||||||
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
|
|
||||||
BUNDLE: ${{ steps.meta.outputs.bundle }}
|
|
||||||
run: |
|
|
||||||
curl -fsSL -H "Authorization: token ${REGISTRY_TOKEN}" \
|
|
||||||
--upload-file "${BUNDLE}" \
|
|
||||||
"${REGISTRY_URL}/api/packages/${OWNER}/generic/vetting/sha-${SHORT_SHA}/vetting-bundle.tar.gz"
|
|
||||||
|
|
||||||
- name: Replace latest alias
|
|
||||||
env:
|
|
||||||
REGISTRY_URL: ${{ vars.REGISTRY_URL }}
|
|
||||||
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
|
|
||||||
OWNER: ${{ gitea.repository_owner }}
|
|
||||||
BUNDLE: ${{ steps.meta.outputs.bundle }}
|
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
# Delete the whole "latest" version, not the file inside it.
|
# Delete the whole "latest" version, not the file inside it.
|
||||||
# Deleting the file leaves a ghost version that makes PUT 404.
|
# Deleting just the file leaves a ghost version that makes
|
||||||
|
# PUT 404.
|
||||||
status=$(curl -sS -o /dev/null -w '%{http_code}' \
|
status=$(curl -sS -o /dev/null -w '%{http_code}' \
|
||||||
-H "Authorization: token ${REGISTRY_TOKEN}" \
|
-H "Authorization: token ${REGISTRY_TOKEN}" \
|
||||||
-X DELETE \
|
-X DELETE \
|
||||||
"${REGISTRY_URL}/api/packages/${OWNER}/generic/vetting/latest")
|
"${REGISTRY_URL}/api/packages/${OWNER}/generic/vetting/latest")
|
||||||
echo "DELETE latest -> ${status}"
|
echo "DELETE vetting/latest -> ${status}"
|
||||||
case "${status}" in
|
case "${status}" in
|
||||||
204|404) ;;
|
204|404) ;;
|
||||||
*) echo "unexpected DELETE status ${status}"; exit 1 ;;
|
*) echo "unexpected DELETE status ${status}"; exit 1 ;;
|
||||||
@@ -106,5 +197,5 @@ jobs:
|
|||||||
# the upload re-creates it under the same name.
|
# the upload re-creates it under the same name.
|
||||||
sleep 2
|
sleep 2
|
||||||
curl -fsSL -H "Authorization: token ${REGISTRY_TOKEN}" \
|
curl -fsSL -H "Authorization: token ${REGISTRY_TOKEN}" \
|
||||||
--upload-file "${BUNDLE}" \
|
--upload-file bin/vetting-bundle.tar.gz \
|
||||||
"${REGISTRY_URL}/api/packages/${OWNER}/generic/vetting/latest/vetting-bundle.tar.gz"
|
"${REGISTRY_URL}/api/packages/${OWNER}/generic/vetting/latest/vetting-bundle.tar.gz"
|
||||||
|
|||||||
@@ -71,18 +71,15 @@ install: orchestrator-linux agent-linux ## Run deploy/install.sh (must be run on
|
|||||||
sudo ./deploy/install.sh --binary ./bin/vetting-linux-amd64 --agent-binary ./bin/vetting-agent.linux-amd64
|
sudo ./deploy/install.sh --binary ./bin/vetting-linux-amd64 --agent-binary ./bin/vetting-agent.linux-amd64
|
||||||
|
|
||||||
.PHONY: release
|
.PHONY: release
|
||||||
release: orchestrator-linux agent-linux live-image ## Build the scp-and-go release tarball (run from Linux/WSL)
|
release: orchestrator-linux agent-linux ## Build the slim release tarball (no live-image files — they're fetched on install)
|
||||||
ifneq ($(findstring Windows,$(UNAME_S))$(findstring MINGW,$(UNAME_S))$(findstring MSYS,$(UNAME_S)),)
|
|
||||||
@echo "ERROR: make release must be run from Linux/WSL (live-image dep needs mkosi)." && exit 1
|
|
||||||
endif
|
|
||||||
@set -e; \
|
@set -e; \
|
||||||
stamp=vetting-bundle-$(GIT_SHA); \
|
stamp=vetting-bundle; \
|
||||||
rm -rf build/$$stamp bin/$$stamp.tar.gz; \
|
rm -rf build/$$stamp bin/$$stamp.tar.gz; \
|
||||||
mkdir -p build/$$stamp/bin build/$$stamp/live-image; \
|
mkdir -p build/$$stamp/bin build/$$stamp/live-image; \
|
||||||
cp bin/vetting-linux-amd64 bin/vetting-agent.linux-amd64 build/$$stamp/bin/; \
|
cp bin/vetting-linux-amd64 bin/vetting-agent.linux-amd64 build/$$stamp/bin/; \
|
||||||
cp live-image/build/vmlinuz live-image/build/initrd.img build/$$stamp/live-image/; \
|
|
||||||
cp deploy/install.sh deploy/pxe-setup.sh deploy/vetting.service \
|
cp deploy/install.sh deploy/pxe-setup.sh deploy/vetting.service \
|
||||||
deploy/vetting.production.yaml deploy/ipxe-shas.txt build/$$stamp/; \
|
deploy/vetting.production.yaml deploy/ipxe-shas.txt build/$$stamp/; \
|
||||||
|
cp live-image/VERSION build/$$stamp/live-image/VERSION; \
|
||||||
echo $(GIT_SHA) > build/$$stamp/VERSION; \
|
echo $(GIT_SHA) > build/$$stamp/VERSION; \
|
||||||
tar -C build -czf bin/$$stamp.tar.gz $$stamp; \
|
tar -C build -czf bin/$$stamp.tar.gz $$stamp; \
|
||||||
echo "wrote bin/$$stamp.tar.gz ($$(du -h bin/$$stamp.tar.gz | cut -f1))"
|
echo "wrote bin/$$stamp.tar.gz ($$(du -h bin/$$stamp.tar.gz | cut -f1))"
|
||||||
|
|||||||
+86
-10
@@ -18,10 +18,15 @@
|
|||||||
# - Fetch TFTP iPXE payloads (that's pxe-setup.sh's job — it also
|
# - Fetch TFTP iPXE payloads (that's pxe-setup.sh's job — it also
|
||||||
# writes the pxe: block of vetting.yaml with first-time args).
|
# writes the pxe: block of vetting.yaml with first-time args).
|
||||||
#
|
#
|
||||||
# When a live-image/{vmlinuz,initrd.img} is present next to this script
|
# Live-image staging has two modes:
|
||||||
# (release bundle) or under ../live-image/build/ (repo checkout), it's
|
# - Release bundle (new format): the bundle carries only a
|
||||||
# staged into --live-dir automatically. This makes the one-liner
|
# live-image/VERSION pointer. We compare it to ${LIVE_DIR}/VERSION
|
||||||
# upgrade loop work end-to-end for PXE-enabled installs.
|
# and, on mismatch, fetch vmlinuz+initrd.img from the Gitea
|
||||||
|
# generic registry at live-image/<VERSION>/. Matched versions
|
||||||
|
# skip the fetch (set FORCE_LIVE_IMAGE=1 to override).
|
||||||
|
# - Repo checkout / legacy bundle: if vmlinuz+initrd.img are present
|
||||||
|
# next to this script (${SCRIPT_DIR}/live-image/) or under
|
||||||
|
# ${REPO_ROOT}/live-image/build/, they're copied straight in.
|
||||||
#
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# sudo ./install.sh [--binary PATH] [--config-dir /etc/vetting]
|
# sudo ./install.sh [--binary PATH] [--config-dir /etc/vetting]
|
||||||
@@ -165,6 +170,59 @@ heal_pxe_config() {
|
|||||||
mv "${tmp}" "${config}"
|
mv "${tmp}" "${config}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# refresh_live_image: pull vmlinuz+initrd.img from the Gitea generic
|
||||||
|
# package registry when the bundle's live-image/VERSION pointer differs
|
||||||
|
# from ${LIVE_DIR}/VERSION. Skips the fetch when versions match unless
|
||||||
|
# FORCE_LIVE_IMAGE=1 (useful when on-disk files got corrupted). Set by
|
||||||
|
# proxmox-install.sh; on a direct `install.sh` invocation the caller
|
||||||
|
# must export REGISTRY_URL (and optionally PACKAGE_OWNER).
|
||||||
|
refresh_live_image() {
|
||||||
|
local pointer="${SCRIPT_DIR}/live-image/VERSION"
|
||||||
|
local bundle_ver
|
||||||
|
bundle_ver="$(tr -d '[:space:]' < "${pointer}" 2>/dev/null || true)"
|
||||||
|
if [[ -z "${bundle_ver}" ]]; then
|
||||||
|
echo "WARN: bundle's ${pointer} is empty; skipping live-image fetch" >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
local installed_ver=""
|
||||||
|
if [[ -f "${LIVE_DIR}/VERSION" ]]; then
|
||||||
|
installed_ver="$(tr -d '[:space:]' < "${LIVE_DIR}/VERSION")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${bundle_ver}" == "${installed_ver}" && "${FORCE_LIVE_IMAGE:-0}" != "1" ]]; then
|
||||||
|
echo "==> live-image already at ${bundle_ver}; skipping fetch (FORCE_LIVE_IMAGE=1 to redownload)"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -z "${REGISTRY_URL:-}" ]]; then
|
||||||
|
echo "WARN: REGISTRY_URL is not set; cannot fetch live-image ${bundle_ver}. Re-run via proxmox-install.sh or export REGISTRY_URL." >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
local owner="${PACKAGE_OWNER:-josh}"
|
||||||
|
local base="${REGISTRY_URL%/}/api/packages/${owner}/generic/live-image/${bundle_ver}"
|
||||||
|
|
||||||
|
echo "==> fetching live-image ${bundle_ver} (was '${installed_ver:-none}') from ${base}"
|
||||||
|
local tmp
|
||||||
|
tmp="$(mktemp -d)"
|
||||||
|
# shellcheck disable=SC2064
|
||||||
|
trap "rm -rf '${tmp}'" RETURN
|
||||||
|
|
||||||
|
# Default curl meter shows rate + ETA, which matters for the ~300 MB
|
||||||
|
# initrd on slow links.
|
||||||
|
curl -fL -o "${tmp}/vmlinuz" "${base}/vmlinuz"
|
||||||
|
curl -fL -o "${tmp}/initrd.img" "${base}/initrd.img"
|
||||||
|
|
||||||
|
install -d -m 0755 -o "${SERVICE_USER}" -g "${SERVICE_USER}" "${LIVE_DIR}"
|
||||||
|
install -m 0644 -o "${SERVICE_USER}" -g "${SERVICE_USER}" \
|
||||||
|
"${tmp}/vmlinuz" "${LIVE_DIR}/vmlinuz"
|
||||||
|
install -m 0644 -o "${SERVICE_USER}" -g "${SERVICE_USER}" \
|
||||||
|
"${tmp}/initrd.img" "${LIVE_DIR}/initrd.img"
|
||||||
|
printf '%s\n' "${bundle_ver}" > "${LIVE_DIR}/VERSION"
|
||||||
|
chown "${SERVICE_USER}:${SERVICE_USER}" "${LIVE_DIR}/VERSION"
|
||||||
|
chmod 0644 "${LIVE_DIR}/VERSION"
|
||||||
|
}
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
REPO_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
@@ -244,12 +302,14 @@ if [[ -f "${SCRIPT_DIR}/pxe-setup.sh" && -f "${SCRIPT_DIR}/ipxe-shas.txt" ]]; th
|
|||||||
ln -sfn /usr/local/share/vetting/pxe-setup.sh /usr/local/sbin/vetting-pxe-setup
|
ln -sfn /usr/local/share/vetting/pxe-setup.sh /usr/local/sbin/vetting-pxe-setup
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# Stage the live image into LIVE_DIR if we can find one. Two layouts:
|
# Stage the live image into LIVE_DIR. Preference order:
|
||||||
# - release bundle: ${SCRIPT_DIR}/live-image/{vmlinuz,initrd.img}
|
# 1. --live-image-src explicitly given, or local files found in the
|
||||||
# - repo-tree dev run: ${REPO_ROOT}/live-image/build/{vmlinuz,initrd.img}
|
# bundle/repo — copy straight in (dev and legacy bundle layouts).
|
||||||
# Silently skipped when no source is found — operators without PXE
|
# 2. Bundle carries only live-image/VERSION — fetch from the Gitea
|
||||||
# don't need it, and dev checkouts that haven't run `make live-image`
|
# generic registry when the pointer differs from ${LIVE_DIR}/VERSION.
|
||||||
# shouldn't fail the install.
|
# 3. Neither — skip quietly (no-PXE installs don't need a live image,
|
||||||
|
# and dev checkouts that haven't run `make live-image` shouldn't
|
||||||
|
# fail the install).
|
||||||
if [[ -z "${LIVE_IMAGE_SRC}" ]]; then
|
if [[ -z "${LIVE_IMAGE_SRC}" ]]; then
|
||||||
for cand in \
|
for cand in \
|
||||||
"${SCRIPT_DIR}/live-image" \
|
"${SCRIPT_DIR}/live-image" \
|
||||||
@@ -268,6 +328,22 @@ if [[ -n "${LIVE_IMAGE_SRC}" ]]; then
|
|||||||
"${LIVE_IMAGE_SRC}/vmlinuz" "${LIVE_DIR}/vmlinuz"
|
"${LIVE_IMAGE_SRC}/vmlinuz" "${LIVE_DIR}/vmlinuz"
|
||||||
install -m 0644 -o "${SERVICE_USER}" -g "${SERVICE_USER}" \
|
install -m 0644 -o "${SERVICE_USER}" -g "${SERVICE_USER}" \
|
||||||
"${LIVE_IMAGE_SRC}/initrd.img" "${LIVE_DIR}/initrd.img"
|
"${LIVE_IMAGE_SRC}/initrd.img" "${LIVE_DIR}/initrd.img"
|
||||||
|
# Record the version that produced these files if the source has
|
||||||
|
# one (bundle with legacy layout carrying VERSION alongside the
|
||||||
|
# kernel; dev tree has live-image/VERSION at repo root). Lets a
|
||||||
|
# future bundle-based install decide whether to refetch.
|
||||||
|
for vcand in \
|
||||||
|
"${LIVE_IMAGE_SRC}/VERSION" \
|
||||||
|
"${SCRIPT_DIR}/live-image/VERSION" \
|
||||||
|
"${REPO_ROOT}/live-image/VERSION"; do
|
||||||
|
if [[ -f "${vcand}" ]]; then
|
||||||
|
install -m 0644 -o "${SERVICE_USER}" -g "${SERVICE_USER}" \
|
||||||
|
"${vcand}" "${LIVE_DIR}/VERSION"
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
elif [[ -f "${SCRIPT_DIR}/live-image/VERSION" ]]; then
|
||||||
|
refresh_live_image
|
||||||
else
|
else
|
||||||
echo "==> no live image found (bundle/live-image or ../live-image/build); skipping live-dir staging"
|
echo "==> no live image found (bundle/live-image or ../live-image/build); skipping live-dir staging"
|
||||||
fi
|
fi
|
||||||
|
|||||||
+41
-29
@@ -3,26 +3,42 @@
|
|||||||
# any Debian/Ubuntu host). Fetches a prebuilt release bundle from the
|
# any Debian/Ubuntu host). Fetches a prebuilt release bundle from the
|
||||||
# Gitea package registry, extracts it, and hands off to install.sh.
|
# Gitea package registry, extracts it, and hands off to install.sh.
|
||||||
#
|
#
|
||||||
|
# The bundle itself is slim (~30 MB: orchestrator + agent + deploy
|
||||||
|
# scripts + a live-image/VERSION pointer). install.sh compares that
|
||||||
|
# pointer against /var/lib/vetting/live/VERSION and fetches the
|
||||||
|
# ~300 MB vmlinuz+initrd.img from the registry only when they differ,
|
||||||
|
# so repeated runs cost ~10 s on no-live-image-change releases.
|
||||||
|
#
|
||||||
# Usage:
|
# Usage:
|
||||||
# curl -fsSL https://gitea.thewrightserver.net/josh/Vetting/raw/branch/main/deploy/proxmox-install.sh | sudo bash
|
# curl -fsSL https://gitea.thewrightserver.net/josh/Vetting/raw/branch/main/deploy/proxmox-install.sh | sudo bash
|
||||||
#
|
#
|
||||||
# To pin a specific build instead of "latest":
|
# Flags / env overrides:
|
||||||
# VETTING_VERSION=sha-abc1234 curl -fsSL .../proxmox-install.sh | sudo bash
|
# REGISTRY_URL base URL of the Gitea instance hosting the
|
||||||
#
|
# package registry (default: https://gitea.thewrightserver.net)
|
||||||
# Env overrides:
|
# PACKAGE_OWNER Gitea owner of the `vetting` package
|
||||||
# REGISTRY_URL base URL of the Gitea instance hosting the package
|
|
||||||
# registry (default: https://gitea.thewrightserver.net)
|
|
||||||
# PACKAGE_OWNER Gitea owner who owns the `vetting` package
|
|
||||||
# (default: josh)
|
# (default: josh)
|
||||||
# VETTING_VERSION package version — either "latest" (rolling) or
|
# FORCE_LIVE_IMAGE=1 or --force-live-image — re-download the live
|
||||||
# "sha-<short-sha>" (immutable). Default: "latest".
|
# image even when the on-disk version matches
|
||||||
|
# (useful when the local files got corrupted).
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
REGISTRY_URL="${REGISTRY_URL:-https://gitea.thewrightserver.net}"
|
REGISTRY_URL="${REGISTRY_URL:-https://gitea.thewrightserver.net}"
|
||||||
PACKAGE_OWNER="${PACKAGE_OWNER:-josh}"
|
PACKAGE_OWNER="${PACKAGE_OWNER:-josh}"
|
||||||
VETTING_VERSION="${VETTING_VERSION:-latest}"
|
FORCE_LIVE_IMAGE="${FORCE_LIVE_IMAGE:-0}"
|
||||||
|
|
||||||
BUNDLE_URL="${REGISTRY_URL}/api/packages/${PACKAGE_OWNER}/generic/vetting/${VETTING_VERSION}/vetting-bundle.tar.gz"
|
for arg in "$@"; do
|
||||||
|
case "${arg}" in
|
||||||
|
--force-live-image) FORCE_LIVE_IMAGE=1 ;;
|
||||||
|
*) echo "unknown arg: ${arg}" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Exported so install.sh (run as a child process inside the extracted
|
||||||
|
# bundle dir) sees them when deciding whether to fetch the live image
|
||||||
|
# and where to fetch it from.
|
||||||
|
export REGISTRY_URL PACKAGE_OWNER FORCE_LIVE_IMAGE
|
||||||
|
|
||||||
|
BUNDLE_URL="${REGISTRY_URL}/api/packages/${PACKAGE_OWNER}/generic/vetting/latest/vetting-bundle.tar.gz"
|
||||||
|
|
||||||
if [[ $EUID -ne 0 ]]; then
|
if [[ $EUID -ne 0 ]]; then
|
||||||
echo "proxmox-install.sh must be run as root (try: sudo bash)" >&2
|
echo "proxmox-install.sh must be run as root (try: sudo bash)" >&2
|
||||||
@@ -38,25 +54,22 @@ apt-get install -y --no-install-recommends \
|
|||||||
tmp="$(mktemp -d)"
|
tmp="$(mktemp -d)"
|
||||||
trap 'rm -rf "${tmp}"' EXIT
|
trap 'rm -rf "${tmp}"' EXIT
|
||||||
|
|
||||||
echo "==> fetching bundle (${VETTING_VERSION}) from ${BUNDLE_URL}"
|
echo "==> fetching bundle from ${BUNDLE_URL}"
|
||||||
# Default curl meter (no -s, no --progress-bar) shows transfer rate
|
# -f fails on HTTP errors; -L follows redirects. Default meter (rate +
|
||||||
# and ETA, which matters now that bundles are ~300 MB — the live
|
# ETA) is fine now that the bundle is ~30 MB.
|
||||||
# image ships the full firmware+rootfs, so a bare percentage bar
|
|
||||||
# with no speed/ETA makes slow links look like hangs.
|
|
||||||
# -f fails on HTTP errors; -L follows redirects.
|
|
||||||
curl -fL "${BUNDLE_URL}" -o "${tmp}/vetting-bundle.tar.gz"
|
curl -fL "${BUNDLE_URL}" -o "${tmp}/vetting-bundle.tar.gz"
|
||||||
|
|
||||||
bundle_size="$(du -h "${tmp}/vetting-bundle.tar.gz" | cut -f1)"
|
bundle_size="$(du -h "${tmp}/vetting-bundle.tar.gz" | cut -f1)"
|
||||||
echo "==> extracting (${bundle_size})"
|
echo "==> extracting (${bundle_size})"
|
||||||
tar -C "${tmp}" -xzf "${tmp}/vetting-bundle.tar.gz"
|
tar -C "${tmp}" -xzf "${tmp}/vetting-bundle.tar.gz"
|
||||||
|
|
||||||
# Bundle extracts to vetting-bundle-<sha>/; glob-match the single
|
# New bundle extracts to vetting-bundle/; legacy bundles used
|
||||||
# top-level directory.
|
# vetting-bundle-<sha>/. Match both so a downgrade-pin still works.
|
||||||
shopt -s nullglob
|
shopt -s nullglob
|
||||||
candidates=( "${tmp}"/vetting-bundle-* )
|
candidates=( "${tmp}"/vetting-bundle "${tmp}"/vetting-bundle-* )
|
||||||
shopt -u nullglob
|
shopt -u nullglob
|
||||||
if [[ ${#candidates[@]} -ne 1 || ! -d "${candidates[0]}" ]]; then
|
if [[ ${#candidates[@]} -ne 1 || ! -d "${candidates[0]}" ]]; then
|
||||||
echo "unexpected bundle layout: expected exactly one vetting-bundle-*/ dir" >&2
|
echo "unexpected bundle layout: expected exactly one vetting-bundle* dir" >&2
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
bundle_dir="${candidates[0]}"
|
bundle_dir="${candidates[0]}"
|
||||||
@@ -67,17 +80,16 @@ bash install.sh \
|
|||||||
--binary "${bundle_dir}/bin/vetting-linux-amd64" \
|
--binary "${bundle_dir}/bin/vetting-linux-amd64" \
|
||||||
--agent-binary "${bundle_dir}/bin/vetting-agent.linux-amd64"
|
--agent-binary "${bundle_dir}/bin/vetting-agent.linux-amd64"
|
||||||
|
|
||||||
|
orch_ver="$(cat "${bundle_dir}/VERSION" 2>/dev/null || echo unknown)"
|
||||||
|
li_ver="$(cat "${bundle_dir}/live-image/VERSION" 2>/dev/null || echo unknown)"
|
||||||
|
|
||||||
cat <<EOF
|
cat <<EOF
|
||||||
|
|
||||||
vetting is installed from bundle $(cat "${bundle_dir}/VERSION" 2>/dev/null || echo unknown).
|
vetting installed: orchestrator ${orch_ver}, live-image ${li_ver}.
|
||||||
|
|
||||||
To upgrade later, just rerun this one-liner; it always pulls "latest"
|
To upgrade later, rerun this one-liner. It always pulls the current
|
||||||
unless VETTING_VERSION is set.
|
latest bundle; the live image is re-downloaded only when its VERSION
|
||||||
|
has bumped (override with --force-live-image).
|
||||||
To pin a specific build:
|
|
||||||
VETTING_VERSION=sha-abc1234 curl -fsSL \\
|
|
||||||
${REGISTRY_URL}/${PACKAGE_OWNER}/Vetting/raw/branch/main/deploy/proxmox-install.sh \\
|
|
||||||
| sudo bash
|
|
||||||
|
|
||||||
For PXE support, run:
|
For PXE support, run:
|
||||||
sudo vetting-pxe-setup \\
|
sudo vetting-pxe-setup \\
|
||||||
|
|||||||
+35
-22
@@ -13,11 +13,17 @@ repaired nodes so DHCP and WoL work.
|
|||||||
|
|
||||||
### One-liner install (recommended)
|
### One-liner install (recommended)
|
||||||
|
|
||||||
Every push to `main` kicks off a Gitea Actions run that builds a full
|
Every push to `main` kicks off a Gitea Actions run that rebuilds the
|
||||||
release bundle (orchestrator + agent + live image + install scripts +
|
slim release bundle (orchestrator + agent + install scripts + a
|
||||||
pinned iPXE SHAs) and publishes it to the Gitea package registry. The
|
pointer file for the live image's version) and publishes it to the
|
||||||
LXC installer fetches the prebuilt tarball — no source clone, no Go
|
Gitea package registry. The ~300 MB live image (`vmlinuz` + `initrd.img`)
|
||||||
toolchain, no `make`, no WSL.
|
is published separately under `live-image/<version>/` and only
|
||||||
|
rebuilds when [`live-image/VERSION`](../live-image/VERSION) changes.
|
||||||
|
|
||||||
|
The LXC installer fetches the slim bundle on every run (~30 MB,
|
||||||
|
fast), then fetches the live image files only when the bundle's
|
||||||
|
pointer differs from what's on disk — no Go toolchain, no `make`,
|
||||||
|
no WSL, and no 300 MB transfer on ordinary releases.
|
||||||
|
|
||||||
On the LXC:
|
On the LXC:
|
||||||
|
|
||||||
@@ -26,16 +32,21 @@ curl -fsSL https://gitea.thewrightserver.net/josh/Vetting/raw/branch/main/deploy
|
|||||||
| sudo bash
|
| sudo bash
|
||||||
```
|
```
|
||||||
|
|
||||||
To pin a specific build instead of the rolling `latest`:
|
Force-refresh the on-disk live image even when versions match
|
||||||
|
(useful if the staged files got corrupted):
|
||||||
|
|
||||||
```
|
```
|
||||||
VETTING_VERSION=sha-abc1234 curl -fsSL .../proxmox-install.sh | sudo bash
|
curl -fsSL .../proxmox-install.sh | sudo bash -s -- --force-live-image
|
||||||
```
|
```
|
||||||
|
|
||||||
`proxmox-install.sh` curls the bundle from
|
`proxmox-install.sh` curls the bundle from
|
||||||
`${REGISTRY_URL}/api/packages/${PACKAGE_OWNER}/generic/vetting/${VETTING_VERSION}/vetting-bundle.tar.gz`,
|
`${REGISTRY_URL}/api/packages/${PACKAGE_OWNER}/generic/vetting/latest/vetting-bundle.tar.gz`,
|
||||||
extracts it, and hands off to the bundled `install.sh` for the base
|
extracts it, and hands off to the bundled `install.sh` for the base
|
||||||
install (user, binaries, config, systemd unit).
|
install (user, binaries, config, systemd unit). `install.sh` then
|
||||||
|
compares `live-image/VERSION` inside the bundle against
|
||||||
|
`/var/lib/vetting/live/VERSION` and fetches
|
||||||
|
`live-image/<version>/{vmlinuz,initrd.img}` from the registry when
|
||||||
|
they differ.
|
||||||
|
|
||||||
If you don't need PXE (e.g. host-mode reporter only, no automated
|
If you don't need PXE (e.g. host-mode reporter only, no automated
|
||||||
live-boots), you can stop here — edit `/etc/vetting/vetting.yaml` to
|
live-boots), you can stop here — edit `/etc/vetting/vetting.yaml` to
|
||||||
@@ -44,17 +55,21 @@ tune `server.bind` / `public_url`, then
|
|||||||
|
|
||||||
### Offline / air-gapped install
|
### Offline / air-gapped install
|
||||||
|
|
||||||
If the LXC can't reach the registry, build the tarball locally and
|
If the LXC can't reach the registry, build the slim bundle locally
|
||||||
`scp` it across:
|
and `scp` it across. The live image files must also be copied in
|
||||||
|
separately (either into the bundle's `live-image/` dir before running
|
||||||
|
install.sh, or into `/var/lib/vetting/live/` directly):
|
||||||
|
|
||||||
```
|
```
|
||||||
make release # on a Linux/WSL workstation
|
make release # on any host with Go + templ
|
||||||
scp bin/vetting-bundle-<sha>.tar.gz lxc:/tmp/
|
scp bin/vetting-bundle.tar.gz lxc:/tmp/
|
||||||
ssh lxc 'cd /tmp && tar xzf vetting-bundle-*.tar.gz \
|
ssh lxc 'cd /tmp && tar xzf vetting-bundle.tar.gz \
|
||||||
&& cd vetting-bundle-* && sudo ./install.sh'
|
&& cp /path/to/vmlinuz /path/to/initrd.img vetting-bundle/live-image/ \
|
||||||
|
&& cd vetting-bundle && sudo ./install.sh'
|
||||||
```
|
```
|
||||||
|
|
||||||
Same bundle layout either way.
|
`install.sh` recognizes local `vmlinuz`/`initrd.img` under
|
||||||
|
`live-image/` and stages them without a registry fetch.
|
||||||
|
|
||||||
### PXE enablement
|
### PXE enablement
|
||||||
|
|
||||||
@@ -232,10 +247,8 @@ curl -fsSL https://gitea.thewrightserver.net/josh/Vetting/raw/branch/main/deploy
|
|||||||
|
|
||||||
That's it — `install.sh` auto-restarts `vetting.service` when it's
|
That's it — `install.sh` auto-restarts `vetting.service` when it's
|
||||||
already enabled, and re-stages `vmlinuz`/`initrd.img` into
|
already enabled, and re-stages `vmlinuz`/`initrd.img` into
|
||||||
`/var/lib/vetting/live/` so PXE-enabled LXCs come back up with the
|
`/var/lib/vetting/live/` only when the bundle points at a new
|
||||||
fresh live image. Watch the logs with `journalctl -fu vetting`.
|
`live-image/VERSION`. Watch the logs with `journalctl -fu vetting`.
|
||||||
|
|
||||||
Pin to a specific build with `VETTING_VERSION=sha-abc1234` if you
|
The DB migration runs at startup and is append-only — no manual
|
||||||
need to roll back or test a commit. The DB migration runs at startup
|
schema work unless a release's notes call it out.
|
||||||
and is append-only — no manual schema work unless a release's notes
|
|
||||||
call it out.
|
|
||||||
|
|||||||
+15
-4
@@ -4,13 +4,24 @@ Debian-based Linux live image that PXE-booted hosts drop into. Runs the
|
|||||||
`vetting-agent` binary under systemd and reaches back to the orchestrator
|
`vetting-agent` binary under systemd and reaches back to the orchestrator
|
||||||
over HTTP+SSE.
|
over HTTP+SSE.
|
||||||
|
|
||||||
## Preferred build path: `make release`
|
## Versioning
|
||||||
|
|
||||||
Run `make release` from the repo root (Linux/WSL) — it builds the live
|
The live image has its own version marker at
|
||||||
image *and* bundles it with the orchestrator binary, install scripts,
|
[VERSION](VERSION). Bump it (`v<major>.<minor>.<patch>`) whenever
|
||||||
and pinned iPXE SHAs into a single `vetting-bundle-<sha>.tar.gz`. See
|
any mkosi input changes; CI only rebuilds and republishes
|
||||||
|
`live-image/<VERSION>/{vmlinuz,initrd.img}` on the push that touches
|
||||||
|
that file. The slim release bundle carries only a pointer to this
|
||||||
|
version, and `install.sh` fetches the actual files from the registry
|
||||||
|
when the target's on-disk copy is stale. See
|
||||||
[../docs/operations.md](../docs/operations.md) for the install flow.
|
[../docs/operations.md](../docs/operations.md) for the install flow.
|
||||||
|
|
||||||
|
## Release bundling
|
||||||
|
|
||||||
|
`make release` from the repo root produces `vetting-bundle.tar.gz` —
|
||||||
|
orchestrator + agent + install scripts + a `live-image/VERSION`
|
||||||
|
pointer. It does **not** embed `vmlinuz`/`initrd.img`; those come
|
||||||
|
from the registry at install time.
|
||||||
|
|
||||||
## Manual build (dev loop)
|
## Manual build (dev loop)
|
||||||
|
|
||||||
On Windows:
|
On Windows:
|
||||||
|
|||||||
@@ -0,0 +1 @@
|
|||||||
|
v0.1.0
|
||||||
Reference in New Issue
Block a user