josh
39563f6934
Wave 1: Plan 01 (scaffold + test infra)
Wave 2: Plans 02 (eslint firewall), 03 (save layer), 04 (content pipeline),
05 (asset provenance — autonomous:false human-curate checkpoint),
06 (doctrine docs)
Wave 3: Plan 07 (CI workflow)
All 16 Phase-1 REQ-IDs covered. Plan-checker found 4 blockers + 6 warnings
on first pass; revision iteration 1 landed all 10 fixes; iteration 2
returned VERIFICATION PASSED. Two orchestrator judgment calls during
revision: (1) implement CORE-04 localStorage fallback in Phase 1 (the
literal requirement and ROADMAP success criterion #2 both call for it),
(2) reclassify STRY-09 as vacuously satisfied in Phase 1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
28 KiB
28 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 01 | 05 | execute | 2 |
|
|
false |
|
|
Purpose: This is the floor of the asset pipeline that Phase 5 will scale up to production volume. CONTEXT D-01/D-02/D-03 lock the shape (sidecar + CI walker, no curator workflow, no two-stage promotion). The 10–20 reference set is the seed against which Phase 5+ asset migrations will be visually regressed. RESEARCH § Pattern 6 provides verbatim code.
Output: A complete asset pipeline floor: validator script, sidecar schema, refused-sample fixture, ~10–20 north-star images with provenance sidecars, and a Vitest test enforcing the gate.
This plan is autonomous: false because Task 2 — committing the 10–20 north-star images — requires human curation per CONTEXT D-01 + D-03 (curation gate IS the human reviewer per CONTEXT). The user must approve which images ship. If the user has no local AI image tool, the fallback (commit licensed-CC-BY photographs of real cottage gardens or hand-painted references with provenance fields filled honestly) is acceptable per RESEARCH Open Question #5 + Environment Availability — but still needs human selection.
<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/01-foundations-and-doctrine/01-CONTEXT.md @.planning/phases/01-foundations-and-doctrine/01-RESEARCH.md @.planning/phases/01-foundations-and-doctrine/01-01-SUMMARY.md @CLAUDE.md @.planning/research/PITFALLS.md Task 1: Provenance validator script + Zod sidecar schema + Vitest enforcement test scripts/validate-assets.mjs, scripts/validate-assets.test.ts, assets/__samples__/refused/no-provenance.png, assets/__samples__/refused/.gitkeep - .planning/phases/01-foundations-and-doctrine/01-RESEARCH.md § "Pattern 6: Provenance Sidecar Validator" (verbatim ~30-line script) and § "Provenance sidecar example" (the JSON shape) and § Open Question #2 (optional `provenance_schema_version` field) - .planning/phases/01-foundations-and-doctrine/01-CONTEXT.md (D-01 — full provenance metadata, 6 fields; D-02 — vendor deferred; D-03 — sidecar + CI walker, refused sample, no curator workflow) - CLAUDE.md "Code Style" — provenance metadata: `{model_id, checkpoint_hash, prompt, seed, sampler, params}` - REQUIREMENTS.md AEST-08 + AEST-09 - package.json (Plan 01 already created the `validate:assets` script that calls this file) - **validate-assets.mjs:** - Recursively walks `process.env.ASSETS_DIR ?? 'assets'` using `node:fs/promises` (Node 20+ supports `readdir({recursive: true, withFileTypes: true})`). - Skips any path under `assets/__samples__/refused/` (the gate proof — these files INTENTIONALLY have no sidecar). - Skips `.gitkeep` files. - Skips files ending in `.provenance.json` (those are sidecars, not assets). - For every other file, requires a sibling `.provenance.json` (e.g., `garden-soil-01.png` requires `garden-soil-01.png.provenance.json` per RESEARCH § Pattern 6 sidecar naming convention decision). - Reads each sidecar and validates against `ProvenanceSchema` (Zod with the 6 required fields + optional `provenance_schema_version`). - On any missing or invalid sidecar, prints a clear error and exits non-zero. - On success, prints `[provenance] all assets carry valid provenance.` and exits 0. - **validate-assets.test.ts:** - **Positive case:** Runs the validator against the real `/assets/` tree (the default — no `ASSETS_DIR` override) and asserts exit 0. This proves the north-star set + refused-sample dir together pass the gate. - **Negative case (BLOCKER 2 fix):** Generates a per-test-run unique tmpdir under `os.tmpdir()` (using `node:os` + `node:path` + `fs.mkdtemp`), drops a single PNG with no sidecar inside, runs the validator with `ASSETS_DIR=` set in the env, asserts exit code !== 0 and stderr/stdout contains the expected error message. Cleans up the tmpdir in `afterAll`. **No risk of polluting `/assets/`, no Ctrl-C cleanup hazard** — even if the test runner is killed mid-run, the OS reclaims the tmpdir on next reboot. **Step 1 — `scripts/validate-assets.mjs`** (per RESEARCH § Pattern 6 verbatim, with one improvement: the optional `provenance_schema_version` per Open Question #2): ```javascript #!/usr/bin/env node // scripts/validate-assets.mjs — Phase 1 asset provenance gate (PIPE-03, AEST-08, AEST-09) // // Walks /assets/ (or process.env.ASSETS_DIR for tests), requires every non-sidecar // non-.gitkeep file to have a sibling .provenance.json validating against // ProvenanceSchema. Excludes /assets/__samples__/refused/ (which intentionally lacks // sidecars to prove the gate). // // Per CONTEXT D-03: minimum-viable. No curator workflow, no two-stage promotion, // no pre-commit hook. Sidecar + this script + CI is the entire pipeline. // // Per CONTEXT D-01: 6 required fields per CLAUDE.md provenance metadata. // Per RESEARCH Open Question #2: optional provenance_schema_version for Phase 5 fwd-compat.import { readdir, readFile } from 'node:fs/promises';
import { join, basename } from 'node:path';
import { z } from 'zod';
const ProvenanceSchema = z.object({
model_id: z.string().min(1),
checkpoint_hash: z.string().min(1),
prompt: z.string().min(1),
seed: z.union([z.string(), z.number()]),
sampler: z.string().min(1),
params: z.record(z.string(), z.unknown()),
provenance_schema_version: z.number().int().positive().optional(),
});
const ASSETS_DIR = process.env.ASSETS_DIR ?? 'assets';
// Refused-sample exclusion is relative to the *real* assets tree; tests pointing
// ASSETS_DIR at a tmpdir won't have these paths so the exclusion is harmless.
const REFUSED_PREFIXES = ['assets/__samples__/refused', 'assets/__test_fixtures__/refused'];
async function* walk(dir) {
let entries;
try {
entries = await readdir(dir, { withFileTypes: true });
} catch (e) {
if (e.code === 'ENOENT') return;
throw e;
}
for (const entry of entries) {
const path = join(dir, entry.name);
if (entry.isDirectory()) {
yield* walk(path);
} else {
yield path;
}
}
}
function normalizePath(p) {
return p.replaceAll('\\', '/');
}
const errors = [];
let assetCount = 0;
for await (const path of walk(ASSETS_DIR)) {
const norm = normalizePath(path);
if (REFUSED_PREFIXES.some((r) => norm.startsWith(r))) continue;
if (norm.endsWith('.provenance.json')) continue;
if (basename(norm) === '.gitkeep') continue;
if (basename(norm) === 'README.md') continue;
assetCount++;
const sidecar = path + '.provenance.json';
try {
const raw = await readFile(sidecar, 'utf8');
const parsed = ProvenanceSchema.safeParse(JSON.parse(raw));
if (!parsed.success) {
errors.push(`${path}: provenance schema validation failed — ${parsed.error.message}`);
}
} catch (e) {
errors.push(`${path}: missing or unreadable provenance sidecar (${sidecar}): ${e.code ?? e.message}`);
}
}
if (errors.length) {
console.error('[provenance] validation failed:');
for (const err of errors) console.error(' ' + err);
process.exit(1);
}
console.log(`[provenance] all ${assetCount} assets carry valid provenance.`);
```
Note the `ASSETS_DIR` env override — this lets the Vitest test point the script at an `os.tmpdir()` fixture directory without modifying production code, **and without leaving stray fixture files in `/assets/`** (BLOCKER 2 fix). The `REFUSED_PREFIXES` list covers paths under the real `assets/` tree; tmpdir-based test runs simply have no such paths and the exclusion is a harmless no-op.
Also note: the script accepts `assets/north-stars/README.md` (skipped by `basename === 'README.md'`) — Task 2 will add this README; without the skip, the validator would demand a sidecar for the README itself.
**Step 2 — Make the script executable (Unix; harmless on Windows):**
```bash
chmod +x scripts/validate-assets.mjs 2>/dev/null || true
```
**Step 3 — Create the refused-sample asset.** Per CONTEXT D-03, a real image file under `assets/__samples__/refused/` proves the gate exists by being intentionally without a sidecar. Use a tiny 1x1 transparent PNG (~70 bytes); generate with Node:
```bash
node -e "const fs = require('fs'); const png = Buffer.from('89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d49444154789c63600100000005000146cd9c5d0000000049454e44ae426082', 'hex'); fs.writeFileSync('assets/__samples__/refused/no-provenance.png', png);"
touch assets/__samples__/refused/.gitkeep
```
The `.gitkeep` ensures the directory persists if the PNG is ever removed; the PNG itself is the gate-proof artifact.
**Step 4 — Run the validator manually:** `node scripts/validate-assets.mjs` should exit 0 (only the refused-sample is in `/assets/`, and it's excluded). Output: `[provenance] all 0 assets carry valid provenance.`
**Step 5 — `scripts/validate-assets.test.ts`** (Vitest integration test). The negative-case fixture is created under `os.tmpdir()` per BLOCKER 2 fix — isolated from the real `/assets/` tree, no orphan-fragility risk, no Ctrl-C cleanup hazard:
```typescript
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { execFile } from 'node:child_process';
import { promisify } from 'node:util';
import { mkdtemp, rm, writeFile } from 'node:fs/promises';
import { join } from 'node:path';
import os from 'node:os';
const exec = promisify(execFile);
const SCRIPT = 'scripts/validate-assets.mjs';
describe('PIPE-03 / AEST-09: asset provenance gate', () => {
it('exits 0 against the real /assets/ tree (refused sample excluded)', async () => {
const result = await exec('node', [SCRIPT]);
expect(result.stdout).toMatch(/all \d+ assets carry valid provenance/);
});
describe('with an isolated tmpdir fixture missing provenance', () => {
let tmpDir: string;
let fixtureFile: string;
beforeAll(async () => {
// Per-test-run unique tmpdir under os.tmpdir() — isolated from /assets/,
// no risk of polluting the real tree even if the runner is killed mid-test.
tmpDir = await mkdtemp(join(os.tmpdir(), 'tlg-provenance-test-'));
fixtureFile = join(tmpDir, 'orphan.png');
// Tiny 1x1 PNG with no sidecar
const png = Buffer.from(
'89504e470d0a1a0a0000000d49484452000000010000000108060000001f15c4890000000d49444154789c63600100000005000146cd9c5d0000000049454e44ae426082',
'hex',
);
await writeFile(fixtureFile, png);
});
afterAll(async () => {
await rm(tmpDir, { recursive: true, force: true });
});
it('exits non-zero with a clear error message when ASSETS_DIR points at the fixture', async () => {
// Run the validator against the isolated tmpdir; the script reads ASSETS_DIR
// from process.env, so the orphan.png is the only file under inspection.
let exitCode = 0;
let combinedOutput = '';
try {
await exec('node', [SCRIPT], { env: { ...process.env, ASSETS_DIR: tmpDir } });
} catch (err: any) {
exitCode = err.code ?? -1;
combinedOutput = (err.stdout ?? '') + (err.stderr ?? '');
}
expect(exitCode).toBe(1);
expect(combinedOutput).toMatch(/validation failed/);
expect(combinedOutput).toMatch(/orphan\.png/);
expect(combinedOutput).toMatch(/missing.*provenance sidecar/i);
});
});
});
```
**Step 6 — Run `npm test` and confirm the validate-assets test passes.** The positive case asserts the real `/assets/` tree (refused-sample dir + Task 2's north-stars) passes the gate; the negative case runs the script against an isolated `os.tmpdir()` fixture with one orphan PNG and asserts exit 1 + the expected error message.
Per RESEARCH § Pattern 6 ("Refused-sample test"), the negative-case fixture proves the gate fires; the BLOCKER 2 fix moves that fixture to `os.tmpdir()` so it cannot pollute the real `/assets/` tree.
**Step 7 — Commit `feat(01-05): asset provenance validator + Zod sidecar schema + refused-sample fixture + PIPE-03 enforcement test (tmpdir-isolated)`.**
Task 2 ships the 10–20 hand-curated north-star reference set per CONTEXT D-01 — the visual ground truth Phase 5+ will regenerate against. Per CONTEXT D-03, the curation gate IS the human reviewer (you), not a workflow document. This task pauses for your hands-on curation.
**Three valid paths** (your call):
**Path A — AI-generated (recommended if you have a tool available):**
1. Use whatever AI image tool you currently have (Claude with image generation, Stable Diffusion + watercolor LoRA, Midjourney, Scenario, etc.).
2. Generate 10–20 watercolor-style images representing the visual north-star: walled cottage gardens, real-but-slightly-wrong wildflowers, golden/autumnal palette for Season 1, hand-painted feel, no fantasy elements (no D&D flora — see PROJECT.md "Out of Scope").
3. For each generated image, write a sibling `<filename>.png.provenance.json` with all 6 required fields filled honestly (the exact `model_id` you used, the prompt verbatim, the seed if your tool surfaces one, etc.).
4. Place the pair under `assets/north-stars/<descriptive-slug>.png` + `assets/north-stars/<descriptive-slug>.png.provenance.json`.
**Path B — Hand-painted / photograph fallback (if no AI tool is available locally):**
Per RESEARCH § Open Question #5 + Environment Availability, the provenance schema accepts arbitrary `model_id` strings, so honest "human-painted" or licensed-photograph entries are valid. For each image:
- `model_id`: `"human"` (or `"photograph:cc-by:<photographer>"`)
- `checkpoint_hash`: `"n/a"`
- `prompt`: a description of what the image is
- `seed`: `0`
- `sampler`: `"n/a"`
- `params`: `{ "notes": "Phase 1 fallback per RESEARCH Open Question #5; replaceable in Phase 5+" }`
**Path C — Defer with explicit IOU:**
If neither A nor B is feasible right now, commit **two** placeholder images with full honest provenance saying "placeholder" — enough to prove the schema accepts real entries — and **record the IOU in a dedicated file** at `.planning/phases/01-foundations-and-doctrine/01-05-IOU.md` (do **not** edit `.planning/STATE.md` from a phase-internal task — STATE.md is owned by the orchestrator, per WARNING 5 fix). The IOU file contains date, owner, and the deferred-work statement; the verification phase / Phase-5 entry will surface this IOU back into STATE.md if it is still open at that point. This still satisfies CONTEXT D-01's "10–20 hand-curated" loosely (with explicit IOU) and keeps the rest of Phase 1 unblocked.
Whichever path you choose, also write `assets/north-stars/README.md` documenting:
- What this directory is (the visual ground truth for Phase 5+ regression)
- Which path was chosen (A/B/C) and why
- How to add new images (sidecar naming convention, the 6 required fields)
- When this set will be revisited (Phase 5 is the planned consolidation point per CONTEXT D-02)
**Date:** <YYYY-MM-DD>
**Owner:** <your name or handle>
**Phase:** 01 (Foundations & Doctrine)
**Plan:** 05 (Asset Provenance)
## Deferred Work
Path C was selected for Plan 05 Task 2: the north-star reference set under
`assets/north-stars/` currently contains <N> placeholder images (target: 10–20).
Expansion to the full curated set must happen before Phase 5 begins, since
Phase 5+ visual regression depends on this seed.
## Trigger
Phase 5 entry; or sooner if an AI image tool / hand-painted batch becomes available.
## Acceptance
- 10–20 final images committed under `assets/north-stars/` with valid provenance sidecars.
- This file deleted on resolution (or marked `RESOLVED` with the resolving commit hash).
```
Type `approved` when done, or describe issues encountered (e.g., "AI tool unavailable, going with Path B / Path C").
<threat_model>
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|---|---|---|---|---|
| T-01-06 | Spoofing | Provenance sidecar fabrication (a contributor adds an asset with fabricated provenance) | accept (out of scope for Phase 1) | Single-developer project in Phase 1; not a real threat. RESEARCH § Security Domain explicitly defers this to Phase 8+ when external contributors enter the picture, with human_reviewed_by field signed by a curator. |
| T-01-07 | Tampering | Path traversal via sidecar filename | accept | Sidecars are walked by the validator using node:fs/promises readdir and reads are confined to paths under /assets/ (or process.env.ASSETS_DIR). The validator never resolves paths from sidecar contents. Not exploitable. |
| </threat_model> |
<success_criteria>
- 30-line standalone Node script validates every non-sidecar asset has a sibling
<filename>.provenance.jsonper the 6-field schema. - Refused-sample fixture proves the gate by being intentionally excluded.
- Vitest integration test creates an
os.tmpdir()-isolated missing-provenance fixture, asserts non-zero exit, cleans up — no real-tree pollution risk. - 10–20 north-star reference images committed with valid sidecars (Path A / B / C per CONTEXT D-01 + RESEARCH Open Question #5).
npm run validate:assets(the package.json script Plan 01 created) exits 0.- Phase 5 has a working seed for visual regression and asset-pipeline scale-up. </success_criteria>