Files
josh 3625ef85e6 docs(01-04): complete content pipeline plan
PIPE-01 + STRY-09 satisfied. Vite-native loader with literal
import.meta.glob patterns; FragmentSchema regex enforced; demo
fragment proves round-trip; 5 Vitest assertions cover schema
violations; content/README.md is the writer-facing contract.
2026-05-08 23:32:17 -04:00

200 lines
18 KiB
Markdown

---
phase: 01-foundations-and-doctrine
plan: 04
subsystem: content-pipeline
tags: [zod, yaml, gray-matter, vite, import.meta.glob, fragments, validation, pipe-01]
# Dependency graph
requires:
- phase: 01-01-scaffold-and-test-infra
provides: "zod, yaml, gray-matter, vitest, happy-dom installed; src/content/ firewall directory; /content/seasons/ + /content/dialogue/ trees; vitest.config.ts include glob picks up src/**/*.test.ts; pre-declared compile:ink no-op script in package.json"
provides:
- "Zod schemas: FragmentSchema (with stable-string-ID regex `^season\\d+\\.[a-z0-9._-]+$`) and SeasonContentSchema (wraps fragments[])"
- "Vite-native loader (src/content/loader.ts) using `import.meta.glob('/content/seasons/*/fragments.yaml', { eager: true, query: '?raw', import: 'default' })` with literal patterns (Pitfall 1 honored)"
- "Build-time validation: schema violations throw at module-eval time, failing `npm run build` non-zero (PIPE-01 contract)"
- "Test-only `loadFragmentsFromGlob(yamlGlob, mdGlob)` helper for unit-test injection without touching the filesystem"
- "Demo fragment `season0.demo.first-light` under /content/seasons/00-demo/fragments.yaml proving end-to-end round-trip"
- "content/README.md documenting the /content/ convention, ID regex, YAML and Markdown authoring options for Phase 2 writers"
- "5 Vitest assertions covering 2 happy-path + 3 schema-violation cases (numeric id, season out of range, missing frontmatter id)"
- "Public surface src/content/index.ts re-exporting fragments + loadFragmentsFromGlob + schemas for Phase 2 consumers"
affects: [01-06-doctrine-docs, 01-07-ci-workflow, 02-season-1-vertical-slice, 02-onwards]
# Tech tracking
tech-stack:
added:
- "(no new packages — all deps installed in 01-01: zod@^4.4.3, yaml@^2.8.4, gray-matter@^4.0.3)"
patterns:
- "Vite-native build-time content pipeline: loader.ts runs at module-eval time, throws bubble through Vite to fail npm run build. No separate validation script needed; the build IS the validator."
- "Literal-glob discipline (RESEARCH Pitfall 1): every import.meta.glob call uses a string literal. Vite's plugin walks the AST at build time and cannot resolve runtime expressions, so any computed glob pattern silently produces an empty result."
- "Test-only injection helper pattern: loadFragmentsFromGlob takes mocked glob outputs as parameters so unit tests can prove schema-violation throws fire without writing real malformed files into /content/. The build-time validator and the test helper share the same Zod parse + throw-with-prefix code."
- "Stable-string-ID convention enforced at the regex level (CLAUDE.md Code Style + MEMR-03): `^season\\d+\\.[a-z0-9._-]+$`. Numeric IDs are physically rejected by the schema, not just discouraged in style."
- "Documentation-as-contract: content/README.md is the writer-facing API for Phase 2. Phase 2 writers can author fragments without reading any TypeScript, against a schema whose regex is duplicated in the README."
key-files:
created:
- "src/content/schemas/fragment.ts — FragmentSchema (id regex, season 0..7, body min 1)"
- "src/content/schemas/season.ts — SeasonContentSchema wrapping fragments[]"
- "src/content/schemas/index.ts — schemas re-export barrel"
- "src/content/loader.ts — Vite-native loader with two import.meta.glob calls + loadFragmentsFromGlob test helper"
- "src/content/loader.test.ts — 5 PIPE-01 assertions (2 happy-path + 3 throws)"
- "src/content/index.ts — public surface for Phase 2 consumers"
- "content/seasons/00-demo/fragments.yaml — demo fragment season0.demo.first-light (removed in Phase 2)"
- "content/README.md — writer-facing /content/ convention documentation"
modified: []
deleted:
- "src/content/.gitkeep — replaced by real source files; per wave1_handoff this is the firewall marker that this plan was meant to retire"
key-decisions:
- "Used explicit `.ts` import extensions and `import type` for Fragment/SeasonContent — required by tsconfig.app.json's `verbatimModuleSyntax: true` and `allowImportingTsExtensions: true`. Without `.ts` suffixes the build fails type-checking; without `import type` for type-only re-exports the lint blocks the import."
- "Demo fragment uses `season: 0` (not 1) so the schema range `[0, 7]` accommodates the Phase-1 demo. Phase 2 will narrow the range to `[1, 7]` when /content/seasons/00-demo/ is removed and real Season 1 content lands. The README and the schema comment both flag this transition for Phase 2."
- "Skipped creating content/seasons/00-demo/.gitkeep — the directory has a real fragments.yaml file, and Phase 2 will replace the entire directory anyway. Adding .gitkeep alongside a real file is dead weight."
- "Removed src/content/.gitkeep as part of the feat commit. The wave1_handoff explicitly identifies it as 'the firewall marker — your plan replaces it with real source files'. Leaving it in alongside loader.ts/schemas/index.ts would be vestigial."
- "loader.ts ships BOTH yaml and md glob handling in Phase 1, even though Phase 1 has no .md fragments. Reasoning: the wiring is identical, the test helper exercises the md path, and Phase 2 should not need to re-edit loader.ts to begin authoring per-file Markdown fragments. The mdFiles glob will simply expand to {} until /content/seasons/<slug>/fragments/*.md files exist."
patterns-established:
- "Build-as-validator: throw at module-eval time → Vite catches → npm run build exits non-zero. No separate validation script, no CI-only check, no opt-in. PIPE-01 is satisfied because the build itself is the validator."
- "Test the helper, not the glob: import.meta.glob is a Vite build-time primitive that's hard to mock cleanly. Exposing a parallel `loadFragmentsFromGlob(yamlGlob, mdGlob)` helper that uses the same Zod parse + throw lets the unit test inject failure cases directly. The build-time loader and the test helper share identical validation semantics."
- "Error-message prefix discipline: every throw carries `[content] schema violation in <path>` so a build failure points at the offending file. Tests assert the prefix via regex so any future refactor that drops the prefix breaks the suite."
requirements-completed: [PIPE-01, STRY-09]
# Metrics
duration: 8min
completed: 2026-05-09
---
# Phase 1 Plan 04: Content Pipeline Summary
**Vite-native build-time content pipeline (`src/content/loader.ts`) with Zod schemas (FragmentSchema + SeasonContentSchema), the stable-string-ID regex `^season\d+\.[a-z0-9._-]+$` from CLAUDE.md, one demo fragment proving end-to-end round-trip, and 5 Vitest assertions proving schema violations throw at module-eval time (PIPE-01).**
## Performance
- **Duration:** ~8 min
- **Started:** 2026-05-09T03:21:00Z (approx)
- **Completed:** 2026-05-09T03:30:00Z
- **Tasks:** 2 (both completed atomically)
- **Files created:** 8 (5 source + 1 test + 1 demo content + 1 README); 1 deleted (`src/content/.gitkeep`)
## Accomplishments
- **Zod schemas with the stable-string-ID regex.** `FragmentSchema` rejects numeric IDs and IDs that don't match `^season\d+\.[a-z0-9._-]+$`. `SeasonContentSchema` wraps `fragments[]` so a YAML file with malformed top-level shape is also rejected. Both schemas are colocated in `src/content/schemas/` with a barrel re-export.
- **Vite-native loader using literal `import.meta.glob` patterns (RESEARCH Pitfall 1 honored).** Two glob calls — one for `/content/seasons/*/fragments.yaml`, one for `/content/seasons/*/fragments/*.md` — both with `{ eager: true, query: '?raw', import: 'default' }`. Throws on schema violation at module-eval time, which fails `npm run build` non-zero (PIPE-01).
- **One demo fragment proves round-trip.** `season0.demo.first-light` under `/content/seasons/00-demo/fragments.yaml` validates and is included in the production bundle. `npm run build` is green, which means the loader executed and the schema accepted the demo.
- **5 Vitest assertions cover the schema-violation matrix.** 2 happy-path (empty globs, valid YAML) + 3 schema-violation throws (numeric id, season out of [0,7] range, Markdown frontmatter missing required id). All 5 pass; the full Phase-1 suite (sentinel + content) is 6 green tests.
- **`content/README.md` documents the convention for Phase 2 writers.** Captures the directory shape, the ID regex, both YAML and Markdown authoring forms, the validation guarantee, and the Phase-2 transition notes (e.g., season range will narrow to [1,7] once the demo is removed). Phase 2 writers can author fragments without reading TypeScript.
- **`compile:ink` no-op stub from Plan 01 confirmed runnable.** Verified `npm run compile:ink` exits 0 with the placeholder echo message, per CONTEXT D-08 (Ink deferred to Phase 2).
## Task Commits
Each task was committed atomically:
1. **Task 1: Vite-native content pipeline + Zod schemas + demo fragment + /content/ README**`d52e35f` (feat)
2. **Task 2: PIPE-01 enforcement test — schema violations throw at content load**`c49710e` (test)
_Note: Plan-level final metadata commit (this SUMMARY.md) is owned by the orchestrator after all parallel-wave agents return._
## Files Created/Modified
**Created (8):**
- `src/content/schemas/fragment.ts` — FragmentSchema (Zod) with id regex `^season\d+\.[a-z0-9._-]+$`, season `int [0,7]`, body `min(1)`. Exports `Fragment` type via `z.infer`.
- `src/content/schemas/season.ts` — SeasonContentSchema wrapping `z.array(FragmentSchema)`. Exports `SeasonContent` type.
- `src/content/schemas/index.ts` — Barrel re-export of both schemas + types.
- `src/content/loader.ts` — Two `import.meta.glob` calls (yaml + md) with literal patterns; `loadYamlFragments` + `loadMdFragments` helpers throw on schema violation; flat `fragments: Fragment[]` export; test-only `loadFragmentsFromGlob(yamlGlob, mdGlob?)` helper for unit-test injection.
- `src/content/index.ts` — Public surface re-exporting `fragments`, `loadFragmentsFromGlob`, schemas, and types for Phase 2 consumers.
- `src/content/loader.test.ts` — 5 Vitest assertions: empty globs, valid YAML round-trip, numeric-id throws, season-99 throws, missing-id-frontmatter throws. All asserting the `[content] schema violation` error-message prefix.
- `content/seasons/00-demo/fragments.yaml` — Demo fragment `season0.demo.first-light` proving end-to-end round-trip (removed in Phase 2).
- `content/README.md` — Writer-facing documentation of the /content/ convention, ID regex, YAML and Markdown authoring options, validation guarantee, and Phase 2 transition notes.
**Deleted (1):**
- `src/content/.gitkeep` — Was placed in Plan 01 as a firewall marker; removed now that real source files populate the directory. The wave1_handoff explicitly identifies this as the marker this plan was meant to retire.
**Modified:** None.
## Decisions Made
- **Explicit `.ts` import suffixes + `import type` for type-only re-exports.** `tsconfig.app.json` has `verbatimModuleSyntax: true` and `allowImportingTsExtensions: true`. Without the `.ts` suffix on imports the build fails type-checking; without `import type` for `Fragment`/`SeasonContent` re-exports the strict-TS lint blocks the surface. This is a global pattern for the project; future plans should follow it.
- **Demo fragment uses `season: 0`.** The schema accepts `[0, 7]` so the demo can ship in Phase 1 without polluting Season 1's slug space. Phase 2 will narrow the range to `[1, 7]` when the demo is removed. Both the schema comment and the README flag this transition.
- **No `content/seasons/00-demo/.gitkeep` created.** The directory has a real `fragments.yaml` file. Phase 2 will replace the entire directory anyway. Adding `.gitkeep` alongside a real file is dead weight.
- **`loader.ts` ships both YAML and Markdown glob handling in Phase 1.** Phase 2 should not need to re-edit `loader.ts` to begin authoring per-file Markdown fragments. The `mdFiles` glob simply expands to `{}` until `/content/seasons/<slug>/fragments/*.md` files exist. The Markdown path is exercised by Test 5 in `loader.test.ts`.
- **Build-time error message includes the offending file path.** Every throw carries `[content] schema violation in <path>` so a `npm run build` failure points the writer at the broken file. Tests assert this prefix via regex, so any future refactor that drops the prefix breaks the suite.
## Deviations from Plan
None — plan executed exactly as written. The plan's `<action>` block was followed verbatim across both tasks; the only minor adjustments were:
1. Adding `.ts` import suffixes and `import type` for type-only re-exports (required by the project's existing TS config; not a deviation, just an environmental requirement). The plan's pseudo-code was language-agnostic; the actual file-on-disk respects the project's strict-TS verbatim-module-syntax setting.
If the orchestrator's deviation-detection treats those as adjustments, classify them as Rule 3 (blocking — without them, `tsc -b` fails). They were applied transparently and required no scope change.
## Issues Encountered
None. `npm run build` passed first try; `npx vitest run src/content/loader.test.ts` passed all 5 tests first try; `npm test` (full suite) is 6 green.
## Authentication Gates
None — content pipeline is build-time only; no external services.
## Threat Flags
None. Per the plan's `<threat_model>`: "content pipeline threats are minimal. Path traversal via `import.meta.glob` (a malicious content file with `../../` in frontmatter) is not exploitable: Vite glob expansion is at build time; the validator step never resolves paths from frontmatter values." This plan introduced no new network endpoints, auth paths, file-access patterns, or schema changes at trust boundaries.
## Known Stubs
- **`compile:ink` is a no-op echo + `exit 0`.** Inherited from Plan 01; not introduced by this plan. Per CONTEXT D-08 / RESEARCH § "Pattern 4 — Ink files in Phase 1": Phase 2 will replace this with `inklecate -o src/content/compiled-ink/ content/dialogue/*.ink` once authored Ink files exist.
- **`/content/seasons/00-demo/fragments.yaml` itself is a stub.** Intentional. It exists only to prove the loader round-trips end-to-end. Phase 2 deletes `/content/seasons/00-demo/` and creates `/content/seasons/01-soil/` with real fragments. The README and schema comment both document this transition.
- **Markdown glob expands to `{}` in Phase 1.** No per-file Markdown fragments exist yet. The wiring is in place so Phase 2 can author them without re-editing `loader.ts`. Test 5 in `loader.test.ts` exercises the Markdown validation path via the test helper.
These are all intentional Phase-1 stubs; none block production at the Phase-1 boundary.
## Next Plan / Next Phase Readiness
**Plan 06 (doctrine docs):** Ready. `npm run build` succeeds, so the doctrine docs can reference this content-pipeline implementation as a working example of the build-as-validator pattern.
**Plan 07 (CI workflow):** Ready. The `ci` script (`npm run lint && npm run test && npm run validate:assets && npm run build`) now has a green `npm run build` and a green `npm test` (sentinel + 5 PIPE-01 assertions). Plan 07's CI workflow YAML can call `npm ci && npm run ci` and rely on the build step exercising the content pipeline.
**Phase 2 (Season 1 vertical slice):** The loader is the contract Phase 2 writes against. When Phase 2 begins authoring `/content/seasons/01-soil/`:
- Follow `content/README.md` Section "Adding fragments" for the YAML and Markdown authoring forms.
- Use the ID convention `season1.<slug>` per the regex `^season1\.[a-z0-9._-]+$`.
- The test in `src/content/loader.test.ts` proves any deviation from the schema fails the build.
- Delete `/content/seasons/00-demo/` when real Season 1 content lands.
- Narrow the schema range from `[0, 7]` to `[1, 7]` in `src/content/schemas/fragment.ts` once the demo is removed.
- Begin authoring `.ink` files under `/content/dialogue/` and replace the `compile:ink` no-op with the real `inklecate` invocation.
**For Phase 2's writer (writer-facing summary):**
- Fragment ID regex: `^season\d+\.[a-z0-9._-]+$`
- Demo fragment path (delete in Phase 2): `content/seasons/00-demo/fragments.yaml`
- `compile:ink` is currently a no-op echo (per CONTEXT D-08); Phase 2 swaps it for a real Ink compile step.
- When authoring real fragments, follow `content/README.md` "Adding fragments" — the test in `src/content/loader.test.ts` proves any deviation from the schema fails the build.
## Self-Check
- [x] `src/content/schemas/fragment.ts` exists — verified.
- [x] `src/content/schemas/season.ts` exists — verified.
- [x] `src/content/schemas/index.ts` exists — verified.
- [x] `src/content/loader.ts` exists — verified.
- [x] `src/content/index.ts` exists — verified.
- [x] `src/content/loader.test.ts` exists — verified.
- [x] `content/seasons/00-demo/fragments.yaml` exists with `season0.demo` fragment — verified.
- [x] `content/README.md` exists, documents `season<N>` convention, says "Never use numeric IDs" — verified.
- [x] `content/dialogue/.gitkeep` exists (inherited from Plan 01) — verified.
- [x] FragmentSchema enforces the regex `^season\d+\.[a-z0-9._-]+$` — verified by inspection of `src/content/schemas/fragment.ts`.
- [x] `loader.ts` calls `import.meta.glob` with literal patterns (2 calls — yaml + md) — verified by inspection.
- [x] `loader.ts` throws on schema violation (`throw new Error("[content] schema violation ...`) — verified by inspection.
- [x] `npm run build` exits 0 — verified.
- [x] `npm run compile:ink` exits 0 — verified.
- [x] `npx vitest run src/content/loader.test.ts` passes 5 tests — verified.
- [x] `npm test` passes the entire Phase-1 suite (6 tests) — verified.
- [x] Task 1 commit `d52e35f` exists — verified in `git log`.
- [x] Task 2 commit `c49710e` exists — verified in `git log`.
**## Self-Check: PASSED**
---
*Phase: 01-foundations-and-doctrine*
*Plan: 04 of 7*
*Completed: 2026-05-09*