Files
gbrain/test
Garry TanandClaude Opus 4.7 adcaf4ac0c feat(ingestion): wintermute-greenfield migration-mode importer (v0.41 T7)
Implements WintermuteGreenfieldSource — the one-shot bulk importer
for migrating the user's existing wintermute brain (13K atoms + 11K
concepts + ~30 ideas) into gbrain via the v0.41 lens packs.

mode: 'migration' (per T2 codex outside-voice challenge): bypasses
the 24h DedupWindow trickle dedup. Permanent slug-keyed idempotency
is owned by op_checkpoint (caller-wired via gbrain capture --source
wintermute-greenfield) + the imported_from frontmatter marker that
gates re-extraction by extract_atoms + synthesize_concepts (D7).

@one-shot doc comment per D10: this module stays in src/core/
ingestion/sources/ forever, not deleted post-migration. Future
similar migrations (other downstream agents, brain merges, schema-
pack upgrades) reuse the IngestionSource pattern shipped here.
Deleting the working example is short-sighted.

Walk:
  - ~/git/brain/atoms/{YYYY-MM-DD}/*.md (atoms, date-bucketed)
  - ~/git/brain/concepts/*.md (concepts, flat)
  - ~/git/brain/ideas/*.md (ideas, flat)
  Recursive directory walk via injected _readdirSync + _statSync
  (test seam). Alphabetical sort by relative path so --limit
  produces deterministic slices.

Per file:
  1. Read content; gray-matter parses frontmatter + body
  2. Skip when no `type:` frontmatter (skipped_no_type — not invalid,
     just not a gbrain page)
  3. Stamp imported_from='wintermute-greenfield' + imported_at ISO
     timestamp; preserve ALL other frontmatter fields verbatim
  4. Re-stringify via matter.stringify
  5. Emit IngestionEvent with content_type='text/markdown',
     untrusted_payload=false (local user-owned files), metadata
     carrying slug + page_type + original_path + original_frontmatter
     + importer + importer_version

Per-row validation failure → JSONL audit at
~/.gbrain/audit/wintermute-greenfield-failures-YYYY-Www.jsonl per
D12. Failed-file processing continues (don't fail-fast on one bad
row). Audit dir created lazily via mkdirSync recursive on first
write.

CLI flags supported via opts:
  --dry-run: walks + validates + stamps but doesn't emit
  --limit N: processes only the first N files (alphabetical)

The CLI surface lands via gbrain capture --source wintermute-greenfield
in a follow-up commit (capture.ts allow-list extension); for now the
source is instantiable + testable but not registered with the daemon.

Tests (test/ingestion/wintermute-greenfield.test.ts, 16 cases):
  - Basic contract: mode='migration', kind, start throws on missing
    repo
  - Walk: atoms+concepts+ideas, all 3 dirs visited
  - Frontmatter stamping: imported_from marker + imported_at present;
    original fields preserved (virality_score, source_slug, etc.)
  - Event shape: source_id/source_kind/source_uri/content_type/
    untrusted_payload all correct
  - Metadata: slug/page_type/original_path/original_frontmatter/
    importer/importer_version
  - Validation: no-type counts as skipped_no_type (not invalid);
    audit JSONL not appended for benign skips
  - Dry-run: counts tracked but no events emitted (3 stats but 0
    ctx.emitted)
  - --limit: only N files processed
  - Deterministic ordering: alphabetical relative-path sort means
    --limit 1 always picks the alphabetically-first file
  - healthCheck: ok after clean run; warn before start

All 16 pass; typecheck clean.

Plan: ~/.claude/plans/system-instruction-you-are-working-toasty-milner.md
Task T7 of 13.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-24 00:21:45 -07:00
..