* feat: dream_verdicts schema + engine methods Adds the v25 schema migration creating the dream_verdicts table (file_path, content_hash, worth_processing, reasons, judged_at; PRIMARY KEY (file_path, content_hash); RLS-enabled when running as a BYPASSRLS role). Distinct from raw_data (which is page-scoped) — transcripts being judged for synthesis aren't pages. The (file_path, content_hash) key means edited transcripts re-judge automatically. BrainEngine gains: - DreamVerdict + DreamVerdictInput types - getDreamVerdict(filePath, contentHash) → DreamVerdict | null - putDreamVerdict(filePath, contentHash, verdict) — ON CONFLICT upsert Both engines implement (postgres-engine.ts, pglite-engine.ts). This commit alone is functionally inert — nothing reads/writes the table yet. The synthesize phase (later commit) is the consumer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: trusted-workspace allow-list for subagent put_page Adds OperationContext.allowedSlugPrefixes — when set, put_page enforces slug membership in the allow-list instead of the legacy wiki/agents/<id>/... namespace. The trust signal is the SUBMITTER (PROTECTED_JOB_NAMES gates subagent submission so MCP can't reach this field), not the runtime ctx.remote flag — every subagent tool call has remote=true for auto-link safety, so basing trust on remote is incoherent. matchesSlugAllowList(slug, prefixes) helper supports glob suffix '/*' (recursive — wiki/originals/* matches ideas/foo/bar) and exact match for unsuffixed entries. put_page check shape: if (viaSubagent && allowedSlugPrefixes set) → allow-list check else if (viaSubagent) → existing namespace check (regression guard) else → no check (regular CLI) Auto-link is re-enabled for the trusted-workspace path so the cycle's extract phase doesn't have to recompute every edge after synthesize writes. Untrusted remote writes still skip auto-link as before. SubagentHandlerData.allowed_slug_prefixes is the wire field; the synthesize/patterns phases (later commit) populate it from a single source of truth in skills/_brain-filing-rules.json's dream_synthesize_paths.globs array. The model's tool schema description mirrors the allow-list so it writes correct slugs on the first try. IRON RULE security tests: - test/operations-allow-list.test.ts: allow-list ALLOW/REJECT, glob semantics, regression guard for the v0.15 namespace fallback when allow-list is unset, FAIL-CLOSED when subagentId is missing. - test/e2e/dream-allow-list-pglite.test.ts: end-to-end on PGLite, poisoned-transcript style write outside allow-list → REJECTED. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: cycle scaffolding — 8-phase order + transcript discovery Extends ALL_PHASES from 6 → 8: synthesize between sync and extract, patterns between extract and embed. Codex finding #7: patterns MUST run after extract because subagent put_page sets ctx.remote=true and skips auto-link/timeline by default — extract is the canonical edge materialization step. Without that ordering, patterns reads stale graph state. Final order: lint → backlinks → sync → synthesize → extract → patterns → embed → orphans CycleOpts gains: - yieldDuringPhase callback — generic in-phase keepalive for long waits (synthesize fan-out, patterns roll-up). Renews cycle-lock TTL + worker job lock. Mirrors yieldBetweenPhases shape. - synthInputFile / synthDate / synthFrom / synthTo — forwarded to runPhaseSynthesize for the CLI's --input/--date/--from/--to flags. CycleReport.totals additively grows (no schema_version bump): transcripts_processed, synth_pages_written, patterns_written. src/core/cycle/transcript-discovery.ts is a pure filesystem walk: - .txt files only, sorted by path for determinism - date-prefixed basename filter (--date / --from / --to) - min_chars filter (default 2000) - exclude_patterns auto-wraps bare words as \b<word>\b regex (Q-3), power users may pass full regex with anchors - compileExcludePatterns is exported for unit tests Phase implementations land in the next commit; this one only adds the dispatcher slots so commit-by-commit bisect doesn't crash on import-not-found. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: synthesize + patterns phases — gbrain dream actually dreams Synthesize phase (src/core/cycle/synthesize.ts) reads conversation transcripts from dream.synthesize.session_corpus_dir and writes brain-native pages: reflections to wiki/personal/reflections/..., originals to wiki/originals/ideas/..., timeline entries on existing people pages. Pipeline: 1. discoverTranscripts (filesystem walk + filters) 2. cooldown check via dream.synthesize.last_completion_ts config (default 12h; bypassed by --input/--date/--from/--to) 3. cheap Haiku verdict per transcript, cached in dream_verdicts table keyed by (file_path, content_hash) — backfill re-runs skip already-judged transcripts at zero cost 4. fan-out: one Sonnet subagent per worth-processing transcript dispatched with allowed_slug_prefixes (read from skills/_brain-filing-rules.json's dream_synthesize_paths.globs) and idempotency_key dream:synth:<file_path>:<content_hash> 5. wait via waitForCompletion; yieldDuringPhase ticks every child terminal so the cycle-lock TTL refreshes on long backfills 6. collect slugs from subagent_tool_executions for each child (codex finding #2: NOT pages.updated_at, which would pick up unrelated writes) 7. orchestrator dual-write — query each new page from DB, reverse-render via serializeMarkdown, write file to brain_dir. Subagent never gets fs-write access. 8. deterministic summary index page at dream-cycle-summaries/<date> (codex finding #4: slug shape is regex-compatible — no underscores, no .md extension) 9. write completion timestamp ONLY on successful runs Patterns phase (src/core/cycle/patterns.ts) runs after extract so the graph state is fresh. Single Sonnet subagent gathers reflections within dream.patterns.lookback_days (default 30); names a pattern only when ≥dream.patterns.min_evidence (default 3) reflections support it. Same allow-list path as synthesize. CLI flags on `gbrain dream` (src/commands/dream.ts): --input <file> ad-hoc transcript synthesis (implies --phase synthesize; bypasses cooldown) --date YYYY-MM-DD restrict synthesize to one date --from <d> --to <d> backfill range --dry-run runs Haiku verdict (cached), skips Sonnet synthesis. NOT zero LLM calls (codex #8). Conflict detection: --input + --date/--from/--to exits 2. ISO 8601 date format validated; range start > end exits 2. Auto-commit / push deferred to v1.1 (codex finding #5). v1 writes files to brain_dir; user or autopilot handles git. Tests: - test/cycle-patterns.test.ts: structural assertions on the patterns phase (queue + waitForCompletion wired, allow-list threading, subagent_tool_executions provenance, no raw_data dependency). - test/dream-cli-flags.test.ts: argv parsing, conflict detection, ISO date validation, --input implies --phase synthesize, dry-run semantics doc string. - test/e2e/dream-synthesize-pglite.test.ts: 8 cases on PGLite in-memory exercising not_configured, empty corpus, no API key skip path, dry-run, cooldown active vs --input bypass, and the dream_verdicts cache hit path. Per-test rig isolation (each test creates and tears down its own engine) avoids cross-test PGLite WASM contention. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: dream cycle v0.27.0 — skills, CLAUDE.md, migration, changelog - skills/maintain/SKILL.md: synthesize + patterns phases documented with quality bar (Iron Law for synthesis), trust boundary, idempotency, cooldown semantics, CLI invocation patterns. New triggers added so "process today's session" / "synthesize my conversations" route here. - skills/RESOLVER.md: dream cycle triggers route to maintain. - skills/_brain-filing-rules.md: directory table for the five output types (reflections, originals, patterns, people enrichment, cycle summary) with slug shape per row; Iron Law repeated. - skills/migrations/v0.27.0.md: agent-readable migration narrative. Schema migration v25 runs automatically on `gbrain apply-migrations`; synthesize ships disabled by default — opt-in via dream.synthesize.session_corpus_dir + dream.synthesize.enabled. - CLAUDE.md: file inventory updated with new files (cycle/synthesize.ts, cycle/patterns.ts, cycle/transcript-discovery.ts), the 8-phase ordering, the trusted-workspace allow-list trust model, and the v25 schema migration line in the migrate.ts entry. - VERSION: 0.20.4 → 0.27.0 - CHANGELOG.md: v0.27.0 release-summary section per CLAUDE.md voice rules (numbers that matter table, what-this-means closer, "to take advantage of" block), followed by the itemized changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: add patterns E2E + 8-phase cycle E2E + bump synth-cooldown timeouts Two new E2E test files on PGLite (no DATABASE_URL or API key required): - test/e2e/dream-patterns-pglite.test.ts (6 cases) — exercises runPhasePatterns skip paths against a real engine: disabled, default-enabled-but-insufficient-evidence, no-API-key, dry-run. Sibling of dream-synthesize-pglite.test.ts; same per-test rig pattern for engine isolation. - test/e2e/dream-cycle-eight-phase-pglite.test.ts (5 cases) — end-to-end runCycle with the v0.27 8-phase order. Asserts: ALL_PHASES is the documented 8 phases in the right sequence, the dry-run report's phases array preserves that order, CycleReport.totals carries the new transcripts_processed / synth_pages_written / patterns_written fields, --phase synthesize and --phase patterns each run only that phase, and synthInputFile is plumbed correctly through runCycle to runPhaseSynthesize. Bump per-test timeout to 30s on the two synthesize-cooldown E2E tests that create two PGLite engines back-to-back. Default Bun 5s budget is tight under sustained suite pressure (PGLite WASM init costs ~1-2s per engine on macOS); each test passes alone but flakes in the full E2E suite. The third arg `30_000` is Bun's standard test-timeout knob. Full E2E suite (test/e2e/) now: 86 pass / 0 fail / 258 skip. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: ship-prep — typecheck fixes, llms.txt regen, 8-phase test update - src/core/cycle/synthesize.ts + patterns.ts: PageType 'default' → 'note' (TS strict typecheck rejected 'default'; 'note' is a valid PageType for orchestrator-written summary index pages and reverse-render fallback). - src/core/pglite-engine.ts: re-import DreamVerdict + DreamVerdictInput types after the master merge dropped them from the import line. - test/e2e/dream-allow-list-pglite.test.ts: ToolCtx now requires remote: true literal; thread it through every put_page tool call. - test/e2e/dream-patterns-pglite.test.ts: PageType 'default' → 'note' in the seedReflections helper. - test/core/cycle.test.ts: bump expected hook-call count + phase count 6 → 8 to match v0.27 ALL_PHASES extension. - llms-full.txt: regenerate against the updated CHANGELOG + CLAUDE.md so the committed snapshot matches what the generator now produces. Full bun test suite: 2793 pass / 0 fail / 258 skip (3051 tests, 177 files). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update README + INSTALL_FOR_AGENTS for v0.27.0 dream cycle README: maintain skill row mentions synthesize/patterns; gbrain dream command-reference block describes the 8-phase pipeline and the new --input/--date/--from/--to flags. INSTALL_FOR_AGENTS: dream cycle bullet calls out v0.27 conversation synthesis + cross-session pattern detection. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * chore: renumber v0.27.0 → v0.23.0 Master is at v0.22.5; v0.23.0 is the next natural slot for the dream-cycle synthesize + patterns release. Bulk rename across VERSION, package.json, CHANGELOG, migration file, source comments, skills, and llms.txt bundles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): bump cycle.test.ts phase count 6 → 8 The dry-run full-cycle test asserted 6 phases. v0.23 added synthesize and patterns, bringing the total to 8. The unit-side equivalent (test/core/cycle.test.ts) was already updated; this catches the E2E sibling that surfaced after the latest master merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
6.1 KiB
Brain Filing Rules -- MANDATORY for all skills that write to the brain
The Rule
The PRIMARY SUBJECT of the content determines where it goes. Not the format, not the source, not the skill that's running.
Decision Protocol
- Identify the primary subject (a person? company? concept? policy issue?)
- File in the directory that matches the subject
- Cross-link from related directories
- When in doubt: what would you search for to find this page again?
Common Misfiling Patterns -- DO NOT DO THESE
| Wrong | Right | Why |
|---|---|---|
Analysis of a topic -> sources/ |
-> appropriate subject directory | sources/ is for raw data only |
Article about a person -> sources/ |
-> people/ |
Primary subject is a person |
Meeting-derived company info -> meetings/ only |
-> ALSO update companies/ |
Entity propagation is mandatory |
Research about a company -> sources/ |
-> companies/ |
Primary subject is a company |
Reusable framework/thesis -> sources/ |
-> concepts/ |
It's a mental model |
Tweet thread about policy -> media/ |
-> civic/ or concepts/ |
media/ is for content ops |
What sources/ Is Actually For
sources/ is ONLY for:
- Bulk data imports (API dumps, CSV exports, snapshots)
- Raw data that feeds multiple brain pages (e.g., a guest export, contact sync)
- Periodic captures (quarterly snapshots, sync exports)
If the content has a clear primary subject (a person, company, concept, policy issue), it does NOT go in sources/. Period.
Notability Gate
Not everything deserves a brain page. Before creating a new entity page:
- People: Will you interact with them again? Are they relevant to your work?
- Companies: Are they relevant to your work or interests?
- Concepts: Is this a reusable mental model worth referencing later?
- When in doubt, DON'T create. A missing page can be created later. A junk page wastes attention and degrades search quality.
Iron Law: Back-Linking (MANDATORY)
Every mention of a person or company with a brain page MUST create a back-link FROM that entity's page TO the page mentioning them. This is bidirectional: the new page links to the entity, AND the entity's page links back.
Format for back-links (append to Timeline or See Also):
- **YYYY-MM-DD** | Referenced in [page title](path/to/page.md) -- brief context
An unlinked mention is a broken brain. The graph is the intelligence.
Citation Requirements (MANDATORY)
Every fact written to a brain page must carry an inline [Source: ...] citation.
Three formats:
- Direct attribution:
[Source: User, {context}, YYYY-MM-DD] - API/external:
[Source: {provider}, YYYY-MM-DD]or[Source: {publication}, {URL}] - Synthesis:
[Source: compiled from {list of sources}]
Source precedence (highest to lowest):
- User's direct statements (highest authority)
- Compiled truth (pre-existing brain synthesis)
- Timeline entries (raw evidence)
- External sources (API enrichment, web search -- lowest)
When sources conflict, note the contradiction with both citations. Don't silently pick one.
Raw Source Preservation
Every ingested item should have its raw source preserved for provenance.
Size routing (automatic via gbrain files upload-raw):
- < 100 MB text/PDF: stays in the brain repo (git-tracked) in a
.raw/sidecar directory alongside the brain page - >= 100 MB OR media files (video, audio, images): uploaded to cloud
storage (Supabase Storage, S3, etc.) with a
.redirect.yamlpointer left in the brain repo. Files >= 100 MB use TUS resumable upload (6 MB chunks with retry) for reliability.
Upload command:
gbrain files upload-raw <file> --page <page-slug> --type <type>
Returns JSON: {storage: "git"} for small files, {storage: "supabase", storagePath, reference} for cloud.
The .redirect.yaml pointer format:
target: supabase://brain-files/page-slug/filename.mp4
bucket: brain-files
storage_path: page-slug/filename.mp4
size: 524288000
size_human: 500 MB
hash: sha256:abc123...
mime: video/mp4
uploaded: 2026-04-11T...
type: transcript
Accessing stored files:
gbrain files signed-url <storage-path> # Generate 1-hour signed URL
gbrain files restore <dir> # Download back to local
This ensures any derived brain page can be traced back to its original source, and large files don't bloat the git repo.
Dream-cycle synthesize / patterns directories (v0.23)
The synthesize and patterns phases of gbrain dream write to a
fixed allow-list of paths sourced from _brain-filing-rules.json's
dream_synthesize_paths.globs array. Editing that JSON is the ONLY way
to add a new directory the synthesis subagent may write to:
| Output type | Slug pattern | What goes here |
|---|---|---|
| Reflection | wiki/personal/reflections/YYYY-MM-DD-<topic>-<hash[:6]> |
Self-knowledge, emotional processing, pattern recognition. Verbatim quotes from the user, with analysis. |
| Original idea | wiki/originals/ideas/YYYY-MM-DD-<idea>-<hash[:6]> |
New frames, theses, mental models, "conceptive ideologist" outputs. Capture the user's exact phrasing — that's the artifact. |
| People enrichment | wiki/people/<existing-slug> |
Timeline entries appended to existing people pages from session mentions. Stub pages for new substantive people. |
| Pattern | wiki/personal/patterns/<theme> |
Cross-session theme detected across ≥3 reflections. Highest-leverage output: a pattern can span 25 years if reflections reference dated content. |
| Cycle summary | dream-cycle-summaries/YYYY-MM-DD |
Index of every page produced by one dream cycle. Auto-written deterministically by the orchestrator. |
Iron Law for synthesize output:
- Quote the user verbatim. Do not paraphrase memorable phrasings.
- Cross-reference compulsively: every new page MUST link to existing brain content.
- Slug discipline: lowercase alphanumeric and hyphens only, slash-separated. NO underscores, NO file extensions.
- Edited transcripts produce NEW slugs (content-hash suffix changes) — never silently overwrite a prior reflection.