diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000..6f0b9fd68 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,59 @@ +# Agents working on GBrain + +This is your install + operating protocol. Claude Code reads `./CLAUDE.md` automatically. +Everyone else (Codex, Cursor, OpenClaw, Aider, Continue, or an LLM fetching via URL): +start here. + +## Install (5 min) + +1. Clone: `git clone https://github.com/garrytan/gbrain ~/gbrain && cd ~/gbrain` +2. Install: `bun install` +3. Init the brain: `gbrain init` (defaults to PGLite, zero-config). For 1000+ files or + multi-machine sync, init suggests Postgres + pgvector via Supabase. +4. Read [`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) for the full 9-step flow + (API keys, identity, cron, verification). + +## Read this order + +1. `./AGENTS.md` (this file) — install + operating protocol. +2. [`./CLAUDE.md`](./CLAUDE.md) — architecture reference, key files, trust boundaries, + test layout. +3. [`./skills/RESOLVER.md`](./skills/RESOLVER.md) — skill dispatcher. Read before any task. + +## Trust boundary (critical) + +GBrain distinguishes **trusted local CLI callers** (`OperationContext.remote = false`, +set by `src/cli.ts`) from **untrusted agent-facing callers** (`remote = true`, set by +`src/mcp/server.ts`). Security-sensitive operations like `file_upload` tighten filesystem +confinement when `remote = true` and default to strict behavior when unset. If you are +writing or reviewing an operation, consult `src/core/operations.ts` for the contract. + +## Common tasks + +- **Configure:** [`docs/ENGINES.md`](./docs/ENGINES.md), + [`docs/guides/live-sync.md`](./docs/guides/live-sync.md), + [`docs/mcp/DEPLOY.md`](./docs/mcp/DEPLOY.md). +- **Debug:** [`docs/GBRAIN_VERIFY.md`](./docs/GBRAIN_VERIFY.md), + [`docs/guides/minions-fix.md`](./docs/guides/minions-fix.md), `gbrain doctor --fix`. +- **Migrate:** [`docs/UPGRADING_DOWNSTREAM_AGENTS.md`](./docs/UPGRADING_DOWNSTREAM_AGENTS.md), + [`skills/migrations/`](./skills/migrations/), `gbrain apply-migrations`. +- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map. + [`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for + single-fetch ingestion. + +## Before shipping + +Run `bun test` plus the E2E lifecycle described in `./CLAUDE.md` (spin up the test +Postgres container, run `bun run test:e2e`, tear it down). Ship via the `/ship` skill, +not by hand. + +## Privacy + +Never commit real names of people, companies, or funds into public artifacts. See the +Privacy rule in `./CLAUDE.md`. GBrain pages reference real contacts; public docs must +use generic placeholders (`alice-example`, `acme-example`, `fund-a`). + +## Forks + +If you are a fork, regenerate `llms.txt` + `llms-full.txt` with your own URL base before +publishing: `LLMS_REPO_BASE=https://raw.githubusercontent.com/your-org/your-fork/main bun run build:llms`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 0944b8374..c8d386375 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,76 @@ All notable changes to GBrain will be documented in this file. +## [0.15.0] - 2026-04-21 + +## **GBrain now talks to LLMs the way modern docs sites do.** +## **One URL, full context. Three files, zero drift.** + +Three new artifacts ship at the repo root: `llms.txt` (llmstxt.org-spec index), `llms-full.txt` (same map with core docs inlined, ~225KB, fits well under a 150k-token context window), and `AGENTS.md` (the non-Claude-agent operating protocol). All three are generator-driven. `scripts/build-llms.ts` reads a curated `scripts/llms-config.ts` and emits `llms.txt` + `llms-full.txt` deterministically; `AGENTS.md` is hand-written and uses relative links so it survives forks and rename. Every agent that clones GBrain now has a one-screen answer to "I just got here, what do I do?" + +README and `INSTALL_FOR_AGENTS.md` now point agents at `AGENTS.md` first. The old install prompt still works, but the leverage point, Codex's read of the plan, was that these files are invisible unless the install path references them. Fixed. + +### The numbers that matter + +Measured on this release: + +| Metric | BEFORE | AFTER | Δ | +|-------------------------------------------------|----------------------------------|-----------------------------------|----------------------------| +| Agent entry points with clear install protocol | 1 (CLAUDE.md, Claude Code only) | 3 (CLAUDE.md + AGENTS.md + llms.txt) | +non-Claude coverage | +| Docs referenced at a single canonical URL | 0 | 20 (across 5 H2 sections) | index exists | +| Full-context fetch round-trips | ~20 (one per doc) | 1 (`llms-full.txt`, 224 KB) | ~20x fewer fetches | +| Tests guarding the doc index | 0 | 7 (paths resolve, idempotent, spec shape, regen-drift, content contract, AGENTS mirror, size budget) | +7 | +| Pre-existing repo bugs found and fixed | — | 1 (`git pull origin main` → `master`) | drive-by | + +The 7 tests enforce content contract: removing `skills/RESOLVER.md` or the Debugging H2 from the config fails `bun test`. Forgetting to rerun `bun run build:llms` after adding a new doc fails `bun test`. The size budget (600KB) fails `bun test` if `llms-full.txt` balloons. + +### What this means for you + +If you're running GBrain: nothing to do. Your agent already has CLAUDE.md. But next time you install GBrain on Codex, Cursor, or OpenClaw, the agent lands on `AGENTS.md` and walks the install without hunting. If you run a fork, regenerate with `LLMS_REPO_BASE=https://raw.githubusercontent.com/your-org/your-fork/main bun run build:llms` to rewrite URLs. If you publish GBrain docs alongside your own, `llms.txt` is the index; `llms-full.txt` is the drop-into-a-context-window bundle. + +Credit to Codex for catching that the original plan's AGENTS.md was underpowered, that the eng review missed a content-contract test, and that the install prompt was the real leverage point. Seven of the fifteen Codex findings landed directly in the plan; three went to user decision; five stayed as intentional NOT-in-scope. + +## To take advantage of this release + +`gbrain upgrade` does not need to do anything. These are new public files; existing installs pick them up on their next pull. + +1. **If you wrote a downstream fork:** regenerate with your URL base. + ```bash + LLMS_REPO_BASE=https://raw.githubusercontent.com/your-org/your-fork/main bun run build:llms + git add llms.txt llms-full.txt && git commit + ``` +2. **If you add a new doc under `docs/`:** add it to `scripts/llms-config.ts`, then + ```bash + bun run build:llms + bun test test/build-llms.test.ts + ``` + CI blocks ship if these drift. +3. **Verify it actually works:** ask a fresh LLM + ``` + Fetch https://raw.githubusercontent.com/garrytan/gbrain/master/llms.txt and tell me + how I'd debug a broken live sync. + ``` + Answer should cite `docs/GBRAIN_VERIFY.md`, `docs/guides/live-sync.md`, and `gbrain doctor`. + +### Itemized changes + +#### Added +- `AGENTS.md` at repo root — ~45-line non-Claude-agent operating protocol. Install, read order, trust boundary, config/debug/migration pointers, fork instructions. Uses relative links so it survives renames. +- `llms.txt` at repo root — llmstxt.org-spec index. H1 + blockquote + 5 required H2 sections (Core entry points, Configuration, Debugging, Migrations) plus an Operational tips block with `gbrain doctor`, `gbrain orphans`, `gbrain repair-jsonb`. ~4KB. +- `llms-full.txt` at repo root — same index with core docs inlined under `## {path}` headings for single-fetch ingestion. ~225KB, under the 600KB `FULL_SIZE_BUDGET`. +- `scripts/llms-config.ts` — curated TS config. `LLMS_REPO_BASE` env var lets forks regenerate with their own URL base. `includeInFull: false` flags entries that should appear in `llms.txt` but not be inlined in `llms-full.txt` (Philosophy, Optional, CHANGELOG). +- `scripts/build-llms.ts` — the generator. Deterministic, no timestamps, sorted by config order. Warns (does not fail) if `llms-full.txt` exceeds `FULL_SIZE_BUDGET` with the biggest entries listed. +- `test/build-llms.test.ts` — 7 cases: paths resolve on disk, generator idempotent, llms.txt spec shape, checked-in files match generator output (drift guard), content contract (RESOLVER / AGENTS / INSTALL_FOR_AGENTS referenced), AGENTS mirrors README+INSTALL install path, size budget enforcement. +- `bun run build:llms` script in `package.json`. + +#### Changed +- `README.md` — adds a one-line LLMs/Agents pointer above the install CTA and a follow-up paragraph under the agent paste block naming `AGENTS.md` + `llms.txt` as fallback entry points for non-Claude agents. +- `INSTALL_FOR_AGENTS.md` — new "Step 0: If you are not Claude Code" prelude points agents at `AGENTS.md` first. +- `CLAUDE.md` — adds `scripts/llms-config.ts`, `scripts/build-llms.ts`, and `AGENTS.md` to Key files. Explicitly notes that committed generator output is NOT analogous to `schema-embedded.ts` (no runtime consumer; committed for GitHub browsing + fork safety). + +#### Fixed +- `INSTALL_FOR_AGENTS.md:136` — `git pull origin main` → `git pull origin master`. Pre-existing drift: README and CI use `master`, `origin/HEAD -> master`, but the upgrade instructions told users to pull from a branch that doesn't exist. Folded into this release as a drive-by fix. + ## [0.14.2] - 2026-04-20 ## **Eight deferred bugs, root-cause fixes, one clean wave.** diff --git a/CLAUDE.md b/CLAUDE.md index 78c29502e..4aa0e517c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -71,6 +71,8 @@ strict behavior when unset. - `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run]`: health checks. v0.12.3 adds two reliability detection checks: `jsonb_integrity` (scans pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata for `jsonb_typeof='string'` rows left over from v0.12.0) and `markdown_body_completeness` (flags pages whose compiled_truth is <30% of raw source when raw has multiple H2/H3 boundaries). Fix hints point at `gbrain repair-jsonb` and `gbrain sync --force`. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. - `src/core/markdown.ts` — Frontmatter parsing + body splitter. `splitBody` requires an explicit timeline sentinel (``, `--- timeline ---`, or `---` immediately before `## Timeline`/`## History`). Plain `---` in body text is a markdown horizontal rule, not a separator. `inferType` auto-types `/wiki/analysis/` → analysis, `/wiki/guides/` → guide, `/wiki/hardware/` → hardware, `/wiki/architecture/` → architecture, `/writing/` → writing (plus the existing people/companies/deals/etc heuristics). - `scripts/check-jsonb-pattern.sh` — CI grep guard. Fails the build if anyone reintroduces the `${JSON.stringify(x)}::jsonb` interpolation pattern (which postgres.js v3 double-encodes). Wired into `bun test`. +- `scripts/llms-config.ts` + `scripts/build-llms.ts` — Generator for `llms.txt` (llmstxt.org-spec web index) + `llms-full.txt` (inlined single-fetch bundle). Curated config drives both. Run `bun run build:llms` after adding a new doc. `LLMS_REPO_BASE` env var lets forks regenerate with their own URL base. `FULL_SIZE_BUDGET` (600KB) caps the inline bundle; generator WARNs if exceeded. Committed output is not analogous to `schema-embedded.ts` (no runtime consumer); we commit for GitHub browsing and fork-safe fetching. +- `AGENTS.md` — Local-clone entry point for non-Claude agents (Codex, Cursor, OpenClaw, Aider). Mirrors `CLAUDE.md` intent via relative links. Claude Code keeps using `CLAUDE.md`. - `docs/UPGRADING_DOWNSTREAM_AGENTS.md` — Patches for downstream agent skill forks to apply when upgrading. Each release appends a new section. v0.10.3 includes diffs for brain-ops, meeting-ingestion, signal-detector, enrich. - `src/core/schema-embedded.ts` — AUTO-GENERATED from schema.sql (run `bun run build:schema`) - `src/schema.sql` — Full Postgres + pgvector DDL (source of truth, generates schema-embedded.ts) @@ -205,7 +207,8 @@ parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redac `test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`), `test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called), `test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present), -`test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics). +`test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics), +`test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases). E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_URL`. - `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes 9 dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's `unnest()` binding is structurally different from PGLite's and gets its own coverage. diff --git a/INSTALL_FOR_AGENTS.md b/INSTALL_FOR_AGENTS.md index af7bac5b0..57ed0c000 100644 --- a/INSTALL_FOR_AGENTS.md +++ b/INSTALL_FOR_AGENTS.md @@ -3,6 +3,17 @@ Read this entire file, then follow the steps. Ask the user for API keys when needed. Target: ~30 minutes to a fully working brain. +## Step 0: If you are not Claude Code + +Read `AGENTS.md` at the repo root first. It's the non-Claude-agent operating +protocol (install, read order, trust boundary, common tasks). Claude Code reads +`CLAUDE.md` automatically and can skip ahead. + +If you fetched this file by URL without cloning yet, the companion files live at: +- `https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` — start here +- `https://raw.githubusercontent.com/garrytan/gbrain/master/llms.txt` — full doc map +- `https://raw.githubusercontent.com/garrytan/gbrain/master/llms-full.txt` — same map, inlined + ## Step 1: Install GBrain ```bash @@ -133,7 +144,7 @@ actually works) is the most important. ## Upgrade ```bash -cd ~/gbrain && git pull origin main && bun install +cd ~/gbrain && git pull origin master && bun install gbrain init # apply schema migrations (idempotent) gbrain post-upgrade # show migration notes for the version range ``` diff --git a/README.md b/README.md index 399d3c01a..2ad2a7ab4 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,8 @@ GBrain is those patterns, generalized. 26 skills. Install in 30 minutes. Your ag > **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys. +> **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code). + ## Install ### On an agent platform (recommended) @@ -28,6 +30,11 @@ https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 26 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes. +If your agent doesn't auto-read `AGENTS.md`, point it at that file first: +`https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` is the non-Claude +agent operating protocol (install, read order, trust boundary, common tasks). For +the full doc map, use `llms.txt` at the same URL root. + ### Standalone CLI (no agent) ```bash diff --git a/VERSION b/VERSION index e867cc2a6..a55105169 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.14.2 +0.15.0 diff --git a/llms-full.txt b/llms-full.txt new file mode 100644 index 000000000..d96bf9cad --- /dev/null +++ b/llms-full.txt @@ -0,0 +1,4404 @@ +# GBrain — Full Context + +> GBrain is a personal knowledge brain and GStack mod for agent platforms. Pluggable engines (PGLite default, Postgres+pgvector for scale), contract-first operations, 26 fat-markdown skills. Teaches agents brain ops, ingestion, enrichment, scheduling, identity, and access control. + +This file concatenates core GBrain documentation for single-fetch ingestion. +For the link-only index, see `llms.txt`. Source of truth: https://github.com/garrytan/gbrain. + +# Core entry points + +## AGENTS.md + +Source: https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md + +# Agents working on GBrain + +This is your install + operating protocol. Claude Code reads `./CLAUDE.md` automatically. +Everyone else (Codex, Cursor, OpenClaw, Aider, Continue, or an LLM fetching via URL): +start here. + +## Install (5 min) + +1. Clone: `git clone https://github.com/garrytan/gbrain ~/gbrain && cd ~/gbrain` +2. Install: `bun install` +3. Init the brain: `gbrain init` (defaults to PGLite, zero-config). For 1000+ files or + multi-machine sync, init suggests Postgres + pgvector via Supabase. +4. Read [`./INSTALL_FOR_AGENTS.md`](./INSTALL_FOR_AGENTS.md) for the full 9-step flow + (API keys, identity, cron, verification). + +## Read this order + +1. `./AGENTS.md` (this file) — install + operating protocol. +2. [`./CLAUDE.md`](./CLAUDE.md) — architecture reference, key files, trust boundaries, + test layout. +3. [`./skills/RESOLVER.md`](./skills/RESOLVER.md) — skill dispatcher. Read before any task. + +## Trust boundary (critical) + +GBrain distinguishes **trusted local CLI callers** (`OperationContext.remote = false`, +set by `src/cli.ts`) from **untrusted agent-facing callers** (`remote = true`, set by +`src/mcp/server.ts`). Security-sensitive operations like `file_upload` tighten filesystem +confinement when `remote = true` and default to strict behavior when unset. If you are +writing or reviewing an operation, consult `src/core/operations.ts` for the contract. + +## Common tasks + +- **Configure:** [`docs/ENGINES.md`](./docs/ENGINES.md), + [`docs/guides/live-sync.md`](./docs/guides/live-sync.md), + [`docs/mcp/DEPLOY.md`](./docs/mcp/DEPLOY.md). +- **Debug:** [`docs/GBRAIN_VERIFY.md`](./docs/GBRAIN_VERIFY.md), + [`docs/guides/minions-fix.md`](./docs/guides/minions-fix.md), `gbrain doctor --fix`. +- **Migrate:** [`docs/UPGRADING_DOWNSTREAM_AGENTS.md`](./docs/UPGRADING_DOWNSTREAM_AGENTS.md), + [`skills/migrations/`](./skills/migrations/), `gbrain apply-migrations`. +- **Everything else:** [`./llms.txt`](./llms.txt) is the full documentation map. + [`./llms-full.txt`](./llms-full.txt) is the same map with core docs inlined for + single-fetch ingestion. + +## Before shipping + +Run `bun test` plus the E2E lifecycle described in `./CLAUDE.md` (spin up the test +Postgres container, run `bun run test:e2e`, tear it down). Ship via the `/ship` skill, +not by hand. + +## Privacy + +Never commit real names of people, companies, or funds into public artifacts. See the +Privacy rule in `./CLAUDE.md`. GBrain pages reference real contacts; public docs must +use generic placeholders (`alice-example`, `acme-example`, `fund-a`). + +## Forks + +If you are a fork, regenerate `llms.txt` + `llms-full.txt` with your own URL base before +publishing: `LLMS_REPO_BASE=https://raw.githubusercontent.com/your-org/your-fork/main bun run build:llms`. + +--- + +## CLAUDE.md + +Source: https://raw.githubusercontent.com/garrytan/gbrain/master/CLAUDE.md + +# CLAUDE.md + +GBrain is a personal knowledge brain and GStack mod for agent platforms. Pluggable +engines: PGLite (embedded Postgres via WASM, zero-config default) or Postgres + pgvector ++ hybrid search in a managed Supabase instance. `gbrain init` defaults to PGLite; +suggests Supabase for 1000+ files. GStack teaches agents how to code. GBrain teaches +agents everything else: brain ops, signal detection, content ingestion, enrichment, +cron scheduling, reports, identity, and access control. + +## Architecture + +Contract-first: `src/core/operations.ts` defines ~41 shared operations (adds `find_orphans` in v0.12.3). CLI and MCP +server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`) +dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat +markdown files (tool-agnostic, work with both CLI and plugin contexts). + +**Trust boundary:** `OperationContext.remote` distinguishes trusted local CLI callers +(`remote: false` set by `src/cli.ts`) from untrusted agent-facing callers +(`remote: true` set by `src/mcp/server.ts`). Security-sensitive operations like +`file_upload` tighten filesystem confinement when `remote=true` and default to +strict behavior when unset. + +## Key files + +- `src/core/operations.ts` — Contract-first operation definitions (the foundation). Also exports upload validators: `validateUploadPath`, `validatePageSlug`, `validateFilename`. `OperationContext.remote` flags untrusted callers. +- `src/core/engine.ts` — Pluggable engine interface (BrainEngine). `clampSearchLimit(limit, default, cap)` takes an explicit cap so per-operation caps can be tighter than `MAX_SEARCH_LIMIT`. Exports `LinkBatchInput` / `TimelineBatchInput` for the v0.12.1 bulk-insert API (`addLinksBatch` / `addTimelineEntriesBatch`). +- `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`) +- `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all 40 BrainEngine methods. `addLinksBatch` / `addTimelineEntriesBatch` use multi-row `unnest()` with manual `$N` placeholders. +- `src/core/pglite-schema.ts` — PGLite-specific DDL (pgvector, pg_trgm, triggers) +- `src/core/postgres-engine.ts` — Postgres + pgvector implementation (Supabase / self-hosted). `addLinksBatch` / `addTimelineEntriesBatch` use `INSERT ... SELECT FROM unnest($1::text[], ...) JOIN pages ON CONFLICT DO NOTHING RETURNING 1` — 4-5 array params regardless of batch size, sidesteps the 65535-parameter cap. As of v0.12.3, `searchKeyword` / `searchVector` scope `statement_timeout` via `sql.begin` + `SET LOCAL` so the GUC dies with the transaction instead of leaking across the pooled postgres.js connection (contributed by @garagon). `getEmbeddingsByChunkIds` uses `tryParseEmbedding` so one corrupt row skips+warns instead of killing the query. +- `src/core/utils.ts` — Shared SQL utilities extracted from postgres-engine.ts. Exports `parseEmbedding(value)` (throws on unknown input, used by migration + ingest paths where data integrity matters) and as of v0.12.3 `tryParseEmbedding(value)` (returns `null` + warns once per process, used by search/rescore paths where availability matters more than strictness). +- `src/core/db.ts` — Connection management, schema initialization +- `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`) +- `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags) +- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion) +- `src/core/storage.ts` — Pluggable storage interface (S3, Supabase Storage, local) +- `src/core/supabase-admin.ts` — Supabase admin API (project discovery, pgvector check) +- `src/core/file-resolver.ts` — File resolution with fallback chain (local -> .redirect.yaml -> .redirect -> .supabase) +- `src/core/chunkers/` — 3-tier chunking (recursive, semantic, LLM-guided) +- `src/core/search/` — Hybrid search: vector + keyword + RRF + multi-query expansion + dedup +- `src/core/search/intent.ts` — Query intent classifier (entity/temporal/event/general → auto-selects detail level) +- `src/core/search/eval.ts` — Retrieval eval harness: P@k, R@k, MRR, nDCG@k metrics + runEval() orchestrator +- `src/commands/eval.ts` — `gbrain eval` command: single-run table + A/B config comparison +- `src/core/embedding.ts` — OpenAI text-embedding-3-large, batch, retry, backoff +- `src/core/check-resolvable.ts` — Resolver validation: reachability, MECE overlap, DRY checks, structured fix objects. v0.14.1: `CROSS_CUTTING_PATTERNS.conventions` is an array (notability gate accepts both `conventions/quality.md` and `_brain-filing-rules.md`). New `extractDelegationTargets()` parses `> **Convention:**`, `> **Filing rule:**`, and inline backtick references. DRY suppression is proximity-based via `DRY_PROXIMITY_LINES = 40`. +- `src/core/dry-fix.ts` — `gbrain doctor --fix` engine. `autoFixDryViolations(fixes, {dryRun})` rewrites inlined rules to `> **Convention:** see [path](path).` callouts via three shape-aware expanders (bullet / blockquote / paragraph). Five guards: working-tree-dirty (`getWorkingTreeStatus()` returns 3-state `'clean' | 'dirty' | 'not_a_repo'`), no-git-backup, inside-code-fence, already-delegated (40-line proximity, consistent with detector), ambiguous-multi-match, block-is-callout. `execFileSync` array args (no shell — no injection surface). EOF newline preserved. +- `src/core/backoff.ts` — Adaptive load-aware throttling: CPU/memory checks, exponential backoff, active hours multiplier +- `src/core/fail-improve.ts` — Deterministic-first, LLM-fallback loop with JSONL failure logging and auto-test generation +- `src/core/transcription.ts` — Audio transcription: Groq Whisper (default), OpenAI fallback, ffmpeg segmentation for >25MB +- `src/core/enrichment-service.ts` — Global enrichment service: entity slug generation, tier auto-escalation, batch throttling +- `src/core/data-research.ts` — Recipe validation, field extraction (MRR/ARR regex), dedup, tracker parsing, HTML stripping +- `src/commands/extract.ts` — `gbrain extract links|timeline|all [--source fs|db]`: batch link/timeline extraction. fs walks markdown files, db walks pages from the engine (mutation-immune snapshot iteration; use this for live brains with no local checkout). As of v0.12.1 there is no in-memory dedup pre-load — candidates are buffered 100 at a time and flushed via `addLinksBatch` / `addTimelineEntriesBatch`; `ON CONFLICT DO NOTHING` enforces uniqueness at the DB layer, and the `created` counter returns real rows inserted (truthful on re-runs). +- `src/commands/graph-query.ts` — `gbrain graph-query [--type T] [--depth N] [--direction in|out|both]`: typed-edge relationship traversal (renders indented tree) +- `src/core/link-extraction.ts` — shared library for the v0.12.0 graph layer. extractEntityRefs (canonical, replaces backlinks.ts duplicate) matches both `[Name](people/slug)` markdown links and Obsidian `[[people/slug|Name]]` wikilinks as of v0.12.3. extractPageLinks, inferLinkType heuristics (attended/works_at/invested_in/founded/advises/source/mentions), parseTimelineEntries, isAutoLinkEnabled config helper. `DIR_PATTERN` covers `people`, `companies`, `deals`, `topics`, `concepts`, `projects`, `entities`, `tech`, `finance`, `personal`, `openclaw`. Used by extract.ts, operations.ts auto-link post-hook, and backlinks.ts. +- `src/core/minions/` — Minions job queue: BullMQ-inspired, Postgres-native (queue, worker, backoff, types) +- `src/core/minions/queue.ts` — MinionQueue class (submit, claim, complete, fail, stall detection, parent-child, depth/child-cap, per-job timeouts, cascade-kill, attachments, idempotency keys, child_done inbox, removeOnComplete/Fail). `add()` takes a 4th `trusted` arg (separate from `opts` to prevent spread leakage); protected names in `PROTECTED_JOB_NAMES` require `{allowProtectedSubmit: true}` and the check runs trim-normalized (whitespace-bypass safe). +- `src/core/minions/worker.ts` — MinionWorker class (handler registry, lock renewal, graceful shutdown, timeout safety net). v0.14.0 abort-path fix: aborted jobs now call `failJob` with reason (`timeout`/`cancel`/`lock-lost`/`shutdown`) instead of returning silently. `shutdownAbort` (instance field) fires on process SIGTERM/SIGINT and propagates to `ctx.shutdownSignal` — shell handler listens to it; non-shell handlers don't. +- `src/core/minions/protected-names.ts` — side-effect-free constant module exporting `PROTECTED_JOB_NAMES` + `isProtectedJobName()`. Kept pure so queue core can import without loading handler modules. +- `src/core/minions/handlers/shell.ts` — `shell` job handler. Spawns `/bin/sh -c cmd` (absolute path, PATH-override-safe) or `argv[0] argv[1..]` (no shell). Env allowlist: `PATH, HOME, USER, LANG, TZ, NODE_ENV` + caller `env:` overrides. UTF-8-safe stdout/stderr tail via `string_decoder.StringDecoder`. Abort (either `ctx.signal` or `ctx.shutdownSignal`) fires SIGTERM → 5s grace → SIGKILL on child. Requires `GBRAIN_ALLOW_SHELL_JOBS=1` on worker (gated by `registerBuiltinHandlers`). +- `src/core/minions/handlers/shell-audit.ts` — per-submission JSONL audit trail at `~/.gbrain/audit/shell-jobs-YYYY-Www.jsonl` (ISO-week rotation; override via `GBRAIN_AUDIT_DIR`). Best-effort: `mkdirSync(recursive)` + `appendFileSync`; failures logged to stderr, submission not blocked. Logs cmd (first 80 chars) or argv (JSON array). Never logs env values. +- `src/core/minions/attachments.ts` — Attachment validation (path traversal, null byte, oversize, base64, duplicate detection) +- `src/commands/jobs.ts` — `gbrain jobs` CLI subcommands + `gbrain jobs work` daemon +- `src/commands/features.ts` — `gbrain features --json --auto-fix`: usage scan + feature adoption salesman +- `src/commands/autopilot.ts` — `gbrain autopilot --install`: self-maintaining brain daemon (sync+extract+embed) +- `src/mcp/server.ts` — MCP stdio server (generated from operations) +- `src/commands/auth.ts` — Standalone token management (create/list/revoke/test) +- `src/commands/upgrade.ts` — Self-update CLI. `runPostUpgrade()` enumerates migrations from the TS registry (src/commands/migrations/index.ts) and tail-calls `runApplyMigrations(['--yes', '--non-interactive'])` so the mechanical side of every outstanding migration runs unconditionally. +- `src/commands/migrations/` — TS migration registry (compiled into the binary; no filesystem walk of `skills/migrations/*.md` needed at runtime). `index.ts` lists migrations in semver order. `v0_11_0.ts` = Minions adoption orchestrator (8 phases). `v0_12_0.ts` = Knowledge Graph auto-wire orchestrator (5 phases: schema → config check → backfill links → backfill timeline → verify). `phaseASchema` has a 600s timeout (bumped from 60s in v0.12.1 for duplicate-heavy brains). `v0_12_2.ts` = JSONB double-encode repair orchestrator (4 phases: schema → repair-jsonb → verify → record). `v0_14_0.ts` = shell-jobs + autopilot cooperative (2 phases: schema ALTER minion_jobs.max_stalled SET DEFAULT 3, pending-host-work ping for skills/migrations/v0.14.0.md). All orchestrators are idempotent and resumable from `partial` status. As of v0.14.2 (Bug 3), the RUNNER owns all ledger writes — orchestrators return `OrchestratorResult` and `apply-migrations.ts` persists a canonical `{version, status, phases}` shape after return. Orchestrators no longer call `appendCompletedMigration` directly. `statusForVersion` prefers `complete` over `partial` (never regresses). 3 consecutive partials → wedged → `--force-retry ` writes a `'retry'` reset marker. +- `src/commands/repair-jsonb.ts` — `gbrain repair-jsonb [--dry-run] [--json]`: rewrites `jsonb_typeof='string'` rows in place across 5 affected columns (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter). Fixes v0.12.0 double-encode bug on Postgres; PGLite no-ops. Idempotent. +- `src/commands/orphans.ts` — `gbrain orphans [--json] [--count] [--include-pseudo]`: surfaces pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. Shipped in v0.12.3 (contributed by @knee5). +- `src/commands/doctor.ts` — `gbrain doctor [--json] [--fast] [--fix] [--dry-run]`: health checks. v0.12.3 adds two reliability detection checks: `jsonb_integrity` (scans pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata for `jsonb_typeof='string'` rows left over from v0.12.0) and `markdown_body_completeness` (flags pages whose compiled_truth is <30% of raw source when raw has multiple H2/H3 boundaries). Fix hints point at `gbrain repair-jsonb` and `gbrain sync --force`. v0.14.1: `--fix` delegates inlined cross-cutting rules to `> **Convention:** see [path](path).` callouts (pipes DRY violations into `src/core/dry-fix.ts`); `--fix --dry-run` previews without writing. +- `src/core/markdown.ts` — Frontmatter parsing + body splitter. `splitBody` requires an explicit timeline sentinel (``, `--- timeline ---`, or `---` immediately before `## Timeline`/`## History`). Plain `---` in body text is a markdown horizontal rule, not a separator. `inferType` auto-types `/wiki/analysis/` → analysis, `/wiki/guides/` → guide, `/wiki/hardware/` → hardware, `/wiki/architecture/` → architecture, `/writing/` → writing (plus the existing people/companies/deals/etc heuristics). +- `scripts/check-jsonb-pattern.sh` — CI grep guard. Fails the build if anyone reintroduces the `${JSON.stringify(x)}::jsonb` interpolation pattern (which postgres.js v3 double-encodes). Wired into `bun test`. +- `scripts/llms-config.ts` + `scripts/build-llms.ts` — Generator for `llms.txt` (llmstxt.org-spec web index) + `llms-full.txt` (inlined single-fetch bundle). Curated config drives both. Run `bun run build:llms` after adding a new doc. `LLMS_REPO_BASE` env var lets forks regenerate with their own URL base. `FULL_SIZE_BUDGET` (600KB) caps the inline bundle; generator WARNs if exceeded. Committed output is not analogous to `schema-embedded.ts` (no runtime consumer); we commit for GitHub browsing and fork-safe fetching. +- `AGENTS.md` — Local-clone entry point for non-Claude agents (Codex, Cursor, OpenClaw, Aider). Mirrors `CLAUDE.md` intent via relative links. Claude Code keeps using `CLAUDE.md`. +- `docs/UPGRADING_DOWNSTREAM_AGENTS.md` — Patches for downstream agent skill forks to apply when upgrading. Each release appends a new section. v0.10.3 includes diffs for brain-ops, meeting-ingestion, signal-detector, enrich. +- `src/core/schema-embedded.ts` — AUTO-GENERATED from schema.sql (run `bun run build:schema`) +- `src/schema.sql` — Full Postgres + pgvector DDL (source of truth, generates schema-embedded.ts) +- `src/commands/integrations.ts` — Standalone integration recipe management (no DB needed). Exports `getRecipeDirs()` (trust-tagged recipe sources), SSRF helpers (`isInternalUrl`, `parseOctet`, `hostnameToOctets`, `isPrivateIpv4`). Only package-bundled recipes are `embedded=true`; `$GBRAIN_RECIPES_DIR` and cwd `./recipes/` are untrusted and cannot run `command`/`http`/string health checks. +- `src/core/search/expansion.ts` — Multi-query expansion via Haiku. Exports `sanitizeQueryForPrompt` + `sanitizeExpansionOutput` (prompt-injection defense-in-depth). Sanitized query is only used for the LLM channel; original query still drives search. +- `recipes/` — Integration recipe files (YAML frontmatter + markdown setup instructions) +- `docs/guides/` — Individual SKILLPACK guides (broken out from monolith) +- `docs/integrations/` — "Getting Data In" guides and integration docs +- `docs/architecture/infra-layer.md` — Shared infrastructure documentation +- `docs/ethos/THIN_HARNESS_FAT_SKILLS.md` — Architecture philosophy essay +- `docs/ethos/MARKDOWN_SKILLS_AS_RECIPES.md` — "Homebrew for Personal AI" essay +- `docs/guides/repo-architecture.md` — Two-repo pattern (agent vs brain) +- `docs/guides/sub-agent-routing.md` — Model routing table for sub-agents +- `docs/guides/skill-development.md` — 5-step skill development cycle + MECE +- `docs/guides/idea-capture.md` — Originality distribution, depth test, cross-linking +- `docs/guides/quiet-hours.md` — Notification hold + timezone-aware delivery +- `docs/guides/diligence-ingestion.md` — Data room to brain pages pipeline +- `docs/designs/HOMEBREW_FOR_PERSONAL_AI.md` — 10-star vision for integration system +- `docs/mcp/` — Per-client setup guides (Claude Desktop, Code, Cowork, Perplexity) +- `docs/benchmarks/` — Search quality benchmark results (reproducible, fictional data) +- `skills/_brain-filing-rules.md` — Cross-cutting brain filing rules (referenced by all brain-writing skills) +- `skills/RESOLVER.md` — Skill routing table (based on the agent-fork AGENTS.md pattern) +- `skills/conventions/` — Cross-cutting rules (quality, brain-first, model-routing, test-before-bulk, cross-modal) +- `skills/_output-rules.md` — Output quality standards (deterministic links, no slop, exact phrasing) +- `skills/signal-detector/SKILL.md` — Always-on idea+entity capture on every message +- `skills/brain-ops/SKILL.md` — Brain-first lookup, read-enrich-write loop, source attribution +- `skills/idea-ingest/SKILL.md` — Links/articles/tweets with author people page mandatory +- `skills/media-ingest/SKILL.md` — Video/audio/PDF/book with entity extraction +- `skills/meeting-ingestion/SKILL.md` — Transcripts with attendee enrichment chaining +- `skills/citation-fixer/SKILL.md` — Citation format auditing and fixing +- `skills/repo-architecture/SKILL.md` — Filing rules by primary subject +- `skills/skill-creator/SKILL.md` — Create conforming skills with MECE check +- `skills/daily-task-manager/SKILL.md` — Task lifecycle with priority levels +- `skills/daily-task-prep/SKILL.md` — Morning prep with calendar context +- `skills/cross-modal-review/SKILL.md` — Quality gate via second model +- `skills/cron-scheduler/SKILL.md` — Schedule staggering, quiet hours, idempotency +- `skills/reports/SKILL.md` — Timestamped reports with keyword routing +- `skills/testing/SKILL.md` — Skill validation framework +- `skills/soul-audit/SKILL.md` — 6-phase interview for SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md +- `skills/webhook-transforms/SKILL.md` — External events to brain signals +- `skills/data-research/SKILL.md` — Structured data research: email-to-tracker pipeline with parameterized YAML recipes +- `skills/minion-orchestrator/SKILL.md` — Background job orchestration: submit, fan out children with depth/cap/timeouts, collect results via child_done inbox +- `templates/` — SOUL.md, USER.md, ACCESS_POLICY.md, HEARTBEAT.md templates +- `skills/migrations/` — Version migration files with feature_pitch YAML frontmatter +- `src/commands/publish.ts` — Deterministic brain page publisher (code+skill pair, zero LLM calls) +- `src/commands/backlinks.ts` — Back-link checker and fixer (enforces Iron Law) +- `src/commands/lint.ts` — Page quality linter (catches LLM artifacts, placeholder dates) +- `src/commands/report.ts` — Structured report saver (audit trail for maintenance/enrichment) +- `openclaw.plugin.json` — ClawHub bundle plugin manifest + +## Commands + +Run `gbrain --help` or `gbrain --tools-json` for full command reference. + +Key commands added in v0.7: +- `gbrain init` — defaults to PGLite (no Supabase needed), scans repo size, suggests Supabase for 1000+ files +- `gbrain migrate --to supabase` / `gbrain migrate --to pglite` — bidirectional engine migration + +Key commands added for Minions (job queue): +- `gbrain jobs submit [--params JSON] [--follow] [--dry-run]` — submit a background job +- `gbrain jobs list [--status S] [--queue Q]` — list jobs with filters +- `gbrain jobs get ` — job details with attempt history +- `gbrain jobs cancel/retry/delete ` — manage job lifecycle +- `gbrain jobs prune [--older-than 30d]` — clean old completed/dead jobs +- `gbrain jobs stats` — job health dashboard +- `gbrain jobs work [--queue Q] [--concurrency N]` — start worker daemon (Postgres only) + +Key commands added in v0.12.2: +- `gbrain repair-jsonb [--dry-run] [--json]` — repair double-encoded JSONB rows left over from v0.12.0-and-earlier Postgres writes. Idempotent; PGLite no-ops. The `v0_12_2` migration runs this automatically on `gbrain upgrade`. + +Key commands added in v0.12.3: +- `gbrain orphans [--json] [--count] [--include-pseudo]` — surface pages with zero inbound wikilinks, grouped by domain. Auto-generated/raw/pseudo pages filtered by default. Also exposed as `find_orphans` MCP operation. The natural consumer of the v0.12.0 knowledge graph layer: once edges are captured, find the gaps. +- `gbrain doctor` gains two new reliability detection checks: `jsonb_integrity` (v0.12.0 Postgres double-encode damage) and `markdown_body_completeness` (pages truncated by the old splitBody bug). Detection only; fix hints point at `gbrain repair-jsonb` and `gbrain sync --force`. + +Key commands added in v0.14.2: +- `gbrain sync --skip-failed` — acknowledge the current set of failed-parse files recorded in `~/.gbrain/sync-failures.jsonl` so the sync bookmark advances past them. Doctor's `sync_failures` check shows previously-skipped as "all acknowledged" instead of warning. +- `gbrain sync --retry-failed` — re-walk the unacknowledged failures and re-attempt parsing. If the files now succeed, they clear from the set and the bookmark advances naturally. +- `gbrain apply-migrations --force-retry ` — reset a wedged migration (3 consecutive partials with no completion) by appending a `'retry'` marker. Next `apply-migrations --yes` treats the version as fresh. `complete` status never regresses to `partial` either before or after a retry marker. +- `GBRAIN_POOL_SIZE` env var — honored by both the singleton pool (`src/core/db.ts`) and the parallel-import worker pool (`src/commands/import.ts`). Default is 10; lower to 2 for Supabase transaction pooler to avoid MaxClients crashes during `gbrain upgrade` subprocess spawns. Read at call time via `resolvePoolSize()`. +- `gbrain doctor` gains two new checks: `sync_failures` (surfaces unacknowledged parse failures with exact paths + fix hints) and `brain_score` (renders the 5-component breakdown when score < 100: embed coverage / 35, link density / 25, timeline coverage / 15, orphans / 15, dead links / 10 — sum equals total). + +## Testing + +`bun test` runs all tests. After the v0.12.1 release: ~75 unit test files + 8 E2E test files (1412 unit pass, 119 E2E when `DATABASE_URL` is set — skip gracefully otherwise). Unit tests run +without a database. E2E tests skip gracefully when `DATABASE_URL` is not set. + +Unit tests: `test/markdown.test.ts` (frontmatter parsing), `test/chunkers/recursive.test.ts` +(chunking), `test/parity.test.ts` (operations contract +parity), `test/cli.test.ts` (CLI structure), `test/config.test.ts` (config redaction), +`test/files.test.ts` (MIME/hash), `test/import-file.test.ts` (import pipeline), +`test/upgrade.test.ts` (schema migrations), +`test/file-migration.test.ts` (file migration), `test/file-resolver.test.ts` (file resolution), +`test/import-resume.test.ts` (import checkpoints), `test/migrate.test.ts` (migration; v8/v9 helper-btree-index SQL structural assertions + 1000-row wall-clock fixtures that guard the O(n²)→O(n log n) fix), +`test/setup-branching.test.ts` (setup flow), `test/slug-validation.test.ts` (slug validation), +`test/storage.test.ts` (storage backends), `test/supabase-admin.test.ts` (Supabase admin), +`test/yaml-lite.test.ts` (YAML parsing), `test/check-update.test.ts` (version check + update CLI), +`test/pglite-engine.test.ts` (PGLite engine, all 40 BrainEngine methods including 11 cases for `addLinksBatch` / `addTimelineEntriesBatch`: empty batch, missing optionals, within-batch dedup via ON CONFLICT, missing-slug rows dropped by JOIN, half-existing batch, batch of 100), +`test/engine-factory.test.ts` (engine factory + dynamic imports), +`test/integrations.test.ts` (recipe parsing, CLI routing, recipe validation), +`test/publish.test.ts` (content stripping, encryption, password generation, HTML output), +`test/backlinks.test.ts` (entity extraction, back-link detection, timeline entry generation), +`test/lint.test.ts` (LLM artifact detection, code fence stripping, frontmatter validation), +`test/report.test.ts` (report format, directory structure), +`test/skills-conformance.test.ts` (skill frontmatter + required sections validation), +`test/resolver.test.ts` (RESOLVER.md coverage, routing validation), +`test/search.test.ts` (RRF normalization, compiled truth boost, cosine similarity, dedup key), +`test/dedup.test.ts` (source-aware dedup, compiled truth guarantee, layer interactions), +`test/intent.test.ts` (query intent classification: entity/temporal/event/general), +`test/eval.test.ts` (retrieval metrics: precisionAtK, recallAtK, mrr, ndcgAtK, parseQrels), +`test/check-resolvable.test.ts` (resolver reachability, MECE overlap, gap detection, DRY checks + v0.14.1 proximity-based DRY detection + `extractDelegationTargets` coverage — 13 DRY cases), +`test/dry-fix.test.ts` (v0.14.1 auto-fix: three shape-aware expander pure-function tests, five guards — working-tree-dirty, no-git-backup, inside-code-fence, already-delegated within 40 lines, ambiguous-multi-match, block-is-callout — 28 cases), +`test/doctor-fix.test.ts` (v0.14.1 `gbrain doctor --fix` CLI integration: dry-run preview, apply path, JSON output shape — 3 cases), +`test/backoff.test.ts` (load-aware throttling, concurrency limits, active hours), +`test/fail-improve.test.ts` (deterministic/LLM cascade, JSONL logging, test generation, rotation), +`test/transcription.test.ts` (provider detection, format validation, API key errors), +`test/enrichment-service.test.ts` (entity slugification, extraction, tier escalation), +`test/data-research.test.ts` (recipe validation, MRR/ARR extraction, dedup, tracker parsing, HTML stripping), +`test/minions.test.ts` (Minions job queue v7: CRUD, state machine, backoff, stall detection, dependencies, worker lifecycle, lock management, claim mechanics, depth/child-cap, timeouts, cascade kill, idempotency, child_done inbox, attachments, removeOnComplete/Fail), +`test/extract.test.ts` (link extraction, timeline extraction, frontmatter parsing, directory type inference), +`test/extract-db.test.ts` (gbrain extract --source db: typed link inference, idempotency, --type filter, --dry-run JSON output), +`test/extract-fs.test.ts` (gbrain extract --source fs: first-run inserts + second-run reports zero, dry-run dedups candidates across files, second-run perf regression guard — the v0.12.1 N+1 dedup bug), +`test/link-extraction.test.ts` (canonical extractEntityRefs both formats, extractPageLinks dedup, inferLinkType heuristics, parseTimelineEntries date variants, isAutoLinkEnabled config), +`test/graph-query.test.ts` (direction in/out/both, type filter, indented tree output), +`test/features.test.ts` (feature scanning, brain_score calculation, CLI routing, persistence), +`test/file-upload-security.test.ts` (symlink traversal, cwd confinement, slug + filename allowlists, remote vs local trust), +`test/query-sanitization.test.ts` (prompt-injection stripping, output sanitization, structural boundary), +`test/search-limit.test.ts` (clampSearchLimit default/cap behavior across list_pages and get_ingest_log), +`test/repair-jsonb.test.ts` (v0.12.2 JSONB repair: TARGETS list, idempotency, engine-awareness), +`test/migrations-v0_12_2.test.ts` (v0.12.2 orchestrator phases: schema → repair → verify → record), +`test/markdown.test.ts` (splitBody sentinel precedence, horizontal-rule preservation, inferType wiki subtypes), +`test/orphans.test.ts` (v0.12.3 orphans command: detection, pseudo filtering, text/json/count outputs, MCP op), +`test/postgres-engine.test.ts` (v0.12.3 statement_timeout scoping: `sql.begin` + `SET LOCAL` shape, source-level grep guardrail against reintroduced bare `SET statement_timeout`), +`test/sync.test.ts` (sync logic + v0.12.3 regression guard asserting top-level `engine.transaction` is not called), +`test/doctor.test.ts` (doctor command + v0.12.3 assertions that `jsonb_integrity` scans the four v0.12.0 write sites and `markdown_body_completeness` is present), +`test/utils.test.ts` (shared SQL utilities + `tryParseEmbedding` null-return and single-warn semantics), +`test/build-llms.test.ts` (llms.txt/llms-full.txt generator: path resolution, idempotence, spec shape, regen-drift guard, content contract, AGENTS.md install-path mirror, size-budget enforcement — 7 cases). + +E2E tests (`test/e2e/`): Run against real Postgres+pgvector. Require `DATABASE_URL`. +- `bun run test:e2e` runs Tier 1 (mechanical, all operations, no API keys). Includes 9 dedicated cases for the postgres-engine `addLinksBatch` / `addTimelineEntriesBatch` bind path — postgres-js's `unnest()` binding is structurally different from PGLite's and gets its own coverage. +- `test/e2e/search-quality.test.ts` runs search quality E2E against PGLite (no API keys, in-memory) +- `test/e2e/graph-quality.test.ts` runs the v0.10.3 knowledge graph pipeline (auto-link via put_page, reconciliation, traversePaths) against PGLite in-memory +- `test/e2e/postgres-jsonb.test.ts` — v0.12.2 regression test. Round-trips all 5 JSONB write sites (pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter) against real Postgres and asserts `jsonb_typeof='object'` plus `->>'key'` returns the expected scalar. The test that should have caught the original double-encode bug. +- `test/e2e/jsonb-roundtrip.test.ts` — v0.12.3 companion regression against the 4 doctor-scanned JSONB sites. Assertion-level overlap with `postgres-jsonb.test.ts` is intentional defense-in-depth: if doctor's scan surface ever drifts from the actual write surface, one of these tests catches it. +- `test/e2e/upgrade.test.ts` runs check-update E2E against real GitHub API (network required) +- Tier 2 (`skills.test.ts`) requires OpenClaw + API keys, runs nightly in CI +- If `.env.testing` doesn't exist in this directory, check sibling worktrees for one: + `find ../ -maxdepth 2 -name .env.testing -print -quit` and copy it here if found. +- Always run E2E tests when they exist. Do not skip them just because DATABASE_URL + is not set. Start the test DB, run the tests, then tear it down. + +### API keys and running ALL tests + +ALWAYS source the user's shell profile before running tests: + +```bash +source ~/.zshrc 2>/dev/null || true +``` + +This loads `OPENAI_API_KEY` and `ANTHROPIC_API_KEY`. Without these, Tier 2 tests +skip silently. Do NOT skip Tier 2 tests just because they require API keys — load +the keys and run them. + +When asked to "run all E2E tests" or "run tests", that means ALL tiers: +- Tier 1: `bun run test:e2e` (mechanical, sync, upgrade — no API keys needed) +- Tier 2: `test/e2e/skills.test.ts` (requires OpenAI + Anthropic + openclaw CLI) +- Always spin up the test DB, source zshrc, run everything, tear down. + +### E2E test DB lifecycle (ALWAYS follow this) + +You are responsible for spinning up and tearing down the test Postgres container. +Do not leave containers running after tests. Do not skip E2E tests. + +1. **Check for `.env.testing`** — if missing, copy from sibling worktree. + Read it to get the DATABASE_URL (it has the port number). +2. **Check if the port is free:** + `docker ps --filter "publish=PORT"` — if another container is on that port, + pick a different port (try 5435, 5436, 5437) and start on that one instead. +3. **Start the test DB:** + ```bash + docker run -d --name gbrain-test-pg \ + -e POSTGRES_USER=postgres -e POSTGRES_PASSWORD=postgres \ + -e POSTGRES_DB=gbrain_test \ + -p PORT:5432 pgvector/pgvector:pg16 + ``` + Wait for ready: `docker exec gbrain-test-pg pg_isready -U postgres` +4. **Run E2E tests:** + `DATABASE_URL=postgresql://postgres:postgres@localhost:PORT/gbrain_test bun run test:e2e` +5. **Tear down immediately after tests finish (pass or fail):** + `docker stop gbrain-test-pg && docker rm gbrain-test-pg` + +Never leave `gbrain-test-pg` running. If you find a stale one from a previous run, +stop and remove it before starting a new one. + +## Skills + +Read the skill files in `skills/` before doing brain operations. GBrain ships 26 skills +organized by `skills/RESOLVER.md`: + +**Original 8 (conformance-migrated):** ingest (thin router), query, maintain, enrich, +briefing, migrate, setup, publish. + +**Brain skills (ported from an upstream agent fork):** signal-detector, brain-ops, idea-ingest, media-ingest, +meeting-ingestion, citation-fixer, repo-architecture, skill-creator, daily-task-manager. + +**Operational + identity:** daily-task-prep, cross-modal-review, cron-scheduler, reports, +testing, soul-audit, webhook-transforms, data-research, minion-orchestrator. + +**Conventions:** `skills/conventions/` has cross-cutting rules (quality, brain-first, +model-routing, test-before-bulk, cross-modal). `skills/_brain-filing-rules.md` and +`skills/_output-rules.md` are shared references. + +## Build + +`bun build --compile --outfile bin/gbrain src/cli.ts` + +## Pre-ship requirements + +Before shipping (/ship) or reviewing (/review), always run the full test suite: +- `bun test` — unit tests (no database required) +- Follow the "E2E test DB lifecycle" steps above to spin up the test DB, + run `bun run test:e2e`, then tear it down. + +Both must pass. Do not ship with failing E2E tests. Do not skip E2E tests. + +## Post-ship requirements (MANDATORY) + +After EVERY /ship, you MUST run /document-release. This is NOT optional. Do NOT +skip it. Do NOT say "docs look fine" without running it. The skill reads every .md +file in the project, cross-references the diff, and updates anything that drifted. + +If /ship's Step 8.5 triggers document-release automatically, that counts. But if +it gets skipped for ANY reason (timeout, error, oversight), you MUST run it manually +before considering the ship complete. + +Files that MUST be checked on every ship: +- README.md — does it reflect new features, commands, or setup steps? +- CLAUDE.md — does it reflect new files, test files, or architecture changes? +- CHANGELOG.md — does it cover every commit? +- TODOS.md — are completed items marked done? +- docs/ — do any guides need updating? + +A ship without updated docs is an incomplete ship. Period. + +## CHANGELOG voice + release-summary format + +Every version entry in `CHANGELOG.md` MUST start with a release-summary section in +the GStack/Garry voice — one viewport's worth of prose + tables that lands like a +verdict, not marketing. The itemized changelog (subsections, bullets, files) goes +BELOW that summary, separated by a `### Itemized changes` header. + +The release-summary section gets read by humans, by the auto-update agent, and by +anyone deciding whether to upgrade. The itemized list is for agents that need to +know exactly what changed. + +### Release-summary template + +Use this structure for the top of every `## [X.Y.Z]` entry: + +1. **Two-line bold headline** (10-14 words total) ... should land like a verdict, not + marketing. Sound like someone who shipped today and cares whether it works. +2. **Lead paragraph** (3-5 sentences) ... what shipped, what changed for the user. + Specific, concrete, no AI vocabulary, no em dashes, no hype. +3. **A "The X numbers that matter" section** with: + - One short setup paragraph naming the source of the numbers (real production + deployment OR a reproducible benchmark ... name the file/command to run). + - A table of 3-6 key metrics with BEFORE / AFTER / Δ columns. + - A second optional table for per-category breakdown if relevant. + - 1-2 sentences interpreting the most striking number in concrete user terms. +4. **A "What this means for [audience]" closing paragraph** (2-4 sentences) tying + the metrics to a real workflow shift. End with what to do. + +Voice rules: +- No em dashes (use commas, periods, "..."). +- No AI vocabulary (delve, robust, comprehensive, nuanced, fundamental, etc.) or + banned phrases ("here's the kicker", "the bottom line", etc.). +- Real numbers, real file names, real commands. Not "fast" but "~30s on 30K pages." +- Short paragraphs, mix one-sentence punches with 2-3 sentence runs. +- Connect to user outcomes: "the agent does ~3x less reading" beats "improved + precision." +- Be direct about quality. "Well-designed" or "this is a mess." No dancing. + +Source material to pull from: +- CHANGELOG.md previous entry for prior context +- `docs/benchmarks/[latest].md` for the headline numbers +- Recent commits (`git log ..HEAD --oneline`) for what shipped +- Don't make up numbers. If a metric isn't in a benchmark or production data, don't + include it. Say "no measurement yet" if asked. + +Target length: ~250-350 words for the summary. Should render as one viewport. + +### "To take advantage of v[version]" block (required, v0.13+) + +After the release-summary and BEFORE `### Itemized changes`, every `## [X.Y.Z]` +entry MUST include a human-readable self-repair block under the heading +`## To take advantage of v[version]`. + +Why: `gbrain upgrade` runs `gbrain post-upgrade` which runs `gbrain apply-migrations`. +This chain has a known weak link — `upgrade.ts` catches post-upgrade failures as +best-effort (so the binary still works). When that chain silently fails, users end +up with half-upgraded brains. The self-repair block gives them a paste-ready +recovery path; the v0.13+ `~/.gbrain/upgrade-errors.jsonl` trail + `gbrain doctor` +integration close the loop. + +Template (adapt the verify commands per release): + +```markdown +## To take advantage of v[version] + +`gbrain upgrade` should do this automatically. If it didn't, or if `gbrain doctor` +warns about a partial migration: + +1. **Run the orchestrator manually:** + ```bash + gbrain apply-migrations --yes + ``` +2. **Your agent reads `skills/migrations/v[version].md` the next time you interact with it.** + [One sentence on whether headless agents need manual action, or whether the + orchestrator already handled the mechanical side.] +3. **Verify the outcome:** + ```bash + [release-specific verify commands, e.g. `gbrain graph ... --depth 2`] + gbrain stats + ``` +4. **If any step fails or the numbers look wrong,** please file an issue: + https://github.com/garrytan/gbrain/issues with: + - output of `gbrain doctor` + - contents of `~/.gbrain/upgrade-errors.jsonl` if it exists + - which step broke + + This feedback loop is how the gbrain maintainers find fragile upgrade paths. Thank you. +``` + +**Skip this block** for patches that are pure bug fixes with zero user-facing action +(rare). If the release has a schema migration, data backfill, or new feature the +user needs to verify, the block is required. + +The v0.13.0 entry in CHANGELOG.md is the canonical example. + +### Itemized changes (the existing rules) + +Below the release summary, write `### Itemized changes` and continue with the +detailed subsections (Knowledge Graph Layer, Schema migrations, Security hardening, +Tests, etc.). Same rules as before: + +- Lead with what the user can now DO that they couldn't before +- Frame as benefits and capabilities, not files changed or code written +- Make the user think "hell yeah, I want that" +- Bad: "Added GBRAIN_VERIFY.md installation verification runbook" +- Good: "Your agent now verifies the entire GBrain installation end-to-end, catching + silent sync failures and stale embeddings before they bite you" +- Bad: "Setup skill Phase H and Phase I added" +- Good: "New installs automatically set up live sync so your brain never falls behind" +- **Always credit community contributions.** When a CHANGELOG entry includes work from + a community PR, name the contributor with `Contributed by @username`. Contributors + did real work. Thank them publicly every time, no exceptions. + +### Reference: v0.12.0 entry as canonical example + +The v0.12.0 entry in CHANGELOG.md is the canonical example of the format. Match its +structure for every future version: bold headline, lead paragraph, "numbers that +matter" with BrainBench-style before/after table, "what this means" closer, then +`### Itemized changes` with the detailed sections below. + +## Version migrations + +Create a migration file at `skills/migrations/v[version].md` when a release +includes changes that existing users need to act on. The auto-update agent +reads these files post-upgrade (Section 17, Step 4) and executes them. + +**You need a migration file when:** +- New setup step that existing installs don't have (e.g., v0.5.0 added live sync, + existing users need to set it up, not just new installs) +- New SKILLPACK section with a MUST ADD setup requirement +- Schema changes that require `gbrain init` or manual SQL +- Changed defaults that affect existing behavior +- Deprecated commands or flags that need replacement +- New verification steps that should run on existing installs +- New cron jobs or background processes that should be registered + +**You do NOT need a migration file when:** +- Bug fixes with no behavior changes +- Documentation-only improvements (the agent re-reads docs automatically) +- New optional features that don't affect existing setups +- Performance improvements that are transparent + +**The key test:** if an existing user upgrades and does nothing else, will their +brain work worse than before? If yes, migration file. If no, skip it. + +Write migration files as agent instructions, not technical notes. Tell the agent +what to do, step by step, with exact commands. See `skills/migrations/v0.5.0.md` +for the pattern. + +## Migration is canonical, not advisory + +GBrain's job is to deliver a canonical, working setup to every user on upgrade. +Anything that looks like a "host-repo change" — AGENTS.md, cron manifests, +launchctl units, config files outside `~/.gbrain/` — is a GBrain migration +step, not a nudge we leave for the host-repo maintainer. Migrations edit host +files (with backups) to make the canonical setup real. Exceptions: changes +that require human judgment (content edits, renames that break semantics, +host-specific handler registration where shell-exec would be an RCE surface). +Everything mechanical ships in the migration. + +**Test:** if shipping a feature requires a sentence that starts with "in +your AGENTS.md, add…" or "in your cron/jobs.json, rewrite…", the migration +orchestrator should be doing that edit, not the user. + +**The exception is host-specific code.** For custom Minion handlers +(host-specific integrations like inbox sweeps or third-party API scanners), shipping them as a +data file the worker would exec is an RCE surface. Those get registered in +the host's own repo via the plugin contract (`docs/guides/plugin-handlers.md`); +the migration orchestrator emits a structured TODO to +`~/.gbrain/migrations/pending-host-work.jsonl` + the host agent walks the +TODOs using `skills/migrations/v0.11.0.md` — stays host-agnostic, still +canonical. + +## Privacy rule: scrub real names from public docs + +**Never reference real people, companies, funds, or private agent names in any +public-facing artifact.** Public artifacts include: `CHANGELOG.md`, `README.md`, +`docs/`, `skills/`, PR titles + bodies, commit messages, and comments in checked-in +code. Query examples, benchmark stories, and migration guides MUST use generic +placeholders. + +Why: gbrain runs a personal knowledge brain containing notes on real people and +real companies (YC founders, portfolio companies, funds, investors, meeting +attendees). When a doc copies a query like `gbrain graph diana-hu --depth 2` or +names a specific agent fork like `Wintermute`, that real name gets indexed by +search engines, surfaced in cross-references, and distributed with every release. + +**Name mapping** to use in examples: +- Agent forks → `your agent fork`, `a downstream agent`, or `agent-fork` +- Example person → `alice-example`, `charlie-example`, or `a-founder` +- Example company → `acme-example`, `widget-co`, or `a-company` +- Example fund → `fund-a`, `fund-b`, `fund-c` +- Example deal → `acme-seed`, `widget-series-a` +- Example meeting → `meetings/2026-04-03` (generic date is fine) +- Example user → `you` or `the user`, never a proper name + +**When in doubt, ask yourself:** "Would this query reveal private information +about the user's contacts, investments, or portfolio if it were read by a +stranger?" If yes, replace with generic placeholders. + +**Illustrative API examples with household-brand companies** (Stripe, Brex, OpenAI, +GitHub, etc.) are fine — they're public entities, not contacts in anyone's brain. +Do not confuse illustrative API examples with queries that reveal real +relationships. + +## Schema state tracking + +`~/.gbrain/update-state.json` tracks which recommended schema directories the user +adopted, declined, or added custom. The auto-update agent (SKILLPACK Section 17) +reads this during upgrades to suggest new schema additions without re-suggesting +things the user already declined. The setup skill writes the initial state during +Phase C/E. Never modify a user's custom directories or re-suggest declined ones. + +## GitHub Actions SHA maintenance + +All GitHub Actions in `.github/workflows/` are pinned to commit SHAs. Before shipping +(`/ship`) or reviewing (`/review`), check for stale pins and update them: + +```bash +for action in actions/checkout oven-sh/setup-bun actions/upload-artifact actions/download-artifact softprops/action-gh-release gitleaks/gitleaks-action; do + tag=$(grep -r "$action@" .github/workflows/ | head -1 | grep -o '#.*' | tr -d '# ') + [ -n "$tag" ] && echo "$action@$tag: $(gh api repos/$action/git/ref/tags/$tag --jq .object.sha 2>/dev/null)" +done +``` + +If any SHA differs from what's in the workflow files, update the pin and version comment. + +## PR descriptions cover the whole branch + +Pull request titles and bodies must describe **everything in the PR diff against the +base branch**, not just the most recent commit you made. When you open or update a +PR, walk the full commit range with `git log --oneline ..` and write the +body to cover all of it. Group by feature area (schema, code, tests, docs) — not +chronologically by commit. + +This matters because reviewers read the PR body to understand what's shipping. If +the body only covers your last commit, they miss everything else and can't review +properly. A 7-commit PR with a body that describes commit 7 is worse than no body +at all — it actively misleads. + +When in doubt, run `gh pr view --json commits --jq '[.commits[].messageHeadline]'` +to see what's actually in the PR before writing the body. + +## Community PR wave process + +Never merge external PRs directly into master. Instead, use the "fix wave" workflow: + +1. **Categorize** — group PRs by theme (bug fixes, features, infra, docs) +2. **Deduplicate** — if two PRs fix the same thing, pick the one that changes fewer + lines. Close the other with a note pointing to the winner. +3. **Collector branch** — create a feature branch (e.g. `garrytan/fix-wave-N`), cherry-pick + or manually re-implement the best fixes from each PR. Do NOT merge PR branches directly — + read the diff, understand the fix, and write it yourself if needed. +4. **Test the wave** — verify with `bun test && bun run test:e2e` (full E2E lifecycle). + Every fix in the wave must have test coverage. +5. **Close with context** — every closed PR gets a comment explaining why and what (if + anything) supersedes it. Contributors did real work; respect that with clear communication + and thank them. +6. **Ship as one PR** — single PR to master with all attributions preserved via + `Co-Authored-By:` trailers. Include a summary of what merged and what closed. + +**Community PR guardrails:** +- Always AskUserQuestion before accepting commits that touch voice, tone, or + promotional material (README intro, CHANGELOG voice, skill templates). +- Never auto-merge PRs that remove YC references or "neutralize" the founder perspective. +- Preserve contributor attribution in commit messages. + +## Skill routing + +When the user's request matches an available skill, ALWAYS invoke it using the Skill +tool as your FIRST action. Do NOT answer directly, do NOT use other tools first. +The skill has specialized workflows that produce better results than ad-hoc answers. + +**NEVER hand-roll ship operations.** Do not manually run git commit + push + gh pr +create when /ship is available. /ship handles VERSION bump, CHANGELOG, document-release, +pre-landing review, test coverage audit, and adversarial review. Manually creating a PR +skips all of these. If the user says "commit and ship", "push and ship", "bisect and +ship", or any combination that ends with shipping — invoke /ship and let it handle +everything including the commits. If the branch name contains a version (e.g. +`v0.5-live-sync`), /ship should use that version for the bump. + +Key routing rules: +- Product ideas, "is this worth building", brainstorming → invoke office-hours +- Bugs, errors, "why is this broken", 500 errors → invoke investigate +- Ship, deploy, push, create PR, "commit and ship", "push and ship" → invoke ship +- QA, test the site, find bugs → invoke qa +- Code review, check my diff → invoke review +- Update docs after shipping → invoke document-release +- Weekly retro → invoke retro +- Design system, brand → invoke design-consultation +- Visual audit, design polish → invoke design-review +- Architecture review → invoke plan-eng-review +- Save progress, checkpoint, resume → invoke checkpoint +- Code quality, health check → invoke health + +--- + +## INSTALL_FOR_AGENTS.md + +Source: https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md + +# GBrain Installation Guide for AI Agents + +Read this entire file, then follow the steps. Ask the user for API keys when needed. +Target: ~30 minutes to a fully working brain. + +## Step 0: If you are not Claude Code + +Read `AGENTS.md` at the repo root first. It's the non-Claude-agent operating +protocol (install, read order, trust boundary, common tasks). Claude Code reads +`CLAUDE.md` automatically and can skip ahead. + +If you fetched this file by URL without cloning yet, the companion files live at: +- `https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` — start here +- `https://raw.githubusercontent.com/garrytan/gbrain/master/llms.txt` — full doc map +- `https://raw.githubusercontent.com/garrytan/gbrain/master/llms-full.txt` — same map, inlined + +## Step 1: Install GBrain + +```bash +git clone https://github.com/garrytan/gbrain.git ~/gbrain && cd ~/gbrain +curl -fsSL https://bun.sh/install | bash +export PATH="$HOME/.bun/bin:$PATH" +bun install && bun link +``` + +Verify: `gbrain --version` should print a version number. If `gbrain` is not found, +restart the shell or add the PATH export to the shell profile. + +## Step 2: API Keys + +Ask the user for these: + +```bash +export OPENAI_API_KEY=sk-... # required for vector search +export ANTHROPIC_API_KEY=sk-ant-... # optional, improves search quality +``` + +Save to shell profile or `.env`. Without OpenAI, keyword search still works. +Without Anthropic, search works but skips query expansion. + +## Step 3: Create the Brain + +```bash +gbrain init # PGLite, no server needed +gbrain doctor --json # verify all checks pass +``` + +The user's markdown files (notes, docs, brain repo) are SEPARATE from this tool repo. +Ask the user where their files are, or create a new brain repo: + +```bash +mkdir -p ~/brain && cd ~/brain && git init +``` + +Read `~/gbrain/docs/GBRAIN_RECOMMENDED_SCHEMA.md` and set up the MECE directory +structure (people/, companies/, concepts/, etc.) inside the user's brain repo, +NOT inside ~/gbrain. + +## Step 4: Import and Index + +```bash +gbrain import ~/brain/ --no-embed # import markdown files +gbrain embed --stale # generate vector embeddings +gbrain query "key themes across these documents?" +``` + +## Step 4.5: Wire the Knowledge Graph + +If the user already had a brain repo (Step 3 imported existing markdown), backfill +the typed-link graph and structured timeline. This populates the `links` and +`timeline_entries` tables that future writes will maintain automatically. + +```bash +gbrain extract links --source db --dry-run | head -20 # preview +gbrain extract links --source db # commit +gbrain extract timeline --source db # dated events +gbrain stats # verify links > 0 +``` + +For brand-new empty brains, skip this step — auto-link populates the graph as the +agent writes pages going forward. There is nothing to backfill yet. + +After this step: +- `gbrain graph-query --depth 2` works (relationship traversal) +- Search ranks well-connected entities higher (backlink boost) +- Every future `put_page` auto-creates typed links and reconciles stale ones + +If a user has a very large brain (>10K pages), `extract --source db` is idempotent +and supports `--since YYYY-MM-DD` for incremental runs. + +## Step 5: Load Skills + +Read `~/gbrain/skills/RESOLVER.md`. This is the skill dispatcher. It tells you which +skill to read for any task. Save this to your memory permanently. + +The three most important skills to adopt immediately: + +1. **Signal detector** (`skills/signal-detector/SKILL.md`) — fire this on EVERY + inbound message. It captures ideas and entities in parallel. The brain compounds. + +2. **Brain-ops** (`skills/brain-ops/SKILL.md`) — brain-first lookup on every response. + Check the brain before any external API call. + +3. **Conventions** (`skills/conventions/quality.md`) — citation format, back-linking + iron law, source attribution. These are non-negotiable quality rules. + +## Step 6: Identity (optional) + +Run the soul-audit skill to customize the agent's identity: + +``` +Read skills/soul-audit/SKILL.md and follow it. +``` + +This generates SOUL.md (agent identity), USER.md (user profile), ACCESS_POLICY.md +(who sees what), and HEARTBEAT.md (operational cadence) from the user's answers. + +If skipped, minimal defaults are installed automatically. + +## Step 7: Recurring Jobs + +Set up using your platform's scheduler (OpenClaw cron, Railway cron, crontab): + +- **Live sync** (every 15 min): `gbrain sync --repo ~/brain && gbrain embed --stale` +- **Auto-update** (daily): `gbrain check-update --json` (tell user, never auto-install) +- **Dream cycle** (nightly): read `docs/guides/cron-schedule.md` for the full protocol. + Entity sweep, citation fixes, memory consolidation. This is what makes the brain + compound. Do not skip it. +- **Weekly**: `gbrain doctor --json && gbrain embed --stale` + +## Step 8: Integrations + +Run `gbrain integrations list`. Each recipe in `~/gbrain/recipes/` is a self-contained +installer. It tells you what credentials to ask for, how to validate, and what cron +to register. Ask the user which integrations they want (email, calendar, voice, Twitter). + +Verify: `gbrain integrations doctor` (after at least one is configured) + +## Step 9: Verify + +Read `docs/GBRAIN_VERIFY.md` and run all 7 verification checks. Check #4 (live sync +actually works) is the most important. + +## Upgrade + +```bash +cd ~/gbrain && git pull origin master && bun install +gbrain init # apply schema migrations (idempotent) +gbrain post-upgrade # show migration notes for the version range +``` + +Then read `~/gbrain/skills/migrations/v.md` (and any intermediate +versions you skipped) and run any backfill or verification steps it lists. Skipping +this is how features ship in the binary but stay dormant in the user's brain. + +For v0.12.0+ specifically: if your brain was created before v0.12.0, run +`gbrain extract links --source db && gbrain extract timeline --source db` to +backfill the new graph layer (see Step 4.5 above). + +For v0.12.2+ specifically: if your brain is Postgres- or Supabase-backed and +predates v0.12.2, the `v0_12_2` migration runs `gbrain repair-jsonb` +automatically during `gbrain post-upgrade` to fix the double-encoded JSONB +columns. PGLite brains no-op. If wiki-style imports were truncated by the old +`splitBody` bug, run `gbrain sync --full` after upgrading to rebuild +`compiled_truth` from source markdown. + +--- + +## skills/RESOLVER.md + +Source: https://raw.githubusercontent.com/garrytan/gbrain/master/skills/RESOLVER.md + +# GBrain Skill Resolver + +This is the dispatcher. Skills are the implementation. **Read the skill file before acting.** If two skills could match, read both. They are designed to chain (e.g., ingest then enrich for each entity). + +## Always-on (every message) + +| Trigger | Skill | +|---------|-------| +| Every inbound message (spawn parallel, don't block) | `skills/signal-detector/SKILL.md` | +| Any brain read/write/lookup/citation | `skills/brain-ops/SKILL.md` | + +## Brain operations + +| Trigger | Skill | +|---------|-------| +| "What do we know about", "tell me about", "search for" | `skills/query/SKILL.md` | +| "Who knows who", "relationship between", "connections", "graph query" | `skills/query/SKILL.md` (use graph-query) | +| Creating/enriching a person or company page | `skills/enrich/SKILL.md` | +| Where does a new file go? Filing rules | `skills/repo-architecture/SKILL.md` | +| Fix broken citations in brain pages | `skills/citation-fixer/SKILL.md` | +| "Research", "track", "extract from email", "investor updates", "donations" | `skills/data-research/SKILL.md` | +| Share a brain page as a link | `skills/publish/SKILL.md` | + +## Content & media ingestion + +| Trigger | Skill | +|---------|-------| +| User shares a link, article, tweet, or idea | `skills/idea-ingest/SKILL.md` | +| Video, audio, PDF, book, YouTube, screenshot | `skills/media-ingest/SKILL.md` | +| Meeting transcript received | `skills/meeting-ingestion/SKILL.md` | +| Generic "ingest this" (auto-routes to above) | `skills/ingest/SKILL.md` | + +## Thinking skills (from GStack) + +| Trigger | Skill | +|---------|-------| +| "Brainstorm", "I have an idea", "office hours" | GStack: office-hours | +| "Review this plan", "CEO review", "poke holes" | GStack: ceo-review | +| "Debug", "fix", "broken", "investigate" | GStack: investigate | +| "Retro", "what shipped", "retrospective" | GStack: retro | + +> These skills come from GStack. If GStack is installed, the agent reads them directly. +> If not, brain-only mode still works (brain skills function without thinking skills). + +## Operational + +| Trigger | Skill | +|---------|-------| +| Task add/remove/complete/defer/review | `skills/daily-task-manager/SKILL.md` | +| Morning prep, meeting context, day planning | `skills/daily-task-prep/SKILL.md` | +| Daily briefing, "what's happening today" | `skills/briefing/SKILL.md` | +| Cron scheduling, quiet hours, job staggering | `skills/cron-scheduler/SKILL.md` | +| Save or load reports | `skills/reports/SKILL.md` | +| "Create a skill", "improve this skill" | `skills/skill-creator/SKILL.md` | +| "Skillify this", "is this a skill?", "make this proper" | `skills/skillify/SKILL.md` | +| "Is gbrain healthy?", morning health check, skillpack-check | `skills/skillpack-check/SKILL.md` | +| Cross-modal review, second opinion | `skills/cross-modal-review/SKILL.md` | +| "Validate skills", skill health check | `skills/testing/SKILL.md` | +| Webhook setup, external event processing | `skills/webhook-transforms/SKILL.md` | +| "Spawn agent", "background task", "parallel tasks", "steer agent", "pause/resume agent" | `skills/minion-orchestrator/SKILL.md` | + +## Setup & migration + +| Trigger | Skill | +|---------|-------| +| "Set up GBrain", first boot | `skills/setup/SKILL.md` | +| "Migrate from Obsidian/Notion/Logseq" | `skills/migrate/SKILL.md` | +| Brain health check, maintenance run | `skills/maintain/SKILL.md` | +| "Extract links", "build link graph", "populate timeline" | `skills/maintain/SKILL.md` (extraction sections) | +| "Brain health", "what features am I missing", "brain score" | Run `gbrain features --json` | +| "Set up autopilot", "run brain maintenance", "keep brain updated" | Run `gbrain autopilot --install --repo ~/brain` | +| Agent identity, "who am I", customize agent | `skills/soul-audit/SKILL.md` | +| "Populate links", "extract links", "backfill graph" | `skills/maintain/SKILL.md` (graph population phase) | +| "Populate timeline", "extract timeline entries" | `skills/maintain/SKILL.md` (graph population phase) | + +## Identity & access (always-on) + +| Trigger | Skill | +|---------|-------| +| Non-owner sends a message | Check `ACCESS_POLICY.md` before responding | +| Agent needs to know its identity/vibe | Read `SOUL.md` | +| Agent needs user context | Read `USER.md` | +| Operational cadence (what to check and when) | Read `HEARTBEAT.md` | + +## Disambiguation rules + +When multiple skills could match: +1. Prefer the most specific skill (meeting-ingestion over ingest) +2. If the user mentions a URL, route by content type (link → idea-ingest, video → media-ingest) +3. If the user mentions a person/company, check if enrich or query fits better +4. Chaining is explicit in each skill's Phases section +5. When in doubt, ask the user + +## Conventions (cross-cutting) + +These apply to ALL brain-writing skills: +- `skills/conventions/quality.md` — citations, back-links, notability gate +- `skills/conventions/brain-first.md` — check brain before external APIs +- `skills/conventions/subagent-routing.md` — when to use Minions vs inline work +- `skills/_brain-filing-rules.md` — where files go +- `skills/_output-rules.md` — output quality standards + +--- + +## README.md + +Source: https://raw.githubusercontent.com/garrytan/gbrain/master/README.md + +# GBrain + +Your AI agent is smart but forgetful. GBrain gives it a brain. + +Built by the President and CEO of Y Combinator to run his actual AI agents. The production brain powering his OpenClaw and Hermes deployments: **17,888 pages, 4,383 people, 723 companies**, 21 cron jobs running autonomously, built in 12 days. The agent ingests meetings, emails, tweets, voice calls, and original ideas while you sleep. It enriches every person and company it encounters. It fixes its own citations and consolidates memory overnight. You wake up and the brain is smarter than when you went to bed. + +The brain wires itself. Every page write extracts entity references and creates typed links (`attended`, `works_at`, `invested_in`, `founded`, `advises`) with zero LLM calls. Hybrid search. Self-wiring knowledge graph. Structured timeline. Backlink-boosted ranking. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked end-to-end: **Recall@5 jumps from 83% to 95%, Precision@5 from 39% to 45%, +30 more correct answers in the agent's top-5 reads** on a 240-page Opus-generated rich-prose corpus. Graph-only F1: **86.6% vs grep's 57.8%** (+28.8 pts). [Full report](docs/benchmarks/2026-04-18-brainbench-v1.md). + +GBrain is those patterns, generalized. 26 skills. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours. + +> **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys. + +> **LLMs:** fetch [`llms.txt`](llms.txt) for the documentation map, or [`llms-full.txt`](llms-full.txt) for the same map with core docs inlined in one fetch. **Agents:** start with [`AGENTS.md`](AGENTS.md) (or [`CLAUDE.md`](CLAUDE.md) if you're Claude Code). + +## Install + +### On an agent platform (recommended) + +GBrain is designed to be installed and operated by an AI agent. If you don't have one running yet: + +- **[OpenClaw](https://openclaw.ai)** ... Deploy [AlphaClaw on Render](https://render.com/deploy?repo=https://github.com/chrysb/alphaclaw) (one click, 8GB+ RAM) +- **[Hermes Agent](https://github.com/NousResearch/hermes-agent)** ... Deploy on [Railway](https://github.com/praveen-ks-2001/hermes-agent-template) (one click) + +Paste this into your agent: + +``` +Retrieve and follow the instructions at: +https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md +``` + +That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 26 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes. + +If your agent doesn't auto-read `AGENTS.md`, point it at that file first: +`https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` is the non-Claude +agent operating protocol (install, read order, trust boundary, common tasks). For +the full doc map, use `llms.txt` at the same URL root. + +### Standalone CLI (no agent) + +```bash +git clone https://github.com/garrytan/gbrain.git && cd gbrain && bun install && bun link +gbrain init # local brain, ready in 2 seconds +gbrain import ~/notes/ # index your markdown +gbrain query "what themes show up across my notes?" +``` + +``` +3 results (hybrid search, 0.12s): + +1. concepts/do-things-that-dont-scale (score: 0.94) + PG's argument that unscalable effort teaches you what users want. + [Source: paulgraham.com, 2013-07-01] + +2. originals/founder-mode-observation (score: 0.87) + Deep involvement isn't micromanagement if it expands the team's thinking. + +3. concepts/build-something-people-want (score: 0.81) + The YC motto. Connected to 12 other brain pages. +``` + +### MCP server (Claude Code, Cursor, Windsurf) + +GBrain exposes 30+ MCP tools via stdio: + +```json +{ + "mcpServers": { + "gbrain": { "command": "gbrain", "args": ["serve"] } + } +} +``` + +Add to `~/.claude/server.json` (Claude Code), Settings > MCP Servers (Cursor), or your client's MCP config. + +### Remote MCP (Claude Desktop, Cowork, Perplexity) + +```bash +ngrok http 8787 --url your-brain.ngrok.app +bun run src/commands/auth.ts create "claude-desktop" +claude mcp add gbrain -t http https://your-brain.ngrok.app/mcp -H "Authorization: Bearer TOKEN" +``` + +Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). ChatGPT requires OAuth 2.1 (not yet implemented). + +## The 26 Skills + +GBrain ships 26 skills organized by `skills/RESOLVER.md`. The resolver tells your agent which skill to read for any task. + +[Skill files are code.](https://x.com/garrytan/status/2042925773300908103) They're the most powerful way to get knowledge work done. A skill file is a fat markdown document that encodes an entire workflow: when to fire, what to check, how to chain with other skills, what quality bar to enforce. The agent reads the skill and executes it. Skills can also call deterministic TypeScript code bundled in GBrain (search, import, embed, sync) for the parts that shouldn't be left to LLM judgment. [Thin harness, fat skills](docs/ethos/THIN_HARNESS_FAT_SKILLS.md): the intelligence lives in the skills, not the runtime. + +### Always-on + +| Skill | What it does | +|-------|-------------| +| **signal-detector** | Fires on every message. Spawns a cheap model in parallel to capture original thinking and entity mentions. The brain compounds on autopilot. | +| **brain-ops** | Brain-first lookup before any external API. The read-enrich-write loop that makes every response smarter. | + +### Content ingestion + +| Skill | What it does | +|-------|-------------| +| **ingest** | Thin router. Detects input type and delegates to the right ingestion skill. | +| **idea-ingest** | Links, articles, tweets become brain pages with analysis, author people pages, and cross-linking. | +| **media-ingest** | Video, audio, PDF, books, screenshots, GitHub repos. Transcripts, entity extraction, backlink propagation. | +| **meeting-ingestion** | Transcripts become brain pages. Every attendee gets enriched. Every company gets a timeline entry. | + +### Brain operations + +| Skill | What it does | +|-------|-------------| +| **enrich** | Tiered enrichment (Tier 1/2/3). Creates and updates person/company pages with compiled truth and timelines. | +| **query** | 3-layer search with synthesis and citations. Says "the brain doesn't have info on X" instead of hallucinating. | +| **maintain** | Periodic health: stale pages, orphans, dead links, citation audit, back-link enforcement, tag consistency. | +| **citation-fixer** | Scans pages for missing or malformed citations. Fixes format to match the standard. | +| **repo-architecture** | Where new brain files go. Decision protocol: primary subject determines directory, not format. | +| **publish** | Share brain pages as password-protected HTML. Zero LLM calls. | +| **data-research** | Structured data research with parameterized YAML recipes. Extract investor updates, expenses, company metrics from email. | + +### Operational + +| Skill | What it does | +|-------|-------------| +| **daily-task-manager** | Task lifecycle with priority levels (P0-P3). Stored as searchable brain pages. | +| **daily-task-prep** | Morning prep: calendar lookahead with brain context per attendee, open threads, task review. | +| **cron-scheduler** | Schedule staggering (5-min offsets), quiet hours (timezone-aware with wake-up override), idempotency. | +| **reports** | Timestamped reports with keyword routing. "What's the latest briefing?" finds it instantly. | +| **cross-modal-review** | Quality gate via second model. Refusal routing: if one model refuses, silently switch. | +| **webhook-transforms** | External events (SMS, meetings, social mentions) converted into brain pages with entity extraction. | +| **testing** | Validates every skill has SKILL.md with frontmatter, manifest coverage, resolver coverage. | +| **skill-creator** | Create new skills following the conformance standard. MECE check against existing skills. | +| **minion-orchestrator** | Long-running agent work as background jobs. Submit, fan out children with depth/cap/timeouts, collect results via child_done inbox. | + +### Identity and setup + +| Skill | What it does | +|-------|-------------| +| **soul-audit** | 6-phase interview generating SOUL.md (agent identity), USER.md (user profile), ACCESS_POLICY.md (4-tier privacy), HEARTBEAT.md (operational cadence). | +| **setup** | Auto-provision PGLite or Supabase. First import. GStack detection. | +| **migrate** | Universal migration from Obsidian, Notion, Logseq, markdown, CSV, JSON, Roam. | +| **briefing** | Daily briefing with meeting context, active deals, and citation tracking. | + +### Conventions + +Cross-cutting rules in `skills/conventions/`: +- **quality.md** ... citations, back-links, notability gate, source attribution +- **brain-first.md** ... 5-step lookup before any external API call +- **model-routing.md** ... which model for which task +- **test-before-bulk.md** ... test 3-5 items before any batch operation +- **cross-modal.yaml** ... review pairs and refusal routing chain + +## How It Works + +``` +Signal arrives (meeting, email, tweet, link) + -> Signal detector captures ideas + entities (parallel, never blocks) + -> Brain-ops: check the brain first (gbrain search, gbrain get) + -> Respond with full context + -> Write: update brain pages with new information + citations + -> Auto-link: typed relationships extracted on every write (zero LLM calls) + -> Sync: gbrain indexes changes for next query +``` + +Every cycle adds knowledge. The agent enriches a person page after a meeting. Next time that person comes up, the agent already has context. The difference compounds daily. + +The system gets smarter on its own. Entity enrichment auto-escalates: a person mentioned once gets a stub page (Tier 3). After 3 mentions across different sources, they get web + social enrichment (Tier 2). After a meeting or 8+ mentions, full pipeline (Tier 1). The brain learns who matters without being told. Deterministic classifiers improve over time via a fail-improve loop that logs every LLM fallback and generates better regex patterns from the failures. `gbrain doctor` shows the trajectory: "intent classifier: 87% deterministic, up from 40% in week 1." + +> "Prep me for my meeting with Jordan in 30 minutes" +> ... pulls dossier, shared history, recent activity, open threads + +> "What have I said about the relationship between shame and founder performance?" +> ... searches YOUR thinking, not the internet + +## Minions: your sub-agents won't drop work anymore + +A durable, Postgres-native job queue built into the brain. Every long-running agent task is now a job that survives gateway restarts, streams progress, gets paused / resumed / steered mid-flight, and shows up in `gbrain jobs list`. Zero infra beyond your existing brain. + +### The production numbers that matter + +Here's my personal OpenClaw deployment: one Render container. Supabase Postgres holding a 45,000-page brain. 19 cron jobs firing on schedule. Real gateway load from real daily work. The task: pull a month of my social posts from an external API and ingest them end-to-end into the brain as a structured page. + +| | Minions | `sessions_spawn` | +|--- |--- |--- | +| Wall time | **753ms** | **>10,000ms** (gateway timeout) | +| Token cost | **$0.00** | ~$0.03 per run | +| Success rate | **100%** | **0%** (couldn't even spawn) | +| Memory/job | ~2 MB | ~80 MB | + +Under that 19-cron load, sub-agent spawn couldn't clear the 10-second gateway wall. Minions landed it in under a second for zero tokens. **Scaling:** 19,240 posts across 36 months, single bash loop, ~15 min total, $0.00. Sub-agents: ~9 min best case, ~$1.08 in tokens, ~40% spawn failure. **Lab:** durability ∞ (SIGKILL mid-flight, 10/10 rescued), throughput ~10× faster, fan-out ~21× with no failure wall, memory ~400× less. + +Full benchmarks: [production](docs/benchmarks/2026-04-18-minions-vs-openclaw-production.md) and [lab](docs/benchmarks/2026-04-18-minions-vs-openclaw-subagents.md). + +### The routing rule + +> **Deterministic** (same input → same steps → same output) → **Minions** +> **Judgment** (input requires assessment or decision) → **Sub-agents** + +Pull posts, parse JSON, write a brain page, run a sync — deterministic. $0 tokens, survives restart, millisecond runtime. Triage the inbox, assess meeting priority, decide if a cold email deserves a reply — judgment. What sub-agents are actually good at. `minion_mode: pain_triggered` (the default) automates the routing. + +### What's fixed + +The six daily pains — spawn storms, agents that stop responding, forgotten dispatches, gateway crashes mid-run, runaway grandchildren, debugging soup — all belonged to the "deterministic work through a reasoning model" mistake. Minions fixes them by not making that mistake: `max_children` cap, `timeout_ms` + AbortSignal, `child_done` inbox, full `parent_job_id`/`depth`/transcript per job, Postgres durability with stall detection, cascade cancel via recursive CTE. Plus idempotency keys, attachment validation, `removeOnComplete`, and `gbrain jobs smoke` that proves the install in half a second. + +```bash +gbrain jobs smoke # verify install +gbrain jobs submit sync --params '{}' # fire a background job +gbrain jobs stats # health dashboard +gbrain jobs work --concurrency 4 # start a worker (Postgres only) +``` + +Read [`skills/minion-orchestrator/SKILL.md`](skills/minion-orchestrator/SKILL.md) for parent-child DAGs, fan-in collection, steering via inbox. + +**Minions is not incrementally better than sub-agents for background work. It's categorically different.** 753ms vs gateway timeout. $0 vs tokens. 100% vs couldn't-spawn. If your agent does deterministic work on a schedule, it runs on Minions now. + +### Health check and self-heal + +Minions is canonical as of v0.11.1 — every `gbrain upgrade` runs the migration automatically (schema → smoke → prefs → host rewrites → env-aware autopilot install). If you ever want to verify manually or wire a cron into your morning briefing: + +```bash +gbrain doctor # half-migrated state? prints loud banner + exits non-zero +gbrain skillpack-check --quiet # exit 0/1/2 for pipeline gating +gbrain skillpack-check | jq # full JSON: {healthy, summary, actions[], doctor, migrations} +``` + +If anything's off, `actions[]` tells you the exact command to run. For deeper troubleshooting: [`docs/guides/minions-fix.md`](docs/guides/minions-fix.md). + +Moving gateway crons to Minions (deterministic scripts, zero LLM tokens per fire): [`docs/guides/minions-shell-jobs.md`](docs/guides/minions-shell-jobs.md). + +## Skillify: your skills tree stops being a black box + +Hermes and similar agent frameworks auto-create skills as a background behavior. Fine until you don't know what the agent shipped. Checklists decay. Tests drift. Resolver entries get stale. Six months later you've got an opaque pile of "skills" that nobody has read, nobody has tested, and nobody is sure still work. + +GBrain ships the same capability. Except the human stays in the loop. + +- **`/skillify`** turns raw code into a properly-skilled feature: SKILL.md + deterministic script + unit tests + integration tests + LLM evals + resolver trigger + resolver trigger eval + E2E smoke + brain filing. Ten items. Every one required. +- **`gbrain check-resolvable`** walks the whole skills tree: reachability, MECE overlap, DRY violations, gap detection, orphaned skills. Exits non-zero if anything is off. +- **`scripts/skillify-check.ts`** — machine-readable audit. `--json` for CI, `--recent` for last-7-days files. + +You decide when and what. The tooling keeps the checklist honest. + +### Why this is the right answer for OpenClaw + +Auto-generated skills are a liability the first time a behavior breaks. Was it the skill? The test? The resolver trigger? The eval? You don't know, because you never read it. Debugging a black box is pure guesswork. + +Skillify makes the black box legible. Every skill in your tree has: a contract (SKILL.md), tests that exercise that contract, an eval that grades LLM output against a rubric, a resolver trigger the user actually types, and a test that confirms the trigger routes right. If something breaks, you know which layer to look at. If anything goes stale, `check-resolvable` says so. + +In practice this combo produces **zero orphaned skills, every feature with tests + evals + resolver triggers + evals of the triggers.** Compounding quality instead of compounding entropy. + +```bash +# Audit a feature's skill completeness (10-item checklist) +bun run scripts/skillify-check.ts src/commands/publish.ts + +# In CI: fail the build when a new feature isn't properly skilled +bun run scripts/skillify-check.ts --json --recent + +# Validate the whole skills tree before shipping +gbrain check-resolvable +``` + +**Skillify is not a nice-to-have. It's the piece that makes the skills tree survive six months of compounding work.** Read [`skills/skillify/SKILL.md`](skills/skillify/SKILL.md) for the full 10-item checklist and the anti-patterns it catches. + +## Getting Data In + +GBrain ships integration recipes that your agent sets up for you. Each recipe tells the agent what credentials to ask for, how to validate, and what cron to register. + +| Recipe | Requires | What It Does | +|--------|----------|-------------| +| [Public Tunnel](recipes/ngrok-tunnel.md) | — | Fixed URL for MCP + voice (ngrok Hobby $8/mo) | +| [Credential Gateway](recipes/credential-gateway.md) | — | Gmail + Calendar access | +| [Voice-to-Brain](recipes/twilio-voice-brain.md) | ngrok-tunnel | Phone calls to brain pages (Twilio + OpenAI Realtime) | +| [Email-to-Brain](recipes/email-to-brain.md) | credential-gateway | Gmail to entity pages | +| [X-to-Brain](recipes/x-to-brain.md) | — | Twitter timeline + mentions + deletions | +| [Calendar-to-Brain](recipes/calendar-to-brain.md) | credential-gateway | Google Calendar to searchable daily pages | +| [Meeting Sync](recipes/meeting-sync.md) | — | Circleback transcripts to brain pages with attendees | + +**Data research recipes** extract structured data from email into tracked brain pages. Built-in recipes for investor updates (MRR, ARR, runway, headcount), expense tracking, and company metrics. Create your own with `gbrain research init`. + +Run `gbrain integrations` to see status. + +## GBrain + GStack + +[GStack](https://github.com/garrytan/gstack) is the engine. GBrain is the mod. + +- **[GStack](https://github.com/garrytan/gstack)** = coding skills (ship, review, QA, investigate, office-hours, retro). 70,000+ stars, 30,000 developers per day. When your agent codes on itself, it uses GStack. +- **GBrain** = everything-else skills (brain ops, signal detection, ingestion, enrichment, cron, reports, identity). When your agent remembers, thinks, and operates, it uses GBrain. +- **`hosts/gbrain.ts`** = the bridge. Tells GStack's coding skills to check the brain before coding. + +`gbrain init` detects if GStack is installed and reports mod status. If GStack isn't there, it tells you how to get it. + +## Architecture + +``` +┌──────────────────┐ ┌───────────────┐ ┌──────────────────┐ +│ Brain Repo │ │ GBrain │ │ AI Agent │ +│ (git) │ │ (retrieval) │ │ (read/write) │ +│ │ │ │ │ │ +│ markdown files │───>│ Postgres + │<──>│ 26 skills │ +│ = source of │ │ pgvector │ │ define HOW to │ +│ truth │ │ │ │ use the brain │ +│ │<───│ hybrid │ │ │ +│ human can │ │ search │ │ RESOLVER.md │ +│ always read │ │ (vector + │ │ routes intent │ +│ & edit │ │ keyword + │ │ to skill │ +│ │ │ RRF) │ │ │ +└──────────────────┘ └───────────────┘ └──────────────────┘ +``` + +The repo is the system of record. GBrain is the retrieval layer. The agent reads and writes through both. Human always wins... edit any markdown file and `gbrain sync` picks up the changes. + +## The Knowledge Model + +Every page follows the compiled truth + timeline pattern: + +```markdown +--- +type: concept +title: Do Things That Don't Scale +tags: [startups, growth, pg-essay] +--- + +Paul Graham's argument that startups should do unscalable things early on. +The key insight: the unscalable effort teaches you what users actually +want, which you can't learn any other way. + +--- + +- 2013-07-01: Published on paulgraham.com +- 2024-11-15: Referenced in batch W25 kickoff talk +``` + +Above the `---`: **compiled truth**. Your current best understanding. Gets rewritten when new evidence changes the picture. Below: **timeline**. Append-only evidence trail. Never edited, only added to. + +## Knowledge Graph + +Pages aren't just text. Every mention of a person, company, or concept becomes a typed link in a structured graph. The brain wires itself. + +``` +Write a meeting page mentioning Alice and Acme AI + -> Auto-link extracts entity refs from content (zero LLM calls) + -> Infers types: meeting page + person ref => `attended` + "CEO of X" pattern => `works_at` + "invested in" => `invested_in` + "advises", "advisor" => `advises` + "founded", "co-founded" => `founded` + -> Reconciles stale links: edits remove links no longer in content + -> Backlinks rank well-connected entities higher in search +``` + +```bash +gbrain graph-query people/alice --type attended --depth 2 +# returns who Alice met with, transitively +``` + +The graph powers questions vector search can't: "who works at Acme AI?", "what has Bob invested in?", "find the connection between Alice and Carol". Backfill an existing brain in one command: + +```bash +gbrain extract links --source db # wire up the existing 29K pages +gbrain extract timeline --source db # extract dated events from markdown timelines +``` + +Then ask graph questions or watch the search ranking improve. Benchmarked: **Recall@5 jumps from 83% to 95%, Precision@5 from 39% to 45%, +30 more correct answers in the agent's top-5 reads** on a 240-page Opus-generated rich-prose corpus. Graph-only F1 hits 86.6% vs grep's 57.8% (+28.8 pts). See [docs/benchmarks/2026-04-18-brainbench-v1.md](docs/benchmarks/2026-04-18-brainbench-v1.md). + +## Search + +Hybrid search: vector + keyword + RRF fusion + multi-query expansion + 4-layer dedup. + +``` +Query + -> Intent classifier (entity? temporal? event? general?) + -> Multi-query expansion (Claude Haiku) + -> Vector search (HNSW cosine) + Keyword search (tsvector) + -> RRF fusion: score = sum(1/(60 + rank)) + -> Cosine re-scoring + compiled truth boost + -> 4-layer dedup + compiled truth guarantee + -> Results +``` + +Keyword alone misses conceptual matches. Vector alone misses exact phrases. RRF gets both. Search quality is benchmarked and reproducible: `gbrain eval --qrels queries.json` measures P@k, Recall@k, MRR, and nDCG@k. A/B test config changes before deploying them. + +## Why it works: many strategies in concert + +The brain isn't one trick. Every retrieval question goes through ~20 deterministic +techniques layered together. No single one is magic; the win comes from stacking +them so each layer covers what the others miss. + +``` +Question + │ + ├─ INGESTION (every put_page) + │ ├─ Recursive markdown chunking (or semantic / LLM-guided) + │ ├─ Embedding cache invalidation on edit + │ └─ Idempotent imports (content-hash dedup) + │ + ├─ GRAPH EXTRACTION (auto-link post-hook, zero LLM) + │ ├─ Entity-ref regex (markdown links + bare slugs) + │ ├─ Code-fence stripping (no false-positive slugs in code blocks) + │ ├─ Typed inference cascade (FOUNDED → INVESTED → ADVISES → WORKS_AT) + │ ├─ Page-role priors (partner-bio language → invested_in) + │ ├─ Within-page dedup (same target collapses to one link) + │ ├─ Stale-link reconciliation (edits remove dropped refs) + │ └─ Multi-type link constraint (same person can works_at AND advises) + │ + ├─ SEARCH PIPELINE (every query) + │ ├─ Intent classifier (entity / temporal / event / general — auto-routes) + │ ├─ Multi-query expansion (Haiku rephrases the question 3 ways) + │ ├─ Vector search (HNSW cosine over OpenAI embeddings) + │ ├─ Keyword search (Postgres tsvector + websearch_to_tsquery) + │ ├─ Reciprocal Rank Fusion (score = sum 1/(60+rank) across both) + │ ├─ Cosine re-scoring (re-rank chunks against actual query embedding) + │ ├─ Compiled-truth boost (assessments outrank timeline noise) + │ ├─ Backlink boost (well-connected entities rank higher) + │ └─ Source-aware dedup (one CT chunk per page guaranteed) + │ + ├─ GRAPH TRAVERSAL (relational queries) + │ ├─ Recursive CTE with cycle prevention (visited-array check) + │ ├─ Type-filtered edges (--type works_at, attended, etc.) + │ ├─ Direction control (in / out / both) + │ └─ Depth-capped (≤10 for remote MCP; DoS prevention) + │ + └─ AGENT WORKFLOW (graph-confident hybrid) + ├─ Graph-query first (high-precision typed answers) + ├─ Grep fallback when graph returns nothing + └─ Graph hits ranked first in top-K (better P@K and R@K) +``` + +End-to-end on the BrainBench v1 corpus (240 rich-prose pages, before/after PR #188): + +| Metric | BEFORE PR #188 | AFTER PR #188 | Δ | +|-------------------------|----------------|---------------|-------------| +| **Precision@5** | 39.2% | **44.7%** | **+5.4 pts**| +| **Recall@5** | 83.1% | **94.6%** | **+11.5 pts**| +| Correct in top-5 | 217 | 247 | **+30** | +| Graph-only F1 (ablation)| 57.8% (grep) | **86.6%** | **+28.8 pts**| + +Plus 5 orthogonal capability checks (identity resolution, temporal queries, +performance at 10K-page scale, robustness to malformed input, MCP operation +contract). All pass. [Full report.](docs/benchmarks/2026-04-18-brainbench-v1.md) + +The point: each technique handles a class of inputs the others miss. Vector +search misses exact slug refs; keyword catches them. Keyword misses conceptual +matches; vector catches them. RRF picks the best of both. Compiled-truth boost +keeps assessments above timeline noise. Auto-link extraction wires the graph +that lets backlink boost rank well-connected entities higher. Graph traversal +answers questions search alone can't reach. The agent picks graph-first for +precision and falls back to keyword for recall. **All deterministic, all in +concert, all measured.** + +## Voice + +Call a phone number. Your AI answers. It knows who's calling, pulls their full context from the brain, and responds like someone who actually knows your world. When the call ends, a brain page appears with the transcript, entity detection, and cross-references. + +

+ Voice client connected +

+ +> [See it in action](https://x.com/garrytan/status/2043022208512172263) + +The voice recipe ships with GBrain: [Voice-to-Brain](recipes/twilio-voice-brain.md). WebRTC works in a browser tab with zero setup. A real phone number is optional. + +## Engine Architecture + +``` +CLI / MCP Server + (thin wrappers, identical operations) + | + BrainEngine interface (pluggable) + | + +--------+--------+ + | | +PGLiteEngine PostgresEngine + (default) (Supabase) + | | +~/.gbrain/ Supabase Pro ($25/mo) +brain.pglite Postgres + pgvector +embedded PG 17.5 + + gbrain migrate --to supabase|pglite + (bidirectional migration) +``` + +PGLite: embedded Postgres, no server, zero config. When your brain outgrows local (1000+ files, multi-device), `gbrain migrate --to supabase` moves everything. + +## File Storage + +Brain repos accumulate binaries. GBrain has a three-stage migration: + +```bash +gbrain files mirror # copy to cloud, local untouched +gbrain files redirect # replace local with .redirect pointers +gbrain files clean # remove pointers, cloud only +gbrain files restore # download everything back (undo) +``` + +Storage backends: S3-compatible (AWS, R2, MinIO), Supabase Storage, or local. + +## Commands + +``` +SETUP + gbrain init [--supabase|--url] Create brain (PGLite default) + gbrain migrate --to supabase|pglite Bidirectional engine migration + gbrain upgrade Self-update with feature discovery + +PAGES + gbrain get Read a page (fuzzy slug matching) + gbrain put [< file.md] Write/update (auto-versions) + gbrain delete Delete a page + gbrain list [--type T] [--tag T] List with filters + +SEARCH + gbrain search Keyword search (tsvector) + gbrain query Hybrid search (vector + keyword + RRF) + +IMPORT + gbrain import [--no-embed] Import markdown (idempotent) + gbrain sync [--repo ] Git-to-brain incremental sync + gbrain export [--dir ./out/] Export to markdown + +FILES + gbrain files list|upload|sync|verify File storage operations + +EMBEDDINGS + gbrain embed [|--all|--stale] Generate/refresh embeddings + +LINKS + GRAPH + gbrain link|unlink|backlinks Cross-reference management + gbrain extract links|timeline|all Batch backfill from existing pages + (--source db|fs, --type, --since, --dry-run) + gbrain graph-query Typed traversal (--type T --depth N + --direction in|out|both) + +JOBS (Minions) + gbrain jobs submit [--params JSON] [--follow] Submit a background job + gbrain jobs list [--status S] [--queue Q] List jobs with filters + gbrain jobs get|cancel|retry|delete Manage job lifecycle + gbrain jobs prune [--older-than 30d] Clean completed/dead jobs + gbrain jobs stats Job health dashboard + gbrain jobs smoke One-command health check + gbrain jobs work [--queue Q] [--concurrency N] Start worker daemon + +ADMIN + gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings) + gbrain doctor --fix [--dry-run] Auto-fix DRY violations (delegate inlined rules to conventions) + gbrain stats Brain statistics + gbrain serve MCP server (stdio) + gbrain integrations Integration recipe dashboard + gbrain check-backlinks check|fix Back-link enforcement + gbrain lint [--fix] LLM artifact detection + gbrain repair-jsonb [--dry-run] Repair v0.12.0 double-encoded JSONB (Postgres) + gbrain orphans [--json] [--count] Find pages with zero inbound wikilinks + gbrain transcribe