diff --git a/CHANGELOG.md b/CHANGELOG.md index 0361e8133..8a6c64193 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,69 @@ All notable changes to GBrain will be documented in this file. +## [0.11.1] - 2026-04-18 + +### Fixed — the v0.11.0 migration mega-bug + +Your v0.11.0 upgrade shipped the Minions schema, worker, queue, and migration skill. It didn't ship the actual migration running on upgrade. If you upgraded and ended up with no `~/.gbrain/preferences.json`, autopilot still running inline, and cron jobs still hitting `agentTurn`'s 300s timeout — that's the bug. This release fixes it and auto-repairs on your next `gbrain upgrade`. + +- **`gbrain apply-migrations` is the canonical repair.** Reads `~/.gbrain/migrations/completed.jsonl`, diffs against the TS migration registry, runs any pending orchestrators. Idempotent: rerunning on a healthy install is cheap and silent. +- **`gbrain upgrade` and `postinstall` now invoke it.** `runPostUpgrade` tail-calls `apply-migrations --yes` unconditionally (Codex caught that the earlier early-return on missing upgrade-state.json left broken-v0.11.0 installs broken forever). `package.json`'s new `postinstall` hook runs it after `bun update gbrain` / `npm i gbrain`. First-install guard keeps postinstall silent when no brain is configured yet. +- **Stopgap for v0.11.0 binaries without this release:** paste `curl -fsSL https://raw.githubusercontent.com/garrytan/gbrain/v0.11.1/scripts/fix-v0.11.0.sh | bash`. It writes `preferences.json` + a `status: "partial"` record so the eventual `apply-migrations --yes` run picks up where it left off — the stopgap does not poison the permanent migration path. + +### Added — autopilot supervises Minions itself, one install step + +Before this release, autopilot + `gbrain jobs work` were two separate processes you had to manage. Now autopilot is the one install step, and it forks the Minions worker as a child with 10s-backoff restart + 5-crash cap + async SIGTERM drain that waits up to 35s for the worker to commit in-flight work before SIGKILL. + +- **Autopilot dispatches each cycle as a single `autopilot-cycle` Minion job** with `idempotency_key: autopilot-cycle:`. A 5-min autopilot + 8-min embed no longer stacks 4 overlapping runs — the queue's unique partial index dedupes at the DB layer. Codex caught that the earlier "parent/child DAG" plan was a category error (parent/child in Minions flips the parent to `waiting-children`, not the child to `waiting-for-parent`, so extract would have run before sync). +- **Per-step partial-failure handling.** Each of sync / extract / embed / backlinks is wrapped in its own try/catch. Handler returns `{ partial: true, failed_steps: [...] }` when any step fails; never throws. An intermittent extract bug no longer blocks every future cycle via Minion retry. +- **Env-aware `gbrain autopilot --install`** picks the right supervisor: launchd on macOS, systemd user unit on Linux-with-systemd (with a stricter `systemctl --user is-system-running` probe — the naive `/run/systemd/system` check was a false-positive magnet), bootstrap hook on ephemeral containers (Render / Railway / Fly / Docker — auto-injects into OpenClaw's `hooks/bootstrap/ensure-services.sh` when detected, use `--no-inject` to opt out), crontab otherwise. `--target` overrides detection. Uninstall mirrors all four targets. +- **Worker child spawn uses `resolveGbrainCliPath()`** — never blindly uses `process.execPath` (on source installs that's the Bun runtime, not `gbrain`). Resolution tries argv[1], then execPath ending `/gbrain`, then `which gbrain`. + +### Added — library-level Core fns so handlers don't kill workers + +Reusing CLI entry-point functions (`runExtract`, `runEmbed`, etc.) as Minion handler bodies was wrong — any `process.exit(1)` on bad args would kill the entire worker process and every in-flight job. New Core fns throw instead: + +- `runExtractCore(engine, opts)` — wraps extract-links + extract-timeline. +- `runEmbedCore(engine, opts)` — accepts `{ slug, slugs, all, stale }`. +- `runBacklinksCore(opts)` — `{ action: 'check' | 'fix', dir, dryRun }`. +- `runLintCore(opts)` — returns counts, doesn't print human detail (CLI wrapper does that). + +CLI wrappers (`runExtract`, `runEmbed`, etc.) stay as thin arg-parsers that catch + `process.exit(1)`. Handlers in `jobs.ts` import the Core fns directly. + +### Added — skillify ships as a first-class gbrain skill + +Ported from Wintermute, proven in production. Paired with `gbrain check-resolvable` gives a user-controllable equivalent of Hermes' auto-skill-creation — you decide when and what, the tooling keeps the 10-item checklist honest. + +- `skills/skillify/SKILL.md` — the meta skill. Triggers: "skillify this", "is this a skill?", "make this proper". +- `scripts/skillify-check.ts` — machine-readable audit. `--json` for CI, `--recent` to check files modified in the last 7 days. +- README now has a short section explaining the Skillify + check-resolvable pair and why user-controlled beats auto-generated. + +### Added — host-agnostic plugin contract (replaces handlers.json) + +An earlier design draft shipped `~/.claude/gbrain-handlers.json` where each entry was a shell command the worker would exec. Codex flagged this as a durable RCE surface. Dropped in favor of a code-level plugin contract: + +- `docs/guides/plugin-handlers.md` — the full contract. Host imports `gbrain/minions`, constructs a `MinionWorker`, calls `worker.register(name, fn)` for every custom handler, calls `worker.start()`. Ships the bootstrap as code in the host repo, same trust model as any other code. +- `skills/conventions/cron-via-minions.md` — the rewrite convention for cron manifests. PGLite branch keeps `--follow` (inline); Postgres branch drops `--follow` + uses `--idempotency-key` on the cycle slot. +- `skills/migrations/v0.11.0.md` — body restored as the host-agent instruction manual. Walks the host through every JSONL TODO using the 10-item skillify checklist. + +### Added — `gbrain init --migrate-only` (the Codex H1 fix) + +Running bare `gbrain init` with no flags defaulted to PGLite and called `saveConfig` — silently clobbering any existing Postgres config. The migration orchestrator now calls `gbrain init --migrate-only` which only applies the schema against the configured engine and NEVER writes a new config. Apply-migrations + stopgap + postinstall all use this flag. Bare `gbrain init` still exists and still defaults to PGLite when you want a fresh install. + +### Changed + +- `runPostUpgrade` is now async + runs `apply-migrations --yes` unconditionally (Codex H8). +- `gbrain upgrade`'s subprocess timeout for `post-upgrade` bumped 30s → 300s so the migration has room to do real work like autopilot install (Codex H7). +- Migration enumeration uses a TS registry at `src/commands/migrations/index.ts` instead of walking `skills/migrations/*.md` on disk — compiled binaries see the same set source installs do (Codex K). +- Migration diff rule: apply when no `status: "complete"` entry exists in `completed.jsonl` AND `version ≤ installed VERSION`. Earlier proposed "version > currentVersion" would have SKIPPED v0.11.0 when running v0.11.1 (Codex H9). +- Autopilot refreshes its lock-file mtime every cycle so a long-lived autopilot doesn't get declared "stale" by the next cron-fired invocation after 10 minutes (Codex C). +- CLAUDE.md gained a new "Migration is canonical, not advisory" section pinning the design principle. + +### Tests + +34 new unit tests across preferences, init-migrate-only, apply-migrations, v0.11.0 orchestrator, handlers, autopilot-resolve-cli, autopilot-install, skillify-check. All 1177 existing tests still green. + ## [0.11.0] - 2026-04-18 ### Added — Minions (agent orchestration primitives) diff --git a/CLAUDE.md b/CLAUDE.md index c623a27f9..fdd0493ca 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -316,6 +316,30 @@ Write migration files as agent instructions, not technical notes. Tell the agent what to do, step by step, with exact commands. See `skills/migrations/v0.5.0.md` for the pattern. +## Migration is canonical, not advisory + +GBrain's job is to deliver a canonical, working setup to every user on upgrade. +Anything that looks like a "host-repo change" — AGENTS.md, cron manifests, +launchctl units, config files outside `~/.gbrain/` — is a GBrain migration +step, not a nudge we leave for the host-repo maintainer. Migrations edit host +files (with backups) to make the canonical setup real. Exceptions: changes +that require human judgment (content edits, renames that break semantics, +host-specific handler registration where shell-exec would be an RCE surface). +Everything mechanical ships in the migration. + +**Test:** if shipping a feature requires a sentence that starts with "in +your AGENTS.md, add…" or "in your cron/jobs.json, rewrite…", the migration +orchestrator should be doing that edit, not the user. + +**The exception is host-specific code.** For custom Minion handlers +(`ea-inbox-sweep`, `frameio-scan`, etc. on Wintermute), shipping them as a +data file the worker would exec is an RCE surface. Those get registered in +the host's own repo via the plugin contract (`docs/guides/plugin-handlers.md`); +the migration orchestrator emits a structured TODO to +`~/.gbrain/migrations/pending-host-work.jsonl` + the host agent walks the +TODOs using `skills/migrations/v0.11.0.md` — stays host-agnostic, still +canonical. + ## Schema state tracking `~/.gbrain/update-state.json` tracks which recommended schema directories the user diff --git a/README.md b/README.md index 933df60b5..0ba4be566 100644 --- a/README.md +++ b/README.md @@ -214,6 +214,37 @@ gbrain jobs work --concurrency 4 # start a worker daemon (Postgres only) Read `skills/minion-orchestrator/SKILL.md` for the full orchestration patterns (parent-child DAGs, fan-in collection, steering via inbox). +### v0.11.0 migration didn't fire on your upgrade? + +If you upgraded to v0.11.0 and Minions is partially set up (no `~/.gbrain/preferences.json`, autopilot still inline, cron jobs still on `agentTurn`), you hit the mega-bug where `runPostUpgrade` printed the feature pitch but never executed the migration. Two repair paths: + +- **v0.11.1 binary installed**: `gbrain upgrade && gbrain apply-migrations --yes`. +- **Stuck on v0.11.0 binary**: paste this stopgap, then upgrade + apply-migrations: + ```bash + curl -fsSL https://raw.githubusercontent.com/garrytan/gbrain/v0.11.1/scripts/fix-v0.11.0.sh | bash + ``` + +Full troubleshooting: [`docs/guides/minions-fix.md`](docs/guides/minions-fix.md). + +## Skillify + check-resolvable: user-controllable auto-skill-creation + +Hermes and similar agent frameworks auto-create skills in the background. That's fine until you don't know what the agent shipped — checklists decay, tests drift, resolver entries get stale. GBrain ships the same capability as two user-controlled tools: + +- **`/skillify`** builds the full skill stack from raw code (SKILL.md + tests + LLM evals + resolver trigger + trigger eval + E2E + brain filing). +- **`gbrain check-resolvable`** validates the whole skill tree: reachability, MECE overlap, DRY, gap detection, orphaned skills. + +You decide when and what. The human keeps judgment; the tooling keeps the checklist honest. Paired on Wintermute, this combo produces zero orphaned skills + every feature with tests + evals + resolver triggers + evals of the triggers. + +```bash +# Audit a feature's skill completeness +bun run scripts/skillify-check.ts src/commands/publish.ts + +# Machine-readable output +bun run scripts/skillify-check.ts --json --recent +``` + +Read `skills/skillify/SKILL.md` for the 10-item checklist. + ## Getting Data In GBrain ships integration recipes that your agent sets up for you. Each recipe tells the agent what credentials to ask for, how to validate, and what cron to register. diff --git a/package.json b/package.json index 137916014..f48ed625e 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "gbrain", - "version": "0.11.0", + "version": "0.11.1", "description": "Postgres-native personal knowledge brain with hybrid RAG search", "type": "module", "main": "src/core/index.ts", diff --git a/scripts/fix-v0.11.0.sh b/scripts/fix-v0.11.0.sh new file mode 100755 index 000000000..ad3a2edac --- /dev/null +++ b/scripts/fix-v0.11.0.sh @@ -0,0 +1,171 @@ +#!/usr/bin/env bash +# fix-v0.11.0.sh — stopgap for broken v0.11.0 installs where the Minions +# migration never fired on upgrade. +# +# Usage: +# curl -fsSL https://raw.githubusercontent.com/garrytan/gbrain/v0.11.1/scripts/fix-v0.11.0.sh | bash +# +# What it does: +# 1. gbrain init --migrate-only — applies schema v7 without touching config. +# 2. gbrain jobs smoke — fails loudly if Minions isn't healthy. +# 3. Prompts for minion_mode (or defaults to pain_triggered on non-TTY). +# 4. Atomically writes ~/.gbrain/preferences.json (0o600). +# 5. Appends ~/.gbrain/migrations/completed.jsonl with status:"partial" and +# apply_migrations_pending: true — the v0.11.1 `apply-migrations` runner +# will pick up where we left off (host rewrites, autopilot install). +# 6. Detects host AGENTS.md / cron/jobs.json and PRINTS the rewrite guidance +# as text. Never auto-edits host files from a curl-piped script — too +# high blast-radius (user trust model is "I pasted this"). +# 7. Final line: tells the user to run `gbrain autopilot --install` as the +# one-stop finisher (autopilot forks the Minions worker as a child). +# +# Retires when v0.11.1 is out: the canonical fix becomes +# gbrain upgrade && gbrain apply-migrations + +set -euo pipefail + +RED=$'\033[1;31m' +GREEN=$'\033[1;32m' +YELLOW=$'\033[1;33m' +NC=$'\033[0m' + +say() { printf "%s%s%s\n" "$1" "$2" "$NC"; } +info() { say "" "$1"; } +ok() { say "$GREEN" "$1"; } +warn() { say "$YELLOW" "$1"; } +die() { say "$RED" "$1"; exit 1; } + +command -v gbrain >/dev/null 2>&1 || die "gbrain not found on \$PATH. Install it first (\`bun add -g gbrain\` or download a binary)." + +GBRAIN_DIR="${HOME}/.gbrain" +PREFS_PATH="${GBRAIN_DIR}/preferences.json" +COMPLETED_PATH="${GBRAIN_DIR}/migrations/completed.jsonl" + +mkdir -p "${GBRAIN_DIR}/migrations" + +# ------------------------------------------------------------ +# Step 1: schema +# ------------------------------------------------------------ +info "[1/8] Applying schema (gbrain init --migrate-only)..." +if ! gbrain init --migrate-only; then + die "Schema migration failed. Check ~/.gbrain/config.json has a valid database_url (or database_path for PGLite), then re-run." +fi +ok " schema ok" + +# ------------------------------------------------------------ +# Step 2: smoke +# ------------------------------------------------------------ +info "[2/8] Running Minions smoke test (gbrain jobs smoke)..." +if ! gbrain jobs smoke; then + die "Smoke test failed. See the error above. Fix before continuing." +fi +ok " smoke ok" + +# ------------------------------------------------------------ +# Step 3: mode prompt +# ------------------------------------------------------------ +info "[3/8] Choose minion_mode..." +MODE="pain_triggered" +if [ -t 0 ] && [ -t 1 ]; then + echo "" + echo " [1] always — route every background task through Minions (most durable)" + echo " [2] pain_triggered — default to native subagents, switch to Minions on pain signals (recommended)" + echo " [3] off — disable Minions; keep native subagents" + echo "" + read -r -p " Choice [2]: " CHOICE + case "${CHOICE:-2}" in + 1) MODE="always" ;; + 3) MODE="off" ;; + *) MODE="pain_triggered" ;; + esac +else + warn " non-interactive shell → defaulting to pain_triggered (change later: \`gbrain config set minion_mode \`)" +fi +ok " mode=${MODE}" + +# ------------------------------------------------------------ +# Step 4: atomic write preferences.json (0o600) +# ------------------------------------------------------------ +info "[4/8] Writing ~/.gbrain/preferences.json..." +NOW_ISO=$(date -u +"%Y-%m-%dT%H:%M:%SZ") +TMP_PREFS=$(mktemp) +cat > "${TMP_PREFS}" <> "${COMPLETED_PATH}" +ok " appended ${COMPLETED_PATH}" + +# ------------------------------------------------------------ +# Step 6: detect AGENTS.md — PRINT guidance, do not auto-edit +# ------------------------------------------------------------ +info "[6/8] Scanning for AGENTS.md..." +AGENTS_FOUND=() +for CANDIDATE in "${HOME}/.claude/AGENTS.md" "${HOME}/.openclaw/AGENTS.md" "${PWD}/AGENTS.md"; do + [ -f "${CANDIDATE}" ] && AGENTS_FOUND+=("${CANDIDATE}") +done +if [ ${#AGENTS_FOUND[@]} -eq 0 ]; then + ok " no AGENTS.md found — nothing to suggest" +else + for F in "${AGENTS_FOUND[@]}"; do + warn " AGENTS.md detected: ${F}" + echo " - Next steps (this script does NOT auto-edit):" + echo " 1. Add a pointer to skills/conventions/subagent-routing.md" + echo " 2. The v0.11.1 binary's \`gbrain apply-migrations --yes\` will inject" + echo " this automatically once v0.11.1 is installed." + done +fi + +# ------------------------------------------------------------ +# Step 7: detect cron/jobs.json and scan for agentTurn +# ------------------------------------------------------------ +info "[7/8] Scanning for cron manifests..." +CRON_FOUND=() +for CANDIDATE in "${HOME}/.claude/cron/jobs.json" "${HOME}/.openclaw/cron/jobs.json" "${PWD}/cron/jobs.json"; do + [ -f "${CANDIDATE}" ] && CRON_FOUND+=("${CANDIDATE}") +done +if [ ${#CRON_FOUND[@]} -eq 0 ]; then + ok " no cron/jobs.json found — nothing to suggest" +else + for F in "${CRON_FOUND[@]}"; do + warn " cron manifest detected: ${F}" + COUNT=$(grep -c 'agentTurn' "${F}" 2>/dev/null || echo 0) + echo " - ${COUNT} agentTurn entries" + echo " - v0.11.1 apply-migrations will:" + echo " * auto-rewrite builtin handlers (sync/embed/lint/import/" + echo " extract/backlinks/autopilot-cycle) to gbrain jobs submit" + echo " * emit a pending-host-work.jsonl TODO for every non-builtin" + echo " handler; host agent walks those per skills/migrations/v0.11.0.md" + done +fi + +# ------------------------------------------------------------ +# Step 8: final line +# ------------------------------------------------------------ +info "[8/8] Done. Next step:" +echo "" +echo " ${GREEN}gbrain autopilot --install${NC}" +echo "" +echo " That ONE command does the rest: supervises autopilot, forks the" +echo " Minions worker, and installs the right entry for your host (launchd" +echo " on macOS, systemd on Linux, bootstrap hook on ephemeral containers)." +echo "" +echo " Once v0.11.1 is out:" +echo " ${GREEN}gbrain upgrade && gbrain apply-migrations${NC}" +echo " becomes the canonical fix. This script retires then."