Files
gbrain/scripts/check-skill-brain-first.sh
T
772253ef44 v0.37.3.0 feat: skill_brain_first doctor check + auto-fix + declarative opt-out (supersedes #1206) (#1215)
* v0.37.1.0 feat: skill_brain_first doctor check + auto-fix + declarative opt-out

Cathedral wave superseding PR #1206. Doctor now scans every SKILL.md for
external-lookup tools (web_search / web_fetch / exa / perplexity / happenstance
/ crustdata / captain_api / firecrawl) and warns when the skill has no brain-
first compliance signal. gbrain doctor --fix auto-inserts the canonical
> **Convention:** see [conventions/brain-first.md](...) callout via the
dry-fix.ts MISSING_RULE_PATTERNS extension (sharing safety gates with the
existing REPLACE patterns).

Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged
Garry's Palantir tweet as risky because no model knew he built it, but the
brain already had "designed the entire Finance product UI" and "150+ PSDs
from April-December 2006." Static check catches authorship; v0.37+ runtime
gate (filed in TODOS.md) closes the dispatch side.

Key design decisions locked via /plan-eng-review + codex outside-voice review:
- A1: frontmatter ships only brain_first: exempt (no required/n/a enum)
- A2: snapshot+diff audit at ~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl
  with transition-only writes (stable brains = 0 lines/run)
- A3: scaffold template pre-inserts callout; skillify check fails (exit 1)
  on external + no callout + no exempt
- A4: position-relative gate is BODY-ONLY (frontmatter tools: [web_search]
  declaration doesn't false-flag the skill)
- Q1: single pure analyzeSkillBrainFirst() helper consumed by 3 surfaces
- CMT1: no upgrade migration — doctor surfaces hint, --fix applies via
  dry-fix safety gates (user stays in loop)
- CMT2: dropped tools+writes_pages auto-exemption (was hiding mixed-class
  skills like idea-ingest/meeting-ingestion/data-research)

Trio: VERSION + package.json + CHANGELOG aligned at 0.37.1.0. 56 unit cases
+ 12 E2E cases pass. 170 related existing tests pass unchanged. Self-dogfood:
gbrain doctor against this repo's skills/ reports skill_brain_first: ok
across 43 skills (compliant or exempt). functional-area-resolver and
strategic-reading skills gained brain_first: exempt to validate the
declarative opt-out in production code (both name perplexity in dispatcher
prose without calling it).

Co-Authored-By: garrytan-agents <noreply@github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update CLAUDE.md for v0.37.1.0 skill_brain_first wave

Added Key Files entries for the four new modules:
- src/core/skill-frontmatter.ts (shared parser)
- src/core/skill-brain-first.ts (analyzer + FORMERLY_HARDCODED_EXEMPT)
- src/core/skill-fix-gates.ts (extracted safety primitives)
- src/core/audit-skill-brain-first.ts (snapshot+diff JSONL)

Extended existing entries:
- src/core/filing-audit.ts: rewired to shared parser
- src/core/dry-fix.ts: MISSING_RULE_PATTERNS INSERT pattern type
- src/commands/doctor.ts: skill_brain_first check + tweet-shield framing
- src/commands/skillify-check.ts: required item 12 + scaffold pre-insert

Added test inventory entries:
- test/skill-brain-first.test.ts (56 unit cases)
- test/e2e/skill-brain-first.test.ts (12 E2E cases)

Regenerated llms-full.txt via bun run build:llms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(ci): skill_brain_first guard uses doctor --fast to skip engine connect

CI run #76881161092 failed because scripts/check-skill-brain-first.sh
invoked plain `gbrain doctor --json`, which routes through connectEngine().
With no ~/.gbrain/config.json present (CI's case — runner is bun-only,
no brain init), connectEngine() exits 1 with "No brain configured." and
emits zero stdout. The python parser sees an empty file and returns
parse_error, failing the verify gate.

Fix: pass --fast to doctor. --fast routes through runDoctor(null, ...)
which runs the filesystem-only check set (resolver_health,
skill_conformance, skill_brain_first) and emits the standard
single-line JSON envelope the parser expects. skill_brain_first is
filesystem-only by design (scans SKILL.md, no DB touch), so --fast is
the correct knob, not a workaround.

Verified by reproducing the CI failure mode locally with
GBRAIN_HOME=/tmp/empty-... — gate now passes both with and without
a configured brain.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: rebump v0.37.1.0 → v0.37.3.0 (queue collision with #1214)

PR #1214 (brainstorm + lsd) claimed v0.37.1.0 concurrently with #1215.
Skipping 0.37.2.0 leaves a buffer for #1214's adjacent slot. Trio
(VERSION + package.json + CHANGELOG header + inline "To take advantage
of v0.37.3.0" block) aligned at 0.37.3.0.

No behavior changes — version metadata only.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: garrytan-agents <noreply@github.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-20 13:55:12 -07:00

101 lines
3.6 KiB
Bash
Executable File

#!/usr/bin/env bash
# CI guard for the v0.36.x skill_brain_first doctor check.
#
# Runs `gbrain doctor --json` against this repo's own skills/ and parses
# the JSON to assert `checks[name=skill_brain_first].status !== "warn"`.
# Doctor's exit code only flags `fail`, not `warn`, so explicit JSON-
# parsing is required to gate `bun run verify` on this warning-class check
# (F15 from /plan-eng-review).
#
# When this fires, the brain-first compliance check found new offenders
# in this repo's skills. Either:
# - add `brain_first: exempt` to the flagged skill's frontmatter (if it
# legitimately doesn't need brain-first), or
# - add a canonical `> **Convention:** see [conventions/brain-first.md]`
# callout near the top of the skill body.
#
# Usage: scripts/check-skill-brain-first.sh
# Exit: 0 on ok; 1 on warn or unexpected.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
cd "$ROOT"
# Run doctor with this repo's own skills as the explicit target.
#
# --fast is REQUIRED here. Without it, doctor calls connectEngine() which
# exits 1 when no ~/.gbrain/config.json exists (the CI runner's case — no
# brain init), producing zero stdout and tripping the parser's
# `parse_error` fallback. --fast routes through runDoctor(null, ...) which
# runs filesystem-only checks (resolver_health, skill_conformance,
# skill_brain_first) and emits the standard JSON envelope. The
# skill_brain_first check is filesystem-only by design, so --fast is the
# correct knob, not a workaround.
#
# Capturing JSON output; redirect stderr to keep progress noise out of the
# parse.
TMPOUT="$(mktemp -t gbrain-doctor-XXXXXXXX)"
# shellcheck disable=SC2064
trap "rm -f \"$TMPOUT\"" EXIT
GBRAIN_SKILLS_DIR="$ROOT/skills" bun run src/cli.ts doctor --fast --json >"$TMPOUT" 2>/dev/null || true
# Extract the skill_brain_first check status. Use python3 (already a
# repo-wide dependency via image-decoders + admin tooling) so we don't
# add jq to the verify chain.
STATUS=$(python3 -c "
import json, sys
with open('$TMPOUT') as fp:
for line in fp:
line = line.strip()
if not (line.startswith('{') and line.endswith('}')):
continue
try:
report = json.loads(line)
except Exception:
continue
for c in report.get('checks', []):
if c.get('name') == 'skill_brain_first':
print(c.get('status', 'missing'))
sys.exit(0)
print('missing')
sys.exit(0)
print('parse_error')
" 2>/dev/null || echo "parse_error")
case "$STATUS" in
ok)
echo "OK: skill_brain_first check passes against this repo's skills/"
exit 0
;;
warn)
echo
echo "ERROR: skill_brain_first check found violations in this repo's skills/."
echo
echo "Re-run for details:"
echo " GBRAIN_SKILLS_DIR=\"\$(pwd)/skills\" bun run src/cli.ts doctor"
echo
echo "Fix options per skill:"
echo " 1. Add 'brain_first: exempt' to frontmatter (declarative opt-out)"
echo " 2. Add a > **Convention:** see [conventions/brain-first.md] callout"
echo " 3. Run 'gbrain doctor --fix' to auto-add the canonical callout"
exit 1
;;
fail)
echo "ERROR: skill_brain_first check returned status=fail (unexpected)."
exit 1
;;
missing)
echo "ERROR: skill_brain_first check not present in doctor output."
echo " This guard expected the check to run. Investigate doctor.ts wiring."
exit 1
;;
*)
echo "ERROR: skill_brain_first guard could not parse doctor --json output."
echo " Status: $STATUS"
cat "$TMPOUT" | head -20
exit 1
;;
esac