v0.37.3.0 feat: skill_brain_first doctor check + auto-fix + declarative opt-out (supersedes #1206) (#1215)

* v0.37.1.0 feat: skill_brain_first doctor check + auto-fix + declarative opt-out Cathedral wave superseding PR #1206. Doctor now scans every SKILL.md for external-lookup tools (web_search / web_fetch / exa / perplexity / happenstance / crustdata / captain_api / firecrawl) and warns when the skill has no brain- first compliance signal. gbrain doctor --fix auto-inserts the canonical > **Convention:** see [conventions/brain-first.md](...) callout via the dry-fix.ts MISSING_RULE_PATTERNS extension (sharing safety gates with the existing REPLACE patterns). Motivated by the 2026-05-19 tweet-shield incident: cross-modal eval flagged Garry's Palantir tweet as risky because no model knew he built it, but the brain already had "designed the entire Finance product UI" and "150+ PSDs from April-December 2006." Static check catches authorship; v0.37+ runtime gate (filed in TODOS.md) closes the dispatch side. Key design decisions locked via /plan-eng-review + codex outside-voice review: - A1: frontmatter ships only brain_first: exempt (no required/n/a enum) - A2: snapshot+diff audit at ~/.gbrain/audit/skill-brain-first-YYYY-Www.jsonl with transition-only writes (stable brains = 0 lines/run) - A3: scaffold template pre-inserts callout; skillify check fails (exit 1) on external + no callout + no exempt - A4: position-relative gate is BODY-ONLY (frontmatter tools: [web_search] declaration doesn't false-flag the skill) - Q1: single pure analyzeSkillBrainFirst() helper consumed by 3 surfaces - CMT1: no upgrade migration — doctor surfaces hint, --fix applies via dry-fix safety gates (user stays in loop) - CMT2: dropped tools+writes_pages auto-exemption (was hiding mixed-class skills like idea-ingest/meeting-ingestion/data-research) Trio: VERSION + package.json + CHANGELOG aligned at 0.37.1.0. 56 unit cases + 12 E2E cases pass. 170 related existing tests pass unchanged. Self-dogfood: gbrain doctor against this repo's skills/ reports skill_brain_first: ok across 43 skills (compliant or exempt). functional-area-resolver and strategic-reading skills gained brain_first: exempt to validate the declarative opt-out in production code (both name perplexity in dispatcher prose without calling it). Co-Authored-By: garrytan-agents <noreply@github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update CLAUDE.md for v0.37.1.0 skill_brain_first wave Added Key Files entries for the four new modules: - src/core/skill-frontmatter.ts (shared parser) - src/core/skill-brain-first.ts (analyzer + FORMERLY_HARDCODED_EXEMPT) - src/core/skill-fix-gates.ts (extracted safety primitives) - src/core/audit-skill-brain-first.ts (snapshot+diff JSONL) Extended existing entries: - src/core/filing-audit.ts: rewired to shared parser - src/core/dry-fix.ts: MISSING_RULE_PATTERNS INSERT pattern type - src/commands/doctor.ts: skill_brain_first check + tweet-shield framing - src/commands/skillify-check.ts: required item 12 + scaffold pre-insert Added test inventory entries: - test/skill-brain-first.test.ts (56 unit cases) - test/e2e/skill-brain-first.test.ts (12 E2E cases) Regenerated llms-full.txt via bun run build:llms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): skill_brain_first guard uses doctor --fast to skip engine connect CI run #76881161092 failed because scripts/check-skill-brain-first.sh invoked plain `gbrain doctor --json`, which routes through connectEngine(). With no ~/.gbrain/config.json present (CI's case — runner is bun-only, no brain init), connectEngine() exits 1 with "No brain configured." and emits zero stdout. The python parser sees an empty file and returns parse_error, failing the verify gate. Fix: pass --fast to doctor. --fast routes through runDoctor(null, ...) which runs the filesystem-only check set (resolver_health, skill_conformance, skill_brain_first) and emits the standard single-line JSON envelope the parser expects. skill_brain_first is filesystem-only by design (scans SKILL.md, no DB touch), so --fast is the correct knob, not a workaround. Verified by reproducing the CI failure mode locally with GBRAIN_HOME=/tmp/empty-... — gate now passes both with and without a configured brain. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: rebump v0.37.1.0 → v0.37.3.0 (queue collision with #1214) PR #1214 (brainstorm + lsd) claimed v0.37.1.0 concurrently with #1215. Skipping 0.37.2.0 leaves a buffer for #1214's adjacent slot. Trio (VERSION + package.json + CHANGELOG header + inline "To take advantage of v0.37.3.0" block) aligned at 0.37.3.0. No behavior changes — version metadata only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: garrytan-agents <noreply@github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-07-31 04:07:52 +00:00 · 2026-05-20 13:55:12 -07:00
parent 9a4ae0962e
commit 772253ef44
31 changed files with 3098 additions and 87 deletions
@@ -0,0 +1,100 @@
+#!/usr/bin/env bash
+# CI guard for the v0.36.x skill_brain_first doctor check.
+#
+# Runs `gbrain doctor --json` against this repo's own skills/ and parses
+# the JSON to assert `checks[name=skill_brain_first].status !== "warn"`.
+# Doctor's exit code only flags `fail`, not `warn`, so explicit JSON-
+# parsing is required to gate `bun run verify` on this warning-class check
+# (F15 from /plan-eng-review).
+#
+# When this fires, the brain-first compliance check found new offenders
+# in this repo's skills. Either:
+#   - add `brain_first: exempt` to the flagged skill's frontmatter (if it
+#     legitimately doesn't need brain-first), or
+#   - add a canonical `> **Convention:** see [conventions/brain-first.md]`
+#     callout near the top of the skill body.
+#
+# Usage: scripts/check-skill-brain-first.sh
+# Exit:  0 on ok; 1 on warn or unexpected.
+
+set -euo pipefail
+
+ROOT="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"
+cd "$ROOT"
+
+# Run doctor with this repo's own skills as the explicit target.
+#
+# --fast is REQUIRED here. Without it, doctor calls connectEngine() which
+# exits 1 when no ~/.gbrain/config.json exists (the CI runner's case — no
+# brain init), producing zero stdout and tripping the parser's
+# `parse_error` fallback. --fast routes through runDoctor(null, ...) which
+# runs filesystem-only checks (resolver_health, skill_conformance,
+# skill_brain_first) and emits the standard JSON envelope. The
+# skill_brain_first check is filesystem-only by design, so --fast is the
+# correct knob, not a workaround.
+#
+# Capturing JSON output; redirect stderr to keep progress noise out of the
+# parse.
+TMPOUT="$(mktemp -t gbrain-doctor-XXXXXXXX)"
+# shellcheck disable=SC2064
+trap "rm -f \"$TMPOUT\"" EXIT
+
+GBRAIN_SKILLS_DIR="$ROOT/skills" bun run src/cli.ts doctor --fast --json >"$TMPOUT" 2>/dev/null || true
+
+# Extract the skill_brain_first check status. Use python3 (already a
+# repo-wide dependency via image-decoders + admin tooling) so we don't
+# add jq to the verify chain.
+STATUS=$(python3 -c "
+import json, sys
+with open('$TMPOUT') as fp:
+    for line in fp:
+        line = line.strip()
+        if not (line.startswith('{') and line.endswith('}')):
+            continue
+        try:
+            report = json.loads(line)
+        except Exception:
+            continue
+        for c in report.get('checks', []):
+            if c.get('name') == 'skill_brain_first':
+                print(c.get('status', 'missing'))
+                sys.exit(0)
+        print('missing')
+        sys.exit(0)
+print('parse_error')
+" 2>/dev/null || echo "parse_error")
+
+case "$STATUS" in
+  ok)
+    echo "OK: skill_brain_first check passes against this repo's skills/"
+    exit 0
+    ;;
+  warn)
+    echo
+    echo "ERROR: skill_brain_first check found violations in this repo's skills/."
+    echo
+    echo "Re-run for details:"
+    echo "  GBRAIN_SKILLS_DIR=\"\$(pwd)/skills\" bun run src/cli.ts doctor"
+    echo
+    echo "Fix options per skill:"
+    echo "  1. Add 'brain_first: exempt' to frontmatter (declarative opt-out)"
+    echo "  2. Add a > **Convention:** see [conventions/brain-first.md] callout"
+    echo "  3. Run 'gbrain doctor --fix' to auto-add the canonical callout"
+    exit 1
+    ;;
+  fail)
+    echo "ERROR: skill_brain_first check returned status=fail (unexpected)."
+    exit 1
+    ;;
+  missing)
+    echo "ERROR: skill_brain_first check not present in doctor output."
+    echo "       This guard expected the check to run. Investigate doctor.ts wiring."
+    exit 1
+    ;;
+  *)
+    echo "ERROR: skill_brain_first guard could not parse doctor --json output."
+    echo "       Status: $STATUS"
+    cat "$TMPOUT" | head -20
+    exit 1
+    ;;
+esac
@@ -0,0 +1,175 @@
+#!/usr/bin/env bun
+/**
+ * scripts/live-brain-first-check.ts — opt-in dev script for the v0.36.x
+ * skill_brain_first wave (T10 from /plan-eng-review).
+ *
+ * Runs the doctor brain-first check against the LIVE OpenClaw deployment
+ * (or whatever `$OPENCLAW_WORKSPACE` points at) and produces a human-
+ * readable + machine-readable report. NOT part of `bun run verify` — the
+ * deployment isn't this repo's content, so coupling CI to it would
+ * produce drift the moment OpenClaw evolves independently.
+ *
+ * Use this manually during dev / QA / after `gbrain doctor --fix` runs
+ * to validate the wave against the real deployment.
+ *
+ * Usage:
+ *   $OPENCLAW_WORKSPACE=~/.openclaw/workspace bun run scripts/live-brain-first-check.ts
+ *   $OPENCLAW_WORKSPACE=~/.openclaw/workspace bun run scripts/live-brain-first-check.ts --json
+ *   $OPENCLAW_WORKSPACE=~/.openclaw/workspace bun run scripts/live-brain-first-check.ts --fix-preview
+ *
+ * Exit codes:
+ *   0 — workspace clean (no violators) OR workspace not configured
+ *   1 — workspace configured + has violators (informational; no CI gate)
+ *   2 — usage error (couldn't resolve workspace, doctor crash, etc.)
+ */
+
+import { existsSync, readFileSync } from 'fs';
+import { join } from 'path';
+
+import { autoDetectSkillsDirReadOnly } from '../src/core/repo-root.ts';
+import { skillBrainFirstCheck } from '../src/commands/doctor.ts';
+import { autoFixDryViolations } from '../src/core/dry-fix.ts';
+import { parseSkillFrontmatter } from '../src/core/skill-frontmatter.ts';
+import {
+  analyzeSkillBrainFirst,
+  buildBrainFirstSummaryLine,
+  FORMERLY_HARDCODED_EXEMPT,
+} from '../src/core/skill-brain-first.ts';
+import { loadOrDeriveManifest } from '../src/core/skill-manifest.ts';
+
+function main(): number {
+  const args = process.argv.slice(2);
+  const jsonMode = args.includes('--json');
+  const fixPreview = args.includes('--fix-preview');
+
+  if (!process.env.OPENCLAW_WORKSPACE && !process.env.GBRAIN_SKILLS_DIR) {
+    process.stderr.write(
+      '[live-brain-first] No skills dir source set. Set OPENCLAW_WORKSPACE or GBRAIN_SKILLS_DIR.\n' +
+      '  $OPENCLAW_WORKSPACE=~/.openclaw/workspace bun run scripts/live-brain-first-check.ts\n',
+    );
+    return 0; // not configured = not a failure
+  }
+
+  const detected = autoDetectSkillsDirReadOnly();
+  if (!detected || !detected.dir || !existsSync(detected.dir)) {
+    process.stderr.write(
+      `[live-brain-first] Could not resolve skills dir from OPENCLAW_WORKSPACE / GBRAIN_SKILLS_DIR / cwd walk-up.\n`,
+    );
+    return 2;
+  }
+
+  process.stderr.write(`[live-brain-first] Scanning ${detected.dir} (source: ${detected.source})\n\n`);
+
+  const check = skillBrainFirstCheck(detected.dir);
+  const violators = check.issues ?? [];
+
+  // Shape assertions for human eyeballs (per T1 from /plan-eng-review):
+  // structurally-compliant skills MUST NOT appear in the violator list.
+  // We surface these as informational warnings if any shape invariant
+  // breaks; the script never fails on shape breach (that's the unit-test
+  // suite's job).
+  const shapeWarnings: string[] = [];
+
+  // Walk manifest once to collect classification info for the report.
+  const manifest = loadOrDeriveManifest(detected.dir);
+  const compliantViaCallout: string[] = [];
+  const exemptByFrontmatter: string[] = [];
+  const exemptByNoExternal: string[] = [];
+  const flaggedFormerly: string[] = [];
+  const flaggedNew: string[] = [];
+
+  for (const entry of manifest.skills) {
+    const skillPath = join(detected.dir, entry.path);
+    if (!existsSync(skillPath)) continue;
+    let content: string;
+    try {
+      content = readFileSync(skillPath, 'utf-8');
+    } catch {
+      continue;
+    }
+    const fm = parseSkillFrontmatter(content);
+    const a = analyzeSkillBrainFirst(content, entry.name, fm);
+    if (a.status === 'ok') {
+      if (a.reason === 'compliant_callout' || a.reason === 'compliant_phase' || a.reason === 'compliant_position') {
+        compliantViaCallout.push(a.skill);
+      } else if (a.reason === 'exempt_explicit') {
+        exemptByFrontmatter.push(a.skill);
+      } else if (a.reason === 'exempt_no_external') {
+        exemptByNoExternal.push(a.skill);
+      }
+    } else {
+      if (a.formerly_hardcoded_exempt) flaggedFormerly.push(a.skill);
+      else flaggedNew.push(a.skill);
+    }
+  }
+
+  if (jsonMode) {
+    const report = {
+      schema_version: 1,
+      workspace: detected.dir,
+      source: detected.source,
+      status: check.status,
+      total_skills: manifest.skills.length,
+      compliant_via_callout: compliantViaCallout.sort(),
+      exempt_by_frontmatter: exemptByFrontmatter.sort(),
+      exempt_by_no_external: exemptByNoExternal.sort(),
+      flagged_formerly_exempt: flaggedFormerly.sort(),
+      flagged_new: flaggedNew.sort(),
+      total_violators: violators.length,
+    };
+    console.log(JSON.stringify(report, null, 2));
+  } else {
+    console.log(`Workspace: ${detected.dir}`);
+    console.log(`Source: ${detected.source}`);
+    console.log(`Total skills scanned: ${manifest.skills.length}`);
+    console.log('');
+    console.log(`Compliant via callout/phase/position: ${compliantViaCallout.length}`);
+    console.log(`Exempt by frontmatter (brain_first: exempt): ${exemptByFrontmatter.length}`);
+    console.log(`Exempt by no-external-pattern: ${exemptByNoExternal.length}`);
+    console.log('');
+    console.log(`Flagged (formerly hardcoded-exempt in PR #1206): ${flaggedFormerly.length}`);
+    if (flaggedFormerly.length > 0) {
+      for (const s of flaggedFormerly.sort()) console.log(`  - ${s}`);
+    }
+    console.log('');
+    console.log(`Flagged (genuinely new violators): ${flaggedNew.length}`);
+    if (flaggedNew.length > 0) {
+      for (const s of flaggedNew.sort()) console.log(`  - ${s}`);
+    }
+    console.log('');
+
+    if (violators.length === 0) {
+      console.log('STATUS: ok — no brain-first violators in the live deployment');
+    } else {
+      console.log('STATUS: warn — fix with:');
+      console.log('  gbrain doctor --fix       # auto-add canonical Convention callout (writes files)');
+      console.log('  gbrain doctor --fix --dry-run   # preview without writing');
+      console.log('  or add `brain_first: exempt` to each flagged skill\'s frontmatter');
+    }
+  }
+
+  // --fix-preview: also run autoFixDryViolations against the live workspace
+  // in dry-run mode to show what callouts the auto-fix would insert.
+  if (fixPreview) {
+    process.stderr.write('\n[live-brain-first] Running auto-fix dry-run preview...\n');
+    const report = autoFixDryViolations(detected.dir, { dryRun: true });
+    const brainFirstProposed = report.fixed.filter(
+      f => f.status === 'proposed' && f.patternLabel === 'brain-first compliance',
+    );
+    if (brainFirstProposed.length === 0) {
+      console.log('No brain-first auto-fix proposals.');
+    } else {
+      console.log(`\nWould insert canonical Convention callout into ${brainFirstProposed.length} skill(s):`);
+      for (const p of brainFirstProposed.slice(0, 10)) {
+        console.log(`  - ${p.skill}`);
+      }
+      if (brainFirstProposed.length > 10) {
+        console.log(`  ... and ${brainFirstProposed.length - 10} more`);
+      }
+    }
+  }
+
+  return violators.length > 0 ? 1 : 0;
+}
+
+process.exit(main());