mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
v0.19.0 check-resolvable: add OpenClaw skills-dir fallback + docs/tests (#326)
* Add OpenClaw skills fallback for check-resolvable * feat: v0.17.0 foundation — errors/warnings split + AGENTS.md support + auto-manifest First two workstreams of the v0.17.0 "skillify end-to-end" release. Landed together because the D-CX-3 exit-code refactor is a prerequisite for W1's warning-surfaced filing audit in Workstream 3. ## D-CX-3: split ResolvableReport into errors[] + warnings[] + --strict Prior: `env.ok = report.issues.length === 0` treated warnings and errors identically for exit status. Any warning forced exit 1, which meant the planned filing-audit (W3) would break CI for every OpenClaw deployment emitting advisory warnings. New contract: - `ResolvableReport.errors[]` and `warnings[]` as separate arrays. - `issues[]` stays as deprecated backcompat union (remove in v0.18). - Default: exit 0 unless any errors. Warnings are advisory. - `--strict` flag promotes warnings to fail CI (explicit opt-in). Files: src/core/check-resolvable.ts, src/commands/check-resolvable.ts (added --strict flag + help text + header doc), src/commands/doctor.ts (use new fields), test/check-resolvable-cli.test.ts (rewrite REGRESSION-GATE to document the new contract, add 3 D-CX-3 cases). ## W1: AGENTS.md support + auto-manifest + priority fix The reference OpenClaw deployment uses AGENTS.md (not RESOLVER.md) at the workspace root, and ships without a manifest.json. check-resolvable silently false-passed against it pre-W1: 0 manifest entries meant 0 reachability iterations meant 0 errors reported. Post-W1 behavior against ~/git/<redacted>/workspace (smoke-tested live): - Detects 102 skills via SKILL.md walk (no manifest.json needed) - Flags 15 unreachable errors (exactly the essay's '~15% dark' finding) - Flags 108 warnings (overlaps, gaps) — advisory, not blocking - Auto-detects via \$OPENCLAW_WORKSPACE without --skills-dir Changes: - NEW src/core/resolver-filenames.ts: one source of truth for the filename policy. \`RESOLVER_FILENAMES = ['RESOLVER.md', 'AGENTS.md']\`. Callers import from here, never hardcode either name. - NEW src/core/skill-manifest.ts: \`loadOrDeriveManifest()\` — reads manifest.json when present+valid, otherwise walks \`skillsDir/*/SKILL.md\` to derive a synthetic manifest. Both check-resolvable.ts AND dry-fix.ts now call this, replacing the two duplicated loaders that silently returned [] on missing file (F-ENG-1, D-CX-12). - src/core/repo-root.ts (rewrite): auto-detect priority changed to put \$OPENCLAW_WORKSPACE ahead of findRepoRoot() walk when explicitly set (D-CX-4). Adds workspace-root AGENTS.md detection — OpenClaw layout places routing at workspace/AGENTS.md with skills/ below. New SkillsDirSource variants \`openclaw_workspace_env_root\` and \`openclaw_workspace_home_root\` for --verbose log clarity. - src/core/check-resolvable.ts: accepts RESOLVER.md or AGENTS.md at the skills dir or one level up (workspace root). Uses loadOrDeriveManifest for reachability. Updated error messages reference both filenames. - src/core/dry-fix.ts: unified manifest loader — auto-fix now works in AGENTS.md-only workspaces where it previously no-op'd silently. - src/commands/check-resolvable.ts: new AUTO_DETECT_HINT import for clearer missing-skills-dir errors; updated sourceLabel map for the two new workspace-root variants. Tests: - test/skill-manifest.test.ts: 14 cases covering explicit-manifest, derived-manifest, malformed JSON, wrong shape, empty explicit array (honored as 'zero skills' declaration), dirname fallback when no name: frontmatter, underscore/dotfile dir skipping. - test/repo-root.test.ts: new tests for the priority swap, AGENTS.md skills-dir variant, AGENTS.md workspace-root variant, both-files present (RESOLVER.md wins). - test/check-resolvable-cli.test.ts: updated regression-gate to the new contract; added three D-CX-3 cases. All 105 tests passing across the foundation surface. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W2 — Check 5 trigger routing eval (structural) Check 5 of the 10-step skillify checklist (the essay's "resolver trigger eval") now runs structurally by default and has a dedicated CLI verb for CI. Ships Layer A; Layer B (LLM tie-break) is reserved for v0.18. ## New module: src/core/routing-eval.ts The harness. Pure functions: - `normalizeText(s)`: lowercase, strip non-alnum to spaces, collapse whitespace. Unicode-friendly, quote-agnostic, punctuation-tolerant. - `extractTriggerPhrases(cellText)`: split quoted alternatives like `"search for", "find me"` into separate normalized phrases; fall back to the whole cell when unquoted (OpenClaw-style descriptions). - `indexResolverTriggers(resolverContent)`: build a skill-slug → normalized-trigger-phrases map from the resolver table. - `structuralRouteMatch(intent, index)`: substring-match the normalized intent against every trigger phrase; return the set of matched skills + whether the match was ambiguous (more than one specific skill, excluding always-on family). - `lintRoutingFixtures`: rejects fixtures whose intent is verbatim-equal to a trigger (D-CX-6: fixtures must paraphrase the framing, not copy the trigger text) and unknown expected_skill references. - `loadRoutingFixtures(skillsDir)`: walks `skills/<name>/routing-eval.jsonl`, handles JSONL line-comments (`//` / `#`), collects malformed lines separately without crashing. - `runRoutingEval(resolver, fixtures)`: pure scoring. Supports negative cases (`expected_skill: null` — nothing should match) and an `ambiguous_with` allow-list for skills that co-fire with always-on handlers (signal-detector, brain-ops, ingest). Outcomes per fixture: `pass`, `missed`, `ambiguous`, `false_positive`. Metrics: `top1Accuracy`, `passed`, `missed`, `ambiguous`, `falsePositives`. ## Integration: check-resolvable runs Layer A by default `checkResolvable()` now loads `routing-eval.jsonl` fixtures from every skill, runs the structural eval, and appends non-pass outcomes as warning-severity issues. New issue types: - `routing_miss` — expected skill did not match - `routing_ambiguous` — expected matched AND unexpected skills - `routing_false_positive` — negative case unexpectedly matched - `routing_fixture_lint` — linter or malformed-JSONL finding All four are warnings — routing issues don't break exit in default mode, but `--strict` promotes them (D-CX-3 contract). Advisories without breaking CI. ## New CLI verb: `gbrain routing-eval` Standalone Check 5 runner. `--json` envelope, `--llm` flag reserved, `--skills-dir` override. Exit codes: 0 clean, 1 any failure/lint, 2 setup error. Suitable for CI gating separately from check-resolvable. Removed from DEFERRED in CLI: `{check: 5, name: trigger_routing_eval}`. Check 6 (brain_filing) still deferred; lands in W3. ## Seed fixtures - skills/query/routing-eval.jsonl - skills/citation-fixer/routing-eval.jsonl (includes a negative case) These are intentionally modest. Additional fixtures per skill are the natural next step; routing-eval itself passes cleanly under check-resolvable default mode even when fixtures surface real gaps (they're warnings, not errors). Running `gbrain routing-eval` reveals the gaps immediately. ## Tests (34 new cases + updated integrations) - test/routing-eval.test.ts: full harness coverage including normalization, trigger extraction (quoted and unquoted), indexer, structural match with ambiguity + always-on exemption, fixture linter (verbatim-equality rule, unknown-skill rule, shape rule, negative-case skip), JSONL loader (comments, malformed lines, missing dirs, underscore/dot skipping), and every runRoutingEval outcome (pass, miss, ambiguous, negative-pass, false-positive, empty). - test/check-resolvable-cli.test.ts: updated DEFERRED unit test + `--json` envelope test + `--verbose` test to reflect Check 5 shipping. 140/140 passing across the W1 + W2 surface. ## Live smoke `gbrain routing-eval --json` against the current gbrain repo: 6 fixtures, 1 passing, 5 missed. The misses correctly surface resolver-trigger narrowness (intents users naturally phrase differently than trigger text). Fixtures will iterate in follow-up PRs; the machinery ships now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W3 — Check 6 brain filing audit Check 6 ships. Every skill that writes brain pages is now audited against a machine-readable filing-rules doc at `skills/_brain-filing-rules.json`. ## New: skills/_brain-filing-rules.json Canonical filing rules, JSON (D-CX-8: the pre-existing yaml-lite parser handles flat maps only, so YAML would have needed a new dependency for one file). The companion `_brain-filing-rules.md` stays as the human explainer. 14 rule entries + explicit `sources_dir` carve-out for bulk/raw data. ## New module: src/core/filing-audit.ts - `loadFilingRules(skillsDir)`: returns parsed doc or null (missing file → no-op; malformed JSON throws loud). - `allowedDirectories(rules)`: normalized set of every rules[] directory + sources_dir. - `runFilingAudit(skillsDir)`: walks skills/*/SKILL.md, parses frontmatter, audits any skill with `writes_pages: true`. Two checks per qualifying skill: 1. `writes_to:` list is non-empty. 2. Every entry in `writes_to:` appears in allowedDirectories. Both failures emit warning-severity issues. No errors — advisories only, per D-CX-3. ## Distinction: writes_pages vs mutating (D-CX-7) v0.17 introduces a new boolean frontmatter field `writes_pages:`. `mutating: true` already means "has any side effect" (cron schedulers, report writers, config mutators). Filing audit targets ONLY skills with `writes_pages: true`, correctly excluding side- effect-but-not-page-writing skills. The codex outside voice caught this: conflating the two fields would drag ~100 skills into filing-audit noise in the reference OpenClaw deployment. ## Integration: check-resolvable runs Check 6 by default `checkResolvable()` calls `runFilingAudit(skillsDir)` and appends issues as warnings. On missing/malformed rules doc, surfaces a single advisory rather than bailing. `DEFERRED` array in the CLI is now empty — v0.17 ships both Check 5 (W2) and Check 6 (W3). The export stays in place (stable --json field) for future deferred checks. ## Seeded frontmatter on 7 canonical writers Added `writes_pages: true` + `writes_to:` to: - brain-ops (people, companies, deals, concepts, meetings) - enrich (people, companies) - ingest (people, companies, concepts, meetings, sources) - idea-ingest (people, concepts, sources) - media-ingest (concepts, people, companies, sources) - meeting-ingestion (meetings, people, companies) - signal-detector (people, companies, concepts) Live smoke: `gbrain check-resolvable --json` on gbrain repo shows `ok: true`, zero filing errors, zero filing warnings on seeded skills. Every other mutating:true skill (citation-fixer, cron-scheduler, data-research, maintain, migrate, minion-orchestrator, reports, setup, skill-creator, soul-audit, webhook-transforms) correctly skipped as side-effectful-but-not-page-writing. ## Tests (17 new cases + 3 updated CLI integrations) test/filing-audit.test.ts covers: - rules loader: missing (null), valid, malformed (throw), non-array rules (throw) - directory normalization (trailing slash, leading slash) - clean case - missing writes_to on writes_pages:true - unknown directory - D-CX-7: mutating:true alone does not trigger audit - writes_pages:false skips - no frontmatter skips - inline `writes_to: [a, b]` syntax - block `writes_to:\n - a` syntax - sources/ allowed - underscore/dot dir skipping - total counts (totalScanned vs writesPagesSkills) - missing dir graceful - action string quality guard Plus: CLI integration tests updated for empty DEFERRED array (Checks 5 and 6 both shipped). 158/158 passing across the v0.17 foundation + W1 + W2 + W3 surface. ## v0.18 preview (D-CX-13) v0.17 filing-audit is declaration-level only. A future `gbrain filing-audit --pages` walks the brain itself, infers primary subject from page content via LLM judgment, and flags actual misfilings vs. declarations. Declaration audit is the leading indicator; pages audit is the ground truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W4 — gbrain skillify {scaffold,check} subcommand namespace The essay's "skillify it!" verb becomes a CLI primitive pair. Two subcommands, both promoted/factored so there's one source of truth: ## `gbrain skillify scaffold <name>` (mechanical) Pure file generation. Zero LLM, zero judgment. Writes 5 stub files atomically: 1. skills/<name>/SKILL.md frontmatter + body template 2. skills/<name>/scripts/<name>.mjs deterministic-code stub 3. skills/<name>/routing-eval.jsonl routing fixture seed 4. test/<name>.test.ts vitest skeleton 5. Appended trigger row to the detected resolver file (RESOLVER.md or AGENTS.md — whatever W1's auto-detect found) Flags: --description (required), --triggers, --writes-to, --writes-pages, --mutating, --force, --dry-run, --json, --skills-dir. Kebab-case name validation (`^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$`). Works against gbrain-native RESOLVER.md layout AND OpenClaw-native AGENTS.md-at-workspace-root layout (W1 interop). ## `gbrain skillify check [path]` (audit) Promoted from scripts/skillify-check.ts per codex D-CX-2. The legacy script stays as a 12-line shim that delegates to the new module so existing callers (docs, cron, tests) keep working. Wrapped in a subcommand namespace: `gbrain skillify {scaffold, check}` is one coherent verb for the whole post-task loop. The essay's "skillify it!" triggers the markdown skill, which orchestrates the CLI primitives. ## Idempotency contract (D-CX-7) `skillify scaffold --force` regenerates stub FILES but never re-appends a resolver row that already references `skills/<name>/SKILL.md`. Unit test pins this: two applies produce one resolver row, not two. ## D-CX-9 SKILLIFY_STUB sentinel Every scaffolded script + SKILL.md body carries a SKILLIFY_STUB sentinel. `check-resolvable` walks every skill's script dir looking for the marker and emits a `skillify_stub_unreplaced` warning when found. Default mode: advisory. `--strict` mode: error, blocks CI. This is the gate that catches "we scaffolded and forgot to implement" — the exact failure codex flagged as "scaffold verification is theater" in the outside-voice review. ## Files - NEW src/core/skillify/templates.ts (template strings) - NEW src/core/skillify/generator.ts (planScaffold / applyScaffold + SkillifyScaffoldError with typed error codes) - NEW src/commands/skillify.ts (top-level dispatcher + scaffold handler) - NEW src/commands/skillify-check.ts (promoted check logic) - scripts/skillify-check.ts: rewritten to 12-line shim - skills/skillify/SKILL.md: Phase 2 now references the scaffold primitive; legacy manual path kept for extending existing skills - src/cli.ts: `skillify` added to CLI_ONLY + dispatcher - src/core/check-resolvable.ts: SKILLIFY_STUB sentinel scan + new issue type `skillify_stub_unreplaced` ## Tests (14 new scaffold cases) test/skillify-scaffold.test.ts covers: - SKILL_NAME_PATTERN validation (kebab-case, no spaces, no leading digit, no underscores/uppercase) - planScaffold against fresh + existing-file + --force paths - SKILLIFY_STUB sentinel presence in SKILL.md AND script stub (both gate paths) - D-CX-7 idempotency: resolverAppend null when row pre-exists, second apply doesn't duplicate the row - TBD-trigger placeholder when --triggers empty - writes_pages / writes_to / mutating flow through to frontmatter - applyScaffold writes files + appends resolver - Full AGENTS.md-layout workspace interop (W1) Existing test/skillify-check.test.ts still passes against the legacy shim — zero regression for downstream consumers. 178/178 passing across v0.17 foundation + W1..W4. ## Live smoke \`gbrain skillify scaffold webhook-verify --description "verify incoming webhook signatures" --triggers "verify webhook,check tunnel" --skills-dir /tmp/smoke --dry-run\` produces the expected 4-file plan plus a 115-byte resolver append. \`--help\` works on both the top-level and scaffold levels. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W5 — gbrain skillpack install (deps closure + lockfile + diff/dry-run) The essay's "drop it into YOUR OpenClaw" promise lands as a CLI verb. One command installs a curated bundle of gbrain skills + the shared convention files they depend on into a target OpenClaw workspace. Data-loss protected, concurrency-safe, atomic on the AGENTS.md managed block. ## openclaw.plugin.json refresh - Bumped version from stale 0.4.1 → 0.17.0 (codex flagged this drift F-ENG-4 / D-CX-4). - Expanded curated skill list from 7 → 25. Uses skills/manifest.json top-level (v0.10.0 sourced) minus setup/migrate/publish (install-time / code+skill pairs) minus private skills. - Added \`shared_deps: [...]\` listing convention files every skill references: conventions/, _brain-filing-rules.{md,json}, _output-rules.md. Installer always pulls these (D-CX-10 dependency closure). - Added \`excluded_from_install: [...]\` for setup/migrate/publish — surfaces the intentional exclusion as data rather than a comment. ## New module: src/core/skillpack/bundle.ts - \`findGbrainRoot(start)\` — walks up looking for openclaw.plugin.json + src/cli.ts. The pair identifies a gbrain checkout. - \`loadBundleManifest(root)\` — strict validation + typed BundleError codes (manifest_not_found, manifest_malformed, skill_not_found). - \`enumerateBundle({gbrainRoot, skillSlug?, manifest})\` — flat list of source → target-relative paths. When skillSlug is set, scopes to that one skill BUT always pulls shared_deps. \`--all\` walks every skill in the manifest. - \`bundledSkillSlugs(manifest)\` — sorted slugs for \`skillpack list\`. ## New module: src/core/skillpack/installer.ts - \`planInstall(opts)\` — builds InstallPlan with per-file existing/identical diff state. Pure; no writes. - \`applyInstall(plan, opts)\` — writes files + managed block with the contracts below. - \`diffSkill(root, slug, skillsDir)\` — read-only per-file status for \`skillpack diff <name>\`. **Per-file diff protection (D-CX-3 / F4):** wrote_new fresh file wrote_overwrite local diff + --overwrite-local passed skipped_identical bytes match the bundle (silent re-install) skipped_locally_modified target differs + no --overwrite-local → PROTECTED DEFAULT **Concurrency + atomic AGENTS.md (D-CX-11):** - \`.gbrain-skillpack.lock\` at workspace root. Acquired on the first write, released in finally. - Lock stale threshold configurable (default 10min). --force-unlock overrides. - Managed-block writes via tmp-file-plus-rename (atomic on POSIX). **Managed-block format:** <!-- gbrain:skillpack:begin --> <!-- Installed by gbrain <version> — do not hand-edit between markers. --> | Trigger | Skill | |---------|-------| | "alpha" | \`skills/alpha/SKILL.md\` | | ... <!-- gbrain:skillpack:end --> extractManagedSlugs() roundtrips: single-skill installs accumulate into the same block rather than overwriting each other. ## New CLI: gbrain skillpack {list, install, diff, check} Namespaced alongside W4's \`gbrain skillify\`. Subcommands: list bundle inventory (human + --json) install <name> single skill + deps closure install --all entire curated bundle diff <name> per-file diff vs target; read-only check delegates to the pre-existing skillpack-check (same CLI just namespaced) Flags on install: --overwrite-local, --force-unlock, --dry-run, --json, --skills-dir, --workspace. Exit codes: 0 clean, 1 files skipped (protected local edits), 2 setup error / lock held. ## Live smoke \`gbrain skillpack list\`: 25 skills. \`skillpack install query --dry-run\` against a fresh temp workspace: 12 files planned (SKILL.md, routing-eval.jsonl, 7 convention files, 3 rule files, managed block to AGENTS.md). All shared_deps flagged [shared]. ## Tests (36 new cases) test/skillpack-install.test.ts: - findGbrainRoot walks up, returns null when absent - loadBundleManifest validates + rejects malformed - enumerateBundle pulls shared_deps on single-skill scope (D-CX-10) - buildManagedBlock + updateManagedBlock: append when absent, in-place replace when present, extractManagedSlugs roundtrip - planInstall + applyInstall: fresh install, dry-run, idempotency (skipped_identical), local-edit protection, --overwrite-local, lock-held concurrency (D-CX-11), --force-unlock, atomic managed-block write, multi-skill accumulation in managed block, AGENTS.md-at-workspace-root interop (W1 cross-check) - diffSkill: missing, identical, differs test/skillpack-sync-guard.test.ts (F-ENG-4): - both manifests exist - every skill in plugin.json exists on disk - every shared_dep exists on disk - plugin.json skills ⊂ skills/manifest.json - excluded skills aren't in the install list - plugin version ≥ 0.17 (kills the 0.4.1 stale drift) 204/204 passing across the v0.17 foundation + W1..W5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 guards — privacy scrub + OpenClaw-reference E2E + v0.16.4 regression Three ship-blocker work items from the eng review + codex outside voice round out v0.17: ## scripts/check-privacy.sh (CLAUDE.md:550 enforcement) Greps for the banned OpenClaw fork name (case-insensitive) across tracked files. Two modes: scripts/check-privacy.sh scan working tree scripts/check-privacy.sh --staged scan git-staged files (pre-commit) Exit 1 on any finding outside the allow-list. Allow-list covers files where the name is legitimately present: this script itself (defines the rule), CLAUDE.md (the canonical rule text), llms-full.txt (auto-generated from CLAUDE.md), the historical upgrade guide, and test/integrations.test.ts (whose personal-info regex ENFORCES the rule against recipes/). Scrubbed existing leaks: - CHANGELOG.md:366 reference in a closes-# line → "from the OpenClaw reference deployment" - test/doctor-minions-check.test.ts:171 comment → "an OpenClaw host's cron script" - test/plugin-loader.test.ts fixture plugin name → "openclaw-ref" ## test/e2e/openclaw-reference-compat.test.ts (ship-blocker gate) The test that proves v0.17 delivers on the headline claim. New fixture at test/fixtures/openclaw-reference-minimal/ mimics the reference OpenClaw deployment layout: AGENTS.md at workspace root, skills/ below, no manifest.json. Four fixture skills (signal-detector, query, brain-ops, context-now). Every v0.17 surface gets exercised end-to-end: - autoDetectSkillsDir with $OPENCLAW_WORKSPACE (D-CX-4 priority) - loadOrDeriveManifest walks SKILL.md (F-ENG-1 auto-manifest) - checkResolvable accepts AGENTS.md at workspace root, all 4 skills reachable via resolver rows, zero errors - Filing audit clean (brain-ops declares writes_pages+writes_to) - CLI subprocess via `--skills-dir` → exit 0 - CLI subprocess via $OPENCLAW_WORKSPACE (no flag) → exit 0, correct skillsDir detection - skillpack install against the layout writes managed block into AGENTS.md at workspace root This is THE ship-blocker test. If the W1 + W5 stack ever regresses against an AGENTS.md-layout workspace, this fails first. ## test/regression-v0_16_4.test.ts (F-ENG-8) Guards v0.17 against adding "surprise" warnings. Builds a clean fixture matching v0.16.4 canonical shape (manifest.json, RESOLVER.md, 2 skills, no routing-eval fixtures, no writes_pages). Runs v0.17 checkResolvable and asserts: - zero errors, zero routing_*/filing_*/skillify_stub_* warnings - JSON envelope keys unchanged (errors, warnings, issues, ok, summary) — deprecated `issues[]` still equals errors ∪ warnings - summary shape unchanged If someone adds a new check that fires unexpectedly on a v0.16.4-era fixture, this test catches it immediately. ## Fixture test/fixtures/openclaw-reference-minimal/ ├── AGENTS.md (4 rows, 3 sections) └── skills/ ├── brain-ops/SKILL.md (writes_pages+writes_to) ├── context-now/SKILL.md ├── query/SKILL.md └── signal-detector/SKILL.md Intentionally small (4 skills, 1 AGENTS.md, ~30 lines total) so the fixture is maintainable. The OPENCLAW-reference deployment has 107 skills — this fixture is the minimum shape that exercises the full v0.17 code path. ## Tests 215/215 passing across the full v0.17 surface: - foundation + W1 + W2 + W3 + W4 + W5 (204) - regression-v0_16_4 (3) - openclaw-reference-compat (7) - privacy guard (separate bash; exits 0 clean) Plus: privacy pre-commit hook is a drop-in wrapper (documented in the script header). Wiring into .github/workflows is a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * release: v0.17.0 — skillify goes end-to-end Every skill. Every check. Every install. One command each. Five workstreams land in one release: - W1: AGENTS.md + auto-manifest + env-priority - W2: Check 5 routing eval - W3: Check 6 brain filing - W4: gbrain skillify {scaffold,check} - W5: gbrain skillpack {list,install,diff} Plus D-CX-3 foundation (errors/warnings split + --strict), plus codex outside-voice fixes (D-CX-1..12 applied), plus privacy pre- commit guard, plus OpenClaw-reference E2E fixture, plus v0.16.4 regression guard. Live against the reference OpenClaw deployment: 102 skills detected via auto-manifest, 15 unreachable errors + 108 warnings surfaced — exactly the essay's "~15% dark" finding. The magic word from the essay finally works the way the essay describes. Tests: 2156 unit (178 new) + 152 E2E Tier 1 + 3 Tier 2 + 8 new openclaw-reference fixture cases. 0 failures across all tiers. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): add missing 'strict' field to 5 Flags literals in check-resolvable-cli.test.ts CI failed `tsc --noEmit` after the D-CX-3 errors/warnings split added `strict: boolean` as a required field on the `Flags` interface. Five test sites in test/check-resolvable-cli.test.ts still construct Flags object literals (for direct `resolveSkillsDir()` calls) and hadn't been updated. Added `strict: false` to all five literals: - line 129 --skills-dir absolute path - line 135 --skills-dir relative path - line 148 no --skills-dir - line 160 no --skills-dir + no env - line 178 --skills-dir + OPENCLAW_WORKSPACE (REGRESSION-GATE) Unit tests: 207/207 pass across the v0.19 surface. tsc --noEmit exits 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: adopt gstack's branch-scoped CHANGELOG rule + rewrite v0.19.0 entry CLAUDE.md gains a new top section before "CHANGELOG voice" that codifies what gstack's CLAUDE.md already says: CHANGELOG is user-facing product release notes, not a log of internal decisions. Every entry describes what THIS branch adds vs master. Plan-file IDs, decision tags (D-CX-#, F-ENG-#), review rounds, test counts as marketing, and contributor- facing metrics don't belong in it. The v0.19.0 entry is rewritten to the new bar: Removed: - Version-collision note about v0.17.0/v0.18.0 shipping on master - All D-CX-## and W# tags (meaningless outside the plan file) - "codex caught" / CEO + Eng review round-up narrative - Plan file path reference - "215 new cases across 13 test files" marketing metrics - W1..W5 bucketing in itemized changes Kept / sharpened: - User-facing headline (what your agent can now do) - Numbers that mean something to users (unreachable-skills count, scaffold timing, pre/post AGENTS.md support) - Upgrade instructions - Added/Changed/Fixed/For-contributors itemized sections (standard keep-a-changelog shape) Version sequence (`grep "^## \["`) is contiguous v0.19.0 → v0.16.4. Privacy guard clean. Tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update README/CLAUDE/TODOS for v0.19.0 skills + skillify loop Skill count was stale (README said 26, actual is 28: skillify + skillpack-check were missing from the tables and count). Corrected throughout. Marked TODOS item "Checks 5 + 6 deferred in PR #325" as completed in v0.19 — they shipped as real implementations, not just filed issues. README: - Skill count 26 → 28 (headline, install flow, table section, architecture diagram) - Added `skillify` + `skillpack-check` rows to the operational skills table - Rewrote the "Skillify" section to lead with the four v0.19 CLI verbs (`gbrain skillify scaffold/check`, `gbrain skillpack list/install/diff`, `gbrain routing-eval`, `gbrain check-resolvable --strict`) instead of describing the pre-v0.19 state. Added the "works on your OpenClaw" pitch around AGENTS.md + auto-manifest. Added the "drop 25 curated skills into your OpenClaw" section for skillpack install. - Added v0.19 skills block + v0.18 multi-source + v0.17 dream to the Commands reference at the bottom. - Standalone instruction sets count: 25 → 28 (with a parenthetical noting the curated 25-skill bundle that `skillpack install` ships). CLAUDE.md: - Skill count 26 → 28 in the Skills section. - New "Skillify loop (v0.19)" sub-bullet listing skillify + skillpack-check. - Noted that `AGENTS.md` is also accepted as a resolver filename. TODOS.md: - Created "## Completed" section at the top. - Moved the "Checks 5 + 6" item there with completion note linking to the actual implementation files (routing-eval.ts + filing-audit.ts). Privacy scan clean. Version sequence contiguous v0.19.0 → v0.16.4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): regenerate llms-full.txt + llms.txt after README/CLAUDE edits CI failed on `build-llms generator > committed llms.txt + llms-full.txt match current generator output`. The drift was expected: the prior commit edited README.md and CLAUDE.md (skill count + skillify section), both of which are inlined into llms-full.txt by `scripts/build-llms.ts`. Fix: `bun run build:llms` + commit the regenerated output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Wintermute <wintermute@garrytan.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Wintermute
Claude Opus 4.7
Wintermute
Claude Opus 4.7
parent
08b3698e90
commit
78ba0b5b53
+108
-4
@@ -2,6 +2,108 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.19.0] - 2026-04-22
|
||||
|
||||
## **Your OpenClaw finally learns. Say "skillify it!" and every new failure becomes a durable skill.**
|
||||
## **AGENTS.md workspaces work out of the box. `gbrain skillpack install` drops 25 curated skills into your OpenClaw.**
|
||||
|
||||
Your agent can now turn any ad-hoc fix into a permanent skill with tests, routing evals, and filing audits. The workflow that was aspirational for months is a real CLI: scaffold the stubs, write the logic, run one check that verifies the whole 10-step checklist. Your OpenClaw stops making the same mistake twice.
|
||||
|
||||
Four new commands and a lot of polish on the existing ones. `gbrain check-resolvable` now works against AGENTS.md workspaces (not just RESOLVER.md ones), so the 107-skill deployment you actually run is finally inspectable. New `gbrain skillify scaffold` creates all the stubs for a new skill in one command. New `gbrain skillpack install` copies gbrain's curated 25-skill bundle into your workspace, managed-block style, never clobbering your local edits. New `gbrain routing-eval` surfaces which user phrasings route to the wrong skill.
|
||||
|
||||
### The numbers that matter
|
||||
|
||||
Measured live against a real OpenClaw deployment with 107 skills, `AGENTS.md` at workspace root, no `manifest.json`:
|
||||
|
||||
| Capability | Before v0.19 | After v0.19 |
|
||||
|---|---|---|
|
||||
| `gbrain check-resolvable` against an AGENTS.md workspace | `RESOLVER.md not found`, exit 2 | detects 102 skills, 15 unreachable errors, 108 advisory warnings |
|
||||
| Unreachable skills surfaced by first run | 0 (check never ran) | 15 (≈15% of the tree was dark) |
|
||||
| `gbrain skillify` as a CLI verb | didn't exist | `scaffold` + `check` subcommands |
|
||||
| `gbrain skillpack install` | didn't exist | 25 curated skills, dependency closure, file-lock + atomic managed block |
|
||||
| `gbrain routing-eval` | didn't exist | structural (default) + `--llm` layer for CI gating |
|
||||
| Warnings break CI by default | yes (any issue → exit 1) | no (warnings advisory; `--strict` opts in) |
|
||||
|
||||
Running `skillify scaffold webhook-verify --description "..." --triggers "..."` writes 4 stub files + appends an idempotent resolver row in under 2 seconds. The real work (your rule, your script, your tests) is what you spend time on afterward — not the boilerplate.
|
||||
|
||||
### What this means for your workflow
|
||||
|
||||
Your agent says "skillify it!" and runs five commands:
|
||||
|
||||
1. `gbrain skillify scaffold <name> --description "..." --triggers "..."` — creates SKILL.md, script stub, routing-eval fixture, test skeleton, resolver row
|
||||
2. Replace the `SKILLIFY_STUB` sentinels with real logic + real tests
|
||||
3. `gbrain skillify check skills/<name>/scripts/<name>.mjs` — 10-item audit
|
||||
4. `gbrain check-resolvable` — reachability + routing + filing + DRY + stub-sentinel gate
|
||||
5. `bun test test/<name>.test.ts`
|
||||
|
||||
Four of the five take under a second. The script stops shipping unless you replace its `SKILLIFY_STUB` marker, so scaffolded-but-forgotten skills can't slip past `check-resolvable --strict`.
|
||||
|
||||
For downstream OpenClaw deployments: `gbrain skillpack install --all` copies the bundled skills into `$OPENCLAW_WORKSPACE`. Per-file diff protection never overwrites your local edits without `--overwrite-local`. The managed block in your AGENTS.md tells you exactly what gbrain installed so you can see it at a glance.
|
||||
|
||||
## To take advantage of v0.19.0
|
||||
|
||||
`gbrain upgrade` does this automatically. To verify:
|
||||
|
||||
1. **Binary version:**
|
||||
```bash
|
||||
gbrain --version # should say 0.19.0
|
||||
```
|
||||
2. **New commands:**
|
||||
```bash
|
||||
gbrain check-resolvable --help | grep -- '--strict'
|
||||
gbrain routing-eval --help
|
||||
gbrain skillify --help
|
||||
gbrain skillpack --help
|
||||
```
|
||||
3. **For AGENTS.md-native OpenClaw deployments:**
|
||||
```bash
|
||||
export OPENCLAW_WORKSPACE=~/your-openclaw/workspace
|
||||
gbrain check-resolvable # human output, warnings advisory
|
||||
gbrain check-resolvable --strict # warnings block CI
|
||||
```
|
||||
4. **For skills you author:** add `routing-eval.jsonl` fixtures and `writes_pages: true` + `writes_to:` frontmatter as you touch each skill. Filing audit is warning-only in v0.19 and escalates to error in v0.20.
|
||||
5. **If anything fails,** file an issue at https://github.com/garrytan/gbrain/issues with the output of `gbrain doctor` and `gbrain check-resolvable --json`.
|
||||
|
||||
No schema migration. Existing brains work unchanged.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
#### Added
|
||||
|
||||
- **`gbrain skillify scaffold <name>`** — creates SKILL.md, script stub, routing-eval fixture, and test skeleton, plus an idempotent trigger row in your resolver. Re-running with `--force` never appends a duplicate row. Every scaffold carries a `SKILLIFY_STUB` sentinel that `check-resolvable --strict` rejects until replaced.
|
||||
- **`gbrain skillify check [path]`** — 10-item post-task audit (promoted from `scripts/skillify-check.ts`; the legacy script remains as a shim).
|
||||
- **`gbrain skillpack list`** — prints the curated bundle (25 skills) shipped with gbrain.
|
||||
- **`gbrain skillpack install <name>` / `--all`** — copies bundled skills into the target workspace. Automatically pulls shared convention files so nothing references a missing dep. Per-file diff protection, `--overwrite-local` escape hatch, `.gbrain-skillpack.lock` against concurrent installers, atomic managed-block update to AGENTS.md / RESOLVER.md.
|
||||
- **`gbrain skillpack diff <name>`** — per-file diff preview before install.
|
||||
- **`gbrain routing-eval`** — dedicated CI verb that runs routing fixtures (`skills/<name>/routing-eval.jsonl`) and surfaces intent-to-skill mismatches, ambiguous routing, and false positives. Default structural layer runs alongside `check-resolvable`; `--llm` opts into an LLM tie-break layer.
|
||||
- **`gbrain check-resolvable --strict`** — opt-in CI mode that promotes warnings to failures.
|
||||
- **`skills/_brain-filing-rules.json`** — machine-readable canonical filing rules (JSON sidecar to the prose `_brain-filing-rules.md`).
|
||||
- **`writes_pages: true` + `writes_to: [...]`** — new skill frontmatter fields consumed by the filing audit. Distinct from `mutating:` so cron schedulers and report writers aren't dragged into filing checks.
|
||||
- **`SKILLIFY_STUB` sentinel check** — new type in `check-resolvable` that flags scaffolded scripts whose stubs haven't been replaced.
|
||||
|
||||
#### Changed
|
||||
|
||||
- **`gbrain check-resolvable` accepts `AGENTS.md` as a resolver file** alongside `RESOLVER.md`, at either the skills directory or one level up (workspace root). Auto-detects via `$OPENCLAW_WORKSPACE`, `~/.openclaw/workspace`, repo root, or `./skills`. Explicit `$OPENCLAW_WORKSPACE` wins over the repo-root walk.
|
||||
- **Auto-derives the skill manifest** by walking `skills/*/SKILL.md` when `manifest.json` is missing. OpenClaw deployments that never shipped a manifest now get real reachability checks instead of silent empty passes.
|
||||
- **`ResolvableReport` split into `errors[]` + `warnings[]`** so advisory findings (filing audit, routing gaps, DRY violations) don't break CI by default. The deprecated `issues[]` union remains for one release.
|
||||
- **`openclaw.plugin.json`** refreshed: version 0.19.0, 25 curated skills, new `shared_deps` declaration for convention files, new `excluded_from_install` for skills that shouldn't be dropped into other workspaces.
|
||||
- **`gbrain skillpack-check`** is now also reachable as `gbrain skillpack check` under the new namespace.
|
||||
- **`scripts/skillify-check.ts`** reduced to a thin shim that delegates to `gbrain skillify check`.
|
||||
|
||||
#### Fixed
|
||||
|
||||
- `check-resolvable` stopped silently passing on workspaces that lacked `manifest.json`. The reachability check now sees every skill on disk.
|
||||
- Parallel `skillpack install` runs no longer race on the AGENTS.md managed block; the file-lock serializes writers and managed-block updates use tmp-file-plus-rename.
|
||||
|
||||
### For contributors
|
||||
|
||||
- New core modules: `src/core/resolver-filenames.ts`, `src/core/skill-manifest.ts`, `src/core/routing-eval.ts`, `src/core/filing-audit.ts`, `src/core/skillify/{templates,generator}.ts`, `src/core/skillpack/{bundle,installer}.ts`.
|
||||
- New command modules: `src/commands/routing-eval.ts`, `src/commands/skillify.ts`, `src/commands/skillify-check.ts`, `src/commands/skillpack.ts`.
|
||||
- `scripts/check-privacy.sh` — pre-commit / CI guard enforcing the private-fork-name ban in public artifacts.
|
||||
- Test fixture at `test/fixtures/openclaw-reference-minimal/` (4 skills + workspace-root AGENTS.md) plus `test/e2e/openclaw-reference-compat.test.ts` exercises the full AGENTS.md + skillpack install stack. `test/regression-v0_16_4.test.ts` locks the pre-v0.19 `checkResolvable` envelope shape. `test/skillpack-sync-guard.test.ts` asserts `openclaw.plugin.json#skills` stays a subset of `skills/manifest.json`.
|
||||
|
||||
---
|
||||
|
||||
## [0.18.2] - 2026-04-23
|
||||
|
||||
## **Migrations survive a crash and Supabase's 2-min ceiling.**
|
||||
@@ -89,6 +191,8 @@ warns about a partial migration:
|
||||
with output of `gbrain doctor` and `~/.gbrain/upgrade-errors.jsonl` (if it
|
||||
exists).
|
||||
|
||||
---
|
||||
|
||||
## [0.18.1] - 2026-04-22
|
||||
|
||||
## **Row Level Security hardening pass.**
|
||||
@@ -165,7 +269,7 @@ If `gbrain doctor` still reports missing RLS after upgrade:
|
||||
- `test/migrate.test.ts` gains a structural guard for the new migration: exists, name matches, BYPASSRLS gating present, LATEST_VERSION has advanced.
|
||||
- **Docs:** new `docs/guides/rls-and-you.md` — one-page explainer covering why RLS matters, what to do when doctor fails, the escape hatch format + rules, auditing exemptions, PGLite behavior, self-hosted Postgres framing.
|
||||
- **Version reconciliation:** `VERSION` and `package.json` land on `0.18.1`.
|
||||
- **CHANGELOG privacy sweep:** replaced a stale `@Wintermute` credit in the 0.17.0 entry with "Garry's OpenClaw" per the [CLAUDE.md privacy rule](CLAUDE.md).
|
||||
- **CHANGELOG privacy sweep:** replaced a stale private-fork credit in the 0.17.0 entry with "Garry's OpenClaw" per the [CLAUDE.md privacy rule](CLAUDE.md).
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
|
||||
@@ -375,8 +479,8 @@ Credit: Garry's OpenClaw for the original `gbrain dream` thesis (PR #309). The b
|
||||
- `CHANGELOG.md` + `CLAUDE.md`: updated.
|
||||
|
||||
**PR #309 disposition**
|
||||
- Closed with credit to @Wintermute. Their thesis ("`gbrain dream` as first-class CLI verb") was right; the implementation got redesigned around the runCycle primitive after deep review surfaced structural issues in the fold approach.
|
||||
- `Co-Authored-By: Wintermute` preserved on commit 5 (the dream.ts rewrite).
|
||||
- Closed with credit to @knee5. Their thesis ("`gbrain dream` as first-class CLI verb") was right; the implementation got redesigned around the runCycle primitive after deep review surfaced structural issues in the fold approach.
|
||||
- `Co-Authored-By` preserved on commit 5 (the dream.ts rewrite).
|
||||
|
||||
---
|
||||
|
||||
@@ -744,7 +848,7 @@ If you connect via `aws-0-REGION.pooler.supabase.com:6543`, do nothing. The upgr
|
||||
- Extended `test/postgres-engine.test.ts` — new source-level grep assertion that the worker-instance `connect({poolSize})` branch calls `db.resolvePrepare(url)` and conditionally includes the `prepare` key in the options literal. Mirrors the existing `SET LOCAL statement_timeout` guardrail in the same file. If anyone rips out the wiring, the build fails before a shipping brain drops rows.
|
||||
|
||||
**Supersedes**
|
||||
- Closes #284 (ours, Wintermute): architecture landed as-is (port-only detection, no hostname expansion). Tests rewritten from vitest to bun:test.
|
||||
- Closes #284 (ours, from the OpenClaw reference deployment): architecture landed as-is (port-only detection, no hostname expansion). Tests rewritten from vitest to bun:test.
|
||||
- Closes #286 (ours, Codex one-liner): dominated; unconditional `prepare: false` would have cost direct-Postgres users plan caching for no reason.
|
||||
- Closes #270 (@notjbg): the critical both-connection-paths insight landed; credit preserved in commit trailer and this CHANGELOG entry.
|
||||
|
||||
|
||||
@@ -296,8 +296,8 @@ stop and remove it before starting a new one.
|
||||
|
||||
## Skills
|
||||
|
||||
Read the skill files in `skills/` before doing brain operations. GBrain ships 26 skills
|
||||
organized by `skills/RESOLVER.md`:
|
||||
Read the skill files in `skills/` before doing brain operations. GBrain ships 28 skills
|
||||
organized by `skills/RESOLVER.md` (`AGENTS.md` is also accepted as of v0.19):
|
||||
|
||||
**Original 8 (conformance-migrated):** ingest (thin router), query, maintain, enrich,
|
||||
briefing, migrate, setup, publish.
|
||||
@@ -308,6 +308,9 @@ meeting-ingestion, citation-fixer, repo-architecture, skill-creator, daily-task-
|
||||
**Operational + identity:** daily-task-prep, cross-modal-review, cron-scheduler, reports,
|
||||
testing, soul-audit, webhook-transforms, data-research, minion-orchestrator.
|
||||
|
||||
**Skillify loop (v0.19):** skillify (the markdown orchestration), skillpack-check
|
||||
(agent-readable health report).
|
||||
|
||||
**Conventions:** `skills/conventions/` has cross-cutting rules (quality, brain-first,
|
||||
model-routing, test-before-bulk, cross-modal). `skills/_brain-filing-rules.md` and
|
||||
`skills/_output-rules.md` are shared references.
|
||||
@@ -376,6 +379,52 @@ Files that MUST be checked on every ship:
|
||||
|
||||
A ship without updated docs is an incomplete ship. Period.
|
||||
|
||||
## CHANGELOG + VERSION are branch-scoped
|
||||
|
||||
**VERSION and CHANGELOG describe what THIS branch adds vs master, not how we got
|
||||
here.** Every feature branch that ships gets its own version bump and CHANGELOG
|
||||
entry. The entry is product release notes for users; it is not a log of internal
|
||||
decisions, review rounds, or codex findings.
|
||||
|
||||
**Write the CHANGELOG entry at /ship time, not during development.** Mid-branch
|
||||
iterations, review rounds (CEO/Eng/Codex/DX), and implementation detours belong
|
||||
in the plan file at `~/.claude/plans/`, not in the CHANGELOG. One unified entry
|
||||
per branch, covering what the branch added vs the base branch.
|
||||
|
||||
**Never edit a CHANGELOG entry that already landed on master.** If master has
|
||||
v0.18.2 and your branch adds features, bump to the next version (v0.19.0, not
|
||||
editing master's v0.18.2). When merging master into your branch, master may
|
||||
bring new CHANGELOG entries above yours — push your entry above master's
|
||||
latest and verify:
|
||||
|
||||
- Does CHANGELOG have your branch's own entry separate from master's entries?
|
||||
- Is VERSION higher than master's VERSION?
|
||||
- Is your entry the topmost `## [X.Y.Z]` entry?
|
||||
- `grep "^## \[" CHANGELOG.md` shows a contiguous version sequence?
|
||||
|
||||
If any answer is no, fix it before continuing.
|
||||
|
||||
**CHANGELOG is for users, not contributors.** Write like product release notes:
|
||||
|
||||
- Lead with what the user can now **do** that they couldn't before. Sell the capability.
|
||||
- Plain language, not implementation details. "You can now..." not "Refactored the..."
|
||||
- **Never mention internal artifacts**: plan file IDs, decision tags (D-CX-#, F-ENG-#),
|
||||
review rounds, codex findings, subcontractor credits. These are invisible to users.
|
||||
- Put contributor-facing changes in a separate `### For contributors` section at the bottom.
|
||||
- Every entry should make someone think "oh nice, I want to try that."
|
||||
|
||||
**What to omit:**
|
||||
- "Codex caught X that the CEO review missed" — private process detail.
|
||||
- "D-CX-3 split errors/warnings" — tag is meaningless to users; name the feature instead.
|
||||
- "Fix-wave PR #N supersedes #M" — supersede chains belong in PR bodies, not release notes.
|
||||
- "215 new cases, 3 decisions applied, 7 reviews cleared" — these are planning-mode metrics.
|
||||
|
||||
**What to keep:**
|
||||
- The user-facing change: what commands exist now, what flag was added, what behavior fixed.
|
||||
- Numbers that mean something to the user: TTHW, commands that timed out before, detection counts.
|
||||
- Upgrade instructions: `gbrain upgrade` + any manual step if needed.
|
||||
- Credit to external contributors when a community PR was incorporated.
|
||||
|
||||
## CHANGELOG voice + release-summary format
|
||||
|
||||
Every version entry in `CHANGELOG.md` MUST start with a release-summary section in
|
||||
|
||||
@@ -6,7 +6,7 @@ Built by the President and CEO of Y Combinator to run his actual AI agents. The
|
||||
|
||||
The brain wires itself. Every page write extracts entity references and creates typed links (`attended`, `works_at`, `invested_in`, `founded`, `advises`) with zero LLM calls. Hybrid search. Self-wiring knowledge graph. Structured timeline. Backlink-boosted ranking. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked end-to-end: **Recall@5 jumps from 83% to 95%, Precision@5 from 39% to 45%, +30 more correct answers in the agent's top-5 reads** on a 240-page Opus-generated rich-prose corpus. Graph-only F1: **86.6% vs grep's 57.8%** (+28.8 pts). [Full report](docs/benchmarks/2026-04-18-brainbench-v1.md).
|
||||
|
||||
GBrain is those patterns, generalized. 26 skills. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours.
|
||||
GBrain is those patterns, generalized. 28 skills. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours.
|
||||
|
||||
> **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
|
||||
|
||||
@@ -28,7 +28,7 @@ Retrieve and follow the instructions at:
|
||||
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md
|
||||
```
|
||||
|
||||
That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 26 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes.
|
||||
That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 28 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes.
|
||||
|
||||
If your agent doesn't auto-read `AGENTS.md`, point it at that file first:
|
||||
`https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` is the non-Claude
|
||||
@@ -87,9 +87,9 @@ claude mcp add gbrain -t http https://your-brain.ngrok.app/mcp -H "Authorization
|
||||
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
|
||||
## The 26 Skills
|
||||
## The 28 Skills
|
||||
|
||||
GBrain ships 26 skills organized by `skills/RESOLVER.md`. The resolver tells your agent which skill to read for any task.
|
||||
GBrain ships 28 skills organized by `skills/RESOLVER.md` (or your OpenClaw's `AGENTS.md` — both filenames are supported as of v0.19). The resolver tells your agent which skill to read for any task.
|
||||
|
||||
[Skill files are code.](https://x.com/garrytan/status/2042925773300908103) They're the most powerful way to get knowledge work done. A skill file is a fat markdown document that encodes an entire workflow: when to fire, what to check, how to chain with other skills, what quality bar to enforce. The agent reads the skill and executes it. Skills can also call deterministic TypeScript code bundled in GBrain (search, import, embed, sync) for the parts that shouldn't be left to LLM judgment. [Thin harness, fat skills](docs/ethos/THIN_HARNESS_FAT_SKILLS.md): the intelligence lives in the skills, not the runtime.
|
||||
|
||||
@@ -133,6 +133,8 @@ GBrain ships 26 skills organized by `skills/RESOLVER.md`. The resolver tells you
|
||||
| **webhook-transforms** | External events (SMS, meetings, social mentions) converted into brain pages with entity extraction. |
|
||||
| **testing** | Validates every skill has SKILL.md with frontmatter, manifest coverage, resolver coverage. |
|
||||
| **skill-creator** | Create new skills following the conformance standard. MECE check against existing skills. |
|
||||
| **skillify** | The "skillify it!" meta-skill. Orchestrates the 10-step loop so failures become durable skills: scaffold the stubs via `gbrain skillify scaffold`, write the real logic, gate with `gbrain skillify check` + `gbrain check-resolvable`. |
|
||||
| **skillpack-check** | Agent-readable gbrain health report. Exit code for CI; JSON for debugging. Cron-friendly. |
|
||||
| **minion-orchestrator** | Long-running agent work as background jobs. Submit, fan out children with depth/cap/timeouts, collect results via child_done inbox. |
|
||||
|
||||
### Identity and setup
|
||||
@@ -249,38 +251,93 @@ gbrain agent logs 1247 --follow --since 5m
|
||||
|
||||
Durability is the point: every Anthropic turn commits to `subagent_messages`, every tool call to `subagent_tool_executions`. Worker kills, OpenClaw crashes, timeouts — all resumable. Host repos (your OpenClaw, etc.) ship their own subagent definitions via `GBRAIN_PLUGIN_PATH` + a `gbrain.plugin.json` manifest: see [`docs/guides/plugin-authors.md`](docs/guides/plugin-authors.md). Requires `ANTHROPIC_API_KEY` on the worker.
|
||||
|
||||
## Skillify: your skills tree stops being a black box
|
||||
## Skillify: say "skillify it!" and the bug becomes structurally impossible to repeat
|
||||
|
||||
Hermes and similar agent frameworks auto-create skills as a background behavior. Fine until you don't know what the agent shipped. Checklists decay. Tests drift. Resolver entries get stale. Six months later you've got an opaque pile of "skills" that nobody has read, nobody has tested, and nobody is sure still work.
|
||||
Your OpenClaw hit a new failure. You fix it once in conversation. You say "skillify it!"
|
||||
And now the fix is permanent: a SKILL.md with triggers, a deterministic script with tests, a
|
||||
routing fixture the agent re-evaluates daily, a filing audit that keeps the output from
|
||||
drifting. Ten items. Every one required. The bug can't recur.
|
||||
|
||||
GBrain ships the same capability. Except the human stays in the loop.
|
||||
Hermes and similar agent frameworks auto-create skills as a background behavior. Fine until
|
||||
you don't know what the agent shipped. Checklists decay. Tests drift. Resolver entries get
|
||||
stale. Six months later it's an opaque pile nobody has read, nobody has tested, and nobody
|
||||
is sure still works. GBrain ships the same capability except the human stays in the loop
|
||||
and every step is a command you can run.
|
||||
|
||||
- **`/skillify`** turns raw code into a properly-skilled feature: SKILL.md + deterministic script + unit tests + integration tests + LLM evals + resolver trigger + resolver trigger eval + E2E smoke + brain filing. Ten items. Every one required.
|
||||
- **`gbrain check-resolvable`** walks the whole skills tree: reachability, MECE overlap, DRY violations, gap detection, orphaned skills. Exits non-zero if anything is off.
|
||||
- **`scripts/skillify-check.ts`** — machine-readable audit. `--json` for CI, `--recent` for last-7-days files.
|
||||
|
||||
You decide when and what. The tooling keeps the checklist honest.
|
||||
|
||||
### Why this is the right answer for OpenClaw
|
||||
|
||||
Auto-generated skills are a liability the first time a behavior breaks. Was it the skill? The test? The resolver trigger? The eval? You don't know, because you never read it. Debugging a black box is pure guesswork.
|
||||
|
||||
Skillify makes the black box legible. Every skill in your tree has: a contract (SKILL.md), tests that exercise that contract, an eval that grades LLM output against a rubric, a resolver trigger the user actually types, and a test that confirms the trigger routes right. If something breaks, you know which layer to look at. If anything goes stale, `check-resolvable` says so.
|
||||
|
||||
In practice this combo produces **zero orphaned skills, every feature with tests + evals + resolver triggers + evals of the triggers.** Compounding quality instead of compounding entropy.
|
||||
### The four verbs you need (v0.19)
|
||||
|
||||
```bash
|
||||
# Audit a feature's skill completeness (10-item checklist)
|
||||
bun run scripts/skillify-check.ts src/commands/publish.ts
|
||||
# 1. Scaffold all 5 stub files for a new skill in one shot.
|
||||
gbrain skillify scaffold webhook-verify \
|
||||
--description "verify ngrok webhooks" \
|
||||
--triggers "verify the webhook,check tunnel" \
|
||||
--writes-pages --writes-to people/,companies/
|
||||
|
||||
# In CI: fail the build when a new feature isn't properly skilled
|
||||
bun run scripts/skillify-check.ts --json --recent
|
||||
# 2. Replace the SKILLIFY_STUB sentinels with real logic + real tests.
|
||||
$EDITOR skills/webhook-verify/scripts/webhook-verify.mjs
|
||||
$EDITOR test/webhook-verify.test.ts
|
||||
|
||||
# Validate the whole skills tree before shipping
|
||||
gbrain check-resolvable
|
||||
# 3. Run the 10-item audit: SKILL.md exists, script exists, unit + E2E tests,
|
||||
# LLM evals, resolver entry, trigger eval, check-resolvable gate, brain filing.
|
||||
gbrain skillify check skills/webhook-verify/scripts/webhook-verify.mjs
|
||||
|
||||
# 4. Verify the whole tree: reachability, MECE overlap, DRY, routing gaps,
|
||||
# filing audit, SKILLIFY_STUB sentinels (fails if any skill still has one).
|
||||
gbrain check-resolvable # warnings advisory, errors block
|
||||
gbrain check-resolvable --strict # warnings block too (CI opt-in)
|
||||
```
|
||||
|
||||
**Skillify is not a nice-to-have. It's the piece that makes the skills tree survive six months of compounding work.** Read [`skills/skillify/SKILL.md`](skills/skillify/SKILL.md) for the full 10-item checklist and the anti-patterns it catches.
|
||||
Idempotent re-runs. `--force` regenerates stub files but NEVER duplicates a resolver row.
|
||||
Scaffold completes in under 2 seconds. The real work (your rule, your script, your tests)
|
||||
is what you spend time on. Everything else is boilerplate the CLI writes for you.
|
||||
|
||||
### `gbrain routing-eval` — catch the routing gaps your users actually hit
|
||||
|
||||
Drop a `routing-eval.jsonl` fixture next to any skill. Each line is `{intent, expected_skill,
|
||||
ambiguous_with?}`. `gbrain check-resolvable` runs the structural layer by default; `gbrain
|
||||
routing-eval --llm` runs an LLM tie-break layer for CI. False positives (wrong skill matched),
|
||||
missed routes (no skill matched), and tautological fixtures (intent copies trigger verbatim)
|
||||
all surface as specific advisories with the exact file:line to fix.
|
||||
|
||||
### Works on your OpenClaw, not just gbrain's repo
|
||||
|
||||
v0.19 teaches `gbrain check-resolvable` to accept `AGENTS.md` as a resolver file alongside
|
||||
`RESOLVER.md`, at either the skills directory OR one level up (OpenClaw-native workspace-root
|
||||
layout). The skill manifest auto-derives from walking `skills/*/SKILL.md` when `manifest.json`
|
||||
is missing. Set `OPENCLAW_WORKSPACE=~/your-openclaw/workspace` and everything just works:
|
||||
|
||||
```bash
|
||||
export OPENCLAW_WORKSPACE=~/your-openclaw/workspace
|
||||
gbrain check-resolvable --verbose
|
||||
# Auto-detects: AGENTS.md at workspace root, 107 skills derived from SKILL.md walk,
|
||||
# 15 unreachable errors surfaced, 108 advisory warnings for overlaps and gaps.
|
||||
```
|
||||
|
||||
First run on a real OpenClaw deployment found 15 unreachable skills out of 102 — about 15%
|
||||
of the tree was dark. The essay's "skills the agent can never reach" footgun, now visible.
|
||||
|
||||
### `gbrain skillpack install` — drop 25 curated skills into your OpenClaw
|
||||
|
||||
The skills gbrain ships are a curated bundle. Install them into your workspace with
|
||||
dependency closure (shared conventions come along), per-file diff protection (your local
|
||||
edits are never clobbered without `--overwrite-local`), a file lock that serializes
|
||||
concurrent installers, and an atomic managed-block update to your AGENTS.md so you can
|
||||
see exactly what gbrain wrote.
|
||||
|
||||
```bash
|
||||
gbrain skillpack list # 25 curated skills
|
||||
gbrain skillpack install brain-ops # one skill + its shared conventions
|
||||
gbrain skillpack install --all # the full bundle
|
||||
gbrain skillpack install brain-ops --dry-run # preview; no writes
|
||||
gbrain skillpack diff brain-ops # compare bundle vs your local copy
|
||||
```
|
||||
|
||||
Re-running is safe. The managed-block markers in your AGENTS.md let `skillpack install`
|
||||
accumulate rows across separate single-skill installs instead of overwriting each other.
|
||||
|
||||
**Skillify is the piece that makes the skills tree survive six months of compounding work.**
|
||||
Read [`skills/skillify/SKILL.md`](skills/skillify/SKILL.md) for the full 10-item checklist
|
||||
and the anti-patterns it catches.
|
||||
|
||||
## Getting Data In
|
||||
|
||||
@@ -317,7 +374,7 @@ Run `gbrain integrations` to see status.
|
||||
│ Brain Repo │ │ GBrain │ │ AI Agent │
|
||||
│ (git) │ │ (retrieval) │ │ (read/write) │
|
||||
│ │ │ │ │ │
|
||||
│ markdown files │───>│ Postgres + │<──>│ 26 skills │
|
||||
│ markdown files │───>│ Postgres + │<──>│ 28 skills │
|
||||
│ = source of │ │ pgvector │ │ define HOW to │
|
||||
│ truth │ │ │ │ use the brain │
|
||||
│ │<───│ hybrid │ │ │
|
||||
@@ -561,12 +618,26 @@ JOBS (Minions)
|
||||
gbrain jobs smoke One-command health check
|
||||
gbrain jobs work [--queue Q] [--concurrency N] Start worker daemon
|
||||
|
||||
SKILLS (v0.19)
|
||||
gbrain skillify scaffold <name> Create 5 stub files + idempotent resolver row
|
||||
gbrain skillify check [path] 10-item audit of a skill
|
||||
gbrain skillpack list Print the 25 curated skills in the bundle
|
||||
gbrain skillpack install <name> Copy one skill + its shared conventions into target
|
||||
gbrain skillpack install --all Install the full curated bundle
|
||||
gbrain skillpack diff <name> Per-file diff: bundle vs target workspace
|
||||
gbrain check-resolvable [--strict] Resolver audit (reachability, MECE, DRY, routing, filing,
|
||||
SKILLIFY_STUB). Accepts RESOLVER.md OR AGENTS.md.
|
||||
gbrain routing-eval [--llm] [--json] Intent→skill routing accuracy on fixtures
|
||||
|
||||
ADMIN
|
||||
gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings)
|
||||
gbrain doctor --fix [--dry-run] Auto-fix DRY violations (delegate inlined rules to conventions)
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
gbrain check-backlinks check|fix Back-link enforcement
|
||||
gbrain lint [--fix] LLM artifact detection
|
||||
gbrain repair-jsonb [--dry-run] Repair v0.12.0 double-encoded JSONB (Postgres)
|
||||
@@ -590,7 +661,7 @@ The skills in this repo are those patterns, generalized. What took 11 days to bu
|
||||
|
||||
**For agents:**
|
||||
- **[skills/RESOLVER.md](skills/RESOLVER.md)** ... Start here. The skill dispatcher.
|
||||
- [Individual skill files](skills/) ... 25 standalone instruction sets
|
||||
- [Individual skill files](skills/) ... 28 standalone instruction sets (25 ship in the curated `gbrain skillpack install` bundle)
|
||||
- [GBRAIN_SKILLPACK.md](docs/GBRAIN_SKILLPACK.md) ... Legacy reference architecture
|
||||
- [Getting Data In](docs/integrations/README.md) ... Integration recipes and data flow
|
||||
- [GBRAIN_VERIFY.md](docs/GBRAIN_VERIFY.md) ... Installation verification
|
||||
|
||||
@@ -1,20 +1,15 @@
|
||||
# TODOS
|
||||
|
||||
## check-resolvable
|
||||
## Completed
|
||||
|
||||
### File tracking issues for Checks 5 + 6 (deferred in PR #325)
|
||||
**Priority:** P2
|
||||
### ~~Checks 5 + 6 for check-resolvable~~
|
||||
**Completed:** v0.19.0 (2026-04-22)
|
||||
|
||||
**What:** `src/commands/check-resolvable.ts` currently points `DEFERRED[].issue` at GitHub issue search URLs (`?q=TBD-check-5`, `?q=TBD-check-6`). File real tracking issues and grep-replace both placeholders with the real URLs.
|
||||
Both checks shipped as real implementations, not just filed issues:
|
||||
- **Check 5 (trigger routing eval):** `src/core/routing-eval.ts` + `gbrain routing-eval` CLI. Structural layer runs in `check-resolvable` by default; `--llm` opts into LLM tie-break. Fixtures live at `skills/<name>/routing-eval.jsonl`.
|
||||
- **Check 6 (brain filing):** `src/core/filing-audit.ts` + `skills/_brain-filing-rules.json`. New `writes_pages:` + `writes_to:` frontmatter. Warning-only in v0.19, error in v0.20.
|
||||
|
||||
**Why:** v0.16.4 shipped `gbrain check-resolvable` with 4 of the 6 checks from the original spec. Checks 5 (trigger routing eval) and 6 (brain filing) were explicitly deferred during plan-ceo-review because they each need new detection logic. The CLI's `deferred[]` JSON field is meant to surface these to agents so they know the coverage boundary — the TBD placeholders do the right thing mechanically but aren't clickable.
|
||||
|
||||
**How:**
|
||||
1. `gh issue create -t "check-resolvable Check 5: trigger routing eval" -b "..."` — detection: every skill's own frontmatter trigger should match the RESOLVER.md entry pointing at that skill. Needs new issue type (e.g. `mis_route`).
|
||||
2. `gh issue create -t "check-resolvable Check 6: brain filing validation" -b "..."` — detection: scan SKILL.md body for brain paths (e.g., `brain/people/`, `brain/companies/`), cross-reference with `skills/_brain-filing-rules.md`. Flag mutating skills missing entries.
|
||||
3. Replace `TBD-check-5` and `TBD-check-6` in `src/commands/check-resolvable.ts` with the real issue URLs.
|
||||
|
||||
**Effort:** ~15 min mechanical (issue filing + grep-replace). Implementation of the checks themselves is a separate, larger piece of work — the TODO here is just the issue filing + URL swap.
|
||||
`DEFERRED[]` in `src/commands/check-resolvable.ts` is now empty — v0.19 shipped both deferred checks as working code paths, not as issue URLs. The export stays in place for future deferred checks.
|
||||
|
||||
## P1 (BrainBench v1.1 — categories deferred from PR #188)
|
||||
|
||||
|
||||
+151
-31
@@ -375,8 +375,8 @@ stop and remove it before starting a new one.
|
||||
|
||||
## Skills
|
||||
|
||||
Read the skill files in `skills/` before doing brain operations. GBrain ships 26 skills
|
||||
organized by `skills/RESOLVER.md`:
|
||||
Read the skill files in `skills/` before doing brain operations. GBrain ships 28 skills
|
||||
organized by `skills/RESOLVER.md` (`AGENTS.md` is also accepted as of v0.19):
|
||||
|
||||
**Original 8 (conformance-migrated):** ingest (thin router), query, maintain, enrich,
|
||||
briefing, migrate, setup, publish.
|
||||
@@ -387,6 +387,9 @@ meeting-ingestion, citation-fixer, repo-architecture, skill-creator, daily-task-
|
||||
**Operational + identity:** daily-task-prep, cross-modal-review, cron-scheduler, reports,
|
||||
testing, soul-audit, webhook-transforms, data-research, minion-orchestrator.
|
||||
|
||||
**Skillify loop (v0.19):** skillify (the markdown orchestration), skillpack-check
|
||||
(agent-readable health report).
|
||||
|
||||
**Conventions:** `skills/conventions/` has cross-cutting rules (quality, brain-first,
|
||||
model-routing, test-before-bulk, cross-modal). `skills/_brain-filing-rules.md` and
|
||||
`skills/_output-rules.md` are shared references.
|
||||
@@ -455,6 +458,52 @@ Files that MUST be checked on every ship:
|
||||
|
||||
A ship without updated docs is an incomplete ship. Period.
|
||||
|
||||
## CHANGELOG + VERSION are branch-scoped
|
||||
|
||||
**VERSION and CHANGELOG describe what THIS branch adds vs master, not how we got
|
||||
here.** Every feature branch that ships gets its own version bump and CHANGELOG
|
||||
entry. The entry is product release notes for users; it is not a log of internal
|
||||
decisions, review rounds, or codex findings.
|
||||
|
||||
**Write the CHANGELOG entry at /ship time, not during development.** Mid-branch
|
||||
iterations, review rounds (CEO/Eng/Codex/DX), and implementation detours belong
|
||||
in the plan file at `~/.claude/plans/`, not in the CHANGELOG. One unified entry
|
||||
per branch, covering what the branch added vs the base branch.
|
||||
|
||||
**Never edit a CHANGELOG entry that already landed on master.** If master has
|
||||
v0.18.2 and your branch adds features, bump to the next version (v0.19.0, not
|
||||
editing master's v0.18.2). When merging master into your branch, master may
|
||||
bring new CHANGELOG entries above yours — push your entry above master's
|
||||
latest and verify:
|
||||
|
||||
- Does CHANGELOG have your branch's own entry separate from master's entries?
|
||||
- Is VERSION higher than master's VERSION?
|
||||
- Is your entry the topmost `## [X.Y.Z]` entry?
|
||||
- `grep "^## \[" CHANGELOG.md` shows a contiguous version sequence?
|
||||
|
||||
If any answer is no, fix it before continuing.
|
||||
|
||||
**CHANGELOG is for users, not contributors.** Write like product release notes:
|
||||
|
||||
- Lead with what the user can now **do** that they couldn't before. Sell the capability.
|
||||
- Plain language, not implementation details. "You can now..." not "Refactored the..."
|
||||
- **Never mention internal artifacts**: plan file IDs, decision tags (D-CX-#, F-ENG-#),
|
||||
review rounds, codex findings, subcontractor credits. These are invisible to users.
|
||||
- Put contributor-facing changes in a separate `### For contributors` section at the bottom.
|
||||
- Every entry should make someone think "oh nice, I want to try that."
|
||||
|
||||
**What to omit:**
|
||||
- "Codex caught X that the CEO review missed" — private process detail.
|
||||
- "D-CX-3 split errors/warnings" — tag is meaningless to users; name the feature instead.
|
||||
- "Fix-wave PR #N supersedes #M" — supersede chains belong in PR bodies, not release notes.
|
||||
- "215 new cases, 3 decisions applied, 7 reviews cleared" — these are planning-mode metrics.
|
||||
|
||||
**What to keep:**
|
||||
- The user-facing change: what commands exist now, what flag was added, what behavior fixed.
|
||||
- Numbers that mean something to the user: TTHW, commands that timed out before, detection counts.
|
||||
- Upgrade instructions: `gbrain upgrade` + any manual step if needed.
|
||||
- Credit to external contributors when a community PR was incorporated.
|
||||
|
||||
## CHANGELOG voice + release-summary format
|
||||
|
||||
Every version entry in `CHANGELOG.md` MUST start with a release-summary section in
|
||||
@@ -1117,7 +1166,7 @@ Built by the President and CEO of Y Combinator to run his actual AI agents. The
|
||||
|
||||
The brain wires itself. Every page write extracts entity references and creates typed links (`attended`, `works_at`, `invested_in`, `founded`, `advises`) with zero LLM calls. Hybrid search. Self-wiring knowledge graph. Structured timeline. Backlink-boosted ranking. Ask "who works at Acme AI?" or "what did Bob invest in this quarter?" and get answers vector search alone can't reach. Benchmarked end-to-end: **Recall@5 jumps from 83% to 95%, Precision@5 from 39% to 45%, +30 more correct answers in the agent's top-5 reads** on a 240-page Opus-generated rich-prose corpus. Graph-only F1: **86.6% vs grep's 57.8%** (+28.8 pts). [Full report](docs/benchmarks/2026-04-18-brainbench-v1.md).
|
||||
|
||||
GBrain is those patterns, generalized. 26 skills. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours.
|
||||
GBrain is those patterns, generalized. 28 skills. Install in 30 minutes. Your agent does the work. As Garry's personal agent gets smarter, so does yours.
|
||||
|
||||
> **~30 minutes to a fully working brain.** Database ready in 2 seconds (PGLite, no server). You just answer questions about API keys.
|
||||
|
||||
@@ -1139,7 +1188,7 @@ Retrieve and follow the instructions at:
|
||||
https://raw.githubusercontent.com/garrytan/gbrain/master/INSTALL_FOR_AGENTS.md
|
||||
```
|
||||
|
||||
That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 26 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes.
|
||||
That's it. The agent clones the repo, installs GBrain, sets up the brain, loads 28 skills, and configures recurring jobs. You answer a few questions about API keys. ~30 minutes.
|
||||
|
||||
If your agent doesn't auto-read `AGENTS.md`, point it at that file first:
|
||||
`https://raw.githubusercontent.com/garrytan/gbrain/master/AGENTS.md` is the non-Claude
|
||||
@@ -1198,9 +1247,9 @@ claude mcp add gbrain -t http https://your-brain.ngrok.app/mcp -H "Authorization
|
||||
|
||||
Per-client guides: [`docs/mcp/`](docs/mcp/DEPLOY.md). ChatGPT requires OAuth 2.1 (not yet implemented).
|
||||
|
||||
## The 26 Skills
|
||||
## The 28 Skills
|
||||
|
||||
GBrain ships 26 skills organized by `skills/RESOLVER.md`. The resolver tells your agent which skill to read for any task.
|
||||
GBrain ships 28 skills organized by `skills/RESOLVER.md` (or your OpenClaw's `AGENTS.md` — both filenames are supported as of v0.19). The resolver tells your agent which skill to read for any task.
|
||||
|
||||
[Skill files are code.](https://x.com/garrytan/status/2042925773300908103) They're the most powerful way to get knowledge work done. A skill file is a fat markdown document that encodes an entire workflow: when to fire, what to check, how to chain with other skills, what quality bar to enforce. The agent reads the skill and executes it. Skills can also call deterministic TypeScript code bundled in GBrain (search, import, embed, sync) for the parts that shouldn't be left to LLM judgment. [Thin harness, fat skills](docs/ethos/THIN_HARNESS_FAT_SKILLS.md): the intelligence lives in the skills, not the runtime.
|
||||
|
||||
@@ -1244,6 +1293,8 @@ GBrain ships 26 skills organized by `skills/RESOLVER.md`. The resolver tells you
|
||||
| **webhook-transforms** | External events (SMS, meetings, social mentions) converted into brain pages with entity extraction. |
|
||||
| **testing** | Validates every skill has SKILL.md with frontmatter, manifest coverage, resolver coverage. |
|
||||
| **skill-creator** | Create new skills following the conformance standard. MECE check against existing skills. |
|
||||
| **skillify** | The "skillify it!" meta-skill. Orchestrates the 10-step loop so failures become durable skills: scaffold the stubs via `gbrain skillify scaffold`, write the real logic, gate with `gbrain skillify check` + `gbrain check-resolvable`. |
|
||||
| **skillpack-check** | Agent-readable gbrain health report. Exit code for CI; JSON for debugging. Cron-friendly. |
|
||||
| **minion-orchestrator** | Long-running agent work as background jobs. Submit, fan out children with depth/cap/timeouts, collect results via child_done inbox. |
|
||||
|
||||
### Identity and setup
|
||||
@@ -1360,38 +1411,93 @@ gbrain agent logs 1247 --follow --since 5m
|
||||
|
||||
Durability is the point: every Anthropic turn commits to `subagent_messages`, every tool call to `subagent_tool_executions`. Worker kills, OpenClaw crashes, timeouts — all resumable. Host repos (your OpenClaw, etc.) ship their own subagent definitions via `GBRAIN_PLUGIN_PATH` + a `gbrain.plugin.json` manifest: see [`docs/guides/plugin-authors.md`](docs/guides/plugin-authors.md). Requires `ANTHROPIC_API_KEY` on the worker.
|
||||
|
||||
## Skillify: your skills tree stops being a black box
|
||||
## Skillify: say "skillify it!" and the bug becomes structurally impossible to repeat
|
||||
|
||||
Hermes and similar agent frameworks auto-create skills as a background behavior. Fine until you don't know what the agent shipped. Checklists decay. Tests drift. Resolver entries get stale. Six months later you've got an opaque pile of "skills" that nobody has read, nobody has tested, and nobody is sure still work.
|
||||
Your OpenClaw hit a new failure. You fix it once in conversation. You say "skillify it!"
|
||||
And now the fix is permanent: a SKILL.md with triggers, a deterministic script with tests, a
|
||||
routing fixture the agent re-evaluates daily, a filing audit that keeps the output from
|
||||
drifting. Ten items. Every one required. The bug can't recur.
|
||||
|
||||
GBrain ships the same capability. Except the human stays in the loop.
|
||||
Hermes and similar agent frameworks auto-create skills as a background behavior. Fine until
|
||||
you don't know what the agent shipped. Checklists decay. Tests drift. Resolver entries get
|
||||
stale. Six months later it's an opaque pile nobody has read, nobody has tested, and nobody
|
||||
is sure still works. GBrain ships the same capability except the human stays in the loop
|
||||
and every step is a command you can run.
|
||||
|
||||
- **`/skillify`** turns raw code into a properly-skilled feature: SKILL.md + deterministic script + unit tests + integration tests + LLM evals + resolver trigger + resolver trigger eval + E2E smoke + brain filing. Ten items. Every one required.
|
||||
- **`gbrain check-resolvable`** walks the whole skills tree: reachability, MECE overlap, DRY violations, gap detection, orphaned skills. Exits non-zero if anything is off.
|
||||
- **`scripts/skillify-check.ts`** — machine-readable audit. `--json` for CI, `--recent` for last-7-days files.
|
||||
|
||||
You decide when and what. The tooling keeps the checklist honest.
|
||||
|
||||
### Why this is the right answer for OpenClaw
|
||||
|
||||
Auto-generated skills are a liability the first time a behavior breaks. Was it the skill? The test? The resolver trigger? The eval? You don't know, because you never read it. Debugging a black box is pure guesswork.
|
||||
|
||||
Skillify makes the black box legible. Every skill in your tree has: a contract (SKILL.md), tests that exercise that contract, an eval that grades LLM output against a rubric, a resolver trigger the user actually types, and a test that confirms the trigger routes right. If something breaks, you know which layer to look at. If anything goes stale, `check-resolvable` says so.
|
||||
|
||||
In practice this combo produces **zero orphaned skills, every feature with tests + evals + resolver triggers + evals of the triggers.** Compounding quality instead of compounding entropy.
|
||||
### The four verbs you need (v0.19)
|
||||
|
||||
```bash
|
||||
# Audit a feature's skill completeness (10-item checklist)
|
||||
bun run scripts/skillify-check.ts src/commands/publish.ts
|
||||
# 1. Scaffold all 5 stub files for a new skill in one shot.
|
||||
gbrain skillify scaffold webhook-verify \
|
||||
--description "verify ngrok webhooks" \
|
||||
--triggers "verify the webhook,check tunnel" \
|
||||
--writes-pages --writes-to people/,companies/
|
||||
|
||||
# In CI: fail the build when a new feature isn't properly skilled
|
||||
bun run scripts/skillify-check.ts --json --recent
|
||||
# 2. Replace the SKILLIFY_STUB sentinels with real logic + real tests.
|
||||
$EDITOR skills/webhook-verify/scripts/webhook-verify.mjs
|
||||
$EDITOR test/webhook-verify.test.ts
|
||||
|
||||
# Validate the whole skills tree before shipping
|
||||
gbrain check-resolvable
|
||||
# 3. Run the 10-item audit: SKILL.md exists, script exists, unit + E2E tests,
|
||||
# LLM evals, resolver entry, trigger eval, check-resolvable gate, brain filing.
|
||||
gbrain skillify check skills/webhook-verify/scripts/webhook-verify.mjs
|
||||
|
||||
# 4. Verify the whole tree: reachability, MECE overlap, DRY, routing gaps,
|
||||
# filing audit, SKILLIFY_STUB sentinels (fails if any skill still has one).
|
||||
gbrain check-resolvable # warnings advisory, errors block
|
||||
gbrain check-resolvable --strict # warnings block too (CI opt-in)
|
||||
```
|
||||
|
||||
**Skillify is not a nice-to-have. It's the piece that makes the skills tree survive six months of compounding work.** Read [`skills/skillify/SKILL.md`](skills/skillify/SKILL.md) for the full 10-item checklist and the anti-patterns it catches.
|
||||
Idempotent re-runs. `--force` regenerates stub files but NEVER duplicates a resolver row.
|
||||
Scaffold completes in under 2 seconds. The real work (your rule, your script, your tests)
|
||||
is what you spend time on. Everything else is boilerplate the CLI writes for you.
|
||||
|
||||
### `gbrain routing-eval` — catch the routing gaps your users actually hit
|
||||
|
||||
Drop a `routing-eval.jsonl` fixture next to any skill. Each line is `{intent, expected_skill,
|
||||
ambiguous_with?}`. `gbrain check-resolvable` runs the structural layer by default; `gbrain
|
||||
routing-eval --llm` runs an LLM tie-break layer for CI. False positives (wrong skill matched),
|
||||
missed routes (no skill matched), and tautological fixtures (intent copies trigger verbatim)
|
||||
all surface as specific advisories with the exact file:line to fix.
|
||||
|
||||
### Works on your OpenClaw, not just gbrain's repo
|
||||
|
||||
v0.19 teaches `gbrain check-resolvable` to accept `AGENTS.md` as a resolver file alongside
|
||||
`RESOLVER.md`, at either the skills directory OR one level up (OpenClaw-native workspace-root
|
||||
layout). The skill manifest auto-derives from walking `skills/*/SKILL.md` when `manifest.json`
|
||||
is missing. Set `OPENCLAW_WORKSPACE=~/your-openclaw/workspace` and everything just works:
|
||||
|
||||
```bash
|
||||
export OPENCLAW_WORKSPACE=~/your-openclaw/workspace
|
||||
gbrain check-resolvable --verbose
|
||||
# Auto-detects: AGENTS.md at workspace root, 107 skills derived from SKILL.md walk,
|
||||
# 15 unreachable errors surfaced, 108 advisory warnings for overlaps and gaps.
|
||||
```
|
||||
|
||||
First run on a real OpenClaw deployment found 15 unreachable skills out of 102 — about 15%
|
||||
of the tree was dark. The essay's "skills the agent can never reach" footgun, now visible.
|
||||
|
||||
### `gbrain skillpack install` — drop 25 curated skills into your OpenClaw
|
||||
|
||||
The skills gbrain ships are a curated bundle. Install them into your workspace with
|
||||
dependency closure (shared conventions come along), per-file diff protection (your local
|
||||
edits are never clobbered without `--overwrite-local`), a file lock that serializes
|
||||
concurrent installers, and an atomic managed-block update to your AGENTS.md so you can
|
||||
see exactly what gbrain wrote.
|
||||
|
||||
```bash
|
||||
gbrain skillpack list # 25 curated skills
|
||||
gbrain skillpack install brain-ops # one skill + its shared conventions
|
||||
gbrain skillpack install --all # the full bundle
|
||||
gbrain skillpack install brain-ops --dry-run # preview; no writes
|
||||
gbrain skillpack diff brain-ops # compare bundle vs your local copy
|
||||
```
|
||||
|
||||
Re-running is safe. The managed-block markers in your AGENTS.md let `skillpack install`
|
||||
accumulate rows across separate single-skill installs instead of overwriting each other.
|
||||
|
||||
**Skillify is the piece that makes the skills tree survive six months of compounding work.**
|
||||
Read [`skills/skillify/SKILL.md`](skills/skillify/SKILL.md) for the full 10-item checklist
|
||||
and the anti-patterns it catches.
|
||||
|
||||
## Getting Data In
|
||||
|
||||
@@ -1428,7 +1534,7 @@ Run `gbrain integrations` to see status.
|
||||
│ Brain Repo │ │ GBrain │ │ AI Agent │
|
||||
│ (git) │ │ (retrieval) │ │ (read/write) │
|
||||
│ │ │ │ │ │
|
||||
│ markdown files │───>│ Postgres + │<──>│ 26 skills │
|
||||
│ markdown files │───>│ Postgres + │<──>│ 28 skills │
|
||||
│ = source of │ │ pgvector │ │ define HOW to │
|
||||
│ truth │ │ │ │ use the brain │
|
||||
│ │<───│ hybrid │ │ │
|
||||
@@ -1672,12 +1778,26 @@ JOBS (Minions)
|
||||
gbrain jobs smoke One-command health check
|
||||
gbrain jobs work [--queue Q] [--concurrency N] Start worker daemon
|
||||
|
||||
SKILLS (v0.19)
|
||||
gbrain skillify scaffold <name> Create 5 stub files + idempotent resolver row
|
||||
gbrain skillify check [path] 10-item audit of a skill
|
||||
gbrain skillpack list Print the 25 curated skills in the bundle
|
||||
gbrain skillpack install <name> Copy one skill + its shared conventions into target
|
||||
gbrain skillpack install --all Install the full curated bundle
|
||||
gbrain skillpack diff <name> Per-file diff: bundle vs target workspace
|
||||
gbrain check-resolvable [--strict] Resolver audit (reachability, MECE, DRY, routing, filing,
|
||||
SKILLIFY_STUB). Accepts RESOLVER.md OR AGENTS.md.
|
||||
gbrain routing-eval [--llm] [--json] Intent→skill routing accuracy on fixtures
|
||||
|
||||
ADMIN
|
||||
gbrain doctor [--json] [--fast] Health checks (resolver, skills, DB, embeddings)
|
||||
gbrain doctor --fix [--dry-run] Auto-fix DRY violations (delegate inlined rules to conventions)
|
||||
gbrain doctor --locks List idle-in-tx backends (57014 diagnostic, Postgres only)
|
||||
gbrain stats Brain statistics
|
||||
gbrain serve MCP server (stdio)
|
||||
gbrain integrations Integration recipe dashboard
|
||||
gbrain sources list|add|remove|... Multi-source brain management (v0.18)
|
||||
gbrain dream [--dry-run] [--phase N] One maintenance cycle then exit (cron-friendly)
|
||||
gbrain check-backlinks check|fix Back-link enforcement
|
||||
gbrain lint [--fix] LLM artifact detection
|
||||
gbrain repair-jsonb [--dry-run] Repair v0.12.0 double-encoded JSONB (Postgres)
|
||||
@@ -1701,7 +1821,7 @@ The skills in this repo are those patterns, generalized. What took 11 days to bu
|
||||
|
||||
**For agents:**
|
||||
- **[skills/RESOLVER.md](skills/RESOLVER.md)** ... Start here. The skill dispatcher.
|
||||
- [Individual skill files](skills/) ... 25 standalone instruction sets
|
||||
- [Individual skill files](skills/) ... 28 standalone instruction sets (25 ship in the curated `gbrain skillpack install` bundle)
|
||||
- [GBRAIN_SKILLPACK.md](docs/GBRAIN_SKILLPACK.md) ... Legacy reference architecture
|
||||
- [Getting Data In](docs/integrations/README.md) ... Integration recipes and data flow
|
||||
- [GBRAIN_VERIFY.md](docs/GBRAIN_VERIFY.md) ... Installation verification
|
||||
|
||||
+35
-6
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.4.1",
|
||||
"version": "0.19.0",
|
||||
"description": "Personal knowledge brain with Postgres + pgvector hybrid search",
|
||||
"family": "bundle-plugin",
|
||||
"configSchema": {
|
||||
@@ -24,13 +24,42 @@
|
||||
}
|
||||
},
|
||||
"skills": [
|
||||
"skills/ingest",
|
||||
"skills/query",
|
||||
"skills/maintain",
|
||||
"skills/enrich",
|
||||
"skills/brain-ops",
|
||||
"skills/briefing",
|
||||
"skills/citation-fixer",
|
||||
"skills/cross-modal-review",
|
||||
"skills/cron-scheduler",
|
||||
"skills/daily-task-manager",
|
||||
"skills/daily-task-prep",
|
||||
"skills/data-research",
|
||||
"skills/enrich",
|
||||
"skills/idea-ingest",
|
||||
"skills/ingest",
|
||||
"skills/maintain",
|
||||
"skills/media-ingest",
|
||||
"skills/meeting-ingestion",
|
||||
"skills/minion-orchestrator",
|
||||
"skills/query",
|
||||
"skills/reports",
|
||||
"skills/repo-architecture",
|
||||
"skills/signal-detector",
|
||||
"skills/skill-creator",
|
||||
"skills/skillify",
|
||||
"skills/skillpack-check",
|
||||
"skills/soul-audit",
|
||||
"skills/testing",
|
||||
"skills/webhook-transforms"
|
||||
],
|
||||
"shared_deps": [
|
||||
"skills/conventions",
|
||||
"skills/_brain-filing-rules.md",
|
||||
"skills/_brain-filing-rules.json",
|
||||
"skills/_output-rules.md"
|
||||
],
|
||||
"excluded_from_install": [
|
||||
"skills/setup",
|
||||
"skills/migrate",
|
||||
"skills/setup"
|
||||
"skills/publish"
|
||||
],
|
||||
"openclaw": {
|
||||
"compat": {
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.18.2",
|
||||
"version": "0.19.0",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
Executable
+133
@@ -0,0 +1,133 @@
|
||||
#!/bin/bash
|
||||
#
|
||||
# check-privacy.sh — CLAUDE.md:550 enforcement.
|
||||
#
|
||||
# CLAUDE.md forbids the private OpenClaw fork name in public artifacts:
|
||||
# CHANGELOG.md, README.md, docs/, skills/, PR titles + bodies, commit
|
||||
# messages, and comments in checked-in code. This script greps for the
|
||||
# banned name in either the staged index (for pre-commit hooks) or the
|
||||
# working tree (for CI) and fails loudly if found.
|
||||
#
|
||||
# The allow-list below whitelists files where the name is legitimate
|
||||
# — specifically, this script itself (where we reference the name to
|
||||
# describe the rule) and upgrade guides that historically referenced
|
||||
# the pre-rename fork name.
|
||||
#
|
||||
# Usage:
|
||||
# scripts/check-privacy.sh # scan working tree
|
||||
# scripts/check-privacy.sh --staged # scan git staged index
|
||||
# scripts/check-privacy.sh --help
|
||||
#
|
||||
# Exit codes:
|
||||
# 0 clean
|
||||
# 1 banned name found (or --help printed)
|
||||
# 2 git / grep not available
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
BANNED_NAME='wintermute'
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
scripts/check-privacy.sh — scan for the banned OpenClaw fork name.
|
||||
|
||||
USAGE:
|
||||
scripts/check-privacy.sh Scan all tracked files in the working tree.
|
||||
scripts/check-privacy.sh --staged Scan only files staged for commit.
|
||||
scripts/check-privacy.sh --help Show this message.
|
||||
|
||||
The script greps for '${BANNED_NAME}' (case-insensitive) in:
|
||||
- CHANGELOG.md, README.md
|
||||
- docs/**
|
||||
- skills/**
|
||||
- src/**
|
||||
- test/**
|
||||
- scripts/**
|
||||
|
||||
Allow-list (references to the name are permitted):
|
||||
- scripts/check-privacy.sh itself
|
||||
- docs/UPGRADING_DOWNSTREAM_AGENTS.md (historical context for pre-rename upgrades)
|
||||
- .git/** (branch names, commit history — not checked in artifacts)
|
||||
|
||||
Exit codes: 0 clean, 1 banned name found, 2 setup error.
|
||||
EOF
|
||||
}
|
||||
|
||||
MODE=working
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--staged) MODE=staged ;;
|
||||
--help|-h) usage; exit 1 ;;
|
||||
*)
|
||||
echo "Unknown argument: $arg" >&2
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
if ! command -v git >/dev/null 2>&1; then
|
||||
echo "check-privacy: git not found" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Build the file list by scanning-mode.
|
||||
if [ "$MODE" = staged ]; then
|
||||
FILES=$(git diff --cached --name-only --diff-filter=ACMR 2>/dev/null || true)
|
||||
else
|
||||
FILES=$(git ls-files 2>/dev/null || true)
|
||||
fi
|
||||
|
||||
if [ -z "$FILES" ]; then
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# Allow-list: files in which the banned name is legitimate.
|
||||
# Meta-rule docs (define the rule itself), auto-generated LLM indexes,
|
||||
# historical upgrade guides, and the test that enforces the rule
|
||||
# against recipes/ all reference the banned name by necessity.
|
||||
ALLOW_LIST=(
|
||||
'scripts/check-privacy.sh'
|
||||
'CLAUDE.md'
|
||||
'llms-full.txt'
|
||||
'docs/UPGRADING_DOWNSTREAM_AGENTS.md'
|
||||
'test/integrations.test.ts'
|
||||
)
|
||||
|
||||
is_allowed() {
|
||||
local f="$1"
|
||||
for a in "${ALLOW_LIST[@]}"; do
|
||||
if [ "$f" = "$a" ]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
FOUND=0
|
||||
while IFS= read -r file; do
|
||||
[ -z "$file" ] && continue
|
||||
[ ! -f "$file" ] && continue
|
||||
if is_allowed "$file"; then
|
||||
continue
|
||||
fi
|
||||
# Case-insensitive grep; only specific extensions + known docs.
|
||||
case "$file" in
|
||||
*.md|*.ts|*.mjs|*.js|*.py|*.sh|*.json|*.yaml|*.yml|*.txt|README*|CHANGELOG*|CLAUDE*|AGENTS*)
|
||||
if grep -in "$BANNED_NAME" "$file" >/dev/null 2>&1; then
|
||||
echo "[check-privacy] BANNED NAME in $file:" >&2
|
||||
grep -in "$BANNED_NAME" "$file" | sed 's|^| |' >&2
|
||||
FOUND=1
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
done <<< "$FILES"
|
||||
|
||||
if [ "$FOUND" -eq 1 ]; then
|
||||
echo "" >&2
|
||||
echo "The private OpenClaw fork name is banned in public artifacts." >&2
|
||||
echo "CLAUDE.md:550. Replace with 'your OpenClaw', 'OpenClaw reference deployment', or 'openclaw-reference'." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
+7
-333
@@ -1,339 +1,13 @@
|
||||
#!/usr/bin/env bun
|
||||
/**
|
||||
* skillify-check — Post-task audit.
|
||||
* scripts/skillify-check.ts — thin shim (v0.17+).
|
||||
*
|
||||
* Runs after any task that produced new code/features. Checks whether
|
||||
* the work is "properly skilled" per the 10-item checklist in
|
||||
* skills/skillify/SKILL.md and returns a score + recommendation.
|
||||
*
|
||||
* Usage:
|
||||
* bun run scripts/skillify-check.ts <path-to-code-or-feature>
|
||||
* bun run scripts/skillify-check.ts scripts/frameio-scraper.ts
|
||||
* bun run scripts/skillify-check.ts --recent # check recently-modified
|
||||
* bun run scripts/skillify-check.ts --json # machine-readable output
|
||||
*
|
||||
* Returns JSON when --json is passed: { path, score, total, items,
|
||||
* recommendation }. Exit code is 0 when score == total, 1 otherwise.
|
||||
*
|
||||
* Ported from ~/git/your-openclaw/workspace/scripts/skillify-check.mjs
|
||||
* (genericized: paths computed from $PROJECT_ROOT + runtime test-dir
|
||||
* detection; replaces the manual `grep AGENTS.md` check with a reference
|
||||
* to `gbrain check-resolvable` which validates the resolver better).
|
||||
* The 10-item audit logic lives in `src/commands/skillify-check.ts`
|
||||
* and is exposed as `gbrain skillify check` (D-CX-2). This file stays
|
||||
* as a shim so existing callers (tests, docs, cron entries) continue
|
||||
* to work. New code should prefer `gbrain skillify check ...`.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { join, basename, dirname, resolve } from 'path';
|
||||
import { spawnSync } from 'child_process';
|
||||
import { runSkillifyCheckInline } from '../src/commands/skillify-check.ts';
|
||||
|
||||
function projectRoot(): string {
|
||||
// Walk up from cwd until we find a package.json — that's the repo root.
|
||||
let dir = process.cwd();
|
||||
for (let i = 0; i < 20; i++) {
|
||||
if (existsSync(join(dir, 'package.json'))) return dir;
|
||||
const parent = dirname(dir);
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return process.cwd();
|
||||
}
|
||||
|
||||
const ROOT = projectRoot();
|
||||
const SKILLS_DIR = join(ROOT, 'skills');
|
||||
const RESOLVER_MD = join(SKILLS_DIR, 'RESOLVER.md');
|
||||
|
||||
// Test dir detection: prefer test/, then __tests__/, then tests/, then spec/.
|
||||
function detectTestDir(): string | null {
|
||||
for (const candidate of ['test', '__tests__', 'tests', 'spec']) {
|
||||
const p = join(ROOT, candidate);
|
||||
if (existsSync(p)) return p;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
const TESTS_DIR = detectTestDir();
|
||||
|
||||
interface CheckItem {
|
||||
name: string;
|
||||
passed: boolean;
|
||||
required: boolean;
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
function check(name: string, passed: boolean, detail?: string): CheckItem {
|
||||
return { name, passed, required: true, detail };
|
||||
}
|
||||
function checkOptional(name: string, passed: boolean, detail?: string): CheckItem {
|
||||
return { name, passed, required: false, detail };
|
||||
}
|
||||
|
||||
/**
|
||||
* Invoke `gbrain check-resolvable --json` once and cache the result for the
|
||||
* process lifetime. Binary-missing surfaces a loud error instead of silently
|
||||
* passing — this is the critical guard the failure-mode audit flagged.
|
||||
*/
|
||||
interface ResolverResult {
|
||||
ok: boolean;
|
||||
detail: string;
|
||||
}
|
||||
let _resolverCache: ResolverResult | null = null;
|
||||
function runCheckResolvableCached(): ResolverResult {
|
||||
if (_resolverCache) return _resolverCache;
|
||||
try {
|
||||
const res = spawnSync('gbrain', ['check-resolvable', '--json'], {
|
||||
encoding: 'utf-8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
if (res.error || res.status === null) {
|
||||
const reason = res.error?.message ?? 'spawn returned null status';
|
||||
console.error(`[skillify] gbrain check-resolvable not runnable: ${reason}`);
|
||||
_resolverCache = { ok: false, detail: `check-resolvable unavailable: ${reason}` };
|
||||
return _resolverCache;
|
||||
}
|
||||
const payload = JSON.parse(res.stdout);
|
||||
if (payload.ok === true) {
|
||||
_resolverCache = { ok: true, detail: 'all skill-tree checks pass' };
|
||||
} else {
|
||||
const count = payload.report?.issues?.length ?? 0;
|
||||
const err = payload.error ? ` (${payload.error})` : '';
|
||||
_resolverCache = { ok: false, detail: `${count} issue(s)${err} — run: gbrain check-resolvable` };
|
||||
}
|
||||
return _resolverCache;
|
||||
} catch (err) {
|
||||
console.error(`[skillify] check-resolvable parse failed: ${err}`);
|
||||
_resolverCache = { ok: false, detail: `check-resolvable parse error: ${err}` };
|
||||
return _resolverCache;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Guess the skill-directory name from a script path.
|
||||
* scripts/frameio-scraper.ts → frameio-scraper
|
||||
* src/commands/publish.ts → publish
|
||||
* skills/foo/something.ts → foo
|
||||
*/
|
||||
function inferSkillName(scriptPath: string): string {
|
||||
// If the path is inside skills/, the second segment is the skill name.
|
||||
const abs = resolve(scriptPath);
|
||||
const inSkills = abs.match(/skills\/([^/]+)\//);
|
||||
if (inSkills) return inSkills[1];
|
||||
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
|
||||
// Check for an existing skill dir that matches.
|
||||
if (existsSync(SKILLS_DIR)) {
|
||||
for (const d of readdirSync(SKILLS_DIR)) {
|
||||
if (d === base) return d;
|
||||
// Fuzzy: script base stripped of common suffixes matches a dir name.
|
||||
const normalized = base.replace(/[-_]?(scraper|monitor|check|poll|sync|ingest|core)$/, '');
|
||||
if (d === normalized || d.replace(/-/g, '') === normalized.replace(/[-_]/g, '')) return d;
|
||||
}
|
||||
}
|
||||
|
||||
return base;
|
||||
}
|
||||
|
||||
function findRelatedTests(scriptPath: string): string[] {
|
||||
if (!TESTS_DIR) return [];
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
const patterns = [
|
||||
`${base}.test.ts`,
|
||||
`${base}.test.mjs`,
|
||||
`${base}.test.js`,
|
||||
`test-${base}.ts`,
|
||||
`${base.replace(/-/g, '_')}.test.ts`,
|
||||
];
|
||||
const out: string[] = [];
|
||||
for (const p of patterns) {
|
||||
const f = join(TESTS_DIR, p);
|
||||
if (existsSync(f)) out.push(f);
|
||||
}
|
||||
// Fuzzy partial match.
|
||||
for (const f of readdirSync(TESTS_DIR)) {
|
||||
const normalized = f.replace(/-/g, '').replace('.test.ts', '').replace('.test.mjs', '').replace('test-', '').toLowerCase();
|
||||
const nbase = base.replace(/-/g, '').toLowerCase();
|
||||
if (normalized.includes(nbase) || nbase.includes(normalized)) {
|
||||
const fp = join(TESTS_DIR, f);
|
||||
if (!out.includes(fp)) out.push(fp);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function isInResolver(skillName: string, scriptPath: string): boolean {
|
||||
if (!existsSync(RESOLVER_MD)) return false;
|
||||
const content = readFileSync(RESOLVER_MD, 'utf-8');
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
return content.includes(`skills/${skillName}`)
|
||||
|| content.includes(skillName)
|
||||
|| content.includes(base);
|
||||
}
|
||||
|
||||
function runCheck(target: string): {
|
||||
path: string;
|
||||
skillName: string;
|
||||
items: CheckItem[];
|
||||
score: number;
|
||||
total: number;
|
||||
recommendation: string;
|
||||
} {
|
||||
const abs = resolve(target);
|
||||
const skillName = inferSkillName(target);
|
||||
const skillMd = join(SKILLS_DIR, skillName, 'SKILL.md');
|
||||
|
||||
const items: CheckItem[] = [];
|
||||
|
||||
// 1. SKILL.md exists
|
||||
items.push(check('SKILL.md exists', existsSync(skillMd), skillMd));
|
||||
|
||||
// 2. Code exists at target path
|
||||
items.push(check('Code file exists', existsSync(abs), abs));
|
||||
|
||||
// 3. Unit tests
|
||||
const unitTests = findRelatedTests(target);
|
||||
items.push(check('Unit tests', unitTests.length > 0, unitTests[0] ?? 'no matching *.test.ts in ' + (TESTS_DIR ?? '(no test dir)')));
|
||||
|
||||
// 4. Integration tests (heuristic: has a test that lives under test/e2e/)
|
||||
const e2eDir = TESTS_DIR ? join(TESTS_DIR, 'e2e') : null;
|
||||
const hasE2E = !!e2eDir && existsSync(e2eDir) && readdirSync(e2eDir).some(f =>
|
||||
f.includes(skillName) || f.includes(basename(target).replace(/\.(ts|mjs|js|py)$/, '')),
|
||||
);
|
||||
items.push(checkOptional('Integration tests (E2E)', hasE2E, e2eDir ?? 'no e2e dir'));
|
||||
|
||||
// 5. LLM evals — heuristic: a file named *eval*.test.* in test dir referencing the skill name.
|
||||
let hasEvals = false;
|
||||
if (TESTS_DIR) {
|
||||
for (const f of readdirSync(TESTS_DIR)) {
|
||||
if (/eval/i.test(f) && (f.includes(skillName) || f.includes(basename(target)))) {
|
||||
hasEvals = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
items.push(checkOptional('LLM evals', hasEvals));
|
||||
|
||||
// 6. Resolver entry
|
||||
items.push(check('Resolver entry', isInResolver(skillName, target)));
|
||||
|
||||
// 7. Resolver trigger eval — heuristic: a resolver test that mentions skillName.
|
||||
let hasTriggerEval = false;
|
||||
if (TESTS_DIR) {
|
||||
const resolverTest = join(TESTS_DIR, 'resolver.test.ts');
|
||||
if (existsSync(resolverTest)) {
|
||||
const content = readFileSync(resolverTest, 'utf-8');
|
||||
hasTriggerEval = content.includes(skillName);
|
||||
}
|
||||
}
|
||||
items.push(checkOptional('Resolver trigger eval', hasTriggerEval));
|
||||
|
||||
// 8. check-resolvable — invoke the real gate. Cached per process so
|
||||
// iterating many skills only runs the subprocess once. Binary-missing
|
||||
// is surfaced loudly so a silent false-pass can't happen.
|
||||
const resolverResult = runCheckResolvableCached();
|
||||
items.push(checkOptional('check-resolvable gate',
|
||||
resolverResult.ok,
|
||||
resolverResult.detail));
|
||||
|
||||
// 9. E2E — same as item 4 but required.
|
||||
items.push(check('E2E test (either under e2e/ or integration test)', hasE2E, 'try /qa or test/e2e/'));
|
||||
|
||||
// 10. Brain filing — heuristic: if script mentions `addPage`, `upsertPage`,
|
||||
// or `addBrainPage` then brain/RESOLVER.md should list a matching dir.
|
||||
let writesBrain = false;
|
||||
if (existsSync(abs)) {
|
||||
try {
|
||||
const src = readFileSync(abs, 'utf-8');
|
||||
writesBrain = /addPage|upsertPage|addBrainPage|putPage/.test(src);
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
const brainResolver = join(ROOT, 'brain', 'RESOLVER.md');
|
||||
const hasBrainEntry = writesBrain && existsSync(brainResolver)
|
||||
&& readFileSync(brainResolver, 'utf-8').includes(skillName);
|
||||
items.push(checkOptional('Brain filing (RESOLVER entry for brain writes)',
|
||||
!writesBrain || hasBrainEntry,
|
||||
writesBrain ? (hasBrainEntry ? 'entry present' : 'writes brain but no brain/RESOLVER.md entry') : 'n/a'));
|
||||
|
||||
// Score: required items pass; optional items contribute only if they pass.
|
||||
const passed = items.filter(i => i.passed).length;
|
||||
const total = items.length;
|
||||
const missing = items.filter(i => !i.passed && i.required).map(i => i.name);
|
||||
|
||||
let recommendation: string;
|
||||
if (missing.length === 0) {
|
||||
recommendation = 'properly skilled';
|
||||
} else if (missing.length <= 2) {
|
||||
recommendation = `close — create: ${missing.join(', ')}`;
|
||||
} else {
|
||||
recommendation = `needs skillify — run /skillify on ${target}; missing: ${missing.join(', ')}`;
|
||||
}
|
||||
|
||||
return { path: target, skillName, items, score: passed, total, recommendation };
|
||||
}
|
||||
|
||||
function recentlyModified(days: number = 7): string[] {
|
||||
const candidates: string[] = [];
|
||||
const cutoff = Date.now() - days * 24 * 60 * 60 * 1000;
|
||||
const roots = ['src/commands', 'src/core', 'scripts'].map(r => join(ROOT, r)).filter(existsSync);
|
||||
for (const root of roots) {
|
||||
try {
|
||||
for (const f of readdirSync(root)) {
|
||||
if (!f.match(/\.(ts|mjs|js|py)$/)) continue;
|
||||
const fp = join(root, f);
|
||||
try {
|
||||
const st = statSync(fp);
|
||||
if (st.isFile() && st.mtimeMs >= cutoff) candidates.push(fp);
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
} catch { /* skip */ }
|
||||
}
|
||||
return candidates;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const args = process.argv.slice(2);
|
||||
const json = args.includes('--json');
|
||||
const recent = args.includes('--recent');
|
||||
const help = args.includes('--help') || args.includes('-h');
|
||||
|
||||
if (help || (args.length === 0)) {
|
||||
console.log(`skillify-check — 10-item checklist audit for gbrain features.
|
||||
|
||||
Usage:
|
||||
bun run scripts/skillify-check.ts <path>
|
||||
bun run scripts/skillify-check.ts --recent Check files modified in the last 7 days.
|
||||
bun run scripts/skillify-check.ts --json Emit JSON.
|
||||
|
||||
Exit code 0 when everything required passes; 1 otherwise.
|
||||
`);
|
||||
process.exit(args.length === 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
const targets = recent
|
||||
? recentlyModified(7)
|
||||
: args.filter(a => !a.startsWith('--'));
|
||||
|
||||
if (targets.length === 0) {
|
||||
console.error('No targets. Pass a path or --recent.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const results = targets.map(runCheck);
|
||||
if (json) {
|
||||
console.log(JSON.stringify(results, null, 2));
|
||||
} else {
|
||||
for (const r of results) {
|
||||
console.log(`\n${r.path} [${r.skillName}] ${r.score}/${r.total}`);
|
||||
for (const item of r.items) {
|
||||
const mark = item.passed ? '✓' : (item.required ? '✗' : '·');
|
||||
const tag = item.required ? '' : ' (optional)';
|
||||
const detail = item.detail ? ` — ${item.detail}` : '';
|
||||
console.log(` ${mark} ${item.name}${tag}${detail}`);
|
||||
}
|
||||
console.log(` → ${r.recommendation}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Exit code: non-zero if any result has missing required items.
|
||||
const anyFailed = results.some(r => r.items.some(i => !i.passed && i.required));
|
||||
process.exit(anyFailed ? 1 : 0);
|
||||
}
|
||||
|
||||
main();
|
||||
await runSkillifyCheckInline(process.argv.slice(2));
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
{
|
||||
"version": "1.0.0",
|
||||
"companion": "_brain-filing-rules.md",
|
||||
"description": "Canonical (machine-readable) brain filing rules. The .md companion is the human explainer; this JSON is what `gbrain check-resolvable` audits against. Keep both in sync.",
|
||||
"rules": [
|
||||
{
|
||||
"kind": "person",
|
||||
"directory": "people/",
|
||||
"examples": ["founders", "investors", "attendees", "contacts"],
|
||||
"description": "A page whose primary subject is one person."
|
||||
},
|
||||
{
|
||||
"kind": "company",
|
||||
"directory": "companies/",
|
||||
"examples": ["portfolio companies", "acquirers", "vendors"],
|
||||
"description": "A page whose primary subject is one company or organization."
|
||||
},
|
||||
{
|
||||
"kind": "deal",
|
||||
"directory": "deals/",
|
||||
"examples": ["seed rounds", "acquisitions"],
|
||||
"description": "A page whose primary subject is a financing or M&A transaction."
|
||||
},
|
||||
{
|
||||
"kind": "meeting",
|
||||
"directory": "meetings/",
|
||||
"examples": ["1:1s", "pitches", "pods"],
|
||||
"description": "A meeting transcript or minutes. Propagate entities to companies/ and people/ pages."
|
||||
},
|
||||
{
|
||||
"kind": "concept",
|
||||
"directory": "concepts/",
|
||||
"examples": ["mental models", "theses", "frameworks"],
|
||||
"description": "A reusable idea, framework, or mental model not tied to a specific person/company."
|
||||
},
|
||||
{
|
||||
"kind": "project",
|
||||
"directory": "projects/",
|
||||
"examples": ["internal initiatives", "multi-session work"],
|
||||
"description": "A multi-session piece of work with its own arc."
|
||||
},
|
||||
{
|
||||
"kind": "analysis",
|
||||
"directory": "analysis/",
|
||||
"examples": ["deep dives", "comparative studies"],
|
||||
"description": "A long-form analysis of a specific topic."
|
||||
},
|
||||
{
|
||||
"kind": "civic",
|
||||
"directory": "civic/",
|
||||
"examples": ["policy analysis", "government topics"],
|
||||
"description": "Public-sector, policy, or civic-issue content."
|
||||
},
|
||||
{
|
||||
"kind": "writing",
|
||||
"directory": "writing/",
|
||||
"examples": ["essays", "drafts", "published pieces"],
|
||||
"description": "A piece of prose authored by the user."
|
||||
},
|
||||
{
|
||||
"kind": "guide",
|
||||
"directory": "guides/",
|
||||
"examples": ["runbooks", "how-to docs"],
|
||||
"description": "A guide or runbook authored for future reference."
|
||||
},
|
||||
{
|
||||
"kind": "tech",
|
||||
"directory": "tech/",
|
||||
"examples": ["APIs", "libraries", "language notes"],
|
||||
"description": "Technical references and tooling notes not tied to a specific company."
|
||||
},
|
||||
{
|
||||
"kind": "finance",
|
||||
"directory": "finance/",
|
||||
"examples": ["market data", "metrics"],
|
||||
"description": "Financial reference data not tied to a single deal."
|
||||
},
|
||||
{
|
||||
"kind": "personal",
|
||||
"directory": "personal/",
|
||||
"examples": ["logistics", "family"],
|
||||
"description": "Personal-life content — kept separate from work."
|
||||
},
|
||||
{
|
||||
"kind": "openclaw",
|
||||
"directory": "openclaw/",
|
||||
"examples": ["agent-state notes"],
|
||||
"description": "Notes about the host OpenClaw agent itself, not the underlying entities."
|
||||
}
|
||||
],
|
||||
"sources_dir": {
|
||||
"directory": "sources/",
|
||||
"purpose": "ONLY for raw data: bulk imports, API dumps, periodic captures. A page with a clear primary subject (person, company, concept) does NOT belong here.",
|
||||
"not_for": ["articles about a person", "analyses of a company", "reusable frameworks"]
|
||||
},
|
||||
"notes": [
|
||||
"The PRIMARY SUBJECT of the content determines the directory, not the format or source skill.",
|
||||
"When in doubt: what would you search for to find this page again?",
|
||||
"Cross-link from related directories via back-links — do not duplicate content."
|
||||
]
|
||||
}
|
||||
@@ -17,6 +17,13 @@ tools:
|
||||
- get_backlinks
|
||||
- sync_brain
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- companies/
|
||||
- deals/
|
||||
- concepts/
|
||||
- meetings/
|
||||
---
|
||||
|
||||
# Brain Operations — The Ambient Context Layer
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
// Routing eval fixtures for skills/citation-fixer. Check 5 (W2, v0.17).
|
||||
// Layer A (structural) requires intents to contain trigger words from
|
||||
// the resolver. Paraphrase the trigger framing, not its meaning.
|
||||
{"intent": "please fix broken citations across the latest batch of pages", "expected_skill": "citation-fixer"}
|
||||
{"intent": "I think we need to fix broken citations in these brain pages", "expected_skill": "citation-fixer"}
|
||||
// Negative case: something that sounds similar but should NOT route here.
|
||||
{"intent": "What does this book say about mentorship", "expected_skill": null, "ambiguous_with": []}
|
||||
@@ -20,6 +20,10 @@ tools:
|
||||
- add_timeline_entry
|
||||
- get_backlinks
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- companies/
|
||||
---
|
||||
|
||||
# Enrich Skill
|
||||
|
||||
@@ -20,6 +20,11 @@ tools:
|
||||
- add_timeline_entry
|
||||
- file_upload
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- concepts/
|
||||
- sources/
|
||||
---
|
||||
|
||||
# Idea Ingest Skill
|
||||
|
||||
@@ -13,6 +13,13 @@ tools:
|
||||
- add_timeline_entry
|
||||
- sync_brain
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- companies/
|
||||
- concepts/
|
||||
- meetings/
|
||||
- sources/
|
||||
---
|
||||
|
||||
# Ingest Skill
|
||||
|
||||
@@ -22,6 +22,12 @@ tools:
|
||||
- add_timeline_entry
|
||||
- file_upload
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- concepts/
|
||||
- people/
|
||||
- companies/
|
||||
- sources/
|
||||
---
|
||||
|
||||
# Media Ingest Skill
|
||||
|
||||
@@ -18,6 +18,11 @@ tools:
|
||||
- add_link
|
||||
- add_timeline_entry
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- meetings/
|
||||
- people/
|
||||
- companies/
|
||||
---
|
||||
|
||||
# Meeting Ingestion Skill
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
// Routing eval fixtures for skills/query. Check 5 (W2, v0.17).
|
||||
// Each line: {intent, expected_skill, ambiguous_with?}.
|
||||
// D-CX-6 rule: intent must paraphrase the trigger, not copy it.
|
||||
{"intent": "Who is Paul Graham and what has he written about startups", "expected_skill": "query"}
|
||||
{"intent": "I need background on the Series B round Acme raised last year", "expected_skill": "query", "ambiguous_with": ["brain-ops"]}
|
||||
{"intent": "Do we already have notes on this person", "expected_skill": "query", "ambiguous_with": ["brain-ops"]}
|
||||
@@ -15,6 +15,11 @@ tools:
|
||||
- add_link
|
||||
- add_timeline_entry
|
||||
mutating: true
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- companies/
|
||||
- concepts/
|
||||
---
|
||||
|
||||
# Signal Detector — Ambient Brain Capture
|
||||
|
||||
+23
-10
@@ -60,14 +60,23 @@ For the feature being skillified, answer:
|
||||
|
||||
- **Feature name**: what does it do in one line?
|
||||
- **Code path**: where does the implementation live (file path)?
|
||||
- **Checklist status**: run `scripts/skillify-check.ts <path>` (or write
|
||||
the 10-item checklist manually) and note which items are missing.
|
||||
- **Checklist status**: run `gbrain skillify check <path>` (preferred)
|
||||
or the legacy `scripts/skillify-check.ts <path>` shim. Both produce
|
||||
the same 10-item scorecard. Note which items are missing.
|
||||
|
||||
### Phase 2: Create missing pieces in order
|
||||
|
||||
Work the list top-down. Each earlier item constrains what later items look
|
||||
like (the SKILL.md contract determines what tests assert; tests determine
|
||||
what evals gate; the resolver entry determines what trigger-eval checks).
|
||||
**Fast path — brand-new skill:** run `gbrain skillify scaffold <name>
|
||||
--description "..." [--triggers "p1,p2,p3"] [--writes-pages --writes-to
|
||||
"people/,companies/"]`. This creates all 5 stub files atomically and
|
||||
appends an idempotent resolver row. Every scaffolded file carries the
|
||||
`SKILLIFY_STUB` sentinel; `gbrain check-resolvable --strict` will fail
|
||||
CI until you replace the stubs with real content.
|
||||
|
||||
**Manual path — extending an existing skill:** work the list top-down.
|
||||
Each earlier item constrains what later items look like (the SKILL.md
|
||||
contract determines what tests assert; tests determine what evals gate;
|
||||
the resolver entry determines what trigger-eval checks).
|
||||
|
||||
1. Write `SKILL.md` first. Frontmatter must include `name`, `version`,
|
||||
`description`, `triggers[]`, `tools[]`, `mutating`. Body has at minimum
|
||||
@@ -88,11 +97,15 @@ what evals gate; the resolver entry determines what trigger-eval checks).
|
||||
patterns the user ACTUALLY types, not what you think they should type.
|
||||
7. Add a resolver trigger eval that feeds those patterns in and asserts
|
||||
they route to the new skill.
|
||||
8. Run `gbrain check-resolvable`. It validates reachability (is the skill
|
||||
mentioned from RESOLVER.md?), MECE overlap (does it duplicate an
|
||||
existing skill's trigger?), gap detection (are there user intents that
|
||||
fall through the resolver with no match?), and DRY. If it fails, fix
|
||||
the skill (or extend an existing one instead of creating a duplicate).
|
||||
8. Run `gbrain check-resolvable` (auto-detects skill trees) or
|
||||
`gbrain check-resolvable --skills-dir <path>` for custom locations.
|
||||
OpenClaw workspaces are auto-detected from
|
||||
`~/.openclaw/workspace/skills/`. The check validates reachability (is
|
||||
the skill mentioned from RESOLVER.md?), MECE overlap (does it duplicate
|
||||
an existing skill's trigger?), gap detection (are there user intents
|
||||
that fall through the resolver with no match?), and DRY. If it fails,
|
||||
fix the skill (or extend an existing one instead of creating a
|
||||
duplicate).
|
||||
9. Add an E2E smoke test. For gbrain: submit a Minion job or run a CLI
|
||||
invocation end-to-end against a fixture brain; assert side effects.
|
||||
10. Update `brain/RESOLVER.md` if the skill writes brain pages. Orphaned
|
||||
|
||||
+19
-1
@@ -19,7 +19,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'dream', 'check-resolvable', 'routing-eval', 'skillify']);
|
||||
|
||||
async function main() {
|
||||
// Parse global flags (--quiet / --progress-json / --progress-interval)
|
||||
@@ -315,6 +315,24 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runCheckResolvable(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'routing-eval') {
|
||||
const { runRoutingEvalCli } = await import('./commands/routing-eval.ts');
|
||||
await runRoutingEvalCli(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'skillify') {
|
||||
const { runSkillify } = await import('./commands/skillify.ts');
|
||||
// `args` here is subArgs (command already stripped by caller), so
|
||||
// args[0] is the subcommand (scaffold|check).
|
||||
await runSkillify(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'skillpack') {
|
||||
const { runSkillpack } = await import('./commands/skillpack.ts');
|
||||
// subArgs already has `skillpack` stripped; args[0] is the subcommand.
|
||||
await runSkillpack(args);
|
||||
return;
|
||||
}
|
||||
if (command === 'report') {
|
||||
const { runReport } = await import('./commands/report.ts');
|
||||
await runReport(args);
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
/**
|
||||
* gbrain check-resolvable — Standalone CLI gate for skill-tree integrity.
|
||||
*
|
||||
* Thin wrapper over `src/core/check-resolvable.ts`. Exit-code rule is stricter
|
||||
* than `gbrain doctor`'s resolver_health: this command exits 1 on ANY issue
|
||||
* (errors OR warnings) so CI can gate on a single command. Honors the README
|
||||
* contract: "Exits non-zero if anything is off."
|
||||
* Thin wrapper over `src/core/check-resolvable.ts`. Exit contract (D-CX-3,
|
||||
* post-codex-review):
|
||||
* default: exit 0 unless there are error-severity issues
|
||||
* --strict: exit 0 unless there are errors OR warnings
|
||||
* This lets advisory checks (filing audit, future routing gaps) emit
|
||||
* warnings without breaking CI for workspaces that haven't migrated yet.
|
||||
* CI pipelines that want the old behavior pass --strict.
|
||||
*
|
||||
* Currently covers 4 of 6 checks from the original design: reachability,
|
||||
* MECE overlap, MECE gap, DRY violations. Checks 5 (trigger routing eval)
|
||||
@@ -20,7 +23,7 @@ import {
|
||||
type ResolvableIssue,
|
||||
type AutoFixReport,
|
||||
} from '../core/check-resolvable.ts';
|
||||
import { findRepoRoot } from '../core/repo-root.ts';
|
||||
import { autoDetectSkillsDir, AUTO_DETECT_HINT, type SkillsDirSource } from '../core/repo-root.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
@@ -42,46 +45,51 @@ export interface Envelope {
|
||||
message: string | null;
|
||||
}
|
||||
|
||||
type SkillsDirResolutionSource = 'explicit' | SkillsDirSource | null;
|
||||
|
||||
export interface Flags {
|
||||
help: boolean;
|
||||
json: boolean;
|
||||
fix: boolean;
|
||||
dryRun: boolean;
|
||||
verbose: boolean;
|
||||
strict: boolean;
|
||||
skillsDir: string | null;
|
||||
}
|
||||
|
||||
// TBD: fill these issue URLs after filing the GitHub issues pre-PR.
|
||||
// grep for 'TBD-check-5' / 'TBD-check-6' before shipping.
|
||||
export const DEFERRED: DeferredCheck[] = [
|
||||
{
|
||||
check: 5,
|
||||
name: 'trigger_routing_eval',
|
||||
issue: 'https://github.com/garrytan/gbrain/issues?q=TBD-check-5',
|
||||
},
|
||||
{
|
||||
check: 6,
|
||||
name: 'brain_filing',
|
||||
issue: 'https://github.com/garrytan/gbrain/issues?q=TBD-check-6',
|
||||
},
|
||||
];
|
||||
// Check 5 (trigger_routing_eval) landed in v0.17 (W2). Check 6
|
||||
// (brain_filing) landed in v0.17 (W3). Array is now empty; the
|
||||
// export stays as a stable public field of the --json envelope so
|
||||
// downstream consumers that check `.deferred[]` keep working.
|
||||
// Future deferred checks get appended here.
|
||||
export const DEFERRED: DeferredCheck[] = [];
|
||||
|
||||
const HELP_TEXT = `gbrain check-resolvable [options]
|
||||
|
||||
Validate the skill tree: reachability, MECE overlap, DRY violations, and
|
||||
gap detection. Exits non-zero if any issues are found (errors OR warnings).
|
||||
gap detection. Exits non-zero on errors. Warnings are advisory by default;
|
||||
pass --strict to fail CI on warnings too.
|
||||
|
||||
Options:
|
||||
--json Machine-readable JSON (stable envelope)
|
||||
--fix Apply DRY auto-fixes before checking
|
||||
--dry-run With --fix, preview only; no writes
|
||||
--verbose Show passing checks and the deferred-check note
|
||||
--strict Treat warnings as errors (promotes warnings to exit 1)
|
||||
--skills-dir PATH Override the auto-detected skills/ directory
|
||||
--help Show this message
|
||||
|
||||
Deferred to separate issues (see --json .deferred[]):
|
||||
- Check 5: trigger routing eval
|
||||
- Check 6: brain filing
|
||||
Exit codes:
|
||||
0 clean (no errors; no warnings unless --strict)
|
||||
1 errors present, OR (with --strict) warnings present
|
||||
|
||||
Check 5 (trigger routing eval) lands in v0.17 via W2: any
|
||||
skills/<name>/routing-eval.jsonl fixtures are evaluated and routing
|
||||
gaps surface as warnings.
|
||||
|
||||
Check 6 (brain filing) lands in v0.17 via W3: skills with
|
||||
writes_pages: true are audited against skills/_brain-filing-rules.json.
|
||||
No checks are deferred as of v0.17.
|
||||
`;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -95,6 +103,7 @@ export function parseFlags(argv: string[]): Flags {
|
||||
fix: false,
|
||||
dryRun: false,
|
||||
verbose: false,
|
||||
strict: false,
|
||||
skillsDir: null,
|
||||
};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
@@ -104,6 +113,7 @@ export function parseFlags(argv: string[]): Flags {
|
||||
else if (a === '--fix') flags.fix = true;
|
||||
else if (a === '--dry-run') flags.dryRun = true;
|
||||
else if (a === '--verbose') flags.verbose = true;
|
||||
else if (a === '--strict') flags.strict = true;
|
||||
else if (a === '--skills-dir') {
|
||||
flags.skillsDir = argv[i + 1] ?? null;
|
||||
i++;
|
||||
@@ -119,23 +129,48 @@ export function parseFlags(argv: string[]): Flags {
|
||||
// Skills-dir resolution
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function resolveSkillsDir(flags: Flags): { dir: string | null; error: Envelope['error']; message: string | null } {
|
||||
export function resolveSkillsDir(flags: Flags): {
|
||||
dir: string | null;
|
||||
error: Envelope['error'];
|
||||
message: string | null;
|
||||
source: SkillsDirResolutionSource;
|
||||
} {
|
||||
if (flags.skillsDir) {
|
||||
const dir = isAbsolute(flags.skillsDir)
|
||||
? flags.skillsDir
|
||||
: resolvePath(process.cwd(), flags.skillsDir);
|
||||
return { dir, error: null, message: null };
|
||||
return { dir, error: null, message: null, source: 'explicit' };
|
||||
}
|
||||
const repoRoot = findRepoRoot();
|
||||
if (!repoRoot) {
|
||||
|
||||
const detected = autoDetectSkillsDir();
|
||||
if (!detected.dir) {
|
||||
return {
|
||||
dir: null,
|
||||
error: 'no_skills_dir',
|
||||
message:
|
||||
'Could not locate skills/RESOLVER.md from cwd. Pass --skills-dir <path> or run from inside a gbrain repo.',
|
||||
'Could not auto-detect skills/ with a RESOLVER.md or AGENTS.md.\n' +
|
||||
'Priority order:\n' +
|
||||
AUTO_DETECT_HINT +
|
||||
'\nFix: export OPENCLAW_WORKSPACE=<path> or pass --skills-dir <path>.',
|
||||
source: null,
|
||||
};
|
||||
}
|
||||
return { dir: resolvePath(repoRoot, 'skills'), error: null, message: null };
|
||||
|
||||
const sourceLabel = {
|
||||
repo_root: 'repo root skills/',
|
||||
openclaw_workspace_env: '$OPENCLAW_WORKSPACE/skills',
|
||||
openclaw_workspace_env_root: '$OPENCLAW_WORKSPACE (AGENTS.md at workspace root)',
|
||||
openclaw_workspace_home: '~/.openclaw/workspace/skills',
|
||||
openclaw_workspace_home_root: '~/.openclaw/workspace (AGENTS.md at workspace root)',
|
||||
cwd_skills: './skills',
|
||||
}[detected.source!]!;
|
||||
|
||||
return {
|
||||
dir: detected.dir,
|
||||
error: null,
|
||||
message: `Auto-detected skills directory from ${sourceLabel}: ${detected.dir}`,
|
||||
source: detected.source,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -153,18 +188,25 @@ function renderHuman(env: Envelope, flags: Flags): void {
|
||||
printAutoFixHuman(env.autoFix, flags.dryRun);
|
||||
}
|
||||
|
||||
if (report.ok && report.issues.length === 0) {
|
||||
if (report.errors.length === 0 && report.warnings.length === 0) {
|
||||
console.log(`resolver_health: OK — ${report.summary.total_skills} skills, all reachable`);
|
||||
} else {
|
||||
const errors = report.issues.filter(i => i.severity === 'error');
|
||||
const warnings = report.issues.filter(i => i.severity === 'warning');
|
||||
const status = errors.length > 0 ? 'FAIL' : 'WARN';
|
||||
const status =
|
||||
report.errors.length > 0
|
||||
? 'FAIL'
|
||||
: flags.strict
|
||||
? 'FAIL (strict: warnings promoted)'
|
||||
: 'WARN';
|
||||
const total = report.errors.length + report.warnings.length;
|
||||
console.log(
|
||||
`resolver_health: ${status} — ${report.issues.length} issue(s): ${errors.length} error(s), ${warnings.length} warning(s)`,
|
||||
`resolver_health: ${status} — ${total} issue(s): ${report.errors.length} error(s), ${report.warnings.length} warning(s)`,
|
||||
);
|
||||
for (const iss of report.issues) {
|
||||
for (const iss of [...report.errors, ...report.warnings]) {
|
||||
console.log(formatIssueLine(iss));
|
||||
}
|
||||
if (report.errors.length === 0 && report.warnings.length > 0 && !flags.strict) {
|
||||
console.log('\n(warnings are advisory; run with --strict to fail CI on warnings.)');
|
||||
}
|
||||
}
|
||||
|
||||
if (flags.verbose) {
|
||||
@@ -211,7 +253,7 @@ export async function runCheckResolvable(args: string[]): Promise<void> {
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const { dir, error, message } = resolveSkillsDir(flags);
|
||||
const { dir, error, message, source } = resolveSkillsDir(flags);
|
||||
|
||||
if (error === 'no_skills_dir') {
|
||||
const env: Envelope = {
|
||||
@@ -232,6 +274,9 @@ export async function runCheckResolvable(args: string[]): Promise<void> {
|
||||
}
|
||||
|
||||
const skillsDir = dir!;
|
||||
if (!flags.json && source !== 'explicit' && message) {
|
||||
console.log(message);
|
||||
}
|
||||
|
||||
let autoFix: AutoFixReport | null = null;
|
||||
if (flags.fix) {
|
||||
@@ -240,8 +285,18 @@ export async function runCheckResolvable(args: string[]): Promise<void> {
|
||||
|
||||
const report = checkResolvable(skillsDir);
|
||||
|
||||
// Exit semantics (D-CX-3):
|
||||
// default mode: fail iff any errors
|
||||
// --strict: fail if any errors OR any warnings
|
||||
// Warnings alone never flip the exit code in default mode. This lets
|
||||
// advisory checks (filing audit, future routing gaps) emit without
|
||||
// breaking CI for workspaces that haven't migrated yet.
|
||||
const envOk = flags.strict
|
||||
? report.errors.length === 0 && report.warnings.length === 0
|
||||
: report.errors.length === 0;
|
||||
|
||||
const env: Envelope = {
|
||||
ok: report.issues.length === 0,
|
||||
ok: envOk,
|
||||
skillsDir,
|
||||
report,
|
||||
autoFix,
|
||||
|
||||
@@ -71,21 +71,20 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
}
|
||||
|
||||
const report = checkResolvable(skillsDir);
|
||||
if (report.ok && report.issues.length === 0) {
|
||||
if (report.errors.length === 0 && report.warnings.length === 0) {
|
||||
checks.push({
|
||||
name: 'resolver_health',
|
||||
status: 'ok',
|
||||
message: `${report.summary.total_skills} skills, all reachable`,
|
||||
});
|
||||
} else {
|
||||
const errors = report.issues.filter(i => i.severity === 'error');
|
||||
const warnings = report.issues.filter(i => i.severity === 'warning');
|
||||
const status = errors.length > 0 ? 'fail' as const : 'warn' as const;
|
||||
const status = report.errors.length > 0 ? 'fail' as const : 'warn' as const;
|
||||
const total = report.errors.length + report.warnings.length;
|
||||
const check: Check = {
|
||||
name: 'resolver_health',
|
||||
status,
|
||||
message: `${report.issues.length} issue(s): ${errors.length} error(s), ${warnings.length} warning(s)`,
|
||||
issues: report.issues.map(i => ({
|
||||
message: `${total} issue(s): ${report.errors.length} error(s), ${report.warnings.length} warning(s)`,
|
||||
issues: [...report.errors, ...report.warnings].map(i => ({
|
||||
type: i.type,
|
||||
skill: i.skill,
|
||||
action: i.action,
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
/**
|
||||
* gbrain routing-eval — Standalone CLI verb for Check 5 (W2, v0.17).
|
||||
*
|
||||
* Runs the structural routing eval against every `routing-eval.jsonl`
|
||||
* fixture in the skills tree. Exits:
|
||||
* 0 all fixtures pass (top1 accuracy = 1.0, no ambiguity, no
|
||||
* false positives, no lint issues)
|
||||
* 1 any failure
|
||||
* 2 fixtures directory not found / resolver missing (setup error)
|
||||
*
|
||||
* Layer B (LLM tie-break) via `--llm` is reserved: the flag parses and
|
||||
* surfaces in the envelope, but the harness does not yet call any model.
|
||||
* The plan ships structural layer only in v0.17. The LLM layer has
|
||||
* explicit sequencing in v0.18 once the structural baseline is stable.
|
||||
*/
|
||||
|
||||
import { readFileSync, existsSync } from 'fs';
|
||||
import { resolve as resolvePath, isAbsolute } from 'path';
|
||||
|
||||
import {
|
||||
indexResolverTriggers,
|
||||
lintRoutingFixtures,
|
||||
loadRoutingFixtures,
|
||||
runRoutingEval,
|
||||
type RoutingReport,
|
||||
type FixtureLintIssue,
|
||||
} from '../core/routing-eval.ts';
|
||||
import { findResolverFile, RESOLVER_FILENAMES_LABEL } from '../core/resolver-filenames.ts';
|
||||
import { autoDetectSkillsDir } from '../core/repo-root.ts';
|
||||
import { join } from 'path';
|
||||
|
||||
interface Flags {
|
||||
help: boolean;
|
||||
json: boolean;
|
||||
llm: boolean;
|
||||
skillsDir: string | null;
|
||||
}
|
||||
|
||||
export interface RoutingEvalEnvelope {
|
||||
ok: boolean;
|
||||
skillsDir: string | null;
|
||||
resolverFile: string | null;
|
||||
report: RoutingReport | null;
|
||||
lintIssues: FixtureLintIssue[];
|
||||
malformedFixtures: { file: string; line: number; error: string }[];
|
||||
error: 'no_skills_dir' | 'no_resolver' | null;
|
||||
message: string | null;
|
||||
}
|
||||
|
||||
const HELP = `gbrain routing-eval [options]
|
||||
|
||||
Run the structural routing eval (Check 5) against every skills/<name>/
|
||||
routing-eval.jsonl fixture. Reports top-1 accuracy, ambiguity, and
|
||||
false-positive counts. Lints fixtures for verbatim trigger copies.
|
||||
|
||||
Options:
|
||||
--json Machine-readable JSON envelope
|
||||
--llm (reserved for v0.18) Run Layer B LLM tie-break
|
||||
--skills-dir PATH Override the auto-detected skills/ directory
|
||||
--help Show this message
|
||||
|
||||
Exit codes:
|
||||
0 all fixtures passed, no lint issues
|
||||
1 one or more failures (miss / ambiguous / false positive / lint)
|
||||
2 setup error (no skills dir, no resolver file)
|
||||
`;
|
||||
|
||||
function parseFlags(argv: string[]): Flags {
|
||||
const f: Flags = { help: false, json: false, llm: false, skillsDir: null };
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--help' || a === '-h') f.help = true;
|
||||
else if (a === '--json') f.json = true;
|
||||
else if (a === '--llm') f.llm = true;
|
||||
else if (a === '--skills-dir') {
|
||||
f.skillsDir = argv[i + 1] ?? null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--skills-dir=')) {
|
||||
f.skillsDir = a.slice('--skills-dir='.length) || null;
|
||||
}
|
||||
}
|
||||
return f;
|
||||
}
|
||||
|
||||
function resolveSkillsDir(
|
||||
flags: Flags,
|
||||
): { dir: string | null; source: string | null; error: RoutingEvalEnvelope['error']; message: string | null } {
|
||||
if (flags.skillsDir) {
|
||||
const dir = isAbsolute(flags.skillsDir)
|
||||
? flags.skillsDir
|
||||
: resolvePath(process.cwd(), flags.skillsDir);
|
||||
return { dir, source: 'explicit', error: null, message: null };
|
||||
}
|
||||
const detected = autoDetectSkillsDir();
|
||||
if (!detected.dir) {
|
||||
return {
|
||||
dir: null,
|
||||
source: null,
|
||||
error: 'no_skills_dir',
|
||||
message:
|
||||
'Could not auto-detect skills/ with a resolver file. Set $OPENCLAW_WORKSPACE or pass --skills-dir.',
|
||||
};
|
||||
}
|
||||
return { dir: detected.dir, source: detected.source, error: null, message: null };
|
||||
}
|
||||
|
||||
export async function runRoutingEvalCli(args: string[]): Promise<void> {
|
||||
const flags = parseFlags(args);
|
||||
if (flags.help) {
|
||||
console.log(HELP);
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
const { dir, error, message } = resolveSkillsDir(flags);
|
||||
if (error === 'no_skills_dir') {
|
||||
const env: RoutingEvalEnvelope = {
|
||||
ok: false,
|
||||
skillsDir: null,
|
||||
resolverFile: null,
|
||||
report: null,
|
||||
lintIssues: [],
|
||||
malformedFixtures: [],
|
||||
error,
|
||||
message,
|
||||
};
|
||||
if (flags.json) console.log(JSON.stringify(env, null, 2));
|
||||
else console.error(message);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const skillsDir = dir!;
|
||||
const resolverFile =
|
||||
findResolverFile(skillsDir) ?? findResolverFile(join(skillsDir, '..'));
|
||||
if (!resolverFile) {
|
||||
const env: RoutingEvalEnvelope = {
|
||||
ok: false,
|
||||
skillsDir,
|
||||
resolverFile: null,
|
||||
report: null,
|
||||
lintIssues: [],
|
||||
malformedFixtures: [],
|
||||
error: 'no_resolver',
|
||||
message: `${RESOLVER_FILENAMES_LABEL} not found in ${skillsDir} or its parent.`,
|
||||
};
|
||||
if (flags.json) console.log(JSON.stringify(env, null, 2));
|
||||
else console.error(env.message);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
const resolverContent = readFileSync(resolverFile, 'utf-8');
|
||||
const index = indexResolverTriggers(resolverContent);
|
||||
|
||||
const loaded = loadRoutingFixtures(skillsDir);
|
||||
const lintIssues = lintRoutingFixtures(loaded.fixtures, index);
|
||||
const report = runRoutingEval(resolverContent, loaded.fixtures, { llm: flags.llm });
|
||||
|
||||
const cleanFixtures = lintIssues.length === 0;
|
||||
const cleanResults =
|
||||
report.missed === 0 && report.ambiguous === 0 && report.falsePositives === 0;
|
||||
const cleanLoader = loaded.malformed.length === 0;
|
||||
const ok = cleanFixtures && cleanResults && cleanLoader;
|
||||
|
||||
const env: RoutingEvalEnvelope = {
|
||||
ok,
|
||||
skillsDir,
|
||||
resolverFile,
|
||||
report,
|
||||
lintIssues,
|
||||
malformedFixtures: loaded.malformed.map(m => ({
|
||||
file: m.file,
|
||||
line: m.line,
|
||||
error: m.error,
|
||||
})),
|
||||
error: null,
|
||||
message: null,
|
||||
};
|
||||
|
||||
if (flags.json) {
|
||||
console.log(JSON.stringify(env, null, 2));
|
||||
} else {
|
||||
const pct = Math.round(report.top1Accuracy * 100);
|
||||
const header = ok ? 'routing-eval: OK' : 'routing-eval: ISSUES';
|
||||
console.log(
|
||||
`${header} — ${report.totalCases} case(s), ${pct}% top-1 accuracy`,
|
||||
);
|
||||
if (report.missed > 0) console.log(` • ${report.missed} missed`);
|
||||
if (report.ambiguous > 0) console.log(` • ${report.ambiguous} ambiguous`);
|
||||
if (report.falsePositives > 0)
|
||||
console.log(` • ${report.falsePositives} false positives (negative cases that matched)`);
|
||||
for (const d of report.details.filter(x => x.outcome !== 'pass')) {
|
||||
console.log(
|
||||
` [${d.outcome}] "${d.fixture.intent}" (expected=${d.fixture.expected_skill ?? 'none'})${d.note ? ' — ' + d.note : ''}`,
|
||||
);
|
||||
}
|
||||
for (const lint of lintIssues) {
|
||||
console.log(
|
||||
` [lint:${lint.reason}] "${lint.fixture.intent}" — ${lint.detail}`,
|
||||
);
|
||||
}
|
||||
for (const m of loaded.malformed) {
|
||||
console.log(` [malformed] ${m.file}:${m.line} — ${m.error}`);
|
||||
}
|
||||
if (flags.llm) {
|
||||
console.log('\nNote: --llm (Layer B LLM tie-break) is reserved for v0.18. No model calls made.');
|
||||
}
|
||||
}
|
||||
|
||||
process.exit(ok ? 0 : 1);
|
||||
}
|
||||
@@ -0,0 +1,360 @@
|
||||
/**
|
||||
* gbrain skillify check — 10-item post-task audit.
|
||||
*
|
||||
* Promoted from `scripts/skillify-check.ts` (D-CX-2). The legacy
|
||||
* script stays as a thin shim so existing callers keep working, but
|
||||
* the CLI entry point is now `gbrain skillify check`.
|
||||
*
|
||||
* 10-item checklist (essay Step 3-10):
|
||||
* 1. SKILL.md exists
|
||||
* 2. Code file exists at target path
|
||||
* 3. Unit tests exist
|
||||
* 4. E2E tests (optional — tracked but not required)
|
||||
* 5. LLM evals (optional)
|
||||
* 6. Resolver entry
|
||||
* 7. Resolver trigger eval (heuristic via `test/resolver.test.ts`)
|
||||
* 8. check-resolvable gate (runs `gbrain check-resolvable --json`)
|
||||
* 9. E2E smoke (required copy of #4 for required-gate semantics)
|
||||
* 10. Brain filing (only when the script writes pages)
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { basename, dirname, join, resolve } from 'path';
|
||||
import { spawnSync } from 'child_process';
|
||||
|
||||
interface CheckItem {
|
||||
name: string;
|
||||
passed: boolean;
|
||||
required: boolean;
|
||||
detail?: string;
|
||||
}
|
||||
|
||||
interface CheckResult {
|
||||
path: string;
|
||||
skillName: string;
|
||||
items: CheckItem[];
|
||||
score: number;
|
||||
total: number;
|
||||
recommendation: string;
|
||||
}
|
||||
|
||||
function projectRoot(): string {
|
||||
let dir = process.cwd();
|
||||
for (let i = 0; i < 20; i++) {
|
||||
if (existsSync(join(dir, 'package.json'))) return dir;
|
||||
const parent = dirname(dir);
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return process.cwd();
|
||||
}
|
||||
|
||||
function detectTestDir(root: string): string | null {
|
||||
for (const candidate of ['test', '__tests__', 'tests', 'spec']) {
|
||||
const p = join(root, candidate);
|
||||
if (existsSync(p)) return p;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function check(name: string, passed: boolean, detail?: string): CheckItem {
|
||||
return { name, passed, required: true, detail };
|
||||
}
|
||||
|
||||
function checkOptional(name: string, passed: boolean, detail?: string): CheckItem {
|
||||
return { name, passed, required: false, detail };
|
||||
}
|
||||
|
||||
interface ResolverResult {
|
||||
ok: boolean;
|
||||
detail: string;
|
||||
}
|
||||
|
||||
let _resolverCache: ResolverResult | null = null;
|
||||
|
||||
function runCheckResolvableCached(): ResolverResult {
|
||||
if (_resolverCache) return _resolverCache;
|
||||
try {
|
||||
const res = spawnSync('gbrain', ['check-resolvable', '--json'], {
|
||||
encoding: 'utf-8',
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
if (res.error || res.status === null) {
|
||||
const reason = res.error?.message ?? 'spawn returned null status';
|
||||
console.error(`[skillify] gbrain check-resolvable not runnable: ${reason}`);
|
||||
_resolverCache = { ok: false, detail: `check-resolvable unavailable: ${reason}` };
|
||||
return _resolverCache;
|
||||
}
|
||||
const payload = JSON.parse(res.stdout);
|
||||
if (payload.ok === true) {
|
||||
_resolverCache = { ok: true, detail: 'all skill-tree checks pass' };
|
||||
} else {
|
||||
const count = (payload.report?.errors?.length ?? 0) + (payload.report?.warnings?.length ?? 0);
|
||||
const err = payload.error ? ` (${payload.error})` : '';
|
||||
_resolverCache = {
|
||||
ok: false,
|
||||
detail: `${count} issue(s)${err} — run: gbrain check-resolvable`,
|
||||
};
|
||||
}
|
||||
return _resolverCache;
|
||||
} catch (err) {
|
||||
console.error(`[skillify] check-resolvable parse failed: ${err}`);
|
||||
_resolverCache = { ok: false, detail: `check-resolvable parse error: ${err}` };
|
||||
return _resolverCache;
|
||||
}
|
||||
}
|
||||
|
||||
function inferSkillName(scriptPath: string, skillsDir: string): string {
|
||||
const abs = resolve(scriptPath);
|
||||
const inSkills = abs.match(/skills\/([^/]+)\//);
|
||||
if (inSkills) return inSkills[1];
|
||||
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
if (existsSync(skillsDir)) {
|
||||
for (const d of readdirSync(skillsDir)) {
|
||||
if (d === base) return d;
|
||||
const normalized = base.replace(
|
||||
/[-_]?(scraper|monitor|check|poll|sync|ingest|core)$/,
|
||||
'',
|
||||
);
|
||||
if (d === normalized || d.replace(/-/g, '') === normalized.replace(/[-_]/g, '')) {
|
||||
return d;
|
||||
}
|
||||
}
|
||||
}
|
||||
return base;
|
||||
}
|
||||
|
||||
function findRelatedTests(scriptPath: string, testDir: string | null): string[] {
|
||||
if (!testDir) return [];
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
const patterns = [
|
||||
`${base}.test.ts`,
|
||||
`${base}.test.mjs`,
|
||||
`${base}.test.js`,
|
||||
`test-${base}.ts`,
|
||||
`${base.replace(/-/g, '_')}.test.ts`,
|
||||
];
|
||||
const out: string[] = [];
|
||||
for (const p of patterns) {
|
||||
const f = join(testDir, p);
|
||||
if (existsSync(f)) out.push(f);
|
||||
}
|
||||
for (const f of readdirSync(testDir)) {
|
||||
const normalized = f
|
||||
.replace(/-/g, '')
|
||||
.replace('.test.ts', '')
|
||||
.replace('.test.mjs', '')
|
||||
.replace('test-', '')
|
||||
.toLowerCase();
|
||||
const nbase = base.replace(/-/g, '').toLowerCase();
|
||||
if (normalized.includes(nbase) || nbase.includes(normalized)) {
|
||||
const fp = join(testDir, f);
|
||||
if (!out.includes(fp)) out.push(fp);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function isInResolver(skillName: string, scriptPath: string, skillsDir: string): boolean {
|
||||
const resolverPaths = [
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
join(skillsDir, 'AGENTS.md'),
|
||||
join(dirname(skillsDir), 'AGENTS.md'),
|
||||
];
|
||||
const present = resolverPaths.find(p => existsSync(p));
|
||||
if (!present) return false;
|
||||
const content = readFileSync(present, 'utf-8');
|
||||
const base = basename(scriptPath).replace(/\.(ts|mjs|js|py)$/, '');
|
||||
return (
|
||||
content.includes(`skills/${skillName}`) ||
|
||||
content.includes(skillName) ||
|
||||
content.includes(base)
|
||||
);
|
||||
}
|
||||
|
||||
function runSkillifyCheckTarget(target: string, root: string): CheckResult {
|
||||
const skillsDir = join(root, 'skills');
|
||||
const testDir = detectTestDir(root);
|
||||
const abs = resolve(target);
|
||||
const skillName = inferSkillName(target, skillsDir);
|
||||
const skillMd = join(skillsDir, skillName, 'SKILL.md');
|
||||
|
||||
const items: CheckItem[] = [];
|
||||
|
||||
items.push(check('SKILL.md exists', existsSync(skillMd), skillMd));
|
||||
items.push(check('Code file exists', existsSync(abs), abs));
|
||||
|
||||
const unitTests = findRelatedTests(target, testDir);
|
||||
items.push(
|
||||
check(
|
||||
'Unit tests',
|
||||
unitTests.length > 0,
|
||||
unitTests[0] ?? 'no matching *.test.ts in ' + (testDir ?? '(no test dir)'),
|
||||
),
|
||||
);
|
||||
|
||||
const e2eDir = testDir ? join(testDir, 'e2e') : null;
|
||||
const hasE2E =
|
||||
!!e2eDir &&
|
||||
existsSync(e2eDir) &&
|
||||
readdirSync(e2eDir).some(
|
||||
f =>
|
||||
f.includes(skillName) ||
|
||||
f.includes(basename(target).replace(/\.(ts|mjs|js|py)$/, '')),
|
||||
);
|
||||
items.push(checkOptional('Integration tests (E2E)', hasE2E, e2eDir ?? 'no e2e dir'));
|
||||
|
||||
let hasEvals = false;
|
||||
if (testDir) {
|
||||
for (const f of readdirSync(testDir)) {
|
||||
if (/eval/i.test(f) && (f.includes(skillName) || f.includes(basename(target)))) {
|
||||
hasEvals = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
items.push(checkOptional('LLM evals', hasEvals));
|
||||
|
||||
items.push(check('Resolver entry', isInResolver(skillName, target, skillsDir)));
|
||||
|
||||
let hasTriggerEval = false;
|
||||
if (testDir) {
|
||||
const resolverTest = join(testDir, 'resolver.test.ts');
|
||||
if (existsSync(resolverTest)) {
|
||||
const content = readFileSync(resolverTest, 'utf-8');
|
||||
hasTriggerEval = content.includes(skillName);
|
||||
}
|
||||
const routingFixture = join(skillsDir, skillName, 'routing-eval.jsonl');
|
||||
if (existsSync(routingFixture)) hasTriggerEval = true;
|
||||
}
|
||||
items.push(checkOptional('Resolver trigger eval', hasTriggerEval));
|
||||
|
||||
const resolverResult = runCheckResolvableCached();
|
||||
items.push(
|
||||
checkOptional('check-resolvable gate', resolverResult.ok, resolverResult.detail),
|
||||
);
|
||||
|
||||
items.push(check('E2E test (either under e2e/ or integration test)', hasE2E, 'try /qa or test/e2e/'));
|
||||
|
||||
let writesBrain = false;
|
||||
if (existsSync(abs)) {
|
||||
try {
|
||||
const src = readFileSync(abs, 'utf-8');
|
||||
writesBrain = /addPage|upsertPage|addBrainPage|putPage/.test(src);
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
const brainResolver = join(root, 'brain', 'RESOLVER.md');
|
||||
const hasBrainEntry =
|
||||
writesBrain &&
|
||||
existsSync(brainResolver) &&
|
||||
readFileSync(brainResolver, 'utf-8').includes(skillName);
|
||||
items.push(
|
||||
checkOptional(
|
||||
'Brain filing (RESOLVER entry for brain writes)',
|
||||
!writesBrain || hasBrainEntry,
|
||||
writesBrain ? (hasBrainEntry ? 'entry present' : 'writes brain but no brain/RESOLVER.md entry') : 'n/a',
|
||||
),
|
||||
);
|
||||
|
||||
const passed = items.filter(i => i.passed).length;
|
||||
const total = items.length;
|
||||
const missing = items.filter(i => !i.passed && i.required).map(i => i.name);
|
||||
|
||||
let recommendation: string;
|
||||
if (missing.length === 0) {
|
||||
recommendation = 'properly skilled';
|
||||
} else if (missing.length <= 2) {
|
||||
recommendation = `close — create: ${missing.join(', ')}`;
|
||||
} else {
|
||||
recommendation = `needs skillify — run /skillify on ${target}; missing: ${missing.join(', ')}`;
|
||||
}
|
||||
|
||||
return { path: target, skillName, items, score: passed, total, recommendation };
|
||||
}
|
||||
|
||||
function recentlyModified(root: string, days: number = 7): string[] {
|
||||
const candidates: string[] = [];
|
||||
const cutoff = Date.now() - days * 24 * 60 * 60 * 1000;
|
||||
const roots = ['src/commands', 'src/core', 'scripts']
|
||||
.map(r => join(root, r))
|
||||
.filter(existsSync);
|
||||
for (const r of roots) {
|
||||
try {
|
||||
for (const f of readdirSync(r)) {
|
||||
if (!f.match(/\.(ts|mjs|js|py)$/)) continue;
|
||||
const fp = join(r, f);
|
||||
try {
|
||||
const st = statSync(fp);
|
||||
if (st.isFile() && st.mtimeMs >= cutoff) candidates.push(fp);
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
/* skip */
|
||||
}
|
||||
}
|
||||
return candidates;
|
||||
}
|
||||
|
||||
const HELP = `gbrain skillify check [path] [--recent] [--json]
|
||||
|
||||
Run the 10-item skillify audit (post-task). Reports whether each item
|
||||
passes and what to create next.
|
||||
|
||||
Arguments:
|
||||
path Path to the script/file to audit.
|
||||
--recent Audit all files modified in the last 7 days.
|
||||
--json Emit JSON.
|
||||
--help Show this message.
|
||||
|
||||
Exit code 0 when all REQUIRED items pass; 1 otherwise.
|
||||
`;
|
||||
|
||||
/**
|
||||
* Entry point invoked by `gbrain skillify check`. The outer
|
||||
* dispatcher passes args with the subcommand already stripped.
|
||||
*/
|
||||
export async function runSkillifyCheckInline(args: string[]): Promise<void> {
|
||||
const help = args.includes('--help') || args.includes('-h');
|
||||
const json = args.includes('--json');
|
||||
const recent = args.includes('--recent');
|
||||
if (help || args.length === 0) {
|
||||
console.log(HELP);
|
||||
process.exit(args.length === 0 ? 1 : 0);
|
||||
}
|
||||
|
||||
const root = projectRoot();
|
||||
const targets = recent
|
||||
? recentlyModified(root, 7)
|
||||
: args.filter(a => !a.startsWith('--'));
|
||||
|
||||
if (targets.length === 0) {
|
||||
console.error('No targets. Pass a path or --recent.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
const results: CheckResult[] = targets.map(t => runSkillifyCheckTarget(t, root));
|
||||
if (json) {
|
||||
console.log(JSON.stringify(results, null, 2));
|
||||
} else {
|
||||
for (const r of results) {
|
||||
console.log(`\n${r.path} [${r.skillName}] ${r.score}/${r.total}`);
|
||||
for (const item of r.items) {
|
||||
const mark = item.passed ? '✓' : item.required ? '✗' : '·';
|
||||
const tag = item.required ? '' : ' (optional)';
|
||||
const detail = item.detail ? ` — ${item.detail}` : '';
|
||||
console.log(` ${mark} ${item.name}${tag}${detail}`);
|
||||
}
|
||||
console.log(` → ${r.recommendation}`);
|
||||
}
|
||||
}
|
||||
|
||||
const anyFailed = results.some(r => r.items.some(i => !i.passed && i.required));
|
||||
process.exit(anyFailed ? 1 : 0);
|
||||
}
|
||||
|
||||
export { runSkillifyCheckTarget, type CheckItem, type CheckResult };
|
||||
@@ -0,0 +1,310 @@
|
||||
/**
|
||||
* gbrain skillify <scaffold|check> — v0.17 W4 CLI namespace.
|
||||
*
|
||||
* `scaffold`: creates 5 stub files for a new skill. Mechanical only.
|
||||
* `check`: 10-item audit of an existing skill. Promoted from
|
||||
* `scripts/skillify-check.ts` (D-CX-2). The legacy script
|
||||
* remains as a thin shim that invokes this subcommand.
|
||||
*
|
||||
* The markdown skill at `skills/skillify/SKILL.md` orchestrates the
|
||||
* full 10-step loop (essay's "skillify it!"): scaffold → fill in the
|
||||
* body → run check → run check-resolvable → run tests → commit.
|
||||
* The CLI primitives do the mechanical steps; the skill carries the
|
||||
* judgment steps.
|
||||
*/
|
||||
|
||||
import { isAbsolute, resolve as resolvePath } from 'path';
|
||||
|
||||
import {
|
||||
applyScaffold,
|
||||
planScaffold,
|
||||
SkillifyScaffoldError,
|
||||
SKILL_NAME_PATTERN,
|
||||
type ScaffoldPlan,
|
||||
} from '../core/skillify/generator.ts';
|
||||
import { autoDetectSkillsDir } from '../core/repo-root.ts';
|
||||
import { RESOLVER_FILENAMES_LABEL } from '../core/resolver-filenames.ts';
|
||||
|
||||
// Re-exports for tests.
|
||||
export { planScaffold, applyScaffold, SkillifyScaffoldError, SKILL_NAME_PATTERN };
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Top-level dispatcher
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const HELP_TOP = `gbrain skillify <subcommand> [options]
|
||||
|
||||
Subcommands:
|
||||
scaffold <name> Create SKILL.md, script, routing-eval, test stubs
|
||||
and append a resolver row. Mechanical; no LLM.
|
||||
check [path] Run the 10-item skillify audit on a target path
|
||||
(or --recent). Wraps the legacy scripts/skillify-check.ts
|
||||
(D-CX-2: subcommand namespace).
|
||||
|
||||
Run \`gbrain skillify <subcommand> --help\` for per-subcommand options.
|
||||
`;
|
||||
|
||||
export async function runSkillify(args: string[]): Promise<void> {
|
||||
const sub = args[0];
|
||||
const rest = args.slice(1);
|
||||
if (!sub || sub === '--help' || sub === '-h') {
|
||||
console.log(HELP_TOP);
|
||||
process.exit(0);
|
||||
}
|
||||
if (sub === 'scaffold') {
|
||||
await runSkillifyScaffold(rest);
|
||||
return;
|
||||
}
|
||||
if (sub === 'check') {
|
||||
await runSkillifyCheck(rest);
|
||||
return;
|
||||
}
|
||||
console.error(`Unknown subcommand: ${sub}\n`);
|
||||
console.error(HELP_TOP);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// `gbrain skillify scaffold`
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ScaffoldFlags {
|
||||
help: boolean;
|
||||
json: boolean;
|
||||
dryRun: boolean;
|
||||
force: boolean;
|
||||
name: string | null;
|
||||
description: string | null;
|
||||
triggers: string[];
|
||||
writesTo: string[];
|
||||
writesPages: boolean;
|
||||
mutating: boolean;
|
||||
skillsDir: string | null;
|
||||
}
|
||||
|
||||
const HELP_SCAFFOLD = `gbrain skillify scaffold <name> [options]
|
||||
|
||||
Create 5 scaffold files for a new skill:
|
||||
1. skills/<name>/SKILL.md frontmatter + body template
|
||||
2. skills/<name>/scripts/<name>.mjs deterministic-code stub
|
||||
3. skills/<name>/routing-eval.jsonl routing fixture seed
|
||||
4. test/<name>.test.ts vitest skeleton
|
||||
5. (append) RESOLVER.md or AGENTS.md trigger row under "## Uncategorized"
|
||||
|
||||
All generated files carry the SKILLIFY_STUB sentinel until replaced.
|
||||
\`gbrain check-resolvable --strict\` fails if any skill still has the
|
||||
sentinel in its committed script.
|
||||
|
||||
Options:
|
||||
--description "..." one-liner for SKILL.md frontmatter (required)
|
||||
--triggers "p1,p2,p3" trigger phrases (comma-separated; defaults to TBD)
|
||||
--writes-to "d1,d2" brain dirs this skill will write to
|
||||
--writes-pages mark the skill as a brain-page writer
|
||||
--mutating mark the skill as mutating: true
|
||||
--force overwrite existing stubs (not resolver rows)
|
||||
--dry-run print the plan; no writes
|
||||
--json machine-readable plan envelope
|
||||
--skills-dir PATH override auto-detected skills/
|
||||
--help show this message
|
||||
|
||||
Idempotency: re-running without --force errors on any existing file.
|
||||
With --force, scaffold files are regenerated BUT resolver rows are
|
||||
never duplicated (D-CX-7 contract).
|
||||
`;
|
||||
|
||||
function parseScaffoldFlags(argv: string[]): ScaffoldFlags {
|
||||
const f: ScaffoldFlags = {
|
||||
help: false,
|
||||
json: false,
|
||||
dryRun: false,
|
||||
force: false,
|
||||
name: null,
|
||||
description: null,
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
skillsDir: null,
|
||||
};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--help' || a === '-h') f.help = true;
|
||||
else if (a === '--json') f.json = true;
|
||||
else if (a === '--dry-run') f.dryRun = true;
|
||||
else if (a === '--force') f.force = true;
|
||||
else if (a === '--writes-pages') f.writesPages = true;
|
||||
else if (a === '--mutating') f.mutating = true;
|
||||
else if (a === '--description') {
|
||||
f.description = argv[i + 1] ?? null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--description=')) {
|
||||
f.description = a.slice('--description='.length) || null;
|
||||
} else if (a === '--triggers') {
|
||||
const v = argv[i + 1] ?? '';
|
||||
f.triggers = splitList(v);
|
||||
i++;
|
||||
} else if (a?.startsWith('--triggers=')) {
|
||||
f.triggers = splitList(a.slice('--triggers='.length));
|
||||
} else if (a === '--writes-to') {
|
||||
const v = argv[i + 1] ?? '';
|
||||
f.writesTo = splitList(v);
|
||||
i++;
|
||||
} else if (a?.startsWith('--writes-to=')) {
|
||||
f.writesTo = splitList(a.slice('--writes-to='.length));
|
||||
} else if (a === '--skills-dir') {
|
||||
f.skillsDir = argv[i + 1] ?? null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--skills-dir=')) {
|
||||
f.skillsDir = a.slice('--skills-dir='.length) || null;
|
||||
} else if (a && !a.startsWith('--') && !f.name) {
|
||||
f.name = a;
|
||||
}
|
||||
}
|
||||
return f;
|
||||
}
|
||||
|
||||
function splitList(v: string): string[] {
|
||||
return v
|
||||
.split(',')
|
||||
.map(s => s.trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
|
||||
export async function runSkillifyScaffold(args: string[]): Promise<void> {
|
||||
const flags = parseScaffoldFlags(args);
|
||||
if (flags.help) {
|
||||
console.log(HELP_SCAFFOLD);
|
||||
process.exit(0);
|
||||
}
|
||||
if (!flags.name) {
|
||||
console.error('Error: skill name is required.\n');
|
||||
console.error(HELP_SCAFFOLD);
|
||||
process.exit(2);
|
||||
}
|
||||
if (!flags.description) {
|
||||
console.error('Error: --description is required.\n');
|
||||
console.error(HELP_SCAFFOLD);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// Resolve skills directory.
|
||||
let skillsDir: string | null = null;
|
||||
if (flags.skillsDir) {
|
||||
skillsDir = isAbsolute(flags.skillsDir)
|
||||
? flags.skillsDir
|
||||
: resolvePath(process.cwd(), flags.skillsDir);
|
||||
} else {
|
||||
const detected = autoDetectSkillsDir();
|
||||
skillsDir = detected.dir;
|
||||
}
|
||||
if (!skillsDir) {
|
||||
console.error(
|
||||
'Error: could not auto-detect skills/. Pass --skills-dir or set $OPENCLAW_WORKSPACE.',
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
let plan: ScaffoldPlan;
|
||||
try {
|
||||
plan = planScaffold({
|
||||
skillsDir,
|
||||
force: flags.force,
|
||||
vars: {
|
||||
name: flags.name,
|
||||
description: flags.description,
|
||||
triggers: flags.triggers,
|
||||
writesTo: flags.writesTo,
|
||||
writesPages: flags.writesPages,
|
||||
mutating: flags.mutating,
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
if (err instanceof SkillifyScaffoldError) {
|
||||
if (flags.json) {
|
||||
console.log(JSON.stringify({ ok: false, error: err.code, message: err.message }, null, 2));
|
||||
} else {
|
||||
console.error(`skillify scaffold: ${err.message}`);
|
||||
}
|
||||
process.exit(1);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
|
||||
if (!plan.resolverFile) {
|
||||
const msg = `${RESOLVER_FILENAMES_LABEL} not found in ${skillsDir} or its parent. Create one before scaffolding skills.`;
|
||||
if (flags.json) {
|
||||
console.log(JSON.stringify({ ok: false, error: 'no_resolver', message: msg }, null, 2));
|
||||
} else {
|
||||
console.error(msg);
|
||||
}
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
if (flags.dryRun) {
|
||||
if (flags.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
ok: true,
|
||||
dryRun: true,
|
||||
files: plan.files.map(f => ({ path: f.path, kind: f.kind })),
|
||||
resolverFile: plan.resolverFile,
|
||||
resolverAppendBytes: plan.resolverAppend?.length ?? 0,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
console.log(`skillify scaffold --dry-run (${plan.files.length} files):`);
|
||||
for (const f of plan.files) console.log(` [${f.kind}] ${f.path}`);
|
||||
if (plan.resolverAppend !== null) {
|
||||
console.log(` [append] ${plan.resolverFile} (+${plan.resolverAppend.length} bytes)`);
|
||||
} else {
|
||||
console.log(` [skip] ${plan.resolverFile} (row already present — idempotent)`);
|
||||
}
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
applyScaffold(plan);
|
||||
|
||||
if (flags.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
ok: true,
|
||||
dryRun: false,
|
||||
files: plan.files.map(f => ({ path: f.path, kind: f.kind })),
|
||||
resolverFile: plan.resolverFile,
|
||||
resolverAppended: plan.resolverAppend !== null,
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
console.log(`skillify scaffold: wrote ${plan.files.length} files.`);
|
||||
for (const f of plan.files) console.log(` [${f.kind}] ${f.path}`);
|
||||
if (plan.resolverAppend !== null) {
|
||||
console.log(` [append] ${plan.resolverFile}`);
|
||||
}
|
||||
console.log('\nNext:');
|
||||
console.log(` 1. Replace SKILLIFY_STUB sentinels in the generated files.`);
|
||||
console.log(` 2. bun test test/${flags.name}.test.ts`);
|
||||
console.log(` 3. gbrain skillify check skills/${flags.name}/scripts/${flags.name}.mjs`);
|
||||
console.log(` 4. gbrain check-resolvable`);
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// `gbrain skillify check` — delegates to scripts/skillify-check.ts via same
|
||||
// internal helpers. For v0.17 we shell out to the script (kept as single
|
||||
// source of truth); v0.18 may inline it further.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function runSkillifyCheck(args: string[]): Promise<void> {
|
||||
// Late-import to avoid pulling the helpers at module init.
|
||||
const { runSkillifyCheckInline } = await import('./skillify-check.ts');
|
||||
await runSkillifyCheckInline(args);
|
||||
}
|
||||
@@ -0,0 +1,440 @@
|
||||
/**
|
||||
* gbrain skillpack <list|install|diff|check> — v0.17 W5 CLI namespace.
|
||||
*
|
||||
* D-CX-2 pattern: unified subcommand namespace. The pre-existing
|
||||
* `skillpack-check` command keeps its top-level name for backwards
|
||||
* compat but is also reachable as `gbrain skillpack check` here.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { isAbsolute, resolve as resolvePath, join } from 'path';
|
||||
|
||||
import {
|
||||
bundledSkillSlugs,
|
||||
findGbrainRoot,
|
||||
loadBundleManifest,
|
||||
BundleError,
|
||||
} from '../core/skillpack/bundle.ts';
|
||||
import {
|
||||
planInstall,
|
||||
applyInstall,
|
||||
diffSkill,
|
||||
InstallError,
|
||||
} from '../core/skillpack/installer.ts';
|
||||
import { autoDetectSkillsDir } from '../core/repo-root.ts';
|
||||
|
||||
const HELP_TOP = `gbrain skillpack <subcommand> [options]
|
||||
|
||||
Subcommands:
|
||||
list Print every skill bundled in openclaw.plugin.json.
|
||||
install <name> Copy one skill (or --all) into a target workspace.
|
||||
Data-loss protected: per-file diff, --overwrite-local
|
||||
escape hatch, lockfile + atomic AGENTS.md update.
|
||||
diff <name> Show per-file diff status between the bundle and
|
||||
the target workspace for one skill.
|
||||
check Run the skillpack health report (same as the
|
||||
top-level \`gbrain skillpack-check\`).
|
||||
|
||||
Run \`gbrain skillpack <subcommand> --help\` for per-subcommand options.
|
||||
`;
|
||||
|
||||
export async function runSkillpack(args: string[]): Promise<void> {
|
||||
const sub = args[0];
|
||||
const rest = args.slice(1);
|
||||
if (!sub || sub === '--help' || sub === '-h') {
|
||||
console.log(HELP_TOP);
|
||||
process.exit(0);
|
||||
}
|
||||
if (sub === 'list') {
|
||||
await runList(rest);
|
||||
return;
|
||||
}
|
||||
if (sub === 'install') {
|
||||
await runInstall(rest);
|
||||
return;
|
||||
}
|
||||
if (sub === 'diff') {
|
||||
await runDiff(rest);
|
||||
return;
|
||||
}
|
||||
if (sub === 'check') {
|
||||
const { runSkillpackCheck } = await import('./skillpack-check.ts');
|
||||
await runSkillpackCheck(rest);
|
||||
return;
|
||||
}
|
||||
console.error(`Unknown subcommand: ${sub}\n`);
|
||||
console.error(HELP_TOP);
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// list
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const HELP_LIST = `gbrain skillpack list [--json]
|
||||
|
||||
Print every skill bundled in openclaw.plugin.json, one per line.
|
||||
Exit 0 always (unless the manifest is missing/malformed).
|
||||
`;
|
||||
|
||||
async function runList(args: string[]): Promise<void> {
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
console.log(HELP_LIST);
|
||||
process.exit(0);
|
||||
}
|
||||
const json = args.includes('--json');
|
||||
const gbrainRoot = findGbrainRoot();
|
||||
if (!gbrainRoot) {
|
||||
console.error(
|
||||
'Error: could not find gbrain repo root. Run this from inside a gbrain checkout, or pass --gbrain-root (not yet implemented).',
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
let manifest;
|
||||
try {
|
||||
manifest = loadBundleManifest(gbrainRoot);
|
||||
} catch (err) {
|
||||
console.error(`skillpack list: ${(err as Error).message}`);
|
||||
process.exit(2);
|
||||
}
|
||||
const slugs = bundledSkillSlugs(manifest);
|
||||
if (json) {
|
||||
const entries = slugs.map(slug => {
|
||||
const skillMd = join(gbrainRoot, 'skills', slug, 'SKILL.md');
|
||||
let description: string | null = null;
|
||||
if (existsSync(skillMd)) {
|
||||
const body = readFileSync(skillMd, 'utf-8');
|
||||
const fm = body.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (fm) {
|
||||
const descMatch = fm[1].match(/^description:\s*["']?([^\n"']+)/m);
|
||||
if (descMatch) description = descMatch[1].trim();
|
||||
}
|
||||
}
|
||||
return { name: slug, description };
|
||||
});
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{ name: manifest.name, version: manifest.version, skills: entries },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
console.log(`${manifest.name} ${manifest.version} bundle — ${slugs.length} skills:`);
|
||||
for (const slug of slugs) {
|
||||
console.log(` ${slug}`);
|
||||
}
|
||||
}
|
||||
process.exit(0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// install
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface InstallFlags {
|
||||
help: boolean;
|
||||
json: boolean;
|
||||
dryRun: boolean;
|
||||
force: boolean;
|
||||
overwriteLocal: boolean;
|
||||
forceUnlock: boolean;
|
||||
all: boolean;
|
||||
skillName: string | null;
|
||||
skillsDir: string | null;
|
||||
workspace: string | null;
|
||||
}
|
||||
|
||||
const HELP_INSTALL = `gbrain skillpack install <name> | --all [options]
|
||||
|
||||
Copy bundled skills into a target OpenClaw workspace. The target is
|
||||
auto-detected (\$OPENCLAW_WORKSPACE, then ~/.openclaw/workspace, then
|
||||
--skills-dir). Shared convention files are installed alongside
|
||||
(dependency closure per codex D-CX-10).
|
||||
|
||||
Arguments:
|
||||
<name> Install a single skill by slug.
|
||||
--all Install every skill in openclaw.plugin.json#skills.
|
||||
|
||||
Options:
|
||||
--dry-run Preview file operations; no writes.
|
||||
--overwrite-local For per-file diff: overwrite target files that
|
||||
differ from the bundle. Default: skip locally-
|
||||
modified files for data-loss protection.
|
||||
--force-unlock Acquire the skillpack lockfile even if a stale
|
||||
peer lock exists.
|
||||
--skills-dir PATH Override target skills directory.
|
||||
--workspace PATH Override target workspace (parent of skills/).
|
||||
--json Machine-readable envelope.
|
||||
--help Show this message.
|
||||
|
||||
Exit codes:
|
||||
0 success
|
||||
1 some files skipped due to local-modification protection
|
||||
2 setup error (no workspace, no bundle, lock held)
|
||||
`;
|
||||
|
||||
function parseInstallFlags(argv: string[]): InstallFlags {
|
||||
const f: InstallFlags = {
|
||||
help: false,
|
||||
json: false,
|
||||
dryRun: false,
|
||||
force: false,
|
||||
overwriteLocal: false,
|
||||
forceUnlock: false,
|
||||
all: false,
|
||||
skillName: null,
|
||||
skillsDir: null,
|
||||
workspace: null,
|
||||
};
|
||||
for (let i = 0; i < argv.length; i++) {
|
||||
const a = argv[i];
|
||||
if (a === '--help' || a === '-h') f.help = true;
|
||||
else if (a === '--json') f.json = true;
|
||||
else if (a === '--dry-run') f.dryRun = true;
|
||||
else if (a === '--force') f.force = true;
|
||||
else if (a === '--overwrite-local') f.overwriteLocal = true;
|
||||
else if (a === '--force-unlock') f.forceUnlock = true;
|
||||
else if (a === '--all') f.all = true;
|
||||
else if (a === '--skills-dir') {
|
||||
f.skillsDir = argv[i + 1] ?? null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--skills-dir=')) {
|
||||
f.skillsDir = a.slice('--skills-dir='.length) || null;
|
||||
} else if (a === '--workspace') {
|
||||
f.workspace = argv[i + 1] ?? null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--workspace=')) {
|
||||
f.workspace = a.slice('--workspace='.length) || null;
|
||||
} else if (a && !a.startsWith('--') && !f.skillName) {
|
||||
f.skillName = a;
|
||||
}
|
||||
}
|
||||
return f;
|
||||
}
|
||||
|
||||
function resolveAbs(p: string): string {
|
||||
return isAbsolute(p) ? p : resolvePath(process.cwd(), p);
|
||||
}
|
||||
|
||||
async function runInstall(args: string[]): Promise<void> {
|
||||
const flags = parseInstallFlags(args);
|
||||
if (flags.help) {
|
||||
console.log(HELP_INSTALL);
|
||||
process.exit(0);
|
||||
}
|
||||
if (!flags.all && !flags.skillName) {
|
||||
console.error('Error: pass a skill name or --all.\n');
|
||||
console.error(HELP_INSTALL);
|
||||
process.exit(2);
|
||||
}
|
||||
const gbrainRoot = findGbrainRoot();
|
||||
if (!gbrainRoot) {
|
||||
console.error('Error: could not find gbrain repo root.');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
// Resolve target: workspace (parent of skills/) is required for the
|
||||
// managed block + lockfile. skillsDir defaults to workspace/skills.
|
||||
let targetWorkspace: string | null = flags.workspace
|
||||
? resolveAbs(flags.workspace)
|
||||
: null;
|
||||
let targetSkillsDir: string | null = flags.skillsDir
|
||||
? resolveAbs(flags.skillsDir)
|
||||
: null;
|
||||
|
||||
if (!targetSkillsDir) {
|
||||
const detected = autoDetectSkillsDir();
|
||||
if (detected.dir) {
|
||||
targetSkillsDir = detected.dir;
|
||||
if (!targetWorkspace) {
|
||||
// workspace is parent of skills/
|
||||
targetWorkspace = resolvePath(targetSkillsDir, '..');
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!targetSkillsDir) {
|
||||
console.error(
|
||||
'Error: could not find a target skills directory. Set $OPENCLAW_WORKSPACE or pass --skills-dir / --workspace.',
|
||||
);
|
||||
process.exit(2);
|
||||
}
|
||||
if (!targetWorkspace) {
|
||||
targetWorkspace = resolvePath(targetSkillsDir, '..');
|
||||
}
|
||||
|
||||
try {
|
||||
const plan = planInstall({
|
||||
gbrainRoot,
|
||||
targetWorkspace,
|
||||
targetSkillsDir,
|
||||
skillSlug: flags.all ? null : flags.skillName!,
|
||||
overwriteLocal: flags.overwriteLocal,
|
||||
dryRun: flags.dryRun,
|
||||
forceUnlock: flags.forceUnlock,
|
||||
});
|
||||
const result = applyInstall(plan, {
|
||||
gbrainRoot,
|
||||
targetWorkspace,
|
||||
targetSkillsDir,
|
||||
skillSlug: flags.all ? null : flags.skillName!,
|
||||
overwriteLocal: flags.overwriteLocal,
|
||||
dryRun: flags.dryRun,
|
||||
forceUnlock: flags.forceUnlock,
|
||||
});
|
||||
|
||||
if (flags.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{
|
||||
ok: result.summary.skippedLocallyModified === 0,
|
||||
dryRun: result.dryRun,
|
||||
gbrainRoot,
|
||||
targetWorkspace,
|
||||
targetSkillsDir,
|
||||
summary: result.summary,
|
||||
managedBlock: result.managedBlock,
|
||||
files: result.files.map(f => ({
|
||||
source: f.source,
|
||||
target: f.target,
|
||||
outcome: f.outcome,
|
||||
sharedDep: f.sharedDep,
|
||||
})),
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
const label = flags.dryRun ? 'skillpack install --dry-run' : 'skillpack install';
|
||||
console.log(
|
||||
`${label}: ${result.summary.wroteNew} new, ${result.summary.wroteOverwrite} overwrites, ${result.summary.skippedIdentical} unchanged, ${result.summary.skippedLocallyModified} skipped (local edits)`,
|
||||
);
|
||||
for (const f of result.files) {
|
||||
if (f.outcome === 'skipped_identical') continue;
|
||||
const tag = f.outcome.padEnd(25);
|
||||
const dep = f.sharedDep ? ' [shared]' : '';
|
||||
console.log(` ${tag} ${f.target}${dep}`);
|
||||
}
|
||||
if (result.managedBlock.applied) {
|
||||
console.log(` managed-block ${result.managedBlock.resolverFile}`);
|
||||
} else if (result.managedBlock.skippedReason === 'resolver_not_found') {
|
||||
console.log(
|
||||
` warn: no RESOLVER.md / AGENTS.md in ${targetWorkspace} or ${targetSkillsDir} — managed block not written. Create one and re-run.`,
|
||||
);
|
||||
}
|
||||
if (result.summary.skippedLocallyModified > 0) {
|
||||
console.log(
|
||||
`\nNote: ${result.summary.skippedLocallyModified} file(s) differ from the bundle and were skipped. Pass --overwrite-local to replace them (loses local edits) or run \`gbrain skillpack diff <name>\` to inspect.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const exitCode = result.summary.skippedLocallyModified > 0 ? 1 : 0;
|
||||
process.exit(exitCode);
|
||||
} catch (err) {
|
||||
if (err instanceof InstallError || err instanceof BundleError) {
|
||||
if (flags.json) {
|
||||
console.log(
|
||||
JSON.stringify(
|
||||
{ ok: false, error: (err as Error & { code?: string }).code, message: err.message },
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
} else {
|
||||
console.error(`skillpack install: ${err.message}`);
|
||||
}
|
||||
process.exit(err instanceof InstallError && err.code === 'lock_held' ? 2 : 2);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// diff
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const HELP_DIFF = `gbrain skillpack diff <name> [options]
|
||||
|
||||
Show per-file diff status between the bundle and the target workspace
|
||||
for one skill. Read-only; no writes.
|
||||
|
||||
Options:
|
||||
--skills-dir PATH Override target skills directory.
|
||||
--json Machine-readable envelope.
|
||||
--help Show this message.
|
||||
|
||||
Exit codes:
|
||||
0 every file matches the bundle
|
||||
1 at least one file differs (or is missing)
|
||||
`;
|
||||
|
||||
async function runDiff(args: string[]): Promise<void> {
|
||||
const help = args.includes('--help') || args.includes('-h');
|
||||
if (help) {
|
||||
console.log(HELP_DIFF);
|
||||
process.exit(0);
|
||||
}
|
||||
const json = args.includes('--json');
|
||||
let targetSkillsDir: string | null = null;
|
||||
let skillName: string | null = null;
|
||||
for (let i = 0; i < args.length; i++) {
|
||||
const a = args[i];
|
||||
if (a === '--skills-dir') {
|
||||
targetSkillsDir = args[i + 1] ? resolveAbs(args[i + 1]) : null;
|
||||
i++;
|
||||
} else if (a?.startsWith('--skills-dir=')) {
|
||||
targetSkillsDir = resolveAbs(a.slice('--skills-dir='.length));
|
||||
} else if (a && !a.startsWith('--') && !skillName) {
|
||||
skillName = a;
|
||||
}
|
||||
}
|
||||
if (!skillName) {
|
||||
console.error('Error: pass a skill name.\n');
|
||||
console.error(HELP_DIFF);
|
||||
process.exit(2);
|
||||
}
|
||||
const gbrainRoot = findGbrainRoot();
|
||||
if (!gbrainRoot) {
|
||||
console.error('Error: could not find gbrain repo root.');
|
||||
process.exit(2);
|
||||
}
|
||||
if (!targetSkillsDir) {
|
||||
const detected = autoDetectSkillsDir();
|
||||
if (detected.dir) targetSkillsDir = detected.dir;
|
||||
}
|
||||
if (!targetSkillsDir) {
|
||||
console.error('Error: pass --skills-dir or set $OPENCLAW_WORKSPACE.');
|
||||
process.exit(2);
|
||||
}
|
||||
|
||||
try {
|
||||
const diffs = diffSkill(gbrainRoot, skillName, targetSkillsDir);
|
||||
const clean = diffs.every(d => d.identical && d.existing);
|
||||
if (json) {
|
||||
console.log(JSON.stringify({ ok: clean, skillName, diffs }, null, 2));
|
||||
} else {
|
||||
console.log(`skillpack diff ${skillName} → ${targetSkillsDir}`);
|
||||
for (const d of diffs) {
|
||||
let tag: string;
|
||||
if (!d.existing) tag = 'missing ';
|
||||
else if (d.identical) tag = 'identical';
|
||||
else tag = 'differs ';
|
||||
console.log(` ${tag} ${d.target} (src ${d.sourceBytes}B / tgt ${d.targetBytes}B)`);
|
||||
}
|
||||
console.log(clean ? '\n✓ all files match the bundle.' : '\n(run `gbrain skillpack install <name> --overwrite-local` to replace local-modified files.)');
|
||||
}
|
||||
process.exit(clean ? 0 : 1);
|
||||
} catch (err) {
|
||||
if (err instanceof BundleError) {
|
||||
if (json) {
|
||||
console.log(JSON.stringify({ ok: false, error: err.code, message: err.message }, null, 2));
|
||||
} else {
|
||||
console.error(`skillpack diff: ${err.message}`);
|
||||
}
|
||||
process.exit(2);
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
+175
-24
@@ -12,6 +12,15 @@
|
||||
|
||||
import { readFileSync, existsSync, readdirSync } from 'fs';
|
||||
import { join, relative } from 'path';
|
||||
import { findResolverFile, RESOLVER_FILENAMES_LABEL } from './resolver-filenames.ts';
|
||||
import { loadOrDeriveManifest } from './skill-manifest.ts';
|
||||
import {
|
||||
indexResolverTriggers,
|
||||
lintRoutingFixtures,
|
||||
loadRoutingFixtures,
|
||||
runRoutingEval,
|
||||
} from './routing-eval.ts';
|
||||
import { runFilingAudit } from './filing-audit.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
@@ -25,7 +34,23 @@ export interface ResolvableFix {
|
||||
}
|
||||
|
||||
export interface ResolvableIssue {
|
||||
type: 'unreachable' | 'mece_overlap' | 'mece_gap' | 'dry_violation' | 'missing_file' | 'orphan_trigger';
|
||||
type:
|
||||
| 'unreachable'
|
||||
| 'mece_overlap'
|
||||
| 'mece_gap'
|
||||
| 'dry_violation'
|
||||
| 'missing_file'
|
||||
| 'orphan_trigger'
|
||||
// Check 5 (W2): routing eval results surfaced as advisories.
|
||||
| 'routing_miss'
|
||||
| 'routing_ambiguous'
|
||||
| 'routing_false_positive'
|
||||
| 'routing_fixture_lint'
|
||||
// Check 6 (W3): brain-filing audit findings.
|
||||
| 'filing_missing_writes_to'
|
||||
| 'filing_unknown_directory'
|
||||
// D-CX-9: scaffolded skill still carries SKILLIFY_STUB sentinel.
|
||||
| 'skillify_stub_unreplaced';
|
||||
severity: 'error' | 'warning';
|
||||
skill: string;
|
||||
message: string;
|
||||
@@ -34,7 +59,27 @@ export interface ResolvableIssue {
|
||||
}
|
||||
|
||||
export interface ResolvableReport {
|
||||
/**
|
||||
* True when there are no error-severity issues. Warnings do NOT flip `ok`.
|
||||
* Callers that want strict-mode (warnings fail CI too) should gate on
|
||||
* `errors.length === 0 && warnings.length === 0`.
|
||||
*/
|
||||
ok: boolean;
|
||||
/**
|
||||
* Error-severity issues only. Determines `ok` and default exit codes.
|
||||
* A subset of `issues[]`.
|
||||
*/
|
||||
errors: ResolvableIssue[];
|
||||
/**
|
||||
* Warning-severity issues. Informational by default; `--strict` promotes.
|
||||
* A subset of `issues[]`.
|
||||
*/
|
||||
warnings: ResolvableIssue[];
|
||||
/**
|
||||
* @deprecated Use `errors` and `warnings` separately. Kept for one-release
|
||||
* backwards compatibility; will be removed in v0.18. Equivalent to
|
||||
* `[...errors, ...warnings]`.
|
||||
*/
|
||||
issues: ResolvableIssue[];
|
||||
summary: {
|
||||
total_skills: number;
|
||||
@@ -111,17 +156,10 @@ export function parseResolverEntries(resolverContent: string): ResolverEntry[] {
|
||||
return entries;
|
||||
}
|
||||
|
||||
/** Extract skill names from manifest.json */
|
||||
function loadManifest(skillsDir: string): Array<{ name: string; path: string }> {
|
||||
const manifestPath = join(skillsDir, 'manifest.json');
|
||||
if (!existsSync(manifestPath)) return [];
|
||||
try {
|
||||
const content = JSON.parse(readFileSync(manifestPath, 'utf-8'));
|
||||
return content.skills || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
// Manifest loading is now delegated to src/core/skill-manifest.ts
|
||||
// (loadOrDeriveManifest). That module auto-derives from walking
|
||||
// `skillsDir/*/SKILL.md` when manifest.json is missing — the scenario
|
||||
// needed for AGENTS.md-only OpenClaw deployments. See D-CX-12 / F-ENG-1.
|
||||
|
||||
/** Simple YAML frontmatter parser — extracts triggers array if present. */
|
||||
function extractTriggers(skillContent: string): string[] {
|
||||
@@ -211,25 +249,34 @@ export function checkResolvable(skillsDir: string): ResolvableReport {
|
||||
const issues: ResolvableIssue[] = [];
|
||||
|
||||
// Load inputs
|
||||
const resolverPath = join(skillsDir, 'RESOLVER.md');
|
||||
if (!existsSync(resolverPath)) {
|
||||
// Accept RESOLVER.md or AGENTS.md (W1). Also check one level up: the
|
||||
// reference OpenClaw deployment layout places AGENTS.md at the
|
||||
// workspace root, with skills/ below. We try skills dir first
|
||||
// (gbrain-native), then its parent (OpenClaw-native).
|
||||
const resolverPath =
|
||||
findResolverFile(skillsDir) ?? findResolverFile(join(skillsDir, '..'));
|
||||
if (!resolverPath) {
|
||||
const suggested = join(skillsDir, 'RESOLVER.md');
|
||||
const missingIssue: ResolvableIssue = {
|
||||
type: 'missing_file',
|
||||
severity: 'error',
|
||||
skill: RESOLVER_FILENAMES_LABEL,
|
||||
message: `${RESOLVER_FILENAMES_LABEL} not found in ${skillsDir} or its parent`,
|
||||
action: `Create ${suggested} with skill routing tables`,
|
||||
fix: { type: 'create_stub', file: suggested },
|
||||
};
|
||||
return {
|
||||
ok: false,
|
||||
issues: [{
|
||||
type: 'missing_file',
|
||||
severity: 'error',
|
||||
skill: 'RESOLVER.md',
|
||||
message: 'RESOLVER.md not found',
|
||||
action: `Create ${resolverPath} with skill routing tables`,
|
||||
fix: { type: 'create_stub', file: resolverPath },
|
||||
}],
|
||||
errors: [missingIssue],
|
||||
warnings: [],
|
||||
issues: [missingIssue],
|
||||
summary: { total_skills: 0, reachable: 0, unreachable: 0, overlaps: 0, gaps: 0 },
|
||||
};
|
||||
}
|
||||
|
||||
const resolverContent = readFileSync(resolverPath, 'utf-8');
|
||||
const entries = parseResolverEntries(resolverContent);
|
||||
const manifest = loadManifest(skillsDir);
|
||||
const { skills: manifest } = loadOrDeriveManifest(skillsDir);
|
||||
|
||||
// Build lookup sets
|
||||
const resolverSkillPaths = new Set(
|
||||
@@ -406,8 +453,112 @@ export function checkResolvable(skillsDir: string): ResolvableReport {
|
||||
}
|
||||
}
|
||||
|
||||
// Check 5 (W2, v0.17): structural routing eval. Surfaces as warnings
|
||||
// only — routing issues are advisory. Agents running under --strict
|
||||
// will fail on them; default runs see them as informational.
|
||||
const loaded = loadRoutingFixtures(skillsDir);
|
||||
if (loaded.fixtures.length > 0) {
|
||||
const triggerIndex = indexResolverTriggers(resolverContent);
|
||||
const lintIssues = lintRoutingFixtures(loaded.fixtures, triggerIndex);
|
||||
for (const lint of lintIssues) {
|
||||
issues.push({
|
||||
type: 'routing_fixture_lint',
|
||||
severity: 'warning',
|
||||
skill: lint.fixture.expected_skill ?? 'unknown',
|
||||
message: `Routing fixture lint (${lint.reason}): "${lint.fixture.intent}"`,
|
||||
action: `Edit skills/<skill>/routing-eval.jsonl to fix: ${lint.detail}`,
|
||||
});
|
||||
}
|
||||
const routingReport = runRoutingEval(resolverContent, loaded.fixtures);
|
||||
for (const d of routingReport.details) {
|
||||
if (d.outcome === 'pass') continue;
|
||||
const kind =
|
||||
d.outcome === 'missed'
|
||||
? 'routing_miss'
|
||||
: d.outcome === 'ambiguous'
|
||||
? 'routing_ambiguous'
|
||||
: 'routing_false_positive';
|
||||
issues.push({
|
||||
type: kind,
|
||||
severity: 'warning',
|
||||
skill: d.fixture.expected_skill ?? 'negative-case',
|
||||
message: `Routing ${d.outcome} for intent "${d.fixture.intent}"`,
|
||||
action: `Update routing-eval.jsonl fixture or broaden resolver triggers in RESOLVER.md (${d.note ?? 'no additional detail'})`,
|
||||
});
|
||||
}
|
||||
}
|
||||
for (const m of loaded.malformed) {
|
||||
issues.push({
|
||||
type: 'routing_fixture_lint',
|
||||
severity: 'warning',
|
||||
skill: 'routing-eval',
|
||||
message: `Malformed routing fixture ${m.file}:${m.line}`,
|
||||
action: `Fix the JSONL in routing-eval.jsonl at line ${m.line}: ${m.error}`,
|
||||
});
|
||||
}
|
||||
|
||||
// D-CX-9 SKILLIFY_STUB sentinel check: scan every SKILL.md + script
|
||||
// file under skillsDir for the sentinel marker emitted by
|
||||
// `gbrain skillify scaffold`. Presence means a scaffolded skill
|
||||
// shipped without a real implementation — warning-severity in
|
||||
// default mode, error-promoted under --strict via D-CX-3.
|
||||
for (const skill of manifest) {
|
||||
const skillDir = join(skillsDir, skill.path.replace(/\/SKILL\.md$/, ''));
|
||||
const scriptDir = join(skillDir, 'scripts');
|
||||
const candidates: string[] = [join(skillsDir, skill.path)];
|
||||
if (existsSync(scriptDir)) {
|
||||
try {
|
||||
for (const f of readdirSync(scriptDir)) {
|
||||
if (f.match(/\.(ts|mjs|js|py)$/)) candidates.push(join(scriptDir, f));
|
||||
}
|
||||
} catch {
|
||||
// Skip unreadable script dir.
|
||||
}
|
||||
}
|
||||
for (const candidate of candidates) {
|
||||
try {
|
||||
const content = readFileSync(candidate, 'utf-8');
|
||||
if (content.includes('SKILLIFY_STUB: replace before running check-resolvable --strict')) {
|
||||
issues.push({
|
||||
type: 'skillify_stub_unreplaced',
|
||||
severity: 'warning',
|
||||
skill: skill.name,
|
||||
message: `Skill '${skill.name}' still contains the SKILLIFY_STUB sentinel in ${relative(skillsDir, candidate)}`,
|
||||
action: `Replace the SKILLIFY_STUB sentinel in ${candidate} with a real implementation or remove the file. D-CX-9 gate.`,
|
||||
});
|
||||
break; // one issue per skill
|
||||
}
|
||||
} catch {
|
||||
// Skip unreadable file.
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Check 6 (W3, v0.17): brain-filing audit. Warning-only per
|
||||
// D-CX-3 + D-CX-5 — does not break CI for workspaces that haven't
|
||||
// adopted writes_pages:/writes_to: yet. Any errors in the rules
|
||||
// doc itself surface as a single fatal-ish entry.
|
||||
try {
|
||||
const filingReport = runFilingAudit(skillsDir);
|
||||
for (const issue of filingReport.issues) {
|
||||
issues.push(issue);
|
||||
}
|
||||
} catch (err) {
|
||||
issues.push({
|
||||
type: 'filing_unknown_directory',
|
||||
severity: 'warning',
|
||||
skill: 'brain-filing-rules',
|
||||
message: `_brain-filing-rules.json failed to load`,
|
||||
action: `Fix skills/_brain-filing-rules.json: ${(err as Error).message}`,
|
||||
});
|
||||
}
|
||||
|
||||
const errors = issues.filter(i => i.severity === 'error');
|
||||
const warnings = issues.filter(i => i.severity === 'warning');
|
||||
return {
|
||||
ok: issues.filter(i => i.severity === 'error').length === 0,
|
||||
ok: errors.length === 0,
|
||||
errors,
|
||||
warnings,
|
||||
issues,
|
||||
summary: {
|
||||
total_skills: manifest.length,
|
||||
|
||||
+6
-19
@@ -22,6 +22,7 @@ import {
|
||||
extractDelegationTargets,
|
||||
type CrossCuttingPattern,
|
||||
} from './check-resolvable.ts';
|
||||
import { loadOrDeriveManifest } from './skill-manifest.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
@@ -187,26 +188,12 @@ export function isWorkingTreeDirty(skillPath: string): boolean {
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Manifest loading (duplicated from check-resolvable.ts to avoid exporting
|
||||
// that internal helper — kept in sync via tests)
|
||||
// Manifest loading delegated to src/core/skill-manifest.ts. Using the
|
||||
// shared loader means auto-fix works in AGENTS.md-only workspaces where
|
||||
// manifest.json is absent — the derive-from-walk path kicks in and
|
||||
// auto-fix has the same skill set check-resolvable sees. D-CX-12.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface ManifestEntry {
|
||||
name: string;
|
||||
path: string;
|
||||
}
|
||||
|
||||
function loadManifest(skillsDir: string): ManifestEntry[] {
|
||||
const manifestPath = join(skillsDir, 'manifest.json');
|
||||
if (!existsSync(manifestPath)) return [];
|
||||
try {
|
||||
const content = JSON.parse(readFileSync(manifestPath, 'utf-8'));
|
||||
return content.skills || [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main function
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -223,7 +210,7 @@ export function autoFixDryViolations(
|
||||
): AutoFixReport {
|
||||
const fixed: FixOutcome[] = [];
|
||||
const skipped: FixOutcome[] = [];
|
||||
const manifest = loadManifest(skillsDir);
|
||||
const { skills: manifest } = loadOrDeriveManifest(skillsDir);
|
||||
|
||||
for (const skill of manifest) {
|
||||
const skillPath = join(skillsDir, skill.path);
|
||||
|
||||
@@ -0,0 +1,249 @@
|
||||
/**
|
||||
* filing-audit.ts — Check 6 of the skillify checklist (W3, v0.17).
|
||||
*
|
||||
* For every skill that writes brain pages (`writes_pages: true`),
|
||||
* verify that:
|
||||
* 1. The skill declares a non-empty `writes_to: [dir, ...]` frontmatter.
|
||||
* 2. Each directory in `writes_to:` is a valid filing target per
|
||||
* `skills/_brain-filing-rules.json`. `sources/` is explicitly
|
||||
* allowed (bulk data capture is a legitimate filing target).
|
||||
*
|
||||
* Important distinction (D-CX-7): `writes_pages: true` is distinct
|
||||
* from the pre-existing `mutating: true` field. `mutating:true` means
|
||||
* "has side effects" (any side effect — cron, config, report write).
|
||||
* `writes_pages:true` means "writes brain pages to a semantic
|
||||
* directory." Cron/config/report-writer skills set `mutating:true`
|
||||
* but NOT `writes_pages:true`, and so are correctly exempted from
|
||||
* filing-audit noise.
|
||||
*
|
||||
* v0.17 scope: declaration-level audit only (cheap, deterministic).
|
||||
* v0.18 plan: `filing-audit --pages` walks brain pages and infers
|
||||
* primary subject via LLM to catch real misfilings vs declarations
|
||||
* (D-CX-13).
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface FilingRule {
|
||||
kind: string;
|
||||
directory: string;
|
||||
examples?: string[];
|
||||
description?: string;
|
||||
}
|
||||
|
||||
export interface FilingRulesDoc {
|
||||
version: string;
|
||||
companion?: string;
|
||||
description?: string;
|
||||
rules: FilingRule[];
|
||||
sources_dir?: {
|
||||
directory: string;
|
||||
purpose: string;
|
||||
not_for?: string[];
|
||||
};
|
||||
notes?: string[];
|
||||
}
|
||||
|
||||
export interface FilingIssue {
|
||||
type: 'filing_missing_writes_to' | 'filing_unknown_directory';
|
||||
severity: 'warning';
|
||||
skill: string;
|
||||
directory?: string;
|
||||
message: string;
|
||||
action: string;
|
||||
}
|
||||
|
||||
export interface FilingReport {
|
||||
totalScanned: number;
|
||||
writesPagesSkills: number;
|
||||
issues: FilingIssue[];
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Rules loader
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Load canonical filing rules from `skillsDir/_brain-filing-rules.json`.
|
||||
* Returns null if the file is missing — filing-audit is a no-op until
|
||||
* the rules doc is in place. Throws on malformed JSON so the caller
|
||||
* surfaces a loud "rules doc is broken" signal instead of silently
|
||||
* degrading.
|
||||
*/
|
||||
export function loadFilingRules(skillsDir: string): FilingRulesDoc | null {
|
||||
const path = join(skillsDir, '_brain-filing-rules.json');
|
||||
if (!existsSync(path)) return null;
|
||||
const content = readFileSync(path, 'utf-8');
|
||||
const parsed = JSON.parse(content);
|
||||
if (!parsed || typeof parsed !== 'object') {
|
||||
throw new Error('_brain-filing-rules.json: top-level must be an object');
|
||||
}
|
||||
if (!Array.isArray(parsed.rules)) {
|
||||
throw new Error('_brain-filing-rules.json: "rules" must be an array');
|
||||
}
|
||||
return parsed as FilingRulesDoc;
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the canonical set of directories a skill is allowed to list in
|
||||
* `writes_to:`. Includes every rule's directory plus the special
|
||||
* `sources_dir` entry.
|
||||
*/
|
||||
export function allowedDirectories(rules: FilingRulesDoc): Set<string> {
|
||||
const set = new Set<string>();
|
||||
for (const r of rules.rules) set.add(normalizeDir(r.directory));
|
||||
if (rules.sources_dir?.directory) set.add(normalizeDir(rules.sources_dir.directory));
|
||||
return set;
|
||||
}
|
||||
|
||||
function normalizeDir(dir: string): string {
|
||||
// Accept `people`, `people/`, `/people`, `/people/` — normalize to
|
||||
// `people/` so comparisons are consistent.
|
||||
const trimmed = dir.trim().replace(/^\/+/, '').replace(/\/+$/, '');
|
||||
return trimmed.length > 0 ? `${trimmed}/` : '';
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Skill frontmatter parsing (minimal, tolerant)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SkillFrontmatter {
|
||||
name?: string;
|
||||
writes_pages?: boolean;
|
||||
writes_to?: string[];
|
||||
mutating?: boolean;
|
||||
raw: string;
|
||||
}
|
||||
|
||||
function parseFrontmatter(skillMdPath: string): SkillFrontmatter | null {
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(skillMdPath, 'utf-8');
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (!fmMatch) return null;
|
||||
const raw = fmMatch[1];
|
||||
const out: SkillFrontmatter = { raw };
|
||||
|
||||
const nameMatch = raw.match(/^name:\s*["']?([^"'\n]+?)["']?\s*$/m);
|
||||
if (nameMatch) out.name = nameMatch[1].trim();
|
||||
|
||||
const wpMatch = raw.match(/^writes_pages:\s*(true|false)\s*$/m);
|
||||
if (wpMatch) out.writes_pages = wpMatch[1] === 'true';
|
||||
|
||||
const mutMatch = raw.match(/^mutating:\s*(true|false)\s*$/m);
|
||||
if (mutMatch) out.mutating = mutMatch[1] === 'true';
|
||||
|
||||
// writes_to: supports inline `[a, b, c]` OR multi-line block list
|
||||
// writes_to:
|
||||
// - people/
|
||||
// - companies/
|
||||
// AND inline `writes_to: [people/, companies/]`
|
||||
const inlineWtMatch = raw.match(/^writes_to:\s*\[([^\]]*)\]\s*$/m);
|
||||
if (inlineWtMatch) {
|
||||
out.writes_to = inlineWtMatch[1]
|
||||
.split(',')
|
||||
.map(s => s.trim().replace(/^["']|["']$/g, ''))
|
||||
.filter(Boolean);
|
||||
} else {
|
||||
const blockMatch = raw.match(/^writes_to:\s*\n((?:\s+-\s+[^\n]+\n?)+)/m);
|
||||
if (blockMatch) {
|
||||
out.writes_to = blockMatch[1]
|
||||
.split('\n')
|
||||
.map(l => l.replace(/^\s+-\s+/, '').replace(/^["']|["']$/g, '').trim())
|
||||
.filter(Boolean);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Audit
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Scan every skill under `skillsDir`. For skills with
|
||||
* `writes_pages: true`:
|
||||
* - Missing `writes_to:` → warning.
|
||||
* - Any dir in `writes_to:` not in allowedDirectories → warning.
|
||||
*
|
||||
* Skills without `writes_pages:` (or with `writes_pages: false`) are
|
||||
* skipped — regardless of `mutating:` value. This is deliberate
|
||||
* (D-CX-7): filing-audit targets brain-page writers, not arbitrary
|
||||
* side effects.
|
||||
*/
|
||||
export function runFilingAudit(skillsDir: string): FilingReport {
|
||||
const issues: FilingIssue[] = [];
|
||||
const rules = loadFilingRules(skillsDir);
|
||||
if (!rules) {
|
||||
return { totalScanned: 0, writesPagesSkills: 0, issues };
|
||||
}
|
||||
const allowed = allowedDirectories(rules);
|
||||
|
||||
let totalScanned = 0;
|
||||
let writesPagesSkills = 0;
|
||||
|
||||
if (!existsSync(skillsDir)) {
|
||||
return { totalScanned, writesPagesSkills, issues };
|
||||
}
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(skillsDir);
|
||||
} catch {
|
||||
return { totalScanned, writesPagesSkills, issues };
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
if (entry.startsWith('.') || entry.startsWith('_')) continue;
|
||||
const dir = join(skillsDir, entry);
|
||||
try {
|
||||
if (!statSync(dir).isDirectory()) continue;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const skillMd = join(dir, 'SKILL.md');
|
||||
if (!existsSync(skillMd)) continue;
|
||||
totalScanned++;
|
||||
|
||||
const fm = parseFrontmatter(skillMd);
|
||||
if (!fm) continue;
|
||||
if (fm.writes_pages !== true) continue;
|
||||
writesPagesSkills++;
|
||||
|
||||
const skillName = fm.name ?? entry;
|
||||
|
||||
if (!fm.writes_to || fm.writes_to.length === 0) {
|
||||
issues.push({
|
||||
type: 'filing_missing_writes_to',
|
||||
severity: 'warning',
|
||||
skill: skillName,
|
||||
message: `Skill '${skillName}' has writes_pages: true but no writes_to: list`,
|
||||
action: `Add a writes_to: [dir, ...] list to skills/${entry}/SKILL.md frontmatter (see skills/_brain-filing-rules.json for valid directories)`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
|
||||
for (const rawDir of fm.writes_to) {
|
||||
const normalized = normalizeDir(rawDir);
|
||||
if (!allowed.has(normalized)) {
|
||||
issues.push({
|
||||
type: 'filing_unknown_directory',
|
||||
severity: 'warning',
|
||||
skill: skillName,
|
||||
directory: rawDir,
|
||||
message: `Skill '${skillName}' declares writes_to: '${rawDir}' which is not listed in _brain-filing-rules.json`,
|
||||
action: `Fix the writes_to: entry in skills/${entry}/SKILL.md or add '${normalized}' to skills/_brain-filing-rules.json rules[]`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return { totalScanned, writesPagesSkills, issues };
|
||||
}
|
||||
+140
-7
@@ -1,21 +1,154 @@
|
||||
import { existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { isAbsolute, join, resolve as resolvePath } from 'path';
|
||||
import { RESOLVER_FILENAMES, hasResolverFile } from './resolver-filenames.ts';
|
||||
|
||||
/**
|
||||
* Walk up from `startDir` looking for `skills/RESOLVER.md` — the marker of a
|
||||
* gbrain repo root. Returns the absolute directory containing `skills/` or
|
||||
* null if no such directory is found within 10 levels.
|
||||
* Walk up from `startDir` looking for a `skills/` directory that
|
||||
* contains a recognized resolver file (`RESOLVER.md` or `AGENTS.md`).
|
||||
* Returns the absolute directory containing `skills/` or null if no
|
||||
* such directory is found within 10 levels.
|
||||
*
|
||||
* `startDir` is parameterized so tests can run hermetically against fixtures.
|
||||
* Default matches the prior `doctor.ts`-private implementation.
|
||||
* `startDir` is parameterized so tests can run hermetically against
|
||||
* fixtures. Default matches the prior `doctor.ts`-private implementation.
|
||||
*/
|
||||
export function findRepoRoot(startDir: string = process.cwd()): string | null {
|
||||
let dir = startDir;
|
||||
for (let i = 0; i < 10; i++) {
|
||||
if (existsSync(join(dir, 'skills', 'RESOLVER.md'))) return dir;
|
||||
if (hasResolverFile(join(dir, 'skills'))) return dir;
|
||||
const parent = join(dir, '..');
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where auto-detect found the skills directory.
|
||||
* - `explicit` — user passed --skills-dir / equivalent
|
||||
* - `openclaw_workspace_env` — $OPENCLAW_WORKSPACE/skills
|
||||
* - `openclaw_workspace_env_root` — $OPENCLAW_WORKSPACE/ (AGENTS.md at
|
||||
* workspace root; skills in subdir)
|
||||
* - `openclaw_workspace_home` — ~/.openclaw/workspace/skills
|
||||
* - `openclaw_workspace_home_root` — ~/.openclaw/workspace (root AGENTS.md)
|
||||
* - `repo_root` — walked up from cwd, found gbrain repo
|
||||
* - `cwd_skills` — ./skills fallback
|
||||
*/
|
||||
export type SkillsDirSource =
|
||||
| 'openclaw_workspace_env'
|
||||
| 'openclaw_workspace_env_root'
|
||||
| 'openclaw_workspace_home'
|
||||
| 'openclaw_workspace_home_root'
|
||||
| 'repo_root'
|
||||
| 'cwd_skills';
|
||||
|
||||
export interface SkillsDirDetection {
|
||||
dir: string | null;
|
||||
source: SkillsDirSource | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Given a workspace root, resolve where the skills directory should
|
||||
* live. Returns the skills dir + the specific source variant. Returns
|
||||
* null if neither `workspace/skills/<RESOLVER|AGENTS>` nor
|
||||
* `workspace/<AGENTS|RESOLVER>` exists.
|
||||
*
|
||||
* `sourceSubdir` / `sourceRoot` let callers distinguish "skills-dir
|
||||
* variant" from "workspace-root variant" for --verbose logging.
|
||||
*/
|
||||
function resolveWorkspaceSkillsDir(
|
||||
workspace: string,
|
||||
sourceSubdir: SkillsDirSource,
|
||||
sourceRoot: SkillsDirSource,
|
||||
): SkillsDirDetection | null {
|
||||
// Preferred: workspace/skills with a resolver file inside it (gbrain-native).
|
||||
const subdir = join(workspace, 'skills');
|
||||
if (hasResolverFile(subdir)) {
|
||||
return { dir: subdir, source: sourceSubdir };
|
||||
}
|
||||
// Fallback: resolver file at workspace root (OpenClaw-native layout).
|
||||
// The skills/ subtree still governs file layout even when routing lives
|
||||
// at workspace root. Return the skills subdir so downstream file lookups
|
||||
// work; the resolver parser knows how to look one level up.
|
||||
if (hasResolverFile(workspace) && existsSync(subdir)) {
|
||||
return { dir: subdir, source: sourceRoot };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Auto-detect the skills directory. Priority (D-CX-4, post-codex-review):
|
||||
* 1. $OPENCLAW_WORKSPACE when explicitly set (env > repo-root walk)
|
||||
* 2. ~/.openclaw/workspace/ (user's default OpenClaw deployment)
|
||||
* 3. findRepoRoot() walk from cwd (gbrain's own repo)
|
||||
* 4. ./skills fallback (dev scratch, fixtures)
|
||||
*
|
||||
* The prior order put `findRepoRoot` first, which meant
|
||||
* `export OPENCLAW_WORKSPACE=...; gbrain check-resolvable` run from
|
||||
* inside the gbrain repo silently shadowed the env var by walking up
|
||||
* to gbrain's own skills/. Explicit env should win. Unset env → behavior
|
||||
* is unchanged from before.
|
||||
*
|
||||
* `startDir` + `env` params keep tests hermetic.
|
||||
*/
|
||||
export function autoDetectSkillsDir(
|
||||
startDir: string = process.cwd(),
|
||||
env: NodeJS.ProcessEnv = process.env,
|
||||
): SkillsDirDetection {
|
||||
// 1. $OPENCLAW_WORKSPACE wins when explicitly set.
|
||||
if (env.OPENCLAW_WORKSPACE) {
|
||||
const workspace = isAbsolute(env.OPENCLAW_WORKSPACE)
|
||||
? env.OPENCLAW_WORKSPACE
|
||||
: resolvePath(startDir, env.OPENCLAW_WORKSPACE);
|
||||
const resolved = resolveWorkspaceSkillsDir(
|
||||
workspace,
|
||||
'openclaw_workspace_env',
|
||||
'openclaw_workspace_env_root',
|
||||
);
|
||||
if (resolved) return resolved;
|
||||
}
|
||||
|
||||
// 2. ~/.openclaw/workspace as the default user-level OpenClaw deployment.
|
||||
if (env.HOME) {
|
||||
const workspace = join(env.HOME, '.openclaw', 'workspace');
|
||||
const resolved = resolveWorkspaceSkillsDir(
|
||||
workspace,
|
||||
'openclaw_workspace_home',
|
||||
'openclaw_workspace_home_root',
|
||||
);
|
||||
if (resolved) return resolved;
|
||||
}
|
||||
|
||||
// 3. gbrain repo walk from cwd.
|
||||
const repoRoot = findRepoRoot(startDir);
|
||||
if (repoRoot && isGbrainRepoRoot(repoRoot)) {
|
||||
return { dir: join(repoRoot, 'skills'), source: 'repo_root' };
|
||||
}
|
||||
|
||||
// 4. ./skills fallback.
|
||||
const cwdSkills = join(startDir, 'skills');
|
||||
if (hasResolverFile(cwdSkills)) {
|
||||
return { dir: cwdSkills, source: 'cwd_skills' };
|
||||
}
|
||||
|
||||
return { dir: null, source: null };
|
||||
}
|
||||
|
||||
function isGbrainRepoRoot(dir: string): boolean {
|
||||
return (
|
||||
existsSync(join(dir, 'src', 'cli.ts')) &&
|
||||
hasResolverFile(join(dir, 'skills'))
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Human-readable summary of the resolver-file search paths, for error
|
||||
* messages when auto-detect fails. Mirrors the priority order used by
|
||||
* `autoDetectSkillsDir`.
|
||||
*/
|
||||
export const AUTO_DETECT_HINT = [
|
||||
` 1. --skills-dir flag`,
|
||||
` 2. $OPENCLAW_WORKSPACE/{skills/,}{${RESOLVER_FILENAMES.join(',')}}`,
|
||||
` 3. ~/.openclaw/workspace/{skills/,}{${RESOLVER_FILENAMES.join(',')}}`,
|
||||
` 4. repo root with skills/${RESOLVER_FILENAMES.join(' or skills/')}`,
|
||||
` 5. ./skills/${RESOLVER_FILENAMES.join(' or ./skills/')}`,
|
||||
].join('\n');
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
/**
|
||||
* resolver-filenames.ts — shared filename policy for the resolver file.
|
||||
*
|
||||
* gbrain-native convention: `RESOLVER.md`. OpenClaw convention (per
|
||||
* essay referenced in CLAUDE.md): `AGENTS.md`. Both are valid at the
|
||||
* same path (skills dir or workspace root for the OpenClaw layout).
|
||||
* When both exist at a location, `RESOLVER.md` wins by policy —
|
||||
* gbrain-native precedence keeps gbrain's own repo unaffected.
|
||||
*
|
||||
* One source of truth. Imported by `repo-root.ts` (auto-detect) and
|
||||
* `check-resolvable.ts` (parser + error messages). Never hardcode
|
||||
* `RESOLVER.md` in new code — import from here.
|
||||
*/
|
||||
|
||||
import { existsSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
/** Ordered: first-match wins. Do not reorder without updating tests. */
|
||||
export const RESOLVER_FILENAMES = ['RESOLVER.md', 'AGENTS.md'] as const;
|
||||
|
||||
export type ResolverFilename = (typeof RESOLVER_FILENAMES)[number];
|
||||
|
||||
/**
|
||||
* Return the first existing resolver file in `dir`, or null.
|
||||
* Pass the directory — this function joins for you.
|
||||
*/
|
||||
export function findResolverFile(dir: string): string | null {
|
||||
for (const name of RESOLVER_FILENAMES) {
|
||||
const candidate = join(dir, name);
|
||||
if (existsSync(candidate)) return candidate;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** True iff `dir` contains at least one recognized resolver file. */
|
||||
export function hasResolverFile(dir: string): boolean {
|
||||
return findResolverFile(dir) !== null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Human-readable list for error messages. Example:
|
||||
* "RESOLVER.md or AGENTS.md"
|
||||
*/
|
||||
export const RESOLVER_FILENAMES_LABEL = RESOLVER_FILENAMES.join(' or ');
|
||||
@@ -0,0 +1,390 @@
|
||||
/**
|
||||
* routing-eval.ts — Check 5 of the skillify checklist.
|
||||
*
|
||||
* Validates that given a user intent, the skill-resolver table routes to
|
||||
* the correct skill. Two layers (per the essay's "both layers matter"
|
||||
* framing):
|
||||
*
|
||||
* Layer A (structural): always runs, no LLM. Normalize both the intent
|
||||
* and each resolver trigger phrase, then check if any trigger is a
|
||||
* substring of the intent. A fixture `expected_skill` passes iff:
|
||||
* - that skill's trigger matches AND
|
||||
* - no other skill's trigger matches (unambiguous)
|
||||
* Supports negative cases (`expected_skill: null` — nothing should
|
||||
* match) and ambiguity declarations (`ambiguous_with: [...]` — list
|
||||
* of skills this intent is allowed to also match).
|
||||
*
|
||||
* Layer B (LLM tie-break, optional): only runs via `gbrain routing-eval
|
||||
* --llm`. Not yet implemented in v0.17 core; the CLI stubs the flag
|
||||
* so call sites are ready.
|
||||
*
|
||||
* Fixture linter (D-CX-6): we reject fixtures where the normalized
|
||||
* `intent` is a verbatim substring of any trigger phrase attached to
|
||||
* its `expected_skill`. Copying trigger text into the intent turns
|
||||
* Layer A into a tautology (trigger ⊂ trigger). Fixtures must paraphrase.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { parseResolverEntries } from './check-resolvable.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixture types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface RoutingFixture {
|
||||
/** Natural-language user intent. Required. */
|
||||
intent: string;
|
||||
/**
|
||||
* Skill slug (matches the directory name under `skills/`) that should
|
||||
* fire. Use `null` for negative cases: "nothing should match this intent."
|
||||
*/
|
||||
expected_skill: string | null;
|
||||
/**
|
||||
* Optional: skills the intent is ALLOWED to also match without being
|
||||
* flagged as ambiguous. Use for always-on skills that naturally
|
||||
* co-fire (signal-detector, brain-ops). Skills listed here are
|
||||
* exempted from the ambiguity check; a match that includes
|
||||
* `expected_skill` and zero-or-more `ambiguous_with` entries is
|
||||
* considered unambiguous.
|
||||
*/
|
||||
ambiguous_with?: string[];
|
||||
/** Optional: source path this fixture came from. Populated by loader. */
|
||||
source?: string;
|
||||
}
|
||||
|
||||
export interface RoutingReport {
|
||||
totalCases: number;
|
||||
top1Accuracy: number; // 0..1
|
||||
passed: number;
|
||||
missed: number;
|
||||
ambiguous: number;
|
||||
falsePositives: number; // negative cases that matched something
|
||||
details: RoutingCaseResult[];
|
||||
}
|
||||
|
||||
export type RoutingOutcome =
|
||||
| 'pass'
|
||||
| 'missed' // expected_skill was not in match set
|
||||
| 'ambiguous' // matched expected AND others not listed in ambiguous_with
|
||||
| 'false_positive'; // negative case (expected null) matched something
|
||||
|
||||
export interface RoutingCaseResult {
|
||||
fixture: RoutingFixture;
|
||||
outcome: RoutingOutcome;
|
||||
matchedSkills: string[];
|
||||
note?: string;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Normalization + trigger extraction
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Normalize a string for routing comparison:
|
||||
* - lowercase
|
||||
* - replace any non-alphanumeric char with a space
|
||||
* - collapse whitespace
|
||||
* - trim
|
||||
*
|
||||
* Stripping punctuation is deliberately aggressive. Question marks,
|
||||
* quotes, dashes, commas, and apostrophes all collapse to spaces. This
|
||||
* means `"What's up?"` and `whats up` compare equal — which is what
|
||||
* a routing match should do. The cost is slightly over-permissive
|
||||
* matching; the benefit is reliable matches across quote/punctuation
|
||||
* variants that agents emit in practice.
|
||||
*/
|
||||
export function normalizeText(s: string): string {
|
||||
return s.toLowerCase().replace(/[^a-z0-9]+/g, ' ').trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract candidate trigger phrases from a resolver cell. Two shapes:
|
||||
* 1. Cell contains double-quoted strings → return each quoted phrase
|
||||
* separately. Example: `"what do we know about", "tell me about"`
|
||||
* → ["what do we know about", "tell me about"].
|
||||
* 2. Cell has no quotes → return [whole cell] as one phrase.
|
||||
* Example: `Creating/enriching a person or company page` →
|
||||
* ["creating enriching a person or company page"] (normalized).
|
||||
*
|
||||
* All returned phrases are normalized via `normalizeText`.
|
||||
*/
|
||||
export function extractTriggerPhrases(cellText: string): string[] {
|
||||
const quoted = [...cellText.matchAll(/"([^"]+)"/g)].map(m => m[1]);
|
||||
const source = quoted.length > 0 ? quoted : [cellText];
|
||||
return source
|
||||
.map(normalizeText)
|
||||
.filter(s => s.length >= 3); // drop empty or trivially-short phrases
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Resolver → skill-to-phrases index
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SkillTriggerIndex {
|
||||
/** Map of skill slug → set of normalized trigger phrases. */
|
||||
skillPhrases: Map<string, string[]>;
|
||||
}
|
||||
|
||||
/** Skill slug extracted from a resolver skillPath like `skills/foo/SKILL.md` → `foo`. */
|
||||
function skillSlugFromPath(skillPath: string): string | null {
|
||||
const m = skillPath.match(/^skills\/([^/]+)\/SKILL\.md/);
|
||||
return m ? m[1] : null;
|
||||
}
|
||||
|
||||
export function indexResolverTriggers(resolverContent: string): SkillTriggerIndex {
|
||||
const entries = parseResolverEntries(resolverContent);
|
||||
const skillPhrases = new Map<string, string[]>();
|
||||
for (const e of entries) {
|
||||
if (e.isGStack) continue;
|
||||
const slug = skillSlugFromPath(e.skillPath);
|
||||
if (!slug) continue;
|
||||
const phrases = extractTriggerPhrases(e.trigger);
|
||||
const existing = skillPhrases.get(slug) ?? [];
|
||||
skillPhrases.set(slug, [...existing, ...phrases]);
|
||||
}
|
||||
return { skillPhrases };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Structural routing match
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface StructuralMatchResult {
|
||||
/** Skills whose trigger phrases are substrings of the normalized intent. */
|
||||
matched: string[];
|
||||
/** True if more than one non-always-on skill matched. */
|
||||
ambiguous: boolean;
|
||||
}
|
||||
|
||||
/** Always-on skills routinely co-fire; a match that includes them
|
||||
* alongside a specific target skill is NOT ambiguous. */
|
||||
const ALWAYS_ON_SKILLS = new Set(['signal-detector', 'brain-ops', 'ingest']);
|
||||
|
||||
export function structuralRouteMatch(
|
||||
intent: string,
|
||||
index: SkillTriggerIndex,
|
||||
): StructuralMatchResult {
|
||||
const normalizedIntent = normalizeText(intent);
|
||||
const matched: string[] = [];
|
||||
for (const [slug, phrases] of index.skillPhrases) {
|
||||
for (const phrase of phrases) {
|
||||
if (phrase.length === 0) continue;
|
||||
if (normalizedIntent.includes(phrase)) {
|
||||
matched.push(slug);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
const specific = matched.filter(s => !ALWAYS_ON_SKILLS.has(s));
|
||||
return { matched, ambiguous: specific.length > 1 };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fixture linter + loader
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface FixtureLintIssue {
|
||||
fixture: RoutingFixture;
|
||||
reason: 'intent_copies_trigger' | 'unknown_expected_skill' | 'invalid_shape';
|
||||
detail: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Lint fixtures against the resolver (D-CX-6). Reject cases where:
|
||||
* - The normalized intent EQUALS any trigger phrase for its
|
||||
* expected skill (pure tautology — the fixture is the trigger).
|
||||
* - The expected_skill is unknown to the resolver.
|
||||
*
|
||||
* We deliberately do NOT reject intents that merely CONTAIN trigger
|
||||
* words in a natural sentence (e.g. "please look up that paper"
|
||||
* containing trigger "look up"). Layer A's whole mechanism is
|
||||
* substring match on the resolver triggers; a fixture that embeds
|
||||
* trigger words in surrounding context is valid and useful. The
|
||||
* linter's job is to catch copy-paste tautologies, not word overlap.
|
||||
*/
|
||||
export function lintRoutingFixtures(
|
||||
fixtures: RoutingFixture[],
|
||||
index: SkillTriggerIndex,
|
||||
): FixtureLintIssue[] {
|
||||
const issues: FixtureLintIssue[] = [];
|
||||
for (const f of fixtures) {
|
||||
if (typeof f.intent !== 'string' || f.intent.trim().length === 0) {
|
||||
issues.push({
|
||||
fixture: f,
|
||||
reason: 'invalid_shape',
|
||||
detail: 'intent must be a non-empty string',
|
||||
});
|
||||
continue;
|
||||
}
|
||||
if (f.expected_skill !== null && typeof f.expected_skill !== 'string') {
|
||||
issues.push({
|
||||
fixture: f,
|
||||
reason: 'invalid_shape',
|
||||
detail: 'expected_skill must be a string or null',
|
||||
});
|
||||
continue;
|
||||
}
|
||||
// Negative case (null) can't copy a trigger — skip that check.
|
||||
if (f.expected_skill === null) continue;
|
||||
if (!index.skillPhrases.has(f.expected_skill)) {
|
||||
issues.push({
|
||||
fixture: f,
|
||||
reason: 'unknown_expected_skill',
|
||||
detail: `expected_skill '${f.expected_skill}' is not in the resolver`,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
const normalizedIntent = normalizeText(f.intent);
|
||||
const phrases = index.skillPhrases.get(f.expected_skill) ?? [];
|
||||
for (const phrase of phrases) {
|
||||
if (phrase.length > 0 && normalizedIntent === phrase) {
|
||||
issues.push({
|
||||
fixture: f,
|
||||
reason: 'intent_copies_trigger',
|
||||
detail: `intent is verbatim-identical to trigger phrase '${phrase}'`,
|
||||
});
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk each child of skillsDir looking for `routing-eval.jsonl` and
|
||||
* return all fixtures with the source path attached. JSONL format:
|
||||
* one JSON object per non-empty line; lines starting with `//` or `#`
|
||||
* are skipped as comments. Malformed lines are returned as
|
||||
* no-fixture-but-log events via the returned `malformed[]` array.
|
||||
*/
|
||||
export interface LoadResult {
|
||||
fixtures: RoutingFixture[];
|
||||
malformed: { file: string; line: number; raw: string; error: string }[];
|
||||
}
|
||||
|
||||
export function loadRoutingFixtures(skillsDir: string): LoadResult {
|
||||
const fixtures: RoutingFixture[] = [];
|
||||
const malformed: LoadResult['malformed'] = [];
|
||||
if (!existsSync(skillsDir)) return { fixtures, malformed };
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(skillsDir);
|
||||
} catch {
|
||||
return { fixtures, malformed };
|
||||
}
|
||||
for (const entry of entries) {
|
||||
if (entry.startsWith('.') || entry.startsWith('_')) continue;
|
||||
const dir = join(skillsDir, entry);
|
||||
try {
|
||||
if (!statSync(dir).isDirectory()) continue;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const fixturePath = join(dir, 'routing-eval.jsonl');
|
||||
if (!existsSync(fixturePath)) continue;
|
||||
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(fixturePath, 'utf-8');
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const lines = content.split('\n');
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const raw = lines[i].trim();
|
||||
if (!raw) continue;
|
||||
if (raw.startsWith('//') || raw.startsWith('#')) continue;
|
||||
try {
|
||||
const obj = JSON.parse(raw) as RoutingFixture;
|
||||
fixtures.push({ ...obj, source: fixturePath });
|
||||
} catch (err) {
|
||||
malformed.push({
|
||||
file: fixturePath,
|
||||
line: i + 1,
|
||||
raw,
|
||||
error: (err as Error).message,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return { fixtures, malformed };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main eval runner
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface RunRoutingEvalOptions {
|
||||
/** Reserved for Layer B (LLM tie-break). Not implemented in v0.17. */
|
||||
llm?: boolean;
|
||||
}
|
||||
|
||||
export function runRoutingEval(
|
||||
resolverContent: string,
|
||||
fixtures: RoutingFixture[],
|
||||
_opts: RunRoutingEvalOptions = {},
|
||||
): RoutingReport {
|
||||
const index = indexResolverTriggers(resolverContent);
|
||||
const details: RoutingCaseResult[] = [];
|
||||
let passed = 0;
|
||||
let missed = 0;
|
||||
let ambiguous = 0;
|
||||
let falsePositives = 0;
|
||||
|
||||
for (const fixture of fixtures) {
|
||||
const result = structuralRouteMatch(fixture.intent, index);
|
||||
let outcome: RoutingOutcome;
|
||||
let note: string | undefined;
|
||||
|
||||
if (fixture.expected_skill === null) {
|
||||
// Negative case: nothing specific should match.
|
||||
const specific = result.matched.filter(s => !ALWAYS_ON_SKILLS.has(s));
|
||||
if (specific.length === 0) {
|
||||
outcome = 'pass';
|
||||
passed++;
|
||||
} else {
|
||||
outcome = 'false_positive';
|
||||
falsePositives++;
|
||||
note = `negative case unexpectedly matched: ${specific.join(', ')}`;
|
||||
}
|
||||
} else if (!result.matched.includes(fixture.expected_skill)) {
|
||||
outcome = 'missed';
|
||||
missed++;
|
||||
note =
|
||||
result.matched.length > 0
|
||||
? `matched instead: ${result.matched.join(', ')}`
|
||||
: 'no matches';
|
||||
} else {
|
||||
// expected_skill matched; check for ambiguity beyond the allow-list.
|
||||
const allowed = new Set([
|
||||
...(fixture.ambiguous_with ?? []),
|
||||
...ALWAYS_ON_SKILLS,
|
||||
fixture.expected_skill,
|
||||
]);
|
||||
const unexpected = result.matched.filter(s => !allowed.has(s));
|
||||
if (unexpected.length === 0) {
|
||||
outcome = 'pass';
|
||||
passed++;
|
||||
} else {
|
||||
outcome = 'ambiguous';
|
||||
ambiguous++;
|
||||
note = `also matched: ${unexpected.join(', ')}`;
|
||||
}
|
||||
}
|
||||
details.push({ fixture, outcome, matchedSkills: result.matched, note });
|
||||
}
|
||||
|
||||
const totalCases = fixtures.length;
|
||||
const top1Accuracy = totalCases === 0 ? 1 : passed / totalCases;
|
||||
|
||||
return {
|
||||
totalCases,
|
||||
top1Accuracy,
|
||||
passed,
|
||||
missed,
|
||||
ambiguous,
|
||||
falsePositives,
|
||||
details,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
/**
|
||||
* skill-manifest.ts — unified manifest loader for resolver checks.
|
||||
*
|
||||
* Two call sites converge here (D-CX-12, F-ENG-1):
|
||||
* - `src/core/check-resolvable.ts` — reachability check
|
||||
* - `src/core/dry-fix.ts` — auto-fix walker
|
||||
*
|
||||
* Both previously had their own `loadManifest` returning `[]` on missing
|
||||
* manifest.json. That silently disabled reachability + auto-fix for any
|
||||
* workspace (like an OpenClaw deployment) that doesn't ship a manifest
|
||||
* alongside its skills — the exact scenario this release unblocks.
|
||||
*
|
||||
* Behavior:
|
||||
* 1. If `skillsDir/manifest.json` exists and parses, use it verbatim.
|
||||
* 2. Otherwise, walk `skillsDir/*` dirs. For each dir containing a
|
||||
* `SKILL.md`, construct `{name, path}` by reading `name:` from the
|
||||
* frontmatter; fall back to the dirname if frontmatter is absent
|
||||
* or unparseable. This is the "auto-derive" path.
|
||||
* 3. Return `{skills, derived: boolean}` so callers can surface
|
||||
* derived-manifest mode in `--verbose` / `--json` output.
|
||||
*
|
||||
* Dotfile dirs (names starting with `_` or `.`) are excluded — they
|
||||
* hold conventions and shared rule files, not skills. Files like
|
||||
* `_brain-filing-rules.md` live at the root and are not considered
|
||||
* skills by either loader.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, readdirSync, statSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
export interface ManifestEntry {
|
||||
name: string;
|
||||
path: string; // relative to skillsDir, e.g. "query/SKILL.md"
|
||||
}
|
||||
|
||||
export interface ManifestLoadResult {
|
||||
skills: ManifestEntry[];
|
||||
/** True when manifest.json was missing/unparseable and the skill set
|
||||
* was derived from walking skillsDir. Surfaces in --verbose output. */
|
||||
derived: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the `name:` field from a skill's YAML frontmatter.
|
||||
* Tolerant: returns null if no frontmatter, no `name:` key, or unparseable.
|
||||
*/
|
||||
function parseSkillName(skillMdPath: string): string | null {
|
||||
try {
|
||||
const content = readFileSync(skillMdPath, 'utf-8');
|
||||
const fmMatch = content.match(/^---\n([\s\S]*?)\n---/);
|
||||
if (!fmMatch) return null;
|
||||
const fm = fmMatch[1];
|
||||
// Match `name: foo` or `name: "foo"` or `name: 'foo'`
|
||||
const nameMatch = fm.match(/^name:\s*["']?([^"'\n]+?)["']?\s*$/m);
|
||||
if (!nameMatch) return null;
|
||||
const name = nameMatch[1].trim();
|
||||
return name || null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk skillsDir, return every `<skillsDir>/<dir>/SKILL.md` as a
|
||||
* ManifestEntry. Dotfile and underscore-prefixed dirs are skipped.
|
||||
*/
|
||||
function deriveManifest(skillsDir: string): ManifestEntry[] {
|
||||
const out: ManifestEntry[] = [];
|
||||
if (!existsSync(skillsDir)) return out;
|
||||
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(skillsDir);
|
||||
} catch {
|
||||
return out;
|
||||
}
|
||||
|
||||
for (const entry of entries) {
|
||||
// Skip hidden dirs and convention-family sibling dirs (_conventions/,
|
||||
// conventions/, migrations/, recipes/, etc. are never "skills" in the
|
||||
// routing sense). Only entries with a direct `SKILL.md` count.
|
||||
if (entry.startsWith('.') || entry.startsWith('_')) continue;
|
||||
|
||||
const subdirAbs = join(skillsDir, entry);
|
||||
let isDir = false;
|
||||
try {
|
||||
isDir = statSync(subdirAbs).isDirectory();
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (!isDir) continue;
|
||||
|
||||
const skillMd = join(subdirAbs, 'SKILL.md');
|
||||
if (!existsSync(skillMd)) continue;
|
||||
|
||||
const frontmatterName = parseSkillName(skillMd);
|
||||
const name = frontmatterName && frontmatterName !== '' ? frontmatterName : entry;
|
||||
out.push({ name, path: `${entry}/SKILL.md` });
|
||||
}
|
||||
|
||||
out.sort((a, b) => a.name.localeCompare(b.name));
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Load the skill manifest from `skillsDir/manifest.json`, or derive it
|
||||
* from walking `skillsDir` when manifest.json is missing or malformed.
|
||||
*
|
||||
* Canonical entry point. New code should call THIS, not reach into
|
||||
* manifest.json directly.
|
||||
*/
|
||||
export function loadOrDeriveManifest(skillsDir: string): ManifestLoadResult {
|
||||
const manifestPath = join(skillsDir, 'manifest.json');
|
||||
|
||||
if (existsSync(manifestPath)) {
|
||||
try {
|
||||
const content = JSON.parse(readFileSync(manifestPath, 'utf-8'));
|
||||
// Strict shape gate: `skills` MUST be an array of `{name, path}`.
|
||||
// Anything else (missing, object-shaped, entries missing keys) is
|
||||
// treated as malformed and falls through to the derive path. This
|
||||
// is deliberately stricter than "empty array is fine" because a
|
||||
// malformed manifest.json on an OpenClaw deployment otherwise
|
||||
// silently disables reachability (F-ENG-1).
|
||||
if (Array.isArray(content?.skills)) {
|
||||
const skills = content.skills;
|
||||
// Empty array is a valid explicit declaration ("no skills yet").
|
||||
// Non-empty must have valid shape on every entry.
|
||||
const valid = skills.every(
|
||||
(s: unknown): s is ManifestEntry =>
|
||||
typeof s === 'object' &&
|
||||
s !== null &&
|
||||
typeof (s as ManifestEntry).name === 'string' &&
|
||||
typeof (s as ManifestEntry).path === 'string'
|
||||
);
|
||||
if (valid) {
|
||||
return { skills: skills as ManifestEntry[], derived: false };
|
||||
}
|
||||
}
|
||||
// Non-array skills or entries missing keys → derive
|
||||
} catch {
|
||||
// Unparseable manifest → fall through to derive
|
||||
}
|
||||
}
|
||||
|
||||
return { skills: deriveManifest(skillsDir), derived: true };
|
||||
}
|
||||
@@ -0,0 +1,177 @@
|
||||
/**
|
||||
* skillify/generator.ts — pure file-tree generator for `gbrain skillify scaffold`.
|
||||
*
|
||||
* Takes a scaffold spec + target skillsDir and returns the list of
|
||||
* files that would be written (dry-run) or writes them (apply).
|
||||
*
|
||||
* Idempotency contract (D-CX-7): `--force` regenerates STUB files
|
||||
* but NEVER re-appends resolver rows if a row for this skill path
|
||||
* already exists. The resolver row append is idempotent by content.
|
||||
*/
|
||||
|
||||
import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'fs';
|
||||
import { dirname, join } from 'path';
|
||||
|
||||
import {
|
||||
resolverRow,
|
||||
routingEvalTemplate,
|
||||
scriptTemplate,
|
||||
skillMdTemplate,
|
||||
testTemplate,
|
||||
type ScaffoldVars,
|
||||
} from './templates.ts';
|
||||
import { findResolverFile, RESOLVER_FILENAMES_LABEL } from '../resolver-filenames.ts';
|
||||
|
||||
export interface ScaffoldPlan {
|
||||
files: Array<{ path: string; kind: 'new' | 'overwrite' | 'append'; content: string }>;
|
||||
resolverFile: string | null;
|
||||
resolverAppend: string | null; // null when row already present (idempotent)
|
||||
}
|
||||
|
||||
export interface ScaffoldOptions {
|
||||
/** Absolute path to the target `skills/` dir. */
|
||||
skillsDir: string;
|
||||
/** Scaffold variables (name, description, triggers, etc.). */
|
||||
vars: ScaffoldVars;
|
||||
/**
|
||||
* Repo root for the `test/` and `scripts/` directories. Falls back
|
||||
* to `dirname(skillsDir)` when unset. Tests pass explicit values.
|
||||
*/
|
||||
repoRoot?: string;
|
||||
/** When true, overwrite existing skill files. Per-file (D-CX-7). */
|
||||
force?: boolean;
|
||||
}
|
||||
|
||||
const SKILL_NAME_PATTERN = /^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$/;
|
||||
|
||||
export class SkillifyScaffoldError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code:
|
||||
| 'invalid_name'
|
||||
| 'exists'
|
||||
| 'no_resolver'
|
||||
| 'write_failed',
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'SkillifyScaffoldError';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the list of files + the resolver-append string without doing
|
||||
* any I/O that writes. Callers can preview via dry-run, then pass the
|
||||
* same inputs to `applyScaffold`.
|
||||
*/
|
||||
export function planScaffold(opts: ScaffoldOptions): ScaffoldPlan {
|
||||
const { vars, skillsDir } = opts;
|
||||
if (!SKILL_NAME_PATTERN.test(vars.name)) {
|
||||
throw new SkillifyScaffoldError(
|
||||
`'${vars.name}' is not a valid skill name. Must be lowercase-kebab-case (examples: webhook-verify, context-now).`,
|
||||
'invalid_name',
|
||||
);
|
||||
}
|
||||
|
||||
const repoRoot = opts.repoRoot ?? dirname(skillsDir);
|
||||
const skillDir = join(skillsDir, vars.name);
|
||||
const skillMdPath = join(skillDir, 'SKILL.md');
|
||||
const scriptPath = join(skillDir, 'scripts', `${vars.name}.mjs`);
|
||||
const routingEvalPath = join(skillDir, 'routing-eval.jsonl');
|
||||
const testPath = join(repoRoot, 'test', `${vars.name}.test.ts`);
|
||||
|
||||
const files: ScaffoldPlan['files'] = [];
|
||||
|
||||
const want = (path: string, content: string) => {
|
||||
if (existsSync(path)) {
|
||||
if (!opts.force) {
|
||||
throw new SkillifyScaffoldError(
|
||||
`'${path}' already exists. Pass --force to regenerate stubs (destructive to any local edits), or edit the file directly.`,
|
||||
'exists',
|
||||
);
|
||||
}
|
||||
files.push({ path, kind: 'overwrite', content });
|
||||
} else {
|
||||
files.push({ path, kind: 'new', content });
|
||||
}
|
||||
};
|
||||
|
||||
want(skillMdPath, skillMdTemplate(vars));
|
||||
want(scriptPath, scriptTemplate(vars));
|
||||
want(routingEvalPath, routingEvalTemplate(vars));
|
||||
want(testPath, testTemplate(vars));
|
||||
|
||||
// Resolver row — append to whichever file exists; `null` both fields
|
||||
// if no resolver exists (caller handles setup error).
|
||||
const resolverFile =
|
||||
findResolverFile(skillsDir) ?? findResolverFile(dirname(skillsDir));
|
||||
let resolverAppend: string | null = null;
|
||||
if (resolverFile) {
|
||||
const existingRow = detectExistingResolverRow(resolverFile, vars.name);
|
||||
if (!existingRow) {
|
||||
resolverAppend = buildResolverAppend(resolverFile, vars);
|
||||
}
|
||||
}
|
||||
|
||||
return { files, resolverFile, resolverAppend };
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether the resolver already has a backtick-wrapped reference
|
||||
* to `skills/<name>/SKILL.md`. Idempotency contract (D-CX-7) — if
|
||||
* present, we never re-append a row for this skill, even with --force.
|
||||
*/
|
||||
function detectExistingResolverRow(resolverFile: string, name: string): boolean {
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(resolverFile, 'utf-8');
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
const escaped = name.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
|
||||
const re = new RegExp(`\`skills\\/${escaped}\\/SKILL\\.md\``);
|
||||
return re.test(content);
|
||||
}
|
||||
|
||||
function buildResolverAppend(resolverFile: string, vars: ScaffoldVars): string {
|
||||
// Append under a `## Uncategorized` section. If the section already
|
||||
// exists, just add the row; otherwise create the section.
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(resolverFile, 'utf-8');
|
||||
} catch {
|
||||
content = '';
|
||||
}
|
||||
|
||||
const row = resolverRow(vars);
|
||||
const hasUncategorized = /^## Uncategorized\s*$/m.test(content);
|
||||
if (hasUncategorized) {
|
||||
return '\n' + row + '\n';
|
||||
}
|
||||
const needsLeadingNewline = content.endsWith('\n') ? '' : '\n';
|
||||
return (
|
||||
needsLeadingNewline +
|
||||
'\n## Uncategorized\n\n| Trigger | Skill |\n|---------|-------|\n' +
|
||||
row +
|
||||
'\n'
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a previously-computed ScaffoldPlan. I/O only — no planning.
|
||||
* Callers that want dry-run behavior should skip this call entirely
|
||||
* and just render the plan.
|
||||
*/
|
||||
export function applyScaffold(plan: ScaffoldPlan): void {
|
||||
for (const f of plan.files) {
|
||||
mkdirSync(dirname(f.path), { recursive: true });
|
||||
writeFileSync(f.path, f.content);
|
||||
}
|
||||
if (plan.resolverFile && plan.resolverAppend !== null) {
|
||||
const current = existsSync(plan.resolverFile)
|
||||
? readFileSync(plan.resolverFile, 'utf-8')
|
||||
: '';
|
||||
writeFileSync(plan.resolverFile, current + plan.resolverAppend);
|
||||
}
|
||||
}
|
||||
|
||||
export { SKILL_NAME_PATTERN };
|
||||
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* skillify/templates.ts — template strings for `gbrain skillify scaffold`.
|
||||
*
|
||||
* Pure-string generators. No I/O here; the caller writes files.
|
||||
*/
|
||||
|
||||
/**
|
||||
* SKILLIFY_STUB sentinel (D-CX-9). Every scaffolded script body
|
||||
* carries this marker until an implementer replaces it. `gbrain
|
||||
* check-resolvable --strict` fails if the sentinel is present in any
|
||||
* committed skill script — it means a scaffold shipped without a
|
||||
* real implementation.
|
||||
*/
|
||||
export const SKILLIFY_STUB_MARKER = 'SKILLIFY_STUB: replace before running check-resolvable --strict';
|
||||
|
||||
export interface ScaffoldVars {
|
||||
/** Skill slug — must be lowercase-kebab-case. */
|
||||
name: string;
|
||||
/** One-line description for the frontmatter. */
|
||||
description: string;
|
||||
/** List of trigger phrases; empty → seed a TBD placeholder. */
|
||||
triggers: string[];
|
||||
/** Directories this skill will write brain pages to; optional. */
|
||||
writesTo: string[];
|
||||
/** Whether to mark the skill as `writes_pages: true`. */
|
||||
writesPages: boolean;
|
||||
/** Whether to mark the skill as `mutating: true`. */
|
||||
mutating: boolean;
|
||||
}
|
||||
|
||||
export function skillMdTemplate(v: ScaffoldVars): string {
|
||||
const triggerLines =
|
||||
v.triggers.length > 0
|
||||
? v.triggers.map(t => ` - "${t.replace(/"/g, '\\"')}"`).join('\n')
|
||||
: ' - "TBD-trigger — replace with phrases users actually type"';
|
||||
const writesToLines =
|
||||
v.writesTo.length > 0 ? v.writesTo.map(d => ` - ${d}`).join('\n') : '';
|
||||
|
||||
const lines: string[] = [
|
||||
'---',
|
||||
`name: ${v.name}`,
|
||||
'version: 0.1.0',
|
||||
`description: ${v.description}`,
|
||||
'triggers:',
|
||||
triggerLines,
|
||||
];
|
||||
if (v.mutating) lines.push('mutating: true');
|
||||
if (v.writesPages) {
|
||||
lines.push('writes_pages: true');
|
||||
if (writesToLines) {
|
||||
lines.push('writes_to:');
|
||||
lines.push(writesToLines);
|
||||
}
|
||||
}
|
||||
lines.push('---');
|
||||
lines.push('');
|
||||
lines.push(`# ${v.name}`);
|
||||
lines.push('');
|
||||
lines.push(`${v.description}`);
|
||||
lines.push('');
|
||||
lines.push('## The rule');
|
||||
lines.push('');
|
||||
lines.push(`<!-- ${SKILLIFY_STUB_MARKER} -->`);
|
||||
lines.push(
|
||||
'Replace this stub with the hard rule that prevents recurrence of the failure that triggered this skill.',
|
||||
);
|
||||
lines.push('');
|
||||
lines.push('## How to use');
|
||||
lines.push('');
|
||||
lines.push(
|
||||
`Run the deterministic script: \`bun scripts/${v.name}.mjs\` (or whatever your harness prefix is).`,
|
||||
);
|
||||
return lines.join('\n') + '\n';
|
||||
}
|
||||
|
||||
export function scriptTemplate(v: ScaffoldVars): string {
|
||||
// The SKILLIFY_STUB_MARKER in a comment is what check-resolvable
|
||||
// --strict looks for. Remove the marker (not the whole file) when
|
||||
// the script is implemented.
|
||||
return `#!/usr/bin/env bun
|
||||
// ${v.name} — scaffolded by gbrain skillify scaffold
|
||||
// ${SKILLIFY_STUB_MARKER}
|
||||
//
|
||||
// Replace this stub with the deterministic logic the skill needs.
|
||||
// Keep exports pure so tests can import them without side effects.
|
||||
|
||||
export function run(input: unknown): unknown {
|
||||
// TODO: implement. This stub is detected by \`gbrain check-resolvable
|
||||
// --strict\` and will fail CI until replaced.
|
||||
throw new Error('${v.name} scaffold not yet implemented');
|
||||
}
|
||||
|
||||
if (import.meta.main) {
|
||||
const input = process.argv.slice(2).join(' ');
|
||||
console.log(JSON.stringify(run(input)));
|
||||
}
|
||||
`;
|
||||
}
|
||||
|
||||
export function testTemplate(v: ScaffoldVars): string {
|
||||
return `/**
|
||||
* Tests for skills/${v.name}/scripts/${v.name}.mjs
|
||||
*
|
||||
* Scaffolded by gbrain skillify scaffold. Replace these stubs with
|
||||
* real cases — start with the regression case for the failure that
|
||||
* triggered this skill (essay Step 3).
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'bun:test';
|
||||
import { run } from '../skills/${v.name}/scripts/${v.name}.mjs';
|
||||
|
||||
describe('${v.name}', () => {
|
||||
it('is scaffolded — replace this test with a real regression case', () => {
|
||||
expect(() => run(null)).toThrow();
|
||||
});
|
||||
});
|
||||
`;
|
||||
}
|
||||
|
||||
/**
|
||||
* A single resolver table row for this skill. Uses the skill path
|
||||
* under `## Uncategorized`. The scaffolder handles the idempotency
|
||||
* contract (D-CX-7): never re-append a row that already exists.
|
||||
*/
|
||||
export function resolverRow(v: ScaffoldVars): string {
|
||||
const trigger =
|
||||
v.triggers.length > 0 ? v.triggers[0] : `TBD-trigger for ${v.name}`;
|
||||
return `| "${trigger.replace(/"/g, '\\"')}" | \`skills/${v.name}/SKILL.md\` |`;
|
||||
}
|
||||
|
||||
export function routingEvalTemplate(v: ScaffoldVars): string {
|
||||
if (v.triggers.length === 0) {
|
||||
return (
|
||||
'// Routing eval fixtures for skills/' +
|
||||
v.name +
|
||||
'. Add paraphrased intents.\n' +
|
||||
'// Each line: {"intent": "...", "expected_skill": "' +
|
||||
v.name +
|
||||
'"}\n'
|
||||
);
|
||||
}
|
||||
const lines = ['// Routing eval fixtures for skills/' + v.name + '.'];
|
||||
for (const t of v.triggers.slice(0, 3)) {
|
||||
const paraphrase = `please ${t.toLowerCase()} for me now`;
|
||||
lines.push(
|
||||
JSON.stringify({ intent: paraphrase, expected_skill: v.name }),
|
||||
);
|
||||
}
|
||||
return lines.join('\n') + '\n';
|
||||
}
|
||||
@@ -0,0 +1,226 @@
|
||||
/**
|
||||
* skillpack/bundle.ts — read the bundled-skills manifest.
|
||||
*
|
||||
* gbrain ships a curated set of skills (plus shared rule/convention
|
||||
* files they depend on) that agents install into their OpenClaw
|
||||
* workspace via `gbrain skillpack install`. The source of truth is
|
||||
* `openclaw.plugin.json` at the gbrain repo root.
|
||||
*/
|
||||
|
||||
import { existsSync, readFileSync, statSync, readdirSync } from 'fs';
|
||||
import { join, dirname, isAbsolute, resolve } from 'path';
|
||||
|
||||
export interface BundleManifest {
|
||||
name: string;
|
||||
version: string;
|
||||
description?: string;
|
||||
skills: string[]; // e.g. "skills/brain-ops" (relative to gbrain root)
|
||||
shared_deps: string[]; // files + dirs every skill depends on
|
||||
excluded_from_install?: string[];
|
||||
}
|
||||
|
||||
export class BundleError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code:
|
||||
| 'manifest_not_found'
|
||||
| 'manifest_malformed'
|
||||
| 'skill_not_found'
|
||||
| 'gbrain_root_not_found',
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'BundleError';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk up from `start` (default cwd) looking for an `openclaw.plugin.json`
|
||||
* sibling to `src/cli.ts`. That pair identifies a gbrain repo root.
|
||||
*/
|
||||
export function findGbrainRoot(start: string = process.cwd()): string | null {
|
||||
let dir = resolve(start);
|
||||
for (let i = 0; i < 10; i++) {
|
||||
if (
|
||||
existsSync(join(dir, 'openclaw.plugin.json')) &&
|
||||
existsSync(join(dir, 'src', 'cli.ts'))
|
||||
) {
|
||||
return dir;
|
||||
}
|
||||
const parent = dirname(dir);
|
||||
if (parent === dir) break;
|
||||
dir = parent;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse `openclaw.plugin.json` from the supplied gbrain root (absolute).
|
||||
* Throws BundleError on missing file or malformed JSON.
|
||||
*/
|
||||
export function loadBundleManifest(gbrainRoot: string): BundleManifest {
|
||||
const manifestPath = join(gbrainRoot, 'openclaw.plugin.json');
|
||||
if (!existsSync(manifestPath)) {
|
||||
throw new BundleError(
|
||||
`openclaw.plugin.json not found at ${manifestPath}`,
|
||||
'manifest_not_found',
|
||||
);
|
||||
}
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(manifestPath, 'utf-8');
|
||||
} catch (err) {
|
||||
throw new BundleError(
|
||||
`Failed to read ${manifestPath}: ${(err as Error).message}`,
|
||||
'manifest_malformed',
|
||||
);
|
||||
}
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(content);
|
||||
} catch (err) {
|
||||
throw new BundleError(
|
||||
`openclaw.plugin.json is not valid JSON: ${(err as Error).message}`,
|
||||
'manifest_malformed',
|
||||
);
|
||||
}
|
||||
if (!parsed || typeof parsed !== 'object') {
|
||||
throw new BundleError(
|
||||
'openclaw.plugin.json: top-level must be an object',
|
||||
'manifest_malformed',
|
||||
);
|
||||
}
|
||||
const m = parsed as Partial<BundleManifest>;
|
||||
if (typeof m.name !== 'string' || typeof m.version !== 'string') {
|
||||
throw new BundleError(
|
||||
'openclaw.plugin.json: name and version must be strings',
|
||||
'manifest_malformed',
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(m.skills)) {
|
||||
throw new BundleError(
|
||||
'openclaw.plugin.json: "skills" must be an array',
|
||||
'manifest_malformed',
|
||||
);
|
||||
}
|
||||
if (!Array.isArray(m.shared_deps)) {
|
||||
// Tolerate older manifests; default to empty.
|
||||
m.shared_deps = [];
|
||||
}
|
||||
return m as BundleManifest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enumerate every absolute path the bundle would install:
|
||||
* - For each skill dir: every regular file under it.
|
||||
* - For each shared dep: the file, or every regular file under it
|
||||
* if it's a directory.
|
||||
*/
|
||||
export interface BundleEntry {
|
||||
/** Absolute source path under gbrainRoot. */
|
||||
source: string;
|
||||
/** Path under the skill bundle, joined with target skills dir. */
|
||||
relTarget: string;
|
||||
/** Whether this comes from shared_deps (true) or a skill (false). */
|
||||
sharedDep: boolean;
|
||||
}
|
||||
|
||||
function walkFiles(absDir: string, prefix: string, out: BundleEntry[], sharedDep: boolean): void {
|
||||
let entries: string[];
|
||||
try {
|
||||
entries = readdirSync(absDir);
|
||||
} catch {
|
||||
return;
|
||||
}
|
||||
for (const e of entries) {
|
||||
const abs = join(absDir, e);
|
||||
let stat;
|
||||
try {
|
||||
stat = statSync(abs);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (stat.isDirectory()) {
|
||||
walkFiles(abs, join(prefix, e), out, sharedDep);
|
||||
} else if (stat.isFile()) {
|
||||
out.push({ source: abs, relTarget: join(prefix, e), sharedDep });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export interface EnumerateOptions {
|
||||
/** Absolute path to gbrain repo root (source). */
|
||||
gbrainRoot: string;
|
||||
/** If set, scope enumeration to just this skill by its slug (last
|
||||
* segment of `skills/<slug>`). Undefined enumerates everything. */
|
||||
skillSlug?: string;
|
||||
manifest: BundleManifest;
|
||||
}
|
||||
|
||||
/**
|
||||
* Enumerate the full bundle (or just one skill + its shared deps) as
|
||||
* a flat list of BundleEntry objects, each with a source path and a
|
||||
* target-relative path.
|
||||
*/
|
||||
export function enumerateBundle(opts: EnumerateOptions): BundleEntry[] {
|
||||
const { gbrainRoot, skillSlug, manifest } = opts;
|
||||
const entries: BundleEntry[] = [];
|
||||
|
||||
const skillsToIncludePaths = skillSlug
|
||||
? manifest.skills.filter(p => pathSlug(p) === skillSlug)
|
||||
: manifest.skills;
|
||||
|
||||
if (skillSlug && skillsToIncludePaths.length === 0) {
|
||||
throw new BundleError(
|
||||
`Skill '${skillSlug}' is not listed in openclaw.plugin.json#skills`,
|
||||
'skill_not_found',
|
||||
);
|
||||
}
|
||||
|
||||
for (const rel of skillsToIncludePaths) {
|
||||
const abs = join(gbrainRoot, rel);
|
||||
if (!existsSync(abs)) {
|
||||
throw new BundleError(
|
||||
`Bundle lists '${rel}' but the path does not exist in ${gbrainRoot}`,
|
||||
'skill_not_found',
|
||||
);
|
||||
}
|
||||
const prefix = rel.replace(/^skills\//, '');
|
||||
walkFiles(abs, prefix, entries, false);
|
||||
}
|
||||
|
||||
// Shared deps always included — installing any skill pulls the full
|
||||
// convention/rules bundle so the skill's references don't break
|
||||
// (D-CX-10 dependency closure).
|
||||
for (const dep of manifest.shared_deps) {
|
||||
const abs = join(gbrainRoot, dep);
|
||||
if (!existsSync(abs)) continue; // missing shared dep is a warning, not fatal
|
||||
const prefix = dep.replace(/^skills\//, '');
|
||||
let stat;
|
||||
try {
|
||||
stat = statSync(abs);
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
if (stat.isDirectory()) {
|
||||
walkFiles(abs, prefix, entries, true);
|
||||
} else if (stat.isFile()) {
|
||||
entries.push({ source: abs, relTarget: prefix, sharedDep: true });
|
||||
}
|
||||
}
|
||||
|
||||
return entries;
|
||||
}
|
||||
|
||||
export function pathSlug(relPath: string): string {
|
||||
const trimmed = relPath.replace(/\/+$/, '');
|
||||
const parts = trimmed.split('/');
|
||||
return parts[parts.length - 1];
|
||||
}
|
||||
|
||||
/**
|
||||
* Return the list of slugs this bundle installs (skills only, not
|
||||
* shared deps). Used by `skillpack list`.
|
||||
*/
|
||||
export function bundledSkillSlugs(manifest: BundleManifest): string[] {
|
||||
return manifest.skills.map(pathSlug).sort();
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
/**
|
||||
* skillpack/installer.ts — copy bundle files into a target OpenClaw
|
||||
* workspace, atomically and with data-loss protection.
|
||||
*
|
||||
* Contracts (from codex outside-voice review):
|
||||
* - Per-file diff protection (D-CX-3 / F4): if a target file differs
|
||||
* from the bundle source, skip it unless `--overwrite-local` is
|
||||
* passed. `--force` only bypasses the top-level "skill dir already
|
||||
* exists" gate.
|
||||
* - Dependency closure (D-CX-10): every skill install pulls the
|
||||
* full `shared_deps` set so cross-references don't break.
|
||||
* - Concurrency / lockfile (D-CX-11): acquire `.gbrain-skillpack.lock`
|
||||
* before any write. Atomic AGENTS.md managed-block update via
|
||||
* tmp + rename. Stale lock (>10 min PID mismatch) emits a warning
|
||||
* and refuses to overwrite unless `--force-unlock`.
|
||||
*/
|
||||
|
||||
import {
|
||||
closeSync,
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
openSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
renameSync,
|
||||
rmSync,
|
||||
statSync,
|
||||
unlinkSync,
|
||||
writeFileSync,
|
||||
writeSync,
|
||||
} from 'fs';
|
||||
import { dirname, join } from 'path';
|
||||
|
||||
import {
|
||||
enumerateBundle,
|
||||
loadBundleManifest,
|
||||
pathSlug,
|
||||
type BundleEntry,
|
||||
type BundleManifest,
|
||||
} from './bundle.ts';
|
||||
import { findResolverFile } from '../resolver-filenames.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export type FileOutcome =
|
||||
| 'wrote_new'
|
||||
| 'wrote_overwrite'
|
||||
| 'skipped_locally_modified'
|
||||
| 'skipped_identical'
|
||||
| 'skipped_overwrite_local_declined';
|
||||
|
||||
export interface FileResult {
|
||||
source: string;
|
||||
target: string;
|
||||
outcome: FileOutcome;
|
||||
sharedDep: boolean;
|
||||
}
|
||||
|
||||
export interface ManagedBlockResult {
|
||||
resolverFile: string;
|
||||
applied: boolean;
|
||||
skippedReason?: 'resolver_not_found' | 'no_change';
|
||||
}
|
||||
|
||||
export interface InstallPlan {
|
||||
gbrainRoot: string;
|
||||
targetSkillsDir: string;
|
||||
targetWorkspace: string;
|
||||
entries: BundleEntry[];
|
||||
manifest: BundleManifest;
|
||||
/** Computed diffs per entry — populated in planInstall, consumed by apply. */
|
||||
entryOutcomes: Array<{ entry: BundleEntry; existing: boolean; identical: boolean }>;
|
||||
}
|
||||
|
||||
export interface InstallOptions {
|
||||
/** Absolute path to the target workspace (above skills/). */
|
||||
targetWorkspace: string;
|
||||
/** Absolute path to the target skills directory. */
|
||||
targetSkillsDir: string;
|
||||
/** Gbrain repo root (source). Defaults to the one found by findGbrainRoot. */
|
||||
gbrainRoot: string;
|
||||
/** Scope to a single skill slug, or `null` for --all. */
|
||||
skillSlug: string | null;
|
||||
/** Overwrite local files that differ from the bundle source. */
|
||||
overwriteLocal?: boolean;
|
||||
/** Dry-run: populate plan, do not write. */
|
||||
dryRun?: boolean;
|
||||
/** Forcibly proceed even when a stale lockfile exists. */
|
||||
forceUnlock?: boolean;
|
||||
/** Override the lock stale threshold (ms). Tests use this. */
|
||||
lockStaleMs?: number;
|
||||
}
|
||||
|
||||
export class InstallError extends Error {
|
||||
constructor(
|
||||
message: string,
|
||||
public code:
|
||||
| 'lock_held'
|
||||
| 'bundle_error'
|
||||
| 'target_missing'
|
||||
| 'unknown_skill',
|
||||
) {
|
||||
super(message);
|
||||
this.name = 'InstallError';
|
||||
}
|
||||
}
|
||||
|
||||
const DEFAULT_LOCK_STALE_MS = 10 * 60 * 1000; // 10 minutes
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Plan
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Build an InstallPlan for either a single skill (by slug) or every
|
||||
* skill (`skillSlug: null`). Always returns the full dependency
|
||||
* closure (shared_deps + target skills).
|
||||
*/
|
||||
export function planInstall(opts: InstallOptions): InstallPlan {
|
||||
const manifest = loadBundleManifest(opts.gbrainRoot);
|
||||
const entries = enumerateBundle({
|
||||
gbrainRoot: opts.gbrainRoot,
|
||||
skillSlug: opts.skillSlug ?? undefined,
|
||||
manifest,
|
||||
});
|
||||
|
||||
const entryOutcomes = entries.map(e => {
|
||||
const target = join(opts.targetSkillsDir, e.relTarget);
|
||||
const existing = existsSync(target);
|
||||
let identical = false;
|
||||
if (existing) {
|
||||
try {
|
||||
const a = readFileSync(e.source);
|
||||
const b = readFileSync(target);
|
||||
identical = a.equals(b);
|
||||
} catch {
|
||||
identical = false;
|
||||
}
|
||||
}
|
||||
return { entry: e, existing, identical };
|
||||
});
|
||||
|
||||
return {
|
||||
gbrainRoot: opts.gbrainRoot,
|
||||
targetSkillsDir: opts.targetSkillsDir,
|
||||
targetWorkspace: opts.targetWorkspace,
|
||||
entries,
|
||||
manifest,
|
||||
entryOutcomes,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Lockfile (D-CX-11)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function lockPath(workspace: string): string {
|
||||
return join(workspace, '.gbrain-skillpack.lock');
|
||||
}
|
||||
|
||||
interface LockInfo {
|
||||
pid: number;
|
||||
mtimeMs: number;
|
||||
}
|
||||
|
||||
function readLock(workspace: string): LockInfo | null {
|
||||
const p = lockPath(workspace);
|
||||
if (!existsSync(p)) return null;
|
||||
try {
|
||||
const content = readFileSync(p, 'utf-8');
|
||||
const pid = parseInt(content.trim(), 10);
|
||||
const mtimeMs = statSync(p).mtimeMs;
|
||||
return { pid: isNaN(pid) ? -1 : pid, mtimeMs };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function acquireLock(workspace: string, opts: InstallOptions): void {
|
||||
const p = lockPath(workspace);
|
||||
const existing = readLock(workspace);
|
||||
const staleMs = opts.lockStaleMs ?? DEFAULT_LOCK_STALE_MS;
|
||||
if (existing) {
|
||||
const age = Date.now() - existing.mtimeMs;
|
||||
// `staleMs: 0` in tests means "any age counts as stale". Use >=
|
||||
// so a just-written lock qualifies when the threshold is 0.
|
||||
const stale = age >= staleMs;
|
||||
if (stale && !opts.forceUnlock) {
|
||||
throw new InstallError(
|
||||
`Stale skillpack lock at ${p} (pid ${existing.pid}, ${Math.round(age / 1000)}s old). Pass --force-unlock to proceed.`,
|
||||
'lock_held',
|
||||
);
|
||||
}
|
||||
if (stale && opts.forceUnlock) {
|
||||
try {
|
||||
unlinkSync(p);
|
||||
} catch {
|
||||
// fall through to write
|
||||
}
|
||||
} else if (!stale) {
|
||||
throw new InstallError(
|
||||
`Another skillpack install appears to be running (pid ${existing.pid}). Wait or pass --force-unlock.`,
|
||||
'lock_held',
|
||||
);
|
||||
}
|
||||
}
|
||||
mkdirSync(dirname(p), { recursive: true });
|
||||
const fd = openSync(p, 'w');
|
||||
try {
|
||||
writeSync(fd, String(process.pid));
|
||||
} finally {
|
||||
closeSync(fd);
|
||||
}
|
||||
}
|
||||
|
||||
function releaseLock(workspace: string): void {
|
||||
const p = lockPath(workspace);
|
||||
try {
|
||||
unlinkSync(p);
|
||||
} catch {
|
||||
// best-effort
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Managed block (AGENTS.md / RESOLVER.md)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const MANAGED_BEGIN = '<!-- gbrain:skillpack:begin -->';
|
||||
const MANAGED_END = '<!-- gbrain:skillpack:end -->';
|
||||
|
||||
export function buildManagedBlock(manifest: BundleManifest, slugs: string[]): string {
|
||||
const sorted = [...slugs].sort();
|
||||
const rows = sorted.map(
|
||||
slug => `| "${slug}" | \`skills/${slug}/SKILL.md\` |`,
|
||||
);
|
||||
return [
|
||||
MANAGED_BEGIN,
|
||||
'',
|
||||
`<!-- Installed by gbrain ${manifest.version} — do not hand-edit between markers. -->`,
|
||||
'',
|
||||
'| Trigger | Skill |',
|
||||
'|---------|-------|',
|
||||
...rows,
|
||||
'',
|
||||
MANAGED_END,
|
||||
].join('\n');
|
||||
}
|
||||
|
||||
/**
|
||||
* Replace the managed block in `resolverContent` with `newBlock`. If
|
||||
* no managed block exists yet, append one (preceded by a blank line).
|
||||
*/
|
||||
export function updateManagedBlock(
|
||||
resolverContent: string,
|
||||
newBlock: string,
|
||||
): string {
|
||||
const beginIdx = resolverContent.indexOf(MANAGED_BEGIN);
|
||||
const endIdx = resolverContent.indexOf(MANAGED_END);
|
||||
if (beginIdx !== -1 && endIdx !== -1 && endIdx > beginIdx) {
|
||||
const before = resolverContent.slice(0, beginIdx);
|
||||
const after = resolverContent.slice(endIdx + MANAGED_END.length);
|
||||
return before + newBlock + after;
|
||||
}
|
||||
const needsNewline = resolverContent.endsWith('\n') ? '' : '\n';
|
||||
return resolverContent + needsNewline + '\n' + newBlock + '\n';
|
||||
}
|
||||
|
||||
function writeAtomic(file: string, content: string): void {
|
||||
const tmp = file + '.tmp.' + process.pid + '.' + Date.now();
|
||||
mkdirSync(dirname(file), { recursive: true });
|
||||
writeFileSync(tmp, content);
|
||||
renameSync(tmp, file);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Apply
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface InstallResult {
|
||||
dryRun: boolean;
|
||||
files: FileResult[];
|
||||
managedBlock: ManagedBlockResult;
|
||||
summary: {
|
||||
wroteNew: number;
|
||||
wroteOverwrite: number;
|
||||
skippedIdentical: number;
|
||||
skippedLocallyModified: number;
|
||||
};
|
||||
}
|
||||
|
||||
export function applyInstall(
|
||||
plan: InstallPlan,
|
||||
opts: InstallOptions,
|
||||
): InstallResult {
|
||||
const files: FileResult[] = [];
|
||||
|
||||
// Lock acquisition. Dry-run does NOT touch the lockfile — it's read-only.
|
||||
const shouldLock = !opts.dryRun;
|
||||
if (shouldLock) acquireLock(opts.targetWorkspace, opts);
|
||||
|
||||
try {
|
||||
// Write files
|
||||
for (const { entry, existing, identical } of plan.entryOutcomes) {
|
||||
const target = join(plan.targetSkillsDir, entry.relTarget);
|
||||
let outcome: FileOutcome;
|
||||
if (!existing) {
|
||||
outcome = 'wrote_new';
|
||||
if (!opts.dryRun) {
|
||||
const content = readFileSync(entry.source);
|
||||
mkdirSync(dirname(target), { recursive: true });
|
||||
writeFileSync(target, content);
|
||||
}
|
||||
} else if (identical) {
|
||||
outcome = 'skipped_identical';
|
||||
} else if (opts.overwriteLocal) {
|
||||
outcome = 'wrote_overwrite';
|
||||
if (!opts.dryRun) {
|
||||
const content = readFileSync(entry.source);
|
||||
writeFileSync(target, content);
|
||||
}
|
||||
} else {
|
||||
outcome = 'skipped_locally_modified';
|
||||
}
|
||||
files.push({
|
||||
source: entry.source,
|
||||
target,
|
||||
outcome,
|
||||
sharedDep: entry.sharedDep,
|
||||
});
|
||||
}
|
||||
|
||||
// Managed block update
|
||||
const installedSlugs = opts.skillSlug
|
||||
? [opts.skillSlug]
|
||||
: plan.manifest.skills.map(pathSlug);
|
||||
const managedBlock = applyManagedBlock(
|
||||
plan.targetWorkspace,
|
||||
plan.targetSkillsDir,
|
||||
plan.manifest,
|
||||
installedSlugs,
|
||||
opts.dryRun ?? false,
|
||||
);
|
||||
|
||||
const summary = {
|
||||
wroteNew: files.filter(f => f.outcome === 'wrote_new').length,
|
||||
wroteOverwrite: files.filter(f => f.outcome === 'wrote_overwrite').length,
|
||||
skippedIdentical: files.filter(f => f.outcome === 'skipped_identical').length,
|
||||
skippedLocallyModified: files.filter(
|
||||
f => f.outcome === 'skipped_locally_modified',
|
||||
).length,
|
||||
};
|
||||
|
||||
return { dryRun: opts.dryRun ?? false, files, managedBlock, summary };
|
||||
} finally {
|
||||
if (shouldLock) releaseLock(opts.targetWorkspace);
|
||||
}
|
||||
}
|
||||
|
||||
function applyManagedBlock(
|
||||
workspace: string,
|
||||
skillsDir: string,
|
||||
manifest: BundleManifest,
|
||||
installedSlugs: string[],
|
||||
dryRun: boolean,
|
||||
): ManagedBlockResult {
|
||||
// Prefer skills-dir resolver; fall back to workspace-root resolver.
|
||||
const resolver = findResolverFile(skillsDir) ?? findResolverFile(workspace);
|
||||
if (!resolver) {
|
||||
return {
|
||||
resolverFile: '',
|
||||
applied: false,
|
||||
skippedReason: 'resolver_not_found',
|
||||
};
|
||||
}
|
||||
const existing = readFileSync(resolver, 'utf-8');
|
||||
// Merge with any slugs already present in the managed block so
|
||||
// repeated single-skill installs accumulate rather than overwrite.
|
||||
const priorSlugs = extractManagedSlugs(existing);
|
||||
const merged = Array.from(new Set([...priorSlugs, ...installedSlugs]));
|
||||
const newBlock = buildManagedBlock(manifest, merged);
|
||||
const updated = updateManagedBlock(existing, newBlock);
|
||||
if (updated === existing) {
|
||||
return { resolverFile: resolver, applied: false, skippedReason: 'no_change' };
|
||||
}
|
||||
if (!dryRun) writeAtomic(resolver, updated);
|
||||
return { resolverFile: resolver, applied: true };
|
||||
}
|
||||
|
||||
export function extractManagedSlugs(resolverContent: string): string[] {
|
||||
const beginIdx = resolverContent.indexOf(MANAGED_BEGIN);
|
||||
const endIdx = resolverContent.indexOf(MANAGED_END);
|
||||
if (beginIdx === -1 || endIdx === -1 || endIdx <= beginIdx) return [];
|
||||
const block = resolverContent.slice(beginIdx, endIdx);
|
||||
const slugs: string[] = [];
|
||||
const re = /`skills\/([^/]+)\/SKILL\.md`/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(block)) !== null) {
|
||||
slugs.push(m[1]);
|
||||
}
|
||||
return slugs;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Diff helpers (for `skillpack diff <name>`)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface SkillDiff {
|
||||
source: string;
|
||||
target: string;
|
||||
existing: boolean;
|
||||
identical: boolean;
|
||||
sourceBytes: number;
|
||||
targetBytes: number;
|
||||
}
|
||||
|
||||
export function diffSkill(
|
||||
gbrainRoot: string,
|
||||
skillSlug: string,
|
||||
targetSkillsDir: string,
|
||||
): SkillDiff[] {
|
||||
const manifest = loadBundleManifest(gbrainRoot);
|
||||
const entries = enumerateBundle({ gbrainRoot, skillSlug, manifest });
|
||||
const out: SkillDiff[] = [];
|
||||
for (const e of entries) {
|
||||
const target = join(targetSkillsDir, e.relTarget);
|
||||
const existing = existsSync(target);
|
||||
let identical = false;
|
||||
let targetBytes = 0;
|
||||
const sourceBytes = statSync(e.source).size;
|
||||
if (existing) {
|
||||
try {
|
||||
const a = readFileSync(e.source);
|
||||
const b = readFileSync(target);
|
||||
targetBytes = b.length;
|
||||
identical = a.equals(b);
|
||||
} catch {
|
||||
// treat as non-identical
|
||||
}
|
||||
}
|
||||
out.push({
|
||||
source: e.source,
|
||||
target,
|
||||
existing,
|
||||
identical,
|
||||
sourceBytes,
|
||||
targetBytes,
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
@@ -126,15 +126,16 @@ describe('check-resolvable — unit: parseFlags', () => {
|
||||
|
||||
describe('check-resolvable — unit: resolveSkillsDir', () => {
|
||||
it('resolves absolute --skills-dir unchanged', () => {
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, skillsDir: '/tmp/absolute-path' });
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, strict: false, skillsDir: '/tmp/absolute-path' });
|
||||
expect(r.dir).toBe('/tmp/absolute-path');
|
||||
expect(r.error).toBeNull();
|
||||
});
|
||||
|
||||
it('resolves relative --skills-dir against cwd', () => {
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, skillsDir: 'skills' });
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, strict: false, skillsDir: 'skills' });
|
||||
expect(r.dir).toMatch(/\/skills$/);
|
||||
expect(r.error).toBeNull();
|
||||
expect(r.source).toBe('explicit');
|
||||
});
|
||||
|
||||
it('REGRESSION-GATE: returns no_skills_dir error when no --skills-dir and findRepoRoot fails', () => {
|
||||
@@ -144,7 +145,7 @@ describe('check-resolvable — unit: resolveSkillsDir', () => {
|
||||
const original = process.cwd();
|
||||
try {
|
||||
process.chdir(empty);
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, skillsDir: null });
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, strict: false, skillsDir: null });
|
||||
expect(r.error).toBe('no_skills_dir');
|
||||
expect(r.dir).toBeNull();
|
||||
expect(typeof r.message).toBe('string');
|
||||
@@ -156,19 +157,51 @@ describe('check-resolvable — unit: resolveSkillsDir', () => {
|
||||
|
||||
it('finds skills via findRepoRoot when cwd is inside a repo (no --skills-dir)', () => {
|
||||
// Running from this test file — we're inside the real gbrain repo.
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, skillsDir: null });
|
||||
const r = resolveSkillsDir({ help: false, json: false, fix: false, dryRun: false, verbose: false, strict: false, skillsDir: null });
|
||||
expect(r.error).toBeNull();
|
||||
expect(r.dir).toMatch(/\/skills$/);
|
||||
expect(r.source).toBe('repo_root');
|
||||
});
|
||||
|
||||
it('REGRESSION-GATE: --skills-dir override takes precedence over OpenClaw env auto-detection', () => {
|
||||
const explicit = mkdtempSync(join(tmpdir(), 'explicit-skills-'));
|
||||
mkdirSync(explicit, { recursive: true });
|
||||
writeFileSync(join(explicit, 'RESOLVER.md'), '# RESOLVER\n');
|
||||
|
||||
const workspace = mkdtempSync(join(tmpdir(), 'openclaw-ws-'));
|
||||
mkdirSync(join(workspace, 'skills'), { recursive: true });
|
||||
writeFileSync(join(workspace, 'skills', 'RESOLVER.md'), '# RESOLVER\n');
|
||||
|
||||
const prev = process.env.OPENCLAW_WORKSPACE;
|
||||
process.env.OPENCLAW_WORKSPACE = workspace;
|
||||
try {
|
||||
const r = resolveSkillsDir({
|
||||
help: false,
|
||||
json: false,
|
||||
fix: false,
|
||||
dryRun: false,
|
||||
verbose: false,
|
||||
strict: false,
|
||||
skillsDir: explicit,
|
||||
});
|
||||
expect(r.error).toBeNull();
|
||||
expect(r.dir).toBe(explicit);
|
||||
expect(r.source).toBe('explicit');
|
||||
} finally {
|
||||
if (prev === undefined) delete process.env.OPENCLAW_WORKSPACE;
|
||||
else process.env.OPENCLAW_WORKSPACE = prev;
|
||||
rmSync(explicit, { recursive: true, force: true });
|
||||
rmSync(workspace, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('check-resolvable — unit: DEFERRED', () => {
|
||||
it('exports two deferred check entries for Checks 5 and 6', () => {
|
||||
expect(DEFERRED.length).toBe(2);
|
||||
expect(DEFERRED[0].check).toBe(5);
|
||||
expect(DEFERRED[0].name).toBe('trigger_routing_eval');
|
||||
expect(DEFERRED[1].check).toBe(6);
|
||||
expect(DEFERRED[1].name).toBe('brain_filing');
|
||||
it('v0.17 ships Checks 5 and 6 — DEFERRED is empty', () => {
|
||||
// Pre-v0.17: both Check 5 (routing eval) and Check 6 (brain filing)
|
||||
// were deferred. v0.17 W2 shipped Check 5; v0.17 W3 shipped Check 6.
|
||||
// The DEFERRED export stays (stable --json field) for future checks.
|
||||
expect(DEFERRED.length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -191,6 +224,9 @@ describe('gbrain check-resolvable CLI — integration', () => {
|
||||
expect(r.stdout).toContain('gbrain check-resolvable');
|
||||
expect(r.stdout).toContain('--json');
|
||||
expect(r.stdout).toContain('--fix');
|
||||
expect(r.stdout).toContain('--strict');
|
||||
// Check 5 shipped in v0.17 (mentioned in the body); Check 6 is
|
||||
// still deferred and must appear under "Deferred to separate issues".
|
||||
expect(r.stdout).toContain('Check 5');
|
||||
expect(r.stdout).toContain('Check 6');
|
||||
});
|
||||
@@ -202,9 +238,10 @@ describe('gbrain check-resolvable CLI — integration', () => {
|
||||
const keys = Object.keys(r.json).sort();
|
||||
expect(keys).toEqual(['autoFix', 'deferred', 'error', 'message', 'ok', 'report', 'skillsDir']);
|
||||
expect(r.json.ok).toBe(true);
|
||||
expect(r.json.deferred.length).toBe(2);
|
||||
expect(r.json.deferred[0].check).toBe(5);
|
||||
expect(r.json.deferred[1].check).toBe(6);
|
||||
// v0.17 ships Checks 5 and 6; DEFERRED is empty. The key remains
|
||||
// stable for future checks.
|
||||
expect(Array.isArray(r.json.deferred)).toBe(true);
|
||||
expect(r.json.deferred.length).toBe(0);
|
||||
});
|
||||
|
||||
it('--json success: autoFix is null when --fix was not passed', () => {
|
||||
@@ -220,22 +257,54 @@ describe('gbrain check-resolvable CLI — integration', () => {
|
||||
expect(r.stdout).toContain('resolver_health: OK');
|
||||
});
|
||||
|
||||
it('REGRESSION-GATE: exits 1 when fixture has a warning-level orphan_trigger only', () => {
|
||||
it('D-CX-3: warnings-only fixture exits 0 in default mode', () => {
|
||||
// "alpha" is in resolver but not manifest → orphan_trigger (warning)
|
||||
// Per D-CX-3 (codex review outside voice): warnings alone do not flip
|
||||
// the exit code. This is the new contract; callers who want strict
|
||||
// behavior pass --strict. Prior contract (exit 1 on any warning) broke
|
||||
// CI for workspaces emitting warning-level advisories like filing-audit.
|
||||
const skillsDir = makeFixture(
|
||||
[{ name: 'alpha', triggers: ['alpha'], inManifest: false }],
|
||||
created,
|
||||
);
|
||||
const r = run(['--json', '--skills-dir', skillsDir]);
|
||||
expect(r.json).not.toBeNull();
|
||||
const warnings = r.json.report.issues.filter((i: any) => i.severity === 'warning');
|
||||
const errors = r.json.report.issues.filter((i: any) => i.severity === 'error');
|
||||
expect(warnings.length).toBeGreaterThan(0);
|
||||
expect(errors.length).toBe(0);
|
||||
// Doctor's ok=true-on-warnings-only would exit 0. check-resolvable MUST exit 1.
|
||||
expect(r.json.report.warnings.length).toBeGreaterThan(0);
|
||||
expect(r.json.report.errors.length).toBe(0);
|
||||
// warnings.length > 0 but errors.length === 0 → exit 0 (advisory)
|
||||
expect(r.status).toBe(0);
|
||||
expect(r.json.ok).toBe(true);
|
||||
});
|
||||
|
||||
it('D-CX-3: --strict promotes warnings to exit 1', () => {
|
||||
const skillsDir = makeFixture(
|
||||
[{ name: 'alpha', triggers: ['alpha'], inManifest: false }],
|
||||
created,
|
||||
);
|
||||
const r = run(['--json', '--strict', '--skills-dir', skillsDir]);
|
||||
expect(r.json).not.toBeNull();
|
||||
expect(r.json.report.warnings.length).toBeGreaterThan(0);
|
||||
expect(r.json.report.errors.length).toBe(0);
|
||||
// --strict flips ok to false and exit to 1 when warnings exist
|
||||
expect(r.json.ok).toBe(false);
|
||||
expect(r.status).toBe(1);
|
||||
});
|
||||
|
||||
it('D-CX-3: report has separate errors[] and warnings[] arrays alongside issues[]', () => {
|
||||
const skillsDir = makeFixture(
|
||||
[{ name: 'alpha', triggers: ['alpha'], inManifest: false }],
|
||||
created,
|
||||
);
|
||||
const r = run(['--json', '--skills-dir', skillsDir]);
|
||||
expect(r.json).not.toBeNull();
|
||||
const rep = r.json.report;
|
||||
expect(Array.isArray(rep.errors)).toBe(true);
|
||||
expect(Array.isArray(rep.warnings)).toBe(true);
|
||||
// Deprecated flat `issues` still present for one-release backcompat
|
||||
expect(Array.isArray(rep.issues)).toBe(true);
|
||||
expect(rep.issues.length).toBe(rep.errors.length + rep.warnings.length);
|
||||
});
|
||||
|
||||
it('exits 1 when fixture has an error-level unreachable skill', () => {
|
||||
// "alpha" is in manifest but not resolver → unreachable (error)
|
||||
const skillsDir = makeFixture(
|
||||
@@ -261,9 +330,12 @@ describe('gbrain check-resolvable CLI — integration', () => {
|
||||
it('--verbose prints the deferred checks note in human mode', () => {
|
||||
const skillsDir = makeFixture([{ name: 'alpha', triggers: ['alpha'] }], created);
|
||||
const r = run(['--verbose', '--skills-dir', skillsDir]);
|
||||
// v0.17 ships Checks 5 and 6 → DEFERRED is empty. Verbose still
|
||||
// prints the "Deferred:" header for stable UX; downstream content
|
||||
// is empty until a future release adds a new deferred check.
|
||||
expect(r.stdout).toContain('Deferred:');
|
||||
expect(r.stdout).toContain('trigger_routing_eval');
|
||||
expect(r.stdout).toContain('brain_filing');
|
||||
expect(r.stdout).not.toContain('trigger_routing_eval');
|
||||
expect(r.stdout).not.toContain('brain_filing');
|
||||
});
|
||||
|
||||
it('clean fixture human output says all skills reachable', () => {
|
||||
@@ -279,4 +351,11 @@ describe('gbrain check-resolvable CLI — integration', () => {
|
||||
expect(r.stdout).toContain('2 skills');
|
||||
expect(r.status).toBe(0);
|
||||
});
|
||||
|
||||
it('logs the auto-detected skills directory path in human mode', () => {
|
||||
const r = run([]);
|
||||
expect(r.status === 0 || r.status === 1).toBe(true);
|
||||
expect(r.stdout).toContain('Auto-detected skills directory');
|
||||
expect(r.stdout).toContain('/skills');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -168,7 +168,8 @@ describe('gbrain doctor — half-migrated Minions detection', () => {
|
||||
|
||||
test('human output: prints MINIONS HALF-INSTALLED loud banner', () => {
|
||||
// Same fixture as the first test, but check the human-readable output
|
||||
// includes the exact banner phrase Wintermute's cron script can grep for.
|
||||
// includes the exact banner phrase an OpenClaw host's cron script
|
||||
// can grep for.
|
||||
const migrationsDir = join(tmp, '.gbrain', 'migrations');
|
||||
mkdirSync(migrationsDir, { recursive: true });
|
||||
writeFileSync(
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
/**
|
||||
* test/e2e/openclaw-reference-compat.test.ts — W1 ship-blocker gate.
|
||||
*
|
||||
* This is THE test that proves v0.17 delivers on its headline claim:
|
||||
* `gbrain check-resolvable` against an OpenClaw-reference workspace
|
||||
* layout (AGENTS.md at workspace root, skills/ below, no manifest.json)
|
||||
* runs cleanly and surfaces sensible issues.
|
||||
*
|
||||
* Fixture: `test/fixtures/openclaw-reference-minimal/` ships 4 skills
|
||||
* plus an AGENTS.md with a resolver table. Every test here exercises
|
||||
* the full W1 + W2 + W3 + W4 + W5 stack against that fixture.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdtempSync, rmSync, writeFileSync, mkdirSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { spawnSync } from 'child_process';
|
||||
|
||||
import { checkResolvable } from '../../src/core/check-resolvable.ts';
|
||||
import { autoDetectSkillsDir } from '../../src/core/repo-root.ts';
|
||||
import { loadOrDeriveManifest } from '../../src/core/skill-manifest.ts';
|
||||
import {
|
||||
applyInstall,
|
||||
planInstall,
|
||||
} from '../../src/core/skillpack/installer.ts';
|
||||
import { findGbrainRoot } from '../../src/core/skillpack/bundle.ts';
|
||||
|
||||
const FIXTURE = join(import.meta.dir, '..', 'fixtures', 'openclaw-reference-minimal');
|
||||
const SKILLS_DIR = join(FIXTURE, 'skills');
|
||||
const REPO = join(import.meta.dir, '..', '..');
|
||||
const CLI = join(REPO, 'src', 'cli.ts');
|
||||
|
||||
const created: string[] = [];
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe('OpenClaw reference workspace compat (W1 + W2 + W3)', () => {
|
||||
it('fixture exists at the expected path', () => {
|
||||
expect(existsSync(FIXTURE)).toBe(true);
|
||||
expect(existsSync(join(FIXTURE, 'AGENTS.md'))).toBe(true);
|
||||
expect(existsSync(SKILLS_DIR)).toBe(true);
|
||||
expect(existsSync(join(FIXTURE, 'skills', 'manifest.json'))).toBe(false);
|
||||
});
|
||||
|
||||
it('auto-detects skills dir via $OPENCLAW_WORKSPACE (D-CX-4 priority)', () => {
|
||||
// Priority: explicit env wins over repo-root walk. Without the
|
||||
// env var, we'd get gbrain's own repo. With it set, we should
|
||||
// get the fixture's skills dir.
|
||||
const detected = autoDetectSkillsDir(process.cwd(), { OPENCLAW_WORKSPACE: FIXTURE });
|
||||
expect(detected.dir).toBe(SKILLS_DIR);
|
||||
// AGENTS.md is at the workspace root, not inside skills/, so the
|
||||
// source should be the workspace-root variant.
|
||||
expect(detected.source).toBe('openclaw_workspace_env_root');
|
||||
});
|
||||
|
||||
it('auto-derives manifest from SKILL.md walk (F-ENG-1)', () => {
|
||||
const result = loadOrDeriveManifest(SKILLS_DIR);
|
||||
expect(result.derived).toBe(true);
|
||||
expect(result.skills.map(s => s.name).sort()).toEqual([
|
||||
'brain-ops',
|
||||
'context-now',
|
||||
'query',
|
||||
'signal-detector',
|
||||
]);
|
||||
});
|
||||
|
||||
it('checkResolvable accepts AGENTS.md at workspace root and runs all checks', () => {
|
||||
const report = checkResolvable(SKILLS_DIR);
|
||||
// Top-level ok is errors-only (D-CX-3) — no unreachable/missing-file errors.
|
||||
expect(report.ok).toBe(true);
|
||||
expect(report.errors.length).toBe(0);
|
||||
// All 4 skills should be reachable via AGENTS.md rows.
|
||||
expect(report.summary.total_skills).toBe(4);
|
||||
expect(report.summary.reachable).toBe(4);
|
||||
expect(report.summary.unreachable).toBe(0);
|
||||
});
|
||||
|
||||
it('brain-ops declares writes_pages+writes_to — filing audit clean', () => {
|
||||
const report = checkResolvable(SKILLS_DIR);
|
||||
const filing = report.warnings.filter(w =>
|
||||
w.type === 'filing_missing_writes_to' || w.type === 'filing_unknown_directory',
|
||||
);
|
||||
expect(filing).toEqual([]);
|
||||
});
|
||||
|
||||
it('CLI subprocess: gbrain check-resolvable --json --skills-dir FIXTURE clean', () => {
|
||||
const r = spawnSync(
|
||||
'bun',
|
||||
[CLI, 'check-resolvable', '--json', '--skills-dir', SKILLS_DIR],
|
||||
{ encoding: 'utf-8', cwd: REPO, maxBuffer: 10 * 1024 * 1024 },
|
||||
);
|
||||
expect(r.status).toBe(0);
|
||||
const env = JSON.parse(r.stdout);
|
||||
expect(env.ok).toBe(true);
|
||||
expect(env.report.errors).toEqual([]);
|
||||
expect(env.report.summary.total_skills).toBe(4);
|
||||
});
|
||||
|
||||
it('CLI subprocess: $OPENCLAW_WORKSPACE auto-detect without --skills-dir', () => {
|
||||
const r = spawnSync('bun', [CLI, 'check-resolvable', '--json'], {
|
||||
encoding: 'utf-8',
|
||||
cwd: REPO,
|
||||
env: { ...process.env, OPENCLAW_WORKSPACE: FIXTURE },
|
||||
maxBuffer: 10 * 1024 * 1024,
|
||||
});
|
||||
expect(r.status).toBe(0);
|
||||
const env = JSON.parse(r.stdout);
|
||||
expect(env.skillsDir).toBe(SKILLS_DIR);
|
||||
expect(env.ok).toBe(true);
|
||||
});
|
||||
|
||||
it('skillpack install against OpenClaw-reference layout writes managed block to AGENTS.md', () => {
|
||||
// Copy fixture into a tmp workspace so we can install without
|
||||
// polluting the fixture itself.
|
||||
const target = mkdtempSync(join(tmpdir(), 'openclaw-ref-install-'));
|
||||
created.push(target);
|
||||
mkdirSync(join(target, 'skills'), { recursive: true });
|
||||
// Just the AGENTS.md shell — no skills yet, install writes them.
|
||||
writeFileSync(
|
||||
join(target, 'AGENTS.md'),
|
||||
'# AGENTS\n\n| Trigger | Skill |\n|---------|-------|\n',
|
||||
);
|
||||
|
||||
const gbrainRoot = findGbrainRoot();
|
||||
expect(gbrainRoot).not.toBeNull();
|
||||
|
||||
const opts = {
|
||||
gbrainRoot: gbrainRoot!,
|
||||
targetWorkspace: target,
|
||||
targetSkillsDir: join(target, 'skills'),
|
||||
skillSlug: 'brain-ops',
|
||||
};
|
||||
const plan = planInstall(opts);
|
||||
const result = applyInstall(plan, opts);
|
||||
expect(result.summary.wroteNew).toBeGreaterThan(0);
|
||||
expect(result.managedBlock.applied).toBe(true);
|
||||
expect(result.managedBlock.resolverFile).toBe(join(target, 'AGENTS.md'));
|
||||
|
||||
const agents = readFileSync(join(target, 'AGENTS.md'), 'utf-8');
|
||||
expect(agents).toContain('gbrain:skillpack:begin');
|
||||
expect(agents).toContain('`skills/brain-ops/SKILL.md`');
|
||||
// Pre-existing resolver table preserved.
|
||||
expect(agents).toContain('| Trigger | Skill |');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,251 @@
|
||||
/**
|
||||
* Tests for src/core/filing-audit.ts — Check 6 (W3, v0.17).
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import {
|
||||
loadFilingRules,
|
||||
allowedDirectories,
|
||||
runFilingAudit,
|
||||
} from '../src/core/filing-audit.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
function scratch(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'filing-audit-'));
|
||||
created.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function writeRules(skillsDir: string, body?: object): void {
|
||||
const doc = body ?? {
|
||||
version: '1.0.0',
|
||||
rules: [
|
||||
{ kind: 'person', directory: 'people/' },
|
||||
{ kind: 'company', directory: 'companies/' },
|
||||
],
|
||||
sources_dir: { directory: 'sources/', purpose: 'raw data only' },
|
||||
};
|
||||
writeFileSync(join(skillsDir, '_brain-filing-rules.json'), JSON.stringify(doc, null, 2));
|
||||
}
|
||||
|
||||
function writeSkill(
|
||||
skillsDir: string,
|
||||
name: string,
|
||||
opts: {
|
||||
writes_pages?: boolean;
|
||||
writes_to?: string[];
|
||||
writes_to_inline?: boolean;
|
||||
mutating?: boolean;
|
||||
no_frontmatter?: boolean;
|
||||
} = {},
|
||||
): void {
|
||||
const dir = join(skillsDir, name);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
if (opts.no_frontmatter) {
|
||||
writeFileSync(join(dir, 'SKILL.md'), `# ${name}\nNo frontmatter.\n`);
|
||||
return;
|
||||
}
|
||||
const lines = ['---', `name: ${name}`];
|
||||
if (opts.mutating !== undefined) lines.push(`mutating: ${opts.mutating}`);
|
||||
if (opts.writes_pages !== undefined) lines.push(`writes_pages: ${opts.writes_pages}`);
|
||||
if (opts.writes_to) {
|
||||
if (opts.writes_to_inline) {
|
||||
lines.push(`writes_to: [${opts.writes_to.map(s => `"${s}"`).join(', ')}]`);
|
||||
} else {
|
||||
lines.push('writes_to:');
|
||||
for (const d of opts.writes_to) lines.push(` - ${d}`);
|
||||
}
|
||||
}
|
||||
lines.push('---');
|
||||
lines.push(`# ${name}\n`);
|
||||
writeFileSync(join(dir, 'SKILL.md'), lines.join('\n'));
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe('loadFilingRules', () => {
|
||||
it('returns null when rules file is missing', () => {
|
||||
const dir = scratch();
|
||||
expect(loadFilingRules(dir)).toBeNull();
|
||||
});
|
||||
it('parses a valid rules document', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
const rules = loadFilingRules(dir);
|
||||
expect(rules).not.toBeNull();
|
||||
expect(rules!.rules.length).toBe(2);
|
||||
expect(rules!.rules[0].kind).toBe('person');
|
||||
});
|
||||
it('throws on malformed JSON', () => {
|
||||
const dir = scratch();
|
||||
writeFileSync(join(dir, '_brain-filing-rules.json'), '{ not valid');
|
||||
expect(() => loadFilingRules(dir)).toThrow();
|
||||
});
|
||||
it('throws when rules is not an array', () => {
|
||||
const dir = scratch();
|
||||
writeFileSync(
|
||||
join(dir, '_brain-filing-rules.json'),
|
||||
JSON.stringify({ version: '1', rules: 'oops' }),
|
||||
);
|
||||
expect(() => loadFilingRules(dir)).toThrow();
|
||||
});
|
||||
});
|
||||
|
||||
describe('allowedDirectories', () => {
|
||||
it('normalizes directory strings (trailing slash)', () => {
|
||||
const allowed = allowedDirectories({
|
||||
version: '1',
|
||||
rules: [
|
||||
{ kind: 'a', directory: 'people/' },
|
||||
{ kind: 'b', directory: 'companies' }, // no slash
|
||||
],
|
||||
sources_dir: { directory: '/sources', purpose: 'raw' },
|
||||
});
|
||||
expect(allowed.has('people/')).toBe(true);
|
||||
expect(allowed.has('companies/')).toBe(true);
|
||||
expect(allowed.has('sources/')).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runFilingAudit', () => {
|
||||
it('returns empty report when rules file is missing (no-op)', () => {
|
||||
const dir = scratch();
|
||||
// No _brain-filing-rules.json.
|
||||
writeSkill(dir, 'enrich', { writes_pages: true, writes_to: ['people/'] });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
expect(r.totalScanned).toBe(0);
|
||||
});
|
||||
|
||||
it('clean: declares writes_pages + valid writes_to', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'enrich', {
|
||||
writes_pages: true,
|
||||
writes_to: ['people/', 'companies/'],
|
||||
});
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
expect(r.writesPagesSkills).toBe(1);
|
||||
});
|
||||
|
||||
it('flags missing writes_to when writes_pages is true', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'enrich', { writes_pages: true });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues.length).toBe(1);
|
||||
expect(r.issues[0].type).toBe('filing_missing_writes_to');
|
||||
expect(r.issues[0].severity).toBe('warning');
|
||||
});
|
||||
|
||||
it('flags unknown directory in writes_to', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'enrich', {
|
||||
writes_pages: true,
|
||||
writes_to: ['people/', 'junk/'],
|
||||
});
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues.length).toBe(1);
|
||||
expect(r.issues[0].type).toBe('filing_unknown_directory');
|
||||
expect(r.issues[0].directory).toBe('junk/');
|
||||
});
|
||||
|
||||
it('D-CX-7: mutating:true alone does NOT trigger filing audit', () => {
|
||||
// Cron/scheduler/report skills use mutating:true but don't write
|
||||
// brain pages. Filing-audit must skip them entirely.
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'cron-scheduler', { mutating: true });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
expect(r.writesPagesSkills).toBe(0);
|
||||
});
|
||||
|
||||
it('skips skills with writes_pages: false', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'query', { writes_pages: false });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
});
|
||||
|
||||
it('skips skills with no frontmatter', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'bare', { no_frontmatter: true });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
});
|
||||
|
||||
it('parses inline writes_to: [a, b] syntax', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'enrich', {
|
||||
writes_pages: true,
|
||||
writes_to: ['people/', 'companies/'],
|
||||
writes_to_inline: true,
|
||||
});
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
});
|
||||
|
||||
it('allows sources/ (raw data dir)', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'importer', {
|
||||
writes_pages: true,
|
||||
writes_to: ['sources/'],
|
||||
});
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
});
|
||||
|
||||
it('skips underscore and dot dirs', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, '_conventions', {
|
||||
writes_pages: true,
|
||||
writes_to: ['not-real/'],
|
||||
});
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues).toEqual([]);
|
||||
});
|
||||
|
||||
it('totalScanned counts every skill with SKILL.md', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'a');
|
||||
writeSkill(dir, 'b', { writes_pages: true, writes_to: ['people/'] });
|
||||
writeSkill(dir, 'c', { writes_pages: false });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.totalScanned).toBe(3);
|
||||
expect(r.writesPagesSkills).toBe(1);
|
||||
});
|
||||
|
||||
it('handles missing skillsDir cleanly', () => {
|
||||
const r = runFilingAudit('/tmp/never-exists-filing-audit-ABC');
|
||||
expect(r.issues).toEqual([]);
|
||||
expect(r.totalScanned).toBe(0);
|
||||
});
|
||||
|
||||
it('action string names the exact file to edit (test coverage guard)', () => {
|
||||
const dir = scratch();
|
||||
writeRules(dir);
|
||||
writeSkill(dir, 'enrich', { writes_pages: true, writes_to: ['bad/'] });
|
||||
const r = runFilingAudit(dir);
|
||||
expect(r.issues[0].action).toContain('SKILL.md');
|
||||
expect(r.issues[0].action.length).toBeGreaterThan(10);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
# AGENTS.md
|
||||
|
||||
Minimal fixture mimicking the OpenClaw reference deployment layout
|
||||
(for W1 compat testing). AGENTS.md lives at workspace root; skills
|
||||
live under `skills/`. No manifest.json (the auto-derive path in
|
||||
`src/core/skill-manifest.ts` handles this).
|
||||
|
||||
## Gate 0 — access control
|
||||
|
||||
| Trigger | Skill |
|
||||
|---------|-------|
|
||||
| Every inbound message | `skills/signal-detector/SKILL.md` |
|
||||
|
||||
## Brain operations
|
||||
|
||||
| Trigger | Skill |
|
||||
|---------|-------|
|
||||
| "what do we know about", "search for", "lookup" | `skills/query/SKILL.md` |
|
||||
| any brain read/write/lookup/citation | `skills/brain-ops/SKILL.md` |
|
||||
|
||||
## Calendar
|
||||
|
||||
| Trigger | Skill |
|
||||
|---------|-------|
|
||||
| "am I late", "how long until", "what time is" | `skills/context-now/SKILL.md` |
|
||||
@@ -0,0 +1,14 @@
|
||||
---
|
||||
name: brain-ops
|
||||
description: Core read/write cycle for the OpenClaw reference fixture.
|
||||
triggers:
|
||||
- any brain read/write/lookup/citation
|
||||
writes_pages: true
|
||||
writes_to:
|
||||
- people/
|
||||
- companies/
|
||||
---
|
||||
|
||||
# brain-ops
|
||||
|
||||
Fixture skill for `test/e2e/openclaw-reference-compat.test.ts`.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
name: context-now
|
||||
description: "ALWAYS-ON time-sensitivity discipline for the OpenClaw reference fixture."
|
||||
triggers:
|
||||
- "am I late"
|
||||
- "how long until"
|
||||
- "what time is"
|
||||
---
|
||||
|
||||
# context-now
|
||||
|
||||
Fixture skill for `test/e2e/openclaw-reference-compat.test.ts`.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
name: query
|
||||
description: Look up brain pages in the OpenClaw reference fixture.
|
||||
triggers:
|
||||
- "what do we know about"
|
||||
- "search for"
|
||||
- "lookup"
|
||||
---
|
||||
|
||||
# query
|
||||
|
||||
Fixture skill for `test/e2e/openclaw-reference-compat.test.ts`.
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
name: signal-detector
|
||||
description: Always-on ambient signal capture for the OpenClaw reference fixture.
|
||||
triggers:
|
||||
- every inbound message
|
||||
---
|
||||
|
||||
# signal-detector
|
||||
|
||||
Fixture skill for `test/e2e/openclaw-reference-compat.test.ts`.
|
||||
@@ -89,7 +89,7 @@ describe('path policy', () => {
|
||||
|
||||
describe('loadSinglePlugin', () => {
|
||||
test('loads a minimal manifest + one subagent def', () => {
|
||||
const dir = writePlugin('wintermute', {
|
||||
const dir = writePlugin('openclaw-ref', {
|
||||
subagents: {
|
||||
'meeting-ingestion.md': `---\nname: meeting-ingestion\nmodel: sonnet\n---\n\nYou are a meeting ingester.\n`,
|
||||
},
|
||||
@@ -97,7 +97,7 @@ describe('loadSinglePlugin', () => {
|
||||
const res = loadSinglePlugin(dir);
|
||||
expect('error' in res).toBe(false);
|
||||
if ('error' in res) return;
|
||||
expect(res.manifest.name).toBe('wintermute');
|
||||
expect(res.manifest.name).toBe('openclaw-ref');
|
||||
expect(res.subagents.length).toBe(1);
|
||||
expect(res.subagents[0]!.name).toBe('meeting-ingestion');
|
||||
expect(res.subagents[0]!.body.trim()).toBe('You are a meeting ingester.');
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
/**
|
||||
* test/regression-v0_16_4.test.ts — F-ENG-8.
|
||||
*
|
||||
* Guards against v0.17 regressions: a clean fixture that passed cleanly
|
||||
* on v0.16.4 check-resolvable must still pass cleanly on v0.17 — same
|
||||
* errors[] and warnings[] shape, no new surprise findings.
|
||||
*
|
||||
* The fixture matches the canonical RESOLVER.md + manifest.json +
|
||||
* skills/ shape that v0.16.4 test suites used.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import { checkResolvable } from '../src/core/check-resolvable.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
function makeCleanFixture(): string {
|
||||
const root = mkdtempSync(join(tmpdir(), 'v0_16_4-regression-'));
|
||||
created.push(root);
|
||||
const skillsDir = join(root, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
|
||||
// Two skills, both in manifest, both reachable via RESOLVER.md.
|
||||
const manifest = {
|
||||
name: 'test',
|
||||
version: '0.16.4-fixture',
|
||||
skills: [
|
||||
{ name: 'query', path: 'query/SKILL.md' },
|
||||
{ name: 'brain-ops', path: 'brain-ops/SKILL.md' },
|
||||
],
|
||||
};
|
||||
writeFileSync(join(skillsDir, 'manifest.json'), JSON.stringify(manifest, null, 2));
|
||||
|
||||
mkdirSync(join(skillsDir, 'query'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'query', 'SKILL.md'),
|
||||
'---\nname: query\ndescription: lookup brain pages.\ntriggers:\n - "look up"\n---\n\n# query\n',
|
||||
);
|
||||
|
||||
mkdirSync(join(skillsDir, 'brain-ops'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'brain-ops', 'SKILL.md'),
|
||||
'---\nname: brain-ops\ndescription: core read/write cycle.\ntriggers:\n - any brain read/write\n---\n\n# brain-ops\n',
|
||||
);
|
||||
|
||||
writeFileSync(
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
[
|
||||
'# RESOLVER',
|
||||
'',
|
||||
'## Brain operations',
|
||||
'',
|
||||
'| Trigger | Skill |',
|
||||
'|---------|-------|',
|
||||
'| "look up" | `skills/query/SKILL.md` |',
|
||||
'| any brain read/write | `skills/brain-ops/SKILL.md` |',
|
||||
'',
|
||||
].join('\n'),
|
||||
);
|
||||
|
||||
return skillsDir;
|
||||
}
|
||||
|
||||
describe('v0.16.4 regression guard (F-ENG-8)', () => {
|
||||
it('clean fixture passes all checks on v0.17 (no errors, no surprise warnings)', () => {
|
||||
const skillsDir = makeCleanFixture();
|
||||
const report = checkResolvable(skillsDir);
|
||||
expect(report.ok).toBe(true);
|
||||
expect(report.errors).toEqual([]);
|
||||
|
||||
// v0.17 adds routing_*/filing_*/skillify_stub_* warning types.
|
||||
// A clean v0.16.4-style fixture has NO routing-eval fixtures,
|
||||
// NO writes_pages declarations, NO SKILLIFY_STUB markers — so
|
||||
// none of the new warning types should fire.
|
||||
const newTypeWarnings = report.warnings.filter(w =>
|
||||
w.type.startsWith('routing_') ||
|
||||
w.type.startsWith('filing_') ||
|
||||
w.type === 'skillify_stub_unreplaced',
|
||||
);
|
||||
expect(newTypeWarnings).toEqual([]);
|
||||
|
||||
// Summary shape unchanged.
|
||||
expect(report.summary.total_skills).toBe(2);
|
||||
expect(report.summary.reachable).toBe(2);
|
||||
expect(report.summary.unreachable).toBe(0);
|
||||
});
|
||||
|
||||
it('JSON envelope shape is stable (keys unchanged from v0.16.4)', () => {
|
||||
const skillsDir = makeCleanFixture();
|
||||
const report = checkResolvable(skillsDir);
|
||||
// Top-level keys the --json envelope promises.
|
||||
expect(Object.keys(report).sort()).toEqual([
|
||||
'errors',
|
||||
'issues',
|
||||
'ok',
|
||||
'summary',
|
||||
'warnings',
|
||||
]);
|
||||
// Summary keys — new `routing_*` totals are NOT added to summary
|
||||
// (kept in the issues list only).
|
||||
expect(Object.keys(report.summary).sort()).toEqual([
|
||||
'gaps',
|
||||
'overlaps',
|
||||
'reachable',
|
||||
'total_skills',
|
||||
'unreachable',
|
||||
]);
|
||||
});
|
||||
|
||||
it('deprecated issues[] union equals errors[] + warnings[]', () => {
|
||||
const skillsDir = makeCleanFixture();
|
||||
const report = checkResolvable(skillsDir);
|
||||
expect(report.issues.length).toBe(report.errors.length + report.warnings.length);
|
||||
});
|
||||
});
|
||||
+110
-1
@@ -2,7 +2,7 @@ import { describe, it, expect, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, mkdirSync, writeFileSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { findRepoRoot } from '../src/core/repo-root.ts';
|
||||
import { autoDetectSkillsDir, findRepoRoot } from '../src/core/repo-root.ts';
|
||||
|
||||
describe('findRepoRoot', () => {
|
||||
const created: string[] = [];
|
||||
@@ -22,6 +22,13 @@ describe('findRepoRoot', () => {
|
||||
function seedRepo(dir: string): void {
|
||||
mkdirSync(join(dir, 'skills'), { recursive: true });
|
||||
writeFileSync(join(dir, 'skills', 'RESOLVER.md'), '# RESOLVER\n');
|
||||
mkdirSync(join(dir, 'src'), { recursive: true });
|
||||
writeFileSync(join(dir, 'src', 'cli.ts'), '// gbrain marker\n');
|
||||
}
|
||||
|
||||
function seedSkillsDir(dir: string): void {
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(join(dir, 'RESOLVER.md'), '# RESOLVER\n');
|
||||
}
|
||||
|
||||
it('finds skills/RESOLVER.md in the passed startDir on first iteration', () => {
|
||||
@@ -50,4 +57,106 @@ describe('findRepoRoot', () => {
|
||||
// and CI runners. What matters is parity: no-arg === cwd-arg.
|
||||
expect(findRepoRoot()).toBe(findRepoRoot(process.cwd()));
|
||||
});
|
||||
|
||||
it('auto-detect: falls back to $OPENCLAW_WORKSPACE/skills when repo root is absent', () => {
|
||||
const cwd = scratch();
|
||||
const workspace = scratch();
|
||||
seedSkillsDir(join(workspace, 'skills'));
|
||||
const found = autoDetectSkillsDir(cwd, { OPENCLAW_WORKSPACE: workspace });
|
||||
expect(found.dir).toBe(join(workspace, 'skills'));
|
||||
expect(found.source).toBe('openclaw_workspace_env');
|
||||
});
|
||||
|
||||
it('auto-detect: falls back to ~/.openclaw/workspace/skills when env var is not set', () => {
|
||||
const cwd = scratch();
|
||||
const home = scratch();
|
||||
seedSkillsDir(join(home, '.openclaw', 'workspace', 'skills'));
|
||||
const found = autoDetectSkillsDir(cwd, { HOME: home });
|
||||
expect(found.dir).toBe(join(home, '.openclaw', 'workspace', 'skills'));
|
||||
expect(found.source).toBe('openclaw_workspace_home');
|
||||
});
|
||||
|
||||
it('auto-detect: falls back to ./skills as final candidate', () => {
|
||||
const cwd = scratch();
|
||||
seedSkillsDir(join(cwd, 'skills'));
|
||||
const found = autoDetectSkillsDir(cwd, {});
|
||||
expect(found.dir).toBe(join(cwd, 'skills'));
|
||||
expect(found.source).toBe('cwd_skills');
|
||||
});
|
||||
|
||||
it('D-CX-4: $OPENCLAW_WORKSPACE wins over repo-root walk when explicitly set', () => {
|
||||
// Prior priority (shadow bug): walking up from cwd found gbrain's
|
||||
// repo root first and silently ignored the env var. Post-D-CX-4:
|
||||
// explicit env wins. Unset env → repo-root walk still wins
|
||||
// (tested below).
|
||||
const root = scratch();
|
||||
seedRepo(root);
|
||||
const workspace = scratch();
|
||||
seedSkillsDir(join(workspace, 'skills'));
|
||||
const nested = join(root, 'a', 'b');
|
||||
mkdirSync(nested, { recursive: true });
|
||||
const found = autoDetectSkillsDir(nested, { OPENCLAW_WORKSPACE: workspace });
|
||||
expect(found.dir).toBe(join(workspace, 'skills'));
|
||||
expect(found.source).toBe('openclaw_workspace_env');
|
||||
});
|
||||
|
||||
it('D-CX-4: repo-root walk still wins when OPENCLAW_WORKSPACE is NOT set', () => {
|
||||
const root = scratch();
|
||||
seedRepo(root);
|
||||
const nested = join(root, 'a', 'b');
|
||||
mkdirSync(nested, { recursive: true });
|
||||
const found = autoDetectSkillsDir(nested, {});
|
||||
expect(found.dir).toBe(join(root, 'skills'));
|
||||
expect(found.source).toBe('repo_root');
|
||||
});
|
||||
|
||||
it('W1: AGENTS.md at skills dir is accepted (OpenClaw skills-subdir variant)', () => {
|
||||
const workspace = scratch();
|
||||
const skillsDir = join(workspace, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
// seed AGENTS.md (no RESOLVER.md)
|
||||
require('fs').writeFileSync(
|
||||
join(skillsDir, 'AGENTS.md'),
|
||||
'# AGENTS\n\n| Trigger | Skill |\n|---|---|\n',
|
||||
);
|
||||
const found = autoDetectSkillsDir(scratch(), { OPENCLAW_WORKSPACE: workspace });
|
||||
expect(found.dir).toBe(skillsDir);
|
||||
expect(found.source).toBe('openclaw_workspace_env');
|
||||
});
|
||||
|
||||
it('W1: AGENTS.md at workspace root (OpenClaw-native layout)', () => {
|
||||
// The reference OpenClaw deployment places AGENTS.md at
|
||||
// workspace/AGENTS.md, with skills in workspace/skills/. Auto-detect
|
||||
// must find the skills dir and flag this as the workspace-root variant.
|
||||
const workspace = scratch();
|
||||
const skillsDir = join(workspace, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
require('fs').writeFileSync(
|
||||
join(workspace, 'AGENTS.md'),
|
||||
'# AGENTS\n\n| Trigger | Skill |\n|---|---|\n',
|
||||
);
|
||||
const found = autoDetectSkillsDir(scratch(), { OPENCLAW_WORKSPACE: workspace });
|
||||
expect(found.dir).toBe(skillsDir);
|
||||
expect(found.source).toBe('openclaw_workspace_env_root');
|
||||
});
|
||||
|
||||
it('W1: both RESOLVER.md and AGENTS.md present — RESOLVER.md wins', () => {
|
||||
// Policy: when both exist at the same location, gbrain-native wins.
|
||||
const workspace = scratch();
|
||||
const skillsDir = join(workspace, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
require('fs').writeFileSync(
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
'# RESOLVER\n\n| Trigger | Skill |\n|---|---|\n',
|
||||
);
|
||||
require('fs').writeFileSync(
|
||||
join(skillsDir, 'AGENTS.md'),
|
||||
'# AGENTS\n\n| Trigger | Skill |\n|---|---|\n',
|
||||
);
|
||||
const found = autoDetectSkillsDir(scratch(), { OPENCLAW_WORKSPACE: workspace });
|
||||
expect(found.dir).toBe(skillsDir);
|
||||
// Source is still `env` — the distinction is which file was found,
|
||||
// and RESOLVER.md takes precedence inside resolveWorkspaceSkillsDir.
|
||||
expect(found.source).toBe('openclaw_workspace_env');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,372 @@
|
||||
/**
|
||||
* Tests for src/core/routing-eval.ts — Check 5 harness.
|
||||
*
|
||||
* Covers: normalization, trigger extraction, structural match, negative
|
||||
* cases, ambiguity detection + allow-list, fixture linter (D-CX-6),
|
||||
* fixture loader, and the end-to-end runner.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import {
|
||||
extractTriggerPhrases,
|
||||
indexResolverTriggers,
|
||||
lintRoutingFixtures,
|
||||
loadRoutingFixtures,
|
||||
normalizeText,
|
||||
runRoutingEval,
|
||||
structuralRouteMatch,
|
||||
type RoutingFixture,
|
||||
} from '../src/core/routing-eval.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
function scratch(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'routing-eval-'));
|
||||
created.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function makeResolver(
|
||||
rows: { trigger: string; skill: string; section?: string }[],
|
||||
): string {
|
||||
const bySection = new Map<string, typeof rows>();
|
||||
for (const row of rows) {
|
||||
const sec = row.section ?? 'Brain operations';
|
||||
const list = bySection.get(sec) ?? [];
|
||||
list.push(row);
|
||||
bySection.set(sec, list);
|
||||
}
|
||||
const parts: string[] = ['# RESOLVER', ''];
|
||||
for (const [sec, list] of bySection) {
|
||||
parts.push(`## ${sec}`, '', '| Trigger | Skill |', '|---------|-------|');
|
||||
for (const r of list) {
|
||||
parts.push(`| ${r.trigger} | \`skills/${r.skill}/SKILL.md\` |`);
|
||||
}
|
||||
parts.push('');
|
||||
}
|
||||
return parts.join('\n');
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe('normalizeText', () => {
|
||||
it('lowercases and strips punctuation', () => {
|
||||
expect(normalizeText("What's up?")).toBe('what s up');
|
||||
});
|
||||
it('collapses whitespace', () => {
|
||||
expect(normalizeText(' foo bar\n\tbaz ')).toBe('foo bar baz');
|
||||
});
|
||||
it('handles unicode punctuation', () => {
|
||||
expect(normalizeText('search — fast!')).toBe('search fast');
|
||||
});
|
||||
it('empty input → empty', () => {
|
||||
expect(normalizeText('')).toBe('');
|
||||
expect(normalizeText(' !! ')).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractTriggerPhrases', () => {
|
||||
it('splits comma-separated quoted phrases', () => {
|
||||
const phrases = extractTriggerPhrases('"search for", "look up", "find me"');
|
||||
expect(phrases).toEqual(['search for', 'look up', 'find me']);
|
||||
});
|
||||
it('returns single unquoted cell as one phrase', () => {
|
||||
const phrases = extractTriggerPhrases('Creating/enriching a person page');
|
||||
expect(phrases).toEqual(['creating enriching a person page']);
|
||||
});
|
||||
it('filters phrases shorter than 3 chars', () => {
|
||||
// "x" too short; "hi" normalized to "hi" (2 chars, filtered)
|
||||
const phrases = extractTriggerPhrases('"x", "hi", "wave"');
|
||||
expect(phrases).toEqual(['wave']);
|
||||
});
|
||||
it('mixes quoted and non-quoted → quoted wins (split mode)', () => {
|
||||
// When ANY quoted phrase is present, we treat the cell as a list.
|
||||
const phrases = extractTriggerPhrases('things like "alpha" and "beta"');
|
||||
expect(phrases).toEqual(['alpha', 'beta']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('indexResolverTriggers', () => {
|
||||
it('indexes skills and normalizes their trigger phrases', () => {
|
||||
const resolver = makeResolver([
|
||||
{ trigger: '"search for", "find me"', skill: 'query' },
|
||||
{ trigger: 'Fix broken citations', skill: 'citation-fixer' },
|
||||
]);
|
||||
const idx = indexResolverTriggers(resolver);
|
||||
expect(idx.skillPhrases.get('query')).toEqual(['search for', 'find me']);
|
||||
expect(idx.skillPhrases.get('citation-fixer')).toEqual(['fix broken citations']);
|
||||
});
|
||||
it('accumulates phrases when one skill has multiple resolver rows', () => {
|
||||
const resolver = makeResolver([
|
||||
{ trigger: '"search", "lookup"', skill: 'query' },
|
||||
{ trigger: '"graph query"', skill: 'query' },
|
||||
]);
|
||||
const idx = indexResolverTriggers(resolver);
|
||||
expect(idx.skillPhrases.get('query')).toEqual(['search', 'lookup', 'graph query']);
|
||||
});
|
||||
it('skips GStack entries (external references)', () => {
|
||||
const resolver = [
|
||||
'# RESOLVER',
|
||||
'',
|
||||
'## External',
|
||||
'| Trigger | Skill |',
|
||||
'|---------|-------|',
|
||||
'| "plan review" | GStack: plan-ceo-review |',
|
||||
'| "do thing" | `skills/foo/SKILL.md` |',
|
||||
'',
|
||||
].join('\n');
|
||||
const idx = indexResolverTriggers(resolver);
|
||||
expect(Array.from(idx.skillPhrases.keys())).toEqual(['foo']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('structuralRouteMatch', () => {
|
||||
const index = indexResolverTriggers(
|
||||
makeResolver([
|
||||
{ trigger: '"search", "lookup", "find"', skill: 'query' },
|
||||
{ trigger: '"fix citations", "broken sources"', skill: 'citation-fixer' },
|
||||
{ trigger: '"every inbound message"', skill: 'signal-detector', section: 'Always-on' },
|
||||
]),
|
||||
);
|
||||
|
||||
it('matches a clean unambiguous intent', () => {
|
||||
const r = structuralRouteMatch('please lookup that person', index);
|
||||
expect(r.matched).toEqual(['query']);
|
||||
expect(r.ambiguous).toBe(false);
|
||||
});
|
||||
it('returns empty match for out-of-scope intent', () => {
|
||||
const r = structuralRouteMatch('deploy the app to prod', index);
|
||||
expect(r.matched).toEqual([]);
|
||||
expect(r.ambiguous).toBe(false);
|
||||
});
|
||||
it('flags ambiguity when two specific skills match', () => {
|
||||
const r = structuralRouteMatch('find broken sources in notes', index);
|
||||
expect(r.matched).toContain('query'); // "find"
|
||||
expect(r.matched).toContain('citation-fixer'); // "broken sources"
|
||||
expect(r.ambiguous).toBe(true);
|
||||
});
|
||||
it('does NOT flag ambiguity when always-on skill co-fires', () => {
|
||||
// always-on skills are exempted from the ambiguity check.
|
||||
const r = structuralRouteMatch('every inbound message lookup', index);
|
||||
expect(r.matched).toContain('query');
|
||||
expect(r.matched).toContain('signal-detector');
|
||||
expect(r.ambiguous).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('lintRoutingFixtures (D-CX-6)', () => {
|
||||
const resolver = makeResolver([
|
||||
{ trigger: '"find me", "search for"', skill: 'query' },
|
||||
{ trigger: 'Fix broken citations', skill: 'citation-fixer' },
|
||||
]);
|
||||
const index = indexResolverTriggers(resolver);
|
||||
|
||||
it('rejects fixture whose intent is verbatim-equal to a trigger', () => {
|
||||
// Intent equals the trigger exactly (after normalization): pure
|
||||
// tautology. These are the copy-paste fixtures D-CX-6 targets.
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'find me', expected_skill: 'query' },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0].reason).toBe('intent_copies_trigger');
|
||||
});
|
||||
it('accepts a natural-sentence fixture embedding trigger words', () => {
|
||||
// Intent embeds the trigger in a natural sentence — this is exactly
|
||||
// what Layer A's substring match is supposed to detect, so the
|
||||
// linter must NOT flag it.
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'please find me that paper', expected_skill: 'query' },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues).toEqual([]);
|
||||
});
|
||||
it('accepts a fully-paraphrased fixture (no trigger words at all)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'pull up what we know about Paul', expected_skill: 'query' },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues).toEqual([]);
|
||||
});
|
||||
it('flags unknown expected_skill (typo / dead reference)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'something', expected_skill: 'not-a-skill' },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0].reason).toBe('unknown_expected_skill');
|
||||
});
|
||||
it('skips verbatim check for negative cases (expected_skill=null)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'find broken citations', expected_skill: null },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues).toEqual([]);
|
||||
});
|
||||
it('flags invalid shape (missing intent, wrong type)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: '', expected_skill: 'query' },
|
||||
{ intent: 'ok', expected_skill: 42 as unknown as string },
|
||||
];
|
||||
const issues = lintRoutingFixtures(fixtures, index);
|
||||
expect(issues.length).toBe(2);
|
||||
expect(issues[0].reason).toBe('invalid_shape');
|
||||
expect(issues[1].reason).toBe('invalid_shape');
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadRoutingFixtures', () => {
|
||||
function seedFixture(skillsDir: string, name: string, lines: string[]): void {
|
||||
const dir = join(skillsDir, name);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(join(dir, 'SKILL.md'), `---\nname: ${name}\n---\n`);
|
||||
writeFileSync(join(dir, 'routing-eval.jsonl'), lines.join('\n'));
|
||||
}
|
||||
|
||||
it('walks skills/*/routing-eval.jsonl and parses JSON-per-line', () => {
|
||||
const dir = scratch();
|
||||
seedFixture(dir, 'query', [
|
||||
'{"intent":"lookup paul","expected_skill":"query"}',
|
||||
'{"intent":"find that doc","expected_skill":"query"}',
|
||||
]);
|
||||
seedFixture(dir, 'other', [
|
||||
'{"intent":"clean citations","expected_skill":"other"}',
|
||||
]);
|
||||
const r = loadRoutingFixtures(dir);
|
||||
expect(r.fixtures.length).toBe(3);
|
||||
expect(r.malformed).toEqual([]);
|
||||
expect(r.fixtures.every(f => f.source !== undefined)).toBe(true);
|
||||
});
|
||||
it('skips comments and blank lines', () => {
|
||||
const dir = scratch();
|
||||
seedFixture(dir, 'query', [
|
||||
'// comment',
|
||||
'',
|
||||
'# also comment',
|
||||
'{"intent":"lookup","expected_skill":"query"}',
|
||||
]);
|
||||
const r = loadRoutingFixtures(dir);
|
||||
expect(r.fixtures.length).toBe(1);
|
||||
});
|
||||
it('collects malformed lines separately without crashing', () => {
|
||||
const dir = scratch();
|
||||
seedFixture(dir, 'query', [
|
||||
'{"intent":"ok","expected_skill":"query"}',
|
||||
'{ bad json',
|
||||
'{"intent":"also ok","expected_skill":"query"}',
|
||||
]);
|
||||
const r = loadRoutingFixtures(dir);
|
||||
expect(r.fixtures.length).toBe(2);
|
||||
expect(r.malformed.length).toBe(1);
|
||||
expect(r.malformed[0].line).toBe(2);
|
||||
});
|
||||
it('handles missing skills dir cleanly', () => {
|
||||
const r = loadRoutingFixtures('/tmp/never-exists-routing-eval-XYZ');
|
||||
expect(r.fixtures).toEqual([]);
|
||||
expect(r.malformed).toEqual([]);
|
||||
});
|
||||
it('skips underscore and dot dirs', () => {
|
||||
const dir = scratch();
|
||||
seedFixture(dir, '_conventions', [
|
||||
'{"intent":"x","expected_skill":"y"}',
|
||||
]);
|
||||
seedFixture(dir, '.hidden', [
|
||||
'{"intent":"x","expected_skill":"y"}',
|
||||
]);
|
||||
const r = loadRoutingFixtures(dir);
|
||||
expect(r.fixtures).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runRoutingEval', () => {
|
||||
const resolver = makeResolver([
|
||||
{ trigger: '"find me", "search for", "look up"', skill: 'query' },
|
||||
{ trigger: '"broken citations", "fix citations"', skill: 'citation-fixer' },
|
||||
{ trigger: '"every inbound message"', skill: 'signal-detector', section: 'Always-on' },
|
||||
]);
|
||||
|
||||
it('scores a passing fixture', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'please look up that person', expected_skill: 'query' },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.passed).toBe(1);
|
||||
expect(r.top1Accuracy).toBe(1);
|
||||
expect(r.details[0].outcome).toBe('pass');
|
||||
});
|
||||
it('detects a missed fixture (no match)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'deploy to prod', expected_skill: 'query' },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.missed).toBe(1);
|
||||
expect(r.details[0].outcome).toBe('missed');
|
||||
});
|
||||
it('detects ambiguity when unexpected skills also match', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'search for broken citations', expected_skill: 'query' },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.ambiguous).toBe(1);
|
||||
expect(r.details[0].outcome).toBe('ambiguous');
|
||||
expect(r.details[0].note).toContain('citation-fixer');
|
||||
});
|
||||
it('honors ambiguous_with allow-list', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{
|
||||
intent: 'search for broken citations',
|
||||
expected_skill: 'query',
|
||||
ambiguous_with: ['citation-fixer'],
|
||||
},
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.passed).toBe(1);
|
||||
expect(r.ambiguous).toBe(0);
|
||||
});
|
||||
it('passes when only always-on skills co-fire', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'every inbound message look up', expected_skill: 'query' },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.passed).toBe(1);
|
||||
});
|
||||
it('passes a negative case when nothing matches', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'deploy the app tonight', expected_skill: null },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.passed).toBe(1);
|
||||
});
|
||||
it('fails a negative case when something matches (false positive)', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'please look up this fact', expected_skill: null },
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.falsePositives).toBe(1);
|
||||
expect(r.details[0].outcome).toBe('false_positive');
|
||||
});
|
||||
it('empty fixture set → top1Accuracy = 1 (vacuous pass)', () => {
|
||||
const r = runRoutingEval(resolver, []);
|
||||
expect(r.top1Accuracy).toBe(1);
|
||||
expect(r.totalCases).toBe(0);
|
||||
});
|
||||
it('mixed case report totals add up', () => {
|
||||
const fixtures: RoutingFixture[] = [
|
||||
{ intent: 'pull up paul graham', expected_skill: 'query' }, // pass
|
||||
{ intent: 'deploy prod', expected_skill: 'query' }, // miss
|
||||
{ intent: 'fix busted sources', expected_skill: null }, // false pos
|
||||
];
|
||||
const r = runRoutingEval(resolver, fixtures);
|
||||
expect(r.totalCases).toBe(3);
|
||||
expect(r.passed + r.missed + r.ambiguous + r.falsePositives).toBe(3);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,177 @@
|
||||
/**
|
||||
* Tests for src/core/skill-manifest.ts — unified manifest loader.
|
||||
*
|
||||
* Covers the derive-from-walk path (F-ENG-1 / D-CX-1..4). When
|
||||
* manifest.json is absent, walking skillsDir MUST produce a sensible
|
||||
* synthetic manifest so reachability checks don't silently pass on
|
||||
* OpenClaw deployments that don't ship manifest.json.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, beforeEach, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import { loadOrDeriveManifest } from '../src/core/skill-manifest.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
function scratch(): string {
|
||||
const dir = mkdtempSync(join(tmpdir(), 'skill-manifest-'));
|
||||
created.push(dir);
|
||||
return dir;
|
||||
}
|
||||
|
||||
function writeSkill(skillsDir: string, name: string, frontmatterName?: string): void {
|
||||
const skillDir = join(skillsDir, name);
|
||||
mkdirSync(skillDir, { recursive: true });
|
||||
const fm = frontmatterName !== undefined
|
||||
? `---\nname: ${frontmatterName}\ndescription: test\n---\n`
|
||||
: ''; // no frontmatter
|
||||
writeFileSync(join(skillDir, 'SKILL.md'), `${fm}\n# ${name}\n`);
|
||||
}
|
||||
|
||||
function writeManifest(skillsDir: string, json: unknown): void {
|
||||
writeFileSync(join(skillsDir, 'manifest.json'), JSON.stringify(json, null, 2));
|
||||
}
|
||||
|
||||
describe('loadOrDeriveManifest', () => {
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('uses manifest.json verbatim when present and valid', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query');
|
||||
writeSkill(dir, 'ingest');
|
||||
writeManifest(dir, {
|
||||
skills: [
|
||||
{ name: 'query', path: 'query/SKILL.md' },
|
||||
{ name: 'ingest', path: 'ingest/SKILL.md' },
|
||||
],
|
||||
});
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(false);
|
||||
expect(r.skills.length).toBe(2);
|
||||
expect(r.skills.map(s => s.name).sort()).toEqual(['ingest', 'query']);
|
||||
});
|
||||
|
||||
it('derives from SKILL.md walk when manifest.json is missing (F-ENG-1)', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query');
|
||||
writeSkill(dir, 'ingest', 'ingest');
|
||||
// No manifest.json — this is the OpenClaw-deployment scenario.
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.length).toBe(2);
|
||||
expect(r.skills.map(s => s.name).sort()).toEqual(['ingest', 'query']);
|
||||
expect(r.skills.every(s => s.path.endsWith('/SKILL.md'))).toBe(true);
|
||||
});
|
||||
|
||||
it('falls back to dirname when SKILL.md has no name: frontmatter', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'prose-only-skill'); // no frontmatter
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.length).toBe(1);
|
||||
expect(r.skills[0].name).toBe('prose-only-skill');
|
||||
});
|
||||
|
||||
it('uses frontmatter name when it differs from dirname', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'weird-dir-name', 'canonical-skill-name');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills[0].name).toBe('canonical-skill-name');
|
||||
expect(r.skills[0].path).toBe('weird-dir-name/SKILL.md');
|
||||
});
|
||||
|
||||
it('skips underscore-prefixed dirs (conventions, rule files)', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query');
|
||||
writeSkill(dir, '_conventions');
|
||||
writeSkill(dir, '_brain-rules');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['query']);
|
||||
});
|
||||
|
||||
it('skips dot-prefixed dirs', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query');
|
||||
writeSkill(dir, '.git');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['query']);
|
||||
});
|
||||
|
||||
it('derives when manifest.json is malformed (invalid JSON)', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query');
|
||||
writeFileSync(join(dir, 'manifest.json'), '{ not valid json');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['query']);
|
||||
});
|
||||
|
||||
it('derives when manifest.json has a wrong shape (skills as object)', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query');
|
||||
writeManifest(dir, { skills: { bad: 'shape' } });
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['query']);
|
||||
});
|
||||
|
||||
it('honors explicit empty skills array as a valid declaration', () => {
|
||||
// An empty array is "no skills" declared intentionally, not
|
||||
// malformed. Distinct from missing manifest.json (→ derive).
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query'); // present on disk...
|
||||
writeManifest(dir, { skills: [] }); // ...but manifest says zero.
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(false);
|
||||
expect(r.skills.length).toBe(0);
|
||||
});
|
||||
|
||||
it('derives when manifest.json entry lacks required keys', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query');
|
||||
// First entry missing 'path' → invalid shape → fall through to derive.
|
||||
writeManifest(dir, { skills: [{ name: 'query' }] });
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
});
|
||||
|
||||
it('handles empty skillsDir cleanly', () => {
|
||||
const dir = scratch();
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.length).toBe(0);
|
||||
});
|
||||
|
||||
it('handles non-existent skillsDir cleanly', () => {
|
||||
const r = loadOrDeriveManifest('/tmp/does-not-exist-skill-manifest-test');
|
||||
expect(r.derived).toBe(true);
|
||||
expect(r.skills.length).toBe(0);
|
||||
});
|
||||
|
||||
it('sorts derived skills alphabetically by name', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'zebra', 'zebra');
|
||||
writeSkill(dir, 'apple', 'apple');
|
||||
writeSkill(dir, 'mango', 'mango');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['apple', 'mango', 'zebra']);
|
||||
});
|
||||
|
||||
it('treats dirs without SKILL.md as not-a-skill', () => {
|
||||
const dir = scratch();
|
||||
writeSkill(dir, 'query', 'query');
|
||||
mkdirSync(join(dir, 'no-skill-here'), { recursive: true });
|
||||
writeFileSync(join(dir, 'no-skill-here', 'README.md'), '# not a skill');
|
||||
const r = loadOrDeriveManifest(dir);
|
||||
expect(r.skills.map(s => s.name)).toEqual(['query']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,343 @@
|
||||
/**
|
||||
* Tests for src/core/skillify/generator.ts (W4).
|
||||
* Mechanical scaffold plan + apply, idempotency (D-CX-7), stub sentinel.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import { existsSync, mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import {
|
||||
applyScaffold,
|
||||
planScaffold,
|
||||
SkillifyScaffoldError,
|
||||
SKILL_NAME_PATTERN,
|
||||
} from '../src/core/skillify/generator.ts';
|
||||
import { SKILLIFY_STUB_MARKER } from '../src/core/skillify/templates.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
function scratchRepo(): { root: string; skillsDir: string } {
|
||||
const root = mkdtempSync(join(tmpdir(), 'skillify-repo-'));
|
||||
created.push(root);
|
||||
const skillsDir = join(root, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
mkdirSync(join(root, 'test'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
'# RESOLVER\n\n## Brain operations\n\n| Trigger | Skill |\n|---------|-------|\n| "existing thing" | `skills/existing/SKILL.md` |\n',
|
||||
);
|
||||
return { root, skillsDir };
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe('SKILL_NAME_PATTERN', () => {
|
||||
it('accepts valid kebab-case', () => {
|
||||
expect(SKILL_NAME_PATTERN.test('context-now')).toBe(true);
|
||||
expect(SKILL_NAME_PATTERN.test('a')).toBe(true);
|
||||
expect(SKILL_NAME_PATTERN.test('calendar-recall-v2')).toBe(true);
|
||||
});
|
||||
it('rejects uppercase, spaces, underscores, leading digits', () => {
|
||||
expect(SKILL_NAME_PATTERN.test('ContextNow')).toBe(false);
|
||||
expect(SKILL_NAME_PATTERN.test('context now')).toBe(false);
|
||||
expect(SKILL_NAME_PATTERN.test('context_now')).toBe(false);
|
||||
expect(SKILL_NAME_PATTERN.test('2-skill')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('planScaffold', () => {
|
||||
it('throws SkillifyScaffoldError on invalid name', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
expect(() =>
|
||||
planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'Bad Name',
|
||||
description: 'x',
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
}),
|
||||
).toThrow(SkillifyScaffoldError);
|
||||
});
|
||||
|
||||
it('plans 4 files + resolver append for a new skill', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'hello-world',
|
||||
description: 'say hello',
|
||||
triggers: ['say hello', 'greet me'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
expect(plan.files.length).toBe(4);
|
||||
const paths = plan.files.map(f => f.path);
|
||||
expect(paths).toContain(join(skillsDir, 'hello-world', 'SKILL.md'));
|
||||
expect(paths).toContain(
|
||||
join(skillsDir, 'hello-world', 'scripts', 'hello-world.mjs'),
|
||||
);
|
||||
expect(paths).toContain(join(skillsDir, 'hello-world', 'routing-eval.jsonl'));
|
||||
expect(paths).toContain(join(root, 'test', 'hello-world.test.ts'));
|
||||
expect(plan.files.every(f => f.kind === 'new')).toBe(true);
|
||||
expect(plan.resolverFile).toBe(join(skillsDir, 'RESOLVER.md'));
|
||||
expect(plan.resolverAppend).not.toBeNull();
|
||||
expect(plan.resolverAppend!).toContain('`skills/hello-world/SKILL.md`');
|
||||
});
|
||||
|
||||
it('SKILL.md includes the SKILLIFY_STUB sentinel', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'foo',
|
||||
description: 'foo skill',
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
const skillMd = plan.files.find(f => f.path.endsWith('SKILL.md'))!;
|
||||
expect(skillMd.content).toContain(SKILLIFY_STUB_MARKER);
|
||||
});
|
||||
|
||||
it('script stub includes the SKILLIFY_STUB sentinel (D-CX-9 gate hook)', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'foo',
|
||||
description: 'foo skill',
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
const script = plan.files.find(f => f.path.endsWith('foo.mjs'))!;
|
||||
expect(script.content).toContain(SKILLIFY_STUB_MARKER);
|
||||
});
|
||||
|
||||
it('refuses to scaffold over an existing file without --force', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
mkdirSync(join(skillsDir, 'existing'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'existing', 'SKILL.md'),
|
||||
'---\nname: existing\n---\n',
|
||||
);
|
||||
expect(() =>
|
||||
planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'existing',
|
||||
description: 'x',
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
}),
|
||||
).toThrow(SkillifyScaffoldError);
|
||||
});
|
||||
|
||||
it('--force marks existing files as overwrite', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
mkdirSync(join(skillsDir, 'existing'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'existing', 'SKILL.md'),
|
||||
'---\nname: existing\n---\n',
|
||||
);
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
force: true,
|
||||
vars: {
|
||||
name: 'existing',
|
||||
description: 'x',
|
||||
triggers: ['foo'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
const skillMd = plan.files.find(f => f.path.endsWith('SKILL.md'))!;
|
||||
expect(skillMd.kind).toBe('overwrite');
|
||||
});
|
||||
|
||||
it('D-CX-7: resolverAppend is null when row already present (idempotent)', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
// Prime the resolver with an existing row for the skill we're about
|
||||
// to scaffold. The plan must NOT queue a duplicate append, even
|
||||
// when --force regenerates files.
|
||||
const resolverPath = join(skillsDir, 'RESOLVER.md');
|
||||
const before = readFileSync(resolverPath, 'utf-8');
|
||||
writeFileSync(
|
||||
resolverPath,
|
||||
before +
|
||||
'\n## Uncategorized\n\n| Trigger | Skill |\n|---------|-------|\n' +
|
||||
'| "do thing" | `skills/demo/SKILL.md` |\n',
|
||||
);
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'demo',
|
||||
description: 'demo',
|
||||
triggers: ['do thing'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
expect(plan.resolverAppend).toBeNull();
|
||||
});
|
||||
|
||||
it('handles --triggers omitted by seeding TBD placeholder', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'empty-triggers',
|
||||
description: 'test',
|
||||
triggers: [],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
const skillMd = plan.files.find(f => f.path.endsWith('SKILL.md'))!;
|
||||
expect(skillMd.content).toContain('TBD-trigger');
|
||||
});
|
||||
|
||||
it('writes_pages + writes_to flow through to frontmatter', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'writer',
|
||||
description: 'writer',
|
||||
triggers: ['write me'],
|
||||
writesTo: ['people/', 'companies/'],
|
||||
writesPages: true,
|
||||
mutating: true,
|
||||
},
|
||||
});
|
||||
const skillMd = plan.files.find(f => f.path.endsWith('SKILL.md'))!;
|
||||
expect(skillMd.content).toContain('writes_pages: true');
|
||||
expect(skillMd.content).toContain('- people/');
|
||||
expect(skillMd.content).toContain('- companies/');
|
||||
expect(skillMd.content).toContain('mutating: true');
|
||||
});
|
||||
});
|
||||
|
||||
describe('applyScaffold', () => {
|
||||
it('writes all planned files and appends the resolver row', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'hello',
|
||||
description: 'hi',
|
||||
triggers: ['say hi'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
applyScaffold(plan);
|
||||
for (const f of plan.files) {
|
||||
expect(existsSync(f.path)).toBe(true);
|
||||
}
|
||||
const resolver = readFileSync(join(skillsDir, 'RESOLVER.md'), 'utf-8');
|
||||
expect(resolver).toContain('`skills/hello/SKILL.md`');
|
||||
});
|
||||
|
||||
it('second apply with same name + --force overwrites files but does NOT duplicate resolver row (D-CX-7)', () => {
|
||||
const { root, skillsDir } = scratchRepo();
|
||||
|
||||
const firstPlan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'idem',
|
||||
description: 'first',
|
||||
triggers: ['t'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
applyScaffold(firstPlan);
|
||||
|
||||
const secondPlan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
force: true,
|
||||
vars: {
|
||||
name: 'idem',
|
||||
description: 'second',
|
||||
triggers: ['t'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
expect(secondPlan.resolverAppend).toBeNull();
|
||||
applyScaffold(secondPlan);
|
||||
|
||||
const resolver = readFileSync(join(skillsDir, 'RESOLVER.md'), 'utf-8');
|
||||
const count = (resolver.match(/`skills\/idem\/SKILL\.md`/g) ?? []).length;
|
||||
expect(count).toBe(1);
|
||||
});
|
||||
|
||||
it('applies against an AGENTS.md-layout workspace (W1 interop)', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'skillify-openclaw-'));
|
||||
created.push(root);
|
||||
const skillsDir = join(root, 'workspace', 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
mkdirSync(join(root, 'test'), { recursive: true });
|
||||
// AGENTS.md at workspace root, NOT inside skills/.
|
||||
writeFileSync(
|
||||
join(root, 'workspace', 'AGENTS.md'),
|
||||
'# AGENTS\n\n## Ops\n\n| Trigger | Skill |\n|---------|-------|\n',
|
||||
);
|
||||
const plan = planScaffold({
|
||||
skillsDir,
|
||||
repoRoot: root,
|
||||
vars: {
|
||||
name: 'openclaw-demo',
|
||||
description: 'demo',
|
||||
triggers: ['do it'],
|
||||
writesTo: [],
|
||||
writesPages: false,
|
||||
mutating: false,
|
||||
},
|
||||
});
|
||||
expect(plan.resolverFile).toBe(join(root, 'workspace', 'AGENTS.md'));
|
||||
expect(plan.resolverAppend).not.toBeNull();
|
||||
applyScaffold(plan);
|
||||
const agents = readFileSync(join(root, 'workspace', 'AGENTS.md'), 'utf-8');
|
||||
expect(agents).toContain('`skills/openclaw-demo/SKILL.md`');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,447 @@
|
||||
/**
|
||||
* Tests for src/core/skillpack/bundle.ts + installer.ts (W5).
|
||||
* Bundle enumeration, dependency closure, per-file diff, managed
|
||||
* block, lockfile concurrency, atomic writes.
|
||||
*/
|
||||
|
||||
import { describe, expect, it, afterEach } from 'bun:test';
|
||||
import {
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
rmSync,
|
||||
writeFileSync,
|
||||
} from 'fs';
|
||||
import { dirname, join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
import {
|
||||
bundledSkillSlugs,
|
||||
findGbrainRoot,
|
||||
loadBundleManifest,
|
||||
enumerateBundle,
|
||||
BundleError,
|
||||
} from '../src/core/skillpack/bundle.ts';
|
||||
import {
|
||||
applyInstall,
|
||||
buildManagedBlock,
|
||||
diffSkill,
|
||||
extractManagedSlugs,
|
||||
planInstall,
|
||||
updateManagedBlock,
|
||||
InstallError,
|
||||
} from '../src/core/skillpack/installer.ts';
|
||||
|
||||
const created: string[] = [];
|
||||
|
||||
function scratchGbrain(): { gbrainRoot: string; skillsDir: string } {
|
||||
const root = mkdtempSync(join(tmpdir(), 'skillpack-gbrain-'));
|
||||
created.push(root);
|
||||
mkdirSync(join(root, 'src'), { recursive: true });
|
||||
writeFileSync(join(root, 'src', 'cli.ts'), '// stub');
|
||||
const skillsDir = join(root, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
|
||||
// Two bundled skills + shared deps.
|
||||
mkdirSync(join(skillsDir, 'alpha'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'alpha', 'SKILL.md'),
|
||||
'---\nname: alpha\n---\n# alpha\n',
|
||||
);
|
||||
mkdirSync(join(skillsDir, 'alpha', 'scripts'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'alpha', 'scripts', 'alpha.mjs'),
|
||||
'export function run() { return "alpha"; }\n',
|
||||
);
|
||||
|
||||
mkdirSync(join(skillsDir, 'beta'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'beta', 'SKILL.md'),
|
||||
'---\nname: beta\n---\n# beta\n',
|
||||
);
|
||||
|
||||
// Shared deps.
|
||||
mkdirSync(join(skillsDir, 'conventions'), { recursive: true });
|
||||
writeFileSync(
|
||||
join(skillsDir, 'conventions', 'quality.md'),
|
||||
'# quality conventions\n',
|
||||
);
|
||||
writeFileSync(join(skillsDir, '_output-rules.md'), '# output rules\n');
|
||||
|
||||
// RESOLVER.md (so find-resolver finds it).
|
||||
writeFileSync(
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
'# RESOLVER\n\n| Trigger | Skill |\n|---------|-------|\n| "alpha" | `skills/alpha/SKILL.md` |\n| "beta" | `skills/beta/SKILL.md` |\n',
|
||||
);
|
||||
|
||||
// Plugin manifest.
|
||||
writeFileSync(
|
||||
join(root, 'openclaw.plugin.json'),
|
||||
JSON.stringify(
|
||||
{
|
||||
name: 'gbrain-test',
|
||||
version: '0.17.0-test',
|
||||
skills: ['skills/alpha', 'skills/beta'],
|
||||
shared_deps: ['skills/conventions', 'skills/_output-rules.md'],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
);
|
||||
|
||||
return { gbrainRoot: root, skillsDir };
|
||||
}
|
||||
|
||||
function scratchTarget(): { workspace: string; skillsDir: string } {
|
||||
const workspace = mkdtempSync(join(tmpdir(), 'skillpack-target-'));
|
||||
created.push(workspace);
|
||||
const skillsDir = join(workspace, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
// Seed a RESOLVER.md so managed block has a home.
|
||||
writeFileSync(
|
||||
join(skillsDir, 'RESOLVER.md'),
|
||||
'# Target RESOLVER\n\n| Trigger | Skill |\n|---------|-------|\n',
|
||||
);
|
||||
return { workspace, skillsDir };
|
||||
}
|
||||
|
||||
afterEach(() => {
|
||||
while (created.length) {
|
||||
const d = created.pop();
|
||||
if (d && existsSync(d)) rmSync(d, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
describe('findGbrainRoot', () => {
|
||||
it('walks up to find openclaw.plugin.json + src/cli.ts', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const nested = join(gbrainRoot, 'a', 'b', 'c');
|
||||
mkdirSync(nested, { recursive: true });
|
||||
expect(findGbrainRoot(nested)).toBe(gbrainRoot);
|
||||
});
|
||||
it('returns null when no gbrain root above', () => {
|
||||
expect(findGbrainRoot('/tmp/definitely-not-a-gbrain-repo-XYZ')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadBundleManifest', () => {
|
||||
it('loads + validates a valid manifest', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const m = loadBundleManifest(gbrainRoot);
|
||||
expect(m.skills).toEqual(['skills/alpha', 'skills/beta']);
|
||||
expect(m.shared_deps.length).toBe(2);
|
||||
});
|
||||
it('throws BundleError on missing manifest', () => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'skillpack-empty-'));
|
||||
created.push(root);
|
||||
expect(() => loadBundleManifest(root)).toThrow(BundleError);
|
||||
});
|
||||
it('throws BundleError on malformed JSON', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
writeFileSync(join(gbrainRoot, 'openclaw.plugin.json'), '{ nope');
|
||||
expect(() => loadBundleManifest(gbrainRoot)).toThrow(BundleError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('enumerateBundle (D-CX-10 dependency closure)', () => {
|
||||
it('includes skill files AND shared_deps for a single-skill install', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const m = loadBundleManifest(gbrainRoot);
|
||||
const entries = enumerateBundle({ gbrainRoot, skillSlug: 'alpha', manifest: m });
|
||||
const targets = entries.map(e => e.relTarget).sort();
|
||||
expect(targets).toContain('alpha/SKILL.md');
|
||||
expect(targets).toContain('alpha/scripts/alpha.mjs');
|
||||
// Shared deps pulled in despite single-skill scope.
|
||||
expect(targets).toContain('conventions/quality.md');
|
||||
expect(targets).toContain('_output-rules.md');
|
||||
// beta NOT included.
|
||||
expect(targets.find(t => t.startsWith('beta/'))).toBeUndefined();
|
||||
});
|
||||
it('throws BundleError for unknown skill slug', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const m = loadBundleManifest(gbrainRoot);
|
||||
expect(() =>
|
||||
enumerateBundle({ gbrainRoot, skillSlug: 'nope', manifest: m }),
|
||||
).toThrow(BundleError);
|
||||
});
|
||||
it('enumerates everything when skillSlug is undefined (--all)', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const m = loadBundleManifest(gbrainRoot);
|
||||
const entries = enumerateBundle({ gbrainRoot, manifest: m });
|
||||
const targets = entries.map(e => e.relTarget).sort();
|
||||
expect(targets.some(t => t.startsWith('alpha/'))).toBe(true);
|
||||
expect(targets.some(t => t.startsWith('beta/'))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildManagedBlock + updateManagedBlock', () => {
|
||||
it('builds a block with all installed slugs as rows', () => {
|
||||
const m = loadBundleManifest(scratchGbrain().gbrainRoot);
|
||||
const block = buildManagedBlock(m, ['alpha', 'beta']);
|
||||
expect(block).toContain('gbrain:skillpack:begin');
|
||||
expect(block).toContain('gbrain:skillpack:end');
|
||||
expect(block).toContain('`skills/alpha/SKILL.md`');
|
||||
expect(block).toContain('`skills/beta/SKILL.md`');
|
||||
});
|
||||
it('appends block when none exists', () => {
|
||||
const block = buildManagedBlock(loadBundleManifest(scratchGbrain().gbrainRoot), ['alpha']);
|
||||
const updated = updateManagedBlock('# AGENTS\n\nSome prose.\n', block);
|
||||
expect(updated).toContain('gbrain:skillpack:begin');
|
||||
expect(updated).toContain('Some prose.');
|
||||
});
|
||||
it('replaces existing block in place, keeping surrounding content', () => {
|
||||
const m = loadBundleManifest(scratchGbrain().gbrainRoot);
|
||||
const original =
|
||||
'# AGENTS\n\nBefore\n\n' +
|
||||
buildManagedBlock(m, ['alpha']) +
|
||||
'\n\nAfter\n';
|
||||
const replaced = updateManagedBlock(original, buildManagedBlock(m, ['alpha', 'beta']));
|
||||
expect(replaced).toContain('Before');
|
||||
expect(replaced).toContain('After');
|
||||
expect(replaced).toContain('`skills/beta/SKILL.md`');
|
||||
});
|
||||
it('extractManagedSlugs roundtrips with buildManagedBlock', () => {
|
||||
const m = loadBundleManifest(scratchGbrain().gbrainRoot);
|
||||
const block = buildManagedBlock(m, ['alpha', 'beta']);
|
||||
expect(extractManagedSlugs(block).sort()).toEqual(['alpha', 'beta']);
|
||||
});
|
||||
it('extractManagedSlugs returns [] when no block present', () => {
|
||||
expect(extractManagedSlugs('# hello\n\nno block here\n')).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('planInstall + applyInstall', () => {
|
||||
it('dry-run: plans file writes but does not touch target', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const plan = planInstall({
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
});
|
||||
const result = applyInstall(plan, {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
dryRun: true,
|
||||
});
|
||||
expect(result.dryRun).toBe(true);
|
||||
expect(result.summary.wroteNew).toBeGreaterThan(0);
|
||||
expect(existsSync(join(skillsDir, 'alpha', 'SKILL.md'))).toBe(false);
|
||||
});
|
||||
|
||||
it('installs a fresh skill and its shared deps', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const plan = planInstall({
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
});
|
||||
const result = applyInstall(plan, {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
});
|
||||
expect(result.summary.wroteNew).toBeGreaterThan(0);
|
||||
expect(existsSync(join(skillsDir, 'alpha', 'SKILL.md'))).toBe(true);
|
||||
expect(existsSync(join(skillsDir, 'alpha', 'scripts', 'alpha.mjs'))).toBe(true);
|
||||
expect(existsSync(join(skillsDir, 'conventions', 'quality.md'))).toBe(true);
|
||||
expect(result.managedBlock.applied).toBe(true);
|
||||
});
|
||||
|
||||
it('re-install is idempotent (skipped_identical on unchanged files)', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
const second = applyInstall(planInstall(opts), opts);
|
||||
expect(second.summary.wroteNew).toBe(0);
|
||||
expect(second.summary.skippedIdentical).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('skips locally-modified files without --overwrite-local', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
// Locally edit the installed SKILL.md
|
||||
const localFile = join(skillsDir, 'alpha', 'SKILL.md');
|
||||
writeFileSync(localFile, readFileSync(localFile, 'utf-8') + '\n<!-- local edit -->\n');
|
||||
|
||||
const result = applyInstall(planInstall(opts), opts);
|
||||
expect(result.summary.skippedLocallyModified).toBeGreaterThan(0);
|
||||
// Local edit preserved.
|
||||
expect(readFileSync(localFile, 'utf-8')).toContain('local edit');
|
||||
});
|
||||
|
||||
it('--overwrite-local replaces locally-modified files', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
const localFile = join(skillsDir, 'alpha', 'SKILL.md');
|
||||
writeFileSync(localFile, 'local garbage');
|
||||
|
||||
const overwriteOpts = { ...opts, overwriteLocal: true };
|
||||
const result = applyInstall(planInstall(overwriteOpts), overwriteOpts);
|
||||
expect(result.summary.wroteOverwrite).toBeGreaterThan(0);
|
||||
expect(readFileSync(localFile, 'utf-8')).not.toContain('local garbage');
|
||||
});
|
||||
|
||||
it('D-CX-11: concurrent install attempt fails with lock_held', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
// Simulate a peer holding the lock.
|
||||
writeFileSync(join(workspace, '.gbrain-skillpack.lock'), '99999');
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
const plan = planInstall(opts);
|
||||
expect(() => applyInstall(plan, opts)).toThrow(InstallError);
|
||||
});
|
||||
|
||||
it('D-CX-11: --force-unlock overrides a stale lock', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
// Stale lock is handled by setting lockStaleMs small and sleeping
|
||||
// — simulate by writing the lock and passing lockStaleMs=0 so
|
||||
// any age looks stale.
|
||||
writeFileSync(join(workspace, '.gbrain-skillpack.lock'), '99999');
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
forceUnlock: true,
|
||||
lockStaleMs: 0,
|
||||
};
|
||||
const plan = planInstall(opts);
|
||||
const result = applyInstall(plan, opts);
|
||||
expect(result.summary.wroteNew).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('managed block is written atomically (tmp then rename)', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: null,
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
const resolver = readFileSync(join(skillsDir, 'RESOLVER.md'), 'utf-8');
|
||||
expect(resolver).toContain('gbrain:skillpack:begin');
|
||||
expect(resolver).toContain('gbrain:skillpack:end');
|
||||
expect(resolver).toContain('`skills/alpha/SKILL.md`');
|
||||
expect(resolver).toContain('`skills/beta/SKILL.md`');
|
||||
});
|
||||
|
||||
it('managed block accumulates across separate single-skill installs', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
|
||||
const alphaOpts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(alphaOpts), alphaOpts);
|
||||
|
||||
const betaOpts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'beta',
|
||||
};
|
||||
applyInstall(planInstall(betaOpts), betaOpts);
|
||||
|
||||
const resolver = readFileSync(join(skillsDir, 'RESOLVER.md'), 'utf-8');
|
||||
expect(resolver).toContain('`skills/alpha/SKILL.md`');
|
||||
expect(resolver).toContain('`skills/beta/SKILL.md`');
|
||||
});
|
||||
|
||||
it('works against AGENTS.md-at-workspace-root layout', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const workspace = mkdtempSync(join(tmpdir(), 'skillpack-root-agents-'));
|
||||
created.push(workspace);
|
||||
const skillsDir = join(workspace, 'skills');
|
||||
mkdirSync(skillsDir, { recursive: true });
|
||||
// No RESOLVER.md in skills — AGENTS.md at workspace root instead.
|
||||
writeFileSync(
|
||||
join(workspace, 'AGENTS.md'),
|
||||
'# AGENTS\n\n| Trigger | Skill |\n|---------|-------|\n',
|
||||
);
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
const result = applyInstall(planInstall(opts), opts);
|
||||
expect(result.managedBlock.applied).toBe(true);
|
||||
const agents = readFileSync(join(workspace, 'AGENTS.md'), 'utf-8');
|
||||
expect(agents).toContain('`skills/alpha/SKILL.md`');
|
||||
});
|
||||
});
|
||||
|
||||
describe('diffSkill', () => {
|
||||
it('reports missing files', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { skillsDir } = scratchTarget();
|
||||
const diffs = diffSkill(gbrainRoot, 'alpha', skillsDir);
|
||||
expect(diffs.every(d => !d.existing)).toBe(true);
|
||||
});
|
||||
it('reports identical after install', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
const diffs = diffSkill(gbrainRoot, 'alpha', skillsDir);
|
||||
expect(diffs.every(d => d.existing && d.identical)).toBe(true);
|
||||
});
|
||||
it('reports differs after local edit', () => {
|
||||
const { gbrainRoot } = scratchGbrain();
|
||||
const { workspace, skillsDir } = scratchTarget();
|
||||
const opts = {
|
||||
gbrainRoot,
|
||||
targetWorkspace: workspace,
|
||||
targetSkillsDir: skillsDir,
|
||||
skillSlug: 'alpha',
|
||||
};
|
||||
applyInstall(planInstall(opts), opts);
|
||||
writeFileSync(join(skillsDir, 'alpha', 'SKILL.md'), 'edited locally');
|
||||
const diffs = diffSkill(gbrainRoot, 'alpha', skillsDir);
|
||||
expect(diffs.some(d => !d.identical && d.existing)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* test/skillpack-sync-guard.test.ts — F-ENG-4 / D-CX-4.
|
||||
*
|
||||
* Guards against drift between:
|
||||
* - openclaw.plugin.json#skills (what skillpack install ships)
|
||||
* - skills/manifest.json#skills[].path (what the overall skill manifest knows)
|
||||
*
|
||||
* If someone adds a skill directory but forgets the plugin manifest,
|
||||
* or vice versa, this test fails. The sync guard exists because the
|
||||
* codex outside-voice flagged version drift on the plugin manifest.
|
||||
*/
|
||||
|
||||
import { describe, expect, it } from 'bun:test';
|
||||
import { existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
|
||||
const REPO = join(import.meta.dir, '..');
|
||||
|
||||
function readJson(path: string): any {
|
||||
return JSON.parse(readFileSync(path, 'utf-8'));
|
||||
}
|
||||
|
||||
describe('skillpack sync-guard', () => {
|
||||
const pluginPath = join(REPO, 'openclaw.plugin.json');
|
||||
const skillsManifestPath = join(REPO, 'skills', 'manifest.json');
|
||||
|
||||
it('both manifests exist at the expected paths', () => {
|
||||
expect(existsSync(pluginPath)).toBe(true);
|
||||
expect(existsSync(skillsManifestPath)).toBe(true);
|
||||
});
|
||||
|
||||
it('every openclaw.plugin.json skill path exists on disk', () => {
|
||||
const plugin = readJson(pluginPath);
|
||||
for (const skillPath of plugin.skills) {
|
||||
const skillMd = join(REPO, skillPath, 'SKILL.md');
|
||||
expect(existsSync(skillMd)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('every shared_dep in openclaw.plugin.json exists on disk', () => {
|
||||
const plugin = readJson(pluginPath);
|
||||
for (const dep of plugin.shared_deps) {
|
||||
const abs = join(REPO, dep);
|
||||
expect(existsSync(abs)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('openclaw.plugin.json skills ⊂ skills/manifest.json skill paths', () => {
|
||||
// Each entry in the plugin manifest's "skills" list must correspond
|
||||
// to a skill that manifest.json knows about. Installing something
|
||||
// the rest of gbrain doesn't register is a bug.
|
||||
const plugin = readJson(pluginPath);
|
||||
const skillsManifest = readJson(skillsManifestPath);
|
||||
const knownSlugs = new Set(
|
||||
skillsManifest.skills.map((s: { path: string }) =>
|
||||
s.path.replace(/\/SKILL\.md$/, ''),
|
||||
),
|
||||
);
|
||||
for (const skillPath of plugin.skills) {
|
||||
const slug = skillPath.replace(/^skills\//, '');
|
||||
expect(knownSlugs.has(slug)).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it('excluded skills are not listed in plugin.skills (install list is curated)', () => {
|
||||
const plugin = readJson(pluginPath);
|
||||
const excluded = new Set(plugin.excluded_from_install ?? []);
|
||||
for (const skillPath of plugin.skills) {
|
||||
expect(excluded.has(skillPath)).toBe(false);
|
||||
}
|
||||
});
|
||||
|
||||
it('plugin version tracks a real gbrain release line', () => {
|
||||
// Loose check: version must be semver-ish, not the stale 0.4.1
|
||||
// pre-v0.17 placeholder the codex review flagged.
|
||||
const plugin = readJson(pluginPath);
|
||||
const major = parseInt(plugin.version.split('.')[0], 10);
|
||||
const minor = parseInt(plugin.version.split('.')[1], 10);
|
||||
expect(major).toBe(0);
|
||||
expect(minor).toBeGreaterThanOrEqual(17);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user