Files
gbrain/test/skill-trigger-index.test.ts
T
32f8be96c2 v0.41.14.0 fix(#1451): close RESOLVER.md drift bug class structurally (#1458)
* feat(core): loadSkillTriggerIndex shared primitive (closes #1451 drift class)

Single loader that unions per-skill SKILL.md frontmatter triggers: with
curated RESOLVER.md / AGENTS.md rows. UNION semantics — explicit
RESOLVER.md rows ADD to frontmatter triggers for the same skill (don't
replace). Dedup keyed on (skillPath, normalized trigger string) so case
or whitespace drift between the two surfaces collapses to one entry.

This is the structural foundation for #1451: pre-fix, gbrain skills
declared triggers in two places (per-skill frontmatter and a curated
RESOLVER.md table) that could silently drift. Three consumers
(checkResolvable, routing-eval CLI, mounts-cache.composeResolvers) each
built their own resolver index from RESOLVER.md only, so fixing
frontmatter would have closed doctor's warning without closing the
other two surfaces. This primitive becomes the single join point for
all three; consumers are wired in the next commit.

Tests: 18 hermetic cases pinning frontmatter auto-registration,
RESOLVER.md/AGENTS.md merge, case-insensitive dedupe, OpenClaw
workspace-root layout (../AGENTS.md), graceful skip of conventions /
deprecated skills / non-directory entries / missing skillsDir, plus
synthesis round-trip and findPrimaryResolverPath.

Plan: ~/.claude/plans/system-instruction-you-are-working-tidy-storm.md

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor: wire 3 consumers through loadSkillTriggerIndex (#1451)

Replace the three independent resolver-content loaders with calls to
the v0.41.11 shared primitive so frontmatter triggers propagate to
every dispatch surface, not just doctor.

Before: checkResolvable, runRoutingEvalCli, and mounts-cache each
walked RESOLVER.md / AGENTS.md files separately. Adding frontmatter
triggers to one consumer (e.g. checkResolvable) wouldn't have reached
the routing-eval CLI or cross-brain composed dispatchers — the same
drift bug class as #1451 in cross-consumer form. Codex caught this in
plan-eng-review.

After: all three consumers fold through loadSkillTriggerIndex. UNION
semantics across both surfaces means new skills with frontmatter
triggers are reachable everywhere without editing RESOLVER.md.

Also updates:
- check-resolvable action text on routing_miss to point at the
  canonical surface (SKILL.md frontmatter triggers) first, with
  RESOLVER.md row as secondary.
- test/resolver-merge.test.ts to test BOTH the legacy
  RESOLVER.md-only authority path (skills with no frontmatter
  triggers) AND the new auto-registration path (skills reachable via
  frontmatter alone, no RESOLVER.md needed).
- 3 routing-eval.jsonl fixtures (voice-note-ingest, brain-taxonomist,
  strategic-reading) gain `ambiguous_with` declarations for skill
  overlaps that auto-registration newly exposes. These overlaps are
  legitimate (voice-note vs idea-ingest on audio notes,
  brain-taxonomist vs repo-architecture on filing, strategic-reading
  vs idea-ingest on reading-through-a-lens) — the agent picks based
  on context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(#1451): broaden skillpack-harvest triggers + negative fixtures + tighten gate

Closes the 7 residual routing_miss warnings on skillpack-harvest that
gbrain doctor reported on every fresh install (resolver_health: WARN,
~5 health-score points).

Three changes:

1. Broaden skills/skillpack-harvest/SKILL.md frontmatter triggers from
   5 narrow to 10 realistic phrasings. Each new trigger is a
   contiguous substring of one of the 7 shipped routing-eval.jsonl
   intents (per kylma-code's design in PR #1331; moved from
   RESOLVER.md to frontmatter under the v0.41.11 frontmatter-
   authoritative contract). Existing RESOLVER.md row stays for
   human-readability of the dispatcher map.

2. Add 4 negative-fixture cases to skills/skillpack-harvest/
   routing-eval.jsonl with expected_skill=null to defend against
   false positives the broader triggers might introduce
   ("publish this report to the team", "promote my role on
   LinkedIn", "bundle these screenshots into a deck", "lift weights
   at the gym"). Two candidate negatives ("save this report as PDF",
   "share this article with the channel") were excluded — they trip
   idea-ingest's existing "save this"/"share" triggers, a real
   overlap but a separate v0.42+ concern.

3. Tighten test/check-resolvable.test.ts's "repo skills/ pass cleanly"
   assertion: the v0.25.1 carve-out that allowed routing_miss as
   informational is removed. The contract is back to zero errors AND
   zero warnings — the CI gate (next commit) enforces this for PRs
   so future drift fails the build instead of degrading user-install
   resolver_health silently.

Co-Authored-By: kylma-code <noreply@github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(cli): register reindex in CLI_ONLY so --help works (closes part of #1354)

Pre-fix: src/cli.ts had a `case 'reindex':` handler at line 1334 that
dispatched to reindex-multimodal or reindex.ts based on flags, but
'reindex' was missing from the CLI_ONLY Set at line 38. The dispatcher
rejected the command with "Unknown command: reindex" before the handler
ever ran.

Post-fix: 'reindex' is in CLI_ONLY (recognized as a registered command).
NOT added to CLI_ONLY_SELF_HELP — the handler doesn't have its own
--help branch, so the dispatcher's generic printCliOnlyHelp() shows
"gbrain reindex - run gbrain --help for the full command list."
Polishing this to per-flag help text (--multimodal, --markdown, --code)
is a follow-up TODO.

Regression test in test/cli.test.ts asserts `'reindex'` is in the
CLI_ONLY Set source string. Mirrors the existing pattern for
'reinit-pglite' in test/v0_37_fix_wave.serial.test.ts:284 and
'book-mirror' in test/book-mirror.test.ts:73.

Cherry-picked from lost9999's PR #1354 (which bundled this fix with
their fixture-rewrite approach to #1451 — the routing-eval half of
that PR was superseded by kylma-code's trigger-broadening direction
in #1331, which we took structurally in the previous commits).

Co-Authored-By: lost9999 <noreply@github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(ci): wire check:resolver into bun run verify

Adds `bun run check:resolver` (= `bun src/cli.ts check-resolvable
--strict --skills-dir skills/`) to package.json scripts and registers
it in scripts/run-verify-parallel.sh's CHECKS array.

This gates PR CI on resolver health: any future drift between a
skill's frontmatter triggers and its routing-eval.jsonl fixtures
fails the build, instead of silently degrading the resolver_health
score on user installs after merge. The --strict flag exits non-zero
on warnings (not just errors), so routing_miss / routing_ambiguous /
routing_false_positive all block.

Closes the CI half of #1451's structural fix: doctor catches drift
at runtime, this gate catches drift at PR time.

Local pre-flight: `bun run check:resolver`.

Codex finding #9 from plan review: scripts/run-verify-parallel.sh
invokes entries as `bun run <script-name>`, not raw shell. The
package.json script name + CHECKS-array entry is the correct shape.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): update CLI unreachable test for v0.41.11 contract change

The prior fixture used `triggers: ['alpha']` + `inResolver: false` to
simulate an unreachable skill. Under v0.41.11's structural fix,
frontmatter triggers auto-register the skill independently of
RESOLVER.md, so this skill is reachable now — the assertion
`errors.length > 0` failed.

Drop the `triggers:` array from the fixture so the skill is genuinely
unreachable (neither frontmatter nor RESOLVER.md row), preserving the
test's regression-guard intent: doctor/check-resolvable still exits 1
when a manifest skill is truly unreachable.

Caught by the full unit test suite after the merge.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: CLAUDE.md Key Files note for skill-trigger-index + regen llms.txt

Document the v0.41.11 shared primitive (loadSkillTriggerIndex) in
the Key Files section so future contributors find it before they
reach for parseResolverEntries directly. Notes the 3 consumers
(checkResolvable, runRoutingEvalCli, mounts-cache.composeResolvers),
the UNION semantics, skip rules, parseSkillFrontmatter dependency,
test coverage, and the CI gate wiring.

Regenerated llms.txt + llms-full.txt per the CLAUDE.md edit rule.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.41.14.0 fix(#1451): close RESOLVER.md drift bug class structurally

Frontmatter triggers + RESOLVER.md / AGENTS.md rows now union into one
unified index via the new loadSkillTriggerIndex primitive, consumed by
all three dispatch surfaces (checkResolvable, routing-eval CLI,
mounts-cache.composeResolvers). Closes the 7 residual routing_miss
warnings #1451 reported on every fresh install, and the drift bug class
that produced them.

Highlights:
- New shared primitive src/core/skill-trigger-index.ts (252 lines + 361
  lines of tests across 18 cases). UNION semantics, case-insensitive
  dedupe keyed on (skillPath, normalized trigger).
- Three consumers wired through the primitive — fixing frontmatter
  triggers for doctor now also fixes routing-eval CLI and
  cross-brain mounted dispatch (codex outside-voice catch).
- skillpack-harvest frontmatter broadened from 5 to 10 triggers per
  kylma-code's design in #1331, plus 4 negative-fixture cases for
  false-positive defense.
- reindex CLI added to CLI_ONLY set so `gbrain reindex --help` works
  instead of "Unknown command: reindex" (lost9999's #1354 hunk).
- check:resolver wired into bun run verify CI gate so future drift
  fails PR CI instead of silently degrading user-install
  resolver_health.
- check-resolvable's repo skills/ test tightened from "warn-tolerant"
  to "zero errors AND zero warnings" — the carve-out was a stop-gap
  pre-structural-fix.

Plan + 5 decisions + codex outside-voice recalibration captured at
~/.claude/plans/system-instruction-you-are-working-tidy-storm.md.

Co-Authored-By: kylma-code <noreply@github.com>
Co-Authored-By: lost9999 <noreply@github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: update project documentation for v0.41.14.0

- CLAUDE.md: tag skill-trigger-index entry with correct shipped version
  (v0.41.14.0, closes #1451) instead of the stale v0.41.11 draft tag.
- CONTRIBUTING.md: list the new `check:resolver` gate in the `bun run
  verify` chain so contributors know to expect resolver-drift failures
  in PR CI.
- llms-full.txt: regenerated from updated CLAUDE.md (mandatory per
  CLAUDE.md's auto-derived files rule; CI shard 1 fails the build
  otherwise).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: kylma-code <noreply@github.com>
2026-05-25 20:00:03 -07:00

362 lines
13 KiB
TypeScript

/**
* Unit suite for `src/core/skill-trigger-index.ts` — the v0.41.11
* shared primitive that unifies frontmatter-declared triggers with
* curated RESOLVER.md / AGENTS.md rows.
*
* Hermetic: every case builds a fresh tempdir skills tree and runs the
* loader against it. No PGLite, no env var mutation (R1/R2 from
* CLAUDE.md test-isolation rules), no mock.module.
*/
import { afterAll, beforeEach, describe, expect, test } from 'bun:test';
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
import { tmpdir } from 'os';
import { join } from 'path';
import {
FRONTMATTER_SECTION,
_resetWarnedSkillsForTests,
entriesToResolverContent,
findPrimaryResolverPath,
loadSkillTriggerIndex,
} from '../src/core/skill-trigger-index.ts';
// ---------------------------------------------------------------------------
// Helpers
// ---------------------------------------------------------------------------
const TEMPDIRS: string[] = [];
function makeSkillsDir(): string {
const dir = mkdtempSync(join(tmpdir(), 'skill-trigger-index-'));
const skillsDir = join(dir, 'skills');
mkdirSync(skillsDir, { recursive: true });
TEMPDIRS.push(dir);
return skillsDir;
}
function writeSkill(skillsDir: string, name: string, content: string): void {
const dir = join(skillsDir, name);
mkdirSync(dir, { recursive: true });
writeFileSync(join(dir, 'SKILL.md'), content);
}
function skillWithTriggers(name: string, triggers: string[]): string {
const triggerLines = triggers.map(t => ` - "${t}"`).join('\n');
return `---
name: ${name}
description: Test skill ${name}.
triggers:
${triggerLines}
---
# ${name}
Test body.
`;
}
afterAll(() => {
for (const d of TEMPDIRS) {
try {
rmSync(d, { recursive: true, force: true });
} catch {
// best-effort cleanup
}
}
});
beforeEach(() => {
_resetWarnedSkillsForTests();
});
// ---------------------------------------------------------------------------
// Frontmatter auto-registration
// ---------------------------------------------------------------------------
describe('loadSkillTriggerIndex — frontmatter auto-registration', () => {
test('skill with block-form triggers, no RESOLVER.md → frontmatter entries appear', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what do we know', 'who is']));
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(2);
expect(entries[0]).toEqual({
trigger: 'what do we know',
skillPath: 'skills/query/SKILL.md',
isGStack: false,
section: FRONTMATTER_SECTION,
source: 'frontmatter',
});
expect(entries[1].trigger).toBe('who is');
expect(entries[1].source).toBe('frontmatter');
});
test('skill with inline-form triggers (triggers: ["a", "b"]) parses the same', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'lint', `---
name: lint
triggers: ["lint this", "audit pages"]
---
body
`);
const entries = loadSkillTriggerIndex(skillsDir);
const triggers = entries.map(e => e.trigger).sort();
expect(triggers).toEqual(['audit pages', 'lint this']);
expect(entries.every(e => e.source === 'frontmatter')).toBe(true);
});
test('skill with no triggers: field → not registered, no error', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'install', '# Deprecated skill — replaced by setup.\n\nNo frontmatter.\n');
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toEqual([]);
});
test('skill with empty triggers: array → not registered', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'empty', `---
name: empty
triggers: []
---
body
`);
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toEqual([]);
});
test('directory without SKILL.md → silently skipped', () => {
const skillsDir = makeSkillsDir();
// `install/` exists in the bundled tree but has no SKILL.md.
mkdirSync(join(skillsDir, 'install'), { recursive: true });
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is']));
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(1);
expect(entries[0].trigger).toBe('what is');
});
test('_brain-filing-rules.md and conventions/ subdirs are ignored', () => {
const skillsDir = makeSkillsDir();
// Underscore-prefixed file at the top level (not a skill).
writeFileSync(join(skillsDir, '_brain-filing-rules.md'), '# Rules\n');
// Underscore-prefixed directory.
mkdirSync(join(skillsDir, '_conventions-private'), { recursive: true });
writeFileSync(
join(skillsDir, '_conventions-private', 'SKILL.md'),
skillWithTriggers('_priv', ['should not appear']),
);
// conventions/ subtree.
mkdirSync(join(skillsDir, 'conventions'), { recursive: true });
writeFileSync(
join(skillsDir, 'conventions', 'SKILL.md'),
skillWithTriggers('conv', ['should not appear either']),
);
// One real skill that SHOULD register.
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is']));
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(1);
expect(entries[0].trigger).toBe('what is');
});
test('non-directory entries in skillsDir → silently skipped', () => {
const skillsDir = makeSkillsDir();
writeFileSync(join(skillsDir, 'README.md'), '# skills/ index\n');
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is']));
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(1);
});
test('skillsDir does not exist → empty array, no throw', () => {
const entries = loadSkillTriggerIndex('/tmp/does-not-exist-' + Date.now());
expect(entries).toEqual([]);
});
});
// ---------------------------------------------------------------------------
// RESOLVER.md merge
// ---------------------------------------------------------------------------
describe('loadSkillTriggerIndex — RESOLVER.md merge (UNION semantics)', () => {
test('skill with frontmatter AND RESOLVER.md row → both contribute; duplicate triggers collapse', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is', 'who is']));
writeFileSync(
join(skillsDir, 'RESOLVER.md'),
`# Test resolver
## Brain operations
| trigger | skill |
| --- | --- |
| what is | \`skills/query/SKILL.md\` |
| tell me about | \`skills/query/SKILL.md\` |
`,
);
const entries = loadSkillTriggerIndex(skillsDir);
const triggers = entries.map(e => e.trigger).sort();
// 'what is' present in BOTH surfaces → one entry (dedup keeps
// frontmatter source since it's loaded first). 'who is' from
// frontmatter only. 'tell me about' from RESOLVER.md only.
expect(triggers).toEqual(['tell me about', 'what is', 'who is']);
const whatIs = entries.find(e => e.trigger === 'what is')!;
expect(whatIs.source).toBe('frontmatter'); // first occurrence wins
const tellMe = entries.find(e => e.trigger === 'tell me about')!;
expect(tellMe.source).toBe('resolver_md');
expect(tellMe.section).toBe('Brain operations');
const whoIs = entries.find(e => e.trigger === 'who is')!;
expect(whoIs.source).toBe('frontmatter');
});
test('case-insensitive dedupe: "What Is" in frontmatter + "what is" in RESOLVER.md → one entry', () => {
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['What Is']));
writeFileSync(
join(skillsDir, 'RESOLVER.md'),
`## Brain operations
| trigger | skill |
| --- | --- |
| what is | \`skills/query/SKILL.md\` |
`,
);
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(1);
expect(entries[0].trigger).toBe('What Is'); // first-occurrence-wins preserves original casing
expect(entries[0].source).toBe('frontmatter');
});
test('AGENTS.md at workspace root (OpenClaw layout) is merged via parent-dir scan', () => {
const skillsDir = makeSkillsDir();
// Put AGENTS.md ONE LEVEL UP from skillsDir (parent of `skills/`).
const workspaceRoot = join(skillsDir, '..');
writeFileSync(
join(workspaceRoot, 'AGENTS.md'),
`## Operational
| trigger | skill |
| --- | --- |
| openclaw dispatch | \`skills/query/SKILL.md\` |
`,
);
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is']));
const entries = loadSkillTriggerIndex(skillsDir);
const triggers = entries.map(e => e.trigger).sort();
expect(triggers).toEqual(['openclaw dispatch', 'what is']);
});
test('skill with RESOLVER.md row but NO frontmatter triggers → still registered from RESOLVER.md', () => {
const skillsDir = makeSkillsDir();
// install/ is the canonical deprecated-skill shape (no frontmatter).
mkdirSync(join(skillsDir, 'install'), { recursive: true });
writeFileSync(join(skillsDir, 'install', 'SKILL.md'), '# Install (Deprecated)\n');
writeFileSync(
join(skillsDir, 'RESOLVER.md'),
`## Setup
| trigger | skill |
| --- | --- |
| install gbrain | \`skills/install/SKILL.md\` |
`,
);
const entries = loadSkillTriggerIndex(skillsDir);
expect(entries).toHaveLength(1);
expect(entries[0].source).toBe('resolver_md');
expect(entries[0].trigger).toBe('install gbrain');
});
});
// ---------------------------------------------------------------------------
// entriesToResolverContent (synthesis for runRoutingEval compat)
// ---------------------------------------------------------------------------
describe('entriesToResolverContent', () => {
test('synthesized markdown is re-parseable by parseResolverEntries', async () => {
const { parseResolverEntries } = await import('../src/core/check-resolvable.ts');
const skillsDir = makeSkillsDir();
writeSkill(skillsDir, 'query', skillWithTriggers('query', ['what is', 'tell me about']));
const entries = loadSkillTriggerIndex(skillsDir);
const synthesized = entriesToResolverContent(entries);
const reparsed = parseResolverEntries(synthesized);
// Same skill paths and trigger strings (modulo source field).
expect(reparsed.map(e => e.trigger).sort()).toEqual(['tell me about', 'what is']);
expect(reparsed.every(e => e.skillPath === 'skills/query/SKILL.md')).toBe(true);
});
test('pipes inside trigger strings are escaped in synthesis (defensive)', () => {
// Defense-in-depth: the synthesizer escapes `|` so it can't break
// the markdown table row. parseResolverEntries does not currently
// unescape (its pipe-split doesn't honor backslashes), so a
// pipe-bearing trigger is lost on re-parse — but it ALSO doesn't
// corrupt neighboring rows, which is the actual risk we're hedging
// against. Real-world triggers don't contain `|` (it'd be unusual
// natural language). Filed as v0.42+ TODO if we ever start
// declaring pipe-bearing triggers.
const synthesized = entriesToResolverContent([
{
trigger: 'option a | option b',
skillPath: 'skills/x/SKILL.md',
isGStack: false,
section: FRONTMATTER_SECTION,
source: 'frontmatter',
},
]);
expect(synthesized).toContain('option a \\| option b');
});
test('GStack/external entries re-emit with prose skillPath (no backticks)', async () => {
const { parseResolverEntries } = await import('../src/core/check-resolvable.ts');
const synthesized = entriesToResolverContent([
{
trigger: 'review this plan',
skillPath: 'GStack: ceo-review',
isGStack: true,
section: 'Thinking',
source: 'resolver_md',
},
]);
const reparsed = parseResolverEntries(synthesized);
expect(reparsed).toHaveLength(1);
expect(reparsed[0].isGStack).toBe(true);
expect(reparsed[0].skillPath).toBe('GStack: ceo-review');
});
});
// ---------------------------------------------------------------------------
// findPrimaryResolverPath
// ---------------------------------------------------------------------------
describe('findPrimaryResolverPath', () => {
test('returns RESOLVER.md path when present in skillsDir', () => {
const skillsDir = makeSkillsDir();
const resolverPath = join(skillsDir, 'RESOLVER.md');
writeFileSync(resolverPath, '# resolver\n');
expect(findPrimaryResolverPath(skillsDir)).toBe(resolverPath);
});
test('returns AGENTS.md path when only that exists', () => {
const skillsDir = makeSkillsDir();
const agentsPath = join(skillsDir, 'AGENTS.md');
writeFileSync(agentsPath, '# agents\n');
expect(findPrimaryResolverPath(skillsDir)).toBe(agentsPath);
});
test('returns null when neither exists in skillsDir or parent', () => {
const skillsDir = makeSkillsDir();
expect(findPrimaryResolverPath(skillsDir)).toBeNull();
});
});