Files
gbrain/test/check-resolvable-openclaw-compact.test.ts
T
374deff579 v0.41.7.0 feat: compact list-format resolver + 300-skill scaling tutorial (#1407)
* feat(check-resolvable): parseResolverEntries accepts compact list format

Add the second parser branch alongside the existing markdown-table branch
so RESOLVER.md and AGENTS.md can use the OpenClaw-native list shape:

    - **skill-name**: trigger1 | trigger2 | trigger3
    - skill-name: trigger1 | trigger2

Constraints:
  - Skill names must be kebab-lowercase ([a-z][a-z0-9-]+). Bold names
    starting with an uppercase letter (e.g. **Note**, **Convention**)
    are deliberately skipped so prose bullets in real-world AGENTS.md
    files don't get mis-parsed as fake skill rows.
  - skillPath is always derived as skills/<name>/SKILL.md. An optional
    arrow suffix (Unicode -> or ASCII ->) is stripped from the trigger
    string but NOT honored as a path. Downstream consumers
    (routing-eval.ts skillSlugFromPath, the manifest check at line 367)
    assume the convention. For non-conventional paths, use the table
    format.
  - Multiple triggers fan out to one entry per trigger. checkResolvable
    dedupes by skillPath downstream, so the reachability count counts
    each skill once regardless of trigger fan-out.

The parser body is restructured to an if/else-if shape so the existing
'continue' on non-table rows no longer short-circuits the list branch.

Unit tests cover 11 new cases: bold + plain name shapes, multi-trigger
fan-out, Unicode and ASCII path-suffix strip, ellipsis filter, empty
pipe segments, mixed-shape files, section tracking, and two D4
regression cases (prose-bullet rejection + convention-violation
silent-skip).

Closes #1370 — credit @garrytan-agents for the original PR that flagged
the parser gap.

* test(check-resolvable): integration fixtures + regression suite for compact format

Two fixtures pin the v0.41.7.0 parser fix at the integration layer:

  test/fixtures/openclaw-compact-resolver/
    List-format only RESOLVER.md with 10 fictional skills (gift-advisor,
    flight-tracker, email-triage, etc.), each with valid frontmatter
    triggers. A trailing 'Notes' section embeds 4 prose bullets
    (- **Note**:, - **Convention**:, - **TODO**:, - **Important**:)
    that pin the D4 kebab-lowercase regex tighten: if the regex ever
    regresses to permissive [\w-]+, those prose bullets would surface
    as orphan_trigger warnings and the test fails loudly.

  test/fixtures/openclaw-mixed-merge/
    Tests the v0.31.7 D-CX-14 multi-resolver merge: workspace-root
    AGENTS.md (compact list, 3 skills) + skills/RESOLVER.md (table
    format, 5 skills). The merge dedups by skillPath and counts each
    skill once.

The regression test (test/check-resolvable-openclaw-compact.test.ts)
runs 8 assertions across both fixtures:

  1. unreachable === 0 on the compact fixture (the 'pre-v0.41.7.0
     reported 238 FAILs on a 306-skill OpenClaw, post-fix 0' headline).
  2. zero error-severity issues; report.ok === true.
  3. zero mece_gap warnings (every stub ships valid triggers).
  4. zero orphan_trigger warnings for the 4 prose-bullet names — D4
     regex regression guard at integration level.
  5. zero missing_file warnings.
  6. mixed-merge: total_skills === 8 (5 table + 3 list), all reachable.
  7. mixed-merge: errors.length === 0; report.ok === true.
  8. mixed-merge: each expected skill from BOTH shapes is non-unreachable
     (catches the bug where one shape silently swallows the other via
     dedup-by-skillPath).

* docs(guides): scaling-skills.md walkthrough for 300-skill agents

Three-tier architecture for agents that have outgrown the always-loaded
skill manifest:

  Tier A — always loaded (~35 skills, in the system prompt every turn)
  Tier B — resolver-routed (~85 skills, looked up via RESOLVER.md/AGENTS.md
            only when no Tier A match)
  Tier C — dormant (~180 skills, on disk but not injected into the prompt)

Real numbers from Garry's 306-skill OpenClaw: 25K tokens of skill
descriptions per turn collapsed to 4K tokens (~21K tokens freed per
turn) with zero capability loss. The compact list-format resolver
(v0.41.7.0) is the parser-level enabler for this pattern.

The guide covers:

  - The scaling wall (when the always-loaded manifest stops working)
  - The three tiers + per-turn token math
  - What the resolver actually does (routing-table-but-cheaper pattern)
  - The compact list format (kebab-lowercase contract, optional path
    suffix, mixed-shape support)
  - The 'gbrain doctor' / 'gbrain check-resolvable --strict' safety net
  - Implementation walkthrough (audit → tier → disable → resolver →
    doctor)
  - The scaling curve (50 → 100 → 200 → 300 → 1000, no ceiling)

Voice + privacy cleanup applied per CLAUDE.md rules:
  - Wintermute → 'Garry's OpenClaw' / 'your OpenClaw'
  - Unicode em dashes stripped; ASCII '--' preserved in command flags
  - Made-up 'check_resolvable' invocation replaced with real
    'gbrain doctor' and 'gbrain check-resolvable --json'/'--strict'
  - Blog-style 'Previous in this series' footer dropped

Wiring:
  - scripts/llms-config.ts registers the new guide in the curated
    array so 'bun run build:llms' picks it up. docs/UPGRADING_
    DOWNSTREAM_AGENTS.md excluded from the inlined bundle to stay
    under the 600KB FULL_SIZE_BUDGET after adding the new content.
  - docs/tutorials/README.md gains a one-line entry pointing at the
    guide under Related documentation.
  - llms.txt + llms-full.txt regenerated.

* chore: bump version and changelog (v0.41.7.0)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* docs: update CLAUDE.md for v0.41.7.0 compact-format resolver

Annotate the src/core/check-resolvable.ts entry with the v0.41.7.0
parseResolverEntries compact list-format support: kebab-lowercase name
gate (closes the prose-bullet false-positive class), path-suffix strip
contract (skillPath always derived as skills/<name>/SKILL.md so
routing-eval and the manifest check don't drift), multi-trigger fan-out
plus checkResolvable downstream dedupe, the 238 FAILs to 0 OpenClaw
headline, the two integration fixtures pinning the regression, and the
docs/guides/scaling-skills.md pointer for the tutorial context.

Regenerate llms-full.txt to match (CLAUDE.md edit chaser, per the
CLAUDE.md own rule about test/build-llms.test.ts catching drift).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-25 13:58:26 -07:00

141 lines
5.5 KiB
TypeScript

/**
* v0.41.7.0 — Regression suite for compact list-format resolvers.
*
* The bisect anchor for the OpenClaw scaling regression: pre-v0.41.7.0,
* any agent that wrote the compact `- **name**: t1 | t2` shape (instead
* of the markdown table) saw every skill reported as unreachable by
* `gbrain doctor`. The OpenClaw deployment regression was 238 FAIL
* errors → 0 errors after the parser fix.
*
* Two fixtures drive three regression tests:
*
* 1. test/fixtures/openclaw-compact-resolver/ — list-format only,
* ~10 skills with valid frontmatter triggers, plus a prose-bullet
* section that pins the D4 kebab-lowercase regex tighten.
*
* 2. test/fixtures/openclaw-mixed-merge/ — table-format
* skills/RESOLVER.md + parent ../AGENTS.md (compact list). Pins
* the v0.31.7 D-CX-14 multi-resolver merge case.
*/
import { describe, test, expect } from "bun:test";
import { join } from "path";
import { checkResolvable } from "../src/core/check-resolvable.ts";
const COMPACT_FIXTURE = join(
import.meta.dir,
"fixtures",
"openclaw-compact-resolver",
"skills"
);
const MIXED_MERGE_FIXTURE = join(
import.meta.dir,
"fixtures",
"openclaw-mixed-merge",
"skills"
);
describe("v0.41.7.0 — compact list-format resolver (PR #1370 regression)", () => {
const report = checkResolvable(COMPACT_FIXTURE);
test("every skill in the manifest is reachable from the list-format resolver", () => {
// The headline assertion: pre-v0.41.7.0 this was unreachable=N for
// every skill in the fixture. Post-fix, unreachable === 0.
expect(report.summary.unreachable).toBe(0);
expect(report.summary.reachable).toBe(report.summary.total_skills);
expect(report.summary.total_skills).toBeGreaterThanOrEqual(10);
});
test("zero error-severity issues", () => {
// The headline 238 FAILs → 0 outcome. errors only; warnings are
// separately gated below.
if (report.errors.length > 0) {
console.error(
"Unexpected errors:\n",
report.errors.map(e => ` - [${e.type}] ${e.skill}: ${e.message}`).join("\n")
);
}
expect(report.errors.length).toBe(0);
expect(report.ok).toBe(true);
});
test("zero mece_gap warnings (fixture stubs ship valid triggers)", () => {
// D5 fixture upgrade: every SKILL.md stub carries valid frontmatter
// triggers, so the mece_gap detection should stay silent. If this
// assertion ever fires, a fixture file lost its triggers: array.
const gaps = report.warnings.filter(w => w.type === "mece_gap");
if (gaps.length > 0) {
console.error(
"Unexpected mece_gap warnings:\n",
gaps.map(w => ` - ${w.skill}: ${w.message}`).join("\n")
);
}
expect(gaps.length).toBe(0);
});
test("D4 REGRESSION: prose bullets do not surface as orphan triggers", () => {
// The compact RESOLVER.md fixture intentionally embeds 4 prose
// bullets (`- **Note**:`, `- **Convention**:`, `- **TODO**:`,
// `- **Important**:`). The kebab-lowercase regex rejects them
// before they reach the resolver entry stream, so we should NOT
// see orphan_trigger warnings naming any of these.
const proseBulletNames = ["Note", "Convention", "TODO", "Important"];
const orphans = report.warnings.filter(w => w.type === "orphan_trigger");
for (const name of proseBulletNames) {
const hit = orphans.find(w => w.skill === name);
expect(hit, `prose bullet "${name}" should not surface as orphan_trigger`).toBeUndefined();
}
});
test("zero missing_file warnings (every list entry resolves to disk)", () => {
const missing = report.warnings.filter(w => w.type === "missing_file");
if (missing.length > 0) {
console.error(
"Unexpected missing_file warnings:\n",
missing.map(w => ` - ${w.skill}: ${w.message}`).join("\n")
);
}
expect(missing.length).toBe(0);
});
});
describe("v0.41.7.0 — D-CX-14 mixed-merge (table + parent AGENTS.md)", () => {
const report = checkResolvable(MIXED_MERGE_FIXTURE);
test("every skill is reachable across both resolver files", () => {
// 5 skills routed from skills/RESOLVER.md (table format)
// + 3 skills routed from ../AGENTS.md (compact list format)
// = 8 total. All reachable via the v0.31.7 multi-file merge.
expect(report.summary.total_skills).toBe(8);
expect(report.summary.unreachable).toBe(0);
expect(report.summary.reachable).toBe(8);
});
test("zero error-severity issues across the merged resolver set", () => {
if (report.errors.length > 0) {
console.error(
"Unexpected errors:\n",
report.errors.map(e => ` - [${e.type}] ${e.skill}: ${e.message}`).join("\n")
);
}
expect(report.errors.length).toBe(0);
expect(report.ok).toBe(true);
});
test("both table and list shapes contribute skills to the merge", () => {
// Sanity check: if the merge silently dropped one shape's entries,
// we'd see unreachable > 0. This test exists to guard against the
// regression where one shape's parser starts swallowing the other's
// output via the dedup-by-skillPath path.
const expectedTableSkills = ["query", "enrich", "briefing", "migrate", "setup"];
const expectedListSkills = ["adversary-tracking", "civic-intelligence", "book-mirror"];
for (const name of [...expectedTableSkills, ...expectedListSkills]) {
const unreachable = report.errors.find(
e => e.type === "unreachable" && e.skill === name
);
expect(unreachable, `${name} should be reachable`).toBeUndefined();
}
});
});