mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-31 04:07:52 +00:00
c2ae4dbfc58d826c4982139d4b685399c0fdd68e
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
c2ae4dbfc5 |
v0.25.1 feat: book-mirror flagship + 8 research skills + skillpack uninstall + post-install advisory (#566)
* v0.25.1 foundation: scaffolds + manifests + filing-doctrine update
Foundation commit for v0.25.1 skills wave (book-mirror flagship + 8 research
pairings). All content is scaffold-stage; subsequent commits port wintermute
SKILL.md content into pure gbrain idiom.
Version bumps:
- VERSION 0.24.0 -> 0.25.1
- package.json: version + engines.bun >= 1.3.10 (D14 PTY harness)
- openclaw.plugin.json inner version 0.19.0 -> 0.25.1
- bun.lock refreshed
9 skill scaffolds via `gbrain skillify scaffold` (frontmatter + RESOLVER row +
routing-eval seed): book-mirror, article-enrichment, strategic-reading,
concept-synthesis, perplexity-research, archive-crawler, academic-verify,
brain-pdf, voice-note-ingest. Stub .mjs scripts and stub .test.ts files
deleted; these are pure-markdown skills, not deterministic-script skills.
Real tests will return when src/commands/book-mirror.ts and the other
runtime pieces land.
skills/manifest.json + openclaw.plugin.json skills[]: 9 new entries
(codex T6 fix; required by test/skillpack-sync-guard.test.ts).
D13 filing-doctrine update:
- skills/_brain-filing-rules.md: carve out media/<format>/<slug> as a
sanctioned exception for sui-generis synthesized output.
- skills/_brain-filing-rules.json: add media/books/ and media/articles/
as `synthesis-output` kind, distinct from raw-ingest filing.
- skills/media-ingest/SKILL.md: refine anti-pattern callout to clarify
that format-prefixed paths are anti-pattern for raw ingest only,
sanctioned for one-of-one synthesis.
Privacy guard hardening (codex T7):
- scripts/check-privacy.sh: extended for /data/brain/ and
/data/.openclaw/ wintermute-specific path patterns. 7 historical
files allow-listed (frozen migrations, test fixtures, env-var
fallbacks). PRIVACY OK passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 book-mirror: trusted CLI with read-only subagent fan-out
Implements `gbrain book-mirror` per the locked v0.25.1 plan (D2/α + codex
HIGH-1 fix). Closes the prompt-injection vector codex flagged on the
earlier `allowedSlugPrefixes: ['media/books/*', 'people/*']` design by
narrowing the trust contract at the tool-allowlist layer instead.
Trust contract:
- Each chapter is analyzed by a separate subagent with allowed_tools
restricted to ['get_page', 'search'] — read-only. Subagents cannot
call put_page or any mutating op. Untrusted EPUB/PDF content cannot
prompt-inject any people/* page because subagents lack write access
entirely.
- Subagents return markdown analysis text via final_message
(SubagentResult.result). The CLI reads each child's job.result and
assembles the final two-column page itself.
- The CLI calls put_page once at the end with operator-level trust
(no viaSubagent flag, no allowedSlugPrefixes). Operator can write
anywhere; the namespace check doesn't fire for direct CLI calls.
Architecture:
- `--chapters-dir` is the input contract. The skill (which has shell +
python access) handles EPUB/PDF extraction; the CLI takes pre-extracted
.txt files. Separation of concerns: skill prepares inputs, CLI is the
trusted runtime.
- Cost-estimate prompt before launching: ~$0.30/chapter × N at Opus,
~$0.06/chapter at Sonnet. Refuses to spend in non-TTY without --yes.
- Idempotency keys on each child: `book-mirror:<slug>:ch-<N>`. Re-running
on same input dedups against the queue; failed chapters retry.
- Partial-failure handling: assembled page renders with completed
chapters and a `## Failed chapters` section listing retries needed.
Exit 1 on any failure; exit 0 only on full success.
- 30-min default per-child timeout (override with --timeout-ms).
CLI wiring:
- `book-mirror` added to CLI_ONLY set in src/cli.ts.
- Lazy-imports src/commands/book-mirror.ts to keep cold-start fast.
Out of scope for this commit (filed for v0.25.1 follow-ons):
- skills/book-mirror/SKILL.md content port (replaces the foundation
scaffold stub).
- test/book-mirror.test.ts (will test arg parsing, validation, mock
fan-out, cost-estimate gating, partial-failure assembly).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 book-mirror: port SKILL.md content + routing-eval
Replaces the foundation scaffold stub with the full ported book-mirror
SKILL.md, pointing the agent at the new `gbrain book-mirror` CLI as the
trusted runtime.
skills/book-mirror/SKILL.md:
- Drops wintermute_only frontmatter; uses gbrain frontmatter shape
(mutating + writes_pages + writes_to: media/books/).
- Documents the trust contract: subagents are read-only, the CLI does
the put_page write itself with operator trust. Closes the codex
HIGH-1 prompt-injection vector at the tool-allowlist layer.
- Replaces /data/brain/ absolute paths with $BRAIN_DIR resolution from
gbrain config.
- Replaces brain-commit-link.sh / direct shell-script writes with the
CLI's single put_page call.
- Documents EPUB/PDF extraction via the agent's shell + python access
(BeautifulSoup4 for EPUB, pdftotext for PDF). The skill prepares
inputs; the CLI is the trusted runtime.
- Privacy scrub clean — no real names, no /data/brain/, no .openclaw/,
no Wintermute literals.
skills/book-mirror/routing-eval.jsonl:
- 5 paraphrased intents per D-CX-6 rule (intent paraphrases the
trigger, doesn't copy it).
- 3 adversarial intents that pattern-match media-ingest's "process
this book" trigger (IRON RULE regression test for the
media-ingest <-> book-mirror routing conflict flagged in R1+R2).
These assert that book-mirror should NOT win on generic ingest
phrasing.
skills/_brain-filing-rules.json: 4 new directory kinds added so
check-resolvable's filing audit passes for the new skills' writes_to
declarations:
- idea (ideas/) — generative ideas to act on later (voice-note-ingest,
archive-crawler).
- research (research/) — web-research deltas, citation-checked claims
(perplexity-research, academic-verify).
- original (originals/) — user-authored thinking the user originated
(voice-note-ingest, archive-crawler, signal-detector).
- voice-note (voice-notes/) — random-thought audio capture pages
(voice-note-ingest).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 ports: article-enrichment + strategic-reading + voice-note-ingest
Replaces SKILLIFY_STUB scaffolds with content-ported SKILL.md files in
pure gbrain idiom:
skills/article-enrichment/SKILL.md:
- Drops wintermute-specific scripts/enrich-article.mjs reference; the
skill is markdown agent instructions, not a deterministic script
pipeline.
- Replaces /data/brain/ paths with relative brain-dir paths.
- Documents the structured output contract (Executive Summary,
Quotable Lines verbatim, Key Insights, Why It Matters, See Also,
details-block source preservation).
- Sonnet by default, Opus for high-value content.
skills/strategic-reading/SKILL.md:
- Generic problem-lens reading flow (book/article/case study x specific
strategic problem -> applied playbook with do/avoid/watch-for).
- Drops Garry-specific oppo example ("Tyler Law/Han Zou gatekeeper
fight"); uses generic "gatekeeper-vs-incumbent fight" framing.
- Files to projects/<slug>/playbook.md (problem-tied) or
concepts/<slug>.md (general strategy) per primary-subject filing rule.
- Cross-references book-mirror as the whole-life-personalization
counterpart.
skills/voice-note-ingest/SKILL.md:
- Iron Law: exact phrasing preserved, never paraphrased. Block-quoted
transcript is sacred; analysis is interpretive.
- 7-step decision tree (originals -> concepts -> people -> companies
-> ideas -> personal -> voice-notes catch-all) per
_brain-filing-rules.md.
- Replaces wintermute's brain-commit-link.sh + Supabase Storage helper
with gbrain transcription + storage interface (pluggable per
src/core/storage.ts).
Each skill ships routing-eval.jsonl with 5 paraphrased intents per
D-CX-6 (intent paraphrases trigger, doesn't copy it). The literal
"please <trigger> for me now" stubs from gbrain skillify scaffold are
replaced with realistic user phrasings.
Privacy scrub clean — no real names, no /data/brain/, no .openclaw/,
no Wintermute literals.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 ports: concept-synthesis + perplexity-research + brain-pdf
Replaces SKILLIFY_STUB scaffolds with content-ported SKILL.md files in
pure gbrain idiom:
skills/concept-synthesis/SKILL.md:
- 4-phase pipeline: dedup -> tier (T1 Canon to T4 Riff) -> synthesize
T1/T2 -> cluster + intellectual map.
- Generic across any concept-stub source (signal-detector,
voice-note-ingest, idea-ingest, archive-crawler).
- Drops wintermute-specific X-pipeline framing (9051 stubs from x-deep-enrich,
scripts/x-concept-compiler.mjs); skill is markdown agent instructions
using gbrain query + put_page.
- Output format: T1 gets full synthesis with evolution table + best
articulation + related-concepts cross-links; T3/T4 stay as stubs.
- Cluster map at concepts/README.md as the master intellectual fingerprint.
skills/perplexity-research/SKILL.md:
- Brain-augmented web research: sends brain context as part of the
Perplexity prompt so the search focuses on what's NEW vs already-known.
- Output structure: Executive Summary + Key New Developments + Confirming
Signals + Contradictions or Updates + Recommended Brain Updates +
Citations.
- Uses Perplexity sonar-pro by default (~$0.04/query); sonar for bulk.
- Drops wintermute-specific scripts/perplexity-research.mjs and
/data/.env path; documents PERPLEXITY_API_KEY in agent env.
- Cross-references academic-verify (which wraps this skill for
citation-checked claim verification per D7/alpha) and enrich (entity
enrichment loop).
skills/brain-pdf/SKILL.md:
- Documents gstack make-pdf as soft prereq with absent-binary detection.
- 4-step workflow: resolve -> strip frontmatter -> render -> deliver.
- Defaults: NO --cover, NO --toc (look corporate and waste space).
- Mandatory CONTAINER=1 for Playwright sandboxing.
- Anti-pattern callout: never use raw MEDIA: tags for Telegram delivery
(they fail silently); use message tool with filePath= attachment.
Each ships routing-eval.jsonl with 5 paraphrased intents per D-CX-6.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 ports: archive-crawler + academic-verify (final SKILL.md batch)
Replaces the last two SKILLIFY_STUB scaffolds. All 9 new skills now
have ported content; `gbrain check-resolvable` reports zero
skillify_stub_unreplaced warnings.
skills/archive-crawler/SKILL.md (D3 + D12):
- Hard safety gate: refuses to run unless `archive-crawler.scan_paths:`
is set in gbrain.yml. Closes the codex HIGH-4 footgun where 'trust
the prompt' was not a control.
- Schema-generic port (D3 user constraint): no hardcoded era folders
(no archive/, post-stanford/, posterous-era/, initialized-era/,
yc-era/). Reads filing rules from _brain-filing-rules.json at
runtime; agent decides per-page filing within sanctioned dirs.
- Drops wintermute-specific scripts and brain-commit-link.sh; uses
gbrain operations for inventory + put_page for ingest.
- File-type handlers preserved (.mbox, .doc/.docx, .pst, .zip, images)
with the exact same shell + python recipes.
- Manifest tracks per-item triage status + exact user reactions per
conventions/quality.md exact-phrasing rule.
skills/academic-verify/SKILL.md (D4 + D7/alpha):
- Drops ALL the wintermute-specific oppo / adversarial framing: no
Goff/Solomon, no CPE, no '48 Hills', no fabrication-detection,
no 'oppo research where the target relies on academic credentials'.
This is the public skillpack — research-not-adversarial bar.
- Pure-routing implementation per D7/alpha: skill is a thin
orchestrator that scopes the claim, invokes
perplexity-research with citation-mode prompt, and formats results
as a verdict-shaped brain page. Zero new infrastructure.
- 5 verdict states (verified / partial / unverifiable / misattributed
/ retracted) replace the 'fabrication suspected' / 'methodologically
flawed' classifications that read like takedown rubric.
- Documents Retraction Watch / PubPeer / OSF / Semantic Scholar /
OpenAlex / Many Labs as the databases the agent uses via
perplexity-research, but doesn't ship its own API integrations.
Each ports a routing-eval.jsonl with 5 paraphrased intents per D-CX-6.
Privacy scrub clean. typecheck OK. Remaining check-resolvable warnings
are routing_miss on the substring matcher (paraphrased intents don't
exact-match the RESOLVER triggers); the LLM tie-break layer is a
v0.26+ enhancement per CLAUDE.md routing-eval section. Warnings are
advisory, not errors.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 drift backports: citation-fixer + testing + cross-modal-review
Pulls the wintermute drift improvements identified by R1's quick audit
into the public skillpack, in pure gbrain idiom (no real names, no
/data/brain/ paths, no Wintermute literals — privacy guard passes).
skills/citation-fixer/SKILL.md (PORT, version 1.0 -> 1.1):
- Adds tweet/post URL resolution: scans pages for broken tweet
references (no x.com URL) and resolves them via the host's X API
integration.
- 5-step pipeline: identify broken refs -> extract searchable content
(handle/quote/date) -> X API search -> verify + extract metadata
-> patch the page with deterministic URL.
- Batch-mode pattern with priority order (recently changed pages
first), rate-limit guidance (~50 pages/run), batch-commit cadence.
- Integration callout: enrich + media-ingest can call
citation-fixer pre-commit to validate output.
- Anti-pattern: never compose tweet URLs by guessing the id;
deterministic links only (per _output-rules.md).
skills/testing/SKILL.md (PORT, version 1.0 -> 1.1):
- Splits into TWO modes: skill conformance validation (original 1.0
scope) AND project test-suite health (v0.25.1 extension).
- Test tiers: unit (<2s, every commit), evals (~60s, daily),
integration (~5m, pre-ship + nightly), system health (<10s).
- Daily run protocol: unit -> evals -> system -> git diff analysis
for regression intelligence.
- Failure classification: REGRESSION / STALE / FLAKE / NEW / INFRA
with markers (red / yellow / warning / green / wrench).
- Auto-fix protocol: explicit DO and DO NOT lists. Security-test
failures always escalate, never auto-fix.
- State tracking at ~/.gbrain/test-state.json for trend analysis,
flake detection, regression velocity.
skills/cross-modal-review/SKILL.md (PORT, version 1.0 -> 1.1):
- Adds explicit "When to invoke" gating (significant code changes 5+
files / 100+ lines, security-sensitive, architecture, churning,
pre-bulk, skill creation, brain-page quality) vs DO NOT invoke
(simple memory writes, typo fixes, routine cron, post-review
commits).
- Adds code-review handoff section: knows WHEN to recommend gstack's
/codex review (independent diff review from a different AI) and how
to frame the cross-model output.
- Adversarial Challenge sub-mode: red-team prompt for security-
sensitive changes; output adds exploitability rating
(CRITICAL/HIGH/MEDIUM/LOW) + mitigations.
- Iron Law: user-sovereignty rule explicitly captured. Reviewer
findings are informational until the user explicitly approves;
cross-model consensus is signal, not permission.
All three pass scripts/check-privacy.sh (no Wintermute literals, no
/data/brain/, no /data/.openclaw/). typecheck OK.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 skillpack uninstall: D6 + D8 + D11 content-hash guard
Implements `gbrain skillpack uninstall <name>` per the locked
v0.25.1 plan. Inverse of install with symmetric data-loss posture:
refuses if the slug isn't in the managed-block's cumulative-slugs
receipt (D8) or if any installed file diverges from the bundle
original (D11). Same --overwrite-local escape hatch as install.
src/core/skillpack/installer.ts:
- New UninstallError class (mirrors InstallError shape) with codes:
lock_held, bundle_error, target_missing, unknown_skill,
user_added_slug (D8), locally_modified (D11), managed_block_missing.
- New types: UninstallFileOutcome, UninstallFileResult,
UninstallResult, UninstallOptions.
- New applyUninstall() function. Steps:
1. Acquire workspace lockfile (same gate as install).
2. D8 check: read managed block; verify slug is in cumulative-slugs
receipt. If user-added or unknown, throw user_added_slug.
3. Enumerate bundle entries scoped to the skill (NOT shared_deps —
other installed skills depend on them).
4. D11 check: hash each existing target file vs bundle original.
Skip removal for divergent files unless --overwrite-local.
5. Atomic: if ANY file would be skipped due to local-mod and the
user did not pass --overwrite-local, refuse the WHOLE uninstall
(no half-uninstall — would desync managed block from filesystem).
6. Rebuild managed block via applyManagedBlockUninstall() (drops
slug from cumulative-slugs, preserves other rows + user-added
unknown rows with stderr warning, atomic write via writeAtomic).
7. Release lock.
src/commands/skillpack.ts:
- Wire `gbrain skillpack uninstall` subcommand. Flags mirror install:
--dry-run, --overwrite-local, --force-unlock, --skills-dir,
--workspace, --json, --help.
- Exit codes: 0 success, 1 refused due to local-mod (recoverable
with --overwrite-local), 2 setup error (slug not in receipt, no
workspace, lock held, etc.).
- Help text documents the symmetric trust contract explicitly.
D6 test slot is filled (smoke test t2 "uninstall changes routing"
will use this command). Per the plan, no `--all` uninstall in v0.25.1
(scope-narrowing; renaming a skill in the bundle should still be the
install --all path that prunes).
Typecheck passes. Privacy guard passes. `gbrain skillpack uninstall
--help` renders correctly.
Out of scope for this commit (next):
- test/skillpack-uninstall.test.ts (D8 + D11 cases, multi-arg,
fail-loud-under-lock, idempotent-when-absent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 archive-crawler safety gate (D12 + codex HIGH-4 fix)
Adds the gbrain.yml `archive-crawler.scan_paths:` allow-list contract
that closes the codex HIGH-4 finding. The archive-crawler skill
refuses to run unless the user has explicitly listed paths the agent
is permitted to scan.
src/core/archive-crawler-config.ts (NEW, 263 lines):
- Sibling to storage-config.ts (separate concern: archive scanning,
not storage tiering; same gbrain.yml file shape).
- Hand-rolled parser for the `archive-crawler:` section (mirrors
storage-config's parsing pattern; same trade-off — narrow-but-
predictable, zero-dep).
- Accepts both `archive-crawler:` and `archive_crawler:` spellings.
- ArchiveCrawlerConfig: { scan_paths: string[]; deny_paths: string[] }
— both normalized to absolute trailing-slashed paths.
- Validation:
* scan_paths MUST be non-empty (D12 contract)
* Every path absolute after ~ expansion (rejects relative)
* Path-traversal rejected (`..` literal in path → invalid_path)
* Trailing-slash normalized for unambiguous prefix matching
- isPathAllowed(candidate, config) helper for runtime per-file gate:
prefix-match against scan_paths, deny_paths overrides. Directory-
boundary safe — /writing/ does NOT match /writing-stuff/.
- ArchiveCrawlerConfigError class with discriminated codes:
missing_section / empty_scan_paths / invalid_path / parse_error.
test/archive-crawler-config.test.ts (NEW, 19 tests):
- D12 missing_section gates: null repoPath, missing gbrain.yml, no
archive-crawler section.
- D12 empty_scan_paths: scan_paths omitted or empty array.
- D12 invalid_path: relative path, ".." traversal in scan_paths,
".." traversal in deny_paths.
- Happy path: normalized paths, ~ expansion, deny_paths optional,
both archive-crawler and archive_crawler key spellings.
- Direct API validation (normalizeAndValidateArchiveCrawlerConfig).
- isPathAllowed: scan_path match, scan_path miss, deny_path override,
directory-boundary correctness (writing/ vs writing-stuff/),
relative-path rejection.
19/19 pass in 17ms. Privacy guard passes. Typecheck OK.
The skills/archive-crawler/SKILL.md (already shipped in earlier
commit) documents the contract; this commit lands the runtime
that enforces it. The skill's safety claim is no longer aspirational.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.25.1 PTY harness port from gstack (D14/C-prime)
Ports gstack's claude-pty-runner.ts (~1300 lines) as a generalized
gbrain harness (~470 lines after trimming gstack-specific
orchestrators). Used by the smoke test E2E to drive interactive
openclaw sessions; future: any CLI command that grows interactive
prompts becomes testable without a refactor.
test/helpers/cli-pty-runner.ts (NEW, 470 lines):
- launchPty(opts): generic CLI spawner via Bun.spawn `terminal:` mode.
Drops gstack's launchClaudePty's --permission-mode plan default;
takes any binary + args.
- resolveBinary(name, override?): finds CLI binaries on PATH with
homebrew/local/bun fallbacks.
- stripAnsi: standard CSI + OSC + charset + DEC-special escape
stripping (verbatim port).
- isNumberedOptionListVisible: cursor + numbered list detection.
- parseNumberedOptions: extracts cursor-anchored numbered AUQ options
(1-based indices, sequential block only). Handles cursor-on-non-1
(user pressed Down) and box-layout AUQs (cursor mid-line after
dividers). Reads only last 4KB to avoid matching stale lists.
- optionsSignature: stable hash for "is this AUQ the same as last
poll?" detection.
- isTrustDialogVisible: matches Claude Code's "trust this folder"
dialog so launchPty can auto-handle it.
- PtyOptions / PtySession types + send / sendKey / mark / visibleSince
/ waitFor / waitForAny primitives.
- launchPty internals: terminal: mode, exit tracking, wall-clock
timeout, autoTrust polling watcher (15s window), graceful close
with SIGINT then SIGKILL fallback.
DROPPED from the gstack original (gstack-specific):
- runPlanSkillObservation, runPlanSkillCounting, invokeAndObserve
(Claude-Code plan-mode test orchestrators).
- isPlanReadyVisible, isPermissionDialogVisible (Claude-Code-specific
dialog detection).
- ceoStep0Boundary, engStep0Boundary, designStep0Boundary,
devexStep0Boundary (per-skill /plan-* boundary predicates).
- MODE_RE, COMPLETION_SUMMARY_RE, parseQuestionPrompt, auqFingerprint,
assertReviewReportAtBottom (gstack plan-review specifics).
- classifyVisible (plan-mode outcome classifier).
If the smoke test ever needs Claude-Code-specific dialog detection,
add a thin wrapper in test/e2e/ — keeping the harness generic.
test/cli-pty-runner.test.ts (NEW, 24 tests, all pass):
- stripAnsi: 6 cases (CSI, OSC-BEL, OSC-ST, charset, DEC-special, plain)
- isNumberedOptionListVisible: 4 cases (match, no-cursor, single-opt,
TTY collapsed-whitespace)
- parseNumberedOptions: 7 cases (3-opt, no-list, single-opt, prose-
gating-pattern, gap-truncation, cursor-on-non-1, last-4KB-only)
- optionsSignature: 2 cases (order-independence, label-changes-sig)
- isTrustDialogVisible: 2 cases (canonical phrase, non-match)
- resolveBinary: 3 cases (override, missing, sh-on-path)
24/24 pass in 14ms. Privacy guard passes. Typecheck OK.
Bun version requirement (D14): engines.bun >= 1.3.10 (set in commit
|
||
|
|
4fc1246606 |
v0.24.0: production-hardening pass on the skillify loop (#387)
* feat: v0.19.0 — skillify loop + AGENTS.md compat + brain-first convention This is the v0.19.0 release. The branch ships four new CLI commands, a refactor to check-resolvable, and an expansion of the brain-first convention for sub-agent tool discovery. The original commit message described only the convention expansion, undercounting the scope by ~5x; this amend captures the full release. NEW COMMANDS - gbrain skillify scaffold <name> — 4 stub files + idempotent resolver row - gbrain skillify check [path] — 10-item post-task audit (promoted) - gbrain skillpack list / install — curated 25-skill bundle, atomic install - gbrain skillpack diff <name> — per-file diff preview - gbrain routing-eval — dedicated CI verb for Check 5 fixtures CHECK-RESOLVABLE REFACTOR - Accepts AGENTS.md as a resolver file alongside RESOLVER.md, at either the skills directory or one level up (workspace root layout). - Auto-derives the skill manifest by walking skills/*/SKILL.md when manifest.json is missing. - Splits ResolvableReport into errors[] + warnings[] so advisory checks (filing audit, routing gaps, DRY violations) don't break CI by default. - New --strict opt-in flag promotes warnings to exit 1. BRAIN-FIRST CONVENTION - skills/conventions/brain-first.md expanded from 5-step lookup guide to full sub-agent reference: tool inventory, lookup chain, score thresholds, authority hierarchy, sync rules, entity page conventions, sub-agent propagation rule. PRODUCTION-READINESS HARDENING (this branch's review pass) - routing-eval --llm: emits stderr placeholder notice + runs structural layer only. README, CHANGELOG, CLI help all rewritten consistently. Was a silent no-op against documented contract. - skillpack installer: receipt comment in fence (cumulative-slugs="...") preserves single-skill-install accumulation while letting install --all prune removed bundle skills cleanly. Unknown rows preserved + stderr warning for the operating agent. Pre-v0.19 fences upgrade silently. - skillify scaffold: resolver-row regex broadened to detect backticked, quoted, and bare path forms. No duplicate row on --force after the user normalizes formatting. - scripts/check-privacy.sh: now wired into package.json test chain so the wintermute-ban rule is actually enforced. New regression test. - E2E Tier 2 (LLM skills) promoted from schedule-only to required per-PR CI. Local Tier 1 + Tier 2 verified clean. - Stale v0.17/v0.18 version labels rewritten across new files. TESTS - test/routing-eval-cli.test.ts: 4 cases covering --llm warn semantics - test/privacy-script-wired.test.ts: regression guard for CI wiring - test/skillpack-install.test.ts: 4 new cases for receipt + cumulative + unknown-row preserve+warn + pre-v0.19 upgrade path - test/skillify-scaffold.test.ts: 4 new cases for broadened regex VERIFICATION - bun test: 2237 pass / 18 known PGLite-contention flakes (CI green; documented as P3 dev-experience in TODOS.md) - bun run typecheck: clean - bun run test:e2e: 18/19 files green (1 pre-existing flake on master, not caused by this branch — verified via git stash) - llms.txt + llms-full.txt regenerated to match README + CHANGELOG Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: scrub banned fork name from public artifacts The privacy guard wired into the test chain in this branch caught 5 pre-existing references to the banned OpenClaw fork name in CHANGELOG.md (2x), skills/migrations/v0.19.0.md (1x), src/cli.ts (1x), and src/commands/sync.ts (1x). All originated in master's v0.19.0 release notes and migration doc when the privacy script existed but wasn't wired into CI yet. Replacements per CLAUDE.md privacy mapping: - Origin-story copy (CHANGELOG layer narratives, code comments naming the production deployment that drove the feature) → "Garry's OpenClaw" - Reader-facing migration step → "your OpenClaw" No code semantics changed. Comments + headings only. Verification: scripts/check-privacy.sh exits 0, full CI guard chain green (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump VERSION to 0.24.0 + new CHANGELOG entry Bump branch version above master's v0.21.0 per CLAUDE.md "CHANGELOG + VERSION are branch-scoped" rule. The new v0.24.0 entry at the top of CHANGELOG covers what THIS branch adds vs master: - routing-eval --llm honesty pass (4-surface contract drift fix) - skillpack installer cumulative-receipt + unknown-row preserve+warn (the Codex-caught regression that would have shipped in master if the original v0.19.0 had landed without this branch's review pass) - skillify scaffold resolver-row regex broadening (backtick + quoted + bare forms; idempotency contract preserved under hand-editing) - 5 banned-name leaks scrubbed from public artifacts - check-privacy.sh wired into CI test chain + regression guard test - 7 stale v0.17/v0.18 version labels rewritten across 5 files - Tier 2 (LLM-skills E2E) promoted from schedule-only to required per-PR VERSION 0.21.0 → 0.24.0 package.json version field synced. llms.txt + llms-full.txt regenerated (no content drift; sizes match). Test suite: 62/62 green across the 5 test files this branch added or extended (routing-eval-cli, privacy-script-wired, skillpack-install, skillify-scaffold, build-llms). CI guards: privacy + jsonb + progress + wasm + typecheck all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.24.0 Auto-discovered drift via /document-release after the v0.24.0 hardening pass landed. All factual corrections clearly warranted by the diff. CLAUDE.md: - Skillpack installer: documented the cumulative-slugs receipt comment, install --all prune semantics, unknown-row preserve+warn behavior, and pre-v0.24 silent upgrade. Was previously vague about "tracks a skill manifest so install --update diffs cleanly" without explaining what the receipt is or why it matters. - routing-eval: replaced the false claim that --llm "opts into a Haiku tie-break layer for CI." Now correctly describes the placeholder semantic landed in v0.24.0 (stderr notice + structural-only run). README.md: - Skillpack section: added one paragraph on the receipt comment + the user-visible stderr message for hand-added rows. Connects the safe rerun promise to the v0.24.0 implementation that actually enforces it. CONTRIBUTING.md: - Running tests section: now recommends `bun run test` (full CI guard chain + typecheck + tests) before pushing. Names each guard so new contributors understand what catches what. The privacy guard (newly wired in v0.24.0) is one of these — without `bun run test` you'd skip it locally and find out from CI. llms-full.txt: regenerated to reflect CLAUDE.md changes. Verification: full guard chain green locally (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garry@ycombinator.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
8468ba25a9 |
v0.22.8 perf: doctor integrity batch-load + multi-source correctness (#393)
* perf: batch-load integrity scan — 500 round-trips → 1 SQL query
doctor's integrity_sample check called getPage() sequentially for 500
pages through PgBouncer transaction-mode pooling. Each call required a
full connection acquire/release cycle, causing doctor to timeout (~90s+)
on production deployments.
Replace with a single SQL query that fetches slug, compiled_truth, and
frontmatter for all candidate pages at once. Falls back to the
sequential path for PGLite or when no DB connection is available.
Before: doctor timeout (killed at 60s)
After: doctor completes in ~6s (full run including all other checks)
143 existing minions tests pass unchanged.
* fix: skillpack acquireLock negative-age on Linux sub-ms fs timestamps
On Linux ext4, statSync().mtimeMs has sub-ms precision while Date.now() is
integer ms. A just-written lockfile can report an mtime ~0.3ms ahead of
Date.now(), making age negative. The acquireLock check `age >= staleMs`
then evaluated false on staleMs:0, falling through the forceUnlock branch
and throwing "Another skillpack install appears to be running" instead of
unlocking. macOS rounds to integer ms so this only surfaced on Linux CI.
Clamp age to zero and add a utimesSync-based regression test that pushes
the lock mtime 10ms into the future to deterministically reproduce the
negative-age case on any platform.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: scanIntegrity batch path scopes by unique slug + Postgres-only gate
Codex review caught that the batch SQL scanned raw (source_id, slug) rows
while sequential's getAllSlugs() returned a Set<string>. On multi-source
brains (UNIQUE(source_id, slug) since v0.18.0), the batch path overcounted
hits and exhausted the LIMIT before covering N distinct pages.
Three changes:
- SELECT DISTINCT ON (slug) ... ORDER BY slug mirrors Set<string>
semantics; multi-source brains now get exact unique-slug counts.
- engine.kind === 'postgres' gate at the call site so PGLite never
enters the batch branch (catch{} fallback was firing on every PGLite
doctor run, polluting the GBRAIN_DEBUG log signal).
- Replace bare catch{} with debug-gated console.error so real Postgres
errors (deadlock, connection drop, SQL bug) are diagnosable instead
of silently swallowed.
Plus inline comments explaining the WHY for DISTINCT ON, the engine.kind
gate, the GBRAIN_DEBUG fallback, and the validate filter divergence
(boolean is the documented contract; stringly-typed handled at lint).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: scanIntegrity batch parity (dedup, hits, validate, topPages)
Real-Postgres E2E tests asserting the batch fast path returns identical
results to the sequential path on the four cases that matter:
- dedup: multi-source duplicate slugs scan once (regression guard for
the codex catch). Raw SQL fixture seeds the alt-source row since
engine.putPage doesn't take a source_id.
- hits: bareHits and externalHits arrays match between paths.
- validate: validate:false (boolean) page is skipped on both paths.
- topPages: ordering matches.
Skip when DATABASE_URL is not set (matches existing test/e2e/ pattern).
Per-test TRUNCATE keeps fixture state isolated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: bump version, changelog, and CLAUDE.md (v0.22.7)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* chore: regenerate llms-full.txt for v0.22.7 CLAUDE.md updates
CLAUDE.md gained the integrity.ts inventory entry and the new
test/e2e/integrity-batch.test.ts test file in commit
|
||
|
|
11abb24ddd |
v0.20.4 feat: merge gbrain-jobs into minion-orchestrator — single unified minions skill (#381)
* feat: merge gbrain-jobs into minion-orchestrator — single unified minions skill
* fix(skill/minion-orchestrator): correct MCP boundary, real handler names, PGLite path
The initial merge commit
|
||
|
|
78ba0b5b53 |
v0.19.0 check-resolvable: add OpenClaw skills-dir fallback + docs/tests (#326)
* Add OpenClaw skills fallback for check-resolvable * feat: v0.17.0 foundation — errors/warnings split + AGENTS.md support + auto-manifest First two workstreams of the v0.17.0 "skillify end-to-end" release. Landed together because the D-CX-3 exit-code refactor is a prerequisite for W1's warning-surfaced filing audit in Workstream 3. ## D-CX-3: split ResolvableReport into errors[] + warnings[] + --strict Prior: `env.ok = report.issues.length === 0` treated warnings and errors identically for exit status. Any warning forced exit 1, which meant the planned filing-audit (W3) would break CI for every OpenClaw deployment emitting advisory warnings. New contract: - `ResolvableReport.errors[]` and `warnings[]` as separate arrays. - `issues[]` stays as deprecated backcompat union (remove in v0.18). - Default: exit 0 unless any errors. Warnings are advisory. - `--strict` flag promotes warnings to fail CI (explicit opt-in). Files: src/core/check-resolvable.ts, src/commands/check-resolvable.ts (added --strict flag + help text + header doc), src/commands/doctor.ts (use new fields), test/check-resolvable-cli.test.ts (rewrite REGRESSION-GATE to document the new contract, add 3 D-CX-3 cases). ## W1: AGENTS.md support + auto-manifest + priority fix The reference OpenClaw deployment uses AGENTS.md (not RESOLVER.md) at the workspace root, and ships without a manifest.json. check-resolvable silently false-passed against it pre-W1: 0 manifest entries meant 0 reachability iterations meant 0 errors reported. Post-W1 behavior against ~/git/<redacted>/workspace (smoke-tested live): - Detects 102 skills via SKILL.md walk (no manifest.json needed) - Flags 15 unreachable errors (exactly the essay's '~15% dark' finding) - Flags 108 warnings (overlaps, gaps) — advisory, not blocking - Auto-detects via \$OPENCLAW_WORKSPACE without --skills-dir Changes: - NEW src/core/resolver-filenames.ts: one source of truth for the filename policy. \`RESOLVER_FILENAMES = ['RESOLVER.md', 'AGENTS.md']\`. Callers import from here, never hardcode either name. - NEW src/core/skill-manifest.ts: \`loadOrDeriveManifest()\` — reads manifest.json when present+valid, otherwise walks \`skillsDir/*/SKILL.md\` to derive a synthetic manifest. Both check-resolvable.ts AND dry-fix.ts now call this, replacing the two duplicated loaders that silently returned [] on missing file (F-ENG-1, D-CX-12). - src/core/repo-root.ts (rewrite): auto-detect priority changed to put \$OPENCLAW_WORKSPACE ahead of findRepoRoot() walk when explicitly set (D-CX-4). Adds workspace-root AGENTS.md detection — OpenClaw layout places routing at workspace/AGENTS.md with skills/ below. New SkillsDirSource variants \`openclaw_workspace_env_root\` and \`openclaw_workspace_home_root\` for --verbose log clarity. - src/core/check-resolvable.ts: accepts RESOLVER.md or AGENTS.md at the skills dir or one level up (workspace root). Uses loadOrDeriveManifest for reachability. Updated error messages reference both filenames. - src/core/dry-fix.ts: unified manifest loader — auto-fix now works in AGENTS.md-only workspaces where it previously no-op'd silently. - src/commands/check-resolvable.ts: new AUTO_DETECT_HINT import for clearer missing-skills-dir errors; updated sourceLabel map for the two new workspace-root variants. Tests: - test/skill-manifest.test.ts: 14 cases covering explicit-manifest, derived-manifest, malformed JSON, wrong shape, empty explicit array (honored as 'zero skills' declaration), dirname fallback when no name: frontmatter, underscore/dotfile dir skipping. - test/repo-root.test.ts: new tests for the priority swap, AGENTS.md skills-dir variant, AGENTS.md workspace-root variant, both-files present (RESOLVER.md wins). - test/check-resolvable-cli.test.ts: updated regression-gate to the new contract; added three D-CX-3 cases. All 105 tests passing across the foundation surface. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W2 — Check 5 trigger routing eval (structural) Check 5 of the 10-step skillify checklist (the essay's "resolver trigger eval") now runs structurally by default and has a dedicated CLI verb for CI. Ships Layer A; Layer B (LLM tie-break) is reserved for v0.18. ## New module: src/core/routing-eval.ts The harness. Pure functions: - `normalizeText(s)`: lowercase, strip non-alnum to spaces, collapse whitespace. Unicode-friendly, quote-agnostic, punctuation-tolerant. - `extractTriggerPhrases(cellText)`: split quoted alternatives like `"search for", "find me"` into separate normalized phrases; fall back to the whole cell when unquoted (OpenClaw-style descriptions). - `indexResolverTriggers(resolverContent)`: build a skill-slug → normalized-trigger-phrases map from the resolver table. - `structuralRouteMatch(intent, index)`: substring-match the normalized intent against every trigger phrase; return the set of matched skills + whether the match was ambiguous (more than one specific skill, excluding always-on family). - `lintRoutingFixtures`: rejects fixtures whose intent is verbatim-equal to a trigger (D-CX-6: fixtures must paraphrase the framing, not copy the trigger text) and unknown expected_skill references. - `loadRoutingFixtures(skillsDir)`: walks `skills/<name>/routing-eval.jsonl`, handles JSONL line-comments (`//` / `#`), collects malformed lines separately without crashing. - `runRoutingEval(resolver, fixtures)`: pure scoring. Supports negative cases (`expected_skill: null` — nothing should match) and an `ambiguous_with` allow-list for skills that co-fire with always-on handlers (signal-detector, brain-ops, ingest). Outcomes per fixture: `pass`, `missed`, `ambiguous`, `false_positive`. Metrics: `top1Accuracy`, `passed`, `missed`, `ambiguous`, `falsePositives`. ## Integration: check-resolvable runs Layer A by default `checkResolvable()` now loads `routing-eval.jsonl` fixtures from every skill, runs the structural eval, and appends non-pass outcomes as warning-severity issues. New issue types: - `routing_miss` — expected skill did not match - `routing_ambiguous` — expected matched AND unexpected skills - `routing_false_positive` — negative case unexpectedly matched - `routing_fixture_lint` — linter or malformed-JSONL finding All four are warnings — routing issues don't break exit in default mode, but `--strict` promotes them (D-CX-3 contract). Advisories without breaking CI. ## New CLI verb: `gbrain routing-eval` Standalone Check 5 runner. `--json` envelope, `--llm` flag reserved, `--skills-dir` override. Exit codes: 0 clean, 1 any failure/lint, 2 setup error. Suitable for CI gating separately from check-resolvable. Removed from DEFERRED in CLI: `{check: 5, name: trigger_routing_eval}`. Check 6 (brain_filing) still deferred; lands in W3. ## Seed fixtures - skills/query/routing-eval.jsonl - skills/citation-fixer/routing-eval.jsonl (includes a negative case) These are intentionally modest. Additional fixtures per skill are the natural next step; routing-eval itself passes cleanly under check-resolvable default mode even when fixtures surface real gaps (they're warnings, not errors). Running `gbrain routing-eval` reveals the gaps immediately. ## Tests (34 new cases + updated integrations) - test/routing-eval.test.ts: full harness coverage including normalization, trigger extraction (quoted and unquoted), indexer, structural match with ambiguity + always-on exemption, fixture linter (verbatim-equality rule, unknown-skill rule, shape rule, negative-case skip), JSONL loader (comments, malformed lines, missing dirs, underscore/dot skipping), and every runRoutingEval outcome (pass, miss, ambiguous, negative-pass, false-positive, empty). - test/check-resolvable-cli.test.ts: updated DEFERRED unit test + `--json` envelope test + `--verbose` test to reflect Check 5 shipping. 140/140 passing across the W1 + W2 surface. ## Live smoke `gbrain routing-eval --json` against the current gbrain repo: 6 fixtures, 1 passing, 5 missed. The misses correctly surface resolver-trigger narrowness (intents users naturally phrase differently than trigger text). Fixtures will iterate in follow-up PRs; the machinery ships now. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W3 — Check 6 brain filing audit Check 6 ships. Every skill that writes brain pages is now audited against a machine-readable filing-rules doc at `skills/_brain-filing-rules.json`. ## New: skills/_brain-filing-rules.json Canonical filing rules, JSON (D-CX-8: the pre-existing yaml-lite parser handles flat maps only, so YAML would have needed a new dependency for one file). The companion `_brain-filing-rules.md` stays as the human explainer. 14 rule entries + explicit `sources_dir` carve-out for bulk/raw data. ## New module: src/core/filing-audit.ts - `loadFilingRules(skillsDir)`: returns parsed doc or null (missing file → no-op; malformed JSON throws loud). - `allowedDirectories(rules)`: normalized set of every rules[] directory + sources_dir. - `runFilingAudit(skillsDir)`: walks skills/*/SKILL.md, parses frontmatter, audits any skill with `writes_pages: true`. Two checks per qualifying skill: 1. `writes_to:` list is non-empty. 2. Every entry in `writes_to:` appears in allowedDirectories. Both failures emit warning-severity issues. No errors — advisories only, per D-CX-3. ## Distinction: writes_pages vs mutating (D-CX-7) v0.17 introduces a new boolean frontmatter field `writes_pages:`. `mutating: true` already means "has any side effect" (cron schedulers, report writers, config mutators). Filing audit targets ONLY skills with `writes_pages: true`, correctly excluding side- effect-but-not-page-writing skills. The codex outside voice caught this: conflating the two fields would drag ~100 skills into filing-audit noise in the reference OpenClaw deployment. ## Integration: check-resolvable runs Check 6 by default `checkResolvable()` calls `runFilingAudit(skillsDir)` and appends issues as warnings. On missing/malformed rules doc, surfaces a single advisory rather than bailing. `DEFERRED` array in the CLI is now empty — v0.17 ships both Check 5 (W2) and Check 6 (W3). The export stays in place (stable --json field) for future deferred checks. ## Seeded frontmatter on 7 canonical writers Added `writes_pages: true` + `writes_to:` to: - brain-ops (people, companies, deals, concepts, meetings) - enrich (people, companies) - ingest (people, companies, concepts, meetings, sources) - idea-ingest (people, concepts, sources) - media-ingest (concepts, people, companies, sources) - meeting-ingestion (meetings, people, companies) - signal-detector (people, companies, concepts) Live smoke: `gbrain check-resolvable --json` on gbrain repo shows `ok: true`, zero filing errors, zero filing warnings on seeded skills. Every other mutating:true skill (citation-fixer, cron-scheduler, data-research, maintain, migrate, minion-orchestrator, reports, setup, skill-creator, soul-audit, webhook-transforms) correctly skipped as side-effectful-but-not-page-writing. ## Tests (17 new cases + 3 updated CLI integrations) test/filing-audit.test.ts covers: - rules loader: missing (null), valid, malformed (throw), non-array rules (throw) - directory normalization (trailing slash, leading slash) - clean case - missing writes_to on writes_pages:true - unknown directory - D-CX-7: mutating:true alone does not trigger audit - writes_pages:false skips - no frontmatter skips - inline `writes_to: [a, b]` syntax - block `writes_to:\n - a` syntax - sources/ allowed - underscore/dot dir skipping - total counts (totalScanned vs writesPagesSkills) - missing dir graceful - action string quality guard Plus: CLI integration tests updated for empty DEFERRED array (Checks 5 and 6 both shipped). 158/158 passing across the v0.17 foundation + W1 + W2 + W3 surface. ## v0.18 preview (D-CX-13) v0.17 filing-audit is declaration-level only. A future `gbrain filing-audit --pages` walks the brain itself, infers primary subject from page content via LLM judgment, and flags actual misfilings vs. declarations. Declaration audit is the leading indicator; pages audit is the ground truth. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W4 — gbrain skillify {scaffold,check} subcommand namespace The essay's "skillify it!" verb becomes a CLI primitive pair. Two subcommands, both promoted/factored so there's one source of truth: ## `gbrain skillify scaffold <name>` (mechanical) Pure file generation. Zero LLM, zero judgment. Writes 5 stub files atomically: 1. skills/<name>/SKILL.md frontmatter + body template 2. skills/<name>/scripts/<name>.mjs deterministic-code stub 3. skills/<name>/routing-eval.jsonl routing fixture seed 4. test/<name>.test.ts vitest skeleton 5. Appended trigger row to the detected resolver file (RESOLVER.md or AGENTS.md — whatever W1's auto-detect found) Flags: --description (required), --triggers, --writes-to, --writes-pages, --mutating, --force, --dry-run, --json, --skills-dir. Kebab-case name validation (`^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$`). Works against gbrain-native RESOLVER.md layout AND OpenClaw-native AGENTS.md-at-workspace-root layout (W1 interop). ## `gbrain skillify check [path]` (audit) Promoted from scripts/skillify-check.ts per codex D-CX-2. The legacy script stays as a 12-line shim that delegates to the new module so existing callers (docs, cron, tests) keep working. Wrapped in a subcommand namespace: `gbrain skillify {scaffold, check}` is one coherent verb for the whole post-task loop. The essay's "skillify it!" triggers the markdown skill, which orchestrates the CLI primitives. ## Idempotency contract (D-CX-7) `skillify scaffold --force` regenerates stub FILES but never re-appends a resolver row that already references `skills/<name>/SKILL.md`. Unit test pins this: two applies produce one resolver row, not two. ## D-CX-9 SKILLIFY_STUB sentinel Every scaffolded script + SKILL.md body carries a SKILLIFY_STUB sentinel. `check-resolvable` walks every skill's script dir looking for the marker and emits a `skillify_stub_unreplaced` warning when found. Default mode: advisory. `--strict` mode: error, blocks CI. This is the gate that catches "we scaffolded and forgot to implement" — the exact failure codex flagged as "scaffold verification is theater" in the outside-voice review. ## Files - NEW src/core/skillify/templates.ts (template strings) - NEW src/core/skillify/generator.ts (planScaffold / applyScaffold + SkillifyScaffoldError with typed error codes) - NEW src/commands/skillify.ts (top-level dispatcher + scaffold handler) - NEW src/commands/skillify-check.ts (promoted check logic) - scripts/skillify-check.ts: rewritten to 12-line shim - skills/skillify/SKILL.md: Phase 2 now references the scaffold primitive; legacy manual path kept for extending existing skills - src/cli.ts: `skillify` added to CLI_ONLY + dispatcher - src/core/check-resolvable.ts: SKILLIFY_STUB sentinel scan + new issue type `skillify_stub_unreplaced` ## Tests (14 new scaffold cases) test/skillify-scaffold.test.ts covers: - SKILL_NAME_PATTERN validation (kebab-case, no spaces, no leading digit, no underscores/uppercase) - planScaffold against fresh + existing-file + --force paths - SKILLIFY_STUB sentinel presence in SKILL.md AND script stub (both gate paths) - D-CX-7 idempotency: resolverAppend null when row pre-exists, second apply doesn't duplicate the row - TBD-trigger placeholder when --triggers empty - writes_pages / writes_to / mutating flow through to frontmatter - applyScaffold writes files + appends resolver - Full AGENTS.md-layout workspace interop (W1) Existing test/skillify-check.test.ts still passes against the legacy shim — zero regression for downstream consumers. 178/178 passing across v0.17 foundation + W1..W4. ## Live smoke \`gbrain skillify scaffold webhook-verify --description "verify incoming webhook signatures" --triggers "verify webhook,check tunnel" --skills-dir /tmp/smoke --dry-run\` produces the expected 4-file plan plus a 115-byte resolver append. \`--help\` works on both the top-level and scaffold levels. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 W5 — gbrain skillpack install (deps closure + lockfile + diff/dry-run) The essay's "drop it into YOUR OpenClaw" promise lands as a CLI verb. One command installs a curated bundle of gbrain skills + the shared convention files they depend on into a target OpenClaw workspace. Data-loss protected, concurrency-safe, atomic on the AGENTS.md managed block. ## openclaw.plugin.json refresh - Bumped version from stale 0.4.1 → 0.17.0 (codex flagged this drift F-ENG-4 / D-CX-4). - Expanded curated skill list from 7 → 25. Uses skills/manifest.json top-level (v0.10.0 sourced) minus setup/migrate/publish (install-time / code+skill pairs) minus private skills. - Added \`shared_deps: [...]\` listing convention files every skill references: conventions/, _brain-filing-rules.{md,json}, _output-rules.md. Installer always pulls these (D-CX-10 dependency closure). - Added \`excluded_from_install: [...]\` for setup/migrate/publish — surfaces the intentional exclusion as data rather than a comment. ## New module: src/core/skillpack/bundle.ts - \`findGbrainRoot(start)\` — walks up looking for openclaw.plugin.json + src/cli.ts. The pair identifies a gbrain checkout. - \`loadBundleManifest(root)\` — strict validation + typed BundleError codes (manifest_not_found, manifest_malformed, skill_not_found). - \`enumerateBundle({gbrainRoot, skillSlug?, manifest})\` — flat list of source → target-relative paths. When skillSlug is set, scopes to that one skill BUT always pulls shared_deps. \`--all\` walks every skill in the manifest. - \`bundledSkillSlugs(manifest)\` — sorted slugs for \`skillpack list\`. ## New module: src/core/skillpack/installer.ts - \`planInstall(opts)\` — builds InstallPlan with per-file existing/identical diff state. Pure; no writes. - \`applyInstall(plan, opts)\` — writes files + managed block with the contracts below. - \`diffSkill(root, slug, skillsDir)\` — read-only per-file status for \`skillpack diff <name>\`. **Per-file diff protection (D-CX-3 / F4):** wrote_new fresh file wrote_overwrite local diff + --overwrite-local passed skipped_identical bytes match the bundle (silent re-install) skipped_locally_modified target differs + no --overwrite-local → PROTECTED DEFAULT **Concurrency + atomic AGENTS.md (D-CX-11):** - \`.gbrain-skillpack.lock\` at workspace root. Acquired on the first write, released in finally. - Lock stale threshold configurable (default 10min). --force-unlock overrides. - Managed-block writes via tmp-file-plus-rename (atomic on POSIX). **Managed-block format:** <!-- gbrain:skillpack:begin --> <!-- Installed by gbrain <version> — do not hand-edit between markers. --> | Trigger | Skill | |---------|-------| | "alpha" | \`skills/alpha/SKILL.md\` | | ... <!-- gbrain:skillpack:end --> extractManagedSlugs() roundtrips: single-skill installs accumulate into the same block rather than overwriting each other. ## New CLI: gbrain skillpack {list, install, diff, check} Namespaced alongside W4's \`gbrain skillify\`. Subcommands: list bundle inventory (human + --json) install <name> single skill + deps closure install --all entire curated bundle diff <name> per-file diff vs target; read-only check delegates to the pre-existing skillpack-check (same CLI just namespaced) Flags on install: --overwrite-local, --force-unlock, --dry-run, --json, --skills-dir, --workspace. Exit codes: 0 clean, 1 files skipped (protected local edits), 2 setup error / lock held. ## Live smoke \`gbrain skillpack list\`: 25 skills. \`skillpack install query --dry-run\` against a fresh temp workspace: 12 files planned (SKILL.md, routing-eval.jsonl, 7 convention files, 3 rule files, managed block to AGENTS.md). All shared_deps flagged [shared]. ## Tests (36 new cases) test/skillpack-install.test.ts: - findGbrainRoot walks up, returns null when absent - loadBundleManifest validates + rejects malformed - enumerateBundle pulls shared_deps on single-skill scope (D-CX-10) - buildManagedBlock + updateManagedBlock: append when absent, in-place replace when present, extractManagedSlugs roundtrip - planInstall + applyInstall: fresh install, dry-run, idempotency (skipped_identical), local-edit protection, --overwrite-local, lock-held concurrency (D-CX-11), --force-unlock, atomic managed-block write, multi-skill accumulation in managed block, AGENTS.md-at-workspace-root interop (W1 cross-check) - diffSkill: missing, identical, differs test/skillpack-sync-guard.test.ts (F-ENG-4): - both manifests exist - every skill in plugin.json exists on disk - every shared_dep exists on disk - plugin.json skills ⊂ skills/manifest.json - excluded skills aren't in the install list - plugin version ≥ 0.17 (kills the 0.4.1 stale drift) 204/204 passing across the v0.17 foundation + W1..W5. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: v0.17.0 guards — privacy scrub + OpenClaw-reference E2E + v0.16.4 regression Three ship-blocker work items from the eng review + codex outside voice round out v0.17: ## scripts/check-privacy.sh (CLAUDE.md:550 enforcement) Greps for the banned OpenClaw fork name (case-insensitive) across tracked files. Two modes: scripts/check-privacy.sh scan working tree scripts/check-privacy.sh --staged scan git-staged files (pre-commit) Exit 1 on any finding outside the allow-list. Allow-list covers files where the name is legitimately present: this script itself (defines the rule), CLAUDE.md (the canonical rule text), llms-full.txt (auto-generated from CLAUDE.md), the historical upgrade guide, and test/integrations.test.ts (whose personal-info regex ENFORCES the rule against recipes/). Scrubbed existing leaks: - CHANGELOG.md:366 reference in a closes-# line → "from the OpenClaw reference deployment" - test/doctor-minions-check.test.ts:171 comment → "an OpenClaw host's cron script" - test/plugin-loader.test.ts fixture plugin name → "openclaw-ref" ## test/e2e/openclaw-reference-compat.test.ts (ship-blocker gate) The test that proves v0.17 delivers on the headline claim. New fixture at test/fixtures/openclaw-reference-minimal/ mimics the reference OpenClaw deployment layout: AGENTS.md at workspace root, skills/ below, no manifest.json. Four fixture skills (signal-detector, query, brain-ops, context-now). Every v0.17 surface gets exercised end-to-end: - autoDetectSkillsDir with $OPENCLAW_WORKSPACE (D-CX-4 priority) - loadOrDeriveManifest walks SKILL.md (F-ENG-1 auto-manifest) - checkResolvable accepts AGENTS.md at workspace root, all 4 skills reachable via resolver rows, zero errors - Filing audit clean (brain-ops declares writes_pages+writes_to) - CLI subprocess via `--skills-dir` → exit 0 - CLI subprocess via $OPENCLAW_WORKSPACE (no flag) → exit 0, correct skillsDir detection - skillpack install against the layout writes managed block into AGENTS.md at workspace root This is THE ship-blocker test. If the W1 + W5 stack ever regresses against an AGENTS.md-layout workspace, this fails first. ## test/regression-v0_16_4.test.ts (F-ENG-8) Guards v0.17 against adding "surprise" warnings. Builds a clean fixture matching v0.16.4 canonical shape (manifest.json, RESOLVER.md, 2 skills, no routing-eval fixtures, no writes_pages). Runs v0.17 checkResolvable and asserts: - zero errors, zero routing_*/filing_*/skillify_stub_* warnings - JSON envelope keys unchanged (errors, warnings, issues, ok, summary) — deprecated `issues[]` still equals errors ∪ warnings - summary shape unchanged If someone adds a new check that fires unexpectedly on a v0.16.4-era fixture, this test catches it immediately. ## Fixture test/fixtures/openclaw-reference-minimal/ ├── AGENTS.md (4 rows, 3 sections) └── skills/ ├── brain-ops/SKILL.md (writes_pages+writes_to) ├── context-now/SKILL.md ├── query/SKILL.md └── signal-detector/SKILL.md Intentionally small (4 skills, 1 AGENTS.md, ~30 lines total) so the fixture is maintainable. The OPENCLAW-reference deployment has 107 skills — this fixture is the minimum shape that exercises the full v0.17 code path. ## Tests 215/215 passing across the full v0.17 surface: - foundation + W1 + W2 + W3 + W4 + W5 (204) - regression-v0_16_4 (3) - openclaw-reference-compat (7) - privacy guard (separate bash; exits 0 clean) Plus: privacy pre-commit hook is a drop-in wrapper (documented in the script header). Wiring into .github/workflows is a follow-up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * release: v0.17.0 — skillify goes end-to-end Every skill. Every check. Every install. One command each. Five workstreams land in one release: - W1: AGENTS.md + auto-manifest + env-priority - W2: Check 5 routing eval - W3: Check 6 brain filing - W4: gbrain skillify {scaffold,check} - W5: gbrain skillpack {list,install,diff} Plus D-CX-3 foundation (errors/warnings split + --strict), plus codex outside-voice fixes (D-CX-1..12 applied), plus privacy pre- commit guard, plus OpenClaw-reference E2E fixture, plus v0.16.4 regression guard. Live against the reference OpenClaw deployment: 102 skills detected via auto-manifest, 15 unreachable errors + 108 warnings surfaced — exactly the essay's "~15% dark" finding. The magic word from the essay finally works the way the essay describes. Tests: 2156 unit (178 new) + 152 E2E Tier 1 + 3 Tier 2 + 8 new openclaw-reference fixture cases. 0 failures across all tiers. Plan + reviews: ~/.claude/plans/p1-lets-just-vast-blanket.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): add missing 'strict' field to 5 Flags literals in check-resolvable-cli.test.ts CI failed `tsc --noEmit` after the D-CX-3 errors/warnings split added `strict: boolean` as a required field on the `Flags` interface. Five test sites in test/check-resolvable-cli.test.ts still construct Flags object literals (for direct `resolveSkillsDir()` calls) and hadn't been updated. Added `strict: false` to all five literals: - line 129 --skills-dir absolute path - line 135 --skills-dir relative path - line 148 no --skills-dir - line 160 no --skills-dir + no env - line 178 --skills-dir + OPENCLAW_WORKSPACE (REGRESSION-GATE) Unit tests: 207/207 pass across the v0.19 surface. tsc --noEmit exits 0. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: adopt gstack's branch-scoped CHANGELOG rule + rewrite v0.19.0 entry CLAUDE.md gains a new top section before "CHANGELOG voice" that codifies what gstack's CLAUDE.md already says: CHANGELOG is user-facing product release notes, not a log of internal decisions. Every entry describes what THIS branch adds vs master. Plan-file IDs, decision tags (D-CX-#, F-ENG-#), review rounds, test counts as marketing, and contributor- facing metrics don't belong in it. The v0.19.0 entry is rewritten to the new bar: Removed: - Version-collision note about v0.17.0/v0.18.0 shipping on master - All D-CX-## and W# tags (meaningless outside the plan file) - "codex caught" / CEO + Eng review round-up narrative - Plan file path reference - "215 new cases across 13 test files" marketing metrics - W1..W5 bucketing in itemized changes Kept / sharpened: - User-facing headline (what your agent can now do) - Numbers that mean something to users (unreachable-skills count, scaffold timing, pre/post AGENTS.md support) - Upgrade instructions - Added/Changed/Fixed/For-contributors itemized sections (standard keep-a-changelog shape) Version sequence (`grep "^## \["`) is contiguous v0.19.0 → v0.16.4. Privacy guard clean. Tests green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update README/CLAUDE/TODOS for v0.19.0 skills + skillify loop Skill count was stale (README said 26, actual is 28: skillify + skillpack-check were missing from the tables and count). Corrected throughout. Marked TODOS item "Checks 5 + 6 deferred in PR #325" as completed in v0.19 — they shipped as real implementations, not just filed issues. README: - Skill count 26 → 28 (headline, install flow, table section, architecture diagram) - Added `skillify` + `skillpack-check` rows to the operational skills table - Rewrote the "Skillify" section to lead with the four v0.19 CLI verbs (`gbrain skillify scaffold/check`, `gbrain skillpack list/install/diff`, `gbrain routing-eval`, `gbrain check-resolvable --strict`) instead of describing the pre-v0.19 state. Added the "works on your OpenClaw" pitch around AGENTS.md + auto-manifest. Added the "drop 25 curated skills into your OpenClaw" section for skillpack install. - Added v0.19 skills block + v0.18 multi-source + v0.17 dream to the Commands reference at the bottom. - Standalone instruction sets count: 25 → 28 (with a parenthetical noting the curated 25-skill bundle that `skillpack install` ships). CLAUDE.md: - Skill count 26 → 28 in the Skills section. - New "Skillify loop (v0.19)" sub-bullet listing skillify + skillpack-check. - Noted that `AGENTS.md` is also accepted as a resolver filename. TODOS.md: - Created "## Completed" section at the top. - Moved the "Checks 5 + 6" item there with completion note linking to the actual implementation files (routing-eval.ts + filing-audit.ts). Privacy scan clean. Version sequence contiguous v0.19.0 → v0.16.4. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): regenerate llms-full.txt + llms.txt after README/CLAUDE edits CI failed on `build-llms generator > committed llms.txt + llms-full.txt match current generator output`. The drift was expected: the prior commit edited README.md and CLAUDE.md (skill count + skillify section), both of which are inlined into llms-full.txt by `scripts/build-llms.ts`. Fix: `bun run build:llms` + commit the regenerated output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Wintermute <wintermute@garrytan.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |