3 Commits

Include renames

Author	SHA1	Message	Date
3 people	1eb430a2df	v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts (#1999) ... * fix(security): scope cross-source reads to the caller grant; close get_page exact-path leak One shared resolveRequestedScope() routes every source-scoped read op (query, code_callers/callees, search_by_image, code_blast/flow, get_page) through a single fail-closed trust+grant ladder: a remote caller's __all__ collapses to its granted sources (never the whole brain) and an explicit out-of-grant source_id is rejected. get_page's exact-match path now honors a federated grant via getPage(sourceIds[]) in both engines. Legacy bearer tokens carry their stored permissions.source_id grant (bounded, never widened). Also retries getConfig on transient connection loss. Closes #1924, #1371, #1393, #1336, #1603. * fix(ingest): non-string frontmatter no longer aborts lint/sync; embed/hook/catalog papercuts Parser coerces a non-string title to a string and falls back to inference for slug/type (never fabricating a "123" slug), with a lint NON_STRING_FIELD finding surfacing the malformed frontmatter; a defensive guard in content-sanity stops a non-string title from crashing the whole lint/sync run brain-wide. Plus: embed --catch-up no longer arms the overflowed 32-bit budget timer (and surfaces unembeddable chunks); the frontmatter pre-commit hook ships a correct .md/.mdx regex; and the skill catalog parses YAML block-scalar descriptions. Closes #1883, #1658, #1556, #1948, #1946, #1840, #1711. * v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: add NON_STRING_FIELD frontmatter validation class to docs for v0.42.37.0 The v0.42.37.0 non-string-frontmatter fix added an eighth validation class (NON_STRING_FIELD / lint code frontmatter-non-string-field). Update the two current-state docs that enumerate the validation classes: - skills/frontmatter-guard/SKILL.md (seven->eight + table row) - docs/integrations/pre-commit.md (seven->eight + table row) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-08 21:19:25 -07:00
4 people	4fc1246606	v0.24.0: production-hardening pass on the skillify loop (#387) ... * feat: v0.19.0 — skillify loop + AGENTS.md compat + brain-first convention This is the v0.19.0 release. The branch ships four new CLI commands, a refactor to check-resolvable, and an expansion of the brain-first convention for sub-agent tool discovery. The original commit message described only the convention expansion, undercounting the scope by ~5x; this amend captures the full release. NEW COMMANDS - gbrain skillify scaffold <name> — 4 stub files + idempotent resolver row - gbrain skillify check [path] — 10-item post-task audit (promoted) - gbrain skillpack list / install — curated 25-skill bundle, atomic install - gbrain skillpack diff <name> — per-file diff preview - gbrain routing-eval — dedicated CI verb for Check 5 fixtures CHECK-RESOLVABLE REFACTOR - Accepts AGENTS.md as a resolver file alongside RESOLVER.md, at either the skills directory or one level up (workspace root layout). - Auto-derives the skill manifest by walking skills/*/SKILL.md when manifest.json is missing. - Splits ResolvableReport into errors[] + warnings[] so advisory checks (filing audit, routing gaps, DRY violations) don't break CI by default. - New --strict opt-in flag promotes warnings to exit 1. BRAIN-FIRST CONVENTION - skills/conventions/brain-first.md expanded from 5-step lookup guide to full sub-agent reference: tool inventory, lookup chain, score thresholds, authority hierarchy, sync rules, entity page conventions, sub-agent propagation rule. PRODUCTION-READINESS HARDENING (this branch's review pass) - routing-eval --llm: emits stderr placeholder notice + runs structural layer only. README, CHANGELOG, CLI help all rewritten consistently. Was a silent no-op against documented contract. - skillpack installer: receipt comment in fence (cumulative-slugs="...") preserves single-skill-install accumulation while letting install --all prune removed bundle skills cleanly. Unknown rows preserved + stderr warning for the operating agent. Pre-v0.19 fences upgrade silently. - skillify scaffold: resolver-row regex broadened to detect backticked, quoted, and bare path forms. No duplicate row on --force after the user normalizes formatting. - scripts/check-privacy.sh: now wired into package.json test chain so the wintermute-ban rule is actually enforced. New regression test. - E2E Tier 2 (LLM skills) promoted from schedule-only to required per-PR CI. Local Tier 1 + Tier 2 verified clean. - Stale v0.17/v0.18 version labels rewritten across new files. TESTS - test/routing-eval-cli.test.ts: 4 cases covering --llm warn semantics - test/privacy-script-wired.test.ts: regression guard for CI wiring - test/skillpack-install.test.ts: 4 new cases for receipt + cumulative + unknown-row preserve+warn + pre-v0.19 upgrade path - test/skillify-scaffold.test.ts: 4 new cases for broadened regex VERIFICATION - bun test: 2237 pass / 18 known PGLite-contention flakes (CI green; documented as P3 dev-experience in TODOS.md) - bun run typecheck: clean - bun run test:e2e: 18/19 files green (1 pre-existing flake on master, not caused by this branch — verified via git stash) - llms.txt + llms-full.txt regenerated to match README + CHANGELOG Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: scrub banned fork name from public artifacts The privacy guard wired into the test chain in this branch caught 5 pre-existing references to the banned OpenClaw fork name in CHANGELOG.md (2x), skills/migrations/v0.19.0.md (1x), src/cli.ts (1x), and src/commands/sync.ts (1x). All originated in master's v0.19.0 release notes and migration doc when the privacy script existed but wasn't wired into CI yet. Replacements per CLAUDE.md privacy mapping: - Origin-story copy (CHANGELOG layer narratives, code comments naming the production deployment that drove the feature) → "Garry's OpenClaw" - Reader-facing migration step → "your OpenClaw" No code semantics changed. Comments + headings only. Verification: scripts/check-privacy.sh exits 0, full CI guard chain green (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump VERSION to 0.24.0 + new CHANGELOG entry Bump branch version above master's v0.21.0 per CLAUDE.md "CHANGELOG + VERSION are branch-scoped" rule. The new v0.24.0 entry at the top of CHANGELOG covers what THIS branch adds vs master: - routing-eval --llm honesty pass (4-surface contract drift fix) - skillpack installer cumulative-receipt + unknown-row preserve+warn (the Codex-caught regression that would have shipped in master if the original v0.19.0 had landed without this branch's review pass) - skillify scaffold resolver-row regex broadening (backtick + quoted + bare forms; idempotency contract preserved under hand-editing) - 5 banned-name leaks scrubbed from public artifacts - check-privacy.sh wired into CI test chain + regression guard test - 7 stale v0.17/v0.18 version labels rewritten across 5 files - Tier 2 (LLM-skills E2E) promoted from schedule-only to required per-PR VERSION 0.21.0 → 0.24.0 package.json version field synced. llms.txt + llms-full.txt regenerated (no content drift; sizes match). Test suite: 62/62 green across the 5 test files this branch added or extended (routing-eval-cli, privacy-script-wired, skillpack-install, skillify-scaffold, build-llms). CI guards: privacy + jsonb + progress + wasm + typecheck all clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.24.0 Auto-discovered drift via /document-release after the v0.24.0 hardening pass landed. All factual corrections clearly warranted by the diff. CLAUDE.md: - Skillpack installer: documented the cumulative-slugs receipt comment, install --all prune semantics, unknown-row preserve+warn behavior, and pre-v0.24 silent upgrade. Was previously vague about "tracks a skill manifest so install --update diffs cleanly" without explaining what the receipt is or why it matters. - routing-eval: replaced the false claim that --llm "opts into a Haiku tie-break layer for CI." Now correctly describes the placeholder semantic landed in v0.24.0 (stderr notice + structural-only run). README.md: - Skillpack section: added one paragraph on the receipt comment + the user-visible stderr message for hand-added rows. Connects the safe rerun promise to the v0.24.0 implementation that actually enforces it. CONTRIBUTING.md: - Running tests section: now recommends `bun run test` (full CI guard chain + typecheck + tests) before pushing. Names each guard so new contributors understand what catches what. The privacy guard (newly wired in v0.24.0) is one of these — without `bun run test` you'd skip it locally and find out from CI. llms-full.txt: regenerated to reflect CLAUDE.md changes. Verification: full guard chain green locally (privacy + jsonb + progress + wasm + typecheck). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Garry Tan <garry@ycombinator.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-30 23:17:54 -07:00
3 people	891c28b582	v0.22.4 feat: frontmatter-guard — 0 resolver warnings + validate/audit/install-hook CLI (#448) ... * fix: resolve check-resolvable warnings on master - skills/maintain/SKILL.md: drop "citation audit" trigger; the focused citation-fixer skill is the single owner. Silences the MECE overlap warning surfaced by src/core/check-resolvable.ts. - skills/RESOLVER.md: add citation-audit disambiguation row pointing citation-fixer (focused fix) and chain-into maintain for broader audit. Broaden query triggers ("who is", "background on", "notes on") so the failing routing-eval fixtures resolve. - skills/enrich/SKILL.md: replace inlined Citation Requirements block with backtick-wrapped `skills/conventions/quality.md` reference (the format extractDelegationTargets recognizes). Silences the dry_violation warning. - skills/citation-fixer/routing-eval.jsonl: rewrite the two failing fixtures to embed "fix citations" verbatim so the substring matcher passes. - skills/query/SKILL.md frontmatter: mirror the broadened RESOLVER.md triggers so the trigger round-trip test passes. Result: gbrain check-resolvable reports 0 warnings, 0 errors against the actual checked-in skills/ tree. * feat: extend parseMarkdown + lint with frontmatter validation surface Add an opt-in validation surface to parseMarkdown(): when called with { validate: true }, returns errors[] populated with seven canonical ParseValidationError codes: MISSING_OPEN, MISSING_CLOSE, YAML_PARSE, SLUG_MISMATCH, NULL_BYTES, NESTED_QUOTES, EMPTY_FRONTMATTER Existing callers are unaffected — validation is opt-in via the new opts argument. The validation logic lives here as the single source of truth for what counts as malformed brain-page frontmatter. src/commands/lint.ts now consumes parseMarkdown(..., { validate: true }) and emits stable lint rule names (frontmatter-missing-close, frontmatter-yaml-parse, frontmatter-null-bytes, frontmatter-nested-quotes, frontmatter-slug-mismatch, frontmatter-empty). MISSING_OPEN is suppressed to avoid double-reporting with the legacy no-frontmatter rule. Tests: test/markdown-validation.test.ts (NEW, all 7 codes) + test/lint-frontmatter.test.ts (NEW, lint integration + suppression). * feat: add brain-writer.ts orchestrator (scan / autoFix / writeBrainPage) Thin orchestrator (~280 lines) on top of parseMarkdown(..., {validate:true}) and isSyncable() (the canonical brain-page filter from src/core/sync.ts). Three consumers call into this module: the gbrain frontmatter CLI, the frontmatter_integrity doctor subcheck, and the v0.22.4 migration audit phase. Single source of truth — no parallel validation stack. Public API: - autoFixFrontmatter(content, opts?): { content, fixes } Mechanical auto-repair for the fixable subset (NULL_BYTES, MISSING_CLOSE, NESTED_QUOTES, SLUG_MISMATCH). Idempotent. - writeBrainPage(filePath, content, opts): path-guarded, .bak backup before any in-place mutation. Path guard refuses writes outside sourcePath. .bak is the safety contract for non-git brain repos. - scanBrainSources(engine, opts?): walks every registered source via direct SQL on sources.local_path, uses isSyncable() to filter, blocks symlinks (matches sync's no-symlink policy), respects AbortSignal. The dirty-tree guard from src/core/dry-fix.ts:getWorkingTreeStatus() is NOT used here — it rejects non-git repos as unsafe, but brain repos aren't always git repos. .bak backups are the contract that works universally. Tests: test/brain-writer.test.ts (NEW, 16 cases) — autoFix idempotency, path-guard reject, .bak backup, per-source rollup, AbortSignal mid-scan, single-source filter, missing-source-path graceful skip, symlink no-loop. * feat: gbrain frontmatter CLI (validate / audit / install-hook) New top-level command surface for the frontmatter-guard feature: gbrain frontmatter validate <path> [--json] [--fix] [--dry-run] Validate one .md file or recursively scan a directory. --fix writes .bak then rewrites in place. No git-tree-clean guard — .bak is the safety contract (works for both git and non-git brain repos). gbrain frontmatter audit [--source <id>] [--json] Read-only scan via scanBrainSources(). Per-source rollup grouped by error code. --fix is intentionally NOT available here; use validate --fix on the source path to repair. gbrain frontmatter install-hook [--source <id>] [--force] [--uninstall] Drops a pre-commit hook in each source that's a git repo (skips non-git sources with a one-line note). Hook script gracefully degrades when gbrain is missing on PATH (prints a warning, exits 0). Refuses to clobber existing hooks without --force; writes <hook>.bak. --uninstall reverses cleanly. src/cli.ts wires frontmatter through handleCliOnly so --help works without a DB connection. The audit subcommand instantiates an engine internally only when needed. Tests: test/frontmatter-cli.test.ts (NEW, 9 cases) + test/frontmatter-install-hook.test.ts (NEW, 6 cases) — --help no-DB, clean/broken validate, --fix dry-run, --fix non-git, --json envelope, recursive directory scan with isSyncable filter parity, hook install + overwrite-protection + --force + --uninstall + silent-refresh. * feat: doctor frontmatter_integrity subcheck Adds a frontmatter_integrity subcheck under gbrain doctor that calls scanBrainSources() (the same shared scanner the CLI and migration use). Reports per-source counts grouped by error code, with a fix hint pointing at `gbrain frontmatter validate <path> --fix`. Wrapped in a doctor progress phase with heartbeat so 50K-page brain scans stay visible. Tests: test/doctor.test.ts (UPDATE) — assertion that the subcheck calls scanBrainSources and the fix hint references the correct CLI. * feat: frontmatter-guard skill (registered in manifest + RESOLVER) New skill at skills/frontmatter-guard/SKILL.md that wraps the gbrain frontmatter CLI for agent-driven workflows. Agent-agnostic — no references to private host libraries. Registered in skills/manifest.json and skills/RESOLVER.md (the trigger row was added in the Part A commit). Triggers: "validate frontmatter", "check frontmatter", "fix frontmatter", "frontmatter audit", "brain lint". Includes routing-eval fixtures that pass the substring matcher. The SKILL.md has the conformance-required Output Format and Anti-Patterns sections. Anti-patterns explicitly call out: don't auto-fix MISSING_OPEN or EMPTY_FRONTMATTER without user input, don't skip .bak backups, don't install the pre-commit hook on non-git brain dirs. * feat: v0.22.4 migration orchestrator (audit-only, source-aware) Adds the v0.22.4 migration that surveys every registered source for frontmatter issues and queues per-source repair commands without ever mutating brain content. Three idempotent phases: - schema: no-op (no DB changes in v0.22.4) - audit: scanBrainSources() across ALL registered sources; writes JSON report to ~/.gbrain/migrations/v0.22.4-audit.json - emit-todo: appends one entry per source-with-issues to ~/.gbrain/migrations/pending-host-work.jsonl, each with the exact `gbrain frontmatter validate <source-path> --fix` command The agent reads skills/migrations/v0.22.4.md after upgrade, surfaces the report counts to the user, and runs the fix command only with explicit consent. `apply-migrations --yes` never silently rewrites brain pages. Filename convention: TS orchestrator at v0_22_4.ts (underscores, since TS module paths can't have dots); user-facing migration doc at skills/migrations/v0.22.4.md (dotted, matches existing convention). The pending-host-work.jsonl skill field references the dotted-path doc. Skips cleanly when no sources are registered (fresh install). Tests: test/migrations-v0_22_4.test.ts (NEW, 9 cases) + updated test/migration-orchestrator-v0_21_0.test.ts to allow v0.22.4 after, test/apply-migrations.test.ts skippedFuture arrays extended to include v0.22.4, test/check-resolvable.test.ts regression guard asserting the actual checked-in skills/ tree has 0 warnings + 0 errors. * docs: pre-commit recipe + downstream agent upgrade notes for v0.22.4 - docs/integrations/pre-commit.md (NEW): recipe doc covering install, bypass (`git commit --no-verify`), uninstall, and downstream-fork integration notes. Includes the full pipeline diagram showing how the hook (write-time gate), doctor (audit gate), and CLI (fix tool) share parseMarkdown(..., {validate:true}) as the single source of truth. - docs/UPGRADING_DOWNSTREAM_AGENTS.md: append v0.22.4 section with the diff pattern for forks that had inline frontmatter validators. Covers the five upgrade actions: replace ad-hoc validators, drop lib/brain-writer.mjs references (it never shipped), wire the doctor subcheck into custom health pipelines, optionally install the pre-commit hook on git-backed brain repos, and walk pending-host-work.jsonl after apply-migrations. - llms.txt + llms-full.txt: regenerated from build:llms script after the new docs landed. * chore: bump version and changelog (v0.22.4) Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix: handle null loadConfig() return in frontmatter + migration paths CI typecheck caught three call sites that passed loadConfig()'s GBrainConfig | null result straight into toEngineConfig() (which expects GBrainConfig, not null): - src/commands/frontmatter.ts:64 (audit subcommand connect) - src/commands/frontmatter-install-hook.ts:86 (install-hook connect) - src/commands/migrations/v0_22_4.ts:59 (audit phase connect) The frontmatter CLI and install-hook paths follow the existing src/commands/repair-jsonb.ts pattern: throw 'No brain configured. Run: gbrain init' so users get an actionable message instead of a TS-shaped runtime crash. The v0.22.4 migration audit phase takes a different shape: a fresh install or test environment running apply-migrations shouldn't fail hard just because there's no brain to scan yet. Return a clean 'skipped: no_brain_configured' phase result so the orchestrator continues normally and the ledger records a complete (skipped) run. * test: add v0.22.4 migration E2E + injection point for testability Closes plan item B14 (the E2E that was promised but not delivered before the original ship). Runs the v0_22_4 orchestrator end-to-end on PGLite against a fixture brain with two registered sources and synthetic malformed pages on disk. Asserts: - audit phase writes ~/.gbrain/migrations/v0.22.4-audit.json with per-source counts (NESTED_QUOTES + NULL_BYTES on alpha, NESTED_QUOTES on beta) - emit-todo phase appends one entry per source-with-issues to pending-host-work.jsonl, each pointing at skills/migrations/v0.22.4.md with the exact `gbrain frontmatter validate <source> --fix` command - the migration is audit-only — no fixture page is mutated during apply-migrations (no .bak created, contents byte-identical) - re-running the orchestrator is idempotent — JSONL stays at 2 lines Adds a small test-injection point to v0_22_4.ts: __setTestEngineOverride(engine: BrainEngine | null): void Mirrors src/commands/repair-jsonb.ts pattern. When set, phaseBAudit uses the injected engine instead of loadConfig + createEngine. Production path is unchanged: the override is null by default and the existing loadConfig logic runs end-to-end. Required because Bun's os.homedir() does not observe mid-process process.env.HOME mutations, so we can't redirect loadConfig's config-file lookup via env-var overrides; the injection point is the only hermetic way to E2E-test the orchestrator without writing to the user's real ~/.gbrain/config.json. Test runs unconditionally in CI's Tier 1 (no DATABASE_URL needed, PGLite in-memory). --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>	2026-04-26 20:45:05 -07:00