mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-31 04:07:52 +00:00
* v0.40.6.0 Phase 1 foundations — pack-lock + mutate-audit + cache invalidation + lint rules + best-effort
Six new primitives that Phase 2's withMutation skeleton (next commit) depends on.
No consumers yet; all callers wire up in Phase 4. Foundations ship first per
codex C1 phase-ordering finding from /plan-eng-review.
1.1 pack-lock.ts (18 cases)
Atomic acquire via openSync(path, 'wx') = O_CREAT|O_EXCL. Kernel-level
atomic, NO TOCTOU window. Codex C8 caught that page-lock.ts:79+96 has
existsSync+writeFileSync (TOCTOU) — we deliberately do NOT copy it.
Stale detection via TTL (60s default) + kill(pid, 0) liveness probe.
TTL refresh every 10s while withPackLock(fn) runs so long DB-aware
lint/stats on big brains don't go stale. --force = "steal stale lock"
(NOT "skip locking"). Lock path per-pack so two packs never block.
1.2 mutate-audit.ts (13 cases)
ISO-week JSONL at ~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl.
Privacy redacted per D20: type names → sha8, prefixes → first slug
segment only. Matches candidate-audit.ts privacy posture. Both verbose
surfaces gate on GBRAIN_SCHEMA_AUDIT_VERBOSE=1 (same env). Logs BOTH
success AND failure events so Phase 9's schema_pack_writability doctor
check has signal to read (closes codex C11). summarizeMutations()
primitive shipped for cross-surface parity between doctor + future
audit CLI.
1.3 registry.ts cache invalidation + stat-mtime TTL (10 cases)
invalidatePackCache(name?) walks the extends-chain reverse-graph
(every cached entry whose chain contains name is evicted). This is the
codex C6 fix — pre-v0.40.6, editing a parent pack silently left
children stale because cache identity was child-bytes-only. New
per-name CacheEntry tracks the file-stat snapshot of every file in
the extends chain. tryCachedPack(name) is the TTL-gated fast path:
inside STAT_TTL_MS (1000ms default, env GBRAIN_PACK_STAT_TTL_MS)
returns cached without statting. Outside the window: stats every file
and cascade-invalidates on any mtime change (D11 cross-process
detection). resolvePack reference-equality preserved on byte-identical
re-build. ASCII state-machine diagram in file header (D9).
1.4 best-effort.ts (4 cases)
loadActivePackBestEffort(ctx) returns ResolvedPack | null. Single
source of truth for the 4 T1.5 wiring sites (Phase 8). null means
"EMPTY FILTER" semantics, NOT "fall back to hardcoded defaults" —
pack-load failure must be loud, per D4. Never throws.
1.5 lint-rules.ts (35 cases)
11 pure rule functions extracted from CLI handlers per codex C13/D16.
9 file-plane rules + 2 DB-aware rules (extractable_empty_corpus, mutation_count_anomaly).
Phase 2 withMutation pre-write gate composes file-plane subset.
runAllLintRules() returns {ok, errors, warnings} structured report
ready for CLI + MCP.
1.6 query-cache-invalidator.ts (4 cases)
invalidateQueryCache(engine, sourceId?) DELETEs query_cache rows so
cached search results bound to old page types don't survive a
schema mutation. Reuses SemanticQueryCache.clear() so we don't
reinvent the PGLite+Postgres parity. Codex C9 fix.
Tests: 84 new cases across 6 test files. All 153 schema-pack tests green.
Plan: ~/.claude/plans/system-instruction-you-are-working-recursive-thacker.md
Closes: half of T2-T7 from the plan's Implementation Tasks JSONL.
Successor to: closed PR #1321 (community PR; author garrytan-agents).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.40.6.0 Phase 2 — mutate.ts withMutation skeleton + 11 primitives
Builds on Phase 1 foundations (pack-lock, mutate-audit, lint-rules,
cache invalidation, query-cache-invalidator).
withMutation(packName, opts, mutator, op, ctx): 8-step skeleton wrapping
every primitive. Atomic .tmp+fsync+rename. Per-pack file lock. Pre-write
file-plane lint validation gate. Audit log on success AND failure. Pack
cache + query cache invalidation hooks. ASCII state-machine diagram in
file header per D9.
11 primitives, each ~5-line wrapper around withMutation:
add_type, remove_type (with codex C14 reference check), update_type
add_alias, remove_alias, add_prefix, remove_prefix
add_link_type (rejects fm_links refs on remove)
remove_link_type, set_extractable, set_expert_routing
Inline minimal JSON→YAML emitter so mutating a YAML pack stays YAML.
The emitter's array-of-mappings nesting was tricky: the first key sits
inline with the `- ` (e.g. `- name: person`), subsequent keys live at
indent+1, and nested arrays inside the mapping keep their relative
depth (the v0.40.6 emitter bug I fixed pre-commit: trim+prefix lost
internal indent of nested arrays like path_prefixes).
YAML round-trip: emitted YAML reparses cleanly through parseYamlMini.
Comments and formatting NOT preserved (documented in plan; pin pack.json
if you care about layout).
Codex C14 reference check: removeType refuses if any other type's
aliases/enrichable_types/link_types/frontmatter_links references the
target. STILL_REFERENCED error names every reference for cleanup.
Validation gate composes runFilePlaneLintRules from Phase 1.5 — a
mutation that would create a dangling ref or prefix collision fails
BEFORE the .tmp write (the invariant: pack file on disk is NEVER
partial).
Tests: 34 cases pinning every primitive + skeleton invariant. Bundled
guard, codex C14, atomicity (crash-mid-write leaves original untouched,
lock auto-released after mutator throw), YAML round-trip, validation
gate firing on prefix collision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.40.6.0 Phase 3 — stats + sync pure core functions
Both ship as runStatsCore / runSyncCore pure functions so Phase 4 CLI
handlers (next commit) and Phase 7 MCP ops (later) both compose without
duplicating logic. Codex C13 / D16 prereq for the MCP exposure phase.
stats.ts (17 cases):
Multi-source aware: sourceIds[] (federated read) OR sourceId (single)
OR neither (whole-brain aggregate). NULLIF(type, '') normalizes
empty-string + NULL to one untyped bucket (pages.type is NOT NULL in
the schema so empty string is the legacy "untyped" representation).
Soft-delete exclusion. by_type sorted by count desc, ties by name asc.
Empty-brain coverage:1.0 (vacuous truth, matches getBrainScore).
Dead-prefix detection: pack-declared prefixes with zero matching
pages surface as DeadPrefixHint[] (agent's drilldown signal for
mis-declared paths). Best-effort: pack-load failure leaves
pack_identity:null + dead_prefixes:[].
sync.ts (13 cases):
D14 chunked UPDATE: 1000-row batches per prefix. Each batch:
WITH win AS (SELECT id FROM pages WHERE untyped+prefix LIMIT $batch),
upd AS (UPDATE ... WHERE id IN win RETURNING 1) SELECT COUNT(*). Loop
until zero rows. Concurrent writers never block on the row-set for
more than ~100ms per batch (vs the multi-second monolithic UPDATE
shape PR #1321 had).
Codex C5 write-side scoping: sourceId param directly, NOT
sourceScopeOpts which is read-side and inherits OAuth federation
reads. Phase 7 MCP op (schema_apply_mutations) enforces at dispatch.
Dry-run by default: per-prefix probe returns would_apply + 10-slug
sample (the drilldown signal). Apply path returns total_applied.
Idempotency contract pinned: second apply finds zero matching rows.
Soft-delete exclusion on both probe + update. Dead-prefix flag set
when probe returns count=0. JSON envelope schema_version:1.
Tests use canonical PGLite block per CLAUDE.md test-isolation rules.
seedPage helper auto-seeds sources(id) row before FK insert.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.40.6.0 Phase 4 — wire 14 new schema CLI verbs
Thin handlers wrapping Phase 2's mutation primitives + Phase 3's
stats/sync cores. CLI is the human surface; Phase 7 wires the same
cores into MCP for agent use.
New verbs:
Authoring:
add-type <name> --primitive P --prefix dir/ [--extractable]
[--expert] [--alias A]* [--pack <name>]
remove-type <name> [--pack <name>]
update-type <name> [--extractable BOOL] [--expert BOOL]
[--primitive P] [--pack <name>]
add-alias <type> <alias> [--pack <name>]
remove-alias <type> <alias> [--pack <name>]
add-prefix <type> <prefix> [--pack <name>]
remove-prefix <type> <prefix> [--pack <name>]
add-link-type <name> [--inverse V] [--page-type T] [--target-type T]
[--pack <name>]
remove-link-type <name> [--pack <name>]
set-extractable <type> BOOL [--pack <name>]
set-expert-routing <type> BOOL [--pack <name>]
Activation:
reload [--pack <name>] Flush in-process cache; --pack scopes
Discovery + repair:
stats [--source <id>] Per-type counts + coverage + dead prefixes
sync [--apply] [--source <id>] Backfill page.type (chunked UPDATE)
cli.ts: schema added to CLI_ONLY_SELF_HELP so `gbrain schema --help`
routes to printHelp() instead of the generic one-line stub.
withConnectedEngine defensive fix retained from PR #1321:
EngineConfig built once and passed to BOTH createEngine and
engine.connect for future-proof against engine implementations that
read URL at connect time.
End-to-end agent journey verified:
fork gbrain-base mine → use mine →
add-type researcher --primitive entity --prefix people/researchers/
--extractable --expert →
active (shows 23 page types) →
stats (shows 100% coverage on empty brain, vacuous truth).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.40.6.0 Phase 5+6+7 — schema lint with rich rules + 9 new MCP ops
This is the marquee commit: Wintermute and any other remote OAuth agent
can now author + introspect schema packs over normal HTTPS MCP. Phases
5+6 collapse into Phase 7 because the new MCP ops compose Phase 1.5's
lint rules and Phase 2/3's mutation/stats/sync cores directly — no
extra extraction needed (D6 from /plan-eng-review).
Phase 5: schema lint CLI wired to runAllLintRules from Phase 1.5
Replaces the prior 2-rule check (duplicate names + missing prefix)
with the full 11-rule suite. New --with-db flag opts into the 2
DB-aware rules (extractable_empty_corpus, mutation_count_anomaly).
JSON envelope shape stable. Exit code 1 on any error.
Phase 7: 9 new MCP operations
Read-scope (NOT localOnly — read scope is safe to expose remote):
get_active_schema_pack — identity packet (pack name, sha8, counts).
list_schema_packs — bundled + installed names.
schema_stats — composes runStatsCore from Phase 3.
schema_lint — composes runAllLintRules; --with-db is
CLI-only (DB-aware rules need engine).
schema_graph — JSON {nodes, edges} from link_types
inference + frontmatter_links.
schema_explain_type — settings for one declared type.
schema_review_orphans — untyped pages drilldown.
Admin-scope (NOT localOnly per D2 — Wintermute reaches via OAuth):
schema_apply_mutations — BATCHED per D10. Single MCP tool taking
a mutations[] array; composes all 11
mutate primitives. Atomic batch_id; outer
withPackLock wraps the whole batch so no
other writer can slip in mid-iteration.
Partial-results returned on mid-batch
failure for forensic agent debugging.
Audit log records actor=mcp:<clientId8>
(D20 privacy-redacted shape).
reload_schema_pack — flush in-process cache + extends-chain
cascade (codex C6 fix from Phase 1.3).
withConnectedEngine defensive fix applied to schema.ts:withConnectedEngine
(PR #1321 closed) — EngineConfig built once and passed to BOTH
createEngine AND engine.connect for defense in depth.
Test seams:
- operationsByName lookup pinned for every new op.
- All 9 ops have scope + localOnly declarations pinned to lock in
the trust posture.
- Batched mutation atomicity tested: partial-failure returns
{error: mutation_failed, partial_results: [...]} with one batch_id
across all results.
- Audit log actor=mcp:<clientId.slice(0,8)> capture verified
end-to-end (audit JSONL read back after the op handler runs).
- Empty mutations[] rejected with invalid_request.
- Unknown op surfaced via SchemaPackMutationError INVALID_RESULT.
Coverage: 23 new cases for the 9 ops (operations-schema-pack.test.ts).
All 255 schema-pack-related tests green.
Plan: ~/.claude/plans/system-instruction-you-are-working-recursive-thacker.md
Successor to: closed PR #1321.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* v0.40.6.0 Phase 8 + 10 + 12 — T1.5 wiring, schema-author skill, ship
Final wave commit. Brings the cathedral from "shipped but undiscoverable"
to "shipped + agents find it + agents use it."
Phase 8 (partial T1.5 wiring — agent-facing surfaces):
- whoknows CLI (src/commands/whoknows.ts:340) consults the active pack
via loadActivePackBestEffort + expertTypesFromPack. Pack-load failure
→ EMPTY filter (NOT hardcoded ['person', 'company'] defaults) per
D4. A researcher type declared --expert in a custom pack now
surfaces in `gbrain whoknows "ML"` results. Pre-v0.40.6 it silently
never matched.
- find_experts MCP op (src/core/operations.ts:2820) same wiring so
OAuth clients (Wintermute etc.) inherit pack-aware expert routing
over HTTP MCP, not just CLI.
- facts/eligibility.ts and enrichment-service.ts union widening
deferred to v0.40.7+ (filed in TODOS.md as 2 follow-up entries) —
larger blast radius than fit this wave's context budget.
Phase 10 (skill + RESOLVER + Convention — the discoverability layer):
- skills/schema-author/SKILL.md — agent dispatcher for "evolve the
schema pack." 36 trigger phrases route here. Explicit Non-goals
section names brain-taxonomist (filing one page) and eiirp
(schema-check during iteration) so agents pick the right surface.
7-phase workflow: brain → assess → propose → apply → sync → verify
→ commit. Lists every gbrain schema CLI verb + every MCP op the
skill uses. brain_first: exempt frontmatter (this skill IS the
brain-first path for schema authoring).
- skills/conventions/schema-evolution.md — decision tree for "when to
add a type vs alias vs prefix." <20 pages → don't pack-codify;
20-100 → alias or narrow prefix; 100+ → first-class type. Don'ts
section + "when to remove a type" + "when to commit the pack" all
answered from one place.
- skills/RESOLVER.md entry with full functional-area dispatcher line
(compressed routing pattern per v0.32.3 dispatcher convention).
- schema-evolution.md added to the cross-cutting Conventions list.
Phase 12 (ship bookkeeping):
- VERSION → 0.40.6.0
- package.json → 0.40.6.0
- CHANGELOG.md entry with ELI10 lead per CLAUDE.md voice rules
(250+ words explaining the wave in plain English before any
file/function name appears), full "To take advantage of v0.40.6.0"
paste-ready commands block, itemized changes by category, credit
to @garrytan-agents (PR #1321 author).
- TODOS.md gains 10 new follow-up entries grouped under
"v0.40.6.0 Schema Cathedral v3 follow-ups (v0.40.7+)" covering:
enrichment-service union widening, facts/eligibility wiring, 3
doctor checks, T16 + T16.1 evals, T19 federated closure, T20
extends merging, T21 YAML comments, T22 admin SPA, T23
schema:write scope, T24 multi-tenant federation.
- llms-full.txt regenerated via bun run build:llms (CLAUDE.md
edits trigger the test/build-llms.test.ts gate — required per
repo discipline).
Verification:
- bun run typecheck clean.
- Full agent journey smoke-tested end-to-end in Phase 4 commit
(fork → use → add-type → active → stats — all green).
- All 255+ schema-pack tests green from Phases 1-7.
Total wave: 6 commits, ~5000 net LOC, 84 new tests, 21 design
decisions captured. PR #1321 closed with successor pointer comment.
Plan: ~/.claude/plans/system-instruction-you-are-working-recursive-thacker.md
Co-Authored-By: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): rename Wintermute → 'your OpenClaw' + add schema-author skill conformance
CI failures from PR #1327 first run:
1. check:privacy script flagged 4 'Wintermute' name leaks (CLAUDE.md:550 rule —
never use the private OpenClaw fork name in public artifacts):
- src/core/operations.ts:3816 → 'your OpenClaw and similar remote agents'
- src/core/operations.ts:4015 → 'your OpenClaw, etc.' (in description)
- src/core/operations.ts:4225 → 'your OpenClaw, etc.' (in comment)
- test/operations-schema-pack.test.ts:325 → clientId 'remoteAgentClient12345678'
(matching audit-actor regex updated: 'mcp:remoteAg' instead of 'mcp:wintermu')
2. skills/manifest.json missing schema-author entry. Added between
brain-taxonomist and skillify per alphabetical-ish grouping.
3. skills/schema-author/SKILL.md missing 3 conformance sections per
test/skills-conformance.test.ts:
- ## Contract (inputs/outputs/side effects/idempotency/trust/atomicity)
- ## Anti-Patterns (don't mutate bundled packs, don't add types for one-off
directories, don't conflate filing vs. schema authoring, etc.)
- ## Output Format (per-mutation JSON, per-batch JSON, stats JSON, sync
dry-run JSON, human format, error envelope codes)
The 3 sections were inserted ABOVE the existing 'Failure modes' section so
the existing failure-mode bullets are still adjacent to the new error
envelope codes in Output Format.
Verified locally:
- bun run check:privacy → clean
- bun test test/skills-conformance.test.ts test/check-resolvable.test.ts test/check-resolvable-cli.test.ts test/regression-v0_22_4.test.ts → 286/286 pass
- bun test test/operations-schema-pack.test.ts → 23/23 pass
- bun run verify → clean (privacy + skill_brain_first + fuzz-purity + typecheck)
llms.txt + llms-full.txt regenerated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: v0.40.7.0 — schema cathedral v3 README + CLAUDE.md annotations
Doc-debt cleanup from the v0.40.7.0 ship (Phase 12 had deferred these to
fit context budget; /document-release surfaced the gap):
- README.md: new "What's new in v0.40.7.0" lead paragraph above the
v0.36.4.0 entry. ELI10 lead: "Your agents can now author your brain's
schema pack themselves" + the agent journey + 14 CLI verbs + 9 MCP
ops + schema-author skill boundary callouts.
- CLAUDE.md: new "Schema Cathedral v3 (v0.40.7.0)" section between the
thin-client routing cluster and the Commands section. 14-bullet
Key Files cluster covering pack-lock / mutate-audit / registry /
best-effort / lint-rules / query-cache-invalidator / mutate / stats /
sync / schema.ts CLI / operations.ts MCP / whoknows T1.5 wiring /
schema-author skill / schema-evolution convention. Each bullet
references the design decisions (D2/D4/D6/D8/D9/D10/D11/D13/D14/D20)
and codex findings (C5/C6/C8/C9/C13/C14) captured during /plan-eng-review.
Closes the "CLAUDE.md has zero v0.40.7.0 mentions" doc debt.
- llms-full.txt + llms.txt regenerated.
Privacy check clean (no Wintermute leaks in the new prose — used "your
OpenClaw" per CLAUDE.md:550 rule). test/build-llms.test.ts 7/7 green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: tutorial — Build your first schema pack (closes v0.40.7+ doc-debt)
Closes the tutorial gap surfaced by /document-release's Diataxis coverage
map. The schema-pack cathedral shipped with reference (CLAUDE.md cluster),
how-to (SKILL.md 7-phase workflow), and explanation (conventions/
schema-evolution.md decision tree), but no tutorial — no concrete
"your first schema mutation" walkthrough.
docs/schema-author-tutorial.md ships exactly that:
- 8 numbered steps, time-to-first-result < 3 (active pack visible by step 2)
- Walks from `gbrain schema fork gbrain-base mine` through `add-type
researcher` + `sync --apply` + proving the T1.5 wiring via `gbrain
whoknows` surfacing the new type
- Every step shows the exact command and expected output
- Placeholder pages (alice-example, bob-example, charlie-example) so any
brain can run the tutorial without affecting real content
- "What you built" section recaps state on disk + active wiring
- "Next steps" cover add-link-type, add-alias, lint --with-db, commit to
source control, MCP path for agents
- "Related docs" cross-links to reference (CLAUDE.md cluster) + how-to
(SKILL.md workflow) + explanation (schema-evolution.md)
Cross-linked:
- README.md "What's new in v0.40.7.0" paragraph gets a "Walkthrough:"
pointer at the end
- skills/schema-author/SKILL.md gets a "## Tutorial" callout just above
the workflow phases — agents that hit the skill via RESOLVER routing
see the tutorial pointer first
Closes the Diataxis quadrant matrix to full coverage:
- Tutorial: ✅ docs/schema-author-tutorial.md (NEW)
- How-to: ✅ skills/schema-author/SKILL.md workflow
- Reference: ✅ CLAUDE.md cluster + gbrain schema --help
- Explanation: ✅ skills/conventions/schema-evolution.md
Privacy check clean. Typecheck clean. llms-full.txt regenerated (545KB).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: what-schemas-unlock — the WHY doc (7 use cases + structural argument)
The schema-author tutorial walks through HOW to mutate a pack. This new
doc explains WHY agents and users should care, with concrete killer use
cases on real corpus shapes:
1. The 4000 invisible meetings — untyped pages skip every structural
surface (whoknows, find_experts, recall, think). Adding a `meeting`
type + sync flips them from invisible to queryable. Same content,
completely different agent experience.
2. The founder ops brain — 4 type-adds + 4 link-types build a
CRM-shaped query surface. `gbrain whoknows "Series A SaaS"` routes
through investor + portco specifically; `graph-query` walks intro
chains. Downstream of notes, not parallel to them.
3. The research brain — researcher / paper / lab / grant / dataset
types + cites / authored / uses link verbs turn a reading-list-as-
markdown into a queryable research graph.
4. The legal brain (or anything where claims have numbers) — typed
`damages=5000000`, `filed_date=...` become comparable across pages
of the same type. Generic note systems can't do this because they
don't know which numbers belong to which type.
5. The team brain — each mounted brain has its own schema pack. Two
engineers searching the same brain get DIFFERENT routing because
their personal packs declare different expert types.
6. The agent-co-curates pattern — the NEW thing in v0.40.7.0. Agent
watches your ingestion stream, runs `gbrain schema detect`
periodically, proposes a new type when a pattern accumulates, applies
it via batched MCP `schema_apply_mutations` after one approval.
Brain learns. Audit log captures the agent's client_id as
`actor: mcp:<clientId8>`.
7. Before-vs-after on real content — pick a corpus, note top-3
whoknows results, add the type via sync, re-run. The numerical
delta IS the win.
Then the structural argument: types matter at query time. Untyped
content is invisible content. The schema is queryable AND mutable AND
auditable — that's the production-system difference from "vibes-based
knowledge management."
Closes with the v0.40.7.0-specific list of what changed (withMutation
skeleton, O_CREAT|O_EXCL atomic lock vs page-lock.ts TOCTOU pattern,
privacy-redacted audit log, 9 MCP ops, T1.5 wiring, cross-process
invalidation via stat-mtime TTL gate).
Cross-linked:
- README.md "What's new in v0.40.7.0" paragraph now has both the
"Why it matters:" pointer (this doc) AND the "Walkthrough:"
pointer (tutorial).
- docs/schema-author-tutorial.md opens with "Want the WHY before the
HOW?" link to this doc.
- skills/schema-author/SKILL.md now has a "Tutorial + vision" section
that points at both, with explicit guidance that agents should read
the WHY doc before pitching schema authoring to a user.
177 lines. Privacy check clean. Typecheck clean. llms-full.txt
regenerated (545KB).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: surface schema docs from README Capabilities + Docs index + llms.txt
The two new schema docs were ONLY linked from the v0.40.7.0 "What's new"
paragraph in README. That paragraph will get pushed down by every future
release and become a worse and worse entry point.
Real discovery paths added:
1. README.md `## Capabilities` section — new "Agent-authored schema
(v0.40.7.0)" bullet between "Brain consistency" and "## Integrations".
Permanent home alongside Hybrid search, Self-wiring graph, Minions,
43 skills, Eval framework. Includes the one-paragraph pitch + 3
pointer links (vision / tutorial / agent skill).
2. README.md `## Docs` index — two new lines added at the top of the
list (right after docs/INSTALL.md, before docs/architecture/):
- docs/what-schemas-unlock.md with one-line description
- docs/schema-author-tutorial.md with one-line description
3. scripts/llms-config.ts `Configuration` section — both docs added to
the curated llms.txt entry list so the LLM-readable map points at
them. Sits right after docs/GBRAIN_RECOMMENDED_SCHEMA.md (topical
grouping). includeInFull defaults to true so they ride in the
single-fetch llms-full.txt bundle.
Result: schema docs are now reachable from 5 entry points instead of 1:
- README "What's new" paragraph (release-pinned, will age out)
- README Capabilities bullet (permanent, top-of-funnel)
- README Docs index (permanent, end-of-page reference)
- llms.txt (LLM-readable curated map)
- llms-full.txt (single-fetch bundle for agents)
Also caught 3 leftover Wintermute leaks in docs/what-schemas-unlock.md
that the privacy check flagged: agent-co-curates pattern now uses "your
OpenClaw"; `register-client wintermute` example renamed to
`register-client my-agent` per CLAUDE.md:550 privacy rule. Privacy
check clean. test/build-llms.test.ts 7/7 green. llms.txt 4314 → 5000
bytes, llms-full.txt 545KB → 572KB.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
1766 lines
75 KiB
TypeScript
Executable File
1766 lines
75 KiB
TypeScript
Executable File
#!/usr/bin/env bun
|
|
|
|
import { installSigchldHandler } from './core/zombie-reap.ts';
|
|
installSigchldHandler();
|
|
|
|
import { readFileSync } from 'fs';
|
|
import { loadConfig, loadConfigWithEngine, toEngineConfig, isThinClient } from './core/config.ts';
|
|
import type { GBrainConfig } from './core/config.ts';
|
|
import type { AIGatewayConfig } from './core/ai/types.ts';
|
|
import type { BrainEngine } from './core/engine.ts';
|
|
import { operations, OperationError } from './core/operations.ts';
|
|
import type { Operation, OperationContext } from './core/operations.ts';
|
|
import { serializeMarkdown } from './core/markdown.ts';
|
|
import { parseGlobalFlags, setCliOptions, getCliOptions } from './core/cli-options.ts';
|
|
import type { CliOptions } from './core/cli-options.ts';
|
|
import { callRemoteTool, RemoteMcpError, unpackToolResult } from './core/mcp-client.ts';
|
|
import { maybePromptForUpgrade } from './core/thin-client-upgrade-prompt.ts';
|
|
import { VERSION } from './version.ts';
|
|
|
|
// Build CLI name -> operation lookup
|
|
const cliOps = new Map<string, Operation>();
|
|
for (const op of operations) {
|
|
const name = op.cliHints?.name;
|
|
if (name && !op.cliHints?.hidden) {
|
|
cliOps.set(name, op);
|
|
}
|
|
}
|
|
|
|
// CLI-only commands that bypass the operation layer
|
|
const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder', 'brainstorm', 'lsd', 'schema', 'capture']);
|
|
// CLI-only commands whose handlers print their own --help text. These are
|
|
// excluded from the generic short-circuit so detailed per-command and
|
|
// per-subcommand usage stays reachable.
|
|
const CLI_ONLY_SELF_HELP = new Set([
|
|
'upgrade', 'post-upgrade', 'check-update',
|
|
'embed', 'config',
|
|
'skillpack', 'skillpack-check',
|
|
'integrations', 'friction',
|
|
'frontmatter', 'check-resolvable',
|
|
'models',
|
|
'cache',
|
|
'brainstorm', 'lsd',
|
|
// v0.39.3.0 WARN-5: capture's detailed HELP constant
|
|
// (src/commands/capture.ts:90+) was unreachable because the dispatcher's
|
|
// generic short-circuit (printCliOnlyHelp at :204-208) fired before
|
|
// runCapture saw --help. brainstorm + lsd were already in the set;
|
|
// capture was the holdout.
|
|
'capture',
|
|
// v0.37 fix wave (Lane D.4 + CDX2-12): sync's --no-embed flag was
|
|
// unreachable via help because the dispatcher's generic CLI-only
|
|
// short-circuit fired before runSync could print its own usage block.
|
|
// Adding `sync` here routes `gbrain sync --help` into runSync.
|
|
'sync',
|
|
// v0.37 fix wave (deferred TODO, shipped): reinit-pglite has its
|
|
// own --help in runReinitPglite. Routing through SELF_HELP avoids
|
|
// the generic short-circuit so the destructive-action warning text
|
|
// reaches the user.
|
|
'reinit-pglite',
|
|
// v0.40.6.0 Schema Cathedral v3 — `gbrain schema --help` should hit
|
|
// schema.ts printHelp() with the full 22+ verb taxonomy, not the
|
|
// generic short-circuit's one-line stub.
|
|
'schema',
|
|
]);
|
|
|
|
async function main() {
|
|
// Parse global flags (--quiet / --progress-json / --progress-interval)
|
|
// BEFORE command dispatch, so `gbrain --progress-json doctor` works.
|
|
// The stripped argv is what the command sees.
|
|
const rawArgs = process.argv.slice(2);
|
|
const { cliOpts, rest: args } = parseGlobalFlags(rawArgs);
|
|
setCliOptions(cliOpts);
|
|
|
|
let command = args[0];
|
|
|
|
if (!command || command === '--help' || command === '-h') {
|
|
printHelp();
|
|
return;
|
|
}
|
|
|
|
if (command === '--version' || command === 'version') {
|
|
console.log(`gbrain ${VERSION}`);
|
|
return;
|
|
}
|
|
|
|
if (command === '--tools-json') {
|
|
const { printToolsJson } = await import('./commands/tools-json.ts');
|
|
printToolsJson();
|
|
return;
|
|
}
|
|
|
|
const subArgs = args.slice(1);
|
|
|
|
// DX alias: `ask` is a natural-language alias for `query`
|
|
if (command === 'ask') {
|
|
command = 'query';
|
|
}
|
|
|
|
// Per-command --help
|
|
if (hasHelpFlag(subArgs)) {
|
|
const op = cliOps.get(command);
|
|
if (op) {
|
|
printOpHelp(op);
|
|
return;
|
|
}
|
|
if (CLI_ONLY.has(command) && !CLI_ONLY_SELF_HELP.has(command)) {
|
|
printCliOnlyHelp(command);
|
|
return;
|
|
}
|
|
}
|
|
|
|
// CLI-only commands
|
|
if (CLI_ONLY.has(command)) {
|
|
await handleCliOnly(command, subArgs);
|
|
return;
|
|
}
|
|
|
|
// Shared operations
|
|
const op = cliOps.get(command);
|
|
if (!op) {
|
|
console.error(`Unknown command: ${command}`);
|
|
console.error('Run gbrain --help for available commands.');
|
|
process.exit(1);
|
|
}
|
|
|
|
// v0.31.1 (Issue #734, CDX-1): parse CLI args BEFORE engine connect so
|
|
// the routing seam below can decide local-vs-remote without paying a
|
|
// PGLite migration replay on thin-client installs. The arg parser, image
|
|
// transform, and required-param check are all engine-free; refactoring
|
|
// them out of the engine try/catch is safe and unlocks routing.
|
|
const params = parseOpArgs(op, subArgs);
|
|
|
|
// v0.27.1 (`gbrain query --image <path>`): swap the `image` param from
|
|
// a filesystem path into base64 bytes + mime. The op accepts base64; the
|
|
// CLI accepts a path. Helper is exported so tests can exercise the
|
|
// transform without spawning a subprocess.
|
|
if (op.name === 'query' && typeof params.image === 'string' && params.image.length > 0) {
|
|
try {
|
|
const { path, base64, mime } = resolveQueryImage(
|
|
params.image as string,
|
|
(params.image_mime as string) || undefined,
|
|
);
|
|
params.image = base64;
|
|
params.image_mime = mime;
|
|
void path;
|
|
} catch (err) {
|
|
console.error(err instanceof Error ? err.message : String(err));
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
// Validate required params before calling handler. v0.27.1: the
|
|
// `query` op's positional `query` is required only when --image is
|
|
// NOT supplied. The runtime altRequired check below overrides the
|
|
// generic required-flag check for that op.
|
|
const queryHasAlt = op.name === 'query' && typeof params.image === 'string' && params.image.length > 0;
|
|
for (const [key, def] of Object.entries(op.params)) {
|
|
if (def.required && params[key] === undefined) {
|
|
if (queryHasAlt && key === 'query') continue;
|
|
const cliName = op.cliHints?.name || op.name;
|
|
const positional = op.cliHints?.positional || [];
|
|
const usage = positional.map(p => `<${p}>`).join(' ');
|
|
console.error(`Usage: gbrain ${cliName} ${usage}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
// v0.31.1 (Issue #734, CDX-1 routing seam): on thin-client installs,
|
|
// route every non-localOnly op through callRemoteTool instead of opening
|
|
// the empty local PGLite. localOnly ops can't run on a thin client at all
|
|
// (no local engine, server intentionally hides them) — refuse with hint.
|
|
// Fix for the silent-empty-results bug class that motivated this whole release.
|
|
const cfgPre = loadConfig();
|
|
if (isThinClient(cfgPre)) {
|
|
if (op.localOnly) {
|
|
refuseThinClient(command, cfgPre!.remote_mcp!.mcp_url);
|
|
}
|
|
await runThinClientRouted(op, params, cfgPre!, cliOpts);
|
|
return;
|
|
}
|
|
|
|
// Local engine path (unchanged behavior for local installs).
|
|
const engine = await connectEngine();
|
|
try {
|
|
const ctx = await makeContext(engine, params);
|
|
const rawResult = await op.handler(ctx, params);
|
|
// ENG-2 (renderer parity by data shape): JSON-round-trip the local-engine
|
|
// path's return value so renderers see the same shape they'd see on the
|
|
// routed path. Date → ISO string; bigint → string (postgres.js shape);
|
|
// Buffer → object. Microsecond-cost; eliminates a whole drift bug class.
|
|
const result = JSON.parse(JSON.stringify(rawResult));
|
|
const output = formatResult(op.name, result);
|
|
if (output) process.stdout.write(output);
|
|
if (op.name === 'query') {
|
|
const { awaitPendingSearchCacheWrites } = await import('./core/search/hybrid.ts');
|
|
await awaitPendingSearchCacheWrites();
|
|
}
|
|
} catch (e: unknown) {
|
|
if (e instanceof OperationError) {
|
|
console.error(`Error [${e.code}]: ${e.message}`);
|
|
if (e.suggestion) console.error(` Fix: ${e.suggestion}`);
|
|
process.exit(1);
|
|
}
|
|
console.error(e instanceof Error ? e.message : String(e));
|
|
process.exit(1);
|
|
} finally {
|
|
await engine.disconnect();
|
|
}
|
|
}
|
|
|
|
function hasHelpFlag(args: string[]): boolean {
|
|
return args.includes('--help') || args.includes('-h');
|
|
}
|
|
|
|
function printCliOnlyHelp(command: string) {
|
|
console.log(`Usage: gbrain ${command}`);
|
|
console.log('');
|
|
console.log(`gbrain ${command} - run gbrain --help for the full command list.`);
|
|
}
|
|
|
|
/**
|
|
* v0.31.1 (Issue #734, CDX-1): route a shared op through the remote MCP
|
|
* server instead of running it locally. Called from main() when
|
|
* `isThinClient(cfg) && !op.localOnly`.
|
|
*
|
|
* Timeout policy (ENG-4): user override via --timeout=Ns wins; otherwise
|
|
* 180s for `think` (LLM calls), 30s for everything else.
|
|
*
|
|
* Error policy (CDX-4): callRemoteTool's hardening pass guarantees every
|
|
* thrown value reaches us as a RemoteMcpError. The switch below is
|
|
* exhaustively typed (TS `never` check); adding a new reason variant fails
|
|
* compilation until this dispatcher knows what to render.
|
|
*
|
|
* Renderer policy: the MCP tool result is unpacked via unpackToolResult
|
|
* (which JSON.parses the text content) and handed to the SAME formatResult
|
|
* the local-engine path uses. Renderer parity is enforced by data shape,
|
|
* not by per-command audit.
|
|
*/
|
|
async function runThinClientRouted(
|
|
op: Operation,
|
|
params: Record<string, unknown>,
|
|
cfg: GBrainConfig,
|
|
cliOpts: CliOptions,
|
|
): Promise<void> {
|
|
// ENG-4: per-op timeout default; user override wins.
|
|
const defaultTimeoutMs = op.name === 'think' ? 180_000 : 30_000;
|
|
const timeoutMs = cliOpts.timeoutMs ?? defaultTimeoutMs;
|
|
|
|
// SIGINT support: aborts in-flight HTTP cleanly (exit 130 is the standard
|
|
// SIGINT exit code; our error switch maps `network/aborted` to that).
|
|
const sigintController = new AbortController();
|
|
const onSigint = () => {
|
|
sigintController.abort(new Error('SIGINT'));
|
|
};
|
|
process.on('SIGINT', onSigint);
|
|
|
|
// v0.31.1 (Issue #734, cherry-pick B): print identity banner to stderr
|
|
// BEFORE the routed call. Banner failure suppresses the banner only —
|
|
// never the underlying command. Suppression honors --quiet, non-TTY,
|
|
// and GBRAIN_NO_BANNER=1.
|
|
await printIdentityBannerBestEffort(cfg, cliOpts, sigintController.signal);
|
|
|
|
try {
|
|
const raw = await callRemoteTool(cfg, op.name, params, {
|
|
timeoutMs,
|
|
signal: sigintController.signal,
|
|
});
|
|
const result = unpackToolResult(raw);
|
|
const output = formatResult(op.name, result);
|
|
if (output) process.stdout.write(output);
|
|
} catch (e: unknown) {
|
|
if (e instanceof RemoteMcpError) {
|
|
const url = cfg.remote_mcp!.mcp_url;
|
|
switch (e.reason) {
|
|
case 'config':
|
|
console.error(e.message);
|
|
break;
|
|
case 'discovery':
|
|
console.error(`OAuth discovery failed at ${cfg.remote_mcp!.issuer_url}.`);
|
|
console.error('Run `gbrain remote doctor` for details.');
|
|
break;
|
|
case 'auth':
|
|
console.error('OAuth auth failed.');
|
|
console.error('On the host, re-register your client:');
|
|
console.error(' gbrain auth register-client <name> --grant-types client_credentials --scopes read,write,admin');
|
|
break;
|
|
case 'auth_after_refresh':
|
|
console.error('OAuth auth failed after token refresh. Credentials may have been revoked.');
|
|
console.error('Run `gbrain remote doctor` to confirm.');
|
|
break;
|
|
case 'network':
|
|
if (e.detail?.kind === 'timeout') {
|
|
const hint = cliOpts.timeoutMs ? '' : ` (default ${defaultTimeoutMs}ms; pass --timeout=Ns to override)`;
|
|
console.error(`Request to ${url} timed out${hint}.`);
|
|
} else if (e.detail?.kind === 'aborted') {
|
|
console.error('Request aborted.');
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(130);
|
|
} else {
|
|
console.error(`Cannot reach ${url}. Run \`gbrain remote doctor\` for details.`);
|
|
}
|
|
break;
|
|
case 'tool_error':
|
|
if (e.detail?.code === 'missing_scope') {
|
|
console.error('Missing OAuth scope on this client.');
|
|
console.error('On the host, re-register the client with broader scopes:');
|
|
console.error(' gbrain auth register-client <name> --grant-types client_credentials --scopes read,write,admin');
|
|
} else {
|
|
console.error(e.message);
|
|
console.error('Run `gbrain remote doctor` if this persists.');
|
|
}
|
|
break;
|
|
case 'parse':
|
|
console.error('Server response was malformed. Run `gbrain remote doctor`.');
|
|
break;
|
|
default: {
|
|
// Exhaustive switch sentinel (TS `never` — fails to build if a
|
|
// new RemoteMcpErrorReason variant is added without a case).
|
|
const _exhaustive: never = e.reason;
|
|
void _exhaustive;
|
|
console.error(`Unhandled remote error: ${e.message}`);
|
|
}
|
|
}
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(1);
|
|
}
|
|
// Defense in depth: callRemoteTool's contract is that everything is
|
|
// RemoteMcpError. If a plain Error escapes, render it generically and
|
|
// exit 1 — but this should never happen post-CDX-4.
|
|
console.error(e instanceof Error ? e.message : String(e));
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(1);
|
|
} finally {
|
|
process.off('SIGINT', onSigint);
|
|
}
|
|
}
|
|
|
|
// ============================================================================
|
|
// v0.31.1 (Issue #734, cherry-pick B): thin-client identity banner.
|
|
//
|
|
// Prints "[thin-client → <host> · brain: 102k pages, 265k chunks · vX.Y.Z]"
|
|
// to stderr before each routed command, so users (and agents) know they're
|
|
// talking to a real remote brain — not the empty local PGLite that motivated
|
|
// this whole release.
|
|
//
|
|
// Cache: 60s TTL, in-memory Map keyed by mcp_url. Cross-process file cache
|
|
// is deferred (marginal benefit; one mint per CLI process is fine).
|
|
// Suppression: --quiet, non-TTY, GBRAIN_NO_BANNER=1.
|
|
// Failure mode: any error in fetching identity → suppress banner; underlying
|
|
// command runs normally. Banner is observability, not load-bearing.
|
|
// ============================================================================
|
|
|
|
export interface BrainIdentity {
|
|
version: string;
|
|
engine: 'postgres' | 'pglite';
|
|
page_count: number;
|
|
chunk_count: number;
|
|
last_sync_iso: string | null;
|
|
}
|
|
|
|
interface CachedIdentity {
|
|
identity: BrainIdentity;
|
|
cached_at_ms: number;
|
|
}
|
|
|
|
const IDENTITY_TTL_MS = 60_000;
|
|
const identityCache = new Map<string, CachedIdentity>();
|
|
|
|
/** Test-only escape hatch — clears the in-memory cache between test runs. */
|
|
export function _clearIdentityCacheForTest(): void {
|
|
identityCache.clear();
|
|
}
|
|
|
|
export function bannerSuppressed(cliOpts: CliOptions): boolean {
|
|
if (cliOpts.quiet) return true;
|
|
if (process.env.GBRAIN_NO_BANNER === '1') return true;
|
|
// Non-TTY default is suppressed (clean pipes); explicit env-flag overrides.
|
|
if (!process.stderr.isTTY && process.env.GBRAIN_BANNER !== '1') return true;
|
|
return false;
|
|
}
|
|
|
|
function formatPageCount(n: number): string {
|
|
if (n >= 1000) {
|
|
const k = (n / 1000).toFixed(n >= 100_000 ? 0 : 1);
|
|
return `${k}k`;
|
|
}
|
|
return String(n);
|
|
}
|
|
|
|
function formatBanner(mcpUrl: string, id: BrainIdentity): string {
|
|
const host = mcpUrl.replace(/^https?:\/\//, '').split('/')[0];
|
|
const counts = `brain: ${formatPageCount(id.page_count)} pages, ${formatPageCount(id.chunk_count)} chunks`;
|
|
return `[thin-client → ${host} · ${counts} · v${id.version}]`;
|
|
}
|
|
|
|
async function fetchIdentity(
|
|
cfg: GBrainConfig,
|
|
signal: AbortSignal,
|
|
): Promise<BrainIdentity> {
|
|
// 2s timeout for the banner fetch — must not delay the underlying command.
|
|
const raw = await callRemoteTool(cfg, 'get_brain_identity', {}, {
|
|
timeoutMs: 2000,
|
|
signal,
|
|
});
|
|
const id = unpackToolResult<BrainIdentity>(raw);
|
|
return id;
|
|
}
|
|
|
|
async function printIdentityBannerBestEffort(
|
|
cfg: GBrainConfig,
|
|
cliOpts: CliOptions,
|
|
signal: AbortSignal,
|
|
): Promise<void> {
|
|
if (bannerSuppressed(cliOpts)) return;
|
|
const mcpUrl = cfg.remote_mcp?.mcp_url;
|
|
if (!mcpUrl) return;
|
|
|
|
// Cache lookup keyed by mcp_url so switching hosts via `gbrain init`
|
|
// invalidates cleanly even within a long-lived process.
|
|
const cached = identityCache.get(mcpUrl);
|
|
if (cached && Date.now() - cached.cached_at_ms < IDENTITY_TTL_MS) {
|
|
process.stderr.write(formatBanner(mcpUrl, cached.identity) + '\n');
|
|
// v0.31.11: detect remote-version drift, prompt user to upgrade.
|
|
// bannerIsSuppressed=false here — the early return above guaranteed it.
|
|
await maybePromptForUpgrade(cfg, cached.identity, cliOpts, false);
|
|
return;
|
|
}
|
|
|
|
// Cache miss — fetch. Failure is non-fatal: banner is observability,
|
|
// never load-bearing for the underlying command.
|
|
try {
|
|
const id = await fetchIdentity(cfg, signal);
|
|
identityCache.set(mcpUrl, { identity: id, cached_at_ms: Date.now() });
|
|
process.stderr.write(formatBanner(mcpUrl, id) + '\n');
|
|
// v0.31.11: detect remote-version drift, prompt user to upgrade.
|
|
await maybePromptForUpgrade(cfg, id, cliOpts, false);
|
|
} catch {
|
|
// Swallow. Banner suppressed; main command continues. The CDX-4
|
|
// hardened callRemoteTool will surface the same error class on the
|
|
// actual command call if the host is genuinely unreachable.
|
|
}
|
|
}
|
|
|
|
/**
|
|
* v0.27.1: shared transform for `gbrain query --image <path>` (and any future
|
|
* CLI surface that takes an image path). Reads the file, base64-encodes,
|
|
* derives MIME from the extension, enforces the 20MB cap. Exported so tests
|
|
* can verify the transform without spawning a subprocess.
|
|
*
|
|
* Throws Error on any failure (file missing, oversized, etc.). Caller is
|
|
* responsible for routing to process.exit(1) with a user-facing message.
|
|
*/
|
|
export function resolveQueryImage(
|
|
imagePath: string,
|
|
explicitMime?: string,
|
|
): { path: string; base64: string; mime: string } {
|
|
const bytes = readFileSync(imagePath);
|
|
if (bytes.length > 20 * 1024 * 1024) {
|
|
throw new Error(`Error: image too large (${bytes.length} bytes, max 20MB).`);
|
|
}
|
|
const base64 = bytes.toString('base64');
|
|
let mime = explicitMime;
|
|
if (!mime) {
|
|
const lower = imagePath.toLowerCase();
|
|
const mimeFromExt: Record<string, string> = {
|
|
'.png': 'image/png',
|
|
'.jpg': 'image/jpeg', '.jpeg': 'image/jpeg',
|
|
'.gif': 'image/gif',
|
|
'.webp': 'image/webp',
|
|
'.heic': 'image/heic', '.heif': 'image/heif',
|
|
'.avif': 'image/avif',
|
|
};
|
|
const ext = Object.keys(mimeFromExt).find(e => lower.endsWith(e));
|
|
mime = ext ? mimeFromExt[ext] : 'image/jpeg';
|
|
}
|
|
return { path: imagePath, base64, mime };
|
|
}
|
|
|
|
export function parseOpArgs(op: Operation, args: string[]): Record<string, unknown> {
|
|
const params: Record<string, unknown> = {};
|
|
const positional = op.cliHints?.positional || [];
|
|
let posIdx = 0;
|
|
|
|
for (let i = 0; i < args.length; i++) {
|
|
const arg = args[i];
|
|
if (arg.startsWith('--')) {
|
|
if (arg.startsWith('--no-')) {
|
|
const positiveKey = arg.slice(5).replace(/-/g, '_');
|
|
const positiveDef = op.params[positiveKey];
|
|
if (positiveDef?.type === 'boolean') {
|
|
params[positiveKey] = false;
|
|
continue;
|
|
}
|
|
}
|
|
const key = arg.slice(2).replace(/-/g, '_');
|
|
const paramDef = op.params[key];
|
|
if (paramDef?.type === 'boolean') {
|
|
params[key] = true;
|
|
} else if (i + 1 < args.length) {
|
|
params[key] = args[++i];
|
|
if (paramDef?.type === 'number') params[key] = Number(params[key]);
|
|
}
|
|
} else if (posIdx < positional.length) {
|
|
const key = positional[posIdx++];
|
|
const paramDef = op.params[key];
|
|
params[key] = paramDef?.type === 'number' ? Number(arg) : arg;
|
|
}
|
|
}
|
|
|
|
// Read stdin for content params
|
|
if (op.cliHints?.stdin && !params[op.cliHints.stdin] && !process.stdin.isTTY) {
|
|
const stdinContent = readFileSync('/dev/stdin', 'utf-8');
|
|
const MAX_STDIN = 5_000_000; // 5MB
|
|
if (Buffer.byteLength(stdinContent, 'utf-8') > MAX_STDIN) {
|
|
console.error(`Error: stdin content exceeds ${MAX_STDIN} bytes. Split into smaller inputs.`);
|
|
process.exit(1);
|
|
}
|
|
params[op.cliHints.stdin] = stdinContent;
|
|
}
|
|
|
|
return params;
|
|
}
|
|
|
|
async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
|
|
// v0.31.8 (D11): resolve sourceId via the canonical 6-tier chain. Honors
|
|
// --source / GBRAIN_SOURCE / .gbrain-source / path-match / brain default /
|
|
// 'default'. Wrapped in try/catch so a doctor / single-source brain that
|
|
// never set up sources still returns 'default' silently.
|
|
let sourceId: string | undefined;
|
|
try {
|
|
const { resolveSourceId } = await import('./core/source-resolver.ts');
|
|
// params.source is set when a CLI flag was parsed for the op (rare; most
|
|
// CLI ops don't take --source). Falls through to env/dotfile/path-match.
|
|
const explicit = (params.source as string | undefined) ?? null;
|
|
sourceId = await resolveSourceId(engine, explicit);
|
|
} catch {
|
|
// Source resolution failed (e.g. sources table doesn't exist on a fresh
|
|
// pre-init brain). Leave sourceId unset; engine read methods fall through
|
|
// to the cross-source view (D16 back-compat path).
|
|
sourceId = undefined;
|
|
}
|
|
return {
|
|
engine,
|
|
config: loadConfig() || { engine: 'postgres' },
|
|
logger: { info: console.log, warn: console.warn, error: console.error },
|
|
dryRun: (params.dry_run as boolean) || false,
|
|
// Local CLI invocation — the user owns the machine; do not apply remote-caller
|
|
// confinement (e.g., cwd-locked file_upload).
|
|
remote: false,
|
|
cliOpts: getCliOptions(),
|
|
// v0.34 D4: sourceId is REQUIRED at the type level. Fall back to 'default'
|
|
// when resolveSourceId returned undefined (fresh pre-init brain, no sources
|
|
// table). Matches dispatch.ts's auto-fill so the contract holds across
|
|
// every transport.
|
|
sourceId: sourceId ?? 'default',
|
|
};
|
|
}
|
|
|
|
function formatResult(opName: string, result: unknown): string {
|
|
switch (opName) {
|
|
case 'get_page': {
|
|
const r = result as any;
|
|
if (r.error === 'ambiguous_slug') {
|
|
return `Ambiguous slug. Did you mean:\n${r.candidates.map((c: string) => ` ${c}`).join('\n')}\n`;
|
|
}
|
|
return serializeMarkdown(r.frontmatter || {}, r.compiled_truth || '', r.timeline || '', {
|
|
type: r.type, title: r.title, tags: r.tags || [],
|
|
});
|
|
}
|
|
case 'list_pages': {
|
|
const pages = result as any[];
|
|
if (pages.length === 0) return 'No pages found.\n';
|
|
return pages.map(p =>
|
|
`${p.slug}\t${p.type}\t${p.updated_at?.toString().slice(0, 10) || '?'}\t${p.title}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'search':
|
|
case 'query': {
|
|
const results = result as any[];
|
|
if (results.length === 0) return 'No results.\n';
|
|
// v0.40.4 — --explain switches to per-stage attribution formatter.
|
|
// Reads CliOptions.explain via the module-level singleton.
|
|
const cliOpts = getCliOptions();
|
|
if (cliOpts.explain) {
|
|
// Lazy import keeps formatResult's startup hot path narrow for
|
|
// the common non-explain case.
|
|
const { formatResultsExplain } = require('./core/search/explain-formatter.ts');
|
|
return formatResultsExplain(results);
|
|
}
|
|
return results.map(r =>
|
|
`[${r.score?.toFixed(4) || '?'}] ${r.slug} -- ${r.chunk_text?.slice(0, 100) || ''}${r.stale ? ' (stale)' : ''}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'get_tags': {
|
|
const tags = result as string[];
|
|
return tags.length > 0 ? tags.join(', ') + '\n' : 'No tags.\n';
|
|
}
|
|
case 'get_stats': {
|
|
const s = result as any;
|
|
const lines = [
|
|
`Pages: ${s.page_count}`,
|
|
`Chunks: ${s.chunk_count}`,
|
|
`Embedded: ${s.embedded_count}`,
|
|
`Links: ${s.link_count}`,
|
|
`Tags: ${s.tag_count}`,
|
|
`Timeline: ${s.timeline_entry_count}`,
|
|
];
|
|
if (s.pages_by_type) {
|
|
lines.push('', 'By type:');
|
|
for (const [k, v] of Object.entries(s.pages_by_type)) {
|
|
lines.push(` ${k}: ${v}`);
|
|
}
|
|
}
|
|
return lines.join('\n') + '\n';
|
|
}
|
|
case 'get_health': {
|
|
const h = result as any;
|
|
// Health score weights: missing_embeddings is the heaviest (2 pts), other
|
|
// graph quality issues are 1 pt each. link_coverage / timeline_coverage below
|
|
// 50% on entity pages indicates the graph needs population.
|
|
const score = Math.max(0, 10
|
|
- (h.missing_embeddings > 0 ? 2 : 0)
|
|
- (h.stale_pages > 0 ? 1 : 0)
|
|
- (h.orphan_pages > 0 ? 1 : 0)
|
|
- ((h.link_coverage ?? 1) < 0.5 ? 1 : 0)
|
|
- ((h.timeline_coverage ?? 1) < 0.5 ? 1 : 0));
|
|
const lines = [
|
|
`Health score: ${score}/10`,
|
|
`Embed coverage: ${(h.embed_coverage * 100).toFixed(1)}%`,
|
|
`Missing embeddings: ${h.missing_embeddings}`,
|
|
`Stale pages: ${h.stale_pages}`,
|
|
`Orphan pages: ${h.orphan_pages}`,
|
|
];
|
|
if (h.link_coverage !== undefined) {
|
|
lines.push(`Link coverage (entities): ${(h.link_coverage * 100).toFixed(1)}%`);
|
|
}
|
|
if (h.timeline_coverage !== undefined) {
|
|
lines.push(`Timeline coverage (entities): ${(h.timeline_coverage * 100).toFixed(1)}%`);
|
|
}
|
|
if (Array.isArray(h.most_connected) && h.most_connected.length > 0) {
|
|
lines.push('Most connected entities:');
|
|
for (const e of h.most_connected) {
|
|
lines.push(` ${e.slug}: ${e.link_count} links`);
|
|
}
|
|
}
|
|
return lines.join('\n') + '\n';
|
|
}
|
|
case 'get_timeline': {
|
|
const entries = result as any[];
|
|
if (entries.length === 0) return 'No timeline entries.\n';
|
|
return entries.map(e =>
|
|
`${e.date} ${e.summary}${e.source ? ` [${e.source}]` : ''}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'get_versions': {
|
|
const versions = result as any[];
|
|
if (versions.length === 0) return 'No versions.\n';
|
|
return versions.map(v =>
|
|
`#${v.id} ${v.snapshot_at?.toString().slice(0, 19) || '?'} ${v.compiled_truth?.slice(0, 60) || ''}...`,
|
|
).join('\n') + '\n';
|
|
}
|
|
default:
|
|
return JSON.stringify(result, null, 2) + '\n';
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Multi-topology v1: thin-client refusal set. These commands require a local
|
|
* engine; if `~/.gbrain/config.json` has `remote_mcp` set, the dispatch guard
|
|
* refuses them with a canonical error pointing at the remote host. The check
|
|
* runs before per-command dispatch so the error message is consistent.
|
|
*
|
|
* `serve` is in this set because `gbrain serve` (stdio or http) requires a
|
|
* local engine to expose. Thin clients don't have one to expose.
|
|
*
|
|
* `doctor` is intentionally NOT in this set — task 4 routes it to
|
|
* `runRemoteDoctor` for thin-client installs.
|
|
*/
|
|
const THIN_CLIENT_REFUSED_COMMANDS = new Set([
|
|
'sync', 'embed', 'extract', 'migrate', 'apply-migrations',
|
|
'repair-jsonb', 'orphans', 'integrity', 'serve',
|
|
// v0.31.1 (CDX-2 op coverage matrix): more local-only commands
|
|
'dream', 'transcripts', 'storage',
|
|
// v0.31.1 CDX-2 audit: takes/sources have multiple subcommands; some
|
|
// (takes_list/takes_search, sources_list/sources_status) have MCP
|
|
// equivalents and others are file-system bound (takes mutate commands
|
|
// edit local .md files). v0.31.1 refuses both at the top level with a
|
|
// hint pointing at the routable MCP tools; per-subcommand splits are
|
|
// a v0.31.x follow-up TODO.
|
|
'takes', 'sources',
|
|
// v0.32 thin-client routing audit (Codex round 2 findings #2, #4):
|
|
// - `pages` purge-deleted is admin+localOnly (operations.ts:856-864)
|
|
// - `files` list / file_url MCP ops are localOnly (operations.ts:1769-1879)
|
|
// - `eval` export/prune/replay have no MCP equivalents
|
|
// - `code-def`/`code-refs`/`code-callers`/`code-callees` have NO MCP ops
|
|
// in operations.ts:2630-2671; cannot be "fixed by routing" yet
|
|
'pages', 'files', 'eval', 'code-def', 'code-refs', 'code-callers', 'code-callees',
|
|
]);
|
|
|
|
/**
|
|
* v0.31.1 (Issue #734, CDX-5 + cherry-pick A): pinpoint refusal hints for
|
|
* local-only commands when running on a thin-client install. Each hint names
|
|
* the closest path (remote MCP call, host-side workflow) so users aren't
|
|
* stuck guessing what to do next.
|
|
*
|
|
* Source-of-truth lives here so adding a new local-only command means
|
|
* adding both the THIN_CLIENT_REFUSED_COMMANDS member AND the hint in one
|
|
* place during code review.
|
|
*/
|
|
const THIN_CLIENT_REFUSE_HINTS: Record<string, string> = {
|
|
sync: 'sync runs on the host. Trigger a remote cycle with `gbrain remote ping` (queues an autopilot-cycle job).',
|
|
embed: 'embed runs on the host as part of the autopilot cycle. `gbrain remote ping` triggers a full cycle including embed.',
|
|
extract: 'extract runs on the host. Use `gbrain remote ping` to trigger a cycle including extract.',
|
|
migrate: "migrate runs on the host's local engine. Run on the host machine.",
|
|
'apply-migrations': 'schema migrations run on the host. SSH and run there.',
|
|
'repair-jsonb': 'repair-jsonb operates on the local DB only.',
|
|
integrity: 'integrity scans local files. Run on the host machine.',
|
|
serve: 'serve starts a server. Run on the host, not the thin client.',
|
|
dream: 'dream runs the autopilot cycle on the host. `gbrain remote ping` queues one. (Native `gbrain dream` thin-client routing planned for v0.31.2.)',
|
|
orphans: "orphans needs the host's brain. Run on the host or use the `find_orphans` MCP tool from your agent.",
|
|
transcripts: 'transcripts is server-private (raw chat exports stay on the host). Read transcripts on the host machine.',
|
|
storage: 'storage operates on the local repo on disk. Run on the host.',
|
|
takes: 'takes mutate subcommands edit local .md files; routing the read subcommands lands in v0.31.x. For now: use `takes_list` and `takes_search` MCP tools from your agent, or run on the host.',
|
|
sources: 'sources commands manage local DB + config rows. Per-subcommand thin-client routing lands in v0.31.x. For now: use `sources_list` / `sources_status` MCP tools, or run on the host.',
|
|
// v0.32 audit additions
|
|
pages: '`pages purge-deleted` is admin+localOnly (hard-deletes from the local DB). Run on the host.',
|
|
files: '`files list` and `files url` MCP ops are localOnly (paths live on the host filesystem). Use `gbrain files` on the host machine.',
|
|
eval: '`eval` export/prune/replay touch the local engine and have no MCP equivalents. Run `gbrain eval` on the host.',
|
|
'code-def': '`code-def` needs symbol-aware lookup that has no MCP op yet. Run on the host or use `search` from your agent with a symbol-shaped query.',
|
|
'code-refs': '`code-refs` has no MCP op yet. Run on the host.',
|
|
'code-callers': '`code-callers` has no MCP op yet. Run on the host.',
|
|
'code-callees': '`code-callees` has no MCP op yet. Run on the host.',
|
|
};
|
|
|
|
/**
|
|
* v0.31.1: emit a pinpoint refusal hint for a thin-client-incompatible
|
|
* command and exit 1. Falls back to the canonical generic message when no
|
|
* specific hint is registered (defensive — every member of
|
|
* THIN_CLIENT_REFUSED_COMMANDS should have a hint).
|
|
*/
|
|
function refuseThinClient(command: string, mcpUrl: string): never {
|
|
const hint = THIN_CLIENT_REFUSE_HINTS[command];
|
|
if (hint) {
|
|
console.error(`\`gbrain ${command}\` is not routable. ${hint}`);
|
|
console.error(`(thin-client of ${mcpUrl})`);
|
|
} else {
|
|
console.error(
|
|
`\`gbrain ${command}\` requires a local engine. This install is a thin client of ${mcpUrl}.\n` +
|
|
`Run \`${command}\` on the remote host, or use the corresponding MCP tool from your agent.`,
|
|
);
|
|
}
|
|
process.exit(1);
|
|
}
|
|
|
|
async function handleCliOnly(command: string, args: string[]) {
|
|
// Thin-client guard: refuse DB-bound commands cleanly with a pinpoint
|
|
// hint instead of letting them fail later inside connectEngine or
|
|
// mid-handler. v0.31.1 routes through `refuseThinClient` so every
|
|
// refusal carries an actionable next-step hint (CDX-5 cherry-pick A).
|
|
if (THIN_CLIENT_REFUSED_COMMANDS.has(command)) {
|
|
const cfg = loadConfig();
|
|
if (isThinClient(cfg)) {
|
|
refuseThinClient(command, cfg!.remote_mcp!.mcp_url);
|
|
}
|
|
}
|
|
|
|
// Commands that don't need a database connection
|
|
if (command === 'schema') {
|
|
const { runSchema } = await import('./commands/schema.ts');
|
|
await runSchema(args);
|
|
return;
|
|
}
|
|
if (command === 'init') {
|
|
const { runInit } = await import('./commands/init.ts');
|
|
await runInit(args);
|
|
return;
|
|
}
|
|
// v0.37 fix wave (deferred TODO, shipped): one-command wipe-and-reinit.
|
|
// Spawns its own engine internally so no pre-bound engine needed.
|
|
if (command === 'reinit-pglite') {
|
|
const { runReinitPglite } = await import('./commands/reinit-pglite.ts');
|
|
await runReinitPglite(args);
|
|
return;
|
|
}
|
|
if (command === 'auth') {
|
|
const { runAuth } = await import('./commands/auth.ts');
|
|
await runAuth(args);
|
|
return;
|
|
}
|
|
if (command === 'remote') {
|
|
// Multi-topology v1 (Tier B): thin-client-only convenience commands.
|
|
// `runRemote` self-checks for remote_mcp config and exits 1 if local-only.
|
|
const { runRemote } = await import('./commands/remote.ts');
|
|
await runRemote(args);
|
|
return;
|
|
}
|
|
if (command === 'upgrade') {
|
|
const { runUpgrade } = await import('./commands/upgrade.ts');
|
|
await runUpgrade(args);
|
|
return;
|
|
}
|
|
if (command === 'post-upgrade') {
|
|
const { runPostUpgrade } = await import('./commands/upgrade.ts');
|
|
await runPostUpgrade(args);
|
|
return;
|
|
}
|
|
if (command === 'check-update') {
|
|
const { runCheckUpdate } = await import('./commands/check-update.ts');
|
|
await runCheckUpdate(args);
|
|
return;
|
|
}
|
|
if (command === 'integrations') {
|
|
const { runIntegrations } = await import('./commands/integrations.ts');
|
|
await runIntegrations(args);
|
|
return;
|
|
}
|
|
if (command === 'providers') {
|
|
const { runProviders } = await import('./commands/providers.ts');
|
|
const [sub, ...rest] = args;
|
|
await runProviders(sub, rest);
|
|
return;
|
|
}
|
|
if (command === 'auth') {
|
|
const { runAuth } = await import('./commands/auth.ts');
|
|
await runAuth(args);
|
|
return;
|
|
}
|
|
if (command === 'resolvers') {
|
|
const { runResolvers } = await import('./commands/resolvers.ts');
|
|
await runResolvers(args);
|
|
return;
|
|
}
|
|
if (command === 'integrity') {
|
|
const { runIntegrity } = await import('./commands/integrity.ts');
|
|
await runIntegrity(args);
|
|
return;
|
|
}
|
|
if (command === 'publish') {
|
|
const { runPublish } = await import('./commands/publish.ts');
|
|
await runPublish(args);
|
|
return;
|
|
}
|
|
if (command === 'check-backlinks') {
|
|
const { runBacklinks } = await import('./commands/backlinks.ts');
|
|
await runBacklinks(args);
|
|
return;
|
|
}
|
|
if (command === 'frontmatter') {
|
|
const { runFrontmatter } = await import('./commands/frontmatter.ts');
|
|
await runFrontmatter(args);
|
|
return;
|
|
}
|
|
if (command === 'lint') {
|
|
const { runLint } = await import('./commands/lint.ts');
|
|
await runLint(args);
|
|
return;
|
|
}
|
|
if (command === 'check-resolvable') {
|
|
const { runCheckResolvable } = await import('./commands/check-resolvable.ts');
|
|
await runCheckResolvable(args);
|
|
return;
|
|
}
|
|
if (command === 'mounts') {
|
|
// No DB needed: mounts.json is a local config file. Registry will
|
|
// connect mount engines lazily on first use by op dispatch.
|
|
const { runMounts } = await import('./commands/mounts.ts');
|
|
await runMounts(args);
|
|
return;
|
|
}
|
|
if (command === 'cache') {
|
|
// v0.32.x search-lite: semantic query cache management. Dispatch the
|
|
// subcommand handler (stats / clear / prune); the handler opens its
|
|
// own engine connection.
|
|
const { runCache } = await import('./commands/cache.ts');
|
|
await runCache(args);
|
|
return;
|
|
}
|
|
if (command === 'routing-eval') {
|
|
const { runRoutingEvalCli } = await import('./commands/routing-eval.ts');
|
|
await runRoutingEvalCli(args);
|
|
return;
|
|
}
|
|
if (command === 'skillify') {
|
|
const { runSkillify } = await import('./commands/skillify.ts');
|
|
// `args` here is subArgs (command already stripped by caller), so
|
|
// args[0] is the subcommand (scaffold|check).
|
|
await runSkillify(args);
|
|
return;
|
|
}
|
|
if (command === 'skillpack') {
|
|
const { runSkillpack } = await import('./commands/skillpack.ts');
|
|
// subArgs already has `skillpack` stripped; args[0] is the subcommand.
|
|
await runSkillpack(args);
|
|
return;
|
|
}
|
|
if (command === 'friction') {
|
|
const { runFriction } = await import('./commands/friction.ts');
|
|
process.exit(runFriction(args));
|
|
}
|
|
if (command === 'claw-test') {
|
|
const { runClawTest } = await import('./commands/claw-test.ts');
|
|
process.exit(await runClawTest(args));
|
|
}
|
|
if (command === 'report') {
|
|
const { runReport } = await import('./commands/report.ts');
|
|
await runReport(args);
|
|
return;
|
|
}
|
|
if (command === 'apply-migrations') {
|
|
// Does not need connectEngine — each phase (schema, smoke, host-rewrite)
|
|
// manages its own subprocess or file-layer access directly. Avoids
|
|
// connecting a second time when the orchestrator shells out to
|
|
// `gbrain init --migrate-only` and `gbrain jobs smoke`.
|
|
const { runApplyMigrations } = await import('./commands/apply-migrations.ts');
|
|
await runApplyMigrations(args);
|
|
return;
|
|
}
|
|
if (command === 'repair-jsonb') {
|
|
const { runRepairJsonbCli } = await import('./commands/repair-jsonb.ts');
|
|
await runRepairJsonbCli(args);
|
|
return;
|
|
}
|
|
if (command === 'skillpack-check') {
|
|
// Agent-readable health report. Shells out to doctor + apply-migrations
|
|
// internally; does not need its own DB connection.
|
|
const { runSkillpackCheck } = await import('./commands/skillpack-check.ts');
|
|
await runSkillpackCheck(args);
|
|
return;
|
|
}
|
|
if (command === 'doctor') {
|
|
// Multi-topology v1: thin-client doctor. When `~/.gbrain/config.json`
|
|
// has remote_mcp set, every DB-bound check is irrelevant. Route to the
|
|
// outbound-HTTP probe set in `src/core/doctor-remote.ts` and return
|
|
// before any local-engine work.
|
|
const cfgForDoctor = loadConfig();
|
|
if (isThinClient(cfgForDoctor)) {
|
|
const { runRemoteDoctor } = await import('./core/doctor-remote.ts');
|
|
await runRemoteDoctor(cfgForDoctor!, args);
|
|
return;
|
|
}
|
|
|
|
// v0.36+ brain-health-100: --remediation-plan and --remediate go
|
|
// through dedicated functions that compute from engine.getHealth()
|
|
// (cheap path D7), NOT the full doctor walk.
|
|
if (args.includes('--remediation-plan')) {
|
|
const { runRemediationPlan } = await import('./commands/doctor.ts');
|
|
const eng = await connectEngine();
|
|
try { await runRemediationPlan(eng, args); } finally { await eng.disconnect(); }
|
|
return;
|
|
}
|
|
if (args.includes('--remediate')) {
|
|
const { runRemediate } = await import('./commands/doctor.ts');
|
|
const eng = await connectEngine();
|
|
try { await runRemediate(eng, args); } finally { await eng.disconnect(); }
|
|
return;
|
|
}
|
|
|
|
// Doctor runs filesystem checks first (no DB needed), then DB checks.
|
|
// --fast skips DB checks entirely.
|
|
const { runDoctor } = await import('./commands/doctor.ts');
|
|
const { getDbUrlSource } = await import('./core/config.ts');
|
|
if (args.includes('--fast')) {
|
|
// Pass the DB URL source so doctor can tell "no config at all" from
|
|
// "user chose --fast while config is present".
|
|
await runDoctor(null, args, getDbUrlSource());
|
|
} else {
|
|
try {
|
|
const eng = await connectEngine();
|
|
await runDoctor(eng, args);
|
|
await eng.disconnect();
|
|
} catch {
|
|
// DB unavailable — still run filesystem checks
|
|
await runDoctor(null, args, getDbUrlSource());
|
|
}
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'ze-switch') {
|
|
// v0.36.0.0 — manual ZE-default switch lever. Owns its own engine lifecycle
|
|
// to mirror the doctor pattern.
|
|
const { runZeSwitch } = await import('./commands/ze-switch.ts');
|
|
const eng = await connectEngine();
|
|
try {
|
|
await runZeSwitch(args, eng);
|
|
} finally {
|
|
await eng.disconnect();
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'smoke-test') {
|
|
// Run smoke tests — no DB connection needed, the script handles its own checks
|
|
const { execSync } = await import('child_process');
|
|
const { resolve, dirname } = await import('path');
|
|
const { fileURLToPath } = await import('url');
|
|
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
|
const scriptPath = resolve(scriptDir, '..', 'scripts', 'smoke-test.sh');
|
|
try {
|
|
execSync(`bash "${scriptPath}"`, { stdio: 'inherit', env: { ...process.env } });
|
|
} catch (e: any) {
|
|
// Non-zero exit = some tests failed (exit code = failure count)
|
|
process.exit(e.status ?? 1);
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'dream') {
|
|
// Dream mirrors doctor's pattern: filesystem phases run without a DB,
|
|
// so an engine connection failure is non-fatal. runCycle honestly
|
|
// reports DB phases as skipped when engine is null.
|
|
const { runDream } = await import('./commands/dream.ts');
|
|
let eng: BrainEngine | null = null;
|
|
try {
|
|
eng = await connectEngine();
|
|
} catch {
|
|
// DB unavailable — lint + backlinks still run against the brain dir.
|
|
}
|
|
try {
|
|
await runDream(eng, args);
|
|
} finally {
|
|
if (eng) await eng.disconnect();
|
|
}
|
|
return;
|
|
}
|
|
|
|
// `eval cross-modal` is a pure API-call command — no DB, no brain. Bypass
|
|
// connectEngine entirely so first-run users (no `gbrain init` yet) can
|
|
// run the quality gate. Mirrors the dream/doctor no-DB pattern but
|
|
// doesn't even attempt the connect (T3=A in plans/radiant-napping-lerdorf.md).
|
|
// The handler self-configures the AI gateway from loadConfig() + process.env.
|
|
if (command === 'eval' && args[0] === 'cross-modal') {
|
|
const { runEvalCrossModal } = await import('./commands/eval-cross-modal.ts');
|
|
process.exit(await runEvalCrossModal(args.slice(1)));
|
|
}
|
|
|
|
// v0.32 EXP-5 (codex review #10): `eval takes-quality replay <receipt>`
|
|
// is the ONLY sub-subcommand that doesn't need a brain — it reads a
|
|
// receipt JSON file from disk and re-renders it. Bypass connectEngine
|
|
// here so users can replay a receipt on a machine without DATABASE_URL.
|
|
// run/trend/regress need the brain and fall through to the regular
|
|
// engine-required path below.
|
|
if (command === 'eval' && args[0] === 'takes-quality' && args[1] === 'replay') {
|
|
const { runReplayNoBrain } = await import('./commands/eval-takes-quality.ts');
|
|
process.exit(await runReplayNoBrain(args.slice(2)));
|
|
}
|
|
|
|
// v0.28.8: longmemeval brings its own in-memory PGLite. Bypassing
|
|
// connectEngine here keeps `gbrain eval longmemeval --help` and benchmark
|
|
// runs working on machines that have no `~/.gbrain/config.json` configured.
|
|
//
|
|
// v0.35.1.1: still need to configureGateway() so the in-memory brain's
|
|
// import + hybridSearch can embed via the configured provider. Reads
|
|
// ~/.gbrain/config.json when present; falls back to env vars otherwise
|
|
// (GBRAIN_EMBEDDING_MODEL / GBRAIN_EMBEDDING_DIMENSIONS).
|
|
if (command === 'eval' && args[0] === 'longmemeval') {
|
|
const { runEvalLongMemEval } = await import('./commands/eval-longmemeval.ts');
|
|
if (!(args.length > 1 && (args[1] === '--help' || args[1] === '-h'))) {
|
|
const config = loadConfig() ?? ({
|
|
embedding_model: process.env.GBRAIN_EMBEDDING_MODEL,
|
|
embedding_dimensions: process.env.GBRAIN_EMBEDDING_DIMENSIONS
|
|
? Number(process.env.GBRAIN_EMBEDDING_DIMENSIONS) : undefined,
|
|
} as GBrainConfig);
|
|
const { configureGateway } = await import('./core/ai/gateway.ts');
|
|
configureGateway(buildGatewayConfig(config));
|
|
}
|
|
await runEvalLongMemEval(args.slice(1));
|
|
return;
|
|
}
|
|
|
|
// v0.33.1.3: `gbrain eval whoknows` on thin-client installs bypasses
|
|
// connectEngine entirely — the eval routes per-query through the remote
|
|
// `find_experts` MCP op (the v0.31.1 routing seam). Local mode falls
|
|
// through to the engine-connected path below.
|
|
if (command === 'eval' && args[0] === 'whoknows') {
|
|
const cfgPre = loadConfig();
|
|
if (isThinClient(cfgPre)) {
|
|
const { runEvalWhoknows } = await import('./commands/eval-whoknows.ts');
|
|
process.exit(await runEvalWhoknows(null, args.slice(1)));
|
|
}
|
|
}
|
|
|
|
// v0.37 fix wave (Lane D.4 + CDX2-12): short-circuit `gbrain sync --help`
|
|
// BEFORE the engine bind. runSync has its own --help branch but can't
|
|
// reach it without an engine — which means a user running `--help` from
|
|
// a fresh tmpdir with no config gets a no-such-config error instead of
|
|
// help text. Importing runSync without the engine + passing null works
|
|
// because runSync's --help path doesn't touch the engine argument.
|
|
if (command === 'sync' && (args.includes('--help') || args.includes('-h'))) {
|
|
const { runSync } = await import('./commands/sync.ts');
|
|
await runSync(null as any, args);
|
|
return;
|
|
}
|
|
|
|
// v0.39.3.0 WARN-5: same pattern for `capture --help`. CLI_ONLY_SELF_HELP
|
|
// now includes 'capture' so the generic short-circuit at :101 stays out
|
|
// of the way, but the dispatch case at :1229 still needs an engine. The
|
|
// pre-engine-bind branch here exposes the HELP constant without requiring
|
|
// a configured brain (fresh-tmpdir parity with brainstorm/lsd/sync).
|
|
if (command === 'capture' && (args.includes('--help') || args.includes('-h'))) {
|
|
const { runCapture } = await import('./commands/capture.ts');
|
|
await runCapture(null, args);
|
|
return;
|
|
}
|
|
|
|
// All remaining CLI-only commands need a DB connection
|
|
const engine = await connectEngine();
|
|
try {
|
|
switch (command) {
|
|
case 'import': {
|
|
const { runImport } = await import('./commands/import.ts');
|
|
await runImport(engine, args);
|
|
break;
|
|
}
|
|
case 'export': {
|
|
const { runExport } = await import('./commands/export.ts');
|
|
await runExport(engine, args);
|
|
break;
|
|
}
|
|
case 'files': {
|
|
const { runFiles } = await import('./commands/files.ts');
|
|
await runFiles(engine, args);
|
|
break;
|
|
}
|
|
case 'embed': {
|
|
const { runEmbed } = await import('./commands/embed.ts');
|
|
await runEmbed(engine, args);
|
|
break;
|
|
}
|
|
case 'serve': {
|
|
const { runServe } = await import('./commands/serve.ts');
|
|
await runServe(engine, args);
|
|
return; // serve doesn't disconnect
|
|
}
|
|
case 'call': {
|
|
const { runCall } = await import('./commands/call.ts');
|
|
await runCall(engine, args);
|
|
break;
|
|
}
|
|
case 'config': {
|
|
const { runConfig } = await import('./commands/config.ts');
|
|
await runConfig(engine, args);
|
|
break;
|
|
}
|
|
// doctor is handled before connectEngine() above
|
|
case 'migrate': {
|
|
const { runMigrateEngine } = await import('./commands/migrate-engine.ts');
|
|
await runMigrateEngine(engine, args);
|
|
break;
|
|
}
|
|
case 'eval': {
|
|
// v0.32 EXP-5: `eval takes-quality {run,trend,regress}` requires a
|
|
// brain (samples takes from DB / reads runs table). `replay` was
|
|
// already routed through the no-DB bypass above and never reaches
|
|
// this case. Other `eval` subcommands (export/prune/replay-capture/
|
|
// longmemeval/cross-modal) go to the generic dispatcher.
|
|
if (args[0] === 'takes-quality') {
|
|
const { runEvalTakesQuality } = await import('./commands/eval-takes-quality.ts');
|
|
await runEvalTakesQuality(engine, args.slice(1));
|
|
break;
|
|
}
|
|
const { runEvalCommand } = await import('./commands/eval.ts');
|
|
await runEvalCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'jobs': {
|
|
const { runJobs } = await import('./commands/jobs.ts');
|
|
await runJobs(engine, args);
|
|
break;
|
|
}
|
|
case 'agent': {
|
|
const { runAgent } = await import('./commands/agent.ts');
|
|
await runAgent(engine, args);
|
|
break;
|
|
}
|
|
case 'book-mirror': {
|
|
const { runBookMirrorCmd } = await import('./commands/book-mirror.ts');
|
|
await runBookMirrorCmd(engine, args);
|
|
break;
|
|
}
|
|
case 'sync': {
|
|
const { runSync } = await import('./commands/sync.ts');
|
|
await runSync(engine, args);
|
|
break;
|
|
}
|
|
case 'extract': {
|
|
const { runExtract } = await import('./commands/extract.ts');
|
|
await runExtract(engine, args);
|
|
break;
|
|
}
|
|
case 'features': {
|
|
const { runFeatures } = await import('./commands/features.ts');
|
|
await runFeatures(engine, args);
|
|
break;
|
|
}
|
|
case 'autopilot': {
|
|
const { runAutopilot } = await import('./commands/autopilot.ts');
|
|
await runAutopilot(engine, args);
|
|
return; // autopilot doesn't disconnect (long-running)
|
|
}
|
|
case 'graph-query': {
|
|
const { runGraphQuery } = await import('./commands/graph-query.ts');
|
|
await runGraphQuery(engine, args);
|
|
break;
|
|
}
|
|
case 'reconcile-links': {
|
|
// v0.20.0 Cathedral II Layer 8 D3: batch-recompute doc↔impl edges
|
|
// for any markdown page that cites code files. Idempotent; safe to
|
|
// re-run. Closes the v0.19.0 Layer 6 order-dependency bug where
|
|
// guides imported before their code never got their edges written.
|
|
const { runReconcileLinksCli } = await import('./commands/reconcile-links.ts');
|
|
await runReconcileLinksCli(engine, args);
|
|
break;
|
|
}
|
|
case 'orphans': {
|
|
const { runOrphans } = await import('./commands/orphans.ts');
|
|
await runOrphans(engine, args);
|
|
break;
|
|
}
|
|
// v0.32.7 CJK wave — post-upgrade markdown re-chunk sweep.
|
|
// v0.36 Phase 3 wave — `gbrain reindex --multimodal` re-embeds content_chunks
|
|
// into the unified Voyage multimodal-3 column.
|
|
case 'reindex': {
|
|
if (args.includes('--multimodal')) {
|
|
const { runReindexMultimodal } = await import('./commands/reindex-multimodal.ts');
|
|
const limitIdx = args.indexOf('--limit');
|
|
const limitVal = limitIdx >= 0 && limitIdx + 1 < args.length ? parseInt(args[limitIdx + 1], 10) : undefined;
|
|
const result = await runReindexMultimodal(engine, {
|
|
limit: Number.isFinite(limitVal as number) ? (limitVal as number) : undefined,
|
|
dryRun: args.includes('--dry-run'),
|
|
costEstimate: args.includes('--cost-estimate'),
|
|
noEmbed: args.includes('--no-embed'),
|
|
json: args.includes('--json'),
|
|
yes: args.includes('--yes'),
|
|
});
|
|
if (args.includes('--json')) {
|
|
console.log(JSON.stringify(result, null, 2));
|
|
} else {
|
|
console.log(`reindex --multimodal: ${result.reembedded} re-embedded, ${result.failed} failed, ${result.pending_after} pending. est. cost: $${result.cost_usd_estimate.toFixed(2)}`);
|
|
}
|
|
break;
|
|
}
|
|
const { runReindex } = await import('./commands/reindex.ts');
|
|
await runReindex(engine, args);
|
|
break;
|
|
}
|
|
// v0.29 — Salience + Anomaly Detection
|
|
case 'salience': {
|
|
const { runSalience } = await import('./commands/salience.ts');
|
|
await runSalience(engine, args);
|
|
break;
|
|
}
|
|
case 'anomalies': {
|
|
const { runAnomalies } = await import('./commands/anomalies.ts');
|
|
await runAnomalies(engine, args);
|
|
break;
|
|
}
|
|
// v0.38 — Capture: single human-facing entrypoint for ingestion.
|
|
case 'capture': {
|
|
const { runCapture } = await import('./commands/capture.ts');
|
|
await runCapture(engine, args);
|
|
break;
|
|
}
|
|
case 'edges-backfill': {
|
|
// v0.34 W6 — operator escape hatch for the symbol-resolution backfill.
|
|
// Resumable via the edges_backfilled_at watermark; per-batch transactions
|
|
// commit so Ctrl-C leaves a clean resumable state.
|
|
const { runEdgesBackfill } = await import('./commands/edges-backfill.ts');
|
|
await runEdgesBackfill(engine, args);
|
|
break;
|
|
}
|
|
case 'whoknows': {
|
|
// v0.33 (Issue #?): expertise + relationship-proximity routing.
|
|
// MCP op `find_experts` (read-scoped) backs the same code path; CLI
|
|
// dispatch here is the user-facing surface. Thin-client routing
|
|
// happens inside runWhoknows via isThinClient(cfg) (v0.31.1 pattern).
|
|
const { runWhoknows } = await import('./commands/whoknows.ts');
|
|
await runWhoknows(engine, args);
|
|
break;
|
|
}
|
|
case 'brainstorm': {
|
|
// v0.37.0 (Open Collider wave): bisociation idea generator grounded
|
|
// in the user's own brain. Prefix-stratified domain-bank (D14) +
|
|
// shared judges + citation transparency (D6). LSD MCP exposure
|
|
// deferred to D7; this is CLI-only.
|
|
const { runBrainstormCommand } = await import('./commands/brainstorm.ts');
|
|
await runBrainstormCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'lsd': {
|
|
// v0.37.0 — Lateral Synaptic Drift. Inverted-judge / stale-bias
|
|
// variant of brainstorm. Shares the orchestrator + judges via
|
|
// LSD_PROFILE config. Local-only by design (cost + weirdness gate).
|
|
const { runLsdCommand } = await import('./commands/lsd.ts');
|
|
await runLsdCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'calibration': {
|
|
// v0.36.1.0 (T7): print/regenerate the active calibration profile.
|
|
// MCP op `get_calibration_profile` (read-scoped) backs the same data path.
|
|
const { runCalibration } = await import('./commands/calibration.ts');
|
|
const calibrationConfig = loadConfig() ?? ({} as never);
|
|
await runCalibration(engine, args, calibrationConfig);
|
|
break;
|
|
}
|
|
case 'transcripts': {
|
|
const { runTranscripts } = await import('./commands/transcripts.ts');
|
|
await runTranscripts(engine, args);
|
|
break;
|
|
}
|
|
case 'models': {
|
|
const { runModels } = await import('./commands/models.ts');
|
|
await runModels(engine, args);
|
|
break;
|
|
}
|
|
case 'search': {
|
|
// v0.32.3 search-lite — `gbrain search modes/stats/tune`.
|
|
const { runSearch } = await import('./commands/search.ts');
|
|
await runSearch(engine, args);
|
|
break;
|
|
}
|
|
case 'takes': {
|
|
const { runTakes } = await import('./commands/takes.ts');
|
|
await runTakes(engine, args);
|
|
break;
|
|
}
|
|
case 'founder': {
|
|
// v0.35.4 (T7) — founder scorecard. `gbrain founder scorecard <slug>`
|
|
// rolls up Phase 2's typed-claim substrate into the four scorecard
|
|
// metrics (claim accuracy, consistency, growth trajectory, red flags).
|
|
// Thin-client routing handled inside the command file.
|
|
const { runFounder } = await import('./commands/founder-scorecard.ts');
|
|
await runFounder(engine, args);
|
|
break;
|
|
}
|
|
case 'think': {
|
|
const { runThinkCli } = await import('./commands/think.ts');
|
|
await runThinkCli(engine, args);
|
|
break;
|
|
}
|
|
case 'recall': {
|
|
// v0.31: hot memory recall surface — `gbrain recall <entity>`,
|
|
// `--since DUR`, `--session ID`, `--today`, `--grep TEXT`,
|
|
// `--supersessions`, `--include-expired`, `--as-context`, `--json`.
|
|
const { runRecall } = await import('./commands/recall.ts');
|
|
await runRecall(engine, args);
|
|
break;
|
|
}
|
|
case 'forget': {
|
|
// v0.31: shorthand for expireFact. `gbrain forget <fact-id>`.
|
|
const { runForget } = await import('./commands/recall.ts');
|
|
await runForget(engine, args);
|
|
break;
|
|
}
|
|
case 'notability-eval': {
|
|
// v0.31.2: notability gate eval suite. Two subcommands:
|
|
// gbrain notability-eval mine — sample paragraphs, write candidates
|
|
// gbrain notability-eval review — TTY hand-confirm tiers
|
|
const { runNotabilityEval } = await import('./commands/notability-eval.ts');
|
|
const subcmd = args[0] || 'help';
|
|
const flags: Record<string, string | boolean> = {};
|
|
for (let i = 1; i < args.length; i++) {
|
|
const a = args[i];
|
|
if (a.startsWith('--')) {
|
|
const key = a.slice(2);
|
|
const next = args[i + 1];
|
|
if (next && !next.startsWith('--')) {
|
|
flags[key] = next;
|
|
i++;
|
|
} else {
|
|
flags[key] = true;
|
|
}
|
|
}
|
|
}
|
|
// sync.repo_path resolution (matches dream phase pattern).
|
|
let repoPath: string | undefined;
|
|
try {
|
|
repoPath = (flags.repo as string) || (await engine.getConfig('sync.repo_path')) || undefined;
|
|
} catch { /* engine may not be connected for help */ }
|
|
await runNotabilityEval({ cmd: subcmd, flags, engine, repoPath });
|
|
break;
|
|
}
|
|
case 'sources': {
|
|
const { runSources } = await import('./commands/sources.ts');
|
|
await runSources(engine, args);
|
|
break;
|
|
}
|
|
case 'pages': {
|
|
// v0.26.5: page-level operator commands (purge-deleted escape hatch).
|
|
const { runPages } = await import('./commands/pages.ts');
|
|
await runPages(engine, args);
|
|
break;
|
|
}
|
|
case 'storage': {
|
|
const { runStorage } = await import('./commands/storage.ts');
|
|
await runStorage(engine, args);
|
|
break;
|
|
}
|
|
case 'code-def': {
|
|
const { runCodeDef } = await import('./commands/code-def.ts');
|
|
await runCodeDef(engine, args);
|
|
break;
|
|
}
|
|
case 'code-refs': {
|
|
const { runCodeRefs } = await import('./commands/code-refs.ts');
|
|
await runCodeRefs(engine, args);
|
|
break;
|
|
}
|
|
case 'reindex-code': {
|
|
// v0.20.0 Cathedral II Layer 13 (E2): explicit code-page reindex
|
|
// for users upgrading from v0.19.0. Cost-preview gated; TTY prompt
|
|
// or ConfirmationRequired envelope for non-TTY/JSON callers.
|
|
const { runReindexCodeCli } = await import('./commands/reindex-code.ts');
|
|
await runReindexCodeCli(engine, args);
|
|
break;
|
|
}
|
|
case 'reindex-frontmatter': {
|
|
// v0.29.1: recovery / explicit-rebuild path for pages.effective_date.
|
|
// Mirror of reindex-code shape. Wraps the shared library function in
|
|
// src/core/backfill-effective-date.ts (same code path the v0.29.1
|
|
// migration orchestrator uses). The orchestrator runs once on
|
|
// upgrade; this command is for after-the-fact frontmatter edits.
|
|
//
|
|
// v0.30.1: still works; canonical entrypoint is now `gbrain backfill
|
|
// effective_date`. This command stays as a thin alias for back-compat.
|
|
const { reindexFrontmatterCli } = await import('./commands/reindex-frontmatter.ts');
|
|
await reindexFrontmatterCli(args);
|
|
return; // reindexFrontmatterCli handles its own engine lifecycle
|
|
}
|
|
case 'backfill': {
|
|
// v0.30.1: first-class generic backfill command. Subcommand dispatch
|
|
// is inside runBackfillCommand (kind | list | --help).
|
|
const { runBackfillCommand } = await import('./commands/backfill.ts');
|
|
await runBackfillCommand(args);
|
|
return;
|
|
}
|
|
case 'code-callers': {
|
|
// v0.20.0 Cathedral II Layer 10 (C4): "who calls <symbol>?"
|
|
const { runCodeCallers } = await import('./commands/code-callers.ts');
|
|
await runCodeCallers(engine, args);
|
|
break;
|
|
}
|
|
case 'code-callees': {
|
|
// v0.20.0 Cathedral II Layer 10 (C5): "what does <symbol> call?"
|
|
const { runCodeCallees } = await import('./commands/code-callees.ts');
|
|
await runCodeCallees(engine, args);
|
|
break;
|
|
}
|
|
case 'repos': {
|
|
// v0.19.0: `gbrain repos ...` is an alias into the v0.18.0 sources
|
|
// subsystem. The repos abstraction (Garry's OpenClaw baseline) was
|
|
// redundant with sources and carried per-user config state that
|
|
// couldn't participate in federation / RLS / multi-tenancy. We
|
|
// keep the alias so scripts like `gbrain repos add .` keep
|
|
// working, with a nudge toward the canonical command.
|
|
console.error('[gbrain] Note: "repos" is an alias for "sources" as of v0.19.0. Prefer `gbrain sources <subcommand>`.');
|
|
const { runSources } = await import('./commands/sources.ts');
|
|
await runSources(engine, args);
|
|
break;
|
|
}
|
|
}
|
|
} finally {
|
|
if (command !== 'serve') await engine.disconnect();
|
|
}
|
|
}
|
|
|
|
// Build the AIGatewayConfig payload from a GBrainConfig. Both configureGateway
|
|
// sites in connectEngine() pass through this helper so adding a new field
|
|
// touches one place. Adding a field to one site but not the other previously
|
|
// required remembering to mirror the change; the helper makes that structural.
|
|
// v0.37.6.0: exported so `test/ai/build-gateway-config.test.ts` can pin the
|
|
// env-baseURL passthrough contract for every `_BASE_URL` env var the CLI
|
|
// reads (LLAMA_SERVER, OLLAMA, LMSTUDIO, LITELLM, OPENROUTER).
|
|
export function buildGatewayConfig(c: GBrainConfig): AIGatewayConfig {
|
|
// v0.32 (#121 reworked): when ~/.gbrain/config.json declares
|
|
// openai_api_key / anthropic_api_key, fold them into the gateway env so
|
|
// recipes that read OPENAI_API_KEY / ANTHROPIC_API_KEY find them. Process
|
|
// env still wins (it's loaded last) — this is a fallback for daemons /
|
|
// launchd-spawned subprocesses that don't propagate ~/.zshrc-sourced keys.
|
|
const envFromConfig: Record<string, string> = {};
|
|
if (c.openai_api_key) envFromConfig.OPENAI_API_KEY = c.openai_api_key;
|
|
if (c.anthropic_api_key) envFromConfig.ANTHROPIC_API_KEY = c.anthropic_api_key;
|
|
// v0.37 fix wave (CDX2-5+6): ZE became the default provider in v0.36 but
|
|
// the env-mapping at this seam never picked it up. `gbrain config set
|
|
// zeroentropy_api_key X` wrote DB plane (ignored by gateway). The file-
|
|
// plane field now exists (GBrainConfig type) and gets mapped here, so
|
|
// setting it via `~/.gbrain/config.json` propagates into the gateway.
|
|
if (c.zeroentropy_api_key) envFromConfig.ZEROENTROPY_API_KEY = c.zeroentropy_api_key;
|
|
|
|
// v0.32 codex finding #4+#5 fix: thread local-server _BASE_URL env vars
|
|
// into base_urls so the gateway hits the user's configured port. Without
|
|
// this, `LLAMA_SERVER_BASE_URL=http://localhost:9000` would let the probe
|
|
// succeed against :9000 but the actual embed call would still go to the
|
|
// recipe's base_url_default (localhost:8080). Same fix applies to
|
|
// OLLAMA_BASE_URL. Caller-provided cfg.provider_base_urls wins.
|
|
const envBaseUrls: Record<string, string> = {};
|
|
if (process.env.LLAMA_SERVER_BASE_URL) envBaseUrls['llama-server'] = process.env.LLAMA_SERVER_BASE_URL;
|
|
if (process.env.OLLAMA_BASE_URL) envBaseUrls['ollama'] = process.env.OLLAMA_BASE_URL;
|
|
if (process.env.LMSTUDIO_BASE_URL) envBaseUrls['lmstudio'] = process.env.LMSTUDIO_BASE_URL;
|
|
if (process.env.LITELLM_BASE_URL) envBaseUrls['litellm'] = process.env.LITELLM_BASE_URL;
|
|
if (process.env.OPENROUTER_BASE_URL) envBaseUrls['openrouter'] = process.env.OPENROUTER_BASE_URL;
|
|
|
|
return {
|
|
embedding_model: c.embedding_model,
|
|
embedding_dimensions: c.embedding_dimensions,
|
|
embedding_multimodal_model: c.embedding_multimodal_model,
|
|
expansion_model: c.expansion_model,
|
|
chat_model: c.chat_model,
|
|
chat_fallback_chain: c.chat_fallback_chain,
|
|
base_urls: { ...envBaseUrls, ...(c.provider_base_urls ?? {}) }, // config wins over env
|
|
env: { ...envFromConfig, ...process.env }, // process.env wins
|
|
};
|
|
}
|
|
|
|
async function connectEngine(opts?: { probeOnly?: boolean }): Promise<BrainEngine> {
|
|
const config = loadConfig();
|
|
if (!config) {
|
|
console.error('No brain configured. Run: gbrain init');
|
|
process.exit(1);
|
|
}
|
|
|
|
// Configure the AI gateway BEFORE engine connect — initSchema needs embedding dims.
|
|
// Env is read once here; the gateway never reads process.env at call time (Codex C3).
|
|
const { configureGateway } = await import('./core/ai/gateway.ts');
|
|
configureGateway(buildGatewayConfig(config));
|
|
|
|
const { createEngine } = await import('./core/engine-factory.ts');
|
|
const engine = await createEngine(toEngineConfig(config));
|
|
const noRetry = process.argv.includes('--no-retry-connect') ||
|
|
process.env.GBRAIN_NO_RETRY_CONNECT === '1';
|
|
const { connectWithRetry } = await import('./core/db.ts');
|
|
await connectWithRetry(engine, toEngineConfig(config), { noRetry });
|
|
|
|
// v0.30.1 (Codex X1 / C2): probeOnly skips both hasPendingMigrations() probe
|
|
// AND initSchema(). Used by `get_health` MCP op + `gbrain upgrade --status`
|
|
// + doctor's migration_wedge check — these surfaces report wedge state and
|
|
// must NEVER themselves start or block on migrations.
|
|
if (opts?.probeOnly === true) {
|
|
return engine;
|
|
}
|
|
|
|
// Auto-apply pending schema migrations on connect (#651). Cheap probe
|
|
// first so already-migrated brains don't pay the bootstrap-probe +
|
|
// SCHEMA_SQL replay + ledger-check cost on every short-lived CLI call.
|
|
// This is the conditional version of #652 (oyi77's investigation):
|
|
// same correctness, no perf regression on the hot path.
|
|
try {
|
|
const { hasPendingMigrations } = await import('./core/migrate.ts');
|
|
if (await hasPendingMigrations(engine)) {
|
|
await engine.initSchema();
|
|
}
|
|
} catch (err) {
|
|
// Non-fatal: if probe or initSchema fails, surface a hint and continue
|
|
// with the connected engine. Subsequent operations will surface the
|
|
// real schema error in context.
|
|
console.warn(` Schema probe/migrate failed: ${(err as Error).message}`);
|
|
console.warn(' Try: gbrain init --migrate-only');
|
|
}
|
|
|
|
// v0.27.1 (F3 fix): re-merge DB-plane config now that the engine is up.
|
|
// Flags like `embedding_multimodal` are user-mutable via `gbrain config set`
|
|
// (DB plane) and need to flow into the gateway after connect. Schema-sizing
|
|
// fields (embedding_dimensions etc.) keep their pre-connect file/env values
|
|
// — those drove initSchema and the merged config respects file/env first.
|
|
try {
|
|
const merged = await loadConfigWithEngine(engine, config);
|
|
if (merged) {
|
|
// Stash gate flags on process.env for downstream readers (import-file.ts
|
|
// dispatches on GBRAIN_EMBEDDING_MULTIMODAL, OCR consumer reads
|
|
// GBRAIN_EMBEDDING_IMAGE_OCR_*). The gateway itself doesn't read these
|
|
// flags; this preserves the contract without changing the gateway shape.
|
|
if (merged.embedding_multimodal !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_MULTIMODAL = String(merged.embedding_multimodal);
|
|
}
|
|
if (merged.embedding_image_ocr !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_IMAGE_OCR = String(merged.embedding_image_ocr);
|
|
}
|
|
if (merged.embedding_image_ocr_model !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_IMAGE_OCR_MODEL = merged.embedding_image_ocr_model;
|
|
}
|
|
// Always re-configure with merged values when DB merge succeeded. The
|
|
// trigger used to be field-name-gated (only when embedding_multimodal_model
|
|
// was set); that coupled the gate to the field set and would silently
|
|
// miss future DB-mutable gateway fields. One extra cache+shrinkState
|
|
// clear per startup is microseconds, no hot path.
|
|
configureGateway(buildGatewayConfig(merged));
|
|
}
|
|
// v0.31.12: re-resolve gateway defaults through resolveModel so
|
|
// `models.tier.*` and `models.default` overrides apply to expansion +
|
|
// chat. Per Codex F3 — configureGateway is sync; this is the async
|
|
// re-stamp seam after engine.connect() makes config reads possible.
|
|
const { reconfigureGatewayWithEngine } = await import('./core/ai/gateway.ts');
|
|
await reconfigureGatewayWithEngine(engine);
|
|
} catch {
|
|
// Non-fatal. Pre-v39 brains may not have a usable config table yet.
|
|
}
|
|
|
|
return engine;
|
|
}
|
|
|
|
function printOpHelp(op: Operation) {
|
|
const positional = (op.cliHints?.positional || []).map(p => `<${p}>`).join(' ');
|
|
const name = op.cliHints?.name || op.name;
|
|
console.log(`Usage: gbrain ${name} ${positional} [options]\n`);
|
|
console.log(op.description + '\n');
|
|
const entries = Object.entries(op.params);
|
|
if (entries.length > 0) {
|
|
console.log('Options:');
|
|
for (const [key, def] of entries) {
|
|
const isPos = op.cliHints?.positional?.includes(key);
|
|
const req = def.required ? ' (required)' : '';
|
|
const prefix = isPos ? ` <${key}>` : ` --${key.replace(/_/g, '-')}`;
|
|
console.log(`${prefix.padEnd(28)} ${def.description || ''}${req}`);
|
|
}
|
|
}
|
|
}
|
|
|
|
function printHelp() {
|
|
// Gather shared operations grouped by category
|
|
const cliNames = Array.from(cliOps.entries())
|
|
.map(([name, op]) => ({ name, desc: op.description }));
|
|
|
|
console.log(`gbrain ${VERSION} -- personal knowledge brain
|
|
|
|
USAGE
|
|
gbrain <command> [options]
|
|
|
|
SETUP
|
|
init [--pglite|--supabase|--url] Create brain (PGLite default, no server)
|
|
migrate --to <supabase|pglite> Transfer brain between engines
|
|
upgrade Self-update
|
|
check-update [--json] Check for new versions
|
|
doctor [--json] [--fast] Health check (resolver, skills, pgvector, RLS, embeddings)
|
|
integrations [subcommand] Manage integration recipes (senses + reflexes)
|
|
|
|
PAGES
|
|
get <slug> Read a page
|
|
put <slug> [< file.md] Write/update a page
|
|
delete <slug> Delete a page
|
|
list [--type T] [--tag T] [-n N] List pages
|
|
|
|
SEARCH
|
|
search <query> Keyword search (tsvector)
|
|
query <question> [--no-expand] Hybrid search (RRF + expansion)
|
|
ask <question> [--no-expand] Alias for query
|
|
|
|
IMPORT/EXPORT
|
|
import <dir> [--no-embed] Import markdown directory
|
|
sync [--repo <path>] [flags] Git-to-brain incremental sync
|
|
sync --watch [--interval N] Continuous sync (loops until stopped)
|
|
sync --install-cron Install persistent sync daemon
|
|
export [--dir ./out/] Export to markdown
|
|
export --restore-only [--repo <p>] Restore missing supabase-only files
|
|
[--type T] [--slug-prefix S] With optional filters
|
|
|
|
FILES
|
|
files list [slug] List stored files
|
|
files upload <file> --page <slug> Upload file to storage
|
|
files upload-raw <file> --page <s> Smart upload (size routing + .redirect.yaml)
|
|
files signed-url <path> Generate signed URL (1-hour)
|
|
files sync <dir> Bulk upload directory
|
|
files verify Verify all uploads
|
|
|
|
EMBEDDINGS
|
|
embed [<slug>|--all|--stale] Generate/refresh embeddings
|
|
|
|
LINKS
|
|
link <from> <to> [--type T] Create typed link
|
|
unlink <from> <to> Remove link
|
|
backlinks <slug> Incoming links
|
|
graph <slug> [--depth N] Traverse link graph (returns nodes)
|
|
graph-query <slug> [--type T] Edge-based traversal with type/direction filters
|
|
[--depth N] [--direction in|out|both]
|
|
|
|
TAGS
|
|
tags <slug> List tags
|
|
tag <slug> <tag> Add tag
|
|
untag <slug> <tag> Remove tag
|
|
|
|
TIMELINE
|
|
timeline [<slug>] View timeline
|
|
timeline-add <slug> <date> <text> Add timeline entry
|
|
|
|
TOOLS
|
|
extract <links|timeline|all> Extract links/timeline (idempotent)
|
|
[--source fs|db] fs (default) walks .md files; db iterates engine pages
|
|
[--dir <brain>] brain dir for fs source
|
|
[--type T] [--since DATE] filters (db source)
|
|
[--dry-run] [--json]
|
|
publish <page.md> [--password] Shareable HTML (strips private data, optional AES-256)
|
|
check-backlinks <check|fix> [dir] Find/fix missing back-links across brain
|
|
lint <dir|file> [--fix] Catch LLM artifacts, placeholder dates, bad frontmatter
|
|
orphans [--json] [--count] Find pages with no inbound wikilinks
|
|
salience [--days N] [--kind P] v0.29: pages ranked by emotional + activity salience
|
|
anomalies [--since D] [--sigma N] v0.29: cohort-based statistical anomalies (tag, type)
|
|
transcripts recent [--days N] v0.29: recent raw .txt transcripts (local-only)
|
|
dream [--dry-run] [--json] Run the overnight maintenance cycle once (cron-friendly).
|
|
See also: autopilot --install (continuous daemon).
|
|
check-resolvable [--json] [--fix] Validate skill tree (reachability/MECE/DRY)
|
|
report --type <name> --content ... Save timestamped report to brain/reports/
|
|
|
|
BRAIN (capture / ideate / explore — v0.37/v0.38)
|
|
capture [content] [--file PATH] Single entrypoint for getting content into the brain
|
|
[--stdin] [--slug s] [--type t] Inline content / file / stdin; writes to inbox/ by default
|
|
[--source ID] [--quiet|--json] Multi-source brains: route to a non-default source
|
|
brainstorm <question> [--json] Bisociation idea generator (hybrid search + far-set + judge)
|
|
[--save|--no-save] [--limit N]
|
|
lsd <question> [--json] Lateral Synaptic Drift: inverted-judge brainstorm
|
|
[--save|--no-save] [--limit N] rewarding far-from-obvious + axiomatic inversions
|
|
|
|
SOURCES (multi-repo / multi-brain)
|
|
sources list Show registered sources
|
|
sources add <id> --path <p> Register a source (id = short name, e.g. 'wiki')
|
|
sources remove <id> Remove a source + its pages
|
|
sync --all Sync all sources with a local_path
|
|
sync --source <id> Sync one specific source
|
|
repos ... DEPRECATED alias for 'sources' (v0.19.0)
|
|
|
|
CODE INDEXING (v0.19.0 / v0.20.0 Cathedral II)
|
|
code-def <symbol> [--lang l] Find the definition of a symbol across code pages
|
|
code-refs <symbol> [--lang l] Find all references to a symbol (JSON-first)
|
|
code-callers <symbol> Who calls this symbol? (v0.20.0 A1)
|
|
code-callees <symbol> What does this symbol call? (v0.20.0 A1)
|
|
query <q> --lang <l> Filter hybrid search to one language (v0.20.0)
|
|
query <q> --symbol-kind <k> Filter to symbol type (function|class|method|...) (v0.20.0)
|
|
reconcile-links [--dry-run] Batch-recompute doc↔impl edges (v0.20.0)
|
|
reindex-code [--source id] [--yes] Explicit code-page reindex (v0.20.0)
|
|
sync --strategy code Sync code files into the brain
|
|
|
|
JOBS (Minions)
|
|
jobs submit <name> [--params JSON] Submit background job [--follow] [--dry-run]
|
|
jobs list [--status S] [--limit N] List jobs
|
|
jobs get <id> Job details + history
|
|
jobs cancel <id> Cancel job
|
|
jobs retry <id> Re-queue failed/dead job
|
|
jobs prune [--older-than 30d] Clean old jobs
|
|
jobs stats Job health dashboard
|
|
jobs work [--queue Q] Start worker daemon (Postgres only)
|
|
|
|
ADMIN
|
|
stats Brain statistics
|
|
health Brain health dashboard
|
|
history <slug> Page version history
|
|
revert <slug> <version-id> Revert to version
|
|
features [--json] [--auto-fix] Scan usage + recommend unused features
|
|
autopilot [--repo] [--interval N] Self-maintaining brain daemon
|
|
config [show|get|set] <key> [val] Brain config
|
|
storage status [--repo <path>] Storage tier status and health
|
|
[--json] (git-tracked vs supabase-only)
|
|
serve MCP server (stdio)
|
|
serve --http [--port N] HTTP MCP server with OAuth 2.1
|
|
--token-ttl N Access token TTL in seconds (default: 3600)
|
|
--enable-dcr Enable Dynamic Client Registration
|
|
--public-url URL Public issuer URL (required behind proxy/tunnel)
|
|
call <tool> '<json>' Raw tool invocation
|
|
version Version info
|
|
--tools-json Tool discovery (JSON)
|
|
|
|
Run gbrain <command> --help for command-specific help.
|
|
`);
|
|
}
|
|
|
|
main().catch(e => {
|
|
console.error(e.message || e);
|
|
process.exit(1);
|
|
});
|