mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 06:23:01 +00:00
* feat(schema): migrations v98/v99/v100 for onboard wave (A6 A10 A11 A13 A25, codex #1 #9 #10 #11 #12) Three schema additions supporting the gbrain onboard wave: v98 — links.link_kind nullable column (A10, codex finding #12). The NER extraction was originally going to add a new link_source='ner' provenance, but that would have forced every existing link_source='mentions' query (backlink-count filter, orphan-ratio, doctor checks) to update or metrics would drift across the cutover. Instead: keep link_source='mentions' for the storage layer AND add a nullable link_kind column. Three kinds: 'plain', 'typed_ner', NULL (legacy/unknown — semantically 'plain'). NOT in the links UNIQUE constraint so the storage shape stays compatible. v99 — timeline_entries dedup widening (A11, codex finding #11). Pre-v99 dedup key was (page_id, date, summary). The new --from-meetings extraction writes timeline entries with source='extract-timeline-from- meetings:<meeting-slug>', and codex caught that two meetings with the same date+summary on the same entity page would silently DO NOTHING — the second meeting's provenance is lost. Widened to (page_id, date, summary, source). Legacy rows (source='') preserve current dedup behavior. v100 — migration_impact_log table + content_chunks_stale_idx partial (A6 + A25 + A13 + codex findings #10 + #9). Bundled because both are consumed by the onboard pipeline and ship together. Impact log captures before/after metric stats so gbrain onboard --history shows real deltas; attribution columns (job_id, source_id, brain_id, started_at, idempotency_key) prevent concurrent runs misattributing to wrong migrations. content_chunks_stale_idx partial WHERE embedding IS NULL supports gbrain embed --stale + --priority recent (outer ORDER BY p.updated_at DESC uses existing idx_pages_updated_at_desc via JOIN). Plain NUMERIC columns; delta computed at read time (NOT a stored GENERATED column per eng-review D2 — zero PGLite parity risk). Slot history note: plan originally proposed v97/v98/v99 but master had already used v95 (links 'mentions' CHECK widening), v96 (facts conversation session index), and v97 (pages_dedup_partial_index) by ship time. Codex caught the collision; renumbered to v98/v99/v100. Test pin: test/schema-bootstrap-coverage.test.ts (100/100 migrations apply clean on PGLite), test/migrate.test.ts (152 cases pass). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(remediation): extract doctor remediation library (A1, codex finding #2) Pre-fix: src/commands/doctor.ts contained two CLI-shaped functions (runRemediationPlan + runRemediate) with hardcoded argv parsing, process.exit calls, and console.log emission. Onboard CLI shell and the upcoming MCP run_onboard op couldn't compose against them — the plan file's "100-LOC thin wrapper" assumption didn't survive codex's review of the actual source. Post-fix: src/core/remediation/ exports a library shape that all three consumers (doctor CLI, onboard CLI, MCP run_onboard) wrap. src/core/remediation/types.ts RemediationPlanOpts, RemediationPlan, RemediationOpts, RemediationResult, StepResult, RemediationHooks (the observability seam — library never calls console.* itself). src/core/remediation/context.ts loadRecommendationContext moved verbatim from doctor.ts. Re-exports RecommendationContext from brain-score-recommendations.ts since that's still the canonical home for the type (consumed by computeRecommendations). src/core/remediation/plan.ts computeRemediationPlan(engine, opts): Promise<RemediationPlan>. Pure read; produces the stable JSON envelope downstream agents bind to. Pulls in computeRecommendations + classifyChecks + maxReachableScore behind one library entry point. src/core/remediation/run.ts runRemediation(engine, opts, hooks): Promise<RemediationResult>. Orchestrator with BudgetTracker, checkpoint resume, D5 dep cascade, D7 per-step recheck. Returns a result object instead of process.exit calls; the CLI shell maps result.budget_exhausted / .target_unreachable / .submitted to exit codes. src/core/remediation/index.ts Barrel for the three modules above. doctor.ts is now a thin wrapper: runRemediationPlan: parse argv → computeRemediationPlan → human/JSON render runRemediate: parse argv → TTY confirm gate → runRemediation(hooks: console.*) The TTY confirmation step deliberately stays in the CLI shell — the library never asks for confirmation; that's a CLI concern. Net: ~340 LOC removed from doctor.ts; ~470 LOC added across the library module (with full JSDoc + per-A-decision rationale comments). Functional behavior preserved bit-for-bit: 67 tests pass across doctor.test.ts + v0_37_gap_fill.serial.test.ts. The Lane E.4 source-text test (test/v0_37_gap_fill.serial.test.ts:329) followed loadRecommendationContext to its new home at src/core/remediation/context.ts — assertions otherwise unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(remediation): generalize computeRecommendations to accept extras (A2, codex finding #3) Pre-fix: computeRecommendations at brain-score-recommendations.ts:170 was a hardcoded planner for 5 synthetic check categories. Adding a Check.remediation field to a new doctor check would NOT auto-wire into --remediation-plan — the planner simply ignored it. Codex caught this when reviewing the plan's "checks ARE specs" framing. Post-fix: optional third arg `extraRemediations: RemediationStep[]` lets callers inject step entries discovered outside the hardcoded planner. The existing 5-category surface is preserved bit-for-bit; on id collision the hardcoded entry wins, so an extra accidentally duplicating a hardcoded id doesn't shadow legacy behavior. RemediationPlanOpts gains the matching field; computeRemediationPlan in src/core/remediation/plan.ts threads opts.extraRemediations through. The 4 new doctor checks (T4) will produce per-check helper functions that return RemediationStep[]; onboard's render layer (T12) aggregates them into the opts.extraRemediations slot. doctor's existing --remediation-plan call passes empty (no behavior change for legacy CLI). 84 tests pass across brain-score-recommendations + doctor suites. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(doctor): 4 new onboard checks (embed_staleness, link_coverage, timeline_coverage, takes_count) (A16, T4) Adds src/core/onboard/checks.ts: 4 check helpers + a runAllOnboardChecks aggregator. Each helper returns {check, remediations}, so doctor pushes the Check entry (for human/JSON rendering) AND onboard's plan path collects the RemediationStep[] (via T3's new extraRemediations seam in computeRecommendations). embed_staleness: COUNT(*) on content_chunks WHERE embedding IS NULL. Cheap thanks to content_chunks_stale_idx partial (v100). warn at 1+ stale, fail at 1000+; remediation points at embed-catch-up handler (built in T6). entity_link_coverage: fraction of entity pages with inbound links. Per A21 + codex #15: TABLESAMPLE BERNOULLI on PG when total_pages > 50K with pinned sample formula (LEAST 100, GREATEST 2, target ~5000 rows) AND ±sqrt(p(1-p)/n) confidence interval embedded in message ("coverage: 31% ± 1.3%") so warn/fail decisions show their margin of error. PGLite path: full scan (rare >50K). warn <70%, fail <40%; remediation points at extract-ner handler. timeline_coverage: same TABLESAMPLE policy. warn <90%, fail <70%; remediation points at extract-timeline-from-meetings handler. takes_count: COUNT(*) on takes table. Per A12 two-gate consent: the remediation only emits when `takes.bootstrap_enabled` config is true. Otherwise the check shows "0 takes (takes.bootstrap_enabled is false; opt in to enable)" without an autopilot-eligible remediation. Prevents unattended LLM-bearing extractions on brains that haven't opted in. runDoctor wires runAllOnboardChecks at the end of the DB-checks block (after stale_locks); fast-mode skipped to preserve --fast UX. Thin-client parity (A16 spec) deferred to T16 — the MCP run_onboard op will run these helpers server-side where engine.executeRaw works, which is the real federated path. Adding them to doctor-remote.ts would duplicate the logic without functional benefit since the helpers are server-side queries. 55 doctor tests pass; typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(engine): listStaleChunks --priority recent + executeRaw AbortSignal (A13/A20, codex #7 #9) Two interface extensions on BrainEngine, with parity across postgres-engine and pglite-engine. Plus a follow-on fix for v99's timeline_entries dedup widening. listStaleChunks gains: - orderBy?: 'page_id' | 'updated_desc' (default 'page_id' = legacy) - afterUpdatedAt?: string | null (composite cursor for updated_desc) When orderBy === 'updated_desc' the query JOINs pages and orders by p.updated_at DESC NULLS LAST, p.id ASC, cc.chunk_index ASC backed by idx_pages_updated_at_desc + content_chunks_stale_idx partial (both indexes added in v100). The cursor "next row" semantic with DESC NULLS LAST + ASC tiebreakers is: (updated_at < prev) OR (updated_at = prev AND page_id > prev_page_id) OR (updated_at = prev AND page_id = prev_page_id AND chunk_index > prev_chunk_index) First page (afterUpdatedAt undefined AND afterPageId 0) bypasses the cursor predicate. Both engines parity-tested via 100/100 pglite-engine tests; Postgres path mirrors the same WHERE clause structure. executeRaw gains: - opts?: {signal?: AbortSignal} Postgres impl: real cancellation via postgres.js's .cancel() on the pending query. Pre-aborted signal short-circuits before the network round-trip; mid-flight abort fires .cancel(). The query throws on abort which the caller catches. PGLite impl: in-process WASM has no kernel-level cancellation. Best-effort: pre-check, then race the query against a signal-rejection promise. The query keeps running in WASM but the awaited result is discarded (DOMException AbortError thrown). Documented gap. ReservedConnection.executeRaw extends the signature for type compatibility but doesn't wire the signal (its only callers are migrations + cycle-lock writes that explicitly don't want cancellation). V99 timeline dedup follow-on: the dedup widening in migration v99 changed the unique index from (page_id, date, summary) to (page_id, date, summary, source). The ON CONFLICT clauses in both engines' addTimelineEntriesBatch + addTimelineEntry impls were still using the old 3-tuple, causing 12 PGLite tests to fail with SQLSTATE 42P10 "no unique constraint matching ON CONFLICT specification". Updated all 4 sites (2 per engine) to the 4-tuple. Typecheck clean, 100/100 PGLite engine tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(embed): --batch-size + --priority recent + --catch-up + embed-catch-up handler (A13) CLI surface on gbrain embed gains 3 flags: --batch-size N Override hardcoded PAGE_SIZE=2000 (clamped 1..10000) --priority recent Walk stale chunks newest-first (page.updated_at DESC) backed by content_chunks_stale_idx + idx_pages_updated_at_desc via T5's listStaleChunks(orderBy='updated_desc') extension. Composite cursor (updated_at, page_id, chunk_index). --catch-up Removes the GBRAIN_EMBED_TIME_BUDGET_MS wall-clock cap; loops until countStaleChunks() returns 0. EmbedOpts gains matching fields; embedAll + embedAllStale plumb them through. The cursor tracking in embedAllStale now advances (afterUpdatedAt, afterPageId, afterChunkIndex) instead of just (afterPageId, afterChunkIndex) when in 'updated_desc' mode. The engine returns p.updated_at as Date|string; the caller normalizes to ISO string for the next page's cursor. New Minion handler `embed-catch-up` registered in jobs.ts. Wraps runEmbedCore with stale=true + catchUp=true + the priority/batchSize the caller supplies. NOT in PROTECTED_JOB_NAMES (embedding spend only — same posture as the existing embed-backfill handler). Consumed by the gbrain onboard remediation pipeline (T11) when embed_staleness check fires. 63 embed tests pass; typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(extract): NER link extraction via schema-pack inference.regex (A10, T7, codex #12) NEW src/core/extract-ner.ts: extractNerLinks(engine, opts). Walks pages, reuses the by-mention gazetteer, applies the active schema-pack's link_types[].inference.regex patterns to assign a typed verb to each mention ("CEO of Acme" + Acme is a company → 'works_at' linking the source page to Acme). Codex finding #12 design: do NOT split link_source='ner' as a new provenance. NER is still mention-derived; splitting would break every existing link_source='mentions' query (backlink-count, orphan-ratio, doctor checks). Instead: keep link_source='mentions' AND set link_kind='typed_ner' (v98 column). LinkBatchInput type gains link_kind field. Both engines' addLinksBatch impls add the column to the INSERT projection + unnest() tuple (column #11). The links UNIQUE constraint excludes link_kind so an existing plain mention row + a typed_ner row for the same (from, to, type, source, origin) collide DO NOTHING; the typed link goes in as a separate row with a DIFFERENT link_type (the inferred verb), so they don't collide on the typical case. CLI: `gbrain extract links --ner` (DB source only). Combined `--by-mention --ner` walk shares ONE gazetteer build across both passes — saves a full walk on big brains. Either flag alone runs its pass solo. Each gets its own --source-id filter inheritance. Minion handler: `extract-ner` (NOT in PROTECTED_JOB_NAMES — regex-only, no LLM spend). Consumed by onboard's entity_link_coverage remediation when coverage <70%. Target-type lookup: one round-trip SELECT slug, source_id, type FROM pages WHERE type IN ('person', 'company', 'organization', 'entity') AND deleted_at IS NULL — built once at extraction start, consulted per-mention. Avoids the N+1 getPage cost. Pack best-effort: when no active pack OR no link_types declared OR no inference.regex on any link_type, returns pack_unavailable=true and 0 created. CLI prints a one-line note; handler returns silently. 122 tests pass (pglite-engine + by-mention); typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(extract): timeline from meetings — gbrain extract timeline --from-meetings (A11, T8, codex #11) NEW src/core/extract-timeline-from-meetings.ts: extractTimelineFromMeetings(engine, opts). Walks meeting pages, finds discussed entities via two sources, writes a timeline entry on each entity page. Discussed-entity sources merged: 1. Existing 'attended' links from the meeting (canonical attendees). One round-trip SELECT pulls all attended edges for the loaded meeting set; in-memory Map<meetingSlug → attendees[]> for O(1) lookup per meeting. 2. Body-text mentions via the existing by-mention gazetteer (findMentionedEntities + cross-source guard). Catches entities discussed in the meeting body even when no explicit 'attended' link exists. De-duped via Map<sourceId::slug → entity> within each meeting so a person who's both an attendee AND mentioned in the body gets exactly one timeline row per meeting, not two. Timeline write uses TimelineBatchInput with: source = 'extract-timeline-from-meetings:<meeting-slug>' summary = 'Discussed in <meeting-title>' date = meeting.effective_date Per v99 dedup widening (codex #11): the source field is now in the uniqueness key (page_id, date, summary, source). Two meetings on the same date with the same summary on the same entity page survive as distinct rows — the second meeting's provenance is no longer silently dropped. CLI: `gbrain extract timeline --from-meetings` (DB source only). Mode dispatch — runs SOLO (does not combine with --by-mention/--ner; those are links passes). Minion handler: `extract-timeline-from-meetings` (NOT in PROTECTED_JOB_NAMES — pure SQL + string scan). Consumed by onboard's timeline_coverage remediation when coverage <90%. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(takes): takes-bootstrap from concept/atom/lore pages (A12, A24, T9) NEW src/core/extract-takes-from-pages.ts: Haiku classifier loop. Walks pages WHERE type IN ('concept','atom','lore','briefing','writing', 'originals') AND deleted_at IS NULL AND length(compiled_truth) > 200, ordered by updated_at DESC. Each page is truncated to 20K chars and sent to Haiku with a strict-JSON classifier prompt: {"claim", "kind": fact|take|bet|hunch, "weight": 0..1} Inserts via addTakesBatch with source='cli:takes-bootstrap-from-pages'. Two-gate consent per A12: 1. `takes.bootstrap_enabled` config (default false) — even the manual CLI refuses without it explicitly set. 2. --yes flag (CLI) — interactive confirmation that this sends content to Haiku. The handler-side gate also reads takes.bootstrap_enabled, so even a trusted local Minion submitter (allowProtectedSubmit=true) cannot fire takes-bootstrap on a brain that hasn't opted in. CLI: `gbrain takes extract --from-pages [--yes] [--dry-run] [--source-id X] [--max-pages N] [--holder name]`. Surfaces consent-gate-blocked vs llm-unavailable distinctly so users see the actual blocker. Minion handler `extract-takes-from-pages` added to PROTECTED_JOB_NAMES. Consumed by onboard's takes_count remediation when count=0 AND takes.bootstrap_enabled=true (handler-side double-check). Per A24: ships with classifier infrastructure ONLY. Per-prompt eval suite deferred to v0.42.1 follow-up; autopilot remediation tier for takes-bootstrap stays manual_only until eval coverage catches up. Manual `gbrain takes extract --from-pages --yes` is the only path that triggers it in v0.42.0. parseClaimsJson exported for unit testing — strict JSON parse + ```json fence strip + kind allowlist filter, returns [] on any parse failure. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(minions): recordMinionJobSpend primitive for MCP client_id attribution (A7+A23, codex finding #4) NEW src/core/minion-spend.ts: small primitive that closes the per-OAuth- client spend chain gap codex flagged when MCP run_onboard submits child Minion jobs. Pre-fix: only subagent loops via budget-meter.ts recorded spend against the originating OAuth client. Generic Minion handlers (embed-catch-up, extract-ner, extract-timeline-from-meetings, extract-takes-from-pages) wrote to the gateway with no per-client attribution — admin-scope tokens would have unbounded indirect spend via the run_onboard fan-out. Convention for v0.42.0 (deferred schema column to v0.42.1): - run_onboard MCP op sets job.data.client_id when submitting each child handler. - Handlers that spend LLM/embedding budget call recordMinionJobSpend(engine, job, {operation, spendCents, ...}) which reads job.data.client_id and writes mcp_spend_log with the right attribution. - Local-submitted jobs (CLI, autopilot tick) pass no client_id; the row still lands with client_id=null for global accounting. Two exports: getJobClientId(job): undefined for local jobs; the OAuth client_id string for MCP-submitted ones. recordMinionJobSpend(engine, job, entry): wraps recordSpend with job-aware attribution. Best-effort throughout — spend telemetry failures MUST NOT fail the user's call. A23 full schema column (minion_jobs.client_id + index) deferred to v0.42.1; today's JSONB-pass-through is sufficient for the MCP run_onboard chain to land per-client attribution end-to-end. Handlers adopt the primitive over time; no behavior change for callers that haven't migrated. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboard): impact capture module + writeImpactLogRow primitive (A6 + A25 + A17, T11) NEW src/core/onboard/impact-capture.ts. Three exports: captureMetric(engine, metric) Pure-ish: returns the current numeric value for one of 5 metrics (orphan_count, stale_count, entity_link_coverage, timeline_coverage, takes_count). Returns null on any throw per A17 best-effort posture — a stat-query failure MUST NOT block the extraction itself. writeImpactLogRow(engine, attribution, metric, before, after, details?) Best-effort INSERT into v100's migration_impact_log table. Attribution columns (job_id, source_id, brain_id, started_at, idempotency_key, applied_by) per A25 + codex finding #10 so concurrent runs can't misattribute deltas. withImpactCapture(engine, attribution, metric, runner, details?) Convenience: capture-before → run → capture-after → write log row. Per A17 the log row lands even when the runner throws (after-on-fail + error in details), so downstream consumers see a "ran but impact unknown" entry instead of silent loss. Designed to be picked up by the 4 new Minion handlers (embed-catch-up, extract-ner, extract-timeline-from-meetings, extract-takes-from-pages) when they wrap their main runner. Handlers stay decoupled from the log-write path — they just call withImpactCapture with the metric they move. Per-handler integration follows in T12/T13/T15 as those wrappers land. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboard): types + render layer (A8, T12) NEW src/core/onboard/types.ts: OnboardRecommendation (extends RemediationStep with apply_policy + prompt_text + migration_id), OnboardReport (stable JSON envelope), OnboardOpts. NEW src/core/onboard/render.ts: toOnboardRecommendation(step): RemediationStep → OnboardRecommendation Sets apply_policy per A8 tiered rules: - protected + job === extract-takes-from-pages → 'manual_only' (A12/A24) - protected + other → 'prompt_required' - non-protected → 'auto_apply' buildOnboardReport(plan, opts?): assembles the stable JSON envelope. renderHuman(report): string. Echoes the "Recommendation + WHY" framing the CEO + Eng + Codex reviews settled on; CLI shell prints to stdout. Stable JSON envelope shape: schema_version: 1 brain_id?: string recommendations: OnboardRecommendation[] summary: { total, auto_eligible, prompt_required, manual_only, est_total_usd } history?: Array<{ remediation_id, metric_name, metric_before, metric_after, delta, applied_at }> Library-shaped — no console.* / process.exit. T13 (onboard CLI shell) calls these from the wrapping CLI. MCP run_onboard (T16) returns the JSON envelope unmodified. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboard): gbrain onboard CLI shell (A1, T13) NEW src/commands/onboard.ts (~180 LOC). Thin wrapper that composes: - T2 library (computeRemediationPlan + runRemediation) - T4 onboard checks (runAllOnboardChecks → extraRemediations) - T12 render layer (buildOnboardReport + renderHuman) Three modes: --check (default): print plan, no submission. Computes plan via T2 library with T4 check-derived extraRemediations. Renders human (default) or JSON envelope (--json). --auto: submit auto_apply tier. Requires --max-usd N (cron-safety per A12 + A20 — refuses without explicit cap to avoid surprise spend). --auto --yes: also submit prompt_required tier. --history: dump last 50 migration_impact_log entries. Library hooks wired into stderr (per CLI/library separation): onStepStart, onStepEnd, onBudgetRefused, onBudgetExhausted, onNothingToDo, onTargetUnreachable. Final JSON envelope (--json) or human summary lands on stdout. CLI dispatch: registered in src/cli.ts CLI_ONLY set + case dispatch between 'takes' and 'founder'. Typecheck clean. Manual smoke-test pending T20 E2E (DATABASE_URL gated). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(onboard): init nudge + upgrade banner (A4, A18, A20, T14) NEW src/core/onboard/init-nudge.ts exports two fail-open hooks: runInitNudge(engine): Post-initSchema 5-query AbortSignal-bound parallel check against a 3-second wallclock budget. Per A20: uses REAL cancellation via the T5 executeRaw signal extension — Promise.race against a timer was codex's #7 wrong shape. Postgres queries actually .cancel(); PGLite documented gap. Partial-results path: if some checks complete and the budget fires on others, prints what landed + a fallthrough hint pointing at `gbrain onboard --check` for the full picture. Per A18: fail-open — ANY throw is caught, logged to stderr, and suppressed so init returns successfully. Bypass: GBRAIN_NO_ONBOARD_NUDGE=1 short-circuits. Non-TTY default short-circuits too (CI/scripted callers see nothing). Nudge format: one-line summary of opportunities ("Brain has opportunities: 23000 stale chunks, link coverage 32%, 0 takes") + a 'gbrain onboard --check' nudge. runUpgradeBanner(_engine): Lighter post-upgrade banner. Doesn't engine-query — just prints a one-line nudge that upgrades may surface new opportunities. Same fail-open posture. Wired into: src/commands/init.ts:initPGLite (end-of-function, after reportModStatus) src/commands/init.ts:initPostgres (same) src/commands/upgrade.ts:runPostUpgrade (end-of-function, after postUpgradeReferenceSweep) Each wire site uses dynamic import + try/catch so even an import failure can't crash init/upgrade. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(autopilot): tick consults onboard recommendations (A5, A19, A22, T15) Pre-fix: autopilot tick's per-source recommendation walk called computeRecommendations(health, ctx) — doctor's hardcoded 5-category planner. The 4 new onboard checks (embed_staleness, entity_link_coverage, timeline_coverage, takes_count) had nowhere to hook in, so even with takes.bootstrap_enabled flipped on, autopilot never noticed 0 takes and never proposed bootstrap. Post-fix: tick body now ALSO calls runAllOnboardChecks(engine) and threads the result's RemediationStep[] into the T3-generalized third arg of computeRecommendations. The planner merges onboard's extras with the legacy hardcoded entries (hardcoded wins on id collision). Per A19 fail-open: any throw in the onboard-checks path is caught, logged to stderr, and suppressed. The legacy plan (without extras) runs as before — autopilot can't crash from an onboard-check failure. A22 (idempotency-key dedupe across concurrent manual + autopilot runs): inherits from the existing computeRecommendations → remediation.idempotency_key chain. T7-T9 handlers each get their content-hash key from the makeRemediationStep factory; an autopilot tick + a manual `gbrain onboard --auto` submitting the same step in the same brain produce the SAME key, so queue.add(...) dedupes. No behavior change for brains where all 4 onboard metrics already look healthy (extras=[]; legacy plan unchanged). Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(mcp): run_onboard op with run_protected_onboard scope binding (A7, T16, codex finding #5) NEW MCP op `run_onboard`. Admin scope (NOT localOnly) so federated / thin-client brain installs can probe brain health + submit auto-eligible remediation handlers over OAuth-authenticated MCP. Two-tier authorization per A7 + codex #5: - Admin scope: sufficient for mode='check' (read-only OnboardReport JSON) AND for submitting non-protected handlers in mode='auto'/'auto-with-prompt'. - run_protected_onboard scope (NEW, additive): MUST be granted in addition to admin for any PROTECTED_JOB_NAMES handler to fire (synthesize, patterns, consolidate, extract-takes-from-pages, contextual_reindex_per_chunk). Without the new scope tier, an admin-scoped OAuth token would silently bypass the same protected-name gate `submit_job` enforces at operations.ts:2288. The codex finding #5 caught this: admin scope alone was insufficient guard. Now the run_onboard op explicitly FILTERS protected extras from the recommendation plan when the caller lacks run_protected_onboard; filtered items appear in the response as skipped_missing_scope[] so the caller knows what would have been available with the right grants. Modes: check — read-only OnboardReport JSON envelope. auto — submits auto_apply tier (plus prompt_required when --yes/auto-with-prompt). auto-with-prompt — adds prompt_required tier. Both auto modes REQUIRE max_usd per A12 + A20 cron-safety (rejects with invalid_params if missing). Per A26 source-scope: future extension will scope plans by ctx.sourceId / ctx.auth.allowedSources. Today the recommendation planner is brain-wide; the source-scope thread doesn't change correctness, just optimization. Per A19 fail-open: any error in runAllOnboardChecks during plan-build caught + suppressed; the plan still returns with extras=[] rather than crashing the op. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(verify): add check-source-scope-onboard lint (A26, T17) NEW scripts/check-source-scope-onboard.sh. Grep guard for SQL sites in onboard surfaces (src/core/onboard/, src/commands/onboard.ts) that touch source_id-bearing tables (pages, content_chunks, takes, links, timeline_entries) WITHOUT either: (a) source_id / sourceIds in the WHERE clause, OR (b) the opt-out marker `sourcescope:brain-wide` within 4 lines above the SQL. File-level opt-out: `sourcescope:file-brain-wide` in the file header (first 30 lines) treats every SQL site in that file as intentionally brain-wide. Used by onboard/checks.ts, onboard/impact-capture.ts, and commands/onboard.ts because the onboard CHECKS are explicitly brain-wide aggregates (orphan_count, stale_count, link_coverage are reported across all sources by design). Wired into bun run verify (23 checks total now, all green). Without this gate, any future onboard SQL touching per-source data without source-scoping would silently leak rows across sources — exactly the class of bug v0.34.1's P0 seal closed at the engine layer. The lint adds an explicit forcing function for new code in the onboard surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(install): onboard surface agent prescription (D13, T18) Adds a v0.42.0+ section to INSTALL_FOR_AGENTS.md describing: - First-connect probe: gbrain onboard --check --json - Post-upgrade re-probe (after gbrain upgrade) - Unattended remediation: gbrain onboard --auto --max-usd 5 - MCP run_onboard op for federated/thin-client installs - run_protected_onboard scope requirement for LLM-bearing handlers - Two-gate consent for takes-bootstrap (takes.bootstrap_enabled + --yes) - GBRAIN_NO_ONBOARD_NUDGE=1 bypass for CI Per D13: agents should run --check on first connect AND after every upgrade as a hygiene step. The autopilot path makes this auto-improve on a 24h cycle; the explicit agent probe surfaces opportunities immediately on connect rather than waiting for the next autopilot tick. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): hermetic onboard surface contracts (T20) NEW test/e2e/onboard-full-flow.test.ts. 13 hermetic PGLite cases (no DATABASE_URL needed) covering the key onboard contracts: captureMetric — all 5 metrics return expected values on empty brain (0 for counts; 1 for coverage = vacuous truth). runAllOnboardChecks — returns exactly 4 results with correct names; empty brain shows stale/link/timeline ok BUT takes_count warns (0 takes); 0 remediations emitted because takes.bootstrap_enabled defaults to false per A12 two-gate consent. computeRemediationPlan — extras (T3 generalization) thread through to plan.plan output; stable schema_version: 2 envelope. buildOnboardReport — stable schema_version: 1 envelope with the right summary fields populated. toOnboardRecommendation tier policy (A8): - non-protected job → auto_apply - extract-takes-from-pages → manual_only (A12 + A24) - other protected jobs (synthesize, patterns, ...) → prompt_required Full DATABASE_URL-gated end-to-end (real Postgres, actual extractions through Minion handlers) deferred to v0.42.1 once the per-handler test seam lands; the hermetic suite covers the data-shape contracts that matter for downstream consumers binding to the JSON envelopes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.42.0.0 gbrain onboard mega PR — activation surface (closes #1383, completes #1409) VERSION + package.json bumped to 0.42.0.0. CHANGELOG with full ELI10 lead + "What you can do that you couldn't before" itemized list + "To take advantage of v0.42.0.0" upgrade steps per CLAUDE.md voice rules. TODOS.md: 9 follow-up items filed (TODO-A through TODO-I) for the v0.42.1+ wave: pack-aware linkable types, LLM-disambiguation NER, onboard --explain, live-brain impact measurement, 100+-case takes classifier eval, admin SPA UI, full DATABASE_URL E2E, minion_jobs client_id schema column, thin-client doctor-remote parity. llms-full.txt regenerated per CLAUDE.md rule (every CHANGELOG edit followed by bun run build:llms in the same commit). 23/23 verify checks pass. Full implementation across 21 commits on this branch (T0-T21): T0 merge master T1 schema migrations v98/v99/v100 T2 extract doctor remediation library T3 generalize computeRecommendations T4 4 new doctor checks T5 engine API: listStaleChunks orderBy + executeRaw AbortSignal T6 embed --batch-size / --priority recent / --catch-up T7 NER extraction + extract-ner handler T8 timeline-from-meetings + extract-timeline-from-meetings handler T9 takes-bootstrap + extract-takes-from-pages handler T10 recordMinionJobSpend primitive T11 impact capture module + writeImpactLogRow T12 onboard render layer (types + render) T13 gbrain onboard CLI shell T14 init nudge + upgrade banner T15 autopilot tick consults onboard T16 MCP run_onboard + run_protected_onboard scope T17 check-source-scope-onboard lint T18 INSTALL_FOR_AGENTS.md agent prescription T20 hermetic PGLite E2E (13 cases) T21 ship (this commit) Reviews: CEO + Eng + Codex on plan ~/.claude/plans/system-instruction-you-are-working-lively-hollerith.md. 27 A-decisions locked; 18 codex findings absorbed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): connection-resilience regex + doctor warn-not-fail + v0.41.18.0 Two CI fixes from PR #1521 + version renumber per user request. Why fix #1 (connection-resilience.test.ts): T5/A20 extended PostgresEngine.executeRaw signature to accept an optional `opts?: { signal?: AbortSignal }` 3rd arg and rewrote the body as multi-line. The regression test's regex was anchored to the legacy single-line `(sql: string, params?: unknown[])` shape and the assertions banned `try {` / `catch` (which T5 legitimately added for AbortSignal cancellation swallow, NOT for retry). Updated regex to tolerate both shapes; replaced the wrong `not.toContain('conn.unsafe( sql, params')` assertion (which incorrectly flagged the legitimate single call) with a count assertion: `conn.unsafe(` must appear exactly ONCE in the body. Preserves the original D3 intent (no per-call retry — recovery is supervisor-driven via reconnect()) while accepting the new try/catch shape that swallows AbortSignal aborts. Why fix #2 (src/core/onboard/checks.ts): Three of the four new onboard doctor checks (entity_link_coverage, timeline_coverage, embed_staleness) emitted `status = 'fail'` on healthy DBs that simply hadn't run extractions yet. This flipped `gbrain doctor`'s exit code to non-zero on freshly initialized brains, breaking test/e2e/mechanical.test.ts:1280 ("gbrain doctor exits 0 on healthy DB"). Downgraded all three to `status = 'warn'` — these are remediation opportunities, not assertion failures. Doctor exit codes are reserved for actual failures; remediation surfaces use warn-level signaling so they can be picked up by `--remediate` without polluting the exit code. Why fix #3 (version renumber 0.42.0.0 → 0.41.18.0): Per user directive, this wave ships as v0.41.18.0 rather than v0.42.0.0. Master is at 0.41.16.0; 0.41.17.0 is reserved for an in-flight wave. Renamed every reference my branch added (54 files touched): VERSION, package.json, CHANGELOG.md header, TODOS.md, plus inline version-stamp comments across src/, test/, and scripts/. Preserved 13 files with PRE-EXISTING `v0.42.0.0` references on master (from earlier waves originally planned for v0.42 that landed at v0.41.x — those stay as historical record). Verified via per-file diff against origin/master: every renamed reference is one I added in this branch. Audit trio aligned: VERSION=0.41.18.0, package.json=0.41.18.0, CHANGELOG topmost entry=[0.41.18.0]. llms-full.txt regenerated to match CLAUDE.md updates. Bisect contract: this commit fixes CI test failures from PR #1521's landing. Typecheck clean; connection-resilience suite 26/26 pass. Refs A20 (executeRaw AbortSignal), A16 (4 new onboard checks), codex #1 (master collision avoidance via renumber). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1992 lines
87 KiB
TypeScript
Executable File
1992 lines
87 KiB
TypeScript
Executable File
#!/usr/bin/env bun
|
|
|
|
import { installSigchldHandler } from './core/zombie-reap.ts';
|
|
installSigchldHandler();
|
|
// v0.41.6.0 D5: cleanup registry + signal handlers for SIGTERM/SIGHUP/SIGPIPE/
|
|
// uncaughtException. NOT SIGINT (the existing AbortController path at :254
|
|
// owns SIGINT). Installed at module load so locks acquired during boot
|
|
// (e.g. during connectEngine's schema-probe path) are covered too.
|
|
import { installSignalHandlers as installCleanupSignalHandlers } from './core/process-cleanup.ts';
|
|
installCleanupSignalHandlers();
|
|
|
|
import { readFileSync } from 'fs';
|
|
import { loadConfig, loadConfigWithEngine, toEngineConfig, isThinClient } from './core/config.ts';
|
|
import type { GBrainConfig } from './core/config.ts';
|
|
import type { AIGatewayConfig } from './core/ai/types.ts';
|
|
import type { BrainEngine } from './core/engine.ts';
|
|
import { operations, OperationError } from './core/operations.ts';
|
|
import type { Operation, OperationContext } from './core/operations.ts';
|
|
import { awaitPendingLastRetrievedWrites, type DrainOutcome } from './core/last-retrieved.ts';
|
|
import { shouldForceExitAfterMain } from './core/cli-force-exit.ts';
|
|
import { serializeMarkdown } from './core/markdown.ts';
|
|
import { parseGlobalFlags, setCliOptions, getCliOptions } from './core/cli-options.ts';
|
|
import type { CliOptions } from './core/cli-options.ts';
|
|
import { callRemoteTool, RemoteMcpError, unpackToolResult } from './core/mcp-client.ts';
|
|
import { maybePromptForUpgrade } from './core/thin-client-upgrade-prompt.ts';
|
|
import { VERSION } from './version.ts';
|
|
|
|
// Build CLI name -> operation lookup
|
|
const cliOps = new Map<string, Operation>();
|
|
for (const op of operations) {
|
|
const name = op.cliHints?.name;
|
|
if (name && !op.cliHints?.hidden) {
|
|
cliOps.set(name, op);
|
|
}
|
|
}
|
|
|
|
// CLI-only commands that bypass the operation layer
|
|
const CLI_ONLY = new Set(['init', 'reinit-pglite', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'extract-conversation-facts', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder', 'brainstorm', 'lsd', 'schema', 'capture', 'onboard', 'conversation-parser']);
|
|
// CLI-only commands whose handlers print their own --help text. These are
|
|
// excluded from the generic short-circuit so detailed per-command and
|
|
// per-subcommand usage stays reachable.
|
|
const CLI_ONLY_SELF_HELP = new Set([
|
|
'upgrade', 'post-upgrade', 'check-update',
|
|
'embed', 'config',
|
|
'skillpack', 'skillpack-check',
|
|
'integrations', 'friction',
|
|
'frontmatter', 'check-resolvable',
|
|
'models',
|
|
'cache',
|
|
'brainstorm', 'lsd',
|
|
// v0.39.3.0 WARN-5: capture's detailed HELP constant
|
|
// (src/commands/capture.ts:90+) was unreachable because the dispatcher's
|
|
// generic short-circuit (printCliOnlyHelp at :204-208) fired before
|
|
// runCapture saw --help. brainstorm + lsd were already in the set;
|
|
// capture was the holdout.
|
|
'capture',
|
|
// v0.37 fix wave (Lane D.4 + CDX2-12): sync's --no-embed flag was
|
|
// unreachable via help because the dispatcher's generic CLI-only
|
|
// short-circuit fired before runSync could print its own usage block.
|
|
// Adding `sync` here routes `gbrain sync --help` into runSync.
|
|
'sync',
|
|
// v0.37 fix wave (deferred TODO, shipped): reinit-pglite has its
|
|
// own --help in runReinitPglite. Routing through SELF_HELP avoids
|
|
// the generic short-circuit so the destructive-action warning text
|
|
// reaches the user.
|
|
'reinit-pglite',
|
|
// v0.40.6.0 Schema Cathedral v3 — `gbrain schema --help` should hit
|
|
// schema.ts printHelp() with the full 22+ verb taxonomy, not the
|
|
// generic short-circuit's one-line stub.
|
|
'schema',
|
|
// v0.41.11.0 — extract-conversation-facts ships its own detailed HELP
|
|
// describing segment splitting + checkpointing + budget caps + the
|
|
// unified types config story. Route around the generic short-circuit.
|
|
'extract-conversation-facts',
|
|
]);
|
|
|
|
async function main() {
|
|
// Parse global flags (--quiet / --progress-json / --progress-interval)
|
|
// BEFORE command dispatch, so `gbrain --progress-json doctor` works.
|
|
// The stripped argv is what the command sees.
|
|
const rawArgs = process.argv.slice(2);
|
|
const { cliOpts, rest: args } = parseGlobalFlags(rawArgs);
|
|
setCliOptions(cliOpts);
|
|
|
|
let command = args[0];
|
|
|
|
if (!command || command === '--help' || command === '-h') {
|
|
printHelp();
|
|
return;
|
|
}
|
|
|
|
if (command === '--version' || command === 'version') {
|
|
console.log(`gbrain ${VERSION}`);
|
|
return;
|
|
}
|
|
|
|
if (command === '--tools-json') {
|
|
const { printToolsJson } = await import('./commands/tools-json.ts');
|
|
printToolsJson();
|
|
return;
|
|
}
|
|
|
|
const subArgs = args.slice(1);
|
|
|
|
// DX alias: `ask` is a natural-language alias for `query`
|
|
if (command === 'ask') {
|
|
command = 'query';
|
|
}
|
|
|
|
// Per-command --help
|
|
if (hasHelpFlag(subArgs)) {
|
|
const op = cliOps.get(command);
|
|
if (op) {
|
|
printOpHelp(op);
|
|
return;
|
|
}
|
|
if (CLI_ONLY.has(command) && !CLI_ONLY_SELF_HELP.has(command)) {
|
|
printCliOnlyHelp(command);
|
|
return;
|
|
}
|
|
}
|
|
|
|
// CLI-only commands
|
|
if (CLI_ONLY.has(command)) {
|
|
await handleCliOnly(command, subArgs);
|
|
return;
|
|
}
|
|
|
|
// Shared operations
|
|
const op = cliOps.get(command);
|
|
if (!op) {
|
|
console.error(`Unknown command: ${command}`);
|
|
console.error('Run gbrain --help for available commands.');
|
|
process.exit(1);
|
|
}
|
|
|
|
// v0.31.1 (Issue #734, CDX-1): parse CLI args BEFORE engine connect so
|
|
// the routing seam below can decide local-vs-remote without paying a
|
|
// PGLite migration replay on thin-client installs. The arg parser, image
|
|
// transform, and required-param check are all engine-free; refactoring
|
|
// them out of the engine try/catch is safe and unlocks routing.
|
|
const params = parseOpArgs(op, subArgs);
|
|
|
|
// v0.27.1 (`gbrain query --image <path>`): swap the `image` param from
|
|
// a filesystem path into base64 bytes + mime. The op accepts base64; the
|
|
// CLI accepts a path. Helper is exported so tests can exercise the
|
|
// transform without spawning a subprocess.
|
|
if (op.name === 'query' && typeof params.image === 'string' && params.image.length > 0) {
|
|
try {
|
|
const { path, base64, mime } = resolveQueryImage(
|
|
params.image as string,
|
|
(params.image_mime as string) || undefined,
|
|
);
|
|
params.image = base64;
|
|
params.image_mime = mime;
|
|
void path;
|
|
} catch (err) {
|
|
console.error(err instanceof Error ? err.message : String(err));
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
// Validate required params before calling handler. v0.27.1: the
|
|
// `query` op's positional `query` is required only when --image is
|
|
// NOT supplied. The runtime altRequired check below overrides the
|
|
// generic required-flag check for that op.
|
|
const queryHasAlt = op.name === 'query' && typeof params.image === 'string' && params.image.length > 0;
|
|
for (const [key, def] of Object.entries(op.params)) {
|
|
if (def.required && params[key] === undefined) {
|
|
if (queryHasAlt && key === 'query') continue;
|
|
const cliName = op.cliHints?.name || op.name;
|
|
const positional = op.cliHints?.positional || [];
|
|
const usage = positional.map(p => `<${p}>`).join(' ');
|
|
console.error(`Usage: gbrain ${cliName} ${usage}`);
|
|
process.exit(1);
|
|
}
|
|
}
|
|
|
|
// v0.31.1 (Issue #734, CDX-1 routing seam): on thin-client installs,
|
|
// route every non-localOnly op through callRemoteTool instead of opening
|
|
// the empty local PGLite. localOnly ops can't run on a thin client at all
|
|
// (no local engine, server intentionally hides them) — refuse with hint.
|
|
// Fix for the silent-empty-results bug class that motivated this whole release.
|
|
const cfgPre = loadConfig();
|
|
if (isThinClient(cfgPre)) {
|
|
if (op.localOnly) {
|
|
refuseThinClient(command, cfgPre!.remote_mcp!.mcp_url);
|
|
}
|
|
await runThinClientRouted(op, params, cfgPre!, cliOpts);
|
|
return;
|
|
}
|
|
|
|
// Local engine path (unchanged behavior for local installs).
|
|
const engine = await connectEngine();
|
|
// v0.41.8.0 (#1247, #1269, #1290): the search / query / get_page
|
|
// op handlers fire-and-forget `bumpLastRetrievedAt` after returning
|
|
// results. On PGLite that IIFE keeps Bun's event loop alive past
|
|
// engine.disconnect(), hanging the CLI at ~95-98% CPU until SIGKILL.
|
|
// Drain the fire-and-forget set BEFORE disconnect; force-exit only
|
|
// if the drain itself times out (preserves stderr diagnostic signal
|
|
// AND guarantees the CLI doesn't re-hang at the disconnect layer).
|
|
//
|
|
// Defense-in-depth (adversarial-review C13): `engine.disconnect()` itself
|
|
// can hang on PGLite (db.close() or releaseLock racing OS-level FS state).
|
|
// Install an unref'd setTimeout hard-exit fallback BEFORE entering the
|
|
// try/catch/finally so a hung disconnect cannot defeat the force-exit
|
|
// contract. Daemons (`serve`) are excluded so they stay alive.
|
|
const DISCONNECT_HARD_DEADLINE_MS = 10_000;
|
|
let forceExitTimer: ReturnType<typeof setTimeout> | undefined;
|
|
if (shouldForceExitAfterMain()) {
|
|
forceExitTimer = setTimeout(() => {
|
|
console.warn(
|
|
`[cli] engine.disconnect() did not return within ${DISCONNECT_HARD_DEADLINE_MS}ms — force-exiting`,
|
|
);
|
|
process.exit(0);
|
|
}, DISCONNECT_HARD_DEADLINE_MS);
|
|
// unref so the timer itself doesn't keep the event loop alive — only
|
|
// the actual pending work (PGLite WASM handle) does. Without unref,
|
|
// we'd block a clean exit by 10s on every successful CLI run.
|
|
forceExitTimer.unref?.();
|
|
}
|
|
|
|
let drainResult: DrainOutcome = { outcome: 'drained', pending: 0 };
|
|
try {
|
|
const ctx = await makeContext(engine, params);
|
|
const rawResult = await op.handler(ctx, params);
|
|
// ENG-2 (renderer parity by data shape): JSON-round-trip the local-engine
|
|
// path's return value so renderers see the same shape they'd see on the
|
|
// routed path. Date → ISO string; bigint → string (postgres.js shape);
|
|
// Buffer → object. Microsecond-cost; eliminates a whole drift bug class.
|
|
const result = JSON.parse(JSON.stringify(rawResult));
|
|
const output = formatResult(op.name, result);
|
|
if (output) process.stdout.write(output);
|
|
if (op.name === 'query') {
|
|
const { awaitPendingSearchCacheWrites } = await import('./core/search/hybrid.ts');
|
|
await awaitPendingSearchCacheWrites();
|
|
}
|
|
// Drain unconditionally for every op — empty-set fast-path is a
|
|
// few microseconds. Not per-op-name gated: that was the original
|
|
// PR #1259 mistake that left search and get_page exposed.
|
|
drainResult = await awaitPendingLastRetrievedWrites();
|
|
} catch (e: unknown) {
|
|
// C9 fix: drain BEFORE process.exit so a successful op that throws
|
|
// during stdout/format still gets its bumpLastRetrievedAt UPDATE
|
|
// a chance to commit. Bounded by the drain's own 5s timeout; the
|
|
// outer hard-exit timer above bounds the disconnect path.
|
|
try { await awaitPendingLastRetrievedWrites(); } catch { /* best-effort */ }
|
|
if (e instanceof OperationError) {
|
|
console.error(`Error [${e.code}]: ${e.message}`);
|
|
if (e.suggestion) console.error(` Fix: ${e.suggestion}`);
|
|
process.exit(1);
|
|
}
|
|
console.error(e instanceof Error ? e.message : String(e));
|
|
process.exit(1);
|
|
} finally {
|
|
await engine.disconnect();
|
|
if (forceExitTimer) clearTimeout(forceExitTimer);
|
|
// Narrow force-exit: only when the drain timed out AND we are NOT
|
|
// running a daemon. The drain helper already stderr-warned with the
|
|
// pending count, so the diagnostic signal is preserved. Without
|
|
// this guard a hung underlying promise can still keep Bun's loop
|
|
// alive past disconnect — Codex outside-voice finding #1.
|
|
if (drainResult.outcome === 'timeout' && shouldForceExitAfterMain()) {
|
|
process.exit(0);
|
|
}
|
|
}
|
|
}
|
|
|
|
|
|
function hasHelpFlag(args: string[]): boolean {
|
|
return args.includes('--help') || args.includes('-h');
|
|
}
|
|
|
|
function printCliOnlyHelp(command: string) {
|
|
console.log(`Usage: gbrain ${command}`);
|
|
console.log('');
|
|
console.log(`gbrain ${command} - run gbrain --help for the full command list.`);
|
|
}
|
|
|
|
/**
|
|
* v0.31.1 (Issue #734, CDX-1): route a shared op through the remote MCP
|
|
* server instead of running it locally. Called from main() when
|
|
* `isThinClient(cfg) && !op.localOnly`.
|
|
*
|
|
* Timeout policy (ENG-4): user override via --timeout=Ns wins; otherwise
|
|
* 180s for `think` (LLM calls), 30s for everything else.
|
|
*
|
|
* Error policy (CDX-4): callRemoteTool's hardening pass guarantees every
|
|
* thrown value reaches us as a RemoteMcpError. The switch below is
|
|
* exhaustively typed (TS `never` check); adding a new reason variant fails
|
|
* compilation until this dispatcher knows what to render.
|
|
*
|
|
* Renderer policy: the MCP tool result is unpacked via unpackToolResult
|
|
* (which JSON.parses the text content) and handed to the SAME formatResult
|
|
* the local-engine path uses. Renderer parity is enforced by data shape,
|
|
* not by per-command audit.
|
|
*/
|
|
async function runThinClientRouted(
|
|
op: Operation,
|
|
params: Record<string, unknown>,
|
|
cfg: GBrainConfig,
|
|
cliOpts: CliOptions,
|
|
): Promise<void> {
|
|
// ENG-4: per-op timeout default; user override wins.
|
|
const defaultTimeoutMs = op.name === 'think' ? 180_000 : 30_000;
|
|
const timeoutMs = cliOpts.timeoutMs ?? defaultTimeoutMs;
|
|
|
|
// SIGINT support: aborts in-flight HTTP cleanly (exit 130 is the standard
|
|
// SIGINT exit code; our error switch maps `network/aborted` to that).
|
|
const sigintController = new AbortController();
|
|
const onSigint = () => {
|
|
sigintController.abort(new Error('SIGINT'));
|
|
};
|
|
process.on('SIGINT', onSigint);
|
|
|
|
// v0.31.1 (Issue #734, cherry-pick B): print identity banner to stderr
|
|
// BEFORE the routed call. Banner failure suppresses the banner only —
|
|
// never the underlying command. Suppression honors --quiet, non-TTY,
|
|
// and GBRAIN_NO_BANNER=1.
|
|
await printIdentityBannerBestEffort(cfg, cliOpts, sigintController.signal);
|
|
|
|
try {
|
|
const raw = await callRemoteTool(cfg, op.name, params, {
|
|
timeoutMs,
|
|
signal: sigintController.signal,
|
|
});
|
|
const result = unpackToolResult(raw);
|
|
const output = formatResult(op.name, result);
|
|
if (output) process.stdout.write(output);
|
|
} catch (e: unknown) {
|
|
if (e instanceof RemoteMcpError) {
|
|
const url = cfg.remote_mcp!.mcp_url;
|
|
switch (e.reason) {
|
|
case 'config':
|
|
console.error(e.message);
|
|
break;
|
|
case 'discovery':
|
|
console.error(`OAuth discovery failed at ${cfg.remote_mcp!.issuer_url}.`);
|
|
console.error('Run `gbrain remote doctor` for details.');
|
|
break;
|
|
case 'auth':
|
|
console.error('OAuth auth failed.');
|
|
console.error('On the host, re-register your client:');
|
|
console.error(' gbrain auth register-client <name> --grant-types client_credentials --scopes read,write,admin');
|
|
break;
|
|
case 'auth_after_refresh':
|
|
console.error('OAuth auth failed after token refresh. Credentials may have been revoked.');
|
|
console.error('Run `gbrain remote doctor` to confirm.');
|
|
break;
|
|
case 'network':
|
|
if (e.detail?.kind === 'timeout') {
|
|
const hint = cliOpts.timeoutMs ? '' : ` (default ${defaultTimeoutMs}ms; pass --timeout=Ns to override)`;
|
|
console.error(`Request to ${url} timed out${hint}.`);
|
|
} else if (e.detail?.kind === 'aborted') {
|
|
console.error('Request aborted.');
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(130);
|
|
} else {
|
|
console.error(`Cannot reach ${url}. Run \`gbrain remote doctor\` for details.`);
|
|
}
|
|
break;
|
|
case 'tool_error':
|
|
if (e.detail?.code === 'missing_scope') {
|
|
console.error('Missing OAuth scope on this client.');
|
|
console.error('On the host, re-register the client with broader scopes:');
|
|
console.error(' gbrain auth register-client <name> --grant-types client_credentials --scopes read,write,admin');
|
|
} else {
|
|
console.error(e.message);
|
|
console.error('Run `gbrain remote doctor` if this persists.');
|
|
}
|
|
break;
|
|
case 'parse':
|
|
console.error('Server response was malformed. Run `gbrain remote doctor`.');
|
|
break;
|
|
default: {
|
|
// Exhaustive switch sentinel (TS `never` — fails to build if a
|
|
// new RemoteMcpErrorReason variant is added without a case).
|
|
const _exhaustive: never = e.reason;
|
|
void _exhaustive;
|
|
console.error(`Unhandled remote error: ${e.message}`);
|
|
}
|
|
}
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(1);
|
|
}
|
|
// Defense in depth: callRemoteTool's contract is that everything is
|
|
// RemoteMcpError. If a plain Error escapes, render it generically and
|
|
// exit 1 — but this should never happen post-CDX-4.
|
|
console.error(e instanceof Error ? e.message : String(e));
|
|
process.off('SIGINT', onSigint);
|
|
process.exit(1);
|
|
} finally {
|
|
process.off('SIGINT', onSigint);
|
|
}
|
|
}
|
|
|
|
// ============================================================================
|
|
// v0.31.1 (Issue #734, cherry-pick B): thin-client identity banner.
|
|
//
|
|
// Prints "[thin-client → <host> · brain: 102k pages, 265k chunks · vX.Y.Z]"
|
|
// to stderr before each routed command, so users (and agents) know they're
|
|
// talking to a real remote brain — not the empty local PGLite that motivated
|
|
// this whole release.
|
|
//
|
|
// Cache: 60s TTL, in-memory Map keyed by mcp_url. Cross-process file cache
|
|
// is deferred (marginal benefit; one mint per CLI process is fine).
|
|
// Suppression: --quiet, non-TTY, GBRAIN_NO_BANNER=1.
|
|
// Failure mode: any error in fetching identity → suppress banner; underlying
|
|
// command runs normally. Banner is observability, not load-bearing.
|
|
// ============================================================================
|
|
|
|
export interface BrainIdentity {
|
|
version: string;
|
|
engine: 'postgres' | 'pglite';
|
|
page_count: number;
|
|
chunk_count: number;
|
|
last_sync_iso: string | null;
|
|
}
|
|
|
|
interface CachedIdentity {
|
|
identity: BrainIdentity;
|
|
cached_at_ms: number;
|
|
}
|
|
|
|
const IDENTITY_TTL_MS = 60_000;
|
|
const identityCache = new Map<string, CachedIdentity>();
|
|
|
|
/** Test-only escape hatch — clears the in-memory cache between test runs. */
|
|
export function _clearIdentityCacheForTest(): void {
|
|
identityCache.clear();
|
|
}
|
|
|
|
export function bannerSuppressed(cliOpts: CliOptions): boolean {
|
|
if (cliOpts.quiet) return true;
|
|
if (process.env.GBRAIN_NO_BANNER === '1') return true;
|
|
// Non-TTY default is suppressed (clean pipes); explicit env-flag overrides.
|
|
if (!process.stderr.isTTY && process.env.GBRAIN_BANNER !== '1') return true;
|
|
return false;
|
|
}
|
|
|
|
function formatPageCount(n: number): string {
|
|
if (n >= 1000) {
|
|
const k = (n / 1000).toFixed(n >= 100_000 ? 0 : 1);
|
|
return `${k}k`;
|
|
}
|
|
return String(n);
|
|
}
|
|
|
|
function formatBanner(mcpUrl: string, id: BrainIdentity): string {
|
|
const host = mcpUrl.replace(/^https?:\/\//, '').split('/')[0];
|
|
const counts = `brain: ${formatPageCount(id.page_count)} pages, ${formatPageCount(id.chunk_count)} chunks`;
|
|
return `[thin-client → ${host} · ${counts} · v${id.version}]`;
|
|
}
|
|
|
|
async function fetchIdentity(
|
|
cfg: GBrainConfig,
|
|
signal: AbortSignal,
|
|
): Promise<BrainIdentity> {
|
|
// 2s timeout for the banner fetch — must not delay the underlying command.
|
|
const raw = await callRemoteTool(cfg, 'get_brain_identity', {}, {
|
|
timeoutMs: 2000,
|
|
signal,
|
|
});
|
|
const id = unpackToolResult<BrainIdentity>(raw);
|
|
return id;
|
|
}
|
|
|
|
async function printIdentityBannerBestEffort(
|
|
cfg: GBrainConfig,
|
|
cliOpts: CliOptions,
|
|
signal: AbortSignal,
|
|
): Promise<void> {
|
|
if (bannerSuppressed(cliOpts)) return;
|
|
const mcpUrl = cfg.remote_mcp?.mcp_url;
|
|
if (!mcpUrl) return;
|
|
|
|
// Cache lookup keyed by mcp_url so switching hosts via `gbrain init`
|
|
// invalidates cleanly even within a long-lived process.
|
|
const cached = identityCache.get(mcpUrl);
|
|
if (cached && Date.now() - cached.cached_at_ms < IDENTITY_TTL_MS) {
|
|
process.stderr.write(formatBanner(mcpUrl, cached.identity) + '\n');
|
|
// v0.31.11: detect remote-version drift, prompt user to upgrade.
|
|
// bannerIsSuppressed=false here — the early return above guaranteed it.
|
|
await maybePromptForUpgrade(cfg, cached.identity, cliOpts, false);
|
|
return;
|
|
}
|
|
|
|
// Cache miss — fetch. Failure is non-fatal: banner is observability,
|
|
// never load-bearing for the underlying command.
|
|
try {
|
|
const id = await fetchIdentity(cfg, signal);
|
|
identityCache.set(mcpUrl, { identity: id, cached_at_ms: Date.now() });
|
|
process.stderr.write(formatBanner(mcpUrl, id) + '\n');
|
|
// v0.31.11: detect remote-version drift, prompt user to upgrade.
|
|
await maybePromptForUpgrade(cfg, id, cliOpts, false);
|
|
} catch {
|
|
// Swallow. Banner suppressed; main command continues. The CDX-4
|
|
// hardened callRemoteTool will surface the same error class on the
|
|
// actual command call if the host is genuinely unreachable.
|
|
}
|
|
}
|
|
|
|
/**
|
|
* v0.27.1: shared transform for `gbrain query --image <path>` (and any future
|
|
* CLI surface that takes an image path). Reads the file, base64-encodes,
|
|
* derives MIME from the extension, enforces the 20MB cap. Exported so tests
|
|
* can verify the transform without spawning a subprocess.
|
|
*
|
|
* Throws Error on any failure (file missing, oversized, etc.). Caller is
|
|
* responsible for routing to process.exit(1) with a user-facing message.
|
|
*/
|
|
export function resolveQueryImage(
|
|
imagePath: string,
|
|
explicitMime?: string,
|
|
): { path: string; base64: string; mime: string } {
|
|
const bytes = readFileSync(imagePath);
|
|
if (bytes.length > 20 * 1024 * 1024) {
|
|
throw new Error(`Error: image too large (${bytes.length} bytes, max 20MB).`);
|
|
}
|
|
const base64 = bytes.toString('base64');
|
|
let mime = explicitMime;
|
|
if (!mime) {
|
|
const lower = imagePath.toLowerCase();
|
|
const mimeFromExt: Record<string, string> = {
|
|
'.png': 'image/png',
|
|
'.jpg': 'image/jpeg', '.jpeg': 'image/jpeg',
|
|
'.gif': 'image/gif',
|
|
'.webp': 'image/webp',
|
|
'.heic': 'image/heic', '.heif': 'image/heif',
|
|
'.avif': 'image/avif',
|
|
};
|
|
const ext = Object.keys(mimeFromExt).find(e => lower.endsWith(e));
|
|
mime = ext ? mimeFromExt[ext] : 'image/jpeg';
|
|
}
|
|
return { path: imagePath, base64, mime };
|
|
}
|
|
|
|
export function parseOpArgs(op: Operation, args: string[]): Record<string, unknown> {
|
|
const params: Record<string, unknown> = {};
|
|
const positional = op.cliHints?.positional || [];
|
|
let posIdx = 0;
|
|
|
|
for (let i = 0; i < args.length; i++) {
|
|
const arg = args[i];
|
|
if (arg.startsWith('--')) {
|
|
if (arg.startsWith('--no-')) {
|
|
const positiveKey = arg.slice(5).replace(/-/g, '_');
|
|
const positiveDef = op.params[positiveKey];
|
|
if (positiveDef?.type === 'boolean') {
|
|
params[positiveKey] = false;
|
|
continue;
|
|
}
|
|
}
|
|
const key = arg.slice(2).replace(/-/g, '_');
|
|
const paramDef = op.params[key];
|
|
if (paramDef?.type === 'boolean') {
|
|
params[key] = true;
|
|
} else if (i + 1 < args.length) {
|
|
params[key] = args[++i];
|
|
if (paramDef?.type === 'number') params[key] = Number(params[key]);
|
|
}
|
|
} else if (posIdx < positional.length) {
|
|
const key = positional[posIdx++];
|
|
const paramDef = op.params[key];
|
|
params[key] = paramDef?.type === 'number' ? Number(arg) : arg;
|
|
}
|
|
}
|
|
|
|
// Read stdin for content params
|
|
if (op.cliHints?.stdin && !params[op.cliHints.stdin] && !process.stdin.isTTY) {
|
|
const stdinContent = readFileSync(0, 'utf-8');
|
|
const MAX_STDIN = 5_000_000; // 5MB
|
|
if (Buffer.byteLength(stdinContent, 'utf-8') > MAX_STDIN) {
|
|
console.error(`Error: stdin content exceeds ${MAX_STDIN} bytes. Split into smaller inputs.`);
|
|
process.exit(1);
|
|
}
|
|
params[op.cliHints.stdin] = stdinContent;
|
|
}
|
|
|
|
return params;
|
|
}
|
|
|
|
async function makeContext(engine: BrainEngine, params: Record<string, unknown>): Promise<OperationContext> {
|
|
// v0.31.8 (D11): resolve sourceId via the canonical 6-tier chain. Honors
|
|
// --source / GBRAIN_SOURCE / .gbrain-source / path-match / brain default /
|
|
// 'default'. Wrapped in try/catch so a doctor / single-source brain that
|
|
// never set up sources still returns 'default' silently.
|
|
let sourceId: string | undefined;
|
|
try {
|
|
const { resolveSourceId } = await import('./core/source-resolver.ts');
|
|
// params.source is set when a CLI flag was parsed for the op (rare; most
|
|
// CLI ops don't take --source). Falls through to env/dotfile/path-match.
|
|
const explicit = (params.source as string | undefined) ?? null;
|
|
sourceId = await resolveSourceId(engine, explicit);
|
|
} catch {
|
|
// Source resolution failed (e.g. sources table doesn't exist on a fresh
|
|
// pre-init brain). Leave sourceId unset; engine read methods fall through
|
|
// to the cross-source view (D16 back-compat path).
|
|
sourceId = undefined;
|
|
}
|
|
return {
|
|
engine,
|
|
config: loadConfig() || { engine: 'postgres' },
|
|
logger: { info: console.log, warn: console.warn, error: console.error },
|
|
dryRun: (params.dry_run as boolean) || false,
|
|
// Local CLI invocation — the user owns the machine; do not apply remote-caller
|
|
// confinement (e.g., cwd-locked file_upload).
|
|
remote: false,
|
|
cliOpts: getCliOptions(),
|
|
// v0.34 D4: sourceId is REQUIRED at the type level. Fall back to 'default'
|
|
// when resolveSourceId returned undefined (fresh pre-init brain, no sources
|
|
// table). Matches dispatch.ts's auto-fill so the contract holds across
|
|
// every transport.
|
|
sourceId: sourceId ?? 'default',
|
|
};
|
|
}
|
|
|
|
function formatResult(opName: string, result: unknown): string {
|
|
switch (opName) {
|
|
case 'get_page': {
|
|
const r = result as any;
|
|
if (r.error === 'ambiguous_slug') {
|
|
return `Ambiguous slug. Did you mean:\n${r.candidates.map((c: string) => ` ${c}`).join('\n')}\n`;
|
|
}
|
|
return serializeMarkdown(r.frontmatter || {}, r.compiled_truth || '', r.timeline || '', {
|
|
type: r.type, title: r.title, tags: r.tags || [],
|
|
});
|
|
}
|
|
case 'list_pages': {
|
|
const pages = result as any[];
|
|
if (pages.length === 0) return 'No pages found.\n';
|
|
return pages.map(p =>
|
|
`${p.slug}\t${p.type}\t${p.updated_at?.toString().slice(0, 10) || '?'}\t${p.title}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'search':
|
|
case 'query': {
|
|
const results = result as any[];
|
|
if (results.length === 0) return 'No results.\n';
|
|
// v0.40.4 — --explain switches to per-stage attribution formatter.
|
|
// Reads CliOptions.explain via the module-level singleton.
|
|
const cliOpts = getCliOptions();
|
|
if (cliOpts.explain) {
|
|
// Lazy import keeps formatResult's startup hot path narrow for
|
|
// the common non-explain case.
|
|
const { formatResultsExplain } = require('./core/search/explain-formatter.ts');
|
|
return formatResultsExplain(results);
|
|
}
|
|
return results.map(r =>
|
|
`[${r.score?.toFixed(4) || '?'}] ${r.slug} -- ${r.chunk_text?.slice(0, 100) || ''}${r.stale ? ' (stale)' : ''}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'get_tags': {
|
|
const tags = result as string[];
|
|
return tags.length > 0 ? tags.join(', ') + '\n' : 'No tags.\n';
|
|
}
|
|
case 'get_stats': {
|
|
const s = result as any;
|
|
const lines = [
|
|
`Pages: ${s.page_count}`,
|
|
`Chunks: ${s.chunk_count}`,
|
|
`Embedded: ${s.embedded_count}`,
|
|
`Links: ${s.link_count}`,
|
|
`Tags: ${s.tag_count}`,
|
|
`Timeline: ${s.timeline_entry_count}`,
|
|
];
|
|
if (s.pages_by_type) {
|
|
lines.push('', 'By type:');
|
|
for (const [k, v] of Object.entries(s.pages_by_type)) {
|
|
lines.push(` ${k}: ${v}`);
|
|
}
|
|
}
|
|
return lines.join('\n') + '\n';
|
|
}
|
|
case 'get_health': {
|
|
const h = result as any;
|
|
// Health score weights: missing_embeddings is the heaviest (2 pts), other
|
|
// graph quality issues are 1 pt each. link_coverage / timeline_coverage below
|
|
// 50% on entity pages indicates the graph needs population.
|
|
const score = Math.max(0, 10
|
|
- (h.missing_embeddings > 0 ? 2 : 0)
|
|
- (h.stale_pages > 0 ? 1 : 0)
|
|
- (h.orphan_pages > 0 ? 1 : 0)
|
|
- ((h.link_coverage ?? 1) < 0.5 ? 1 : 0)
|
|
- ((h.timeline_coverage ?? 1) < 0.5 ? 1 : 0));
|
|
const lines = [
|
|
`Health score: ${score}/10`,
|
|
`Embed coverage: ${(h.embed_coverage * 100).toFixed(1)}%`,
|
|
`Missing embeddings: ${h.missing_embeddings}`,
|
|
`Stale pages: ${h.stale_pages}`,
|
|
`Orphan pages: ${h.orphan_pages}`,
|
|
];
|
|
if (h.link_coverage !== undefined) {
|
|
lines.push(`Link coverage (entities): ${(h.link_coverage * 100).toFixed(1)}%`);
|
|
}
|
|
if (h.timeline_coverage !== undefined) {
|
|
lines.push(`Timeline coverage (entities): ${(h.timeline_coverage * 100).toFixed(1)}%`);
|
|
}
|
|
if (Array.isArray(h.most_connected) && h.most_connected.length > 0) {
|
|
lines.push('Most connected entities:');
|
|
for (const e of h.most_connected) {
|
|
lines.push(` ${e.slug}: ${e.link_count} links`);
|
|
}
|
|
}
|
|
return lines.join('\n') + '\n';
|
|
}
|
|
case 'get_timeline': {
|
|
const entries = result as any[];
|
|
if (entries.length === 0) return 'No timeline entries.\n';
|
|
return entries.map(e =>
|
|
`${e.date} ${e.summary}${e.source ? ` [${e.source}]` : ''}`,
|
|
).join('\n') + '\n';
|
|
}
|
|
case 'get_versions': {
|
|
const versions = result as any[];
|
|
if (versions.length === 0) return 'No versions.\n';
|
|
return versions.map(v =>
|
|
`#${v.id} ${v.snapshot_at?.toString().slice(0, 19) || '?'} ${v.compiled_truth?.slice(0, 60) || ''}...`,
|
|
).join('\n') + '\n';
|
|
}
|
|
default:
|
|
return JSON.stringify(result, null, 2) + '\n';
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Multi-topology v1: thin-client refusal set. These commands require a local
|
|
* engine; if `~/.gbrain/config.json` has `remote_mcp` set, the dispatch guard
|
|
* refuses them with a canonical error pointing at the remote host. The check
|
|
* runs before per-command dispatch so the error message is consistent.
|
|
*
|
|
* `serve` is in this set because `gbrain serve` (stdio or http) requires a
|
|
* local engine to expose. Thin clients don't have one to expose.
|
|
*
|
|
* `doctor` is intentionally NOT in this set — task 4 routes it to
|
|
* `runRemoteDoctor` for thin-client installs.
|
|
*/
|
|
const THIN_CLIENT_REFUSED_COMMANDS = new Set([
|
|
'sync', 'embed', 'extract', 'extract-conversation-facts', 'migrate', 'apply-migrations',
|
|
'repair-jsonb', 'orphans', 'integrity', 'serve',
|
|
// v0.31.1 (CDX-2 op coverage matrix): more local-only commands
|
|
'dream', 'transcripts', 'storage',
|
|
// v0.31.1 CDX-2 audit: takes/sources have multiple subcommands; some
|
|
// (takes_list/takes_search, sources_list/sources_status) have MCP
|
|
// equivalents and others are file-system bound (takes mutate commands
|
|
// edit local .md files). v0.31.1 refuses both at the top level with a
|
|
// hint pointing at the routable MCP tools; per-subcommand splits are
|
|
// a v0.31.x follow-up TODO.
|
|
'takes', 'sources',
|
|
// v0.32 thin-client routing audit (Codex round 2 findings #2, #4):
|
|
// - `pages` purge-deleted is admin+localOnly (operations.ts:856-864)
|
|
// - `files` list / file_url MCP ops are localOnly (operations.ts:1769-1879)
|
|
// - `eval` export/prune/replay have no MCP equivalents
|
|
// - `code-def`/`code-refs`/`code-callers`/`code-callees` have NO MCP ops
|
|
// in operations.ts:2630-2671; cannot be "fixed by routing" yet
|
|
'pages', 'files', 'eval', 'code-def', 'code-refs', 'code-callers', 'code-callees',
|
|
]);
|
|
|
|
/**
|
|
* v0.31.1 (Issue #734, CDX-5 + cherry-pick A): pinpoint refusal hints for
|
|
* local-only commands when running on a thin-client install. Each hint names
|
|
* the closest path (remote MCP call, host-side workflow) so users aren't
|
|
* stuck guessing what to do next.
|
|
*
|
|
* Source-of-truth lives here so adding a new local-only command means
|
|
* adding both the THIN_CLIENT_REFUSED_COMMANDS member AND the hint in one
|
|
* place during code review.
|
|
*/
|
|
const THIN_CLIENT_REFUSE_HINTS: Record<string, string> = {
|
|
sync: 'sync runs on the host. Trigger a remote cycle with `gbrain remote ping` (queues an autopilot-cycle job).',
|
|
embed: 'embed runs on the host as part of the autopilot cycle. `gbrain remote ping` triggers a full cycle including embed.',
|
|
extract: 'extract runs on the host. Use `gbrain remote ping` to trigger a cycle including extract.',
|
|
'extract-conversation-facts': 'extract-conversation-facts runs on the host (requires local engine + chat gateway). Run on the host machine.',
|
|
migrate: "migrate runs on the host's local engine. Run on the host machine.",
|
|
'apply-migrations': 'schema migrations run on the host. SSH and run there.',
|
|
'repair-jsonb': 'repair-jsonb operates on the local DB only.',
|
|
integrity: 'integrity scans local files. Run on the host machine.',
|
|
serve: 'serve starts a server. Run on the host, not the thin client.',
|
|
dream: 'dream runs the autopilot cycle on the host. `gbrain remote ping` queues one. (Native `gbrain dream` thin-client routing planned for v0.31.2.)',
|
|
orphans: "orphans needs the host's brain. Run on the host or use the `find_orphans` MCP tool from your agent.",
|
|
transcripts: 'transcripts is server-private (raw chat exports stay on the host). Read transcripts on the host machine.',
|
|
storage: 'storage operates on the local repo on disk. Run on the host.',
|
|
takes: 'takes mutate subcommands edit local .md files; routing the read subcommands lands in v0.31.x. For now: use `takes_list` and `takes_search` MCP tools from your agent, or run on the host.',
|
|
sources: 'sources commands manage local DB + config rows. Per-subcommand thin-client routing lands in v0.31.x. For now: use `sources_list` / `sources_status` MCP tools, or run on the host.',
|
|
// v0.32 audit additions
|
|
pages: '`pages purge-deleted` is admin+localOnly (hard-deletes from the local DB). Run on the host.',
|
|
files: '`files list` and `files url` MCP ops are localOnly (paths live on the host filesystem). Use `gbrain files` on the host machine.',
|
|
eval: '`eval` export/prune/replay touch the local engine and have no MCP equivalents. Run `gbrain eval` on the host.',
|
|
'code-def': '`code-def` needs symbol-aware lookup that has no MCP op yet. Run on the host or use `search` from your agent with a symbol-shaped query.',
|
|
'code-refs': '`code-refs` has no MCP op yet. Run on the host.',
|
|
'code-callers': '`code-callers` has no MCP op yet. Run on the host.',
|
|
'code-callees': '`code-callees` has no MCP op yet. Run on the host.',
|
|
};
|
|
|
|
/**
|
|
* v0.31.1: emit a pinpoint refusal hint for a thin-client-incompatible
|
|
* command and exit 1. Falls back to the canonical generic message when no
|
|
* specific hint is registered (defensive — every member of
|
|
* THIN_CLIENT_REFUSED_COMMANDS should have a hint).
|
|
*/
|
|
function refuseThinClient(command: string, mcpUrl: string): never {
|
|
const hint = THIN_CLIENT_REFUSE_HINTS[command];
|
|
if (hint) {
|
|
console.error(`\`gbrain ${command}\` is not routable. ${hint}`);
|
|
console.error(`(thin-client of ${mcpUrl})`);
|
|
} else {
|
|
console.error(
|
|
`\`gbrain ${command}\` requires a local engine. This install is a thin client of ${mcpUrl}.\n` +
|
|
`Run \`${command}\` on the remote host, or use the corresponding MCP tool from your agent.`,
|
|
);
|
|
}
|
|
process.exit(1);
|
|
}
|
|
|
|
async function handleCliOnly(command: string, args: string[]) {
|
|
// Thin-client guard: refuse DB-bound commands cleanly with a pinpoint
|
|
// hint instead of letting them fail later inside connectEngine or
|
|
// mid-handler. v0.31.1 routes through `refuseThinClient` so every
|
|
// refusal carries an actionable next-step hint (CDX-5 cherry-pick A).
|
|
if (THIN_CLIENT_REFUSED_COMMANDS.has(command)) {
|
|
const cfg = loadConfig();
|
|
if (isThinClient(cfg)) {
|
|
refuseThinClient(command, cfg!.remote_mcp!.mcp_url);
|
|
}
|
|
}
|
|
|
|
// Commands that don't need a database connection
|
|
if (command === 'schema') {
|
|
const { runSchema } = await import('./commands/schema.ts');
|
|
await runSchema(args);
|
|
return;
|
|
}
|
|
if (command === 'init') {
|
|
const { runInit } = await import('./commands/init.ts');
|
|
await runInit(args);
|
|
return;
|
|
}
|
|
// v0.37 fix wave (deferred TODO, shipped): one-command wipe-and-reinit.
|
|
// Spawns its own engine internally so no pre-bound engine needed.
|
|
if (command === 'reinit-pglite') {
|
|
const { runReinitPglite } = await import('./commands/reinit-pglite.ts');
|
|
await runReinitPglite(args);
|
|
return;
|
|
}
|
|
if (command === 'auth') {
|
|
const { runAuth } = await import('./commands/auth.ts');
|
|
await runAuth(args);
|
|
return;
|
|
}
|
|
if (command === 'remote') {
|
|
// Multi-topology v1 (Tier B): thin-client-only convenience commands.
|
|
// `runRemote` self-checks for remote_mcp config and exits 1 if local-only.
|
|
const { runRemote } = await import('./commands/remote.ts');
|
|
await runRemote(args);
|
|
return;
|
|
}
|
|
if (command === 'upgrade') {
|
|
const { runUpgrade } = await import('./commands/upgrade.ts');
|
|
await runUpgrade(args);
|
|
return;
|
|
}
|
|
if (command === 'post-upgrade') {
|
|
const { runPostUpgrade } = await import('./commands/upgrade.ts');
|
|
await runPostUpgrade(args);
|
|
return;
|
|
}
|
|
if (command === 'check-update') {
|
|
const { runCheckUpdate } = await import('./commands/check-update.ts');
|
|
await runCheckUpdate(args);
|
|
return;
|
|
}
|
|
if (command === 'integrations') {
|
|
const { runIntegrations } = await import('./commands/integrations.ts');
|
|
await runIntegrations(args);
|
|
return;
|
|
}
|
|
if (command === 'providers') {
|
|
const { runProviders } = await import('./commands/providers.ts');
|
|
const [sub, ...rest] = args;
|
|
await runProviders(sub, rest);
|
|
return;
|
|
}
|
|
if (command === 'auth') {
|
|
const { runAuth } = await import('./commands/auth.ts');
|
|
await runAuth(args);
|
|
return;
|
|
}
|
|
if (command === 'resolvers') {
|
|
const { runResolvers } = await import('./commands/resolvers.ts');
|
|
await runResolvers(args);
|
|
return;
|
|
}
|
|
if (command === 'integrity') {
|
|
const { runIntegrity } = await import('./commands/integrity.ts');
|
|
await runIntegrity(args);
|
|
return;
|
|
}
|
|
if (command === 'publish') {
|
|
const { runPublish } = await import('./commands/publish.ts');
|
|
await runPublish(args);
|
|
return;
|
|
}
|
|
if (command === 'check-backlinks') {
|
|
const { runBacklinks } = await import('./commands/backlinks.ts');
|
|
await runBacklinks(args);
|
|
return;
|
|
}
|
|
if (command === 'frontmatter') {
|
|
const { runFrontmatter } = await import('./commands/frontmatter.ts');
|
|
await runFrontmatter(args);
|
|
return;
|
|
}
|
|
if (command === 'lint') {
|
|
const { runLint } = await import('./commands/lint.ts');
|
|
await runLint(args);
|
|
return;
|
|
}
|
|
if (command === 'check-resolvable') {
|
|
const { runCheckResolvable } = await import('./commands/check-resolvable.ts');
|
|
await runCheckResolvable(args);
|
|
return;
|
|
}
|
|
if (command === 'mounts') {
|
|
// No DB needed: mounts.json is a local config file. Registry will
|
|
// connect mount engines lazily on first use by op dispatch.
|
|
const { runMounts } = await import('./commands/mounts.ts');
|
|
await runMounts(args);
|
|
return;
|
|
}
|
|
if (command === 'cache') {
|
|
// v0.32.x search-lite: semantic query cache management. Dispatch the
|
|
// subcommand handler (stats / clear / prune); the handler opens its
|
|
// own engine connection.
|
|
const { runCache } = await import('./commands/cache.ts');
|
|
await runCache(args);
|
|
return;
|
|
}
|
|
if (command === 'routing-eval') {
|
|
const { runRoutingEvalCli } = await import('./commands/routing-eval.ts');
|
|
await runRoutingEvalCli(args);
|
|
return;
|
|
}
|
|
if (command === 'skillify') {
|
|
const { runSkillify } = await import('./commands/skillify.ts');
|
|
// `args` here is subArgs (command already stripped by caller), so
|
|
// args[0] is the subcommand (scaffold|check).
|
|
await runSkillify(args);
|
|
return;
|
|
}
|
|
if (command === 'skillpack') {
|
|
const { runSkillpack } = await import('./commands/skillpack.ts');
|
|
// subArgs already has `skillpack` stripped; args[0] is the subcommand.
|
|
await runSkillpack(args);
|
|
return;
|
|
}
|
|
if (command === 'friction') {
|
|
const { runFriction } = await import('./commands/friction.ts');
|
|
process.exit(runFriction(args));
|
|
}
|
|
if (command === 'claw-test') {
|
|
const { runClawTest } = await import('./commands/claw-test.ts');
|
|
process.exit(await runClawTest(args));
|
|
}
|
|
if (command === 'report') {
|
|
const { runReport } = await import('./commands/report.ts');
|
|
await runReport(args);
|
|
return;
|
|
}
|
|
if (command === 'apply-migrations') {
|
|
// Does not need connectEngine — each phase (schema, smoke, host-rewrite)
|
|
// manages its own subprocess or file-layer access directly. Avoids
|
|
// connecting a second time when the orchestrator shells out to
|
|
// `gbrain init --migrate-only` and `gbrain jobs smoke`.
|
|
const { runApplyMigrations } = await import('./commands/apply-migrations.ts');
|
|
await runApplyMigrations(args);
|
|
return;
|
|
}
|
|
if (command === 'repair-jsonb') {
|
|
const { runRepairJsonbCli } = await import('./commands/repair-jsonb.ts');
|
|
await runRepairJsonbCli(args);
|
|
return;
|
|
}
|
|
if (command === 'skillpack-check') {
|
|
// Agent-readable health report. Shells out to doctor + apply-migrations
|
|
// internally; does not need its own DB connection.
|
|
const { runSkillpackCheck } = await import('./commands/skillpack-check.ts');
|
|
await runSkillpackCheck(args);
|
|
return;
|
|
}
|
|
if (command === 'doctor') {
|
|
// Multi-topology v1: thin-client doctor. When `~/.gbrain/config.json`
|
|
// has remote_mcp set, every DB-bound check is irrelevant. Route to the
|
|
// outbound-HTTP probe set in `src/core/doctor-remote.ts` and return
|
|
// before any local-engine work.
|
|
const cfgForDoctor = loadConfig();
|
|
if (isThinClient(cfgForDoctor)) {
|
|
const { runRemoteDoctor } = await import('./core/doctor-remote.ts');
|
|
await runRemoteDoctor(cfgForDoctor!, args);
|
|
return;
|
|
}
|
|
|
|
// v0.36+ brain-health-100: --remediation-plan and --remediate go
|
|
// through dedicated functions that compute from engine.getHealth()
|
|
// (cheap path D7), NOT the full doctor walk.
|
|
if (args.includes('--remediation-plan')) {
|
|
const { runRemediationPlan } = await import('./commands/doctor.ts');
|
|
const eng = await connectEngine();
|
|
try { await runRemediationPlan(eng, args); } finally { await eng.disconnect(); }
|
|
return;
|
|
}
|
|
if (args.includes('--remediate')) {
|
|
const { runRemediate } = await import('./commands/doctor.ts');
|
|
const eng = await connectEngine();
|
|
try { await runRemediate(eng, args); } finally { await eng.disconnect(); }
|
|
return;
|
|
}
|
|
|
|
// Doctor runs filesystem checks first (no DB needed), then DB checks.
|
|
// --fast skips DB checks entirely.
|
|
const { runDoctor } = await import('./commands/doctor.ts');
|
|
const { getDbUrlSource } = await import('./core/config.ts');
|
|
if (args.includes('--fast')) {
|
|
// Pass the DB URL source so doctor can tell "no config at all" from
|
|
// "user chose --fast while config is present".
|
|
await runDoctor(null, args, getDbUrlSource());
|
|
} else {
|
|
try {
|
|
const eng = await connectEngine();
|
|
await runDoctor(eng, args);
|
|
await eng.disconnect();
|
|
} catch {
|
|
// DB unavailable — still run filesystem checks
|
|
await runDoctor(null, args, getDbUrlSource());
|
|
}
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'ze-switch') {
|
|
// v0.36.0.0 — manual ZE-default switch lever. Owns its own engine lifecycle
|
|
// to mirror the doctor pattern.
|
|
const { runZeSwitch } = await import('./commands/ze-switch.ts');
|
|
const eng = await connectEngine();
|
|
try {
|
|
await runZeSwitch(args, eng);
|
|
} finally {
|
|
await eng.disconnect();
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'smoke-test') {
|
|
// Run smoke tests — no DB connection needed, the script handles its own checks
|
|
const { execSync } = await import('child_process');
|
|
const { resolve, dirname } = await import('path');
|
|
const { fileURLToPath } = await import('url');
|
|
const scriptDir = dirname(fileURLToPath(import.meta.url));
|
|
const scriptPath = resolve(scriptDir, '..', 'scripts', 'smoke-test.sh');
|
|
try {
|
|
execSync(`bash "${scriptPath}"`, { stdio: 'inherit', env: { ...process.env } });
|
|
} catch (e: any) {
|
|
// Non-zero exit = some tests failed (exit code = failure count)
|
|
process.exit(e.status ?? 1);
|
|
}
|
|
return;
|
|
}
|
|
|
|
if (command === 'dream') {
|
|
// Dream mirrors doctor's pattern: filesystem phases run without a DB,
|
|
// so an engine connection failure is non-fatal. runCycle honestly
|
|
// reports DB phases as skipped when engine is null. v0.41.13 (#1422):
|
|
// bind + surface the error on stderr so the user knows WHY DB phases
|
|
// were skipped instead of seeing a silent "lint + backlinks done"
|
|
// and assuming the cycle actually ran. Pre-fix, foxhoundinc reported
|
|
// the cycle exiting 0 on PostgreSQL with every DB phase silently no-op.
|
|
const { runDream } = await import('./commands/dream.ts');
|
|
let eng: BrainEngine | null = null;
|
|
try {
|
|
eng = await connectEngine();
|
|
} catch (err) {
|
|
const msg = err instanceof Error ? err.message : String(err);
|
|
process.stderr.write(
|
|
`[dream] WARNING: could not connect to DB (${msg}). ` +
|
|
`Running filesystem-only phases (lint, backlinks, extract). ` +
|
|
`DB-dependent phases (sync, embed, synthesize, etc.) will report as skipped.\n`
|
|
);
|
|
}
|
|
try {
|
|
await runDream(eng, args);
|
|
} finally {
|
|
if (eng) await eng.disconnect();
|
|
}
|
|
return;
|
|
}
|
|
|
|
// `eval cross-modal` is a pure API-call command — no DB, no brain. Bypass
|
|
// connectEngine entirely so first-run users (no `gbrain init` yet) can
|
|
// run the quality gate. Mirrors the dream/doctor no-DB pattern but
|
|
// doesn't even attempt the connect (T3=A in plans/radiant-napping-lerdorf.md).
|
|
// The handler self-configures the AI gateway from loadConfig() + process.env.
|
|
if (command === 'eval' && args[0] === 'cross-modal') {
|
|
const { runEvalCrossModal } = await import('./commands/eval-cross-modal.ts');
|
|
process.exit(await runEvalCrossModal(args.slice(1)));
|
|
}
|
|
|
|
// v0.32 EXP-5 (codex review #10): `eval takes-quality replay <receipt>`
|
|
// is the ONLY sub-subcommand that doesn't need a brain — it reads a
|
|
// receipt JSON file from disk and re-renders it. Bypass connectEngine
|
|
// here so users can replay a receipt on a machine without DATABASE_URL.
|
|
// run/trend/regress need the brain and fall through to the regular
|
|
// engine-required path below.
|
|
if (command === 'eval' && args[0] === 'takes-quality' && args[1] === 'replay') {
|
|
const { runReplayNoBrain } = await import('./commands/eval-takes-quality.ts');
|
|
process.exit(await runReplayNoBrain(args.slice(2)));
|
|
}
|
|
|
|
// v0.28.8: longmemeval brings its own in-memory PGLite. Bypassing
|
|
// connectEngine here keeps `gbrain eval longmemeval --help` and benchmark
|
|
// runs working on machines that have no `~/.gbrain/config.json` configured.
|
|
//
|
|
// v0.35.1.1: still need to configureGateway() so the in-memory brain's
|
|
// import + hybridSearch can embed via the configured provider. Reads
|
|
// ~/.gbrain/config.json when present; falls back to env vars otherwise
|
|
// (GBRAIN_EMBEDDING_MODEL / GBRAIN_EMBEDDING_DIMENSIONS).
|
|
if (command === 'eval' && args[0] === 'longmemeval') {
|
|
const { runEvalLongMemEval } = await import('./commands/eval-longmemeval.ts');
|
|
if (!(args.length > 1 && (args[1] === '--help' || args[1] === '-h'))) {
|
|
const config = loadConfig() ?? ({
|
|
embedding_model: process.env.GBRAIN_EMBEDDING_MODEL,
|
|
embedding_dimensions: process.env.GBRAIN_EMBEDDING_DIMENSIONS
|
|
? Number(process.env.GBRAIN_EMBEDDING_DIMENSIONS) : undefined,
|
|
} as GBrainConfig);
|
|
const { configureGateway } = await import('./core/ai/gateway.ts');
|
|
configureGateway(buildGatewayConfig(config));
|
|
}
|
|
await runEvalLongMemEval(args.slice(1));
|
|
return;
|
|
}
|
|
|
|
// v0.41.13.0: `gbrain eval conversation-parser` is pure-function
|
|
// (parses fixture JSONL, runs parseConversation, scores results).
|
|
// No DB access; bypass connectEngine entirely so the CI fixture
|
|
// gate runs on machines with no `~/.gbrain/config.json`.
|
|
if (command === 'eval' && args[0] === 'conversation-parser') {
|
|
const { runEvalConversationParser } = await import('./commands/eval-conversation-parser.ts');
|
|
process.exit(await runEvalConversationParser(args.slice(1)));
|
|
}
|
|
|
|
// v0.41.13.0: `gbrain conversation-parser list-builtins | validate
|
|
// | --help` are pure (no DB access). Bypass connectEngine so the
|
|
// operator can run them on machines with no brain configured.
|
|
// `scan <slug>` needs a brain and falls through.
|
|
if (
|
|
command === 'conversation-parser' &&
|
|
(args.length === 0 ||
|
|
args[0] === '--help' ||
|
|
args[0] === '-h' ||
|
|
args[0] === 'list-builtins' ||
|
|
args[0] === 'validate')
|
|
) {
|
|
const { runConversationParser } = await import('./commands/conversation-parser.ts');
|
|
await runConversationParser(null, args);
|
|
return;
|
|
}
|
|
|
|
// v0.33.1.3: `gbrain eval whoknows` on thin-client installs bypasses
|
|
// connectEngine entirely — the eval routes per-query through the remote
|
|
// `find_experts` MCP op (the v0.31.1 routing seam). Local mode falls
|
|
// through to the engine-connected path below.
|
|
if (command === 'eval' && args[0] === 'whoknows') {
|
|
const cfgPre = loadConfig();
|
|
if (isThinClient(cfgPre)) {
|
|
const { runEvalWhoknows } = await import('./commands/eval-whoknows.ts');
|
|
process.exit(await runEvalWhoknows(null, args.slice(1)));
|
|
}
|
|
}
|
|
|
|
// v0.37 fix wave (Lane D.4 + CDX2-12): short-circuit `gbrain sync --help`
|
|
// BEFORE the engine bind. runSync has its own --help branch but can't
|
|
// reach it without an engine — which means a user running `--help` from
|
|
// a fresh tmpdir with no config gets a no-such-config error instead of
|
|
// help text. Importing runSync without the engine + passing null works
|
|
// because runSync's --help path doesn't touch the engine argument.
|
|
if (command === 'sync' && (args.includes('--help') || args.includes('-h'))) {
|
|
const { runSync } = await import('./commands/sync.ts');
|
|
await runSync(null as any, args);
|
|
return;
|
|
}
|
|
|
|
// v0.39.3.0 WARN-5: same pattern for `capture --help`. CLI_ONLY_SELF_HELP
|
|
// now includes 'capture' so the generic short-circuit at :101 stays out
|
|
// of the way, but the dispatch case at :1229 still needs an engine. The
|
|
// pre-engine-bind branch here exposes the HELP constant without requiring
|
|
// a configured brain (fresh-tmpdir parity with brainstorm/lsd/sync).
|
|
if (command === 'capture' && (args.includes('--help') || args.includes('-h'))) {
|
|
const { runCapture } = await import('./commands/capture.ts');
|
|
await runCapture(null, args);
|
|
return;
|
|
}
|
|
|
|
// v0.41.6.0 D3 (per outside-voice F1): connect-time + dispatch-time wallclock
|
|
// timeouts for read-only commands whose hang would otherwise spin at 100% CPU
|
|
// (the production "10-day zombie gbrain search ping" bug class). The wrap
|
|
// covers connectEngine (so a hung schema probe / PgBouncer freeze actually
|
|
// surfaces a timeout) AND the dispatch body (so a wedged runSearch /
|
|
// runList honors the same deadline).
|
|
// Per-command default: search 30s, sources list 10s. User --timeout=Ns wins.
|
|
// Other commands (import, embed, doctor, etc.) keep their existing
|
|
// unbounded connect — destructive / long-running commands shouldn't get
|
|
// a default kill switch.
|
|
const readOnlyDefaultTimeoutMs =
|
|
command === 'search' ? 30_000 :
|
|
command === 'sources' && (args[0] === 'list' || args[0] === undefined) ? 10_000 :
|
|
null;
|
|
const cliOptsResolved = getCliOptions();
|
|
const userTimeoutMs = cliOptsResolved.timeoutMs;
|
|
const readOnlyTimeoutMs = userTimeoutMs ?? readOnlyDefaultTimeoutMs;
|
|
|
|
if (readOnlyTimeoutMs !== null) {
|
|
const { withTimeout, OperationTimeoutError } = await import('./core/timeout.ts');
|
|
const label = `gbrain ${command}`;
|
|
let engine: BrainEngine;
|
|
try {
|
|
engine = await withTimeout(connectEngine(), readOnlyTimeoutMs, `${label}: connect`);
|
|
} catch (e) {
|
|
if (e instanceof OperationTimeoutError) {
|
|
const hint = userTimeoutMs ? '' : ` (default ${e.ms}ms; pass --timeout=Ns to override)`;
|
|
console.error(`${e.label} timed out${hint}.`);
|
|
process.exit(124);
|
|
}
|
|
throw e;
|
|
}
|
|
try {
|
|
await withTimeout(dispatchReadOnlyCommand(engine, command, args), readOnlyTimeoutMs, label);
|
|
} catch (e) {
|
|
if (e instanceof OperationTimeoutError) {
|
|
const hint = userTimeoutMs ? '' : ` (default ${e.ms}ms; pass --timeout=Ns to override)`;
|
|
console.error(`${e.label} timed out${hint}.`);
|
|
process.exit(124);
|
|
}
|
|
throw e;
|
|
} finally {
|
|
try { await engine.disconnect(); } catch { /* best-effort */ }
|
|
}
|
|
return;
|
|
}
|
|
|
|
// All remaining CLI-only commands need a DB connection
|
|
const engine = await connectEngine();
|
|
try {
|
|
switch (command) {
|
|
case 'import': {
|
|
const { runImport } = await import('./commands/import.ts');
|
|
// v0.41 (Codex r2 #3 fix): honor errors counter for exit code.
|
|
// runImport's per-file catch already records failures, but the
|
|
// CLI was discarding the result so the process exited 0 even
|
|
// when files failed (e.g. content-sanity hard-block throws,
|
|
// size-cap throws, parse errors). Surface non-zero on errors > 0
|
|
// so wrappers (sync, CI scripts, `&& gbrain doctor`) propagate.
|
|
const importResult = await runImport(engine, args);
|
|
if (importResult.errors > 0) {
|
|
process.exitCode = 1;
|
|
}
|
|
break;
|
|
}
|
|
case 'export': {
|
|
const { runExport } = await import('./commands/export.ts');
|
|
await runExport(engine, args);
|
|
break;
|
|
}
|
|
case 'files': {
|
|
const { runFiles } = await import('./commands/files.ts');
|
|
await runFiles(engine, args);
|
|
break;
|
|
}
|
|
case 'embed': {
|
|
const { runEmbed } = await import('./commands/embed.ts');
|
|
await runEmbed(engine, args);
|
|
break;
|
|
}
|
|
case 'serve': {
|
|
const { runServe } = await import('./commands/serve.ts');
|
|
await runServe(engine, args);
|
|
return; // serve doesn't disconnect
|
|
}
|
|
case 'call': {
|
|
const { runCall } = await import('./commands/call.ts');
|
|
await runCall(engine, args);
|
|
break;
|
|
}
|
|
case 'config': {
|
|
const { runConfig } = await import('./commands/config.ts');
|
|
await runConfig(engine, args);
|
|
break;
|
|
}
|
|
// doctor is handled before connectEngine() above
|
|
case 'migrate': {
|
|
const { runMigrateEngine } = await import('./commands/migrate-engine.ts');
|
|
await runMigrateEngine(engine, args);
|
|
break;
|
|
}
|
|
case 'eval': {
|
|
// v0.32 EXP-5: `eval takes-quality {run,trend,regress}` requires a
|
|
// brain (samples takes from DB / reads runs table). `replay` was
|
|
// already routed through the no-DB bypass above and never reaches
|
|
// this case. Other `eval` subcommands (export/prune/replay-capture/
|
|
// longmemeval/cross-modal) go to the generic dispatcher.
|
|
if (args[0] === 'takes-quality') {
|
|
const { runEvalTakesQuality } = await import('./commands/eval-takes-quality.ts');
|
|
await runEvalTakesQuality(engine, args.slice(1));
|
|
break;
|
|
}
|
|
const { runEvalCommand } = await import('./commands/eval.ts');
|
|
await runEvalCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'jobs': {
|
|
const { runJobs } = await import('./commands/jobs.ts');
|
|
await runJobs(engine, args);
|
|
break;
|
|
}
|
|
case 'agent': {
|
|
const { runAgent } = await import('./commands/agent.ts');
|
|
await runAgent(engine, args);
|
|
break;
|
|
}
|
|
case 'book-mirror': {
|
|
const { runBookMirrorCmd } = await import('./commands/book-mirror.ts');
|
|
await runBookMirrorCmd(engine, args);
|
|
break;
|
|
}
|
|
case 'sync': {
|
|
const { runSync } = await import('./commands/sync.ts');
|
|
await runSync(engine, args);
|
|
break;
|
|
}
|
|
case 'extract': {
|
|
const { runExtract } = await import('./commands/extract.ts');
|
|
await runExtract(engine, args);
|
|
break;
|
|
}
|
|
case 'extract-conversation-facts': {
|
|
const { runExtractConversationFacts } = await import('./commands/extract-conversation-facts.ts');
|
|
await runExtractConversationFacts(engine, args);
|
|
break;
|
|
}
|
|
case 'features': {
|
|
const { runFeatures } = await import('./commands/features.ts');
|
|
await runFeatures(engine, args);
|
|
break;
|
|
}
|
|
case 'autopilot': {
|
|
const { runAutopilot } = await import('./commands/autopilot.ts');
|
|
await runAutopilot(engine, args);
|
|
return; // autopilot doesn't disconnect (long-running)
|
|
}
|
|
case 'graph-query': {
|
|
const { runGraphQuery } = await import('./commands/graph-query.ts');
|
|
await runGraphQuery(engine, args);
|
|
break;
|
|
}
|
|
case 'reconcile-links': {
|
|
// v0.20.0 Cathedral II Layer 8 D3: batch-recompute doc↔impl edges
|
|
// for any markdown page that cites code files. Idempotent; safe to
|
|
// re-run. Closes the v0.19.0 Layer 6 order-dependency bug where
|
|
// guides imported before their code never got their edges written.
|
|
const { runReconcileLinksCli } = await import('./commands/reconcile-links.ts');
|
|
await runReconcileLinksCli(engine, args);
|
|
break;
|
|
}
|
|
case 'orphans': {
|
|
const { runOrphans } = await import('./commands/orphans.ts');
|
|
await runOrphans(engine, args);
|
|
break;
|
|
}
|
|
// v0.32.7 CJK wave — post-upgrade markdown re-chunk sweep.
|
|
// v0.36 Phase 3 wave — `gbrain reindex --multimodal` re-embeds content_chunks
|
|
// into the unified Voyage multimodal-3 column.
|
|
case 'reindex': {
|
|
if (args.includes('--multimodal')) {
|
|
const { runReindexMultimodal } = await import('./commands/reindex-multimodal.ts');
|
|
const { parseWorkers } = await import('./core/sync-concurrency.ts');
|
|
const limitIdx = args.indexOf('--limit');
|
|
const limitVal = limitIdx >= 0 && limitIdx + 1 < args.length ? parseInt(args[limitIdx + 1], 10) : undefined;
|
|
// v0.41.15.0 (T9, D9): --workers N for parallel UPDATEs within
|
|
// each Voyage batch. Honored by the inner write loop only;
|
|
// the outer batch loop is one Voyage round-trip per batch.
|
|
const workersIdx = args.indexOf('--workers');
|
|
const concurrencyIdx = args.indexOf('--concurrency');
|
|
const workersValIdx = workersIdx >= 0 ? workersIdx + 1 : (concurrencyIdx >= 0 ? concurrencyIdx + 1 : -1);
|
|
const workers = workersValIdx > 0 && workersValIdx < args.length
|
|
? parseWorkers(args[workersValIdx])
|
|
: undefined;
|
|
const result = await runReindexMultimodal(engine, {
|
|
limit: Number.isFinite(limitVal as number) ? (limitVal as number) : undefined,
|
|
dryRun: args.includes('--dry-run'),
|
|
costEstimate: args.includes('--cost-estimate'),
|
|
noEmbed: args.includes('--no-embed'),
|
|
json: args.includes('--json'),
|
|
yes: args.includes('--yes'),
|
|
workers,
|
|
});
|
|
if (args.includes('--json')) {
|
|
console.log(JSON.stringify(result, null, 2));
|
|
} else {
|
|
console.log(`reindex --multimodal: ${result.reembedded} re-embedded, ${result.failed} failed, ${result.pending_after} pending. est. cost: $${result.cost_usd_estimate.toFixed(2)}`);
|
|
}
|
|
break;
|
|
}
|
|
const { runReindex } = await import('./commands/reindex.ts');
|
|
await runReindex(engine, args);
|
|
break;
|
|
}
|
|
// v0.29 — Salience + Anomaly Detection
|
|
case 'salience': {
|
|
const { runSalience } = await import('./commands/salience.ts');
|
|
await runSalience(engine, args);
|
|
break;
|
|
}
|
|
case 'anomalies': {
|
|
const { runAnomalies } = await import('./commands/anomalies.ts');
|
|
await runAnomalies(engine, args);
|
|
break;
|
|
}
|
|
// v0.38 — Capture: single human-facing entrypoint for ingestion.
|
|
case 'capture': {
|
|
const { runCapture } = await import('./commands/capture.ts');
|
|
await runCapture(engine, args);
|
|
break;
|
|
}
|
|
case 'conversation-parser': {
|
|
// v0.41.13.0 — debug + introspection CLI for the new parser
|
|
// cathedral. `scan <slug>` requires a connected brain; the
|
|
// other subcommands are pure (`list-builtins`, `validate`).
|
|
const { runConversationParser } = await import('./commands/conversation-parser.ts');
|
|
await runConversationParser(engine, args);
|
|
break;
|
|
}
|
|
case 'edges-backfill': {
|
|
// v0.34 W6 — operator escape hatch for the symbol-resolution backfill.
|
|
// Resumable via the edges_backfilled_at watermark; per-batch transactions
|
|
// commit so Ctrl-C leaves a clean resumable state.
|
|
const { runEdgesBackfill } = await import('./commands/edges-backfill.ts');
|
|
await runEdgesBackfill(engine, args);
|
|
break;
|
|
}
|
|
case 'whoknows': {
|
|
// v0.33 (Issue #?): expertise + relationship-proximity routing.
|
|
// MCP op `find_experts` (read-scoped) backs the same code path; CLI
|
|
// dispatch here is the user-facing surface. Thin-client routing
|
|
// happens inside runWhoknows via isThinClient(cfg) (v0.31.1 pattern).
|
|
const { runWhoknows } = await import('./commands/whoknows.ts');
|
|
await runWhoknows(engine, args);
|
|
break;
|
|
}
|
|
case 'brainstorm': {
|
|
// v0.37.0 (Open Collider wave): bisociation idea generator grounded
|
|
// in the user's own brain. Prefix-stratified domain-bank (D14) +
|
|
// shared judges + citation transparency (D6). LSD MCP exposure
|
|
// deferred to D7; this is CLI-only.
|
|
const { runBrainstormCommand } = await import('./commands/brainstorm.ts');
|
|
await runBrainstormCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'lsd': {
|
|
// v0.37.0 — Lateral Synaptic Drift. Inverted-judge / stale-bias
|
|
// variant of brainstorm. Shares the orchestrator + judges via
|
|
// LSD_PROFILE config. Local-only by design (cost + weirdness gate).
|
|
const { runLsdCommand } = await import('./commands/lsd.ts');
|
|
await runLsdCommand(engine, args);
|
|
break;
|
|
}
|
|
case 'calibration': {
|
|
// v0.36.1.0 (T7): print/regenerate the active calibration profile.
|
|
// MCP op `get_calibration_profile` (read-scoped) backs the same data path.
|
|
const { runCalibration } = await import('./commands/calibration.ts');
|
|
const calibrationConfig = loadConfig() ?? ({} as never);
|
|
await runCalibration(engine, args, calibrationConfig);
|
|
break;
|
|
}
|
|
case 'transcripts': {
|
|
const { runTranscripts } = await import('./commands/transcripts.ts');
|
|
await runTranscripts(engine, args);
|
|
break;
|
|
}
|
|
case 'models': {
|
|
const { runModels } = await import('./commands/models.ts');
|
|
await runModels(engine, args);
|
|
break;
|
|
}
|
|
case 'search': {
|
|
// v0.32.3 search-lite — `gbrain search modes/stats/tune`.
|
|
const { runSearch } = await import('./commands/search.ts');
|
|
await runSearch(engine, args);
|
|
break;
|
|
}
|
|
case 'takes': {
|
|
const { runTakes } = await import('./commands/takes.ts');
|
|
await runTakes(engine, args);
|
|
break;
|
|
}
|
|
case 'onboard': {
|
|
// v0.41.18.0 (T13) — gbrain onboard. Thin shell over T2 library
|
|
// + T4 onboard checks + T12 render layer.
|
|
const { runOnboard } = await import('./commands/onboard.ts');
|
|
await runOnboard(engine, args);
|
|
break;
|
|
}
|
|
case 'founder': {
|
|
// v0.35.4 (T7) — founder scorecard. `gbrain founder scorecard <slug>`
|
|
// rolls up Phase 2's typed-claim substrate into the four scorecard
|
|
// metrics (claim accuracy, consistency, growth trajectory, red flags).
|
|
// Thin-client routing handled inside the command file.
|
|
const { runFounder } = await import('./commands/founder-scorecard.ts');
|
|
await runFounder(engine, args);
|
|
break;
|
|
}
|
|
case 'think': {
|
|
const { runThinkCli } = await import('./commands/think.ts');
|
|
await runThinkCli(engine, args);
|
|
break;
|
|
}
|
|
case 'recall': {
|
|
// v0.31: hot memory recall surface — `gbrain recall <entity>`,
|
|
// `--since DUR`, `--session ID`, `--today`, `--grep TEXT`,
|
|
// `--supersessions`, `--include-expired`, `--as-context`, `--json`.
|
|
const { runRecall } = await import('./commands/recall.ts');
|
|
await runRecall(engine, args);
|
|
break;
|
|
}
|
|
case 'forget': {
|
|
// v0.31: shorthand for expireFact. `gbrain forget <fact-id>`.
|
|
const { runForget } = await import('./commands/recall.ts');
|
|
await runForget(engine, args);
|
|
break;
|
|
}
|
|
case 'notability-eval': {
|
|
// v0.31.2: notability gate eval suite. Two subcommands:
|
|
// gbrain notability-eval mine — sample paragraphs, write candidates
|
|
// gbrain notability-eval review — TTY hand-confirm tiers
|
|
const { runNotabilityEval } = await import('./commands/notability-eval.ts');
|
|
const subcmd = args[0] || 'help';
|
|
const flags: Record<string, string | boolean> = {};
|
|
for (let i = 1; i < args.length; i++) {
|
|
const a = args[i];
|
|
if (a.startsWith('--')) {
|
|
const key = a.slice(2);
|
|
const next = args[i + 1];
|
|
if (next && !next.startsWith('--')) {
|
|
flags[key] = next;
|
|
i++;
|
|
} else {
|
|
flags[key] = true;
|
|
}
|
|
}
|
|
}
|
|
// sync.repo_path resolution (matches dream phase pattern).
|
|
let repoPath: string | undefined;
|
|
try {
|
|
repoPath = (flags.repo as string) || (await engine.getConfig('sync.repo_path')) || undefined;
|
|
} catch { /* engine may not be connected for help */ }
|
|
await runNotabilityEval({ cmd: subcmd, flags, engine, repoPath });
|
|
break;
|
|
}
|
|
case 'sources': {
|
|
const { runSources } = await import('./commands/sources.ts');
|
|
await runSources(engine, args);
|
|
break;
|
|
}
|
|
case 'pages': {
|
|
// v0.26.5: page-level operator commands (purge-deleted escape hatch).
|
|
const { runPages } = await import('./commands/pages.ts');
|
|
await runPages(engine, args);
|
|
break;
|
|
}
|
|
case 'storage': {
|
|
const { runStorage } = await import('./commands/storage.ts');
|
|
await runStorage(engine, args);
|
|
break;
|
|
}
|
|
case 'code-def': {
|
|
const { runCodeDef } = await import('./commands/code-def.ts');
|
|
await runCodeDef(engine, args);
|
|
break;
|
|
}
|
|
case 'code-refs': {
|
|
const { runCodeRefs } = await import('./commands/code-refs.ts');
|
|
await runCodeRefs(engine, args);
|
|
break;
|
|
}
|
|
case 'reindex-code': {
|
|
// v0.20.0 Cathedral II Layer 13 (E2): explicit code-page reindex
|
|
// for users upgrading from v0.19.0. Cost-preview gated; TTY prompt
|
|
// or ConfirmationRequired envelope for non-TTY/JSON callers.
|
|
const { runReindexCodeCli } = await import('./commands/reindex-code.ts');
|
|
await runReindexCodeCli(engine, args);
|
|
break;
|
|
}
|
|
case 'reindex-frontmatter': {
|
|
// v0.29.1: recovery / explicit-rebuild path for pages.effective_date.
|
|
// Mirror of reindex-code shape. Wraps the shared library function in
|
|
// src/core/backfill-effective-date.ts (same code path the v0.29.1
|
|
// migration orchestrator uses). The orchestrator runs once on
|
|
// upgrade; this command is for after-the-fact frontmatter edits.
|
|
//
|
|
// v0.30.1: still works; canonical entrypoint is now `gbrain backfill
|
|
// effective_date`. This command stays as a thin alias for back-compat.
|
|
const { reindexFrontmatterCli } = await import('./commands/reindex-frontmatter.ts');
|
|
await reindexFrontmatterCli(args);
|
|
return; // reindexFrontmatterCli handles its own engine lifecycle
|
|
}
|
|
case 'backfill': {
|
|
// v0.30.1: first-class generic backfill command. Subcommand dispatch
|
|
// is inside runBackfillCommand (kind | list | --help).
|
|
const { runBackfillCommand } = await import('./commands/backfill.ts');
|
|
await runBackfillCommand(args);
|
|
return;
|
|
}
|
|
case 'code-callers': {
|
|
// v0.20.0 Cathedral II Layer 10 (C4): "who calls <symbol>?"
|
|
const { runCodeCallers } = await import('./commands/code-callers.ts');
|
|
await runCodeCallers(engine, args);
|
|
break;
|
|
}
|
|
case 'code-callees': {
|
|
// v0.20.0 Cathedral II Layer 10 (C5): "what does <symbol> call?"
|
|
const { runCodeCallees } = await import('./commands/code-callees.ts');
|
|
await runCodeCallees(engine, args);
|
|
break;
|
|
}
|
|
case 'repos': {
|
|
// v0.19.0: `gbrain repos ...` is an alias into the v0.18.0 sources
|
|
// subsystem. The repos abstraction (Garry's OpenClaw baseline) was
|
|
// redundant with sources and carried per-user config state that
|
|
// couldn't participate in federation / RLS / multi-tenancy. We
|
|
// keep the alias so scripts like `gbrain repos add .` keep
|
|
// working, with a nudge toward the canonical command.
|
|
console.error('[gbrain] Note: "repos" is an alias for "sources" as of v0.19.0. Prefer `gbrain sources <subcommand>`.');
|
|
const { runSources } = await import('./commands/sources.ts');
|
|
await runSources(engine, args);
|
|
break;
|
|
}
|
|
}
|
|
} finally {
|
|
if (command !== 'serve') await engine.disconnect();
|
|
}
|
|
}
|
|
|
|
/**
|
|
* v0.41.6.0 D3: dispatch helper for the read-only commands that take a
|
|
* default wallclock timeout (`gbrain search`, `gbrain sources list`).
|
|
* Keeps the timeout-wrap site in main() small and the per-command
|
|
* dispatch logic colocated for easy extension. Pure dispatcher; no engine
|
|
* lifecycle (caller owns connect/disconnect).
|
|
*/
|
|
async function dispatchReadOnlyCommand(engine: BrainEngine, command: string, args: string[]): Promise<void> {
|
|
switch (command) {
|
|
case 'search': {
|
|
const { runSearch } = await import('./commands/search.ts');
|
|
await runSearch(engine, args);
|
|
return;
|
|
}
|
|
case 'sources': {
|
|
const { runSources } = await import('./commands/sources.ts');
|
|
await runSources(engine, args);
|
|
return;
|
|
}
|
|
default:
|
|
throw new Error(`dispatchReadOnlyCommand: unsupported command "${command}"`);
|
|
}
|
|
}
|
|
|
|
// Build the AIGatewayConfig payload from a GBrainConfig. Both configureGateway
|
|
// sites in connectEngine() pass through this helper so adding a new field
|
|
// touches one place. Adding a field to one site but not the other previously
|
|
// required remembering to mirror the change; the helper makes that structural.
|
|
// v0.37.6.0: exported so `test/ai/build-gateway-config.test.ts` can pin the
|
|
// env-baseURL passthrough contract for every `_BASE_URL` env var the CLI
|
|
// reads (LLAMA_SERVER, OLLAMA, LMSTUDIO, LITELLM, OPENROUTER).
|
|
export function buildGatewayConfig(c: GBrainConfig): AIGatewayConfig {
|
|
// v0.32 (#121 reworked): when ~/.gbrain/config.json declares
|
|
// openai_api_key / anthropic_api_key, fold them into the gateway env so
|
|
// recipes that read OPENAI_API_KEY / ANTHROPIC_API_KEY find them. Process
|
|
// env still wins (it's loaded last) — this is a fallback for daemons /
|
|
// launchd-spawned subprocesses that don't propagate ~/.zshrc-sourced keys.
|
|
const envFromConfig: Record<string, string> = {};
|
|
if (c.openai_api_key) envFromConfig.OPENAI_API_KEY = c.openai_api_key;
|
|
if (c.anthropic_api_key) envFromConfig.ANTHROPIC_API_KEY = c.anthropic_api_key;
|
|
// v0.37 fix wave (CDX2-5+6): ZE became the default provider in v0.36 but
|
|
// the env-mapping at this seam never picked it up. `gbrain config set
|
|
// zeroentropy_api_key X` wrote DB plane (ignored by gateway). The file-
|
|
// plane field now exists (GBrainConfig type) and gets mapped here, so
|
|
// setting it via `~/.gbrain/config.json` propagates into the gateway.
|
|
if (c.zeroentropy_api_key) envFromConfig.ZEROENTROPY_API_KEY = c.zeroentropy_api_key;
|
|
|
|
// v0.32 codex finding #4+#5 fix: thread local-server _BASE_URL env vars
|
|
// into base_urls so the gateway hits the user's configured port. Without
|
|
// this, `LLAMA_SERVER_BASE_URL=http://localhost:9000` would let the probe
|
|
// succeed against :9000 but the actual embed call would still go to the
|
|
// recipe's base_url_default (localhost:8080). Same fix applies to
|
|
// OLLAMA_BASE_URL. Caller-provided cfg.provider_base_urls wins.
|
|
const envBaseUrls: Record<string, string> = {};
|
|
if (process.env.LLAMA_SERVER_BASE_URL) envBaseUrls['llama-server'] = process.env.LLAMA_SERVER_BASE_URL;
|
|
// v0.40.6.1: sibling recipe for llama-server in reranking mode. Separate
|
|
// env var because --reranking and --embeddings are mutually exclusive at
|
|
// server launch — users running both will have two llama-server processes
|
|
// on different ports.
|
|
if (process.env.LLAMA_SERVER_RERANKER_BASE_URL) envBaseUrls['llama-server-reranker'] = process.env.LLAMA_SERVER_RERANKER_BASE_URL;
|
|
if (process.env.OLLAMA_BASE_URL) envBaseUrls['ollama'] = process.env.OLLAMA_BASE_URL;
|
|
if (process.env.LMSTUDIO_BASE_URL) envBaseUrls['lmstudio'] = process.env.LMSTUDIO_BASE_URL;
|
|
if (process.env.LITELLM_BASE_URL) envBaseUrls['litellm'] = process.env.LITELLM_BASE_URL;
|
|
if (process.env.OPENROUTER_BASE_URL) envBaseUrls['openrouter'] = process.env.OPENROUTER_BASE_URL;
|
|
|
|
return {
|
|
embedding_model: c.embedding_model,
|
|
embedding_dimensions: c.embedding_dimensions,
|
|
embedding_multimodal_model: c.embedding_multimodal_model,
|
|
expansion_model: c.expansion_model,
|
|
chat_model: c.chat_model,
|
|
chat_fallback_chain: c.chat_fallback_chain,
|
|
base_urls: { ...envBaseUrls, ...(c.provider_base_urls ?? {}) }, // config wins over env
|
|
env: { ...envFromConfig, ...process.env }, // process.env wins
|
|
};
|
|
}
|
|
|
|
async function connectEngine(opts?: { probeOnly?: boolean }): Promise<BrainEngine> {
|
|
const config = loadConfig();
|
|
if (!config) {
|
|
console.error('No brain configured. Run: gbrain init');
|
|
process.exit(1);
|
|
}
|
|
|
|
// Configure the AI gateway BEFORE engine connect — initSchema needs embedding dims.
|
|
// Env is read once here; the gateway never reads process.env at call time (Codex C3).
|
|
const { configureGateway } = await import('./core/ai/gateway.ts');
|
|
configureGateway(buildGatewayConfig(config));
|
|
|
|
const { createEngine } = await import('./core/engine-factory.ts');
|
|
const engine = await createEngine(toEngineConfig(config));
|
|
const noRetry = process.argv.includes('--no-retry-connect') ||
|
|
process.env.GBRAIN_NO_RETRY_CONNECT === '1';
|
|
const { connectWithRetry } = await import('./core/db.ts');
|
|
await connectWithRetry(engine, toEngineConfig(config), { noRetry });
|
|
|
|
// v0.30.1 (Codex X1 / C2): probeOnly skips both hasPendingMigrations() probe
|
|
// AND initSchema(). Used by `get_health` MCP op + `gbrain upgrade --status`
|
|
// + doctor's migration_wedge check — these surfaces report wedge state and
|
|
// must NEVER themselves start or block on migrations.
|
|
if (opts?.probeOnly === true) {
|
|
return engine;
|
|
}
|
|
|
|
// v0.41.6.0 D4: race-tolerant CLI-side migration runner. Replaces the
|
|
// pre-v0.41.6.0 `try { hasPendingMigrations && initSchema() } catch warn`
|
|
// block that fired the alarming "Schema probe/migrate failed: deadlock
|
|
// detected" warning on EVERY sync when two CLIs raced on schema probe.
|
|
// The retry+poll loop quiets the warning when the race resolves
|
|
// itself (the common case); the revised wording fires only when
|
|
// migrations are genuinely stuck.
|
|
try {
|
|
const { tryRunPendingMigrations } = await import('./core/migrate.ts');
|
|
const result = await tryRunPendingMigrations(engine);
|
|
if (result.status === 'persistent') {
|
|
console.warn(
|
|
' Schema migrations are pending. Another process attempted to apply them ' +
|
|
'but the migration didn\'t complete within the retry window. This is usually transient.',
|
|
);
|
|
console.warn(' If it persists:');
|
|
console.warn(' 1. Check `gbrain doctor` for stale locks or stuck advisory locks.');
|
|
console.warn(' 2. Check `gbrain jobs supervisor status` for crashed migration workers.');
|
|
console.warn(' 3. Re-run: `gbrain apply-migrations --yes`');
|
|
} else if (result.status === 'error') {
|
|
// Non-deadlock error during initSchema. Surface the message and continue;
|
|
// subsequent operations will resurface the real schema error in context.
|
|
console.warn(` Schema probe failed: ${result.error.message}`);
|
|
console.warn(' Re-run: `gbrain apply-migrations --yes`');
|
|
}
|
|
// 'ok', 'not_needed', 'race_resolved' → silent (the common-case outcomes).
|
|
} catch (err) {
|
|
// Last-resort defense in case the helper itself throws unexpectedly.
|
|
console.warn(` Schema probe failed (unexpected): ${(err as Error).message}`);
|
|
console.warn(' Re-run: `gbrain apply-migrations --yes`');
|
|
}
|
|
|
|
// v0.27.1 (F3 fix): re-merge DB-plane config now that the engine is up.
|
|
// Flags like `embedding_multimodal` are user-mutable via `gbrain config set`
|
|
// (DB plane) and need to flow into the gateway after connect. Schema-sizing
|
|
// fields (embedding_dimensions etc.) keep their pre-connect file/env values
|
|
// — those drove initSchema and the merged config respects file/env first.
|
|
try {
|
|
const merged = await loadConfigWithEngine(engine, config);
|
|
if (merged) {
|
|
// Stash gate flags on process.env for downstream readers (import-file.ts
|
|
// dispatches on GBRAIN_EMBEDDING_MULTIMODAL, OCR consumer reads
|
|
// GBRAIN_EMBEDDING_IMAGE_OCR_*). The gateway itself doesn't read these
|
|
// flags; this preserves the contract without changing the gateway shape.
|
|
if (merged.embedding_multimodal !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_MULTIMODAL = String(merged.embedding_multimodal);
|
|
}
|
|
if (merged.embedding_image_ocr !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_IMAGE_OCR = String(merged.embedding_image_ocr);
|
|
}
|
|
if (merged.embedding_image_ocr_model !== undefined) {
|
|
process.env.GBRAIN_EMBEDDING_IMAGE_OCR_MODEL = merged.embedding_image_ocr_model;
|
|
}
|
|
// Always re-configure with merged values when DB merge succeeded. The
|
|
// trigger used to be field-name-gated (only when embedding_multimodal_model
|
|
// was set); that coupled the gate to the field set and would silently
|
|
// miss future DB-mutable gateway fields. One extra cache+shrinkState
|
|
// clear per startup is microseconds, no hot path.
|
|
configureGateway(buildGatewayConfig(merged));
|
|
}
|
|
// v0.31.12: re-resolve gateway defaults through resolveModel so
|
|
// `models.tier.*` and `models.default` overrides apply to expansion +
|
|
// chat. Per Codex F3 — configureGateway is sync; this is the async
|
|
// re-stamp seam after engine.connect() makes config reads possible.
|
|
const { reconfigureGatewayWithEngine } = await import('./core/ai/gateway.ts');
|
|
await reconfigureGatewayWithEngine(engine);
|
|
} catch {
|
|
// Non-fatal. Pre-v39 brains may not have a usable config table yet.
|
|
}
|
|
|
|
return engine;
|
|
}
|
|
|
|
function printOpHelp(op: Operation) {
|
|
const positional = (op.cliHints?.positional || []).map(p => `<${p}>`).join(' ');
|
|
const name = op.cliHints?.name || op.name;
|
|
console.log(`Usage: gbrain ${name} ${positional} [options]\n`);
|
|
console.log(op.description + '\n');
|
|
const entries = Object.entries(op.params);
|
|
if (entries.length > 0) {
|
|
console.log('Options:');
|
|
for (const [key, def] of entries) {
|
|
const isPos = op.cliHints?.positional?.includes(key);
|
|
const req = def.required ? ' (required)' : '';
|
|
const prefix = isPos ? ` <${key}>` : ` --${key.replace(/_/g, '-')}`;
|
|
console.log(`${prefix.padEnd(28)} ${def.description || ''}${req}`);
|
|
}
|
|
}
|
|
}
|
|
|
|
function printHelp() {
|
|
// Gather shared operations grouped by category
|
|
const cliNames = Array.from(cliOps.entries())
|
|
.map(([name, op]) => ({ name, desc: op.description }));
|
|
|
|
console.log(`gbrain ${VERSION} -- personal knowledge brain
|
|
|
|
USAGE
|
|
gbrain <command> [options]
|
|
|
|
SETUP
|
|
init [--pglite|--supabase|--url] Create brain (PGLite default, no server)
|
|
migrate --to <supabase|pglite> Transfer brain between engines
|
|
upgrade Self-update
|
|
check-update [--json] Check for new versions
|
|
doctor [--json] [--fast] Health check (resolver, skills, pgvector, RLS, embeddings)
|
|
integrations [subcommand] Manage integration recipes (senses + reflexes)
|
|
|
|
PAGES
|
|
get <slug> Read a page
|
|
put <slug> [< file.md] Write/update a page
|
|
delete <slug> Delete a page
|
|
list [--type T] [--tag T] [-n N] List pages
|
|
|
|
SEARCH
|
|
search <query> Keyword search (tsvector)
|
|
query <question> [--no-expand] Hybrid search (RRF + expansion)
|
|
ask <question> [--no-expand] Alias for query
|
|
|
|
IMPORT/EXPORT
|
|
import <dir> [--no-embed] Import markdown directory
|
|
sync [--repo <path>] [flags] Git-to-brain incremental sync
|
|
sync --watch [--interval N] Continuous sync (loops until stopped)
|
|
sync --install-cron Install persistent sync daemon
|
|
export [--dir ./out/] Export to markdown
|
|
export --restore-only [--repo <p>] Restore missing supabase-only files
|
|
[--type T] [--slug-prefix S] With optional filters
|
|
|
|
FILES
|
|
files list [slug] List stored files
|
|
files upload <file> --page <slug> Upload file to storage
|
|
files upload-raw <file> --page <s> Smart upload (size routing + .redirect.yaml)
|
|
files signed-url <path> Generate signed URL (1-hour)
|
|
files sync <dir> Bulk upload directory
|
|
files verify Verify all uploads
|
|
|
|
EMBEDDINGS
|
|
embed [<slug>|--all|--stale] Generate/refresh embeddings
|
|
|
|
LINKS
|
|
link <from> <to> [--type T] Create typed link
|
|
unlink <from> <to> Remove link
|
|
backlinks <slug> Incoming links
|
|
graph <slug> [--depth N] Traverse link graph (returns nodes)
|
|
graph-query <slug> [--type T] Edge-based traversal with type/direction filters
|
|
[--depth N] [--direction in|out|both]
|
|
|
|
TAGS
|
|
tags <slug> List tags
|
|
tag <slug> <tag> Add tag
|
|
untag <slug> <tag> Remove tag
|
|
|
|
TIMELINE
|
|
timeline [<slug>] View timeline
|
|
timeline-add <slug> <date> <text> Add timeline entry
|
|
|
|
TOOLS
|
|
extract <links|timeline|all> Extract links/timeline (idempotent)
|
|
[--source fs|db] fs (default) walks .md files; db iterates engine pages
|
|
[--dir <brain>] brain dir for fs source
|
|
[--type T] [--since DATE] filters (db source)
|
|
[--dry-run] [--json]
|
|
publish <page.md> [--password] Shareable HTML (strips private data, optional AES-256)
|
|
check-backlinks <check|fix> [dir] Find/fix missing back-links across brain
|
|
lint <dir|file> [--fix] Catch LLM artifacts, placeholder dates, bad frontmatter
|
|
orphans [--json] [--count] Find pages with no inbound wikilinks
|
|
salience [--days N] [--kind P] v0.29: pages ranked by emotional + activity salience
|
|
anomalies [--since D] [--sigma N] v0.29: cohort-based statistical anomalies (tag, type)
|
|
transcripts recent [--days N] v0.29: recent raw .txt transcripts (local-only)
|
|
dream [--dry-run] [--json] Run the overnight maintenance cycle once (cron-friendly).
|
|
See also: autopilot --install (continuous daemon).
|
|
check-resolvable [--json] [--fix] Validate skill tree (reachability/MECE/DRY)
|
|
report --type <name> --content ... Save timestamped report to brain/reports/
|
|
|
|
BRAIN (capture / ideate / explore — v0.37/v0.38)
|
|
capture [content] [--file PATH] Single entrypoint for getting content into the brain
|
|
[--stdin] [--slug s] [--type t] Inline content / file / stdin; writes to inbox/ by default
|
|
[--source ID] [--quiet|--json] Multi-source brains: route to a non-default source
|
|
brainstorm <question> [--json] Bisociation idea generator (hybrid search + far-set + judge)
|
|
[--save|--no-save] [--limit N]
|
|
lsd <question> [--json] Lateral Synaptic Drift: inverted-judge brainstorm
|
|
[--save|--no-save] [--limit N] rewarding far-from-obvious + axiomatic inversions
|
|
|
|
SOURCES (multi-repo / multi-brain)
|
|
sources list Show registered sources
|
|
sources add <id> --path <p> Register a source (id = short name, e.g. 'wiki')
|
|
sources remove <id> Remove a source + its pages
|
|
sync --all Sync all sources with a local_path
|
|
sync --source <id> Sync one specific source
|
|
repos ... DEPRECATED alias for 'sources' (v0.19.0)
|
|
|
|
CODE INDEXING (v0.19.0 / v0.20.0 Cathedral II)
|
|
code-def <symbol> [--lang l] Find the definition of a symbol across code pages
|
|
code-refs <symbol> [--lang l] Find all references to a symbol (JSON-first)
|
|
code-callers <symbol> Who calls this symbol? (v0.20.0 A1)
|
|
code-callees <symbol> What does this symbol call? (v0.20.0 A1)
|
|
query <q> --lang <l> Filter hybrid search to one language (v0.20.0)
|
|
query <q> --symbol-kind <k> Filter to symbol type (function|class|method|...) (v0.20.0)
|
|
reconcile-links [--dry-run] Batch-recompute doc↔impl edges (v0.20.0)
|
|
reindex-code [--source id] [--yes] Explicit code-page reindex (v0.20.0)
|
|
sync --strategy code Sync code files into the brain
|
|
|
|
JOBS (Minions)
|
|
jobs submit <name> [--params JSON] Submit background job [--follow] [--dry-run]
|
|
jobs list [--status S] [--limit N] List jobs
|
|
jobs get <id> Job details + history
|
|
jobs cancel <id> Cancel job
|
|
jobs retry <id> Re-queue failed/dead job
|
|
jobs prune [--older-than 30d] Clean old jobs
|
|
jobs stats Job health dashboard
|
|
jobs work [--queue Q] Start worker daemon (Postgres only)
|
|
|
|
ADMIN
|
|
stats Brain statistics
|
|
health Brain health dashboard
|
|
history <slug> Page version history
|
|
revert <slug> <version-id> Revert to version
|
|
features [--json] [--auto-fix] Scan usage + recommend unused features
|
|
autopilot [--repo] [--interval N] Self-maintaining brain daemon
|
|
config [show|get|set] <key> [val] Brain config
|
|
storage status [--repo <path>] Storage tier status and health
|
|
[--json] (git-tracked vs supabase-only)
|
|
serve MCP server (stdio)
|
|
serve --http [--port N] HTTP MCP server with OAuth 2.1
|
|
--token-ttl N Access token TTL in seconds (default: 3600)
|
|
--enable-dcr Enable Dynamic Client Registration
|
|
--public-url URL Public issuer URL (required behind proxy/tunnel)
|
|
call <tool> '<json>' Raw tool invocation
|
|
version Version info
|
|
--tools-json Tool discovery (JSON)
|
|
|
|
Run gbrain <command> --help for command-specific help.
|
|
`);
|
|
}
|
|
|
|
main().catch(e => {
|
|
console.error(e.message || e);
|
|
process.exit(1);
|
|
});
|