Files
gbrain/src/core/operations.ts
T
f0825018dd v0.26.3 feat(admin): observability + per-agent config + auth hardening (#586)
* feat(admin): legacy API keys alongside OAuth clients in dashboard

Adds API key management to the admin dashboard:

Server (serve-http.ts):
- GET /admin/api/api-keys — list legacy access_tokens with status
- POST /admin/api/api-keys — create new bearer token
- POST /admin/api/api-keys/revoke — revoke by name
- Stats endpoint now includes active_api_keys count

Admin UI (Agents.tsx):
- Tabbed view: 'OAuth Clients' | 'API Keys'
- API Keys tab: table with name, status, created, last used, revoke button
- Create API Key modal with name input
- Token reveal modal with copy button + warning
- Badge showing active key count on tab

Both auth methods (OAuth 2.1 client_credentials and legacy bearer tokens)
now visible and manageable from a single admin surface.

* feat(admin): remember admin token in localStorage + auto-reauth

Login flow:
- First login: paste token, saved to localStorage
- Subsequent visits: auto-login from localStorage (no paste needed)
- Shows 'Authenticating...' spinner during auto-login
- If saved token is stale (server restarted), clears it and shows login form

Session recovery:
- If session cookie expires mid-use (server restart, 24h expiry), the API
  layer auto-reauths with the saved token before redirecting to login
- Transparent to the user — one failed request triggers reauth + retry
- Only falls back to login page if the saved token itself is invalid

Security:
- Token stored in localStorage (same-origin, tailnet-only deployment)
- Cleared automatically when token becomes invalid
- Cookie remains HttpOnly + SameSite=Strict for the actual session

* feat(admin): rich request logging + agent activity tracking

Server:
- mcp_request_log now captures params (jsonb) and error_message (text)
- Agents API returns last_used_at, total_requests, requests_today
- Request log API supports agent/operation/status filtering via query params
- SSE broadcast includes params and error details

Agents page:
- Shows 'Requests today / total' and 'Last used' (relative time) per agent
- Removed Client ID column (low signal, shown in drawer)

Request Log page:
- New 'Params' column — shows query text, slug, or param count inline
- Click any row to expand full details (params JSON, error message, timestamps)
- Click agent name to filter all requests by that agent
- Agent filter dropdown in header
- Error messages shown in red in expanded view

What this means: when Claude Code searches for 'pedro franceschi',
the admin dashboard shows the search query, which agent ran it,
how long it took, and whether it succeeded — all clickable.

* feat(admin): magic link login — ask your agent for the URL

New flow:
1. User opens /admin → sees 'This is a protected dashboard'
2. UI tells them: 'Ask your AI agent for the admin login link'
3. Agent generates: https://host:port/admin/auth/<token>
4. User clicks the link → auto-authenticates → redirects to dashboard
5. Session lasts 7 days (magic link) vs 24h (manual token paste)

Server: GET /admin/auth/:token validates the bootstrap token, sets
HttpOnly cookie, redirects to /admin/. Invalid tokens get a plain
text error telling them to ask their agent for a fresh link.

Login page: primary UX is the 'ask your agent' prompt with example.
Manual token paste collapsed under a <details> disclosure.

* feat(admin): config export for Claude Code, ChatGPT, Claude.ai, Cursor, Perplexity

Agent drawer now shows setup instructions for 5 clients + raw JSON:
- Claude Code: .mcp.json with bearer token + curl to mint
- ChatGPT: Settings → Tools → MCP with OAuth discovery
- Claude.ai (Cowork): Connected Apps → MCP with OAuth
- Cursor: .cursor/mcp.json with OAuth config
- Perplexity: Connectors with client ID/secret
- JSON: raw config with all URLs (server, token, discovery)

All snippets use the actual server URL (window.location.origin)
instead of placeholder YOUR_SERVER. Client ID pre-filled.

* feat(admin): per-client token TTL — configurable token lifetime

Problem: OAuth tokens expire in 1 hour (hardcoded). Claude Code's built-in
OAuth client doesn't auto-refresh, so users get 401s every hour.

Fix: per-client token_ttl column on oauth_clients table. Set at registration
time or updated later via the admin dashboard.

Server:
- oauth_clients.token_ttl column (nullable integer, seconds)
- exchangeClientCredentials reads per-client TTL, falls back to server default
- POST /admin/api/register-client accepts tokenTtl param
- POST /admin/api/update-client-ttl for existing clients
- Agents API returns token_ttl for display

Admin UI:
- Register modal: Token Lifetime dropdown (1h, 24h, 7d, 30d, 1y, no expiry)
- Agent drawer: shows current TTL in Details section

Presets: gstack-desktop and garry-claude-code set to 30-day tokens.

* fix(admin): request log shows agent name instead of truncated client_id

Resolves client_id → client_name via LEFT JOIN on oauth_clients (and
access_tokens for legacy keys). Agent column now shows 'gstack-desktop'
instead of 'd0db7692caf5…'. Clickable to filter by agent.

* feat(admin): DESIGN.md + left-align everything

DESIGN.md establishes the admin dashboard design system:
- Left-align all text (Garry preference)
- Inter + JetBrains Mono (shared DNA with GStack)
- No accent color — semantic badges carry all color
- Dense utilitarian ops dashboard
- Component specs and anti-patterns documented

CSS: login-box text-align center → left

* feat(admin): unified agent view + resolved agent names in request log

Agent names stored at log time (agent_name column). Agents page shows
OAuth clients and API keys in one unified table. Request log shows
human-readable names. Backfilled 1,114 existing entries.

* feat(admin): working Revoke Agent button + e2e tests

Bugs fixed:
- Revoke Agent button was a no-op (no onClick handler, no API endpoint)
- Legacy API key tokens got 401 at /mcp (missing expiresAt in AuthInfo)
- token_ttl and deleted_at queries failed on PGLite (columns don't exist)

Server:
- POST /admin/api/revoke-client: soft-deletes oauth_clients + purges tokens
- exchangeClientCredentials checks deleted_at (graceful if column missing)
- Legacy token verify returns expiresAt (1yr future) for SDK compat

UI:
- Revoke button: confirm dialog → revoke → close drawer → reload table
- Shows 'This agent has been revoked' for revoked agents

E2E tests (2 new cases, 17 total):
- revoke client via admin API invalidates all tokens (mint → use → revoke → verify rejected → mint fails)
- revoke API key via admin API (create → use at /mcp → revoke → verify rejected)

52 tests, 0 failures, 213 assertions across unit + e2e.

* fix(test): e2e tests clean up after themselves — no more orphan clients

Problem: every test run left e2e-oauth-test, e2e-revoke-test, and
e2e-revoke-key-test rows in oauth_clients and access_tokens. The CLI-based
cleanup in afterAll was failing silently.

Fix:
- beforeAll: SQL DELETE of any e2e-* orphans from previous crashed runs
- afterAll: direct SQL cleanup of oauth_tokens, oauth_clients, access_tokens,
  mcp_request_log — all rows matching 'e2e-%' pattern
- No reliance on CLI commands for cleanup (they fail silently)

Verified: 52 tests pass, 0 test rows remain after run.

* feat(admin): hide revoked toggle on Agents page

* fix(admin): styled error page for expired magic links

Matches the login page aesthetic instead of plain text. Dark theme,
GBrain logo, explains the link expired, tells user to ask their agent.

* fix(admin): clean config export — auth-type-aware Claude Code instructions

* fix(admin): rewrite all config exports — command language, auth-type-aware, verified syntax

* fix(admin): API key rows clickable with revoke + sync all fixes from master

Syncs all accumulated fixes onto the PR branch:
- API key rows in agents table now open drawer with Revoke button
- API keys show bearer token usage hint instead of config export tabs
- Config export snippets use command language directed at the AI agent
- Styled expired magic link error page
- Hide revoked toggle
- Test cleanup via direct SQL
- All v0.26.2 upstream fixes incorporated

* fix(oauth): port coerceTimestamp helper from master 1055e10c

Tests in test/oauth.test.ts (already on this branch) import coerceTimestamp
from oauth-provider.ts. The import was synced from master via PR commit 16
("sync all fixes from master") but the production-code change to
oauth-provider.ts was not. Result: bun test fails at module load with
"coerceTimestamp is not exported".

This commit ports the helper directly instead of merging master, avoiding
VERSION/CHANGELOG/dist conflicts.

Boundary helper for postgres.js BIGINT-as-string (auto-detected on
Supabase pgbouncer / port 6543). Throws on non-finite so corrupt rows
fail loud at the SELECT-row -> JS-number boundary. Returns undefined
for SQL NULL; comparison sites treat NULL as expired (fail-closed).

Refactors 4 sites:
- getClient: DCR response numeric-shape compliance per RFC 7591 §3.2.1
- exchangeRefreshToken: NULL -> expired fail-closed
- verifyAccessToken: single guard, narrowed return; folds in v0.26.1's
  inline Number(...) at the return site

Originally landed on master as part of #593 (v0.26.2). Ported here so
PR #586 (v0.26.3) can build standalone without a master merge.

* feat(schema): migration v33 — admin dashboard columns

Adds the 5 columns + new index referenced by PR #586 admin dashboard work
that landed without a corresponding schema migration:

  oauth_clients.token_ttl       INTEGER     -- per-client OAuth TTL override
  oauth_clients.deleted_at      TIMESTAMPTZ -- soft-delete for revoke
  mcp_request_log.agent_name    TEXT        -- resolved client_name for log
  mcp_request_log.params        JSONB       -- captured request params
  mcp_request_log.error_message TEXT        -- captured error text on failure
  idx_mcp_log_agent_time        INDEX       -- supports new agent filter

Without v33 on existing brains:
- /admin/api/agents 503s (SELECT references token_ttl + deleted_at)
- POST /admin/api/revoke-client throws 500 (UPDATE deleted_at)
- POST /admin/api/update-client-ttl throws 500 (UPDATE token_ttl)
- mcp_request_log INSERTs silently swallow column-doesn't-exist errors,
  request log appears empty to the operator

All ALTERs use ADD COLUMN IF NOT EXISTS so re-running the migration is
a no-op on a brain that already has v33.

Includes inline UPDATE backfill of agent_name on existing rows via
COALESCE on oauth_clients.client_name → access_tokens.name → token_name.

Updates:
- src/core/migrate.ts: v33 migration entry
- src/schema.sql: source-of-truth schema for fresh installs
- src/core/pglite-schema.ts: PGLite mirror
- src/core/schema-embedded.ts: regenerated via bun run build:schema
- test/migrate.test.ts: 5 SQL-shape assertions pinning the v33 contract

* refactor(serve-http): parameterize request-log filter; kill dead vars

Three issues in the prior /admin/api/requests handler:

1. sql.unsafe() with manual single-quote escape on user input:
     conditions.push(`token_name = '${agent.replace(/'/g, "''")}'`);
   Works under standard_conforming_strings=on (PG default since 9.1) but
   pattern is a footgun — any future contributor adding a filter without
   escaping breaks the dam. Backslashes are not escaped. Mitigated by
   requireAdmin but defense-in-depth says don't ship the pattern.

2. Dead variables (lines 348-357 of the prior code): `query`, `params`,
   `paramIdx` were built up with $N placeholders and then never used
   when the function fell through to sql.unsafe with manually-escaped
   strings. Confusing leftovers from an earlier parameterization attempt.

3. Unused `values: unknown[] = []` in the conditions block.

Fix: replace the entire dynamic-WHERE construction with postgres.js
tagged-template fragments. Each filter expands to either
`AND col = ${val}` (true parameter binding via the postgres-js driver)
or an empty fragment. `WHERE 1=1` lets us always have a WHERE clause
and unconditionally append AND-prefixed fragments. No string
interpolation, no manual escaping, no sql.unsafe.

Net change: -27 lines (from 30 lines of broken/dead code to 17 lines
of clean parameterized fragments).

* perf(oauth): thread client_name through AuthInfo; drop per-request lookup

PR #586's serve-http.ts /mcp handler did one extra DB roundtrip per
authenticated request to resolve client_id → client_name for logging:

  let agentName = authInfo.clientId;
  try {
    const [client] = await sql`SELECT client_name FROM oauth_clients
                                 WHERE client_id = ${authInfo.clientId}`;
    if (client) agentName = client.client_name;
  } catch { /* best effort */ }

On a busy brain (Perplexity Computer doing inline research, Claude Code
searching) that is ~50–100ms extra per /mcp request — wasted on a static
lookup that doesn't change between requests.

Codex's review reframed the planned cache+invalidation approach: the
right fix is to fold the name resolution into verifyAccessToken's
existing oauth_tokens SELECT via a LEFT JOIN on oauth_clients. One query
that was already running, returns the name as a bonus column, no module-
scope cache to maintain, no invalidation contract for future contributors
to remember.

Changes:
- AuthInfo (src/core/operations.ts): add optional clientName field with
  doc explaining why it's threaded here.
- verifyAccessToken (src/core/oauth-provider.ts): SELECT becomes
    SELECT t.client_id, t.scopes, t.expires_at, t.resource, c.client_name
    FROM oauth_tokens t
    LEFT JOIN oauth_clients c ON c.client_id = t.client_id
    WHERE t.token_hash = ${tokenHash} AND t.token_type = 'access'
  Returns clientName in AuthInfo.
- Legacy access_tokens path: clientName = name (single identifier).
- serve-http.ts /mcp handler: read authInfo.clientName directly,
  fall back to clientId. Per-request lookup removed.

Net change: -8 LOC. Eliminates the per-request DB roundtrip while
keeping the same behavior surface.

* security(serve-http): timingSafeEqual on admin token hash compare

Both /admin/login (POST, JSON body) and /admin/auth/:token (GET, magic
link) compared the sha256 of the operator-supplied token against the
known bootstrapHash via JS string `===`, which short-circuits at the
first mismatched character. The inputs are SHA-256 outputs so the
practical timing leak only reveals hash bits (not raw token bits, since
SHA-256 isn't invertible) — but defense-in-depth on the highest-
privileged URLs the server exposes is the right call.

New helper safeHexEqual(a, b):
- Length-equal check first (both are 64-char hex)
- Buffer.from(hex, 'hex') decodes each side to 32 bytes
- crypto.timingSafeEqual returns the constant-time compare result

Also tightens the POST handler's input validation: requires token to
be a string before passing to createHash (prior code only checked
truthiness, would have crashed on object-typed bodies even with
express.json's parser).

Used at both magic-link and password-style admin auth sites.

* security(serve-http): rate-limit /admin/auth/:token at 10/min/IP

Defense-in-depth on the magic-link endpoint. A misconfigured client
looping on /admin/auth/:bad would otherwise consume CPU on sha256 +
the inline HTML 401 response without bound. Brute-forcing the 64-char
hex bootstrap token is computationally infeasible regardless, so this
is about denial-of-service, not auth bypass.

Reuses the existing express-rate-limit dep already wiring /token's
client-credentials limiter. New adminAuthRateLimiter shares the same
configuration shape (standardHeaders, legacyHeaders) for consistency.

windowMs: 60_000 (1 minute)
max: 10
message: plain string ("Too many magic-link attempts. Wait a minute
before trying again.") instead of JSON envelope, matching the
endpoint's HTML response style.

* security(admin): kill JS-state token; single-use magic links; sign out everywhere

Resolves D11 + D12 from the codex-pushback review. Closes the actual
trust boundary instead of the persistence layer (sessionStorage was
security theater per codex finding #7).

# Single-use magic links (D11=C)

The bootstrap token is no longer the magic-link path component. New
flow:

  agent has bootstrap token (read from server stderr)
    -> POST /admin/api/issue-magic-link
       Authorization: Bearer <bootstrap>
    -> server returns one-time nonce URL
    -> operator clicks /admin/auth/<nonce>
    -> server consumes nonce, sets cookie, redirects to dashboard

Server state (in-memory):
- magicLinkNonces: Map<nonce, expiresAt> (5-minute TTL)
- consumedNonces:  Set<nonce> (LRU cap 1000 to bound memory)
- pruneExpiredNonces() best-effort GC on each issue/redeem

Each redemption marks the nonce consumed. Second click on the same URL
gets the styled 401 page. Leaked URL grants exactly one extra session
before dying. The bootstrap token never appears in a URL — no leakage
via browser history, proxy access logs, or Referer headers.

# Kill JS-state bootstrap token (D12=B)

admin/src/pages/Login.tsx + admin/src/api.ts:
- All localStorage reads/writes removed
- Auto-reauth-via-saved-token logic deleted
- Token only lives in form state during submit, cleared after
- 401 redirects straight to login — no cache to retry against

The HttpOnly cookie is the only session credential after successful
authentication. Closing the tab ends the session. Reopening shows the
login page. Operator asks the agent for a fresh magic link (or pastes
the bootstrap token from the server terminal).

# Sign out everywhere

POST /admin/api/sign-out-everywhere (admin-cookie-required) calls
adminSessions.clear() and returns {revoked_sessions: count}. Every
browser/tab fails its next request, gets 401, redirects to login.
Bootstrap token unaffected — still valid for new magic-link mints.

UI: button in the sidebar footer with a confirm() guard ("Sign out
every active admin session, including other browsers and tabs?").

# Notes

admin/dist is gitignored on this branch (master's v0.26.2 removed that
line; the merge to master will reconcile). After /ship's merge step,
rebuild admin/dist with `cd admin && bun run build` to capture the new
sign-out button + simplified login page.

* fix(admin): rename loadApiKeys() to loadAgents() in Agents.tsx onCreated

The Create API Key flow's onCreated callback called loadApiKeys() but
no such function exists in this file. The unified /admin/api/agents
endpoint (added in PR commit 14) returns BOTH OAuth clients AND legacy
API keys, so loadAgents() is the right call.

User-visible bug: clicking "+ API Key" -> filling in the name ->
clicking Create would mint the key on the server but throw
ReferenceError: loadApiKeys is not defined in the React onCreated
callback. The token-reveal modal would still appear (because
setShowApiKeyToken runs before the loadApiKeys call), but the agents
table wouldn't refresh, leaving the new key invisible until manual
page reload.

Five Claude review passes missed this. Codex caught it in one pass.

1-line fix.

* fix(admin): empty-state placeholder when filtered Agents result is empty

Pre-fix: the empty-state guard checked the unfiltered agents array.
If every agent was revoked AND the "Hide revoked" toggle was on
(default), the table rendered a header row with zero body rows and
no placeholder — looked like a broken / empty / loading state.

Two cases to render distinctly:

1. agents.length === 0 (truly no agents)
   "No agents registered. Register your first agent to get started."

2. visibleAgents.length === 0 BUT agents.length > 0
   (all agents are revoked, hideRevoked filter hides them all)
   "All agents are revoked. Uncheck "Hide revoked" to view them."

Refactored the table render into an IIFE so the filter expression is
computed once and shared between the empty-state guard and the row
map. Drops the prior inline `agents.filter(...).map(...)` pattern.

(F2.2 from the eng review pass #2.)

* fix(admin): restore Claude Code + Cursor tabs for API-key agents

Wintermute's commit 16 (3d5d0f87) wrapped the entire Config Export
section in {isOAuth && (...)}, hiding ALL tabs for api_key agents and
replacing them with a single line of plain instruction. That dropped
the working auth-type-aware Claude Code + Cursor snippets (added by
his own commit 15) along with the genuinely OAuth-only ChatGPT /
Claude.ai / Perplexity ones.

Codex review pass D5 settled on option C: per-tab branching. Two
clients (Claude Code, Cursor) accept raw bearer tokens in their MCP
config, so their snippets render normally for api_key agents (commit
15's auth-type-aware branching does the right thing). Three clients
(ChatGPT, Claude.ai, Perplexity) only speak OAuth 2.0 client_credentials
and reject raw bearer; for api_key agents they render an explanatory
message naming the client and pointing the operator at registering an
OAuth client instead.

JSON tab continues to render its raw structured metadata unconditionally.

Layout: removed the `{isOAuth && (...)}` outer wrap; tab list now
always visible. The body of each tab is selected via an IIFE that
checks (auth_type === 'api_key' && tab in oauthOnlyTabs).

Net change: +24 lines (the warning panel + IIFE branch logic).

* feat(admin): read -s prompt OAuth Claude Code snippet + 2-step curl fallback

Wintermute's commit 15 inlined client_secret into a long compound
`claude mcp add --header "Authorization: Bearer $(curl -d '...
client_secret=PASTE_HERE')"` line. When the operator replaces PASTE
with their real secret, that secret lands in ~/.zsh_history and
appears in `ps` output for the lifetime of the curl process.

D13=C from the eng review: ship both shapes.

Default (read -s prompt-based, ~17 lines):
- read -rs prompts for the secret without echo, stores in
  $GBRAIN_CS scoped to the shell session
- curl uses --data-urlencode "client_secret=$GBRAIN_CS" — variable
  substitution at exec time, so the secret enters the curl process's
  argv at the moment of the call, but the shell history records
  literally `--data-urlencode "client_secret=$GBRAIN_CS"`, not the
  value
- unset GBRAIN_CS afterwards to scrub the env

Fallback (2-step curl + paste, for shells without read -s):
- one curl command to mint the token (PASTE_YOUR_CLIENT_SECRET_HERE
  in the body — secret hits history but in one short isolated line
  that's easy to scrub)
- second `claude mcp add` command with PASTE_TOKEN_FROM_ABOVE — the
  bearer token, not the long-lived client secret
- bash + zsh history-deletion hint at the bottom

Both shapes preserve the agent-facing voice ("The user wants to
connect GBrain MCP to your context. Here's how.") and the token-TTL
rendering ("will last 30 days") that commit 15 added.

Net change: +25 lines in the configSnippets['claude-code'] OAuth
branch. API-key branch unchanged (single paste, no secret).

* chore(ci): gate admin React build via scripts/check-admin-build.sh

Codex review pass #6 finding #3 caught loadApiKeys() referenced but
undefined in Agents.tsx — a real shipping bug that 5 Claude review
passes missed. Root cause: the bash test pipeline never compiled the
React admin app, so missing-symbol errors only surfaced during a
deliberate `cd admin && bun run build`.

This commit threads the admin build into the standard test gate. Any
future TypeScript error or missing symbol in admin/src/ now fails
`bun run test` alongside the other shell guards (privacy, jsonb,
progress-stdout, etc.) and the typecheck step.

Behavior:
- scripts/check-admin-build.sh runs `bun install --silent` (idempotent,
  ~50ms on no-op) then `bun run build` in admin/.
- Vite's build runs `tsc -b && vite build` so type errors fail the
  pipeline, not just bundling errors.
- GBRAIN_SKIP_ADMIN_BUILD=1 escape hatch for fast inner-loop test runs
  that don't touch admin/. Production CI MUST NOT set this.
- Skips silently if admin/ doesn't exist (handles slim-clone scenarios).

Wired into both:
- "test" script: full pipeline now includes admin build before bun test
- "check:admin-build" script: invoke standalone for debugging

* test(e2e): v0.26.3 coverage — column round-trip, injection probe, TTL, magic-link

Folds together the planned fix-up commits #8-#11 since they all live in
the same E2E file and share the spawned-server harness. Each test block
is independently bisect-readable.

# Test 1: mcp_request_log new column round-trip (pins migration v33)

Wipes log rows for the e2e-oauth-test client, makes a successful
tools/list call + a failed tools/call (nonexistent tool name), then
asserts:
  - rows persisted (count >= 2) — proves the INSERT wasn't silently
    swallowed by the "best effort" try/catch on a column-doesn't-exist
    error
  - agent_name column resolves to 'e2e-oauth-test' on every row (proves
    the JOIN in verifyAccessToken or the v33 backfill path)
  - params column persisted as JSONB on tools/call
  - error_message column populated on the status='error' row

Without migration v33, every assertion fails — the column doesn't exist
so the INSERT throws, gets swallowed, and rows.length === 0.

# Test 2: request-log filter injection probe

Sends `?agent=alice'%20OR%201%3D1` to /admin/api/requests. Pre-fix,
the sql.unsafe path would have crashed the server with malformed SQL
on the way to the auth check (or worse, returned all rows under broken
escaping). Post-fix (parameterized fragments), the unauthenticated
request hits 401 without ever touching SQL.

Asserts:
  - 401 (not 500) on the injection input
  - server still responsive on /health afterwards (didn't crash)

# Test 3: per-client token_ttl flow

Registers e2e-test-ttl, sets oauth_clients.token_ttl, mints a token,
asserts response's expires_in matches. Cycles through three states:
  - token_ttl = 86400 → expires_in = 86400 (24h custom override)
  - token_ttl = 7200  → expires_in = 7200 (2h different custom)
  - token_ttl = NULL  → expires_in = 3600 (server default fallback)

Pins the per-client TTL feature added in PR #586 commit 6 (e7989e97).

# Test 4: magic-link styled 401 page + single-use semantic

(a) Invalid nonce returns Content-Type: text/html with a body that
    contains "expired" and "GBrain" — pins the styled error page from
    PR commit 13 (f8f5cfe8).

(b) Single-use semantic: extract bootstrap token from server stderr
    (best-effort; skips gracefully if not extractable), POST to
    /admin/api/issue-magic-link to mint a one-time nonce URL, click
    once (gets 302 + cookie), click again (gets styled 401). Pins the
    D11=C single-use rotation logic.

# Test 5: agent_name resolution path

Makes an OAuth request and asserts mcp_request_log.agent_name resolves
to the OAuth client_name (not the truncated client_id). Pins the JOIN
introduced in fix-up #4 + the v33 backfill path.

# Test 6: register-client missing-name returns 400 (basic input validation)

Hits /admin/api/register-client without auth — must 401 (not crash 500).

# Other changes

- Renamed describe header from `(v0.26.1 + v0.26.2)` to
  `(v0.26.1 + v0.26.2 + v0.26.3)` — F6.5.
- All postgres.js sql tag bindings on `clientId` / `clientSecret` use
  the `!` non-null assertion since these are typed `string | undefined`
  in the test fixture but always assigned before each test block runs.
- Result casts go through `as unknown as ...` per postgres.js's RowList
  typing (the lib's structural type doesn't unify with bare interface
  arrays).

* chore: privacy sweep + integrity.ts on getconnection allow-list

Two pre-existing CI failures uncovered while running `bun run test`
on this branch — unrelated to v0.26.3 substance but blocking the
pipeline.

# Privacy sweep (src/core/mounts-cache.ts)

Two references to the private agent fork name in code comments,
violating CLAUDE.md privacy rule ("never reference real people,
companies, funds, or private agent names in any public-facing
artifact"). Both authored in v0.26.0 commit 3c032d79e.

  - line 6 (docblock):
    "Host agents (Wintermute / OpenClaw / any Claude Code install) read"
    -> "Host agents (your OpenClaw / any Claude Code install) read"
  - line 324 (RESOLVER preamble emitter):
    "Host agents (Wintermute/OpenClaw/Claude Code) should prefer this file over"
    -> "Host agents (your OpenClaw / Claude Code) should prefer this file over"

Per the documented substitution: "your OpenClaw" for reader-facing copy
covers any downstream OpenClaw deployment (Wintermute, Hermes, AlphaClaw,
etc.) without leaking the private name into search engines or release
artifacts.

# integrity.ts on the getconnection allow-list

`scripts/check-no-legacy-getconnection.sh` flags `db.getConnection()`
calls outside `src/core/db.ts` to enforce the multi-brain routing
contract. `src/commands/integrity.ts:355` (scanIntegrityBatch) was
introduced in v0.22.16 commit 8468ba25a — the check ran clean at the
time because the file wasn't on the allow-list yet, but PR #586's
test pipeline catches it.

Adds the file to ALLOWED with a "PR 1 cleanup" note matching the
existing entries' pattern. The proper fix (refactor to accept engine
from OperationContext) is out of v0.26.3 scope and tracked alongside
the other PR 1 entries.

* chore: bump v0.26.2 -> v0.26.3 + CHANGELOG

VERSION + package.json already at 0.26.3 from the initial bump on this
branch (see commit history). This commit lands the rewritten CHANGELOG
entry covering everything that actually shipped in v0.26.3 — well past
the original "legacy API keys" framing.

What lands in v0.26.3:

# Headline (admin trust model)

Bootstrap token never persists in browser JS state (no localStorage,
no sessionStorage). Magic-link URLs use single-use server-issued
nonces — bootstrap token never appears in a URL. Cookie sessions are
HttpOnly + SameSite=Strict. "Sign out everywhere" button revokes every
active admin session in one click.

# Schema

Migration v33 adds 5 columns referenced by PR #586's admin-dashboard
work that landed without a corresponding migration. Without v33,
existing brains 503 on /admin/api/agents and silently empty their
request log. Backfill of agent_name from oauth_clients.client_name
-> access_tokens.name -> token_name baked into the migration.

# Performance

verifyAccessToken JOINs oauth_clients in its existing token SELECT
and returns clientName on AuthInfo. Removes the per-MCP-request DB
roundtrip that was happening on every authenticated /mcp call.

# Security

- crypto.timingSafeEqual on admin token hash compare
- /admin/auth/:nonce rate-limited at 10/min/IP
- Single-use nonces with 5-minute TTL
- Request-log filter parameterized via postgres.js tagged-template
  fragments (sql.unsafe + manual escape removed)
- Per-client OAuth token TTL (1h, 24h, 7d, 30d, 1y, no expiry)
- Ported coerceTimestamp helper from master v0.26.2 (BIGINT-as-string fix)

# UI

- API keys + OAuth clients in one unified Agents table
- Auth-type-aware Config Export tabs
- Claude Code OAuth: read -s prompt-based snippet (default) +
  2-step curl fallback (D13=C)
- Cursor: OAuth discovery URL OR raw bearer based on auth type
- ChatGPT/Cowork/Perplexity: "OAuth client required" CTA on api_key agents
- Hide-revoked toggle + empty-state placeholder for filtered-empty
- Bug fix: loadApiKeys -> loadAgents (codex caught what 5 review
  passes missed; Create-API-Key flow was broken)

# Tests + CI

- New E2E coverage: column round-trip, injection probe, per-client
  TTL, magic-link single-use, styled 401, agent_name resolution
- Admin React build is now a CI gate (catches missing-symbol bugs
  before E2E)
- check-no-legacy-getconnection allowlist updated for integrity.ts

Branch shape: 16 author commits + 13 fix-up commits = 29 commits on
PR. Commit-by-commit bisect-friendly.

Plan + codex review pass artifacts at
~/.claude/plans/check-this-out-and-breezy-forest.md.

---------

Co-authored-by: Wintermute <wintermute@garrytan.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-05-03 16:49:21 -07:00

1562 lines
59 KiB
TypeScript

/**
* Contract-first operation definitions. Single source of truth for CLI, MCP, and tools-json.
* Each operation defines its schema, handler, and optional CLI hints.
*/
import { lstatSync, realpathSync } from 'fs';
import { resolve, relative, sep } from 'path';
import type { BrainEngine } from './engine.ts';
import { clampSearchLimit } from './engine.ts';
import type { GBrainConfig } from './config.ts';
import type { PageType } from './types.ts';
import { importFromContent } from './import-file.ts';
import { hybridSearch } from './search/hybrid.ts';
import { expandQuery } from './search/expansion.ts';
import { dedupResults } from './search/dedup.ts';
import { captureEvalCandidate, isEvalCaptureEnabled, isEvalScrubEnabled } from './eval-capture.ts';
import type { HybridSearchMeta } from './types.ts';
import { extractPageLinks, isAutoLinkEnabled, isAutoTimelineEnabled, parseTimelineEntries, makeResolver, type UnresolvedFrontmatterRef } from './link-extraction.ts';
import * as db from './db.ts';
// --- Types ---
export type ErrorCode =
| 'page_not_found'
| 'invalid_params'
| 'embedding_failed'
| 'storage_error'
| 'bucket_not_found'
| 'database_error'
| 'permission_denied';
export class OperationError extends Error {
constructor(
public code: ErrorCode,
message: string,
public suggestion?: string,
public docs?: string,
) {
super(message);
this.name = 'OperationError';
}
toJSON() {
return {
error: this.code,
message: this.message,
suggestion: this.suggestion,
docs: this.docs,
};
}
}
// --- Upload validators (Fix 1 / B5 / H5 / M4) ---
/**
* Validate an upload path. Two modes:
* - strict (remote=true): confines the resolved path to `root` and rejects symlinks.
* Used when the caller is untrusted (MCP over stdio/HTTP, agent-facing).
* - loose (remote=false): only verifies the file exists and is not a symlink whose
* target escapes the filesystem (no path traversal protection). Used for local CLI
* where the user owns the filesystem.
*
* Either way: symlinks in the final component are always rejected (prevents
* transparent redirection to a different file than the user typed).
*
* @param filePath caller-supplied path
* @param root confinement root (only used when strict=true)
* @param strict true → enforce cwd confinement (B5 + H1). false → allow any accessible path.
* @throws OperationError(invalid_params) on symlink escape, traversal, or missing file
*/
export function validateUploadPath(filePath: string, root: string, strict = true): string {
let real: string;
try {
real = realpathSync(resolve(filePath));
} catch (e: unknown) {
const msg = e instanceof Error ? e.message : String(e);
if (msg.includes('ENOENT')) {
throw new OperationError('invalid_params', `File not found: ${filePath}`);
}
throw new OperationError('invalid_params', `Cannot resolve path: ${filePath}`);
}
// Always reject final-component symlinks (basic safety for both modes).
try {
if (lstatSync(resolve(filePath)).isSymbolicLink()) {
throw new OperationError('invalid_params', `Symlinks are not allowed for upload: ${filePath}`);
}
} catch (e) {
if (e instanceof OperationError) throw e;
// lstat race with unlink — pass if realpath already succeeded.
}
if (!strict) return real;
// Strict mode: confine to root via realpath + path.relative (catches parent-dir symlinks per B5).
let realRoot: string;
try {
realRoot = realpathSync(root);
} catch {
throw new OperationError('invalid_params', `Confinement root not accessible: ${root}`);
}
const rel = relative(realRoot, real);
if (rel === '' || rel.startsWith('..') || rel.startsWith(`..${sep}`) || resolve(realRoot, rel) !== real) {
throw new OperationError('invalid_params', `Upload path must be within the working directory: ${filePath}`);
}
return real;
}
/**
* Allowlist validator for page slugs. Rejects URL-encoded traversal, backslashes,
* control chars, RTL overrides, Unicode lookalikes — anything outside the allowlist.
* Format: lowercase alphanumeric + hyphen segments separated by single forward slashes.
*/
export function validatePageSlug(slug: string): void {
if (typeof slug !== 'string' || slug.length === 0) {
throw new OperationError('invalid_params', 'page_slug must be a non-empty string');
}
if (slug.length > 255) {
throw new OperationError('invalid_params', 'page_slug exceeds 255 characters');
}
if (!/^[a-z0-9][a-z0-9\-]*(\/[a-z0-9][a-z0-9\-]*)*$/i.test(slug)) {
throw new OperationError('invalid_params', `Invalid page_slug: ${slug} (allowed: alphanumeric, hyphens, forward-slash separated segments)`);
}
}
/**
* Match a slug against a list of allow-list prefix globs.
*
* Glob form: `<prefix>/*` matches any slug starting with `<prefix>/` and
* having at least one more segment (single or multi). Bare `<prefix>` (no
* trailing `/*`) matches that exact slug only. The `*` is intentionally
* permissive — depth is unbounded, so `wiki/originals/*` matches both
* `wiki/originals/idea-x` and `wiki/originals/ideas/2026-04-25-idea-y`.
*
* Used by the v0.23 dream-cycle trusted-workspace path. Order doesn't
* matter; the first match wins (returns true on any match).
*/
export function matchesSlugAllowList(slug: string, prefixes: readonly string[]): boolean {
for (const p of prefixes) {
if (p.endsWith('/*')) {
const base = p.slice(0, -2);
if (slug === base) continue;
if (slug.startsWith(base + '/')) return true;
} else if (p === slug) {
return true;
}
}
return false;
}
/**
* Allowlist validator for uploaded file basenames. Rejects control chars, backslashes,
* RTL overrides (\u202E), leading dot (hidden files) and leading dash (CLI flag confusion).
* Allows extension dots and underscores. Max 255 chars.
*/
export function validateFilename(name: string): void {
if (typeof name !== 'string' || name.length === 0) {
throw new OperationError('invalid_params', 'Filename must be a non-empty string');
}
if (name.length > 255) {
throw new OperationError('invalid_params', 'Filename exceeds 255 characters');
}
if (!/^[a-zA-Z0-9][a-zA-Z0-9._\-]*$/.test(name)) {
throw new OperationError('invalid_params', `Invalid filename: ${name} (allowed: alphanumeric, dot, underscore, hyphen — no leading dot/dash, no control chars or backslash)`);
}
}
export interface ParamDef {
type: 'string' | 'number' | 'boolean' | 'object' | 'array';
required?: boolean;
description?: string;
default?: unknown;
enum?: string[];
items?: ParamDef;
}
export interface Logger {
info(msg: string): void;
warn(msg: string): void;
error(msg: string): void;
}
export interface AuthInfo {
token: string;
clientId: string;
/**
* Human-readable agent name resolved at token-verification time.
* For OAuth clients this is `oauth_clients.client_name`; for legacy
* bearer tokens it is `access_tokens.name`. Threading this through
* AuthInfo eliminates a per-request DB roundtrip in the /mcp handler
* (was: SELECT client_name FROM oauth_clients WHERE client_id = ?
* on every request — see PR #586 review note D14=B).
*/
clientName?: string;
scopes: string[];
expiresAt?: number;
}
export interface OperationContext {
engine: BrainEngine;
config: GBrainConfig;
logger: Logger;
dryRun: boolean;
/**
* OAuth auth info (v0.8+). Present when the caller authenticated via OAuth 2.1
* through `gbrain serve --http`. Contains clientId and granted scopes for
* per-operation scope enforcement.
*/
auth?: AuthInfo;
/**
* True when the caller is remote/untrusted (MCP over stdio/HTTP, or any agent-facing entry point).
* False for local CLI invocations by the owner of the machine.
*
* Security-sensitive operations (e.g., file_upload) tighten their filesystem
* confinement when remote=true and allow unrestricted local-filesystem access
* when remote=false.
*
* When unset, operations MUST default to the stricter (remote=true) behavior.
*/
remote?: boolean;
/**
* Subagent runtime context (v0.16+). Set by the subagent tool dispatcher when
* dispatching an op as a tool call from an LLM loop. Used to enforce per-op
* agent policy (e.g. put_page namespace rule).
*
* `viaSubagent` is the FAIL-CLOSED flag: when true, agent-facing policy MUST
* be enforced even if `subagentId` happens to be undefined (a bug in the
* dispatcher must not bypass the guard). `subagentId` is the owning subagent
* job id; `jobId` is the current Minion job id (aggregator or subagent).
*/
jobId?: number;
subagentId?: number;
viaSubagent?: boolean;
/**
* Trusted-workspace allow-list (v0.23 dream cycle). When the cycle's
* synthesize/patterns phases dispatch a subagent, they thread an
* explicit list of slug-prefix globs (e.g. "wiki/personal/reflections/*")
* through this field. put_page enforces it BEFORE the legacy
* `wiki/agents/<id>/...` namespace check.
*
* Trust comes from the SUBMITTER (subagent jobs are gated by
* PROTECTED_JOB_NAMES — MCP cannot submit them), not from `remote`.
* Every subagent tool call has `remote=true` for auto-link safety,
* so basing trust on `remote` is incoherent (would always reject).
*
* Empty / unset → fall back to the legacy namespace check (existing
* v0.15 behavior; pure addition, no regression).
*/
allowedSlugPrefixes?: string[];
/**
* Resolved global CLI options (--quiet / --progress-json / --progress-interval).
* CLI callers populate this from `getCliOptions()`. MCP / library callers
* may leave it undefined — consumers default to quiet/no-progress for
* background work.
*/
cliOpts?: { quiet: boolean; progressJson: boolean; progressInterval: number };
/**
* Connected-gbrains brain id (v0.19+). Identifies which brain this op is
* targeting. 'host' for the default brain configured in ~/.gbrain/config.json;
* otherwise a mount id registered in ~/.gbrain/mounts.json.
*
* `ctx.engine` is the resolved BrainEngine for this id (populated by
* BrainRegistry at dispatch time). `brainId` exists alongside for:
* - audit logging (mount-ops JSONL carries the id)
* - subagent inheritance (child jobs receive the parent's brainId)
* - cross-brain citation prefixes in agent output
*
* Orthogonal to v0.18.0's source_id, which scopes per-repo WITHIN a brain.
* See docs/architecture/brains-and-sources.md for the mental model.
*
* Omitted = 'host' (pre-v0.19 callers + single-brain deployments keep
* working without change).
*/
brainId?: string;
}
export interface Operation {
name: string;
description: string;
params: Record<string, ParamDef>;
handler: (ctx: OperationContext, params: Record<string, unknown>) => Promise<unknown>;
mutating?: boolean;
scope?: 'read' | 'write' | 'admin';
localOnly?: boolean;
cliHints?: {
name?: string;
positional?: string[];
stdin?: string;
hidden?: boolean;
};
}
// --- Page CRUD ---
const get_page: Operation = {
name: 'get_page',
description: 'Read a page by slug (supports optional fuzzy matching)',
params: {
slug: { type: 'string', required: true, description: 'Page slug' },
fuzzy: { type: 'boolean', description: 'Enable fuzzy slug resolution (default: false)' },
},
handler: async (ctx, p) => {
const slug = p.slug as string;
const fuzzy = (p.fuzzy as boolean) || false;
let page = await ctx.engine.getPage(slug);
let resolved_slug: string | undefined;
if (!page && fuzzy) {
const candidates = await ctx.engine.resolveSlugs(slug);
if (candidates.length === 1) {
page = await ctx.engine.getPage(candidates[0]);
resolved_slug = candidates[0];
} else if (candidates.length > 1) {
return { error: 'ambiguous_slug', candidates };
}
}
if (!page) {
throw new OperationError('page_not_found', `Page not found: ${slug}`, 'Check the slug or use fuzzy: true');
}
const tags = await ctx.engine.getTags(page.slug);
return { ...page, tags, ...(resolved_slug ? { resolved_slug } : {}) };
},
scope: 'read',
cliHints: { name: 'get', positional: ['slug'] },
};
const put_page: Operation = {
name: 'put_page',
description: 'Write/update a page (markdown with frontmatter). Chunks, embeds, reconciles tags, and (when auto_link/auto_timeline are enabled) extracts + reconciles graph links and timeline entries.',
params: {
slug: { type: 'string', required: true, description: 'Page slug' },
content: { type: 'string', required: true, description: 'Full markdown content with YAML frontmatter' },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
const slug = p.slug as string;
// Subagent namespace enforcement (v0.15+). Runs BEFORE the dry-run
// short-circuit so preview calls surface the same rejection. Confines
// LLM-driven writes to wiki/agents/<subagentId>/... — no leading slash
// (slug grammar rejects that), anchored, slash-boundary to defeat prefix
// collisions like `wiki/agents/12evil/*` impersonating subagent 12.
//
// FAIL-CLOSED: `viaSubagent=true` enforces the check even if the
// dispatcher forgot to populate `subagentId`. Agent-originated writes
// without an owning subagent id are rejected outright.
if (ctx.viaSubagent === true) {
if (typeof ctx.subagentId !== 'number' || Number.isNaN(ctx.subagentId)) {
throw new OperationError('permission_denied', 'put_page via subagent requires ctx.subagentId');
}
const allowList = ctx.allowedSlugPrefixes;
if (allowList && allowList.length > 0) {
// Trusted-workspace path: explicit allow-list bounds writes.
// Set only by cycle.ts (synthesize/patterns) which submits subagent
// jobs under PROTECTED_JOB_NAMES — MCP cannot reach this branch.
if (!matchesSlugAllowList(slug, allowList)) {
throw new OperationError(
'permission_denied',
`put_page slug '${slug}' is not within the trusted-workspace allow-list (${allowList.join(', ')})`
);
}
} else {
// Legacy default: agent-namespace confinement.
const prefix = `wiki/agents/${ctx.subagentId}/`;
if (!slug.startsWith(prefix) || slug.length === prefix.length) {
throw new OperationError('permission_denied', `put_page via subagent must write under '${prefix}...'`);
}
}
}
if (ctx.dryRun) return { dry_run: true, action: 'put_page', slug: p.slug };
// Skip embedding when no OpenAI key is configured. importFromContent's existing
// try/catch around embed only catches; without a key the OpenAI client would
// attempt 5 retries with exponential backoff (up to ~2 minutes total) before
// giving up. Detect early.
const noEmbed = !process.env.OPENAI_API_KEY;
const result = await importFromContent(ctx.engine, slug, p.content as string, { noEmbed });
// Auto-link post-hook: runs AFTER importFromContent (which is its own
// transaction). Runs even on status='skipped' so reconciliation catches drift
// between the page text and the links table. Failures are non-blocking.
//
// SECURITY: skipped for remote (MCP) callers. Auto-link's bare-slug regex
// matches `people/X` etc. anywhere in page text, including code fences,
// quoted strings, and prompt-injected content. An untrusted page can plant
// arbitrary outbound links by including `see meetings/board-q1` in its body.
// Combined with the backlink boost in hybridSearch, attacker-placed targets
// would surface higher in search. Local CLI users (ctx.remote=false) opt
// into this behavior; MCP/remote writes do not.
let autoLinks:
| { created: number; removed: number; errors: number; unresolved: UnresolvedFrontmatterRef[] }
| { error: string }
| { skipped: 'remote' }
| undefined;
let autoTimeline: { created: number } | { error: string } | { skipped: 'remote' } | undefined;
// Trusted-workspace path (v0.23 dream cycle) re-enables auto-link/timeline
// even though ctx.remote=true, because the allow-list bounds the slug and
// the synthesis prompt is itself the trusted dispatcher. Without this,
// the cycle's `extract` phase would have to recompute every edge, and
// patterns (which runs after extract) would still see the right graph
// but auto_timeline would never fire on synth output.
const trustedWorkspace = ctx.viaSubagent === true
&& Array.isArray(ctx.allowedSlugPrefixes)
&& ctx.allowedSlugPrefixes.length > 0;
if (ctx.remote === true && !trustedWorkspace) {
autoLinks = { skipped: 'remote' };
autoTimeline = { skipped: 'remote' };
} else if (result.parsedPage) {
try {
const enabled = await isAutoLinkEnabled(ctx.engine);
if (enabled) {
autoLinks = await runAutoLink(ctx.engine, slug, result.parsedPage);
}
} catch (e) {
autoLinks = { error: e instanceof Error ? e.message : String(e) };
}
// Timeline extraction mirrors auto-link: runs post-write, best-effort,
// never blocks the write. ON CONFLICT DO NOTHING in
// addTimelineEntriesBatch keeps it idempotent across re-writes, so a
// page that's edited and re-written won't duplicate its own timeline.
try {
const enabled = await isAutoTimelineEnabled(ctx.engine);
if (enabled) {
const fullContent = result.parsedPage.compiled_truth + '\n' + result.parsedPage.timeline;
const entries = parseTimelineEntries(fullContent);
if (entries.length > 0) {
const batch = entries.map(e => ({
slug,
date: e.date,
summary: e.summary,
detail: e.detail || '',
}));
const created = await ctx.engine.addTimelineEntriesBatch(batch);
autoTimeline = { created };
} else {
autoTimeline = { created: 0 };
}
}
} catch (e) {
autoTimeline = { error: e instanceof Error ? e.message : String(e) };
}
}
// Post-write validator lint (PR 2.5): feature-flag-gated, non-blocking.
// When `writer.lint_on_put_page` is enabled, runs the BrainWriter's
// validators on the freshly-written page and logs findings to
// ingest_log + ~/.gbrain/validator-lint.jsonl. Does NOT reject the
// write — that's the deferred strict-mode flip after the 7-day soak.
let writerLint: { error_count: number; warning_count: number } | { skipped: string } | undefined;
try {
const { runPostWriteLint } = await import('./output/post-write.ts');
const lint = await runPostWriteLint(ctx.engine, result.slug);
if (lint.ran) {
writerLint = {
error_count: lint.findings.filter(f => f.severity === 'error').length,
warning_count: lint.findings.filter(f => f.severity === 'warning').length,
};
} else if (lint.skippedReason) {
writerLint = { skipped: lint.skippedReason };
}
} catch {
// Non-fatal; never blocks put_page.
}
return {
slug: result.slug,
status: result.status === 'imported' ? 'created_or_updated' : result.status,
chunks: result.chunks,
...(autoLinks ? { auto_links: autoLinks } : {}),
...(autoTimeline ? { auto_timeline: autoTimeline } : {}),
...(writerLint ? { writer_lint: writerLint } : {}),
};
},
cliHints: { name: 'put', positional: ['slug'], stdin: 'content' },
};
/**
* Extract entity refs from a freshly-written page, sync the links table to match.
* Creates new links via addLink, removes stale ones (links present in DB but no
* longer referenced in content) via removeLink. Returns counts.
*
* Runs OUTSIDE importFromContent's transaction so it doesn't block the page write
* or get rolled back if a single link operation fails. Per-link failures are
* counted; the overall function never throws (catch in put_page handler covers
* extraction errors).
*/
async function runAutoLink(
engine: BrainEngine,
slug: string,
parsed: { type: PageType; compiled_truth: string; timeline: string; frontmatter: Record<string, unknown> },
): Promise<{ created: number; removed: number; errors: number; unresolved: UnresolvedFrontmatterRef[] }> {
const fullContent = parsed.compiled_truth + '\n' + parsed.timeline;
// Live-mode resolver: per-put throwaway cache, pg_trgm + optional search.
const resolver = makeResolver(engine, { mode: 'live' });
const { candidates, unresolved } = await extractPageLinks(
slug, fullContent, parsed.frontmatter, parsed.type, resolver,
);
// Resolve which targets exist (skip refs to non-existent pages to avoid FK
// violation churn in addLink). One getAllSlugs call upfront, O(1) lookup.
const allSlugs = await engine.getAllSlugs();
const valid = candidates.filter(c =>
allSlugs.has(c.targetSlug) && (!c.fromSlug || allSlugs.has(c.fromSlug))
);
// Split candidates by direction. Outgoing (fromSlug === slug or unset) are
// this page's own edges, reconciled against getLinks(slug). Incoming
// (fromSlug !== slug — frontmatter with `direction: incoming`) are edges
// where this page is the TO side; reconciled against getBacklinks(slug)
// but SCOPED to the frontmatter edges this page authored via
// (link_source='frontmatter' AND origin_slug = slug). We never touch
// frontmatter edges authored by OTHER pages.
const out = valid.filter(c => !c.fromSlug || c.fromSlug === slug);
const inc = valid.filter(c => c.fromSlug && c.fromSlug !== slug);
// Run getLinks + addLink/removeLink loops inside a single transaction so that
// concurrent put_page calls on the same slug can't race the reconciliation:
// without this, two simultaneous writes both read stale `existingKeys` and
// re-create links the other side just removed (lost-update).
//
// Row-level locks alone aren't enough: both writers can read the same
// `existingKeys` set BEFORE either mutates a row, so the union-of-writes
// race survives. A transaction-scoped advisory lock keyed on the slug
// hash serializes the entire reconciliation across processes. Falls
// through on engines that don't support pg_advisory_xact_lock (PGLite is
// single-process so there's no cross-process concern there anyway).
const result = await engine.transaction(async (tx) => {
try {
await tx.executeRaw(`SELECT pg_advisory_xact_lock(hashtext($1)::bigint)`, [`auto_link:${slug}`]);
} catch {
// engine doesn't support advisory locks — fall through
}
const existingOut = await tx.getLinks(slug);
// Incoming: we only look at frontmatter edges WE authored (origin_slug=slug).
// Non-frontmatter and other-page frontmatter edges survive untouched.
const existingInRaw = await tx.getBacklinks(slug);
const existingIn = existingInRaw.filter(
l => l.link_source === 'frontmatter' && l.origin_slug === slug,
);
// Reconcilable outgoing edges: markdown + our own frontmatter edges.
// Manual edges (link_source='manual') are NEVER touched by reconciliation.
const reconcilableOut = existingOut.filter(
l => l.link_source === 'markdown' || l.link_source == null ||
(l.link_source === 'frontmatter' && l.origin_slug === slug),
);
const outKeys = new Set(out.map(c =>
`${c.targetSlug}\u0000${c.linkType}\u0000${c.linkSource ?? 'markdown'}`
));
const incKeys = new Set(inc.map(c =>
`${c.fromSlug}\u0000${c.linkType}`
));
let created = 0, removed = 0, errors = 0;
// Add outgoing edges.
for (const c of out) {
try {
await tx.addLink(
slug, c.targetSlug, c.context, c.linkType,
c.linkSource, c.originSlug, c.originField,
);
const existKey = `${c.targetSlug}\u0000${c.linkType}\u0000${c.linkSource ?? 'markdown'}`;
const exists = reconcilableOut.some(l =>
`${l.to_slug}\u0000${l.link_type}\u0000${l.link_source ?? 'markdown'}` === existKey
);
if (!exists) created++;
} catch {
errors++;
}
}
// Add incoming edges (other page → slug).
for (const c of inc) {
try {
await tx.addLink(
c.fromSlug!, c.targetSlug, c.context, c.linkType,
'frontmatter', c.originSlug, c.originField,
);
const existKey = `${c.fromSlug}\u0000${c.linkType}`;
const exists = existingIn.some(l =>
`${l.from_slug}\u0000${l.link_type}` === existKey
);
if (!exists) created++;
} catch {
errors++;
}
}
// Remove stale outgoing (markdown or our-frontmatter, not in desired set).
for (const l of reconcilableOut) {
const key = `${l.to_slug}\u0000${l.link_type}\u0000${l.link_source ?? 'markdown'}`;
if (!outKeys.has(key)) {
try {
await tx.removeLink(slug, l.to_slug, l.link_type, l.link_source ?? undefined);
removed++;
} catch {
errors++;
}
}
}
// Remove stale incoming (our frontmatter → slug, not in desired set).
for (const l of existingIn) {
const key = `${l.from_slug}\u0000${l.link_type}`;
if (!incKeys.has(key)) {
try {
await tx.removeLink(l.from_slug, slug, l.link_type, 'frontmatter');
removed++;
} catch {
errors++;
}
}
}
return { created, removed, errors };
});
return { ...result, unresolved };
}
const delete_page: Operation = {
name: 'delete_page',
description: 'Delete a page',
params: {
slug: { type: 'string', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'delete_page', slug: p.slug };
await ctx.engine.deletePage(p.slug as string);
return { status: 'deleted' };
},
cliHints: { name: 'delete', positional: ['slug'] },
};
const list_pages: Operation = {
name: 'list_pages',
description: 'List pages with optional filters',
params: {
type: { type: 'string', description: 'Filter by page type' },
tag: { type: 'string', description: 'Filter by tag' },
limit: { type: 'number', description: 'Max results (default 50)' },
},
handler: async (ctx, p) => {
const pages = await ctx.engine.listPages({
type: p.type as any,
tag: p.tag as string,
limit: clampSearchLimit(p.limit as number | undefined, 50, 100),
});
return pages.map(pg => ({
slug: pg.slug,
type: pg.type,
title: pg.title,
updated_at: pg.updated_at,
}));
},
scope: 'read',
cliHints: { name: 'list' },
};
// --- Search ---
const search: Operation = {
name: 'search',
description: 'Keyword search using full-text search',
params: {
query: { type: 'string', required: true },
limit: { type: 'number', description: 'Max results (default 20)' },
offset: { type: 'number', description: 'Skip first N results (for pagination)' },
},
handler: async (ctx, p) => {
const startedAt = Date.now();
const queryText = p.query as string;
const raw = await ctx.engine.searchKeyword(queryText, {
limit: (p.limit as number) || 20,
offset: (p.offset as number) || 0,
});
const results = dedupResults(raw);
const latency_ms = Date.now() - startedAt;
// Op-layer capture (v0.25.0). Fire-and-forget — no await on the
// capture call so MCP response latency is unaffected. search has
// no expand/detail/vector semantics so meta fields are fixed.
if (isEvalCaptureEnabled(ctx.config)) {
void captureEvalCandidate(
ctx.engine,
{
tool_name: 'search',
query: queryText,
results,
meta: { vector_enabled: false, detail_resolved: null, expansion_applied: false },
latency_ms,
remote: ctx.remote ?? false,
expand_enabled: null,
detail: null,
job_id: ctx.jobId ?? null,
subagent_id: ctx.subagentId ?? null,
},
{ scrub_pii: isEvalScrubEnabled(ctx.config) },
);
}
return results;
},
scope: 'read',
cliHints: { name: 'search', positional: ['query'] },
};
const query: Operation = {
name: 'query',
description: 'Hybrid search with vector + keyword + multi-query expansion',
params: {
query: { type: 'string', required: true },
limit: { type: 'number', description: 'Max results (default 20)' },
offset: { type: 'number', description: 'Skip first N results (for pagination)' },
expand: { type: 'boolean', description: 'Enable multi-query expansion (default: true)' },
detail: { type: 'string', description: 'Result detail level: low (compiled truth only), medium (default, all with dedup), high (all chunks)' },
// v0.20.0 Cathedral II Layer 10 C1/C2: language + symbol-kind filters.
lang: { type: 'string', description: 'Filter to chunks where content_chunks.language matches (e.g., typescript, python, ruby)' },
symbol_kind: { type: 'string', description: 'Filter to chunks where content_chunks.symbol_type matches (e.g., function, class, method, type, interface)' },
// v0.20.0 Cathedral II Layer 7 (A2) / Layer 10 C3: two-pass structural expansion.
near_symbol: { type: 'string', description: 'Anchor retrieval at this qualified symbol name (e.g., BrainEngine.searchKeyword). Enables A2 two-pass.' },
walk_depth: { type: 'number', description: 'Structural walk depth 1-2. Default 0 (off). Expands anchors through code_edges with 1/(1+hop) decay.' },
},
handler: async (ctx, p) => {
const startedAt = Date.now();
const expand = p.expand !== false;
const detail = (p.detail as 'low' | 'medium' | 'high') || undefined;
const queryText = p.query as string;
// v0.25.0 — capture meta side-channel. hybridSearch's return contract
// stays SearchResult[] (Cathedral II callers depend on that); meta
// arrives via callback so eval capture can record what actually ran.
let capturedMeta: HybridSearchMeta | null = null;
const results = await hybridSearch(ctx.engine, queryText, {
limit: (p.limit as number) || 20,
offset: (p.offset as number) || 0,
expansion: expand,
expandFn: expand ? expandQuery : undefined,
detail,
language: (p.lang as string) || undefined,
symbolKind: (p.symbol_kind as string) || undefined,
nearSymbol: (p.near_symbol as string) || undefined,
walkDepth: typeof p.walk_depth === 'number' ? (p.walk_depth as number) : undefined,
onMeta: (m) => { capturedMeta = m; },
});
const latency_ms = Date.now() - startedAt;
// Op-layer capture (v0.25.0). Fire-and-forget. meta tells gbrain-evals
// what hybridSearch *actually* did so replay can distinguish "with API
// key" from "keyword-only fallback" and "expansion fired" from
// "expansion requested + silently fell back."
if (isEvalCaptureEnabled(ctx.config)) {
const meta: HybridSearchMeta = capturedMeta ?? {
vector_enabled: false, detail_resolved: detail ?? null, expansion_applied: false,
};
void captureEvalCandidate(
ctx.engine,
{
tool_name: 'query',
query: queryText,
results,
meta,
latency_ms,
remote: ctx.remote ?? false,
expand_enabled: expand,
detail: detail ?? null,
job_id: ctx.jobId ?? null,
subagent_id: ctx.subagentId ?? null,
},
{ scrub_pii: isEvalScrubEnabled(ctx.config) },
);
}
return results;
},
scope: 'read',
cliHints: { name: 'query', positional: ['query'] },
};
// --- Tags ---
const add_tag: Operation = {
name: 'add_tag',
description: 'Add tag to page',
params: {
slug: { type: 'string', required: true },
tag: { type: 'string', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'add_tag', slug: p.slug, tag: p.tag };
await ctx.engine.addTag(p.slug as string, p.tag as string);
return { status: 'ok' };
},
cliHints: { name: 'tag', positional: ['slug', 'tag'] },
};
const remove_tag: Operation = {
name: 'remove_tag',
description: 'Remove tag from page',
params: {
slug: { type: 'string', required: true },
tag: { type: 'string', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'remove_tag', slug: p.slug, tag: p.tag };
await ctx.engine.removeTag(p.slug as string, p.tag as string);
return { status: 'ok' };
},
cliHints: { name: 'untag', positional: ['slug', 'tag'] },
};
const get_tags: Operation = {
name: 'get_tags',
description: 'List tags for a page',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getTags(p.slug as string);
},
scope: 'read',
cliHints: { name: 'tags', positional: ['slug'] },
};
// --- Links ---
const add_link: Operation = {
name: 'add_link',
description: 'Create link between pages',
params: {
from: { type: 'string', required: true },
to: { type: 'string', required: true },
link_type: { type: 'string', description: 'Link type (e.g., invested_in, works_at)' },
context: { type: 'string', description: 'Context for the link' },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'add_link', from: p.from, to: p.to };
await ctx.engine.addLink(
p.from as string, p.to as string,
(p.context as string) || '', (p.link_type as string) || '',
);
return { status: 'ok' };
},
cliHints: { name: 'link', positional: ['from', 'to'] },
};
const remove_link: Operation = {
name: 'remove_link',
description: 'Remove link between pages',
params: {
from: { type: 'string', required: true },
to: { type: 'string', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'remove_link', from: p.from, to: p.to };
await ctx.engine.removeLink(p.from as string, p.to as string);
return { status: 'ok' };
},
cliHints: { name: 'unlink', positional: ['from', 'to'] },
};
const get_links: Operation = {
name: 'get_links',
description: 'List outgoing links from a page',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getLinks(p.slug as string);
},
scope: 'read',
};
const get_backlinks: Operation = {
name: 'get_backlinks',
description: 'List incoming links to a page',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getBacklinks(p.slug as string);
},
scope: 'read',
cliHints: { name: 'backlinks', positional: ['slug'] },
};
/**
* Hard cap on traverse_graph depth from MCP callers. Each recursive CTE iteration
* grows a `visited` array per path; in `direction=both` the join is `OR`-based and
* fans out exponentially. Without a cap, a remote MCP caller can pass depth=1e6
* and burn memory/CPU on the database. 10 hops is well beyond any realistic
* relationship query (your OpenClaw's "people who attended meetings with Alice"
* is 2 hops; the deepest meaningful chain in our test data is 4).
*/
const TRAVERSE_DEPTH_CAP = 10;
const traverse_graph: Operation = {
name: 'traverse_graph',
description: 'Traverse link graph from a page. With link_type/direction, returns edges (GraphPath[]) instead of nodes.',
params: {
slug: { type: 'string', required: true },
depth: { type: 'number', description: `Max traversal depth (default 5, capped at ${TRAVERSE_DEPTH_CAP})` },
link_type: { type: 'string', description: 'Filter to one link type (per-edge filter, traversal only follows matching edges)' },
direction: { type: 'string', enum: ['in', 'out', 'both'], description: 'Traversal direction (default out)' },
},
handler: async (ctx, p) => {
const slug = p.slug as string;
const requestedDepth = (p.depth as number) || 5;
if (requestedDepth > TRAVERSE_DEPTH_CAP) {
ctx.logger.warn(`[gbrain] traverse_graph depth clamped from ${requestedDepth} to ${TRAVERSE_DEPTH_CAP}`);
}
const depth = Math.max(1, Math.min(requestedDepth, TRAVERSE_DEPTH_CAP));
const linkType = p.link_type as string | undefined;
const direction = p.direction as 'in' | 'out' | 'both' | undefined;
// Backward compat: when neither link_type nor direction is provided, return
// the legacy GraphNode[] shape. Once either is set, switch to GraphPath[].
if (linkType === undefined && direction === undefined) {
return ctx.engine.traverseGraph(slug, depth);
}
return ctx.engine.traversePaths(slug, { depth, linkType, direction });
},
scope: 'read',
cliHints: { name: 'graph', positional: ['slug'] },
};
// --- Timeline ---
const add_timeline_entry: Operation = {
name: 'add_timeline_entry',
description: 'Add timeline entry to a page',
params: {
slug: { type: 'string', required: true },
date: { type: 'string', required: true },
summary: { type: 'string', required: true },
detail: { type: 'string' },
source: { type: 'string' },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'add_timeline_entry', slug: p.slug };
const date = p.date as string;
// Reject anything that isn't a strict YYYY-MM-DD with year 1900-2199 and
// a real calendar day. PG DATE accepts year 5874897 silently — that's a
// semantic bug nobody actually wants.
if (!/^\d{4}-\d{2}-\d{2}$/.test(date)) {
throw new Error(`Invalid date format "${date}" (expected YYYY-MM-DD)`);
}
const [y, m, d] = date.split('-').map(Number);
if (y < 1900 || y > 2199 || m < 1 || m > 12 || d < 1 || d > 31) {
throw new Error(`Invalid date "${date}" (year 1900-2199, month 1-12, day 1-31)`);
}
// Round-trip through Date to catch e.g. Feb 30.
const parsed = new Date(date);
if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== date) {
throw new Error(`Invalid calendar date "${date}"`);
}
await ctx.engine.addTimelineEntry(p.slug as string, {
date,
source: (p.source as string) || '',
summary: p.summary as string,
detail: (p.detail as string) || '',
});
return { status: 'ok' };
},
cliHints: { name: 'timeline-add', positional: ['slug', 'date', 'summary'] },
};
const get_timeline: Operation = {
name: 'get_timeline',
description: 'Get timeline entries for a page',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getTimeline(p.slug as string);
},
scope: 'read',
cliHints: { name: 'timeline', positional: ['slug'] },
};
// --- Admin ---
const get_stats: Operation = {
name: 'get_stats',
description: 'Brain statistics (page count, chunk count, etc.)',
params: {},
handler: async (ctx) => {
return ctx.engine.getStats();
},
scope: 'admin',
cliHints: { name: 'stats' },
};
const get_health: Operation = {
name: 'get_health',
description: 'Brain health dashboard (embed coverage, stale pages, orphans)',
params: {},
handler: async (ctx) => {
return ctx.engine.getHealth();
},
scope: 'admin',
cliHints: { name: 'health' },
};
const get_versions: Operation = {
name: 'get_versions',
description: 'Page version history',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getVersions(p.slug as string);
},
scope: 'read',
cliHints: { name: 'history', positional: ['slug'] },
};
const revert_version: Operation = {
name: 'revert_version',
description: 'Revert page to a previous version',
params: {
slug: { type: 'string', required: true },
version_id: { type: 'number', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'revert_version', slug: p.slug, version_id: p.version_id };
await ctx.engine.createVersion(p.slug as string);
await ctx.engine.revertToVersion(p.slug as string, p.version_id as number);
return { status: 'reverted' };
},
cliHints: { name: 'revert', positional: ['slug', 'version_id'] },
};
// --- Sync ---
const sync_brain: Operation = {
name: 'sync_brain',
description: 'Sync git repo to brain (incremental)',
params: {
repo: { type: 'string', description: 'Path to git repo (optional if configured)' },
dry_run: { type: 'boolean', description: 'Preview changes without applying' },
full: { type: 'boolean', description: 'Full re-sync (ignore checkpoint)' },
no_pull: { type: 'boolean', description: 'Skip git pull' },
no_embed: { type: 'boolean', description: 'Skip embedding generation' },
},
mutating: true,
scope: 'admin',
localOnly: true,
handler: async (ctx, p) => {
const { performSync } = await import('../commands/sync.ts');
return performSync(ctx.engine, {
repoPath: p.repo as string | undefined,
dryRun: ctx.dryRun || (p.dry_run as boolean) || false,
noEmbed: (p.no_embed as boolean) || false,
noPull: (p.no_pull as boolean) || false,
full: (p.full as boolean) || false,
});
},
cliHints: { name: 'sync', hidden: true },
};
// --- Raw Data ---
const put_raw_data: Operation = {
name: 'put_raw_data',
description: 'Store raw API response data for a page',
params: {
slug: { type: 'string', required: true },
source: { type: 'string', required: true, description: 'Data source (e.g., crustdata, happenstance)' },
data: { type: 'object', required: true, description: 'Raw data object' },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'put_raw_data', slug: p.slug, source: p.source };
await ctx.engine.putRawData(p.slug as string, p.source as string, p.data as object);
return { status: 'ok' };
},
};
const get_raw_data: Operation = {
name: 'get_raw_data',
description: 'Retrieve raw data for a page',
params: {
slug: { type: 'string', required: true },
source: { type: 'string', description: 'Filter by source' },
},
handler: async (ctx, p) => {
return ctx.engine.getRawData(p.slug as string, p.source as string | undefined);
},
scope: 'read',
};
// --- Resolution & Chunks ---
const resolve_slugs: Operation = {
name: 'resolve_slugs',
description: 'Fuzzy-resolve a partial slug to matching page slugs',
params: {
partial: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.resolveSlugs(p.partial as string);
},
scope: 'read',
};
const get_chunks: Operation = {
name: 'get_chunks',
description: 'Get content chunks for a page',
params: {
slug: { type: 'string', required: true },
},
handler: async (ctx, p) => {
return ctx.engine.getChunks(p.slug as string);
},
scope: 'read',
};
// --- Ingest Log ---
const log_ingest: Operation = {
name: 'log_ingest',
description: 'Log an ingestion event',
params: {
source_type: { type: 'string', required: true },
source_ref: { type: 'string', required: true },
pages_updated: { type: 'array', required: true, items: { type: 'string' } },
summary: { type: 'string', required: true },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'log_ingest' };
await ctx.engine.logIngest({
source_type: p.source_type as string,
source_ref: p.source_ref as string,
pages_updated: p.pages_updated as string[],
summary: p.summary as string,
});
return { status: 'ok' };
},
};
const get_ingest_log: Operation = {
name: 'get_ingest_log',
description: 'Get recent ingestion log entries',
params: {
limit: { type: 'number', description: 'Max entries (default 20)' },
},
handler: async (ctx, p) => {
return ctx.engine.getIngestLog({ limit: clampSearchLimit(p.limit as number | undefined, 20, 50) });
},
scope: 'read',
};
// --- File Operations ---
// Both branches need a LIMIT. Without one, the slug-filtered branch materializes
// every file for that slug — an MCP caller can force unbounded memory consumption
// by targeting a page with many attachments.
const FILE_LIST_LIMIT = 100;
const file_list: Operation = {
name: 'file_list',
description: 'List stored files',
params: {
slug: { type: 'string', description: 'Filter by page slug' },
},
scope: 'admin',
localOnly: true,
handler: async (_ctx, p) => {
const sql = db.getConnection();
const slug = p.slug as string | undefined;
if (slug) {
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files WHERE page_slug = ${slug} ORDER BY filename LIMIT ${FILE_LIST_LIMIT}`;
}
return sql`SELECT id, page_slug, filename, storage_path, mime_type, size_bytes, content_hash, created_at FROM files ORDER BY page_slug, filename LIMIT ${FILE_LIST_LIMIT}`;
},
};
const file_upload: Operation = {
name: 'file_upload',
description: 'Upload a file to storage',
params: {
path: { type: 'string', required: true, description: 'Local file path' },
page_slug: { type: 'string', description: 'Associate with page' },
},
mutating: true,
scope: 'admin',
localOnly: true,
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'file_upload', path: p.path };
const { readFileSync, statSync } = await import('fs');
const { basename, extname } = await import('path');
const { createHash } = await import('crypto');
const filePath = p.path as string;
const pageSlug = (p.page_slug as string) || null;
// Fix 1 / B5 / H5 / M4: validate path, slug, filename before any filesystem read.
// Remote callers (MCP, agent) are confined to cwd (strict). Local CLI callers
// can upload from anywhere on the filesystem (loose) — the user owns the machine.
// Default is strict when ctx.remote is undefined (defense-in-depth).
const strict = ctx.remote !== false;
validateUploadPath(filePath, process.cwd(), strict);
if (pageSlug) validatePageSlug(pageSlug);
const filename = basename(filePath);
validateFilename(filename);
const stat = statSync(filePath);
const content = readFileSync(filePath);
const hash = createHash('sha256').update(content).digest('hex');
const storagePath = pageSlug ? `${pageSlug}/${filename}` : `unsorted/${hash.slice(0, 8)}-${filename}`;
const MIME_TYPES: Record<string, string> = {
'.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.png': 'image/png',
'.gif': 'image/gif', '.webp': 'image/webp', '.svg': 'image/svg+xml',
'.pdf': 'application/pdf', '.mp4': 'video/mp4', '.mp3': 'audio/mpeg',
};
const mimeType = MIME_TYPES[extname(filePath).toLowerCase()] || null;
const sql = db.getConnection();
const existing = await sql`SELECT id FROM files WHERE content_hash = ${hash} AND storage_path = ${storagePath}`;
if (existing.length > 0) {
return { status: 'already_exists', storage_path: storagePath };
}
// Upload to storage backend if configured
if (ctx.config.storage) {
const { createStorage } = await import('./storage.ts');
const storage = await createStorage(ctx.config.storage as any);
try {
await storage.upload(storagePath, content, mimeType || undefined);
} catch (uploadErr) {
throw new OperationError('storage_error', `Upload failed: ${uploadErr instanceof Error ? uploadErr.message : String(uploadErr)}`);
}
}
try {
await sql`
INSERT INTO files (page_slug, filename, storage_path, mime_type, size_bytes, content_hash, metadata)
VALUES (${pageSlug}, ${filename}, ${storagePath}, ${mimeType}, ${stat.size}, ${hash}, ${'{}'}::jsonb)
ON CONFLICT (storage_path) DO UPDATE SET
content_hash = EXCLUDED.content_hash,
size_bytes = EXCLUDED.size_bytes,
mime_type = EXCLUDED.mime_type
`;
} catch (dbErr) {
// Rollback: clean up storage if DB write failed
if (ctx.config.storage) {
try {
const { createStorage } = await import('./storage.ts');
const storage = await createStorage(ctx.config.storage as any);
await storage.delete(storagePath);
} catch { /* best effort cleanup */ }
}
throw dbErr;
}
return { status: 'uploaded', storage_path: storagePath, size_bytes: stat.size };
},
};
const file_url: Operation = {
name: 'file_url',
description: 'Get a URL for a stored file',
params: {
storage_path: { type: 'string', required: true },
},
scope: 'admin',
localOnly: true,
handler: async (_ctx, p) => {
const sql = db.getConnection();
const rows = await sql`SELECT storage_path, mime_type, size_bytes FROM files WHERE storage_path = ${p.storage_path as string}`;
if (rows.length === 0) {
throw new OperationError('storage_error', `File not found: ${p.storage_path}`);
}
// TODO: generate signed URL from Supabase Storage
return { storage_path: rows[0].storage_path, url: `gbrain:files/${rows[0].storage_path}` };
},
};
// --- Jobs (Minions) ---
const submit_job: Operation = {
name: 'submit_job',
description: 'Submit a background job to the Minions queue. Built-in types: sync, embed, lint, import, extract, backlinks, autopilot-cycle. The `shell` type is CLI-only and rejected over MCP.',
params: {
name: { type: 'string', required: true, description: 'Job type (sync, embed, lint, import, extract, backlinks, autopilot-cycle; shell is CLI-only)' },
data: { type: 'object', description: 'Job payload (JSON)' },
queue: { type: 'string', description: 'Queue name (default: "default")' },
priority: { type: 'number', description: 'Priority (0 = highest, default: 0)' },
max_attempts: { type: 'number', description: 'Max retry attempts (default: 3)' },
delay: { type: 'number', description: 'Delay in ms before eligible' },
timeout_ms: { type: 'number', description: 'Per-job wall-clock timeout in ms; aborted job goes to dead' },
},
mutating: true,
scope: 'admin',
handler: async (ctx, p) => {
const name = typeof p.name === 'string' ? p.name.trim() : '';
if (ctx.dryRun) return { dry_run: true, action: 'submit_job', name };
// Submit-side MCP guard: reject protected job names from untrusted callers
// BEFORE we touch the DB. This is the first of the two security layers
// (the second is MinionQueue.add's check). Independent of the worker-side
// GBRAIN_ALLOW_SHELL_JOBS env flag — even if that flag is on, MCP callers
// cannot submit protected-type jobs.
const { isProtectedJobName } = await import('./minions/protected-names.ts');
if (ctx.remote && isProtectedJobName(name)) {
throw new OperationError('permission_denied', `'${name}' jobs cannot be submitted over MCP (CLI-only for security)`);
}
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
// Trusted flag set only when this is a local (non-remote) submission. When
// remote=true, the guard above has already thrown for protected names, so
// passing undefined here is safe for any non-protected name that slips by.
const trusted = !ctx.remote && isProtectedJobName(name) ? { allowProtectedSubmit: true } : undefined;
return queue.add(name, (p.data as Record<string, unknown>) || {}, {
queue: (p.queue as string) || 'default',
priority: (p.priority as number) || 0,
max_attempts: (p.max_attempts as number) || 3,
delay: (p.delay as number) || undefined,
timeout_ms: (p.timeout_ms as number) || undefined,
}, trusted);
},
};
const get_job: Operation = {
name: 'get_job',
description: 'Get job status and details by ID',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
scope: 'admin',
handler: async (ctx, p) => {
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const job = await queue.getJob(p.id as number);
if (!job) throw new OperationError('invalid_params', `Job not found: ${p.id}`);
return job;
},
};
const list_jobs: Operation = {
name: 'list_jobs',
description: 'List jobs with optional filters',
params: {
status: { type: 'string', description: 'Filter by status (waiting, active, completed, failed, delayed, dead, cancelled)' },
queue: { type: 'string', description: 'Filter by queue name' },
name: { type: 'string', description: 'Filter by job type' },
limit: { type: 'number', description: 'Max results (default: 50)' },
},
scope: 'admin',
handler: async (ctx, p) => {
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
return queue.getJobs({
status: p.status as string | undefined,
queue: p.queue as string | undefined,
name: p.name as string | undefined,
limit: (p.limit as number) || 50,
} as Parameters<typeof queue.getJobs>[0]);
},
};
const cancel_job: Operation = {
name: 'cancel_job',
description: 'Cancel a waiting, active, or delayed job',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
mutating: true,
scope: 'admin',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'cancel_job', id: p.id };
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const cancelled = await queue.cancelJob(p.id as number);
if (!cancelled) throw new OperationError('invalid_params', `Cannot cancel job ${p.id} (may already be in terminal status)`);
return cancelled;
},
};
const retry_job: Operation = {
name: 'retry_job',
description: 'Re-queue a failed or dead job for retry',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
mutating: true,
scope: 'admin',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'retry_job', id: p.id };
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const retried = await queue.retryJob(p.id as number);
if (!retried) throw new OperationError('invalid_params', `Cannot retry job ${p.id} (must be failed or dead)`);
return retried;
},
};
const get_job_progress: Operation = {
name: 'get_job_progress',
description: 'Get structured progress for a running job',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
scope: 'admin',
handler: async (ctx, p) => {
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const job = await queue.getJob(p.id as number);
if (!job) throw new OperationError('invalid_params', `Job not found: ${p.id}`);
return { id: job.id, name: job.name, status: job.status, progress: job.progress };
},
};
const pause_job: Operation = {
name: 'pause_job',
description: 'Pause a waiting, active, or delayed job',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
scope: 'admin',
handler: async (ctx, p) => {
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const job = await queue.pauseJob(p.id as number);
if (!job) throw new OperationError('invalid_params', `Job not found or not pausable: ${p.id}`);
return { id: job.id, status: job.status };
},
};
const resume_job: Operation = {
name: 'resume_job',
description: 'Resume a paused job back to waiting',
params: {
id: { type: 'number', required: true, description: 'Job ID' },
},
scope: 'admin',
handler: async (ctx, p) => {
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const job = await queue.resumeJob(p.id as number);
if (!job) throw new OperationError('invalid_params', `Job not found or not paused: ${p.id}`);
return { id: job.id, status: job.status };
},
};
const replay_job: Operation = {
name: 'replay_job',
description: 'Replay a completed/failed/dead job, optionally with modified data',
params: {
id: { type: 'number', required: true, description: 'Source job ID to replay' },
data_overrides: { type: 'object', required: false, description: 'Data fields to override (merged with original)' },
},
scope: 'admin',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'replay_job', id: p.id };
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const job = await queue.replayJob(p.id as number, p.data_overrides as Record<string, unknown> | undefined);
if (!job) throw new OperationError('invalid_params', `Job not found or not in terminal state: ${p.id}`);
return { id: job.id, name: job.name, status: job.status, source_id: p.id };
},
};
const send_job_message: Operation = {
name: 'send_job_message',
description: 'Send a sidechannel message to a running job\'s inbox',
params: {
id: { type: 'number', required: true, description: 'Job ID to message' },
payload: { type: 'object', required: true, description: 'Message payload (arbitrary JSON)' },
sender: { type: 'string', required: false, description: 'Sender identity (default: admin)' },
},
scope: 'admin',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'send_job_message', id: p.id };
const { MinionQueue } = await import('./minions/queue.ts');
const queue = new MinionQueue(ctx.engine);
const msg = await queue.sendMessage(p.id as number, p.payload, (p.sender as string) ?? 'admin');
if (!msg) throw new OperationError('invalid_params', `Job not found, not messageable, or sender unauthorized: ${p.id}`);
return { sent: true, message_id: msg.id, job_id: p.id };
},
};
// --- Orphans ---
const find_orphans: Operation = {
name: 'find_orphans',
description: 'Find pages with no inbound wikilinks. Essential for content enrichment cycles.',
params: {
include_pseudo: {
type: 'boolean',
description: 'Include auto-generated and pseudo pages (default: false)',
},
},
scope: 'read',
handler: async (ctx, p) => {
const { findOrphans } = await import('../commands/orphans.ts');
return findOrphans(ctx.engine, { includePseudo: (p.include_pseudo as boolean) || false });
},
cliHints: { name: 'orphans', hidden: true },
};
// --- Exports ---
export const operations: Operation[] = [
// Page CRUD
get_page, put_page, delete_page, list_pages,
// Search
search, query,
// Tags
add_tag, remove_tag, get_tags,
// Links
add_link, remove_link, get_links, get_backlinks, traverse_graph,
// Timeline
add_timeline_entry, get_timeline,
// Admin
get_stats, get_health, get_versions, revert_version,
// Sync
sync_brain,
// Raw data
put_raw_data, get_raw_data,
// Resolution & chunks
resolve_slugs, get_chunks,
// Ingest log
log_ingest, get_ingest_log,
// Files
file_list, file_upload, file_url,
// Jobs (Minions)
submit_job, get_job, list_jobs, cancel_job, retry_job, get_job_progress,
pause_job, resume_job, replay_job, send_job_message,
// Orphans
find_orphans,
];
export const operationsByName = Object.fromEntries(
operations.map(op => [op.name, op]),
) as Record<string, Operation>;