Files
gbrain/test/e2e/serve-http-oauth.test.ts
T
89ae720959 v0.31.0 feat: hot memory — facts hook + recall CLI + MCP _meta + consolidate phase (#785)
* v0.31 feat(migrate): facts hot memory schema (migration v40)

Phase 1 of v0.31 hot-memory.

- New facts table with source_id (TEXT FK to sources, per-source isolation),
  kind CHECK (event/preference/commitment/belief/fact), visibility CHECK
  (private/world for takes-style ACL parity), valid_from/valid_until/
  expired_at/superseded_by for temporal + supersession audit, and
  consolidated_at/consolidated_into pointing at takes(id) for the dream-
  cycle hot→cold bridge.
- Embedding column dim resolved at migration time from
  config.embedding_dimensions so non-OpenAI brains (Voyage etc) work
  out-of-the-box. HALFVEC where pgvector >= 0.7; falls back to VECTOR
  with stderr warn on older versions. Matching opclass per column type
  (halfvec_cosine_ops vs vector_cosine_ops).
- 5 partial indexes leading on source_id so every read uses the trust
  boundary as part of the index, not a callback. HNSW partial index
  excludes expired/null rows so footprint stays proportional to active
  fact count.
- RLS DO-block matches takes pattern (Postgres BYPASSRLS gate; PGLite
  no-op).
- v0_31_0.ts orchestrator follows v0_28_0.ts pattern — phase A asserts
  schema version >= 40 + facts table presence; runner owns ledger.

All 87 existing migrate.test.ts cases pass. PGLite smoke test confirms
table + indexes + CHECK constraints + ON DELETE CASCADE all behave.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 chore(version): bump VERSION + package.json to 0.31.0

Phase 1 closer. CHANGELOG entry written when Phase 7 lands.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 feat(engine): facts hot memory engine API (Phase 2)

Phase 2 of v0.31 hot-memory.

Adds 8 facts methods to BrainEngine implemented on both PGLite and
Postgres engines:

- insertFact(input, ctx) — INSERT with optional supersedeId; expires the
  named row in the same transaction. Per-entity advisory lock on Postgres
  (`pg_advisory_xact_lock(hashtextextended(source_id::text || ':' ||
  entity_slug, 0))`) for the dedup window. PGLite is single-process so
  the lock is a no-op.
- expireFact(id, opts) — sets expired_at + optional superseded_by.
  Idempotent-as-false (already-expired returns false).
- listFactsByEntity / listFactsSince / listFactsBySession — list surfaces
  with FactListOpts filters (activeOnly, kinds, visibility, limit/offset).
  Every query starts WHERE source_id = $X so the trust boundary is part
  of the index path.
- listSupersessions — audit log; activeOnly:false + expired_at IS NOT NULL
  + superseded_by IS NOT NULL.
- findCandidateDuplicates(source_id, entity_slug, factText, k) —
  entity-prefiltered (mandatory), k=5 default, hard cap 20. Embedding-
  cosine ordering when caller supplies an embedding, recency fallback
  otherwise. Bounds the contradiction-classifier blast radius.
- consolidateFact(id, takeId) — sets consolidated_at + consolidated_into.
  Never DELETE; facts stay as audit trail for the resulting take.
- getFactsHealth(source_id) — per-source counters consumed by `gbrain
  doctor` facts_health check.

Public types in engine.ts: FactKind (5-value union), FactVisibility,
FactInsertStatus, FactRow, NewFact, FactListOpts, FactsHealth.

PGLite + Postgres helpers: rowToFact / rowToFactPg parse the
text-format pgvector embedding back into Float32Array; toPgVectorLiteral
encodes for the supersede-path INSERT (postgres-js can't bind Float32Array
directly to a vector column without an explicit literal cast).

Smoke test confirms every method end-to-end on PGLite. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 feat(facts): extraction code path (Phase 3)

Phase 3 of v0.31 hot-memory.

Five new modules under src/core/facts/ + src/core/entities/:

- src/core/facts/decay.ts — pure helper. effectiveConfidence(fact, now)
  applies confidence × exp(-age/halflife) with per-kind halflife table
  (event 7d, commitment 90d, preference 90d, belief 365d, fact 365d).
  Returns 0 for expired or past-valid_until rows. Single source of truth
  consumed by recall, supersession audit, facts_health, and the MCP _meta
  injector (eD8 DRY).

- src/core/facts/queue.ts — bounded in-memory queue. Cap 100 default,
  drop-oldest on overflow with counter. Per-session in-flight=1 serializes
  burst chat. AbortSignal threading from server SIGTERM (mirrors minion
  worker pattern per eD7): 5s grace for in-flight, then drop pending with
  counter. getFactsQueue() process-singleton; __resetFactsQueueForTests
  for hermetic tests.

- src/core/facts/classify.ts — contradiction classifier with cosine
  fast-path (D13: ≥0.95 → duplicate, skip LLM) and classifier-failure
  fallback (D12: cosine ≥0.92 → duplicate, else INSERT). Pure cosine
  helper exported. JSON-strict output with 4-strategy parse fallback;
  refusal stop-reason maps to fallback path. Caller-provided abort
  signal propagated to the gateway chat call.

- src/core/facts/extract.ts — Haiku turn-extractor. Reuses
  INJECTION_PATTERNS from src/core/think/sanitize.ts on the way IN
  (turn_text) AND on the way OUT (each fact). Tight system prompt with
  5-kind taxonomy, 0..1 confidence scoring, entity slug or display name.
  Anti-loop check on isDreamGenerated (reuses v0.23.2 marker semantics).
  Synchronous embedOne() per fact via the gateway so classifier paths
  have embeddings available; AbortError re-thrown explicitly so SIGTERM
  during embed never writes a NULL-embedding row meant to be cancelled
  (eE8 distinction).

- src/core/entities/resolve.ts — slug canonicalization shared by
  signal-detector AND facts. Resolution order: exact slug match →
  pg_trgm fuzzy match (similarity ≥0.4) → deterministic slugify
  fallback. slugify exported standalone for tests + callers that want
  the floor.

Smoke tests confirm decay table, cosine math, slugify rules, queue
drop-oldest under overflow, and shutdown grace + drop-pending semantics.
Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 feat(mcp+cli): MCP ops + recall CLI + _meta + transport refactor (Phase 4)

Phase 4 of v0.31 hot-memory.

Three new MCP ops on the contract-first surface:

- `extract_facts` (write scope, localOnly:false): extracts facts from a
  conversation turn via the Haiku extractor, runs the cosine fast-path
  dedup, INSERTs into per-source hot memory. Returns counts +
  fact_ids[]. Skips on is_dream_generated:true (anti-loop).
- `recall` (read scope): query the per-source hot memory by
  entity / since / session / supersessions / grep filter. Visibility-
  aware: remote callers see visibility='world' rows only (takes-style
  ACL parity, eD21). Returns most-recent first; pagination via limit.
- `forget_fact` (write scope): expireFact wrapper. Idempotent-as-error
  on unknown id; uses the new 'fact_not_found' ErrorCode.

ErrorCode union opened (eD6 / eE7): TS forward-compat via the
`(string & {})` autocomplete-friendly hack so downstream consumers
(gbrain-evals etc) don't break their typecheck on every new code.
Three new codes: 'rate_limited', 'extraction_failed', 'fact_not_found'.

OperationContext gains source_id?:string (eD4 / eE2 — TEXT not INTEGER
per schema reality). Resolved once in buildOperationContext from
DispatchOpts.sourceId. Stdio MCP defaults to GBRAIN_SOURCE env or
'default'; HTTP MCP reads it from the per-token sources scope (eE3).

ToolResult gains _meta?: Record<string, unknown> (eD3). Dispatcher
calls a configurable metaHook AFTER op.handler succeeds, wrapped in
its own try/catch so a DB blip degrades to no-_meta rather than
flipping the whole tool call to error (eE4).

New module src/core/facts/meta-hook.ts:
- getBrainHotMemoryMeta(name, ctx) builds the _meta.brain_hot_memory
  payload. Cache key (source_id, session_id, hash(takesHoldersAllowList
  sorted)) (eD10 / eE5). 30s TTL per session. Visibility filter applies:
  remote → world only; local → all. Top-K=10 ranked by effective
  confidence (decay). Skips injection on recall/extract_facts/forget_fact
  themselves. bumpHotMemoryCache() invalidates per (source_id,
  session_id) on extraction event.

D12 (eE1) accepted: serve-http.ts:801 inlined dispatch path REFACTORED
to call dispatchToolCall. HTTP MCP now inherits source_id, _meta
injection, error envelope unification, and OperationContext shape from
the same code path stdio uses. Scope check + mcp_request_log + SSE
broadcast stay in serve-http.ts (HTTP-specific concerns); the dispatcher
returns ToolResult and the HTTP handler reads isError + content + _meta
to fan into the audit + broadcast paths.

put_page compliance backstop (D23): when a conversation-shape page is
written (note/meeting/slack/email/calendar-event/source/writing) with
a substantive body (>=80 chars) on a non-subagent slug AND no
dream_generated:true marker, fire-and-forget enqueue an extraction job
into the bounded queue. Never blocks the put_page response. Skipped
reasons (no_parsed_page / subagent_namespace / dream_generated /
kind:* / too_short / queue_shutdown / backstop_error) are stable
strings consumed by tests.

`gbrain recall` + `gbrain forget` CLI commands (src/commands/recall.ts):
- recall <entity> | --since DUR | --session ID | --today (markdown
  with kind icons 📅🎯🤝💭📌) | --grep TEXT | --supersessions |
  --include-expired | --as-context (prompt-injection-ready) | --json
- forget <fact-id> shorthand for expireFact

Wired into src/cli.ts dispatch table next to takes / think.

Smoke tests confirm: dispatch surfaces (extract_facts → ops →
listFactsByEntity), forget_fact + idempotent re-call, _meta visibility
filter (remote sees world only, local sees all), CLI markdown render
with kind icons + age strings + decayed confidence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 feat(cycle): consolidate phase — facts → takes promotion (Phase 5)

Phase 5 of v0.31 hot-memory.

New 10th cycle phase `consolidate` between `patterns` and `embed`:

- src/core/cycle.ts:
  * CyclePhase union extended with 'consolidate'
  * ALL_PHASES gets 'consolidate' between patterns and embed (graph-fresh
    after patterns; embed runs after so the new takes get embedded
    same-cycle)
  * NEEDS_LOCK_PHASES gets 'consolidate' (writes takes + UPDATEs facts)
  * CycleReport.totals gains facts_consolidated + consolidate_takes_written
  * runCycle dispatches the new phase via dynamic import

- src/core/cycle/phases/consolidate.ts (new):
  * Scans (source_id, entity_slug) buckets where COUNT(unconsolidated
    facts) >= 3 (uses idx_facts_unconsolidated partial index)
  * Skips buckets where the OLDEST fact is < 24h old (gives signal time
    to settle before locking it into cold memory)
  * Greedy cosine clustering at threshold 0.85; head-element centroid
    keeps it deterministic + cheap. Singletons (no embedding) stay
    unconsolidated this cycle.
  * For each cluster size >= 2: picks the highest-confidence fact's text
    as the take claim (v0.31 deterministic; v0.32 swaps to Sonnet
    synthesis pass). avg confidence → take weight, earliest valid_from →
    take since_date, concatenated source_sessions → take.source.
  * Resolves entity_slug → page_id via pages.slug (per source). Skips
    cluster if page is missing in this source — no auto-page-creation
    in v0.31.
  * INSERT into takes(kind='fact', holder='self') with row_num =
    MAX(existing) + 1.
  * UPDATE contributing facts: consolidated_at = now() +
    consolidated_into = takes.id. NEVER DELETE — facts are the audit
    trail for the resulting take.
  * dryRun honored: pretends the writes happened; counters still tick
    so operators can preview load before the first real run.
  * yieldDuringPhase keepalive between buckets so the Minions worker
    job lock + cycle-lock TTL don't drift on long runs.

Smoke test on PGLite confirms: 4 unconsolidated facts → clustered
(cosine 1.0 since same vector) → 1 take row created → all 4 facts
marked consolidated_into. runCycle({phases:['consolidate']}) wires
through to the report totals. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 test: 18 facts test files (Phase 6)

Phase 6 of v0.31 hot-memory: comprehensive coverage across the new
substrate. 110 unit tests pass; 5 E2E test files added (skip gracefully
without DATABASE_URL).

Unit tests (PGLite in-memory, no DATABASE_URL):
- test/facts-decay.test.ts (12 cases) — HALFLIFE_DAYS pinned per kind,
  effectiveConfidence math: age=0 / age=halflife (~1/e) / age=2×halflife
  (~1/e²) / expired returns 0 / valid_until past returns 0 /
  preference-vs-event slower decay / belief-vs-commitment crossover.
- test/facts-queue.test.ts (10 cases) — FIFO within session, drop-oldest
  on overflow, per-session in-flight=1 serializes, different sessions
  parallelize, failed jobs counter, shutdown grace + drop_pending +
  external AbortController triggers shutdown.
- test/facts-classify.test.ts (8 cases) — cosineSimilarity edge cases,
  empty candidates → independent, cheap fast-path ≥0.95 → duplicate
  no LLM, threshold-configurable cosine_fallback path.
- test/facts-engine.test.ts (13 cases) — every BrainEngine fact method
  end-to-end: insertFact (insert/supersede), expireFact idempotency,
  list*, findCandidateDuplicates entity-prefiltered + k cap + cosine
  ordering, consolidateFact never DELETE, getFactsHealth shape +
  total_today ⊆ total_week.
- test/facts-multi-tenant.test.ts (6 cases) — cross-source isolation
  on every list method + CASCADE delete on sources.
- test/facts-visibility.test.ts (6 cases) — visibility column private/
  world; remote=true filters to world-only via dispatchToolCall;
  remote=false sees all.
- test/facts-canonicality.test.ts (10 cases) — slugify rules including
  NFKD diacritic strip ("Crème Brûlée" → "creme-brulee"), exact slug
  match, fallback to slugify when no fuzzy match.
- test/facts-extract.test.ts (4 cases) — empty turn returns [], dream-
  generated short-circuit, graceful no-API-key return.
- test/facts-backstop-gating.test.ts (5 cases) — put_page backstop:
  too_short, subagent_namespace, dream_generated, eligible note path,
  non-eligible kind:guide.
- test/facts-anti-loop.test.ts (4 cases) — extractor + put_page both
  respect dream_generated:true marker.
- test/facts-doctor-shape.test.ts (4 cases) — facts_health JSON shape
  pinned for downstream consumers.
- test/facts-mcp-allowlist.serial.test.ts (5 cases) — extract_facts
  write-scope, recall read-scope, forget_fact write-scope, forget_fact
  fact_not_found error code, extract_facts no-API-key zero counts.
- test/facts-context-injection.serial.test.ts (6 cases) — _meta
  injection on success, world-only filter under remote=true, anti-loop
  on facts ops themselves, best-effort degrade on hook error,
  cache-key includes allow-list hash.
- test/facts-separation-pglite.test.ts (2 cases) — Garry's Separation
  Test as primary ship gate, plus expired hidden-by-default contract.
- test/facts-recall-render.test.ts (3 cases) — --today markdown render
  with all 5 kind icons, --json shape with effective_confidence,
  --as-context emits comment-wrapped block.
- test/facts-migration-dim.test.ts (4 cases) — embedding column type
  is HALFVEC/VECTOR (not arbitrary), dim matches gateway-configured
  embedding_dimensions, HNSW opclass agrees with column type, idempotent
  re-init.
- test/cycle-consolidate.test.ts (5 cases) — below-count + below-age
  thresholds skip, happy path 4 facts → 1 take + all consolidated never
  DELETE, dryRun honored, missing page → bucket skipped.

E2E tests (skip gracefully on DATABASE_URL unset; required gates by
CLAUDE.md test policy):
- test/e2e/facts-separation-postgres.test.ts — Postgres parity for the
  ship gate.
- test/e2e/facts-cross-source-isolation.test.ts — cross-source ACL on PG
  + CASCADE delete.
- test/e2e/facts-forget.test.ts — full forget_fact MCP roundtrip.
- test/e2e/facts-context-injection-postgres.test.ts — _meta injection
  end-to-end on PG.
- test/e2e/facts-recall-render.test.ts — recall --today markdown on PG.
- test/e2e/serve-http-meta.test.ts — eE1 regression: HTTP MCP transport
  inherits _meta + sourceId + scope correctness via dispatchToolCall.

Side-effect: src/core/entities/resolve.ts NFKD post-decompose strips
combining marks (U+0300..U+036F) before hyphenating non-alphanumerics,
so "Crème" → "creme", not "cre-me-".

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 feat(operational): kill switch + doctor check + CHANGELOG + README (Phase 7)

Phase 7 of v0.31 hot-memory.

- src/core/facts/extract.ts: new isFactsExtractionEnabled(engine) helper
  reads `facts.extraction_enabled` config row. Defaults to TRUE; flip to
  'false'/'0'/'no'/'off' (case-insensitive) via `gbrain config set
  facts.extraction_enabled false` to kill extraction across the brain
  without binary downgrade.
- extract_facts MCP op short-circuits with zero-counts envelope + a
  'skipped: extraction_disabled' field when the flag is off (clean
  success, not permission_denied).
- put_page facts backstop respects the same flag — eligibility check now
  returns 'extraction_disabled' as the skipped reason.
- src/commands/doctor.ts: new facts_health check (runs after queue_health,
  before index_audit). Probes for the facts table existence (post-v40
  guard), then surfaces total_active / total_today / total_week /
  total_consolidated + top-3 entities for the default source. Pre-v0.31
  brains report "facts table not present (pre-v0.31 brain or migration
  pending)".
- CHANGELOG.md: full v0.31.0 entry in the GStack release-summary voice.
  Headline + numbers-table + what-it-ships + itemized changes + "To take
  advantage of v0.31" upgrade block + out-of-scope. Honest about the
  HALFVEC + serve-http refactor + ErrorCode-open-union complications.
- README.md: cycle phase list updated 8 → 10 (consolidate + purge). New
  "v0.31 Hot Memory" command block under Commands with recall + forget
  variants, kind icons, --as-context surface for headless agents.

Test gates: 28 facts unit tests pass after the kill-switch wiring + doctor
check ride-along. Typecheck clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix(migrate): add facts→sources FK explicitly via ALTER TABLE

The inline column-level FK declaration on facts.source_id worked on
PGLite but silently got dropped on Postgres in the v0.31 e2e run —
the migration handler ran via postgres-js's `unsafe()` multi-statement
path and the resulting facts table came back without the
`facts_source_id_fkey` constraint. Same psql input run directly
against the same database produced the FK; the difference was the
unsafe() pipeline, not the SQL itself.

Splitting the FK into a separate ALTER TABLE inside a DO block makes
the constraint declaration explicit and idempotent: the named
constraint either exists or it doesn't, the ALTER is a no-op on
re-runs, and the failure mode is loud rather than silently leaving
a CASCADE-less foreign key behind.

Without this fix, deleting a source row leaves orphaned facts rows
(test/e2e/facts-cross-source-isolation.test.ts CASCADE-on-sources-
delete case caught it). With this fix the constraint is in place,
the cascade fires, and both PG + PGLite e2e suites stay green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 test: update phase-count assertions for the new consolidate phase

Three e2e/unit tests pinned the cycle phase count or order, all now
updated to reflect v0.31's 10-phase cycle:

- test/e2e/dream-cycle-eight-phase-pglite.test.ts:
  describe rename "8-phase cycle" → "10-phase cycle"; ALL_PHASES
  expectation extended to include 'consolidate' (between patterns +
  embed) and 'purge' (the v0.26.5 addition that was already in
  ALL_PHASES but missing from the test's assertion list). totals
  match adds the new facts_consolidated + consolidate_takes_written
  fields plus the pre-existing purged_sources_count + purged_pages_count
  that should have been added when v0.26.5 landed.

- test/e2e/cycle.test.ts: dry-run full cycle now expects
  report.phases.length === 10 (was 9).

- test/core/cycle.serial.test.ts: yieldBetweenPhases hook count + full
  cycle phases.length both updated 9 → 10. Comments call out the
  v0.31 addition lineage so the next person to add a phase sees the
  precedent.

These are mechanical assertion bumps. The tests pass against the
updated assertions on PGLite and Postgres.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix(test): truncate facts table between e2e describe blocks

setupDB() truncates ALL_TABLES between every describe block's
beforeAll() hook. The list missed the new v0.31 facts table, so
facts seeded by an earlier describe block leaked into Garry's
Separation Test on Postgres — listFactsByEntity('travel') returned
2 rows instead of 1 because a prior facts-context-injection test had
also seeded a 'travel' fact.

Adding 'facts' to the truncate list (before 'pages' to respect FK
ordering) makes every describe-block start from an empty facts table.

Pinned by re-running the e2e file ordering that originally caught it
(facts-recall-render → cross-source-isolation → serve-http-meta →
context-injection → separation-postgres → facts-forget) — 13 pass /
0 fail after the fix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 test: meta-hook cache + Postgres consolidate phase coverage

Two net-new test files filling real coverage gaps the earlier sweep missed:

- test/facts-meta-cache.test.ts (5 cases) — pins the eD3/eD10 cache
  contract that the dispatcher relies on. 30s TTL hit path, post-bump
  fresh-query, scoped invalidation (bump for sess-A leaves sess-B cache
  warm — closes the cross-source leak risk codex F5 originally surfaced
  on the recall payload), facts-self ops skip injection (anti-loop on
  recall / extract_facts / forget_fact), distinct allow-lists produce
  distinct cache entries.

- test/e2e/cycle-consolidate-postgres.test.ts (3 cases) — Postgres
  parity for the dream-cycle consolidate phase. Mirrors the PGLite
  unit test but exercises the real postgres-engine codepaths: sql.begin
  transactions, advisory locks on insertFact's entity-slug dedup window,
  unsafe('::vector') casts on findCandidateDuplicates ordering,
  addTakesBatch postgres-js unnest path. Happy path (4 facts → 1 take +
  all consolidated_into set), age-threshold skip, dry-run no-write.

All 5 unit + 3 e2e tests pass. Closes the unit-only gap on the
consolidate phase (was only PGLite-tested) and pins meta-cache
invariants the dispatcher depends on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix: thread auth + sourceId, JSON-shape every error envelope

Three bugs surfaced during the full e2e sweep that all trace back to my
v0.31 dispatch refactor (D12/eE1) silently dropping auth threading +
non-OperationError exceptions emitting plain strings:

1. **HTTP MCP transport lost ctx.auth.** Refactoring serve-http.ts to call
   dispatchToolCall meant auth had to come through DispatchOpts, but the
   field didn't exist yet. Every HTTP whoami call returned
   `unknown_transport` because ctx.auth was undefined. Added `auth?:
   AuthInfo` to DispatchOpts, plumbed it through buildOperationContext,
   and updated serve-http.ts:816 to pass `auth: authInfo` alongside
   sourceId/takesHoldersAllowList. Pinned by sources-remote-mcp e2e
   `whoami reports oauth transport + sources_admin scope`.

2. **Non-OperationError exceptions emitted plain strings, not JSON.**
   The pre-v0.31 serve-http.ts always wrapped errors in JSON envelope
   `{error, message}`; my dispatch refactor missed the unknown-tool +
   uncaught-throw paths and emitted `Error: ${msg}` text content. Every
   caller that did `JSON.parse(content)` (sources-remote-mcp callMcp
   helper at line 104) crashed with `Unexpected identifier "Error"`.
   Both error paths in dispatchToolCall now return JSON-shaped content
   matching the OperationError pattern.

3. **Files→sources FK silently lost on rewound bootstrap path.**
   test/e2e/postgres-bootstrap.test.ts simulates a pre-v0.21 brain by
   `DROP TABLE IF EXISTS sources CASCADE` which removes
   files_source_id_fkey while leaving files.source_id intact. The v23
   migration's `ALTER TABLE files ADD COLUMN IF NOT EXISTS source_id ...
   REFERENCES sources(id) ON DELETE CASCADE` is a no-op when the column
   exists, so the FK never came back on upgrade — and any sources-remove
   afterward stopped cascading to files. Added a defensive
   `IF NOT EXISTS files_source_id_fkey ... ALTER TABLE ADD CONSTRAINT`
   block inside v23's handler. Pinned by `multi-source — cascade delete
   covers every dependent row` after running postgres-bootstrap.

Plus: src/core/preferences.ts now honors GBRAIN_HOME for
`~/.gbrain/migrations/completed.jsonl`. Without this, the doctor
exits-0 mechanical test inherits the developer machine's stale
partial-migration ledger entries (0.21.0, 0.22.4, 0.28.0, 0.29.1
prior dev work) and surfaces them as the [FAIL] minions_migration check.
GBRAIN_HOME-scoped tempdir per test now isolates this state cleanly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 chore: scrub personal references from public artifacts

Per the CLAUDE.md privacy rule on `Garry's Separation Test`, replace
personally-coded references in v0.31 artifacts with neutral examples:

- CHANGELOG.md v0.31 entry: rename "Garry's Separation Test" header to
  "The cross-session test" + drop the "topic-2659/topic-1941, 7 AM/2 PM,
  flying to Tokyo" narrative.
- src/commands/migrations/v0_31_0.ts feature pitch: same scrub.
- test/facts-separation-pglite.test.ts + test/e2e/facts-separation-postgres.test.ts:
  rename describe blocks; replace specific topic-NNNN session ids with
  session-A / session-B; replace personal sample fact with
  "sample event Tuesday".
- src/core/facts/extract.ts extractor system prompt example slugs:
  people/sam-altman → people/alice-example; companies/anthropic → companies/acme.
- src/core/entities/resolve.ts comment: Sam Altman → Alice Example.
- All v0.31 test fixtures: people/sam → people/alice-example,
  Sam Altman → Alice Example, sam-the-cofounder → alice-the-cofounder.
  Test names referencing real-world entities replaced with neutral slugs.

Pre-existing references to "Garry" elsewhere in CHANGELOG (v0.17, v0.19,
v0.21+ entries) are untouched — that's a separate scope from this v0.31
ship.

Plus: the truncate fix for the Bun-script-induced syntax error in
test/e2e/mechanical.test.ts (cliEnv arrow function had ", 30_000)" tacked
onto its closing brace by the bulk-add-timeouts script — repaired to a
clean function definition).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix(test): bump E2E phase-count assertions for 11-phase cycle

Two E2E tests still asserted the v0.31 pre-merge 10-phase shape
(consolidate inserted, but recompute_emotional_weight from v0.29 not yet
absorbed). With master's v0.29 work merged in, the cycle is now 11 phases:
lint → backlinks → sync → synthesize → extract → patterns →
recompute_emotional_weight → consolidate → embed → orphans → purge.

- test/e2e/cycle.test.ts: 10 → 11
- test/e2e/dream-cycle-eight-phase-pglite.test.ts: ALL_PHASES + dry-run order

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix(merge): close brace between v44 and v45 migration objects

The v0.30.2 merge resolution stitched master's v40-v44 migrations onto
HEAD's v45 (facts hot memory) migration but lost the closing `},` between
v44 and v45. tsc caught it as TS1136 Property assignment expected at
migrate.ts:2188.

This is a one-line bracket fix; the rest of the merge resolution is
correct and tests pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 fix: put_page cliHints + buildPlan v0.31.0 in skippedFuture

Two unit-test failures surfaced after the v0.30.2 merge:

1. operations.ts: put_page had `cliHints: { name: 'put', positional: ['stdin'] }`
   from earlier v0.31 development. The parity test enforces that every name
   in `positional` is a real param. Restored master's correct shape:
   `{ name: 'put', positional: ['slug'], stdin: 'content' }`.

2. test/apply-migrations.test.ts: the H9 regression tests pin the exact
   skippedFuture list. Adding v0.31.0 to the registry meant the list grew
   by one. Updated both `expect(...).toEqual([...])` assertions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* v0.31 docs: clarify consolidate is 11th phase + regen llms-full.txt

CHANGELOG.md narrative said "new 10th phase consolidate"; with v0.29's
recompute_emotional_weight already on master, consolidate is the 11th phase
(between recompute and embed). Schema migration is v45, not v40, after the
merge resolution renumbered it to clear master's v40-v44.

llms-full.txt regenerated to reflect the README's 11-phase dream-cycle
phrasing (the build-llms test enforces commit-time parity).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 16:57:47 -07:00

879 lines
40 KiB
TypeScript

/**
* E2E tests for serve-http.ts OAuth 2.1 fixes (v0.26.1).
*
* Spins up a real `gbrain serve --http` against real Postgres, registers an
* OAuth client, mints tokens, and exercises the full MCP JSON-RPC pipeline
* end-to-end. Catches the three bugs fixed in v0.26.1:
*
* 1. client_credentials tokens rejected at /mcp (expiresAt string vs number)
* 2. OAuth metadata missing client_credentials grant type
* 3. Express 5 trust proxy + admin SPA wildcard
*
* Run: GBRAIN_DATABASE_URL=... bun test test/e2e/serve-http-oauth.test.ts
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { hasDatabase } from './helpers.ts';
const skip = !hasDatabase();
const describeE2E = skip ? describe.skip : describe;
if (skip) {
console.log('Skipping E2E serve-http-oauth tests (DATABASE_URL not set)');
}
const PORT = 19131; // Avoid collision with production 3131
const BASE = `http://localhost:${PORT}`;
describeE2E('serve-http OAuth 2.1 E2E (v0.26.1 + v0.26.2 + v0.26.3)', () => {
let serverProcess: ReturnType<typeof import('child_process').spawn> | null = null;
let clientId: string | undefined;
let clientSecret: string | undefined;
// DCR-registered clients accumulate here so afterAll can revoke them too
// (one per test that posts to /register).
const dcrClientIds: string[] = [];
beforeAll(async () => {
const { execSync, spawn } = await import('child_process');
// Register a test OAuth client via CLI.
// env: { ...process.env } is required: bun's execSync does NOT inherit
// env mutations done via `process.env.X = ...` (only OS-level env from
// before bun started). helpers.ts loads .env.testing and sets DATABASE_URL
// via process.env mutation, which is invisible to subprocesses unless we
// explicitly re-pass process.env. Same pattern applies to every execSync
// in this file.
// v0.28.10: register with admin scope so the F7 protected-name guard
// tests can mint admin-scoped tokens that actually exercise the guard
// at operations.ts:1527. Without admin in the client's allowed scopes,
// submit_job for a protected name (`shell`, `subagent`) gets rejected
// by hasScope() in serve-http.ts BEFORE reaching the F7 guard, so the
// test was validating scope enforcement instead of the RCE protection.
// Other tests that mint specific subsets ('read', 'read write') still
// get the subset they ask for — adding admin to the client's allowed
// ceiling does not auto-grant it to every minted token.
const regOutput = execSync(
'bun run src/cli.ts auth register-client e2e-oauth-test --grant-types client_credentials --scopes "read write admin"',
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } }
);
const idMatch = regOutput.match(/Client ID:\s+(gbrain_cl_\S+)/);
const secretMatch = regOutput.match(/Client Secret:\s+(gbrain_cs_\S+)/);
if (!idMatch || !secretMatch) throw new Error('Failed to register test client:\n' + regOutput);
clientId = idMatch[1];
clientSecret = secretMatch[1];
// Start the HTTP server. v0.26.2 adds --enable-dcr so the /register
// endpoint is reachable for the DCR response-shape test.
serverProcess = spawn('bun', [
'run', 'src/cli.ts', 'serve', '--http',
'--port', String(PORT),
'--public-url', `http://localhost:${PORT}`,
'--enable-dcr',
], {
cwd: process.cwd(),
env: process.env,
stdio: ['ignore', 'pipe', 'pipe'],
});
// Collect stderr for debugging failures
let stderr = '';
serverProcess.stderr?.on('data', (d: Buffer) => { stderr += d.toString(); });
// Wait for server to be ready (up to 15s)
let ready = false;
for (let i = 0; i < 30; i++) {
try {
const res = await fetch(`${BASE}/health`);
if (res.ok) { ready = true; break; }
} catch {}
await new Promise(r => setTimeout(r, 500));
}
if (!ready) throw new Error('Server failed to start within 15s.\nstderr: ' + stderr.slice(-500));
}, 30_000);
afterAll(async () => {
// Kill server first so it can't issue more tokens during cleanup.
if (serverProcess) {
serverProcess.kill('SIGTERM');
await new Promise(r => setTimeout(r, 1000));
if (!serverProcess.killed) serverProcess.kill('SIGKILL');
}
// v0.26.2 cleanup contract: only revoke if registration succeeded
// (clientId guard) and surface any cleanup failure to stderr without
// throwing — a real test failure is more interesting than the cleanup
// error that follows it. Same shape applies to DCR-registered clients
// tracked in dcrClientIds.
const { execSync } = await import('child_process');
const toRevoke = [...(clientId ? [clientId] : []), ...dcrClientIds];
for (const id of toRevoke) {
try {
execSync(`bun run src/cli.ts auth revoke-client "${id}"`,
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } });
} catch (e: any) {
// eslint-disable-next-line no-console
console.error(`[afterAll] revoke-client cleanup failed for ${id}: ${e.message}`);
}
}
}, 30_000);
// Helper: mint a token with given scopes
async function mintToken(scope = 'read write'): Promise<{ access_token: string; expires_in: number; scope: string }> {
const res = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${clientId}&client_secret=${clientSecret}&scope=${encodeURIComponent(scope)}`,
});
expect(res.ok).toBe(true);
return res.json() as any;
}
// Helper: call MCP JSON-RPC with a bearer token
async function mcpCall(token: string, method: string, params?: any): Promise<Response> {
return fetch(`${BASE}/mcp`, {
method: 'POST',
headers: {
'Authorization': `Bearer ${token}`,
'Content-Type': 'application/json',
'Accept': 'application/json, text/event-stream',
},
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, ...(params ? { params } : {}) }),
});
}
// =========================================================================
// Fix 1: client_credentials tokens validate at /mcp
// =========================================================================
test('mint token via client_credentials grant', async () => {
const data = await mintToken('read write');
expect(data.access_token).toMatch(/^gbrain_at_/);
expect(data.expires_in).toBe(3600);
expect(data.scope).toContain('read');
});
test('minted token is accepted at /mcp — tools/list returns tools', async () => {
const { access_token } = await mintToken('read');
const res = await mcpCall(access_token, 'tools/list');
// Before v0.26.1 fix: 401 {"error":"invalid_token","error_description":"Token has no expiration time"}
expect(res.status).not.toBe(401);
const body = await res.text();
expect(body).toContain('tools');
expect(body).toContain('search'); // search tool should be in the list
expect(body).toContain('query'); // query tool too
}, 15_000);
test('minted token works for tools/call — search executes', async () => {
const { access_token } = await mintToken('read');
const res = await mcpCall(access_token, 'tools/call', {
name: 'search',
arguments: { query: 'gbrain', limit: 1 },
});
expect(res.status).not.toBe(401);
const body = await res.text();
// Should contain search results, not an auth error
expect(body).not.toContain('invalid_token');
expect(body).toContain('result');
}, 15_000);
test('expired/invalid token is rejected at /mcp', async () => {
const res = await mcpCall('gbrain_at_totally_fake_token', 'tools/list');
// Invalid tokens should not return 200 with tool results
const body = await res.text();
expect(body).not.toContain('"tools"');
// Should be an error status (401, 403, or 500 depending on SDK error mapping)
expect(res.status).toBeGreaterThanOrEqual(400);
});
test('missing Authorization header returns 401', async () => {
const res = await fetch(`${BASE}/mcp`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json, text/event-stream',
},
body: JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'tools/list' }),
});
expect(res.status).toBe(401);
});
// =========================================================================
// Fix 2: OAuth metadata includes client_credentials
// =========================================================================
test('OAuth AS metadata includes all three grant types', async () => {
const res = await fetch(`${BASE}/.well-known/oauth-authorization-server`);
expect(res.ok).toBe(true);
const meta = await res.json() as any;
expect(meta.grant_types_supported).toContain('authorization_code');
expect(meta.grant_types_supported).toContain('refresh_token');
expect(meta.grant_types_supported).toContain('client_credentials');
});
test('OAuth metadata issuer matches public URL', async () => {
const res = await fetch(`${BASE}/.well-known/oauth-authorization-server`);
const meta = await res.json() as any;
expect(meta.issuer).toBe(`http://localhost:${PORT}/`);
expect(meta.token_endpoint).toContain('/token');
expect(meta.scopes_supported).toContain('read');
expect(meta.scopes_supported).toContain('write');
expect(meta.scopes_supported).toContain('admin');
});
// T2 (eng-review): scopes_supported advertises the full ALLOWED_SCOPES_LIST
// so MCP clients (Claude Desktop, ChatGPT, Perplexity) can discover the
// v0.28 sources_admin and users_admin scopes via standard discovery.
// Pre-v0.28 the list was hardcoded to ['read','write','admin'] in
// serve-http.ts:195 and this assertion would have failed.
test('OAuth metadata advertises all 5 v0.28 scopes (sources_admin + users_admin)', async () => {
const res = await fetch(`${BASE}/.well-known/oauth-authorization-server`);
const meta = await res.json() as any;
expect(meta.scopes_supported).toContain('sources_admin');
expect(meta.scopes_supported).toContain('users_admin');
expect(meta.scopes_supported).toEqual(
expect.arrayContaining(['admin', 'read', 'sources_admin', 'users_admin', 'write']),
);
});
// =========================================================================
// Fix 3: Express 5 compatibility
// =========================================================================
test('admin dashboard serves SPA index.html (not Express error)', async () => {
const res = await fetch(`${BASE}/admin/`);
const html = await res.text();
expect(html).toContain('GBrain Admin');
expect(html).not.toContain('<pre>Cannot GET');
});
test('admin sub-routes serve SPA fallback', async () => {
const res = await fetch(`${BASE}/admin/agents`);
const html = await res.text();
expect(html).toContain('GBrain Admin');
});
test('X-Forwarded-For header does not crash server', async () => {
const res = await fetch(`${BASE}/health`, {
headers: { 'X-Forwarded-For': '10.0.0.1, 172.16.0.1' },
});
expect(res.ok).toBe(true);
const data = await res.json() as any;
expect(data.status).toBe('ok');
});
// =========================================================================
// Scope enforcement
// =========================================================================
test('read-only token is rejected for write operations', async () => {
const { access_token } = await mintToken('read');
const res = await mcpCall(access_token, 'tools/call', {
name: 'put_page',
arguments: { slug: 'e2e-scope-test', content: '---\ntitle: test\n---\ntest' },
});
const body = await res.text();
// Should be rejected via scope check (403 or JSON-RPC error with scope message)
expect(res.status === 403 || body.includes('scope') || body.includes('Insufficient')).toBe(true);
}, 15_000);
test('write-scoped token can call read operations', async () => {
const { access_token } = await mintToken('read write');
const res = await mcpCall(access_token, 'tools/call', {
name: 'search',
arguments: { query: 'test', limit: 1 },
});
expect(res.status).not.toBe(401);
expect(res.status).not.toBe(403);
const body = await res.text();
// Should get a result, not an auth error
expect(body).not.toContain('invalid_token');
expect(body).not.toContain('insufficient_scope');
}, 15_000);
// =========================================================================
// Health endpoint (no auth required) — v0.28.10 made /health liveness-only;
// engine stats moved to /admin/api/full-stats behind requireAdmin so a
// saturated pool can't pin /health and trigger orchestrator restart cascades.
// =========================================================================
test('v0.28.10: /health returns liveness-only body (no engine stats)', async () => {
const res = await fetch(`${BASE}/health`);
expect(res.ok).toBe(true);
const data = await res.json() as any;
expect(data.status).toBe('ok');
expect(data.version).toBeDefined();
expect(data.engine).toBeDefined();
// Regression: pre-v0.28.10 /health spread getStats() (page_count,
// chunk_count, etc.) into the body. The whole point of the v0.28.10
// split is that /health stops touching those tables. If page_count
// ever reappears here, the heavy probe leaked back into the public
// route and the original DoS surface is back.
expect(data.page_count).toBeUndefined();
expect(data.chunk_count).toBeUndefined();
expect(data.embedded_count).toBeUndefined();
// Body shape is exactly {status, version, engine}.
expect(Object.keys(data).sort()).toEqual(['engine', 'status', 'version']);
});
test('v0.28.10: /admin/api/full-stats without admin cookie returns 401', async () => {
const res = await fetch(`${BASE}/admin/api/full-stats`);
expect(res.status).toBe(401);
const data = await res.json() as any;
expect(data.error).toBe('Admin authentication required');
});
test('v0.28.10: /admin/api/full-stats with valid admin cookie returns getStats() body', async () => {
// Same magic-link cookie dance the existing single-use test uses.
// Skip gracefully if the bootstrap token isn't extractable — the 401
// case above pins the auth gate; this test pins the happy path.
const stderrBuf = (serverProcess as any)?._stderrBuffer || '';
const tokenMatch = String(stderrBuf).match(/Admin Token[\s\S]*?([a-f0-9]{32,64})/);
if (!tokenMatch) {
console.warn('[e2e] skipped /admin/api/full-stats happy path: could not extract bootstrap token');
return;
}
const bootstrapToken = tokenMatch[1];
const issueRes = await fetch(`${BASE}/admin/api/issue-magic-link`, {
method: 'POST',
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${bootstrapToken}` },
body: '{}',
});
expect(issueRes.ok).toBe(true);
const { url } = await issueRes.json() as any;
const click = await fetch(url, { redirect: 'manual' });
expect(click.status).toBe(302);
const setCookie = click.headers.get('set-cookie') || '';
const cookieMatch = setCookie.match(/gbrain_admin=([^;]+)/);
expect(cookieMatch).toBeTruthy();
const cookieValue = cookieMatch![1];
const statsRes = await fetch(`${BASE}/admin/api/full-stats`, {
headers: { Cookie: `gbrain_admin=${cookieValue}` },
});
expect(statsRes.ok).toBe(true);
const stats = await statsRes.json() as any;
expect(stats.status).toBe('ok');
expect(stats.version).toBeDefined();
expect(stats.engine).toBeDefined();
// The full-stats body is probeHealth's spread of getStats() — page_count
// is the canonical signal that we're hitting the heavy path here.
expect(typeof stats.page_count).toBe('number');
expect(stats.page_count).toBeGreaterThanOrEqual(0);
}, 15_000);
// =========================================================================
// Token lifecycle
// =========================================================================
test('multiple tokens can be minted and used independently', async () => {
const t1 = await mintToken('read');
const t2 = await mintToken('read write');
// Both should work
const r1 = await mcpCall(t1.access_token, 'tools/list');
const r2 = await mcpCall(t2.access_token, 'tools/list');
expect(r1.status).not.toBe(401);
expect(r2.status).not.toBe(401);
}, 15_000);
test('wrong client_secret is rejected at token endpoint', async () => {
const res = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${clientId}&client_secret=gbrain_cs_wrong_secret&scope=read`,
});
expect(res.ok).toBe(false);
const data = await res.json() as any;
expect(data.error).toBe('invalid_grant');
});
// =========================================================================
// v0.26.2: DCR /register response shape (RFC 7591 §3.2.1 number contract)
// =========================================================================
//
// The user-visible bug v0.26.2 protects against: postgres.js with
// `prepare: false` returns BIGINT columns as strings, and an RFC-strict
// DCR client (Claude Code, Cursor) parses the /register response as JSON
// and rejects timestamps that aren't numbers. This is the HTTP-level test;
// the internal-store shape test in test/oauth.test.ts is not enough on its
// own (Codex flagged it as the wrong seam).
test('DCR /register returns numeric client_id_issued_at (RFC 7591 §3.2.1)', async () => {
const res = await fetch(`${BASE}/register`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
client_name: 'e2e-dcr-shape',
redirect_uris: ['https://example.com/cb'],
grant_types: ['authorization_code'],
token_endpoint_auth_method: 'client_secret_basic',
scope: 'read',
}),
});
expect(res.ok).toBe(true);
const body = await res.json() as any;
// Track for cleanup before any assertion that could throw.
if (body.client_id) dcrClientIds.push(body.client_id);
// The contract: client_id_issued_at is REQUIRED to be a JSON number per
// RFC 7591. Pre-v0.26.2 with prepare:false returned this as a string
// (e.g., "1735689600") and strict clients rejected the registration.
expect(typeof body.client_id_issued_at).toBe('number');
expect(Number.isFinite(body.client_id_issued_at)).toBe(true);
expect(body.client_id_issued_at).toBeGreaterThan(0);
// client_secret_expires_at is OPTIONAL. If present, it must also be a
// number. Undefined/missing means "does not expire" per the spec.
if (body.client_secret_expires_at !== undefined) {
expect(typeof body.client_secret_expires_at).toBe('number');
expect(Number.isFinite(body.client_secret_expires_at)).toBe(true);
}
}, 15_000);
// =========================================================================
// v0.26.2: revoke-client CLI subprocess test
// =========================================================================
//
// Validates the actual CLI router in src/commands/auth.ts, not just the
// database deletion semantics. Codex flagged that a unit test in
// test/oauth.test.ts proves DB DELETE works but does NOT prove the
// subcommand exists or routes correctly.
test('auth revoke-client (CLI) deletes client + cascades to tokens', async () => {
const { execSync } = await import('child_process');
// Step 1: register a throwaway client via CLI.
// env: { ...process.env } per the bun execSync inheritance fix above.
const regOutput = execSync(
'bun run src/cli.ts auth register-client e2e-revoke-cli --grant-types client_credentials --scopes read',
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } }
);
const idMatch = regOutput.match(/Client ID:\s+(gbrain_cl_\S+)/);
const secretMatch = regOutput.match(/Client Secret:\s+(gbrain_cs_\S+)/);
expect(idMatch).not.toBeNull();
expect(secretMatch).not.toBeNull();
const id = idMatch![1];
const secret = secretMatch![1];
// Step 2: mint a token through the live server.
const tokenRes = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${id}&client_secret=${secret}&scope=read`,
});
expect(tokenRes.ok).toBe(true);
const { access_token } = await tokenRes.json() as any;
// Sanity: the freshly-minted token works at /mcp.
const before = await mcpCall(access_token, 'tools/list');
expect(before.status).not.toBe(401);
// Step 3: revoke via the CLI subprocess.
const revokeOutput = execSync(
`bun run src/cli.ts auth revoke-client "${id}"`,
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } }
);
// The handler prints the human confirmation lines. No exit code != 0
// here since execSync would throw.
expect(revokeOutput).toMatch(/OAuth client revoked/);
expect(revokeOutput).toMatch(/cascade/i);
// Step 4: previously-minted token must now be rejected at /mcp. Cascade
// wiped the oauth_tokens row; verifyAccessToken throws "Invalid token".
// Match the existing pattern at line 156: SDK error mapping varies
// (401/403/500), so we assert non-success status + non-success body
// rather than a single status code.
const after = await mcpCall(access_token, 'tools/list');
expect(after.status).toBeGreaterThanOrEqual(400);
const afterBody = await after.text();
expect(afterBody).not.toContain('"tools":[');
// Step 5: re-running revoke-client on the now-deleted id must exit 1.
let secondRunFailed = false;
let secondRunStderr = '';
try {
execSync(`bun run src/cli.ts auth revoke-client "${id}"`,
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } });
} catch (e: any) {
secondRunFailed = true;
secondRunStderr = (e.stderr || '').toString() + (e.stdout || '').toString();
}
expect(secondRunFailed).toBe(true);
expect(secondRunStderr).toMatch(/No client found/);
}, 30_000);
// =========================================================================
// v0.26.3: Migration v33 round-trip — pins the 5 new columns
// =========================================================================
//
// PR #586 referenced oauth_clients.{token_ttl, deleted_at} +
// mcp_request_log.{agent_name, params, error_message} without an
// accompanying migration. v33 adds them. This test pins the round-trip:
// make a /mcp call -> assert all three new mcp_request_log columns
// persisted correctly. Without v33, the INSERT silently swallows
// column-doesn't-exist errors via the existing best-effort try/catch
// and the row never appears.
test('v0.26.3: /mcp request persists agent_name + params + error_message', async () => {
const postgres = (await import('postgres')).default;
const sql = postgres(process.env.GBRAIN_DATABASE_URL || process.env.DATABASE_URL || '', { prepare: false });
try {
// Wipe any prior log rows for our test client so we can assert exact counts.
await sql`DELETE FROM mcp_request_log WHERE token_name = ${clientId!}`;
// Mint a fresh write-scoped token and make a successful tools/list call.
const tokenRes = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${clientId!}&client_secret=${clientSecret!}&scope=read`,
});
expect(tokenRes.ok).toBe(true);
const { access_token } = await tokenRes.json() as any;
const okRes = await mcpCall(access_token, 'tools/list');
expect(okRes.status).not.toBe(401);
// Trigger an error path so the error_message column gets a value too.
// Request a tool that doesn't exist — v0.28.10 logs unknown-op attempts
// with operation = the attempted name and error_message starting with
// 'unknown_operation:'.
await mcpCall(access_token, 'tools/call', { name: 'this_tool_does_not_exist', arguments: {} });
// Allow async best-effort INSERT to flush.
await new Promise(r => setTimeout(r, 250));
const rows = await sql`
SELECT operation, status, agent_name, params, error_message
FROM mcp_request_log
WHERE token_name = ${clientId!}
ORDER BY created_at ASC
` as unknown as Array<Record<string, unknown>>;
expect(rows.length).toBeGreaterThanOrEqual(2);
// Agent name resolved from oauth_clients.client_name (the JOIN in
// verifyAccessToken or the agent_name backfill path).
for (const row of rows) {
expect(row.agent_name).toBe('e2e-oauth-test');
}
// v0.28.10: tools/list logs as operation='tools/list' (the JSON-RPC
// method name). tools/call success/error logs as operation=<inner
// tool name> (the convention preserved from pre-v0.28.10 dispatch
// logging — agents querying mcp_request_log filter by tool name, not
// by JSON-RPC method).
const listRow = rows.find(r => r.operation === 'tools/list');
expect(listRow).toBeDefined();
expect(listRow!.status).toBe('success');
// The unknown-op call shows up with operation = the attempted name.
const callRow = rows.find(r => r.operation === 'this_tool_does_not_exist');
expect(callRow).toBeDefined();
expect(callRow!.status).toBe('error');
// error_message populated on the failed call.
const errorRow = rows.find(r => r.status === 'error');
expect(errorRow).toBeDefined();
expect(errorRow!.error_message).toBeTruthy();
expect(typeof errorRow!.error_message).toBe('string');
expect(errorRow!.error_message as string).toContain('unknown_operation');
} finally {
await sql.end();
}
}, 30_000);
// =========================================================================
// v0.26.3: request-log filter injection probe
// =========================================================================
//
// Pre-fix: /admin/api/requests built WHERE clauses via sql.unsafe() with
// single-quote escape (`token_name = '${agent.replace(/'/g, "''")}'`).
// Post-fix: postgres.js tagged-template fragments. This probe sends a
// payload that, under broken escaping, would short-circuit to TRUE and
// return all rows. Under correct parameterization, it matches no rows.
test("v0.26.3: request-log filter rejects injection attempt (' OR 1=1)", async () => {
// Use a plain admin session via /admin/login + bootstrap token. This
// test covers the unauthenticated SQL-injection vector via the agent
// query parameter — even though the endpoint is admin-gated, defense-
// in-depth on parameterization matters.
//
// Extract the admin bootstrap token from the spawned server's stderr.
const probe = "alice'%20OR%201%3D1";
// We don't have a clean way to pull the admin token from the spawned
// process here (commit 16 deleted the regex extraction). The injection
// probe still works WITHOUT auth — the endpoint requires it via 401.
// We assert that the 401 lands BEFORE any SQL gets built, so we don't
// crash the server with malformed SQL on the way to the auth check.
const res = await fetch(`${BASE}/admin/api/requests?agent=${probe}`, {
method: 'GET',
});
// No admin cookie — must hit 401, not 500 (no SQL crash).
expect(res.status).toBe(401);
// Server is still alive (didn't crash on the malformed input).
const health = await fetch(`${BASE}/health`);
expect(health.ok).toBe(true);
});
// =========================================================================
// v0.26.3: per-client TTL flow
// =========================================================================
//
// PR #586 added `tokenTtl` per OAuth client. exchangeClientCredentials
// reads oauth_clients.token_ttl (per-client override) and falls back to
// the server default. This test registers a client with a custom TTL,
// mints a token, and asserts the response's expires_in matches.
test('v0.26.3: per-client token_ttl is honored on token mint', async () => {
const postgres = (await import('postgres')).default;
const sql = postgres(process.env.GBRAIN_DATABASE_URL || process.env.DATABASE_URL || '', { prepare: false });
try {
// Register a client + set a custom token_ttl (24 hours = 86400 seconds).
const { execSync } = await import('child_process');
const regOutput = execSync(
'bun run src/cli.ts auth register-client e2e-test-ttl --grant-types client_credentials --scopes read',
{ cwd: process.cwd(), encoding: 'utf8', env: { ...process.env } }
);
const idMatch = regOutput.match(/Client ID:\s+(gbrain_cl_\S+)/);
const secretMatch = regOutput.match(/Client Secret:\s+(gbrain_cs_\S+)/);
expect(idMatch).not.toBeNull();
expect(secretMatch).not.toBeNull();
const id = idMatch![1];
const secret = secretMatch![1];
dcrClientIds.push(id); // afterAll cleanup
// Set a 24-hour TTL.
await sql`UPDATE oauth_clients SET token_ttl = 86400 WHERE client_id = ${id}`;
// Mint a token. Response must include expires_in close to 86400.
const tokenRes = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${id}&client_secret=${secret}&scope=read`,
});
expect(tokenRes.ok).toBe(true);
const body = await tokenRes.json() as any;
expect(body.expires_in).toBe(86400);
// Update TTL to a different value mid-test, mint again, assert new value.
await sql`UPDATE oauth_clients SET token_ttl = 7200 WHERE client_id = ${id}`;
const tokenRes2 = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${id}&client_secret=${secret}&scope=read`,
});
expect(tokenRes2.ok).toBe(true);
const body2 = await tokenRes2.json() as any;
expect(body2.expires_in).toBe(7200);
// NULL token_ttl falls back to server default (3600 = 1 hour).
await sql`UPDATE oauth_clients SET token_ttl = NULL WHERE client_id = ${id}`;
const tokenRes3 = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${id}&client_secret=${secret}&scope=read`,
});
expect(tokenRes3.ok).toBe(true);
const body3 = await tokenRes3.json() as any;
expect(body3.expires_in).toBe(3600);
} finally {
await sql.end();
}
}, 30_000);
// =========================================================================
// v0.26.3: magic-link single-use + 401 styled error page
// =========================================================================
//
// D11=C: /admin/auth/:nonce is single-use. First click consumes the nonce,
// second click fails with the styled 401 page. No bootstrap token in URL.
//
// Also covers F6.5: server returns Content-Type: text/html on the 401
// path (Express auto-sets this for HTML body) so browsers render the
// styled page instead of treating it as plain text.
test('v0.26.3: invalid magic-link nonce returns styled 401 HTML page', async () => {
const res = await fetch(`${BASE}/admin/auth/garbage_nonce_that_does_not_exist`, { redirect: 'manual' });
expect(res.status).toBe(401);
const ct = res.headers.get('content-type') || '';
expect(ct).toContain('text/html');
const body = await res.text();
expect(body).toContain('expired');
expect(body).toContain('GBrain');
});
test('v0.26.3: magic-link nonce is single-use (second click fails)', async () => {
// Get a real bootstrap token from the spawned server's environment.
// The server prints it to stderr at startup but commit 16 removed our
// regex extractor. Use the issue-magic-link endpoint directly with the
// bootstrap token from process env — except that env var doesn't exist
// in the test fixture. The portable approach: extract from the server
// process's stderr.
// Pull the bootstrap token from server stderr by re-reading the
// spawn handle. The spawn already started so stderr has flushed.
// Skip if we can't extract — the test is best-effort coverage of the
// single-use semantic; the styled-401 test above covers the negative path.
const stderrBuf = (serverProcess as any)?._stderrBuffer || '';
const tokenMatch = String(stderrBuf).match(/Admin Token[\s\S]*?([a-f0-9]{32,64})/);
if (!tokenMatch) {
// No way to get the bootstrap token in this test fixture — skip gracefully.
// The unit-level coverage for nonce single-use is in oauth.test.ts and
// the styled-401 test above pins the consumed-nonce path.
console.warn('[e2e] skipped magic-link single-use: could not extract bootstrap token');
return;
}
const bootstrapToken = tokenMatch[1];
// Mint a one-time nonce.
const issueRes = await fetch(`${BASE}/admin/api/issue-magic-link`, {
method: 'POST',
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${bootstrapToken}` },
body: '{}',
});
expect(issueRes.ok).toBe(true);
const { url } = await issueRes.json() as any;
expect(url).toContain('/admin/auth/');
// First click — should set cookie + redirect (302 to /admin/).
const first = await fetch(url, { redirect: 'manual' });
expect(first.status).toBe(302);
const cookie = first.headers.get('set-cookie') || '';
expect(cookie).toContain('gbrain_admin=');
// Second click on the same URL — must fail (single-use consumed).
const second = await fetch(url, { redirect: 'manual' });
expect(second.status).toBe(401);
const secondBody = await second.text();
expect(secondBody).toContain('GBrain');
}, 15_000);
// =========================================================================
// v0.26.3: agent_name backfill across oauth_clients + access_tokens
// =========================================================================
//
// Migration v33 backfills mcp_request_log.agent_name using
// COALESCE(oauth_clients.client_name, access_tokens.name, token_name)
// This test confirms the agent_name is correctly resolved across both
// auth lanes (oauth client + legacy api key).
test('v0.26.3: agent_name resolves correctly for OAuth + legacy paths', async () => {
const postgres = (await import('postgres')).default;
const sql = postgres(process.env.GBRAIN_DATABASE_URL || process.env.DATABASE_URL || '', { prepare: false });
try {
// Make an OAuth-authenticated request — agent_name should be the OAuth client_name.
const tokenRes = await fetch(`${BASE}/token`, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: `grant_type=client_credentials&client_id=${clientId!}&client_secret=${clientSecret!}&scope=read`,
});
const { access_token } = await tokenRes.json() as any;
await mcpCall(access_token, 'tools/list');
await new Promise(r => setTimeout(r, 250));
const oauthRows = await sql`
SELECT agent_name FROM mcp_request_log
WHERE token_name = ${clientId!}
ORDER BY created_at DESC LIMIT 1
` as unknown as Array<{ agent_name: string }>;
expect(oauthRows.length).toBeGreaterThan(0);
expect(oauthRows[0].agent_name).toBe('e2e-oauth-test');
} finally {
await sql.end();
}
}, 15_000);
// =========================================================================
// v0.26.3: register-client missing-name returns 400
// =========================================================================
//
// Defense-in-depth: the admin register-client endpoint must validate
// input. Pre-fix would have crashed or returned 500.
test('v0.26.3: /admin/api/register-client without name returns 400', async () => {
// Endpoint is admin-cookie-gated. Without auth we should get 401, not 500.
// Without a name in the body (with auth) we should get 400. We test the
// 401 path here as a basic input-validation smoke; the 400 path requires
// an admin session which the test fixture doesn't easily produce.
const res = await fetch(`${BASE}/admin/api/register-client`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: '{}',
});
expect(res.status).toBe(401);
});
// =========================================================================
// F7 + F7b: HTTP MCP shell-job RCE regression
// =========================================================================
//
// The headline trust-boundary fix. Pre-fix, the inlined OperationContext
// literal in serve-http.ts forgot to set `remote: true`, which meant
// operations.ts:1391's protected-job-name guard (`if (ctx.remote && ...)`)
// saw a falsy undefined and skipped. An HTTP MCP caller with a write-scoped
// token could then submit `{name: "shell", params: {cmd: "id"}}` over /mcp
// and execute arbitrary commands on the gbrain host.
//
// The fix is two-layered:
// 1) F7 — serve-http.ts sets `remote: true` explicitly.
// 2) F7b — operations.ts:1391 + :1400 use `ctx.remote !== false` /
// `ctx.remote === false` so undefined fails closed even if a
// future transport bypasses the type via cast.
//
// Together they close the path even if either layer regresses alone.
test('F7: HTTP MCP cannot submit shell jobs (RCE regression)', async () => {
// v0.28.10: must mint admin scope. submit_job's required scope is
// 'admin'; without it, hasScope() rejects with insufficient_scope BEFORE
// the F7 protected-name guard at operations.ts:1527 fires. To validate
// the actual RCE protection (the protected-name guard), the token has
// to clear the scope check first.
const { access_token } = await mintToken('admin');
const res = await mcpCall(access_token, 'tools/call', {
name: 'submit_job',
arguments: { name: 'shell', data: { cmd: 'id' } },
});
const body = await res.text();
// Must reject. Either HTTP 4xx, or a JSON-RPC envelope carrying an
// OperationError with code permission_denied. The exact wire shape
// depends on SDK error mapping — assert the negative invariant
// (no command executed) and the positive invariant (rejection signal).
const rejected =
res.status >= 400 ||
body.includes('permission_denied') ||
body.includes('cannot be submitted over MCP');
expect(rejected).toBe(true);
// Negative: response must NOT contain a successful submit_job result
// (which would surface a job_id field). If a job ID came back the
// privesc landed.
expect(body).not.toMatch(/"job_id"\s*:\s*"?\d+/);
}, 15_000);
test('F7: HTTP MCP cannot submit subagent jobs (protected name)', async () => {
// Same admin-scope requirement as the shell-job sibling test above.
const { access_token } = await mintToken('admin');
const res = await mcpCall(access_token, 'tools/call', {
name: 'submit_job',
arguments: { name: 'subagent', data: { prompt: 'noop' } },
});
const body = await res.text();
const rejected =
res.status >= 400 ||
body.includes('permission_denied') ||
body.includes('cannot be submitted over MCP');
expect(rejected).toBe(true);
expect(body).not.toMatch(/"job_id"\s*:\s*"?\d+/);
}, 15_000);
});