Files
gbrain/test/e2e/http-transport.test.ts
T
d3b52edeba v0.22.7 fix: built-in HTTP transport with bearer auth for remote MCP (#483)
* fix: add built-in HTTP transport with bearer auth for remote MCP

Adds `gbrain serve --http` with token-based authentication using the
existing access_tokens table. Eliminates the need for standalone OAuth
wrappers that may have insecure open registration endpoints.

- New: src/mcp/http-transport.ts — HTTP+SSE transport with bearer auth
- New: SECURITY.md — security advisory for remote MCP deployments
- Updated: serve command accepts --http and --port flags
- Updated: DEPLOY.md recommends --http for remote access
- Bump: 0.22.4 → 0.22.5

* chore: extract shared MCP dispatch + rate-limit modules

dispatch.ts is the single source of truth for stdio + HTTP transport: validateParams,
OperationContext build, handler invocation, error formatting. Server.ts refactored to
use it. Prevents the F1-F3 transport-drift bugs where stdio and HTTP independently
implemented dispatch logic differently (reversed args, missing context fields, no
param validation).

rate-limit.ts: bounded-LRU token-bucket. Tracks lastTouchedMs separately from
lastRefillMs so an exhausted key can't be reset by hammering past the TTL.

* feat: HTTP transport hardening + F1-F3 dispatch bug fixes

Rewrite of src/mcp/http-transport.ts on top of the new dispatch.ts and rate-limit.ts:

- F1 fix: dispatch via shared dispatchToolCall(ctx, params) — was reversed args
  (params, ctx) before, would have crashed every real tools/call.
- F2 fix: full OperationContext (engine, config, logger, dryRun, remote) — was
  only {engine, remote: true} before.
- F3 fix: validateParams runs on HTTP path — was skipped before.
- Engine.kind fail-fast: clear error message on PGLite (access_tokens table is
  Postgres-only by design).
- CORS: default-deny via GBRAIN_HTTP_CORS_ORIGIN allowlist.
- Body cap: stream-counted via req.body reader, catches chunked transfers
  without Content-Length. Default 1 MiB via GBRAIN_HTTP_MAX_BODY_BYTES.
- Rate limit: pre-auth IP bucket fires BEFORE DB lookup (limits brute-force
  load), post-auth token-id bucket fires after auth (limits runaway clients).
  Both bounded LRU with TTL prune.
- mcp_request_log: per-request audit row reusing the existing schema (v4).
- last_used_at SQL-level debounce: WHERE last_used_at < now() - interval
  '60 seconds'. Race-tolerant under PgBouncer.
- Response shape: application/json (gbrain MCP tools don't stream).
  Streamable-HTTP transport spec compliant for non-streaming responses.
- X-Forwarded-For honored only when GBRAIN_HTTP_TRUST_PROXY=1.

* feat: wire gbrain auth into the main CLI

The original PR's docs referenced 'gbrain auth create/list/revoke' but auth.ts
was a standalone script never wired to the CLI dispatcher. Running 'gbrain auth'
from the compiled binary returned 'Unknown command'.

- auth.ts: extract the dispatch into runAuth(args) + import.meta.main guard
  so direct-script invocation still works (bun run src/commands/auth.ts ...).
- cli.ts: add 'auth' to CLI_ONLY set + handler in handleCliOnly that imports
  runAuth and dispatches without requiring an engine connection (auth.ts
  manages its own postgres() connection).

* test: HTTP transport unit + E2E coverage (23 + 8 cases)

test/http-transport.test.ts — 23 unit cases against mocked engine.sql:
  - Auth: valid/missing/no-Bearer/unknown/revoked/health-bypass (1-6)
  - F1+F2 round-trip via dispatch.ts (7) — regression guard for reversed args
  - F3 invalid_params via validateParams (8) — regression guard
  - Response Content-Type application/json, not SSE (9)
  - CORS default-deny + allowlist + non-match (10-12)
  - Body cap: Content-Length + chunked-transfer (13-14)
  - Rate limit: refill, exhaust+Retry-After, LRU eviction, TTL prune,
    pre-auth IP fires before DB, /health bypasses (15-20)
  - mcp_request_log audit: success row + auth_failed row (21-22)

test/e2e/http-transport.test.ts — 8 cases against real Postgres:
  - /health, tools/list, tools/call list_pages (real op round-trip),
    revoked → 401, last_used_at debounce within 60s (asserts ONE update),
    debounce 65s gap (asserts TWO updates), mcp_request_log row check,
    invalid_params via real handler.

* docs: v0.22.7 CHANGELOG + SECURITY.md + DEPLOY.md

CHANGELOG: v0.22.7 release notes covering the F1-F3 dispatch fixes, the full
hardening surface (CORS default-deny, two-bucket rate limit, body cap, audit
log), and the upgrade path. Master's v0.22.6 schema-verify entry stitched in
above (preserving merge ordering).

SECURITY.md: full hardening reference for gbrain serve --http — Postgres-only
caveat, CORS allowlist, rate limit + tunnel caveat, body cap, audit log query,
GBRAIN_HTTP_TRUST_PROXY warning.

docs/mcp/DEPLOY.md: Postgres-only call-out, env var summary, fail-fast behavior
on PGLite.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: codex review follow-ups (DB-probing /health + XFF trust safety contract)

- /health now does SELECT 1 against Postgres and returns 503 + status:unhealthy
  when the DB is unreachable. Prevents the failure mode where orchestration
  sees green pods while clients get misleading 401s during a DB outage.
- SECURITY.md: tighten the GBRAIN_HTTP_TRUST_PROXY=1 guidance with the explicit
  two-condition safety contract — gbrain bound to a private interface AND the
  proxy strips client-supplied XFF. Without both, the flag enables IP spoofing
  past the pre-auth rate limit.
- Tests: add 6b (/health DB-down → 503) + assert db:'ok' on the happy path.

Caught by codex adversarial review during /ship Step 11.

* docs: TODOS.md — v0.22.7 follow-ups (audit volume, validateParams enums, SSE, scopes)

* docs: update project documentation for v0.22.7

CLAUDE.md: document src/mcp/dispatch.ts, src/mcp/rate-limit.ts, and the
rewritten src/mcp/http-transport.ts in the Key files section. Add
test/http-transport.test.ts (23 unit cases) and test/e2e/http-transport.test.ts
(8 E2E cases) to the test inventories.

CHANGELOG.md: fix copy-paste version mismatches inside the v0.22.7 entry that
referenced v0.22.5 (header line + "To take advantage of" block).

README.md: replace the standalone bun-run auth invocation with the wired-in
gbrain auth CLI; add gbrain serve --http startup step to the Remote MCP
example; surface gbrain auth in the admin command list; link SECURITY.md
from the Remote MCP section so it's discoverable.

SECURITY.md: align "as of v0.22.5" callouts with the actual release version
(v0.22.7).

docs/mcp/DEPLOY.md: align v0.22.5+ callout with v0.22.7+; switch token-management
examples from `bun run src/commands/auth.ts` to `gbrain auth` now that auth is
in the main CLI.

docs/mcp/ALTERNATIVES.md: drop the "planned but not yet implemented" note for
gbrain serve --http; document that the built-in HTTP transport is the
recommended path.

docs/mcp/{CLAUDE_DESKTOP,CLAUDE_COWORK,CLAUDE_CODE,PERPLEXITY}.md: switch
token-creation examples from `bun run src/commands/auth.ts create` to
`gbrain auth create` to match the wired-in CLI.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix: typecheck — cast CallToolRequestSchema handler return to any

MCP SDK 1.29 widened the response type for setRequestHandler(CallToolRequestSchema, ...)
to require a 'task' field for managed-task responses. gbrain ops are synchronous and
return the legacy { content, isError? } shape, which is still valid via the SDK's
ServerResult union. Casting the handler return type to any silences the narrowing
that broke after dispatch.ts was extracted (the original inline handler dodged this
because TypeScript inferred its return as any from the function body).

CI failure: src/mcp/server.ts(25,51): error TS2345 — Property 'task' is missing in
type 'ToolResult' but required in type '{ ...; task: { taskId: string; ... }; ... }'.
Caught by the 'test' job's bun run typecheck step at PR #483 commit 65ea9e7.

* docs: regenerate llms-full.txt after master merge

The build-llms regen-drift guard fails when committed llms.txt + llms-full.txt
don't match what scripts/build-llms.ts produces from current source. Master's
v0.22.6.1 merge brought in new content (CLAUDE.md entries, CHANGELOG, etc.)
that hadn't been folded into the bundle. Running 'bun run build:llms' to sync.

llms.txt unchanged; llms-full.txt picks up the new entries.

* docs: CHANGELOG — scrub attack-surface enumeration from v0.22.7 entry

Per CLAUDE.md responsible-disclosure rule: 'when a release fixes a security
gap or a user-impacting bug, describe the fix functionally. Do not enumerate
the attack surface, quantify the exposure window, or highlight the most
sensitive records by name in public-facing artifacts.'

Removed:
- Lead-paragraph attack-chain ('attacker who discovers URL → POST /register
  → client_credentials → read entire brain'). Public-doc readers don't need
  the directed probe path.
- 'Bug fixes folded in' section that itemized prior-version failure modes.
  Reframed as a 'transport refactor' note in the For Contributors section,
  describing the dispatch consolidation functionally without claiming the
  prior version was broken in specific ways.
- 'Without the OAuth footgun' lead headline. The fix's mechanism (built-in
  bearer auth via access_tokens) is already self-evident from the headline.
- F1/F2/F3 internal labels and 'caught by codex outside-voice during
  planning' parenthetical.

Kept:
- The full hardening reference table (configuration / behavior, not exposure).
- 'gbrain serve --http' user-facing operator ergonomics.
- 'Postgres-only by design' known-limit framing.
- Dispatch consolidation as a contributor-facing single-source-of-truth note.

SECURITY.md left intact: its OAuth-deployment guidance is generic 'if you
deploy MCP behind a custom HTTP wrapper, here are the rules' framing, not
gbrain-version-specific exposure. That's defensible under the same rule.

* docs: SECURITY.md — drop unverified security@garrytan.com address

The address was in the original PR's SECURITY.md commit (6e740590, author
'root <root@localhost>' — machine-generated) and never verified to exist or
forward anywhere. A non-monitored disclosure address is worse than no address
at all: reports go to a black hole.

Keep the GitHub private security advisory link as the sole disclosure channel.
GitHub Security Advisories is the working path most researchers reach for
first anyway — restricted-access by default, scopes the conversation to
maintainers, and integrates with CVE issuance when needed.

---------

Co-authored-by: root <root@localhost>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-28 17:01:40 -07:00

234 lines
8.8 KiB
TypeScript

/**
* E2E tests for src/mcp/http-transport.ts against real Postgres.
*
* Catches schema drift (column-name typos that would slip past the unit suite's
* stubbed engine.sql) and proves the F1+F2+F3 dispatch pipeline works against a
* real handler doing real DB work. Also exercises the SQL-level last_used_at
* debounce against real Postgres semantics.
*
* Run: DATABASE_URL=... bun test test/e2e/http-transport.test.ts
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { createHash, randomBytes } from 'crypto';
import { startHttpTransport } from '../../src/mcp/http-transport.ts';
import { hasDatabase, setupDB, teardownDB, getEngine, getConn } from './helpers.ts';
const skip = !hasDatabase();
const describeE2E = skip ? describe.skip : describe;
if (skip) {
console.log('Skipping E2E http-transport tests (DATABASE_URL not set)');
}
interface ServerHandle {
port: number;
stop: () => Promise<void>;
}
function generateToken(): string {
return 'gbrain_test_' + randomBytes(16).toString('hex');
}
function hashToken(token: string): string {
return createHash('sha256').update(token).digest('hex');
}
async function startServer(): Promise<ServerHandle> {
const engine = getEngine();
const server = await startHttpTransport({ port: 0, engine: engine as any });
return {
port: (server as any).port,
stop: async () => { (server as any).stop(true); },
};
}
function rpc(method: string, params?: unknown, id: number = 1) {
return JSON.stringify({ jsonrpc: '2.0', id, method, ...(params !== undefined ? { params } : {}) });
}
describeE2E('http-transport E2E (real Postgres)', () => {
let srv: ServerHandle;
let validToken: string;
let revokedToken: string;
let validTokenName: string;
beforeAll(async () => {
await setupDB();
const conn = getConn();
// Seed a valid + revoked token directly via SQL (mirrors auth.ts's create path).
validToken = generateToken();
validTokenName = 'e2e-valid-' + randomBytes(4).toString('hex');
await conn.unsafe(
'INSERT INTO access_tokens (name, token_hash) VALUES ($1, $2)',
[validTokenName, hashToken(validToken)],
);
revokedToken = generateToken();
await conn.unsafe(
'INSERT INTO access_tokens (name, token_hash, revoked_at) VALUES ($1, $2, now())',
['e2e-revoked-' + randomBytes(4).toString('hex'), hashToken(revokedToken)],
);
srv = await startServer();
});
afterAll(async () => {
if (srv) await srv.stop();
await teardownDB();
});
test('1. /health → 200 with expected JSON shape', async () => {
const r = await fetch(`http://localhost:${srv.port}/health`);
expect(r.status).toBe(200);
const body = await r.json();
expect(body.status).toBe('ok');
expect(body.transport).toBe('http');
expect(body.version).toBeString();
});
test('2. /mcp tools/list with valid Bearer → 200 + ops list', async () => {
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
expect(r.status).toBe(200);
const body = await r.json();
expect(body.result.tools).toBeArray();
expect(body.result.tools.length).toBeGreaterThan(5);
expect(r.headers.get('content-type')).toContain('application/json');
});
test('3. /mcp tools/call (real op: list_pages) round-trips successfully — F1+F2+F3 guard', async () => {
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/call', { name: 'list_pages', arguments: { limit: 5 } }),
});
expect(r.status).toBe(200);
const body = await r.json();
expect(body.jsonrpc).toBe('2.0');
expect(body.result.content).toBeArray();
// Should NOT be an error — handler ran successfully against the real engine.
expect(body.result.isError).toBeUndefined();
// Result text should parse as JSON (list_pages returns an object/array)
const resultText = body.result.content[0].text;
const parsed = JSON.parse(resultText);
expect(parsed).toBeDefined();
});
test('4. revoked token → 401', async () => {
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${revokedToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
expect(r.status).toBe(401);
});
test('5. last_used_at debounce: two consecutive valid calls → only one UPDATE within 60s', async () => {
const conn = getConn();
// Reset last_used_at to NULL so the first call definitely updates
await conn.unsafe('UPDATE access_tokens SET last_used_at = NULL WHERE name = $1', [validTokenName]);
// First request — should update last_used_at
await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
// Give the fire-and-forget UPDATE a moment to land
await new Promise(r => setTimeout(r, 50));
const [row1] = await conn.unsafe(
'SELECT last_used_at FROM access_tokens WHERE name = $1',
[validTokenName],
) as { last_used_at: Date | null }[];
expect(row1.last_used_at).not.toBeNull();
const firstUpdate = row1.last_used_at;
// Second request immediately — should NOT trigger another UPDATE (debounced by SQL WHERE)
await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
await new Promise(r => setTimeout(r, 50));
const [row2] = await conn.unsafe(
'SELECT last_used_at FROM access_tokens WHERE name = $1',
[validTokenName],
) as { last_used_at: Date | null }[];
// Same timestamp = same UPDATE = debounce held
expect(row2.last_used_at?.getTime()).toBe(firstUpdate?.getTime());
});
test('6. last_used_at debounce: simulating 65s gap → second request DOES update', async () => {
const conn = getConn();
// Set last_used_at to 65 seconds ago — simulates the time gap without waiting in real time
await conn.unsafe(
`UPDATE access_tokens SET last_used_at = now() - interval '65 seconds' WHERE name = $1`,
[validTokenName],
);
const [before] = await conn.unsafe(
'SELECT last_used_at FROM access_tokens WHERE name = $1',
[validTokenName],
) as { last_used_at: Date | null }[];
await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
await new Promise(r => setTimeout(r, 50));
const [after] = await conn.unsafe(
'SELECT last_used_at FROM access_tokens WHERE name = $1',
[validTokenName],
) as { last_used_at: Date | null }[];
expect(after.last_used_at?.getTime()).toBeGreaterThan(before.last_used_at!.getTime());
});
test('7. mcp_request_log gets a row per request', async () => {
const conn = getConn();
const beforeRows = await conn.unsafe('SELECT count(*)::int AS n FROM mcp_request_log') as { n: number }[];
const beforeN = beforeRows[0].n;
await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/list'),
});
// Fire-and-forget audit insert — give it a tick
await new Promise(r => setTimeout(r, 100));
const afterRows = await conn.unsafe('SELECT count(*)::int AS n FROM mcp_request_log') as { n: number }[];
expect(afterRows[0].n).toBeGreaterThan(beforeN);
const [row] = await conn.unsafe(
`SELECT token_name, operation, status, latency_ms FROM mcp_request_log
WHERE token_name = $1 ORDER BY created_at DESC LIMIT 1`,
[validTokenName],
) as { token_name: string; operation: string; status: string; latency_ms: number }[];
expect(row.token_name).toBe(validTokenName);
expect(row.operation).toBe('tools/list');
expect(row.status).toBe('success');
expect(row.latency_ms).toBeGreaterThanOrEqual(0);
});
test('8. tools/call with malformed params → isError result with invalid_params', async () => {
const r = await fetch(`http://localhost:${srv.port}/mcp`, {
method: 'POST',
headers: { 'Authorization': `Bearer ${validToken}`, 'Content-Type': 'application/json' },
body: rpc('tools/call', { name: 'get_page', arguments: { slug: 42 } }),
});
expect(r.status).toBe(200);
const body = await r.json();
expect(body.result.isError).toBe(true);
expect(body.result.content[0].text).toContain('invalid_params');
});
});