mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat(jobs): niceness core, worker registry, shared supervisor-pid reader OS scheduling-priority primitives for issue #1815: - niceness.ts: parseNiceValue (whole-string), applyNiceness (re-reads effective in success AND failure paths), getEffectiveNiceness, formatNice. - worker-registry.ts: live workers self-register pid + requested/effective nice under gbrainPath('workers'); readWorkers prunes ESRCH (keeps EPERM) with a pid-reuse start-time guard. - supervisor-pid.ts: readSupervisorPid extracted from the copy-pasted PID-file + liveness block. * feat(jobs): --nice flag for jobs work/supervisor + doctor niceness check Wires the --nice <n> flag (and GBRAIN_NICE env) through the CLI (issue #1815): - jobs work: applies niceness + registers the worker; cleanup on finally and process.on('exit'). - jobs supervisor: applies in the foreground-start path only (after the --detach fork), passes the apply result into MinionSupervisor. - supervisor.ts: nice opts, extracted testable buildWorkerArgs (appends --nice), emits niceness on started/worker_spawned audit events. - jobs stats / supervisor status: surface effective worker + supervisor nice. - doctor: separate supervisor_niceness check (warns on requested != effective) so it can't clobber the supervisor crash-check precedence; registered in doctor-categories. * test(jobs): cover niceness, worker registry, supervisor-pid, build args Unit tests for issue #1815: parseNiceValue rejects 3.5/10abc that parseInt would accept; applyNiceness re-reads effective on EPERM; registry ESRCH/EPERM + pid-reuse guard + brain-isolated path; readSupervisorPid states; parseNiceFlag flag>env precedence; buildWorkerArgs --nice propagation. * chore: bump version and changelog (v0.42.23.0) --nice flag for jobs work/supervisor (issue #1815). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: document --nice flag for jobs work/supervisor (v0.42.23.0) - minions-deployment.md: niceness tuning section (full concurrency, low priority). - KEY_FILES.md: entries for niceness.ts, worker-registry.ts, supervisor-pid.ts; supervisor.ts entry notes buildWorkerArgs + nice opts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): add enrich_thin to dream cycle EXPECTED_PHASES The enrich_thin cycle phase (src/core/cycle.ts ALL_PHASES, between conversation_facts_backfill and skillopt) shipped without updating the e2e phase-order expectation, so dream-cycle-phase-order-pglite failed on master. Sync the expected list to the real ALL_PHASES order. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): align sync-lock-recovery with the shipped --break-lock --all contract v0.41.13.0 intentionally dropped the "--break-lock + --all is refused" guard so cron can self-heal every source in one call (sync.ts runBreakLock iterates sources under --all). The e2e test still asserted the old exit-1 refusal and failed on master. Assert the current contract: the combination is accepted and takes the iterate / no-active-sources path (exit 0, no refusal message). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): de-flake ingestion-roundtrip chokidar first-drop race The native fsevents watcher occasionally missed a freshly written file, timing out the 15s waitFor (~1/3 on master under load). Three fixes: - inject a polling chokidar watcher via the source's _watchFactory seam (usePolling, 20ms interval) so detection never depends on fsevents timing; - drop deterministic fixtures BEFORE start so the initial scan (ignoreInitial:false) emits them, keeping live-watch coverage only where it's robust; - poll for the dedup hit instead of a fixed 600ms sleep. 15/15 green under stress. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): make hermetic-PGLite serve tests actually hermetic connect-bearer and serve-stdio-roundtrip init a PGLite brain and spawn serve, but passed {...process.env} through — leaking an ambient DATABASE_URL / GBRAIN_DATABASE_URL into the subprocess, which then came up on Postgres and failed the `engine: pglite` assertion. Strip both DB vars from the spawned env so the tests are deterministic whether or not the shell/CI has a DB URL set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(e2e): type the hermetic-PGLite env so tsc passes The DATABASE_URL/GBRAIN_DATABASE_URL strip used `delete` on a narrowly-typed env literal (tsc-only failure; bun test doesn't typecheck). Annotate connect-bearer's env as Record<string,string|undefined> and build serve-stdio's as a concrete Record<string,string> (StdioClientTransport.env rejects undefined). Runtime behavior unchanged (7/7 + 3/3 green). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test: quarantine worker-registry to *.serial (R1 env-mutation isolation) worker-registry.test.ts sets process.env.GBRAIN_HOME per-test so gbrainPath resolves to a temp dir, then lazy-imports the module — a process-global mutation the parallel isolation lint (rule R1) forbids. Rename to worker-registry.serial.test.ts: it runs in the serial pass (own bun process, max-concurrency=1) where env mutation is safe, and the lint skips *.serial files. No logic change (6/6 green); fixes the failing `verify` CI job. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
214 lines
10 KiB
TypeScript
214 lines
10 KiB
TypeScript
/**
|
|
* E2E for `gbrain connect`'s D4 raw-bearer smoke probe (connect-probe.ts).
|
|
*
|
|
* Spins up a real `gbrain serve --http` against a hermetic PGLite brain (no
|
|
* Postgres / Docker), mints a legacy bearer token via `gbrain auth create`,
|
|
* then drives the real MCP SDK probe against `/mcp`:
|
|
* - real token → ok, returns get_brain_identity payload
|
|
* - wrong token → not ok, reason 'auth'
|
|
* - unreachable → not ok, reason 'unreachable' | 'timeout'
|
|
*
|
|
* This is the integration coverage the unit tests (injected deps) can't give:
|
|
* the actual StreamableHTTP initialize handshake + tools/call over bearer auth.
|
|
*/
|
|
|
|
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
|
import { spawn, spawnSync, execFileSync, type ChildProcess } from 'child_process';
|
|
import { mkdtempSync, rmSync } from 'fs';
|
|
import { tmpdir } from 'os';
|
|
import { join } from 'path';
|
|
import { probeBrainIdentity } from '../../src/core/connect-probe.ts';
|
|
import { discoverOAuth, mintClientCredentialsToken } from '../../src/core/remote-mcp-probe.ts';
|
|
|
|
const PORT = 19735; // avoid the production 3131 + the oauth E2E's 19131
|
|
const BASE = `http://127.0.0.1:${PORT}`;
|
|
const MCP_URL = `${BASE}/mcp`;
|
|
|
|
describe('connect bearer probe E2E (PGLite + real serve --http)', () => {
|
|
let home: string;
|
|
let server: ChildProcess | null = null;
|
|
let token = '';
|
|
let oauthClientId = '';
|
|
let oauthClientSecret = '';
|
|
let serverReady = false;
|
|
|
|
beforeAll(async () => {
|
|
home = mkdtempSync(join(tmpdir(), 'gbrain-connect-e2e-'));
|
|
// Hermetic PGLite: strip any ambient Postgres URL so `init --pglite` and the
|
|
// spawned `serve` actually use PGLite. Without this, a dev/CI shell with
|
|
// DATABASE_URL set leaks it into the subprocess and the brain comes up on
|
|
// Postgres, breaking the `engine: pglite` assertion.
|
|
const env: Record<string, string | undefined> = { ...process.env, GBRAIN_HOME: home };
|
|
delete env.DATABASE_URL;
|
|
delete env.GBRAIN_DATABASE_URL;
|
|
|
|
execFileSync('bun', ['run', 'src/cli.ts', 'init', '--pglite', '--no-embedding', '--non-interactive'], {
|
|
cwd: process.cwd(), env, stdio: 'ignore',
|
|
});
|
|
const authOut = execFileSync('bun', ['run', 'src/cli.ts', 'auth', 'create', 'e2e-connect'], {
|
|
cwd: process.cwd(), env, encoding: 'utf8',
|
|
});
|
|
token = (authOut.match(/gbrain_[a-f0-9]{64}/) ?? [''])[0];
|
|
if (!token) throw new Error(`auth create did not yield a token:\n${authOut}`);
|
|
|
|
// Register the OAuth client BEFORE spawning serve — PGLite is single-writer,
|
|
// so register-client can't open the brain once the server holds it.
|
|
const regOut = execFileSync('bun', [
|
|
'run', 'src/cli.ts', 'auth', 'register-client', 'e2e-perplexity-oauth',
|
|
'--grant-types', 'client_credentials', '--scopes', 'read write',
|
|
'--token-endpoint-auth-method', 'client_secret_post',
|
|
], { cwd: process.cwd(), env, encoding: 'utf8' });
|
|
oauthClientId = (regOut.match(/Client ID:\s+(\S+)/) ?? ['', ''])[1];
|
|
oauthClientSecret = (regOut.match(/Client Secret:\s+(\S+)/) ?? ['', ''])[1];
|
|
if (!oauthClientId || !oauthClientSecret) throw new Error(`register-client did not yield creds:\n${regOut}`);
|
|
|
|
server = spawn('bun', [
|
|
'run', 'src/cli.ts', 'serve', '--http',
|
|
'--bind', '127.0.0.1', '--port', String(PORT),
|
|
'--public-url', BASE,
|
|
], { cwd: process.cwd(), env, stdio: ['ignore', 'pipe', 'pipe'] });
|
|
let serr = '';
|
|
server.stderr?.on('data', (d: Buffer) => { serr += d.toString(); });
|
|
|
|
for (let i = 0; i < 60; i++) {
|
|
try {
|
|
const res = await fetch(`${BASE}/health`);
|
|
if (res.ok) { serverReady = true; break; }
|
|
} catch { /* not up yet */ }
|
|
await new Promise((r) => setTimeout(r, 500));
|
|
}
|
|
if (!serverReady) throw new Error(`serve --http did not become ready:\n${serr}`);
|
|
}, 60_000);
|
|
|
|
afterAll(() => {
|
|
if (server) { try { server.kill('SIGTERM'); } catch { /* best-effort */ } }
|
|
if (home) { try { rmSync(home, { recursive: true, force: true }); } catch { /* best-effort */ } }
|
|
});
|
|
|
|
test('real bearer token round-trips get_brain_identity', async () => {
|
|
expect(serverReady).toBe(true);
|
|
const r = await probeBrainIdentity(MCP_URL, token, { timeoutMs: 15_000 });
|
|
expect(r.ok).toBe(true);
|
|
if (r.ok) {
|
|
// get_brain_identity returns the version/engine counter packet.
|
|
expect(r.identity).toMatch(/version/);
|
|
expect(r.identity).toMatch(/pglite/);
|
|
}
|
|
}, 30_000);
|
|
|
|
test('wrong token classifies as auth', async () => {
|
|
expect(serverReady).toBe(true);
|
|
const r = await probeBrainIdentity(MCP_URL, 'gbrain_deadbeef', { timeoutMs: 15_000 });
|
|
expect(r.ok).toBe(false);
|
|
if (!r.ok) expect(r.reason).toBe('auth');
|
|
}, 30_000);
|
|
|
|
test('unreachable host classifies as unreachable or timeout', async () => {
|
|
// 127.0.0.1:1 is reserved/closed — connection refused or fast timeout.
|
|
const r = await probeBrainIdentity('http://127.0.0.1:1/mcp', token, { timeoutMs: 4_000 });
|
|
expect(r.ok).toBe(false);
|
|
if (!r.ok) expect(['unreachable', 'timeout']).toContain(r.reason);
|
|
}, 15_000);
|
|
|
|
// -------------------------------------------------------------------------
|
|
// Real-CLI coverage: drive the actual `claude` / `codex` binaries through
|
|
// `gbrain connect --install` against the live server. Sandboxed via HOME /
|
|
// CODEX_HOME so the dev machine's real agent config is untouched. Skips
|
|
// gracefully when a binary isn't on PATH (e.g. CI without the CLIs).
|
|
// -------------------------------------------------------------------------
|
|
|
|
const hasBin = (b: string): boolean => {
|
|
try { execFileSync(b, ['--version'], { stdio: 'ignore' }); return true; } catch { return false; }
|
|
};
|
|
const HAS_CLAUDE = hasBin('claude');
|
|
const HAS_CODEX = hasBin('codex');
|
|
|
|
// Run `gbrain connect <args>` as a subprocess with extra env (HOME/CODEX_HOME
|
|
// sandbox + GBRAIN_REMOTE_TOKEN). spawnSync captures stderr too — connect's
|
|
// "Verified" / "Added" lines go to stderr.
|
|
const runConnectCli = (args: string[], extraEnv: Record<string, string>): { code: number; out: string } => {
|
|
const r = spawnSync('bun', ['run', 'src/cli.ts', 'connect', ...args], {
|
|
cwd: process.cwd(),
|
|
encoding: 'utf8',
|
|
env: { ...process.env, GBRAIN_HOME: home, ...extraEnv },
|
|
});
|
|
return { code: r.status ?? 1, out: `${r.stdout ?? ''}\n${r.stderr ?? ''}` };
|
|
};
|
|
|
|
(HAS_CLAUDE ? test : test.skip)('claude-code --install registers + connects against the live server', () => {
|
|
expect(serverReady).toBe(true);
|
|
const claudeHome = mkdtempSync(join(tmpdir(), 'gb-claude-'));
|
|
try {
|
|
const r = runConnectCli([MCP_URL, '--token', token, '--install', '--yes'], { HOME: claudeHome });
|
|
expect(r.code).toBe(0);
|
|
expect(r.out).toMatch(/Verified/);
|
|
// The real `claude` CLI actually registered the server.
|
|
const got = spawnSync('claude', ['mcp', 'get', 'gbrain'], { encoding: 'utf8', env: { ...process.env, HOME: claudeHome } });
|
|
expect(got.status).toBe(0);
|
|
expect(`${got.stdout ?? ''}${got.stderr ?? ''}`).toContain(`:${PORT}/mcp`);
|
|
} finally {
|
|
try { spawnSync('claude', ['mcp', 'remove', 'gbrain'], { env: { ...process.env, HOME: claudeHome } }); } catch { /* best-effort */ }
|
|
rmSync(claudeHome, { recursive: true, force: true });
|
|
}
|
|
}, 60_000);
|
|
|
|
(HAS_CODEX ? test : test.skip)('codex --install registers the env-var bearer against the live server', () => {
|
|
expect(serverReady).toBe(true);
|
|
const codexHome = mkdtempSync(join(tmpdir(), 'gb-codex-'));
|
|
try {
|
|
const r = runConnectCli([MCP_URL, '--token', token, '--agent', 'codex', '--install', '--yes'], { CODEX_HOME: codexHome, GBRAIN_REMOTE_TOKEN: token });
|
|
expect(r.code).toBe(0);
|
|
expect(r.out).toMatch(/Verified/);
|
|
// The real `codex` CLI registered the streamable-http server with the
|
|
// env-var bearer — and the token never lands in Codex config.
|
|
const got = spawnSync('codex', ['mcp', 'get', 'gbrain'], { encoding: 'utf8', env: { ...process.env, CODEX_HOME: codexHome } });
|
|
expect(got.status).toBe(0);
|
|
const text = `${got.stdout ?? ''}${got.stderr ?? ''}`;
|
|
expect(text).toContain('GBRAIN_REMOTE_TOKEN');
|
|
expect(text).not.toContain(token);
|
|
expect(text).toContain(`:${PORT}/mcp`);
|
|
} finally {
|
|
rmSync(codexHome, { recursive: true, force: true });
|
|
}
|
|
}, 60_000);
|
|
|
|
// Perplexity Computer is a GUI connector (Settings → Connectors, Pro account):
|
|
// there is no CLI to wire E2E. We can only assert `connect` prints the exact
|
|
// values the user pastes into the GUI.
|
|
test('perplexity print mode yields the GUI connector values (no CLI to wire E2E)', () => {
|
|
expect(serverReady).toBe(true);
|
|
const r = runConnectCli([MCP_URL, '--token', token, '--agent', 'perplexity'], {});
|
|
expect(r.code).toBe(0);
|
|
expect(r.out).toContain(`:${PORT}/mcp`);
|
|
expect(r.out).toContain(token); // print mode shows the token to paste into the connector
|
|
expect(r.out).toMatch(/Settings.+Connectors/);
|
|
}, 30_000);
|
|
|
|
// The OAuth path Perplexity actually uses, proven end-to-end against the live
|
|
// server: register a client → connect --oauth formats it → mint a real
|
|
// client-credentials access token via OAuth discovery + /token → call
|
|
// get_brain_identity with that token. This exercises the whole chain a
|
|
// Perplexity OAuth connector walks.
|
|
test('perplexity OAuth: connect --oauth → mint client-credentials token → tool call', async () => {
|
|
expect(serverReady).toBe(true);
|
|
// The OAuth client was registered in beforeAll (before serve took the
|
|
// PGLite write lock). Here: format it, then walk the connector's flow.
|
|
// 1. `connect --oauth` formats the connector block with the right issuer.
|
|
const conn = runConnectCli([MCP_URL, '--agent', 'perplexity', '--oauth', '--client-id', oauthClientId, '--client-secret', oauthClientSecret], {});
|
|
expect(conn.code).toBe(0);
|
|
expect(conn.out).toContain(`Issuer URL: ${BASE}`);
|
|
expect(conn.out).toContain(`Client ID: ${oauthClientId}`);
|
|
|
|
// 2. Mint a real access token the way a connector does, then call a tool.
|
|
const disco = await discoverOAuth(BASE, { timeoutMs: 10_000 });
|
|
expect(disco.ok).toBe(true);
|
|
if (!disco.ok) return;
|
|
const minted = await mintClientCredentialsToken(disco.metadata.token_endpoint, oauthClientId, oauthClientSecret, { scope: 'read write', timeoutMs: 10_000 });
|
|
expect(minted.ok).toBe(true);
|
|
if (!minted.ok) return;
|
|
const probed = await probeBrainIdentity(MCP_URL, minted.token.access_token, { timeoutMs: 15_000 });
|
|
expect(probed.ok).toBe(true);
|
|
if (probed.ok) expect(probed.identity).toMatch(/version/);
|
|
}, 60_000);
|
|
});
|