mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
v0.22.16 feat: gbrain claw-test — end-to-end fresh-install friction harness (#522)
* feat: hermeticity migration — every $GBRAIN_HOME write site honors the env override
configDir() in src/core/config.ts already implemented $GBRAIN_HOME as a
parent-dir override (returns <override>/.gbrain), but ~12 consumers built paths
from os.homedir() directly and bypassed it. Critically, loadConfig/saveConfig
themselves used a private getConfigDir() that ignored the env. Fixed.
Migrated every write site to gbrainPath() — fail-improve, validator-lint, cycle
lock, shell-audit, backpressure-audit, sync-failures, integrity logs,
integrations heartbeat, init pglite path, migrate-engine manifest, import
checkpoint, v0_13_1 rollback, v0_14_0 host-work. Read-side host-detection in
init.ts (~/.claude / ~/.openclaw probes) intentionally NOT migrated; that's a
v1.1 follow-up under a separate $GBRAIN_HOST_HOME override.
Adds gbrainPath(...segments) sugar plus path validation: $GBRAIN_HOME must be
absolute and contain no '..' segments (throws GbrainHomeInvalidError).
test/gbrain-home-isolation.test.ts proves write-isolation across all migrated
sites. test/migrations-v0_14_0.test.ts updated to use $GBRAIN_HOME instead of
the old HOME-swap pattern.
Closes part of the claw-test E2E harness preconditions (D13 + D21).
* feat: gbrain friction {log,render,list,summary} — agent friction reporter
Append-only JSONL writer at $GBRAIN_HOME/friction/<run-id>.jsonl. Schema is a
flat extension of StructuredAgentError (D20), one envelope shape across both
agent-emitted entries and harness-wrapped command failures. Run-id resolves
from --run-id > $GBRAIN_FRICTION_RUN_ID > 'standalone'.
Subcommands stay ≤30 LOC each; core lives in src/core/friction.ts (writer +
reader + renderer + redactor). render --redact (default for md output) strips
\$HOME / \$CWD to placeholders so reports paste safely in PRs/issues.
Severity: confused | error | blocker | nit. Kind: friction | delight (D7) |
phase-marker | interrupted. Readers tolerate malformed lines (skip + warn).
40 unit tests; this is the channel the claw-test harness writes to and that
agents emit through during live-mode runs.
* feat: gbrain claw-test — end-to-end fresh-install friction harness
Two modes: scripted (CI gate, no agent) and --live (real agent subprocess).
Phases: setup → install_brain (gbrain init --pglite) → import (--no-embed) →
query → extract all --source fs → verify (gbrain doctor --json, asserts
status==='ok' and progress.jsonl phase coverage).
AgentRunner interface + registry — interface stays narrow (detect, invoke,
optional postInstallHook). v1 ships only OpenClawRunner; the registry pattern
lets v1.1 land hermes/codex as ~50-line additions without refactoring callers.
OpenClaw invocation: 'openclaw agent --local --agent <name> --message <brief>'
matching test/e2e/skills.test.ts (NOT --prompt-file, which doesn't exist).
transcript-capture: spawns child with piped stdio, async-drains via
fs.createWriteStream + 'drain' events so 256KB+ bursts don't stall the child
(D17 backpressure). Writes <run>/transcript.jsonl with schema_version + ts +
channel + byte_offset + bytes_b64. Friction entries' transcript_offset field
references byte offsets here so render --transcripts can resolve back.
progress-tail: parses gbrain's --progress-json events out of child stderr.
Phase verification asserts each scenario.expected_phases entry (dotted names
like import.files, extract.links_fs, doctor.db_checks) saw at least one event
from the actual command — proves the COMMAND ran, not that the agent obeyed
prompts.
seed-pglite: ~50 LOC SQL replay primitive for the upgrade-from-v0.18 scenario.
Existing migration helpers (test/e2e/helpers.ts) are Postgres-only; PGLite has
no equivalent. seedPglite opens a fresh PGLite, executes each statement
individually (errors name the failing one), then disconnects so gbrain init
can take over and walk forward.
53 unit tests covering registry selection, runner detection, multi-byte UTF-8
chunk-boundary safety, PIPE buffer drain, scenario load+validate, progress
event parsing, and SQL splitter.
* feat: claw-test scenario fixtures + friction-protocol skills convention
Two scenarios ship in v1 — fresh-install and upgrade-from-v0.18. Each is a
self-contained directory: brain/ (markdown pages), BRIEF.md (live-mode prompt),
expected.json (scripted-mode assertions), scenario.json (kind, expected_phases,
optional from_version + seed paths). Schema is owned by src/core/claw-test/
scenarios.ts.
upgrade-from-v0.18 ships scaffolded — seed/dump.sql is the v1.1 follow-up
(needs a real v0.18-shape PGLite dump; seed/README.md documents the gen
procedure). The harness gracefully no-ops the seed phase when dump.sql is
absent.
skills/_friction-protocol.md is a cross-cutting convention skill (like
_brain-filing-rules.md). Tells agents when to call gbrain friction log and how
to choose severity. Skills the claw-test exercises will gain a > Convention:
callout pointing here in a v1.1 sweep.
13 unit tests for the scenario loader + 'shipped scenarios load cleanly' for
both.
* feat: register gbrain claw-test + gbrain friction; CLAUDE.md + llms sync
Wires both commands into src/cli.ts CLI_ONLY allow-list and adds dispatch
in handleCliOnly so neither command requires a brain engine connection.
CLAUDE.md gains entries for src/commands/{friction,claw-test}.ts +
src/core/claw-test/ + skills/_friction-protocol.md, and a Commands section
listing all 8 new gbrain claw-test ... and gbrain friction ... invocations
with the v0.23 marker. Documents the GBRAIN_HOME write-isolation contract
and the v1 caveat (read-side host-fingerprint detection deferred to v1.1).
llms.txt + llms-full.txt regenerated via 'bun run build:llms' so the
committed generator-output gate passes.
test/e2e/claw-test.test.ts is the scripted-mode E2E. Builds a tiny shim that
delegates to 'bun run src/cli.ts' (NOT bun --compile, which doesn't bundle
PGLite's runtime assets), points the harness at it via GBRAIN_BIN_OVERRIDE,
runs --scenario fresh-install end-to-end. Asserts exit 0, zero error/blocker
friction. Includes a deliberate-break test that proves the friction signal
fires when a phase command rejects.
test/claw-test-cli.test.ts covers shipped-scenario load + agent registry +
OpenClawRunner detection (relative-path / .. / missing-bin guards) + the
GBRAIN_FRICTION_RUN_ID env handoff between harness and friction CLI.
Closes the v0.23 claw-test E2E feature.
* chore: bump version and changelog (v0.24.0)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tests): typecheck failures + spawnWithCapture timeout headroom in CI
Three CI fixes after PR #522 landed:
1. test/agent-runner.test.ts:89 — UnavailableRunner.invoke() returns
Promise<void> by default but the AgentRunner contract requires
Promise<InvokeResult>. Annotate the throw-only invoke explicitly so tsc
sees the contract is satisfied (the throw makes the body unreachable as
far as the return type is concerned).
2. test/seed-pglite.test.ts — bun:test signature is test(name, fn, timeoutMs:
number), not test(name, opts: {timeout}, fn). The {timeout: 30_000} object
form was a guess that tsc on bun 1.3.13 rejects. Move the 30s cap to the
trailing positional number arg on each PGLite-using test.
3. test/transcript-capture.test.ts — `spawnWithCapture > timeout fires
SIGTERM/SIGKILL` blew the 10s outer cap on the GitHub runner. Two fixes:
(a) use `exec sleep` so the child we spawn IS sleep — SIGTERM goes
directly to it, no `/bin/sh` fork-vs-exec process-group ambiguity that
could orphan the sleep and force the SIGKILL grace path. (b) bump outer
cap to 30s for headroom even when the runner is slow and SIGKILL after
the 5s grace is what actually ends the child.
* chore: rebump to v0.22.16 (next free 0.22.x patch slot per queue)
PR #506 claims v0.22.15, PR #521 claims v0.22.10, intermediate slots
(.11/.12/.13/.14) are claimed by other open PRs. v0.22.16 is the next
clean PATCH slot. v0.23.0 is claimed by PR #462 so MINOR isn't free.
This release fits the 0.22.x train; v0.23.0 lands when #462 ships.
Updates VERSION, package.json, CHANGELOG.md header, TODOS.md follow-up
labels. Code is unchanged.
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
17c3c43783
commit
83e55ffcdb
@@ -0,0 +1,97 @@
|
||||
/**
|
||||
* AgentRunner registry + selection tests. Proves the harness contract is
|
||||
* truly agent-agnostic via a fake-runner integration.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import {
|
||||
registerAgentRunner, resolveAgentRunner, listRegisteredAgents,
|
||||
_resetRegistryForTests,
|
||||
type AgentRunner, type DetectResult, type InvokeOpts, type InvokeResult, type TranscriptSink,
|
||||
} from '../src/core/claw-test/agent-runner.ts';
|
||||
|
||||
class FakeRunner implements AgentRunner {
|
||||
readonly name: string;
|
||||
invocations = 0;
|
||||
detected: DetectResult = { available: true, binPath: '/usr/bin/fake-agent' };
|
||||
|
||||
constructor(name: string) { this.name = name; }
|
||||
|
||||
async detect(): Promise<DetectResult> { return this.detected; }
|
||||
async invoke(_opts: InvokeOpts): Promise<InvokeResult> {
|
||||
this.invocations++;
|
||||
return { exitCode: 0, durationMs: 1 };
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
_resetRegistryForTests();
|
||||
});
|
||||
|
||||
describe('registry', () => {
|
||||
test('register + resolve roundtrips', () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const r = resolveAgentRunner('fake');
|
||||
expect(r.name).toBe('fake');
|
||||
});
|
||||
|
||||
test('resolve unknown agent throws with helpful list', () => {
|
||||
registerAgentRunner('alpha', () => new FakeRunner('alpha'));
|
||||
registerAgentRunner('beta', () => new FakeRunner('beta'));
|
||||
expect(() => resolveAgentRunner('gamma')).toThrow(/registered: alpha, beta/);
|
||||
});
|
||||
|
||||
test('listRegisteredAgents returns sorted names', () => {
|
||||
registerAgentRunner('zeta', () => new FakeRunner('zeta'));
|
||||
registerAgentRunner('alpha', () => new FakeRunner('alpha'));
|
||||
expect(listRegisteredAgents()).toEqual(['alpha', 'zeta']);
|
||||
});
|
||||
|
||||
test('factory pattern produces independent instances', () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const a = resolveAgentRunner('fake') as FakeRunner;
|
||||
const b = resolveAgentRunner('fake') as FakeRunner;
|
||||
expect(a).not.toBe(b);
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent-agnosticism guard', () => {
|
||||
test('a fake runner can satisfy the AgentRunner contract end-to-end', async () => {
|
||||
registerAgentRunner('fake', () => new FakeRunner('fake'));
|
||||
const runner = resolveAgentRunner('fake');
|
||||
|
||||
// The harness contract: detect → invoke. Nothing else.
|
||||
const detected = await runner.detect();
|
||||
expect(detected.available).toBe(true);
|
||||
expect(detected.binPath).toBe('/usr/bin/fake-agent');
|
||||
|
||||
let written = 0;
|
||||
const sink: TranscriptSink = {
|
||||
write: () => { written++; },
|
||||
nextOffset: () => 0,
|
||||
close: async () => { /* noop */ },
|
||||
};
|
||||
|
||||
const result = await runner.invoke({
|
||||
cwd: '/tmp',
|
||||
brief: 'hello',
|
||||
env: {},
|
||||
timeoutMs: 1000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
});
|
||||
|
||||
test('a runner reporting unavailable still satisfies the contract', async () => {
|
||||
class UnavailableRunner implements AgentRunner {
|
||||
name = 'gone';
|
||||
async detect() { return { available: false, reason: 'not installed' } as DetectResult; }
|
||||
async invoke(): Promise<InvokeResult> { throw new Error('should not be called'); }
|
||||
}
|
||||
registerAgentRunner('gone', () => new UnavailableRunner());
|
||||
const r = resolveAgentRunner('gone');
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toBe('not installed');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,165 @@
|
||||
/**
|
||||
* gbrain claw-test CLI dispatch tests.
|
||||
*
|
||||
* These tests exercise the harness's argument parsing, scenario loading,
|
||||
* agent registry resolution, and friction-report path. They do NOT spawn
|
||||
* real gbrain commands (no built binary in CI yet); the canonical scripted
|
||||
* E2E that walks `gbrain init → import → query → extract → verify` lives
|
||||
* in test/e2e/claw-test.test.ts and gates on a built binary.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, existsSync, readFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { runFriction } from '../src/commands/friction.ts';
|
||||
import { listScenarios, loadScenario } from '../src/core/claw-test/scenarios.ts';
|
||||
import {
|
||||
registerAgentRunner, resolveAgentRunner, listRegisteredAgents,
|
||||
_resetRegistryForTests,
|
||||
type AgentRunner, type DetectResult, type InvokeOpts, type InvokeResult,
|
||||
} from '../src/core/claw-test/agent-runner.ts';
|
||||
|
||||
let tmp: string;
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'claw-test-cli-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
_resetRegistryForTests();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('shipped scenarios are loadable', () => {
|
||||
test('default fixtures root contains both v1 scenarios', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const names = listScenarios();
|
||||
expect(names).toContain('fresh-install');
|
||||
expect(names).toContain('upgrade-from-v0.18');
|
||||
});
|
||||
|
||||
test('fresh-install has expected_phases', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const cfg = loadScenario('fresh-install');
|
||||
expect(cfg.expectedPhases).toContain('import.files');
|
||||
expect(cfg.expectedPhases).toContain('extract.links_fs');
|
||||
expect(cfg.expectedPhases).toContain('doctor.db_checks');
|
||||
});
|
||||
|
||||
test('upgrade-from-v0.18 declares from_version', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
const cfg = loadScenario('upgrade-from-v0.18');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
expect(cfg.seedRelative).toBe('seed');
|
||||
});
|
||||
});
|
||||
|
||||
describe('agent registry — fake-runner integration', () => {
|
||||
test('a fake runner can be registered, resolved, and detect/invoke called', async () => {
|
||||
let invokeCount = 0;
|
||||
class FakeRunner implements AgentRunner {
|
||||
readonly name = 'fake';
|
||||
async detect(): Promise<DetectResult> { return { available: true, binPath: '/usr/bin/fake' }; }
|
||||
async invoke(_opts: InvokeOpts): Promise<InvokeResult> {
|
||||
invokeCount++;
|
||||
return { exitCode: 0, durationMs: 1 };
|
||||
}
|
||||
}
|
||||
registerAgentRunner('fake', () => new FakeRunner());
|
||||
expect(listRegisteredAgents()).toContain('fake');
|
||||
|
||||
const r = resolveAgentRunner('fake');
|
||||
const detected = await r.detect();
|
||||
expect(detected.available).toBe(true);
|
||||
|
||||
const result = await r.invoke({
|
||||
cwd: tmp,
|
||||
brief: 'test',
|
||||
env: {},
|
||||
timeoutMs: 1000,
|
||||
transcriptSink: { write: () => {}, nextOffset: () => 0, close: async () => {} },
|
||||
});
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(invokeCount).toBe(1);
|
||||
});
|
||||
|
||||
test('resolveAgentRunner with unknown name throws with registered list', () => {
|
||||
registerAgentRunner('alpha', () => ({} as AgentRunner));
|
||||
expect(() => resolveAgentRunner('unknown')).toThrow(/registered: alpha/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('friction CLI integrates with harness run-id env', () => {
|
||||
test('GBRAIN_FRICTION_RUN_ID populates harness-style run-ids', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'claw-test-20260428-fake-abcd1234';
|
||||
try {
|
||||
const code = runFriction(['log', '--phase', 'install', '--message', 'simulated harness write']);
|
||||
expect(code).toBe(0);
|
||||
const expectedFile = join(tmp, '.gbrain', 'friction', 'claw-test-20260428-fake-abcd1234.jsonl');
|
||||
expect(existsSync(expectedFile)).toBe(true);
|
||||
const raw = readFileSync(expectedFile, 'utf-8');
|
||||
const entry = JSON.parse(raw.split('\n')[0]);
|
||||
expect(entry.run_id).toBe('claw-test-20260428-fake-abcd1234');
|
||||
expect(entry.message).toBe('simulated harness write');
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('OpenClawRunner detection (reliable on box without openclaw)', () => {
|
||||
test('detect returns unavailable when OPENCLAW_BIN missing', async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
delete process.env.OPENCLAW_BIN;
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
// Either unavailable, or available if openclaw IS on PATH for the dev — both states are valid.
|
||||
// We only assert the contract shape.
|
||||
expect(typeof d.available).toBe('boolean');
|
||||
if (!d.available) {
|
||||
expect(typeof d.reason).toBe('string');
|
||||
} else {
|
||||
expect(d.binPath?.startsWith('/')).toBe(true);
|
||||
}
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
}
|
||||
});
|
||||
|
||||
test('detect rejects relative OPENCLAW_BIN', async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
process.env.OPENCLAW_BIN = 'relative/openclaw';
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toMatch(/absolute/);
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
else delete process.env.OPENCLAW_BIN;
|
||||
}
|
||||
});
|
||||
|
||||
test("detect rejects '..' segments in OPENCLAW_BIN", async () => {
|
||||
const orig = process.env.OPENCLAW_BIN;
|
||||
process.env.OPENCLAW_BIN = '/tmp/foo/../bar';
|
||||
try {
|
||||
const { OpenClawRunner } = await import('../src/core/claw-test/runners/openclaw.ts');
|
||||
const r = new OpenClawRunner();
|
||||
const d = await r.detect();
|
||||
expect(d.available).toBe(false);
|
||||
expect(d.reason).toMatch(/'\.\.' segments/);
|
||||
} finally {
|
||||
if (orig !== undefined) process.env.OPENCLAW_BIN = orig;
|
||||
else delete process.env.OPENCLAW_BIN;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,134 @@
|
||||
/**
|
||||
* gbrain claw-test scripted-mode E2E.
|
||||
*
|
||||
* Invokes the harness via `bun run src/cli.ts` (NOT a compiled binary —
|
||||
* `bun build --compile` doesn't bundle PGLite's runtime assets like
|
||||
* pglite.data, so a compiled gbrain can't init a fresh PGLite brain).
|
||||
* Uses a tiny shim script that the harness can spawn as if it were the
|
||||
* gbrain binary.
|
||||
*
|
||||
* Asserts:
|
||||
* - exit code 0 on a clean tree
|
||||
* - the friction JSONL has zero error/blocker entries
|
||||
* - the harness recorded progress events for the expected phases
|
||||
*
|
||||
* Tagged-skip env: CLAW_TEST_SKIP_E2E=1 to opt out (e.g. when PGLite
|
||||
* WASM is broken on the host — the macOS 26.3 #223 bug class).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll } from 'bun:test';
|
||||
import { execFileSync, spawnSync } from 'child_process';
|
||||
import { mkdirSync, existsSync, mkdtempSync, rmSync, readFileSync, readdirSync, writeFileSync, chmodSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join, resolve } from 'path';
|
||||
|
||||
const REPO_ROOT = resolve(import.meta.dir, '..', '..');
|
||||
const BIN_CACHE = join(REPO_ROOT, 'test', '.cache');
|
||||
const BIN_PATH = join(BIN_CACHE, 'gbrain.sh');
|
||||
const SCENARIOS_DIR = join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios');
|
||||
|
||||
beforeAll(() => {
|
||||
if (!existsSync(BIN_CACHE)) mkdirSync(BIN_CACHE, { recursive: true });
|
||||
// Shim that delegates to `bun run src/cli.ts` so PGLite assets resolve from
|
||||
// the source tree (bun --compile doesn't bundle them). Marked executable so
|
||||
// child_process.spawn can run it directly.
|
||||
const shim = `#!/bin/sh\nexec bun run "${join(REPO_ROOT, 'src', 'cli.ts')}" "$@"\n`;
|
||||
writeFileSync(BIN_PATH, shim, 'utf-8');
|
||||
chmodSync(BIN_PATH, 0o755);
|
||||
}, 30_000);
|
||||
|
||||
describe('gbrain claw-test --scenario fresh-install (scripted)', () => {
|
||||
test('runs end-to-end clean and produces zero error/blocker friction', () => {
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-fresh-'));
|
||||
try {
|
||||
const result = spawnSync(BIN_PATH, ['claw-test', '--scenario', 'fresh-install', '--keep-tempdir'], {
|
||||
cwd: REPO_ROOT,
|
||||
env: {
|
||||
...process.env,
|
||||
GBRAIN_HOME: tmp,
|
||||
GBRAIN_BIN_OVERRIDE: BIN_PATH,
|
||||
GBRAIN_CLAW_SCENARIOS_DIR: join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios'),
|
||||
},
|
||||
encoding: 'utf-8',
|
||||
timeout: 120_000,
|
||||
});
|
||||
if (result.status !== 0) {
|
||||
console.error('STDOUT:', result.stdout);
|
||||
console.error('STDERR:', result.stderr);
|
||||
}
|
||||
expect(result.status).toBe(0);
|
||||
|
||||
// Inspect the friction JSONL the harness wrote.
|
||||
const frictionDir = join(tmp, '.gbrain', 'friction');
|
||||
expect(existsSync(frictionDir)).toBe(true);
|
||||
const files = readdirSync(frictionDir).filter(f => f.endsWith('.jsonl'));
|
||||
expect(files.length).toBeGreaterThan(0);
|
||||
const runFile = join(frictionDir, files[0]);
|
||||
const lines = readFileSync(runFile, 'utf-8').split('\n').filter(l => l.trim());
|
||||
const entries = lines.map(l => JSON.parse(l));
|
||||
const blockers = entries.filter(e => e.kind === 'friction' && (e.severity === 'error' || e.severity === 'blocker'));
|
||||
if (blockers.length > 0) {
|
||||
console.error('unexpected friction entries:', blockers);
|
||||
}
|
||||
expect(blockers.length).toBe(0);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 180_000);
|
||||
|
||||
test('break path: an invented command produces an error friction entry and exits non-zero', () => {
|
||||
// We do this by setting GBRAIN_BIN_OVERRIDE to a script that pretends to be gbrain
|
||||
// and rejects the `import` subcommand specifically.
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-break-'));
|
||||
const fakeBin = join(tmp, 'fake-gbrain');
|
||||
try {
|
||||
// Write a shim that delegates to real gbrain but rejects 'import' to simulate breakage.
|
||||
const shimContent = `#!/bin/sh\nif [ "$1" = "import" ]; then echo "fake import error" >&2; exit 17; fi\nexec "${BIN_PATH}" "$@"\n`;
|
||||
const { writeFileSync, chmodSync } = require('fs');
|
||||
writeFileSync(fakeBin, shimContent, 'utf-8');
|
||||
chmodSync(fakeBin, 0o755);
|
||||
|
||||
const result = spawnSync(BIN_PATH, ['claw-test', '--scenario', 'fresh-install', '--keep-tempdir'], {
|
||||
cwd: REPO_ROOT,
|
||||
env: {
|
||||
...process.env,
|
||||
GBRAIN_HOME: tmp,
|
||||
GBRAIN_BIN_OVERRIDE: fakeBin,
|
||||
GBRAIN_CLAW_SCENARIOS_DIR: join(REPO_ROOT, 'test', 'fixtures', 'claw-test-scenarios'),
|
||||
},
|
||||
encoding: 'utf-8',
|
||||
timeout: 60_000,
|
||||
});
|
||||
expect(result.status).not.toBe(0);
|
||||
|
||||
// The friction log should have an error-severity entry for the 'import' phase.
|
||||
const frictionDir = join(tmp, '.gbrain', 'friction');
|
||||
const files = readdirSync(frictionDir).filter(f => f.endsWith('.jsonl'));
|
||||
const lines = readFileSync(join(frictionDir, files[0]), 'utf-8').split('\n').filter(l => l.trim());
|
||||
const entries = lines.map(l => JSON.parse(l));
|
||||
const importErrors = entries.filter(e => e.phase === 'import' && e.severity === 'error');
|
||||
expect(importErrors.length).toBeGreaterThan(0);
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 90_000);
|
||||
});
|
||||
|
||||
describe('gbrain friction render integration', () => {
|
||||
test('render produces a markdown report with the redact placeholder', () => {
|
||||
const tmp = mkdtempSync(join(tmpdir(), 'claw-test-e2e-render-'));
|
||||
try {
|
||||
// Log a friction entry with $HOME embedded, then render --redact md
|
||||
const home = process.env.HOME ?? '/tmp';
|
||||
const env = { ...process.env, GBRAIN_HOME: tmp, GBRAIN_FRICTION_RUN_ID: 'render-e2e' };
|
||||
execFileSync(BIN_PATH, ['friction', 'log', '--phase', 'p', '--message', `error at ${home}/.gbrain/x`], { env, encoding: 'utf-8' });
|
||||
const out = execFileSync(BIN_PATH, ['friction', 'render', '--run-id', 'render-e2e'], { env, encoding: 'utf-8' });
|
||||
expect(out).toContain('# Friction report');
|
||||
expect(out).toContain('<HOME>');
|
||||
// --redact is the default for md, so home itself should not appear.
|
||||
expect(out).not.toContain(home + '/.gbrain');
|
||||
} finally {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
@@ -0,0 +1,32 @@
|
||||
# Claw-test brief — fresh-install
|
||||
|
||||
You are testing gbrain on a brand-new install. The user just ran `gbrain init` for the first time. Walk through the canonical first-day flow:
|
||||
|
||||
1. **Verify install:** confirm `gbrain --version` works and `gbrain doctor --json` returns a valid JSON object with a `status` field.
|
||||
2. **Install skillpack:** run `gbrain skillpack install --workspace $PWD`. The workspace already has an `AGENTS.md` routing file.
|
||||
3. **Import the brain:** run `gbrain import ./brain --no-embed --progress-json`. There are 3 small markdown pages already there.
|
||||
4. **Query the brain:** run `gbrain query "alice"` and verify >0 results.
|
||||
5. **Extract links:** run `gbrain extract --source fs --progress-json`.
|
||||
6. **Verify health:** run `gbrain doctor --json`. The `status` field should be `"ok"`.
|
||||
|
||||
## Friction protocol
|
||||
|
||||
If anything is confusing, missing, surprising, or wrong, run:
|
||||
|
||||
```
|
||||
gbrain friction log --severity {confused|error|blocker|nit} --phase <which-step> --message "<what-happened>" [--hint "<what-could-be-better>"]
|
||||
```
|
||||
|
||||
Severity guide:
|
||||
- `blocker` — couldn't proceed at all
|
||||
- `error` — command failed unexpectedly
|
||||
- `confused` — docs said one thing, the tool did another, or a step felt unclear
|
||||
- `nit` — minor polish opportunity
|
||||
|
||||
If something *just worked* and was nicer than expected, log a delight too:
|
||||
|
||||
```
|
||||
gbrain friction log --kind delight --phase <step> --message "<what-was-nice>"
|
||||
```
|
||||
|
||||
We want to know what didn't work, not just whether commands exited zero. Be specific.
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
---
|
||||
type: company
|
||||
name: Acme Example
|
||||
founded: 2024
|
||||
founders:
|
||||
- alice-example
|
||||
---
|
||||
|
||||
# Acme Example
|
||||
|
||||
Fictional company used for claw-test fixtures. Founded 2024 by [Alice](people/alice-example).
|
||||
|
||||
## What they do
|
||||
|
||||
Acme builds an agentic-workflow product on top of [retrieval-augmented-generation](concepts/retrieval-augmented-generation). Early traction comes from a developer-tools wedge.
|
||||
+10
@@ -0,0 +1,10 @@
|
||||
---
|
||||
type: concept
|
||||
name: Agentic Workflows
|
||||
---
|
||||
|
||||
# Agentic Workflows
|
||||
|
||||
Workflows where an LLM-driven agent plans, executes, and revises a sequence of steps with minimal human supervision per step. Key constraints: cost, latency, and observability of the loop.
|
||||
|
||||
Companies building in this space include [acme-example](companies/acme-example).
|
||||
+13
@@ -0,0 +1,13 @@
|
||||
---
|
||||
type: person
|
||||
name: Alice Example
|
||||
x_handle: alice_example
|
||||
---
|
||||
|
||||
# Alice Example
|
||||
|
||||
Alice is a fictional founder used for claw-test fixtures. She started [acme-example](companies/acme-example) in 2024.
|
||||
|
||||
## Background
|
||||
|
||||
Alice has spent 10 years in software and 2 years in AI tooling. She is exploring product-market fit for an [agentic-workflow](concepts/agentic-workflows) tool.
|
||||
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"min_pages_after_import": 3,
|
||||
"min_query_results": 1,
|
||||
"min_links_after_extract": 0,
|
||||
"doctor_status": "ok"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"kind": "fresh-install",
|
||||
"description": "Canonical 5-minute first-day flow: init → import → query → extract → verify",
|
||||
"expected_phases": [
|
||||
"import.files",
|
||||
"extract.links_fs",
|
||||
"doctor.db_checks"
|
||||
],
|
||||
"brain": "brain"
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
# Claw-test brief — upgrade-from-v0.18
|
||||
|
||||
You inherit a gbrain v0.18 brain (the harness has already replayed a seed SQL dump into a PGLite database). Walk through the upgrade path:
|
||||
|
||||
1. **Run `gbrain doctor --json`** first. Note any warnings or fix-hints.
|
||||
2. **Run `gbrain init --pglite`** with the existing database path. The migration chain should detect the old `schema_version` and walk forward to the latest.
|
||||
3. **Run `gbrain doctor --json` again.** The `status` field should be `"ok"`.
|
||||
4. **Verify queries still work:** `gbrain query "alice"` should return results from the seeded brain.
|
||||
|
||||
## Friction protocol
|
||||
|
||||
If anything is confusing, missing, surprising, or wrong (especially around the migration steps — these are the highest-historical-pain regression points), run:
|
||||
|
||||
```
|
||||
gbrain friction log --severity {confused|error|blocker|nit} --phase <which-step> --message "<what-happened>" [--hint "<what-could-be-better>"]
|
||||
```
|
||||
|
||||
Common upgrade-flow friction patterns to watch for:
|
||||
|
||||
- The migration chain failed at a specific schema version (capture the version + error)
|
||||
- Doctor flagged an issue but the fix-hint wasn't actionable
|
||||
- `gbrain init --pglite` didn't recognize the existing brain
|
||||
- Manual SQL was needed to unblock something
|
||||
|
||||
If something just worked, log a delight. We're tuning the upgrade flow toward zero-friction.
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
---
|
||||
type: person
|
||||
name: Alice Example
|
||||
---
|
||||
|
||||
# Alice Example
|
||||
|
||||
Same brain content as the fresh-install scenario; this scenario tests upgrade flow rather than ingest. After the migration chain walks forward, agents query and the page must be findable.
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"min_pages_after_migration": 1,
|
||||
"doctor_status": "ok"
|
||||
}
|
||||
@@ -0,0 +1,10 @@
|
||||
{
|
||||
"kind": "upgrade",
|
||||
"from_version": "0.18.0",
|
||||
"description": "Pre-v0.18 brain shape replayed via PGLite SQL dump; migration chain walks forward to LATEST",
|
||||
"expected_phases": [
|
||||
"doctor.db_checks"
|
||||
],
|
||||
"seed": "seed",
|
||||
"brain": "brain"
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
# v0.18 seed
|
||||
|
||||
This directory ships in v1 as **scaffolding only** — `dump.sql` will contain a real v0.18-shape PGLite SQL dump in v1.1. Until then the harness treats the absent dump as a no-op seed and the upgrade scenario behaves like a fresh-install scenario for the test gate.
|
||||
|
||||
## Generating a real v0.18 seed
|
||||
|
||||
To produce an authentic seed:
|
||||
|
||||
1. Check out gbrain at the v0.18 release (`git checkout v0.18.0`).
|
||||
2. Run `gbrain init --pglite --path /tmp/v0.18-seed.pglite` against a small fixture brain.
|
||||
3. Run `gbrain import <fixture-brain>` to populate it.
|
||||
4. Dump the PGLite as SQL: PGLite supports `pg_dump`-style export via the `executeRaw('SELECT * FROM pg_dump(...)')` extension or via direct file copy. If neither path works, run `pglite-tools dump /tmp/v0.18-seed.pglite > dump.sql`.
|
||||
5. Place `dump.sql` here.
|
||||
6. Update `expected.json::min_pages_after_migration` to match your dump's page count.
|
||||
|
||||
## What gets tested
|
||||
|
||||
When `dump.sql` exists, the harness:
|
||||
|
||||
- Runs `seedPgliteFromFile()` to replay the dump into a fresh `<tempdir>/.gbrain/brain.pglite`
|
||||
- Then runs `gbrain init --pglite` so the migration chain detects the old schema_version and walks forward to LATEST
|
||||
- Asserts `gbrain doctor --json` returns `status: 'ok'` after the walk
|
||||
|
||||
This is the regression gate for the upgrade-wedge bug class (#239/#243/#266/#357/#366/#374/#375/#378/#395/#396) — every gbrain release that adds a column-with-index in the embedded schema blob without a corresponding bootstrap retriggered the same wedge family.
|
||||
@@ -0,0 +1,196 @@
|
||||
/**
|
||||
* Friction CLI dispatch tests. Exercises the thin command layer (each
|
||||
* subcommand stays ≤ 30 LOC per the DRY contract from the eng review).
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, readFileSync, existsSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { runFriction } from '../src/commands/friction.ts';
|
||||
import { frictionFile, frictionDir } from '../src/core/friction.ts';
|
||||
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
const ORIG_RUN_ID = process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
let tmp: string;
|
||||
let stdoutLines: string[];
|
||||
let stderrLines: string[];
|
||||
let origStdoutWrite: typeof process.stdout.write;
|
||||
let origConsoleLog: typeof console.log;
|
||||
let origConsoleError: typeof console.error;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'friction-cli-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
stdoutLines = [];
|
||||
stderrLines = [];
|
||||
origStdoutWrite = process.stdout.write.bind(process.stdout);
|
||||
origConsoleLog = console.log;
|
||||
origConsoleError = console.error;
|
||||
process.stdout.write = ((chunk: string) => { stdoutLines.push(String(chunk)); return true; }) as any;
|
||||
console.log = (...args: unknown[]) => { stdoutLines.push(args.join(' ') + '\n'); };
|
||||
console.error = (...args: unknown[]) => { stderrLines.push(args.join(' ') + '\n'); };
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
if (ORIG_RUN_ID !== undefined) process.env.GBRAIN_FRICTION_RUN_ID = ORIG_RUN_ID;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
process.stdout.write = origStdoutWrite;
|
||||
console.log = origConsoleLog;
|
||||
console.error = origConsoleError;
|
||||
});
|
||||
|
||||
describe('dispatch', () => {
|
||||
test('--help returns 0 and prints subcommand list', () => {
|
||||
const code = runFriction(['--help']);
|
||||
expect(code).toBe(0);
|
||||
expect(stdoutLines.join('')).toContain('Subcommands');
|
||||
expect(stdoutLines.join('')).toContain('log');
|
||||
expect(stdoutLines.join('')).toContain('render');
|
||||
expect(stdoutLines.join('')).toContain('list');
|
||||
expect(stdoutLines.join('')).toContain('summary');
|
||||
});
|
||||
|
||||
test('unknown subcommand returns 2', () => {
|
||||
const code = runFriction(['nonsense']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('unknown subcommand');
|
||||
});
|
||||
});
|
||||
|
||||
describe('log subcommand', () => {
|
||||
test('writes a friction entry under GBRAIN_HOME', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-1', '--phase', 'install', '--message', 'something broke', '--severity', 'error']);
|
||||
expect(code).toBe(0);
|
||||
const path = frictionFile('cli-1');
|
||||
expect(existsSync(path)).toBe(true);
|
||||
expect(path.startsWith(tmp)).toBe(true);
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
expect(raw).toContain('something broke');
|
||||
});
|
||||
|
||||
test('missing --phase returns 2 with usage', () => {
|
||||
const code = runFriction(['log', '--message', 'foo']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('usage');
|
||||
});
|
||||
|
||||
test('missing --message returns 2 with usage', () => {
|
||||
const code = runFriction(['log', '--phase', 'p']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('usage');
|
||||
});
|
||||
|
||||
test('invalid --severity returns 2', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-2', '--phase', 'p', '--message', 'm', '--severity', 'panicking']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('invalid --severity');
|
||||
});
|
||||
|
||||
test('invalid --kind returns 2', () => {
|
||||
const code = runFriction(['log', '--run-id', 'cli-3', '--phase', 'p', '--message', 'm', '--kind', 'bogus']);
|
||||
expect(code).toBe(2);
|
||||
expect(stderrLines.join('')).toContain('invalid --kind');
|
||||
});
|
||||
|
||||
test('--kind delight is recorded', () => {
|
||||
runFriction(['log', '--run-id', 'cli-4', '--phase', 'p', '--message', 'great', '--kind', 'delight']);
|
||||
const raw = readFileSync(frictionFile('cli-4'), 'utf-8');
|
||||
expect(raw).toContain('"kind":"delight"');
|
||||
});
|
||||
});
|
||||
|
||||
describe('render subcommand', () => {
|
||||
test('renders markdown by default', () => {
|
||||
runFriction(['log', '--run-id', 'cli-r', '--phase', 'install', '--message', 'beep', '--severity', 'error']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['render', '--run-id', 'cli-r']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('# Friction report');
|
||||
expect(out).toContain('## error');
|
||||
});
|
||||
|
||||
test('--json emits parseable JSON', () => {
|
||||
runFriction(['log', '--run-id', 'cli-r2', '--phase', 'p', '--message', 'beep']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['render', '--run-id', 'cli-r2', '--json']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('').trim();
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.run_id).toBe('cli-r2');
|
||||
expect(parsed.entries.length).toBe(1);
|
||||
});
|
||||
|
||||
test('missing run-id returns 1 with actionable error', () => {
|
||||
const code = runFriction(['render', '--run-id', 'no-such-run']);
|
||||
expect(code).toBe(1);
|
||||
expect(stderrLines.join('')).toContain('not found');
|
||||
});
|
||||
});
|
||||
|
||||
describe('list subcommand', () => {
|
||||
test('reports no runs initially', () => {
|
||||
const code = runFriction(['list']);
|
||||
expect(code).toBe(0);
|
||||
expect(stdoutLines.join('')).toContain('no runs');
|
||||
});
|
||||
|
||||
test('lists logged runs with counts', () => {
|
||||
runFriction(['log', '--run-id', 'a', '--phase', 'p', '--message', 'm', '--severity', 'error']);
|
||||
runFriction(['log', '--run-id', 'b', '--phase', 'p', '--message', 'm', '--kind', 'delight']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['list']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('a');
|
||||
expect(out).toContain('b');
|
||||
});
|
||||
|
||||
test('--json emits parseable JSON array', () => {
|
||||
runFriction(['log', '--run-id', 'jl', '--phase', 'p', '--message', 'm']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['list', '--json']);
|
||||
expect(code).toBe(0);
|
||||
const parsed = JSON.parse(stdoutLines.join('').trim());
|
||||
expect(Array.isArray(parsed)).toBe(true);
|
||||
expect(parsed[0].runId).toBe('jl');
|
||||
});
|
||||
});
|
||||
|
||||
describe('summary subcommand', () => {
|
||||
test('renders friction + delight columns', () => {
|
||||
runFriction(['log', '--run-id', 'sum-1', '--phase', 'p', '--message', 'broken thing']);
|
||||
runFriction(['log', '--run-id', 'sum-1', '--phase', 'p', '--message', 'nice thing', '--kind', 'delight']);
|
||||
stdoutLines.length = 0;
|
||||
const code = runFriction(['summary', '--run-id', 'sum-1']);
|
||||
expect(code).toBe(0);
|
||||
const out = stdoutLines.join('');
|
||||
expect(out).toContain('friction (1)');
|
||||
expect(out).toContain('delight (1)');
|
||||
expect(out).toContain('broken thing');
|
||||
expect(out).toContain('nice thing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('GBRAIN_FRICTION_RUN_ID fallback (D19)', () => {
|
||||
test('log without --run-id uses standalone', () => {
|
||||
const code = runFriction(['log', '--phase', 'p', '--message', 'fallback']);
|
||||
expect(code).toBe(0);
|
||||
const path = frictionFile('standalone');
|
||||
expect(existsSync(path)).toBe(true);
|
||||
expect(readFileSync(path, 'utf-8')).toContain('fallback');
|
||||
});
|
||||
|
||||
test('log honors $GBRAIN_FRICTION_RUN_ID', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'env-run';
|
||||
try {
|
||||
runFriction(['log', '--phase', 'p', '--message', 'env']);
|
||||
expect(existsSync(frictionFile('env-run'))).toBe(true);
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,232 @@
|
||||
/**
|
||||
* Friction core: writer + reader + renderer + redactor.
|
||||
*
|
||||
* These tests are pure local-fs (no DB, no subprocess). They run under
|
||||
* GBRAIN_HOME=<tmp> for hermeticity — see test/gbrain-home-isolation.test.ts
|
||||
* for the regression gate proving every consumer honors that env.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, existsSync, readFileSync, appendFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import {
|
||||
logFriction, readFriction, listRuns, renderReport, renderSummary,
|
||||
redactEntry, frictionFile, frictionDir, activeRunId,
|
||||
type FrictionEntry,
|
||||
} from '../src/core/friction.ts';
|
||||
|
||||
const ORIG_HOME = process.env.GBRAIN_HOME;
|
||||
const ORIG_RUN_ID = process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
let tmp: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'friction-test-'));
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
process.env.GBRAIN_HOME = ORIG_HOME;
|
||||
if (ORIG_RUN_ID !== undefined) process.env.GBRAIN_FRICTION_RUN_ID = ORIG_RUN_ID;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('writer', () => {
|
||||
test('logFriction appends one JSONL line and roundtrips through reader', () => {
|
||||
logFriction({ runId: 'run-a', phase: 'install', message: 'first', severity: 'error' });
|
||||
const { entries, malformed } = readFriction('run-a');
|
||||
expect(malformed).toBe(0);
|
||||
expect(entries).toHaveLength(1);
|
||||
expect(entries[0].message).toBe('first');
|
||||
expect(entries[0].severity).toBe('error');
|
||||
expect(entries[0].kind).toBe('friction');
|
||||
expect(entries[0].schema_version).toBe('1');
|
||||
expect(entries[0].run_id).toBe('run-a');
|
||||
});
|
||||
|
||||
test('multiple entries append in order', () => {
|
||||
logFriction({ runId: 'run-b', phase: 'p1', message: 'one', severity: 'nit' });
|
||||
logFriction({ runId: 'run-b', phase: 'p2', message: 'two', severity: 'blocker' });
|
||||
const { entries } = readFriction('run-b');
|
||||
expect(entries.map(e => e.message)).toEqual(['one', 'two']);
|
||||
});
|
||||
|
||||
test('long messages are truncated', () => {
|
||||
const long = 'x'.repeat(5000);
|
||||
logFriction({ runId: 'run-c', phase: 'p', message: long });
|
||||
const { entries } = readFriction('run-c');
|
||||
expect(entries[0].message.length).toBeLessThan(5000);
|
||||
expect(entries[0].message.endsWith('[truncated]')).toBe(true);
|
||||
});
|
||||
|
||||
test('kind: delight is recorded distinctly', () => {
|
||||
logFriction({ runId: 'run-d', phase: 'verify', message: 'this just worked', kind: 'delight' });
|
||||
const { entries } = readFriction('run-d');
|
||||
expect(entries[0].kind).toBe('delight');
|
||||
});
|
||||
|
||||
test('phase-marker entry roundtrips', () => {
|
||||
logFriction({ runId: 'run-e', phase: 'extract', message: 'phase started', kind: 'phase-marker', marker: 'start' });
|
||||
const { entries } = readFriction('run-e');
|
||||
expect(entries[0].kind).toBe('phase-marker');
|
||||
expect(entries[0].marker).toBe('start');
|
||||
});
|
||||
|
||||
test('error envelope fields flatten in (D20)', () => {
|
||||
logFriction({
|
||||
runId: 'run-f',
|
||||
phase: 'install',
|
||||
message: 'spawn failed',
|
||||
severity: 'blocker',
|
||||
errorClass: 'AgentSpawnError',
|
||||
errorCode: 'spawn_enoent',
|
||||
docsUrl: 'https://example.test/docs',
|
||||
});
|
||||
const { entries } = readFriction('run-f');
|
||||
expect(entries[0].class).toBe('AgentSpawnError');
|
||||
expect(entries[0].code).toBe('spawn_enoent');
|
||||
expect(entries[0].docs_url).toBe('https://example.test/docs');
|
||||
});
|
||||
|
||||
test('rejects invalid run-id', () => {
|
||||
expect(() => logFriction({ runId: 'has space', phase: 'p', message: 'm' })).toThrow(/invalid run-id/);
|
||||
expect(() => logFriction({ runId: '../escape', phase: 'p', message: 'm' })).toThrow(/invalid run-id/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('activeRunId', () => {
|
||||
test('falls back to standalone when env unset (D19)', () => {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
expect(activeRunId()).toBe('standalone');
|
||||
});
|
||||
|
||||
test('reads GBRAIN_FRICTION_RUN_ID', () => {
|
||||
process.env.GBRAIN_FRICTION_RUN_ID = 'my-run';
|
||||
try {
|
||||
expect(activeRunId()).toBe('my-run');
|
||||
} finally {
|
||||
delete process.env.GBRAIN_FRICTION_RUN_ID;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('reader', () => {
|
||||
test('skips malformed lines and counts them', () => {
|
||||
logFriction({ runId: 'run-g', phase: 'p', message: 'good' });
|
||||
appendFileSync(frictionFile('run-g'), 'this is not json\n', 'utf-8');
|
||||
appendFileSync(frictionFile('run-g'), '{"ts":"only","kind":"friction"}\n', 'utf-8');
|
||||
logFriction({ runId: 'run-g', phase: 'p', message: 'good2' });
|
||||
const { entries, malformed } = readFriction('run-g');
|
||||
expect(entries).toHaveLength(2);
|
||||
expect(malformed).toBe(2);
|
||||
});
|
||||
|
||||
test('throws on missing run-id', () => {
|
||||
expect(() => readFriction('does-not-exist')).toThrow(/not found/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('listRuns', () => {
|
||||
test('lists runs sorted most-recent-first', () => {
|
||||
logFriction({ runId: 'old-run', phase: 'p', message: 'a' });
|
||||
// Sleep one millisecond worth via busy-wait so mtime differs reliably
|
||||
const t0 = Date.now();
|
||||
while (Date.now() - t0 < 10) { /* spin */ }
|
||||
logFriction({ runId: 'new-run', phase: 'p', message: 'b' });
|
||||
const runs = listRuns();
|
||||
expect(runs.length).toBe(2);
|
||||
expect(runs[0].runId).toBe('new-run');
|
||||
expect(runs[1].runId).toBe('old-run');
|
||||
});
|
||||
|
||||
test('reports per-run counts and interrupted flag', () => {
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'a', severity: 'error' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'b', severity: 'error' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'c', kind: 'delight' });
|
||||
logFriction({ runId: 'run-h', phase: 'p', message: 'killed', kind: 'interrupted' });
|
||||
const runs = listRuns();
|
||||
const r = runs.find(x => x.runId === 'run-h')!;
|
||||
expect(r.counts.friction).toBe(2);
|
||||
expect(r.counts.delight).toBe(1);
|
||||
expect(r.counts.interrupted).toBe(true);
|
||||
expect(r.counts.bySeverity.error).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('renderer', () => {
|
||||
test('markdown groups by severity then phase', () => {
|
||||
logFriction({ runId: 'run-r', phase: 'install', message: 'a', severity: 'blocker' });
|
||||
logFriction({ runId: 'run-r', phase: 'install', message: 'b', severity: 'error' });
|
||||
logFriction({ runId: 'run-r', phase: 'verify', message: 'c', severity: 'error' });
|
||||
logFriction({ runId: 'run-r', phase: 'verify', message: 'positive', kind: 'delight' });
|
||||
const md = renderReport('run-r', { format: 'md', redact: false });
|
||||
expect(md).toContain('# Friction report');
|
||||
expect(md).toContain('## blocker');
|
||||
expect(md).toContain('## error');
|
||||
expect(md).toContain('### `install`');
|
||||
expect(md).toContain('### `verify`');
|
||||
// Blocker section comes before error section
|
||||
expect(md.indexOf('## blocker')).toBeLessThan(md.indexOf('## error'));
|
||||
});
|
||||
|
||||
test('json output is valid and includes entries', () => {
|
||||
logFriction({ runId: 'run-j', phase: 'p', message: 'one' });
|
||||
const out = renderReport('run-j', { format: 'json' });
|
||||
const parsed = JSON.parse(out);
|
||||
expect(parsed.run_id).toBe('run-j');
|
||||
expect(parsed.entries).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('redact strips homedir and cwd from message + cwd field', () => {
|
||||
const home = process.env.HOME ?? '';
|
||||
const fakeCwd = process.cwd();
|
||||
logFriction({
|
||||
runId: 'run-red',
|
||||
phase: 'p',
|
||||
message: `error at ${home}/.gbrain/foo and ${fakeCwd}/bar.ts`,
|
||||
});
|
||||
const md = renderReport('run-red', { format: 'md', redact: true });
|
||||
expect(md).not.toContain(home + '/.gbrain');
|
||||
expect(md).toContain('<HOME>');
|
||||
expect(md).toContain('<CWD>');
|
||||
});
|
||||
|
||||
test('--no-redact path preserves homedir', () => {
|
||||
const home = process.env.HOME ?? '/tmp/none';
|
||||
logFriction({ runId: 'run-noredact', phase: 'p', message: `at ${home}/foo` });
|
||||
const md = renderReport('run-noredact', { format: 'md', redact: false });
|
||||
expect(md).toContain(home);
|
||||
});
|
||||
|
||||
test('interrupted run shows banner', () => {
|
||||
logFriction({ runId: 'run-i', phase: 'p', message: 'partial' });
|
||||
logFriction({ runId: 'run-i', phase: 'p', message: 'killed', kind: 'interrupted' });
|
||||
const md = renderReport('run-i', { format: 'md', redact: false });
|
||||
expect(md).toContain('Run was interrupted');
|
||||
});
|
||||
});
|
||||
|
||||
describe('summary', () => {
|
||||
test('two columns, friction + delight side-by-side', () => {
|
||||
logFriction({ runId: 'run-s', phase: 'p', message: 'bad-thing' });
|
||||
logFriction({ runId: 'run-s', phase: 'p', message: 'good-thing', kind: 'delight' });
|
||||
const md = renderSummary('run-s', { format: 'md' });
|
||||
expect(md).toContain('| friction (1) | delight (1) |');
|
||||
expect(md).toContain('bad-thing');
|
||||
expect(md).toContain('good-thing');
|
||||
});
|
||||
});
|
||||
|
||||
describe('redactEntry pure function', () => {
|
||||
test('replaces homedir occurrences', () => {
|
||||
const home = process.env.HOME ?? '/x';
|
||||
const e: FrictionEntry = {
|
||||
schema_version: '1', ts: 'now', run_id: 'r', phase: 'p', kind: 'friction',
|
||||
message: `${home}/secret/file.txt`, source: 'claw', cwd: '/cwd', gbrain_version: 'test',
|
||||
};
|
||||
const r = redactEntry(e);
|
||||
expect(r.message).toContain('<HOME>');
|
||||
expect(r.cwd).toBe('<CWD>');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,141 @@
|
||||
/**
|
||||
* Hermeticity test: every site that writes under `~/.gbrain` must honor
|
||||
* `GBRAIN_HOME=<tmp>` and write under `<tmp>/.gbrain` instead of the developer's
|
||||
* real home.
|
||||
*
|
||||
* Why this exists: `src/core/config.ts::configDir()` already supports
|
||||
* `GBRAIN_HOME` as a parent-dir override (returns `<override>/.gbrain`), but
|
||||
* historically many call sites built paths from `os.homedir()` directly,
|
||||
* bypassing the override. The hermeticity migration migrated every write-side
|
||||
* caller to `gbrainPath(...)`. This test is the regression gate.
|
||||
*
|
||||
* Scope: write-isolation only. Read-side host detection in
|
||||
* `src/commands/init.ts` (reading `~/.claude`, `~/.openclaw`, etc. for module
|
||||
* fingerprinting) is the documented v1 caveat and is NOT asserted here.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { mkdtempSync, existsSync, readdirSync, statSync, rmSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
|
||||
// Save original env so we don't leak between tests.
|
||||
const ORIG_GBRAIN_HOME = process.env.GBRAIN_HOME;
|
||||
|
||||
function fresh(): string {
|
||||
return mkdtempSync(join(tmpdir(), 'gbrain-home-isolation-'));
|
||||
}
|
||||
|
||||
describe('GBRAIN_HOME write-side isolation', () => {
|
||||
test('configDir() returns <GBRAIN_HOME>/.gbrain when override is set', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { configDir, gbrainPath } = await import('../src/core/config.ts');
|
||||
expect(configDir()).toBe(join(tmp, '.gbrain'));
|
||||
expect(gbrainPath('foo', 'bar.json')).toBe(join(tmp, '.gbrain', 'foo', 'bar.json'));
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('configDir() falls back to homedir when GBRAIN_HOME unset', async () => {
|
||||
delete process.env.GBRAIN_HOME;
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
const result = configDir();
|
||||
// Should NOT contain the test tmpdir; should resolve to a real homedir path.
|
||||
expect(result.endsWith('.gbrain')).toBe(true);
|
||||
expect(result.startsWith('/tmp/')).toBe(false);
|
||||
} finally {
|
||||
if (ORIG_GBRAIN_HOME !== undefined) process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test('rejects relative GBRAIN_HOME', async () => {
|
||||
process.env.GBRAIN_HOME = 'relative/path';
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
expect(() => configDir()).toThrow(/absolute path/);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test("rejects GBRAIN_HOME containing '..' segments", async () => {
|
||||
process.env.GBRAIN_HOME = '/tmp/foo/../bar';
|
||||
try {
|
||||
const { configDir } = await import('../src/core/config.ts');
|
||||
expect(() => configDir()).toThrow(/'\.\.' segments/);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
}
|
||||
});
|
||||
|
||||
test('saveConfig/loadConfig honor GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { saveConfig, loadConfig } = await import('../src/core/config.ts');
|
||||
const cfg = { engine: 'pglite' as const, database_path: join(tmp, '.gbrain', 'brain.pglite') };
|
||||
saveConfig(cfg);
|
||||
// Config file should exist under the override, NOT under real ~/.gbrain.
|
||||
expect(existsSync(join(tmp, '.gbrain', 'config.json'))).toBe(true);
|
||||
|
||||
// Round-trip: loadConfig() finds it back via the override.
|
||||
const loaded = loadConfig();
|
||||
expect(loaded?.engine).toBe('pglite');
|
||||
expect(loaded?.database_path).toBe(cfg.database_path);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('integrity, sync-failures, integrations heartbeat resolve under GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
try {
|
||||
const { gbrainPath } = await import('../src/core/config.ts');
|
||||
// Spot-check a representative set of paths used across the migrated sites.
|
||||
const paths = [
|
||||
gbrainPath('integrity-review.md'), // src/commands/integrity.ts
|
||||
gbrainPath('sync-failures.jsonl'), // src/core/sync.ts
|
||||
gbrainPath('integrations', 'recipe-x'), // src/commands/integrations.ts
|
||||
gbrainPath('migrate-manifest.json'), // src/commands/migrate-engine.ts
|
||||
gbrainPath('import-checkpoint.json'), // src/commands/import.ts
|
||||
gbrainPath('migrations', 'v0_13_1-rollback.jsonl'), // src/commands/migrations/v0_13_1.ts
|
||||
gbrainPath('migrations', 'pending-host-work.jsonl'), // src/commands/migrations/v0_14_0.ts
|
||||
gbrainPath('audit'), // shell-audit / backpressure-audit
|
||||
gbrainPath('cycle.lock'), // src/core/cycle.ts
|
||||
gbrainPath('fail-improve'), // src/core/fail-improve.ts
|
||||
gbrainPath('validator-lint.jsonl'), // src/core/output/post-write.ts
|
||||
gbrainPath('brain.pglite'), // init pglite default
|
||||
];
|
||||
for (const p of paths) {
|
||||
expect(p.startsWith(join(tmp, '.gbrain'))).toBe(true);
|
||||
}
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('GBRAIN_AUDIT_DIR override still wins over GBRAIN_HOME', async () => {
|
||||
const tmp = fresh();
|
||||
const auditTmp = fresh();
|
||||
process.env.GBRAIN_HOME = tmp;
|
||||
process.env.GBRAIN_AUDIT_DIR = auditTmp;
|
||||
try {
|
||||
const { resolveAuditDir } = await import('../src/core/minions/handlers/shell-audit.ts');
|
||||
// Per the docstring: GBRAIN_AUDIT_DIR is the explicit override and wins.
|
||||
expect(resolveAuditDir()).toBe(auditTmp);
|
||||
} finally {
|
||||
process.env.GBRAIN_HOME = ORIG_GBRAIN_HOME;
|
||||
delete process.env.GBRAIN_AUDIT_DIR;
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
rmSync(auditTmp, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -16,16 +16,17 @@ import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
let tmpHome: string;
|
||||
const originalHome = process.env.HOME;
|
||||
const originalGbrainHome = process.env.GBRAIN_HOME;
|
||||
|
||||
beforeEach(() => {
|
||||
tmpHome = mkdtempSync(join(tmpdir(), 'gbrain-v0_14_0-'));
|
||||
process.env.HOME = tmpHome;
|
||||
// GBRAIN_HOME is the parent dir; configDir() appends '.gbrain' itself.
|
||||
process.env.GBRAIN_HOME = tmpHome;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (originalHome) process.env.HOME = originalHome;
|
||||
else delete process.env.HOME;
|
||||
if (originalGbrainHome !== undefined) process.env.GBRAIN_HOME = originalGbrainHome;
|
||||
else delete process.env.GBRAIN_HOME;
|
||||
try { rmSync(tmpHome, { recursive: true, force: true }); } catch { /* ignore */ }
|
||||
});
|
||||
|
||||
|
||||
@@ -0,0 +1,81 @@
|
||||
/**
|
||||
* progress-tail tests — parse --progress-json events out of mixed stderr.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { parseProgressEvents, eventsByPhase, verifyExpectedPhases } from '../src/core/claw-test/progress-tail.ts';
|
||||
|
||||
describe('parseProgressEvents', () => {
|
||||
test('extracts JSON event lines from mixed stderr', () => {
|
||||
const stderr = [
|
||||
'starting up',
|
||||
'{"phase":"import.files","event":"start"}',
|
||||
'warning: deprecated flag X',
|
||||
'{"phase":"import.files","event":"tick","done":3,"total":10}',
|
||||
'random text',
|
||||
'{"phase":"import.files","event":"finish"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events).toHaveLength(3);
|
||||
expect(events.map(e => e.event)).toEqual(['start', 'tick', 'finish']);
|
||||
});
|
||||
|
||||
test('ignores malformed JSON lines silently', () => {
|
||||
const stderr = [
|
||||
'{"phase":"a","event":"start"}',
|
||||
'{"phase":', // truncated JSON
|
||||
'not json at all',
|
||||
'{"phase":"b","event":"start"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('ignores objects without phase field', () => {
|
||||
const stderr = [
|
||||
'{"phase":"a","event":"start"}',
|
||||
'{"foo":"bar"}',
|
||||
'{"phase":"b","event":"start"}',
|
||||
].join('\n');
|
||||
const events = parseProgressEvents(stderr);
|
||||
expect(events.map(e => e.phase)).toEqual(['a', 'b']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('eventsByPhase', () => {
|
||||
test('groups by phase name', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files', event: 'start' },
|
||||
{ phase: 'import.files', event: 'finish' },
|
||||
{ phase: 'extract.links_fs', event: 'start' },
|
||||
];
|
||||
const grouped = eventsByPhase(events);
|
||||
expect(grouped.get('import.files')).toHaveLength(2);
|
||||
expect(grouped.get('extract.links_fs')).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('verifyExpectedPhases', () => {
|
||||
test('returns empty when all expected phases present', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files' },
|
||||
{ phase: 'extract.links_fs' },
|
||||
{ phase: 'doctor.db_checks' },
|
||||
];
|
||||
const missing = verifyExpectedPhases(events, ['import.files', 'doctor.db_checks']);
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
|
||||
test('returns missing phase names when some are absent', () => {
|
||||
const events = [
|
||||
{ phase: 'import.files' },
|
||||
];
|
||||
const missing = verifyExpectedPhases(events, ['import.files', 'extract.links_fs', 'doctor.db_checks']);
|
||||
expect(missing).toEqual(['extract.links_fs', 'doctor.db_checks']);
|
||||
});
|
||||
|
||||
test('returns full expected list when no events at all', () => {
|
||||
const missing = verifyExpectedPhases([], ['a', 'b']);
|
||||
expect(missing).toEqual(['a', 'b']);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,132 @@
|
||||
/**
|
||||
* Scenario loader tests — proves scenario.json parsing + validation work.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, mkdirSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { listScenarios, loadScenario, readBrief } from '../src/core/claw-test/scenarios.ts';
|
||||
|
||||
const ORIG_ROOT = process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
let root: string;
|
||||
|
||||
beforeEach(() => {
|
||||
root = mkdtempSync(join(tmpdir(), 'scenarios-'));
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
if (ORIG_ROOT !== undefined) process.env.GBRAIN_CLAW_SCENARIOS_DIR = ORIG_ROOT;
|
||||
else delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
rmSync(root, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
function scaffoldScenario(name: string, scenarioJson: string, briefContent = '# Brief'): void {
|
||||
const dir = join(root, name);
|
||||
mkdirSync(dir, { recursive: true });
|
||||
writeFileSync(join(dir, 'scenario.json'), scenarioJson);
|
||||
writeFileSync(join(dir, 'BRIEF.md'), briefContent);
|
||||
}
|
||||
|
||||
describe('listScenarios', () => {
|
||||
test('returns empty when no scenarios exist', () => {
|
||||
expect(listScenarios()).toEqual([]);
|
||||
});
|
||||
|
||||
test('returns directories that contain scenario.json, sorted', () => {
|
||||
scaffoldScenario('beta', '{"kind":"fresh-install","expected_phases":[]}');
|
||||
scaffoldScenario('alpha', '{"kind":"fresh-install","expected_phases":[]}');
|
||||
mkdirSync(join(root, 'incomplete'), { recursive: true }); // no scenario.json
|
||||
expect(listScenarios()).toEqual(['alpha', 'beta']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('loadScenario', () => {
|
||||
test('parses a valid fresh-install scenario', () => {
|
||||
scaffoldScenario('demo', JSON.stringify({
|
||||
kind: 'fresh-install',
|
||||
expected_phases: ['import.files', 'doctor.db_checks'],
|
||||
description: 'demo',
|
||||
brain: 'brain',
|
||||
}));
|
||||
const cfg = loadScenario('demo');
|
||||
expect(cfg.name).toBe('demo');
|
||||
expect(cfg.kind).toBe('fresh-install');
|
||||
expect(cfg.expectedPhases).toEqual(['import.files', 'doctor.db_checks']);
|
||||
expect(cfg.description).toBe('demo');
|
||||
expect(cfg.brainRelative).toBe('brain');
|
||||
});
|
||||
|
||||
test('parses an upgrade scenario with from_version + seed', () => {
|
||||
scaffoldScenario('upgrade-x', JSON.stringify({
|
||||
kind: 'upgrade',
|
||||
from_version: '0.18.0',
|
||||
expected_phases: ['doctor.db_checks'],
|
||||
seed: 'seed',
|
||||
}));
|
||||
mkdirSync(join(root, 'upgrade-x', 'seed'), { recursive: true });
|
||||
const cfg = loadScenario('upgrade-x');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
expect(cfg.seedRelative).toBe('seed');
|
||||
});
|
||||
|
||||
test('throws on missing scenario directory', () => {
|
||||
expect(() => loadScenario('does-not-exist')).toThrow(/not found/);
|
||||
});
|
||||
|
||||
test('throws on malformed JSON', () => {
|
||||
scaffoldScenario('bad', 'not json {');
|
||||
expect(() => loadScenario('bad')).toThrow(/malformed/);
|
||||
});
|
||||
|
||||
test('throws on unknown kind', () => {
|
||||
scaffoldScenario('weird', JSON.stringify({ kind: 'mystery', expected_phases: [] }));
|
||||
expect(() => loadScenario('weird')).toThrow(/unknown kind/);
|
||||
});
|
||||
|
||||
test('throws on non-array expected_phases', () => {
|
||||
scaffoldScenario('bad-phases', JSON.stringify({ kind: 'fresh-install', expected_phases: 'oops' }));
|
||||
expect(() => loadScenario('bad-phases')).toThrow(/expected_phases/);
|
||||
});
|
||||
|
||||
test('throws when BRIEF.md missing', () => {
|
||||
const dir = join(root, 'no-brief');
|
||||
mkdirSync(dir);
|
||||
writeFileSync(join(dir, 'scenario.json'), JSON.stringify({ kind: 'fresh-install', expected_phases: [] }));
|
||||
expect(() => loadScenario('no-brief')).toThrow(/BRIEF\.md missing/);
|
||||
});
|
||||
});
|
||||
|
||||
describe('readBrief', () => {
|
||||
test('returns BRIEF.md content', () => {
|
||||
scaffoldScenario('reads-brief', '{"kind":"fresh-install","expected_phases":[]}', '# Hello world');
|
||||
const cfg = loadScenario('reads-brief');
|
||||
expect(readBrief(cfg)).toBe('# Hello world');
|
||||
});
|
||||
});
|
||||
|
||||
describe('shipped scenarios load cleanly', () => {
|
||||
test('fresh-install loads from default fixtures root', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
try {
|
||||
const cfg = loadScenario('fresh-install');
|
||||
expect(cfg.kind).toBe('fresh-install');
|
||||
expect(cfg.expectedPhases.length).toBeGreaterThan(0);
|
||||
} finally {
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
}
|
||||
});
|
||||
|
||||
test('upgrade-from-v0.18 loads from default fixtures root', () => {
|
||||
delete process.env.GBRAIN_CLAW_SCENARIOS_DIR;
|
||||
try {
|
||||
const cfg = loadScenario('upgrade-from-v0.18');
|
||||
expect(cfg.kind).toBe('upgrade');
|
||||
expect(cfg.fromVersion).toBe('0.18.0');
|
||||
} finally {
|
||||
process.env.GBRAIN_CLAW_SCENARIOS_DIR = root;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,130 @@
|
||||
/**
|
||||
* seed-pglite tests — exercises the SQL replay primitive that powers the
|
||||
* upgrade-from-v0.18 scenario. Pure PGLite in-memory; no real DB needed.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { seedPglite, seedPgliteFromFile, _internal } from '../src/core/claw-test/seed-pglite.ts';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
|
||||
let tmp: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'seed-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('splitStatements', () => {
|
||||
const split = _internal.splitStatements;
|
||||
|
||||
test('splits on semicolons', () => {
|
||||
expect(split('CREATE TABLE a(x int); INSERT INTO a VALUES (1);').length).toBe(2);
|
||||
});
|
||||
|
||||
test('respects single-quoted strings', () => {
|
||||
const sql = "INSERT INTO t VALUES ('a;b'); INSERT INTO t VALUES ('c');";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(2);
|
||||
expect(stmts[0]).toContain("'a;b'");
|
||||
});
|
||||
|
||||
test('respects -- line comments', () => {
|
||||
const sql = "-- a comment with ; semicolon\nCREATE TABLE x(id int);";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(1);
|
||||
});
|
||||
|
||||
test('handles escaped quotes (doubled apostrophe)', () => {
|
||||
const sql = "INSERT INTO t VALUES ('it''s ok');";
|
||||
const stmts = split(sql);
|
||||
expect(stmts.length).toBe(1);
|
||||
expect(stmts[0]).toContain("it''s ok");
|
||||
});
|
||||
|
||||
test('returns empty list for empty input', () => {
|
||||
expect(split('').length).toBe(0);
|
||||
expect(split(' \n').length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('seedPglite', () => {
|
||||
test('replays a SQL dump into a fresh PGLite database', async () => {
|
||||
const dbPath = join(tmp, 'brain.pglite');
|
||||
const sql = `
|
||||
CREATE TABLE seeded(id INT PRIMARY KEY, name TEXT);
|
||||
INSERT INTO seeded(id, name) VALUES (1, 'alice');
|
||||
INSERT INTO seeded(id, name) VALUES (2, 'bob');
|
||||
`;
|
||||
await seedPglite({ dbPath, sql });
|
||||
|
||||
// Re-open the seeded database and verify content survived.
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const rows: any = await (engine as any).db.query('SELECT id, name FROM seeded ORDER BY id');
|
||||
expect(rows.rows).toEqual([
|
||||
{ id: 1, name: 'alice' },
|
||||
{ id: 2, name: 'bob' },
|
||||
]);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('throws with a useful message when SQL is invalid', async () => {
|
||||
const dbPath = join(tmp, 'bad.pglite');
|
||||
const sql = 'INVALID SQL HERE;';
|
||||
await expect(seedPglite({ dbPath, sql })).rejects.toThrow(/SQL execution failed/);
|
||||
}, 30_000);
|
||||
|
||||
test('creates parent directories when needed', async () => {
|
||||
const dbPath = join(tmp, 'nested', 'deeper', 'brain.pglite');
|
||||
await seedPglite({ dbPath, sql: 'CREATE TABLE x(y int);' });
|
||||
// No throw means the dir was created.
|
||||
expect(true).toBe(true);
|
||||
}, 30_000);
|
||||
|
||||
test('empty SQL is a no-op (just creates the .pglite)', async () => {
|
||||
const dbPath = join(tmp, 'empty.pglite');
|
||||
await seedPglite({ dbPath, sql: '' });
|
||||
// Verify the database is openable but empty.
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const r: any = await (engine as any).db.query("SELECT COUNT(*)::int AS c FROM information_schema.tables WHERE table_schema='public'");
|
||||
expect(r.rows[0].c).toBe(0);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
});
|
||||
|
||||
describe('seedPgliteFromFile', () => {
|
||||
test('reads SQL from disk and replays', async () => {
|
||||
const sqlPath = join(tmp, 'dump.sql');
|
||||
const dbPath = join(tmp, 'brain.pglite');
|
||||
writeFileSync(sqlPath, 'CREATE TABLE z(id int); INSERT INTO z VALUES (42);');
|
||||
await seedPgliteFromFile({ dbPath, sqlPath });
|
||||
const engine = new PGLiteEngine();
|
||||
try {
|
||||
await engine.connect({ engine: 'pglite', database_path: dbPath });
|
||||
const r: any = await (engine as any).db.query('SELECT id FROM z');
|
||||
expect(r.rows).toEqual([{ id: 42 }]);
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}, 30_000);
|
||||
|
||||
test('throws on missing SQL file', async () => {
|
||||
await expect(seedPgliteFromFile({
|
||||
dbPath: join(tmp, 'x.pglite'),
|
||||
sqlPath: join(tmp, 'nope.sql'),
|
||||
})).rejects.toThrow(/seed SQL not found/);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* Transcript capture tests — async drain, byte offsets, multi-byte safety,
|
||||
* spawn-with-capture happy + timeout paths.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, readFileSync, existsSync } from 'fs';
|
||||
import { tmpdir } from 'os';
|
||||
import { join } from 'path';
|
||||
import { createTranscriptSink, spawnWithCapture } from '../src/core/claw-test/transcript-capture.ts';
|
||||
|
||||
let tmp: string;
|
||||
let path: string;
|
||||
|
||||
beforeEach(() => {
|
||||
tmp = mkdtempSync(join(tmpdir(), 'transcript-'));
|
||||
path = join(tmp, 'transcript.jsonl');
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
rmSync(tmp, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
describe('createTranscriptSink', () => {
|
||||
test('writes events as JSONL lines with byte_offset', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('hello') });
|
||||
sink.write({ ts: 2, channel: 'stderr', bytes: Buffer.from('world') });
|
||||
await sink.close();
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n').map(l => JSON.parse(l));
|
||||
expect(lines).toHaveLength(2);
|
||||
expect(lines[0].channel).toBe('stdout');
|
||||
expect(lines[0].byte_offset).toBe(0);
|
||||
expect(lines[1].channel).toBe('stderr');
|
||||
expect(lines[1].byte_offset).toBeGreaterThan(0);
|
||||
expect(Buffer.from(lines[0].bytes_b64, 'base64').toString('utf-8')).toBe('hello');
|
||||
expect(Buffer.from(lines[1].bytes_b64, 'base64').toString('utf-8')).toBe('world');
|
||||
});
|
||||
|
||||
test('preserves multi-byte UTF-8 (no chunk-boundary corruption)', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// Split a 4-byte emoji across two writes to simulate stdio chunk boundaries.
|
||||
const emoji = '🌍';
|
||||
const buf = Buffer.from(emoji, 'utf-8');
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: buf.slice(0, 2) });
|
||||
sink.write({ ts: 2, channel: 'stdout', bytes: buf.slice(2) });
|
||||
await sink.close();
|
||||
|
||||
const lines = readFileSync(path, 'utf-8').trim().split('\n').map(l => JSON.parse(l));
|
||||
const concatenated = Buffer.concat([
|
||||
Buffer.from(lines[0].bytes_b64, 'base64'),
|
||||
Buffer.from(lines[1].bytes_b64, 'base64'),
|
||||
]).toString('utf-8');
|
||||
expect(concatenated).toBe(emoji);
|
||||
});
|
||||
|
||||
test('byte_offset is monotonic and matches the actual file position', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const before1 = sink.nextOffset();
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('a') });
|
||||
const before2 = sink.nextOffset();
|
||||
sink.write({ ts: 2, channel: 'stdout', bytes: Buffer.from('b') });
|
||||
await sink.close();
|
||||
|
||||
expect(before1).toBe(0);
|
||||
expect(before2).toBeGreaterThan(0);
|
||||
|
||||
// Verify the offsets recorded in lines match the actual file substring offsets.
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n').map(l => JSON.parse(l));
|
||||
const expectedOffsets = [0, Buffer.byteLength(raw.split('\n')[0] + '\n')];
|
||||
expect(lines[0].byte_offset).toBe(expectedOffsets[0]);
|
||||
expect(lines[1].byte_offset).toBe(expectedOffsets[1]);
|
||||
});
|
||||
|
||||
test('survives bursty writes (drain handling)', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// 256KB of payload across 256 1KB writes — exceeds default pipe buffer
|
||||
const chunk = Buffer.alloc(1024, 0x61); // 'a' * 1024
|
||||
for (let i = 0; i < 256; i++) {
|
||||
sink.write({ ts: i, channel: 'stdout', bytes: chunk });
|
||||
}
|
||||
await sink.close();
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const lines = raw.trim().split('\n');
|
||||
expect(lines.length).toBe(256);
|
||||
});
|
||||
|
||||
test('close is idempotent', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
sink.write({ ts: 1, channel: 'stdout', bytes: Buffer.from('x') });
|
||||
await sink.close();
|
||||
// Second close should not throw — the writeStream's `end` won't fire 'close' a second time
|
||||
// but we can call without error in our own wrapper.
|
||||
// (Implementation note: we don't expose a closed flag; idempotent via stream's no-op behavior.)
|
||||
expect(existsSync(path)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('spawnWithCapture', () => {
|
||||
test('captures stdout from a small command', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'printf hi'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 5_000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.exitCode).toBe(0);
|
||||
expect(result.timedOut).toBe(false);
|
||||
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const captured = raw.split('\n').filter(Boolean).map(l => JSON.parse(l));
|
||||
const stdoutBytes = captured.filter(e => e.channel === 'stdout')
|
||||
.map(e => Buffer.from(e.bytes_b64, 'base64').toString('utf-8'))
|
||||
.join('');
|
||||
expect(stdoutBytes).toBe('hi');
|
||||
});
|
||||
|
||||
test('non-zero exit propagates', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'exit 7'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 5_000,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.exitCode).toBe(7);
|
||||
expect(result.timedOut).toBe(false);
|
||||
});
|
||||
|
||||
test('timeout fires SIGTERM/SIGKILL', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
// `exec sleep` replaces sh with sleep so the child we spawn IS sleep —
|
||||
// SIGTERM goes directly to it, no shell-vs-child process-group ambiguity.
|
||||
// CI runners are slower than local, so the test cap is 30s with headroom
|
||||
// even if SIGTERM is missed and SIGKILL has to run after the 5s grace.
|
||||
const result = await spawnWithCapture('/bin/sh', ['-c', 'exec sleep 30'], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 200,
|
||||
transcriptSink: sink,
|
||||
});
|
||||
await sink.close();
|
||||
expect(result.timedOut).toBe(true);
|
||||
expect(result.exitCode).not.toBe(0);
|
||||
}, 30_000);
|
||||
|
||||
test('rejects when the binary does not exist', async () => {
|
||||
const sink = createTranscriptSink(path);
|
||||
await expect(
|
||||
spawnWithCapture('/no/such/binary', [], {
|
||||
cwd: tmp,
|
||||
env: { PATH: process.env.PATH ?? '' },
|
||||
timeoutMs: 1_000,
|
||||
transcriptSink: sink,
|
||||
})
|
||||
).rejects.toThrow();
|
||||
await sink.close();
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user