mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 19:49:14 +00:00
* feat(dream): --once for one-shot phase runs without toggling config gates Fixes the "toggle enabled true, run, toggle back to false" workaround that gbrain doctor's extract_atoms_backlog message implicitly recommends and that #2860's reporter had to script around: with an external orchestrator running `gbrain dream --phase patterns` on a cadence outside the autopilot, the only way to run patterns once was `config set dream.patterns.enabled true` -> run -> `config set ... false`. A crash between steps left the flag stuck true, and the autopilot (which polls the same flag) re-enqueued patterns every cycle -- 119 LLM jobs / ~$400 over 24h before it was caught. Root cause: `--phase X` only controls which phase FUNCTION cycle.ts calls; it does not bypass that phase's own `dream.<phase>.enabled` / `cycle.<phase>.enabled` config read. Each gated phase (patterns, synthesize, conversation_facts_backfill, enrich_thin, skillopt) reads its enabled flag internally and skips regardless of how the phase was selected -- confirmed by reading each phase module, not assumed. extract_atoms/synthesize_concepts are a DIFFERENT mechanism entirely (pack-declaration via packDeclaresPhase, not a config .enabled read) and already have a working one-shot escape hatch: `--drain`. The existing doctor message for extract_atoms already says `--phase extract_atoms --drain --window 120`, so no doctor text needed updating there -- verified by reading src/commands/doctor.ts directly rather than assuming the paraphrase in the issue was literal. Design: `gbrain dream --phase <name> --once`. Requires an explicit --phase (bare --once is a usage error, exit 2) so it can never force-enable every disabled phase at once in a full/default cycle -- that would recreate the same unbounded-spend risk the flag exists to prevent. Threaded through CycleOpts as `onceForPhase?: CyclePhase` (the literal phase name, not a boolean) so the bypass can never leak to a phase other than the one named, even if a future programmatic caller passes a wider `phases` array than the CLI does. Never reads or writes config -- the phase still evaluates its .enabled gate every call; --once only overrides the boolean OUTCOME for that one invocation, mirroring the existing --unsafe-bypass-dream-guard / --input precedents (stderr warning at the bypass point, no new config-touching code path). Rejected alternatives (documented per task instructions): - Making explicit --phase X always bypass .enabled: breaking change for existing crons that rely on the disabled flag as a cheap no-op; an upgrade would silently start running LLM/write phases. - A new subcommand: adds a whole dispatch/help/arg surface that internally routes through the same override anyway. - Extending --once to also bypass packDeclaresPhase for extract_atoms/synthesize_concepts: conflates two different gating mechanisms (config toggle vs. pack membership) under one flag; extract_atoms already has --drain, which is purpose-built for its batched/windowed execution model. Design was cross-validated by an independent second-model review (external design consultation) before implementation; its recommendation to also update the extract_atoms doctor message to `--once` was NOT adopted because that phase has no .enabled gate to bypass -- doing so would be a documented no-op, contradicted by reading src/commands/doctor.ts:3264 directly. Tests: 9 new (structural CLI-flag wiring in dream-cli-flags.test.ts; a real PGLite E2E test in dream-patterns-pglite.test.ts proving the bypass fires AND that dream.patterns.enabled is never written; 4 runCycle-level tests in cycle.serial.test.ts proving onceForPhase does not leak across phases). Verified 6 of 9 fail against the pre-fix source (via git stash of source-only changes) to confirm they're meaningful regressions, not tautologies. Closes #2860 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(dream): --help short-circuits before --once usage validation Codex review finding (P2): `gbrain dream --help --once` (no --phase) called process.exit(2) from the new --once usage-error check inside parseArgs before runDream's documented IRON RULE ("--help short-circuits BEFORE any engine-bearing work") ever got a chance to run -- parseArgs computes ALL its validations unconditionally before runDream checks opts.help. Repo precedent for this ordering already exists as a pinned regression test (test/dream.test.ts's "--help --source whatever prints help and exits 0"). Fix: compute wantsHelp once in parseArgs and exempt the --once validation when it's set, mirroring that precedent. Added the same class of pinned tests here: bare `--once` still exits 2 with the usage hint, `--help --once` prints help and exits 0, and a real --phase patterns --once run against a PGLite engine proves the bypass actually fires (falls through to insufficient_evidence instead of disabled) without writing dream.patterns.enabled. Also fixed the structural test in dream-cli-flags.test.ts that asserted the exact pre-fix guard-condition source text. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * fix(dream): --once must require an EXPLICIT --phase, not a derived one Codex review finding (P3): the --once validation checked the derived `phase` value, but `phase` gets defaulted implicitly by --input (implies --phase synthesize) and --drain (implies --phase extract_atoms) BEFORE that check ran. So `gbrain dream --input <f> --once` and `gbrain dream --drain --once` both slipped past the "explicit --phase required" contract silently -- and --once became a true no-op in both cases: --drain returns from runDream before onceForPhase is ever read (the drain path doesn't call runCycle at all), and --input already bypasses the synthesize enabled-gate on its own via the existing opts.inputFile check, so onceForPhase would never even be consulted. Fix: capture `phaseWasExplicit = phaseIdx !== -1` at the very top of parseArgs, before the --input/--drain defaulting blocks run, and validate --once against that instead of the derived `phase`. Updated the usage-error message and --help text to say "an explicit --phase" so a user hitting this understands why `--input ... --once` doesn't count. Tests: 2 new pins in test/dream.test.ts exercising runDream directly (--input <file> --once exits 2; --drain --once exits 2), plus a structural test in dream-cli-flags.test.ts pinning that phaseWasExplicit is captured before both implicit-defaulting blocks. Updated the two existing structural/behavioral tests whose literal guard-condition / error-message assertions changed shape. Verified: dream-cli-flags.test.ts 27/27, dream.test.ts 31/31, typecheck clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: masashiono0611 <masashi.ono.0611@gmail.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> Co-authored-by: Garry Tan <garrytan@gmail.com>
217 lines
8.1 KiB
TypeScript
217 lines
8.1 KiB
TypeScript
/**
|
|
* E2E patterns phase — PGLite, no API key required.
|
|
*
|
|
* Mirrors the per-test-rig pattern from dream-synthesize-pglite.test.ts.
|
|
* Each test creates and tears down its own PGLite engine to avoid
|
|
* cross-test contention (CLAUDE.md issue #223 macOS WASM bug).
|
|
*
|
|
* Covers the runPhasePatterns skip paths that don't require a real
|
|
* Anthropic call:
|
|
* - disabled: dream.patterns.enabled=false → skipped
|
|
* - insufficient_evidence: <min_evidence reflections → skipped
|
|
* - no_provider: enough reflections, no reachable provider for the
|
|
* resolved patterns model (default: Anthropic with no key in env OR
|
|
* config) → skipped
|
|
* - dry-run: passes through with reflections_considered + zero pages
|
|
*
|
|
* The Sonnet detection path is structurally covered in
|
|
* test/cycle-patterns.test.ts (asserts queue + waitForCompletion are
|
|
* wired, allow-list reads from filing-rules JSON, slug provenance from
|
|
* subagent_tool_executions, no raw_data dependency).
|
|
*
|
|
* Run: bun test test/e2e/dream-patterns-pglite.test.ts
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
import { PGLiteEngine } from '../../src/core/pglite-engine.ts';
|
|
import { runPhasePatterns } from '../../src/core/cycle/patterns.ts';
|
|
import { withoutAnthropicKey } from '../helpers/no-anthropic-key.ts';
|
|
|
|
interface TestRig {
|
|
engine: PGLiteEngine;
|
|
brainDir: string;
|
|
cleanup: () => Promise<void>;
|
|
}
|
|
|
|
async function setupRig(): Promise<TestRig> {
|
|
const engine = new PGLiteEngine();
|
|
await engine.connect({ engine: 'pglite' } as never);
|
|
await engine.initSchema();
|
|
return {
|
|
engine,
|
|
brainDir: '/tmp/gbrain-patterns-test',
|
|
cleanup: async () => {
|
|
try { await engine.disconnect(); } catch { /* */ }
|
|
},
|
|
};
|
|
}
|
|
|
|
/**
|
|
* Insert N reflection pages directly via engine.putPage so the patterns
|
|
* gather query has data without going through the synthesize phase.
|
|
* Slugs follow the v0.23 wiki/personal/reflections/<topic>-<hash> shape.
|
|
*/
|
|
async function seedReflections(engine: PGLiteEngine, count: number): Promise<void> {
|
|
for (let i = 0; i < count; i++) {
|
|
const slug = `wiki/personal/reflections/2026-04-${String(15 + i).padStart(2, '0')}-test-pattern-aaa${i}`;
|
|
await engine.putPage(slug, {
|
|
type: 'note',
|
|
title: `Reflection ${i}`,
|
|
compiled_truth: `Sample reflection content ${i} discussing recurring theme of work-life balance.`,
|
|
timeline: '',
|
|
frontmatter: { type: 'note', title: `Reflection ${i}` },
|
|
});
|
|
}
|
|
}
|
|
|
|
describe('E2E patterns — disabled', () => {
|
|
test('skipped when dream.patterns.enabled=false', async () => {
|
|
// 30s timeout: a fresh PGLiteEngine + initSchema (36 migrations,
|
|
// pgvector WASM cold start) clears in ~3s but spikes to 6-15s under
|
|
// full-e2e-suite load contention. Default 5s timeout was eating the
|
|
// happy path.
|
|
const rig = await setupRig();
|
|
try {
|
|
await rig.engine.setConfig('dream.patterns.enabled', 'false');
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
expect((result.details as { reason?: string }).reason).toBe('disabled');
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
|
|
test('default-enabled when config key unset', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
// No reflections seeded → falls through to insufficient_evidence,
|
|
// not disabled. Confirms the default-true semantics.
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
|
|
// issue #2860 — `gbrain dream --phase patterns --once` must bypass the
|
|
// enabled gate WITHOUT touching config (the whole point: the old
|
|
// toggle-on/run/toggle-off workaround left `enabled` stuck true forever
|
|
// if the process died between steps, running patterns hourly and
|
|
// burning ~$400 in LLM spend before it was caught).
|
|
test('once:true bypasses the disabled gate for this call only', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
await rig.engine.setConfig('dream.patterns.enabled', 'false');
|
|
|
|
// Without --once: still skipped as 'disabled' (unchanged behavior).
|
|
const gated = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(gated.status).toBe('skipped');
|
|
expect((gated.details as { reason?: string }).reason).toBe('disabled');
|
|
|
|
// With --once: the disabled gate no longer short-circuits — the call
|
|
// proceeds past it to the NEXT gate (insufficient_evidence, since no
|
|
// reflections were seeded). If --once didn't work, this would still
|
|
// report 'disabled'.
|
|
const forced = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
once: true,
|
|
});
|
|
expect(forced.status).toBe('skipped');
|
|
expect((forced.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
|
|
|
// Config was NEVER written — the override is call-scoped only, unlike
|
|
// the toggle-on/toggle-off workaround the issue exists to replace.
|
|
const stillDisabled = await rig.engine.getConfig('dream.patterns.enabled');
|
|
expect(stillDisabled).toBe('false');
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
});
|
|
|
|
describe('E2E patterns — insufficient_evidence', () => {
|
|
test('skipped with 0 reflections', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
|
|
test('skipped with reflections below min_evidence', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
await rig.engine.setConfig('dream.patterns.min_evidence', '5');
|
|
await seedReflections(rig.engine, 3); // below 5
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
expect((result.details as { reason?: string }).reason).toBe('insufficient_evidence');
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
});
|
|
|
|
describe('E2E patterns — no reachable provider', () => {
|
|
test('enough reflections, no Anthropic key in env OR config → skipped no_provider', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
await seedReflections(rig.engine, 5); // above default min_evidence (3)
|
|
// Default patterns model resolves to Anthropic; with no key reachable
|
|
// from EITHER source (env + config file — the shared helper neuters
|
|
// both) the gateway probe reports the provider unavailable. A
|
|
// non-Anthropic stack (litellm, deepseek, ...) passes this gate and
|
|
// dispatches through the gateway instead (PR #2279).
|
|
await withoutAnthropicKey(async () => {
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: false,
|
|
});
|
|
expect(result.status).toBe('skipped');
|
|
expect((result.details as { reason?: string }).reason).toBe('no_provider');
|
|
});
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
});
|
|
|
|
describe('E2E patterns — dry-run', () => {
|
|
test('dry-run returns ok with reflections_considered and zero patterns_written', async () => {
|
|
const rig = await setupRig();
|
|
try {
|
|
await seedReflections(rig.engine, 5);
|
|
const result = await runPhasePatterns(rig.engine, {
|
|
brainDir: rig.brainDir,
|
|
dryRun: true,
|
|
});
|
|
expect(result.status).toBe('ok');
|
|
expect((result.details as { dryRun: boolean }).dryRun).toBe(true);
|
|
expect((result.details as { reflections_considered: number }).reflections_considered).toBe(5);
|
|
expect((result.details as { patterns_written: number }).patterns_written).toBe(0);
|
|
} finally {
|
|
await rig.cleanup();
|
|
}
|
|
}, 30_000);
|
|
});
|