Files
gbrain/test/ze-switch-cli.test.ts
T
cdba533a04 v0.36.2.0 feat: ZeroEntropy as default + zero-based README rewrite (#1136)
* feat(dims): OpenAI text-embedding-3 Matryoshka range validation (D13)

dimsProviderOptions now fail-loud at the embed boundary when the
configured embedding_dimensions is outside the model's native range
(1..1536 for -small, 1..3072 for -large). Paste-ready fix hint in the
AIConfigError.fix field. Closes the silent-HTTP-400 path that would
have bit OpenAI-fallback users on v0.36.0.0 ZE-default installs.

16 new test cases in test/ai/dims-openai.test.ts pinning the contract
across native-openai and openai-compatible adapter paths.

* feat(ai): flip defaults to ZeroEntropy zembed-1 1280d + zerank-2 reranker

Default embedding model is now zeroentropyai:zembed-1 at 1280d via
Matryoshka. Real-corpus benchmark: 2.2x faster than OpenAI, 2.6x
cheaper at regular pricing, wins 11/20 head-to-head queries.

1280 is the closest valid ZE Matryoshka step to the prior OpenAI 1536d
default (valid set: 2560/1280/640/320/160/80/40). 1024 (Voyage's step)
is NOT on ZE's list — pinned by AIConfigError fail-loud in dims.ts.

balanced mode bundle now defaults reranker_enabled=true. zerank-2
reshuffles 60% of top-1 results in benchmarks. Missing-key fail-open
contract in src/core/search/rerank.ts handles unauthenticated cases.
Opt out with: gbrain config set search.reranker.enabled false

Existing tests updated (gateway.test.ts, search-mode.test.ts) and a
new test/balanced-reranker-default.test.ts (10 cases) pins the fail-
open invariants.

* feat(retrieval-upgrade): RetrievalUpgradePlanner + interactive prompt UX

New src/core/retrieval-upgrade-planner.ts is the consolidated planner
that computes the brain's pending retrieval-upgrade work (chunker
bumps + ZE switch) in one pass and applies the schema transition +
config updates atomically.

Tagged-union ApplyResult enum (D15): 'applied' | 'skipped_already_
applied' | 'skipped_no_work' | 'declined' | 'planned' | 'failed'.
No string-parsing reasons.

Three config keys (D12): ze_switch_prompt_shown (UI state),
ze_switch_requested (user intent), ze_switch_applied (work done).
Plus ze_switch_previous_snapshot (JSON, full prior config for --undo
per D16) and ze_switch_declined_at (90-day re-ask window).

Schema transition (D18) is atomic: DROP indexes + ALTER COLUMN +
CREATE INDEX inside a single engine.transaction(). HNSW recreation
is part of the same transaction — no silent slow-search window.

C3 eligibility logic: ze_switch_offered iff NOT on ZE + NOT declined
recently + NOT applied + (legacy default OR >100 pages).

C4 cost math: MAX(chunker_pending, dim_pending) not SUM — one
re-embed pass invalidates both surfaces simultaneously.

New src/core/retrieval-upgrade-prompt.ts wires the planner to a
TTY-only interactive prompt with two-line cost split (D10) and
privacy callout for the reranker flip.

Tests: test/retrieval-upgrade-planner.test.ts (24 cases) pins the
state machine. test/asymmetric-encoding-contract.test.ts (6 cases)
pins D17: search read path uses gateway.embedQuery() not embed(),
asserted via __setEmbedTransportForTests mock.

* feat(cli): gbrain ze-switch — manual lever for the ZE switch

New gbrain ze-switch CLI with --dry-run, --json, --resume, --force,
--undo, --non-interactive, --confirm-reembed, --ignore-missing-key
flags. Mirrors the upgrade prompt's UX symmetry: --undo presents a
cost-warning before re-embedding back to the prior width.

src/cli.ts: dispatch case + CLI_ONLY entry. ze-switch owns its own
engine lifecycle (mirrors the doctor pattern).

test/ze-switch-cli.test.ts (11 cases): --help, --dry-run, --json,
--non-interactive, --ignore-missing-key, --resume, --undo,
--confirm-reembed. Uses captureExit harness to test process.exit()
paths without breaking the test process.

* feat(doctor): ze_embedding_health + embedding_width_consistency checks

Two new doctor checks (D-A5):

ze_embedding_health: when embedding_model starts with zeroentropyai:,
verify ZEROENTROPY_API_KEY is set (env or config). Paste-ready setup
hint with the signup URL on failure.

embedding_width_consistency: cross-check that the configured
embedding_dimensions matches the actual vector(N) column width on
content_chunks.embedding. Catches the half-applied switch state
(schema migrated but config write crashed) with a paste-ready
gbrain ze-switch --resume hint.

Wired into runDoctor between reranker_health and the existing
sync_freshness checks. Both checks gracefully no-op on non-ZE
embedding configs.

test/doctor-ze-checks.test.ts (8 cases) pins both checks across
happy + missing-key + missing-config + drift paths. Uses withEnv()
helper to clear ZEROENTROPY_API_KEY for the no-key path so tests
are hermetic against contributor env state.

test/e2e/v0_28_5-fix-wave.test.ts + test/openai-compat-multimodal.test.ts:
updated to explicit-configure the gateway when the test depends on
specific dims that diverge from the v0.36.0.0 default (1280d).

* docs: README zero-based rewrite (884 -> 139 lines) + new docs files

Strip 4 months of accreted "New in v0.X.Y" hero blocks and reorganize
around what gbrain does today. 33 H2s -> 8. The Commands section
(136 lines duplicating gbrain --help) moved out; the 6-table skills
enumeration collapsed to a one-paragraph capability description with
a link to skills/RESOLVER.md.

Hero retains load-bearing facts: OpenClaw + Hermes credit, production
numbers (17,888 pages / 4,383 people / 723 companies), BrainBench
numbers (P@5 49.1% / R@5 97.9% / +31.4 lift), ZE comparison numbers,
30-min install claim. Adds one paragraph announcing the v0.36.0.0 ZE
default with the explicit gbrain config set escape for OpenAI/Voyage
users.

New files:
- docs/INSTALL.md: every install path consolidated (agent platform,
  CLI standalone, MCP server). Thin-client mode covered.
- docs/architecture/RETRIEVAL.md: why the hybrid + graph stack works.
  BrainBench numbers, why each strategy alone fails, the source-aware
  ranking + intent classification + multi-query expansion story.
- docs/ethos/ORIGIN.md: origin story lifted from the old README so
  the front door stays factual + concrete.

test/readme-hero-anchors.test.ts (5 cases) is the D9 regression
guard. Five load-bearing strings: OpenClaw, Hermes, ZE,
production-numbers regex, P@5/R@5. Light anchors that let voice/
structure evolve but block accidental loss of headline facts.

scripts/check-test-real-names.sh: allowlist entries for OpenClaw +
Hermes literals in the anchor test (it explicitly asserts those
strings appear in README).

* chore: bump version and changelog (v0.36.0.0)

ZeroEntropy as the new default for embedding (zembed-1 at 1280d via
Matryoshka) and reranker (zerank-2 cross-encoder, on by default in
balanced mode bundle). README zero-based rewrite (884 -> 139 lines).
3 new docs files. Two new doctor checks. New gbrain ze-switch CLI
with --undo for symmetric reversibility.

skills/migrations/v0.36.0.0.md tells the agent how to surface the
retrieval-upgrade prompt post-upgrade.

llms-full.txt regenerated via bun run build:llms.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(docs): scrub Wintermute from RETRIEVAL.md per privacy rule

* chore: rebump version 0.36.0.0 → 0.36.2.0 (queue collision)

Three open PRs were claiming v0.36.0.0 (#1130 skillpack, #1139
hindsight, #1136 this PR). Ship-aware queue allocator says this
branch lands at v0.36.2.0.

Trio audit:
  VERSION       0.36.2.0
  package.json  0.36.2.0
  CHANGELOG     ## [0.36.2.0] - 2026-05-17

Updates: VERSION, package.json, CHANGELOG header + body refs,
README "New default in v0.36.2.0" announcement + credit line,
skills/migrations/v0.36.0.0.md renamed to v0.36.2.0.md with
frontmatter + body refs updated. llms-full.txt regenerated.

* fix(test): pin gateway dim=1536 in cross-file-stateful PGLite tests

CI shard 1 reported 10 failures across `query-cache.test.ts` (6) and
`consolidate-valid-until.test.ts` (4). Both files hardcode 1536-dim
vectors but rely on `PGLiteEngine.initSchema()` to size
`vector(__EMBEDDING_DIMS__)` at the right width.

Root cause: v0.36.2.0 flipped DEFAULT_EMBEDDING_DIMENSIONS from 1536
to 1280 (ZE Matryoshka step). The gateway module is process-singleton;
when ANOTHER test file in the same shard's bun-test process configures
the gateway before us, `pglite-engine.ts:216` reads
`getEmbeddingDimensions() === 1280` and sizes the schema columns at
vector(1280). The hardcoded 1536-dim INSERTs then fail with
"expected 1280 dimensions, not 1536".

Locally these tests pass in isolation because the gateway falls back
through the try/catch at pglite-engine.ts:218 (1536 default). CI runs
multiple test files in one process, so cross-file state poisons the
schema width.

Fix: explicit `resetGateway()` + `configureGateway({embedding_dimensions:
1536, ...})` at the top of `beforeAll`, plus `resetGateway()` in
`afterAll`. Pins the schema width regardless of cross-file state.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-18 21:11:02 -07:00

239 lines
8.3 KiB
TypeScript

/**
* v0.36.0.0 (T3) — `gbrain ze-switch` CLI tests.
*
* Pins:
* - --dry-run prints a plan, applies nothing
* - --non-interactive without key exits 1 (unless --ignore-missing-key)
* - --non-interactive --ignore-missing-key applies + exits 0
* - --json envelope shape: {status: ..., plan: {...}}
* - --resume completes a half-applied switch
* - --undo without snapshot exits 1
* - --help exits 0 without touching the engine
*
* Engine lifecycle: each test creates + disconnects its own PGLite engine
* to keep process.exit() semantics clean. The CLI calls process.exit at
* the end of every path; we intercept via a stub.
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { resetPgliteState } from './helpers/reset-pglite.ts';
import { withEnv } from './helpers/with-env.ts';
import { runZeSwitch } from '../src/commands/ze-switch.ts';
import {
KEY_APPLIED,
KEY_REQUESTED,
KEY_PREVIOUS_SNAPSHOT,
ZE_TARGET_EMBEDDING_DIM,
} from '../src/core/retrieval-upgrade-planner.ts';
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
});
afterAll(async () => {
await engine.disconnect();
});
beforeEach(async () => {
await resetPgliteState(engine);
});
// Helpers: capture stdout/stderr/exitCode without actually exiting.
function captureExit<T>(fn: () => Promise<T>): Promise<{ exitCode: number; stdout: string; stderr: string }> {
return new Promise(async (resolve) => {
const origExit = process.exit;
const origStdoutWrite = process.stdout.write.bind(process.stdout);
const origStderrWrite = process.stderr.write.bind(process.stderr);
const origConsoleLog = console.log;
const origConsoleError = console.error;
let stdout = '';
let stderr = '';
let exitCode = 0;
process.exit = ((code?: number) => {
exitCode = code ?? 0;
throw new Error('__captured_exit__');
}) as any;
process.stdout.write = ((chunk: any) => {
stdout += typeof chunk === 'string' ? chunk : chunk.toString();
return true;
}) as any;
process.stderr.write = ((chunk: any) => {
stderr += typeof chunk === 'string' ? chunk : chunk.toString();
return true;
}) as any;
console.log = (...args: any[]) => { stdout += args.join(' ') + '\n'; };
console.error = (...args: any[]) => { stderr += args.join(' ') + '\n'; };
try {
await fn();
} catch (e: any) {
if (e?.message !== '__captured_exit__') {
stderr += `Unexpected: ${e?.message ?? String(e)}\n`;
exitCode = exitCode || 1;
}
} finally {
process.exit = origExit;
process.stdout.write = origStdoutWrite;
process.stderr.write = origStderrWrite;
console.log = origConsoleLog;
console.error = origConsoleError;
resolve({ exitCode, stdout, stderr });
}
});
}
async function seedPages(n: number) {
for (let i = 0; i < n; i++) {
await engine.putPage(`seed/page-${i}`, {
title: `Seed ${i}`,
compiled_truth: `Body text ${i} with enough chars to flow through cost math.`,
timeline: '',
type: 'note',
});
}
}
async function setLegacyConfig() {
await engine.setConfig('embedding_model', 'openai:text-embedding-3-large');
await engine.setConfig('embedding_dimensions', '1536');
}
describe('--help', () => {
test('exits 0 with usage text', async () => {
const r = await captureExit(() => runZeSwitch(['--help'], engine));
expect(r.exitCode).toBe(0);
expect(r.stdout).toContain('gbrain ze-switch');
expect(r.stdout).toContain('--dry-run');
expect(r.stdout).toContain('--undo');
});
});
describe('--dry-run', () => {
test('human output prints plan, changes nothing', async () => {
await setLegacyConfig();
await seedPages(150);
const r = await captureExit(() => runZeSwitch(['--dry-run'], engine));
expect(r.exitCode).toBe(0);
expect(r.stdout).toContain('Current model');
expect(r.stdout).toContain('Target model');
// Nothing changed:
expect(await engine.getConfig('embedding_model')).toBe('openai:text-embedding-3-large');
expect(await engine.getConfig(KEY_APPLIED)).toBeNull();
});
test('--json output emits a planned envelope', async () => {
await setLegacyConfig();
await seedPages(150);
const r = await captureExit(() => runZeSwitch(['--dry-run', '--json'], engine));
expect(r.exitCode).toBe(0);
const env = JSON.parse(r.stdout);
expect(env.status).toBe('planned');
expect(env.plan).toBeDefined();
expect(env.plan.target_embedding_model).toBe('zeroentropyai:zembed-1');
expect(env.plan.target_dim).toBe(ZE_TARGET_EMBEDDING_DIM);
});
});
describe('--non-interactive', () => {
test('without ZE key + without --ignore-missing-key: exits 1', async () => {
await setLegacyConfig();
await seedPages(150);
// Clear the env var so the test runs the no-key path even when the
// contributor has ZEROENTROPY_API_KEY set in their shell.
await withEnv({ ZEROENTROPY_API_KEY: undefined }, async () => {
const r = await captureExit(() => runZeSwitch(['--non-interactive'], engine));
expect(r.exitCode).toBe(1);
expect(r.stderr).toContain('ZEROENTROPY_API_KEY');
});
});
test('without ZE key + with --ignore-missing-key: applies, exits 0', async () => {
await setLegacyConfig();
await seedPages(150);
const r = await captureExit(() =>
runZeSwitch(['--non-interactive', '--ignore-missing-key'], engine),
);
expect(r.exitCode).toBe(0);
expect(await engine.getConfig(KEY_APPLIED)).toBe('true');
expect(await engine.getConfig('embedding_model')).toBe('zeroentropyai:zembed-1');
expect(await engine.getConfig('embedding_dimensions')).toBe(String(ZE_TARGET_EMBEDDING_DIM));
});
test('with env ZE key set: applies', async () => {
await setLegacyConfig();
await seedPages(150);
await withEnv({ ZEROENTROPY_API_KEY: 'sk-fake' }, async () => {
const r = await captureExit(() => runZeSwitch(['--non-interactive'], engine));
expect(r.exitCode).toBe(0);
expect(await engine.getConfig(KEY_APPLIED)).toBe('true');
});
});
test('--json + --non-interactive: emits {status: "applied"}', async () => {
await setLegacyConfig();
await seedPages(150);
const r = await captureExit(() =>
runZeSwitch(['--non-interactive', '--ignore-missing-key', '--json'], engine),
);
expect(r.exitCode).toBe(0);
const env = JSON.parse(r.stdout);
expect(env.status).toBe('applied');
});
});
describe('--resume', () => {
test('completes a half-applied switch', async () => {
await setLegacyConfig();
await seedPages(150);
// Simulate crash partway: requested but not applied.
await engine.setConfig(KEY_REQUESTED, 'true');
const r = await captureExit(() => runZeSwitch(['--resume'], engine));
expect(r.exitCode).toBe(0);
expect(await engine.getConfig(KEY_APPLIED)).toBe('true');
expect(await engine.getConfig('embedding_model')).toBe('zeroentropyai:zembed-1');
});
});
describe('--undo', () => {
test('without snapshot exits 1', async () => {
const r = await captureExit(() =>
runZeSwitch(['--undo', '--non-interactive', '--confirm-reembed'], engine),
);
expect(r.exitCode).toBe(1);
});
test('--non-interactive without --confirm-reembed exits 1', async () => {
const r = await captureExit(() => runZeSwitch(['--undo', '--non-interactive'], engine));
expect(r.exitCode).toBe(1);
expect(r.stderr).toContain('confirm-reembed');
});
test('with snapshot + --confirm-reembed: reverses the switch', async () => {
// Set up: apply switch, then undo.
await setLegacyConfig();
await seedPages(150);
await captureExit(() =>
runZeSwitch(['--non-interactive', '--ignore-missing-key'], engine),
);
expect(await engine.getConfig(KEY_APPLIED)).toBe('true');
const r = await captureExit(() =>
runZeSwitch(['--undo', '--non-interactive', '--confirm-reembed'], engine),
);
expect(r.exitCode).toBe(0);
// Reverted to prior model.
expect(await engine.getConfig('embedding_model')).toBe('openai:text-embedding-3-large');
expect(await engine.getConfig('embedding_dimensions')).toBe('1536');
expect(await engine.getConfig(KEY_APPLIED)).toBeNull();
});
});