mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
v0.37.1.0 feat: brainstorm + lsd — bisociation idea generator grounded in your own brain (#1214)
* feat: brainstorm + lsd (v0.37 wave, pre-merge snapshot) Brainstorm + LSD bisociation idea generator. Will rebase + bump after master merge. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * docs: CHANGELOG voice tightening — strip meta-content from v0.37.1.0 entry CLAUDE.md gains an IRON RULE for CHANGELOG entries: the changelog describes what the user gets, not how the work happened. No mentions of review processes, plan files, decision tags, migration version drama, or 'what we caught and fixed before merging.' If a fact only exists because of the development workflow, it does not belong in release notes. Rewrite v0.37.1.0 entry to comply: cut the 'what we caught' section (architectural review drama), the 'plan + reviews' bullet, and the migration-renumbering aside. Entry shrinks 67 → 56 lines, every sentence now answers 'what can I do / how do I use it / what should I watch for.' Regenerated llms-full.txt to absorb the CLAUDE.md voice update. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
bc9f7774bf
commit
39e14cd50e
@@ -2,6 +2,61 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.37.1.0] - 2026-05-19
|
||||
|
||||
**Your brain can now collide its own ideas.**
|
||||
|
||||
You type `gbrain brainstorm "why are AI coding tools converging on the same UX?"` and get back five ideas. Each one cites two pages from your own notes: one that's close to the question, and one from a part of your brain that has no business being there. You see the slugs, you see how far the collision actually traveled (a 0-1 distance score next to each idea), and you can click through to read the source pages. Five minutes ago you didn't know your brain had those connections. The judge filters out anything trivial. Cost about 10 cents.
|
||||
|
||||
Then you try `gbrain lsd` on the same question. Lateral Synaptic Drift. The dial is turned to 11: twelve pages from twelve different parts of your brain, the judge is inverted (rejects ideas that score too high on coherence — "too obvious, you'd have thought of this without LSD"), and the system explicitly prefers pages you haven't looked at in 90 days. Most of what comes back is noise. One of them is the thing your brain noticed in its sleep.
|
||||
|
||||
This is bisociation, the Arthur Koestler thing. The Open Collider project demonstrated 4-13x distance shift over default prompting. Their version invents distant domains from training data. Ours uses what you already wrote down. That difference matters: every idea cites a real slug in your brain. You can audit it. You can trust it.
|
||||
|
||||
**How to turn it on**
|
||||
|
||||
```bash
|
||||
gbrain upgrade # applies migration v79
|
||||
gbrain brainstorm "your question here" # ~$0.10 per run
|
||||
gbrain lsd "your question here" # ~$0.30 per run
|
||||
gbrain doctor # see brainstorm_health check
|
||||
```
|
||||
|
||||
By default `brainstorm` saves to `wiki/ideas/<date>-brainstorm-<slug>.md` and `lsd` does not save (its output is ephemeral by design — pass `--save` if an idea lands). The dream cycle skips `mode: lsd` pages automatically so noise does not pollute the calibration profile.
|
||||
|
||||
**The numbers that matter**
|
||||
|
||||
| Mode | Close set | Far set | Ideas per cross | Judge threshold | Vibe |
|
||||
|---|---|---|---|---|---|
|
||||
| brainstorm | 4 | 6 | 3 | weighted 4.0/5 | Analyst riffing with their own notes. Defensible. |
|
||||
| lsd | 2 | 12 | 4 | inverted: rejects resistance >4.5 | Your brain at 3am noticing what it forgot. |
|
||||
|
||||
**Things to watch**
|
||||
|
||||
- The "far set" comes from prefix-stratified sampling. One page per top-level prefix (`wiki/vc`, `wiki/biology`, `concepts/`, etc), tiebroken by inbound link count. If your brain has only one or two top-level prefixes you will see a stderr warning that the bank was narrower than usual. Add more cross-domain content via `gbrain import` and the next run gets sharper.
|
||||
- LSD's stale-page bias works off a new column `pages.last_retrieved_at` bumped when `search`/`query`/`get_page` returns results. Default-on. If you do not want per-search writes, `gbrain config set search.track_retrieval false` opts out (LSD then runs without the stale-preference signal but still works). Internal callers (sync, migrations, dream cycle) never bump the column — the signal stays clean.
|
||||
- Calibration cold-start: if your `calibration_profiles` row has no `active_bias_tags`, the judge runs without anti-bias context and stderr-warns. Both commands still work; the judge just is not penalizing your known biases.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
**New commands** — `gbrain brainstorm <question>` and `gbrain lsd <question>`, both CLI-only. `gbrain eval brainstorm <fixture.jsonl>` is the three-axis evaluation gate (distance + usefulness + grounding, all three must clear).
|
||||
|
||||
**New module** — `src/core/brainstorm/{domain-bank,orchestrator,judges}.ts`. One `judges.ts` exports a shared `runJudge(config, ideas)` plus `BRAINSTORM_JUDGE_CONFIG` + `LSD_JUDGE_CONFIG`, so the two modes share judge plumbing.
|
||||
|
||||
**New engine surface** — `BrainEngine.listPrefixSampledPages(opts)` + `BrainEngine.listCorpusSample(opts)`, parity on both Postgres and PGLite. Both accept `sourceId?` and `sourceIds?` for federated-read scoping.
|
||||
|
||||
**Schema migration v79** — `pages.last_retrieved_at TIMESTAMPTZ NULL` + full B-tree index on `pages (last_retrieved_at)`. Forward-reference bootstrap added on both engines so pre-v79 brains upgrade cleanly.
|
||||
|
||||
**Op-layer write-back** — `src/core/last-retrieved.ts` exports `bumpLastRetrievedAt(engine, pageIds)`. Called fire-and-forget from the `search`/`query`/`get_page` op handlers after results return. 5-minute throttle via the SQL `WHERE last_retrieved_at IS NULL OR last_retrieved_at < NOW() - INTERVAL '5 minutes'` clause skips ~90% of writes on heavily-searched brains.
|
||||
|
||||
**Dream-cycle skip** — `src/core/cycle/transcript-discovery.ts` extends `isDreamOutput(content)` to also skip pages with `mode: lsd` frontmatter (noise-by-design). New exports `isLsdOutput()` and `isBrainstormOutput()` for downstream callers.
|
||||
|
||||
**Doctor check** — `brainstorm_health` surfaces three signals: migration v79 applied, `search.track_retrieval` setting, calibration cold-start status. Paste-ready fix hints per signal.
|
||||
|
||||
**Tests** — 38 new unit tests across `test/brainstorm/{distance,lsd-mode-skip,eval-brainstorm}.test.ts`. Pin distance normalization (same-vector → 0, orthogonal → 0.5, opposite → 1, dimension mismatch throws), LSD frontmatter detection, eval verdict logic, fixture parsing.
|
||||
|
||||
## To take advantage of v0.37.1.0
|
||||
|
||||
`gbrain upgrade` should do this automatically. If it did not:
|
||||
## [0.37.0.0] - 2026-05-19
|
||||
|
||||
**Skillpacks become a real ecosystem. You can ship one, someone else can install it, and gbrain has an opinion about quality.**
|
||||
@@ -150,6 +205,22 @@ Auto-flip prompt fires at coverage=100% so the last step is suggested inline.
|
||||
```bash
|
||||
gbrain apply-migrations --yes
|
||||
```
|
||||
2. **Verify migration v79 landed:**
|
||||
```bash
|
||||
gbrain doctor --json | grep '"schema_version"'
|
||||
```
|
||||
Should show: `"Version 79 (latest: 79)"`
|
||||
3. **Try it:**
|
||||
```bash
|
||||
gbrain brainstorm "your question here"
|
||||
gbrain lsd "your question here" --save
|
||||
```
|
||||
4. **Check brainstorm_health:**
|
||||
```bash
|
||||
gbrain doctor --json | grep brainstorm_health
|
||||
```
|
||||
Should show: `"Migration v79 applied; tracking enabled..."`
|
||||
5. **If anything fails,** file an issue at https://github.com/garrytan/gbrain/issues with the output of `gbrain doctor` and `~/.gbrain/upgrade-errors.jsonl` if present.
|
||||
2. **Verify the new surface:**
|
||||
```bash
|
||||
gbrain doctor | grep -E 'modal|cross'
|
||||
|
||||
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "gbrain",
|
||||
"version": "0.37.0.0",
|
||||
"version": "0.37.1.0",
|
||||
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
|
||||
"type": "module",
|
||||
"main": "src/core/index.ts",
|
||||
|
||||
+19
-1
@@ -27,7 +27,7 @@ for (const op of operations) {
|
||||
}
|
||||
|
||||
// CLI-only commands that bypass the operation layer
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder']);
|
||||
const CLI_ONLY = new Set(['init', 'upgrade', 'post-upgrade', 'check-update', 'integrations', 'publish', 'check-backlinks', 'lint', 'report', 'import', 'export', 'files', 'embed', 'serve', 'call', 'config', 'doctor', 'migrate', 'eval', 'sync', 'extract', 'features', 'autopilot', 'graph-query', 'jobs', 'agent', 'apply-migrations', 'skillpack-check', 'skillpack', 'resolvers', 'integrity', 'repair-jsonb', 'orphans', 'sources', 'mounts', 'dream', 'check-resolvable', 'routing-eval', 'skillify', 'smoke-test', 'providers', 'storage', 'repos', 'code-def', 'code-refs', 'reindex-code', 'reindex-frontmatter', 'code-callers', 'code-callees', 'frontmatter', 'auth', 'friction', 'claw-test', 'book-mirror', 'takes', 'think', 'salience', 'anomalies', 'transcripts', 'models', 'remote', 'recall', 'forget', 'edges-backfill', 'cache', 'ze-switch', 'founder', 'brainstorm', 'lsd']);
|
||||
// CLI-only commands whose handlers print their own --help text. These are
|
||||
// excluded from the generic short-circuit so detailed per-command and
|
||||
// per-subcommand usage stays reachable.
|
||||
@@ -39,6 +39,7 @@ const CLI_ONLY_SELF_HELP = new Set([
|
||||
'frontmatter', 'check-resolvable',
|
||||
'models',
|
||||
'cache',
|
||||
'brainstorm', 'lsd',
|
||||
]);
|
||||
|
||||
async function main() {
|
||||
@@ -1206,6 +1207,23 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
await runWhoknows(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'brainstorm': {
|
||||
// v0.37.0 (Open Collider wave): bisociation idea generator grounded
|
||||
// in the user's own brain. Prefix-stratified domain-bank (D14) +
|
||||
// shared judges + citation transparency (D6). LSD MCP exposure
|
||||
// deferred to D7; this is CLI-only.
|
||||
const { runBrainstormCommand } = await import('./commands/brainstorm.ts');
|
||||
await runBrainstormCommand(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'lsd': {
|
||||
// v0.37.0 — Lateral Synaptic Drift. Inverted-judge / stale-bias
|
||||
// variant of brainstorm. Shares the orchestrator + judges via
|
||||
// LSD_PROFILE config. Local-only by design (cost + weirdness gate).
|
||||
const { runLsdCommand } = await import('./commands/lsd.ts');
|
||||
await runLsdCommand(engine, args);
|
||||
break;
|
||||
}
|
||||
case 'calibration': {
|
||||
// v0.36.1.0 (T7): print/regenerate the active calibration profile.
|
||||
// MCP op `get_calibration_profile` (read-scoped) backs the same data path.
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
/**
|
||||
* gbrain brainstorm — bisociation-style idea generation grounded in your
|
||||
* own notes.
|
||||
*
|
||||
* v0.37.0 wave (D14 + D6 + D11 + D12). Pulls a small close-set via
|
||||
* hybridSearch, a far-set via prefix-stratified domain-bank, crosses them
|
||||
* via gateway.chat, and judges via the shared 5-axis rubric. Output cites
|
||||
* close + far slugs with a 0-1 distance score (D6 transparency).
|
||||
*
|
||||
* For "Lateral Synaptic Drift" — the inverted-judge / stale-bias variant —
|
||||
* see `src/commands/lsd.ts` which calls the same orchestrator with the
|
||||
* LSD_PROFILE config object.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import {
|
||||
runBrainstorm,
|
||||
formatBrainstormMarkdown,
|
||||
buildBrainstormFrontmatter,
|
||||
BRAINSTORM_PROFILE,
|
||||
LSD_PROFILE,
|
||||
type BrainstormProfile,
|
||||
} from '../core/brainstorm/orchestrator.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
|
||||
export interface BrainstormCliArgs {
|
||||
question?: string;
|
||||
json: boolean;
|
||||
save?: boolean;
|
||||
yes: boolean;
|
||||
limit?: number;
|
||||
help: boolean;
|
||||
error?: string;
|
||||
}
|
||||
|
||||
export function parseBrainstormArgs(args: string[]): BrainstormCliArgs {
|
||||
const out: BrainstormCliArgs = { json: false, yes: false, help: false };
|
||||
const positional: string[] = [];
|
||||
let i = 0;
|
||||
while (i < args.length) {
|
||||
const arg = args[i];
|
||||
if (arg === '--help' || arg === '-h') {
|
||||
out.help = true;
|
||||
} else if (arg === '--json') {
|
||||
out.json = true;
|
||||
} else if (arg === '--save') {
|
||||
out.save = true;
|
||||
} else if (arg === '--no-save') {
|
||||
out.save = false;
|
||||
} else if (arg === '--yes' || arg === '-y') {
|
||||
out.yes = true;
|
||||
} else if (arg === '--limit') {
|
||||
const v = args[++i];
|
||||
const n = v ? parseInt(v, 10) : NaN;
|
||||
if (!Number.isFinite(n) || n <= 0) {
|
||||
out.error = `--limit requires a positive integer (got ${v})`;
|
||||
return out;
|
||||
}
|
||||
out.limit = n;
|
||||
} else if (arg.startsWith('--')) {
|
||||
out.error = `unknown flag: ${arg}`;
|
||||
return out;
|
||||
} else {
|
||||
positional.push(arg);
|
||||
}
|
||||
i++;
|
||||
}
|
||||
if (positional.length > 0) {
|
||||
out.question = positional.join(' ');
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
const BRAINSTORM_HELP = `Usage: gbrain brainstorm <question> [options]
|
||||
|
||||
Bisociation idea generator grounded in your own notes. Pulls a close-set
|
||||
via hybrid search and a far-set via prefix-stratified domain-bank, crosses
|
||||
them, judges with a 5-axis rubric. Output cites close + far slugs with a
|
||||
0-1 distance score so you can see how far each collision actually traveled.
|
||||
|
||||
Options:
|
||||
--json Emit BrainstormResult as JSON (for agents)
|
||||
--save Save to wiki/ideas/<date>-brainstorm-<slug>.md (default ON)
|
||||
--no-save Don't save; print only
|
||||
--yes, -y Skip the 10s cost-preview wait (TTY only)
|
||||
--limit N Override the far-bank size (default 6 brainstorm / 12 LSD)
|
||||
--help, -h Show this help
|
||||
|
||||
Examples:
|
||||
gbrain brainstorm "why are AI coding tools converging on the same UX?"
|
||||
gbrain brainstorm "what's the real bottleneck on lab automation" --json
|
||||
|
||||
Cost: ~$0.05-0.15 per run. Set GBRAIN_NO_BRAINSTORM_PREVIEW=1 or pass --yes
|
||||
to skip the TTY grace window in scripted callers.
|
||||
|
||||
See also: gbrain lsd — Lateral Synaptic Drift, the inverted-judge variant
|
||||
that prefers forgotten pages and rejects ideas that are "too obvious."
|
||||
`;
|
||||
|
||||
const LSD_HELP = `Usage: gbrain lsd <question> [options]
|
||||
|
||||
LSD = Lateral Synaptic Drift. Same bisociation engine as \`gbrain brainstorm\`
|
||||
with the distance dial maxed: bigger far-bank (12 pages), smaller close-set
|
||||
(2 pages), forgotten pages preferred via the stale-bias signal, inverted
|
||||
judge that REJECTS ideas scoring too high on coherence ("too obvious — you'd
|
||||
have thought of this without LSD"), every idea must invert at least one
|
||||
implicit axiom. Output is ephemeral by default — pass --save if an idea lands.
|
||||
|
||||
Options:
|
||||
--json Emit BrainstormResult as JSON
|
||||
--save Persist to wiki/ideas/<date>-lsd-<slug>.md (default OFF)
|
||||
--yes, -y Skip the 10s cost-preview wait (TTY only)
|
||||
--limit N Override the far-bank size (default 12)
|
||||
--help, -h Show this help
|
||||
|
||||
Examples:
|
||||
gbrain lsd "why are AI coding tools converging on the same UX?"
|
||||
gbrain lsd "the unspoken assumption in venture pricing" --save
|
||||
|
||||
Cost: ~$0.20-0.40 per run.
|
||||
|
||||
See also: gbrain brainstorm — the sober, cite-heavy default variant.
|
||||
`;
|
||||
|
||||
/** Shared body: brainstorm.ts → runBrainstormCli(BRAINSTORM_PROFILE); lsd.ts → runBrainstormCli(LSD_PROFILE). */
|
||||
async function runBrainstormCli(
|
||||
engine: BrainEngine,
|
||||
args: string[],
|
||||
profile: BrainstormProfile,
|
||||
help: string,
|
||||
): Promise<void> {
|
||||
const parsed = parseBrainstormArgs(args);
|
||||
if (parsed.help) {
|
||||
console.log(help);
|
||||
return;
|
||||
}
|
||||
if (parsed.error) {
|
||||
console.error(`gbrain ${profile.label}: ${parsed.error}`);
|
||||
console.error(help);
|
||||
process.exit(2);
|
||||
return;
|
||||
}
|
||||
if (!parsed.question || parsed.question.trim().length === 0) {
|
||||
console.error(`gbrain ${profile.label}: question required`);
|
||||
console.error(help);
|
||||
process.exit(2);
|
||||
return;
|
||||
}
|
||||
|
||||
const config = loadConfig() ?? {};
|
||||
// Honor env-var skip for scripted environments that can't easily pass --yes.
|
||||
const skipPreview = parsed.yes || process.env.GBRAIN_NO_BRAINSTORM_PREVIEW === '1';
|
||||
|
||||
// --limit override: replace m_far on a shallow copy of the profile.
|
||||
const effectiveProfile: BrainstormProfile = parsed.limit
|
||||
? { ...profile, m_far: parsed.limit }
|
||||
: profile;
|
||||
|
||||
const result = await runBrainstorm(engine, config, {
|
||||
question: parsed.question,
|
||||
profile: effectiveProfile,
|
||||
skipCostPreview: skipPreview,
|
||||
});
|
||||
|
||||
if (parsed.json) {
|
||||
console.log(JSON.stringify(result, null, 2));
|
||||
return;
|
||||
}
|
||||
|
||||
// Human-readable: render via formatBrainstormMarkdown. `onlyPassed: true`
|
||||
// by default — passes-only output is the headline; failed ideas live in
|
||||
// the JSON / saved page for triage.
|
||||
const md = formatBrainstormMarkdown(result, { onlyPassed: true, includeMeta: true });
|
||||
console.log(md);
|
||||
|
||||
// Save policy: brainstorm defaults to save-on; lsd defaults to save-off.
|
||||
// CLI --save / --no-save overrides the default.
|
||||
const shouldSave = parsed.save ?? profile.default_save;
|
||||
if (shouldSave) {
|
||||
const slug = buildIdeaSlug(parsed.question, profile.label);
|
||||
const frontmatter = buildBrainstormFrontmatter(result, { slug });
|
||||
// Re-render content for save: include filtered ideas too so --retry-judge
|
||||
// (when implemented) has the full set to re-score.
|
||||
const body = formatBrainstormMarkdown(result, { onlyPassed: false, includeMeta: true });
|
||||
const content = frontmatter + body;
|
||||
try {
|
||||
await engine.putPage(slug, {
|
||||
title: `${profile.label === 'lsd' ? 'LSD' : 'Brainstorm'}: ${parsed.question.slice(0, 100)}`,
|
||||
type: 'note',
|
||||
compiled_truth: content,
|
||||
frontmatter: {
|
||||
mode: profile.frontmatter_mode,
|
||||
generated_at: new Date().toISOString(),
|
||||
question: parsed.question,
|
||||
judge_failed: result.judge_failed,
|
||||
unscored: result.judge_failed,
|
||||
close_slugs: result.close_set.map((c) => c.slug),
|
||||
far_slugs: result.far_set.map((f) => f.slug),
|
||||
},
|
||||
timeline: '',
|
||||
});
|
||||
console.log(`\n_Saved to \`${slug}\`._`);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
console.error(`gbrain ${profile.label}: save failed: ${msg}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Slugify the question for the saved page path. Capped + collision-resistant via date prefix. */
|
||||
function buildIdeaSlug(question: string, label: 'brainstorm' | 'lsd'): string {
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
const stem = question
|
||||
.toLowerCase()
|
||||
.replace(/[^a-z0-9]+/g, '-')
|
||||
.replace(/^-+|-+$/g, '')
|
||||
.slice(0, 60)
|
||||
.replace(/^-+|-+$/g, '');
|
||||
return `wiki/ideas/${date}-${label}-${stem || 'untitled'}`;
|
||||
}
|
||||
|
||||
/** CLI entry: `gbrain brainstorm`. */
|
||||
export async function runBrainstormCommand(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
return runBrainstormCli(engine, args, BRAINSTORM_PROFILE, BRAINSTORM_HELP);
|
||||
}
|
||||
|
||||
/** CLI entry: `gbrain lsd`. */
|
||||
export async function runLsdCommand(engine: BrainEngine, args: string[]): Promise<void> {
|
||||
return runBrainstormCli(engine, args, LSD_PROFILE, LSD_HELP);
|
||||
}
|
||||
@@ -573,6 +573,13 @@ export async function doctorReportRemote(engine: BrainEngine): Promise<DoctorRep
|
||||
// things when reranker is on vs off.
|
||||
checks.push(await checkRerankerHealth(engine));
|
||||
|
||||
// 9b. v0.37.0 brainstorm_health: surfaces three brainstorm/lsd readiness
|
||||
// signals: (a) migration v79 applied (last_retrieved_at column exists),
|
||||
// (b) calibration cold-start status (active_bias_tags empty), (c)
|
||||
// search.track_retrieval enabled/disabled. Each surfaces a paste-ready
|
||||
// fix hint.
|
||||
checks.push(await checkBrainstormHealth(engine));
|
||||
|
||||
// 10. v0.36.1.0 Hindsight calibration wave (T12) — four new checks:
|
||||
// - abandoned_threads: high-conviction takes never revisited
|
||||
// - calibration_freshness: profile is older than 7 days
|
||||
@@ -837,6 +844,112 @@ export async function checkRerankerHealth(engine: BrainEngine): Promise<Check> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.37.0 brainstorm_health doctor check.
|
||||
*
|
||||
* Surfaces three readiness signals for `gbrain brainstorm` / `gbrain lsd`:
|
||||
*
|
||||
* 1. Migration v79 applied — the `pages.last_retrieved_at` column exists.
|
||||
* If missing, LSD's stale-page signal degrades silently (corpus-sampling
|
||||
* fallback only). Fix: `gbrain apply-migrations --yes`.
|
||||
*
|
||||
* 2. search.track_retrieval — when explicitly off, LSD never accumulates
|
||||
* stale signal (every page stays at NULL last_retrieved_at). Default-on
|
||||
* is fine; explicit-off is a warning so the user notices the setting.
|
||||
* Fix: `gbrain config set search.track_retrieval true`.
|
||||
*
|
||||
* 3. Calibration cold-start — the latest calibration profile has empty
|
||||
* `active_bias_tags`. brainstorm + LSD judge fall back to no-anti-bias
|
||||
* mode with a stderr warning at run time; this surfaces it earlier.
|
||||
* Fix: `gbrain calibration --regenerate` once enough takes are resolved.
|
||||
*
|
||||
* Returns the FIRST non-ok signal as the status — column-missing dominates,
|
||||
* then disabled-tracking, then cold-start. All three are non-blocking warnings;
|
||||
* brainstorm + LSD still work, just with degraded signal.
|
||||
*/
|
||||
export async function checkBrainstormHealth(engine: BrainEngine): Promise<Check> {
|
||||
// (1) Column probe — fast, single-query.
|
||||
try {
|
||||
const probeRows = await engine.executeRaw<{ exists: boolean }>(
|
||||
`SELECT EXISTS (
|
||||
SELECT 1 FROM information_schema.columns
|
||||
WHERE table_name = 'pages' AND column_name = 'last_retrieved_at'
|
||||
) AS exists`,
|
||||
[]
|
||||
);
|
||||
const columnPresent = probeRows[0]?.exists === true;
|
||||
if (!columnPresent) {
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'warn',
|
||||
message: `pages.last_retrieved_at column missing. LSD stale-bias degraded to corpus-sampling. Fix: \`gbrain apply-migrations --yes\``,
|
||||
};
|
||||
}
|
||||
} catch (e) {
|
||||
// Information schema may not be queryable on every engine variant.
|
||||
// Don't fail the doctor over this — degrade to skip.
|
||||
const msg = e instanceof Error ? e.message : String(e);
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'warn',
|
||||
message: `Could not probe pages.last_retrieved_at (${msg}); brainstorm/lsd may run with degraded signal.`,
|
||||
};
|
||||
}
|
||||
|
||||
// (2) search.track_retrieval — explicit-off surfaces as a warning.
|
||||
try {
|
||||
const trackCfg = await engine.getConfig('search.track_retrieval');
|
||||
if (trackCfg === 'false' || trackCfg === '0' || trackCfg === 'off' || trackCfg === 'no') {
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'warn',
|
||||
message: `search.track_retrieval is explicitly off — LSD's stale-page signal never accumulates. Fix: \`gbrain config set search.track_retrieval true\` (or accept and use brainstorm only).`,
|
||||
};
|
||||
}
|
||||
} catch {
|
||||
// Config read miss is benign; default-on applies.
|
||||
}
|
||||
|
||||
// (3) Calibration cold-start — empty active_bias_tags.
|
||||
try {
|
||||
const calibRows = await engine.executeRaw<{ active_bias_tags: string[] | null }>(
|
||||
`SELECT active_bias_tags
|
||||
FROM calibration_profiles
|
||||
ORDER BY generated_at DESC
|
||||
LIMIT 1`,
|
||||
[]
|
||||
);
|
||||
if (calibRows.length === 0) {
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'ok',
|
||||
message: `Migration v79 applied; tracking enabled. Calibration profile not yet generated — brainstorm/lsd will run unbiased until enough takes are resolved.`,
|
||||
};
|
||||
}
|
||||
const tags = calibRows[0].active_bias_tags;
|
||||
if (!Array.isArray(tags) || tags.length === 0) {
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'ok',
|
||||
message: `Migration v79 applied; tracking enabled. Calibration cold-start (no active_bias_tags) — judge runs unbiased. Fix when ready: \`gbrain calibration --regenerate\`.`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'ok',
|
||||
message: `Migration v79 applied; tracking enabled; calibration profile with ${tags.length} bias tag(s) loaded.`,
|
||||
};
|
||||
} catch {
|
||||
// Pre-v0.36.1 brain (no calibration_profiles table). Brainstorm/lsd still
|
||||
// work without anti-bias context — orchestrator stderr-warns at run time.
|
||||
return {
|
||||
name: 'brainstorm_health',
|
||||
status: 'ok',
|
||||
message: `Migration v79 applied; tracking enabled. calibration_profiles table missing (pre-v0.36.1 brain) — judge runs unbiased.`,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.36.0.0 (A5): ze_embedding_health doctor check.
|
||||
*
|
||||
@@ -3391,6 +3504,9 @@ export async function runDoctor(engine: BrainEngine | null, args: string[], dbSo
|
||||
// v0.35.0.0+ reranker_health — read JSONL audit; warn on auth or volume.
|
||||
progress.heartbeat('reranker_health');
|
||||
checks.push(await checkRerankerHealth(engine));
|
||||
// v0.37.0 brainstorm_health — migration v79, track_retrieval, calibration cold-start.
|
||||
progress.heartbeat('brainstorm_health');
|
||||
checks.push(await checkBrainstormHealth(engine));
|
||||
// v0.36.0.0 (A5): ZE embedding key health + schema/config width consistency.
|
||||
progress.heartbeat('ze_embedding_health');
|
||||
checks.push(await checkZeEmbeddingHealth(engine));
|
||||
|
||||
@@ -0,0 +1,399 @@
|
||||
/**
|
||||
* `gbrain eval brainstorm` — three-axis evaluation gate for the v0.37.0
|
||||
* brainstorm + LSD wave (D3 + codex round 2 #11).
|
||||
*
|
||||
* Three independent axes (conjunctive pass — ALL three must clear):
|
||||
*
|
||||
* 1. DISTANCE — mean(distance_score) of passing ideas. Anchored against
|
||||
* Open Collider's published 4-13× distance shift over default-prompt
|
||||
* baseline. Target ≥0.4 (a normalized 0-1 distance; the 4× floor in
|
||||
* Open Collider's space maps roughly to the upper half of our [0,1]).
|
||||
*
|
||||
* 2. USEFULNESS — mean weighted-score from the brainstorm judge over the
|
||||
* held-out fixture. Target ≥3.5/5. Below this the ideas may be far
|
||||
* but they aren't worth reading.
|
||||
*
|
||||
* 3. GROUNDING — fraction of ideas that cite ≥1 real slug from the test
|
||||
* brain. Target 1.0 (every idea must cite a real slug). Without this
|
||||
* we ship a feature that's gameable: a model returning "fluff that
|
||||
* doesn't reference your notes" would still pass distance + usefulness.
|
||||
*
|
||||
* Distance alone is gameable (codex r2 #11) — that's why this is a
|
||||
* three-axis gate rather than the single-axis "4-13× distance shift" that
|
||||
* Open Collider's paper anchors on.
|
||||
*
|
||||
* The fixture is JSONL with one object per line: `{question}` (minimum).
|
||||
* Optional `expected_far_prefixes` for spot-check assertions on which
|
||||
* brain regions the domain-bank should surface.
|
||||
*
|
||||
* Exit codes mirror gbrain's eval convention:
|
||||
* 0 = pass (all three thresholds clear)
|
||||
* 1 = fail (at least one axis below threshold)
|
||||
* 2 = inconclusive (fewer than 2 fixtures produced parseable results)
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../core/engine.ts';
|
||||
import { readFileSync, existsSync } from 'fs';
|
||||
import { runBrainstorm, BRAINSTORM_PROFILE } from '../core/brainstorm/orchestrator.ts';
|
||||
import { loadConfig } from '../core/config.ts';
|
||||
|
||||
export interface BrainstormEvalFixture {
|
||||
question: string;
|
||||
/** Optional: prefixes the domain-bank should surface (asserted in --strict mode). */
|
||||
expected_far_prefixes?: string[];
|
||||
notes?: string;
|
||||
}
|
||||
|
||||
export interface BrainstormEvalThresholds {
|
||||
/** Mean normalized distance of passing ideas. Default 0.4. */
|
||||
distance_min: number;
|
||||
/** Mean judge weighted_score of passing ideas. Default 3.5. */
|
||||
usefulness_min: number;
|
||||
/** Fraction of ideas that cite a real slug from the brain. Default 1.0. */
|
||||
grounding_min: number;
|
||||
}
|
||||
|
||||
export const DEFAULT_BRAINSTORM_THRESHOLDS: BrainstormEvalThresholds = Object.freeze({
|
||||
distance_min: 0.4,
|
||||
usefulness_min: 3.5,
|
||||
grounding_min: 1.0,
|
||||
});
|
||||
|
||||
export interface PerFixtureResult {
|
||||
question: string;
|
||||
/** Number of generated ideas that passed the judge threshold. */
|
||||
pass_count: number;
|
||||
/** Number of ideas total (before judge filtering). */
|
||||
total_ideas: number;
|
||||
/** Mean distance_score over passing ideas. */
|
||||
mean_distance: number;
|
||||
/** Mean judge weighted_score over passing ideas (NaN if no judge scores). */
|
||||
mean_usefulness: number;
|
||||
/** Fraction of ideas citing a real slug. */
|
||||
grounding_rate: number;
|
||||
/** Did the domain-bank surface enough prefixes? D11 sparse signal. */
|
||||
short_of_target: boolean;
|
||||
/** Cost in USD (actual). */
|
||||
cost_usd: number;
|
||||
/** True iff judge failed mid-run for this fixture. */
|
||||
judge_failed: boolean;
|
||||
}
|
||||
|
||||
export interface BrainstormEvalReport {
|
||||
schema_version: 1;
|
||||
fixture_path: string;
|
||||
total_fixtures: number;
|
||||
parseable_fixtures: number;
|
||||
thresholds: BrainstormEvalThresholds;
|
||||
per_fixture: PerFixtureResult[];
|
||||
aggregate: {
|
||||
distance: number;
|
||||
usefulness: number;
|
||||
grounding: number;
|
||||
};
|
||||
/** pass / fail / inconclusive. */
|
||||
verdict: 'pass' | 'fail' | 'inconclusive';
|
||||
/** Reasons supporting the verdict. */
|
||||
reasons: string[];
|
||||
total_cost_usd: number;
|
||||
}
|
||||
|
||||
export interface EvalBrainstormCliArgs {
|
||||
fixture?: string;
|
||||
json: boolean;
|
||||
limit?: number;
|
||||
help: boolean;
|
||||
error?: string;
|
||||
distance_min?: number;
|
||||
usefulness_min?: number;
|
||||
grounding_min?: number;
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): EvalBrainstormCliArgs {
|
||||
const out: EvalBrainstormCliArgs = { json: false, help: false };
|
||||
const positional: string[] = [];
|
||||
let i = 0;
|
||||
while (i < args.length) {
|
||||
const a = args[i];
|
||||
if (a === '--help' || a === '-h') {
|
||||
out.help = true;
|
||||
} else if (a === '--json') {
|
||||
out.json = true;
|
||||
} else if (a === '--limit') {
|
||||
const v = args[++i];
|
||||
const n = v ? parseInt(v, 10) : NaN;
|
||||
if (!Number.isFinite(n) || n <= 0) {
|
||||
out.error = `--limit requires a positive integer (got ${v})`;
|
||||
return out;
|
||||
}
|
||||
out.limit = n;
|
||||
} else if (a === '--distance-min') {
|
||||
const n = parseFloat(args[++i]);
|
||||
if (!Number.isFinite(n)) { out.error = '--distance-min requires a number'; return out; }
|
||||
out.distance_min = n;
|
||||
} else if (a === '--usefulness-min') {
|
||||
const n = parseFloat(args[++i]);
|
||||
if (!Number.isFinite(n)) { out.error = '--usefulness-min requires a number'; return out; }
|
||||
out.usefulness_min = n;
|
||||
} else if (a === '--grounding-min') {
|
||||
const n = parseFloat(args[++i]);
|
||||
if (!Number.isFinite(n)) { out.error = '--grounding-min requires a number'; return out; }
|
||||
out.grounding_min = n;
|
||||
} else if (a.startsWith('--')) {
|
||||
out.error = `unknown flag: ${a}`;
|
||||
return out;
|
||||
} else {
|
||||
positional.push(a);
|
||||
}
|
||||
i++;
|
||||
}
|
||||
if (positional.length > 0) out.fixture = positional[0];
|
||||
return out;
|
||||
}
|
||||
|
||||
const HELP = `Usage: gbrain eval brainstorm <fixture.jsonl> [options]
|
||||
|
||||
Three-axis evaluation gate for \`gbrain brainstorm\`. Runs the fixture's
|
||||
questions through the brainstorm orchestrator, then scores DISTANCE +
|
||||
USEFULNESS + GROUNDING against pass thresholds. All three must clear.
|
||||
|
||||
Fixture format (JSONL, one object per line):
|
||||
{"question": "why are AI coding tools converging on the same UX?"}
|
||||
{"question": "the unspoken assumption in venture pricing"}
|
||||
|
||||
Options:
|
||||
--limit N Cap to N fixtures (default: all)
|
||||
--json Emit BrainstormEvalReport as JSON
|
||||
--distance-min X Override distance threshold (default 0.4)
|
||||
--usefulness-min X Override usefulness threshold (default 3.5)
|
||||
--grounding-min X Override grounding threshold (default 1.0)
|
||||
--help, -h Show this help
|
||||
|
||||
Exit codes: 0 pass, 1 fail, 2 inconclusive (< 2 fixtures parseable).
|
||||
Cost: ~$0.05-0.15 per fixture × N — budget accordingly.
|
||||
`;
|
||||
|
||||
/** Parse JSONL fixture file. Skips blank lines and lines that fail JSON.parse. */
|
||||
export function readBrainstormEvalFixture(path: string): BrainstormEvalFixture[] {
|
||||
if (!existsSync(path)) {
|
||||
throw new Error(`fixture not found: ${path}`);
|
||||
}
|
||||
const text = readFileSync(path, 'utf8');
|
||||
const fixtures: BrainstormEvalFixture[] = [];
|
||||
for (const line of text.split(/\r?\n/)) {
|
||||
const trimmed = line.trim();
|
||||
if (!trimmed) continue;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(trimmed);
|
||||
} catch {
|
||||
continue; // skip malformed rows; we report parseable_fixtures separately
|
||||
}
|
||||
if (typeof parsed !== 'object' || parsed === null) continue;
|
||||
const obj = parsed as Record<string, unknown>;
|
||||
if (typeof obj.question !== 'string' || obj.question.trim().length === 0) continue;
|
||||
fixtures.push({
|
||||
question: obj.question,
|
||||
expected_far_prefixes: Array.isArray(obj.expected_far_prefixes)
|
||||
? obj.expected_far_prefixes.filter((x): x is string => typeof x === 'string')
|
||||
: undefined,
|
||||
notes: typeof obj.notes === 'string' ? obj.notes : undefined,
|
||||
});
|
||||
}
|
||||
return fixtures;
|
||||
}
|
||||
|
||||
/** Verify grounding: a real slug means a slug present in `realSlugs`. Returns the fraction of ideas citing ≥1 real slug. */
|
||||
export function computeGroundingRate(
|
||||
ideas: Array<{ close_slug: string; far_slug: string }>,
|
||||
realSlugs: Set<string>
|
||||
): number {
|
||||
if (ideas.length === 0) return 0;
|
||||
let grounded = 0;
|
||||
for (const idea of ideas) {
|
||||
const closeReal = realSlugs.has(idea.close_slug);
|
||||
const farReal = realSlugs.has(idea.far_slug);
|
||||
if (closeReal || farReal) grounded++;
|
||||
}
|
||||
return grounded / ideas.length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Aggregate one fixture's brainstorm result into the three-axis metrics.
|
||||
* `realSlugs` is the set of slugs known to exist in the test brain (so we
|
||||
* can detect hallucinated citations — codex r2 #11's grounding signal).
|
||||
*/
|
||||
export function summarizeFixture(
|
||||
question: string,
|
||||
result: Awaited<ReturnType<typeof runBrainstorm>>,
|
||||
realSlugs: Set<string>
|
||||
): PerFixtureResult {
|
||||
const passing = result.ideas.filter((i) => i.passes);
|
||||
const meanDistance = passing.length === 0
|
||||
? 0
|
||||
: passing.reduce((s, i) => s + i.distance_score, 0) / passing.length;
|
||||
const judgedPassing = passing.filter((i) => i.judge !== undefined);
|
||||
const meanUsefulness = judgedPassing.length === 0
|
||||
? Number.NaN
|
||||
: judgedPassing.reduce((s, i) => s + (i.judge!.weighted_score), 0) / judgedPassing.length;
|
||||
const grounding = computeGroundingRate(result.ideas, realSlugs);
|
||||
|
||||
return {
|
||||
question,
|
||||
pass_count: passing.length,
|
||||
total_ideas: result.ideas.length,
|
||||
mean_distance: meanDistance,
|
||||
mean_usefulness: meanUsefulness,
|
||||
grounding_rate: grounding,
|
||||
short_of_target: result.short_of_target,
|
||||
cost_usd: result.cost.actual_usd,
|
||||
judge_failed: result.judge_failed,
|
||||
};
|
||||
}
|
||||
|
||||
/** Compute the three aggregate axes across all parseable fixtures + the verdict. */
|
||||
export function computeVerdict(
|
||||
perFixture: PerFixtureResult[],
|
||||
thresholds: BrainstormEvalThresholds
|
||||
): { aggregate: { distance: number; usefulness: number; grounding: number }; verdict: 'pass' | 'fail' | 'inconclusive'; reasons: string[] } {
|
||||
const usable = perFixture.filter((r) => r.pass_count > 0 && !r.judge_failed);
|
||||
if (usable.length < 2) {
|
||||
return {
|
||||
aggregate: { distance: 0, usefulness: 0, grounding: 0 },
|
||||
verdict: 'inconclusive',
|
||||
reasons: [`Only ${usable.length} fixture(s) produced parseable, judged ideas. Need at least 2 to compute meaningful aggregates.`],
|
||||
};
|
||||
}
|
||||
const distance = usable.reduce((s, r) => s + r.mean_distance, 0) / usable.length;
|
||||
const validUseful = usable.filter((r) => Number.isFinite(r.mean_usefulness));
|
||||
const usefulness = validUseful.length === 0
|
||||
? 0
|
||||
: validUseful.reduce((s, r) => s + r.mean_usefulness, 0) / validUseful.length;
|
||||
const grounding = usable.reduce((s, r) => s + r.grounding_rate, 0) / usable.length;
|
||||
|
||||
const reasons: string[] = [];
|
||||
if (distance < thresholds.distance_min) {
|
||||
reasons.push(`distance ${distance.toFixed(3)} < ${thresholds.distance_min} (ideas too close to the question — domain-bank not surfacing distant pages)`);
|
||||
}
|
||||
if (usefulness < thresholds.usefulness_min) {
|
||||
reasons.push(`usefulness ${usefulness.toFixed(2)} < ${thresholds.usefulness_min} (ideas far but low judge score)`);
|
||||
}
|
||||
if (grounding < thresholds.grounding_min) {
|
||||
reasons.push(`grounding ${grounding.toFixed(3)} < ${thresholds.grounding_min} (some ideas cite non-existent slugs — hallucination signal)`);
|
||||
}
|
||||
if (reasons.length > 0) {
|
||||
return { aggregate: { distance, usefulness, grounding }, verdict: 'fail', reasons };
|
||||
}
|
||||
return {
|
||||
aggregate: { distance, usefulness, grounding },
|
||||
verdict: 'pass',
|
||||
reasons: [`all three axes cleared: distance ${distance.toFixed(3)} >= ${thresholds.distance_min}, usefulness ${usefulness.toFixed(2)} >= ${thresholds.usefulness_min}, grounding ${grounding.toFixed(3)} >= ${thresholds.grounding_min}`],
|
||||
};
|
||||
}
|
||||
|
||||
/** Build the set of slugs known to exist in the brain (for grounding check). */
|
||||
async function loadRealSlugs(engine: BrainEngine): Promise<Set<string>> {
|
||||
const refs = await engine.listAllPageRefs();
|
||||
return new Set(refs.map((r) => r.slug));
|
||||
}
|
||||
|
||||
/** CLI entry. Exits with 0/1/2 mirroring eval convention. */
|
||||
export async function runEvalBrainstorm(engine: BrainEngine, args: string[]): Promise<number> {
|
||||
const parsed = parseArgs(args);
|
||||
if (parsed.help) {
|
||||
console.log(HELP);
|
||||
return 0;
|
||||
}
|
||||
if (parsed.error) {
|
||||
console.error(`gbrain eval brainstorm: ${parsed.error}`);
|
||||
console.error(HELP);
|
||||
return 2;
|
||||
}
|
||||
if (!parsed.fixture) {
|
||||
console.error('gbrain eval brainstorm: fixture path required');
|
||||
console.error(HELP);
|
||||
return 2;
|
||||
}
|
||||
let fixtures: BrainstormEvalFixture[];
|
||||
try {
|
||||
fixtures = readBrainstormEvalFixture(parsed.fixture);
|
||||
} catch (err) {
|
||||
console.error(`gbrain eval brainstorm: ${err instanceof Error ? err.message : err}`);
|
||||
return 2;
|
||||
}
|
||||
const thresholds: BrainstormEvalThresholds = {
|
||||
distance_min: parsed.distance_min ?? DEFAULT_BRAINSTORM_THRESHOLDS.distance_min,
|
||||
usefulness_min: parsed.usefulness_min ?? DEFAULT_BRAINSTORM_THRESHOLDS.usefulness_min,
|
||||
grounding_min: parsed.grounding_min ?? DEFAULT_BRAINSTORM_THRESHOLDS.grounding_min,
|
||||
};
|
||||
const slice = parsed.limit ? fixtures.slice(0, parsed.limit) : fixtures;
|
||||
|
||||
const config = loadConfig() ?? {};
|
||||
const realSlugs = await loadRealSlugs(engine);
|
||||
|
||||
const perFixture: PerFixtureResult[] = [];
|
||||
let totalCost = 0;
|
||||
for (const [idx, fix] of slice.entries()) {
|
||||
if (!parsed.json) {
|
||||
console.error(`[eval-brainstorm] ${idx + 1}/${slice.length}: ${fix.question.slice(0, 60)}...`);
|
||||
}
|
||||
try {
|
||||
const result = await runBrainstorm(engine, config, {
|
||||
question: fix.question,
|
||||
profile: BRAINSTORM_PROFILE,
|
||||
skipCostPreview: true,
|
||||
});
|
||||
const summary = summarizeFixture(fix.question, result, realSlugs);
|
||||
perFixture.push(summary);
|
||||
totalCost += summary.cost_usd;
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
if (!parsed.json) {
|
||||
console.error(`[eval-brainstorm] fixture ${idx + 1} failed: ${msg}`);
|
||||
}
|
||||
perFixture.push({
|
||||
question: fix.question,
|
||||
pass_count: 0,
|
||||
total_ideas: 0,
|
||||
mean_distance: 0,
|
||||
mean_usefulness: Number.NaN,
|
||||
grounding_rate: 0,
|
||||
short_of_target: false,
|
||||
cost_usd: 0,
|
||||
judge_failed: true,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const { aggregate, verdict, reasons } = computeVerdict(perFixture, thresholds);
|
||||
const report: BrainstormEvalReport = {
|
||||
schema_version: 1,
|
||||
fixture_path: parsed.fixture,
|
||||
total_fixtures: fixtures.length,
|
||||
parseable_fixtures: perFixture.filter((r) => !r.judge_failed && r.total_ideas > 0).length,
|
||||
thresholds,
|
||||
per_fixture: perFixture,
|
||||
aggregate,
|
||||
verdict,
|
||||
reasons,
|
||||
total_cost_usd: totalCost,
|
||||
};
|
||||
|
||||
if (parsed.json) {
|
||||
console.log(JSON.stringify(report, null, 2));
|
||||
} else {
|
||||
console.log(`\n=== gbrain eval brainstorm ===`);
|
||||
console.log(`Fixture: ${parsed.fixture}`);
|
||||
console.log(`Parseable: ${report.parseable_fixtures}/${report.total_fixtures}`);
|
||||
console.log(`Distance: ${aggregate.distance.toFixed(3)} (threshold ${thresholds.distance_min})`);
|
||||
console.log(`Usefulness: ${aggregate.usefulness.toFixed(2)} (threshold ${thresholds.usefulness_min})`);
|
||||
console.log(`Grounding: ${aggregate.grounding.toFixed(3)} (threshold ${thresholds.grounding_min})`);
|
||||
console.log(`Cost: $${totalCost.toFixed(2)}`);
|
||||
console.log(`Verdict: ${verdict.toUpperCase()}`);
|
||||
for (const r of reasons) {
|
||||
console.log(` - ${r}`);
|
||||
}
|
||||
}
|
||||
|
||||
return verdict === 'pass' ? 0 : verdict === 'fail' ? 1 : 2;
|
||||
}
|
||||
@@ -53,6 +53,14 @@ export async function runEvalCommand(engine: BrainEngine, args: string[]): Promi
|
||||
const { runEvalCodeRetrieval } = await import('./eval-code-retrieval.ts');
|
||||
return runEvalCodeRetrieval(engine, args.slice(1));
|
||||
}
|
||||
if (sub === 'brainstorm') {
|
||||
// v0.37.0 (D3 + codex r2 #11) — three-axis evaluation gate for the
|
||||
// brainstorm + LSD wave. Engine connected (calls hybridSearch +
|
||||
// listAllPageRefs for grounding signal). Exit code mirrors eval
|
||||
// convention: 0 pass, 1 fail, 2 inconclusive.
|
||||
const { runEvalBrainstorm } = await import('./eval-brainstorm.ts');
|
||||
process.exit(await runEvalBrainstorm(engine, args.slice(1)));
|
||||
}
|
||||
if (sub === 'whoknows') {
|
||||
// v0.33 two-layer eval gate (ENG-D2): hand-labeled fixture =
|
||||
// quality, eval_candidates replay = regression. Pass criteria
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
/**
|
||||
* gbrain lsd — Lateral Synaptic Drift.
|
||||
*
|
||||
* Thin re-export of the LSD profile entry point from brainstorm.ts. The
|
||||
* orchestrator + judges are shared (per D6 + D14); only the profile defaults
|
||||
* differ. Lives in its own file so `gbrain lsd --help` reads cleanly via the
|
||||
* CLI dispatch table and the user sees `lsd` as a first-class command, not
|
||||
* a flag on brainstorm.
|
||||
*
|
||||
* See `src/commands/brainstorm.ts` for the shared CLI body and
|
||||
* `src/core/brainstorm/orchestrator.ts` for the profile configs.
|
||||
*/
|
||||
|
||||
export { runLsdCommand } from './brainstorm.ts';
|
||||
@@ -0,0 +1,437 @@
|
||||
/**
|
||||
* v0.37.0 — domain-bank: prefix-stratified far-page retrieval for
|
||||
* `gbrain brainstorm` + `gbrain lsd` (D14).
|
||||
*
|
||||
* Orthogonal to hybridSearch (codex round 1 #1+#2+#3 fix). Does NOT touch
|
||||
* the search cache, SearchOpts, or post-fusion stages. Pulls "far" pages
|
||||
* directly from the corpus via two complementary strategies:
|
||||
*
|
||||
* 1. PRIMARY: prefix-stratified sampling. One page per distinct top-level
|
||||
* slug prefix (`wiki/vc`, `wiki/biology`, `concepts/`, etc) — the
|
||||
* user's own brain organization IS the domain bank. Tiebroken by
|
||||
* `connection_count` (inbound link count via JOIN to page_links,
|
||||
* D10). LSD mode adds stale-bias (D5 / D15) preferring forgotten pages.
|
||||
*
|
||||
* 2. FALLBACK: corpus-sampling. When primary returns < M (small brain,
|
||||
* single-prefix corpus, or close-set ate all the prefixes), random-
|
||||
* sample additional pages to fill the target. Distance-filtered against
|
||||
* the question + close-set so we still get "far" pages, just not from
|
||||
* distinct prefixes.
|
||||
*
|
||||
* 3. SPARSE WARNING: when even fallback can't fill M (D11), emit a
|
||||
* data-driven stderr warning and proceed with what we have. Do NOT
|
||||
* fall back to LLM-invented domains — that undercuts the brain-native
|
||||
* thesis.
|
||||
*
|
||||
* Prefix enumeration cached in the `config` table with 1h TTL (D3 +
|
||||
* codex round 2 #7 — source-scoped). Cache miss does a full
|
||||
* `SELECT DISTINCT substring(slug from '^[^/]+/[^/]+') FROM pages` (~100-500ms
|
||||
* on 30K-page brains); 99%+ of brainstorm calls hit the cache.
|
||||
*
|
||||
* Far-page content sanitized via INJECTION_PATTERNS from `think/sanitize.ts`
|
||||
* — same trust boundary as takes content (v0.28.8).
|
||||
*
|
||||
* Distance score normalized per codex round 2 #9:
|
||||
* `distance_score = 1 - clamp(cosine_distance, 0, 2) / 2`
|
||||
* Range [0, 1] where 1 = orthogonal/opposite, 0 = identical. Powers D6's
|
||||
* citation badges so users see how far each collision actually traveled.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type { DomainBankRow } from '../types.ts';
|
||||
import { INJECTION_PATTERNS } from '../think/sanitize.ts';
|
||||
|
||||
/** Default 1-hour TTL for the prefix-enumeration cache (D3). */
|
||||
export const PREFIX_CACHE_TTL_MS = 60 * 60 * 1000;
|
||||
|
||||
/** Per-far-page content cap before injection into the LLM prompt. */
|
||||
const FAR_CONTENT_LENGTH_CAP = 4000;
|
||||
|
||||
/** Close-set ref the orchestrator passes for distance calc + prefix exclusion. */
|
||||
export interface CloseRef {
|
||||
slug: string;
|
||||
/** Used to exclude this prefix from primary path so close + far don't overlap. */
|
||||
prefix?: string | null;
|
||||
/** Used as distance reference; null when the close-page has no embedded chunks. */
|
||||
representative_chunk_id?: number | null;
|
||||
}
|
||||
|
||||
/** Caller-facing options for the domain-bank orchestrator entry point. */
|
||||
export interface FetchFarOpts {
|
||||
/** Target far-page count. brainstorm=6, lsd=12 (per D14 / plan). */
|
||||
m: number;
|
||||
/** Close-set from hybridSearch (used for exclusion + distance ref). */
|
||||
closeSet: CloseRef[];
|
||||
/** Question embedding; used as the distance anchor when close-set is empty (LSD K=0). */
|
||||
questionEmbedding: Float32Array | null;
|
||||
/** When true (LSD), prefer never-retrieved or stale-by-N-days pages. */
|
||||
staleBias?: boolean;
|
||||
/** Stale-bias day threshold. Default 90. */
|
||||
staleThresholdDays?: number;
|
||||
/** Source scope (canonical scalar). */
|
||||
sourceId?: string;
|
||||
/** Federated read scope (array). */
|
||||
sourceIds?: string[];
|
||||
/** Override the prefix-cache TTL (tests only). */
|
||||
prefixCacheTtlMs?: number;
|
||||
/** Override the prefix list (tests — bypasses cache + enumeration). */
|
||||
prefixListOverride?: string[];
|
||||
/** Default embedding column for distance calc + getEmbeddingsByChunkIds lookup. */
|
||||
embeddingColumn?: string;
|
||||
}
|
||||
|
||||
/** One far-page result enriched with distance + provenance. */
|
||||
export interface FarPage {
|
||||
slug: string;
|
||||
source_id: string;
|
||||
prefix: string | null;
|
||||
page_id: number;
|
||||
title: string | null;
|
||||
/** INJECTION_PATTERNS-sanitized, length-capped. Safe to embed in an LLM prompt. */
|
||||
content: string;
|
||||
/** Cosine distance from this page to the closest of the close-set (or question if close-set empty). Normalized 0-1 per codex r2 #9. */
|
||||
distance_score: number;
|
||||
/** Inbound link count via page_links (tiebreaker, exposed for citation transparency). */
|
||||
connection_count: number;
|
||||
/** When this page was last surfaced by a user-facing op. Null = never retrieved (LSD's prized signal). */
|
||||
last_retrieved_at: Date | null;
|
||||
/** Which sampling strategy produced this page. */
|
||||
source: 'prefix-stratified' | 'corpus-sample';
|
||||
}
|
||||
|
||||
/** Top-level orchestrator return. */
|
||||
export interface FetchFarResult {
|
||||
pages: FarPage[];
|
||||
/** Distinct prefixes available in the brain's source scope (after close-set exclusion). */
|
||||
available_prefixes: number;
|
||||
/** Distinct prefixes total before close-set exclusion. */
|
||||
total_prefixes: number;
|
||||
/** True iff corpus-sampling fallback fired. */
|
||||
used_fallback: boolean;
|
||||
/** True iff result still short of `m` after fallback. Triggers D11 stderr warn. */
|
||||
short_of_target: boolean;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prefix cache (D3 + codex r2 #7 source-scoped)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
interface PrefixCacheEntry {
|
||||
prefixes: string[];
|
||||
cached_at: number;
|
||||
}
|
||||
|
||||
function prefixCacheKey(sourceId: string | undefined): string {
|
||||
// Source-scoped per codex round 2 #7. Federated/mounted brains MUST use
|
||||
// the per-source key to avoid serving prefixes from a foreign source.
|
||||
return `brainstorm.domain_bank.prefixes:${sourceId ?? 'default'}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Read + validate the cached prefix list for `sourceId`. Returns null on
|
||||
* cache miss, expired entry, parse failure, or invalid shape.
|
||||
*/
|
||||
async function readPrefixCache(
|
||||
engine: BrainEngine,
|
||||
sourceId: string | undefined,
|
||||
ttlMs: number
|
||||
): Promise<string[] | null> {
|
||||
let raw: string | null;
|
||||
try {
|
||||
raw = await engine.getConfig(prefixCacheKey(sourceId));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (!raw) return null;
|
||||
let parsed: unknown;
|
||||
try {
|
||||
parsed = JSON.parse(raw);
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (
|
||||
typeof parsed !== 'object'
|
||||
|| parsed === null
|
||||
|| !Array.isArray((parsed as PrefixCacheEntry).prefixes)
|
||||
|| typeof (parsed as PrefixCacheEntry).cached_at !== 'number'
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
const entry = parsed as PrefixCacheEntry;
|
||||
if (Date.now() - entry.cached_at > ttlMs) return null;
|
||||
// Type-narrow: every entry must be a string.
|
||||
for (const p of entry.prefixes) {
|
||||
if (typeof p !== 'string') return null;
|
||||
}
|
||||
return entry.prefixes;
|
||||
}
|
||||
|
||||
async function writePrefixCache(
|
||||
engine: BrainEngine,
|
||||
sourceId: string | undefined,
|
||||
prefixes: string[]
|
||||
): Promise<void> {
|
||||
const entry: PrefixCacheEntry = { prefixes, cached_at: Date.now() };
|
||||
try {
|
||||
await engine.setConfig(prefixCacheKey(sourceId), JSON.stringify(entry));
|
||||
} catch (err) {
|
||||
// Cache-write failures are non-fatal — next call re-enumerates.
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
console.warn(`[domain-bank] prefix-cache write failed (non-fatal): ${msg}`);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Enumerate distinct top-level prefixes from `pages.slug`. The regex
|
||||
* `^[^/]+/[^/]+` captures the first two segments. Single-segment slugs
|
||||
* (e.g. `alice`) match nothing and are excluded from the bank — they
|
||||
* can't be cross-referenced because they have no domain to compare.
|
||||
*
|
||||
* codex round 2 #7: source-scoped via `WHERE source_id = ANY($1::text[])`
|
||||
* when scope is set; cross-source view otherwise.
|
||||
*/
|
||||
export async function enumeratePrefixes(
|
||||
engine: BrainEngine,
|
||||
opts: { sourceId?: string; sourceIds?: string[] }
|
||||
): Promise<string[]> {
|
||||
const sourceIds = opts.sourceIds ?? null;
|
||||
const sourceId = opts.sourceId ?? null;
|
||||
const rows = await engine.executeRaw<{ prefix: string | null }>(
|
||||
`SELECT DISTINCT substring(slug from '^[^/]+/[^/]+') AS prefix
|
||||
FROM pages
|
||||
WHERE deleted_at IS NULL
|
||||
AND substring(slug from '^[^/]+/[^/]+') IS NOT NULL
|
||||
AND (
|
||||
($1::text[] IS NOT NULL AND source_id = ANY($1::text[]))
|
||||
OR ($1::text[] IS NULL AND $2::text IS NOT NULL AND source_id = $2)
|
||||
OR ($1::text[] IS NULL AND $2::text IS NULL)
|
||||
)
|
||||
ORDER BY prefix`,
|
||||
[sourceIds, sourceId]
|
||||
);
|
||||
return rows
|
||||
.map((r) => r.prefix)
|
||||
.filter((p): p is string => typeof p === 'string' && p.length > 0);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Content sanitization + length cap
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Apply INJECTION_PATTERNS (same as takes content per v0.28.8) and cap at
|
||||
* FAR_CONTENT_LENGTH_CAP. Far-page content goes into the LLM prompt as
|
||||
* "you wrote ...", so the same trust boundary applies.
|
||||
*/
|
||||
function sanitizeFarContent(raw: string): string {
|
||||
let text = raw;
|
||||
for (const p of INJECTION_PATTERNS) {
|
||||
if (p.rx.test(text)) {
|
||||
text = text.replace(p.rx, p.replacement);
|
||||
}
|
||||
}
|
||||
if (text.length > FAR_CONTENT_LENGTH_CAP) {
|
||||
text = text.slice(0, FAR_CONTENT_LENGTH_CAP - 3) + '...';
|
||||
}
|
||||
return text;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cosine distance + normalization (codex round 2 #9)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Cosine distance between two embeddings, normalized to [0, 1] where
|
||||
* 1 = orthogonal/opposite, 0 = identical.
|
||||
*
|
||||
* Test cases pinned per codex r2 #9:
|
||||
* - same-vector → 0
|
||||
* - orthogonal → 0.5
|
||||
* - opposite → 1
|
||||
* - missing-vector → caller skips (returns null at retrieval time)
|
||||
*
|
||||
* Internal cosine_distance = 1 - cos_sim, range [0, 2] for unit-norm
|
||||
* vectors. We clamp + halve so the final distance_score is well-bounded.
|
||||
*/
|
||||
export function normalizedCosineDistance(a: Float32Array, b: Float32Array): number {
|
||||
if (a.length !== b.length) {
|
||||
throw new Error(`normalizedCosineDistance: dim mismatch (${a.length} vs ${b.length})`);
|
||||
}
|
||||
let dot = 0, na = 0, nb = 0;
|
||||
for (let i = 0; i < a.length; i++) {
|
||||
dot += a[i] * b[i];
|
||||
na += a[i] * a[i];
|
||||
nb += b[i] * b[i];
|
||||
}
|
||||
const denom = Math.sqrt(na) * Math.sqrt(nb);
|
||||
if (denom === 0) return 0.5; // zero-vector edge: neutral distance.
|
||||
const cosSim = dot / denom;
|
||||
const cosDist = 1 - cosSim; // [0, 2] for unit-norm; can drift slightly outside.
|
||||
// Clamp to [0, 2] then halve → [0, 1].
|
||||
const clamped = Math.max(0, Math.min(2, cosDist));
|
||||
return clamped / 2;
|
||||
}
|
||||
|
||||
/**
|
||||
* Distance from `farEmbed` to the closest of `refEmbeds`. If `refEmbeds`
|
||||
* is empty, fall back to questionEmbedding. If both empty, return 0.5
|
||||
* (neutral — no reference available).
|
||||
*/
|
||||
function distanceFromClosest(
|
||||
farEmbed: Float32Array | null,
|
||||
refEmbeds: Float32Array[],
|
||||
questionEmbed: Float32Array | null
|
||||
): number {
|
||||
if (!farEmbed) return 0.5; // no embedding on far page; can't compute.
|
||||
if (refEmbeds.length === 0) {
|
||||
if (!questionEmbed) return 0.5;
|
||||
return normalizedCosineDistance(farEmbed, questionEmbed);
|
||||
}
|
||||
let minDist = Infinity;
|
||||
for (const ref of refEmbeds) {
|
||||
const d = normalizedCosineDistance(farEmbed, ref);
|
||||
if (d < minDist) minDist = d;
|
||||
}
|
||||
return minDist;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public orchestrator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Pull M far pages from the brain's source scope. Returns `pages.length <= m`;
|
||||
* caller emits the D11 sparse warning when `short_of_target === true`.
|
||||
*
|
||||
* Empty-brain handling (codex round 2 #6 — refuse cleanly): when the brain
|
||||
* has <K+M total pages OR zero usable prefixes AND the corpus is empty,
|
||||
* we still return an empty result with `short_of_target=true`. The CLI
|
||||
* surfaces the paste-ready hint ("Brain has <K+M pages; try gbrain import").
|
||||
*/
|
||||
export async function fetchFar(
|
||||
engine: BrainEngine,
|
||||
opts: FetchFarOpts
|
||||
): Promise<FetchFarResult> {
|
||||
const m = opts.m;
|
||||
if (m <= 0) {
|
||||
return {
|
||||
pages: [],
|
||||
available_prefixes: 0,
|
||||
total_prefixes: 0,
|
||||
used_fallback: false,
|
||||
short_of_target: false,
|
||||
};
|
||||
}
|
||||
const ttlMs = opts.prefixCacheTtlMs ?? PREFIX_CACHE_TTL_MS;
|
||||
const embeddingColumn = opts.embeddingColumn;
|
||||
|
||||
// ---- Step 1: prefix enumeration (cache → DB) ----
|
||||
let allPrefixes: string[];
|
||||
if (opts.prefixListOverride) {
|
||||
allPrefixes = opts.prefixListOverride;
|
||||
} else {
|
||||
const cached = await readPrefixCache(engine, opts.sourceId, ttlMs);
|
||||
if (cached) {
|
||||
allPrefixes = cached;
|
||||
} else {
|
||||
allPrefixes = await enumeratePrefixes(engine, {
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
void writePrefixCache(engine, opts.sourceId, allPrefixes);
|
||||
}
|
||||
}
|
||||
const totalPrefixes = allPrefixes.length;
|
||||
|
||||
// ---- Step 2: filter prefixes that overlap with close-set ----
|
||||
const closePrefixSet = new Set<string>();
|
||||
for (const c of opts.closeSet) {
|
||||
if (c.prefix) closePrefixSet.add(c.prefix);
|
||||
}
|
||||
const candidatePrefixes = allPrefixes.filter((p) => !closePrefixSet.has(p));
|
||||
const availablePrefixes = candidatePrefixes.length;
|
||||
const closeSlugs = opts.closeSet.map((c) => c.slug);
|
||||
|
||||
// ---- Step 3: primary path — listPrefixSampledPages ----
|
||||
let primaryRows: DomainBankRow[] = [];
|
||||
if (candidatePrefixes.length > 0) {
|
||||
primaryRows = await engine.listPrefixSampledPages({
|
||||
prefixes: candidatePrefixes,
|
||||
excludeSlugs: closeSlugs,
|
||||
staleBias: opts.staleBias,
|
||||
staleThresholdDays: opts.staleThresholdDays,
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
}
|
||||
|
||||
// ---- Step 4: fallback if primary didn't fill M ----
|
||||
let fallbackRows: DomainBankRow[] = [];
|
||||
let usedFallback = false;
|
||||
if (primaryRows.length < m) {
|
||||
const need = m - primaryRows.length;
|
||||
const excludeForFallback = [
|
||||
...closeSlugs,
|
||||
...primaryRows.map((r) => r.slug),
|
||||
];
|
||||
fallbackRows = await engine.listCorpusSample({
|
||||
n: need,
|
||||
excludeSlugs: excludeForFallback,
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
usedFallback = fallbackRows.length > 0;
|
||||
}
|
||||
|
||||
// ---- Step 5: hydrate embeddings for distance calc ----
|
||||
const allRows: Array<{ row: DomainBankRow; src: 'prefix-stratified' | 'corpus-sample' }> = [
|
||||
...primaryRows.map((row) => ({ row, src: 'prefix-stratified' as const })),
|
||||
...fallbackRows.map((row) => ({ row, src: 'corpus-sample' as const })),
|
||||
];
|
||||
|
||||
// Build the chunk-id list for one batched embedding lookup. Skip rows
|
||||
// without a representative chunk (no embedded content).
|
||||
const closeChunkIds = opts.closeSet
|
||||
.map((c) => c.representative_chunk_id)
|
||||
.filter((id): id is number => typeof id === 'number');
|
||||
const farChunkIds = allRows
|
||||
.map((r) => r.row.representative_chunk_id)
|
||||
.filter((id): id is number => typeof id === 'number');
|
||||
const chunkIds = [...new Set([...closeChunkIds, ...farChunkIds])];
|
||||
let embeddings: Map<number, Float32Array> = new Map();
|
||||
if (chunkIds.length > 0) {
|
||||
embeddings = await engine.getEmbeddingsByChunkIds(chunkIds, embeddingColumn);
|
||||
}
|
||||
|
||||
const refEmbeds: Float32Array[] = closeChunkIds
|
||||
.map((id) => embeddings.get(id))
|
||||
.filter((e): e is Float32Array => e !== undefined);
|
||||
|
||||
// ---- Step 6: build FarPage results with normalized distance ----
|
||||
const pages: FarPage[] = allRows.map(({ row, src }) => {
|
||||
const farEmbed = row.representative_chunk_id != null
|
||||
? embeddings.get(row.representative_chunk_id) ?? null
|
||||
: null;
|
||||
const distance_score = distanceFromClosest(farEmbed, refEmbeds, opts.questionEmbedding);
|
||||
return {
|
||||
slug: row.slug,
|
||||
source_id: row.source_id,
|
||||
prefix: row.prefix,
|
||||
page_id: row.page_id,
|
||||
title: row.title,
|
||||
content: sanitizeFarContent(row.compiled_truth),
|
||||
distance_score,
|
||||
connection_count: row.connection_count,
|
||||
last_retrieved_at: row.last_retrieved_at,
|
||||
source: src,
|
||||
};
|
||||
});
|
||||
|
||||
return {
|
||||
pages,
|
||||
available_prefixes: availablePrefixes,
|
||||
total_prefixes: totalPrefixes,
|
||||
used_fallback: usedFallback,
|
||||
short_of_target: pages.length < m,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
/**
|
||||
* v0.37.0 — brainstorm + LSD shared judge (D6 — single file, two configs).
|
||||
*
|
||||
* One `runJudge(config, ideas)` function + two exported configs that flip the
|
||||
* threshold and the inversion rule. ~150 LOC vs the 200 LOC of two parallel
|
||||
* files (~70 LOC duplication eliminated). The brainstorm vs LSD contrast is
|
||||
* visible side-by-side at the config-export site.
|
||||
*
|
||||
* Rubric adapted from Open Collider's judge.md (CL-ML/open-collider, MIT).
|
||||
* Five axes scored 1-5; weighted average; per-config threshold.
|
||||
*
|
||||
* Brainstorm: standard rubric, threshold 4.0 (Open Collider uses 4.2 on
|
||||
* training-data inputs; brain-grounded inputs are inherently more constrained
|
||||
* so we relax slightly).
|
||||
*
|
||||
* LSD: inverted rubric. Rejects ideas with resistance > 4.5 ("too obvious,
|
||||
* you would have thought of this without LSD"). The "productive dissonance"
|
||||
* axis (cognitive_load) is the only one weighted heavily; everything else
|
||||
* permissive. Every output must invert at least one implicit axiom.
|
||||
*
|
||||
* Provider-neutral via `gateway.chat()`. Hermetically testable via the
|
||||
* `chatFn` injection point.
|
||||
*/
|
||||
|
||||
import { chat as defaultChat, type ChatResult, type ChatOpts } from '../ai/gateway.ts';
|
||||
|
||||
export const PROMPT_VERSION = 'brainstorm-judge-v1';
|
||||
|
||||
/** One idea handed to the judge. The orchestrator builds these from the cross output. */
|
||||
export interface JudgeIdea {
|
||||
/** Stable id within this run (e.g. "01", "02"). */
|
||||
id: string;
|
||||
/** Free-form idea text (2-4 sentences). */
|
||||
text: string;
|
||||
/** The (close, far) pair that produced this idea. Surfaces in the prompt for context. */
|
||||
close_slug: string;
|
||||
far_slug: string;
|
||||
}
|
||||
|
||||
/** Per-axis 1-5 score from the LLM. */
|
||||
export interface JudgeAxisScores {
|
||||
/** Is the underlying thesis genuinely new? */
|
||||
originality: number;
|
||||
/** Does the core thesis hold up against the strongest counterargument? */
|
||||
resistance: number;
|
||||
/** Could the idea be formulated as a single testable + refutable thesis? */
|
||||
thesis_density: number;
|
||||
/** Could the idea rely on a specific fact, figure, or named situation? */
|
||||
concrete_grounding: number;
|
||||
/** Does the idea force productive dissonance, or is it immediately expected? */
|
||||
cognitive_load: number;
|
||||
}
|
||||
|
||||
/** Per-idea verdict the orchestrator consumes. */
|
||||
export interface JudgeIdeaResult {
|
||||
id: string;
|
||||
scores: JudgeAxisScores;
|
||||
/** Weighted aggregate per the config's axis weights. */
|
||||
weighted_score: number;
|
||||
/** True iff this idea passes the config's threshold (after inversion rule for LSD). */
|
||||
passes: boolean;
|
||||
/** One-sentence judge note (main strength or rejection reason). */
|
||||
note: string;
|
||||
}
|
||||
|
||||
/** Top-level judge response (one batch). */
|
||||
export interface JudgeResult {
|
||||
ideas: JudgeIdeaResult[];
|
||||
/** Number of input ideas that passed the threshold. */
|
||||
pass_count: number;
|
||||
/** Provider:model that answered (for cost accounting / debugging). */
|
||||
model: string;
|
||||
usage: ChatResult['usage'];
|
||||
}
|
||||
|
||||
/** Brainstorm vs LSD config delta. */
|
||||
export interface JudgeConfig {
|
||||
/** Stable label — flows into the cache key and the run report. */
|
||||
label: 'brainstorm' | 'lsd';
|
||||
/** Axis weights — must sum to 1.0 (validated at module load). */
|
||||
weights: JudgeAxisScores;
|
||||
/** Threshold on the weighted average; ideas below this are filtered. */
|
||||
threshold: number;
|
||||
/**
|
||||
* LSD-only: reject ideas whose `resistance` (coherence) axis exceeds
|
||||
* `rejectIfResistanceAbove`. The Open Collider inversion: in LSD mode,
|
||||
* "too obvious" is the failure mode, not "too incoherent."
|
||||
* Undefined on brainstorm (standard rubric — high resistance is good).
|
||||
*/
|
||||
rejectIfResistanceAbove?: number;
|
||||
/** Append to the rubric prompt — e.g. "every output must invert at least one axiom." */
|
||||
extraInstructions?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Brainstorm config. Mirrors Open Collider's judge.md exactly:
|
||||
* originality 0.25, resistance 0.20, thesis_density 0.20,
|
||||
* concrete_grounding 0.20, cognitive_load 0.15
|
||||
* Threshold 4.0 (relaxed from Open Collider's 4.2 — brain-grounded ideas
|
||||
* carry inherent constraint, so we don't need the extra stringency).
|
||||
*/
|
||||
export const BRAINSTORM_JUDGE_CONFIG: JudgeConfig = Object.freeze({
|
||||
label: 'brainstorm',
|
||||
weights: {
|
||||
originality: 0.25,
|
||||
resistance: 0.20,
|
||||
thesis_density: 0.20,
|
||||
concrete_grounding: 0.20,
|
||||
cognitive_load: 0.15,
|
||||
},
|
||||
threshold: 4.0,
|
||||
});
|
||||
|
||||
/**
|
||||
* LSD config. The "Lateral Synaptic Drift" inversion:
|
||||
* - cognitive_load (productive dissonance) is the only axis weighted heavily.
|
||||
* - resistance > 4.5 ("too obvious") is an automatic rejection — these are
|
||||
* the ideas you'd have surfaced without LSD mode.
|
||||
* - axiomatic inversions required on every output.
|
||||
* - threshold relaxed to 3.5 so weird-but-defensible ideas survive.
|
||||
*/
|
||||
export const LSD_JUDGE_CONFIG: JudgeConfig = Object.freeze({
|
||||
label: 'lsd',
|
||||
weights: {
|
||||
// The "productive dissonance" axis dominates the average.
|
||||
originality: 0.20,
|
||||
resistance: 0.05,
|
||||
thesis_density: 0.15,
|
||||
concrete_grounding: 0.10,
|
||||
cognitive_load: 0.50,
|
||||
},
|
||||
threshold: 3.5,
|
||||
rejectIfResistanceAbove: 4.5,
|
||||
extraInstructions:
|
||||
'Every kept idea MUST invert at least one implicit axiom (X is good → X is the problem; everyone does Y → the opposite; dominant narrative says Z → the hidden cause).',
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Module-load validation: each config's axis weights must sum to 1.0.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function validateConfig(config: JudgeConfig): void {
|
||||
const sum =
|
||||
config.weights.originality +
|
||||
config.weights.resistance +
|
||||
config.weights.thesis_density +
|
||||
config.weights.concrete_grounding +
|
||||
config.weights.cognitive_load;
|
||||
// Allow tiny floating-point tolerance.
|
||||
if (Math.abs(sum - 1.0) > 1e-9) {
|
||||
throw new Error(
|
||||
`[brainstorm/judges] ${config.label} axis weights must sum to 1.0 (got ${sum.toFixed(6)})`
|
||||
);
|
||||
}
|
||||
}
|
||||
validateConfig(BRAINSTORM_JUDGE_CONFIG);
|
||||
validateConfig(LSD_JUDGE_CONFIG);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Prompt construction
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const FENCE_RE = /```(?:json)?\s*\n?([\s\S]*?)```/i;
|
||||
|
||||
function buildJudgePrompt(config: JudgeConfig, ideas: JudgeIdea[]): string {
|
||||
const w = config.weights;
|
||||
const ideasBlock = ideas
|
||||
.map(
|
||||
(idea) =>
|
||||
`## Idea ${idea.id}\n(close=${idea.close_slug} × far=${idea.far_slug})\n${idea.text}`
|
||||
)
|
||||
.join('\n\n');
|
||||
|
||||
const inversionRule = config.rejectIfResistanceAbove !== undefined
|
||||
? `\n\n## LSD INVERSION RULE\nAny idea with resistance > ${config.rejectIfResistanceAbove.toFixed(1)} is REJECTED regardless of weighted score — these are the ideas the user would surface without LSD. "Too obvious" is the failure mode here.`
|
||||
: '';
|
||||
|
||||
const extras = config.extraInstructions ? `\n\n## ADDITIONAL CONSTRAINT\n${config.extraInstructions}` : '';
|
||||
|
||||
return `You are a structural evaluator filtering brainstorm ideas. Score each idea on the underlying potential, not the current wording.
|
||||
|
||||
## AXES (each scored 1-5)
|
||||
|
||||
**Originality (weight ${w.originality.toFixed(2)})** — Is the underlying thesis genuinely new?
|
||||
5: thesis never seen formulated this way
|
||||
3: known angle with new packaging
|
||||
1: reformulation of standard advice
|
||||
|
||||
**Resistance (weight ${w.resistance.toFixed(2)})** — Does the core thesis hold up against the strongest possible objection?
|
||||
5: holds up even against the strongest counterargument
|
||||
3: substance recoverable, current wording doesn't resist
|
||||
1: a single objection collapses the entire idea
|
||||
|
||||
**Thesis density (weight ${w.thesis_density.toFixed(2)})** — Could it be formulated as a single testable + refutable thesis?
|
||||
5: precise thesis identifiable, directly attackable
|
||||
3: implicit thesis, recoverable with reformulation
|
||||
1: observation or anecdote from which no thesis can be extracted
|
||||
|
||||
**Concrete grounding (weight ${w.concrete_grounding.toFixed(2)})** — Could the idea rely on a specific fact, figure, or named situation?
|
||||
5: grounding already present, or obvious + immediately findable evidence
|
||||
3: grounding possible but requires non-trivial research
|
||||
1: pure abstraction, no real data could support it
|
||||
|
||||
**Cognitive load (weight ${w.cognitive_load.toFixed(2)})** — Does the idea force reconstruction, or is it immediately expected?
|
||||
5: productive dissonance — the reader must stop and think
|
||||
3: slightly counter-intuitive
|
||||
1: expected information, no friction${inversionRule}${extras}
|
||||
|
||||
## IDEAS TO EVALUATE
|
||||
|
||||
${ideasBlock}
|
||||
|
||||
## OUTPUT FORMAT (strict JSON, no prose outside the JSON)
|
||||
|
||||
\`\`\`json
|
||||
{
|
||||
"ideas": [
|
||||
{
|
||||
"id": "<idea id>",
|
||||
"scores": {
|
||||
"originality": <1-5>,
|
||||
"resistance": <1-5>,
|
||||
"thesis_density": <1-5>,
|
||||
"concrete_grounding": <1-5>,
|
||||
"cognitive_load": <1-5>
|
||||
},
|
||||
"note": "<one sentence — main strength if passing, rejection reason if not>"
|
||||
}
|
||||
]
|
||||
}
|
||||
\`\`\`
|
||||
|
||||
Respond with ONLY the JSON block, nothing before or after.`;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// JSON parsing — 3-strategy fallback. Throws on unparseable rather than
|
||||
// fabricating a verdict.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function parseJudgeJSON(text: string): unknown {
|
||||
if (!text) throw new Error('parseJudgeJSON: empty response');
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
const fenceMatch = text.match(FENCE_RE);
|
||||
if (fenceMatch && fenceMatch[1]) {
|
||||
try {
|
||||
return JSON.parse(fenceMatch[1].trim());
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
}
|
||||
// Common-repairs pass.
|
||||
const cleaned = text
|
||||
.replace(FENCE_RE, (_, inner) => inner)
|
||||
.replace(/,(\s*[}\]])/g, '$1')
|
||||
.trim();
|
||||
const braceMatch = cleaned.match(/\{[\s\S]*\}/);
|
||||
if (braceMatch) {
|
||||
try {
|
||||
return JSON.parse(braceMatch[0]);
|
||||
} catch {
|
||||
// fall through
|
||||
}
|
||||
}
|
||||
throw new Error('parseJudgeJSON: no strategy produced valid JSON');
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Score-shape validation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
function isAxisScoreInRange(n: unknown): n is number {
|
||||
return typeof n === 'number' && Number.isFinite(n) && n >= 1 && n <= 5;
|
||||
}
|
||||
|
||||
function validateIdeaShape(raw: unknown): { id: string; scores: JudgeAxisScores; note: string } | null {
|
||||
if (typeof raw !== 'object' || raw === null) return null;
|
||||
const r = raw as Record<string, unknown>;
|
||||
if (typeof r.id !== 'string') return null;
|
||||
const note = typeof r.note === 'string' ? r.note : '';
|
||||
const s = r.scores;
|
||||
if (typeof s !== 'object' || s === null) return null;
|
||||
const sr = s as Record<string, unknown>;
|
||||
if (
|
||||
!isAxisScoreInRange(sr.originality)
|
||||
|| !isAxisScoreInRange(sr.resistance)
|
||||
|| !isAxisScoreInRange(sr.thesis_density)
|
||||
|| !isAxisScoreInRange(sr.concrete_grounding)
|
||||
|| !isAxisScoreInRange(sr.cognitive_load)
|
||||
) return null;
|
||||
return {
|
||||
id: r.id,
|
||||
scores: {
|
||||
originality: sr.originality,
|
||||
resistance: sr.resistance,
|
||||
thesis_density: sr.thesis_density,
|
||||
concrete_grounding: sr.concrete_grounding,
|
||||
cognitive_load: sr.cognitive_load,
|
||||
},
|
||||
note,
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Weighted score computation
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export function weightedScore(scores: JudgeAxisScores, weights: JudgeAxisScores): number {
|
||||
return (
|
||||
scores.originality * weights.originality
|
||||
+ scores.resistance * weights.resistance
|
||||
+ scores.thesis_density * weights.thesis_density
|
||||
+ scores.concrete_grounding * weights.concrete_grounding
|
||||
+ scores.cognitive_load * weights.cognitive_load
|
||||
);
|
||||
}
|
||||
|
||||
/** Per-config passing rule. LSD additionally enforces the inversion-rule resistance ceiling. */
|
||||
export function ideaPasses(idea: JudgeIdeaResult, config: JudgeConfig): boolean {
|
||||
if (idea.weighted_score < config.threshold) return false;
|
||||
if (
|
||||
config.rejectIfResistanceAbove !== undefined
|
||||
&& idea.scores.resistance > config.rejectIfResistanceAbove
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public orchestrator
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Test seam — swap the chat call without touching the real gateway. */
|
||||
export type ChatFn = (opts: ChatOpts) => Promise<ChatResult>;
|
||||
|
||||
export interface RunJudgeOptions {
|
||||
/** Override the gateway's default chat model (e.g. for `gbrain models doctor` probes). */
|
||||
modelOverride?: string;
|
||||
/** Hermetic test seam. Production callers MUST NOT pass this. */
|
||||
chatFn?: ChatFn;
|
||||
/** Anti-bias context from the user's calibration profile (D4 + codex #8). */
|
||||
activeBiasTags?: string[];
|
||||
/** AbortSignal for Ctrl-C / shutdown propagation. */
|
||||
abortSignal?: AbortSignal;
|
||||
}
|
||||
|
||||
/**
|
||||
* Single batch — caller chunks large idea sets to keep prompt size bounded.
|
||||
* Throws on parse failure (caller maps to judge_failed:true + saves unscored,
|
||||
* per D12).
|
||||
*/
|
||||
export async function runJudge(
|
||||
config: JudgeConfig,
|
||||
ideas: JudgeIdea[],
|
||||
options: RunJudgeOptions = {}
|
||||
): Promise<JudgeResult> {
|
||||
if (ideas.length === 0) {
|
||||
// Empty input is a no-op success; callers can short-circuit too but
|
||||
// returning a well-formed empty result is more ergonomic.
|
||||
return { ideas: [], pass_count: 0, model: 'noop', usage: { input_tokens: 0, output_tokens: 0, cache_read_tokens: 0, cache_creation_tokens: 0 } };
|
||||
}
|
||||
const chat = options.chatFn ?? defaultChat;
|
||||
const prompt = buildJudgePrompt(config, ideas);
|
||||
|
||||
// Anti-bias context (D4 + codex #8): inject the user's known biases so
|
||||
// the judge penalizes ideas that play to them. Cold-start (empty array)
|
||||
// falls through with no anti-bias context — orchestrator stderr-warns.
|
||||
const system = (options.activeBiasTags && options.activeBiasTags.length > 0)
|
||||
? `You are scoring ideas for a user with the following known biases: ${options.activeBiasTags.join(', ')}. Penalize the originality axis when an idea closely matches a known bias pattern.`
|
||||
: undefined;
|
||||
|
||||
const result = await chat({
|
||||
model: options.modelOverride,
|
||||
system,
|
||||
messages: [{ role: 'user', content: prompt }],
|
||||
// Judge runs cold (T=0.1) for consistency; the gateway's temperature
|
||||
// knob isn't on ChatOpts (it's set per-provider in instantiateChat),
|
||||
// so we rely on the default. If we ever need temperature control here
|
||||
// we'd extend ChatOpts.
|
||||
maxTokens: 4000,
|
||||
abortSignal: options.abortSignal,
|
||||
});
|
||||
|
||||
const parsed = parseJudgeJSON(result.text);
|
||||
if (typeof parsed !== 'object' || parsed === null || !Array.isArray((parsed as { ideas?: unknown }).ideas)) {
|
||||
throw new Error(`runJudge: response missing 'ideas' array. Got: ${JSON.stringify(parsed).slice(0, 200)}`);
|
||||
}
|
||||
const rawIdeas = (parsed as { ideas: unknown[] }).ideas;
|
||||
|
||||
const ideaResults: JudgeIdeaResult[] = [];
|
||||
for (const raw of rawIdeas) {
|
||||
const validated = validateIdeaShape(raw);
|
||||
if (!validated) {
|
||||
// Skip malformed rows — the orchestrator surfaces a stderr warning if
|
||||
// fewer ideas come back than were submitted.
|
||||
continue;
|
||||
}
|
||||
const weighted_score = weightedScore(validated.scores, config.weights);
|
||||
const result: JudgeIdeaResult = {
|
||||
id: validated.id,
|
||||
scores: validated.scores,
|
||||
weighted_score,
|
||||
passes: false, // filled below
|
||||
note: validated.note,
|
||||
};
|
||||
result.passes = ideaPasses(result, config);
|
||||
ideaResults.push(result);
|
||||
}
|
||||
|
||||
return {
|
||||
ideas: ideaResults,
|
||||
pass_count: ideaResults.filter((i) => i.passes).length,
|
||||
model: result.model,
|
||||
usage: result.usage,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,730 @@
|
||||
/**
|
||||
* v0.37.0 — brainstorm + LSD orchestrator.
|
||||
*
|
||||
* Shared 4-phase pipeline driven by a `BrainstormProfile` config object
|
||||
* (D1 fold — output formatter + cost preview + types live in this file).
|
||||
*
|
||||
* Phase 1: retrieve close-set via hybridSearch (K=4 brainstorm, K=2 LSD).
|
||||
* Phase 2: fetch far-set via domain-bank (M=6 brainstorm, M=12 LSD).
|
||||
* Phase 3: cross-generate ideas via gateway.chat (one call per close x far).
|
||||
* Phase 4: judge via runJudge (D6 single-file, two configs).
|
||||
*
|
||||
* Trust boundary:
|
||||
* - Far-page content goes through INJECTION_PATTERNS in domain-bank.ts.
|
||||
* - Calibration anti-bias context (D4 + codex #8) consulted with cold-start
|
||||
* fallback when active_bias_tags is empty.
|
||||
* - Judge mid-run failure (D12) saves ideas with judge_failed:true.
|
||||
*
|
||||
* Cost guard (D8 + codex r2 #10):
|
||||
* - Pre-run estimate to stderr + 10s TTY grace window (skipped non-TTY).
|
||||
* - End-of-run actuals: "actual cost: $X (estimated $Y)" — operators can
|
||||
* grep for `actual cost:` to track real spend.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { chat as defaultChat, embedQuery, type ChatResult, type ChatOpts } from '../ai/gateway.ts';
|
||||
import { hybridSearch, hybridSearchCached } from '../search/hybrid.ts';
|
||||
import { fetchFar, type CloseRef, type FarPage } from './domain-bank.ts';
|
||||
import {
|
||||
runJudge,
|
||||
BRAINSTORM_JUDGE_CONFIG,
|
||||
LSD_JUDGE_CONFIG,
|
||||
type JudgeIdea,
|
||||
type JudgeIdeaResult,
|
||||
type JudgeConfig,
|
||||
type ChatFn,
|
||||
} from './judges.ts';
|
||||
import { ANTHROPIC_PRICING } from '../anthropic-pricing.ts';
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Profile (BrainstormProfile is the brainstorm vs LSD config object)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface BrainstormProfile {
|
||||
/** Stable label — used in stderr lines, frontmatter, audit. */
|
||||
label: 'brainstorm' | 'lsd';
|
||||
/** Close-set size from hybridSearch. brainstorm=4, lsd=2. */
|
||||
k_close: number;
|
||||
/** Far-set size from domain-bank. brainstorm=6, lsd=12. */
|
||||
m_far: number;
|
||||
/** Ideas to generate per (close × far) cross. */
|
||||
ideas_per_cross: number;
|
||||
/** Generation temperature. brainstorm=0.7 (steady), lsd=0.95 (loose). */
|
||||
temperature: number;
|
||||
/** Domain-bank stale-bias toggle. LSD only. */
|
||||
stale_bias: boolean;
|
||||
/** Judge config (rubric + threshold + LSD inversion rule). */
|
||||
judge_config: JudgeConfig;
|
||||
/** Whether to save by default. brainstorm=true (defensible output), lsd=false (ephemeral). */
|
||||
default_save: boolean;
|
||||
/** Frontmatter `mode:` value the dream-cycle hook reads (D4). */
|
||||
frontmatter_mode: 'brainstorm' | 'lsd';
|
||||
/** Generator system-prompt suffix — what's the vibe? */
|
||||
generator_voice: string;
|
||||
/** Optional generator-side constraint (LSD: axiomatic inversions required). */
|
||||
generator_constraint?: string;
|
||||
}
|
||||
|
||||
export const BRAINSTORM_PROFILE: BrainstormProfile = Object.freeze({
|
||||
label: 'brainstorm',
|
||||
k_close: 4,
|
||||
m_far: 6,
|
||||
ideas_per_cross: 3,
|
||||
temperature: 0.7,
|
||||
stale_bias: false,
|
||||
judge_config: BRAINSTORM_JUDGE_CONFIG,
|
||||
default_save: true,
|
||||
frontmatter_mode: 'brainstorm',
|
||||
generator_voice: 'Defensible, cite-heavy. An analyst riffing with their own notes.',
|
||||
});
|
||||
|
||||
export const LSD_PROFILE: BrainstormProfile = Object.freeze({
|
||||
label: 'lsd',
|
||||
k_close: 2,
|
||||
m_far: 12,
|
||||
ideas_per_cross: 4,
|
||||
temperature: 0.95,
|
||||
stale_bias: true,
|
||||
judge_config: LSD_JUDGE_CONFIG,
|
||||
default_save: false,
|
||||
frontmatter_mode: 'lsd',
|
||||
generator_voice: 'Your brain at 3am noticing a connection between things it has no business connecting.',
|
||||
generator_constraint: 'Every idea MUST invert at least one implicit axiom (X is good → X is the problem; everyone does Y → opposite; dominant narrative → hidden cause).',
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Caller-facing options + result shape
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface BrainstormOptions {
|
||||
question: string;
|
||||
/** Profile selects brainstorm vs LSD; defaults to BRAINSTORM_PROFILE. */
|
||||
profile?: BrainstormProfile;
|
||||
/** Override the default chat model (subject to provider auth). */
|
||||
modelOverride?: string;
|
||||
/** Skip the cost-preview TTY grace window. Required for non-interactive callers. */
|
||||
skipCostPreview?: boolean;
|
||||
/** When set, force the user holder for calibration profile lookup. Falls back to config (`emotional_weight.user_holder`) then `'garry'`. */
|
||||
holderOverride?: string;
|
||||
/** Source scope. */
|
||||
sourceId?: string;
|
||||
sourceIds?: string[];
|
||||
/** AbortSignal for Ctrl-C / shutdown. */
|
||||
abortSignal?: AbortSignal;
|
||||
/** Override the gateway chat fn (tests only). */
|
||||
chatFn?: ChatFn;
|
||||
/** Override the gateway embedQuery fn (tests only). */
|
||||
embedQueryFn?: (text: string) => Promise<Float32Array>;
|
||||
/** Stderr sink — defaults to process.stderr.write. Tests pipe into a buffer. */
|
||||
stderrWrite?: (s: string) => void;
|
||||
}
|
||||
|
||||
/** One idea emitted to the user, with citation transparency (D6). */
|
||||
export interface BrainstormIdea {
|
||||
/** "01" .. "NN", stable within this run. */
|
||||
id: string;
|
||||
/** Free-form idea body (2-4 sentences). */
|
||||
text: string;
|
||||
/** Citation: close-set page slug. */
|
||||
close_slug: string;
|
||||
/** Citation: far-set page slug. */
|
||||
far_slug: string;
|
||||
/** D6 transparency badge — how far this collision actually traveled. */
|
||||
distance_score: number;
|
||||
/** Scoring from the judge. Absent when `judge_failed === true`. */
|
||||
judge?: JudgeIdeaResult;
|
||||
/** True iff this idea passed the judge threshold. */
|
||||
passes: boolean;
|
||||
/** True iff the judge call failed mid-run (D12). When true `judge` is absent and `passes=false`. */
|
||||
judge_failed?: boolean;
|
||||
}
|
||||
|
||||
export interface BrainstormResult {
|
||||
profile_label: 'brainstorm' | 'lsd';
|
||||
question: string;
|
||||
/** Question embedding model used for distance calc. */
|
||||
embedding_model: string | null;
|
||||
/** All generated ideas (passed + filtered). Callers can render only `.filter(i => i.passes)`. */
|
||||
ideas: BrainstormIdea[];
|
||||
/** Close-set citations for the run header. */
|
||||
close_set: Array<{ slug: string; title: string | null }>;
|
||||
/** Far-set citations for the run header. */
|
||||
far_set: Array<{ slug: string; title: string | null; distance_score: number; source: 'prefix-stratified' | 'corpus-sample' }>;
|
||||
/** Calibration context applied during judging; `null` on cold-start. */
|
||||
active_bias_tags: string[] | null;
|
||||
/** D11 sparse signal — true when domain-bank couldn't fill m_far. */
|
||||
short_of_target: boolean;
|
||||
/** True iff judge phase failed and ideas were saved unscored (D12). */
|
||||
judge_failed: boolean;
|
||||
/** Cost actuals (codex r2 #10). */
|
||||
cost: {
|
||||
estimated_usd: number;
|
||||
actual_usd: number;
|
||||
input_tokens: number;
|
||||
output_tokens: number;
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cost preview (D8 + codex r2 #10) — TTY grace + actuals
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Per-profile cost estimate. brainstorm: ~$0.05-0.15. lsd: ~$0.20-0.40.
|
||||
* Real numbers depend on configured model; we anchor on Sonnet pricing.
|
||||
* The estimate is informational — operators see actuals printed at run-end.
|
||||
*/
|
||||
export function estimateCost(profile: BrainstormProfile, model: string): number {
|
||||
const crosses = profile.k_close * profile.m_far;
|
||||
const ideas = crosses * profile.ideas_per_cross;
|
||||
// Rough per-cross budget: ~3K in, ~1.5K out (prompt + ideas).
|
||||
const inTokens = crosses * 3000;
|
||||
const outTokens = ideas * 250;
|
||||
// Judge: one batch ~ all ideas in, ~200 tokens per scored idea out.
|
||||
const judgeIn = ideas * 350;
|
||||
const judgeOut = ideas * 200;
|
||||
|
||||
const pricing = ANTHROPIC_PRICING[model] ?? { input: 3, output: 15 };
|
||||
const inCost = ((inTokens + judgeIn) / 1_000_000) * pricing.input;
|
||||
const outCost = ((outTokens + judgeOut) / 1_000_000) * pricing.output;
|
||||
return inCost + outCost;
|
||||
}
|
||||
|
||||
/** Pretty-print a USD cost with 2 decimals + leading dollar. */
|
||||
function fmtUsd(n: number): string {
|
||||
return `$${n.toFixed(2)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Print the cost estimate + 10s TTY grace window. Non-TTY (cron, scripted)
|
||||
* auto-proceeds. `--yes` short-circuits via `skipCostPreview: true`.
|
||||
*
|
||||
* Returns true iff the user pressed Ctrl-C during the grace window.
|
||||
*/
|
||||
export async function previewCostAndWait(opts: {
|
||||
profile: BrainstormProfile;
|
||||
model: string;
|
||||
skip: boolean;
|
||||
stderrWrite: (s: string) => void;
|
||||
/** Test seam — override the wait so suites don't hang. */
|
||||
graceMs?: number;
|
||||
}): Promise<{ aborted: boolean; estimate: number }> {
|
||||
const estimate = estimateCost(opts.profile, opts.model);
|
||||
const isTTY = typeof process !== 'undefined' && process.stderr?.isTTY === true;
|
||||
opts.stderrWrite(
|
||||
`[${opts.profile.label}] estimated cost: ${fmtUsd(estimate)} (${opts.profile.k_close}×${opts.profile.m_far} = ${opts.profile.k_close * opts.profile.m_far} crosses × ${opts.profile.ideas_per_cross} ideas + judge)\n`
|
||||
);
|
||||
if (opts.skip || !isTTY) {
|
||||
return { aborted: false, estimate };
|
||||
}
|
||||
opts.stderrWrite(`[${opts.profile.label}] Press Ctrl-C within 10s to abort, or wait to proceed...\n`);
|
||||
const ms = opts.graceMs ?? 10_000;
|
||||
return new Promise((resolve) => {
|
||||
const t = setTimeout(() => {
|
||||
cleanup();
|
||||
resolve({ aborted: false, estimate });
|
||||
}, ms);
|
||||
const onSigint = () => {
|
||||
cleanup();
|
||||
opts.stderrWrite(`[${opts.profile.label}] Aborted by user.\n`);
|
||||
resolve({ aborted: true, estimate });
|
||||
};
|
||||
function cleanup() {
|
||||
clearTimeout(t);
|
||||
process.off?.('SIGINT', onSigint);
|
||||
}
|
||||
process.on?.('SIGINT', onSigint);
|
||||
});
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Calibration profile load (D4 + codex #8 cold-start fallback)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Load the latest published calibration profile for the user holder.
|
||||
* Cold-start gate: returns null when no row exists OR active_bias_tags is empty.
|
||||
* The orchestrator stderr-warns when null so users know the judge ran unbiased.
|
||||
*/
|
||||
export async function loadCalibrationContext(
|
||||
engine: BrainEngine,
|
||||
opts: { holder: string; sourceId?: string }
|
||||
): Promise<{ active_bias_tags: string[]; pattern_statements: string[] } | null> {
|
||||
const sourceClause = opts.sourceId
|
||||
? `AND source_id = '${opts.sourceId.replace(/'/g, "''")}'`
|
||||
: '';
|
||||
let rows: Array<{ active_bias_tags: string[]; pattern_statements: string[] }>;
|
||||
try {
|
||||
rows = await engine.executeRaw(
|
||||
`SELECT active_bias_tags, pattern_statements
|
||||
FROM calibration_profiles
|
||||
WHERE holder = $1 ${sourceClause}
|
||||
ORDER BY generated_at DESC
|
||||
LIMIT 1`,
|
||||
[opts.holder]
|
||||
) as Array<{ active_bias_tags: string[]; pattern_statements: string[] }>;
|
||||
} catch {
|
||||
// Pre-v0.36.1 brains: calibration_profiles table doesn't exist.
|
||||
return null;
|
||||
}
|
||||
if (rows.length === 0) return null;
|
||||
const row = rows[0];
|
||||
const tags = Array.isArray(row.active_bias_tags) ? row.active_bias_tags : [];
|
||||
const patterns = Array.isArray(row.pattern_statements) ? row.pattern_statements : [];
|
||||
if (tags.length === 0) return null; // codex #8 cold-start gate
|
||||
return { active_bias_tags: tags, pattern_statements: patterns };
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Idea generation prompts + response parsing
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Build a single (close × far) cross-generation prompt. */
|
||||
function buildCrossPrompt(opts: {
|
||||
profile: BrainstormProfile;
|
||||
question: string;
|
||||
close: { slug: string; title: string | null; content: string };
|
||||
far: FarPage;
|
||||
}): { system: string; user: string } {
|
||||
const system = `You are an idea generator using bisociation (Arthur Koestler, 1964). You surface non-trivial ideas by colliding two pages from a user's own knowledge brain.
|
||||
|
||||
Voice: ${opts.profile.generator_voice}
|
||||
|
||||
Style rules:
|
||||
- Short, assertive sentences. Zero hedging.
|
||||
- Each idea starts from a principle, not anecdote.
|
||||
- Cite BOTH the close and far slug verbatim — these are the user's own notes.
|
||||
- Never fabricate facts, figures, or quotes. Stay grounded in the cited pages.${opts.profile.generator_constraint ? `\n- ${opts.profile.generator_constraint}` : ''}`;
|
||||
|
||||
const user = `QUESTION:
|
||||
${opts.question}
|
||||
|
||||
CLOSE PAGE (related to the question — context anchor):
|
||||
[${opts.close.slug}] ${opts.close.title ?? '(untitled)'}
|
||||
${opts.close.content.slice(0, 1500)}
|
||||
|
||||
FAR PAGE (from a distant region of the user's brain — the collision partner):
|
||||
[${opts.far.slug}] ${opts.far.title ?? '(untitled)'}
|
||||
${opts.far.content}
|
||||
|
||||
Generate exactly ${opts.profile.ideas_per_cross} ideas from cross-pollinating these pages.
|
||||
|
||||
Output format (one idea per ## block, no JSON):
|
||||
## Idea 1
|
||||
[2-4 sentences. Reference [${opts.close.slug}] and [${opts.far.slug}].]
|
||||
|
||||
## Idea 2
|
||||
[2-4 sentences. Reference [${opts.close.slug}] and [${opts.far.slug}].]
|
||||
|
||||
(Continue for all ${opts.profile.ideas_per_cross} ideas.)`;
|
||||
|
||||
return { system, user };
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse the generator's idea output. Tolerant: matches `## Idea N`, `### Idea N`,
|
||||
* or `## N.` headings. Falls back to splitting on double newlines if no headers
|
||||
* match (very loose, last resort).
|
||||
*/
|
||||
export function parseIdeaResponse(text: string): string[] {
|
||||
if (!text || text.trim().length === 0) return [];
|
||||
// Primary: split on `## Idea N` or `### Idea N` (case-insensitive).
|
||||
const headerRe = /^#{2,4}\s*(?:idea\s+)?\d+[.:\s\-]*/gim;
|
||||
const parts = text.split(headerRe).map((p) => p.trim()).filter((p) => p.length > 0);
|
||||
if (parts.length >= 2) return parts; // first chunk might be a preamble or first idea; keep all non-empty.
|
||||
// Fallback: split on numbered list `1. ... 2. ... 3. ...`.
|
||||
const numberedRe = /^\s*\d+\.\s+/gm;
|
||||
const numbered = text.split(numberedRe).map((p) => p.trim()).filter((p) => p.length > 0);
|
||||
if (numbered.length >= 2) return numbered;
|
||||
// Last resort: split on blank lines.
|
||||
const paragraphs = text.split(/\n{2,}/).map((p) => p.trim()).filter((p) => p.length > 30);
|
||||
return paragraphs;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Bounded parallelism via a simple counting semaphore
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
async function mapWithConcurrency<I, O>(
|
||||
items: I[],
|
||||
limit: number,
|
||||
worker: (item: I, idx: number) => Promise<O>
|
||||
): Promise<O[]> {
|
||||
const results: O[] = new Array(items.length);
|
||||
let cursor = 0;
|
||||
const workers: Promise<void>[] = [];
|
||||
for (let w = 0; w < Math.min(limit, items.length); w++) {
|
||||
workers.push((async () => {
|
||||
while (true) {
|
||||
const i = cursor++;
|
||||
if (i >= items.length) break;
|
||||
results[i] = await worker(items[i], i);
|
||||
}
|
||||
})());
|
||||
}
|
||||
await Promise.all(workers);
|
||||
return results;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Public orchestrator entry point
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const DEFAULT_PARALLELISM = 4;
|
||||
|
||||
/**
|
||||
* Run the brainstorm or LSD pipeline. The CLI command (`gbrain brainstorm` /
|
||||
* `gbrain lsd`) calls this with the question + profile; renders the result
|
||||
* via formatBrainstormMarkdown; optionally saves via put_page.
|
||||
*/
|
||||
export async function runBrainstorm(
|
||||
engine: BrainEngine,
|
||||
config: { embedding_model?: string; emotional_weight?: { user_holder?: string } },
|
||||
opts: BrainstormOptions
|
||||
): Promise<BrainstormResult> {
|
||||
const profile = opts.profile ?? BRAINSTORM_PROFILE;
|
||||
const stderr = opts.stderrWrite ?? ((s: string) => { process.stderr.write(s); });
|
||||
const chat = opts.chatFn ?? defaultChat;
|
||||
const embedFn = opts.embedQueryFn ?? embedQuery;
|
||||
|
||||
// ---- Phase 0: cost preview + TTY grace ----
|
||||
const modelStr = opts.modelOverride ?? 'anthropic:claude-sonnet-4-6';
|
||||
const { aborted, estimate } = await previewCostAndWait({
|
||||
profile,
|
||||
model: modelStr,
|
||||
skip: opts.skipCostPreview === true,
|
||||
stderrWrite: stderr,
|
||||
});
|
||||
if (aborted) {
|
||||
throw new Error('brainstorm: aborted before run (Ctrl-C during cost preview window)');
|
||||
}
|
||||
|
||||
// ---- Phase 1: question embedding + close-set retrieval ----
|
||||
let questionEmbedding: Float32Array | null = null;
|
||||
try {
|
||||
questionEmbedding = await embedFn(opts.question);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
stderr(`[${profile.label}] WARN: question embedding failed (${msg}); distance scores will be neutral.\n`);
|
||||
}
|
||||
|
||||
// hybridSearch for close-set. Limit to profile.k_close. Source-scoped.
|
||||
let closeResults = await hybridSearch(engine, opts.question, {
|
||||
limit: profile.k_close,
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
// Defensive: dedup by slug.
|
||||
const seen = new Set<string>();
|
||||
closeResults = closeResults.filter((r) => {
|
||||
if (seen.has(r.slug)) return false;
|
||||
seen.add(r.slug);
|
||||
return true;
|
||||
});
|
||||
if (closeResults.length === 0) {
|
||||
// K=0 LSD case is intentional; for brainstorm an empty close-set means
|
||||
// "no related pages" — proceed, all crosses use the question as the
|
||||
// sole anchor (no close-page context).
|
||||
stderr(`[${profile.label}] WARN: no close-set pages matched the question; proceeding with empty anchor.\n`);
|
||||
}
|
||||
const closeSet: CloseRef[] = closeResults.map((r) => ({
|
||||
slug: r.slug,
|
||||
prefix: extractPrefix(r.slug),
|
||||
}));
|
||||
|
||||
// ---- Phase 2: domain-bank fetch ----
|
||||
const farResult = await fetchFar(engine, {
|
||||
m: profile.m_far,
|
||||
closeSet,
|
||||
questionEmbedding,
|
||||
staleBias: profile.stale_bias,
|
||||
sourceId: opts.sourceId,
|
||||
sourceIds: opts.sourceIds,
|
||||
});
|
||||
if (farResult.short_of_target) {
|
||||
// D11 data-driven warning text.
|
||||
stderr(
|
||||
`[${profile.label}] WARN: Only ${farResult.available_prefixes} distinct prefixes available, expected ${profile.m_far} — ideas will be drawn from a narrower domain bank than usual.\n`
|
||||
);
|
||||
}
|
||||
if (farResult.pages.length === 0) {
|
||||
throw new Error(
|
||||
`${profile.label}: brain has no usable far pages. Try \`gbrain import <dir>\` to seed cross-domain content, or check the prefix cache via \`gbrain doctor\`.`
|
||||
);
|
||||
}
|
||||
|
||||
// ---- Phase 3: calibration context (cold-start fallback) ----
|
||||
const holder = opts.holderOverride ?? config.emotional_weight?.user_holder ?? 'garry';
|
||||
const calibContext = await loadCalibrationContext(engine, {
|
||||
holder,
|
||||
sourceId: opts.sourceId,
|
||||
});
|
||||
const activeBiasTags = calibContext?.active_bias_tags ?? null;
|
||||
if (!activeBiasTags) {
|
||||
stderr(`[${profile.label}] calibration cold-start, judging without bias context.\n`);
|
||||
}
|
||||
|
||||
// Map close slugs → titles for the cross-prompt (we have slug from
|
||||
// hybridSearch; we don't have the page bodies for close-set, so we read
|
||||
// them now from the engine — small cost, ~K pages, no domain-bank lookup).
|
||||
const closeFull = await Promise.all(
|
||||
closeResults.map(async (r) => {
|
||||
const page = await engine.getPage(r.slug, opts.sourceId ? { sourceId: opts.sourceId } : undefined);
|
||||
return {
|
||||
slug: r.slug,
|
||||
title: page?.title ?? null,
|
||||
content: page?.compiled_truth ?? '',
|
||||
};
|
||||
})
|
||||
);
|
||||
|
||||
// ---- Phase 3.5: generate ideas across (close × far) crosses ----
|
||||
// When closeSet is empty, fabricate a single "anchor-less" close entry so
|
||||
// the cross still happens (LSD K=0 path).
|
||||
const closesForCross = closeFull.length > 0
|
||||
? closeFull
|
||||
: [{ slug: '(no anchor)', title: 'question only', content: opts.question }];
|
||||
type Cross = {
|
||||
close: { slug: string; title: string | null; content: string };
|
||||
far: FarPage;
|
||||
};
|
||||
const crosses: Cross[] = [];
|
||||
for (const c of closesForCross) {
|
||||
for (const f of farResult.pages) {
|
||||
crosses.push({ close: c, far: f });
|
||||
}
|
||||
}
|
||||
|
||||
let totalUsage = { input_tokens: 0, output_tokens: 0 };
|
||||
let crossModel = modelStr;
|
||||
|
||||
// Parallelize chat calls bounded at DEFAULT_PARALLELISM.
|
||||
const rawIdeasByCross = await mapWithConcurrency(crosses, DEFAULT_PARALLELISM, async (cross) => {
|
||||
const { system, user } = buildCrossPrompt({
|
||||
profile,
|
||||
question: opts.question,
|
||||
close: cross.close,
|
||||
far: cross.far,
|
||||
});
|
||||
const chatOpts: ChatOpts = {
|
||||
model: opts.modelOverride,
|
||||
system,
|
||||
messages: [{ role: 'user', content: user }],
|
||||
maxTokens: 1500,
|
||||
abortSignal: opts.abortSignal,
|
||||
};
|
||||
try {
|
||||
const result = await chat(chatOpts);
|
||||
totalUsage.input_tokens += result.usage.input_tokens;
|
||||
totalUsage.output_tokens += result.usage.output_tokens;
|
||||
crossModel = result.model;
|
||||
const parsed = parseIdeaResponse(result.text);
|
||||
return parsed.slice(0, profile.ideas_per_cross).map((text) => ({
|
||||
text,
|
||||
close_slug: cross.close.slug,
|
||||
far_slug: cross.far.slug,
|
||||
distance_score: cross.far.distance_score,
|
||||
}));
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
stderr(`[${profile.label}] WARN: cross [${cross.close.slug}] × [${cross.far.slug}] failed: ${msg}\n`);
|
||||
return [];
|
||||
}
|
||||
});
|
||||
|
||||
// Flatten + assign stable ids.
|
||||
const allRawIdeas: Array<{ id: string; text: string; close_slug: string; far_slug: string; distance_score: number }> = [];
|
||||
for (const ideas of rawIdeasByCross) {
|
||||
for (const idea of ideas) {
|
||||
const id = String(allRawIdeas.length + 1).padStart(2, '0');
|
||||
allRawIdeas.push({ id, ...idea });
|
||||
}
|
||||
}
|
||||
|
||||
if (allRawIdeas.length === 0) {
|
||||
throw new Error(
|
||||
`${profile.label}: no ideas generated across ${crosses.length} crosses. Check API keys via \`gbrain models doctor\`.`
|
||||
);
|
||||
}
|
||||
|
||||
// ---- Phase 4: judge ----
|
||||
let judgeFailed = false;
|
||||
let judgedById: Map<string, JudgeIdeaResult> = new Map();
|
||||
let judgeUsage = { input_tokens: 0, output_tokens: 0 };
|
||||
try {
|
||||
const judgeInput: JudgeIdea[] = allRawIdeas.map((i) => ({
|
||||
id: i.id,
|
||||
text: i.text,
|
||||
close_slug: i.close_slug,
|
||||
far_slug: i.far_slug,
|
||||
}));
|
||||
const judgeResult = await runJudge(profile.judge_config, judgeInput, {
|
||||
modelOverride: opts.modelOverride,
|
||||
chatFn: opts.chatFn,
|
||||
activeBiasTags: activeBiasTags ?? undefined,
|
||||
abortSignal: opts.abortSignal,
|
||||
});
|
||||
for (const idea of judgeResult.ideas) {
|
||||
judgedById.set(idea.id, idea);
|
||||
}
|
||||
judgeUsage = {
|
||||
input_tokens: judgeResult.usage.input_tokens,
|
||||
output_tokens: judgeResult.usage.output_tokens,
|
||||
};
|
||||
} catch (err) {
|
||||
judgeFailed = true;
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
stderr(`[${profile.label}] WARN: judge phase failed (${msg}); saving ideas unscored. Re-run with --retry-judge to score.\n`);
|
||||
}
|
||||
|
||||
// ---- Phase 5: assemble BrainstormResult ----
|
||||
const ideas: BrainstormIdea[] = allRawIdeas.map((raw) => {
|
||||
const j = judgedById.get(raw.id);
|
||||
return {
|
||||
id: raw.id,
|
||||
text: raw.text,
|
||||
close_slug: raw.close_slug,
|
||||
far_slug: raw.far_slug,
|
||||
distance_score: raw.distance_score,
|
||||
judge: j,
|
||||
passes: j?.passes ?? false,
|
||||
judge_failed: judgeFailed,
|
||||
};
|
||||
});
|
||||
|
||||
// Cost actuals (codex r2 #10).
|
||||
const totalIn = totalUsage.input_tokens + judgeUsage.input_tokens;
|
||||
const totalOut = totalUsage.output_tokens + judgeUsage.output_tokens;
|
||||
const pricing = ANTHROPIC_PRICING[crossModel] ?? { input: 3, output: 15 };
|
||||
const actual = (totalIn / 1_000_000) * pricing.input + (totalOut / 1_000_000) * pricing.output;
|
||||
stderr(`[${profile.label}] actual cost: ${fmtUsd(actual)} (estimated ${fmtUsd(estimate)}) — in=${totalIn} out=${totalOut} tokens\n`);
|
||||
|
||||
return {
|
||||
profile_label: profile.label,
|
||||
question: opts.question,
|
||||
embedding_model: config.embedding_model ?? null,
|
||||
ideas,
|
||||
close_set: closeFull.map((c) => ({ slug: c.slug, title: c.title })),
|
||||
far_set: farResult.pages.map((f) => ({
|
||||
slug: f.slug,
|
||||
title: f.title,
|
||||
distance_score: f.distance_score,
|
||||
source: f.source,
|
||||
})),
|
||||
active_bias_tags: activeBiasTags,
|
||||
short_of_target: farResult.short_of_target,
|
||||
judge_failed: judgeFailed,
|
||||
cost: {
|
||||
estimated_usd: estimate,
|
||||
actual_usd: actual,
|
||||
input_tokens: totalIn,
|
||||
output_tokens: totalOut,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Extract the 2-segment top-level prefix from a slug. Returns null for slugs
|
||||
* that don't match the `^[^/]+/[^/]+` pattern (single-segment, empty).
|
||||
* Mirrors the SQL `substring(slug from '^[^/]+/[^/]+')` so the orchestrator
|
||||
* and the engine agree on what counts as a "prefix."
|
||||
*/
|
||||
export function extractPrefix(slug: string): string | null {
|
||||
const m = slug.match(/^([^/]+\/[^/]+)/);
|
||||
return m ? m[1] : null;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Output formatter (D6 citation badges, D1 fold)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Render BrainstormResult as user-facing markdown. Citation badges per D6:
|
||||
* each idea cites close + far slugs AND its normalized distance score.
|
||||
*
|
||||
* @param onlyPassed When true (default), filter to `passes === true` ideas.
|
||||
* Pass false to see the full set including filtered-out + judge-failed rows.
|
||||
*/
|
||||
export function formatBrainstormMarkdown(
|
||||
result: BrainstormResult,
|
||||
opts: { onlyPassed?: boolean; includeMeta?: boolean } = {}
|
||||
): string {
|
||||
const onlyPassed = opts.onlyPassed ?? true;
|
||||
const includeMeta = opts.includeMeta ?? true;
|
||||
const ideasToShow = onlyPassed
|
||||
? result.ideas.filter((i) => i.passes)
|
||||
: result.ideas;
|
||||
|
||||
const lines: string[] = [];
|
||||
|
||||
if (includeMeta) {
|
||||
lines.push(`# ${result.profile_label === 'lsd' ? 'LSD' : 'Brainstorm'}: ${result.question}`);
|
||||
lines.push('');
|
||||
if (result.judge_failed) {
|
||||
lines.push('> **Judge phase failed mid-run** — ideas below are unscored. Re-run with `--retry-judge` to score.');
|
||||
lines.push('');
|
||||
}
|
||||
if (result.short_of_target) {
|
||||
lines.push(`> _Note: domain bank was narrower than usual — see stderr warning._`);
|
||||
lines.push('');
|
||||
}
|
||||
if (result.active_bias_tags === null) {
|
||||
lines.push(`> _Note: calibration cold-start — ideas were judged without anti-bias context._`);
|
||||
lines.push('');
|
||||
}
|
||||
lines.push(`**Close set** (${result.close_set.length}):`);
|
||||
for (const c of result.close_set) {
|
||||
lines.push(`- \`${c.slug}\`${c.title ? ` — ${c.title}` : ''}`);
|
||||
}
|
||||
lines.push('');
|
||||
lines.push(`**Far set** (${result.far_set.length}, ${result.far_set.filter((f) => f.source === 'corpus-sample').length} via corpus-sample fallback):`);
|
||||
for (const f of result.far_set) {
|
||||
lines.push(`- \`${f.slug}\` — distance ${f.distance_score.toFixed(2)}${f.title ? ` — ${f.title}` : ''}`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
lines.push(`## Ideas (${ideasToShow.length}${onlyPassed && result.ideas.length !== ideasToShow.length ? ` of ${result.ideas.length}` : ''})`);
|
||||
lines.push('');
|
||||
for (const idea of ideasToShow) {
|
||||
const scoreSuffix = idea.judge
|
||||
? ` _(score ${idea.judge.weighted_score.toFixed(2)})_`
|
||||
: (idea.judge_failed ? ` _(unscored — judge failed)_` : '');
|
||||
lines.push(`### Idea ${idea.id}${scoreSuffix}`);
|
||||
lines.push('');
|
||||
lines.push(idea.text);
|
||||
lines.push('');
|
||||
lines.push(`_Citation: \`${idea.close_slug}\` × \`${idea.far_slug}\` — distance ${idea.distance_score.toFixed(2)}_`);
|
||||
if (idea.judge?.note) {
|
||||
lines.push(`_Judge note: ${idea.judge.note}_`);
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
||||
/** Frontmatter for a saved brainstorm/lsd page. */
|
||||
export function buildBrainstormFrontmatter(result: BrainstormResult, opts: { slug: string }): string {
|
||||
const date = new Date().toISOString().slice(0, 10);
|
||||
const judgeFailed = result.judge_failed ? '\njudge_failed: true' : '';
|
||||
const unscored = result.judge_failed ? '\nunscored: true' : '';
|
||||
return `---
|
||||
title: "${result.profile_label === 'lsd' ? 'LSD' : 'Brainstorm'}: ${result.question.replace(/"/g, '\\"').slice(0, 100)}"
|
||||
mode: ${result.profile_label}
|
||||
generated_at: ${new Date().toISOString()}
|
||||
date: ${date}
|
||||
question: "${result.question.replace(/"/g, '\\"').slice(0, 200)}"
|
||||
close_slugs: [${result.close_set.map((c) => `"${c.slug}"`).join(', ')}]
|
||||
far_slugs: [${result.far_set.map((f) => `"${f.slug}"`).join(', ')}]
|
||||
short_of_target: ${result.short_of_target}
|
||||
calibration_cold_start: ${result.active_bias_tags === null}${judgeFailed}${unscored}
|
||||
cost_usd: ${result.cost.actual_usd.toFixed(4)}
|
||||
---
|
||||
|
||||
`;
|
||||
}
|
||||
@@ -72,7 +72,49 @@ const DREAM_MARKER_REGEX_SRC =
|
||||
'^\\uFEFF?-{3}\\r?\\n[\\s\\S]{0,2000}?dream_generated\\s*:\\s*true\\b';
|
||||
export const DREAM_OUTPUT_MARKER_RE = new RegExp(DREAM_MARKER_REGEX_SRC, 'i');
|
||||
|
||||
/**
|
||||
* v0.37.0 (D9 / D4): brainstorm + LSD frontmatter markers. `mode: lsd`
|
||||
* pages are noise-by-design and must NEVER be re-ingested by the synthesize
|
||||
* phase (they're inverted-judge experiments, not user-validated knowledge).
|
||||
* `mode: brainstorm` pages stamp the saved-page provenance; they're not
|
||||
* auto-skipped at this layer because the corpus walker doesn't currently
|
||||
* read wiki/ideas/ — full loop closure (synthesize mines `mode: brainstorm`
|
||||
* pages for patterns) is filed as a v0.37.1 follow-up.
|
||||
*/
|
||||
const LSD_MODE_MARKER_REGEX_SRC =
|
||||
'^\\uFEFF?-{3}\\r?\\n[\\s\\S]{0,2000}?mode\\s*:\\s*(?:"|\\\'|)lsd(?:"|\\\'|)\\s*(?:\\r?\\n|$)';
|
||||
export const LSD_OUTPUT_MARKER_RE = new RegExp(LSD_MODE_MARKER_REGEX_SRC, 'i');
|
||||
|
||||
const BRAINSTORM_MODE_MARKER_REGEX_SRC =
|
||||
'^\\uFEFF?-{3}\\r?\\n[\\s\\S]{0,2000}?mode\\s*:\\s*(?:"|\\\'|)brainstorm(?:"|\\\'|)\\s*(?:\\r?\\n|$)';
|
||||
export const BRAINSTORM_OUTPUT_MARKER_RE = new RegExp(BRAINSTORM_MODE_MARKER_REGEX_SRC, 'i');
|
||||
|
||||
/** True iff this content carries the LSD frontmatter marker (D4 noise-by-design skip). */
|
||||
export function isLsdOutput(content: string): boolean {
|
||||
return LSD_OUTPUT_MARKER_RE.test(content);
|
||||
}
|
||||
|
||||
/** True iff this content carries the brainstorm frontmatter marker (saved by `gbrain brainstorm --save`). */
|
||||
export function isBrainstormOutput(content: string): boolean {
|
||||
return BRAINSTORM_OUTPUT_MARKER_RE.test(content);
|
||||
}
|
||||
|
||||
/**
|
||||
* Self-consumption guard: identity-marker check against the synthesize phase's
|
||||
* dream output, EXTENDED in v0.37.0 to also skip `mode: lsd` pages per D4.
|
||||
* The synthesize corpus walker now sees three categories:
|
||||
* - dream output (its own writes): always skipped
|
||||
* - LSD output: skipped (noise-by-design)
|
||||
* - everything else (transcripts, manual notes, brainstorm output): processed
|
||||
*
|
||||
* `bypass` is the explicit `--unsafe-bypass-dream-guard` escape hatch; it bypasses
|
||||
* the dream-output check but NOT the LSD skip — there's no operator scenario
|
||||
* where re-ingesting LSD output is desired (LSD is ephemeral by definition).
|
||||
*/
|
||||
export function isDreamOutput(content: string, bypass = false): boolean {
|
||||
// LSD output ALWAYS skipped — bypass flag is for self-consumption only,
|
||||
// not for re-ingesting LSD experiments into the pattern extractor.
|
||||
if (isLsdOutput(content)) return true;
|
||||
if (bypass) return false;
|
||||
return DREAM_OUTPUT_MARKER_RE.test(content);
|
||||
}
|
||||
|
||||
@@ -14,6 +14,7 @@ import type {
|
||||
EvalCaptureFailure, EvalCaptureFailureReason,
|
||||
SalienceOpts, SalienceResult, AnomaliesOpts, AnomalyResult,
|
||||
EmotionalWeightInputRow, EmotionalWeightWriteRow,
|
||||
DomainBankSampleOpts, CorpusSampleOpts, DomainBankRow,
|
||||
} from './types.ts';
|
||||
|
||||
/**
|
||||
@@ -575,6 +576,36 @@ export interface BrainEngine {
|
||||
*/
|
||||
listAllPageRefs(): Promise<Array<{ slug: string; source_id: string }>>;
|
||||
|
||||
/**
|
||||
* v0.37.0 — prefix-stratified page sampling for `gbrain brainstorm` / `gbrain lsd`
|
||||
* domain-bank module. Takes a caller-supplied prefix list (cached at the domain-bank
|
||||
* layer per D3), returns one page per prefix tiebroken by `connection_count`
|
||||
* (LEFT JOIN to page_links, count of inbound links).
|
||||
*
|
||||
* Stale-bias (D5 / LSD): when `opts.staleBias === true`, ROW_NUMBER() ORDER BY
|
||||
* prefers pages with `last_retrieved_at IS NULL` (never retrieved) > pages older
|
||||
* than `staleThresholdDays` (default 90) > recently-retrieved.
|
||||
*
|
||||
* Source scoping (D5, codex r2 #2 fix): `sourceId` (scalar) and `sourceIds`
|
||||
* (array, wins over scalar) per the [source-id-canonical-thread] pattern.
|
||||
* Both threaded from day 1 even though v0.37.0 callers are CLI-local — D7
|
||||
* MCP exposure ships zero-refactor.
|
||||
*
|
||||
* Soft-deleted pages (deleted_at IS NOT NULL) excluded automatically.
|
||||
*/
|
||||
listPrefixSampledPages(opts: DomainBankSampleOpts): Promise<DomainBankRow[]>;
|
||||
|
||||
/**
|
||||
* v0.37.0 — corpus-sampling fallback for `gbrain brainstorm` when prefix-stratified
|
||||
* can't fill M (small brain, single-prefix corpus). Random sample of N pages with
|
||||
* the same exclusion + source-scope semantics as `listPrefixSampledPages`.
|
||||
* Deterministic with `opts.seed` set; falls back to RANDOM() otherwise.
|
||||
*
|
||||
* Returns the same `DomainBankRow` shape so the orchestrator can union both
|
||||
* sources of pages and dedup by slug+source_id.
|
||||
*/
|
||||
listCorpusSample(opts: CorpusSampleOpts): Promise<DomainBankRow[]>;
|
||||
|
||||
// Search
|
||||
searchKeyword(query: string, opts?: SearchOpts): Promise<SearchResult[]>;
|
||||
searchVector(embedding: Float32Array, opts?: SearchOpts): Promise<SearchResult[]>;
|
||||
|
||||
@@ -0,0 +1,105 @@
|
||||
/**
|
||||
* v0.37.0 — `pages.last_retrieved_at` write-back for the LSD stale-page signal.
|
||||
*
|
||||
* Architecture (codex round 2 #3 + D11 + D2 + D13):
|
||||
*
|
||||
* - Op-layer, NOT engine-layer. This module is called from the `search` /
|
||||
* `query` / `get_page` op handlers in `operations.ts` AFTER the engine
|
||||
* returns. Internal callers (sync, migrations, helper flows) bypass the
|
||||
* op layer entirely, so this never fires from `import-file.ts`, the
|
||||
* dream cycle, doctor probes, etc. Pure signal: "a user-facing surface
|
||||
* just surfaced this page."
|
||||
*
|
||||
* - 5-min throttle (D2). The UPDATE includes a `WHERE last_retrieved_at IS
|
||||
* NULL OR last_retrieved_at < NOW() - INTERVAL '5 minutes'` clause so
|
||||
* hot pages surfaced by many concurrent searches don't pile up MVCC
|
||||
* row versions. ~90% of writes skipped in steady state on a heavily-
|
||||
* searched brain. Mirrors `embedded_at` reset gating in `upsertChunks`.
|
||||
*
|
||||
* - Default-on with `search.track_retrieval` config escape hatch (D13).
|
||||
* Operators worried about per-search write amplification can opt out:
|
||||
* `gbrain config set search.track_retrieval false`. `gbrain doctor`'s
|
||||
* brainstorm_health check surfaces the setting.
|
||||
*
|
||||
* - Best-effort. Any error (column missing, network blip, statement
|
||||
* timeout) is swallowed with a stderr warn. The op result is unaffected.
|
||||
* Two failure modes deserve graceful degradation: a pre-v77 brain that
|
||||
* somehow reaches this code (column missing → SQLSTATE 42703) and a
|
||||
* transient connection error.
|
||||
*
|
||||
* - Fire-and-forget. Caller does NOT await; the UPDATE runs concurrently
|
||||
* with response serialization. If the caller awaited, a slow UPDATE
|
||||
* would add latency to the visible response. Best-effort + concurrent =
|
||||
* the user never sees the write-back cost in the response time.
|
||||
*/
|
||||
|
||||
import type { BrainEngine } from './engine.ts';
|
||||
import { isUndefinedColumnError } from './utils.ts';
|
||||
|
||||
let _trackRetrievalCache: { ts: number; enabled: boolean } | null = null;
|
||||
const TRACK_RETRIEVAL_CACHE_TTL_MS = 30_000;
|
||||
|
||||
/**
|
||||
* Resolve `search.track_retrieval` config with a 30s in-process cache so
|
||||
* hot-path callers don't pay a SELECT per search. Default-on: missing
|
||||
* config OR unparseable value → true (D13 default).
|
||||
*/
|
||||
async function isTrackingEnabled(engine: BrainEngine): Promise<boolean> {
|
||||
const now = Date.now();
|
||||
if (_trackRetrievalCache && now - _trackRetrievalCache.ts < TRACK_RETRIEVAL_CACHE_TTL_MS) {
|
||||
return _trackRetrievalCache.enabled;
|
||||
}
|
||||
let enabled = true;
|
||||
try {
|
||||
const raw = await engine.getConfig('search.track_retrieval');
|
||||
if (raw === 'false' || raw === '0' || raw === 'off' || raw === 'no') {
|
||||
enabled = false;
|
||||
}
|
||||
} catch {
|
||||
// getConfig miss / connection blip → default to enabled (D13 default).
|
||||
}
|
||||
_trackRetrievalCache = { ts: now, enabled };
|
||||
return enabled;
|
||||
}
|
||||
|
||||
/** Test seam — drops the cache so subsequent calls re-read config. */
|
||||
export function _resetTrackRetrievalCacheForTests(): void {
|
||||
_trackRetrievalCache = null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Bump `last_retrieved_at` on the given page_ids. Fire-and-forget — caller
|
||||
* MUST NOT await this for the op response. Empty ids list is a no-op.
|
||||
*
|
||||
* @param engine The BrainEngine handling the op.
|
||||
* @param pageIds The page ids surfaced by the op (search hits, query results,
|
||||
* or the single id returned by get_page).
|
||||
*/
|
||||
export function bumpLastRetrievedAt(engine: BrainEngine, pageIds: number[]): void {
|
||||
if (pageIds.length === 0) return;
|
||||
// Fire-and-forget on purpose. We deliberately do NOT return the promise.
|
||||
void (async () => {
|
||||
try {
|
||||
const enabled = await isTrackingEnabled(engine);
|
||||
if (!enabled) return;
|
||||
// 5-minute throttle (D2) + best-effort. The UPDATE is idempotent:
|
||||
// setting last_retrieved_at = NOW() multiple times in a row is the
|
||||
// same as setting it once (TIMESTAMPTZ comparison is monotonic).
|
||||
await engine.executeRaw(
|
||||
`UPDATE pages
|
||||
SET last_retrieved_at = NOW()
|
||||
WHERE id = ANY($1::int[])
|
||||
AND (last_retrieved_at IS NULL
|
||||
OR last_retrieved_at < NOW() - INTERVAL '5 minutes')`,
|
||||
[pageIds]
|
||||
);
|
||||
} catch (err) {
|
||||
// Pre-v77 brain (column missing) falls through silently — the search
|
||||
// op already returned, the LSD signal just stays NULL until upgrade.
|
||||
if (isUndefinedColumnError(err, 'last_retrieved_at')) return;
|
||||
// Other errors: stderr-warn but don't break the op response.
|
||||
const msg = err instanceof Error ? err.message : String(err);
|
||||
console.warn(`[last-retrieved] write-back failed (best-effort): ${msg}`);
|
||||
}
|
||||
})();
|
||||
}
|
||||
@@ -3684,6 +3684,35 @@ export const MIGRATIONS: Migration[] = [
|
||||
pglite: '',
|
||||
},
|
||||
},
|
||||
{
|
||||
version: 79,
|
||||
name: 'pages_last_retrieved_at',
|
||||
// v0.37.1.0 brainstorm/lsd wave (D15 + D11 + D12):
|
||||
// Originally planned as v77 but v77 + v78 were claimed by the v0.37.0.0
|
||||
// skillpack-registry + cross-modal waves landing on master first.
|
||||
//
|
||||
// Adds `pages.last_retrieved_at TIMESTAMPTZ NULL` — the real stale-page
|
||||
// signal for `gbrain lsd`'s "your brain at 3am noticing what it forgot"
|
||||
// mode. Bumped by op-layer write-back inside the `search` / `query` /
|
||||
// `get_page` op handlers AFTER results return (NOT inside the engine
|
||||
// methods — internal callers like sync / migrations / tests must not
|
||||
// pollute the signal per codex round 2 #3).
|
||||
//
|
||||
// Full index, no partial WHERE per D12 + codex round 2 #6: LSD's primary
|
||||
// query is `WHERE last_retrieved_at IS NULL OR last_retrieved_at < NOW()
|
||||
// - INTERVAL '90 days'`. Postgres B-tree indexes handle NULL (sorted to
|
||||
// one end), so one index supports both branches. A partial `WHERE NOT
|
||||
// NULL` would miss LSD's prioritized never-retrieved branch.
|
||||
//
|
||||
// ADD COLUMN with no DEFAULT (NULL) is metadata-only on Postgres 11+
|
||||
// and PGLite 17.5; instant on tables of any size.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE pages ADD COLUMN IF NOT EXISTS last_retrieved_at TIMESTAMPTZ NULL;
|
||||
CREATE INDEX IF NOT EXISTS pages_last_retrieved_at_idx
|
||||
ON pages (last_retrieved_at);
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -19,6 +19,7 @@ import { extractPageLinks, isAutoLinkEnabled, isAutoTimelineEnabled, parseTimeli
|
||||
import { isFactsBackstopEligible } from './facts/eligibility.ts';
|
||||
import { stripTakesFence } from './takes-fence.ts';
|
||||
import { stripFactsFence } from './facts-fence.ts';
|
||||
import { bumpLastRetrievedAt } from './last-retrieved.ts';
|
||||
import { CJK_SLUG_CHARS } from './cjk.ts';
|
||||
import * as db from './db.ts';
|
||||
import { VERSION } from '../version.ts';
|
||||
@@ -481,6 +482,13 @@ const get_page: Operation = {
|
||||
throw new OperationError('page_not_found', `Page not found: ${slug}`, includeDeleted ? 'Check the slug or use fuzzy: true' : 'Page may be soft-deleted; pass include_deleted: true to verify');
|
||||
}
|
||||
|
||||
// v0.37.0 (D11): op-layer write-back for the `last_retrieved_at` stale
|
||||
// signal. Fire-and-forget — caller does NOT await. Internal callers
|
||||
// (sync, migrations, dream cycle) bypass this op handler so the signal
|
||||
// stays clean. Throttled to ~1 write / 5 min per page via the SQL clause
|
||||
// inside bumpLastRetrievedAt (D2).
|
||||
bumpLastRetrievedAt(ctx.engine, [page.id]);
|
||||
|
||||
const tags = await ctx.engine.getTags(page.slug, sourceOpts);
|
||||
// Privacy boundary for the per-token allow-list (v0.28.6 for takes,
|
||||
// v0.32.2 for facts).
|
||||
@@ -1054,6 +1062,11 @@ const search: Operation = {
|
||||
const results = dedupResults(raw);
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
|
||||
// v0.37.0 (D11): op-layer last_retrieved_at write-back. Fire-and-forget;
|
||||
// results already returned by engine, this just marks them as user-surfaced
|
||||
// for LSD's stale-page signal. 5-min throttle inside bumpLastRetrievedAt.
|
||||
bumpLastRetrievedAt(ctx.engine, results.map((r) => r.page_id));
|
||||
|
||||
// Op-layer capture (v0.25.0). Fire-and-forget — no await on the
|
||||
// capture call so MCP response latency is unaffected. search has
|
||||
// no expand/detail/vector semantics so meta fields are fixed.
|
||||
@@ -1250,6 +1263,10 @@ const query: Operation = {
|
||||
});
|
||||
const latency_ms = Date.now() - startedAt;
|
||||
|
||||
// v0.37.0 (D11): op-layer last_retrieved_at write-back. Same shape as the
|
||||
// search handler — fire-and-forget, internal callers bypass this path.
|
||||
bumpLastRetrievedAt(ctx.engine, results.map((r) => r.page_id));
|
||||
|
||||
// Op-layer capture (v0.25.0). Fire-and-forget. meta tells gbrain-evals
|
||||
// what hybridSearch *actually* did so replay can distinguish "with API
|
||||
// key" from "keyword-only fallback" and "expansion fired" from
|
||||
|
||||
+165
-2
@@ -33,6 +33,7 @@ import type {
|
||||
EvalCaptureFailure, EvalCaptureFailureReason,
|
||||
SalienceOpts, SalienceResult, AnomaliesOpts, AnomalyResult,
|
||||
EmotionalWeightInputRow, EmotionalWeightWriteRow,
|
||||
DomainBankSampleOpts, CorpusSampleOpts, DomainBankRow,
|
||||
} from './types.ts';
|
||||
import { validateSlug, contentHash, rowToPage, rowToChunk, rowToSearchResult, takeRowToTake } from './utils.ts';
|
||||
import { deriveResolutionTuple, finalizeScorecard } from './takes-resolution.ts';
|
||||
@@ -315,7 +316,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='sources' AND column_name='archived_at') AS sources_archived_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='sources' AND column_name='archive_expires_at') AS sources_archive_expires_at_exists
|
||||
WHERE table_schema='public' AND table_name='sources' AND column_name='archive_expires_at') AS sources_archive_expires_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema='public' AND table_name='pages' AND column_name='last_retrieved_at') AS pages_last_retrieved_at_exists
|
||||
`);
|
||||
const probe = rows[0] as {
|
||||
pages_exists: boolean;
|
||||
@@ -346,6 +349,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
sources_archived_exists: boolean;
|
||||
sources_archived_at_exists: boolean;
|
||||
sources_archive_expires_at_exists: boolean;
|
||||
pages_last_retrieved_at_exists: boolean;
|
||||
};
|
||||
|
||||
const needsPagesBootstrap = probe.pages_exists && !probe.source_id_exists;
|
||||
@@ -388,6 +392,9 @@ export class PGLiteEngine implements BrainEngine {
|
||||
&& (!probe.sources_archived_exists
|
||||
|| !probe.sources_archived_at_exists
|
||||
|| !probe.sources_archive_expires_at_exists);
|
||||
// v0.37.0 (v79): pages_last_retrieved_at_idx in PGLITE_SCHEMA_SQL
|
||||
// references last_retrieved_at. Pre-v79 brains crash without the column.
|
||||
const needsPagesLastRetrievedAt = probe.pages_exists && !probe.pages_last_retrieved_at_exists;
|
||||
|
||||
// Fresh installs (no tables yet) and modern brains both no-op.
|
||||
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
|
||||
@@ -395,7 +402,7 @@ export class PGLiteEngine implements BrainEngine {
|
||||
&& !needsMcpLogBootstrap && !needsSubagentProviderId
|
||||
&& !needsPagesRecency && !needsIngestLogSourceId
|
||||
&& !needsFilesBootstrap && !needsOauthClientsBootstrap
|
||||
&& !needsSourcesArchive) return;
|
||||
&& !needsSourcesArchive && !needsPagesLastRetrievedAt) return;
|
||||
|
||||
console.log(' Pre-v0.21 brain detected, applying forward-reference bootstrap');
|
||||
|
||||
@@ -574,6 +581,16 @@ export class PGLiteEngine implements BrainEngine {
|
||||
ALTER TABLE sources ADD COLUMN IF NOT EXISTS archive_expires_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsPagesLastRetrievedAt) {
|
||||
// v79 (pages_last_retrieved_at): adds the stale-page signal column +
|
||||
// full B-tree index. PGLITE_SCHEMA_SQL's CREATE INDEX
|
||||
// pages_last_retrieved_at_idx crashes without the column. v79 runs
|
||||
// later via runMigrations and is idempotent.
|
||||
await this.db.exec(`
|
||||
ALTER TABLE pages ADD COLUMN IF NOT EXISTS last_retrieved_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
async withReservedConnection<T>(fn: (conn: ReservedConnection) => Promise<T>): Promise<T> {
|
||||
@@ -854,6 +871,152 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return (rows as { slug: string; source_id: string }[]).map(r => ({ slug: r.slug, source_id: r.source_id }));
|
||||
}
|
||||
|
||||
// v0.37.0 — domain-bank engine methods (D14 + D5 + D10).
|
||||
// See postgres-engine.ts:listPrefixSampledPages for the ranking + source-scope rationale.
|
||||
// PGLite runs the same SQL (Postgres 17.5 under the hood) with positional `$N` binding.
|
||||
async listPrefixSampledPages(opts: DomainBankSampleOpts): Promise<DomainBankRow[]> {
|
||||
if (opts.prefixes.length === 0) return [];
|
||||
const exclude = opts.excludeSlugs ?? [];
|
||||
const staleBias = opts.staleBias === true;
|
||||
const staleThreshold = opts.staleThresholdDays ?? 90;
|
||||
const sourceIds = opts.sourceIds ?? null;
|
||||
const sourceId = opts.sourceId ?? null;
|
||||
const { rows } = await this.db.query(
|
||||
`WITH prefix_pages AS (
|
||||
SELECT
|
||||
p.id AS page_id,
|
||||
p.slug,
|
||||
p.source_id,
|
||||
p.title,
|
||||
p.compiled_truth,
|
||||
p.last_retrieved_at,
|
||||
substring(p.slug from '^[^/]+/[^/]+') AS prefix,
|
||||
COUNT(pl.id) AS connection_count
|
||||
FROM pages p
|
||||
LEFT JOIN page_links pl ON pl.to_page_id = p.id
|
||||
WHERE p.deleted_at IS NULL
|
||||
AND substring(p.slug from '^[^/]+/[^/]+') = ANY($1::text[])
|
||||
AND (cardinality($2::text[]) = 0 OR NOT (p.slug = ANY($2::text[])))
|
||||
AND (
|
||||
($3::text[] IS NOT NULL AND p.source_id = ANY($3::text[]))
|
||||
OR ($3::text[] IS NULL AND $4::text IS NOT NULL AND p.source_id = $4)
|
||||
OR ($3::text[] IS NULL AND $4::text IS NULL)
|
||||
)
|
||||
GROUP BY p.id, p.slug, p.source_id, p.title, p.compiled_truth, p.last_retrieved_at
|
||||
),
|
||||
ranked AS (
|
||||
SELECT
|
||||
pp.*,
|
||||
(CASE WHEN $5::boolean THEN
|
||||
CASE
|
||||
WHEN pp.last_retrieved_at IS NULL THEN 2
|
||||
WHEN pp.last_retrieved_at < NOW() - ($6::int * INTERVAL '1 day') THEN 1
|
||||
ELSE 0
|
||||
END
|
||||
ELSE 0
|
||||
END) AS stale_score,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY pp.prefix
|
||||
ORDER BY
|
||||
(CASE WHEN $5::boolean THEN
|
||||
CASE
|
||||
WHEN pp.last_retrieved_at IS NULL THEN 2
|
||||
WHEN pp.last_retrieved_at < NOW() - ($6::int * INTERVAL '1 day') THEN 1
|
||||
ELSE 0
|
||||
END
|
||||
ELSE 0
|
||||
END) DESC,
|
||||
pp.connection_count DESC,
|
||||
pp.slug ASC
|
||||
) AS rn
|
||||
FROM prefix_pages pp
|
||||
),
|
||||
with_chunk AS (
|
||||
SELECT
|
||||
r.*,
|
||||
(
|
||||
SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = r.page_id AND cc.embedding IS NOT NULL
|
||||
ORDER BY cc.chunk_index ASC
|
||||
LIMIT 1
|
||||
) AS representative_chunk_id
|
||||
FROM ranked r
|
||||
WHERE r.rn = 1
|
||||
)
|
||||
SELECT page_id, slug, source_id, title, compiled_truth, last_retrieved_at,
|
||||
prefix, connection_count, representative_chunk_id
|
||||
FROM with_chunk
|
||||
ORDER BY prefix`,
|
||||
[opts.prefixes, exclude, sourceIds, sourceId, staleBias, staleThreshold]
|
||||
);
|
||||
return (rows as Array<Record<string, unknown>>).map((r): DomainBankRow => ({
|
||||
slug: r.slug as string,
|
||||
source_id: r.source_id as string,
|
||||
prefix: r.prefix as string | null,
|
||||
page_id: Number(r.page_id),
|
||||
title: r.title as string | null,
|
||||
compiled_truth: (r.compiled_truth as string | null) ?? '',
|
||||
connection_count: Number(r.connection_count),
|
||||
last_retrieved_at: r.last_retrieved_at == null ? null : new Date(r.last_retrieved_at as string),
|
||||
representative_chunk_id: r.representative_chunk_id == null ? null : Number(r.representative_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
async listCorpusSample(opts: CorpusSampleOpts): Promise<DomainBankRow[]> {
|
||||
if (opts.n <= 0) return [];
|
||||
const exclude = opts.excludeSlugs ?? [];
|
||||
const sourceIds = opts.sourceIds ?? null;
|
||||
const sourceId = opts.sourceId ?? null;
|
||||
if (typeof opts.seed === 'number') {
|
||||
const clamped = Math.max(-1, Math.min(1, opts.seed));
|
||||
await this.db.query('SELECT setseed($1::float8)', [clamped]);
|
||||
}
|
||||
const { rows } = await this.db.query(
|
||||
`WITH sampled AS (
|
||||
SELECT
|
||||
p.id AS page_id,
|
||||
p.slug,
|
||||
p.source_id,
|
||||
p.title,
|
||||
p.compiled_truth,
|
||||
p.last_retrieved_at,
|
||||
substring(p.slug from '^[^/]+/[^/]+') AS prefix,
|
||||
(SELECT COUNT(*) FROM page_links pl WHERE pl.to_page_id = p.id) AS connection_count
|
||||
FROM pages p
|
||||
WHERE p.deleted_at IS NULL
|
||||
AND (cardinality($1::text[]) = 0 OR NOT (p.slug = ANY($1::text[])))
|
||||
AND (
|
||||
($2::text[] IS NOT NULL AND p.source_id = ANY($2::text[]))
|
||||
OR ($2::text[] IS NULL AND $3::text IS NOT NULL AND p.source_id = $3)
|
||||
OR ($2::text[] IS NULL AND $3::text IS NULL)
|
||||
)
|
||||
ORDER BY RANDOM()
|
||||
LIMIT $4
|
||||
)
|
||||
SELECT
|
||||
s.*,
|
||||
(
|
||||
SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = s.page_id AND cc.embedding IS NOT NULL
|
||||
ORDER BY cc.chunk_index ASC
|
||||
LIMIT 1
|
||||
) AS representative_chunk_id
|
||||
FROM sampled s`,
|
||||
[exclude, sourceIds, sourceId, opts.n]
|
||||
);
|
||||
return (rows as Array<Record<string, unknown>>).map((r): DomainBankRow => ({
|
||||
slug: r.slug as string,
|
||||
source_id: r.source_id as string,
|
||||
prefix: r.prefix as string | null,
|
||||
page_id: Number(r.page_id),
|
||||
title: r.title as string | null,
|
||||
compiled_truth: (r.compiled_truth as string | null) ?? '',
|
||||
connection_count: Number(r.connection_count),
|
||||
last_retrieved_at: r.last_retrieved_at == null ? null : new Date(r.last_retrieved_at as string),
|
||||
representative_chunk_id: r.representative_chunk_id == null ? null : Number(r.representative_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
async resolveSlugs(partial: string): Promise<string[]> {
|
||||
// Try exact match first
|
||||
const exact = await this.db.query('SELECT slug FROM pages WHERE slug = $1', [partial]);
|
||||
|
||||
@@ -81,6 +81,9 @@ CREATE TABLE IF NOT EXISTS pages (
|
||||
effective_date_source TEXT,
|
||||
import_filename TEXT,
|
||||
salience_touched_at TIMESTAMPTZ,
|
||||
-- v0.37.0 (migration v79): real stale-page signal for gbrain lsd
|
||||
-- (mirrors src/schema.sql). NULL = never retrieved.
|
||||
last_retrieved_at TIMESTAMPTZ,
|
||||
CONSTRAINT pages_source_slug_key UNIQUE (source_id, slug)
|
||||
);
|
||||
|
||||
@@ -94,6 +97,10 @@ CREATE INDEX IF NOT EXISTS pages_deleted_at_purge_idx
|
||||
-- v0.29.1: expression index for since/until date-range filters.
|
||||
CREATE INDEX IF NOT EXISTS pages_coalesce_date_idx
|
||||
ON pages ((COALESCE(effective_date, updated_at)));
|
||||
-- v0.37.0: full B-tree index on last_retrieved_at supports LSD's stale-page
|
||||
-- query (mirrors src/schema.sql). Postgres handles NULL in B-tree indexes.
|
||||
CREATE INDEX IF NOT EXISTS pages_last_retrieved_at_idx
|
||||
ON pages (last_retrieved_at);
|
||||
|
||||
-- ============================================================
|
||||
-- content_chunks: chunked content with embeddings
|
||||
|
||||
+183
-2
@@ -11,6 +11,9 @@ import type {
|
||||
FactRow, FactKind, FactVisibility, FactInsertStatus,
|
||||
NewFact, FactListOpts, FactsHealth,
|
||||
} from './engine.ts';
|
||||
import type {
|
||||
DomainBankSampleOpts, CorpusSampleOpts, DomainBankRow,
|
||||
} from './types.ts';
|
||||
import { MAX_SEARCH_LIMIT, clampSearchLimit } from './engine.ts';
|
||||
import { deriveResolutionTuple, finalizeScorecard } from './takes-resolution.ts';
|
||||
import { normalizeWeightForStorage } from './takes-fence.ts';
|
||||
@@ -403,7 +406,9 @@ export class PostgresEngine implements BrainEngine {
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archived_at') AS sources_archived_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archive_expires_at') AS sources_archive_expires_at_exists
|
||||
WHERE table_schema = current_schema() AND table_name = 'sources' AND column_name = 'archive_expires_at') AS sources_archive_expires_at_exists,
|
||||
EXISTS (SELECT 1 FROM information_schema.columns
|
||||
WHERE table_schema = current_schema() AND table_name = 'pages' AND column_name = 'last_retrieved_at') AS pages_last_retrieved_at_exists
|
||||
`;
|
||||
const probe = probeRows[0]!;
|
||||
|
||||
@@ -451,12 +456,18 @@ export class PostgresEngine implements BrainEngine {
|
||||
&& (!probe.sources_archived_exists
|
||||
|| !probe.sources_archived_at_exists
|
||||
|| !probe.sources_archive_expires_at_exists);
|
||||
// v0.37.0 (v79): pages_last_retrieved_at_idx in SCHEMA_SQL references
|
||||
// last_retrieved_at. Pre-v79 brains crash without the column; bootstrap
|
||||
// adds it before SCHEMA_SQL replay creates the index. v79 runs later
|
||||
// via runMigrations and is idempotent.
|
||||
const needsPagesLastRetrievedAt = probe.pages_exists && !(probe as { pages_last_retrieved_at_exists?: boolean }).pages_last_retrieved_at_exists;
|
||||
|
||||
if (!needsPagesBootstrap && !needsLinksBootstrap && !needsChunksBootstrap
|
||||
&& !needsPagesDeletedAt && !needsMcpLogBootstrap && !needsSubagentProviderId
|
||||
&& !needsChunksEmbeddingImage && !needsPagesRecency
|
||||
&& !needsIngestLogSourceId && !needsFilesBootstrap
|
||||
&& !needsOauthClientsBootstrap && !needsSourcesArchive) return;
|
||||
&& !needsOauthClientsBootstrap && !needsSourcesArchive
|
||||
&& !needsPagesLastRetrievedAt) return;
|
||||
|
||||
console.log(' Pre-v0.21 brain detected, applying forward-reference bootstrap');
|
||||
|
||||
@@ -635,6 +646,16 @@ export class PostgresEngine implements BrainEngine {
|
||||
ALTER TABLE sources ADD COLUMN IF NOT EXISTS archive_expires_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
|
||||
if (needsPagesLastRetrievedAt) {
|
||||
// v79 (pages_last_retrieved_at): adds the real stale-page signal column
|
||||
// + full B-tree index. SCHEMA_SQL's CREATE INDEX
|
||||
// pages_last_retrieved_at_idx crashes without the column. v79 runs
|
||||
// later via runMigrations and is idempotent.
|
||||
await conn.unsafe(`
|
||||
ALTER TABLE pages ADD COLUMN IF NOT EXISTS last_retrieved_at TIMESTAMPTZ;
|
||||
`);
|
||||
}
|
||||
}
|
||||
|
||||
async transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T> {
|
||||
@@ -904,6 +925,166 @@ export class PostgresEngine implements BrainEngine {
|
||||
return rows.map((r) => ({ slug: r.slug as string, source_id: r.source_id as string }));
|
||||
}
|
||||
|
||||
// v0.37.0 — domain-bank engine methods (D14 + D5 + D10).
|
||||
//
|
||||
// `listPrefixSampledPages`: one page per prefix, tiebroken by inbound-link
|
||||
// count (connection_count via LEFT JOIN to page_links). Stale-bias optional
|
||||
// for LSD mode (D5). Source-scoped (D5). Excludes close-set slugs.
|
||||
//
|
||||
// Ranking inside each prefix partition:
|
||||
// 1. stale_score DESC (when staleBias) — never-retrieved beats >90d-stale beats fresh
|
||||
// 2. connection_count DESC — structural-centrality tiebreaker (D10)
|
||||
// 3. slug ASC — deterministic for tests
|
||||
async listPrefixSampledPages(opts: DomainBankSampleOpts): Promise<DomainBankRow[]> {
|
||||
const sql = this.sql;
|
||||
if (opts.prefixes.length === 0) return [];
|
||||
const exclude = opts.excludeSlugs ?? [];
|
||||
const staleBias = opts.staleBias === true;
|
||||
const staleThreshold = opts.staleThresholdDays ?? 90;
|
||||
// Source scoping (D5, codex r2 #2 — federated array wins over scalar).
|
||||
const sourceIds = opts.sourceIds ?? null;
|
||||
const sourceId = opts.sourceId ?? null;
|
||||
const rows = await sql`
|
||||
WITH prefix_pages AS (
|
||||
SELECT
|
||||
p.id AS page_id,
|
||||
p.slug,
|
||||
p.source_id,
|
||||
p.title,
|
||||
p.compiled_truth,
|
||||
p.last_retrieved_at,
|
||||
substring(p.slug from '^[^/]+/[^/]+') AS prefix,
|
||||
COUNT(pl.id) AS connection_count
|
||||
FROM pages p
|
||||
LEFT JOIN page_links pl ON pl.to_page_id = p.id
|
||||
WHERE p.deleted_at IS NULL
|
||||
AND substring(p.slug from '^[^/]+/[^/]+') = ANY(${opts.prefixes}::text[])
|
||||
AND (cardinality(${exclude}::text[]) = 0 OR NOT (p.slug = ANY(${exclude}::text[])))
|
||||
AND (
|
||||
(${sourceIds}::text[] IS NOT NULL AND p.source_id = ANY(${sourceIds}::text[]))
|
||||
OR (${sourceIds}::text[] IS NULL AND ${sourceId}::text IS NOT NULL AND p.source_id = ${sourceId})
|
||||
OR (${sourceIds}::text[] IS NULL AND ${sourceId}::text IS NULL)
|
||||
)
|
||||
GROUP BY p.id, p.slug, p.source_id, p.title, p.compiled_truth, p.last_retrieved_at
|
||||
),
|
||||
ranked AS (
|
||||
SELECT
|
||||
pp.*,
|
||||
(CASE WHEN ${staleBias}::boolean THEN
|
||||
CASE
|
||||
WHEN pp.last_retrieved_at IS NULL THEN 2
|
||||
WHEN pp.last_retrieved_at < NOW() - (${staleThreshold}::int * INTERVAL '1 day') THEN 1
|
||||
ELSE 0
|
||||
END
|
||||
ELSE 0
|
||||
END) AS stale_score,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY pp.prefix
|
||||
ORDER BY
|
||||
(CASE WHEN ${staleBias}::boolean THEN
|
||||
CASE
|
||||
WHEN pp.last_retrieved_at IS NULL THEN 2
|
||||
WHEN pp.last_retrieved_at < NOW() - (${staleThreshold}::int * INTERVAL '1 day') THEN 1
|
||||
ELSE 0
|
||||
END
|
||||
ELSE 0
|
||||
END) DESC,
|
||||
pp.connection_count DESC,
|
||||
pp.slug ASC
|
||||
) AS rn
|
||||
FROM prefix_pages pp
|
||||
),
|
||||
with_chunk AS (
|
||||
SELECT
|
||||
r.*,
|
||||
(
|
||||
SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = r.page_id AND cc.embedding IS NOT NULL
|
||||
ORDER BY cc.chunk_index ASC
|
||||
LIMIT 1
|
||||
) AS representative_chunk_id
|
||||
FROM ranked r
|
||||
WHERE r.rn = 1
|
||||
)
|
||||
SELECT page_id, slug, source_id, title, compiled_truth, last_retrieved_at,
|
||||
prefix, connection_count, representative_chunk_id
|
||||
FROM with_chunk
|
||||
ORDER BY prefix
|
||||
`;
|
||||
return rows.map((r): DomainBankRow => ({
|
||||
slug: r.slug as string,
|
||||
source_id: r.source_id as string,
|
||||
prefix: r.prefix as string | null,
|
||||
page_id: Number(r.page_id),
|
||||
title: r.title as string | null,
|
||||
compiled_truth: (r.compiled_truth as string | null) ?? '',
|
||||
connection_count: Number(r.connection_count),
|
||||
last_retrieved_at: r.last_retrieved_at as Date | null,
|
||||
representative_chunk_id: r.representative_chunk_id == null ? null : Number(r.representative_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
// v0.37.0 — corpus-sampling fallback when prefix-stratified can't fill M.
|
||||
// Deterministic with opts.seed (setseed before SELECT); random otherwise.
|
||||
async listCorpusSample(opts: CorpusSampleOpts): Promise<DomainBankRow[]> {
|
||||
const sql = this.sql;
|
||||
if (opts.n <= 0) return [];
|
||||
const exclude = opts.excludeSlugs ?? [];
|
||||
const sourceIds = opts.sourceIds ?? null;
|
||||
const sourceId = opts.sourceId ?? null;
|
||||
// setseed deterministic path: use SELECT setseed($1) + RANDOM(). PGLite/Postgres
|
||||
// both honor setseed for the same session/transaction. For tests this gives
|
||||
// identical ordering across runs.
|
||||
if (typeof opts.seed === 'number') {
|
||||
// Clamp to [-1, 1] required by setseed.
|
||||
const clamped = Math.max(-1, Math.min(1, opts.seed));
|
||||
await sql`SELECT setseed(${clamped}::float8)`;
|
||||
}
|
||||
const rows = await sql`
|
||||
WITH sampled AS (
|
||||
SELECT
|
||||
p.id AS page_id,
|
||||
p.slug,
|
||||
p.source_id,
|
||||
p.title,
|
||||
p.compiled_truth,
|
||||
p.last_retrieved_at,
|
||||
substring(p.slug from '^[^/]+/[^/]+') AS prefix,
|
||||
(SELECT COUNT(*) FROM page_links pl WHERE pl.to_page_id = p.id) AS connection_count
|
||||
FROM pages p
|
||||
WHERE p.deleted_at IS NULL
|
||||
AND (cardinality(${exclude}::text[]) = 0 OR NOT (p.slug = ANY(${exclude}::text[])))
|
||||
AND (
|
||||
(${sourceIds}::text[] IS NOT NULL AND p.source_id = ANY(${sourceIds}::text[]))
|
||||
OR (${sourceIds}::text[] IS NULL AND ${sourceId}::text IS NOT NULL AND p.source_id = ${sourceId})
|
||||
OR (${sourceIds}::text[] IS NULL AND ${sourceId}::text IS NULL)
|
||||
)
|
||||
ORDER BY RANDOM()
|
||||
LIMIT ${opts.n}
|
||||
)
|
||||
SELECT
|
||||
s.*,
|
||||
(
|
||||
SELECT cc.id FROM content_chunks cc
|
||||
WHERE cc.page_id = s.page_id AND cc.embedding IS NOT NULL
|
||||
ORDER BY cc.chunk_index ASC
|
||||
LIMIT 1
|
||||
) AS representative_chunk_id
|
||||
FROM sampled s
|
||||
`;
|
||||
return rows.map((r): DomainBankRow => ({
|
||||
slug: r.slug as string,
|
||||
source_id: r.source_id as string,
|
||||
prefix: r.prefix as string | null,
|
||||
page_id: Number(r.page_id),
|
||||
title: r.title as string | null,
|
||||
compiled_truth: (r.compiled_truth as string | null) ?? '',
|
||||
connection_count: Number(r.connection_count),
|
||||
last_retrieved_at: r.last_retrieved_at as Date | null,
|
||||
representative_chunk_id: r.representative_chunk_id == null ? null : Number(r.representative_chunk_id),
|
||||
}));
|
||||
}
|
||||
|
||||
async resolveSlugs(partial: string): Promise<string[]> {
|
||||
const sql = this.sql;
|
||||
|
||||
|
||||
@@ -103,6 +103,11 @@ CREATE TABLE IF NOT EXISTS pages (
|
||||
effective_date_source TEXT,
|
||||
import_filename TEXT,
|
||||
salience_touched_at TIMESTAMPTZ,
|
||||
-- v0.37.0 (migration v79): real stale-page signal for \`gbrain lsd\`. Bumped
|
||||
-- by op-layer write-back inside \`search\`/\`query\`/\`get_page\` op handlers
|
||||
-- (NOT inside engine methods — internal callers must not pollute the
|
||||
-- signal). NULL = never retrieved (LSD prioritizes these first).
|
||||
last_retrieved_at TIMESTAMPTZ,
|
||||
CONSTRAINT pages_source_slug_key UNIQUE (source_id, slug)
|
||||
);
|
||||
|
||||
@@ -120,6 +125,13 @@ CREATE INDEX IF NOT EXISTS idx_pages_source_id ON pages(source_id);
|
||||
-- stays low. Don't add a regular \`(deleted_at)\` index without measuring.
|
||||
CREATE INDEX IF NOT EXISTS pages_deleted_at_purge_idx
|
||||
ON pages (deleted_at) WHERE deleted_at IS NOT NULL;
|
||||
-- v0.37.0: full B-tree index on last_retrieved_at supports LSD's stale-page
|
||||
-- query \`WHERE last_retrieved_at IS NULL OR last_retrieved_at < NOW() -
|
||||
-- INTERVAL '90 days'\`. Postgres handles NULL in B-tree indexes (sorted to
|
||||
-- one end) so one index covers both branches. A partial WHERE NOT NULL
|
||||
-- would miss the NULL branch that LSD prioritizes (codex round 2 #6).
|
||||
CREATE INDEX IF NOT EXISTS pages_last_retrieved_at_idx
|
||||
ON pages (last_retrieved_at);
|
||||
-- v0.29.1: expression index used by since/until date-range filters that read
|
||||
-- COALESCE(effective_date, updated_at). A partial index on effective_date
|
||||
-- alone would NOT help — the planner can't use it for the negative side of
|
||||
|
||||
@@ -228,6 +228,60 @@ export const PAGE_SORT_SQL: Record<NonNullable<PageFilters['sort']>, string> = {
|
||||
* See `src/core/cycle/emotional-weight.ts` for the score formula and
|
||||
* `engine.getRecentSalience` for the SQL.
|
||||
*/
|
||||
/**
|
||||
* v0.37.0 — domain-bank sampling for `gbrain brainstorm` / `gbrain lsd`.
|
||||
* Pulls one page per prefix from a caller-supplied prefix list.
|
||||
* Tiebreaker via JOIN to page_links (inbound link count = "structural centrality").
|
||||
* Stale-bias optional (LSD mode prefers forgotten pages via `last_retrieved_at`).
|
||||
* sourceId/sourceIds threaded from day 1 per [source-id-canonical-thread].
|
||||
*/
|
||||
export interface DomainBankSampleOpts {
|
||||
/** Top-level slug prefixes to sample from (e.g. ['wiki/vc', 'wiki/biology']). */
|
||||
prefixes: string[];
|
||||
/** Slugs to exclude (typically the close-set from hybridSearch). */
|
||||
excludeSlugs?: string[];
|
||||
/** When true, prefer pages with NULL last_retrieved_at or > staleThresholdDays old. */
|
||||
staleBias?: boolean;
|
||||
/** Days threshold for "stale" classification. Default 90. */
|
||||
staleThresholdDays?: number;
|
||||
/** Single-source scope (canonical scalar form). */
|
||||
sourceId?: string;
|
||||
/** Federated read scope (array form, wins over scalar). */
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
/** v0.37.0 — corpus-sampling fallback for `gbrain brainstorm` when prefix-stratified can't fill M. */
|
||||
export interface CorpusSampleOpts {
|
||||
/** Number of pages to sample. */
|
||||
n: number;
|
||||
/** Slugs to exclude (close-set + already-picked-by-prefix-stratified). */
|
||||
excludeSlugs?: string[];
|
||||
/** Stable seed for deterministic sampling in tests. Falls back to random when omitted. */
|
||||
seed?: number;
|
||||
/** Single-source scope. */
|
||||
sourceId?: string;
|
||||
/** Federated read scope. */
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
/** v0.37.0 — one row per page returned by domain-bank's prefix/corpus sampling. */
|
||||
export interface DomainBankRow {
|
||||
slug: string;
|
||||
source_id: string;
|
||||
/** Top-level prefix (`^[^/]+/[^/]+`) or null for short slugs. */
|
||||
prefix: string | null;
|
||||
page_id: number;
|
||||
title: string | null;
|
||||
/** Page body — what gets injected into the brainstorm prompt as "the user wrote..." */
|
||||
compiled_truth: string;
|
||||
/** COUNT(page_links.id WHERE to_page_id = this) — inbound link count, the "structural centrality" tiebreaker (D10). */
|
||||
connection_count: number;
|
||||
/** When this page was last surfaced by a user-facing search/query. Powers LSD stale-bias. */
|
||||
last_retrieved_at: Date | null;
|
||||
/** Lowest chunk_index with non-null embedding on the default embedding_column. Null if no embedded chunks. */
|
||||
representative_chunk_id: number | null;
|
||||
}
|
||||
|
||||
export interface SalienceOpts {
|
||||
/** Window in days. Default 14. */
|
||||
days?: number;
|
||||
|
||||
@@ -99,6 +99,11 @@ CREATE TABLE IF NOT EXISTS pages (
|
||||
effective_date_source TEXT,
|
||||
import_filename TEXT,
|
||||
salience_touched_at TIMESTAMPTZ,
|
||||
-- v0.37.0 (migration v79): real stale-page signal for `gbrain lsd`. Bumped
|
||||
-- by op-layer write-back inside `search`/`query`/`get_page` op handlers
|
||||
-- (NOT inside engine methods — internal callers must not pollute the
|
||||
-- signal). NULL = never retrieved (LSD prioritizes these first).
|
||||
last_retrieved_at TIMESTAMPTZ,
|
||||
CONSTRAINT pages_source_slug_key UNIQUE (source_id, slug)
|
||||
);
|
||||
|
||||
@@ -116,6 +121,13 @@ CREATE INDEX IF NOT EXISTS idx_pages_source_id ON pages(source_id);
|
||||
-- stays low. Don't add a regular `(deleted_at)` index without measuring.
|
||||
CREATE INDEX IF NOT EXISTS pages_deleted_at_purge_idx
|
||||
ON pages (deleted_at) WHERE deleted_at IS NOT NULL;
|
||||
-- v0.37.0: full B-tree index on last_retrieved_at supports LSD's stale-page
|
||||
-- query `WHERE last_retrieved_at IS NULL OR last_retrieved_at < NOW() -
|
||||
-- INTERVAL '90 days'`. Postgres handles NULL in B-tree indexes (sorted to
|
||||
-- one end) so one index covers both branches. A partial WHERE NOT NULL
|
||||
-- would miss the NULL branch that LSD prioritizes (codex round 2 #6).
|
||||
CREATE INDEX IF NOT EXISTS pages_last_retrieved_at_idx
|
||||
ON pages (last_retrieved_at);
|
||||
-- v0.29.1: expression index used by since/until date-range filters that read
|
||||
-- COALESCE(effective_date, updated_at). A partial index on effective_date
|
||||
-- alone would NOT help — the planner can't use it for the negative side of
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* v0.37.0 — domain-bank distance normalization (D6 + codex r2 #9).
|
||||
*
|
||||
* Pinned cases from the codex-r2 fix: same-vector → 0, orthogonal → 0.5,
|
||||
* opposite → 1, missing-vector → caller responsibility (skipped at retrieval).
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { normalizedCosineDistance } from '../../src/core/brainstorm/domain-bank.ts';
|
||||
|
||||
function v(...nums: number[]): Float32Array {
|
||||
return new Float32Array(nums);
|
||||
}
|
||||
|
||||
describe('normalizedCosineDistance — codex r2 #9 pinned cases', () => {
|
||||
test('same vector → distance 0 (identical)', () => {
|
||||
const a = v(0.6, 0.8, 0.0);
|
||||
expect(normalizedCosineDistance(a, a)).toBeCloseTo(0, 6);
|
||||
});
|
||||
|
||||
test('orthogonal unit vectors → distance 0.5 (neutral)', () => {
|
||||
const x = v(1, 0, 0);
|
||||
const y = v(0, 1, 0);
|
||||
expect(normalizedCosineDistance(x, y)).toBeCloseTo(0.5, 6);
|
||||
});
|
||||
|
||||
test('opposite unit vectors → distance 1 (maximally far)', () => {
|
||||
const x = v(1, 0, 0);
|
||||
const y = v(-1, 0, 0);
|
||||
expect(normalizedCosineDistance(x, y)).toBeCloseTo(1, 6);
|
||||
});
|
||||
|
||||
test('45-degree separation lands between 0 and 0.5', () => {
|
||||
// cos(45°) ≈ 0.707, so cosDist ≈ 0.293, halved → 0.146
|
||||
const x = v(1, 0);
|
||||
const y = v(Math.sqrt(0.5), Math.sqrt(0.5));
|
||||
const d = normalizedCosineDistance(x, y);
|
||||
expect(d).toBeGreaterThan(0.1);
|
||||
expect(d).toBeLessThan(0.2);
|
||||
});
|
||||
|
||||
test('zero-vector edge → 0.5 (neutral, no division-by-zero)', () => {
|
||||
const z = v(0, 0, 0);
|
||||
const a = v(1, 1, 1);
|
||||
expect(normalizedCosineDistance(z, a)).toBeCloseTo(0.5, 6);
|
||||
});
|
||||
|
||||
test('dimension mismatch throws', () => {
|
||||
expect(() => normalizedCosineDistance(v(1, 0), v(1, 0, 0))).toThrow(/dim mismatch/);
|
||||
});
|
||||
|
||||
test('result is symmetric: d(a,b) === d(b,a)', () => {
|
||||
const a = v(0.3, 0.4, 0.5);
|
||||
const b = v(0.7, 0.1, 0.2);
|
||||
expect(normalizedCosineDistance(a, b)).toBeCloseTo(normalizedCosineDistance(b, a), 6);
|
||||
});
|
||||
|
||||
test('result is bounded [0, 1] even on non-unit vectors', () => {
|
||||
const a = v(10, 0, 0);
|
||||
const b = v(0, 5, 0);
|
||||
const d = normalizedCosineDistance(a, b);
|
||||
expect(d).toBeGreaterThanOrEqual(0);
|
||||
expect(d).toBeLessThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,201 @@
|
||||
/**
|
||||
* v0.37.0 — `gbrain eval brainstorm` pure-function tests (D3 + codex r2 #11).
|
||||
*
|
||||
* The orchestrator + judge themselves are exercised in E2E with a real
|
||||
* brain; here we pin the eval math (grounding rate + verdict computation +
|
||||
* threshold semantics) since those are what gate the eval suite.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
computeGroundingRate,
|
||||
computeVerdict,
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
readBrainstormEvalFixture,
|
||||
type PerFixtureResult,
|
||||
} from '../../src/commands/eval-brainstorm.ts';
|
||||
import { writeFileSync, mkdtempSync, rmSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
|
||||
describe('computeGroundingRate', () => {
|
||||
const real = new Set(['wiki/vc/alice', 'wiki/biology/bee', 'wiki/hardware/asic']);
|
||||
|
||||
test('100% grounding when every idea cites a real slug', () => {
|
||||
const ideas = [
|
||||
{ close_slug: 'wiki/vc/alice', far_slug: 'wiki/biology/bee' },
|
||||
{ close_slug: 'wiki/biology/bee', far_slug: 'wiki/hardware/asic' },
|
||||
];
|
||||
expect(computeGroundingRate(ideas, real)).toBe(1.0);
|
||||
});
|
||||
|
||||
test('50% grounding when half cite hallucinated slugs', () => {
|
||||
const ideas = [
|
||||
{ close_slug: 'wiki/vc/alice', far_slug: 'wiki/biology/bee' },
|
||||
{ close_slug: 'wiki/fake/no-such-page', far_slug: 'wiki/also-fake' },
|
||||
];
|
||||
expect(computeGroundingRate(ideas, real)).toBe(0.5);
|
||||
});
|
||||
|
||||
test('one-real-citation counts as grounded (close OR far real)', () => {
|
||||
const ideas = [
|
||||
{ close_slug: 'wiki/vc/alice', far_slug: 'wiki/hallucinated' },
|
||||
];
|
||||
expect(computeGroundingRate(ideas, real)).toBe(1.0);
|
||||
});
|
||||
|
||||
test('0% grounding when all slugs are hallucinated', () => {
|
||||
const ideas = [
|
||||
{ close_slug: 'wiki/fake/one', far_slug: 'wiki/fake/two' },
|
||||
];
|
||||
expect(computeGroundingRate(ideas, real)).toBe(0);
|
||||
});
|
||||
|
||||
test('empty ideas array → 0 (no division by zero)', () => {
|
||||
expect(computeGroundingRate([], real)).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
function mkFixture(partial: Partial<PerFixtureResult>): PerFixtureResult {
|
||||
return {
|
||||
question: 'test',
|
||||
pass_count: 5,
|
||||
total_ideas: 5,
|
||||
mean_distance: 0.5,
|
||||
mean_usefulness: 4.0,
|
||||
grounding_rate: 1.0,
|
||||
short_of_target: false,
|
||||
cost_usd: 0.10,
|
||||
judge_failed: false,
|
||||
...partial,
|
||||
};
|
||||
}
|
||||
|
||||
describe('computeVerdict', () => {
|
||||
test('pass when all three axes clear', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ mean_distance: 0.5, mean_usefulness: 4.0, grounding_rate: 1.0 }),
|
||||
mkFixture({ mean_distance: 0.6, mean_usefulness: 4.2, grounding_rate: 1.0 }),
|
||||
],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('pass');
|
||||
});
|
||||
|
||||
test('fail when distance below threshold', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ mean_distance: 0.2, mean_usefulness: 4.0, grounding_rate: 1.0 }),
|
||||
mkFixture({ mean_distance: 0.25, mean_usefulness: 4.0, grounding_rate: 1.0 }),
|
||||
],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('fail');
|
||||
expect(res.reasons.some((r) => r.includes('distance'))).toBe(true);
|
||||
});
|
||||
|
||||
test('fail when usefulness below threshold (codex r2 #11 — distance alone is gameable)', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ mean_distance: 0.6, mean_usefulness: 2.5, grounding_rate: 1.0 }),
|
||||
mkFixture({ mean_distance: 0.6, mean_usefulness: 2.8, grounding_rate: 1.0 }),
|
||||
],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('fail');
|
||||
expect(res.reasons.some((r) => r.includes('usefulness'))).toBe(true);
|
||||
});
|
||||
|
||||
test('fail when grounding below 1.0 (every idea must cite a real slug)', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ mean_distance: 0.6, mean_usefulness: 4.0, grounding_rate: 0.7 }),
|
||||
mkFixture({ mean_distance: 0.6, mean_usefulness: 4.0, grounding_rate: 0.9 }),
|
||||
],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('fail');
|
||||
expect(res.reasons.some((r) => r.includes('grounding'))).toBe(true);
|
||||
});
|
||||
|
||||
test('inconclusive when <2 fixtures usable', () => {
|
||||
const res = computeVerdict(
|
||||
[mkFixture({ pass_count: 5 })],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('inconclusive');
|
||||
});
|
||||
|
||||
test('inconclusive when all fixtures have judge_failed', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ judge_failed: true }),
|
||||
mkFixture({ judge_failed: true }),
|
||||
],
|
||||
DEFAULT_BRAINSTORM_THRESHOLDS,
|
||||
);
|
||||
expect(res.verdict).toBe('inconclusive');
|
||||
});
|
||||
|
||||
test('threshold overrides honored', () => {
|
||||
const res = computeVerdict(
|
||||
[
|
||||
mkFixture({ mean_distance: 0.3, mean_usefulness: 4.0, grounding_rate: 1.0 }),
|
||||
mkFixture({ mean_distance: 0.35, mean_usefulness: 4.0, grounding_rate: 1.0 }),
|
||||
],
|
||||
{ distance_min: 0.25, usefulness_min: 3.5, grounding_min: 1.0 },
|
||||
);
|
||||
expect(res.verdict).toBe('pass');
|
||||
});
|
||||
});
|
||||
|
||||
describe('readBrainstormEvalFixture', () => {
|
||||
const tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-eval-brainstorm-'));
|
||||
|
||||
test('parses valid JSONL with one question per line', () => {
|
||||
const f = join(tmpDir, 'good.jsonl');
|
||||
writeFileSync(f, [
|
||||
'{"question": "why is X"}',
|
||||
'{"question": "what about Y"}',
|
||||
'',
|
||||
'{"question": "and Z"}',
|
||||
].join('\n'));
|
||||
const out = readBrainstormEvalFixture(f);
|
||||
expect(out.length).toBe(3);
|
||||
expect(out[0].question).toBe('why is X');
|
||||
expect(out[2].question).toBe('and Z');
|
||||
});
|
||||
|
||||
test('skips malformed JSON lines', () => {
|
||||
const f = join(tmpDir, 'mixed.jsonl');
|
||||
writeFileSync(f, [
|
||||
'{"question": "good one"}',
|
||||
'not json at all',
|
||||
'{"question": "another good"}',
|
||||
'{"no_question_field": "skipped"}',
|
||||
].join('\n'));
|
||||
const out = readBrainstormEvalFixture(f);
|
||||
expect(out.length).toBe(2);
|
||||
expect(out.map((f) => f.question)).toEqual(['good one', 'another good']);
|
||||
});
|
||||
|
||||
test('honors expected_far_prefixes when present', () => {
|
||||
const f = join(tmpDir, 'prefixes.jsonl');
|
||||
writeFileSync(f, JSON.stringify({
|
||||
question: 'cross-pollinate this',
|
||||
expected_far_prefixes: ['wiki/biology', 'wiki/hardware'],
|
||||
}));
|
||||
const out = readBrainstormEvalFixture(f);
|
||||
expect(out[0].expected_far_prefixes).toEqual(['wiki/biology', 'wiki/hardware']);
|
||||
});
|
||||
|
||||
test('throws on missing file', () => {
|
||||
expect(() => readBrainstormEvalFixture('/no/such/path.jsonl')).toThrow(/not found/);
|
||||
});
|
||||
|
||||
// Cleanup at end
|
||||
test('_cleanup_', () => {
|
||||
rmSync(tmpDir, { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,104 @@
|
||||
/**
|
||||
* v0.37.0 (D9 / D4) — dream cycle hook: synthesize phase MUST skip pages
|
||||
* with `mode: lsd` frontmatter (noise-by-design). Pinned via the same
|
||||
* `isDreamOutput` helper that drives the self-consumption guard.
|
||||
*/
|
||||
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import {
|
||||
isDreamOutput,
|
||||
isLsdOutput,
|
||||
isBrainstormOutput,
|
||||
} from '../../src/core/cycle/transcript-discovery.ts';
|
||||
|
||||
const LSD_FRONTMATTER = `---
|
||||
title: "LSD: why are AI coding tools converging on the same UX?"
|
||||
mode: lsd
|
||||
generated_at: 2026-05-19T12:00:00Z
|
||||
question: "why are AI coding tools converging on the same UX?"
|
||||
---
|
||||
|
||||
# Some inverted-judge idea here.
|
||||
`;
|
||||
|
||||
const BRAINSTORM_FRONTMATTER = `---
|
||||
title: "Brainstorm: lab automation"
|
||||
mode: brainstorm
|
||||
generated_at: 2026-05-19T12:00:00Z
|
||||
question: "what's the bottleneck on lab automation?"
|
||||
---
|
||||
|
||||
# Some defensible idea here.
|
||||
`;
|
||||
|
||||
const DREAM_OUTPUT_FRONTMATTER = `---
|
||||
title: "Dream: monthly synthesis"
|
||||
dream_generated: true
|
||||
dream_cycle_date: 2026-05-01
|
||||
---
|
||||
|
||||
Body.
|
||||
`;
|
||||
|
||||
const REGULAR_TRANSCRIPT = `# Some meeting transcript
|
||||
|
||||
Person A: We should talk about lab automation.
|
||||
Person B: Agreed.
|
||||
`;
|
||||
|
||||
describe('v0.37.0 — LSD frontmatter skip in dream-cycle', () => {
|
||||
test('LSD page is detected by isLsdOutput', () => {
|
||||
expect(isLsdOutput(LSD_FRONTMATTER)).toBe(true);
|
||||
});
|
||||
|
||||
test('brainstorm page is NOT detected by isLsdOutput', () => {
|
||||
expect(isLsdOutput(BRAINSTORM_FRONTMATTER)).toBe(false);
|
||||
});
|
||||
|
||||
test('regular transcript is NOT detected by isLsdOutput', () => {
|
||||
expect(isLsdOutput(REGULAR_TRANSCRIPT)).toBe(false);
|
||||
});
|
||||
|
||||
test('brainstorm page is detected by isBrainstormOutput', () => {
|
||||
expect(isBrainstormOutput(BRAINSTORM_FRONTMATTER)).toBe(true);
|
||||
});
|
||||
|
||||
test('LSD page is NOT detected by isBrainstormOutput', () => {
|
||||
expect(isBrainstormOutput(LSD_FRONTMATTER)).toBe(false);
|
||||
});
|
||||
|
||||
test('isDreamOutput SKIPS LSD pages (D4 noise-by-design)', () => {
|
||||
expect(isDreamOutput(LSD_FRONTMATTER)).toBe(true);
|
||||
});
|
||||
|
||||
test('isDreamOutput still skips legitimate dream output', () => {
|
||||
expect(isDreamOutput(DREAM_OUTPUT_FRONTMATTER)).toBe(true);
|
||||
});
|
||||
|
||||
test('isDreamOutput does NOT skip brainstorm pages (they are user-validated content)', () => {
|
||||
expect(isDreamOutput(BRAINSTORM_FRONTMATTER)).toBe(false);
|
||||
});
|
||||
|
||||
test('isDreamOutput does NOT skip regular transcripts', () => {
|
||||
expect(isDreamOutput(REGULAR_TRANSCRIPT)).toBe(false);
|
||||
});
|
||||
|
||||
test('--unsafe-bypass-dream-guard does NOT bypass LSD skip', () => {
|
||||
// Bypass is for self-consumption recovery only; LSD must always be skipped.
|
||||
expect(isDreamOutput(LSD_FRONTMATTER, true)).toBe(true);
|
||||
});
|
||||
|
||||
test('--unsafe-bypass-dream-guard DOES bypass dream output skip', () => {
|
||||
expect(isDreamOutput(DREAM_OUTPUT_FRONTMATTER, true)).toBe(false);
|
||||
});
|
||||
|
||||
test('LSD marker tolerates double-quoted value', () => {
|
||||
const dq = LSD_FRONTMATTER.replace('mode: lsd', 'mode: "lsd"');
|
||||
expect(isLsdOutput(dq)).toBe(true);
|
||||
});
|
||||
|
||||
test('LSD marker tolerates single-quoted value', () => {
|
||||
const sq = LSD_FRONTMATTER.replace('mode: lsd', "mode: 'lsd'");
|
||||
expect(isLsdOutput(sq)).toBe(true);
|
||||
});
|
||||
});
|
||||
Vendored
+5
@@ -0,0 +1,5 @@
|
||||
{"question": "why are AI coding tools converging on the same UX?"}
|
||||
{"question": "the unspoken assumption in early-stage venture pricing"}
|
||||
{"question": "what's the real bottleneck on lab automation"}
|
||||
{"question": "how should we think about ML model fine-tuning in 2026"}
|
||||
{"question": "what makes a developer experience feel inevitable"}
|
||||
@@ -129,6 +129,11 @@ const REQUIRED_BOOTSTRAP_COVERAGE: ForwardReference[] = [
|
||||
{ kind: 'column', table: 'sources', column: 'archived' },
|
||||
{ kind: 'column', table: 'sources', column: 'archived_at' },
|
||||
{ kind: 'column', table: 'sources', column: 'archive_expires_at' },
|
||||
// v0.37.0 (v79) — forward-referenced by `CREATE INDEX
|
||||
// pages_last_retrieved_at_idx ON pages (last_retrieved_at)`. Pre-v79 brains
|
||||
// have pages without this column; bootstrap adds it before SCHEMA_SQL
|
||||
// replay creates the index.
|
||||
{ kind: 'column', table: 'pages', column: 'last_retrieved_at' },
|
||||
];
|
||||
|
||||
test('applyForwardReferenceBootstrap covers every forward reference declared in REQUIRED_BOOTSTRAP_COVERAGE', async () => {
|
||||
|
||||
Reference in New Issue
Block a user