Files
gbrain/src/core/engine.ts
T
d6db3f0ce3 v0.41.34.0 feat(search): retrieval cathedral — max-pool + title + alias + evidence (#1657)
* feat(search): per-page max-pool in searchVector (both engines)

T1 of the retrieval-cathedral wave (supersedes #1616). Vector search returned
chunk-grain top-k with no DISTINCT ON, so a page could be represented by a
weak chunk while a hub page's chunks crowded a distinct page's strong chunk
out of the candidate set entirely. Keyword search always pooled per page; the
vector path did not.

- New shared buildBestPerPagePoolCte() in sql-ranking.ts — single source of
  truth consumed by searchKeyword + searchVector across postgres + pglite, so
  the two engines can't drift (the recurring parity bug class).
- searchVector both engines: compute score as a select-list expr (HNSW
  ORDER BY stays pure-distance), pool DISTINCT ON (slug) over the full
  candidate set before the user LIMIT, deterministic tiebreak
  (slug, score DESC, page_id ASC, chunk_id ASC).
- All keyword pooling blocks refactored onto the shared builder (DRY).
- Regression test: a hub page's chunks no longer crowd out a distinct page's
  strong chunk; results are one-per-page by best chunk. Fails on old path.

Verified: real-Postgres engine-parity 22/22, PGLite hermetic suite green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): title-phrase boost (page.title first-class signal)

T2 of the retrieval-cathedral wave. A query that is a phrase from a page's
title ("Greek amphitheater" -> "The Mingtang - Indoor Greek Amphitheater")
matched a weak body chunk instead of being recognized as a title hit. Names
of things deserve weight.

- New pure title-match.ts: isTitlePhraseMatch (contiguous token-run inside
  page.title OR exact full-title match). Precision guards: >= 2 content
  tokens OR exact full-title; stopword filter; token-boundary match (no raw
  substring). Reused by the eval later so production + bench can't drift.
- applyTitleBoost post-fusion stage in hybrid.ts: reads page.title (not the
  brittle "first chunk"), floor-ratio-gated, stamps title_match_boost for
  --explain, never touches base_score (the agent's dedup confidence).
- ModeBundle.title_boost knob (1.25, on in all modes - cheap gated
  correctness fix), search.title_boost config key, dashboard description.
- KNOBS_HASH_VERSION 6 -> 7 so a boost-on cache write can't serve a
  boost-off lookup; all version-pin + canonical-bundle assertions updated.
- 18 new tests (matcher 13 + stage 5); typecheck clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): page_aliases data layer (T3 foundation)

Free-text alias resolution for search. gbrain stored a page's chosen names
in pages.frontmatter `aliases:` JSONB but search never consulted them, so a
query like "Hall of Light" or "明堂" couldn't surface the "Mingtang" page.

DELIBERATELY SEPARATE from slug_aliases (re-grounded against current code):
  - slug_aliases:  old-slug -> canonical-slug (wikilink/get_page redirect,
    populated only from concept-redirect conversions)
  - page_aliases:  normalized free-text name -> canonical slug (search hop)
Overloading slug_aliases would muddy two distinct semantics, so this is a
new table, not an extension (honors DRY by keeping concepts separate).

- src/core/search/alias-normalize.ts: ONE normalizeAlias() (NFKC + lowercase
  + ws-collapse + quote-strip) + normalizeAliasList() shared by the write
  (ingest) and read (search) paths so they match on the same key (CQ2).
- Migration v108 page_aliases (source_id, alias_norm, slug); btree
  (source_id, alias_norm) for indexed-equality hop, NOT ILIKE; unique TRIPLE
  (not source_id+alias_norm) so two pages may claim one alias — collisions
  reported + resolved at query time, not blocked at ingest (Codex#8).
  Mirror in pglite-schema.ts; Postgres fresh gets it from the migration.
- engine.resolveAliases(aliasNorms, {sourceId|sourceIds}) read +
  setPageAliases(slug, source, aliasNorms) write, both engines, source-scoped.
- 17 tests: normalize round-trip, collision, source-scope, replace, clear.

Ingest projection + the hybridSearch alias hop land next (T3 wiring).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): alias hop + ingest projection (T3 wiring)

Wires the page_aliases data layer into ingest (write) and hybridSearch (read)
so a query that is a page's declared chosen name surfaces that page — the
named-thing class neither max-pool nor title-boost can fix (true synonyms with
zero surface overlap: "Hall of Light" / "明堂" -> the Mingtang page).

- Ingest projection (import-file.ts): after the page write commits,
  normalizeAliasList(frontmatter.aliases) -> engine.setPageAliases. Always
  called (even []) so removing an alias clears its row; content_hash includes
  non-timestamp frontmatter so alias edits reach this path, not the skip branch.
  Fail-soft + pre-v108-safe (isUndefinedTableError swallowed).
- applyAliasHop (hybrid.ts), AFTER rerank so a named query reliably surfaces
  its page: FULL normalized-query exact match only (no substring/n-grams),
  skip >6-token prose queries, present-boost 1.10x / inject absent canonical at
  top-of-organic + epsilon (never absolute 1.0, D3), collisions alpha-ordered +
  capped at 3, fail-open on pre-v108 / lookup error (D9). Stamps alias_hit for
  the T4 evidence contract.
- SearchResult.alias_hit attribution field.
- 8 tests: inject/boost/CJK/no-match/long-skip/collision + ingest projection
  round-trip + alias-removal-clears. 73 pass across the T1/T2/T3 + import suite.

Backfill of existing pages' aliases lands as T8 (reindex --aliases).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): evidence/create_safety contract + search→cheap-hybrid + per-call mode (T4)

The agent-facing fix for the incident's ROOT behavior: tonight the agent read a
single blended 0.64 score, decided "no strong match, safe to write a new page",
and wrote a duplicate on a developed concept page. A blended RRF/cosine score is
not a calibrated probability, so the don't-duplicate decision must key off WHY a
page matched, not a raw number.

- evidence.ts: classifyEvidence (alias_hit > exact_title_match > high_vector_match
  > keyword_exact > weak_semantic) + createSafetyFor (exists | probable | unknown).
  stampEvidence runs at the end of every hybrid return path (main + both keyword
  fallbacks). SearchResult gains evidence + create_safety. The agent keys
  don't-duplicate off create_safety='exists', not a score threshold.
- search op → cheap-hybrid everywhere (D4/D15): full vector+keyword+RRF+pool+
  title+alias, expansion OFF (no per-call LLM cost); `query` stays full-control.
  search.mcp_keyword_only escape hatch (D17) keeps the old keyword-only behavior
  for operators who don't want query text sent to an embedding provider.
- Alias hop + evidence now also run on the keyword-only fallback paths (the
  named-thing fix is most valuable exactly when vector is unavailable).
- Per-call `mode` (D5): honored ONLY for local/trusted callers (ctx.remote===
  false) so a remote OAuth client can't escalate to costly tokenmax; local +
  unknown mode rejects loudly; threaded into resolveSearchMode + the cache key.
- 30 tests (evidence classifier incl. before/after-incident cases, per-call mode
  gate, alias hop). Updated mcp-eval-capture to the new cheap-hybrid contract.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): reconcile `gbrain search` dispatch (T5)

After T4 made the `search` op cheap-hybrid, `gbrain search "x"` already does the
right thing — but `gbrain search modes/stats/tune` would have run a hybrid search
for the literal word "modes" instead of opening the config dashboard (the op
intercepts before the unreachable handleCliOnly dashboard path).

Add a pre-dispatch interception in main(): `search` + subArgs[0] in
{modes,stats,tune} → runSearch dashboard (with the v0.41.6.0 read-only connect+
dispatch 10s timeout preserved); everything else (free-text) falls through to the
cheap-hybrid `search` op. Subprocess test pins all three routes:
modes/stats → dashboard, free-text → search op ("No results", not "Unknown
subcommand").

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(eval): NamedThingBench retrieval-quality gate (T6)

The eval that makes the retrieval-maxpool incident impossible to reintroduce
silently. 7 query families, each a failure class the incident exposed:
title-substring, generic-to-named, alias-synonym, multi-chunk-dilution,
short-vs-rich, graph-relationship, hard-negative.

- src/eval/retrieval-quality/harness.ts: pure scoring (Hit@1/Hit@3/MRR per
  family) + injected SearchFn (CLI uses hybridSearch; tests stub it) +
  evaluateGate. D12 gate: hard-gate the families that ARE the bug from day one
  (title-substring Hit@1>=0.95, alias-synonym Hit@1>=0.98, dilution Hit@3=1.0),
  warn-then-enforce the softer families. Env-overridable floors.
- `gbrain eval retrieval-quality <fixture.jsonl> [--json] [--source]` +
  dispatch in eval.ts. Exit 0 PASS / 1 FAIL / 2 USAGE.
- Synthetic fixture (placeholder names only, privacy-grep guarded) + hermetic
  gate test: seeds a synthetic brain, forces the keyword+title+alias path
  (embed transport stubbed to throw — free, deterministic), asserts the bug
  families pass. The vector max-pool guarantee is pinned separately by
  searchvector-maxpool.test.ts.
- CI gate: the hermetic test is a normal unit test, so it runs in every PR
  shard — the gate is live on every change.
- 23 tests (harness unit + hermetic gate + fixture privacy guard).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(telemetry): rank-1 score drift signal (T7)

Standing observability so a retrieval regression is caught before a human hits
it in chat (like tonight). Aggregate columns on search_telemetry (NOT per-query
rows, D10): sum_rank1_score + count_rank1 + 3 coarse buckets (<0.6 / 0.6-0.85 /
>=0.85). The mean rank-1 base_score is the headline; a downward drift = retrieval
quality regressing.

- hybrid.ts: capture rank-1 base_score at all three return paths, thread through
  emitMeta → recordSearchTelemetry opts (like results_count).
- telemetry.ts: Bucket + record + flush ON CONFLICT-add + readSearchStats expose
  avg_rank1_score (null when no samples — no NaN) + rank1_distribution.
- Migration v109 ADD COLUMN IF NOT EXISTS (both engines; search_telemetry lives
  only in migration v57, so the v57+v109 chain covers fresh + upgrade). Columns
  exempted in schema-bootstrap-coverage (no forward-ref index → no bootstrap need).
- `gbrain search stats` surfaces the avg + bucket line; JSON envelope auto-carries
  the fields. "true-positive" wording dropped per Codex#14 — production has no
  labels, so this is an unlabeled rank-1 score histogram; labeled calibration
  lives in NamedThingBench (T6).
- 3 round-trip tests (mean+buckets, no-result excluded, empty=null).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(reindex): gbrain reindex --aliases backfill (T8)

Import-time projection (T3) covers new + changed pages; this backfills EXISTING
pages whose frontmatter `aliases:` predate v108 / the projection. Walks
listAllPageRefs (cheap cross-source (source_id, slug) enumeration), reads each
page's frontmatter aliases, writes page_aliases via setPageAliases.

Idempotent (setPageAliases replaces) so re-running is convergent — no op-checkpoint
needed (fast, no embedding). --dry-run reports would-write counts, --source
narrows, --limit caps, --json envelope, progress reporter. Wired into the
`reindex` dispatch alongside --markdown / --multimodal.

4 tests: backfill from array + comma-scalar frontmatter, --dry-run writes
nothing, idempotent second run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(search): pre-migration fail-open regression (T9)

Pins that pre-v108 brains (no page_aliases table) keep working: applyAliasHop
returns input unchanged + doesn't throw, importFromContent with frontmatter
aliases still imports (projection swallows table-missing via isUndefinedTableError),
and resolveAliases surfaces the error for the caller to catch.

Completes the T9 mandatory regression set (dilution → searchvector-maxpool,
dispatch → cli-search-dispatch, MCP contract → mcp-eval-capture, engine parity
→ engine-parity 22/22, pre-migration → here).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(search): Phase-0 retrieval diagnostic — `gbrain search diagnose` (T0)

The operator-facing trace the user runs against the production brain to pin
which retrieval layer surfaces (or misses) a target page — the diagnostic the
plan front-loaded so we don't ship a fix that doesn't move the incident.

`gbrain search diagnose "<query>" --target <slug> [--json] [--source]` reports,
for the target: keyword rank+score, vector rank+score (skipped/graceful if no
embedding provider), whether the query is a registered alias, and the hybrid
final rank + evidence + create_safety + which boosts fired (title/alias). The
verdict names the layer that surfaces the target at rank 1 (or "none"), telling
you whether the lever is max-pool/innerLimit (vector) vs title/alias.

Wired into the `search` dispatch alongside modes/stats/tune (60s timeout since
it runs real retrieval). 2 hermetic tests (alias-query trace + title-phrase
trace). For the Mingtang incident, run:
  gbrain search diagnose "Greek amphitheater" --target projects/new-greek-theater/concept_v0

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(retrieval): corrected incident record + named-thing layers + glossary (T10)

- RETRIEVAL_MAXPOOL_INCIDENT.md: replaces closed PR #1616's RFC with the
  verified record — what happened, the disease, the corrections to the RFC's
  mechanics (search was keyword-only, --mode unthreaded, hybrid already pooled
  at dedup, aliases dead to search), the four-layer fix that shipped, and the
  triage commands (search diagnose / reindex --aliases / search stats / eval
  retrieval-quality).
- RETRIEVAL.md: new "Named-thing retrieval" section documenting per-page pool +
  title boost + alias hop + the evidence contract, reconciling the doc with the
  shipped pipeline (closes the doc/reality gap).
- metric-glossary.ts + regenerated METRIC_GLOSSARY.md: Hit@1, Hit@3,
  avg_rank1_score (drift signal, not labeled accuracy), and create_safety
  (the evidence contract) now carry plain-English glossary entries.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(eval): NamedThingBench fixture privacy guard via slug-shape (T6 fixup)

The banned-name literal list itself tripped check-privacy/check-test-real-names.
Replace it with the load-bearing assertion: every fixture slug must be an
*-example placeholder (no real brain page can be referenced).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(search): source-isolation in per-page pool + alias hop (P0, codex adversarial)

Codex outside-voice caught two source-isolation P0s in the retrieval wave — the
exact class the v0.34.1 seal guards. Both fixed before merge.

P0-1: buildBestPerPagePoolCte pooled on `slug` alone. In a federated brain, two
pages with the same slug in different sources collapsed before ranking/pagination
(the neighbor-source page dropped). Now DISTINCT ON (COALESCE(source_id,'default'),
slug) — composite key matching dedup.ts's pageKey. Also fixes the PRE-EXISTING
keyword-path bug (best_per_page was slug-only before this wave); real-PG parity 23/23.

P0-2: the alias hop dropped source_id. resolveAliases returned bare slugs and
applyAliasHop hydrated via getPage(slug, undefined), so a federated caller could
get the default-source page injected or the right allowed-source page suppressed.
resolveAliases now returns {slug, source_id} pairs; applyAliasHop matches by
(source_id, slug) and fetches each canonical in its OWN source.

Regression tests: alias hop boosts only the aliased source (not same-slug in
another source); resolveAliases keeps cross-source same-slug distinct.

Deferred as documented tradeoffs (TODO): evidence high_vector_match label uses
blended base_score not pure cosine; deep-pagination candidate budget is
chunk-bounded; telemetry writes swallow errors pre-v109 on rolling deploys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: bump version and changelog (v0.41.30.0)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: v0.41.30.0 retrieval cathedral — CLAUDE.md key files + llms regen

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: renumber release v0.41.30.0 → v0.41.34.0 (queue moved)

Version trio + CHANGELOG header + CLAUDE.md key-file annotations + TODOS
heading + regenerated llms bundles, all moved to 0.41.34.0.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): restore glossary roster + harden facts-anti-loop hook budget

Two CI failures surfaced after the master merges that brought the branch to
111 migrations:

1. shard 1 — `ALL_METRICS roster > matches the renderer output (no orphans)`:
   the merge took master's `renderMetricGlossaryMarkdown` whose `groups`
   array lacked this branch's 4 retrieval-quality keys (hit@1, hit@3,
   avg_rank1_score, create_safety). `ALL_METRICS` (derived via Object.keys)
   kept them, so the roster test saw 4 orphans. The freshness check
   (check:eval-glossary) passed because renderer-output == committed doc —
   it can't catch a renderer that drops a metric; the roster test can.
   Restored the "Retrieval-Quality / Evidence Metrics (NamedThingBench)"
   group + regenerated docs/eval/METRIC_GLOSSARY.md.

2. shard 2 — facts-anti-loop's two engine-dependent put_page tests failed
   while the two engine-free extractFactsFromTurn tests passed (the
   signature of a partially-failed beforeAll). This file has a documented
   PGLite-cold-start-under-deep-shard-load timeout history; the 30s budget
   was tuned for 95 migrations and the chain is now 111. Bumped to 60s.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): isolate facts-anti-loop in its own process (serial)

Follow-up to the prior hook-timeout bump, which was the wrong theory: the
[58ms]/[71ms] body times in the re-run prove beforeAll did NOT time out —
the engine connects and the two put_page tests run and fail for real, while
the two engine-free extractFactsFromTurn tests in the same file pass.

put_page (via dispatchToolCall) touches process-global singletons (the
facts queue + the AI gateway used by importFromContent's embed step). Some
sibling file in the 78-file shard-2 process leaves residual global state
that makes put_page's pre-backstop path fail on the CI runner. The failure
is NOT reproducible alone, in a Linux oven/bun:1 container, or in a full
local shard-2 run (1172 pass) — only on the GitHub runner, deterministically.

Per CLAUDE.md's test-isolation rules, a test coupled to shared process
state belongs in its own process. Renamed to *.serial.test.ts so it runs
in the dedicated serial-tests job (scripts/run-serial-tests.sh spawns a
fresh `bun test` per serial file), where it passes deterministically;
test-shard.sh excludes serial files from the matrix. Updated the comment
to reflect the real cause and refreshed the test-weights.json key.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(ci): close cross-file gateway-config pollution in test shards

The prior serial-move theory was incomplete. The real, single root cause
behind all three shard failures (2, 5, 10) is cross-file AI-gateway config
pollution within a shard's bun process:

- A test calls configureGateway() and doesn't restore the gateway on exit.
  The legacy-embedding preload pins OpenAI/1536 ONCE at process start and
  re-pins per-test ONLY when the gateway slot is empty — so a leaker that
  reconfigured the gateway to the v0.37 default (zeroentropyai:zembed-1 /
  1280-d) and never reset poisons every later file in the shard.
- Victim A (shard 5, test/search/searchvector-maxpool.test.ts): runs
  initSchema in beforeAll under the leaked gateway → content_chunks.embedding
  becomes vector(1280) → inserting its hardcoded 1536-d basis vectors throws
  pgvector CheckExpectedDim.
- Victims B/C (shard 10 facts-backstop-gating, shard 2 facts-anti-loop):
  put_page's importFromContent embeds by design (embed failure PROPAGATES,
  Codex C2). Under a leaked fake-key gateway the embed step 401s and put_page
  returns isError → the backstop assertions fail.

My branch's shard re-partition (added test files + weight changes) merely
co-located leakers with victims; the hazard was latent.

Fixes (root cause + self-sufficient victims):
- test/search/rerank.test.ts (the shard-5 leaker): add afterAll(resetGateway).
  Its stub omits embedding_model, so it fell back to the ZE/1280 default;
  now it restores the empty slot so the preload re-pins legacy for the next
  file.
- test/search/searchvector-maxpool.test.ts: pin configureGateway(openai/1536)
  in beforeAll BEFORE initSchema (initSchema runs before any preload
  beforeEach, so it can't rely on the inherited slot).
- test/facts-backstop-gating.test.ts + test/facts-anti-loop.test.ts: reset
  the gateway in beforeEach so put_page's embed is a graceful no-op; reverted
  anti-loop from the serial quarantine back into the matrix (the serial move
  was the wrong fix for a gateway-state problem).

Validated deterministically: a non-resetting leaker that poisons the gateway
to ZE, run first in one bun process, no longer breaks any of the three
victims (14/14 pass). verify 29/29, typecheck clean, isolation lint clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-05-30 12:07:38 -07:00

1991 lines
87 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import type {
Page, PageInput, PageFilters, GetPageOpts,
Chunk, ChunkInput, StaleChunkRow,
SearchResult, SearchOpts,
Link, GraphNode, GraphPath,
TimelineEntry, TimelineInput, TimelineOpts,
RawData,
PageVersion,
BrainStats, BrainHealth,
IngestLogEntry, IngestLogInput,
EngineConfig,
CodeEdgeInput, CodeEdgeResult,
EvalCandidate, EvalCandidateInput,
EvalCaptureFailure, EvalCaptureFailureReason,
SalienceOpts, SalienceResult, AnomaliesOpts, AnomalyResult,
EmotionalWeightInputRow, EmotionalWeightWriteRow,
DomainBankSampleOpts, CorpusSampleOpts, DomainBankRow,
AdjacencyRow,
} from './types.ts';
/**
* v0.27.1: file row for binary-asset metadata. Mirrors the `files` table
* shape on both engines (Postgres has had it since v0.18; PGLite gets it
* via migration v36).
*/
/**
* Options for `traverseGraph`.
*
* `frontierCap`: when set, the BFS recursive term applies a parenthesized
* `LIMIT N ORDER BY slug,id` so each iteration emits at most N rows. This
* is the "approximately per-layer" cap discussed in the T8 plan — Postgres'
* recursive CTE caps per ITERATION, not strictly per BFS LAYER (BFS layer
* boundaries map to recursive iterations only when fan-out is bounded).
* For hub-fanout graphs the cap fires early and bounds the work. Default:
* unset = no cap (back-compat; existing callers see no change).
*
* NOTE: a truncation-detection signal (`onTruncation` callback) was
* designed but the v1 algorithm had both false-positive (organic count ==
* cap) and false-negative (LIMIT-before-DISTINCT in diamond graphs) cases
* caught by adversarial review. The signal is deferred until a
* dedupe-then-cap SQL rewrite + real Postgres parity coverage lands. See
* TODOS.md → "T8 truncation signal" entry. Callers that need to detect
* truncation can compare `result.length` against expected fanout bounds
* as a coarse-but-honest signal in the interim.
*/
/**
* v0.38: bare row shape returned by `BrainEngine.listAllSources()`.
* Kept lean (no per-source page_count) so the autopilot tick stays O(1)
* SQL queries regardless of source count. `sources-ops.SourceListEntry`
* is the enriched application-layer shape.
*/
export interface SourceRow {
id: string;
name: string | null;
local_path: string | null;
last_sync_at: Date | null;
config: Record<string, unknown>;
}
export interface TraverseGraphOpts {
sourceId?: string;
sourceIds?: string[];
frontierCap?: number;
}
export interface FileRow {
id: number;
source_id: string;
page_slug: string | null;
page_id: number | null;
filename: string;
storage_path: string;
mime_type: string | null;
size_bytes: number | null;
content_hash: string;
metadata: Record<string, unknown>;
created_at: Date;
}
/**
* v0.27.1: spec for upsertFile. Identity is (source_id, storage_path).
* Re-upserting the same identity with a different content_hash updates the
* row in place (image was replaced); same content_hash is a no-op.
*/
export interface FileSpec {
source_id?: string;
page_slug?: string | null;
page_id?: number | null;
filename: string;
storage_path: string;
mime_type?: string | null;
size_bytes?: number | null;
content_hash: string;
metadata?: Record<string, unknown>;
}
/**
* v0.41.18.0 — shared opts for engine batch primitives that self-retry on
* transient connection errors. Threaded through addLinksBatch /
* addTimelineEntriesBatch / upsertChunks.
*
* Retry semantics: each batch primitive wraps its internal SQL in
* `withRetry(BULK_RETRY_OPTS)` (default `{maxRetries:3, delayMs:1000,
* delayMaxMs:10000, jitter:'decorrelated'}`). Callers MUST NOT add their own
* `withRetry` wrapper around these methods — that produces 3×3=9 retry
* attempts under failure, amplifying load on a recovering circuit breaker.
* CI lint guard `scripts/check-no-double-retry.sh` enforces the rule.
*
* - `auditSite`: typed label for the JSONL audit emission (`~/.gbrain/audit/
* batch-retry-YYYY-Www.jsonl`). Must be a member of `BATCH_AUDIT_SITES`
* in `src/core/retry.ts`. The CI lint guard `scripts/check-batch-audit-
* site.sh` validates every string-literal value at build time.
* - `signal`: AbortSignal that aborts mid-retry-sleep on SIGTERM/SIGINT.
* `MinionWorker.shutdownAbort.signal` is the canonical source.
*/
import type { BatchAuditSite } from './retry.ts';
export interface BatchOpts {
auditSite?: BatchAuditSite;
signal?: AbortSignal;
}
/** Input row for addLinksBatch. Optional fields default to '' (matches NOT NULL DDL). */
export interface LinkBatchInput {
from_slug: string;
to_slug: string;
link_type?: string;
context?: string;
/**
* Provenance (v0.13+). Pass 'frontmatter' for edges derived from YAML
* frontmatter, 'markdown' for [Name](path) refs, 'manual' for user-created.
* NULL means "legacy / unknown" and is only used by pre-v0.13 rows; new
* writes should always set this. Missing on input defaults to 'markdown'.
*/
link_source?: string;
/** For link_source='frontmatter': slug of the page whose frontmatter created this edge. */
origin_slug?: string;
/** Frontmatter field name (e.g. 'key_people', 'investors'). */
origin_field?: string;
/**
* v0.18.0: source id for each endpoint. When omitted, the engine JOINs
* against `source_id='default'`. Pass explicit values when the edge
* lives in a non-default source OR crosses sources.
*
* Without these fields, the batch JOIN `pages.slug = v.from_slug` fans
* out across every source containing that slug, silently creating wrong
* edges in a multi-source brain. The source_id filter eliminates the
* fan-out. Origin pages (frontmatter provenance) get their own
* source_id so reconciliation can't delete edges from another source's
* frontmatter.
*/
from_source_id?: string;
to_source_id?: string;
origin_source_id?: string;
/**
* v0.41.18.0 (A10, codex finding #12): distinguishes "plain body mention"
* (NULL or 'plain') from "verb-pattern-derived typed NER" ('typed_ner')
* within link_source='mentions'. Backed by v98 schema column. NOT in
* the links UNIQUE constraint — same (from, to, type, source, origin)
* tuple with different link_kind collides DO NOTHING. Default NULL =
* legacy / unknown / pre-v98 semantics.
*/
link_kind?: string;
}
/** Input row for addTimelineEntriesBatch. Optional fields default to '' (matches NOT NULL DDL). */
export interface TimelineBatchInput {
slug: string;
date: string;
source?: string;
summary: string;
detail?: string;
/**
* v0.18.0: source id for the owning page. When omitted, the engine JOINs
* against `source_id='default'`. Without this, two pages sharing the
* same slug across sources would fan out timeline rows to both.
*/
source_id?: string;
}
/**
* A single dedicated database connection, isolated from the engine's pool.
*
* Used by migration paths that need session-level GUCs (e.g.
* `SET statement_timeout = '600000'` before a `CREATE INDEX CONCURRENTLY`)
* without leaking into the shared pool, and by write-quiesce designs
* that need a session-lifetime Postgres advisory lock that survives
* across transaction boundaries.
*
* On Postgres: backed by postgres-js `sql.reserve()`; the same backend
* process serves every `executeRaw` call within the callback. Released
* automatically when the callback returns or throws.
*
* On PGLite: a thin pass-through. PGLite has no pool, so every call is
* already on the single backing connection. The interface is still
* exposed so cross-engine callers don't need to branch.
*
* Not safe to call from inside `transaction()`. The transaction holds a
* different backend; reserving a second one can deadlock on a row the
* transaction itself is waiting to write.
*/
export interface ReservedConnection {
/**
* v0.41.18.0 (A20, codex #7): optional 3rd-arg `opts.signal` lets callers
* actually cancel a running query. Init nudge (3s wallclock cap) wires an
* AbortController whose timer fires at 3s; queries that haven't returned
* by then get cancelled (Postgres: query.cancel(); PGLite: in-process,
* Promise.race against signal-rejection — documented gap because PGLite
* has no kernel-level cancellation).
*/
executeRaw<T = Record<string, unknown>>(
sql: string,
params?: unknown[],
opts?: { signal?: AbortSignal },
): Promise<T[]>;
}
/**
* v0.28: Takes — typed/weighted/attributed claims, indexed in Postgres.
* Markdown is source of truth (fenced table on the page); this row is the
* derived index. Page-scoped via page_id (NOT slug — slug is unique only
* within a source). `(page_id, row_num)` is the natural unique key.
*/
// v0.38: TakeKind opens from closed 4-element union to string (T3/T10).
// Pre-v0.38, kinds {fact|take|bet|hunch} were enforced by DB CHECK
// (migrations v41/v48) AND by this TS closed union. Codex outside-voice
// review caught that dropping the CHECK without also widening the TS
// type "moves inconsistency around" — raw SQL and old clients could
// poison rows that runtime-validate cleanly. v0.38 migration v76 drops
// the CHECK; this widens the type. Runtime validation moves to the
// active schema pack's `takes_kinds:` declaration. The annotation
// primitive's seed list in gbrain-base reproduces {fact|take|bet|hunch}
// so existing behavior is unchanged; packs can extend to {finding|
// hypothesis|observation|...} per domain.
export interface TakeKindLiteral { kind: string }
export type TakeKind = string;
/** Input row for addTakesBatch. */
export interface TakeBatchInput {
page_id: number;
row_num: number;
claim: string;
kind: TakeKind;
holder: string;
weight?: number; // 0..1, default 0.5; clamped server-side
since_date?: string; // ISO date 'YYYY-MM-DD'
until_date?: string;
source?: string;
superseded_by?: number | null;
active?: boolean; // default true
}
/** Take row as returned by listTakes / searchTakes. */
export interface Take {
id: number;
page_id: number;
page_slug: string; // joined from pages
row_num: number;
claim: string;
kind: TakeKind;
holder: string;
weight: number;
since_date: string | null;
until_date: string | null;
source: string | null;
superseded_by: number | null;
active: boolean;
resolved_at: string | null;
resolved_outcome: boolean | null;
/**
* v0.30.0: 3-state outcome label. v0.36.1.1 added 'unresolvable' as a 4th
* state for verdicts where evidence was insufficient to grade. Sits
* alongside `resolved_outcome` for back-compat. New writes populate both;
* legacy v0.28-resolved rows have `resolved_quality` backfilled by
* migration v40 from the boolean. Null on unresolved rows. Schema CHECK
* (widened in v74) enforces (quality, outcome) consistency:
* `correct` ↔ `outcome=true`, `incorrect` ↔ `outcome=false`,
* `partial` ↔ `outcome=NULL`, `unresolvable` ↔ `outcome=NULL`.
*/
resolved_quality: 'correct' | 'incorrect' | 'partial' | 'unresolvable' | null;
resolved_value: number | null;
resolved_unit: string | null;
resolved_source: string | null;
resolved_by: string | null;
created_at: string;
updated_at: string;
}
export interface TakesListOpts {
page_id?: number;
page_slug?: string; // resolved via JOIN
holder?: string;
kind?: TakeKind;
active?: boolean; // default true (only active rows)
resolved?: boolean; // true = only resolved; false = only unresolved; undefined = both
/** Per-token MCP allow-list. Server applies AND holder = ANY($takesHoldersAllowList) when set. */
takesHoldersAllowList?: string[];
sortBy?: 'weight' | 'since_date' | 'created_at';
limit?: number;
offset?: number;
}
/** Search result row from searchTakes / searchTakesVector. */
export interface TakeHit {
take_id: number;
page_id: number;
page_slug: string;
row_num: number;
claim: string;
kind: TakeKind;
holder: string;
weight: number;
score: number; // search rank score (ts_rank for keyword, 1-cos_dist for vector)
}
/** v0.28 stale-takes row (mirrors StaleChunkRow shape). Embedding column intentionally omitted. */
export interface StaleTakeRow {
take_id: number;
page_slug: string;
row_num: number;
claim: string;
}
/** Resolution metadata for resolveTake. */
export interface TakeResolution {
/**
* v0.30.0: primary 3-state input; v0.36.1.1 widened to 4-state with
* 'unresolvable'. When set, takes precedence over `outcome` and the engine
* writes both columns (quality directly; outcome derived:
* `correct→true`, `incorrect→false`, `partial→null`, `unresolvable→null`).
* `unresolvable` marks rows where the judge ran but evidence was
* insufficient to grade; surfaces in `TakesScorecard.unresolvable_count`.
*/
quality?: 'correct' | 'incorrect' | 'partial' | 'unresolvable';
/**
* v0.28 back-compat input. Keep submitting for v0.28 callers; the engine
* derives quality (`true→correct`, `false→incorrect`). When `quality` is
* also set, `quality` wins. When neither is set, the engine throws.
* Mutually-exclusive with `quality === 'partial'` because partial isn't
* binary.
*/
outcome?: boolean;
value?: number;
unit?: string; // 'usd' | 'pct' | 'count' | other
source?: string;
resolvedBy: string; // slug or 'garry'
}
/** v0.30.0: scorecard aggregate. */
export interface TakesScorecard {
total_bets: number;
/**
* Count of resolved rows where `resolved_quality IN
* ('correct','incorrect','partial')`. v0.36.1.1 deliberately keeps this
* 3-state semantic to preserve historical comparisons. Unresolvable rows
* land in the sibling `unresolvable_count` field instead.
*/
resolved: number;
correct: number;
incorrect: number;
partial: number;
/** Accuracy = correct / (correct + incorrect). NULL when n=0. */
accuracy: number | null;
/**
* Brier score over rows where `resolved_quality IN ('correct','incorrect')`.
* Maps `correct→1`, `incorrect→0`, computes `mean((weight outcome)²)`.
* Lower is better; 0 = perfect; 0.25 = always-50% baseline.
* Excludes partial AND unresolvable — both hide signal; the dedicated
* `partial_rate` and `unresolvable_rate` fields surface them separately.
* NULL when no correct+incorrect rows.
*/
brier: number | null;
/** partial / resolved. NULL when n=0. */
partial_rate: number | null;
/**
* v0.36.1.1: count of rows where `resolved_quality = 'unresolvable'`.
* Sibling field to `resolved` so historical comparisons against pre-v80
* scorecards stay valid; `resolved` retains its 3-state meaning, and
* unresolvable rows count here separately. Optional for SDK back-compat —
* downstream consumers constructing TakesScorecard fixtures shouldn't have
* to update on a hotfix. `finalizeScorecard` always populates it.
*/
unresolvable_count?: number;
/**
* v0.37.2.0: `unresolvable_count / (resolved + unresolvable_count)`. NULL
* when both are 0. Surfaces the spec's headline calibration signal:
* "what fraction of grade-attempted takes couldn't be graded?" — high
* values signal weak evidence retrieval rather than wrong predictions.
* Optional for SDK back-compat; see `unresolvable_count` note above.
*/
unresolvable_rate?: number | null;
}
export interface TakesScorecardOpts {
holder?: string;
domainPrefix?: string; // e.g. 'companies/' to scope the scorecard
since?: string; // ISO date 'YYYY-MM-DD'
until?: string; // ISO date 'YYYY-MM-DD'
}
/** v0.30.0: calibration curve bucket. */
export interface CalibrationBucket {
/** Lower bound of the weight bucket, inclusive. */
bucket_lo: number;
/** Upper bound, exclusive (except for the final bucket which is inclusive of 1.0). */
bucket_hi: number;
/** Count of resolved correct+incorrect bets falling in this weight range. */
n: number;
/** correct / n. NULL when n=0. */
observed: number | null;
/** mean(weight) within the bucket — what was predicted on average. NULL when n=0. */
predicted: number | null;
}
export interface CalibrationCurveOpts {
holder?: string;
bucketSize?: number; // default 0.1
}
/** Synthesis evidence row input (provenance from think synthesis pages). */
export interface SynthesisEvidenceInput {
synthesis_page_id: number;
take_page_id: number;
take_row_num: number;
citation_index: number;
}
/** Dream-cycle Haiku verdict on whether a transcript is worth processing. */
export interface DreamVerdict {
worth_processing: boolean;
reasons: string[];
judged_at: string;
}
/** Input shape for putDreamVerdict — judged_at defaults to now() server-side. */
export interface DreamVerdictInput {
worth_processing: boolean;
reasons: string[];
}
// ============================================================
// v0.31 Hot Memory: facts table + recall surface
// ============================================================
/** Allowed `facts.kind` values. Different decay halflives apply per kind. */
export type FactKind = 'event' | 'preference' | 'commitment' | 'belief' | 'fact';
export const ALL_FACT_KINDS: readonly FactKind[] = [
'event', 'preference', 'commitment', 'belief', 'fact',
] as const;
/** Visibility tier on a fact row. Mirrors takes' world-default ACL contract (D21). */
export type FactVisibility = 'private' | 'world';
/** Status returned by insertFact. */
export type FactInsertStatus = 'inserted' | 'duplicate' | 'superseded';
/** A fact row read from the facts table. */
export interface FactRow {
id: number;
source_id: string;
entity_slug: string | null;
fact: string;
kind: FactKind;
visibility: FactVisibility;
/**
* v0.31.2: salience tier the LLM assigned at extraction time. Surfaces
* to consumers (recall response, daily-page writer, admin dashboard,
* agents reading via MCP `_meta.brain_hot_memory`). Pre-v45 brains had
* no notability column; migration v46 backfills with default 'medium'.
*/
notability: 'high' | 'medium' | 'low';
context: string | null;
valid_from: Date;
valid_until: Date | null;
expired_at: Date | null;
superseded_by: number | null;
consolidated_at: Date | null;
consolidated_into: number | null;
source: string;
source_session: string | null;
confidence: number;
embedding: Float32Array | null;
embedded_at: Date | null;
created_at: Date;
}
/** Input for insertFact. source_id supplied via the ctx arg. */
export interface NewFact {
fact: string;
kind?: FactKind; // default 'fact'
entity_slug?: string | null;
visibility?: FactVisibility; // default 'private'
context?: string | null;
valid_from?: Date; // default now()
valid_until?: Date | null;
source: string; // 'mcp:put_page' | 'mcp:extract_facts' | 'cli:think' | etc
source_session?: string | null;
confidence?: number; // [0,1], default 1.0
notability?: 'high' | 'medium' | 'low'; // salience filter for extraction gate
embedding?: Float32Array | null; // pre-computed; if null, insertFact computes via gateway
/**
* v0.35.4 (D-CDX-5) — typed-claim fields. Optional. When populated,
* `gbrain eval trajectory` + `find_trajectory` MCP op consume them for
* chronological regression detection and drift_score. `claim_metric` is
* normalized to lowercase snake_case by the extraction layer before
* this method sees it; the engine stores verbatim.
*/
claim_metric?: string | null;
claim_value?: number | null;
claim_unit?: string | null;
claim_period?: string | null;
/**
* v0.40.2.0 — event-shaped row marker ('meeting', 'job_change',
* 'location_change', etc). Mutually informational with `claim_metric`:
* a row can have either, both, or neither. Persisted into
* `facts.event_type` (migration v89). Existing callers don't need to
* set this — leaving it undefined preserves pre-v0.40 behavior.
*/
event_type?: string | null;
}
/** Options shared by list-facts methods. */
export interface FactListOpts {
/** Hide expired_at IS NOT NULL rows. Default true. */
activeOnly?: boolean;
limit?: number;
offset?: number;
/** Restrict to specific kinds. Default: all kinds. */
kinds?: FactKind[];
/**
* Visibility filter. When undefined, returns all. When set, only matches
* are returned. Remote (untrusted) callers must supply ['world'].
*/
visibility?: FactVisibility[];
}
/** Per-source operational health snapshot consumed by `gbrain doctor`. */
export interface FactsHealth {
source_id: string;
total_active: number; // facts where expired_at IS NULL
total_today: number; // created in last 24h
total_week: number; // created in last 7d
total_expired: number; // expired_at IS NOT NULL
total_consolidated: number; // consolidated_at IS NOT NULL
top_entities: Array<{ entity_slug: string; count: number }>;
/** Optional counters fed by the queue / classifier — populated when those modules report. */
drop_counter?: number;
classifier_fail_counter?: number;
p50_latency_ms?: number;
p99_latency_ms?: number;
}
/**
* v0.35.4 (D-CDX-6) — Options for `BrainEngine.findTrajectory`.
*
* `sourceId` (scalar fast path) and `sourceIds` (federated array) follow
* the v0.34.1.0 search* pattern: when `sourceIds` is set the engine
* applies `WHERE source_id = ANY($N::text[])`; otherwise scalar predicate
* with `sourceId ?? 'default'`.
*
* `remote` (D-CDX-1) gates the visibility filter: when true the engine
* adds `AND visibility = 'world'`, mirroring `recall`'s posture for
* untrusted callers. Local CLI keeps `remote: false` and sees both
* private + world facts.
*/
export interface TrajectoryOpts {
entitySlug: string;
/** Single-source scope; default 'default' when both this and sourceIds are unset. */
sourceId?: string;
/** Federated array scope (mutually exclusive with sourceId; the array wins when set). */
sourceIds?: string[];
/** When true, filters to visibility='world' only. Set by MCP layer from ctx.remote. */
remote?: boolean;
/** Metric filter. When set, only facts with this canonical metric label participate. */
metric?: string;
/**
* v0.40.2.0 — kind filter. Default 'all'. Defensive opt that future-proofs
* the API now that event_type rows live alongside metric rows in the same
* table. Existing callers (founder-scorecard, eval-trajectory) pass
* 'metric' explicitly for clarity (no behavior change since their
* downstream math already skips NULL-metric rows). Richer event-shape
* filtering (job_change vs meeting vs location) is a v0.40.3+ TODO once
* the event schema gets structured fields.
* - 'metric': only rows with claim_metric IS NOT NULL
* - 'event': only rows with event_type IS NOT NULL
* - 'all': both (default)
*/
kind?: 'metric' | 'event' | 'all';
/** Lower bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */
since?: string | Date;
/** Upper bound on valid_from (inclusive). YYYY-MM-DD or full ISO. */
until?: string | Date;
/** Cap on points returned. Default 100, max 500. */
limit?: number;
}
/**
* A single point in an entity's claim trajectory. Carries the typed-claim
* fields when populated (drives regression detection), the underlying
* fact text (for display), provenance (source_session, source_markdown_slug),
* and the raw embedding so the caller can compute drift_score without a
* second SQL round-trip.
*/
export interface TrajectoryPoint {
fact_id: number;
valid_from: Date;
metric: string | null;
value: number | null;
unit: string | null;
period: string | null;
/**
* v0.40.2.0 — event-shaped row marker (e.g. 'meeting', 'job_change',
* 'location_change'). Mutually informational with metric: a row can have
* (a) metric set + event_type null (typed claim like MRR=$50K),
* (b) metric null + event_type set (event like "last met Marco"), or
* (c) both null (legacy free-text fact row from pre-v0.35.4 brains).
* Both founder-scorecard's per-metric math and eval-trajectory's
* regression analysis already skip null-metric rows, so event-only
* rows ride through invisibly to those callers.
*/
event_type: string | null;
text: string;
source_session: string | null;
source_markdown_slug: string | null;
/** Raw embedding for drift computation; null when the fact was inserted without one. */
embedding: Float32Array | null;
}
/** Maximum results returned by search operations. Internal bulk operations (listPages) are not clamped. */
export const MAX_SEARCH_LIMIT = 100;
/** Clamp a user-provided search limit to a safe range. */
export function clampSearchLimit(limit: number | undefined, defaultLimit = 20, cap = MAX_SEARCH_LIMIT): number {
if (limit === undefined || limit === null || !Number.isFinite(limit) || Number.isNaN(limit)) return defaultLimit;
if (limit <= 0) return defaultLimit;
return Math.min(Math.floor(limit), cap);
}
export interface BrainEngine {
/** Discriminator: lets migrations and other consumers branch on engine kind without instanceof + dynamic imports. */
readonly kind: 'postgres' | 'pglite';
// Lifecycle
connect(config: EngineConfig): Promise<void>;
disconnect(): Promise<void>;
initSchema(): Promise<void>;
transaction<T>(fn: (engine: BrainEngine) => Promise<T>): Promise<T>;
/**
* Run `fn` with a dedicated connection (Postgres: reserved backend;
* PGLite: pass-through). See `ReservedConnection` for semantics and
* usage constraints. Release is automatic.
*/
withReservedConnection<T>(fn: (conn: ReservedConnection) => Promise<T>): Promise<T>;
// Pages CRUD
/**
* Fetch a page by slug.
* v0.26.5: by default soft-deleted rows return null (matches the search
* filter contract). Pass `opts.includeDeleted: true` to surface them with
* `deleted_at` populated — used by `gbrain pages purge-deleted` listing,
* by `restore_page` flow, and by operator diagnostics.
*/
getPage(slug: string, opts?: GetPageOpts): Promise<Page | null>;
/**
* Insert or update a page. When `opts.sourceId` is omitted, the row is
* written under the schema DEFAULT ('default'). When provided, `source_id`
* is included in the INSERT column list so ON CONFLICT (source_id, slug)
* DO UPDATE actually targets the intended row instead of fabricating a
* duplicate at (default, slug). Multi-source brains MUST pass sourceId.
*/
putPage(slug: string, page: PageInput, opts?: { sourceId?: string }): Promise<Page>;
/**
* v0.41.13 (#1309) — identity-based dedup pre-check for the import pipeline.
*
* Returns the first matching `{slug, id}` whose `(source_id, …)` matches
* the supplied identity signal, OR null when nothing matches.
*
* Identity precedence (a row matches if EITHER fires):
* - `content_hash = $hash` AND `deleted_at IS NULL`
* - `frontmatter->>'id' = $frontmatterId` AND `$frontmatterId IS NOT NULL`
* AND `deleted_at IS NULL`
*
* Background: the overlapping-ingest-roots bug class (infiniteGameExp,
* issue #1309) created two pages per file when a user ran `gbrain import
* /vault/Subdir/` then `gbrain import /vault/` — the slug-shape changed
* but the content + external ID were identical. Pre-fix, the import
* pipeline dedup-checked by `getPage(slug)` alone and missed the
* cross-slug duplicate. This method gives the importer a deterministic
* way to identify true duplicates BEFORE insert.
*
* Per codex review: the optional `?` shape lets existing test doubles
* compile without changes. Callers must defensively check
* `engine.findDuplicatePage?.(...)` and fall through on undefined.
* `deleted_at IS NULL` is deliberate — a soft-deleted page should NOT
* block a legitimate re-import under a new slug.
*/
findDuplicatePage?(
sourceId: string,
opts: { hash: string; frontmatterId?: string | null },
): Promise<{ slug: string; id: number } | null>;
/**
* Hard-delete a page row. Cascades to content_chunks, page_links,
* chunk_relations via existing FK ON DELETE CASCADE.
*
* v0.26.5: this is no longer the public-facing `delete_page` op handler —
* the op now soft-deletes via `softDeletePage` instead. `deletePage` stays
* as the underlying primitive used by `purgeDeletedPages` and by callers
* that explicitly want hard-delete semantics (e.g. test setup teardown).
*/
/**
* v0.18.0+ multi-source: `opts.sourceId` scopes the DELETE so a source-A
* delete doesn't hard-delete the same-slug pages in sources B/C/D. Without
* it, the bare DELETE matches every row with that slug across all sources.
* Cascades through content_chunks / page_links / chunk_relations via FKs.
*
* v0.41.19.0 (CDX-11): single-row primitive used by `purgeDeletedPages`,
* `gbrain sync` (one path per call), test setup teardown, and the v0.41.19.0
* sync-delete decompose path (when `deletePages` throws on a 500-row batch,
* the sync loop falls back to per-slug `deletePage` to log unrecoverable
* failures to `failedFiles`). `gbrain sync` calls this on EVERY run that
* sees a deleted file — it is NOT admin-only.
*/
deletePage(slug: string, opts?: { sourceId?: string }): Promise<void>;
/**
* v0.41.19.0 — batch delete: single SQL round-trip via
* `DELETE FROM pages WHERE slug = ANY($1::text[]) AND source_id = $2
* RETURNING slug`. Cascades through content_chunks / page_links (×3) /
* tags / raw_data / timeline_entries / page_versions via FKs declared in
* `src/schema.sql`. `files.page_id` and `links.origin_page_id` go SET
* NULL per their FK definitions.
*
* SINGLE-BATCH PRIMITIVE: caller is responsible for chunking the input to
* `<= DELETE_BATCH_SIZE` entries per call (see
* `src/core/engine-constants.ts`). Matches the `addLinksBatch` convention
* — engine assumes well-behaved input, caller owns the slicing.
*
* Returns the slugs of rows ACTUALLY DELETED (order undefined). Callers
* use this to filter their own `pagesAffected` tracking so downstream
* phases don't waste lookups on phantom slugs (paths that were in the
* deletion list but had no DB row).
*
* ATOMICITY: one statement, one transaction. The whole batch commits or
* the whole batch rolls back. Coarser than the per-row `deletePage`
* cadence — a mid-loop abort or transient connection failure can roll
* back up to `DELETE_BATCH_SIZE - 1` successful deletes from the
* in-flight batch. `gbrain sync` is idempotent (next run picks them up
* via git diff); other callers should account for the contract.
*
* sourceId is REQUIRED (no `'default'` fallback). This is asymmetric with
* `deletePage` (which keeps the optional/'default' fallback for back-
* compat). Filed as v0.42+ TODO to tighten `deletePage` to match once a
* full caller audit confirms every site threads `sourceId`.
*/
deletePages(slugs: string[], opts: { sourceId: string }): Promise<string[]>;
/**
* v0.41.19.0 — batch path → slug resolution. Single SQL round-trip via
* `SELECT slug, source_path FROM pages WHERE source_path = ANY($1::text[])
* AND source_id = $2`. Returns `Map<path, slug>`; paths NOT in the map
* have no `source_path` row in the DB and the caller is expected to fall
* back to `resolveSlugForPath(path)` for the path-derived slug.
*
* Mirrors the contract of the single-call `resolveSlugByPathOrSourcePath`
* helper in `src/commands/sync.ts`, batched. As of v0.41.19.0, that
* single-call helper is implemented on top of this method (one Map
* allocation per single-path call; negligible cost; one owner of the SQL
* + fallback semantics).
*
* SINGLE-BATCH PRIMITIVE: caller chunks to `<= DELETE_BATCH_SIZE`.
*
* Empty `paths` short-circuits to an empty Map without touching the DB.
*/
resolveSlugsByPaths(
paths: string[],
opts: { sourceId: string },
): Promise<Map<string, string>>;
/**
* v0.26.5 — set `deleted_at = now()` on a page. Returns the slug if a row
* was soft-deleted, null if no row matched (already soft-deleted OR not found).
* Idempotent-as-null. The page stays in the DB and cascade rows (chunks,
* links) stay intact; the autopilot purge phase hard-deletes after 72h.
*/
softDeletePage(slug: string, opts?: { sourceId?: string }): Promise<{ slug: string } | null>;
/**
* v0.26.5 — clear `deleted_at` on a soft-deleted page. Returns true iff a
* row was restored. False if the slug is unknown OR the page is not
* currently soft-deleted (idempotent-as-false).
*/
restorePage(slug: string, opts?: { sourceId?: string }): Promise<boolean>;
/**
* v0.26.5 — hard-delete pages whose `deleted_at` is older than the cutoff.
* Called by the autopilot purge phase and by the `gbrain pages purge-deleted`
* CLI escape hatch. Cascades through existing FKs.
*/
purgeDeletedPages(olderThanHours: number): Promise<{ slugs: string[]; count: number }>;
/**
* v0.26.5: by default `listPages` excludes soft-deleted rows. Set
* `filters.includeDeleted: true` to surface them.
*/
listPages(filters?: PageFilters): Promise<Page[]>;
/**
* Fuzzy slug resolver.
*
* v0.41.13 (#1436): `opts.sourceId` scopes the search to a single source;
* `opts.sourceIds` to an array (federated_read OAuth tier). Pre-fix the
* resolver was unscoped, so MCP `get_page` with `fuzzy: true` would
* return candidates from sources the caller couldn't actually access.
* Source-bleed via fuzzy resolution was the bug class infiniteGameExp
* reported as #1436. When neither opt is set, the original unscoped
* behavior is preserved for back-compat with internal callers (the
* `gbrain query --resolve` CLI path, etc.). Field names match the
* `sourceScopeOpts(ctx)` helper output so callers can spread directly.
*/
resolveSlugs(partial: string, opts?: { sourceId?: string; sourceIds?: string[] }): Promise<string[]>;
/**
* Returns the slug of every page in the brain. Used by batch commands as a
* mutation-immune iteration source (alternative to listPages OFFSET pagination,
* which is unstable when ordering by updated_at and writes are happening).
*
* v0.31.8 (D12): `opts.sourceId` scopes the result to a single source
* (used by the source-aware reconcileLinks path so wikilink resolution
* doesn't span unrelated sources). When omitted, returns the union of
* slugs across every source (pre-v0.31.8 behavior).
*/
getAllSlugs(opts?: { sourceId?: string }): Promise<Set<string>>;
/**
* v0.32.8: cross-source page enumeration. Returns one row per (slug,
* source_id) pair across the brain, ordered by (source_id, slug) for
* deterministic iteration on large brains. Used by extract-takes,
* extract, and integrity to replace the `getAllSlugs() → getPage(slug)`
* N+1 pattern, which silently defaulted to source_id='default' and
* skipped non-default-source pages.
*
* Cheap by design: only slug + source_id, not the full Page row. For
* loops that need page.compiled_truth / timeline / frontmatter, use
* `forEachPage` from src/core/engine-iter.ts instead.
*/
listAllPageRefs(): Promise<Array<{ slug: string; source_id: string }>>;
/**
* v0.38 — lean per-source enumeration for hot-loop callers (autopilot
* dispatch, doctor freshness check). Returns the bare row shape sources-ops
* needs without the N+1 per-source page_count enrichment in
* `sources-ops.listSources`.
*
* Defaults filter out archived sources. When `localPathOnly` is true,
* also filters `local_path IS NOT NULL` so the autopilot fan-out doesn't
* dispatch jobs for pure-DB sources whose handler would fall back to
* the global sync.repo_path (codex r1 P1-4).
*
* `config` is returned as `Record<string, unknown>` — both engines
* already parse the JSONB at the boundary (Postgres-js returns
* parsed objects; PGLite returns objects via its built-in JSONB
* codec). Callers reading `config['last_full_cycle_at']` get a string.
*/
listAllSources(opts?: {
includeArchived?: boolean;
localPathOnly?: boolean;
}): Promise<SourceRow[]>;
/**
* v0.38 — atomic JSONB merge into sources.config. Uses Postgres's
* `config || $patch::jsonb` operator so concurrent writers don't
* stomp each other (last write wins, but no read-modify-write race).
*
* Primary caller: runCycle's exit hook writes
* { last_full_cycle_at: '<ISO>' }
* after a successful per-source cycle so autopilot's freshness gate
* can read it next tick. Resolves codex round-1 P0-5 (write site for
* last_full_cycle_at was unspecified pre-PR).
*
* Returns true if a row was updated (source exists), false otherwise
* (silently no-ops on unknown sourceId — caller decides whether that's
* a problem).
*/
updateSourceConfig(sourceId: string, patch: Record<string, unknown>): Promise<boolean>;
/**
* v0.37.0 — prefix-stratified page sampling for `gbrain brainstorm` / `gbrain lsd`
* domain-bank module. Takes a caller-supplied prefix list (cached at the domain-bank
* layer per D3), returns one page per prefix tiebroken by `connection_count`
* (LEFT JOIN to page_links, count of inbound links).
*
* Stale-bias (D5 / LSD): when `opts.staleBias === true`, ROW_NUMBER() ORDER BY
* prefers pages with `last_retrieved_at IS NULL` (never retrieved) > pages older
* than `staleThresholdDays` (default 90) > recently-retrieved.
*
* Source scoping (D5, codex r2 #2 fix): `sourceId` (scalar) and `sourceIds`
* (array, wins over scalar) per the [source-id-canonical-thread] pattern.
* Both threaded from day 1 even though v0.37.0 callers are CLI-local — D7
* MCP exposure ships zero-refactor.
*
* Soft-deleted pages (deleted_at IS NOT NULL) excluded automatically.
*/
listPrefixSampledPages(opts: DomainBankSampleOpts): Promise<DomainBankRow[]>;
/**
* v0.37.0 — corpus-sampling fallback for `gbrain brainstorm` when prefix-stratified
* can't fill M (small brain, single-prefix corpus). Random sample of N pages with
* the same exclusion + source-scope semantics as `listPrefixSampledPages`.
* Deterministic with `opts.seed` set; falls back to RANDOM() otherwise.
*
* Returns the same `DomainBankRow` shape so the orchestrator can union both
* sources of pages and dedup by slug+source_id.
*/
listCorpusSample(opts: CorpusSampleOpts): Promise<DomainBankRow[]>;
// Search
searchKeyword(query: string, opts?: SearchOpts): Promise<SearchResult[]>;
searchVector(embedding: Float32Array, opts?: SearchOpts): Promise<SearchResult[]>;
/**
* Hydrate embeddings for chunks already known by id. v0.36 (D9):
* optional `column` parameter selects which content_chunks column to
* fetch from (default 'embedding'). The dynamic-embedding-column
* search path hands its resolved column name here so cosineReScore
* rehydrates in the right embedding space — otherwise vector search
* against `embedding_voyage` would HNSW-rank against Voyage but
* rescore against OpenAI vectors (NaN / wrong rankings).
*
* The column name MUST be regex-validated by the caller (resolveEmbed-
* dingColumn rejects bad names). Engines identifier-quote on
* interpolation as defense in depth (D12).
*/
getEmbeddingsByChunkIds(ids: number[], column?: string): Promise<Map<number, Float32Array>>;
// Chunks
/**
* Replace the chunk set for a page. Internal page-id lookup is sourceId-
* scoped when `opts.sourceId` is given; without it, the schema DEFAULT
* matches and bare-slug lookup blows up if the same slug exists in
* multiple sources (Postgres 21000).
*/
/**
* v0.41.18.0: internal SQL wrapped in `withRetry(BULK_RETRY_OPTS)` against
* transient connection errors (Supavisor circuit-breaker recovery).
* Idempotent under replay via single-statement DELETE+INSERT in implicit tx
* — Postgres rolls back automatically on conn drop, so commit-ambiguous
* failure replays to the same end state. Callers MUST NOT wrap externally;
* see {@link BatchOpts} retry-contract block.
*/
upsertChunks(slug: string, chunks: ChunkInput[], opts?: { sourceId?: string } & BatchOpts): Promise<void>;
/**
* Read every chunk for a page. `opts.sourceId` source-scopes the page
* lookup; without it, multi-source brains return chunks from every
* same-slug source (importCodeFile uses this for incremental embedding
* reuse, which would then attach the wrong source's embeddings).
*/
getChunks(slug: string, opts?: { sourceId?: string }): Promise<Chunk[]>;
/**
* Count chunks across the brain where embedding IS NULL.
* Pre-flight short-circuit for `embed --stale` so a 100%-embedded brain
* does no further work after a single SELECT count(*) (~50 bytes wire).
*
* `opts.sourceId` scopes the count to a single source. When omitted,
* counts across every source in the brain. Operators running
* `gbrain embed --stale --source media-corpus` expect only that
* source's NULLs touched; the caller threads `sourceId` here.
*/
countStaleChunks(opts?: { sourceId?: string; signature?: string }): Promise<number>;
/**
* Sum of LENGTH(chunk_text) over stale chunks — the character-count
* backlog the embed phase / embed-backfill will process. Sibling of
* countStaleChunks (same stale predicate + embed_skip filter + optional
* sourceId scope); used by the `gbrain sync --all` cost preview to price
* the embedding backlog via estimateCostFromChars. Returns 0 on an
* empty/fully-embedded brain.
*
* v0.41.31: `signature` (optional) widens "stale" to ALSO include chunks
* whose page `embedding_signature` is set AND differs from the current
* model signature (a model/dims swap). NULL signature is GRANDFATHERED
* (never counted) so the post-migration corpus isn't flagged en masse.
* Omit `signature` for the legacy `embedding IS NULL`-only count.
*/
sumStaleChunkChars(opts?: { sourceId?: string; signature?: string }): Promise<number>;
/**
* Stamp `pages.embedding_signature = signature` for one page. Called after
* a page's chunks are (re)embedded so a later model swap can detect it as
* stale. Idempotent. No-op if the page doesn't exist.
*/
setPageEmbeddingSignature(slug: string, opts: { sourceId?: string; signature: string }): Promise<void>;
/**
* NULL out the embeddings (and embedded_at) of every chunk whose page
* `embedding_signature` is set AND differs from `signature` — i.e. pages
* embedded under a now-stale model. Returns the chunk count invalidated.
* The embed-stale loop calls this BEFORE listStaleChunks so signature-
* drift pages flow through the existing NULL-embedding cursor (keeps
* listStaleChunks's keyset pagination untouched). GRANDFATHER: NULL
* signature is never invalidated. `sourceId` scopes the sweep.
*/
invalidateStaleSignatureEmbeddings(opts: { signature: string; sourceId?: string }): Promise<number>;
/**
* Return every chunk where embedding IS NULL, with the metadata needed
* to call embedBatch + upsertChunks. The `embedding` column is omitted
* by design — stale rows have NULL embeddings, so shipping them wastes
* wire bytes for no gain. Caller groups by slug, embeds, and re-upserts.
*
* v0.33.3: cursor-paginated — yields up to `batchSize` rows per call
* (default 2000) to stay within Supabase's statement_timeout. Pass the
* last row's `(page_id, chunk_index)` as `afterPageId`/`afterChunkIndex`
* to fetch the next page. When fewer than `batchSize` rows come back,
* the caller has reached the end.
*
* `opts.sourceId` scopes the scan to a single source (matches the
* countStaleChunks contract). Paired with embedAllStale's --source
* support.
*/
listStaleChunks(opts?: {
batchSize?: number;
afterPageId?: number;
afterChunkIndex?: number;
sourceId?: string;
// v0.41.18.0 (A13, codex #9): pagination order. Default 'page_id'
// (legacy stable cursor). 'updated_desc' joins pages and orders by
// p.updated_at DESC NULLS LAST, p.id, cc.chunk_index — backed by
// idx_pages_updated_at_desc + content_chunks_stale_idx partial.
orderBy?: 'page_id' | 'updated_desc';
// For 'updated_desc' cursor: previous row's updated_at, page_id, chunk_index.
// ISO-8601 string for cross-engine compatibility (postgres.js + PGLite
// both round-trip TIMESTAMPTZ as Date | string; ISO string is the
// common denominator on the wire).
afterUpdatedAt?: string | null;
}): Promise<StaleChunkRow[]>;
/**
* Delete every chunk for a page. Internal page-id lookup is sourceId-scoped
* when `opts.sourceId` is given; otherwise the bare-slug subquery returns
* the wrong row count in multi-source brains.
*/
deleteChunks(slug: string, opts?: { sourceId?: string }): Promise<void>;
// Links
/**
* Single-row link insert. linkSource defaults to 'markdown' for back-compat
* with pre-v0.13 callers. Pass 'frontmatter' + originSlug + originField for
* frontmatter-derived edges; 'manual' for user-initiated edges.
*/
/**
* v0.18.0+ multi-source: each endpoint can live in a different source.
* `opts.fromSourceId` / `opts.toSourceId` / `opts.originSourceId` default to
* 'default'. Without these, the original cross-product `FROM pages f, pages t`
* fanned out across every source containing the slug.
*/
addLink(
from: string,
to: string,
context?: string,
linkType?: string,
linkSource?: string,
originSlug?: string,
originField?: string,
opts?: { fromSourceId?: string; toSourceId?: string; originSourceId?: string },
): Promise<void>;
/**
* Bulk insert links via a single multi-row INSERT...SELECT FROM (VALUES) JOIN pages
* statement with ON CONFLICT DO NOTHING. Returns the count of rows actually inserted
* (RETURNING clause excludes conflicts and JOIN-dropped rows whose slugs don't exist).
* Used by extract.ts to avoid 47K sequential round-trips on large brains.
*/
/**
* v0.41.18.0: internal SQL wrapped in `withRetry(BULK_RETRY_OPTS)`.
* Idempotent via `ON CONFLICT (from_page_id, to_page_id, link_type,
* link_source, origin_page_id) DO NOTHING` — composite key is the semantic
* uniqueness. Replay-after-partial-success: 2nd attempt finds conflicts,
* returns 0 from RETURNING. Caller-visible edge: linksCreated undercounts
* on commit-ambiguous replay. Cosmetic (audit JSONL captures the truth).
* Callers MUST NOT wrap externally; see {@link BatchOpts} retry contract.
*/
addLinksBatch(links: LinkBatchInput[], opts?: BatchOpts): Promise<number>;
/**
* Remove links from `from` to `to`. If linkType is provided, only that specific
* (from, to, type) row is removed. If omitted, ALL link types between the pair
* are removed (matches pre-multi-type-link behavior). linkSource additionally
* constrains the delete to a specific provenance ('frontmatter', 'markdown',
* 'manual') — used by runAutoLink reconciliation to avoid deleting edges from
* other provenances when pruning frontmatter-derived edges.
*/
removeLink(
from: string,
to: string,
linkType?: string,
linkSource?: string,
opts?: { fromSourceId?: string; toSourceId?: string },
): Promise<void>;
/**
* v0.31.8 (D12 + D16): `opts.sourceId` source-scopes the from-page lookup.
* When omitted, the read returns links from every same-slug page across
* sources (pre-v0.31.8 behavior; preserved via two-branch query in both
* engines). When set, the from-page filter becomes
* `WHERE f.slug = $1 AND f.source_id = $X`.
*/
getLinks(slug: string, opts?: { sourceId?: string }): Promise<Link[]>;
/**
* v0.31.8 (D12 + D16): same `opts.sourceId` semantics as `getLinks`,
* applied to the to-page side of the join.
*/
getBacklinks(slug: string, opts?: { sourceId?: string }): Promise<Link[]>;
/**
* Fuzzy-match a display name to a page slug using pg_trgm similarity.
* Zero embedding cost, zero LLM cost — designed for the v0.13 resolver used
* during migration/batch backfill where 5K+ lookups must stay sub-second.
*
* Returns the best match whose title similarity is at or above `minSimilarity`
* (default 0.55). If `dirPrefix` is given (e.g. 'people' or 'companies'),
* only slugs starting with that prefix are considered. Returns null when no
* page meets the threshold.
*
* Uses the `%` trigram operator (GIN-indexed) + the standard `similarity()`
* function. Both engines support pg_trgm (PGLite 0.3+, Postgres always).
*/
findByTitleFuzzy(
name: string,
dirPrefix?: string,
minSimilarity?: number,
): Promise<{ slug: string; similarity: number } | null>;
/**
* v0.34.1 (#861 — P0 leak seal): `opts.sourceId` / `opts.sourceIds`
* constrain visited nodes to a single source or array of sources.
* Pre-fix, the walk ignored source scope and an authenticated MCP
* client could enumerate cross-source topology + page metadata via
* the graph op. MCP-bound callers MUST pass the auth'd scope; local
* CLI callers omit it for the historical unscoped behavior.
*/
traverseGraph(
slug: string,
depth?: number,
opts?: TraverseGraphOpts,
): Promise<GraphNode[]>;
/**
* Edge-based graph traversal with optional type and direction filters.
* Returns a list of edges (GraphPath[]) instead of nodes. Supports:
* - linkType: per-edge filter, only follows matching edges (per-edge semantics)
* - direction: 'in' (follow to->from), 'out' (follow from->to), 'both'
* - depth: max depth from root (default 5)
* - sourceId/sourceIds: v0.34.1 source-isolation filter, see traverseGraph
* Uses cycle prevention (visited array in recursive CTE).
*/
traversePaths(
slug: string,
opts?: { depth?: number; linkType?: string; direction?: 'in' | 'out' | 'both'; sourceId?: string; sourceIds?: string[] },
): Promise<GraphPath[]>;
/**
* For a list of slugs, return how many inbound links each has.
* Used by hybrid search backlink boost. Single SQL query, not N+1.
* Slugs with zero inbound links are present in the map with value 0.
*/
getBacklinkCounts(slugs: string[]): Promise<Map<string, number>>;
/**
* v0.40.4 — for a list of page_ids, return adjacency aggregates
* restricted to the subgraph induced by them. Returns ALL pages with
* `hits >= 1` (callers apply their own threshold). Empty input → empty
* Map, no SQL.
*
* Returned shape per page (AdjacencyRow):
* - `hits`: distinct from_page_id count, in-set
* - `cross_source_hits`: distinct OTHER source_ids count (excluding
* target's own source), in-set
*
* SOURCE-SCOPE CONTRACT: pageIds MUST already be source-scoped by the
* caller. This method does NOT filter by source_id. Adjacency is
* page-id keyed and the in-set restriction makes cross-source leakage
* impossible BY CONSTRUCTION (a leaked-in page_id from another source
* would have to also appear in the caller's input set, which the
* caller is responsible for preventing). The only consumer in v0.40.4
* is hybridSearch via runPostFusionStages, which is source-scoped
* upstream. Same trust posture as `cosineReScore`'s chunk_id handling.
*
* Known limitation: cross_source_hits doesn't distinguish "genuinely
* linked from another team" from "mirrored imports from another
* source" (codex outside-voice #15). T-todo-4 captures the v0.41+
* sync-topology-aware refinement.
*/
getAdjacencyBoosts(pageIds: number[]): Promise<Map<number, AdjacencyRow>>;
/**
* v0.27.0: for a list of slugs, return their updated_at timestamps (or created_at fallback).
* Used by hybrid search recency boost. Single SQL query, not N+1.
* Slugs with no timestamp get no entry in the map.
*
* @deprecated v0.29.1: prefer getEffectiveDates (composite-keyed, multi-source-safe).
* Kept for back-compat with PR #618 callers.
*/
getPageTimestamps(slugs: string[]): Promise<Map<string, Date>>;
/**
* v0.29.1: for a list of (slug, source_id) refs, return COALESCE(effective_date,
* updated_at) per ref. Single SQL query. Composite-keyed map (key format:
* `${source_id}::${slug}`) so multi-source brains don't conflate pages with
* the same slug across sources (codex pass-1 finding #3).
*
* Drives the new applyRecencyBoost post-fusion stage. Returns NULL for refs
* with no row; map omits them.
*/
getEffectiveDates(refs: Array<{slug: string; source_id: string}>): Promise<Map<string, Date>>;
/**
* v0.29.1: for a list of (slug, source_id) refs, return the salience score
* (emotional_weight × 5 + ln(1 + take_count)) per ref. Single SQL query.
* Composite-keyed (`${source_id}::${slug}`) like getEffectiveDates.
*
* Drives the new applySalienceBoost post-fusion stage. Pages with no row
* (or zero emotional_weight + zero takes) get score = 0; the boost stage
* skips them.
*/
getSalienceScores(refs: Array<{slug: string; source_id: string}>): Promise<Map<string, number>>;
/**
* Return every page with no inbound links.
* Domain comes from the frontmatter `domain` field (null if unset).
* The caller filters pseudo-pages + derives display domain.
* Used by `gbrain orphans` and `runCycle`'s orphan sweep phase.
*
* v0.41.29.0: scopes the CANDIDATE set to one source (`sourceId`, from
* `gbrain doctor/orphans --source` + single-source MCP clients) or a
* federated set (`sourceIds`, from `allowedSources` MCP clients). The
* inbound-link side is NOT scoped — a page in source X linked FROM
* source Y is genuinely reachable, so it is not an orphan of X. Omit
* `opts` for the brain-wide behavior (unchanged). When both are set,
* `sourceIds` wins (mirrors `sourceScopeOpts` precedence).
*/
findOrphanPages(opts?: {
sourceId?: string;
sourceIds?: string[];
}): Promise<Array<{ slug: string; title: string; domain: string | null }>>;
// Tags
/**
* v0.18.0+ multi-source: `opts.sourceId` scopes the page-id lookup. When
* omitted, the schema DEFAULT 'default' applies; in multi-source brains
* with the same slug across sources the bare-slug lookup returns >1 row
* and the INSERT/DELETE fails with Postgres 21000.
*/
addTag(slug: string, tag: string, opts?: { sourceId?: string }): Promise<void>;
removeTag(slug: string, tag: string, opts?: { sourceId?: string }): Promise<void>;
getTags(slug: string, opts?: { sourceId?: string }): Promise<string[]>;
// Timeline
/**
* Insert a timeline entry. By default verifies the page exists and throws if not.
* Pass opts.skipExistenceCheck=true for batch operations where the slug is already
* known to exist (e.g., from a getAllSlugs() snapshot). Duplicates are silently
* deduplicated by the (page_id, date, summary) UNIQUE index (ON CONFLICT DO NOTHING).
*/
/**
* Insert a timeline entry. By default verifies the page exists and throws if not.
* `opts.skipExistenceCheck` skips the pre-check for batch loops where the slug
* is already known to exist. `opts.sourceId` source-scopes both the existence
* check AND the page-id lookup inside the INSERT — required for multi-source
* brains where the slug exists in 2+ sources.
*/
addTimelineEntry(
slug: string,
entry: TimelineInput,
opts?: { skipExistenceCheck?: boolean; sourceId?: string },
): Promise<void>;
/**
* Bulk insert timeline entries via a single multi-row INSERT...SELECT FROM (VALUES)
* JOIN pages statement with ON CONFLICT DO NOTHING. Returns the count of rows
* actually inserted (RETURNING excludes conflicts and JOIN-dropped rows whose
* slugs don't exist). Used by extract.ts to avoid sequential round-trips.
*/
/**
* v0.41.18.0: internal SQL wrapped in `withRetry(BULK_RETRY_OPTS)`.
* Idempotent via composite-key conflict (page_id, kind, when_text, body).
* Same caller-visible undercount caveat as {@link addLinksBatch}.
* Callers MUST NOT wrap externally; see {@link BatchOpts} retry contract.
*/
addTimelineEntriesBatch(entries: TimelineBatchInput[], opts?: BatchOpts): Promise<number>;
getTimeline(slug: string, opts?: TimelineOpts): Promise<TimelineEntry[]>;
// Raw data
/**
* v0.31.8 (D21): `opts.sourceId` source-scopes the page-id lookup. When
* omitted, the write targets the bare slug (pre-v0.31.8 behavior); the
* Postgres 21000 hazard for multi-source brains exists on this path.
* Multi-source callers MUST pass sourceId to land on the intended row.
*/
putRawData(slug: string, source: string, data: object, opts?: { sourceId?: string }): Promise<void>;
/**
* v0.31.8 (D21): `opts.sourceId` source-scopes the page-id lookup. Without
* it, multi-source brains return raw_data rows from every same-slug page
* (preserved via two-branch query for back-compat).
*/
getRawData(slug: string, source?: string, opts?: { sourceId?: string }): Promise<RawData[]>;
// Files (v0.27.1: binary asset metadata + storage_path. Image bytes never
// enter the DB; storage_path references a path inside the brain repo or an
// external store).
upsertFile(spec: FileSpec): Promise<{ id: number; created: boolean }>;
getFile(sourceId: string, storagePath: string): Promise<FileRow | null>;
listFilesForPage(pageId: number): Promise<FileRow[]>;
// ============================================================
// v0.28: Takes (typed/weighted/attributed claims) + synthesis evidence
// ============================================================
/**
* Bulk insert/upsert takes. Uses `unnest()` (Postgres) or manual `$N`
* placeholders (PGLite). Idempotency: ON CONFLICT (page_id, row_num) DO UPDATE
* — re-extract on a changed claim/weight updates the row in place.
* Returns the number of rows inserted OR updated.
*
* Weight outside [0, 1] is clamped server-side and surfaces a stderr
* warning per call (`TAKES_WEIGHT_CLAMPED`). Invalid `kind` values
* fail the whole batch via the CHECK constraint — caller is responsible
* for parser validation upstream.
*/
addTakesBatch(rows: TakeBatchInput[]): Promise<number>;
/** List takes filtered by holder/kind/active/etc. Resolves page_slug via JOIN. */
listTakes(opts?: TakesListOpts): Promise<Take[]>;
/**
* Keyword search across active takes. Uses pg_trgm similarity over claim text.
* Honors `takesHoldersAllowList` via WHERE filter so MCP-bound calls cannot
* retrieve holders outside the token's allow-list.
*/
searchTakes(query: string, opts?: SearchOpts & { takesHoldersAllowList?: string[] }): Promise<TakeHit[]>;
/**
* Vector search across active takes. Cosine distance against `embedding`.
* Skipped (returns []) when no embedding column has been populated yet.
*/
searchTakesVector(
embedding: Float32Array,
opts?: SearchOpts & { takesHoldersAllowList?: string[] },
): Promise<TakeHit[]>;
/** Look up embeddings by take id (mirrors getEmbeddingsByChunkIds). */
getTakeEmbeddings(ids: number[]): Promise<Map<number, Float32Array>>;
/** Pre-flight count for `gbrain embed --stale`. WHERE active AND embedding IS NULL. */
countStaleTakes(): Promise<number>;
/** List stale takes (no embedding column in payload — same pattern as listStaleChunks). */
listStaleTakes(): Promise<StaleTakeRow[]>;
/**
* Update a take's mutable fields. May NOT change claim/kind/holder per the
* supersession invariants — those route through supersedeTake. Throws
* `TAKE_ROW_NOT_FOUND` when (page_id, row_num) doesn't exist.
*/
updateTake(
pageId: number,
rowNum: number,
fields: { weight?: number; since_date?: string; source?: string },
): Promise<void>;
/**
* Supersede the take at (page_id, oldRow). Marks old row active=false +
* sets superseded_by; appends new row at the next row_num for the page;
* returns both row_nums. Atomic (transactional). Cycle prevention: if newRow
* sets superseded_by pointing to a chain that comes back to oldRow, throws
* `TAKES_SUPERSEDE_CYCLE`. Resolved bets (`resolved_at IS NOT NULL`) cannot
* be superseded — throws `TAKE_RESOLVED_IMMUTABLE`.
*/
supersedeTake(
pageId: number,
oldRow: number,
newRow: Omit<TakeBatchInput, 'page_id' | 'row_num' | 'superseded_by'>,
): Promise<{ oldRow: number; newRow: number }>;
/**
* Resolve a bet (or take). Sets resolved_* columns. Immutable: re-resolve
* attempts throw `TAKE_ALREADY_RESOLVED`. Use supersede to express a new bet.
*
* v0.30.0: accepts either `quality` (3-state, primary) or `outcome` (boolean,
* back-compat). When both set, `quality` wins. The engine writes BOTH columns
* derived from whichever input was given: `quality='correct'/'incorrect'` →
* `outcome=true/false`; `quality='partial'` → `outcome=NULL`. The schema
* `takes_resolution_consistency` CHECK constraint catches contradictory
* states at the DB layer as a defense-in-depth backstop.
*/
resolveTake(pageId: number, rowNum: number, resolution: TakeResolution): Promise<void>;
/**
* v0.30.0: aggregate calibration scorecard. Pure SQL aggregation; no LLM.
* Counts resolved bets, computes accuracy, Brier score (correct+incorrect
* only), and `partial_rate`. Filtering: `holder` scopes to one identity;
* `domainPrefix` scopes to a slug-prefix (e.g. `companies/`); `since`/`until`
* scope to a `since_date` window.
*
* Privacy (D4 from plan): `allowList` is REQUIRED in the TS signature.
* The engine applies `WHERE holder = ANY($allowList)` INSIDE the GROUP BY
* so hidden-holder rows contribute zero to aggregates. Pass an empty array
* to enforce zero-results; pass `undefined` only from server-side trusted
* callers that have already verified the request is unrestricted.
*/
getScorecard(opts: TakesScorecardOpts, allowList: string[] | undefined): Promise<TakesScorecard>;
/**
* v0.30.0: calibration curve. Bins resolved correct+incorrect bets by stated
* weight (default bucket size 0.1) and reports observed vs predicted frequency
* per bucket. Same allow-list contract as `getScorecard`. Excludes partial
* (consistent with Brier — partial has no binary outcome to compare against).
*/
getCalibrationCurve(opts: CalibrationCurveOpts, allowList: string[] | undefined): Promise<CalibrationBucket[]>;
/** Persist think provenance. ON CONFLICT DO NOTHING; returns rows inserted. */
addSynthesisEvidence(rows: SynthesisEvidenceInput[]): Promise<number>;
// Dream-cycle significance verdict cache (v0.23).
// Keyed by (file_path, content_hash). Distinct from raw_data, which is
// page-scoped — transcripts being judged aren't pages yet.
getDreamVerdict(filePath: string, contentHash: string): Promise<DreamVerdict | null>;
putDreamVerdict(filePath: string, contentHash: string, verdict: DreamVerdictInput): Promise<void>;
// ============================================================
// v0.32.6 Contradiction probe — batched takes fetch + cache + trends
// ============================================================
/**
* Batch fetch: for each page_id in the input array, return the page's
* currently-active takes. Single query under the hood (`WHERE page_id =
* ANY($1) AND active = true`); replaces the O(K) loop of listTakes calls
* the contradiction probe would otherwise pay per probe-query.
*
* Returns a Map keyed on page_id; pages with no active takes get an empty
* array (NOT undefined) so callers can avoid existence checks.
*
* Honors `takesHoldersAllowList` for MCP scope enforcement (mirrors
* listTakes contract). Pass undefined from trusted local callers.
*/
listActiveTakesForPages(
pageIds: number[],
opts?: { takesHoldersAllowList?: string[] },
): Promise<Map<number, Take[]>>;
/**
* Persist a single contradiction-probe run row. Caller supplies a full
* `ContradictionsRunRow`-shaped object; the engine inserts as-is.
*
* Idempotent on `run_id`: re-inserting an existing run_id is a no-op
* (caller passes ISO-timestamp-shaped run_ids that won't collide
* unintentionally). Returns true iff a row was inserted.
*/
writeContradictionsRun(row: {
run_id: string;
judge_model: string;
prompt_version: string;
queries_evaluated: number;
queries_with_contradiction: number;
total_contradictions_flagged: number;
wilson_ci_lower: number;
wilson_ci_upper: number;
judge_errors_total: number;
cost_usd_total: number;
duration_ms: number;
source_tier_breakdown: Record<string, unknown>;
report_json: Record<string, unknown>;
}): Promise<boolean>;
/**
* Load contradiction-probe run history within the last N days, ordered
* newest first. Used by `gbrain eval suspected-contradictions trend` and
* by the doctor `contradictions` check. `report_json` and
* `source_tier_breakdown` are parsed JSONB columns.
*/
loadContradictionsTrend(days: number): Promise<Array<{
run_id: string;
ran_at: string;
judge_model: string;
queries_evaluated: number;
queries_with_contradiction: number;
total_contradictions_flagged: number;
wilson_ci_lower: number;
wilson_ci_upper: number;
judge_errors_total: number;
cost_usd_total: number;
duration_ms: number;
source_tier_breakdown: Record<string, unknown>;
report_json: Record<string, unknown>;
}>>;
/**
* Cache lookup for the contradiction probe's persistent judge cache (P2).
* Returns the verdict JSON if a row exists with matching key AND non-expired
* `expires_at`. NULL means cache miss (judge call needed).
*
* Key shape mirrors the table primary key: (chunk_a_hash, chunk_b_hash,
* model_id, prompt_version, truncation_policy). Codex's outside-voice
* critique fixed the key to include prompt_version + truncation_policy so
* prompt edits cleanly invalidate prior verdicts.
*/
getContradictionCacheEntry(key: {
chunk_a_hash: string;
chunk_b_hash: string;
model_id: string;
prompt_version: string;
truncation_policy: string;
}): Promise<Record<string, unknown> | null>;
/**
* Upsert a contradiction-probe judge verdict into the persistent cache.
* `ttl_seconds` controls expires_at (default 30 days from now). Caller
* supplies pre-hashed chunk text + the verdict to cache.
*
* ON CONFLICT DO UPDATE so re-runs refresh expires_at; this is the simplest
* shape for "I judged the same pair again with the same config, slide the
* TTL forward."
*/
putContradictionCacheEntry(opts: {
chunk_a_hash: string;
chunk_b_hash: string;
model_id: string;
prompt_version: string;
truncation_policy: string;
verdict: Record<string, unknown>;
ttl_seconds?: number;
}): Promise<void>;
/**
* Sweep expired cache entries. Returns count deleted. Periodic call from
* cache.ts — keeps the table bounded without requiring a cron.
*/
sweepContradictionCache(): Promise<number>;
// ============================================================
// v0.31 Hot memory — facts table operations
// ============================================================
/**
* Insert a fact into the per-source hot memory. The handler:
* 1. canonicalizes entity_slug against pages (caller may pre-canonicalize)
* 2. queries findCandidateDuplicates (entity-prefiltered, k=5 cap)
* 3. cosine ≥0.95 fast-path → mark duplicate, skip classifier
* 4. else classifier (caller's job; this engine method handles the
* DB-side INSERT/UPDATE only). On insert.status === 'duplicate' or
* 'superseded' the engine returns the existing/superseding row id.
* Per-entity advisory lock on Postgres serializes the dedup window.
* PGLite no-op for the lock (single-process).
*
* `status` reflects what the engine wrote:
* 'inserted' → row inserted
* 'duplicate' → no new row (returns the matching candidate id)
* 'superseded' → new row inserted; old row got expired_at + superseded_by
*/
insertFact(
input: NewFact,
ctx: { source_id: string; supersedeId?: number },
): Promise<{ id: number; status: FactInsertStatus }>;
/**
* v0.32.2: batch insert for fence-extracted fact rows. Persists the
* v51 fence columns (`row_num`, `source_markdown_slug`) alongside the
* standard NewFact fields.
*
* Designed for the `extract_facts` cycle phase: wipe-then-batch-insert
* per page. No dedup is performed here — callers (the cycle phase via
* `deleteFactsForPage` + this) own that contract. Bypasses the
* single-row supersede flow because fence reconciliation is the canonical
* source-of-truth direction, not the consolidator path.
*
* Insertion is atomic per call: all rows commit in a single transaction
* or none commit (the transaction rolls back on any constraint
* violation, e.g. the v51 partial UNIQUE index on
* `(source_id, source_markdown_slug, row_num)`).
*
* Returns the inserted ids in input-order so callers can correlate
* fence-row → DB-id without a separate lookup.
*/
insertFacts(
rows: Array<NewFact & { row_num: number; source_markdown_slug: string }>,
ctx: { source_id: string },
): Promise<{ inserted: number; ids: number[] }>;
/**
* v0.32.2: hard-delete every fact row scoped to a single fence page.
*
* Keyed on `(source_id, source_markdown_slug)`. Used by the
* `extract_facts` cycle phase before re-inserting from the fence — the
* fence is canonical, the DB is the derived index, so each phase run
* wipes the page-scoped index and rebuilds it from the markdown.
*
* Hard DELETE (not soft-delete via `expired_at`). A fence row that
* disappears from markdown corresponds to a fact the user removed
* entirely from history; the DB mirrors that. Forgotten facts that
* stay in the fence as strikethrough rows survive the wipe because
* the re-insert puts them back with `valid_until = today` per the
* `extract-from-fence` derivation contract.
*
* Pre-v51 rows (NULL `source_markdown_slug`) are NEVER deleted by this
* call — the partial UNIQUE index on `row_num IS NOT NULL` is the
* structural guarantee that legacy rows live in a different keyspace
* until the v0_32_2 migration backfills them. Cycle-phase callers in
* commit 7 add the empty-fence-guard as a belt-and-suspenders check.
*/
deleteFactsForPage(slug: string, source_id: string): Promise<{ deleted: number }>;
/**
* Mark a fact expired. Never DELETE. Returns true iff a row was updated.
* Idempotent-as-false (already expired returns false without changing state).
*/
expireFact(id: number, opts?: { supersededBy?: number; at?: Date }): Promise<boolean>;
/** List active facts about an entity within a source, newest first. */
listFactsByEntity(
source_id: string,
entitySlug: string,
opts?: FactListOpts,
): Promise<FactRow[]>;
/** List facts created since a given timestamp within a source. */
listFactsSince(
source_id: string,
since: Date,
opts?: FactListOpts & { entitySlug?: string },
): Promise<FactRow[]>;
/** List facts captured under a session id within a source. */
listFactsBySession(
source_id: string,
sessionId: string,
opts?: FactListOpts,
): Promise<FactRow[]>;
/**
* Audit log: facts that were superseded (expired_at + superseded_by both set),
* newest first. Drives `gbrain recall --supersessions`.
*/
listSupersessions(
source_id: string,
opts?: { since?: Date; limit?: number },
): Promise<FactRow[]>;
/**
* v0.32: count facts that haven't been promoted to takes by the consolidate
* phase yet (active + unconsolidated). Drives `gbrain recall --pending`.
* Single SQL: COUNT(*) WHERE consolidated_at IS NULL AND expired_at IS NULL.
*/
countUnconsolidatedFacts(source_id: string): Promise<number>;
/**
* Find candidate duplicates for a new fact within a source+entity bucket.
* Entity-prefilter is mandatory (bounds the contradiction-classifier blast
* radius). Hard cap k=5 by default. Embedding-cosine when both sides have
* embeddings; recency fallback otherwise.
*/
findCandidateDuplicates(
source_id: string,
entitySlug: string,
factText: string,
opts?: { k?: number; embedding?: Float32Array },
): Promise<FactRow[]>;
/**
* Mark a fact as consolidated into a take. Sets consolidated_at + consolidated_into.
* Never DELETE — facts stay as audit trail.
*/
consolidateFact(id: number, takeId: number): Promise<void>;
/**
* v0.35.4 (D-CDX-1 + D-CDX-6) — chronological fact trajectory for an
* entity. Returns points ordered by (valid_from ASC, id ASC) so the
* caller can compute regressions and drift_score deterministically.
*
* - Source-scoped via `sourceId` (scalar) OR `sourceIds` (federated array).
* - Visibility-filtered: when `opts.remote=true`, only `visibility='world'`
* facts are returned. Trusted local callers see both private + world.
* - Optional metric filter restricts to a single normalized metric label.
* - Active-only by default (expired_at IS NULL); soft-deleted entities
* on the pages side are NOT filtered here — trajectory is a facts-table
* query and doesn't JOIN pages.
*/
findTrajectory(opts: TrajectoryOpts): Promise<TrajectoryPoint[]>;
/** Per-source operational metrics for `gbrain doctor` facts_health check. */
getFactsHealth(source_id: string): Promise<FactsHealth>;
// Versions
/**
* Snapshot a page row into page_versions. Source-scoped via `opts.sourceId`;
* without it the bare-slug lookup snapshots whichever row Postgres returns
* first when the slug exists across multiple sources.
*/
createVersion(slug: string, opts?: { sourceId?: string }): Promise<PageVersion>;
/**
* v0.31.8 (D12 + D16): `opts.sourceId` source-scopes the page-id lookup.
* When omitted, returns versions for every same-slug page across sources
* (pre-v0.31.8 behavior; preserved via two-branch query).
*/
getVersions(slug: string, opts?: { sourceId?: string }): Promise<PageVersion[]>;
/**
* v0.31.8 (D12): `opts.sourceId` source-scopes both the version lookup
* and the page revert. Without it, multi-source brains can revert the
* wrong row when the slug exists in 2+ sources.
*/
revertToVersion(slug: string, versionId: number, opts?: { sourceId?: string }): Promise<void>;
// Stats + health
getStats(): Promise<BrainStats>;
getHealth(): Promise<BrainHealth>;
// Ingest log
logIngest(entry: IngestLogInput): Promise<void>;
getIngestLog(opts?: { limit?: number }): Promise<IngestLogEntry[]>;
// Sync
/**
* Rename a page's slug (chunks + links + tags + timeline + versions all
* preserved via stable page_id). `opts.sourceId` scopes the UPDATE — without
* it, the bare `WHERE slug = old` matches every row across every source and
* would either rename them all OR violate the (source_id, slug) UNIQUE.
*/
updateSlug(oldSlug: string, newSlug: string, opts?: { sourceId?: string }): Promise<void>;
rewriteLinks(oldSlug: string, newSlug: string): Promise<void>;
/**
* v0.42 type-unification (T2, plan D1+F10). Returns the canonical slug if
* `slug` is registered in `slug_aliases` for any of the provided source(s);
* otherwise returns `slug` unchanged. Defense-in-depth: also returns the
* input when the table doesn't exist yet (pre-v104 brains).
*
* Accepts either a single sourceId (scalar) OR a sourceIds array
* (federated reads). Multi-source ambiguity: when the same alias_slug
* exists in two registered sources, returns the first match in array
* order and emits a once-per-process stderr `multi_match` warning.
*
* Callers (the cluster the alias-table primitive is meant for):
* - src/core/entities/resolve.ts: wikilink resolver short-circuit
* (alias-table is authoritative; runs BEFORE fuzzy/prefix cascade)
* - MCP `read_page` op (canonical lookup)
* - Search rank stage `applyAliasResolvedBoost` (knows whether a
* top-K result was reached via an alias)
*
* Source-scoped throughout per F12 (codex outside voice) — no cross-source
* false-positive resolution. v0.42 ships this method on both engines.
*/
resolveSlugWithAlias(
slug: string,
sourceOrSources: string | readonly string[],
): Promise<string>;
/**
* T3 retrieval-cathedral — free-text alias resolution for SEARCH.
* Given a set of NORMALIZED alias strings (normalizeAlias() output),
* return a map alias_norm -> canonical slugs from page_aliases. An alias
* may resolve to MORE THAN ONE slug (two pages claimed the same name) —
* the caller reports the collision and resolves deterministically.
*
* Source-scoped: pass scalar `sourceId` (single-source) OR `sourceIds`
* (federated read) — array wins when both set, matching sourceScopeOpts.
* Empty input → empty map (no query). Throws "relation page_aliases does
* not exist" on pre-v110 brains; the alias-hop caller wraps in try/catch
* (isUndefinedColumnError) and degrades to no-injection (D9 fail-open).
*/
resolveAliases(
aliasNorms: string[],
opts?: { sourceId?: string; sourceIds?: string[] },
): Promise<Map<string, Array<{ slug: string; source_id: string }>>>;
/**
* T3 retrieval-cathedral — WRITE side of the alias layer. Replace the full
* alias set for one (slug, source) with `aliasNorms` (already normalized):
* delete the page's existing page_aliases rows, insert the new set. Empty
* `aliasNorms` just clears them. Idempotent (the unique triple). Called by
* the ingest projection in importFromContent and the `reindex --aliases`
* backfill so a page's declared aliases stay in lockstep with its frontmatter.
*/
setPageAliases(
slug: string,
sourceId: string,
aliasNorms: string[],
): Promise<void>;
/**
* v0.35.5 — narrow UPDATE of `pages.compiled_truth`, `pages.timeline`, and
* `pages.content_hash` for a single slug+source. NO chunking, NO embedding,
* NO link reconcile, NO `updated_at` advance beyond the trivial bump.
*
* Used by the phantom-redirect pass in `extract_facts` after appending
* migrated fact rows to a canonical page's disk fence: we just rewrote the
* `.md` on disk, so the DB body must match before the next reconcile reads
* stale state. content_hash is included so the next `gbrain sync` sees the
* canonical as unchanged and skips re-import (round-14 + codex #7 — the
* "second cycle is a no-op" premise depends on all three columns moving
* together).
*
* Skips soft-deleted rows (deleted_at filter). Idempotent — second call
* with the same args produces the same row state.
*/
refreshPageBody(
slug: string,
sourceId: string,
compiledTruth: string,
timeline: string,
contentHash: string,
): Promise<void>;
/**
* v0.40.3.0 — narrow UPDATE that stamps the two CR-state columns
* (`contextual_retrieval_mode`, `corpus_generation`) plus
* `updated_at = now()` and nothing else.
*
* Used by `src/core/contextual-retrieval-service.ts:reembedPageWithContextualRetrieval`
* at the end of its PHASE 2 transaction. Why narrow instead of routing
* through `putPage`: stamping the CR state alone shouldn't trigger the
* full page-version snapshot machinery (createVersion fires on every
* putPage with an existing row, which would bloat page_versions on every
* tier upgrade).
*
* Skips soft-deleted rows (deleted_at filter). Idempotent — same args
* twice produces the same row state. Both columns are NULL-tolerant
* (callers pass NULL for `corpusGeneration` only on the 'none' tier
* path; 'title' and 'per_chunk_synopsis' always supply a hash).
*/
updatePageContextualRetrievalState(
slug: string,
sourceId: string,
mode: string,
corpusGeneration: string | null,
): Promise<void>;
/**
* v0.35.5 — lossless DB-side migration of fact rows from one slug to
* another within a single source. UPDATEs `entity_slug` and
* `source_markdown_slug` on every active fact row whose
* `source_markdown_slug` matches the phantom slug. Every other column
* (embedding, valid_from, valid_until, kind, notability, confidence,
* source_session, status, etc.) is preserved verbatim — codex #3 fix
* for the writeFactsToFence lossy-migration trap.
*
* Idempotent: re-run after success finds no rows to update and returns
* `{migrated: 0}`. Hard-deletes are out of scope; the caller wipes the
* phantom's `.md` file separately. Scoped to one source by design —
* cross-source migration is a separate concern.
*/
migrateFactsToCanonical(
phantomSlug: string,
canonicalSlug: string,
sourceId: string,
): Promise<{ migrated: number }>;
// Config
getConfig(key: string): Promise<string | null>;
setConfig(key: string, value: string): Promise<void>;
/**
* v0.32.3 — delete a config row. Returns the number of rows deleted (0 or 1).
* No-op when the key doesn't exist. Used by `gbrain config unset` and by
* `gbrain search modes --reset`. Engine-agnostic.
*/
unsetConfig(key: string): Promise<number>;
/**
* v0.32.3 — list config keys matching a literal prefix (e.g. "search.").
* Used by `gbrain config unset --pattern` and the search-modes --reset path.
* Does NOT support glob/regex on purpose — the caller knows the prefix.
*/
listConfigKeys(prefix: string): Promise<string[]>;
// Migration support
runMigration(version: number, sql: string): Promise<void>;
getChunksWithEmbeddings(slug: string, opts?: { sourceId?: string }): Promise<Chunk[]>;
// Raw SQL (for Minions job queue and other internal modules)
/**
* v0.41.18.0 (A20, codex #7): optional 3rd-arg `opts.signal` lets callers
* actually cancel a running query. Init nudge (3s wallclock cap) wires an
* AbortController whose timer fires at 3s; queries that haven't returned
* by then get cancelled (Postgres: query.cancel(); PGLite: in-process,
* Promise.race against signal-rejection — documented gap because PGLite
* has no kernel-level cancellation).
*/
executeRaw<T = Record<string, unknown>>(
sql: string,
params?: unknown[],
opts?: { signal?: AbortSignal },
): Promise<T[]>;
// ============================================================
// v0.20.0 Cathedral II: code edges (Layer 5 populates, Layer 7 consumes)
// ============================================================
/**
* Bulk-insert code edges. Resolved edges (to_chunk_id set) land in
* code_edges_chunk; unresolved refs (to_chunk_id null, to_symbol_qualified
* set) land in code_edges_symbol. ON CONFLICT DO NOTHING handles idempotency.
* Returns count of rows actually inserted.
*/
addCodeEdges(edges: CodeEdgeInput[]): Promise<number>;
/**
* Delete all code edges involving these chunk IDs, in BOTH directions, across
* both code_edges_chunk and code_edges_symbol. Called by importCodeFile on
* per-chunk invalidation (codex SP-2): when a chunk's text changed, stale
* inbound edges from other pages pointing at the old symbol must wipe before
* new edges write.
*/
deleteCodeEdgesForChunks(chunkIds: number[]): Promise<void>;
/**
* "Who calls this symbol?" Returns UNION of code_edges_chunk +
* code_edges_symbol matching `to_symbol_qualified = qualifiedName`.
* Source scoping (codex SP-3): if opts.sourceId is set, filter by the
* anchor chunk's source; if opts.allSources, ignore scoping.
*/
getCallersOf(
qualifiedName: string,
opts?: { sourceId?: string; allSources?: boolean; limit?: number },
): Promise<CodeEdgeResult[]>;
/**
* "What does this symbol call?" Returns edges from chunks whose
* from_symbol_qualified = qualifiedName. Same source-scoping semantics
* as getCallersOf.
*/
getCalleesOf(
qualifiedName: string,
opts?: { sourceId?: string; allSources?: boolean; limit?: number },
): Promise<CodeEdgeResult[]>;
/**
* All edges touching a chunk in the given direction. Used by A2 two-pass
* retrieval to expand from anchor chunks. direction='in' returns edges
* pointing AT the chunk; 'out' returns edges FROM it; 'both' unions.
*/
getEdgesByChunk(
chunkId: number,
opts?: { direction?: 'in' | 'out' | 'both'; edgeType?: string; limit?: number },
): Promise<CodeEdgeResult[]>;
/**
* Chunk-grain keyword search. Ranks by content_chunks.search_vector
* without the dedup-to-page pass that searchKeyword applies. Consumed
* by A2 two-pass retrieval as its anchor source. Most callers should
* prefer searchKeyword (external contract: page-grain best-chunk-per-page).
*/
searchKeywordChunks(query: string, opts?: SearchOpts): Promise<SearchResult[]>;
// Eval capture (v0.25.0 — BrainBench-Real substrate).
// Captured at the op-layer wrapper in src/core/operations.ts; reads via
// `gbrain eval export` (NDJSON) for sibling gbrain-evals consumption.
// Adding these to BrainEngine is a breaking-interface change for third-
// party engine implementers — this is why v0.25.0 is a minor bump.
/** Insert a captured candidate. Returns the new row id. Best-effort: callers swallow failures and route them through `logEvalCaptureFailure`. */
logEvalCandidate(input: EvalCandidateInput): Promise<number>;
/** Read candidates by time window / limit / tool filter. Used by `gbrain eval export`. */
listEvalCandidates(filter?: { since?: Date; limit?: number; tool?: 'query' | 'search' }): Promise<EvalCandidate[]>;
/** Delete candidates created before `date`. Returns rows deleted. Used by `gbrain eval prune`. */
deleteEvalCandidatesBefore(date: Date): Promise<number>;
/** Log a capture failure so `gbrain doctor` can surface drops cross-process. Best-effort; symmetric with logEvalCandidate (failure-of-failure is lost). */
logEvalCaptureFailure(reason: EvalCaptureFailureReason): Promise<void>;
/** Read capture failures within an optional time window. Used by `gbrain doctor`. */
listEvalCaptureFailures(filter?: { since?: Date }): Promise<EvalCaptureFailure[]>;
// ============================================================
// v0.29 — Salience + Anomaly Detection
// ============================================================
// The brain surfaces what's unusual and emotionally charged without being
// asked. Cost: ~zero at query time (deterministic SQL), with backfill done
// during the new `recompute_emotional_weight` cycle phase.
/**
* Batch-load tag + take inputs for the emotional-weight formula. One CTE-shaped
* query: `pages` LEFT JOIN aggregated `tags` and aggregated `takes` (each
* pre-aggregated in its own CTE so the page × N tags × M takes cartesian
* product is avoided).
*
* If `slugs` is undefined, returns inputs for every page in the brain
* (full-mode backfill). If provided, returns only matching slugs (incremental
* recompute after sync / synthesize touched specific pages).
*
* Multi-source-aware: each row carries its `source_id` so the matching
* `setEmotionalWeightBatch` UPDATE can composite-key correctly.
*/
batchLoadEmotionalInputs(slugs?: string[]): Promise<EmotionalWeightInputRow[]>;
/**
* Apply pre-computed emotional weights in a single UPDATE. Composite-keyed
* on `(slug, source_id)` because `pages.slug` is only unique within a
* source — a slug-only UPDATE would fan out across sources, the same bug
* that the v0.18.0 link batches fixed for cross-source edges.
*
* Returns the count of rows actually updated. Pages whose `(slug, source_id)`
* tuple doesn't exist (race with delete) are silently skipped.
*/
setEmotionalWeightBatch(rows: EmotionalWeightWriteRow[]): Promise<number>;
/**
* Salience query: pages recently touched, ranked by a deterministic
* `(emotional_weight * 5) + ln(1 + take_count) + recency_decay` score.
*
* The handler computes the time boundary in JS (`now - days * 86400000`)
* and binds it as TIMESTAMPTZ so the SQL is identical across PGLite +
* Postgres (eng review D5 — avoids dialect drift on `interval` binding).
*/
getRecentSalience(opts: SalienceOpts): Promise<SalienceResult[]>;
/**
* Anomaly detection: cohorts (tag, type) with unusually-high page activity
* on a target day vs baseline mean+stddev over the previous N days. Year
* cohort is deferred to v0.30 (slug-regex year extraction is fragile).
*
* Baseline densifies the day series via `generate_series` zero-fill so
* sparse-day rare cohorts don't look "normally active" — a sparse-day cohort
* with one touch in 30 days has a low baseline mean and high sigma at 7 touches,
* not a misleading mean of 1.
*/
findAnomalies(opts: AnomaliesOpts): Promise<AnomalyResult[]>;
}