mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 21:19:18 +00:00
* feat(links): relax link_source CHECK to kebab-case provenance + migration v114 Open link_source from a closed allowlist to a kebab-case format gate (^[a-z][a-z0-9]*(-[a-z0-9]+)*$, char_length<=64) so external derivers stamp their own provenance (e.g. citation-graph) without a per-deriver migration. Migration v114: Postgres NOT VALID + VALIDATE (lock-friendly, transaction:false); PGLite plain DROP+ADD. Updates the schema.sql + engine provenance contract comments. (#1941) * feat(links): expose link provenance on link ops + link-add/link-rm/link-sources add_link/remove_link now accept --link-source/--link-type; add_link guards the reconciliation-managed built-ins (markdown/frontmatter/mentions/ wikilink-resolved) and defaults omitted provenance to 'manual' (was the misleading engine default 'markdown'). New cliHints.aliases mechanism with a startup collision guard registers link-add/link-rm; printOpHelp shows the invoked alias name. New list_link_sources read op + listLinkSources engine method (both engines, {sourceId?,sourceIds?}, deterministic order) powers `gbrain link-sources`, added to the minion read allowlist. (#1941) * test(links): kebab provenance, op guard, link-sources, aliases + parity Covers the v114 regex/length boundaries, upgrade-path constraint swap on existing data, the managed-built-in op guard + manual default, remove_link type/source filters, list_link_sources scoping (scalar + federated) and PG/PGLite parity, and alias resolution/collision/help. Fixes the prior 'inferred'-rejection assertion (now valid kebab) in the mentions test. (#1941) * chore: bump version and changelog (v0.42.31.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: update KEY_FILES for v0.42.31.0 link provenance surface KEY_FILES.md current-state updates for #1941: link_source now an open kebab-case provenance (migration v114), the add_link/remove_link guard + defaults, list_link_sources + listLinkSources, and cliHints.aliases. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(minions): supervisor queue-singleton keying + pidfile cleanup (follow-up #1849) Two correctness bugs in the v0.42.29.0 supervisor-singleton work, caught by adversarial review: - supervisorLockId mixed a config-derived DB identity into the key, but the lock row already lives inside the target database. Two supervisors on the same physical DB via different-but-equivalent URLs (pooler vs direct port, host alias, trailing params) computed different ids and BOTH acquired the "singleton" lock. Key on the queue alone; the database half of the mutex is physical. Removes the now-dead currentDbIdentity() from worker-registry. - The pidfile-cleanup process.on('exit') listener was installed AFTER the DB-lock acquire, so the LOCK_HELD early-exit stranded the pidfile this process had just created. Install the listener first. Regression test pins the listener-before-lock ordering; updates the lockId test to the queue-only invariant; KEY_FILES updated to current state. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
f8d4ce6fc4
commit
f401d7407e
@@ -2,6 +2,27 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.31.0] - 2026-06-07
|
||||
|
||||
**You can now write typed graph edges with your own provenance straight from the CLI — `gbrain link-add a b --link-type relies-on --link-source citation-graph` — and an external edge-writer (a citation-graph ingester, an importer, a classifier) no longer needs a gbrain schema migration to register a new provenance.** Two ergonomics gaps for tools that compute edges out-of-band, filed by a downstream agent building a citation-graph ingester (#1941).
|
||||
|
||||
Before this, `link_source` was a closed allowlist: anything outside `markdown`/`frontmatter`/`manual`/`mentions`/`wikilink-resolved` was rejected by a CHECK constraint, so a deriver had to either patch gbrain's schema or stamp its machine-derived edges `manual` — which made them indistinguishable from hand-entered ones. `link_source` is now an open, format-validated provenance: any lowercase kebab-case tag up to 64 chars (`citation-graph`, `relies-on-graph`, your-tag) is valid, no migration needed. The format gate still rejects garbage (uppercase, spaces, underscores, leading/trailing/double dashes).
|
||||
|
||||
The CLI gap is closed too. `gbrain link` / `gbrain unlink` now take `--link-source` and `--link-type`, with `link-add` / `link-rm` aliases for discoverability. A new `gbrain link-sources` lists the distinct provenances a brain carries (with counts) — the read-side replacement for the discoverability the old allowlist gave you for free. CLI-created edges now record `manual` provenance by default instead of masquerading as parsed-from-`markdown`, and the reconciliation-managed provenances stay reserved for the writers that own their semantics.
|
||||
|
||||
### Added
|
||||
- **`gbrain link-add` / `link-rm` / `link-sources`** plus `--link-source` and `--link-type` on the link ops. Write typed, provenance-tagged, source-scoped edges from the CLI and list which provenances a brain holds. Provenance is any kebab-case tag; removals can filter by provenance so machine-derived edges delete cleanly without touching hand-entered ones. (#1941)
|
||||
|
||||
### Changed
|
||||
- **`link_source` is an open kebab-case provenance, not a closed allowlist (migration v114).** External edge-writers register a new provenance with no gbrain migration. Existing provenances are unaffected; the migration is lock-friendly on Postgres (validates without blocking writes) and applies automatically on upgrade. CLI-created links now default to `manual` provenance.
|
||||
|
||||
### Fixed
|
||||
- **Supervisor queue-singleton hardening (follow-up to #1849).** Two supervisors pointed at the same database via different-but-equivalent connection strings could each acquire the "one per queue" lock; the lock is now keyed on the queue alone (its row already lives in the target database), so same-database + same-queue collides correctly. A supervisor that loses the lock race on startup also no longer leaves its pidfile behind to block the next start.
|
||||
|
||||
### To take advantage of v0.42.31.0
|
||||
|
||||
`gbrain upgrade`. The constraint migration runs automatically. To write edges from a tool or the CLI: `gbrain link-add <from> <to> --link-type <verb> --link-source <your-tag>`; `gbrain link-sources` shows what's in the graph; `gbrain link-rm <from> <to> --link-source <your-tag>` removes only that provenance's edges.
|
||||
|
||||
## [0.42.29.0] - 2026-06-07
|
||||
|
||||
**The background-job queue stops thrashing on long jobs, the cycle stops wedging itself, and you can no longer run two supervisors against one queue by accident.** Three fixes plus a voice-agent feature.
|
||||
|
||||
File diff suppressed because one or more lines are too long
+1
-1
@@ -143,5 +143,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.29.0"
|
||||
"version": "0.42.31.0"
|
||||
}
|
||||
|
||||
+41
-12
@@ -94,6 +94,25 @@ const CLI_ONLY_SELF_HELP = new Set([
|
||||
'connect',
|
||||
]);
|
||||
|
||||
// v114 (#1941): alias -> operation lookup, kept separate from `cliOps` so
|
||||
// aliases don't double-list in printHelp's auto-generated section. Collisions
|
||||
// with a primary CLI name, a CLI_ONLY command, or another alias throw at module
|
||||
// load — a silent route-shadow is worse than a loud boot failure. Placed after
|
||||
// CLI_ONLY so the collision check can see it.
|
||||
export const cliAliases = new Map<string, Operation>();
|
||||
for (const op of operations) {
|
||||
if (op.cliHints?.hidden) continue;
|
||||
for (const alias of op.cliHints?.aliases ?? []) {
|
||||
if (cliOps.has(alias) || CLI_ONLY.has(alias) || cliAliases.has(alias)) {
|
||||
throw new Error(
|
||||
`CLI alias collision: '${alias}' (op '${op.name}') conflicts with an existing ` +
|
||||
`command or alias. Rename the alias in src/core/operations.ts.`,
|
||||
);
|
||||
}
|
||||
cliAliases.set(alias, op);
|
||||
}
|
||||
}
|
||||
|
||||
// v0.42 self-upgrade: commands that must NOT trigger the startup update-check
|
||||
// (they ARE the update path, or are trivial/no-DB) and which set
|
||||
// GBRAIN_SKIP_STARTUP_HOOKS for any children they spawn.
|
||||
@@ -248,9 +267,9 @@ async function main() {
|
||||
|
||||
// Per-command --help
|
||||
if (hasHelpFlag(subArgs)) {
|
||||
const op = cliOps.get(command);
|
||||
const op = cliOps.get(command) ?? cliAliases.get(command);
|
||||
if (op) {
|
||||
printOpHelp(op);
|
||||
printOpHelp(op, command);
|
||||
return;
|
||||
}
|
||||
if (CLI_ONLY.has(command) && !CLI_ONLY_SELF_HELP.has(command)) {
|
||||
@@ -265,8 +284,8 @@ async function main() {
|
||||
return;
|
||||
}
|
||||
|
||||
// Shared operations
|
||||
const op = cliOps.get(command);
|
||||
// Shared operations (fall through to aliases, e.g. link-add -> add_link)
|
||||
const op = cliOps.get(command) ?? cliAliases.get(command);
|
||||
if (!op) {
|
||||
console.error(`Unknown command: ${command}`);
|
||||
console.error('Run gbrain --help for available commands.');
|
||||
@@ -2058,9 +2077,11 @@ async function connectEngine(opts?: { probeOnly?: boolean }): Promise<BrainEngin
|
||||
return engine;
|
||||
}
|
||||
|
||||
function printOpHelp(op: Operation) {
|
||||
export function printOpHelp(op: Operation, invokedName?: string) {
|
||||
const positional = (op.cliHints?.positional || []).map(p => `<${p}>`).join(' ');
|
||||
const name = op.cliHints?.name || op.name;
|
||||
// v114 (#1941): when invoked via an alias (e.g. `gbrain link-add --help`),
|
||||
// show the alias the user typed, not the primary op name.
|
||||
const name = invokedName || op.cliHints?.name || op.name;
|
||||
console.log(`Usage: gbrain ${name} ${positional} [options]\n`);
|
||||
console.log(op.description + '\n');
|
||||
const entries = Object.entries(op.params);
|
||||
@@ -2125,8 +2146,11 @@ EMBEDDINGS
|
||||
embed [<slug>|--all|--stale] Generate/refresh embeddings
|
||||
|
||||
LINKS
|
||||
link <from> <to> [--type T] Create typed link
|
||||
unlink <from> <to> Remove link
|
||||
link <from> <to> Create typed link (alias: link-add)
|
||||
[--link-type T] [--link-source S] provenance defaults to 'manual'
|
||||
unlink <from> <to> Remove link (alias: link-rm)
|
||||
[--link-type T] [--link-source S] filter which edges to remove
|
||||
link-sources List provenances in use, with edge counts
|
||||
backlinks <slug> Incoming links
|
||||
graph <slug> [--depth N] Traverse link graph (returns nodes)
|
||||
graph-query <slug> [--type T] Edge-based traversal with type/direction filters
|
||||
@@ -2222,7 +2246,12 @@ Run gbrain <command> --help for command-specific help.
|
||||
`);
|
||||
}
|
||||
|
||||
main().catch(e => {
|
||||
console.error(e.message || e);
|
||||
process.exit(1);
|
||||
});
|
||||
// Only auto-run when invoked as the entry point (the compiled binary or
|
||||
// `bun src/cli.ts`). Guarded so tests can import cliAliases / printOpHelp
|
||||
// without triggering argv parsing + main(). v114 (#1941).
|
||||
if (import.meta.main) {
|
||||
main().catch(e => {
|
||||
console.error(e.message || e);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
|
||||
+20
-4
@@ -127,10 +127,16 @@ export interface LinkBatchInput {
|
||||
link_type?: string;
|
||||
context?: string;
|
||||
/**
|
||||
* Provenance (v0.13+). Pass 'frontmatter' for edges derived from YAML
|
||||
* frontmatter, 'markdown' for [Name](path) refs, 'manual' for user-created.
|
||||
* NULL means "legacy / unknown" and is only used by pre-v0.13 rows; new
|
||||
* writes should always set this. Missing on input defaults to 'markdown'.
|
||||
* Provenance (v0.13+; opened to kebab tags in v114 / #1941). Any lowercase
|
||||
* kebab-case value <=64 chars is DB-valid (CHECK `^[a-z][a-z0-9]*(-[a-z0-9]+)*$`),
|
||||
* so external derivers stamp their own tag (e.g. 'citation-graph'). The
|
||||
* reconciliation-managed built-ins are 'markdown' ([Name](path) refs),
|
||||
* 'frontmatter' (YAML-derived, see origin_*), 'mentions', 'wikilink-resolved';
|
||||
* 'manual' is for user/tool-created edges. NULL = legacy/unknown (pre-v0.13).
|
||||
* Missing on this batch input defaults to 'markdown'. NOTE: the add_link OP
|
||||
* (not this engine method) forbids callers from passing the four managed
|
||||
* built-ins and defaults omitted to 'manual' — internal callers use the
|
||||
* engine directly and keep writing the managed values.
|
||||
*/
|
||||
link_source?: string;
|
||||
/** For link_source='frontmatter': slug of the page whose frontmatter created this edge. */
|
||||
@@ -1137,6 +1143,16 @@ export interface BrainEngine {
|
||||
* applied to the to-page side of the join.
|
||||
*/
|
||||
getBacklinks(slug: string, opts?: { sourceId?: string }): Promise<Link[]>;
|
||||
/**
|
||||
* v114 (#1941): distinct link_source provenances with edge counts, for
|
||||
* `gbrain link-sources`. Source-scoped via `{sourceId?, sourceIds?}` (both
|
||||
* forms, so federated `allowedSources` reads don't leak cross-source counts).
|
||||
* Deterministic order `count DESC, link_source ASC NULLS LAST` for PG/PGLite
|
||||
* parity. `link_source` may be NULL (legacy/unknown rows).
|
||||
*/
|
||||
listLinkSources(
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<{ link_source: string | null; count: number }[]>;
|
||||
/**
|
||||
* Fuzzy-match a display name to a page slug using pg_trgm similarity.
|
||||
* Zero embedding cost, zero LLM cost — designed for the v0.13 resolver used
|
||||
|
||||
@@ -5120,6 +5120,46 @@ export const MIGRATIONS: Migration[] = [
|
||||
`,
|
||||
},
|
||||
},
|
||||
{
|
||||
version: 114,
|
||||
name: 'links_link_source_check_kebab_regex',
|
||||
// Issue #1941: open link_source from a closed allowlist to a kebab-case
|
||||
// format gate so external derivers (e.g. 'citation-graph') stamp their own
|
||||
// provenance without a per-deriver gbrain migration. Format: lowercase
|
||||
// kebab `^[a-z][a-z0-9]*(-[a-z0-9]+)*$` (rejects UPPER, leading digit/dash,
|
||||
// trailing/double dash, underscore, space) + char_length <= 64 cap on the
|
||||
// indexed free-text column. The five prior built-ins all satisfy the regex,
|
||||
// so existing rows pass `VALIDATE` and the constraint swap never fails.
|
||||
//
|
||||
// DELIBERATELY diverges from the v95/v113 plain DROP+ADD pattern: on real
|
||||
// Postgres a plain `ADD CONSTRAINT ... CHECK` takes ACCESS EXCLUSIVE + a
|
||||
// full-table validation scan, which can stall writes on a large `links`
|
||||
// table. The postgres branch instead does `ADD ... NOT VALID` (instant,
|
||||
// no scan) then `VALIDATE CONSTRAINT` (scans under SHARE UPDATE EXCLUSIVE,
|
||||
// does not block reads/writes). That two-phase form requires running
|
||||
// OUTSIDE a transaction → `transaction: false`. PGLite (single-writer WASM,
|
||||
// no lock concern) keeps the plain one-shot DROP+ADD, and is the branch the
|
||||
// schema-version hash reads (pglite-engine.ts).
|
||||
//
|
||||
// Idempotent via DROP ... IF EXISTS; no-ops on installs that never created
|
||||
// the constraint and safe to re-run.
|
||||
idempotent: true,
|
||||
transaction: false,
|
||||
sql: '', // engine-specific via sqlFor (postgres two-phase vs pglite one-shot)
|
||||
sqlFor: {
|
||||
postgres: `
|
||||
ALTER TABLE links DROP CONSTRAINT IF EXISTS links_link_source_check;
|
||||
ALTER TABLE links ADD CONSTRAINT links_link_source_check
|
||||
CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)) NOT VALID;
|
||||
ALTER TABLE links VALIDATE CONSTRAINT links_link_source_check;
|
||||
`,
|
||||
pglite: `
|
||||
ALTER TABLE links DROP CONSTRAINT IF EXISTS links_link_source_check;
|
||||
ALTER TABLE links ADD CONSTRAINT links_link_source_check
|
||||
CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64));
|
||||
`,
|
||||
},
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -44,7 +44,7 @@ import {
|
||||
import { dirname } from 'path';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { tryAcquireDbLock, type DbLockHandle } from '../db-lock.ts';
|
||||
import { currentDbIdentity, currentBrainId } from './worker-registry.ts';
|
||||
import { currentBrainId } from './worker-registry.ts';
|
||||
|
||||
export type SupervisorEvent =
|
||||
| 'started'
|
||||
@@ -298,14 +298,20 @@ const SUPERVISOR_LOCK_REFRESH_MS = 60_000;
|
||||
const SUPERVISOR_LOCK_REFRESH_MAX_FAILURES = 3; // 3 × 60s = 180s < 5min TTL
|
||||
|
||||
/**
|
||||
* #1849: the queue-scoped supervisor singleton lock id. Keyed on the raw DB
|
||||
* identity (T2) + queue so the mutex domain is the (database, queue) pair —
|
||||
* not the pidfile path. Exported so `gbrain doctor` queries the same row to
|
||||
* surface the holder + effective --max-rss. Pass an explicit dbIdentity
|
||||
* (defaults to `currentDbIdentity()`, which reads config without a DB connect).
|
||||
* #1849: the queue-scoped supervisor singleton lock id. Keyed ONLY on the
|
||||
* queue, because the lock ROW lives inside the target database — the (database)
|
||||
* half of the mutex domain is physical, not part of the key. Keying on a
|
||||
* config-derived DB identity (the prior `currentDbIdentity()`) was a bug: two
|
||||
* supervisors pointed at the SAME physical database via different-but-equivalent
|
||||
* URLs/config paths (pooler vs direct port, host alias, trailing params) hashed
|
||||
* to different ids and BOTH acquired the "singleton" lock in the one shared
|
||||
* locks table. Queue-only keying makes same-DB + same-queue collide correctly,
|
||||
* while different physical databases never collide (separate locks tables).
|
||||
* Exported so `gbrain doctor` queries the same row to surface the holder +
|
||||
* effective --max-rss.
|
||||
*/
|
||||
export function supervisorLockId(queue: string, dbIdentity: string = currentDbIdentity()): string {
|
||||
return `gbrain-supervisor:${dbIdentity}:${queue}`;
|
||||
export function supervisorLockId(queue: string): string {
|
||||
return `gbrain-supervisor:${queue}`;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -495,24 +501,13 @@ export class MinionSupervisor {
|
||||
process.exit(ExitCodes.PID_UNWRITABLE);
|
||||
}
|
||||
|
||||
// 1b. #1849: queue-scoped DB singleton lock — the REAL authority. A second
|
||||
// supervisor with a different $HOME / --pid-file passes the pidfile check
|
||||
// above but loses here, so it can't run a conflicting --max-rss worker on
|
||||
// the same (db, queue). Keyed on the raw DB identity (not the lossy
|
||||
// currentBrainId hash) per T2.
|
||||
this.dbLock = await tryAcquireDbLock(this.engine, this.supervisorLockId(), SUPERVISOR_LOCK_TTL_MIN);
|
||||
if (!this.dbLock) {
|
||||
console.error(
|
||||
`Supervisor already running for queue '${this.opts.queue}' on this database ` +
|
||||
`(another supervisor holds the queue lock, regardless of pidfile path). Exiting.`,
|
||||
);
|
||||
process.exit(ExitCodes.LOCK_HELD);
|
||||
}
|
||||
// Refresh the lock on its own timer (independent of healthInterval, which
|
||||
// can be 0/disabled) so the TTL never lapses while we're alive.
|
||||
this.lockRefreshTimer = setInterval(() => { void this.refreshDbLock(); }, SUPERVISOR_LOCK_REFRESH_MS);
|
||||
|
||||
// 2. Cleanup on process exit (covers any exit path including process.exit).
|
||||
// Installed BEFORE the DB-lock acquisition below: acquirePidLock just
|
||||
// wrote OUR pid into the pidfile, so any early `process.exit` after this
|
||||
// point (notably the LOCK_HELD path in 1b) MUST clean it up or it leaves
|
||||
// a stale pidfile that blocks the next start on this path. The listener
|
||||
// only unlinks when the file still holds our pid, so it's a no-op on the
|
||||
// 'held'/'unwritable' paths above (those never created our pidfile).
|
||||
this.exitListener = () => {
|
||||
try {
|
||||
if (existsSync(this.opts.pidFile)) {
|
||||
@@ -525,6 +520,24 @@ export class MinionSupervisor {
|
||||
};
|
||||
process.on('exit', this.exitListener);
|
||||
|
||||
// 1b. #1849: queue-scoped DB singleton lock — the REAL authority. A second
|
||||
// supervisor with a different $HOME / --pid-file passes the pidfile check
|
||||
// above but loses here, so it can't run a conflicting --max-rss worker on
|
||||
// the same (db, queue). Keyed on the queue alone; the database half of the
|
||||
// mutex is physical (the lock row lives in this DB).
|
||||
this.dbLock = await tryAcquireDbLock(this.engine, this.supervisorLockId(), SUPERVISOR_LOCK_TTL_MIN);
|
||||
if (!this.dbLock) {
|
||||
console.error(
|
||||
`Supervisor already running for queue '${this.opts.queue}' on this database ` +
|
||||
`(another supervisor holds the queue lock, regardless of pidfile path). Exiting.`,
|
||||
);
|
||||
// The exit listener installed above removes the pidfile we just created.
|
||||
process.exit(ExitCodes.LOCK_HELD);
|
||||
}
|
||||
// Refresh the lock on its own timer (independent of healthInterval, which
|
||||
// can be 0/disabled) so the TTL never lapses while we're alive.
|
||||
this.lockRefreshTimer = setInterval(() => { void this.refreshDbLock(); }, SUPERVISOR_LOCK_REFRESH_MS);
|
||||
|
||||
// 3. Signal handlers (tracked refs; removed on shutdown for test lifecycle hygiene).
|
||||
this.sigtermListener = () => { void this.shutdown('SIGTERM', ExitCodes.CLEAN); };
|
||||
this.sigintListener = () => { void this.shutdown('SIGINT', ExitCodes.CLEAN); };
|
||||
|
||||
@@ -54,6 +54,10 @@ export const BRAIN_TOOL_ALLOWLIST: ReadonlySet<string> = new Set([
|
||||
'file_url',
|
||||
'get_backlinks',
|
||||
'traverse_graph',
|
||||
// v114 (#1941): read-only provenance discovery. Edge-WRITE ops (add_link /
|
||||
// remove_link) are deliberately NOT allowlisted — exposing graph writes to
|
||||
// subagents is a separate trust decision.
|
||||
'list_link_sources',
|
||||
'resolve_slugs',
|
||||
'get_ingest_log',
|
||||
'put_page',
|
||||
@@ -89,6 +93,7 @@ export const BRAIN_TOOL_USAGE_HINTS: Readonly<Record<string, string>> = {
|
||||
file_url: 'Get a presigned URL for a brain-stored file. Read-only; expires.',
|
||||
get_backlinks: 'List every page that links TO the given slug. Use for "what references this".',
|
||||
traverse_graph: 'Walk the typed-edge graph starting from a slug (e.g. `works_at`, `founded`, `invested_in`). Use for relationship queries.',
|
||||
list_link_sources: 'List the distinct link provenances in the brain with edge counts (e.g. `citation-graph`, `manual`). Use to discover which edge-writers have populated the graph.',
|
||||
resolve_slugs: 'Resolve free-form entity names to canonical slugs (e.g. "Alice" → `people/alice-example`). Use before any tool that takes a slug if the user gave a name not a slug.',
|
||||
get_ingest_log: 'Read the brain ingestion log for diagnostic / verification queries.',
|
||||
put_page: 'Write a markdown page to the gbrain DATABASE (NOT the local filesystem). Page becomes searchable + linkable. Slug must match the agent\'s allowed namespace.',
|
||||
|
||||
@@ -75,24 +75,6 @@ export function currentBrainId(): string {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The RAW database identity string (url or path), unhashed. Used as the
|
||||
* authoritative key for the #1849 queue-scoped supervisor singleton DB lock:
|
||||
* the protected resource is the (database, queue) pair, so the lock id must
|
||||
* key on the real DB identity, not the lossy djb2 of {@link currentBrainId}
|
||||
* (T2 — removes any hash-collision question). Reads config only (no DB
|
||||
* connection), so it's safe to call before the engine connects. Falls back
|
||||
* to 'default' so two unconfigured brains under one HOME still serialize.
|
||||
*/
|
||||
export function currentDbIdentity(): string {
|
||||
try {
|
||||
const cfg = loadConfig();
|
||||
return cfg?.database_url ?? cfg?.database_path ?? 'default';
|
||||
} catch {
|
||||
return 'default';
|
||||
}
|
||||
}
|
||||
|
||||
function entryPath(pid: number): string {
|
||||
return join(workerRegistryDir(), `worker-${pid}.json`);
|
||||
}
|
||||
|
||||
+57
-5
@@ -498,6 +498,12 @@ export interface Operation {
|
||||
localOnly?: boolean;
|
||||
cliHints?: {
|
||||
name?: string;
|
||||
/**
|
||||
* Alternate CLI command names that dispatch to this same op (v114 / #1941).
|
||||
* e.g. `link-add` aliasing `link`. Registered in `cli.ts`'s `cliAliases`
|
||||
* map; collisions with primary names or CLI_ONLY commands throw at startup.
|
||||
*/
|
||||
aliases?: string[];
|
||||
positional?: string[];
|
||||
stdin?: string;
|
||||
hidden?: boolean;
|
||||
@@ -1793,6 +1799,18 @@ const get_tags: Operation = {
|
||||
|
||||
// --- Links ---
|
||||
|
||||
/**
|
||||
* v114 (#1941): reconciliation-managed provenances a CALLER must not forge via
|
||||
* the add_link op. Internal writers (import-file frontmatter reconciliation,
|
||||
* extract --by-mention, wikilink resolution) write these straight through the
|
||||
* engine — they're excluded here, not at the DB CHECK. A hand-created edge
|
||||
* tagged 'frontmatter' with no origin_page_id would be a phantom that put_page
|
||||
* reconciliation (link_source='frontmatter' AND origin_page_id=written_page)
|
||||
* never cleans (see src/schema.sql). `manual` is intentionally absent — it IS
|
||||
* the user-facing provenance and the default for omitted link_source.
|
||||
*/
|
||||
export const MANAGED_LINK_SOURCES = ['markdown', 'frontmatter', 'mentions', 'wikilink-resolved'];
|
||||
|
||||
const add_link: Operation = {
|
||||
name: 'add_link',
|
||||
description: 'Create link between pages',
|
||||
@@ -1801,11 +1819,22 @@ const add_link: Operation = {
|
||||
to: { type: 'string', required: true },
|
||||
link_type: { type: 'string', description: 'Link type (e.g., invested_in, works_at)' },
|
||||
context: { type: 'string', description: 'Context for the link' },
|
||||
link_source: { type: 'string', description: "Provenance tag (kebab-case, e.g. 'citation-graph'). Defaults to 'manual'. Reconciliation-managed built-ins (markdown/frontmatter/mentions/wikilink-resolved) are rejected." },
|
||||
},
|
||||
mutating: true,
|
||||
scope: 'write',
|
||||
handler: async (ctx, p) => {
|
||||
if (ctx.dryRun) return { dry_run: true, action: 'add_link', from: p.from, to: p.to };
|
||||
// v114 (#1941): default omitted provenance to 'manual' (NOT the engine's
|
||||
// 'markdown' default) so hand/tool-created CLI edges are honestly manual,
|
||||
// and forbid forging the reconciliation-managed built-ins.
|
||||
const linkSource = ((p.link_source as string) || 'manual').trim();
|
||||
if (MANAGED_LINK_SOURCES.includes(linkSource)) {
|
||||
throw new Error(
|
||||
`link_source '${linkSource}' is reconciliation-managed and cannot be set manually; ` +
|
||||
`use 'manual' (the default) or a custom kebab tag like 'citation-graph'`,
|
||||
);
|
||||
}
|
||||
// v0.31.8 (D7): single ctx.sourceId scopes both endpoints + origin. Cross-
|
||||
// source link creation is out of scope for this wave; use the engine API
|
||||
// directly for that edge case.
|
||||
@@ -1815,12 +1844,12 @@ const add_link: Operation = {
|
||||
await ctx.engine.addLink( // gbrain-allow-direct-insert: add_link MCP op is the explicit canonical surface for manual link creation; auto-link reconciliation runs separately via auto_link post-hook
|
||||
p.from as string, p.to as string,
|
||||
(p.context as string) || '', (p.link_type as string) || '',
|
||||
undefined, undefined, undefined,
|
||||
linkSource, undefined, undefined,
|
||||
linkOpts,
|
||||
);
|
||||
return { status: 'ok' };
|
||||
},
|
||||
cliHints: { name: 'link', positional: ['from', 'to'] },
|
||||
cliHints: { name: 'link', aliases: ['link-add'], positional: ['from', 'to'] },
|
||||
};
|
||||
|
||||
const remove_link: Operation = {
|
||||
@@ -1829,6 +1858,8 @@ const remove_link: Operation = {
|
||||
params: {
|
||||
from: { type: 'string', required: true },
|
||||
to: { type: 'string', required: true },
|
||||
link_type: { type: 'string', description: 'Only remove edges of this link type (omit = all types)' },
|
||||
link_source: { type: 'string', description: 'Only remove edges of this provenance (e.g. citation-graph); omit = any provenance' },
|
||||
},
|
||||
mutating: true,
|
||||
scope: 'write',
|
||||
@@ -1837,10 +1868,15 @@ const remove_link: Operation = {
|
||||
const linkOpts = ctx.sourceId
|
||||
? { fromSourceId: ctx.sourceId, toSourceId: ctx.sourceId }
|
||||
: undefined;
|
||||
await ctx.engine.removeLink(p.from as string, p.to as string, undefined, undefined, linkOpts);
|
||||
await ctx.engine.removeLink(
|
||||
p.from as string, p.to as string,
|
||||
(p.link_type as string) || undefined,
|
||||
(p.link_source as string) || undefined,
|
||||
linkOpts,
|
||||
);
|
||||
return { status: 'ok' };
|
||||
},
|
||||
cliHints: { name: 'unlink', positional: ['from', 'to'] },
|
||||
cliHints: { name: 'unlink', aliases: ['link-rm'], positional: ['from', 'to'] },
|
||||
};
|
||||
|
||||
const get_links: Operation = {
|
||||
@@ -1872,6 +1908,22 @@ const get_backlinks: Operation = {
|
||||
cliHints: { name: 'backlinks', positional: ['slug'] },
|
||||
};
|
||||
|
||||
const list_link_sources: Operation = {
|
||||
name: 'list_link_sources',
|
||||
// v114 (#1941): the read-side counterpart to link-add/link-rm. Since
|
||||
// link_source is now an open kebab provenance (no allowlist), this is how an
|
||||
// agent discovers which provenances a brain actually carries.
|
||||
description: 'List distinct link_source provenances in the brain with edge counts (e.g. citation-graph, manual, markdown)',
|
||||
params: {},
|
||||
handler: async (ctx) => {
|
||||
// Route through sourceScopeOpts so the read honors both scalar ctx.sourceId
|
||||
// and federated ctx.auth.allowedSources (no cross-source provenance leak).
|
||||
return ctx.engine.listLinkSources(sourceScopeOpts(ctx));
|
||||
},
|
||||
scope: 'read',
|
||||
cliHints: { name: 'link-sources' },
|
||||
};
|
||||
|
||||
/**
|
||||
* Hard cap on traverse_graph depth from MCP callers. Each recursive CTE iteration
|
||||
* grows a `visited` array per path; in `direction=both` the join is `OR`-based and
|
||||
@@ -4677,7 +4729,7 @@ export const operations: Operation[] = [
|
||||
// Tags
|
||||
add_tag, remove_tag, get_tags,
|
||||
// Links
|
||||
add_link, remove_link, get_links, get_backlinks, traverse_graph,
|
||||
add_link, remove_link, get_links, get_backlinks, list_link_sources, traverse_graph,
|
||||
// Timeline
|
||||
add_timeline_entry, get_timeline,
|
||||
// Admin
|
||||
|
||||
@@ -2576,6 +2576,32 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return rows as unknown as Link[];
|
||||
}
|
||||
|
||||
async listLinkSources(
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<{ link_source: string | null; count: number }[]> {
|
||||
// v114 (#1941): distinct provenances + counts for `gbrain link-sources`.
|
||||
// Scope by the FROM page's source (consistent with getLinks). Federated
|
||||
// {sourceIds} takes precedence over scalar {sourceId}; neither = unscoped.
|
||||
const params: unknown[] = [];
|
||||
let where = '';
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
where = `JOIN pages f ON f.id = l.from_page_id WHERE f.source_id = ANY($${params.length}::text[])`;
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
where = `JOIN pages f ON f.id = l.from_page_id WHERE f.source_id = $${params.length}`;
|
||||
}
|
||||
const { rows } = await this.db.query(
|
||||
`SELECT l.link_source, COUNT(*)::int AS count
|
||||
FROM links l
|
||||
${where}
|
||||
GROUP BY l.link_source
|
||||
ORDER BY count DESC, l.link_source ASC NULLS LAST`,
|
||||
params,
|
||||
);
|
||||
return rows as unknown as { link_source: string | null; count: number }[];
|
||||
}
|
||||
|
||||
async findByTitleFuzzy(
|
||||
name: string,
|
||||
dirPrefix?: string,
|
||||
|
||||
@@ -255,12 +255,11 @@ CREATE TABLE IF NOT EXISTS links (
|
||||
to_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
|
||||
link_type TEXT NOT NULL DEFAULT '',
|
||||
context TEXT NOT NULL DEFAULT '',
|
||||
-- v0.41.18.0: 'mentions' added for auto-linked body-text mentions
|
||||
-- (gbrain extract links --by-mention). Filtered OUT of backlink-count
|
||||
-- for search ranking; only counts toward orphan-ratio + graph traversal.
|
||||
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
|
||||
-- wikilink resolution. See src/schema.sql for full rationale.
|
||||
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
|
||||
-- v114 (#1941): open kebab-case provenance (not a closed allowlist) so
|
||||
-- external derivers stamp their own tag. Format gate only: lowercase kebab,
|
||||
-- <=64 chars. Managed built-ins still satisfy this; see src/schema.sql for
|
||||
-- full rationale + the add_link op-layer guard against forging them.
|
||||
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
|
||||
-- v0.41.18.0 (codex finding #12): nullable link_kind distinguishes
|
||||
-- "plain body mention" from "verb-pattern-derived typed link" within
|
||||
-- link_source='mentions'. See src/schema.sql for full rationale.
|
||||
|
||||
@@ -2663,6 +2663,30 @@ export class PostgresEngine implements BrainEngine {
|
||||
return rows as unknown as Link[];
|
||||
}
|
||||
|
||||
async listLinkSources(
|
||||
opts?: { sourceId?: string; sourceIds?: string[] },
|
||||
): Promise<{ link_source: string | null; count: number }[]> {
|
||||
const sql = this.sql;
|
||||
// v114 (#1941): distinct provenances + counts for `gbrain link-sources`.
|
||||
// Scope by the FROM page's source (consistent with getLinks). Federated
|
||||
// {sourceIds} takes precedence over scalar {sourceId}; neither = unscoped.
|
||||
const sourceCondition =
|
||||
opts?.sourceIds && opts.sourceIds.length > 0
|
||||
? sql`WHERE f.source_id = ANY(${opts.sourceIds}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`WHERE f.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
const rows = await sql`
|
||||
SELECT l.link_source, COUNT(*)::int AS count
|
||||
FROM links l
|
||||
JOIN pages f ON f.id = l.from_page_id
|
||||
${sourceCondition}
|
||||
GROUP BY l.link_source
|
||||
ORDER BY count DESC, l.link_source ASC NULLS LAST
|
||||
`;
|
||||
return rows as unknown as { link_source: string | null; count: number }[];
|
||||
}
|
||||
|
||||
async findByTitleFuzzy(
|
||||
name: string,
|
||||
dirPrefix?: string,
|
||||
|
||||
@@ -431,12 +431,14 @@ CREATE TABLE IF NOT EXISTS links (
|
||||
to_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
|
||||
link_type TEXT NOT NULL DEFAULT '',
|
||||
context TEXT NOT NULL DEFAULT '',
|
||||
-- v0.41.18.0: 'mentions' added for auto-linked body-text mentions
|
||||
-- (gbrain extract links --by-mention). Filtered OUT of backlink-count
|
||||
-- for search ranking; only counts toward orphan-ratio + graph traversal.
|
||||
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
|
||||
-- wikilink resolution (bare [[name]] resolved by slug tail).
|
||||
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
|
||||
-- v114 (#1941): link_source is an open kebab-case provenance, not a closed
|
||||
-- allowlist — external derivers (e.g. 'citation-graph') stamp their own tag
|
||||
-- without a gbrain migration. Format gate only: lowercase kebab, <=64 chars.
|
||||
-- The reconciliation-managed built-ins ('markdown','frontmatter','mentions',
|
||||
-- 'wikilink-resolved') still satisfy this and are written internally; the
|
||||
-- add_link op forbids CALLERS from forging them (see operations.ts). 'manual'
|
||||
-- is the user-facing default for hand/tool-created edges.
|
||||
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
|
||||
-- v0.41.18.0: nullable link_kind distinguishes "plain body mention" from
|
||||
-- "verb-pattern-derived typed link" within link_source='mentions'.
|
||||
-- Codex finding #12 design: keep link_source stable; add link_kind
|
||||
|
||||
+22
-8
@@ -407,12 +407,24 @@ CREATE INDEX IF NOT EXISTS idx_code_edges_symbol_to
|
||||
-- ============================================================
|
||||
-- links: cross-references between pages
|
||||
-- ============================================================
|
||||
-- Provenance model (v0.13, extended issue #972):
|
||||
-- link_source — 'markdown' | 'frontmatter' | 'manual' | 'wikilink-resolved' | NULL
|
||||
-- 'wikilink-resolved' is the opt-in
|
||||
-- (link_resolution.global_basename) basename-match
|
||||
-- provenance — see issue #972 / migration v113.
|
||||
-- (NULL = legacy row written before v0.13; unknown source)
|
||||
-- Provenance model (v0.13; opened to kebab provenance in v114 / issue #1941):
|
||||
-- link_source — open kebab-case provenance tag, NOT a closed allowlist.
|
||||
-- Format gate (CHECK): ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ and
|
||||
-- char_length <= 64. NULL = legacy row (pre-v0.13).
|
||||
-- Reconciliation-managed built-ins written internally:
|
||||
-- 'markdown' — body markdown links
|
||||
-- 'frontmatter' — YAML frontmatter edges (see origin_*)
|
||||
-- 'mentions' — auto-linked body-text mentions
|
||||
-- 'wikilink-resolved'— opt-in global-basename [[name]] (#972)
|
||||
-- User/tool-facing:
|
||||
-- 'manual' — hand- or tool-created edges (the
|
||||
-- add_link op default + CLI link-add)
|
||||
-- '<your-tag>' — external derivers, e.g. 'citation-graph',
|
||||
-- stamp their own kebab tag (no migration).
|
||||
-- The add_link OP forbids callers from passing the four
|
||||
-- managed built-ins (they imply reconciliation semantics a
|
||||
-- hand-created row can't honor); the DB CHECK still admits
|
||||
-- them because internal writers use them. See operations.ts.
|
||||
-- origin_page_id — for link_source='frontmatter', the page whose YAML
|
||||
-- frontmatter created this edge; scopes reconciliation
|
||||
-- origin_field — the frontmatter field name (e.g. 'key_people')
|
||||
@@ -420,7 +432,9 @@ CREATE INDEX IF NOT EXISTS idx_code_edges_symbol_to
|
||||
-- The unique constraint includes link_source + origin_page_id so a manual edge
|
||||
-- and a frontmatter-derived edge with the same (from, to, type) tuple coexist.
|
||||
-- Reconciliation on put_page filters by (link_source='frontmatter' AND
|
||||
-- origin_page_id = written_page) — never touches other pages' edges.
|
||||
-- origin_page_id = written_page) — never touches other pages' edges. (This is
|
||||
-- exactly why a CLI-forged 'frontmatter' row with NULL origin would be a phantom
|
||||
-- edge reconciliation never cleans — hence the op-layer guard.)
|
||||
CREATE TABLE IF NOT EXISTS links (
|
||||
id SERIAL PRIMARY KEY,
|
||||
from_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
|
||||
@@ -432,7 +446,7 @@ CREATE TABLE IF NOT EXISTS links (
|
||||
-- for search ranking; only counts toward orphan-ratio + graph traversal.
|
||||
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
|
||||
-- wikilink resolution (bare [[name]] resolved by slug tail).
|
||||
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
|
||||
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
|
||||
-- v0.41.18.0: nullable link_kind distinguishes "plain body mention" from
|
||||
-- "verb-pattern-derived typed link" within link_source='mentions'.
|
||||
-- Codex finding #12 design: keep link_source stable; add link_kind
|
||||
|
||||
@@ -44,11 +44,13 @@ describe('BRAIN_TOOL_ALLOWLIST', () => {
|
||||
expect(missing).toEqual([]);
|
||||
});
|
||||
|
||||
test('contains the v0.15 read-only 10 + put_page + v0.29 salience pair', () => {
|
||||
test('contains the v0.15 read-only 10 + put_page + v0.29 salience pair + v114 list_link_sources', () => {
|
||||
// v0.29 added get_recent_salience + find_anomalies (read-only).
|
||||
// get_recent_transcripts is deliberately excluded — subagent calls always
|
||||
// have ctx.remote=true, and the v0.29 trust gate rejects remote callers.
|
||||
expect(BRAIN_TOOL_ALLOWLIST.size).toBe(13);
|
||||
// v114 (#1941) added list_link_sources (read-only provenance discovery);
|
||||
// the edge-WRITE ops add_link/remove_link stay out (separate trust call).
|
||||
expect(BRAIN_TOOL_ALLOWLIST.size).toBe(14);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('query')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('search')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('get_page')).toBe(true);
|
||||
@@ -56,6 +58,9 @@ describe('BRAIN_TOOL_ALLOWLIST', () => {
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('put_page')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('get_recent_salience')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('find_anomalies')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('list_link_sources')).toBe(true);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('add_link')).toBe(false);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('remove_link')).toBe(false);
|
||||
expect(BRAIN_TOOL_ALLOWLIST.has('get_recent_transcripts')).toBe(false);
|
||||
});
|
||||
|
||||
|
||||
@@ -339,6 +339,31 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
|
||||
}
|
||||
});
|
||||
|
||||
test('v114 (#1941) listLinkSources parity: same ordered provenance counts on both engines', async () => {
|
||||
const mk = async (eng: BrainEngine) => {
|
||||
for (const s of ['lsp-a', 'lsp-b', 'lsp-c']) {
|
||||
await eng.putPage(s, { type: 'note', title: s, compiled_truth: 'b', timeline: '' });
|
||||
}
|
||||
// citation-graph:2, manual:1 — exercises count DESC + the kebab regex.
|
||||
await eng.addLink('lsp-a', 'lsp-b', '', 'cites', 'citation-graph');
|
||||
await eng.addLink('lsp-a', 'lsp-c', '', 'cites', 'citation-graph');
|
||||
await eng.addLink('lsp-b', 'lsp-c', '', 'rel', 'manual');
|
||||
};
|
||||
await mk(pgEngine);
|
||||
await mk(pgliteEngine);
|
||||
|
||||
const pg = await pgEngine.listLinkSources({ sourceId: 'default' });
|
||||
const pglite = await pgliteEngine.listLinkSources({ sourceId: 'default' });
|
||||
|
||||
const norm = (rows: { link_source: string | null; count: number }[]) =>
|
||||
rows.filter(r => r.link_source === 'citation-graph' || r.link_source === 'manual');
|
||||
expect(norm(pg)).toEqual(norm(pglite));
|
||||
// citation-graph (2) must order before manual (1) on both engines.
|
||||
const cgIdx = pg.findIndex(r => r.link_source === 'citation-graph');
|
||||
const mIdx = pg.findIndex(r => r.link_source === 'manual');
|
||||
expect(cgIdx).toBeLessThan(mIdx);
|
||||
});
|
||||
|
||||
test('v0.41.19.0 resolveSlugsByPaths parity: same Map on both engines', async () => {
|
||||
const seedSql = `
|
||||
INSERT INTO pages (source_id, slug, source_path, type, title, compiled_truth, timeline, frontmatter)
|
||||
|
||||
@@ -0,0 +1,306 @@
|
||||
/**
|
||||
* v114 (#1941): link_source opened from a closed allowlist to a kebab-case
|
||||
* regex + length cap, provenance exposed on the link ops with a managed-
|
||||
* built-in guard, a new `list_link_sources` read op, and `link-add`/`link-rm`
|
||||
* CLI aliases.
|
||||
*
|
||||
* Coverage:
|
||||
* - Migration v114 shape (two engine branches, transaction:false, idempotent)
|
||||
* - DB CHECK boundary: kebab accepted (incl. all 5 built-ins), garbage rejected
|
||||
* - Upgrade-path: existing built-in row survives the constraint swap (re-run)
|
||||
* - add_link op: provenance threaded, default 'manual', managed-built-in guard
|
||||
* - remove_link op: link_type / link_source / both filters
|
||||
* - list_link_sources op: counts, deterministic order, scalar + federated scope
|
||||
* - CLI aliases resolve; printOpHelp shows the invoked alias name; no collisions
|
||||
*
|
||||
* Hermetic via PGLite. No DATABASE_URL needed.
|
||||
*/
|
||||
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { MIGRATIONS, LATEST_VERSION } from '../src/core/migrate.ts';
|
||||
import {
|
||||
operations,
|
||||
operationsByName,
|
||||
MANAGED_LINK_SOURCES,
|
||||
} from '../src/core/operations.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
import { cliAliases, printOpHelp } from '../src/cli.ts';
|
||||
|
||||
const V = 114;
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
}, 60_000);
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
function makeCtx(overrides: Partial<OperationContext> = {}): OperationContext {
|
||||
return {
|
||||
engine,
|
||||
remote: false,
|
||||
config: {},
|
||||
logger: console,
|
||||
dryRun: false,
|
||||
...overrides,
|
||||
} as unknown as OperationContext;
|
||||
}
|
||||
|
||||
// Two pages every link test reuses.
|
||||
async function seedPages() {
|
||||
await engine.putPage('lsrc-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
|
||||
await engine.putPage('lsrc-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
|
||||
}
|
||||
|
||||
describe('migration v114 — links_link_source_check_kebab_regex', () => {
|
||||
test('registered with expected version + name', () => {
|
||||
const m = MIGRATIONS.find(m => m.version === V);
|
||||
expect(m).toBeDefined();
|
||||
expect(m!.name).toBe('links_link_source_check_kebab_regex');
|
||||
});
|
||||
|
||||
test('LATEST_VERSION >= 114', () => {
|
||||
expect(LATEST_VERSION).toBeGreaterThanOrEqual(V);
|
||||
});
|
||||
|
||||
test('both engine branches present; transaction:false; idempotent', () => {
|
||||
const m = MIGRATIONS.find(m => m.version === V)!;
|
||||
expect(m.transaction).toBe(false);
|
||||
expect(m.idempotent).toBe(true);
|
||||
expect(m.sqlFor?.postgres).toBeTruthy();
|
||||
expect(m.sqlFor?.pglite).toBeTruthy();
|
||||
});
|
||||
|
||||
test('postgres branch uses NOT VALID + VALIDATE (lock-friendly)', () => {
|
||||
const m = MIGRATIONS.find(m => m.version === V)!;
|
||||
const pg = m.sqlFor!.postgres!;
|
||||
expect(pg).toMatch(/DROP CONSTRAINT IF EXISTS links_link_source_check/i);
|
||||
expect(pg).toContain('NOT VALID');
|
||||
expect(pg).toMatch(/VALIDATE CONSTRAINT links_link_source_check/i);
|
||||
expect(pg).toContain("'^[a-z][a-z0-9]*(-[a-z0-9]+)*$'");
|
||||
expect(pg).toContain('char_length(link_source) <= 64');
|
||||
});
|
||||
|
||||
test('pglite branch is plain DROP+ADD with the same regex', () => {
|
||||
const m = MIGRATIONS.find(m => m.version === V)!;
|
||||
const pl = m.sqlFor!.pglite!;
|
||||
expect(pl).toMatch(/DROP CONSTRAINT IF EXISTS links_link_source_check/i);
|
||||
expect(pl).not.toContain('NOT VALID');
|
||||
expect(pl).toContain("'^[a-z][a-z0-9]*(-[a-z0-9]+)*$'");
|
||||
});
|
||||
});
|
||||
|
||||
describe('DB CHECK — kebab format gate', () => {
|
||||
beforeAll(seedPages);
|
||||
|
||||
const ACCEPTED = [
|
||||
'citation-graph', 'derived', 'a',
|
||||
// OV7 — all 5 reconciliation-managed built-ins must stay DB-valid
|
||||
'markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved',
|
||||
];
|
||||
const REJECTED: Array<[string, string]> = [
|
||||
['UPPER', 'uppercase'],
|
||||
['has_underscore', 'underscore'],
|
||||
['has space', 'space'],
|
||||
['2bad', 'leading digit'],
|
||||
['-lead', 'leading dash'],
|
||||
['trail-', 'trailing dash'],
|
||||
['a--b', 'double dash'],
|
||||
['', 'empty'],
|
||||
['a'.repeat(65), '65 chars (over cap)'],
|
||||
];
|
||||
|
||||
for (const v of ACCEPTED) {
|
||||
test(`accepts '${v}'`, async () => {
|
||||
// engine.addLink writes link_source straight through (no op guard) — this
|
||||
// is the DB CHECK under test, not the op layer.
|
||||
await expect(
|
||||
engine.addLink('lsrc-a', 'lsrc-b', '', `t-${v}`, v),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
}
|
||||
|
||||
for (const [v, why] of REJECTED) {
|
||||
test(`rejects '${v}' (${why})`, async () => {
|
||||
await expect(
|
||||
engine.addLink('lsrc-a', 'lsrc-b', '', `t-rej-${why}`, v),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
describe('upgrade-path — existing rows survive the constraint swap', () => {
|
||||
test('built-in row present, re-running v114 succeeds, new kebab tag inserts', async () => {
|
||||
await seedPages();
|
||||
// A row tagged with a pre-existing built-in value (would have been valid
|
||||
// under the old allowlist too).
|
||||
await engine.addLink('lsrc-a', 'lsrc-b', '', 't-upgrade', 'markdown');
|
||||
const m = MIGRATIONS.find(m => m.version === V)!;
|
||||
// Re-apply the migration against the table that now holds data: ADD/VALIDATE
|
||||
// must not fail on the existing row.
|
||||
await expect(engine.runMigration(V, m.sqlFor!.pglite!)).resolves.toBeUndefined();
|
||||
// And a brand-new external provenance still inserts afterward.
|
||||
await expect(
|
||||
engine.addLink('lsrc-a', 'lsrc-b', '', 't-upgrade', 'citation-graph'),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('add_link op — provenance + managed-built-in guard (OV4A)', () => {
|
||||
beforeAll(seedPages);
|
||||
|
||||
test('threads custom provenance through to the row', async () => {
|
||||
const op = operationsByName['add_link'];
|
||||
await op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'cites', link_source: 'citation-graph' });
|
||||
const links = await engine.getLinks('lsrc-a');
|
||||
expect(links.some(l => (l as any).to_slug === 'lsrc-b' && (l as any).link_source === 'citation-graph' && (l as any).link_type === 'cites')).toBe(true);
|
||||
});
|
||||
|
||||
test("omitted link_source defaults to 'manual' (not 'markdown')", async () => {
|
||||
const op = operationsByName['add_link'];
|
||||
await op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'default-prov' });
|
||||
const links = await engine.getLinks('lsrc-a');
|
||||
const row = links.find(l => (l as any).link_type === 'default-prov');
|
||||
expect((row as any)?.link_source).toBe('manual');
|
||||
});
|
||||
|
||||
for (const managed of MANAGED_LINK_SOURCES) {
|
||||
test(`rejects managed built-in '${managed}'`, async () => {
|
||||
const op = operationsByName['add_link'];
|
||||
await expect(
|
||||
op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'g', link_source: managed }),
|
||||
).rejects.toThrow(/reconciliation-managed/);
|
||||
});
|
||||
}
|
||||
|
||||
test("'manual' is allowed (not in the managed set)", async () => {
|
||||
const op = operationsByName['add_link'];
|
||||
await expect(
|
||||
op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'man', link_source: 'manual' }),
|
||||
).resolves.toMatchObject({ status: 'ok' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('remove_link op — type/source filters', () => {
|
||||
async function seedTwoProvenances() {
|
||||
await engine.putPage('rm-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
|
||||
await engine.putPage('rm-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
|
||||
await engine.addLink('rm-a', 'rm-b', '', 'cites', 'manual');
|
||||
await engine.addLink('rm-a', 'rm-b', '', 'cites', 'citation-graph');
|
||||
}
|
||||
|
||||
test('--link-source filter deletes only that provenance', async () => {
|
||||
await seedTwoProvenances();
|
||||
const op = operationsByName['remove_link'];
|
||||
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_source: 'citation-graph' });
|
||||
const links = await engine.getLinks('rm-a');
|
||||
const sources = links.filter(l => (l as any).to_slug === 'rm-b').map(l => (l as any).link_source);
|
||||
expect(sources).toContain('manual');
|
||||
expect(sources).not.toContain('citation-graph');
|
||||
});
|
||||
|
||||
test('--link-type-only filter deletes every provenance of that type', async () => {
|
||||
await seedTwoProvenances();
|
||||
const op = operationsByName['remove_link'];
|
||||
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_type: 'cites' });
|
||||
const links = await engine.getLinks('rm-a');
|
||||
expect(links.filter(l => (l as any).to_slug === 'rm-b' && (l as any).link_type === 'cites').length).toBe(0);
|
||||
});
|
||||
|
||||
test('both filters delete only the type+source match', async () => {
|
||||
await seedTwoProvenances();
|
||||
const op = operationsByName['remove_link'];
|
||||
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_type: 'cites', link_source: 'manual' });
|
||||
const links = await engine.getLinks('rm-a');
|
||||
const sources = links.filter(l => (l as any).to_slug === 'rm-b').map(l => (l as any).link_source);
|
||||
expect(sources).toContain('citation-graph');
|
||||
expect(sources).not.toContain('manual');
|
||||
});
|
||||
});
|
||||
|
||||
describe('list_link_sources op (B4 + OV8 + OV9)', () => {
|
||||
test('op declaration: read scope, link-sources CLI name, not localOnly', () => {
|
||||
const op = operationsByName['list_link_sources'];
|
||||
expect(op).toBeDefined();
|
||||
expect(op.scope).toBe('read');
|
||||
expect(op.localOnly).not.toBe(true);
|
||||
expect(op.cliHints?.name).toBe('link-sources');
|
||||
});
|
||||
|
||||
test('returns distinct provenances with counts, ordered count DESC then name ASC', async () => {
|
||||
await engine.putPage('ls-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
|
||||
await engine.putPage('ls-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
|
||||
await engine.putPage('ls-c', { type: 'note', title: 'C', compiled_truth: 'c', timeline: '', frontmatter: {} });
|
||||
// zeta:1, alpha:2 — alpha sorts first by count; within ties, name ASC.
|
||||
await engine.addLink('ls-a', 'ls-b', '', 'r1', 'alpha-src');
|
||||
await engine.addLink('ls-a', 'ls-c', '', 'r2', 'alpha-src');
|
||||
await engine.addLink('ls-b', 'ls-c', '', 'r3', 'zeta-src');
|
||||
|
||||
const op = operationsByName['list_link_sources'];
|
||||
const rows = (await op.handler(makeCtx(), {})) as Array<{ link_source: string | null; count: number }>;
|
||||
const map = new Map(rows.map(r => [r.link_source, r.count]));
|
||||
expect(map.get('alpha-src')).toBe(2);
|
||||
expect(map.get('zeta-src')).toBe(1);
|
||||
// Deterministic order: counts non-increasing.
|
||||
for (let i = 1; i < rows.length; i++) {
|
||||
expect(rows[i - 1].count >= rows[i].count).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
test('scalar ctx.sourceId scopes; a non-matching source returns nothing', async () => {
|
||||
const op = operationsByName['list_link_sources'];
|
||||
// Pages above were written to the 'default' source.
|
||||
const inDefault = (await op.handler(makeCtx({ sourceId: 'default' } as any), {})) as unknown[];
|
||||
expect(inDefault.length).toBeGreaterThan(0);
|
||||
const inOther = (await op.handler(makeCtx({ sourceId: 'no-such-source' } as any), {})) as unknown[];
|
||||
expect(inOther.length).toBe(0);
|
||||
});
|
||||
|
||||
test('federated allowedSources ({sourceIds}) scopes (OV8)', async () => {
|
||||
const op = operationsByName['list_link_sources'];
|
||||
const inDefault = (await op.handler(makeCtx({ auth: { allowedSources: ['default'] } } as any), {})) as unknown[];
|
||||
expect(inDefault.length).toBeGreaterThan(0);
|
||||
const inOther = (await op.handler(makeCtx({ auth: { allowedSources: ['no-such-source'] } } as any), {})) as unknown[];
|
||||
expect(inOther.length).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CLI aliases + help (OV10 + OV11)', () => {
|
||||
test('link-add resolves to add_link, link-rm to remove_link', () => {
|
||||
expect(cliAliases.get('link-add')?.name).toBe('add_link');
|
||||
expect(cliAliases.get('link-rm')?.name).toBe('remove_link');
|
||||
});
|
||||
|
||||
test('no alias collides with any primary CLI name or another alias', () => {
|
||||
const primaries = new Set(operations.map(o => o.cliHints?.name).filter(Boolean) as string[]);
|
||||
const seen = new Set<string>();
|
||||
for (const op of operations) {
|
||||
for (const alias of op.cliHints?.aliases ?? []) {
|
||||
expect(primaries.has(alias)).toBe(false);
|
||||
expect(seen.has(alias)).toBe(false);
|
||||
seen.add(alias);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('printOpHelp shows the invoked alias name, not the primary (OV10)', () => {
|
||||
const op = operationsByName['add_link'];
|
||||
const lines: string[] = [];
|
||||
const orig = console.log;
|
||||
console.log = (...a: unknown[]) => { lines.push(a.join(' ')); };
|
||||
try {
|
||||
printOpHelp(op, 'link-add');
|
||||
} finally {
|
||||
console.log = orig;
|
||||
}
|
||||
const out = lines.join('\n');
|
||||
expect(out).toContain('Usage: gbrain link-add');
|
||||
expect(out).not.toContain('Usage: gbrain link ');
|
||||
});
|
||||
});
|
||||
@@ -97,19 +97,21 @@ describe('fresh-init brain (post-migration v95) accepts link_source=mentions', (
|
||||
expect(rows.some(r => r.link_source === 'mentions')).toBe(true);
|
||||
});
|
||||
|
||||
test('CHECK still rejects an unknown source value (widening did not nullify the gate)', async () => {
|
||||
test('CHECK still rejects a malformed source value (gate not nullified)', async () => {
|
||||
const slugA = `bad-source-a-${Math.random().toString(36).slice(2, 8)}`;
|
||||
const slugB = `bad-source-b-${Math.random().toString(36).slice(2, 8)}`;
|
||||
await engine.putPage(slugA, { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
|
||||
await engine.putPage(slugB, { type: 'person', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
|
||||
// 'inferred' is NOT in allow-list ∪ {'mentions'} — must reject.
|
||||
// v114 (#1941): the closed allowlist became a kebab-case regex, so a once-
|
||||
// illegal-but-kebab value like 'inferred' is now ACCEPTED. The gate still
|
||||
// bites on MALFORMED values — 'Inferred_X' has an uppercase + underscore.
|
||||
await expect(
|
||||
engine.addLinksBatch([
|
||||
{
|
||||
from_slug: slugA,
|
||||
to_slug: slugB,
|
||||
link_type: 'mentions',
|
||||
link_source: 'inferred' as any,
|
||||
link_source: 'Inferred_X' as any,
|
||||
context: 'should reject',
|
||||
},
|
||||
]),
|
||||
|
||||
@@ -10,6 +10,9 @@
|
||||
* - refresh-failure past the threshold fails SAFE (exits non-zero) (F1A)
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach, spyOn } from 'bun:test';
|
||||
import { existsSync, unlinkSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { tryAcquireDbLock } from '../src/core/db-lock.ts';
|
||||
import { MinionSupervisor, ExitCodes, supervisorLockId, classifySupervisorSingleton } from '../src/core/minions/supervisor.ts';
|
||||
@@ -32,12 +35,16 @@ beforeEach(async () => {
|
||||
});
|
||||
|
||||
describe('#1849 supervisorLockId', () => {
|
||||
test('keys on DB identity AND queue', () => {
|
||||
expect(supervisorLockId('default', 'postgres://x')).toBe('gbrain-supervisor:postgres://x:default');
|
||||
expect(supervisorLockId('shell', 'postgres://x')).toBe('gbrain-supervisor:postgres://x:shell');
|
||||
// Different DB identity → different lock even for the same queue.
|
||||
expect(supervisorLockId('default', 'postgres://a'))
|
||||
.not.toBe(supervisorLockId('default', 'postgres://b'));
|
||||
test('keys on queue ONLY (DB scoping is physical — the lock row lives in the DB)', () => {
|
||||
expect(supervisorLockId('default')).toBe('gbrain-supervisor:default');
|
||||
expect(supervisorLockId('shell')).toBe('gbrain-supervisor:shell');
|
||||
// Different queues → different locks.
|
||||
expect(supervisorLockId('default')).not.toBe(supervisorLockId('shell'));
|
||||
// Regression (the bug this fixes): the id must NOT depend on how the same
|
||||
// physical DB was addressed. Two supervisors on one DB via different URLs
|
||||
// must compute the SAME id so they collide on the one shared locks table.
|
||||
// The function takes no DB-identity arg precisely so it can't diverge.
|
||||
expect(supervisorLockId.length).toBe(1);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -75,7 +82,7 @@ describe('#1849 classifySupervisorSingleton (doctor)', () => {
|
||||
|
||||
describe('#1849 DB lock is the real singleton', () => {
|
||||
test('second acquire of the same (db, queue) lock is refused', async () => {
|
||||
const id = supervisorLockId('default', 'pglite:test');
|
||||
const id = supervisorLockId('default');
|
||||
const first = await tryAcquireDbLock(engine, id, 5);
|
||||
expect(first).not.toBeNull();
|
||||
// A second supervisor (different pidfile, same db+queue) gets null → exit 2.
|
||||
@@ -89,8 +96,8 @@ describe('#1849 DB lock is the real singleton', () => {
|
||||
});
|
||||
|
||||
test('different queues on the same DB do not collide', async () => {
|
||||
const a = await tryAcquireDbLock(engine, supervisorLockId('default', 'pglite:test'), 5);
|
||||
const b = await tryAcquireDbLock(engine, supervisorLockId('shell', 'pglite:test'), 5);
|
||||
const a = await tryAcquireDbLock(engine, supervisorLockId('default'), 5);
|
||||
const b = await tryAcquireDbLock(engine, supervisorLockId('shell'), 5);
|
||||
expect(a).not.toBeNull();
|
||||
expect(b).not.toBeNull();
|
||||
await a!.release();
|
||||
@@ -98,6 +105,48 @@ describe('#1849 DB lock is the real singleton', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('#1849 LOCK_HELD path does not strand the pidfile', () => {
|
||||
test('the pidfile-cleanup exit listener is installed BEFORE the DB-lock acquire', async () => {
|
||||
// Supervisor A already holds the queue lock.
|
||||
const holderA = await tryAcquireDbLock(engine, supervisorLockId('default'), 5);
|
||||
expect(holderA).not.toBeNull();
|
||||
|
||||
const pidFile = join(tmpdir(), `gbrain-sup-stranded-${process.pid}-${Math.random().toString(36).slice(2)}.pid`);
|
||||
const sup = new MinionSupervisor(engine, { cliPath: '/bin/sh', healthInterval: 0, json: true, pidFile });
|
||||
|
||||
// Capture the 'exit' listener start() registers (if any) and stop execution
|
||||
// at the first process.exit (the LOCK_HELD path) the way the real exit would.
|
||||
let exitListener: ((...a: unknown[]) => void) | null = null;
|
||||
const onSpy = spyOn(process, 'on').mockImplementation(((event: string, cb: (...a: unknown[]) => void) => {
|
||||
if (event === 'exit') exitListener = cb;
|
||||
return process;
|
||||
}) as never);
|
||||
const exitSpy = spyOn(process, 'exit').mockImplementation(((code?: number) => {
|
||||
throw new Error(`exit:${code}`);
|
||||
}) as never);
|
||||
|
||||
try {
|
||||
try { await sup.start(); } catch { /* exit stub throws at LOCK_HELD */ }
|
||||
|
||||
expect(exitSpy).toHaveBeenCalledWith(ExitCodes.LOCK_HELD);
|
||||
// The bug: the exit listener was registered AFTER the DB-lock exit, so
|
||||
// start() threw before reaching it and the pidfile this process created
|
||||
// is stranded. The fix installs it first → it's captured here.
|
||||
expect(exitListener).not.toBeNull();
|
||||
// And it actually cleans up the pidfile we created (contents match our pid).
|
||||
expect(existsSync(pidFile)).toBe(true);
|
||||
exitListener!();
|
||||
expect(existsSync(pidFile)).toBe(false);
|
||||
} finally {
|
||||
onSpy.mockRestore();
|
||||
exitSpy.mockRestore();
|
||||
if (existsSync(pidFile)) unlinkSync(pidFile);
|
||||
await holderA!.release();
|
||||
}
|
||||
});
|
||||
|
||||
});
|
||||
|
||||
describe('#1849 refresh-failure fails safe (F1A)', () => {
|
||||
test('exits LOCK_LOST after the failure threshold; tolerates a single blip', async () => {
|
||||
const sup = new MinionSupervisor(engine, { cliPath: '/bin/sh', healthInterval: 0, json: true });
|
||||
|
||||
Reference in New Issue
Block a user