Merge remote-tracking branch 'origin/master' into garrytan/fix-issue-1939

# Conflicts:
#	CHANGELOG.md
#	VERSION
#	docs/architecture/KEY_FILES.md
#	package.json
This commit is contained in:
Garry Tan
2026-06-07 19:15:02 -07:00
27 changed files with 1166 additions and 105 deletions
+21
View File
@@ -20,6 +20,27 @@ Two fixes. First, a non-string title/slug/type now coerces deterministically at
### To take advantage of v0.42.32.0
Upgrade and run `gbrain sync` once. Any pages that previously wedged the importer (bare date/number titles) now import on their own. If a file still genuinely can't parse, sync tells you which one; fix it, or let the auto-skip valve move past it after a few runs and watch for it in `gbrain doctor`. Tune the threshold with `GBRAIN_SYNC_AUTOSKIP_AFTER` (set `0` to keep the strict fail-closed behavior).
## [0.42.31.0] - 2026-06-07
**You can now write typed graph edges with your own provenance straight from the CLI — `gbrain link-add a b --link-type relies-on --link-source citation-graph` — and an external edge-writer (a citation-graph ingester, an importer, a classifier) no longer needs a gbrain schema migration to register a new provenance.** Two ergonomics gaps for tools that compute edges out-of-band, filed by a downstream agent building a citation-graph ingester (#1941).
Before this, `link_source` was a closed allowlist: anything outside `markdown`/`frontmatter`/`manual`/`mentions`/`wikilink-resolved` was rejected by a CHECK constraint, so a deriver had to either patch gbrain's schema or stamp its machine-derived edges `manual` — which made them indistinguishable from hand-entered ones. `link_source` is now an open, format-validated provenance: any lowercase kebab-case tag up to 64 chars (`citation-graph`, `relies-on-graph`, your-tag) is valid, no migration needed. The format gate still rejects garbage (uppercase, spaces, underscores, leading/trailing/double dashes).
The CLI gap is closed too. `gbrain link` / `gbrain unlink` now take `--link-source` and `--link-type`, with `link-add` / `link-rm` aliases for discoverability. A new `gbrain link-sources` lists the distinct provenances a brain carries (with counts) — the read-side replacement for the discoverability the old allowlist gave you for free. CLI-created edges now record `manual` provenance by default instead of masquerading as parsed-from-`markdown`, and the reconciliation-managed provenances stay reserved for the writers that own their semantics.
### Added
- **`gbrain link-add` / `link-rm` / `link-sources`** plus `--link-source` and `--link-type` on the link ops. Write typed, provenance-tagged, source-scoped edges from the CLI and list which provenances a brain holds. Provenance is any kebab-case tag; removals can filter by provenance so machine-derived edges delete cleanly without touching hand-entered ones. (#1941)
### Changed
- **`link_source` is an open kebab-case provenance, not a closed allowlist (migration v114).** External edge-writers register a new provenance with no gbrain migration. Existing provenances are unaffected; the migration is lock-friendly on Postgres (validates without blocking writes) and applies automatically on upgrade. CLI-created links now default to `manual` provenance.
### Fixed
- **Supervisor queue-singleton hardening (follow-up to #1849).** Two supervisors pointed at the same database via different-but-equivalent connection strings could each acquire the "one per queue" lock; the lock is now keyed on the queue alone (its row already lives in the target database), so same-database + same-queue collides correctly. A supervisor that loses the lock race on startup also no longer leaves its pidfile behind to block the next start.
### To take advantage of v0.42.31.0
`gbrain upgrade`. The constraint migration runs automatically. To write edges from a tool or the CLI: `gbrain link-add <from> <to> --link-type <verb> --link-source <your-tag>`; `gbrain link-sources` shows what's in the graph; `gbrain link-rm <from> <to> --link-source <your-tag>` removes only that provenance's edges.
## [0.42.29.0] - 2026-06-07
**The background-job queue stops thrashing on long jobs, the cycle stops wedging itself, and you can no longer run two supervisors against one queue by accident.** Three fixes plus a voice-agent feature.
File diff suppressed because one or more lines are too long
@@ -0,0 +1,211 @@
---
title: "feat: Add idea-lineage thinking skill"
type: feat
status: completed
date: 2026-06-03
---
# feat: Add idea-lineage thinking skill
## Summary
Add an `idea-lineage` thinking skill that traces how one idea has evolved through a user's brain: first mention, best articulation, related concepts, reversals, contradictions, abandoned branches, and the current live version. The contribution should start as a read-only skill with routing and conformance coverage, not as a new CLI or MCP operation.
## Problem Frame
GBrain already has two adjacent capabilities that are easy to conflate with this feature:
- `skills/concept-synthesis/SKILL.md` is a mutating, batch-oriented concept map builder. It deduplicates many concept stubs, tiers them, writes concept pages, and creates an intellectual universe.
- `find_trajectory` and `gbrain eval trajectory` are structured entity trajectories over typed facts and events. They work best for questions like metric history, founder consistency, role/status changes, and event timelines.
`idea-lineage` should occupy the narrow space between them: a query-time, single-idea, citation-backed synthesis of conceptual evolution. It should help a user ask "how has my thinking about this idea changed?" without running a global concept-synthesis job or forcing the idea into an entity/metric trajectory model.
## Requirements
**Behavior**
- R1. The skill accepts a single idea, topic, concept phrase, or nearby concept page and produces a focused lineage for that idea only.
- R2. The output identifies first mention, best articulation, related concepts, reversals, contradictions, abandoned branches, and current live version when evidence supports each category.
- R3. Every lineage claim is grounded in existing brain evidence: page links, dates, verbatim snippets, timeline entries, takes, contradiction findings, or trajectory points when applicable.
- R4. The skill distinguishes evidence strength. Missing or weak evidence should be reported as a gap, not filled with plausible narrative.
- R5. The default workflow is read-only and does not write or mutate brain pages.
**Routing**
- R6. Routing should prefer `idea-lineage` for single-idea evolution requests such as "how has my thinking about X changed?".
- R7. Routing should keep broad corpus/map requests on `concept-synthesis`.
- R8. Routing should keep structured entity metric/status questions on `find_trajectory`, `gbrain eval trajectory`, or `gbrain think` trajectory injection.
**Privacy and portability**
- R9. The skill and fixtures must use public, generic examples only.
- R10. The plan and implementation must avoid private fork names, real people, real companies, funds, or host-specific filesystem paths in public artifacts.
## Scope Boundaries
### In Scope
- A new bundled skill under `skills/idea-lineage/`.
- Resolver, manifest, and plugin-bundle wiring.
- Routing fixtures that prove the new intent is reachable and does not swallow `concept-synthesis` or trajectory-shaped prompts.
- Documentation inside the skill body that explains when to use `search`, `query`, `get_page`, `list_pages`, `takes_search`, `find_contradictions`, and optionally `find_trajectory`.
- Focused conformance, resolver, and routing verification.
### Deferred to Follow-Up Work
- A first-class `idea_lineage` MCP operation.
- A `gbrain idea lineage <query>` CLI.
- Persisting lineage reports back into the brain.
- New database tables, schema-pack fields, or concept lineage graph primitives.
- Automated contradiction-probe reruns. The skill should read cached contradiction findings if available, not trigger expensive probes.
### Outside This Contribution
- Replacing `concept-synthesis`.
- Changing the facts/takes epistemology model.
- Changing `find_trajectory`'s entity-slug contract.
- Implementing the broader taxonomy redesign tracked by issue #1668.
## Key Technical Decisions
- **Start as a markdown skill:** GBrain's architecture treats skills as fat markdown workflows. This feature can be useful by orchestrating existing read operations, so a CLI/MCP surface would add contract weight before the behavior is proven.
- **Make the skill non-mutating by default:** The user intent is investigative. Writing lineage pages should remain a later explicit mode after routing and output quality are established.
- **Use evidence buckets rather than a single narrative pass:** The output should force the agent to separately evaluate first mention, articulation, current version, reversals, contradictions, and abandoned branches. That reduces the risk of smoothing over conflict.
- **Keep `find_trajectory` as an optional side-channel:** It is valuable when an idea query resolves to an entity attribute or status history, but `idea-lineage` should not depend on typed facts being present.
- **Avoid the existing "trace idea evolution" trigger phrase:** That phrase already routes to `concept-synthesis`; adding it to the new skill would create avoidable resolver ambiguity.
## High-Level Technical Design
```mermaid
flowchart TB
A["User asks about one idea"] --> B{"Intent shape"}
B -->|"whole corpus / map"| C["concept-synthesis"]
B -->|"entity metric / status over time"| D["trajectory surfaces"]
B -->|"single conceptual idea"| E["idea-lineage skill"]
E --> F["Resolve idea candidates"]
F --> G["Gather evidence via search/query/pages/takes"]
G --> H["Classify lineage moments"]
H --> I["Synthesize cited answer with confidence gaps"]
```
## Implementation Units
### U1. Add the `idea-lineage` Skill
- **Goal:** Create the read-only skill contract and workflow.
- **Requirements:** R1, R2, R3, R4, R5, R9, R10
- **Dependencies:** None
- **Files:**
- `skills/idea-lineage/SKILL.md`
- `test/skills-conformance.test.ts`
- **Approach:** Create a new skill with required frontmatter and conformance sections. The skill should define its workflow in phases: clarify the target idea, resolve likely concept/page anchors, collect evidence, classify lineage moments, produce a cited synthesis, and state gaps. Frontmatter should set `mutating: false` and list read operations only.
- **Patterns to follow:**
- `skills/strategic-reading/SKILL.md` for a read-only thinking-skill shape with related-skill boundaries.
- `skills/query/SKILL.md` for search/query/get-page guidance.
- `skills/concept-synthesis/SKILL.md` for contrast, not for behavior reuse.
- **Test scenarios:**
- A new `SKILL.md` with frontmatter, `## Contract`, `## Output Format`, and `## Anti-Patterns` passes conformance.
- The frontmatter declares a unique `name: idea-lineage`.
- The skill body references only portable, synthetic examples.
- **Verification:** `bun test test/skills-conformance.test.ts` passes.
### U2. Wire Resolver, Manifest, and Bundle Metadata
- **Goal:** Make the skill discoverable by bundled skill users and resolvable by agents.
- **Requirements:** R6, R7, R8, R9, R10
- **Dependencies:** U1
- **Files:**
- `skills/RESOLVER.md`
- `skills/manifest.json`
- `openclaw.plugin.json`
- `test/resolver.test.ts`
- `test/skillpack-reference.test.ts`
- **Approach:** Add `idea-lineage` to the skill manifest and plugin skill list. Add a resolver row in the thinking or uncategorized section with narrow user phrases such as "how has my thinking about", "trace the lineage of this idea", "what is my current version of", and "show reversals in my thinking about". Keep broad concept-map phrases routed to `concept-synthesis`.
- **Patterns to follow:**
- `skills/RESOLVER.md` rows for `strategic-reading`, `concept-synthesis`, and `perplexity-research`.
- Existing sorted `openclaw.plugin.json` skill list.
- **Test scenarios:**
- Every quoted resolver trigger fuzzy-matches a frontmatter trigger in `skills/idea-lineage/SKILL.md`.
- `idea-lineage` is listed in `skills/manifest.json`.
- `idea-lineage` is listed in `openclaw.plugin.json` if the contribution ships as part of the bundled OpenClaw skillpack.
- Existing skills remain reachable.
- **Verification:** `bun test test/resolver.test.ts` passes.
### U3. Add Routing Eval Fixtures
- **Goal:** Prove the new routing boundary against adjacent skills.
- **Requirements:** R6, R7, R8
- **Dependencies:** U1, U2
- **Files:**
- `skills/idea-lineage/routing-eval.jsonl`
- `skills/concept-synthesis/routing-eval.jsonl`
- `src/core/routing-eval.ts`
- **Approach:** Add positive fixtures for single-idea lineage prompts and negative or ambiguity-declared fixtures around adjacent surfaces. The fixture text should paraphrase triggers rather than copy them exactly, because the routing fixture linter rejects tautological trigger copies.
- **Test scenarios:**
- "Show how my thinking about founder-led sales changed over time" routes to `idea-lineage`.
- "What is my current version of the compounding trust idea?" routes to `idea-lineage`.
- "Synthesize my concepts into a tiered intellectual map" stays on `concept-synthesis`.
- "How has acme-example MRR trended since January?" does not route to `idea-lineage`.
- Negative fixtures avoid false positives for generic "publish this report" or "what is this concept?" prompts.
- **Verification:** `gbrain routing-eval --json` reports no new misses, false positives, or unapproved ambiguity for the added fixtures.
### U4. Add Output Contract and Citation Discipline
- **Goal:** Make the skill's user-facing answer shape predictable and reviewable.
- **Requirements:** R2, R3, R4, R5
- **Dependencies:** U1
- **Files:**
- `skills/idea-lineage/SKILL.md`
- `skills/conventions/quality.md`
- `skills/brain-ops/SKILL.md`
- **Approach:** Define the output format directly in the skill body. The recommended shape should include a compact current answer, evidence timeline, lineage buckets, contradictions/reversals, abandoned branches, related concepts, and confidence gaps. Require page/date/snippet evidence for each non-gap claim. Preserve quote fidelity and avoid hallucinated dates.
- **Patterns to follow:**
- `skills/conventions/quality.md` for citation and quote-fidelity expectations.
- `skills/brain-ops/SKILL.md` for source attribution and source-id formatting.
- `docs/takes-vs-facts.md` for not conflating holder-attributed takes with the brain owner's facts.
- **Test scenarios:**
- Test expectation: none beyond conformance for the markdown-only contract; routing and conformance tests cover the machine-checkable surface.
- **Verification:** Manual review confirms the skill body tells the agent how to cite, label gaps, and separate facts/takes/trajectory evidence.
### U5. Refresh Generated Documentation If Required
- **Goal:** Keep generated LLM-facing docs consistent if the test suite requires it.
- **Requirements:** R9, R10
- **Dependencies:** U1, U2, U3
- **Files:**
- `llms.txt`
- `llms-full.txt`
- `test/build-llms.test.ts`
- **Approach:** Run the build-llms test after adding the skill. If it fails because committed docs are stale, regenerate with the existing generator and include the generated diff. If it passes without regeneration, leave these files unchanged.
- **Patterns to follow:**
- `package.json` script `build:llms`.
- `test/build-llms.test.ts` failure message.
- **Test scenarios:**
- Committed `llms.txt` and `llms-full.txt` match generator output.
- `llms-full.txt` remains within the size budget.
- **Verification:** `bun test test/build-llms.test.ts` passes.
## Acceptance Examples
- AE1. When the user asks "How has my thinking about founder-led sales changed over time?", the agent routes to `idea-lineage`, searches for evidence, and returns a cited lineage rather than running `concept-synthesis`.
- AE2. When the user asks "Run concept synthesis across my notes", the agent routes to `concept-synthesis`, not `idea-lineage`.
- AE3. When the user asks "How did acme-example's MRR trend?", the agent uses trajectory surfaces rather than `idea-lineage`.
- AE4. When the evidence does not support an "abandoned branch" claim, the output includes a gap instead of inventing one.
## Risks & Dependencies
- **Resolver overlap risk:** `concept-synthesis` already uses "trace idea evolution". Mitigate by avoiding that exact trigger and adding routing fixtures around the boundary.
- **Narrative overreach risk:** The feature invites story-making. Mitigate by requiring dates, snippets, links, and explicit gaps for unsupported categories.
- **Privacy risk:** Skill examples can easily drift into real-brain language. Use synthetic examples only and rely on existing privacy checks.
- **Generated-doc churn risk:** Adding a bundled skill may require `llms.txt` and `llms-full.txt` regeneration. Treat generated-doc changes as mechanical and separate from the skill design during review.
- **Future taxonomy dependency:** Issue #1668 may eventually change concept filing and identity. This plan avoids new schema assumptions so the contribution remains compatible with the current repo.
## Sources & Research
- `skills/concept-synthesis/SKILL.md` defines the existing batch, mutating, concept-map surface.
- `skills/RESOLVER.md` and `skills/manifest.json` define current skill reachability and bundle metadata.
- `docs/architecture/lens-packs.md` shows that atoms and concepts are already part of the lens-pack/dream-cycle substrate.
- `docs/proposals/temporal-contradiction-probe.md` and `docs/takes-vs-facts.md` define the temporal and epistemic boundaries this skill must not blur.
- `src/core/operations.ts`, `src/core/trajectory.ts`, `src/commands/eval-trajectory.ts`, and `test/operations-find-trajectory.test.ts` define the current `find_trajectory` contract.
- Pull requests #1131, #1296, and #1364 provide the recent trajectory, think-routing, and lens-pack context.
- Issue #1668 is related future taxonomy work, but not a prerequisite for this contribution.
+1
View File
@@ -1373,6 +1373,7 @@ These apply to ALL brain-writing skills:
| "enrich this article", "enrich brain pages", "batch enrich", "make brain pages useful" | `skills/article-enrichment/SKILL.md` |
| "strategic reading", "read this through the lens of", "apply this to my problem", "what can I learn from this about", "extract a playbook from" | `skills/strategic-reading/SKILL.md` |
| "concept synthesis", "synthesize my concepts", "find patterns across my notes", "build my intellectual map", "trace idea evolution" | `skills/concept-synthesis/SKILL.md` |
| "idea lineage", "trace the lineage of this idea", "how my thinking about", "how has my thinking about", "what is my current version of", "show reversals in my thinking about", "where did this idea come from" | `skills/idea-lineage/SKILL.md` |
| "perplexity research", "what's new about", "current state of", "web research", "what changed about" | `skills/perplexity-research/SKILL.md` |
| "crawl my archive", "find gold in my archive", "archive crawler", "scan my dropbox for", "mine my old files for" | `skills/archive-crawler/SKILL.md` |
| "verify this academic claim", "check this study", "academic verify", "validate citation", "is this study real" | `skills/academic-verify/SKILL.md` |
+1
View File
@@ -47,6 +47,7 @@
"skills/enrich",
"skills/functional-area-resolver",
"skills/idea-ingest",
"skills/idea-lineage",
"skills/ingest",
"skills/maintain",
"skills/media-ingest",
+1 -1
View File
@@ -125,6 +125,7 @@ These apply to ALL brain-writing skills:
| "enrich this article", "enrich brain pages", "batch enrich", "make brain pages useful" | `skills/article-enrichment/SKILL.md` |
| "strategic reading", "read this through the lens of", "apply this to my problem", "what can I learn from this about", "extract a playbook from" | `skills/strategic-reading/SKILL.md` |
| "concept synthesis", "synthesize my concepts", "find patterns across my notes", "build my intellectual map", "trace idea evolution" | `skills/concept-synthesis/SKILL.md` |
| "idea lineage", "trace the lineage of this idea", "how my thinking about", "how has my thinking about", "what is my current version of", "show reversals in my thinking about", "where did this idea come from" | `skills/idea-lineage/SKILL.md` |
| "perplexity research", "what's new about", "current state of", "web research", "what changed about" | `skills/perplexity-research/SKILL.md` |
| "crawl my archive", "find gold in my archive", "archive crawler", "scan my dropbox for", "mine my old files for" | `skills/archive-crawler/SKILL.md` |
| "verify this academic claim", "check this study", "academic verify", "validate citation", "is this study real" | `skills/academic-verify/SKILL.md` |
@@ -132,4 +133,3 @@ These apply to ALL brain-writing skills:
| "voice note", "ingest this voice memo", "transcribe and file", "voice note ingest", "save this audio note" | `skills/voice-note-ingest/SKILL.md` |
| "add a page type", "add a type to my schema", "schema author", "schema mutate", "schema pack add", "my brain has untyped pages", "propose new types from my corpus", "backfill page types", "evolve my schema", "researcher type", "make X an expert type" (dispatcher for: gbrain schema active/list/show/validate/graph/lint/stats/explain/use/downgrade/reload/init/fork/edit/diff/add-type/remove-type/update-type/add-alias/remove-alias/add-prefix/remove-prefix/add-link-type/remove-link-type/set-extractable/set-expert-routing/detect/suggest/review-candidates/review-orphans/sync) | `skills/schema-author/SKILL.md` |
| "unify my types", "migrate to gbrain-base-v2", "94 types to 14", "apply canonical taxonomy", "clean up my page types", "pack upgrade", "shrink type proliferation", "consolidate page types", "retype pages to canonical" (dispatcher for: gbrain onboard --check, gbrain onboard --check --explain, gbrain jobs submit unify-types, gbrain pages restore) | `skills/schema-unify/SKILL.md` |
@@ -5,3 +5,4 @@
{"intent":"Find patterns across my notes and group them into clusters","expected_skill":"concept-synthesis"}
{"intent":"Build my intellectual map — what's canon vs riff","expected_skill":"concept-synthesis"}
{"intent":"Trace idea evolution across years of my reflections","expected_skill":"concept-synthesis"}
{"intent":"Trace idea evolution across years of my reflections and cluster the themes","expected_skill":"concept-synthesis"}
+222
View File
@@ -0,0 +1,222 @@
---
name: idea-lineage
version: 0.1.0
description: |
Trace one idea's evolution through the brain: first mention, best
articulation, related concepts, reversals, contradictions, abandoned
branches, and the current live version. Use for single-idea conceptual
lineage, not broad concept-map synthesis or structured entity metrics.
triggers:
- "idea lineage"
- "trace the lineage of this idea"
- "how my thinking about"
- "how has my thinking about"
- "current version of this idea"
- "what is my current version of"
- "show reversals in my thinking about"
- "where did this idea come from"
tools:
- search
- query
- get_page
- list_pages
- takes_search
- find_contradictions
- find_trajectory
mutating: false
---
# idea-lineage - Single-Idea Evolution Through the Brain
> **Convention:** see [conventions/quality.md](../conventions/quality.md) for
> citation rules, quote fidelity, and source-backed claims.
>
> **Boundary:** see [docs/takes-vs-facts.md](../../docs/takes-vs-facts.md) for
> the distinction between holder-attributed takes and the brain owner's hot
> facts. Do not collapse those layers when summarizing lineage.
## What this solves
Users often want to understand how one idea changed across time: when it first
appeared, when it became sharp, what it displaced, what it contradicted, and
what version is alive now. That is different from building a whole concept map
and different from charting an entity's metric trajectory.
Use this skill when the user asks about one idea, topic, phrase, or concept
page and wants its evolution through the brain.
Canonical examples:
- "Run idea lineage on founder-led sales."
- "How has my thinking about compounding trust changed?"
- "What is my current version of this idea?"
- "Where did this idea come from, and what did I abandon along the way?"
## What this is not
- Not `concept-synthesis`: that skill deduplicates many concept stubs, tiers
them, writes concept pages, and builds a broad intellectual map.
- Not `find_trajectory`: that operation charts typed facts or event rows for
an entity, such as MRR, role, location, or status over time.
- Not a contradiction-probe runner: this skill may read cached contradiction
findings when available, but it does not launch expensive probes.
- Not a writing mode by default: do not write a lineage page unless the user
explicitly asks for a saved artifact after seeing the read-only answer.
## Contract
This skill guarantees:
- A single-idea scope is preserved. Broad corpus or "map my concepts" prompts
route to `skills/concept-synthesis/SKILL.md` instead.
- Every lineage claim cites existing brain evidence: page slug, source id when
present, date, and short quote or snippet.
- Missing evidence is labeled as a gap, not patched with plausible narrative.
- Contradictions, reversals, and abandoned branches are separated from normal
temporal evolution.
- The default mode is read-only and does not mutate brain pages.
## Phases
### Phase 1: Resolve the idea target
1. Restate the idea in one sentence.
2. Search for exact phrase variants with `search`.
3. Run one semantic `query` for the natural-language version.
4. Check `list_pages` for concept pages when the idea has an obvious concept
slug or title.
5. If results point to an entity/metric/status trajectory rather than a concept,
hand off to `find_trajectory` or the normal query/think trajectory path.
If multiple distinct ideas share the same phrase, ask the user to choose the
intended one before synthesizing.
### Phase 2: Gather evidence
Collect enough evidence to support or reject each output bucket:
- Search chunks with dates and source slugs.
- Full pages via `get_page` for the top relevant concept, note, transcript,
meeting, article, or project pages.
- Related concept pages through backlinks, `related` frontmatter, or repeated
co-occurrence in search results.
- Takes via `takes_search` when the idea appears as a belief, bet, hunch, or
attributed claim.
- Cached contradiction findings via `find_contradictions` when the user asks
about inconsistency or the search results show obvious conflict.
- `find_trajectory` only when the evidence is entity/attribute-shaped, such as
a role/status/metric evolution that is relevant to the idea's story.
Prefer fewer high-quality sources over a long unsorted pile. Read full pages
when snippets imply a lineage milestone.
### Phase 3: Classify lineage moments
Classify evidence into these buckets:
1. **First mention** - earliest dated evidence where the idea appears.
2. **Best articulation** - the clearest or most complete expression, not
necessarily the newest.
3. **Current live version** - the most recent high-authority version that still
appears active.
4. **Reversals** - places where the user's stance changed direction.
5. **Contradictions** - claims that cannot both be true at the same time or
under the same assumptions. Distinguish these from legitimate temporal
supersession.
6. **Abandoned branches** - promising variants that appear and then disappear,
lose support, or are explicitly rejected.
7. **Related concepts** - nearby ideas that shaped or inherited part of the
original idea.
When a bucket has no evidence, write "No clear evidence found" with a brief note
about what was checked.
### Phase 4: Synthesize the lineage
Write the answer in the output format below. Keep the synthesis proportional to
the evidence. Do not overfit a smooth evolution if the evidence is sparse,
messy, or contradictory.
### Phase 5: Suggest optional next action
If useful, offer one concrete follow-up:
- Save the lineage as a brain page.
- Run broad `concept-synthesis` if the user actually wants the whole concept
map refreshed.
- Run or inspect trajectory data if the idea turned out to depend on structured
entity facts.
- Run a contradiction probe only when stale cached findings are insufficient
and the user explicitly wants that heavier pass.
## Output Format
Use this shape for normal answers:
```markdown
## Current Live Version
[1-3 sentences. Include confidence: high / medium / low.]
## Lineage
- First mention: [date] - [claim] ([source-id:slug], "short quote")
- Best articulation: [date] - [claim] ([source-id:slug], "short quote")
- Turning point: [date] - [what changed] ([source-id:slug])
## Reversals and Contradictions
- Reversal: [what changed, with before/after evidence]
- Contradiction: [what conflicts, or "No clear evidence found"]
## Abandoned Branches
- [branch] - [why it appears abandoned, with evidence]
## Related Concepts
- [concept slug or title] - [relationship]
## Evidence Gaps
- [bucket or claim] - [what was checked and what is missing]
```
For short answers, collapse sections, but keep the same distinctions. Always
cite the source for each non-gap claim.
## Quality Rules
- Quote exact text when naming first mention or best articulation.
- Include dates when the source has dates. If no date is available, say
"undated" rather than guessing.
- Treat the user's direct statements as highest authority for the user's own
current view.
- Treat holder-attributed takes as beliefs by that holder, not automatically
as facts about the world or the brain owner.
- Mark confidence low when evidence comes from a single weak snippet, an
undated page, or a fuzzy semantic match.
- Preserve source ids in citations when search or page payloads include them.
## Anti-Patterns
- Running `concept-synthesis` for a single-idea question.
- Presenting an entity's MRR, ARR, role, or status trajectory as conceptual
lineage without explaining the distinction.
- Treating normal temporal evolution as contradiction.
- Inventing abandoned branches because the story would be more interesting.
- Saving or rewriting brain pages without explicit user instruction.
- Using real names, companies, funds, or fork-specific examples in bundled
fixtures or documentation.
## Related Skills and Operations
- `skills/concept-synthesis/SKILL.md` - broad mutating concept-map synthesis.
- `skills/query/SKILL.md` - general brain search and cited answers.
- `skills/brain-ops/SKILL.md` - source attribution and brain-first behavior.
- `find_trajectory` - structured typed-fact and event timelines for entities.
- `find_contradictions` - cached suspected contradiction findings.
## Tools Used
- `search` - keyword search for exact phrase variants and dated mentions.
- `query` - semantic search for conceptual matches.
- `get_page` - full context for candidate source pages.
- `list_pages` - concept-page discovery and scoped page enumeration.
- `takes_search` - holder-attributed beliefs, bets, hunches, and facts.
- `find_contradictions` - cached contradiction findings when relevant.
- `find_trajectory` - optional structured entity trajectory side-channel.
+10
View File
@@ -0,0 +1,10 @@
// Routing eval fixtures for skills/idea-lineage. Positive cases exercise
// single-idea conceptual lineage. Negative cases protect adjacent
// concept-synthesis and trajectory surfaces.
{"intent":"Run idea lineage on founder-led sales and show the earliest version","expected_skill":"idea-lineage"}
{"intent":"Show how my thinking about compounding trust changed over time","expected_skill":"idea-lineage"}
{"intent":"What is my current version of the invisible college idea?","expected_skill":"idea-lineage"}
{"intent":"Where did this idea come from in my notes, and what did I abandon?","expected_skill":"idea-lineage"}
{"intent":"Show reversals in my thinking about founder-led sales","expected_skill":"idea-lineage"}
{"intent":"How has acme-example MRR trended since January?","expected_skill":null}
{"intent":"Build my intellectual map across all my recurring frameworks","expected_skill":"concept-synthesis"}
+5
View File
@@ -194,6 +194,11 @@
"path": "concept-synthesis/SKILL.md",
"description": "Deduplicate and synthesize raw concept stubs into a tiered intellectual map (T1 Canon to T4 Riff), tracing idea evolution across sources over time."
},
{
"name": "idea-lineage",
"path": "idea-lineage/SKILL.md",
"description": "Trace one idea's evolution through the brain: first mention, best articulation, reversals, contradictions, abandoned branches, related concepts, and current live version."
},
{
"name": "perplexity-research",
"path": "perplexity-research/SKILL.md",
+41 -12
View File
@@ -94,6 +94,25 @@ const CLI_ONLY_SELF_HELP = new Set([
'connect',
]);
// v114 (#1941): alias -> operation lookup, kept separate from `cliOps` so
// aliases don't double-list in printHelp's auto-generated section. Collisions
// with a primary CLI name, a CLI_ONLY command, or another alias throw at module
// load — a silent route-shadow is worse than a loud boot failure. Placed after
// CLI_ONLY so the collision check can see it.
export const cliAliases = new Map<string, Operation>();
for (const op of operations) {
if (op.cliHints?.hidden) continue;
for (const alias of op.cliHints?.aliases ?? []) {
if (cliOps.has(alias) || CLI_ONLY.has(alias) || cliAliases.has(alias)) {
throw new Error(
`CLI alias collision: '${alias}' (op '${op.name}') conflicts with an existing ` +
`command or alias. Rename the alias in src/core/operations.ts.`,
);
}
cliAliases.set(alias, op);
}
}
// v0.42 self-upgrade: commands that must NOT trigger the startup update-check
// (they ARE the update path, or are trivial/no-DB) and which set
// GBRAIN_SKIP_STARTUP_HOOKS for any children they spawn.
@@ -248,9 +267,9 @@ async function main() {
// Per-command --help
if (hasHelpFlag(subArgs)) {
const op = cliOps.get(command);
const op = cliOps.get(command) ?? cliAliases.get(command);
if (op) {
printOpHelp(op);
printOpHelp(op, command);
return;
}
if (CLI_ONLY.has(command) && !CLI_ONLY_SELF_HELP.has(command)) {
@@ -265,8 +284,8 @@ async function main() {
return;
}
// Shared operations
const op = cliOps.get(command);
// Shared operations (fall through to aliases, e.g. link-add -> add_link)
const op = cliOps.get(command) ?? cliAliases.get(command);
if (!op) {
console.error(`Unknown command: ${command}`);
console.error('Run gbrain --help for available commands.');
@@ -2058,9 +2077,11 @@ async function connectEngine(opts?: { probeOnly?: boolean }): Promise<BrainEngin
return engine;
}
function printOpHelp(op: Operation) {
export function printOpHelp(op: Operation, invokedName?: string) {
const positional = (op.cliHints?.positional || []).map(p => `<${p}>`).join(' ');
const name = op.cliHints?.name || op.name;
// v114 (#1941): when invoked via an alias (e.g. `gbrain link-add --help`),
// show the alias the user typed, not the primary op name.
const name = invokedName || op.cliHints?.name || op.name;
console.log(`Usage: gbrain ${name} ${positional} [options]\n`);
console.log(op.description + '\n');
const entries = Object.entries(op.params);
@@ -2125,8 +2146,11 @@ EMBEDDINGS
embed [<slug>|--all|--stale] Generate/refresh embeddings
LINKS
link <from> <to> [--type T] Create typed link
unlink <from> <to> Remove link
link <from> <to> Create typed link (alias: link-add)
[--link-type T] [--link-source S] provenance defaults to 'manual'
unlink <from> <to> Remove link (alias: link-rm)
[--link-type T] [--link-source S] filter which edges to remove
link-sources List provenances in use, with edge counts
backlinks <slug> Incoming links
graph <slug> [--depth N] Traverse link graph (returns nodes)
graph-query <slug> [--type T] Edge-based traversal with type/direction filters
@@ -2222,7 +2246,12 @@ Run gbrain <command> --help for command-specific help.
`);
}
main().catch(e => {
console.error(e.message || e);
process.exit(1);
});
// Only auto-run when invoked as the entry point (the compiled binary or
// `bun src/cli.ts`). Guarded so tests can import cliAliases / printOpHelp
// without triggering argv parsing + main(). v114 (#1941).
if (import.meta.main) {
main().catch(e => {
console.error(e.message || e);
process.exit(1);
});
}
+20 -4
View File
@@ -127,10 +127,16 @@ export interface LinkBatchInput {
link_type?: string;
context?: string;
/**
* Provenance (v0.13+). Pass 'frontmatter' for edges derived from YAML
* frontmatter, 'markdown' for [Name](path) refs, 'manual' for user-created.
* NULL means "legacy / unknown" and is only used by pre-v0.13 rows; new
* writes should always set this. Missing on input defaults to 'markdown'.
* Provenance (v0.13+; opened to kebab tags in v114 / #1941). Any lowercase
* kebab-case value <=64 chars is DB-valid (CHECK `^[a-z][a-z0-9]*(-[a-z0-9]+)*$`),
* so external derivers stamp their own tag (e.g. 'citation-graph'). The
* reconciliation-managed built-ins are 'markdown' ([Name](path) refs),
* 'frontmatter' (YAML-derived, see origin_*), 'mentions', 'wikilink-resolved';
* 'manual' is for user/tool-created edges. NULL = legacy/unknown (pre-v0.13).
* Missing on this batch input defaults to 'markdown'. NOTE: the add_link OP
* (not this engine method) forbids callers from passing the four managed
* built-ins and defaults omitted to 'manual' — internal callers use the
* engine directly and keep writing the managed values.
*/
link_source?: string;
/** For link_source='frontmatter': slug of the page whose frontmatter created this edge. */
@@ -1137,6 +1143,16 @@ export interface BrainEngine {
* applied to the to-page side of the join.
*/
getBacklinks(slug: string, opts?: { sourceId?: string }): Promise<Link[]>;
/**
* v114 (#1941): distinct link_source provenances with edge counts, for
* `gbrain link-sources`. Source-scoped via `{sourceId?, sourceIds?}` (both
* forms, so federated `allowedSources` reads don't leak cross-source counts).
* Deterministic order `count DESC, link_source ASC NULLS LAST` for PG/PGLite
* parity. `link_source` may be NULL (legacy/unknown rows).
*/
listLinkSources(
opts?: { sourceId?: string; sourceIds?: string[] },
): Promise<{ link_source: string | null; count: number }[]>;
/**
* Fuzzy-match a display name to a page slug using pg_trgm similarity.
* Zero embedding cost, zero LLM cost — designed for the v0.13 resolver used
+40
View File
@@ -5120,6 +5120,46 @@ export const MIGRATIONS: Migration[] = [
`,
},
},
{
version: 114,
name: 'links_link_source_check_kebab_regex',
// Issue #1941: open link_source from a closed allowlist to a kebab-case
// format gate so external derivers (e.g. 'citation-graph') stamp their own
// provenance without a per-deriver gbrain migration. Format: lowercase
// kebab `^[a-z][a-z0-9]*(-[a-z0-9]+)*$` (rejects UPPER, leading digit/dash,
// trailing/double dash, underscore, space) + char_length <= 64 cap on the
// indexed free-text column. The five prior built-ins all satisfy the regex,
// so existing rows pass `VALIDATE` and the constraint swap never fails.
//
// DELIBERATELY diverges from the v95/v113 plain DROP+ADD pattern: on real
// Postgres a plain `ADD CONSTRAINT ... CHECK` takes ACCESS EXCLUSIVE + a
// full-table validation scan, which can stall writes on a large `links`
// table. The postgres branch instead does `ADD ... NOT VALID` (instant,
// no scan) then `VALIDATE CONSTRAINT` (scans under SHARE UPDATE EXCLUSIVE,
// does not block reads/writes). That two-phase form requires running
// OUTSIDE a transaction → `transaction: false`. PGLite (single-writer WASM,
// no lock concern) keeps the plain one-shot DROP+ADD, and is the branch the
// schema-version hash reads (pglite-engine.ts).
//
// Idempotent via DROP ... IF EXISTS; no-ops on installs that never created
// the constraint and safe to re-run.
idempotent: true,
transaction: false,
sql: '', // engine-specific via sqlFor (postgres two-phase vs pglite one-shot)
sqlFor: {
postgres: `
ALTER TABLE links DROP CONSTRAINT IF EXISTS links_link_source_check;
ALTER TABLE links ADD CONSTRAINT links_link_source_check
CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)) NOT VALID;
ALTER TABLE links VALIDATE CONSTRAINT links_link_source_check;
`,
pglite: `
ALTER TABLE links DROP CONSTRAINT IF EXISTS links_link_source_check;
ALTER TABLE links ADD CONSTRAINT links_link_source_check
CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64));
`,
},
},
];
export const LATEST_VERSION = MIGRATIONS.length > 0
+38 -25
View File
@@ -44,7 +44,7 @@ import {
import { dirname } from 'path';
import type { BrainEngine } from '../engine.ts';
import { tryAcquireDbLock, type DbLockHandle } from '../db-lock.ts';
import { currentDbIdentity, currentBrainId } from './worker-registry.ts';
import { currentBrainId } from './worker-registry.ts';
export type SupervisorEvent =
| 'started'
@@ -298,14 +298,20 @@ const SUPERVISOR_LOCK_REFRESH_MS = 60_000;
const SUPERVISOR_LOCK_REFRESH_MAX_FAILURES = 3; // 3 × 60s = 180s < 5min TTL
/**
* #1849: the queue-scoped supervisor singleton lock id. Keyed on the raw DB
* identity (T2) + queue so the mutex domain is the (database, queue) pair —
* not the pidfile path. Exported so `gbrain doctor` queries the same row to
* surface the holder + effective --max-rss. Pass an explicit dbIdentity
* (defaults to `currentDbIdentity()`, which reads config without a DB connect).
* #1849: the queue-scoped supervisor singleton lock id. Keyed ONLY on the
* queue, because the lock ROW lives inside the target database — the (database)
* half of the mutex domain is physical, not part of the key. Keying on a
* config-derived DB identity (the prior `currentDbIdentity()`) was a bug: two
* supervisors pointed at the SAME physical database via different-but-equivalent
* URLs/config paths (pooler vs direct port, host alias, trailing params) hashed
* to different ids and BOTH acquired the "singleton" lock in the one shared
* locks table. Queue-only keying makes same-DB + same-queue collide correctly,
* while different physical databases never collide (separate locks tables).
* Exported so `gbrain doctor` queries the same row to surface the holder +
* effective --max-rss.
*/
export function supervisorLockId(queue: string, dbIdentity: string = currentDbIdentity()): string {
return `gbrain-supervisor:${dbIdentity}:${queue}`;
export function supervisorLockId(queue: string): string {
return `gbrain-supervisor:${queue}`;
}
/**
@@ -495,24 +501,13 @@ export class MinionSupervisor {
process.exit(ExitCodes.PID_UNWRITABLE);
}
// 1b. #1849: queue-scoped DB singleton lock — the REAL authority. A second
// supervisor with a different $HOME / --pid-file passes the pidfile check
// above but loses here, so it can't run a conflicting --max-rss worker on
// the same (db, queue). Keyed on the raw DB identity (not the lossy
// currentBrainId hash) per T2.
this.dbLock = await tryAcquireDbLock(this.engine, this.supervisorLockId(), SUPERVISOR_LOCK_TTL_MIN);
if (!this.dbLock) {
console.error(
`Supervisor already running for queue '${this.opts.queue}' on this database ` +
`(another supervisor holds the queue lock, regardless of pidfile path). Exiting.`,
);
process.exit(ExitCodes.LOCK_HELD);
}
// Refresh the lock on its own timer (independent of healthInterval, which
// can be 0/disabled) so the TTL never lapses while we're alive.
this.lockRefreshTimer = setInterval(() => { void this.refreshDbLock(); }, SUPERVISOR_LOCK_REFRESH_MS);
// 2. Cleanup on process exit (covers any exit path including process.exit).
// Installed BEFORE the DB-lock acquisition below: acquirePidLock just
// wrote OUR pid into the pidfile, so any early `process.exit` after this
// point (notably the LOCK_HELD path in 1b) MUST clean it up or it leaves
// a stale pidfile that blocks the next start on this path. The listener
// only unlinks when the file still holds our pid, so it's a no-op on the
// 'held'/'unwritable' paths above (those never created our pidfile).
this.exitListener = () => {
try {
if (existsSync(this.opts.pidFile)) {
@@ -525,6 +520,24 @@ export class MinionSupervisor {
};
process.on('exit', this.exitListener);
// 1b. #1849: queue-scoped DB singleton lock — the REAL authority. A second
// supervisor with a different $HOME / --pid-file passes the pidfile check
// above but loses here, so it can't run a conflicting --max-rss worker on
// the same (db, queue). Keyed on the queue alone; the database half of the
// mutex is physical (the lock row lives in this DB).
this.dbLock = await tryAcquireDbLock(this.engine, this.supervisorLockId(), SUPERVISOR_LOCK_TTL_MIN);
if (!this.dbLock) {
console.error(
`Supervisor already running for queue '${this.opts.queue}' on this database ` +
`(another supervisor holds the queue lock, regardless of pidfile path). Exiting.`,
);
// The exit listener installed above removes the pidfile we just created.
process.exit(ExitCodes.LOCK_HELD);
}
// Refresh the lock on its own timer (independent of healthInterval, which
// can be 0/disabled) so the TTL never lapses while we're alive.
this.lockRefreshTimer = setInterval(() => { void this.refreshDbLock(); }, SUPERVISOR_LOCK_REFRESH_MS);
// 3. Signal handlers (tracked refs; removed on shutdown for test lifecycle hygiene).
this.sigtermListener = () => { void this.shutdown('SIGTERM', ExitCodes.CLEAN); };
this.sigintListener = () => { void this.shutdown('SIGINT', ExitCodes.CLEAN); };
@@ -54,6 +54,10 @@ export const BRAIN_TOOL_ALLOWLIST: ReadonlySet<string> = new Set([
'file_url',
'get_backlinks',
'traverse_graph',
// v114 (#1941): read-only provenance discovery. Edge-WRITE ops (add_link /
// remove_link) are deliberately NOT allowlisted — exposing graph writes to
// subagents is a separate trust decision.
'list_link_sources',
'resolve_slugs',
'get_ingest_log',
'put_page',
@@ -89,6 +93,7 @@ export const BRAIN_TOOL_USAGE_HINTS: Readonly<Record<string, string>> = {
file_url: 'Get a presigned URL for a brain-stored file. Read-only; expires.',
get_backlinks: 'List every page that links TO the given slug. Use for "what references this".',
traverse_graph: 'Walk the typed-edge graph starting from a slug (e.g. `works_at`, `founded`, `invested_in`). Use for relationship queries.',
list_link_sources: 'List the distinct link provenances in the brain with edge counts (e.g. `citation-graph`, `manual`). Use to discover which edge-writers have populated the graph.',
resolve_slugs: 'Resolve free-form entity names to canonical slugs (e.g. "Alice" → `people/alice-example`). Use before any tool that takes a slug if the user gave a name not a slug.',
get_ingest_log: 'Read the brain ingestion log for diagnostic / verification queries.',
put_page: 'Write a markdown page to the gbrain DATABASE (NOT the local filesystem). Page becomes searchable + linkable. Slug must match the agent\'s allowed namespace.',
-18
View File
@@ -75,24 +75,6 @@ export function currentBrainId(): string {
}
}
/**
* The RAW database identity string (url or path), unhashed. Used as the
* authoritative key for the #1849 queue-scoped supervisor singleton DB lock:
* the protected resource is the (database, queue) pair, so the lock id must
* key on the real DB identity, not the lossy djb2 of {@link currentBrainId}
* (T2 — removes any hash-collision question). Reads config only (no DB
* connection), so it's safe to call before the engine connects. Falls back
* to 'default' so two unconfigured brains under one HOME still serialize.
*/
export function currentDbIdentity(): string {
try {
const cfg = loadConfig();
return cfg?.database_url ?? cfg?.database_path ?? 'default';
} catch {
return 'default';
}
}
function entryPath(pid: number): string {
return join(workerRegistryDir(), `worker-${pid}.json`);
}
+57 -5
View File
@@ -498,6 +498,12 @@ export interface Operation {
localOnly?: boolean;
cliHints?: {
name?: string;
/**
* Alternate CLI command names that dispatch to this same op (v114 / #1941).
* e.g. `link-add` aliasing `link`. Registered in `cli.ts`'s `cliAliases`
* map; collisions with primary names or CLI_ONLY commands throw at startup.
*/
aliases?: string[];
positional?: string[];
stdin?: string;
hidden?: boolean;
@@ -1793,6 +1799,18 @@ const get_tags: Operation = {
// --- Links ---
/**
* v114 (#1941): reconciliation-managed provenances a CALLER must not forge via
* the add_link op. Internal writers (import-file frontmatter reconciliation,
* extract --by-mention, wikilink resolution) write these straight through the
* engine — they're excluded here, not at the DB CHECK. A hand-created edge
* tagged 'frontmatter' with no origin_page_id would be a phantom that put_page
* reconciliation (link_source='frontmatter' AND origin_page_id=written_page)
* never cleans (see src/schema.sql). `manual` is intentionally absent — it IS
* the user-facing provenance and the default for omitted link_source.
*/
export const MANAGED_LINK_SOURCES = ['markdown', 'frontmatter', 'mentions', 'wikilink-resolved'];
const add_link: Operation = {
name: 'add_link',
description: 'Create link between pages',
@@ -1801,11 +1819,22 @@ const add_link: Operation = {
to: { type: 'string', required: true },
link_type: { type: 'string', description: 'Link type (e.g., invested_in, works_at)' },
context: { type: 'string', description: 'Context for the link' },
link_source: { type: 'string', description: "Provenance tag (kebab-case, e.g. 'citation-graph'). Defaults to 'manual'. Reconciliation-managed built-ins (markdown/frontmatter/mentions/wikilink-resolved) are rejected." },
},
mutating: true,
scope: 'write',
handler: async (ctx, p) => {
if (ctx.dryRun) return { dry_run: true, action: 'add_link', from: p.from, to: p.to };
// v114 (#1941): default omitted provenance to 'manual' (NOT the engine's
// 'markdown' default) so hand/tool-created CLI edges are honestly manual,
// and forbid forging the reconciliation-managed built-ins.
const linkSource = ((p.link_source as string) || 'manual').trim();
if (MANAGED_LINK_SOURCES.includes(linkSource)) {
throw new Error(
`link_source '${linkSource}' is reconciliation-managed and cannot be set manually; ` +
`use 'manual' (the default) or a custom kebab tag like 'citation-graph'`,
);
}
// v0.31.8 (D7): single ctx.sourceId scopes both endpoints + origin. Cross-
// source link creation is out of scope for this wave; use the engine API
// directly for that edge case.
@@ -1815,12 +1844,12 @@ const add_link: Operation = {
await ctx.engine.addLink( // gbrain-allow-direct-insert: add_link MCP op is the explicit canonical surface for manual link creation; auto-link reconciliation runs separately via auto_link post-hook
p.from as string, p.to as string,
(p.context as string) || '', (p.link_type as string) || '',
undefined, undefined, undefined,
linkSource, undefined, undefined,
linkOpts,
);
return { status: 'ok' };
},
cliHints: { name: 'link', positional: ['from', 'to'] },
cliHints: { name: 'link', aliases: ['link-add'], positional: ['from', 'to'] },
};
const remove_link: Operation = {
@@ -1829,6 +1858,8 @@ const remove_link: Operation = {
params: {
from: { type: 'string', required: true },
to: { type: 'string', required: true },
link_type: { type: 'string', description: 'Only remove edges of this link type (omit = all types)' },
link_source: { type: 'string', description: 'Only remove edges of this provenance (e.g. citation-graph); omit = any provenance' },
},
mutating: true,
scope: 'write',
@@ -1837,10 +1868,15 @@ const remove_link: Operation = {
const linkOpts = ctx.sourceId
? { fromSourceId: ctx.sourceId, toSourceId: ctx.sourceId }
: undefined;
await ctx.engine.removeLink(p.from as string, p.to as string, undefined, undefined, linkOpts);
await ctx.engine.removeLink(
p.from as string, p.to as string,
(p.link_type as string) || undefined,
(p.link_source as string) || undefined,
linkOpts,
);
return { status: 'ok' };
},
cliHints: { name: 'unlink', positional: ['from', 'to'] },
cliHints: { name: 'unlink', aliases: ['link-rm'], positional: ['from', 'to'] },
};
const get_links: Operation = {
@@ -1872,6 +1908,22 @@ const get_backlinks: Operation = {
cliHints: { name: 'backlinks', positional: ['slug'] },
};
const list_link_sources: Operation = {
name: 'list_link_sources',
// v114 (#1941): the read-side counterpart to link-add/link-rm. Since
// link_source is now an open kebab provenance (no allowlist), this is how an
// agent discovers which provenances a brain actually carries.
description: 'List distinct link_source provenances in the brain with edge counts (e.g. citation-graph, manual, markdown)',
params: {},
handler: async (ctx) => {
// Route through sourceScopeOpts so the read honors both scalar ctx.sourceId
// and federated ctx.auth.allowedSources (no cross-source provenance leak).
return ctx.engine.listLinkSources(sourceScopeOpts(ctx));
},
scope: 'read',
cliHints: { name: 'link-sources' },
};
/**
* Hard cap on traverse_graph depth from MCP callers. Each recursive CTE iteration
* grows a `visited` array per path; in `direction=both` the join is `OR`-based and
@@ -4677,7 +4729,7 @@ export const operations: Operation[] = [
// Tags
add_tag, remove_tag, get_tags,
// Links
add_link, remove_link, get_links, get_backlinks, traverse_graph,
add_link, remove_link, get_links, get_backlinks, list_link_sources, traverse_graph,
// Timeline
add_timeline_entry, get_timeline,
// Admin
+26
View File
@@ -2576,6 +2576,32 @@ export class PGLiteEngine implements BrainEngine {
return rows as unknown as Link[];
}
async listLinkSources(
opts?: { sourceId?: string; sourceIds?: string[] },
): Promise<{ link_source: string | null; count: number }[]> {
// v114 (#1941): distinct provenances + counts for `gbrain link-sources`.
// Scope by the FROM page's source (consistent with getLinks). Federated
// {sourceIds} takes precedence over scalar {sourceId}; neither = unscoped.
const params: unknown[] = [];
let where = '';
if (opts?.sourceIds && opts.sourceIds.length > 0) {
params.push(opts.sourceIds);
where = `JOIN pages f ON f.id = l.from_page_id WHERE f.source_id = ANY($${params.length}::text[])`;
} else if (opts?.sourceId) {
params.push(opts.sourceId);
where = `JOIN pages f ON f.id = l.from_page_id WHERE f.source_id = $${params.length}`;
}
const { rows } = await this.db.query(
`SELECT l.link_source, COUNT(*)::int AS count
FROM links l
${where}
GROUP BY l.link_source
ORDER BY count DESC, l.link_source ASC NULLS LAST`,
params,
);
return rows as unknown as { link_source: string | null; count: number }[];
}
async findByTitleFuzzy(
name: string,
dirPrefix?: string,
+5 -6
View File
@@ -255,12 +255,11 @@ CREATE TABLE IF NOT EXISTS links (
to_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
link_type TEXT NOT NULL DEFAULT '',
context TEXT NOT NULL DEFAULT '',
-- v0.41.18.0: 'mentions' added for auto-linked body-text mentions
-- (gbrain extract links --by-mention). Filtered OUT of backlink-count
-- for search ranking; only counts toward orphan-ratio + graph traversal.
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
-- wikilink resolution. See src/schema.sql for full rationale.
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
-- v114 (#1941): open kebab-case provenance (not a closed allowlist) so
-- external derivers stamp their own tag. Format gate only: lowercase kebab,
-- <=64 chars. Managed built-ins still satisfy this; see src/schema.sql for
-- full rationale + the add_link op-layer guard against forging them.
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
-- v0.41.18.0 (codex finding #12): nullable link_kind distinguishes
-- "plain body mention" from "verb-pattern-derived typed link" within
-- link_source='mentions'. See src/schema.sql for full rationale.
+24
View File
@@ -2663,6 +2663,30 @@ export class PostgresEngine implements BrainEngine {
return rows as unknown as Link[];
}
async listLinkSources(
opts?: { sourceId?: string; sourceIds?: string[] },
): Promise<{ link_source: string | null; count: number }[]> {
const sql = this.sql;
// v114 (#1941): distinct provenances + counts for `gbrain link-sources`.
// Scope by the FROM page's source (consistent with getLinks). Federated
// {sourceIds} takes precedence over scalar {sourceId}; neither = unscoped.
const sourceCondition =
opts?.sourceIds && opts.sourceIds.length > 0
? sql`WHERE f.source_id = ANY(${opts.sourceIds}::text[])`
: opts?.sourceId
? sql`WHERE f.source_id = ${opts.sourceId}`
: sql``;
const rows = await sql`
SELECT l.link_source, COUNT(*)::int AS count
FROM links l
JOIN pages f ON f.id = l.from_page_id
${sourceCondition}
GROUP BY l.link_source
ORDER BY count DESC, l.link_source ASC NULLS LAST
`;
return rows as unknown as { link_source: string | null; count: number }[];
}
async findByTitleFuzzy(
name: string,
dirPrefix?: string,
+8 -6
View File
@@ -431,12 +431,14 @@ CREATE TABLE IF NOT EXISTS links (
to_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
link_type TEXT NOT NULL DEFAULT '',
context TEXT NOT NULL DEFAULT '',
-- v0.41.18.0: 'mentions' added for auto-linked body-text mentions
-- (gbrain extract links --by-mention). Filtered OUT of backlink-count
-- for search ranking; only counts toward orphan-ratio + graph traversal.
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
-- wikilink resolution (bare [[name]] resolved by slug tail).
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
-- v114 (#1941): link_source is an open kebab-case provenance, not a closed
-- allowlist — external derivers (e.g. 'citation-graph') stamp their own tag
-- without a gbrain migration. Format gate only: lowercase kebab, <=64 chars.
-- The reconciliation-managed built-ins ('markdown','frontmatter','mentions',
-- 'wikilink-resolved') still satisfy this and are written internally; the
-- add_link op forbids CALLERS from forging them (see operations.ts). 'manual'
-- is the user-facing default for hand/tool-created edges.
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
-- v0.41.18.0: nullable link_kind distinguishes "plain body mention" from
-- "verb-pattern-derived typed link" within link_source='mentions'.
-- Codex finding #12 design: keep link_source stable; add link_kind
+22 -8
View File
@@ -407,12 +407,24 @@ CREATE INDEX IF NOT EXISTS idx_code_edges_symbol_to
-- ============================================================
-- links: cross-references between pages
-- ============================================================
-- Provenance model (v0.13, extended issue #972):
-- link_source — 'markdown' | 'frontmatter' | 'manual' | 'wikilink-resolved' | NULL
-- 'wikilink-resolved' is the opt-in
-- (link_resolution.global_basename) basename-match
-- provenance — see issue #972 / migration v113.
-- (NULL = legacy row written before v0.13; unknown source)
-- Provenance model (v0.13; opened to kebab provenance in v114 / issue #1941):
-- link_source — open kebab-case provenance tag, NOT a closed allowlist.
-- Format gate (CHECK): ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ and
-- char_length <= 64. NULL = legacy row (pre-v0.13).
-- Reconciliation-managed built-ins written internally:
-- 'markdown' — body markdown links
-- 'frontmatter' — YAML frontmatter edges (see origin_*)
-- 'mentions' — auto-linked body-text mentions
-- 'wikilink-resolved'— opt-in global-basename [[name]] (#972)
-- User/tool-facing:
-- 'manual' — hand- or tool-created edges (the
-- add_link op default + CLI link-add)
-- '<your-tag>' — external derivers, e.g. 'citation-graph',
-- stamp their own kebab tag (no migration).
-- The add_link OP forbids callers from passing the four
-- managed built-ins (they imply reconciliation semantics a
-- hand-created row can't honor); the DB CHECK still admits
-- them because internal writers use them. See operations.ts.
-- origin_page_id — for link_source='frontmatter', the page whose YAML
-- frontmatter created this edge; scopes reconciliation
-- origin_field — the frontmatter field name (e.g. 'key_people')
@@ -420,7 +432,9 @@ CREATE INDEX IF NOT EXISTS idx_code_edges_symbol_to
-- The unique constraint includes link_source + origin_page_id so a manual edge
-- and a frontmatter-derived edge with the same (from, to, type) tuple coexist.
-- Reconciliation on put_page filters by (link_source='frontmatter' AND
-- origin_page_id = written_page) — never touches other pages' edges.
-- origin_page_id = written_page) — never touches other pages' edges. (This is
-- exactly why a CLI-forged 'frontmatter' row with NULL origin would be a phantom
-- edge reconciliation never cleans — hence the op-layer guard.)
CREATE TABLE IF NOT EXISTS links (
id SERIAL PRIMARY KEY,
from_page_id INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
@@ -432,7 +446,7 @@ CREATE TABLE IF NOT EXISTS links (
-- for search ranking; only counts toward orphan-ratio + graph traversal.
-- v0.40.8.2 (#972): 'wikilink-resolved' added for opt-in global-basename
-- wikilink resolution (bare [[name]] resolved by slug tail).
link_source TEXT CHECK (link_source IS NULL OR link_source IN ('markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved')),
link_source TEXT CHECK (link_source IS NULL OR (link_source ~ '^[a-z][a-z0-9]*(-[a-z0-9]+)*$' AND char_length(link_source) <= 64)),
-- v0.41.18.0: nullable link_kind distinguishes "plain body mention" from
-- "verb-pattern-derived typed link" within link_source='mentions'.
-- Codex finding #12 design: keep link_source stable; add link_kind
+7 -2
View File
@@ -44,11 +44,13 @@ describe('BRAIN_TOOL_ALLOWLIST', () => {
expect(missing).toEqual([]);
});
test('contains the v0.15 read-only 10 + put_page + v0.29 salience pair', () => {
test('contains the v0.15 read-only 10 + put_page + v0.29 salience pair + v114 list_link_sources', () => {
// v0.29 added get_recent_salience + find_anomalies (read-only).
// get_recent_transcripts is deliberately excluded — subagent calls always
// have ctx.remote=true, and the v0.29 trust gate rejects remote callers.
expect(BRAIN_TOOL_ALLOWLIST.size).toBe(13);
// v114 (#1941) added list_link_sources (read-only provenance discovery);
// the edge-WRITE ops add_link/remove_link stay out (separate trust call).
expect(BRAIN_TOOL_ALLOWLIST.size).toBe(14);
expect(BRAIN_TOOL_ALLOWLIST.has('query')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('search')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('get_page')).toBe(true);
@@ -56,6 +58,9 @@ describe('BRAIN_TOOL_ALLOWLIST', () => {
expect(BRAIN_TOOL_ALLOWLIST.has('put_page')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('get_recent_salience')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('find_anomalies')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('list_link_sources')).toBe(true);
expect(BRAIN_TOOL_ALLOWLIST.has('add_link')).toBe(false);
expect(BRAIN_TOOL_ALLOWLIST.has('remove_link')).toBe(false);
expect(BRAIN_TOOL_ALLOWLIST.has('get_recent_transcripts')).toBe(false);
});
+25
View File
@@ -339,6 +339,31 @@ describeBoth('Engine parity — Postgres vs PGLite', () => {
}
});
test('v114 (#1941) listLinkSources parity: same ordered provenance counts on both engines', async () => {
const mk = async (eng: BrainEngine) => {
for (const s of ['lsp-a', 'lsp-b', 'lsp-c']) {
await eng.putPage(s, { type: 'note', title: s, compiled_truth: 'b', timeline: '' });
}
// citation-graph:2, manual:1 — exercises count DESC + the kebab regex.
await eng.addLink('lsp-a', 'lsp-b', '', 'cites', 'citation-graph');
await eng.addLink('lsp-a', 'lsp-c', '', 'cites', 'citation-graph');
await eng.addLink('lsp-b', 'lsp-c', '', 'rel', 'manual');
};
await mk(pgEngine);
await mk(pgliteEngine);
const pg = await pgEngine.listLinkSources({ sourceId: 'default' });
const pglite = await pgliteEngine.listLinkSources({ sourceId: 'default' });
const norm = (rows: { link_source: string | null; count: number }[]) =>
rows.filter(r => r.link_source === 'citation-graph' || r.link_source === 'manual');
expect(norm(pg)).toEqual(norm(pglite));
// citation-graph (2) must order before manual (1) on both engines.
const cgIdx = pg.findIndex(r => r.link_source === 'citation-graph');
const mIdx = pg.findIndex(r => r.link_source === 'manual');
expect(cgIdx).toBeLessThan(mIdx);
});
test('v0.41.19.0 resolveSlugsByPaths parity: same Map on both engines', async () => {
const seedSql = `
INSERT INTO pages (source_id, slug, source_path, type, title, compiled_truth, timeline, frontmatter)
+306
View File
@@ -0,0 +1,306 @@
/**
* v114 (#1941): link_source opened from a closed allowlist to a kebab-case
* regex + length cap, provenance exposed on the link ops with a managed-
* built-in guard, a new `list_link_sources` read op, and `link-add`/`link-rm`
* CLI aliases.
*
* Coverage:
* - Migration v114 shape (two engine branches, transaction:false, idempotent)
* - DB CHECK boundary: kebab accepted (incl. all 5 built-ins), garbage rejected
* - Upgrade-path: existing built-in row survives the constraint swap (re-run)
* - add_link op: provenance threaded, default 'manual', managed-built-in guard
* - remove_link op: link_type / link_source / both filters
* - list_link_sources op: counts, deterministic order, scalar + federated scope
* - CLI aliases resolve; printOpHelp shows the invoked alias name; no collisions
*
* Hermetic via PGLite. No DATABASE_URL needed.
*/
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { MIGRATIONS, LATEST_VERSION } from '../src/core/migrate.ts';
import {
operations,
operationsByName,
MANAGED_LINK_SOURCES,
} from '../src/core/operations.ts';
import type { OperationContext } from '../src/core/operations.ts';
import { cliAliases, printOpHelp } from '../src/cli.ts';
const V = 114;
let engine: PGLiteEngine;
beforeAll(async () => {
engine = new PGLiteEngine();
await engine.connect({});
await engine.initSchema();
}, 60_000);
afterAll(async () => {
await engine.disconnect();
});
function makeCtx(overrides: Partial<OperationContext> = {}): OperationContext {
return {
engine,
remote: false,
config: {},
logger: console,
dryRun: false,
...overrides,
} as unknown as OperationContext;
}
// Two pages every link test reuses.
async function seedPages() {
await engine.putPage('lsrc-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
await engine.putPage('lsrc-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
}
describe('migration v114 — links_link_source_check_kebab_regex', () => {
test('registered with expected version + name', () => {
const m = MIGRATIONS.find(m => m.version === V);
expect(m).toBeDefined();
expect(m!.name).toBe('links_link_source_check_kebab_regex');
});
test('LATEST_VERSION >= 114', () => {
expect(LATEST_VERSION).toBeGreaterThanOrEqual(V);
});
test('both engine branches present; transaction:false; idempotent', () => {
const m = MIGRATIONS.find(m => m.version === V)!;
expect(m.transaction).toBe(false);
expect(m.idempotent).toBe(true);
expect(m.sqlFor?.postgres).toBeTruthy();
expect(m.sqlFor?.pglite).toBeTruthy();
});
test('postgres branch uses NOT VALID + VALIDATE (lock-friendly)', () => {
const m = MIGRATIONS.find(m => m.version === V)!;
const pg = m.sqlFor!.postgres!;
expect(pg).toMatch(/DROP CONSTRAINT IF EXISTS links_link_source_check/i);
expect(pg).toContain('NOT VALID');
expect(pg).toMatch(/VALIDATE CONSTRAINT links_link_source_check/i);
expect(pg).toContain("'^[a-z][a-z0-9]*(-[a-z0-9]+)*$'");
expect(pg).toContain('char_length(link_source) <= 64');
});
test('pglite branch is plain DROP+ADD with the same regex', () => {
const m = MIGRATIONS.find(m => m.version === V)!;
const pl = m.sqlFor!.pglite!;
expect(pl).toMatch(/DROP CONSTRAINT IF EXISTS links_link_source_check/i);
expect(pl).not.toContain('NOT VALID');
expect(pl).toContain("'^[a-z][a-z0-9]*(-[a-z0-9]+)*$'");
});
});
describe('DB CHECK — kebab format gate', () => {
beforeAll(seedPages);
const ACCEPTED = [
'citation-graph', 'derived', 'a',
// OV7 — all 5 reconciliation-managed built-ins must stay DB-valid
'markdown', 'frontmatter', 'manual', 'mentions', 'wikilink-resolved',
];
const REJECTED: Array<[string, string]> = [
['UPPER', 'uppercase'],
['has_underscore', 'underscore'],
['has space', 'space'],
['2bad', 'leading digit'],
['-lead', 'leading dash'],
['trail-', 'trailing dash'],
['a--b', 'double dash'],
['', 'empty'],
['a'.repeat(65), '65 chars (over cap)'],
];
for (const v of ACCEPTED) {
test(`accepts '${v}'`, async () => {
// engine.addLink writes link_source straight through (no op guard) — this
// is the DB CHECK under test, not the op layer.
await expect(
engine.addLink('lsrc-a', 'lsrc-b', '', `t-${v}`, v),
).resolves.toBeUndefined();
});
}
for (const [v, why] of REJECTED) {
test(`rejects '${v}' (${why})`, async () => {
await expect(
engine.addLink('lsrc-a', 'lsrc-b', '', `t-rej-${why}`, v),
).rejects.toThrow();
});
}
});
describe('upgrade-path — existing rows survive the constraint swap', () => {
test('built-in row present, re-running v114 succeeds, new kebab tag inserts', async () => {
await seedPages();
// A row tagged with a pre-existing built-in value (would have been valid
// under the old allowlist too).
await engine.addLink('lsrc-a', 'lsrc-b', '', 't-upgrade', 'markdown');
const m = MIGRATIONS.find(m => m.version === V)!;
// Re-apply the migration against the table that now holds data: ADD/VALIDATE
// must not fail on the existing row.
await expect(engine.runMigration(V, m.sqlFor!.pglite!)).resolves.toBeUndefined();
// And a brand-new external provenance still inserts afterward.
await expect(
engine.addLink('lsrc-a', 'lsrc-b', '', 't-upgrade', 'citation-graph'),
).resolves.toBeUndefined();
});
});
describe('add_link op — provenance + managed-built-in guard (OV4A)', () => {
beforeAll(seedPages);
test('threads custom provenance through to the row', async () => {
const op = operationsByName['add_link'];
await op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'cites', link_source: 'citation-graph' });
const links = await engine.getLinks('lsrc-a');
expect(links.some(l => (l as any).to_slug === 'lsrc-b' && (l as any).link_source === 'citation-graph' && (l as any).link_type === 'cites')).toBe(true);
});
test("omitted link_source defaults to 'manual' (not 'markdown')", async () => {
const op = operationsByName['add_link'];
await op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'default-prov' });
const links = await engine.getLinks('lsrc-a');
const row = links.find(l => (l as any).link_type === 'default-prov');
expect((row as any)?.link_source).toBe('manual');
});
for (const managed of MANAGED_LINK_SOURCES) {
test(`rejects managed built-in '${managed}'`, async () => {
const op = operationsByName['add_link'];
await expect(
op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'g', link_source: managed }),
).rejects.toThrow(/reconciliation-managed/);
});
}
test("'manual' is allowed (not in the managed set)", async () => {
const op = operationsByName['add_link'];
await expect(
op.handler(makeCtx(), { from: 'lsrc-a', to: 'lsrc-b', link_type: 'man', link_source: 'manual' }),
).resolves.toMatchObject({ status: 'ok' });
});
});
describe('remove_link op — type/source filters', () => {
async function seedTwoProvenances() {
await engine.putPage('rm-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
await engine.putPage('rm-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
await engine.addLink('rm-a', 'rm-b', '', 'cites', 'manual');
await engine.addLink('rm-a', 'rm-b', '', 'cites', 'citation-graph');
}
test('--link-source filter deletes only that provenance', async () => {
await seedTwoProvenances();
const op = operationsByName['remove_link'];
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_source: 'citation-graph' });
const links = await engine.getLinks('rm-a');
const sources = links.filter(l => (l as any).to_slug === 'rm-b').map(l => (l as any).link_source);
expect(sources).toContain('manual');
expect(sources).not.toContain('citation-graph');
});
test('--link-type-only filter deletes every provenance of that type', async () => {
await seedTwoProvenances();
const op = operationsByName['remove_link'];
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_type: 'cites' });
const links = await engine.getLinks('rm-a');
expect(links.filter(l => (l as any).to_slug === 'rm-b' && (l as any).link_type === 'cites').length).toBe(0);
});
test('both filters delete only the type+source match', async () => {
await seedTwoProvenances();
const op = operationsByName['remove_link'];
await op.handler(makeCtx(), { from: 'rm-a', to: 'rm-b', link_type: 'cites', link_source: 'manual' });
const links = await engine.getLinks('rm-a');
const sources = links.filter(l => (l as any).to_slug === 'rm-b').map(l => (l as any).link_source);
expect(sources).toContain('citation-graph');
expect(sources).not.toContain('manual');
});
});
describe('list_link_sources op (B4 + OV8 + OV9)', () => {
test('op declaration: read scope, link-sources CLI name, not localOnly', () => {
const op = operationsByName['list_link_sources'];
expect(op).toBeDefined();
expect(op.scope).toBe('read');
expect(op.localOnly).not.toBe(true);
expect(op.cliHints?.name).toBe('link-sources');
});
test('returns distinct provenances with counts, ordered count DESC then name ASC', async () => {
await engine.putPage('ls-a', { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
await engine.putPage('ls-b', { type: 'note', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
await engine.putPage('ls-c', { type: 'note', title: 'C', compiled_truth: 'c', timeline: '', frontmatter: {} });
// zeta:1, alpha:2 — alpha sorts first by count; within ties, name ASC.
await engine.addLink('ls-a', 'ls-b', '', 'r1', 'alpha-src');
await engine.addLink('ls-a', 'ls-c', '', 'r2', 'alpha-src');
await engine.addLink('ls-b', 'ls-c', '', 'r3', 'zeta-src');
const op = operationsByName['list_link_sources'];
const rows = (await op.handler(makeCtx(), {})) as Array<{ link_source: string | null; count: number }>;
const map = new Map(rows.map(r => [r.link_source, r.count]));
expect(map.get('alpha-src')).toBe(2);
expect(map.get('zeta-src')).toBe(1);
// Deterministic order: counts non-increasing.
for (let i = 1; i < rows.length; i++) {
expect(rows[i - 1].count >= rows[i].count).toBe(true);
}
});
test('scalar ctx.sourceId scopes; a non-matching source returns nothing', async () => {
const op = operationsByName['list_link_sources'];
// Pages above were written to the 'default' source.
const inDefault = (await op.handler(makeCtx({ sourceId: 'default' } as any), {})) as unknown[];
expect(inDefault.length).toBeGreaterThan(0);
const inOther = (await op.handler(makeCtx({ sourceId: 'no-such-source' } as any), {})) as unknown[];
expect(inOther.length).toBe(0);
});
test('federated allowedSources ({sourceIds}) scopes (OV8)', async () => {
const op = operationsByName['list_link_sources'];
const inDefault = (await op.handler(makeCtx({ auth: { allowedSources: ['default'] } } as any), {})) as unknown[];
expect(inDefault.length).toBeGreaterThan(0);
const inOther = (await op.handler(makeCtx({ auth: { allowedSources: ['no-such-source'] } } as any), {})) as unknown[];
expect(inOther.length).toBe(0);
});
});
describe('CLI aliases + help (OV10 + OV11)', () => {
test('link-add resolves to add_link, link-rm to remove_link', () => {
expect(cliAliases.get('link-add')?.name).toBe('add_link');
expect(cliAliases.get('link-rm')?.name).toBe('remove_link');
});
test('no alias collides with any primary CLI name or another alias', () => {
const primaries = new Set(operations.map(o => o.cliHints?.name).filter(Boolean) as string[]);
const seen = new Set<string>();
for (const op of operations) {
for (const alias of op.cliHints?.aliases ?? []) {
expect(primaries.has(alias)).toBe(false);
expect(seen.has(alias)).toBe(false);
seen.add(alias);
}
}
});
test('printOpHelp shows the invoked alias name, not the primary (OV10)', () => {
const op = operationsByName['add_link'];
const lines: string[] = [];
const orig = console.log;
console.log = (...a: unknown[]) => { lines.push(a.join(' ')); };
try {
printOpHelp(op, 'link-add');
} finally {
console.log = orig;
}
const out = lines.join('\n');
expect(out).toContain('Usage: gbrain link-add');
expect(out).not.toContain('Usage: gbrain link ');
});
});
@@ -97,19 +97,21 @@ describe('fresh-init brain (post-migration v95) accepts link_source=mentions', (
expect(rows.some(r => r.link_source === 'mentions')).toBe(true);
});
test('CHECK still rejects an unknown source value (widening did not nullify the gate)', async () => {
test('CHECK still rejects a malformed source value (gate not nullified)', async () => {
const slugA = `bad-source-a-${Math.random().toString(36).slice(2, 8)}`;
const slugB = `bad-source-b-${Math.random().toString(36).slice(2, 8)}`;
await engine.putPage(slugA, { type: 'note', title: 'A', compiled_truth: 'a', timeline: '', frontmatter: {} });
await engine.putPage(slugB, { type: 'person', title: 'B', compiled_truth: 'b', timeline: '', frontmatter: {} });
// 'inferred' is NOT in allow-list {'mentions'} — must reject.
// v114 (#1941): the closed allowlist became a kebab-case regex, so a once-
// illegal-but-kebab value like 'inferred' is now ACCEPTED. The gate still
// bites on MALFORMED values — 'Inferred_X' has an uppercase + underscore.
await expect(
engine.addLinksBatch([
{
from_slug: slugA,
to_slug: slugB,
link_type: 'mentions',
link_source: 'inferred' as any,
link_source: 'Inferred_X' as any,
context: 'should reject',
},
]),
+58 -9
View File
@@ -10,6 +10,9 @@
* - refresh-failure past the threshold fails SAFE (exits non-zero) (F1A)
*/
import { describe, test, expect, beforeAll, afterAll, beforeEach, spyOn } from 'bun:test';
import { existsSync, unlinkSync } from 'fs';
import { join } from 'path';
import { tmpdir } from 'os';
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
import { tryAcquireDbLock } from '../src/core/db-lock.ts';
import { MinionSupervisor, ExitCodes, supervisorLockId, classifySupervisorSingleton } from '../src/core/minions/supervisor.ts';
@@ -32,12 +35,16 @@ beforeEach(async () => {
});
describe('#1849 supervisorLockId', () => {
test('keys on DB identity AND queue', () => {
expect(supervisorLockId('default', 'postgres://x')).toBe('gbrain-supervisor:postgres://x:default');
expect(supervisorLockId('shell', 'postgres://x')).toBe('gbrain-supervisor:postgres://x:shell');
// Different DB identity → different lock even for the same queue.
expect(supervisorLockId('default', 'postgres://a'))
.not.toBe(supervisorLockId('default', 'postgres://b'));
test('keys on queue ONLY (DB scoping is physical — the lock row lives in the DB)', () => {
expect(supervisorLockId('default')).toBe('gbrain-supervisor:default');
expect(supervisorLockId('shell')).toBe('gbrain-supervisor:shell');
// Different queues → different locks.
expect(supervisorLockId('default')).not.toBe(supervisorLockId('shell'));
// Regression (the bug this fixes): the id must NOT depend on how the same
// physical DB was addressed. Two supervisors on one DB via different URLs
// must compute the SAME id so they collide on the one shared locks table.
// The function takes no DB-identity arg precisely so it can't diverge.
expect(supervisorLockId.length).toBe(1);
});
});
@@ -75,7 +82,7 @@ describe('#1849 classifySupervisorSingleton (doctor)', () => {
describe('#1849 DB lock is the real singleton', () => {
test('second acquire of the same (db, queue) lock is refused', async () => {
const id = supervisorLockId('default', 'pglite:test');
const id = supervisorLockId('default');
const first = await tryAcquireDbLock(engine, id, 5);
expect(first).not.toBeNull();
// A second supervisor (different pidfile, same db+queue) gets null → exit 2.
@@ -89,8 +96,8 @@ describe('#1849 DB lock is the real singleton', () => {
});
test('different queues on the same DB do not collide', async () => {
const a = await tryAcquireDbLock(engine, supervisorLockId('default', 'pglite:test'), 5);
const b = await tryAcquireDbLock(engine, supervisorLockId('shell', 'pglite:test'), 5);
const a = await tryAcquireDbLock(engine, supervisorLockId('default'), 5);
const b = await tryAcquireDbLock(engine, supervisorLockId('shell'), 5);
expect(a).not.toBeNull();
expect(b).not.toBeNull();
await a!.release();
@@ -98,6 +105,48 @@ describe('#1849 DB lock is the real singleton', () => {
});
});
describe('#1849 LOCK_HELD path does not strand the pidfile', () => {
test('the pidfile-cleanup exit listener is installed BEFORE the DB-lock acquire', async () => {
// Supervisor A already holds the queue lock.
const holderA = await tryAcquireDbLock(engine, supervisorLockId('default'), 5);
expect(holderA).not.toBeNull();
const pidFile = join(tmpdir(), `gbrain-sup-stranded-${process.pid}-${Math.random().toString(36).slice(2)}.pid`);
const sup = new MinionSupervisor(engine, { cliPath: '/bin/sh', healthInterval: 0, json: true, pidFile });
// Capture the 'exit' listener start() registers (if any) and stop execution
// at the first process.exit (the LOCK_HELD path) the way the real exit would.
let exitListener: ((...a: unknown[]) => void) | null = null;
const onSpy = spyOn(process, 'on').mockImplementation(((event: string, cb: (...a: unknown[]) => void) => {
if (event === 'exit') exitListener = cb;
return process;
}) as never);
const exitSpy = spyOn(process, 'exit').mockImplementation(((code?: number) => {
throw new Error(`exit:${code}`);
}) as never);
try {
try { await sup.start(); } catch { /* exit stub throws at LOCK_HELD */ }
expect(exitSpy).toHaveBeenCalledWith(ExitCodes.LOCK_HELD);
// The bug: the exit listener was registered AFTER the DB-lock exit, so
// start() threw before reaching it and the pidfile this process created
// is stranded. The fix installs it first → it's captured here.
expect(exitListener).not.toBeNull();
// And it actually cleans up the pidfile we created (contents match our pid).
expect(existsSync(pidFile)).toBe(true);
exitListener!();
expect(existsSync(pidFile)).toBe(false);
} finally {
onSpy.mockRestore();
exitSpy.mockRestore();
if (existsSync(pidFile)) unlinkSync(pidFile);
await holderA!.release();
}
});
});
describe('#1849 refresh-failure fails safe (F1A)', () => {
test('exits LOCK_LOST after the failure threshold; tolerates a single blip', async () => {
const sup = new MinionSupervisor(engine, { cliPath: '/bin/sh', healthInterval: 0, json: true });