admin/gbrain
1
0
Fork 0
mirror of https://github.com/garrytan/gbrain.git synced 2026-07-27 22:15:33 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
f09f9177a965122ab8e8f7ba8478c0c9756c6237
gbrain/package.json
T
Russell SmithGitHubClaude Opus 4.7Garry Tan
 f09f9177a9 v0.42.10.0 feat(extract): opt-in global-basename wikilink resolution (closes #972) (#1388) 
* v0.40.8.2 fix(extract): opt-in global-basename wikilink resolution (#972)

Bare wikilinks like [[struktura]] that point at pages in another folder
were silently dropped from the graph. The issue reporter saw 71 wikilinks
in Obsidian render to 12 in gbrain (~83% lost). Symptoms downstream:
`gbrain graph` returns thin neighborhoods, `gbrain backlinks` undercounts.

This release adds an opt-in mode that resolves bare wikilinks by basename
match, covers all three resolver surfaces (FS-source extract, DB-source
extract, put_page auto-link), and emits one edge per match — no silent
winner on ambiguity. `gbrain doctor` surfaces a paste-ready enable hint
when ≥5 bare wikilinks would resolve under the new mode.

Enable with:
  gbrain config set link_resolution.global_basename true
  gbrain extract links

Default stays off. Existing brains see zero behavior change on upgrade.

Closes #972. Adapts PR #1233 from @rayers (regex shape + slug-tail index)
into a multi-match, opt-in form with FS-source coverage that the original
PR explicitly skipped.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: document opt-in global-basename wikilink resolution (#972)

The #972 feature shipped with no user-facing docs — only CHANGELOG + CLAUDE.md.
Anyone migrating an Obsidian/Notion vault with bare [[name]] wikilinks couldn't
discover the link_resolution.global_basename flag unless gbrain doctor happened
to surface its hint.

- README "Self-wiring knowledge graph": one sentence on the opt-in mode for
  Obsidian-style cross-folder bare wikilinks + the doctor pre-check, linking to
  the install step.
- INSTALL_FOR_AGENTS Step 4.5 (Wire the Knowledge Graph): a dedicated agent-
  facing subsection — when bare [[name]] links need it, the enable command,
  re-running extract, the doctor opportunity hint, and the multi-match behavior.
- Regenerated llms-full.txt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): resolve aliased wikilinks by target slug, not display text

Codex outside-voice [P1]: `[[struktura|the project]]` resolved the basename
"the project" (the alias) instead of `struktura` (the target), because
extractPageLinks called resolveBasenameMatches(ref.name) and the doctor check
keyed basenameIndex.get(e.name). ref.name is the display alias (match[2]);
ref.slug is the wikilink target (match[1]).

- extractPageLinks resolves ref.slug; context excerpt locates ref.slug.
- doctor link_resolution_opportunity keys e.slug so its estimate matches
  what extraction actually resolves.
- Test: aliased wikilink calls resolveBasenameMatches with the target, never
  the display text.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): reconcile wikilink-resolved edges in put_page auto-link

Codex outside-voice [P1]: put_page's reconcilableOut filter excluded
link_source='wikilink-resolved', so a basename edge written by auto-link
survived after the bare wikilink was deleted from the page OR the
link_resolution.global_basename flag was turned off (the stale-removal loop
only iterates reconcilableOut). Add 'wikilink-resolved' to the reconcilable
set; manual edges still untouched.

Test: write page with [[struktura]] (flag on) → edge lands; re-put without
the wikilink → edge reconciled away.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): source-scope basename resolution (no cross-source edges)

Codex outside-voice [P1]: makeResolver.resolveBasenameMatches called
engine.getAllSlugs() unscoped, so a bare [[name]] could resolve to a
same-tail page in a DIFFERENT source and create a cross-source edge. The
engine exposes getAllSlugs({sourceId}) precisely to prevent this. #972 is
"global basename across folders," not "cross-source federation" — the
canonical gbrain multi-source bug class.

- makeResolver gains opts.sourceId; ensureBasenameIndex passes it to
  getAllSlugs (unscoped only when sourceId omitted — back-compat).
- runAutoLink (put_page) passes opts.sourceId; extractLinksFromDB passes
  sourceIdFilter. FS extract is already single-source (walks one dir).
- Tests: scoped index returns only the source's slugs (no cross-source);
  unscoped call stays brain-wide.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): FS-source basename edges carry link_source='wikilink-resolved'

The FS extract path is the issue's default repro (gbrain extract links with no
--source db). ExtractedLink had no link_source field, so FS basename edges
landed with the engine default ('markdown') instead of the 'wikilink-resolved'
provenance the DB / put_page paths set and the docs promise. The e2e FS test
only asserted link_type, so it was blind to this.

- ExtractedLink gains link_source?; extractLinksFromFile sets it to
  'wikilink-resolved' on basename edges (undefined for ordinary markdown).
- Carries through the addLinksBatch snapshots automatically (LinkBatchInput
  already has link_source); single-row addLink fallback now passes it too.
- e2e FS repro asserts link_source === 'wikilink-resolved'.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(#972): one shared basename matcher across resolver/FS/doctor

Codex outside-voice [P2] DRY: three surfaces each hand-rolled a basename
matcher with divergent key sets — the doctor omitted the slugified key, so its
link_resolution_opportunity estimate undercounted what extraction resolves, and
the resolver returned matches in unsorted getAllSlugs bucket order.

New shared exports in link-extraction.ts: buildBasenameIndex(slugs) +
queryBasenameIndex(index, name) (keys raw/lower/slugified tail; stable sort
shorter-first then lexical) + normalizeBasename.

- makeResolver.resolveBasenameMatches → queryBasenameIndex (now stable-sorted).
- extract.ts resolveBasenameMatchesFromSlugs → delegates to the shared pair.
- doctor link_resolution_opportunity → shared builder/query (slugified key
  added; estimate now matches extraction).
- Test: doctor counts a slugified-only match ([[Fast Weigh]] → companies/fast-weigh).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): P2 cluster — masking, code-fence, self-link, dedup decision

Codex outside-voice P2 findings:
- P2a markdown-label masking: a wikilink inside a markdown-link label
  ([see [[acme]]](companies/acme.md)) spawned a stray generic basename ref.
  Pass-1 can't match the nested brackets, so a new MARKDOWN_LABEL_WIKILINK_RE
  masks those spans out of pass 2c. Inner [[acme]] is now inert.
- P2b FS code-fence: the FS path (extractMarkdownLinks on raw content) didn't
  strip code blocks like the DB path. extractLinksFromFile now scans
  stripCodeBlocks(content) so [[name]] inside a fence creates no FS edge.
- P2c self-link guard: a basename [[own-tail]] on its own page resolved back
  to itself. Dropped in both extractPageLinks and the FS path.
- P2d dedup: documented the decision to KEEP qualified + bare edges to the
  same target as separate rows (distinct provenance/audit trail).
- P2e: skipFrontmatter unresolved-contract tests added.

Tests: P2a inert-label, P2c self-link drop, P2b code-fence, P2e unresolved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* perf(#972): bound the doctor link_resolution_opportunity scan

The check did listAllPageRefs() + a getPage() per page under a 60s budget.
On a large brain (the eng-review concern) it hit the budget every non-fast
doctor run and returned a perpetual partial, adding ~60s.

Now batch-loads the 1000 most-recent pages in ONE query
(ORDER BY id DESC LIMIT SAMPLE_LIMIT) and scans in memory, with the 60s cap
kept as a backstop. Mirrors the v0.40.9 sampling convention. The estimate
message names the bound when the brain exceeds the sample
("scanned the 1000 most-recent of N pages").

Test: source-grep pins the bounded query + the absence of the per-page
getPage walk.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(#972): reconcile stale version/migration references to v112 / 0.42.6.0

Merge churn left intermediate refs: schema.sql + schema-embedded.ts said
"migration v93", CLAUDE.md said "v0.41.32.0 / Migration v109", CHANGELOG said
"Migration v93". Reconciled all to migration v112 / shipping 0.42.6.0. The
CLAUDE.md annotation is also refreshed to describe the final behavior (shared
matcher, source-scoping, alias-by-target, stale-edge reconciliation, bounded
doctor scan) and credit @rayers + @ukd1. Regenerated schema-embedded + llms.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(#972): register doctor check category + bump llms budget to 800KB

Two full-suite gate failures from the re-sync:
- doctor-categories drift guard: the new `link_resolution_opportunity` check
  wasn't in any category set. Added to BRAIN_CHECK_NAMES (alongside
  graph_coverage / orphan_ratio — it's a graph-quality signal).
- build-llms size budget: the #972 Key Files annotation (landing with master's
  #1696/#1699 waves) pushed llms-full.txt past 750KB. Bumped FULL_SIZE_BUDGET
  750KB→800KB, the established "budget tracks CLAUDE.md's legitimate per-feature
  growth" pattern (600→700→750→800 across releases).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Garry Tan <garrytan@gmail.com>
2026-06-02 16:33:06 -07:00

147 lines
7.4 KiB
JSON
Raw Blame History

{
"name": "gbrain",
"description": "Postgres-native personal knowledge brain with hybrid RAG search",
"type": "module",
"main": "src/core/index.ts",
"bin": {
"gbrain": "src/cli.ts"
},
"exports": {
".": "./src/core/index.ts",
"./engine": "./src/core/engine.ts",
"./types": "./src/core/types.ts",
"./operations": "./src/core/operations.ts",
"./minions": "./src/core/minions/index.ts",
"./engine-factory": "./src/core/engine-factory.ts",
"./pglite-engine": "./src/core/pglite-engine.ts",
"./link-extraction": "./src/core/link-extraction.ts",
"./import-file": "./src/core/import-file.ts",
"./transcription": "./src/core/transcription.ts",
"./embedding": "./src/core/embedding.ts",
"./config": "./src/core/config.ts",
"./markdown": "./src/core/markdown.ts",
"./backoff": "./src/core/backoff.ts",
"./search/hybrid": "./src/core/search/hybrid.ts",
"./search/expansion": "./src/core/search/expansion.ts",
"./ai/gateway": "./src/core/ai/gateway.ts",
"./extract": "./src/commands/extract.ts",
"./ingestion": "./src/core/ingestion/index.ts",
"./ingestion/test-harness": "./src/core/ingestion/test-harness.ts"
},
"scripts": {
"dev": "bun run src/cli.ts",
"build": "bun build --compile --outfile bin/gbrain src/cli.ts",
"build:all": "bun build --compile --target=bun-darwin-arm64 --outfile bin/gbrain-darwin-arm64 src/cli.ts && bun build --compile --target=bun-linux-x64 --outfile bin/gbrain-linux-x64 src/cli.ts",
"build:admin": "cd admin && bun run build && cd .. && bun run scripts/build-admin-embedded.ts",
"build:admin-embedded": "bun run scripts/build-admin-embedded.ts",
"build:schema": "bash scripts/build-schema.sh",
"build:llms": "bun run scripts/build-llms.ts",
"build:pglite-snapshot": "bun run scripts/build-pglite-snapshot.ts",
"test": "bash scripts/run-unit-parallel.sh",
"eval:autocut": "bun test test/search/autocut-eval.test.ts",
"test:full": "bun run verify && bash scripts/run-unit-parallel.sh && bun run test:slow && ([ -n \"$DATABASE_URL\" ] && bash scripts/run-e2e.sh || echo '[test:full] skipped E2E (no DATABASE_URL); run docker-compose -f docker-compose.ci.yml up + bun run test:e2e to include' 1>&2)",
"verify": "bash scripts/run-verify-parallel.sh",
"check:source-config-leak": "scripts/check-source-config-leak.sh",
"check:no-pii-agent-voice": "scripts/check-no-pii-in-agent-voice.sh",
"check:synthetic-corpus-privacy": "scripts/check-synthetic-corpus-privacy.sh",
"check:system-of-record": "scripts/check-system-of-record.sh",
"check:admin-scope-drift": "scripts/check-admin-scope-drift.sh",
"check:cli-exec": "scripts/check-cli-executable.sh",
"check:all": "scripts/check-privacy.sh && scripts/check-proposal-pii.sh && scripts/check-test-real-names.sh && scripts/check-jsonb-pattern.sh && scripts/check-source-id-projection.sh && scripts/check-source-config-leak.sh && scripts/check-progress-to-stdout.sh && scripts/check-no-legacy-getconnection.sh && scripts/check-test-isolation.sh && scripts/check-trailing-newline.sh && scripts/check-wasm-embedded.sh && scripts/check-exports-count.sh && scripts/check-admin-build.sh && scripts/check-admin-scope-drift.sh && scripts/check-cli-executable.sh && scripts/check-skill-brain-first.sh && scripts/check-operations-filter-bypass.sh && scripts/check-gateway-routed-no-direct-anthropic.sh && scripts/check-worker-pool-atomicity.sh && scripts/check-no-double-retry.sh && scripts/check-batch-audit-site.sh",
"check:gateway-routed": "scripts/check-gateway-routed-no-direct-anthropic.sh",
"check:worker-pool-atomicity": "scripts/check-worker-pool-atomicity.sh",
"check:resolver": "bun src/cli.ts check-resolvable --strict --skills-dir skills/",
"check:skill-brain-first": "scripts/check-skill-brain-first.sh",
"check:wasm": "scripts/check-wasm-embedded.sh",
"check:newlines": "scripts/check-trailing-newline.sh",
"test:e2e": "bash scripts/run-e2e.sh",
"test:slow": "bash scripts/run-slow-tests.sh",
"test:heavy": "bash scripts/run-heavy.sh",
"test:profile": "bash scripts/profile-tests.sh",
"test:serial": "bash scripts/run-serial-tests.sh",
"ci:local": "bash scripts/ci-local.sh",
"ci:local:diff": "bash scripts/ci-local.sh --diff",
"ci:select-e2e": "bun run scripts/select-e2e.ts",
"typecheck": "tsc --noEmit",
"check:jsonb": "scripts/check-jsonb-pattern.sh",
"check:no-double-retry": "scripts/check-no-double-retry.sh",
"check:batch-audit-site": "scripts/check-batch-audit-site.sh",
"check:worker-lock-renewal-shape": "scripts/check-worker-lock-renewal-shape.sh",
"check:source-id-projection": "scripts/check-source-id-projection.sh",
"check:privacy": "scripts/check-privacy.sh",
"check:proposal-pii": "scripts/check-proposal-pii.sh",
"check:eval-glossary": "scripts/check-eval-glossary-fresh.sh",
"check:test-names": "scripts/check-test-real-names.sh",
"check:progress": "scripts/check-progress-to-stdout.sh",
"check:exports-count": "scripts/check-exports-count.sh",
"check:admin-build": "scripts/check-admin-build.sh",
"check:admin-embedded": "scripts/check-admin-embedded.sh",
"check:test-isolation": "scripts/check-test-isolation.sh",
"check:fuzz-purity": "scripts/check-fuzz-purity.sh",
"check:operations-filter-bypass": "scripts/check-operations-filter-bypass.sh",
"check:fixture-privacy": "scripts/check-fixture-privacy.sh",
"check:conversation-parser": "bun src/cli.ts eval conversation-parser test/fixtures/conversation-formats/all.jsonl --no-llm",
"check:source-scope-onboard": "scripts/check-source-scope-onboard.sh",
"postinstall": "command -v gbrain >/dev/null 2>&1 && gbrain apply-migrations --yes --non-interactive || echo '[gbrain] postinstall skipped. If installed via bun install -g github:...: run `gbrain doctor` and `gbrain apply-migrations --yes` manually. See https://github.com/garrytan/gbrain/issues/218' 1>&2",
"prepublish:clawhub": "bun run build:all",
"publish:clawhub": "clawhub package publish . --family bundle-plugin"
},
"openclaw": {
"compat": {
"pluginApi": ">=2026.4.0"
},
"extensions": [
"./src/openclaw-context-engine.ts"
]
},
"dependencies": {
"@ai-sdk/anthropic": "^3.0.71",
"@ai-sdk/google": "^3.0.64",
"@ai-sdk/openai": "^3.0.53",
"@ai-sdk/openai-compatible": "^2.0.41",
"@anthropic-ai/sdk": "^0.30.0",
"@aws-sdk/client-s3": "^3.1028.0",
"@dqbd/tiktoken": "^1.0.22",
"@electric-sql/pglite": "0.4.3",
"@jsquash/avif": "^2.1.1",
"@jsquash/png": "^3.1.1",
"@modelcontextprotocol/sdk": "1.29.0",
"ai": "^6.0.168",
"chokidar": "^4.0.3",
"cookie-parser": "^1.4.7",
"cors": "^2.8.5",
"eventsource-parser": "^3.0.8",
"exifr": "^7.1.3",
"express": "^5.1.0",
"express-rate-limit": "^7.5.0",
"gray-matter": "^4.0.3",
"heic-decode": "^2.1.0",
"js-yaml": "^3.14.2",
"marked": "^18.0.0",
"openai": "^4.0.0",
"pgvector": "^0.2.0",
"postgres": "^3.4.0",
"tree-sitter-wasms": "0.1.13",
"web-tree-sitter": "0.22.6",
"zod": "^4.3.6"
},
"devDependencies": {
"@types/bun": "latest",
"@types/cookie-parser": "^1.4.7",
"@types/cors": "^2.8.19",
"@types/express": "^5.0.6",
"@types/js-yaml": "^3.12.10",
"bun-types": "^1.3.13",
"fast-check": "^4.8.0",
"typescript": "^5.6.0"
},
"trustedDependencies": [
"@electric-sql/pglite"
],
"engines": {
"bun": ">=1.3.10"
},
"license": "MIT",
"version": "0.42.10.0"
}
Reference in New Issue View Git Blame Copy Permalink