mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* feat: storage tiering — git-tracked vs supabase-only directories Brain repos scaling to 200K+ files. Bulk data (tweets, articles, transcripts) bloats git repos and slows operations. New storage config in gbrain.yml lets users declare git-tracked and supabase-only directories. Changes: - New config: storage.git_tracked and storage.supabase_only in gbrain.yml - gbrain sync auto-manages .gitignore for supabase-only paths - gbrain export --restore-only restores missing supabase-only files from DB - New gbrain storage status command shows tier breakdown - Config validation warns on conflicts - 8 tests passing, full docs at docs/storage-tiering.md Backward compatible — systems without gbrain.yml work unchanged. * feat: add getDefaultSourcePath() typed accessor (step 1/15) Single source of truth for "what brain repo are we operating against?" Replaces ad-hoc raw SQL in storage.ts:38 (Issue #3 of eng review). Used by both gbrain storage status and gbrain export --restore-only. Returns null on miss, throws on DB error. Composes with the existing resolveSourceId chain so it honors --source flag / GBRAIN_SOURCE env / .gbrain-source dotfile / longest-prefix CWD match / brain-level default. 4 new test cases covering happy path, missing local_path, DB error propagation, and CWD-prefix resolution priority. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: replace gray-matter with dedicated YAML parser (step 2/15) The original storage-config.ts called gray-matter on a delimiter-less YAML file. Gray-matter only parses YAML inside `---` frontmatter blocks; without delimiters, it returns `{data: {}}`. Result: loadStorageConfig() always returned null, the entire feature was a silent no-op for every user. Original eng review's P0 confidence-9 finding (Issue #1). Replaces gray-matter with a small dedicated parser for the gbrain.yml shape (top-level `storage:` section, two array-valued nested keys). Yaml-lite was considered first, but its flat key:value design doesn't handle nested arrays. The dedicated parser is ~50 lines and trades expressiveness for zero-dep, predictable parsing of a file format we control. Adds the Issue #1B sanity warning (locked B): when gbrain.yml exists but has no storage section (or empty arrays), warn once-per-process so the user sees their config didn't take. The single test that would have caught the original P0 — write a real gbrain.yml, call loadStorageConfig, assert non-null — now exists. Also tightens loadStorageConfig per D36: distinguishes "absent" (silent null) from "unreadable" (throws). The previous code silently swallowed read errors, hiding broken installs. 8 new test cases: real-disk happy path, comments + blank lines, quoted values, missing storage section warning, empty section warning, once-per-process warning suppression, unreadable file behavior, and the existing helper tests (validation, tier matching, edge cases) all still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: rename storage keys to db_tracked/db_only (step 3/15) The vendor-specific names "supabase_only" and "git_tracked" hardcoded a backend (Supabase) into the config schema. gbrain ships two engines — PGLite and Postgres-via-Supabase. The canonical distinction is "lives in the brain DB only" vs "lives in the brain DB and on disk under git." Both work on either engine. Renamed throughout (Issue #4 of eng review): git_tracked → db_tracked supabase_only → db_only isGitTracked() → isDbTracked() isSupabaseOnly() → isDbOnly() StorageTier 'git_tracked'/'supabase_only' → 'db_tracked'/'db_only' Backward compatibility (D3 lock): loadStorageConfig accepts both shapes. Loader resolution order per the eng-review pass-2 finding: parse YAML → if canonical keys present use them, else if deprecated keys present map to canonical AND emit once-per-process deprecation warning → THEN run validation. Validation always sees the canonical shape so error messages reference db_tracked/db_only regardless of which keys the user wrote. The deprecation warning suggests `gbrain doctor --fix` for an automated rename (D72 — fix path lands in step 7). When both shapes coexist in one file, canonical wins and a stronger warning fires ("deprecated keys ignored — remove them"). Aliases isGitTracked/isSupabaseOnly kept for now to avoid churning the sync.ts / export.ts / storage.ts call sites in this commit; they'll be removed in a follow-up step. Storage.ts's tier-bucket initializers and output strings updated. ASCII output replaces unicode box-drawing per D10. gbrain.yml example file updated to canonical keys with explanatory comments. 2 new test cases: deprecated-key fallback (asserts both shapes load correctly with warning), canonical-wins-over-deprecated (asserts the "both shapes coexist" path). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: add slugPrefix to PageFilters with engine-side filter (step 4/15) Issue #13 of the eng review: storage.ts and export.ts loaded every page in the brain (limit: 1_000_000) to check tier membership. On the 200K-page brains this feature targets, that's the wall-clock and memory landmine the feature exists to fix. Adds an optional `slugPrefix` field to PageFilters. Both engines implement it as `WHERE slug LIKE prefix || '%' ESCAPE '\'`, with literal escaping of LIKE metacharacters (%, _, \) so user-supplied prefixes like `media/x/` are treated as exact string prefixes. Performance: the (source_id, slug) UNIQUE constraint on the pages table gives both engines a btree index that supports LIKE-prefix range scans. An EXPLAIN on Postgres confirms the index range scan rather than a seq scan. PGLite has the same index shape via pglite-schema.ts. Consumers updated: - export.ts: --slug-prefix flag now goes engine-side (no in-memory .filter(...)). The --restore-only path queries each db_only directory with slugPrefix in a loop instead of one full-table scan, with seen-set deduplication and disk-existence check inline. - storage.ts: keeps the full-scan path because storage-status needs the "unspecified" bucket count, which can't be computed without enumerating every page. Comment notes that step 5 (single-walk filesystem scan) will reduce per-page disk syscall cost. 2 new test cases on PGLiteEngine: slugPrefix happy path (3 tier dirs, asserts only matching slugs return) and metacharacter escape regression (asserts safe/ doesn't match unrelated slugs). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * perf: single-walk filesystem scan via walkBrainRepo() (step 5/15) Issue #14 of the eng review: storage.ts called existsSync + statSync per-page in a synchronous loop. On a 200K-page brain that's 400K syscalls serialized. Wall-clock landmine. Adds src/core/disk-walk.ts with walkBrainRepo(repoPath) — one recursive readdirSync walk, builds a Map<slug, {size, mtimeMs}>. Storage.ts looks up each DB page in the map (O(1)) instead of stat-checking on demand. Slug derivation matches the pages-table convention: people/alice.md on disk becomes people/alice as the map key. Skipped during walk: - dot-directories (.git, .gbrain, .vscode, etc) — not part of the brain namespace - node_modules — guards against accidentally walking into imported repos - non-.md files (sidecar JSON, binaries) — tracked by the brain through the files table, not by slug Reusable: future commands (gbrain doctor's storage_tiering check, the optional autopilot tier-fix path) get the same walk for free. 9 new test cases: empty dir, nonexistent dir, top-level files, nested dirs, dot-dir skipping, node_modules skipping, non-.md filtering, size capture, mtimeMs capture. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: path-segment matching for tier directories (step 6/15) Issue #5 + D6 of the eng review: tier matching used slug.startsWith(dir), which falsely matches 'media/xerox/foo' against 'media/x' if a user wrote the directory without a trailing slash. The new matcher requires the configured directory to end with `/` and treats it as a canonical path-segment ancestor: media/x/ matches media/x/tweet-1 ✓ media/x/ doesn't media/xerox/foo ✗ media/x refused media/x/tweet-1 (matcher requires trailing /) Non-canonical input (no trailing slash) is refused outright. Step 7's auto-normalizing validator converts user-written 'media/x' → 'media/x/' on load, so the matcher never sees non-canonical input from real configs. The behavior tested here is the strict matcher's contract. Regression test pins the media/xerox collision case explicitly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: auto-normalize trailing-slash, throw on tier overlap (step 7/15) D7+D8 of the eng review: validation was warnings-only. Users miss warnings. Now: - Cosmetic: missing trailing slash auto-corrected, one-time info note showing what changed ("normalized 2 storage paths: 'people' → 'people/', 'media/x' → 'media/x/'"). Once-per-process to keep noise low. - Semantic: same directory in both tiers throws StorageConfigError. Ambiguous routing — does media/ win as db_tracked or db_only? — is a real bug the user must fix. Caller propagates to the CLI for a clean exit-1 with actionable message. loadStorageConfig now applies normalize+validate after merging deprecated keys, so the path-segment matcher (step 6) only ever sees canonical trailing-slash directories. The pure validateStorageConfig kept for callers who want the warnings list without the auto-fix side effects (gbrain doctor's reporting path). 2 new test cases: auto-normalize round-trip with warning text assertion, overlap throws StorageConfigError. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: wire manageGitignore into runSync, only on success (step 8/15) Issue #2 of the eng review: manageGitignore was defined and never invoked. Docs claimed "auto-managed by gbrain" — false. Users hit a .gitignore that never updated and committed db_only directories anyway. Wire-up: runSync now calls manageGitignore after each successful performSync return, in both watch and one-shot modes. Eng review pass-2 finding #1: skip on dry_run AND blocked_by_failures status. A sync that aborted partway has stale state; mutating .gitignore based on a partially-loaded config invites drift. Failure-skip test added (uses .gitignore-as-a-directory to simulate write failure; asserts warning fired and disk wasn't corrupted). Hardened manageGitignore itself with three additional behaviors: - GBRAIN_NO_GITIGNORE=1 escape hatch (D23) for shared-repo setups where a maintainer wants gbrain to leave .gitignore alone. - Submodule detection (D49). When repoPath/.git is a regular file (gitdir: ... pointer), the repo is a git submodule. Submodule .gitignore changes don't survive parent submodule updates, so we skip with an actionable warning ("add db_only directories to your parent repo's .gitignore manually"). - Graceful failure (D9). Read errors, write errors, and StorageConfigError (overlap from step 7) all log a warning and return — sync's primary job (moving data) shouldn't die because of a side-effect on .gitignore. manageGitignore is now exported (previously private) so the storage-sync test file can hit it directly without spinning up sync. 9 new test cases: no-op without gbrain.yml, no-op with empty db_only, happy-path append, idempotency (run twice, single entry), preservation of user-written rules, GBRAIN_NO_GITIGNORE skip, submodule skip, .git-directory normal path, write-failure graceful warning. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: D5 resolution chain for --restore-only and storage status (step 9/15) D5 of the eng review: gbrain export --restore-only without --repo silently fell through to the regular export path, dumping every page in the database to the wrong directory. Hard regression risk. Now exits 1 with an actionable message when --restore-only has no --repo AND no configured default source. Resolution order: 1. Explicit --repo flag 2. Typed sources.getDefault() (reuses step 1's accessor) 3. Hard error — never fall through to cwd storage.ts:38 also bypassed BrainEngine with raw SQL and a bare try/catch (Issue #3 + Issue #9). Replaced with the same typed getDefaultSourcePath() — single source of truth, errors propagate cleanly to the user, no silent cwd fallback. Regular export (no --restore-only) keeps its current behavior per D26: exports include everything, --repo is optional. 4 new test cases on PGLite in-memory: - hard-errors with no --repo + no default - explicit --repo wins - falls back to sources default local_path - non-restore export does not require --repo Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: split storage.ts into pure data + JSON + human formatters (step 10/15) Issue #10 of the eng review: getStorageStatus and runStorageStatus mixed data gathering, JSON serialization, and human-readable output in one function. Hard to test, hard to reuse, mismatched the orphans.ts pattern that CLAUDE.md cites as the precedent. Now three pure functions + a thin dispatcher: getStorageStatus(engine, repoPath) — async, returns StorageStatusResult. Side effects: engine.listPages + one walkBrainRepo (Issue #14). Exported so MCP exposure (D14) and gbrain doctor (D13) can consume the same data without re-running the loop. formatStorageStatusJson(result) — pure, returns indented JSON. Stable contract on the StorageStatusResult shape, suitable for orchestrators. formatStorageStatusHuman(result) — pure, returns ASCII text (D10 — no unicode box-drawing). Composable into other commands later. runStorageStatus(engine, args) — thin dispatcher: parses --repo / --json, calls getStorageStatus, picks a formatter, prints. 8 new test cases on the formatters: JSON parse round-trip, null-config fallback, missing-files capped at 10 with rollup, ASCII-only assertion (D10 regression guard), warnings inline, configuration listing, disk- usage block omitted when zero bytes. The StorageStatusResult interface is now exported as a public type, so gbrain doctor's storage_tiering check can build its own findings from the same shape. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * types: distinct PageCountsByTier and DiskUsageByTier (step 11/15) Issue #11 of the eng review: pagesByTier (page counts) and diskUsageByTier (byte totals) shared the same structural type (Record<StorageTier, number>). Both are tier-keyed numeric maps but carry semantically different units. A future bug that swaps them at a call site (e.g., displaying disk bytes where the count belongs) wouldn't trip the compiler. Replaced with distinct nominal types via a brand field. Structurally identical at runtime (no overhead) but compile-time disjoint — TypeScript catches accidental cross-assignment. PageCountsByTier { db_tracked, db_only, unspecified } : numbers (count) DiskUsageByTier { db_tracked, db_only, unspecified } : numbers (bytes) Both initialized in getStorageStatus, both threaded into StorageStatusResult, both consumed by formatStorageStatusHuman / formatStorageStatusJson without further changes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: PGLite soft-warn + full lifecycle test (step 12/15) D4: storage tiering on PGLite is a partial feature. The "DB" the pages live in IS the local file gbrain uses for everything else, so "db_only" has no real offload effect. The .gitignore management still helps (keeps bulk content out of git history), so we warn and proceed — not refuse. Two warning sites (once-per-process each via module-local flags): - storage status: warns at runStorageStatus entry - sync: warns inside manageGitignore when engineKind='pglite' and config has db_only entries Both phrased actionably ("To get full tiering, migrate to Postgres with `gbrain migrate --to supabase`"). manageGitignore signature now takes an optional `engineKind` param. runSync passes engine.kind. Stand-alone callers (tests, future gbrain doctor --fix path) can omit it. New test: test/storage-pglite.test.ts — D8 + D4 lifecycle. 6 cases: engine.kind assertion, getStorageStatus loading gbrain.yml + reporting tier counts, manageGitignore PGLite-warn (once per process), Postgres no-warn, slugPrefix on PGLite, end-to-end (config + putPage + status + gitignore). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: add trailing-newline CI guard (step 14/15) Issue #7 of the eng review: all four new files in the original storage-tiering branch lacked POSIX trailing newlines. Linters complain, git diffs phantom-flag every future edit. We've been adding newlines as each file landed; this commit catches the regression class. scripts/check-trailing-newline.sh: - sibling to check-jsonb-pattern.sh / check-progress-to-stdout.sh per CLAUDE.md's CI guard pattern - portable to bash 3.2 (macOS default; no mapfile, no associative arrays) - covers src/**, test/**, gbrain.yml, top-level *.md - reports each missing file by path and exits 1 Wired into `bun run test` between progress-to-stdout and typecheck. Also fixed docs/storage-tiering.md (pre-existing missing newline from the original branch — caught by the new guard on first run). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: v0.23.0 — VERSION, CHANGELOG, README, CLAUDE.md, storage-tiering.md (step 15/15) VERSION → 0.23.0 (minor bump for new feature surface). CHANGELOG entry in Garry voice with the canonical format: - Two-line bold headline ("Storage tiering, finally working...") - Lead paragraph naming what was broken before and what users get now - "Numbers that matter" before/after table for the 6 things that actually changed - "What this means for your brain" closer - "To take advantage of v0.23.0" self-repair block (per CLAUDE.md convention) — 6 numbered steps users can follow - Itemized changes split into critical fixes / new+renamed surface / architecture cleanup / tests + CI guards CLAUDE.md "Key files" gains four new entries: storage-config.ts, disk-walk.ts, the v0.23.0 storage.ts shape, and gbrain.yml itself. README.md gains a new "Storage tiering" section between Skillify and Getting Data In with the canonical example + commands + link to the full guide. docs/storage-tiering.md rewritten end-to-end with canonical key names (db_tracked / db_only), v0.23.0 hardening details (idempotency, submodule detection, GBRAIN_NO_GITIGNORE, dry-run gating), the resolution chain for --restore-only, the auto-normalize + throw-on-overlap validator, and the PGLite engine note. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test: e2e Postgres lifecycle for storage tiering (step 16/16) Per the v0.23.0 plan: full lifecycle E2E against real Postgres. - engine.kind === 'postgres' assertion - Full lifecycle: write 4 pages (1 db_tracked, 2 db_only, 1 unspecified) → getStorageStatus reports correct tier counts → human formatter renders → manageGitignore writes managed block → idempotency check → getDefaultSourcePath() resolves the configured local_path. - Container restart simulation: 2 db_only pages in DB, files missing on disk → status.missingFiles.length === 2 → slugPrefix engine filter on Postgres returns exactly the tier slugs. - slugPrefix index-based range scan regression: 50 media/x/* + 50 people/p-* pages → slugPrefix='media/x/' returns exactly 50. - getDefaultSourcePath returns null when default source has no local_path (the hard-error path that replaces the original silent cwd fallback). - manageGitignore on Postgres engine does NOT emit the PGLite soft-warn (cross-engine assertion). Skips gracefully when DATABASE_URL is unset, per CLAUDE.md E2E pattern. Run via: DATABASE_URL=... bun test test/e2e/storage-tiering.test.ts Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: rebump version 0.23.0 → 0.22.9 Reverts the minor bump back to a patch-style version on the v0.22 line. Storage tiering ships within the v0.22.x train alongside the recent fix waves. Updates VERSION, package.json, CHANGELOG header + body refs, CLAUDE.md Key files annotations, README.md section heading, and the docs/storage-tiering.md backward-compat note. * chore: bump version 0.22.9 → 0.22.11 Sibling workspaces claimed v0.22.10 in the queue. This branch advances to v0.22.11 to keep the version monotonic on master. Updates VERSION, package.json, CHANGELOG header + body refs, CLAUDE.md Key files annotations, README.md section heading, and the docs/storage-tiering.md backward-compat note. * fix: address Codex pre-landing review findings (4 fixes) Codex found 4 real issues during pre-landing review of v0.22.11 diff: [P0] export --restore-only fell through to full export when storageConfig was null (no gbrain.yml present). On older or misconfigured brains, the recovery command would silently dump the entire database. src/commands/export.ts now refuses with an actionable error before any page query fires — matches the D5 lock spirit ("never silently fall through"). [P1] manageGitignore wire-up only fired when --repo was passed explicitly. performSync resolves the repo from sync.repo_path or sources.local_path, so the common `gbrain sync` path (after setup, no flag) never updated .gitignore. src/commands/sync.ts now uses the same source-resolver chain as the rest of /ship: opts.repoPath → getDefaultSourcePath → null. Fires in both watch and one-shot modes. [P2] getDefaultSourcePath only consulted sources.local_path, missing the legacy global sync.repo_path config key that pre-v0.18 brains use. Added a fallback to engine.getConfig('sync.repo_path') when the sources row has NULL local_path. Pre-v0.18 brains now work without forcing a `gbrain sources add . --path .` migration. [P2] sync --all multi-source loop never called manageGitignore even though src.local_path was already known. Each source now gets its own gitignore update on successful sync. Tests: - test/storage-export.test.ts: replaced the old "falls through to full export" test with one that asserts the new refusal path (storage-tiering config required for --restore-only). - test/source-resolver.test.ts: added a fallback test exercising the legacy sync.repo_path code path for pre-v0.18 brains. - All 78 storage-tiering tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: regenerate llms.txt + llms-full.txt for v0.22.11 Per CLAUDE.md: "Run `bun run build:llms` after adding a new doc." The README's new Storage tiering section + the rewritten docs/storage-tiering.md changed the inlined bundle. test/build-llms.test.ts catches the drift and was failing on master pre-regen. * fix: typecheck error in disk-walk.ts (CI #73350475897) tsc --noEmit failed in CI because ReturnType<typeof readdirSync> with withFileTypes:true picks an overload union that includes Dirent<Buffer<ArrayBufferLike>>. Strict tsc treats entry.name as Buffer, so .startsWith / .endsWith / string comparisons all blew up. Annotate the variable as Dirent[] (string-based) and cast through unknown, matching the pattern sync.ts already uses for its own filesystem walk. Same runtime behavior; clean typecheck. Tests still 9/9. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> EOF --------- Co-authored-by: root <root@localhost> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
288 lines
11 KiB
TypeScript
288 lines
11 KiB
TypeScript
/**
|
|
* v0.18.0 Step 6 — source resolution priority tests.
|
|
*
|
|
* Priority order (highest first):
|
|
* 1. Explicit --source flag
|
|
* 2. GBRAIN_SOURCE env var
|
|
* 3. .gbrain-source dotfile walk-up
|
|
* 4. Registered source whose local_path contains CWD (longest prefix wins)
|
|
* 5. Brain-level `sources.default` config key
|
|
* 6. Fallback: literal 'default'
|
|
*/
|
|
|
|
import { describe, test, expect, beforeEach, afterEach } from 'bun:test';
|
|
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'fs';
|
|
import { join } from 'path';
|
|
import { tmpdir } from 'os';
|
|
import { resolveSourceId, getDefaultSourcePath, __testing } from '../src/core/source-resolver.ts';
|
|
import type { BrainEngine } from '../src/core/engine.ts';
|
|
|
|
// ── Stub engine ────────────────────────────────────────────
|
|
|
|
function makeStub(registeredSources: string[], paths: Array<{ id: string; local_path: string }>, defaultKey: string | null): BrainEngine {
|
|
return {
|
|
kind: 'pglite',
|
|
executeRaw: async <T>(sql: string, params?: unknown[]): Promise<T[]> => {
|
|
if (sql.includes('SELECT id FROM sources WHERE id = $1')) {
|
|
const target = params?.[0];
|
|
return (registeredSources.includes(target as string)
|
|
? [{ id: target } as unknown as T]
|
|
: []);
|
|
}
|
|
if (sql.includes('SELECT id, local_path FROM sources')) {
|
|
return paths as unknown as T[];
|
|
}
|
|
return [];
|
|
},
|
|
getConfig: async (key: string) => (key === 'sources.default' ? defaultKey : null),
|
|
} as unknown as BrainEngine;
|
|
}
|
|
|
|
// ── Priority 1: explicit flag ──────────────────────────────
|
|
|
|
describe('resolveSourceId priority 1 — explicit flag', () => {
|
|
test('wins over every other signal', async () => {
|
|
const engine = makeStub(['default', 'gstack', 'wiki'], [{ id: 'wiki', local_path: '/tmp' }], 'gstack');
|
|
process.env.GBRAIN_SOURCE = 'wiki';
|
|
try {
|
|
const id = await resolveSourceId(engine, 'gstack', '/tmp/whatever');
|
|
expect(id).toBe('gstack');
|
|
} finally {
|
|
delete process.env.GBRAIN_SOURCE;
|
|
}
|
|
});
|
|
|
|
test('rejects unregistered explicit source with actionable error', async () => {
|
|
const engine = makeStub(['default'], [], null);
|
|
await expect(resolveSourceId(engine, 'ghost')).rejects.toThrow(/not found/);
|
|
});
|
|
|
|
test('rejects invalid format', async () => {
|
|
const engine = makeStub(['default'], [], null);
|
|
await expect(resolveSourceId(engine, 'WRONG-case!')).rejects.toThrow(/Invalid --source/);
|
|
});
|
|
});
|
|
|
|
// ── Priority 2: env var ────────────────────────────────────
|
|
|
|
describe('resolveSourceId priority 2 — GBRAIN_SOURCE env', () => {
|
|
test('wins over dotfile / registered-path / default', async () => {
|
|
const engine = makeStub(['default', 'env-wins'], [{ id: 'other', local_path: '/tmp' }], 'default');
|
|
process.env.GBRAIN_SOURCE = 'env-wins';
|
|
try {
|
|
const id = await resolveSourceId(engine, null, '/tmp/x');
|
|
expect(id).toBe('env-wins');
|
|
} finally {
|
|
delete process.env.GBRAIN_SOURCE;
|
|
}
|
|
});
|
|
});
|
|
|
|
// ── Priority 3: dotfile walk-up ────────────────────────────
|
|
|
|
describe('resolveSourceId priority 3 — .gbrain-source dotfile walk-up', () => {
|
|
let tmpdirPath: string;
|
|
|
|
beforeEach(() => {
|
|
tmpdirPath = mkdtempSync(join(tmpdir(), 'gbrain-resolver-test-'));
|
|
});
|
|
afterEach(() => {
|
|
rmSync(tmpdirPath, { recursive: true, force: true });
|
|
});
|
|
|
|
test('finds dotfile in CWD', async () => {
|
|
writeFileSync(join(tmpdirPath, '.gbrain-source'), 'gstack\n');
|
|
const engine = makeStub(['default', 'gstack'], [], null);
|
|
const id = await resolveSourceId(engine, null, tmpdirPath);
|
|
expect(id).toBe('gstack');
|
|
});
|
|
|
|
test('walks up ancestors to find dotfile', async () => {
|
|
writeFileSync(join(tmpdirPath, '.gbrain-source'), 'wiki\n');
|
|
const deep = join(tmpdirPath, 'a', 'b', 'c');
|
|
mkdirSync(deep, { recursive: true });
|
|
const engine = makeStub(['default', 'wiki'], [], null);
|
|
const id = await resolveSourceId(engine, null, deep);
|
|
expect(id).toBe('wiki');
|
|
});
|
|
|
|
test('ignores dotfile with invalid content', async () => {
|
|
writeFileSync(join(tmpdirPath, '.gbrain-source'), 'INVALID!\n');
|
|
const engine = makeStub(['default'], [], null);
|
|
const id = await resolveSourceId(engine, null, tmpdirPath);
|
|
expect(id).toBe('default');
|
|
});
|
|
});
|
|
|
|
// ── Priority 4: registered local_path match (longest prefix) ──
|
|
|
|
describe('resolveSourceId priority 4 — registered local_path longest-prefix match', () => {
|
|
test('picks registered source whose local_path contains CWD', async () => {
|
|
const engine = makeStub(
|
|
['default', 'gstack'],
|
|
[{ id: 'gstack', local_path: '/tmp/gstack' }],
|
|
null,
|
|
);
|
|
const id = await resolveSourceId(engine, null, '/tmp/gstack/plans/foo');
|
|
expect(id).toBe('gstack');
|
|
});
|
|
|
|
test('longest prefix wins when paths are nested (per Codex second pass)', async () => {
|
|
// Codex flagged: overlapping paths need longest-prefix resolution.
|
|
// If gstack at /tmp/gstack and plans at /tmp/gstack/plans both
|
|
// exist, CWD inside plans/ must pick plans.
|
|
const engine = makeStub(
|
|
['default', 'gstack', 'plans'],
|
|
[
|
|
{ id: 'gstack', local_path: '/tmp/gstack' },
|
|
{ id: 'plans', local_path: '/tmp/gstack/plans' },
|
|
],
|
|
null,
|
|
);
|
|
const id = await resolveSourceId(engine, null, '/tmp/gstack/plans/deeper');
|
|
expect(id).toBe('plans');
|
|
});
|
|
|
|
test("CWD outside any registered path falls through to default", async () => {
|
|
const engine = makeStub(
|
|
['default', 'gstack'],
|
|
[{ id: 'gstack', local_path: '/tmp/gstack' }],
|
|
null,
|
|
);
|
|
const id = await resolveSourceId(engine, null, '/some/other/dir');
|
|
expect(id).toBe('default');
|
|
});
|
|
});
|
|
|
|
// ── Priority 5: brain-level default ────────────────────────
|
|
|
|
describe('resolveSourceId priority 5 — sources.default config key', () => {
|
|
test("returns configured default when no higher signal present", async () => {
|
|
const engine = makeStub(['default', 'custom'], [], 'custom');
|
|
const id = await resolveSourceId(engine, null, '/some/random/dir');
|
|
expect(id).toBe('custom');
|
|
});
|
|
});
|
|
|
|
// ── Priority 6: fallback ────────────────────────────────────
|
|
|
|
describe('resolveSourceId priority 6 — fallback', () => {
|
|
test("returns 'default' when no signal at all", async () => {
|
|
const engine = makeStub(['default'], [], null);
|
|
const id = await resolveSourceId(engine, null, '/random/dir');
|
|
expect(id).toBe('default');
|
|
});
|
|
});
|
|
|
|
// ── getDefaultSourcePath ───────────────────────────────────
|
|
|
|
describe('getDefaultSourcePath', () => {
|
|
function makeStubWithPaths(
|
|
registeredSources: string[],
|
|
sourcePaths: Record<string, string | null>,
|
|
defaultKey: string | null,
|
|
): BrainEngine {
|
|
return {
|
|
kind: 'pglite',
|
|
executeRaw: async <T>(sql: string, params?: unknown[]): Promise<T[]> => {
|
|
if (sql.includes('SELECT id FROM sources WHERE id = $1')) {
|
|
const target = params?.[0];
|
|
return (registeredSources.includes(target as string)
|
|
? [{ id: target } as unknown as T]
|
|
: []);
|
|
}
|
|
if (sql.includes('SELECT local_path FROM sources WHERE id = $1')) {
|
|
const target = params?.[0] as string;
|
|
if (target in sourcePaths) {
|
|
return [{ local_path: sourcePaths[target] } as unknown as T];
|
|
}
|
|
return [];
|
|
}
|
|
if (sql.includes('SELECT id, local_path FROM sources')) {
|
|
return Object.entries(sourcePaths)
|
|
.filter(([_, p]) => p !== null)
|
|
.map(([id, local_path]) => ({ id, local_path }) as unknown as T);
|
|
}
|
|
return [];
|
|
},
|
|
getConfig: async (key: string) => (key === 'sources.default' ? defaultKey : null),
|
|
} as unknown as BrainEngine;
|
|
}
|
|
|
|
test('returns local_path of resolved default source', async () => {
|
|
const engine = makeStubWithPaths(['default'], { default: '/path/to/brain' }, null);
|
|
const path = await getDefaultSourcePath(engine, '/random/dir');
|
|
expect(path).toBe('/path/to/brain');
|
|
});
|
|
|
|
test('returns null when source has no local_path', async () => {
|
|
const engine = makeStubWithPaths(['default'], { default: null }, null);
|
|
const path = await getDefaultSourcePath(engine, '/random/dir');
|
|
expect(path).toBeNull();
|
|
});
|
|
|
|
test('throws on DB error (does not silently swallow)', async () => {
|
|
const engine = {
|
|
kind: 'pglite',
|
|
executeRaw: async () => {
|
|
throw new Error('connection refused');
|
|
},
|
|
getConfig: async () => null,
|
|
} as unknown as BrainEngine;
|
|
await expect(getDefaultSourcePath(engine, '/random/dir')).rejects.toThrow(/connection refused/);
|
|
});
|
|
|
|
test('falls back to legacy sync.repo_path config when sources.local_path is null', async () => {
|
|
// Pre-v0.18 brains: 'default' source exists but local_path is NULL; the
|
|
// repo path lives in the global config table under sync.repo_path.
|
|
const engine = {
|
|
kind: 'pglite',
|
|
executeRaw: async <T>(sql: string, params?: unknown[]): Promise<T[]> => {
|
|
if (sql.includes('SELECT id FROM sources WHERE id = $1')) {
|
|
return [{ id: params?.[0] } as unknown as T];
|
|
}
|
|
if (sql.includes('SELECT local_path FROM sources WHERE id = $1')) {
|
|
return [{ local_path: null } as unknown as T];
|
|
}
|
|
if (sql.includes('SELECT id, local_path FROM sources')) {
|
|
return [];
|
|
}
|
|
return [];
|
|
},
|
|
getConfig: async (key: string) => {
|
|
if (key === 'sources.default') return null;
|
|
if (key === 'sync.repo_path') return '/legacy/brain/path';
|
|
return null;
|
|
},
|
|
} as unknown as BrainEngine;
|
|
const path = await getDefaultSourcePath(engine, '/random/dir');
|
|
expect(path).toBe('/legacy/brain/path');
|
|
});
|
|
|
|
test('respects source resolution chain (registered local_path wins over default)', async () => {
|
|
// CWD is inside /custom/path → wiki source matches by path → wiki's local_path returned.
|
|
const engine = makeStubWithPaths(
|
|
['default', 'wiki'],
|
|
{ default: '/default/path', wiki: '/custom/path' },
|
|
'default',
|
|
);
|
|
const path = await getDefaultSourcePath(engine, '/custom/path/sub');
|
|
expect(path).toBe('/custom/path');
|
|
});
|
|
});
|
|
|
|
// ── Regex validation ───────────────────────────────────────
|
|
|
|
describe('SOURCE_ID_RE', () => {
|
|
test('accepts valid ids', () => {
|
|
for (const id of ['default', 'wiki', 'gstack', 'yc-media', 'garrys-list', 'a', '123']) {
|
|
expect(__testing.SOURCE_ID_RE.test(id)).toBe(true);
|
|
}
|
|
});
|
|
test('rejects invalid ids', () => {
|
|
for (const id of ['', 'a'.repeat(33), 'Upper', 'has_underscore', 'trailing-', '-leading', 'with spaces', 'with.dots']) {
|
|
expect(__testing.SOURCE_ID_RE.test(id)).toBe(false);
|
|
}
|
|
});
|
|
});
|