3 Commits

Include renames

Author	SHA1	Message	Date
3 people	2934c53c1d	fix(sources): validate --path is a git repo with committed content at registration (#2707) (#2975) ... * fix(sources): validate --path is a git repo at registration time (#2707) `sources add --path <dir>` accepted any existing non-git directory with zero validation, deferring the failure to the first `gbrain sync` ("Not inside a git repository: ..."). By the time that surfaces, the source has been silently stale for however long nobody read the sync logs. Add a registration-time check (git-remote.ts:isInsideGitRepo, mirroring sync.ts's discoverGitRoot walk-up so subdir-of-git-repo sources still pass) that rejects an existing-but-non-git --path directory with an actionable error pointing at `git init && git add -A && git commit`. Non-existent paths are unaffected (out of scope — different, pre-existing failure mode) and `--force` opts out for callers who want to register before git-init exists. This is registration-time validation ONLY — it never auto-`git init`s the directory, preserving the consent boundary #2967 established for sync-time self-heal (a --path source is the user's own external directory; gbrain must not mutate it without explicit ask). Also documents the git requirement (docs/guides/multi-source-brains.md), including the "files must be committed, not just present" gotcha and that a stale/unreachable sync anchor already self-heals on plain `gbrain sync` (verified manually against HEAD — no reset-anchor command needed). * fix(sources): require a committed HEAD + shell-quote remediation cmd (codex round 1) Codex review round 1 on #2707 found two real gaps: 1. isInsideGitRepo alone accepts a `git init`ed-but-never-committed directory (rev-parse --show-toplevel succeeds with no HEAD), so registration would still pass a source that fails sync's own "No commits in repo ... Make at least one commit before syncing." Add hasGitCommits (git rev-parse HEAD) as a second required check. 2. The remediation command in the error message interpolated the raw path unquoted — spaces, $(), backticks, etc. would break or, worse, execute unintended shell syntax if pasted. POSIX single-quote it (mirrors src/commands/connect.ts:shellQuote; duplicated locally rather than imported, since commands/ depends on core/ not the reverse). * fix(sources): require tracked content in HEAD, not just a resolvable HEAD (codex round 2) Codex review round 2 P1: hasGitCommits (rev-parse HEAD) accepted a repo with an empty commit (git commit --allow-empty) followed by untracked files — HEAD resolves fine (to git's well-known empty-tree object), so registration passed, but the first sync would "succeed" importing nothing and then silently never notice the untracked files change. The exact same gap applied to an untracked subdirectory of an otherwise- real git repo (monorepo case). Replace hasGitCommits with hasTrackedContent (`git ls-tree HEAD -- .`, non-recursive — one entry is enough, no need to walk the whole subtree). `-C path` + pathspec `.` scopes correctly to both a repo toplevel and a subdirectory-of-a-repo source, and an empty tree lists zero entries where a bare `rev-parse HEAD` would still succeed. Also subsumes the "no commits at all" case hasGitCommits covered (ls-tree on an unborn repo fails the same way), so this is one check instead of two. Updated the error copy and docs/guides/multi-source-brains.md to match what's actually verified now. * fix(sources): O(1)-output tree-emptiness probe, avoid maxBuffer overflow (codex round 3) Codex review round 3 found the round-2 `git ls-tree HEAD -- .` listing buffers the whole (non-recursive) tree — a real repo with ~17-20K directly-tracked entries exceeds execFileSync's default 1 MiB maxBuffer, throws ENOBUFS, and the catch-all incorrectly rejects a perfectly valid registration. Replace the listing with `git rev-parse --verify HEAD:./` (resolves the tree object for `path` specifically, correct for both toplevel and subdirectory sources same as before) compared against git's canonical empty-tree SHA-1 (4b825dc6...) — a fixed ~40-byte read regardless of how many entries the tree has, structurally immune to this class of bug rather than just raising the threshold. Added a 300-file regression test locking this in. Declined a second round-3 finding (P1: reject a tree if ANY untracked file exists anywhere under the path, not just when the tree is entirely empty) — untracked files never being synced is standard, existing git-source behavior throughout this codebase (identical for --url managed clones), not a bug specific to this validation. Enforcing zero-untracked-files at registration would reject ordinary repos with gitignored build output, .DS_Store, editor swapfiles, etc. Out of scope relative to what #2707 actually asks for (a directory with real, committed content that will sync) and how every other git source in this system already behaves. * fix(sources): derive empty-tree OID per repo instead of hardcoding SHA-1 (codex round 4) Codex review round 4 P2, confirmed by directly testing against a `git init --object-format=sha256` repo: the hardcoded SHA-1 empty-tree constant only matches SHA-1 repositories. An empty SHA-256 repo's real empty-tree OID is a different (64-char) hash, so the SHA-1 comparison silently mismatched and let an empty/untracked SHA-256 source through — exactly the case this validation exists to catch. Replace the constant with `emptyTreeOid()`: `git hash-object -t tree --stdin < /dev/null` computed in the target repo's own context, so it returns the correct empty-tree OID for whichever object format that repo actually uses, without gbrain needing to know or care which one. Added gated regression tests (git 2.29+ / --object-format=sha256, test.skipIf on older git) for both the empty-repo-rejected and real-content-registers-fine cases. Converging here (4 review rounds; this is the last outstanding finding from round 4, and round 4 raised only this one issue). * fix(test): --force the incidental non-git second source in #1434 routing test (#2707) CI caught a real regression from this PR's registration-time git validation: test/sync-sole-non-default-routing.test.ts's "2+ non-default sources" case registers a bare mkdtempSync temp dir (no git init) as a second source purely to have 2 sources present — the directory's content was never meant to be exercised, only its existence as a distinct local_path. #2707's new validation correctly rejects that dir at registration time, since nothing else in the test suite told it otherwise. --force is the right fix, not adding unnecessary git-init/commit boilerplate to secondRepo: it documents that this specific registration intentionally doesn't care about git-validity, matching what a real caller opting into the legacy lenient behavior would do. Verified: the specific test (3/3 pass), plus every other test file in the repo using `sources add --path` (sources.test.ts, sources-ops.test.ts already covered by the PR's own commits; repos-alias.test.ts, sync-cost-gate.serial.test.ts — 11/11 pass, no similar fixture gap). typecheck clean, verify 31/31. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-20 16:28:53 -07:00
3 people	7ea92d602c	v0.42.48.0 feat(durability): auto-harden brain repos for git durability on PAT+URL (#2241) ... * feat(git): divergence-safe pull, push-probe, default-branch detection for brain durability Add GIT_ENV_AUTH + divergenceSafePull (skip-on-dirty, conflict-abort-clean, never-mid-rebase), detectDefaultBranch, pushProbe, and an env-gated GBRAIN_GIT_ALLOW_FILE_TRANSPORT escape hatch. Export GIT_ENV. pullRepo's --ff-only contract is unchanged. * feat(durability): brain-repo hardening core (hook, helper, cron, PAT, AGENTS rules) hardenBrainRepo/unhardenBrainRepo: local untracked post-commit hook + committed brain-commit-push.sh (one shared push-retry template), repo-scoped credential with existing-helper reuse, push-probe verify, active-resolver-file rules with taxonomy from _brain-filing-rules.json, minimal DB-free pull cron. PAT redaction via redactSecretsInText. * feat(sources): harden/pull/unharden commands + auto-harden on add --url sources harden/pull/unharden subcommands; --pat-file/--no-harden on add; auto-harden managed clones on add; unharden-before-remove. cli.ts pre-connect early-exit for DB-free 'sources pull --path' (the cron entry, never opens PGLite). * test(durability): unit + integration coverage for brain-repo durability git helpers, core harden/unharden, hook+helper E2E (real background push), cron generators. 41 tests across 4 files. * chore: bump version and changelog (v0.42.48.0) Brain-repo git durability: auto-harden a brain's working tree (local auto-push hook, committed commit-push helper, always-on agent rules, DB-free pull cron, repo-scoped credential, push-probe verify) the moment gbrain gets a PAT + URL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(sources): route harden exit code through setCliExitVerdict A raw process.exitCode write is zeroed by the owned-verdict flush-exit (#2084 PGLite-Emscripten pollution defense); cli-exit-verdict-pin guard caught it. Use setCliExitVerdict(3) so 'sources harden' actually reports needs-attention to cron/automation. * docs: document brain-repo durability (KEY_FILES + multi-source guide) KEY_FILES: extend git-remote.ts entry (divergenceSafePull, pushProbe, detectDefaultBranch, GIT_ENV_AUTH, GBRAIN_GIT_ALLOW_FILE_TRANSPORT) + add brain-repo-durability.ts/sources-harden.ts entry. multi-source-brains.md: add a Durability (auto-harden) how-to covering sources harden/pull/unharden, --pat-file, the guarantees, and the security posture. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-06-17 06:41:37 -07:00
3 people	90c5d93fce	feat: v0.18.0 — multi-source brains (one DB, many repos, federation + dotfile resolution) (#337) ... * feat(v0.17.0 step 1/9): sources primitive — additive-only multi-source foundation Lane A of the multi-repo plan. Installs the sources table and seeds a 'default' row that inherits sync.repo_path/last_commit from existing config. This is the bisectable foundation every later step builds on; the breaking schema changes (composite UNIQUE, files FK rewrite, resolution_type, ingest_log.source_id) land with their paired code rewrites in Steps 2/4/5/7 so no single commit breaks the engine. - migration v16 (sources_table_additive) + v0_17_0 orchestrator skeleton - sort-by-version guard in runMigrations (array insertion order can never cause a later migration to skip a lower one again) - default source seeded with config '{"federated": true}' so pre-v0.17 brains keep single-namespace search semantics after upgrade - orchestrator phase B detects absence of file_migration_ledger and no-ops until Step 7 lands it - 8 new structural tests in test/migrate.test.ts (shape, idempotency, scope-guard that nothing else was smuggled into v16) - apply-migrations tests include v0.17.0 in the registered list Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 2/9): pages.source_id + composite UNIQUE (Lane B) Migration v17 adds pages.source_id with DEFAULT 'default' and swaps the global UNIQUE(slug) for composite UNIQUE(source_id, slug). Ships atomically with the engine's ON CONFLICT rewrite so the constraint swap and the code that writes under it land in the same commit — no window where the engine sees one shape and the schema has another. Minimum-surface engine change: only putPage's ON CONFLICT target needs re-targeting. Other slug-based queries work unchanged because single- source brains (the only brain shape pre-Step-5) have exactly one source 'default', so slug remains effectively unique within it. Step 5+ will surface an explicit sourceId param on putPage for cross-source sync. - migration v17 (pages_source_id_composite_unique) in src/core/migrate.ts - pages.source_id + composite UNIQUE added to schema.sql + pglite-schema.ts for fresh installs - ON CONFLICT (slug) → ON CONFLICT (source_id, slug) in both pglite-engine and postgres-engine putPage - DEFAULT 'default' closes the Codex-flagged race where an INSERT between ADD COLUMN and SET NOT NULL could leave source_id NULL - 5 new v17 structural tests (29 pass / 0 fail in migrate.test.ts) - Full suite: 1979 pass / 3 fail (same as baseline — no regressions) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 6/9): sources CLI + source-resolver (Lane C) Adds the CLI surface for multi-source management. Users can now register, list, rename, federate/unfederate, and attach-to-directory a source. The source-resolver is the shared 6-priority helper that Steps 4/5 will use when they start surfacing an explicit --source flag on sync/extract/query. Commands: gbrain sources add <id> --path <p> [--name <n>] [--federated|--no-federated] gbrain sources list [--json] gbrain sources remove <id> [--yes] [--dry-run] [--keep-storage] gbrain sources rename <id> <new-name> gbrain sources default <id> gbrain sources attach <id> — writes .gbrain-source in CWD gbrain sources detach gbrain sources federate <id> / unfederate <id> Resolution priority (source-resolver.ts) — highest first: 1. --source flag 2. GBRAIN_SOURCE env 3. .gbrain-source dotfile walk-up 4. longest-prefix match on registered local_path (Codex #2 fix) 5. sources.default config 6. fallback 'default' - add: validates id format (kebab-case alnum, 1-32), rejects overlapping paths (eng review §4 finding 4.1), supports federated default opt-in - remove: guards against --yes omission + refuses to remove 'default', supports --dry-run, reports cascade page count - attach/detach: matches kubectl/terraform context-pinning semantics - Throws on overlap rather than process.exit() so the CLI error wrapper reports it consistently (also makes unit testing clean) 28 new tests across sources.test.ts (dispatcher + validation + overlap guard) and source-resolver.test.ts (full 6-priority coverage including longest-prefix). Full suite: 2012 pass / 3 fail (pre-existing PGLite infra timeouts). NOT in scope for Step 6 (deferred): - import-from-github (SSRF + clone integration) - prune (retention/TTL, lands v0.18) - MCP tool-defs regen for source-scoping on read ops (Step 5) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(v0.17.0 step 8/9): getting-started guide + migration skill + citation rule Step 8 (Lane F) documents what Steps 1+2+6 have shipped and sets up the agent-facing rules for multi-source. New files: - skills/migrations/v0.17.0.md — migration skill read by host agents after `gbrain apply-migrations`. Covers the v16+v17 chain, what's in v0.17.0 vs what lands later (v0.17.1 ACL, v0.18 sessions), and the new sources CLI surface. Cites docs/guides/multi-source-brains.md as the recipe. - docs/guides/multi-source-brains.md — getting-started for end users. Three canonical scenarios (unified wiki+gstack / purpose-separated yc-media+garrys-list / mixed), full resolution priority, federation flag semantics, command reference, and citation format. skills/brain-ops/SKILL.md — new "Cross-source citation format" section mandating `[source-id:slug]` when the brain has multiple sources. Matches the contract the /plan-devex-review DX review pinned down (DX Finding 5: surface source_id in every page payload + citation contract). Key must be sources.id (immutable), never sources.name. No behavior change — this is pure documentation for what already exists in the binary. 144 skills conformance tests still pass. NOT in this commit (deferred to later steps): - docs/guides/repo-architecture.md rewrite (lands with the full v0.17.0 PR description + release notes) - skills/_brain-filing-rules.md "which source to file into" guidance (lands with Step 5 when sync surfaces --source) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 5/9): sync --source <id> routes through sources table (Lane D) Adds the --source flag to `gbrain sync`. When set, sync reads local_path + last_commit from the matching sources(id) row instead of the global sync.repo_path / sync.last_commit config keys, and writes last_commit + last_sync_at back to the same row. Backward compat: --source omitted = pre-v0.17 behavior exactly, global config path unchanged. - SyncOpts.sourceId threaded through performSync + performFullSync - readSyncAnchor/writeSyncAnchor helpers centralize the sources-vs-config branch so every read/write goes through one decision point. Makes Step 5's later per-source sync-failures tracking a one-file change. - --source resolved via src/core/source-resolver.ts (Step 6), so any command that shell-exposes resolveSourceId gets env var + dotfile walk-up + longest-prefix for free. - Error message for missing source local_path is actionable: Source "gstack" has no local_path. Run: gbrain sources add gstack --path <path> - last_sync_at auto-updates on every last_commit advance so `gbrain sources list` shows real recency. No regression: 2012 pass / 3 fail (same as baseline). NOT in this commit (deferred per plan): - Per-source failure tracking (~/.gbrain/sources/<id>/sync-failures.jsonl) - runImport source-awareness (import.ts path — Step 5 continuation) - Partial-success semantics when walking N sources — single-source flow today, multi-walk lands when the top-level `gbrain sync` without --source starts iterating all sources. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 4/9): qualified [[source:slug]] + links.resolution_type (Lane B) Adds source-pinned wikilink syntax and records the resolution kind on each edge so `gbrain extract --refresh-unqualified` (future) can re-resolve bare references when the source topology changes. Wikilink syntax extension: [[concepts/ai]] — unqualified; resolves via local-first fallback [[wiki:concepts/ai]] — qualified; target pinned to sources.id='wiki' [[gstack:projects/foo|Display]] — qualified + display name The qualified regex runs first and masks matched spans so the unqualified pass can't double-emit. Source id format enforced to match the sources CLI validation: [a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])? Schema: - migration v18 adds links.resolution_type TEXT with CHECK constraint ('qualified'|'unqualified' or NULL for legacy/manual/frontmatter edges) - schema.sql + pglite-schema.ts updated for fresh installs EntityRef type: - sourceId is OPTIONAL (only set on qualified wikilinks). Markdown [Name](path) and unqualified wikilinks omit it so strict toEqual tests pre-v0.17 keep working (69 existing tests still pass). Tests: - 5 new qualified-wikilink extraction tests + 1 migration v18 structural assertion. 75 tests in test/link-extraction.test.ts (up from 69). - Full suite: 2018 pass / 3 fail (pre-existing PGLite infra timeouts). NOT in this commit (deferred to Step 3 / Step 5 continuation): - Writing resolution_type to the DB (addLink / addLinksBatch don't carry the field yet — that's the plumb-through that lands with Step 3 when search/dedup also needs source-aware result keys). - `gbrain extract --refresh-unqualified` re-resolver. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 3/9): source-aware search dedup composite keys (Lane B) Search dedup now keys on (source_id, slug) instead of slug alone. Pre- v0.17 would collapse two same-slug pages in different sources into one, destroying cross-source recall. Codex outside-voice review flagged this as regression-critical — this commit ships the fix plus tests that lock the invariant in. Dedup pipeline (src/core/search/dedup.ts): - pageKey(r) helper — one canonical composite-key derivation. Falls back to source_id='default' for pre-v0.17 rows so single-source brains behave identically to before. - Layer 1 (dedupBySource): group-by composite key. - Layer 4 (capPerPage): count-by composite key. - guaranteeCompiledTruth: swap scoped to matching (source_id, slug), so wiki:topics/ai can't accidentally pull gstack:topics/ai's compiled_truth chunk. SearchResult type gains optional source_id — populated by SQL JOINs in both engines, falls through as 'default' for legacy callers. Engine SQL: - pglite-engine.ts + postgres-engine.ts: search SELECTs add p.source_id - rowToSearchResult (utils.ts): maps row.source_id → result.source_id when present. Shape stays backward compatible (field optional). Tests — 4 new in test/dedup.test.ts: - same-slug-different-source does NOT collapse (the critical regression guard Codex called out) - same-slug-same-source DOES still collapse (no over-correction) - missing source_id falls back to 'default' for pre-v0.17 compat - compiled_truth guarantee scopes to composite key (Codex second pass caught this specific path would leak otherwise) Full suite: 2022 pass / 3 fail (3 pre-existing PGLite infra timeouts). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(v0.17.0 step 7/9): file_migration_ledger + phase-B storage backfill (Lane E) Adds files.source_id + files.page_id + the file_migration_ledger state machine that drives storage object rewrites. Each per-file transition is its own transaction so crash-point recovery is a ledger read, not a filesystem inspection. Codex second-pass review flagged that "skip if already has source prefix" was an unsafe heuristic — the ledger replaces it with explicit state tracking. Schema: - migration v19 (files_source_id_page_id_ledger): handler-only (PGLite has no files table; Postgres-only gate). ADDs source_id + page_id to files, backfills page_id from page_slug scoped to source_id='default', creates file_migration_ledger with PK on file_id (Codex: not storage_path_old — two sources can share an old path during migration). - schema.sql updated for fresh Postgres installs; file_migration_ledger gets RLS alongside other tables. Runtime: - src/commands/migrations/v0_17_0-storage-backfill.ts: drives the ledger state machine pending → copy_done → db_updated → complete. Idempotent per row: re-running resumes from whichever state crashed. Old objects preserved (no delete) so operators can verify the soak window before a future cleanup release. - phase B in v0_17_0.ts orchestrator: wires the storage backend (Supabase/S3/local) through createStorage, runs runStorageBackfill, reports per-state counts + first-three error details. Tests — 13 new in test/storage-backfill.test.ts: - pending → copy_done → db_updated → complete happy path - 3 crash-point recovery tests (resume from copy_done, resume from db_updated, failed rows don't auto-retry) - already-complete rows are skipped with zero side effects - idempotent re-upload (exists-check skips redundant upload) - dry-run mode (no storage, reports counts without mutating) Plus 5 new migrate.test.ts assertions for v19 structure (handler- only, PGLite gate, source_id + page_id + ledger DDL, default-source backfill scope, state machine values). Full suite: 2035 pass / 3 fail (3 pre-existing PGLite infra timeouts). NOT in this commit (explicitly deferred): - DROP old page_slug column — kept for backward compat until operators have time to verify page_id everywhere. - DROP old UNIQUE(storage_path) in favor of UNIQUE(source_id, storage_path) — same reason, deferred to later cleanup. - Actual cleanup phase that deletes old objects post-soak. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(v0.17.0 step 9/9): full multi-source PGLite integration suite (Lane G) End-to-end exercise of every v0.17.0 surface against real PGLite (in-memory, fast — no DATABASE_URL needed). The migration chain v2→v19 runs start-to-finish and the test asserts each Step's invariants hold together. 16 new integration tests across 7 describes: 1. Migration-installed state: - sources('default') exists with federated=true config - pages.source_id column has DEFAULT 'default' - composite UNIQUE (source_id, slug) is installed 2. Default-source write path: - putPage without explicit source → source_id='default' via schema default clause (no engine API change needed for single-source brains) 3. Composite UNIQUE regression guards (Codex-flagged): - Same slug in two different sources coexists - Third insert with same (source_id, slug) hits the UNIQUE constraint 4. sources CLI round-trip: - federate / unfederate flips config.federated - rename changes display, id stays immutable 5. Source resolution priority (integration): - Explicit flag > env var > fallback to default - Unregistered explicit source errors with actionable message 6. Cascade semantics: - sources remove cascades to pages; default source untouched 7. links.resolution_type (Step 4): - Qualified/unqualified values accepted - CHECK constraint rejects invalid values All 16 tests pass. Full suite: 2042 pass / 4 fail (4 pre-existing PGLite beforeEach timeouts in test/wait-for-completion, test/extract-fs, test/e2e/search-quality, test/e2e/graph-quality — count fluctuated 3-5 on baseline from variance alone). Total new tests across Steps 1-9: ~85 unit + integration tests (sources, source-resolver, migrate v16/v17/v18/v19 structural, link-extraction qualified wikilinks, dedup regression-critical, storage-backfill state machine + crash recovery, full multi-source PGLite integration). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump to v0.18.0 + CHANGELOG entry (multi-source brains) One-viewport release summary + itemized changes covering all 9 steps of the multi-source primitive. Notes the v0.17 → v0.18 version bump rationale (master shipped gbrain dream as v0.17 while this branch was in flight). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): v0_18_0 orchestrator TS narrow + mechanical test ON CONFLICT Two CI failures on PR #337: 1. tsc TS2367 at src/commands/migrations/v0_18_0.ts:190 — after the early-return on `a.status === 'failed'` (line 179), TypeScript narrows `a.status` to `'skipped' | 'complete'`, so the subsequent `a.status === 'failed' ? 'failed' :` branch was dead code and refused to compile. Dropped the redundant check. 2. E2E `file_list LIMIT enforcement` at test/e2e/mechanical.test.ts:636 — the test pre-seeded a pages row with `ON CONFLICT (slug) DO NOTHING` but v21 swapped the global UNIQUE for `UNIQUE (source_id, slug)`, so Postgres rejects with "no unique or exclusion constraint matching". Updated the conflict target to the composite key. Tier-1 E2E had only this one failing test; everything else passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(e2e): v0.18.0 multi-source against real Postgres (v20-v23 schema + cascade + sync) Closes the three biggest confidence gaps the author flagged in the self-audit of PR #337: 1. No real Postgres E2E — PGLite has no files table, so v23's files.source_id + files.page_id rewrite + file_migration_ledger seed was NEVER executed against the real DB. This file covers it. 2. `gbrain sync --source <id>` had zero direct tests. Now has two: one that asserts performSync({sourceId}) reads local_path from the sources row (not the global config), one that asserts no-sourceId falls back to the global sync.repo_path. 3. Cascade delete coverage — previously verified only pages count after source removal. Now verifies pages + content_chunks + timeline_entries + links + files ALL cascade-delete when a source is removed. 6 describes, 16 tests total: - Schema shape (fresh install): 6 tests confirming sources('default'), pages.source_id NOT NULL with DEFAULT, composite UNIQUE pages (source_id, slug) replaces global UNIQUE(slug), links.resolution_type column + CHECK, files.source_id + page_id columns, file_migration_ledger table + status CHECK. - Composite UNIQUE semantics: 3 tests confirming same-slug in two sources coexists (Codex-critical regression guard), duplicate (source_id, slug) hits the UNIQUE, putPage targets default source by schema DEFAULT. - Cascade delete: 1 test building a fully populated source (2 pages, chunks, timeline, links, files) then removing it + asserting every dependent row is gone. - Sync routing: 2 tests confirming performSync({sourceId}) reads per-source local_path vs global config. - Sources surface: 3 tests for federate/unfederate flipping + rename preserving id. - Storage backfill: 1 end-to-end test seeding ledger + running runStorageBackfill against a stub StorageBackend, asserting pending → complete transition and files.storage_path rewrite. Gated by DATABASE_URL per CLAUDE.md E2E lifecycle. Each describe's beforeAll defensively DELETEs non-default sources + file_migration_ledger rows so reruns are hermetic (sources isn't in helpers.ALL_TABLES). Verified: 16/16 pass on first run AND second run (residual-state fix holds). Full E2E suite still green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(ci): TS2352 in multi-source E2E — cast postgres.js RowList via unknown tsc rejects the direct `(rows as { column_name: string }[]).map(...)` cast because postgres.js RowList rows have an iterable-row shape that doesn't overlap with the plain-object target. Standard fix: cast via `unknown` first so the narrowing is explicit. Verified: `bunx tsc --noEmit` clean (ignoring the pre-existing baseUrl deprecation warning). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(v0.18.0): addLinksBatch + addTimelineEntriesBatch source-aware JOINs Batch APIs JOINed on pages.slug globally, so two pages sharing the same slug across sources would silently fan out — addLinksBatch(['a->b']) in a brain with 'a' in both 'default' and 'alt' wrote 2 edges instead of 1. Same bug on addTimelineEntriesBatch. Fix: - LinkBatchInput + TimelineBatchInput gain optional source_id fields (from_source_id, to_source_id, origin_source_id for links; source_id for timeline). All default to 'default' so existing callers are backward-compatible on single-source brains. - pglite-engine + postgres-engine batch JOINs now composite-key on (slug, source_id). Postgres adds 3 more unnest arrays for links + 1 for timeline — still one bind per column, no 65535-param cap risk. - LEFT JOIN for origin pages also source-qualified so frontmatter- provenance edges don't cross-pollinate across sources. Regression coverage: - test/pglite-engine.test.ts: 5 new tests covering default-path isolation, explicit alt-source writes, and cross-source edges. - test/e2e/multi-source.test.ts: 4 new tests against real Postgres so postgres-js's unnest() bind path is exercised (structurally different from PGLite's). Gap #4 from the PR self-audit — latent bug, not previously reachable because every existing caller wrote to the default source only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-04-22 16:24:23 -07:00