* fix(sync): self-heal a never-git-initialized default brain dir (#2964)
The dream cycle's sync phase throws unconditionally on a legacy
sync.repo_path-anchored default brain dir that was never git init-ed
(predates git-backed sync, or was rsync'd without its .git), failing
every nightly run with no recovery. doctor's sync_freshness/
sync_consolidation checks report "ok" for this exact brain, but only
because they query the sources table (0 rows for a legacy default
brain) — a coincidental false-negative, not a real diagnosis.
Self-heal by git-initializing the dir and capturing the current
on-disk state as the sync baseline, scoped to !opts.sourceId only —
gbrain owns this directory outright, unlike a registered local source
(sources add --path, no --url) which is the user's own external
directory and should keep failing loudly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): dry-run no-write contract, unborn-HEAD recovery, no-gpg-sign (#2964)
Codex review on the initial self-heal patch (b1671ee) found 3 real gaps:
- P1: the self-heal ran even under --dry-run, mutating the filesystem
during what's documented as a preview-only command. Gated the whole
self-heal (both discoverGitRoot and the headCommit read) on
!opts.dryRun, same as the existing !opts.sourceId ownership check.
- P2: if `git init` succeeded but the process died before the baseline
commit landed, the next run's discoverGitRoot would succeed (`.git`
exists) and skip recovery entirely, permanently wedging on "No
commits in repo" forever. Added the same self-heal at the
`git rev-parse HEAD` catch site, sharing a new createSyncBaselineCommit
helper with the discoverGitRoot catch.
- P2: the baseline commit inherited the operator's global
commit.gpgSign, which can block headless cron/launchd runs on an
unavailable signing agent/pinentry. Added --no-gpg-sign.
Two new tests cover dry-run no-mutation and unborn-HEAD recovery.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): restrict git auto-init self-heal to the anchor-resolved path only (#2964)
Second Codex review round (a10eeab) found the ownership check still too
loose:
- P1 (security): !opts.sourceId alone isn't proof gbrain owns repoPath.
jobs.ts's `sync` job handler leaves sourceId undefined whenever
job.data.repoPath doesn't match a registered source's local_path, so
an admin-scope submit_job({name:'sync', data:{repoPath}}) MCP call
could point the self-heal at an arbitrary directory and have it
silently git-init + commit + ingest it. Gated both self-heal sites on
!opts.repoPath too — only the path resolved from gbrain's own
sync.repo_path anchor (never a caller-supplied one) is eligible.
- P2: the unborn-HEAD recovery site calls discoverGitRoot, which walks
UP from repoPath and can resolve to an ANCESTOR repo for a
--src-subpath/subdir-as-repoPath sync with an unborn HEAD. Committing
there would `git add -A` sibling files well outside the sync scope.
Added a check that gitContextRoot === realpathSync(repoPath) before
self-healing; refuses (falls through to the original error) otherwise.
Tests rewritten to exercise the true self-heal-eligible path (anchor
config via engine.setConfig('sync.repo_path', dir), no repoPath/sourceId
passed) instead of an explicit repoPath, plus a new test asserting a
caller-supplied repoPath with no sourceId still throws.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): prove self-heal ownership by anchor VALUE, not field presence (#2964)
Third Codex review round (613ad5b) found the previous round's fix broke
the very call site it was meant to repair, plus a second scope gap:
- P1 (critical): !opts.repoPath rejected self-heal on the REAL production
callers too. runPhaseSync (dream cycle's sync phase, cycle.ts) always
passes `repoPath: brainDir` explicitly after resolving it upstream, and
the CLI's bare `gbrain sync` resolves sourceId='default'. Both made the
ownership gate rethrow, leaving `gbrain dream` and `gbrain sync`
wedged on the exact non-git legacy brain this fix targets — only
synthetic callers that omitted both fields ever healed.
Fixed by proving ownership by VALUE instead of by field absence: a new
isAnchorOwnedSyncPath() re-reads gbrain's own persisted
sync.repo_path config and requires the resolved repoPath to equal it
exactly, regardless of whether the caller passed it explicitly or let
it default. An attacker-supplied arbitrary path (e.g. via
submit_job({name:'sync', data:{repoPath}})) only self-heals if it
happens to already equal gbrain's own anchor — which is the
legitimate case, not an escalation. opts.sourceId and opts.srcSubpath
still disqualify unconditionally (registered/subpath-scoped syncs are
a different ownership context).
- P2: a --src-subpath sync with an unborn parent-repo HEAD would commit
the whole ancestor root, capturing sibling files outside the scope.
isAnchorOwnedSyncPath's opts.srcSubpath check closes this; the
existing gitContextRoot === repoPath check stays as defense in depth.
- P2: manageGitignore's "warn and return" contract (a deliberate side-
effect that must never kill the sync job for its OTHER callers) meant
a broken gbrain.yml or unwritable .gitignore would silently let the
baseline `git add -A` commit db_only content. createSyncBaselineCommit
now recomputes db_only exclusion directly from loadStorageConfig and
passes it to `git add` as pathspecs, independent of the .gitignore
write's success — true fail-closed. (A redundant pathspec exclude for
a path .gitignore ALREADY covers makes git's -A bail with "paths
ignored, use -f" even though the negation is correct, so each dir is
check-ignore'd first and only pathspec-excluded when NOT already
covered.)
Tests rewritten around the anchor-VALUE model: the critical regression
case (explicit repoPath matching the anchor still heals — the exact
scenario Codex proved was broken) plus a true negative (a caller-supplied
path that does NOT match the anchor still throws), --src-subpath refusal,
and db_only fail-closed exclusion.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): allow default-source ownership, realpath compare, generous timeout, --no-verify (#2964)
Fourth Codex review round (9c1d461) found the previous round's ownership
gate still didn't match the REAL installed-brain shape, plus 3 more gaps:
- P1 (critical): rejecting all non-empty opts.sourceId meant self-heal
still never fired on a real brain. Migration sources_table_additive
seeds a 'default' source row whose local_path mirrors sync.repo_path
on every brain that's run it (virtually all of them), so
resolveSourceForDir (dream cycle) and bare `gbrain sync` both resolve
sourceId:'default' in practice, never undefined. isAnchorOwnedSyncPath
now permits sourceId undefined OR exactly 'default' (gbrain's own
bootstrap identity, never something a caller names) and proves
ownership by rereading the LIVE anchor for that same identity
(sources.default.local_path vs config.sync.repo_path).
- P2: compared raw anchor/repoPath strings, so a cosmetic difference
(trailing slash, ..) between the stored anchor and dream.ts's
path.resolve()-normalized brainDir would defeat the match. Now
realpath-compares both sides (fail-closed on ENOENT/dangling).
- P2: the shared git() helper's 30s timeout could abort the baseline
`git add -A` on a large legacy brain mid-way, after `git init` already
created `.git` — leaving an unborn repo every subsequent sync would
retry and time out identically forever. Added an optional timeoutMs
param (default unchanged at 30s); the baseline add call uses 10min.
- P2: the baseline commit could trigger an operator's global
core.hooksPath/init.templateDir hooks (pre-commit/commit-msg),
breaking headless recovery if those hooks need project tooling or
prompt. Added --no-verify.
Tests: rewrote the mis-scoped "registered source" test (it used
sourceId:'default', which is now correctly permitted) into two — a new
regression test proving sourceId='default' + matching local_path heals
(the actual production shape), and a corrected non-default-sourceId
refusal test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): defer .gitignore write past first import, rebuild index, fail closed on unparseable db_only (#2964)
Fifth Codex review round (c17cd23) found the baseline-commit helper
interacting badly with the pre-existing db_only storage-tiering feature:
- P1 (data loss): createSyncBaselineCommit called manageGitignore BEFORE
performFullSync's collectSyncableFiles ran. collectSyncableFiles
enumerates via `git ls-files --cached --others --exclude-standard`, so
writing db_only entries into .gitignore first would silently exclude
those pages from the DATABASE, not just from git — on a brain's very
first sync. This is the exact bug class runSync's existing "manage
.gitignore ONLY on successful sync" ordering (this file, ~line 4540,
itself a prior Codex P1 fix, comment literally says so) was written to
prevent — my new code reintroduced it in a different spot. Fix: stopped
calling manageGitignore inside the self-heal at all. db_only exclusion
for the COMMIT still happens via the existing pathspec computation
(independent of .gitignore); .gitignore itself gets written by the
already-existing post-sync flow once this sync completes, same as any
other sync.
- P1 (data leak): the unborn-HEAD recovery site can reach
createSyncBaselineCommit with a repo whose INDEX already has entries
staged from some prior operation (manual `git add`, interrupted
workflow) before gbrain's self-heal ever touched it. `git add -A`
only adds/updates — it doesn't drop an already-staged path our
exclusion pathspecs now want excluded. Added `git read-tree --empty`
to reset the index before staging (no-op on a freshly-`git init`-ed
repo, whose index is already empty).
- P2: loadStorageConfig warns-and-returns an EMPTY config (not a throw)
for syntactically-valid-but-unsupported YAML (e.g. flow-style
`db_only: [dir/]` — the narrow custom parser only handles block-style
lists), which would silently resolve zero exclusions from a gbrain.yml
that clearly intended some. Added a sniff-test: if gbrain.yml exists
and mentions db_only but nothing resolved from it, refuse the baseline
commit rather than guess "genuinely empty" vs "syntax silently
ignored" (git init may already have run by this point — same "unborn,
retry on next sync" recovery path handles it, and will hit this same
guard again until the user fixes gbrain.yml).
Tests: a positive regression proving db_only markdown IS imported into
the DB on first sync (the actual data-loss scenario), the sniff-test
refusal, and the stale-staged-content-gets-dropped case for the index
rebuild.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): post-heal .gitignore write, supabase_only alias, honor abort signal (#2964)
Sixth Codex review round (e687913), 3 P2s:
- Dream-cycle callers (cycle.ts:runPhaseSync) invoke performSync directly
and never run runSync's CLI-only post-success manageGitignoreAtGitRoot.
A brain self-healed only via the dream cycle would have db_only content
correctly excluded from the baseline commit (createSyncBaselineCommit's
pathspec exclusion) but no .gitignore ever written, leaving the user's
own future manual git add/commit unprotected. Added
performFullSyncAndMaybeGitignore, a thin wrapper around the 3
post-self-heal performFullSync call sites that writes .gitignore
(same success-status gate runSync already uses) only when didSelfHeal
is true — a no-op for the normal path, which still relies on runSync
exactly as before.
- The fail-closed sniff-test only checked the canonical `db_only` key;
the deprecated-but-still-supported `supabase_only` alias (same
keep-out-of-git semantics) could silently bypass it. Now checks both.
- Self-heal didn't check opts.signal?.aborted before starting the
(now up to 10-minute) git init + baseline commit, so a cancelled sync
could still mutate disk and overrun its budget instead of returning
partial. Added the check at both self-heal sites, before any git
operation runs.
New test proves .gitignore gets written after a bare performSync call
(no runSync wrapper) — the actual dream-cycle shape.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): neutralize a leftover .gitignore during the self-heal first sync (#2964)
Seventh Codex review round (27337b0) ran an actual repro and caught the
primary motivating scenario still broken: a brain rsync'd from another
machine without its .git can retain that machine's old auto-managed
.gitignore. collectSyncableFiles (inside performFullSync) enumerates via
`git ls-files --exclude-standard`, so a leftover db_only ignore rule
would silently omit those pages from THIS first sync's DATABASE import —
the same bug class the round-6 ordering fix prevented for a .gitignore
gbrain would have written itself, just triggered by a pre-existing file
this time.
Fix: performFullSyncAndMaybeGitignore now neutralizes any existing
.gitignore for the duration of the one first-sync call — read, delete,
restore byte-for-byte immediately after (even on error) — before
manageGitignore re-merges the managed db_only block onto the restored
original content. This matches exactly what a truly fresh brain with no
.gitignore at all already does on its first sync (nothing to suppress
collection there either); db_only content stays out of the git COMMIT
independently via createSyncBaselineCommit's pathspec exclusion, which
never depended on .gitignore.
Test proves both halves: db_only markdown IS imported despite a leftover
ignore rule, AND the user's own unrelated .gitignore lines (e.g.
.DS_Store) survive the restore intact.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): simplify — drop db_only-import machinery, isolate hooks fully (#2964)
Eighth Codex review round (54c1e6f) found MORE problems with round 7's
.gitignore-neutralization fix (deleting the whole file loses the user's
own unrelated ignore rules; a multi-sync retry scenario could silently
skip a still-broken db_only file while advancing the bookmark) plus 2
more issues in existing code. Rather than patch those too, stepped back
and checked the actual documented semantics of db_only
(docs/storage-tiering.md): it's for "bulk machine-generated content...
written to disk as a local cache" — DB is the source of truth, disk is a
cache populated FROM the DB (`export --restore-only` restores it), never
the other way. Nothing in the docs says `gbrain sync`'s git-diff-based
file collection is how db_only content is supposed to reach the
database — that's ingest-specific tooling's job. Confirmed directly:
`loadStorageConfig` returns the byte-identical `{db_tracked:[],
db_only:[]}` for a malformed flow-style array AND a literal empty
`db_only: []`, so rounds 6-7's "ensure db_only markdown gets imported on
this first sync" chase was solving a problem outside sync's actual scope
in the first place, on an increasingly complex, adversarially-discovered-
edge-case foundation.
Reverted: performFullSyncAndMaybeGitignore (the wrapper + didSelfHeal
tracking + .gitignore neutralize/restore dance + post-success
manageGitignore call). After self-heal, import and any subsequent
.gitignore management now behave EXACTLY like any other brain,
self-healed or not — runSync's existing post-success
manageGitignoreAtGitRoot covers the CLI path identically either way; the
dream cycle not calling it is a separate, pre-existing characteristic of
the dream cycle in general (applies equally to an already-git-initialized
brain going through the same path), not something this fix introduces.
Kept (still correct, self-contained, don't depend on the reverted
machinery): createSyncBaselineCommit's pathspec-based db_only exclusion
for the COMMIT itself (matches the documented "not committed to git"
requirement), the fail-closed sniff-test guard (now documents its
known, structurally-unavoidable false-positive on a genuinely-empty
`db_only: []` — the trade-off is deliberate: low-cost, self-resolving
false positive vs. high-cost, hard-to-undo false negative), the index
rebuild, and the 600s add timeout.
Improved (round 8, P2): hooks isolation. --no-verify only skips
pre-commit/commit-msg; added `-c core.hooksPath=/dev/null` for the
baseline commit, which disables prepare-commit-msg and post-commit too
(the latter runs synchronously inside the same git invocation and could
otherwise hang past the timeout without even being the slow step).
Tests: removed the 3 that exercised the reverted db_only-import
machinery; the remaining 12 (ownership, dry-run, index rebuild, sniff
test, commit-exclusion, unborn-HEAD recovery) are unaffected by the
simplification.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
* fix(sync): unconditional db_only exclusion, literal pathspecs, precise sniff test (#2964)
Ninth Codex review round (a6e07f6):
- P1: the check-ignore pre-filter (skip pathspec-excluding a dir already
covered by .gitignore) could be defeated by a pre-existing .gitignore
that ignores a db_only tree with a wildcard but re-includes a child via
negation (e.g. `private-cache/*` + `!private-cache/index.md`) —
check-ignore on the directory still reports "ignored", so the filter
skipped the pathspec exclusion, and `git add -A` staged the re-included
child anyway. Fixed by making exclusion unconditional: every db_only
dir is always pathspec-excluded now, never pre-filtered against
.gitignore state at all — our own pathspec doesn't consult .gitignore,
so no .gitignore content (negated or not) can defeat it. The advisory
"paths ignored... use -f" error this can now trigger when a dir IS also
already .gitignore'd (verified: git still stages everything else
correctly despite the nonzero exit) is caught and swallowed by matching
its exact stderr text; anything else rethrows.
- P2: `:!dir` pathspec shorthand reinterprets a dir name that itself
starts with a pathspec magic character (e.g. `:private/`) instead of
excluding it literally. Switched to `:(exclude,literal)dir`.
- P2: the fail-closed sniff-test's bare substring search on gbrain.yml's
raw content could trip on a comment or unrelated prose mentioning
"db_only" even when there's no real storage section at all, refusing
self-heal forever on an unrelated false positive. Now requires an
actual YAML key line (`db_only:`/`supabase_only:`, trimmed, ignoring
`#` comments) — the round-8-documented "genuinely empty db_only: []"
false positive is unchanged and remains an accepted trade-off (still
structurally indistinguishable from unsupported syntax at the
loadStorageConfig API boundary), but comment/prose mentions no longer
false-positive.
Not fixed (deliberately, documented trade-off — see PR description):
Codex's other P1 this round (refuse baselining when other git refs/
history exist alongside an unborn HEAD) is a narrow, non-destructive
scenario — self-heal only ever acts on the current branch ref when it's
provably commit-less, never touches or deletes any other ref (remote-
tracking, other branches), so at worst it creates a possibly-unexpected
extra commit on an otherwise-empty branch the user hadn't checked out
yet. Chasing it further trades diminishing real-world risk reduction
against unbounded scope growth in what's fundamentally still the
self-heal fix from round 1.
Two new tests: unconditional exclusion despite a matching pre-existing
.gitignore (proves the advisory-swallow path), and a comment-only
gbrain.yml no longer false-positives the sniff test.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCVEvUVm15bFuKqskWgfNS
---------
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
