mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
chore: bump version and changelog (v0.42.15.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
920e5f6ad5
commit
4caa38da2c
@@ -2,6 +2,54 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.15.0] - 2026-06-02
|
||||
|
||||
**Your nightly `gbrain dream` stops silently losing every database phase.** If you run gbrain on Postgres (local or Supabase), the dream cycle has been quietly failing: the `lint` and `backlinks` phases work, then `sync`, `synthesize`, `embed`, and the rest all blow up with `No database connection: connect() has not been called`. The extract phase reports "created 0 links" while actually dropping every row. Run the same phases one at a time in separate commands and they all work — only the full cycle breaks. The result: your brain quietly stops staying up to date, and the cycle leaves a stuck lock behind.
|
||||
|
||||
The cause turned out to be a single sentence of logic. The dream cycle opens one long-lived database connection and reuses it for the whole run. But along the way, small helper steps (the lint config probe, doctor checks) briefly open their own connection handle. When those short-lived helpers finished and closed *their* handle, they were actually closing the *shared* connection the rest of the cycle still needed. Every later phase then found the connection gone.
|
||||
|
||||
This release teaches gbrain who actually owns the shared connection. Only the engine that opened it is allowed to close it; the short-lived helpers leave it alone. The fix lands automatically:
|
||||
|
||||
```
|
||||
gbrain upgrade
|
||||
gbrain dream --dir <your-brain> # every DB phase now reports ✓, lock releases cleanly
|
||||
```
|
||||
|
||||
Nothing to configure. If your dream cycle was the broken kind, it just starts working.
|
||||
|
||||
### Under the hood
|
||||
|
||||
The shared connection is the module-level `sql` singleton in `src/core/db.ts`. It was only ever nulled by `db.disconnect()` — postgres.js auto-reconnects its own internal pool and never touches our reference — so the singleton going null mid-cycle was always a "borrower" engine's disconnect cascading into `db.disconnect()`, never an idle-pooler drop. `PostgresEngine.disconnect()` called `db.disconnect()` for any `_connectionStyle === 'module'` engine with no check for whether that engine actually created the singleton.
|
||||
|
||||
The fix adds an ownership token. `db.connect()` now returns whether THIS call created the singleton — decided atomically inside `connect()`, since there is no `await` between its `if (sql)` null-check and the synchronous `sql = postgres(...)` assignment, so two connects can't both claim creation. `PostgresEngine` stores that as `_ownsModuleSingleton` and only calls `db.disconnect()` when it owns the connection; borrowers clear their own marker and leave the shared pool alone. Two adjacent hardenings landed in the same pass: `db.disconnect()` now snapshots and nulls `sql` *before* awaiting `end()` (so a concurrent connect can't join a pool that's already closing), and `reconnect()` uses a shared in-flight promise so concurrent callers await the same reconnect instead of racing a half-rebuilt pool.
|
||||
|
||||
Earlier partial fixes addressed symptoms: v0.41.27.0's retry-with-reconnect rescued the batch-write phases (extract) but not `sync`/`synthesize`, which don't go through the retry path; v0.42.5.0 stopped the lint phase from being a borrower but left the structural hole open for every other helper. This closes the hole at the source.
|
||||
|
||||
### To take advantage of v0.42.15.0
|
||||
|
||||
`gbrain upgrade` applies this automatically — there are no migrations or config to set. To confirm it took:
|
||||
|
||||
1. Run a cycle and watch the phases:
|
||||
```bash
|
||||
gbrain dream --dir <your-brain> --dry-run
|
||||
```
|
||||
Every DB phase should report `✓`, not `✗ ... connect() has not been called`.
|
||||
2. Confirm no stuck lock:
|
||||
```bash
|
||||
psql <your-db> -c "SELECT count(*) FROM gbrain_cycle_locks;" # expect 0
|
||||
```
|
||||
3. If a DB phase still fails, please file an issue at https://github.com/garrytan/gbrain/issues with the output of `gbrain doctor` and the cycle's stderr.
|
||||
|
||||
### Itemized changes
|
||||
|
||||
- **`gbrain dream` on Postgres completes every phase.** The module-singleton ownership fix (`_ownsModuleSingleton` in `src/core/postgres-engine.ts`, `db.connect()` returning the creator token in `src/core/db.ts`) means a short-lived probe engine's `disconnect()` can no longer null the shared connection the cycle is using. Closes #1404, #1471, #1619 (and the symptom tracked in #1570/#1535).
|
||||
- **Connection teardown is concurrency-safe.** `db.disconnect()` snapshots + nulls the singleton before awaiting `end()`; `PostgresEngine.reconnect()` shares one in-flight `_reconnectPromise` so concurrent callers await it instead of racing.
|
||||
- **Tests:** new `test/postgres-engine-singleton-ownership.test.ts` (source-level guardrails for the ownership contract), an expanded DB-gated behavioral matrix in `test/e2e/postgres-engine-disconnect-idempotency.test.ts` (owner/borrower, creation-not-role, symmetric CLI-exit, owner-reconnect-with-live-borrower), a module-style asymmetry case in `test/postgres-engine-getter-selfheal.test.ts`, and the #1570 shared-recovery regression updated to assert the fixed contract.
|
||||
|
||||
### For contributors
|
||||
|
||||
This is the canonical landing for a long-standing class with ~15 competing community PRs and zero merged. Thanks to **@nullhex-io** (#1651) and **@joelwp** (#1667) — the ownership-flag approach that shipped, and to **@BrendanGahan**, **@xaviroblessarries**, and the other #1471 reporters for the precise root-cause walkthroughs. Three follow-ups are filed in `TODOS.md` (connection-lifecycle hardening under concurrent module connects, facts-queue drain on the dream/fall-through paths, stale ConnectionManager refresh after reconnect) — all pre-existing and not reachable in current gbrain.
|
||||
|
||||
## [0.42.10.0] - 2026-06-02
|
||||
|
||||
**Wikilinks like `[[struktura]]` that point at pages in another folder finally connect.** Until now, if you wrote `[[struktura]]` in `concepts/knowledge-graph.md` and the actual page lived at `projects/struktura.md`, GBrain silently dropped the link from its graph. Obsidian users saw a dense web of connections in their vault and a thin, broken graph inside GBrain. The issue reporter had 71 wikilinks across 20 pages — GBrain captured 12.
|
||||
|
||||
@@ -62,7 +62,7 @@ strict behavior when unset.
|
||||
- `src/commands/reindex-code.ts` (v0.21.0 Cathedral II E2, extended v0.37.3.0) — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. **v0.37.3.0:** cost-preview model field now reads `getEmbeddingModelName()` from the gateway instead of the back-compat `EMBEDDING_MODEL` constant — preview reflects what the gateway will actually embed with. Same wave adds an informational stderr nudge inside `runReindexCode` (not the CLI wrapper, so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist currently `{'voyage-code-3'}`, case-insensitive bare match against gateway-returned name), prints a 4-line recommendation to switch to `voyage:voyage-code-3`. Suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` helper returns a tagged `NudgeDecision` union; the helper takes the bare model name (matches gateway return shape) and emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line. Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, and `test/reindex-code-model-source.serial.test.ts` (the latter is the IRON-RULE regression for the cost-preview fix).
|
||||
- `src/commands/sync.ts:resolveSlugByPathOrSourcePath` (v0.32.7 CJK wave, codex post-merge F4) — Resolves a slug by `pages.source_path` first (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back to `resolveSlugForPath(path)`. Threaded into all 4 delete/rename call sites (`performSync`'s un-syncable cleanup at ~:531, deletes at ~:603, rename oldSlug at ~:622). Without this, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (the delete path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path.
|
||||
- `src/core/utils.ts` — Shared SQL utilities extracted from postgres-engine.ts. Exports `parseEmbedding(value)` (throws on unknown input, used by migration + ingest paths where data integrity matters) and as of v0.12.3 `tryParseEmbedding(value)` (returns `null` + warns once per process, used by search/rescore paths where availability matters more than strictness). **v0.26.9 (D14):** adds `isUndefinedColumnError(err)` predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated. Replaces bare `catch {}` blocks in `oauth-provider.ts` so genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback path. Reusable from any future code that needs the same column-existence probe semantics. **v0.32.8 (PR #860):** adds `validateSourceId(id)` that throws on anything outside `^[a-z0-9_-]+$`. Used by the per-source disk-layout fix in patterns.ts/synthesize.ts before any `join(brainDir, '.sources', source_id, slug+'.md')` call so source_id can't traverse out of brainDir. `rowToPage` updated to populate the now-required `Page.source_id` field from the SELECT projection (`scripts/check-source-id-projection.sh` enforces that every projection feeding `rowToPage` includes the column).
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim).
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim). **v0.42.15.0 (#1404/#1471/#1619 module-singleton ownership):** `connect()` returns `Promise<boolean>` — `true` iff THIS call created the module singleton, `false` if it joined an existing one. The create-vs-join decision is atomic (no `await` between the `if (sql)` null-check and the synchronous `sql = postgres(url, opts)` assignment), so two concurrent module connects can't both claim creation. Callers (`PostgresEngine`) store the return as their `_ownsModuleSingleton` token; only the creator may later `disconnect()`. `disconnect()` rewritten to snapshot + null `sql` BEFORE awaiting `s.end()` (a concurrent connect can't join a pool that's already closing; mirrors the v0.41.8.0 PGLite snapshot+early-null pattern). The module `sql` is ONLY ever nulled by `db.disconnect()` (postgres.js auto-reconnects its own internal pool and never touches our reference), so the dream-cycle "connect() has not been called" failure was always a borrower probe engine's disconnect cascading into `db.disconnect()` — closed at the source by the ownership token. `PostgresEngine.disconnect()` guards `db.disconnect()` behind `_ownsModuleSingleton` (borrowers no-op); `PostgresEngine.reconnect()` uses a shared in-flight `_reconnectPromise` (replaces the `_reconnecting` boolean) so concurrent callers await the single reconnect. Pinned by `test/postgres-engine-singleton-ownership.test.ts` + the expanded e2e matrix in `test/e2e/postgres-engine-disconnect-idempotency.test.ts`.
|
||||
- `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`)
|
||||
- `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags). **v0.41.31.0:** `importFromContent` and `importCodeFile` stamp `pages.embedding_signature` via `setPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()})` when the import actually embedded (not `--no-embed`) — covers the inline import/sync path so a model/dims swap is detectable as stale. `importCodeFile` only stamps when every chunk was freshly embedded this call (`needsEmbedIndexes.length === chunks.length`); mixed reuse-by-hash pages stay unstamped (`reindex --code --force` / `embed --stale` handle the swap for those). **v0.41.37.0 (#1621):** `importFromContent`'s tag reconciliation is now ADD-ONLY. The pre-fix "remove every existing tag not in current frontmatter, then add current" logic wiped ALL DB-side enrichment tags (auto-tag / dream synthesize / signal-detector) on every re-import — most visibly under `gbrain reindex --markdown`, which re-imports every page with `forceRechunk`. The `tags` table has no provenance column and frontmatter tags are stripped from stored `pages.frontmatter` (markdown.ts:118), so at re-import time a frontmatter-origin tag can't be distinguished from a DB-enrichment tag — deletion is unsafe. Now it only `addTag` (idempotent, ON CONFLICT DO NOTHING). Accepted trade-off: removing a tag from frontmatter no longer removes it from the DB on next sync (needs a `tag_source` provenance column, deferred — TODOS.md #1621-followup). Pinned by `test/reindex-preserve-tags.test.ts` + `test/import-file.test.ts`.
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). **v0.35.5.0:** new exported `pruneDir(name: string): boolean` helper is the single source of truth for descent-time directory exclusion across walkers. Blocks `node_modules` (no leading dot, so pre-v0.35.5 walkers slipped through and inflated MISSING_OPEN counts via vendor packages), dot-prefix dirs, `ops/`, and `*.raw` sidecars. `isSyncable` now applies it per path segment; `walkMarkdownFiles` in `src/commands/extract.ts` and `listTextFiles` in `src/core/cycle/transcript-discovery.ts` consult it BEFORE recursing so the IO cost of walking thousands of vendor files is saved. Closes #923 + #202. `manageGitignore` worktree fix in same wave: discriminator now matches the gitdir path segment (`/modules/<name>` = submodule, `/worktrees/<name>` = worktree, per Git's documented layout) instead of the legacy absolute-vs-relative check that misclassified absorbed submodules and worktrees both. Conductor worktrees are first-class repos and now get `.gitignore` management for storage-tiering. Closes #889. v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500.
|
||||
|
||||
@@ -1,5 +1,66 @@
|
||||
# TODOS
|
||||
|
||||
## v0.42.15.0 module-singleton ownership follow-ups (v0.42+)
|
||||
|
||||
Filed from the v0.42.15.0 wave (#1404/#1471/#1619 — the dream-cycle
|
||||
"connect() has not been called" class, fixed via `_ownsModuleSingleton`).
|
||||
Surfaced by the Codex outside-voice review (finding #4) and deliberately scoped
|
||||
OUT — pre-existing, and the ownership fix *reduces* its window. See plan +
|
||||
GSTACK REVIEW REPORT at
|
||||
`~/.claude/plans/system-instruction-you-are-working-lazy-allen.md`.
|
||||
|
||||
- [ ] **P3 — Stale `ConnectionManager` read-pool after an owner `reconnect()`.**
|
||||
A module-style borrower engine caches the singleton at connect time via
|
||||
`connectionManager.setReadPool(db.getConnection())` (`postgres-engine.ts:~208`).
|
||||
When the OWNER engine calls `reconnect()` (the batchRetry path), it tears down
|
||||
the old module singleton and builds a fresh one — but the borrower's
|
||||
`connectionManager` still holds the OLD (ended) pool. The borrower's normal
|
||||
query path is fine (`this.sql` → `db.getConnection()` resolves the NEW
|
||||
singleton), so this is invisible on read/write. The edge is
|
||||
`initSchema()`, which routes DDL through `connectionManager.ddl()`
|
||||
(`postgres-engine.ts:~253`) — a borrower running initSchema after an owner
|
||||
reconnect would hit the dead pool. Pre-existing (not introduced by #1471), and
|
||||
the ownership fix makes owner reconnects *rarer* (the singleton no longer gets
|
||||
nulled by borrowers, so reconnect only fires on genuine transient drops), which
|
||||
shrinks the window. Real fix: refresh a borrower's `connectionManager` read
|
||||
pool lazily from `db.getConnection()` on use, or have `db.connect()`/reconnect
|
||||
publish a generation counter the manager checks. Defer until a borrower is
|
||||
observed running `initSchema()` mid-process (no current caller does).
|
||||
|
||||
- [ ] **P2 — Ownership state can desync from the shared singleton under
|
||||
CONCURRENT module connect/reconnect.** Both adversarial reviewers (Codex +
|
||||
Claude) independently flagged this. `_ownsModuleSingleton` is per-engine state
|
||||
about a shared (module-level) resource, so it can migrate: if a borrower calls
|
||||
`connect()`/`reconnect()` during the window when an owner's `reconnect()` has
|
||||
nulled `sql` (`db.ts` snapshot-early-null) but not yet rebuilt it, the borrower
|
||||
creates the new singleton and becomes owner; the owner re-connects as a
|
||||
borrower; the short-lived borrower's later `disconnect()` then closes the live
|
||||
pool the demoted owner still uses — the original bug, in reverse. ALSO: the
|
||||
audit-import + `connectionManager.disconnect()` awaits in `PostgresEngine.disconnect()`
|
||||
and the publish-before-`SELECT 1` window in `db.connect()` let a concurrent
|
||||
connect join a dying/unverified pool. NOT REACHABLE in current gbrain — cycle
|
||||
phases are sequential on one awaited engine, borrowers are nested within a
|
||||
phase, the parallel-sync worker pool uses INSTANCE engines (not the singleton),
|
||||
and facts/last-retrieved background writes reuse the owner engine (no second
|
||||
module engine). The ownership fix is correct for every reachable path and is
|
||||
fully tested. The structural fix (which removes the unenforced "no concurrent
|
||||
module connect" invariant) is the refcount/lease-in-db.ts approach Codex argued
|
||||
in the plan review: keep the lifecycle state WITH the shared resource so it
|
||||
can't desync per-engine, bounded against CLI-hang by a top-level forced
|
||||
cleanup. Do this BEFORE introducing any concurrent module-engine connect path.
|
||||
|
||||
- [ ] **P3 — `dream` + CLI_ONLY fall-through paths don't drain the facts /
|
||||
last-retrieved queues before the owner disconnect.** The op-dispatch path
|
||||
(`cli.ts:~282-314`) drains `getFactsQueue().drainPending()` +
|
||||
`awaitPendingLastRetrievedWrites()` before `engine.disconnect()`; the `dream`
|
||||
owner-disconnect (`cli.ts:~1164`) and the fall-through owner-disconnect
|
||||
(`cli.ts:~1785`) do not. If the dream cycle ever enqueues a facts:absorb /
|
||||
last-retrieved write that's still in flight at disconnect, the owner nulls the
|
||||
singleton and the write throws "No database connection". Pre-existing (not
|
||||
introduced by the #1471 ownership fix), surfaced by the Claude adversarial
|
||||
review (F5). Fix: hoist the same drain-before-disconnect block the op-dispatch
|
||||
path uses into a shared helper and call it on all three owner-disconnect sites.
|
||||
|
||||
## v0.42.7.0 extract-in-default-loop follow-ups (v0.42+)
|
||||
|
||||
Filed from the v0.42.2.0 wave (#1696 link/timeline extraction freshness
|
||||
|
||||
+1
-1
@@ -204,7 +204,7 @@ strict behavior when unset.
|
||||
- `src/commands/reindex-code.ts` (v0.21.0 Cathedral II E2, extended v0.37.3.0) — `gbrain reindex --code [--source ID] [--dry-run] [--yes] [--json] [--force] [--no-embed]`. Walks `pages WHERE type = 'code'` in 100-row batches, replays through `importCodeFile` for chunk + embed + content_hash folding. Idempotent unless `--force` bypasses the content_hash early-return. **v0.37.3.0:** cost-preview model field now reads `getEmbeddingModelName()` from the gateway instead of the back-compat `EMBEDDING_MODEL` constant — preview reflects what the gateway will actually embed with. Same wave adds an informational stderr nudge inside `runReindexCode` (not the CLI wrapper, so dry-run + execute both surface it): when the configured embedding model isn't code-tuned (allowlist currently `{'voyage-code-3'}`, case-insensitive bare match against gateway-returned name), prints a 4-line recommendation to switch to `voyage:voyage-code-3`. Suppress with `GBRAIN_NO_CODE_MODEL_NUDGE=1`, `--no-embed`, or `--json`. Pure `shouldNudgeCodeModel(bareName)` helper returns a tagged `NudgeDecision` union; the helper takes the bare model name (matches gateway return shape) and emits qualified `voyage:voyage-code-3` for the paste-ready `gbrain config set` line. Pinned by `test/ai/voyage-code-3-recipe.test.ts`, `test/reindex-code-nudge.serial.test.ts`, and `test/reindex-code-model-source.serial.test.ts` (the latter is the IRON-RULE regression for the cost-preview fix).
|
||||
- `src/commands/sync.ts:resolveSlugByPathOrSourcePath` (v0.32.7 CJK wave, codex post-merge F4) — Resolves a slug by `pages.source_path` first (returns the stored slug for frontmatter-fallback pages whose path doesn't derive a slug), then falls back to `resolveSlugForPath(path)`. Threaded into all 4 delete/rename call sites (`performSync`'s un-syncable cleanup at ~:531, deletes at ~:603, rename oldSlug at ~:622). Without this, emoji-only / Thai / Arabic filenames whose slug came from frontmatter would orphan on delete/rename (the delete path would compute the wrong path-derived slug). Best-effort query — pre-migration brains fall through to the legacy path.
|
||||
- `src/core/utils.ts` — Shared SQL utilities extracted from postgres-engine.ts. Exports `parseEmbedding(value)` (throws on unknown input, used by migration + ingest paths where data integrity matters) and as of v0.12.3 `tryParseEmbedding(value)` (returns `null` + warns once per process, used by search/rescore paths where availability matters more than strictness). **v0.26.9 (D14):** adds `isUndefinedColumnError(err)` predicate — pattern-matches Postgres SQLSTATE 42703 / "column ... does not exist" with engine-driver shape variation tolerated. Replaces bare `catch {}` blocks in `oauth-provider.ts` so genuine errors (lock timeout, network blip, permission denied) propagate while column-missing falls through to the legacy fallback path. Reusable from any future code that needs the same column-existence probe semantics. **v0.32.8 (PR #860):** adds `validateSourceId(id)` that throws on anything outside `^[a-z0-9_-]+$`. Used by the per-source disk-layout fix in patterns.ts/synthesize.ts before any `join(brainDir, '.sources', source_id, slug+'.md')` call so source_id can't traverse out of brainDir. `rowToPage` updated to populate the now-required `Page.source_id` field from the SELECT projection (`scripts/check-source-id-projection.sh` enforces that every projection feeding `rowToPage` includes the column).
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim).
|
||||
- `src/core/db.ts` — Connection management, schema initialization. v0.22.1 (#363, contributed by @orendi84): `resolveSessionTimeouts()` returns `statement_timeout` + `idle_in_transaction_session_timeout` (defaults: 5min each, env-overridable via `GBRAIN_STATEMENT_TIMEOUT` / `GBRAIN_IDLE_TX_TIMEOUT` / `GBRAIN_CLIENT_CHECK_INTERVAL`). Both `connect()` (module singleton) and `PostgresEngine.connect()` (worker pool) consume the result via postgres.js's `connection` option, sending GUCs as startup parameters that survive PgBouncer transaction mode (unlike the prior `setSessionDefaults` post-pool SET, kept as a back-compat no-op shim). **v0.42.15.0 (#1404/#1471/#1619 module-singleton ownership):** `connect()` returns `Promise<boolean>` — `true` iff THIS call created the module singleton, `false` if it joined an existing one. The create-vs-join decision is atomic (no `await` between the `if (sql)` null-check and the synchronous `sql = postgres(url, opts)` assignment), so two concurrent module connects can't both claim creation. Callers (`PostgresEngine`) store the return as their `_ownsModuleSingleton` token; only the creator may later `disconnect()`. `disconnect()` rewritten to snapshot + null `sql` BEFORE awaiting `s.end()` (a concurrent connect can't join a pool that's already closing; mirrors the v0.41.8.0 PGLite snapshot+early-null pattern). The module `sql` is ONLY ever nulled by `db.disconnect()` (postgres.js auto-reconnects its own internal pool and never touches our reference), so the dream-cycle "connect() has not been called" failure was always a borrower probe engine's disconnect cascading into `db.disconnect()` — closed at the source by the ownership token. `PostgresEngine.disconnect()` guards `db.disconnect()` behind `_ownsModuleSingleton` (borrowers no-op); `PostgresEngine.reconnect()` uses a shared in-flight `_reconnectPromise` (replaces the `_reconnecting` boolean) so concurrent callers await the single reconnect. Pinned by `test/postgres-engine-singleton-ownership.test.ts` + the expanded e2e matrix in `test/e2e/postgres-engine-disconnect-idempotency.test.ts`.
|
||||
- `src/commands/migrate-engine.ts` — Bidirectional engine migration (`gbrain migrate --to supabase/pglite`)
|
||||
- `src/core/import-file.ts` — importFromFile + importFromContent (chunk + embed + tags). **v0.41.31.0:** `importFromContent` and `importCodeFile` stamp `pages.embedding_signature` via `setPageEmbeddingSignature(slug, {sourceId, signature: currentEmbeddingSignature()})` when the import actually embedded (not `--no-embed`) — covers the inline import/sync path so a model/dims swap is detectable as stale. `importCodeFile` only stamps when every chunk was freshly embedded this call (`needsEmbedIndexes.length === chunks.length`); mixed reuse-by-hash pages stay unstamped (`reindex --code --force` / `embed --stale` handle the swap for those). **v0.41.37.0 (#1621):** `importFromContent`'s tag reconciliation is now ADD-ONLY. The pre-fix "remove every existing tag not in current frontmatter, then add current" logic wiped ALL DB-side enrichment tags (auto-tag / dream synthesize / signal-detector) on every re-import — most visibly under `gbrain reindex --markdown`, which re-imports every page with `forceRechunk`. The `tags` table has no provenance column and frontmatter tags are stripped from stored `pages.frontmatter` (markdown.ts:118), so at re-import time a frontmatter-origin tag can't be distinguished from a DB-enrichment tag — deletion is unsafe. Now it only `addTag` (idempotent, ON CONFLICT DO NOTHING). Accepted trade-off: removing a tag from frontmatter no longer removes it from the DB on next sync (needs a `tag_source` provenance column, deferred — TODOS.md #1621-followup). Pinned by `test/reindex-preserve-tags.test.ts` + `test/import-file.test.ts`.
|
||||
- `src/core/sync.ts` — Pure sync functions (manifest parsing, filtering, slug conversion). **v0.35.5.0:** new exported `pruneDir(name: string): boolean` helper is the single source of truth for descent-time directory exclusion across walkers. Blocks `node_modules` (no leading dot, so pre-v0.35.5 walkers slipped through and inflated MISSING_OPEN counts via vendor packages), dot-prefix dirs, `ops/`, and `*.raw` sidecars. `isSyncable` now applies it per path segment; `walkMarkdownFiles` in `src/commands/extract.ts` and `listTextFiles` in `src/core/cycle/transcript-discovery.ts` consult it BEFORE recursing so the IO cost of walking thousands of vendor files is saved. Closes #923 + #202. `manageGitignore` worktree fix in same wave: discriminator now matches the gitdir path segment (`/modules/<name>` = submodule, `/worktrees/<name>` = worktree, per Git's documented layout) instead of the legacy absolute-vs-relative check that misclassified absorbed submodules and worktrees both. Conductor worktrees are first-class repos and now get `.gitignore` management for storage-tiering. Closes #889. v0.22.12 (#500, foundation by @wintermute via #501): `classifyErrorCode(errorMsg)` regex-based classifier with 12 codes (`SLUG_MISMATCH`, `YAML_PARSE`, `YAML_DUPLICATE_KEY`, `MISSING_OPEN`, `MISSING_CLOSE`, `NESTED_QUOTES`, `EMPTY_FRONTMATTER`, `NULL_BYTES`, `INVALID_UTF8`, `STATEMENT_TIMEOUT`, `FILE_TOO_LARGE`, `SYMLINK_NOT_ALLOWED`) plus `UNKNOWN` fallback. `summarizeFailuresByCode(failures)` returns sorted `[{code, count}]`. `code?` optional field on `SyncFailure`; backfilled at ack time on pre-v0.22.12 entries. `acknowledgeSyncFailures()` returns `AcknowledgeResult { count, summary }`. Three regexes (`MISSING_OPEN`, `MISSING_CLOSE`, `EMPTY_FRONTMATTER`) broadened to match actual `markdown.ts:159-244` validator message strings, not just the literal code-name prefix. `FILE_TOO_LARGE` covers all three production size sites in `import-file.ts:199, 352, 401`; `SYMLINK_NOT_ALLOWED` covers the rejection at `:347`. Closes the silent-skip pattern that motivated #500.
|
||||
|
||||
+1
-1
@@ -142,5 +142,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.10.0"
|
||||
"version": "0.42.15.0"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user