mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 11:22:34 +00:00
2d9f4ec243c19fbd4e2a43e95f13e18e0cb8d1e4
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
146a8f1eed |
v0.41.31.0 feat(embed): delta-aware sync --all cost gate + real stale-embedding semantics (#1632)
* fix(cost): embedding cost preview uses configured model rate, not hardcoded OpenAI
The sync --all cost gate computed spend from a hardcoded
EMBEDDING_COST_PER_1K_TOKENS = 0.00013 (OpenAI text-embedding-3-large)
and labeled the preview with the back-compat EMBEDDING_MODEL constant,
regardless of the actually-configured embedding model. A brain running a
cheaper model (e.g. zeroentropyai:zembed-1 @ $0.05/Mtok) saw a preview
that named the wrong provider and over-stated spend ~2.6x ($337 vs $130
on a 2.6B-token corpus).
estimateEmbeddingCostUsd now resolves the live model via the gateway and
prices it through embedding-pricing.ts (the existing per-provider:model
table), falling back to the OpenAI rate only when the gateway is
unconfigured (unit-test context) or the model is unknown. sync.ts surfaces
the real model name in the preview message and JSON.
Regression test pins model-aware pricing: openai 3-large vs zembed-1 must
produce materially different previews; collapsing both to the OpenAI number
fails the assertion.
* fix(cost): sync --all gate is informational when embed is deferred; delta-aware inline gate
Under federated_v2 (default), sync --all DEFERS embedding to per-source
embed-backfill jobs that already cap spend at $25/source/24h. The v0.20
cost gate predated that cap and fired ConfirmationRequired + exit 2 on
EVERY non-TTY sync --all without --yes, regardless of cost — blocking
nightly crons over already-synced corpora and forcing permanent --yes.
The gate is now mode-aware:
- Deferred embed (v2 default): print an FYI deferred notice (cap-aware,
"not charged by this sync") + the stale-chunk backlog estimate, and
NEVER exit 2. The backfill cap is the real money gate.
- Inline embed (v2 off, or --serial without --no-embed): keep the
blocking gate, but estimate the actual delta — full-tree ceiling for
changed sources (unchanged sources contribute 0 via the same git +
chunker_version "do work?" gate doctor/sync use) + stale backlog — and
block only when it exceeds the new configurable floor
sync.cost_gate_min_usd (default $0.50).
New pure helpers in embedding.ts (willEmbedSynchronously, shouldBlockSync)
keep the decision logic hermetically testable. New engine method
sumStaleChunkChars (both engines) prices the embedding backlog via
estimateCostFromChars. estimateSyncAllCost's per-source walk extracted to
estimateSourceTreeTokens (reused by the inline estimator).
Regressions pinned: R-1 deferred non-TTY never exit 2 (headline), R-2
inline above-floor still exit 2 (protection), plus the willEmbedSynchronously
/ shouldBlockSync matrix and sumStaleChunkChars engine + scope + embed_skip
coverage.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(embed): real stale semantics — re-embed on model/dims swap (migration v108)
Pre-v0.41.30 "stale" meant only `embedding IS NULL`, so swapping the
embedding model or dimensions left the whole corpus silently embedded under
the OLD model — `embed --stale` ignored it and search quality quietly
degraded.
New `pages.embedding_signature` (TEXT, migration v108) stamps the embedding
provenance (`<provider:model>:<dims>`) whenever a page's chunks are embedded.
A later model/dims swap makes the stored signature differ from the current
one, which the embed paths now detect and re-embed.
GRANDFATHER (critical): the stale predicate is
`embedding IS NULL OR (embedding_signature IS NOT NULL AND <> $current)`
so a NULL signature is NEVER stale. After the migration every existing page
has NULL → none flagged → the next `embed --stale` does NOT re-embed the
whole corpus. Signatures are stamped going forward only.
Surface:
- countStaleChunks / sumStaleChunkChars gain an optional `signature` opt
that widens staleness (read-only; used by the dry-run preview + the
sync cost preview, which is now signature-aware).
- invalidateStaleSignatureEmbeddings(signature, sourceId?) NULLs the
embeddings of signature-mismatched pages so the EXISTING NULL-embedding
cursor (listStaleChunks, untouched) re-embeds them — keeps the keyset
pagination logic intact.
- setPageEmbeddingSignature stamps after a page's chunks land.
- Both embed loops wired: `gbrain embed --stale`/`--all` (embed.ts) and the
embed-backfill minion (embed-stale.ts) invalidate-then-stamp.
Migration v108 + bootstrap probe (both engines) + REQUIRED_BOOTSTRAP_COVERAGE
entry. Pinned by test/embedding-signature-stale.test.ts (R-4 grandfather,
mismatch detection, matching no-op, scoped invalidate, stamp) + the
bootstrap-coverage gate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(sync): surface embed-backfill job state in sources status + deferred notice
Under federated_v2, `sync --all` exits 0 and embedding lags behind in
embed-backfill jobs (subject to cooldown + the per-source 24h cap). Pre-fix
an operator had no signal those jobs were queued or lagging — the sync looked
"done" while embeddings trickled in later.
`gbrain sources status` now shows a BACKFILL column per source
(active(N)/queued(N)/idle) plus the last completion timestamp, read from
minion_jobs. The deferred-sync notice appends "N backfill job(s) queued" so a
cron operator sees work is enqueued, not lost. Both reads are best-effort —
a brain that never ran a worker (no minion_jobs table) reports idle/0 instead
of crashing the dashboard.
SyncStatusReportSource gains backfill_queued / backfill_active /
backfill_last_completed_at (additive; JSON envelope schema_version unchanged).
Pinned by a new case in test/e2e/sync-status-pglite.test.ts.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: add currentEmbeddingSignature to embedding.ts mocks + sync stale EXPECTED_PHASES
Commit 3 made embed.ts import currentEmbeddingSignature from embedding.ts.
Four tests mock.module the whole embedding.ts and omitted the new export, so
embed.ts (imported transitively) failed at load with "Export named
'currentEmbeddingSignature' not found". Add the export to each mock:
embed.serial.test.ts, e2e/cycle.test.ts, e2e/dream.test.ts,
e2e/dream-cycle-phase-order-pglite.test.ts.
Also sync the stale EXPECTED_PHASES in dream-cycle-phase-order-pglite.test.ts
to match cycle.ts ALL_PHASES — extract_atoms, synthesize_concepts, and
conversation_facts_backfill drifted in after the test was last touched
(v0.41.0.0) and were never added, so both phase-order assertions were failing
on the branch before this wave (confirmed against
|
||
|
|
89ae720959 |
v0.31.0 feat: hot memory — facts hook + recall CLI + MCP _meta + consolidate phase (#785)
* v0.31 feat(migrate): facts hot memory schema (migration v40) Phase 1 of v0.31 hot-memory. - New facts table with source_id (TEXT FK to sources, per-source isolation), kind CHECK (event/preference/commitment/belief/fact), visibility CHECK (private/world for takes-style ACL parity), valid_from/valid_until/ expired_at/superseded_by for temporal + supersession audit, and consolidated_at/consolidated_into pointing at takes(id) for the dream- cycle hot→cold bridge. - Embedding column dim resolved at migration time from config.embedding_dimensions so non-OpenAI brains (Voyage etc) work out-of-the-box. HALFVEC where pgvector >= 0.7; falls back to VECTOR with stderr warn on older versions. Matching opclass per column type (halfvec_cosine_ops vs vector_cosine_ops). - 5 partial indexes leading on source_id so every read uses the trust boundary as part of the index, not a callback. HNSW partial index excludes expired/null rows so footprint stays proportional to active fact count. - RLS DO-block matches takes pattern (Postgres BYPASSRLS gate; PGLite no-op). - v0_31_0.ts orchestrator follows v0_28_0.ts pattern — phase A asserts schema version >= 40 + facts table presence; runner owns ledger. All 87 existing migrate.test.ts cases pass. PGLite smoke test confirms table + indexes + CHECK constraints + ON DELETE CASCADE all behave. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 chore(version): bump VERSION + package.json to 0.31.0 Phase 1 closer. CHANGELOG entry written when Phase 7 lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 feat(engine): facts hot memory engine API (Phase 2) Phase 2 of v0.31 hot-memory. Adds 8 facts methods to BrainEngine implemented on both PGLite and Postgres engines: - insertFact(input, ctx) — INSERT with optional supersedeId; expires the named row in the same transaction. Per-entity advisory lock on Postgres (`pg_advisory_xact_lock(hashtextextended(source_id::text || ':' || entity_slug, 0))`) for the dedup window. PGLite is single-process so the lock is a no-op. - expireFact(id, opts) — sets expired_at + optional superseded_by. Idempotent-as-false (already-expired returns false). - listFactsByEntity / listFactsSince / listFactsBySession — list surfaces with FactListOpts filters (activeOnly, kinds, visibility, limit/offset). Every query starts WHERE source_id = $X so the trust boundary is part of the index path. - listSupersessions — audit log; activeOnly:false + expired_at IS NOT NULL + superseded_by IS NOT NULL. - findCandidateDuplicates(source_id, entity_slug, factText, k) — entity-prefiltered (mandatory), k=5 default, hard cap 20. Embedding- cosine ordering when caller supplies an embedding, recency fallback otherwise. Bounds the contradiction-classifier blast radius. - consolidateFact(id, takeId) — sets consolidated_at + consolidated_into. Never DELETE; facts stay as audit trail for the resulting take. - getFactsHealth(source_id) — per-source counters consumed by `gbrain doctor` facts_health check. Public types in engine.ts: FactKind (5-value union), FactVisibility, FactInsertStatus, FactRow, NewFact, FactListOpts, FactsHealth. PGLite + Postgres helpers: rowToFact / rowToFactPg parse the text-format pgvector embedding back into Float32Array; toPgVectorLiteral encodes for the supersede-path INSERT (postgres-js can't bind Float32Array directly to a vector column without an explicit literal cast). Smoke test confirms every method end-to-end on PGLite. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 feat(facts): extraction code path (Phase 3) Phase 3 of v0.31 hot-memory. Five new modules under src/core/facts/ + src/core/entities/: - src/core/facts/decay.ts — pure helper. effectiveConfidence(fact, now) applies confidence × exp(-age/halflife) with per-kind halflife table (event 7d, commitment 90d, preference 90d, belief 365d, fact 365d). Returns 0 for expired or past-valid_until rows. Single source of truth consumed by recall, supersession audit, facts_health, and the MCP _meta injector (eD8 DRY). - src/core/facts/queue.ts — bounded in-memory queue. Cap 100 default, drop-oldest on overflow with counter. Per-session in-flight=1 serializes burst chat. AbortSignal threading from server SIGTERM (mirrors minion worker pattern per eD7): 5s grace for in-flight, then drop pending with counter. getFactsQueue() process-singleton; __resetFactsQueueForTests for hermetic tests. - src/core/facts/classify.ts — contradiction classifier with cosine fast-path (D13: ≥0.95 → duplicate, skip LLM) and classifier-failure fallback (D12: cosine ≥0.92 → duplicate, else INSERT). Pure cosine helper exported. JSON-strict output with 4-strategy parse fallback; refusal stop-reason maps to fallback path. Caller-provided abort signal propagated to the gateway chat call. - src/core/facts/extract.ts — Haiku turn-extractor. Reuses INJECTION_PATTERNS from src/core/think/sanitize.ts on the way IN (turn_text) AND on the way OUT (each fact). Tight system prompt with 5-kind taxonomy, 0..1 confidence scoring, entity slug or display name. Anti-loop check on isDreamGenerated (reuses v0.23.2 marker semantics). Synchronous embedOne() per fact via the gateway so classifier paths have embeddings available; AbortError re-thrown explicitly so SIGTERM during embed never writes a NULL-embedding row meant to be cancelled (eE8 distinction). - src/core/entities/resolve.ts — slug canonicalization shared by signal-detector AND facts. Resolution order: exact slug match → pg_trgm fuzzy match (similarity ≥0.4) → deterministic slugify fallback. slugify exported standalone for tests + callers that want the floor. Smoke tests confirm decay table, cosine math, slugify rules, queue drop-oldest under overflow, and shutdown grace + drop-pending semantics. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 feat(mcp+cli): MCP ops + recall CLI + _meta + transport refactor (Phase 4) Phase 4 of v0.31 hot-memory. Three new MCP ops on the contract-first surface: - `extract_facts` (write scope, localOnly:false): extracts facts from a conversation turn via the Haiku extractor, runs the cosine fast-path dedup, INSERTs into per-source hot memory. Returns counts + fact_ids[]. Skips on is_dream_generated:true (anti-loop). - `recall` (read scope): query the per-source hot memory by entity / since / session / supersessions / grep filter. Visibility- aware: remote callers see visibility='world' rows only (takes-style ACL parity, eD21). Returns most-recent first; pagination via limit. - `forget_fact` (write scope): expireFact wrapper. Idempotent-as-error on unknown id; uses the new 'fact_not_found' ErrorCode. ErrorCode union opened (eD6 / eE7): TS forward-compat via the `(string & {})` autocomplete-friendly hack so downstream consumers (gbrain-evals etc) don't break their typecheck on every new code. Three new codes: 'rate_limited', 'extraction_failed', 'fact_not_found'. OperationContext gains source_id?:string (eD4 / eE2 — TEXT not INTEGER per schema reality). Resolved once in buildOperationContext from DispatchOpts.sourceId. Stdio MCP defaults to GBRAIN_SOURCE env or 'default'; HTTP MCP reads it from the per-token sources scope (eE3). ToolResult gains _meta?: Record<string, unknown> (eD3). Dispatcher calls a configurable metaHook AFTER op.handler succeeds, wrapped in its own try/catch so a DB blip degrades to no-_meta rather than flipping the whole tool call to error (eE4). New module src/core/facts/meta-hook.ts: - getBrainHotMemoryMeta(name, ctx) builds the _meta.brain_hot_memory payload. Cache key (source_id, session_id, hash(takesHoldersAllowList sorted)) (eD10 / eE5). 30s TTL per session. Visibility filter applies: remote → world only; local → all. Top-K=10 ranked by effective confidence (decay). Skips injection on recall/extract_facts/forget_fact themselves. bumpHotMemoryCache() invalidates per (source_id, session_id) on extraction event. D12 (eE1) accepted: serve-http.ts:801 inlined dispatch path REFACTORED to call dispatchToolCall. HTTP MCP now inherits source_id, _meta injection, error envelope unification, and OperationContext shape from the same code path stdio uses. Scope check + mcp_request_log + SSE broadcast stay in serve-http.ts (HTTP-specific concerns); the dispatcher returns ToolResult and the HTTP handler reads isError + content + _meta to fan into the audit + broadcast paths. put_page compliance backstop (D23): when a conversation-shape page is written (note/meeting/slack/email/calendar-event/source/writing) with a substantive body (>=80 chars) on a non-subagent slug AND no dream_generated:true marker, fire-and-forget enqueue an extraction job into the bounded queue. Never blocks the put_page response. Skipped reasons (no_parsed_page / subagent_namespace / dream_generated / kind:* / too_short / queue_shutdown / backstop_error) are stable strings consumed by tests. `gbrain recall` + `gbrain forget` CLI commands (src/commands/recall.ts): - recall <entity> | --since DUR | --session ID | --today (markdown with kind icons 📅🎯🤝💭📌) | --grep TEXT | --supersessions | --include-expired | --as-context (prompt-injection-ready) | --json - forget <fact-id> shorthand for expireFact Wired into src/cli.ts dispatch table next to takes / think. Smoke tests confirm: dispatch surfaces (extract_facts → ops → listFactsByEntity), forget_fact + idempotent re-call, _meta visibility filter (remote sees world only, local sees all), CLI markdown render with kind icons + age strings + decayed confidence. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 feat(cycle): consolidate phase — facts → takes promotion (Phase 5) Phase 5 of v0.31 hot-memory. New 10th cycle phase `consolidate` between `patterns` and `embed`: - src/core/cycle.ts: * CyclePhase union extended with 'consolidate' * ALL_PHASES gets 'consolidate' between patterns and embed (graph-fresh after patterns; embed runs after so the new takes get embedded same-cycle) * NEEDS_LOCK_PHASES gets 'consolidate' (writes takes + UPDATEs facts) * CycleReport.totals gains facts_consolidated + consolidate_takes_written * runCycle dispatches the new phase via dynamic import - src/core/cycle/phases/consolidate.ts (new): * Scans (source_id, entity_slug) buckets where COUNT(unconsolidated facts) >= 3 (uses idx_facts_unconsolidated partial index) * Skips buckets where the OLDEST fact is < 24h old (gives signal time to settle before locking it into cold memory) * Greedy cosine clustering at threshold 0.85; head-element centroid keeps it deterministic + cheap. Singletons (no embedding) stay unconsolidated this cycle. * For each cluster size >= 2: picks the highest-confidence fact's text as the take claim (v0.31 deterministic; v0.32 swaps to Sonnet synthesis pass). avg confidence → take weight, earliest valid_from → take since_date, concatenated source_sessions → take.source. * Resolves entity_slug → page_id via pages.slug (per source). Skips cluster if page is missing in this source — no auto-page-creation in v0.31. * INSERT into takes(kind='fact', holder='self') with row_num = MAX(existing) + 1. * UPDATE contributing facts: consolidated_at = now() + consolidated_into = takes.id. NEVER DELETE — facts are the audit trail for the resulting take. * dryRun honored: pretends the writes happened; counters still tick so operators can preview load before the first real run. * yieldDuringPhase keepalive between buckets so the Minions worker job lock + cycle-lock TTL don't drift on long runs. Smoke test on PGLite confirms: 4 unconsolidated facts → clustered (cosine 1.0 since same vector) → 1 take row created → all 4 facts marked consolidated_into. runCycle({phases:['consolidate']}) wires through to the report totals. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 test: 18 facts test files (Phase 6) Phase 6 of v0.31 hot-memory: comprehensive coverage across the new substrate. 110 unit tests pass; 5 E2E test files added (skip gracefully without DATABASE_URL). Unit tests (PGLite in-memory, no DATABASE_URL): - test/facts-decay.test.ts (12 cases) — HALFLIFE_DAYS pinned per kind, effectiveConfidence math: age=0 / age=halflife (~1/e) / age=2×halflife (~1/e²) / expired returns 0 / valid_until past returns 0 / preference-vs-event slower decay / belief-vs-commitment crossover. - test/facts-queue.test.ts (10 cases) — FIFO within session, drop-oldest on overflow, per-session in-flight=1 serializes, different sessions parallelize, failed jobs counter, shutdown grace + drop_pending + external AbortController triggers shutdown. - test/facts-classify.test.ts (8 cases) — cosineSimilarity edge cases, empty candidates → independent, cheap fast-path ≥0.95 → duplicate no LLM, threshold-configurable cosine_fallback path. - test/facts-engine.test.ts (13 cases) — every BrainEngine fact method end-to-end: insertFact (insert/supersede), expireFact idempotency, list*, findCandidateDuplicates entity-prefiltered + k cap + cosine ordering, consolidateFact never DELETE, getFactsHealth shape + total_today ⊆ total_week. - test/facts-multi-tenant.test.ts (6 cases) — cross-source isolation on every list method + CASCADE delete on sources. - test/facts-visibility.test.ts (6 cases) — visibility column private/ world; remote=true filters to world-only via dispatchToolCall; remote=false sees all. - test/facts-canonicality.test.ts (10 cases) — slugify rules including NFKD diacritic strip ("Crème Brûlée" → "creme-brulee"), exact slug match, fallback to slugify when no fuzzy match. - test/facts-extract.test.ts (4 cases) — empty turn returns [], dream- generated short-circuit, graceful no-API-key return. - test/facts-backstop-gating.test.ts (5 cases) — put_page backstop: too_short, subagent_namespace, dream_generated, eligible note path, non-eligible kind:guide. - test/facts-anti-loop.test.ts (4 cases) — extractor + put_page both respect dream_generated:true marker. - test/facts-doctor-shape.test.ts (4 cases) — facts_health JSON shape pinned for downstream consumers. - test/facts-mcp-allowlist.serial.test.ts (5 cases) — extract_facts write-scope, recall read-scope, forget_fact write-scope, forget_fact fact_not_found error code, extract_facts no-API-key zero counts. - test/facts-context-injection.serial.test.ts (6 cases) — _meta injection on success, world-only filter under remote=true, anti-loop on facts ops themselves, best-effort degrade on hook error, cache-key includes allow-list hash. - test/facts-separation-pglite.test.ts (2 cases) — Garry's Separation Test as primary ship gate, plus expired hidden-by-default contract. - test/facts-recall-render.test.ts (3 cases) — --today markdown render with all 5 kind icons, --json shape with effective_confidence, --as-context emits comment-wrapped block. - test/facts-migration-dim.test.ts (4 cases) — embedding column type is HALFVEC/VECTOR (not arbitrary), dim matches gateway-configured embedding_dimensions, HNSW opclass agrees with column type, idempotent re-init. - test/cycle-consolidate.test.ts (5 cases) — below-count + below-age thresholds skip, happy path 4 facts → 1 take + all consolidated never DELETE, dryRun honored, missing page → bucket skipped. E2E tests (skip gracefully on DATABASE_URL unset; required gates by CLAUDE.md test policy): - test/e2e/facts-separation-postgres.test.ts — Postgres parity for the ship gate. - test/e2e/facts-cross-source-isolation.test.ts — cross-source ACL on PG + CASCADE delete. - test/e2e/facts-forget.test.ts — full forget_fact MCP roundtrip. - test/e2e/facts-context-injection-postgres.test.ts — _meta injection end-to-end on PG. - test/e2e/facts-recall-render.test.ts — recall --today markdown on PG. - test/e2e/serve-http-meta.test.ts — eE1 regression: HTTP MCP transport inherits _meta + sourceId + scope correctness via dispatchToolCall. Side-effect: src/core/entities/resolve.ts NFKD post-decompose strips combining marks (U+0300..U+036F) before hyphenating non-alphanumerics, so "Crème" → "creme", not "cre-me-". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 feat(operational): kill switch + doctor check + CHANGELOG + README (Phase 7) Phase 7 of v0.31 hot-memory. - src/core/facts/extract.ts: new isFactsExtractionEnabled(engine) helper reads `facts.extraction_enabled` config row. Defaults to TRUE; flip to 'false'/'0'/'no'/'off' (case-insensitive) via `gbrain config set facts.extraction_enabled false` to kill extraction across the brain without binary downgrade. - extract_facts MCP op short-circuits with zero-counts envelope + a 'skipped: extraction_disabled' field when the flag is off (clean success, not permission_denied). - put_page facts backstop respects the same flag — eligibility check now returns 'extraction_disabled' as the skipped reason. - src/commands/doctor.ts: new facts_health check (runs after queue_health, before index_audit). Probes for the facts table existence (post-v40 guard), then surfaces total_active / total_today / total_week / total_consolidated + top-3 entities for the default source. Pre-v0.31 brains report "facts table not present (pre-v0.31 brain or migration pending)". - CHANGELOG.md: full v0.31.0 entry in the GStack release-summary voice. Headline + numbers-table + what-it-ships + itemized changes + "To take advantage of v0.31" upgrade block + out-of-scope. Honest about the HALFVEC + serve-http refactor + ErrorCode-open-union complications. - README.md: cycle phase list updated 8 → 10 (consolidate + purge). New "v0.31 Hot Memory" command block under Commands with recall + forget variants, kind icons, --as-context surface for headless agents. Test gates: 28 facts unit tests pass after the kill-switch wiring + doctor check ride-along. Typecheck clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix(migrate): add facts→sources FK explicitly via ALTER TABLE The inline column-level FK declaration on facts.source_id worked on PGLite but silently got dropped on Postgres in the v0.31 e2e run — the migration handler ran via postgres-js's `unsafe()` multi-statement path and the resulting facts table came back without the `facts_source_id_fkey` constraint. Same psql input run directly against the same database produced the FK; the difference was the unsafe() pipeline, not the SQL itself. Splitting the FK into a separate ALTER TABLE inside a DO block makes the constraint declaration explicit and idempotent: the named constraint either exists or it doesn't, the ALTER is a no-op on re-runs, and the failure mode is loud rather than silently leaving a CASCADE-less foreign key behind. Without this fix, deleting a source row leaves orphaned facts rows (test/e2e/facts-cross-source-isolation.test.ts CASCADE-on-sources- delete case caught it). With this fix the constraint is in place, the cascade fires, and both PG + PGLite e2e suites stay green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 test: update phase-count assertions for the new consolidate phase Three e2e/unit tests pinned the cycle phase count or order, all now updated to reflect v0.31's 10-phase cycle: - test/e2e/dream-cycle-eight-phase-pglite.test.ts: describe rename "8-phase cycle" → "10-phase cycle"; ALL_PHASES expectation extended to include 'consolidate' (between patterns + embed) and 'purge' (the v0.26.5 addition that was already in ALL_PHASES but missing from the test's assertion list). totals match adds the new facts_consolidated + consolidate_takes_written fields plus the pre-existing purged_sources_count + purged_pages_count that should have been added when v0.26.5 landed. - test/e2e/cycle.test.ts: dry-run full cycle now expects report.phases.length === 10 (was 9). - test/core/cycle.serial.test.ts: yieldBetweenPhases hook count + full cycle phases.length both updated 9 → 10. Comments call out the v0.31 addition lineage so the next person to add a phase sees the precedent. These are mechanical assertion bumps. The tests pass against the updated assertions on PGLite and Postgres. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix(test): truncate facts table between e2e describe blocks setupDB() truncates ALL_TABLES between every describe block's beforeAll() hook. The list missed the new v0.31 facts table, so facts seeded by an earlier describe block leaked into Garry's Separation Test on Postgres — listFactsByEntity('travel') returned 2 rows instead of 1 because a prior facts-context-injection test had also seeded a 'travel' fact. Adding 'facts' to the truncate list (before 'pages' to respect FK ordering) makes every describe-block start from an empty facts table. Pinned by re-running the e2e file ordering that originally caught it (facts-recall-render → cross-source-isolation → serve-http-meta → context-injection → separation-postgres → facts-forget) — 13 pass / 0 fail after the fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 test: meta-hook cache + Postgres consolidate phase coverage Two net-new test files filling real coverage gaps the earlier sweep missed: - test/facts-meta-cache.test.ts (5 cases) — pins the eD3/eD10 cache contract that the dispatcher relies on. 30s TTL hit path, post-bump fresh-query, scoped invalidation (bump for sess-A leaves sess-B cache warm — closes the cross-source leak risk codex F5 originally surfaced on the recall payload), facts-self ops skip injection (anti-loop on recall / extract_facts / forget_fact), distinct allow-lists produce distinct cache entries. - test/e2e/cycle-consolidate-postgres.test.ts (3 cases) — Postgres parity for the dream-cycle consolidate phase. Mirrors the PGLite unit test but exercises the real postgres-engine codepaths: sql.begin transactions, advisory locks on insertFact's entity-slug dedup window, unsafe('::vector') casts on findCandidateDuplicates ordering, addTakesBatch postgres-js unnest path. Happy path (4 facts → 1 take + all consolidated_into set), age-threshold skip, dry-run no-write. All 5 unit + 3 e2e tests pass. Closes the unit-only gap on the consolidate phase (was only PGLite-tested) and pins meta-cache invariants the dispatcher depends on. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix: thread auth + sourceId, JSON-shape every error envelope Three bugs surfaced during the full e2e sweep that all trace back to my v0.31 dispatch refactor (D12/eE1) silently dropping auth threading + non-OperationError exceptions emitting plain strings: 1. **HTTP MCP transport lost ctx.auth.** Refactoring serve-http.ts to call dispatchToolCall meant auth had to come through DispatchOpts, but the field didn't exist yet. Every HTTP whoami call returned `unknown_transport` because ctx.auth was undefined. Added `auth?: AuthInfo` to DispatchOpts, plumbed it through buildOperationContext, and updated serve-http.ts:816 to pass `auth: authInfo` alongside sourceId/takesHoldersAllowList. Pinned by sources-remote-mcp e2e `whoami reports oauth transport + sources_admin scope`. 2. **Non-OperationError exceptions emitted plain strings, not JSON.** The pre-v0.31 serve-http.ts always wrapped errors in JSON envelope `{error, message}`; my dispatch refactor missed the unknown-tool + uncaught-throw paths and emitted `Error: ${msg}` text content. Every caller that did `JSON.parse(content)` (sources-remote-mcp callMcp helper at line 104) crashed with `Unexpected identifier "Error"`. Both error paths in dispatchToolCall now return JSON-shaped content matching the OperationError pattern. 3. **Files→sources FK silently lost on rewound bootstrap path.** test/e2e/postgres-bootstrap.test.ts simulates a pre-v0.21 brain by `DROP TABLE IF EXISTS sources CASCADE` which removes files_source_id_fkey while leaving files.source_id intact. The v23 migration's `ALTER TABLE files ADD COLUMN IF NOT EXISTS source_id ... REFERENCES sources(id) ON DELETE CASCADE` is a no-op when the column exists, so the FK never came back on upgrade — and any sources-remove afterward stopped cascading to files. Added a defensive `IF NOT EXISTS files_source_id_fkey ... ALTER TABLE ADD CONSTRAINT` block inside v23's handler. Pinned by `multi-source — cascade delete covers every dependent row` after running postgres-bootstrap. Plus: src/core/preferences.ts now honors GBRAIN_HOME for `~/.gbrain/migrations/completed.jsonl`. Without this, the doctor exits-0 mechanical test inherits the developer machine's stale partial-migration ledger entries (0.21.0, 0.22.4, 0.28.0, 0.29.1 prior dev work) and surfaces them as the [FAIL] minions_migration check. GBRAIN_HOME-scoped tempdir per test now isolates this state cleanly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 chore: scrub personal references from public artifacts Per the CLAUDE.md privacy rule on `Garry's Separation Test`, replace personally-coded references in v0.31 artifacts with neutral examples: - CHANGELOG.md v0.31 entry: rename "Garry's Separation Test" header to "The cross-session test" + drop the "topic-2659/topic-1941, 7 AM/2 PM, flying to Tokyo" narrative. - src/commands/migrations/v0_31_0.ts feature pitch: same scrub. - test/facts-separation-pglite.test.ts + test/e2e/facts-separation-postgres.test.ts: rename describe blocks; replace specific topic-NNNN session ids with session-A / session-B; replace personal sample fact with "sample event Tuesday". - src/core/facts/extract.ts extractor system prompt example slugs: people/sam-altman → people/alice-example; companies/anthropic → companies/acme. - src/core/entities/resolve.ts comment: Sam Altman → Alice Example. - All v0.31 test fixtures: people/sam → people/alice-example, Sam Altman → Alice Example, sam-the-cofounder → alice-the-cofounder. Test names referencing real-world entities replaced with neutral slugs. Pre-existing references to "Garry" elsewhere in CHANGELOG (v0.17, v0.19, v0.21+ entries) are untouched — that's a separate scope from this v0.31 ship. Plus: the truncate fix for the Bun-script-induced syntax error in test/e2e/mechanical.test.ts (cliEnv arrow function had ", 30_000)" tacked onto its closing brace by the bulk-add-timeouts script — repaired to a clean function definition). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix(test): bump E2E phase-count assertions for 11-phase cycle Two E2E tests still asserted the v0.31 pre-merge 10-phase shape (consolidate inserted, but recompute_emotional_weight from v0.29 not yet absorbed). With master's v0.29 work merged in, the cycle is now 11 phases: lint → backlinks → sync → synthesize → extract → patterns → recompute_emotional_weight → consolidate → embed → orphans → purge. - test/e2e/cycle.test.ts: 10 → 11 - test/e2e/dream-cycle-eight-phase-pglite.test.ts: ALL_PHASES + dry-run order Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix(merge): close brace between v44 and v45 migration objects The v0.30.2 merge resolution stitched master's v40-v44 migrations onto HEAD's v45 (facts hot memory) migration but lost the closing `},` between v44 and v45. tsc caught it as TS1136 Property assignment expected at migrate.ts:2188. This is a one-line bracket fix; the rest of the merge resolution is correct and tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 fix: put_page cliHints + buildPlan v0.31.0 in skippedFuture Two unit-test failures surfaced after the v0.30.2 merge: 1. operations.ts: put_page had `cliHints: { name: 'put', positional: ['stdin'] }` from earlier v0.31 development. The parity test enforces that every name in `positional` is a real param. Restored master's correct shape: `{ name: 'put', positional: ['slug'], stdin: 'content' }`. 2. test/apply-migrations.test.ts: the H9 regression tests pin the exact skippedFuture list. Adding v0.31.0 to the registry meant the list grew by one. Updated both `expect(...).toEqual([...])` assertions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * v0.31 docs: clarify consolidate is 11th phase + regen llms-full.txt CHANGELOG.md narrative said "new 10th phase consolidate"; with v0.29's recompute_emotional_weight already on master, consolidate is the 11th phase (between recompute and embed). Schema migration is v45, not v40, after the merge resolution renumbered it to clear master's v40-v44. llms-full.txt regenerated to reflect the README's 11-phase dream-cycle phrasing (the build-llms test enforces commit-time parity). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
|
55ca4984b2 |
feat: v0.17.0 — gbrain dream + runCycle primitive (one cycle, two CLIs) (#321)
* fix(sync): honor --dry-run in full-sync path + expose embedded count
Precondition for v0.17 brain maintenance cycle (runCycle primitive).
The full-sync path (performFullSync) previously called runImport() even
when opts.dryRun was true, silently writing to the DB and advancing
sync.last_commit. `gbrain sync --dry-run` on a fresh brain (or with
--full) would mutate state without warning.
Fix:
- performFullSync now early-returns a `dry_run` SyncResult when
opts.dryRun is set. Walks the repo via collectMarkdownFiles +
isSyncable to count what WOULD be imported. No writes, no git
state advance.
- SyncResult gains an `embedded: number` field (required). Tracks
pages re-embedded during the sync's auto-embed step. Existing
return sites set 0; the synced + first_sync paths set real counts
(best-estimate until commit 2 sharpens runEmbedCore's return type).
- first_sync path now returns real added + chunksCreated counts
from runImport instead of hardcoded zeros.
- printSyncResult shows embedded count in human output.
Tests (test/sync.test.ts, new `performSync dry-run never writes`
block, PGLite + temp git repo, no DATABASE_URL required):
- first-sync --dry-run: no pages, no sync.last_commit
- incremental --dry-run after real sync: bookmark unchanged
- --full --dry-run: no reimport, bookmark unchanged
- SyncResult.embedded is a number
Codex outside-voice caught this. Would have shipped silent DB writes
on dry-run for anyone using `gbrain sync --dry-run --full`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(embed): add dry-run mode + return EmbedResult with counts
Precondition for v0.17 brain maintenance cycle (runCycle primitive).
runEmbedCore previously returned Promise<void> and had no dry-run mode.
That made it impossible for runCycle to (a) report accurate embedded
counts or (b) honor --dry-run without also skipping the entire embed
phase (which would have required runCycle to know embed's internal
semantics — a layering violation).
Changes:
- EmbedOpts gains `dryRun?: boolean`. When set, embedPage and
embedAll enumerate stale chunks (or would-be-created chunks for
unchunked pages, via local chunkText without engine.upsertChunks)
but never call embedBatch and never write to the engine.
- runEmbedCore: Promise<void> -> Promise<EmbedResult>. Result shape:
{ embedded, skipped, would_embed, total_chunks, pages_processed,
dryRun }.
embedded = chunks newly embedded (0 in dryRun).
would_embed = chunks that WOULD be embedded (0 in non-dryRun).
skipped = chunks with pre-existing embeddings.
- runEmbed CLI wrapper honors --dry-run flag and returns the result
through. `gbrain embed --stale --dry-run` is now a safe preview.
- Callers ignoring the return value (sync auto-embed, autopilot
inline fallback, jobs.ts handlers, CLI) keep compiling — the new
return type is additive for `await` callers.
Tests (test/embed.test.ts, new `runEmbedCore --dry-run` block, uses
the existing mock.module embedBatch pattern, no API key required):
- dry-run --all: zero embedBatch calls, zero upsertChunks calls,
would_embed matches stale chunk total
- dry-run --stale correctly splits stale vs already-embedded counts
- dry-run --slugs on a single page tallies per-chunk counts
- non-dry-run regression guard: embedded count matches across
concurrent workers
Codex outside-voice flagged the Promise<void> return as a blocker for
accurate CycleReport.totals.pages_embedded.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* refactor(orphans): engine-injected queries, drop db.getConnection() global
Precondition for v0.17 brain maintenance cycle (runCycle primitive).
findOrphans + queryOrphanPages previously reached into the postgres-js
singleton via db.getConnection(), which (a) didn't compose with
runCycle's explicit-engine contract and (b) was wrong for PGLite test
fixtures and for any caller not using the default global connection.
Codex outside-voice flagged this as a blocker.
Changes:
- BrainEngine interface gains findOrphanPages() — returns pages with
no inbound links via the same NOT EXISTS anti-join. Implemented on
both postgres-engine (sql tag) and pglite-engine (db.query).
- findOrphans signature: findOrphans(engine, { includePseudo }).
Engine is required. Uses engine.findOrphanPages() and
engine.getStats().page_count instead of raw SQL + global counts.
- queryOrphanPages signature: queryOrphanPages(engine). Delegates to
engine.findOrphanPages().
- src/commands/orphans.ts drops the `import * as db` — no more
global-state coupling.
- Callers updated: src/core/operations.ts find_orphans handler now
passes ctx.engine through; runOrphans CLI entry uses its engine arg.
- No signature change needed in cli.ts (it was already passing engine
via CLI_ONLY dispatch).
Tests (test/orphans.test.ts, new `findOrphans (engine-injected)`
describe block, PGLite in-memory, no DATABASE_URL required):
- links correctly scope orphans (alice links to bob -> bob not
an orphan; alice is)
- includePseudo:true surfaces _atlas-style pages
- queryOrphanPages delegates to passed engine
- empty brain returns {orphans: [], total_pages: 0} without crashing
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cycle): add runCycle primitive in src/core/cycle.ts
The brain maintenance cycle as a single function. Six phases in
semantically-driven order (fix files → sync → extract → embed →
report orphans). Pure composition of existing library calls — no
execSync, no subprocess anti-patterns, no regex-parsed output.
┌───────────────────────────────────────────────────┐
│ runCycle(engine, opts) → CycleReport │
│ Phase 1: lint --fix (fs writes) │
│ Phase 2: backlinks --fix (fs writes) │
│ Phase 3: sync (DB picks up 1+2) │
│ Phase 4: extract (DB picks up links) │
│ Phase 5: embed --stale (DB writes) │
│ Phase 6: orphans (DB read, report) │
└───────────────────────────────────────────────────┘
Why the commit-4 primitive:
- CEO + Eng + Codex reviews all converged on "extract one cycle
function, wire both dream and autopilot through it." Two CLIs,
one definition of what the brain does overnight.
- Phase order was wrong in PR #309's original dream.ts (sync
before lint+backlinks lost the "fix files, then index them"
semantic).
- This commit is the bisectable foundation; commit 5 (dream)
and commit 6 (autopilot+jobs) just call into it.
Coordination — the codex-flagged blocker:
Session-scoped pg_try_advisory_lock does not survive PgBouncer
transaction pooling (the v0.15.4 fix made pooled connections the
default). Replaced with a DB lock table (gbrain_cycle_locks) that
works through every pooler:
- Acquire: INSERT ... ON CONFLICT DO UPDATE ... WHERE ttl < NOW()
- Refresh: UPDATE ttl_expires_at between phases via hook
- Release: DELETE in finally{}
- TTL: 30 min; crashed holders auto-release
PGLite / engine=null path uses a file lock at ~/.gbrain/cycle.lock
with PID liveness check. kill(pid, 0) with EPERM treated as alive
(so init/launchd-pid holders aren't mis-classified as stale).
Lock-skip: only phases that mutate state (lint, backlinks, sync,
extract, embed) trigger lock acquisition. orphans is read-only.
Single-phase --phase orphans runs never block on a held lock.
Engine-null mode preserved: filesystem phases run, DB phases skip
with {status:'skipped', reason:'no_database'}. Matches current
dream's capability that would have been lost if runCycle required
a connected engine.
Contract details:
- CycleReport has schema_version:"1" (stable, additive) so agents
consuming --json can rely on the shape
- status: 'ok' | 'clean' | 'partial' | 'skipped' | 'failed'.
'clean' = ran successfully with zero activity; agents trivially
detect a healthy brain.
- PhaseResult.error: { class, code, message, hint?, docs_url? }
(Stripe-API-tier structured failure info) when status='fail'
- yieldBetweenPhases hook: awaited between EVERY phase and before
return, runs even after phase failure, exceptions logged but
non-fatal. Required so the Minions autopilot-cycle handler can
renew its job lock between phases (prevents the v0.14 stall-death
regression codex flagged).
- git pull explicit: opts.pull defaults to false (cron-safe).
Autopilot daemon callers opt in if user configured it.
- extract phase doesn't have a dry-run mode in the underlying
library function, so runCycle honestly skips extract when
dryRun=true (status:'skipped', reason:'no_dry_run_support').
Schema migration v16: gbrain_cycle_locks table + idx_cycle_locks_ttl.
Also appended to src/schema.sql and src/core/pglite-schema.ts for
fresh installs. schema-embedded.ts regenerated via build:schema.
Tests (test/core/cycle.test.ts, PGLite in-memory + mocked library
functions, no DATABASE_URL required):
- dryRun × phases matrix: dryRun:true reaches lint/backlinks/sync/
embed; extract is honestly skipped
- Phase selection: default runs all 6 in order; --phase lint runs
only lint; --phase orphans runs only orphans
- Lock semantics: acquire + release on mutating phases, skip
entirely for read-only selections
- cycle_already_running: seeded live-holder lock → status:skipped,
zero phase runs; TTL-expired holder → auto-claimed
- Engine null: filesystem phases run, DB phases skip
- File lock (engine=null) blocks when PID 1 holds lock with fresh
mtime — exercises the PID liveness branch including EPERM
- Status derivation: 'ok' vs 'clean' vs 'partial' vs 'skipped'
- yieldBetweenPhases called N times, hook exceptions non-fatal
Next: commit 5 rewrites dream.ts as a thin CLI alias over runCycle,
commit 6 migrates autopilot daemon + jobs.ts handler to delegate to
runCycle too.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(dream): add gbrain dream CLI as a thin alias over runCycle
`gbrain dream` is the README brand-promise command: "the agent runs
while I sleep, the dream cycle ... I wake up and the brain is smarter."
Cron-friendly, JSON-reportable, phase-selectable. Same maintenance
cycle as `gbrain autopilot`, just scheduled differently — both
converge on runCycle (added in commit 4) so there's one source of
truth for what happens overnight.
Contract:
gbrain dream # full 6-phase cycle
gbrain dream --dry-run # preview, no writes
gbrain dream --json # CycleReport JSON (agent-readable)
gbrain dream --phase <name> # single-phase run
gbrain dream --pull # git pull before syncing
gbrain dream --dir /path/to/brain # explicit brain location
Cron: 0 2 * * * gbrain dream --json >> /var/log/gbrain-dream.log
Behavior details:
- Brain-dir resolution: requires explicit --dir OR sync.repo_path
in engine config. No more walk-up-cwd-for-.git footgun that
PR #309's original dream.ts had (would lint unrelated git repos).
- engine=null mode preserved via cli.ts's try/catch around
connectEngine — filesystem phases (lint, backlinks) still run
without a DB, DB phases report skipped/no_database in the output.
- status=clean prints "Brain is healthy. N phase(s) checked in Ns."
status=skipped prints the reason (cycle_already_running, etc.).
Partial/failed prints the phase-by-phase detail.
- Exit code 1 when status=failed (cron spots real problems).
'partial' is not a failure — warnings shouldn't page you.
- --help text cross-references `autopilot --install` for users
who want continuous maintenance as a daemon.
CLI registration (src/cli.ts):
- 'dream' added to CLI_ONLY
- handleCliOnly has a pre-engine branch mirroring doctor's pattern:
try connectEngine() → ok path; catch → runDream(null, args) so
filesystem phases still run when DB is down
- Help text updated with one-line dream entry and autopilot cross-ref
Tests (test/dream.test.ts, real PGLite + real library calls, no mocks
to avoid `mock.module` leakage across test files):
- brainDir resolution: explicit --dir wins, engine config fallback,
missing + nonexistent errors
- phase selection: --phase lint|orphans produces single-phase report
- phase validation: --phase garbage exits 1
- output: --json parses as CycleReport with schema_version:"1"
- human output mentions "Brain is healthy" on clean status
- dry-run: cycle runs but DB stays untouched
- exit code: clean/ok/partial do not call process.exit
Also (test/core/cycle.test.ts): refactored to use beforeAll/afterAll
with one shared PGLite engine per describe + truncateCycleLocks
between tests. Cuts test time from ~11s to ~4s; avoids the 15-migration
penalty per test that was causing parallel-suite timeout flakes.
Co-Authored-By: Wintermute <wintermute@garrytan.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat: v0.17.0 — autopilot + jobs delegate to runCycle (unifies the cycle)
Autopilot daemon (`--inline` path) and Minions `autopilot-cycle`
handler both now delegate to `runCycle` (introduced in commit 4).
Three callers, one cycle definition:
1. `gbrain dream` — one-shot cron cycle
2. `gbrain autopilot` daemon inline path — scheduled cycles
3. `autopilot-cycle` Minions handler — durable queue with retry
All three share:
- Same 6 phases in same order (lint → backlinks → sync → extract →
embed → orphans)
- Same DB lock table coordination (`gbrain_cycle_locks`)
- Same yieldBetweenPhases discipline (prevents v0.14 stall-death)
- Same structured CycleReport output
Autopilot inline path gains lint + orphan sweep that the old path
skipped. Minions autopilot-cycle handler also gains lint + orphans.
Users who run `gbrain autopilot --install` see 6-phase reports in
`gbrain jobs get <id>` starting on next interval. No config change
required.
Changes:
- `src/commands/autopilot.ts`: inline fallback path (~20 lines)
replaces the ~22-line sync+extract+embed sequence with a single
runCycle call. Uses pull:true (matches pre-v0.17 autopilot
behavior). Uses setImmediate yield hook. Status/failure reporting
derives from CycleReport.status. `--help` cross-references `gbrain
dream` for one-shot use.
- `src/commands/jobs.ts:579` (`autopilot-cycle` handler): replaces
the 4-step try/catch sequence with a runCycle call. Returns
`{ partial, status, report }` so `gbrain jobs get <id>` shows the
full structured CycleReport. Preserves partial-failure semantic
(one phase failing does NOT throw; next cycle still runs).
yieldBetweenPhases yields the event loop between phases for the
worker's lock-renewal timer.
Release scaffolding:
- VERSION: 0.16.0 → 0.17.0
- CHANGELOG.md: v0.17.0 entry in GStack voice — headline, numbers
table, "what this means" paragraph, "To take advantage" block
per CLAUDE.md post-ship rules. Itemized changes below the fold.
Credit to @Wintermute for the original PR #309 thesis.
- skills/migrations/v0.17.0.md: documents what changed for
upgrading users. No mechanical action required — schema migration
v16 (cycle locks table) + handler delegation both apply
automatically. Includes opt-out paths for users who don't want
their daemon modifying files (use `dream --phase orphans` in cron
and skip autopilot-install, or other explicit configs).
- CLAUDE.md: new entries for `src/core/cycle.ts` and
`src/commands/dream.ts` with contract details.
Tests: no new test file needed for this commit — the cycle primitive
is extensively tested in test/core/cycle.test.ts (18 cases), dream
in test/dream.test.ts (11), and autopilot's delegation is mechanical
(calls runCycle with specific opts). The handler contract is covered
implicitly: if runCycle returns a CycleReport, the handler wraps it
in `{ partial, status, report }` — nothing else to assert.
Verified:
- `bun test test/autopilot-install.test.ts test/autopilot-resolve-cli.test.ts test/core/cycle.test.ts test/dream.test.ts` → 37 pass, 0 fail
Completes the v0.17.0 feature: 6 bisectable commits on one branch
(garrytan/v0.17-dream-cycle), ready to push as one PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(e2e): add runCycle + dream E2E coverage against real Postgres
Gap from the v0.17 commit series: PR #321 shipped unit-level tests
for runCycle (test/core/cycle.test.ts) and dream (test/dream.test.ts)
but no E2E coverage that exercises the real Postgres paths. Filling
that in before merge.
test/e2e/cycle.test.ts (6 cases):
- schema migration v16 created gbrain_cycle_locks + index
- dry-run full cycle: zero DB writes + lock table empty after
- live cycle: pages + chunks materialize, sync.last_commit set
- concurrent cycle blocked by lock → status:'skipped'
- TTL-expired lock auto-claimed (crashed-holder recovery)
- --phase orphans skips lock entirely (read-only optimization)
test/e2e/dream.test.ts (3 cases):
- dream --dry-run --json emits valid CycleReport + DB stays empty
- dream (no --dry-run) syncs pages into real DB
- dream --phase orphans doesn't touch the cycle-lock table
Both files mock embedBatch via mock.module so the embed phase never
calls OpenAI even when the full 6-phase cycle runs (zero API cost,
zero flakiness from network calls).
Verified locally:
- `docker run pgvector/pgvector:pg16` on port 5434
- `DATABASE_URL=... bun test test/e2e/cycle.test.ts test/e2e/dream.test.ts` → 9 pass, 0 fail
- Full E2E suite (`bun run test:e2e`): 16 files, 150 tests, 0 fail
- Container torn down after: `docker stop + rm gbrain-test-pg`
Per CLAUDE.md E2E test DB lifecycle. These tests skip gracefully when
DATABASE_URL isn't set (via hasDatabase() helper + describe.skip).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Wintermute <wintermute@garrytan.com>
|