mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 11:22:34 +00:00
Merge remote-tracking branch 'origin/master' into garrytan/cathedral-2
# Conflicts: # CHANGELOG.md # TODOS.md # VERSION # package.json
This commit is contained in:
Garry Tan
@@ -38,6 +38,34 @@ Every scoreboard row carries a `seam` label: the `openclaw` row exercises the sh
|
||||
- CLI exit codes for the new command route through the shared write-fence + aliveness-grace exit seam, so PGLite's WASM exit-code stomping and Bun's exit-time stdout discard can't corrupt the CI contract.
|
||||
|
||||
To take advantage of v0.44.0.0: run `gbrain eval brainbench` — no setup, no keys, no brain required. If it ever reports something broken after an upgrade, `bun evals/brainbench/generator/gen.ts` rebuilds the corpus byte-identically and `gbrain eval brainbench --update-baseline` re-derives the baseline from an actual run; both are safe to re-run any time.
|
||||
## [0.42.57.0] - 2026-07-02
|
||||
|
||||
**PGLite incident fix: a busy `gbrain dream` (or `embed`) could have its data-directory lock stolen and get its brain corrupted beyond in-place repair. The lock will no longer be taken from a process that is alive, and an already-corrupted store now tells you exactly how to recover.**
|
||||
|
||||
### Fixed
|
||||
- **A live PGLite holder is never stolen.** The data-directory lock used to be reaped if the holder's heartbeat went stale past a grace window. But the heartbeat runs on the JS event loop, which is blocked during long synchronous WASM imports/checkpoints, so a genuinely working `gbrain dream`/`embed` could look stale while fully alive. Reaping it let a second process open the same store and corrupt the catalog + pgvector extension (surfacing later as `relation "content_chunks" does not exist` / `type "vector" does not exist`, only recoverable by wipe-and-restore). The lock is now reaped only when the holder process is actually dead; a wedged-but-alive or PID-reused holder makes the acquire time out with a clear message naming the PID, instead of risking corruption.
|
||||
- **A corrupted PGLite store now explains how to recover.** When the store's catalog or pgvector extension can no longer load, the error names the cause and points at `gbrain reinit-pglite --embedding-model <id> --embedding-dimensions <N>` (or restoring a backup), instead of the unrelated "macOS WASM bug" hint. It also notes that deleting the lock dir or `postmaster.pid` does not fix it.
|
||||
|
||||
### To take advantage of v0.42.57.0
|
||||
`gbrain upgrade`. No migration. New corruption is prevented going forward. A brain already corrupted by a prior concurrent open cannot be repaired in place; the upgraded error message walks you through `gbrain reinit-pglite` or restoring a backup.
|
||||
|
||||
## [0.42.56.0] - 2026-07-02
|
||||
|
||||
**Life Chronicle: gbrain gains a temporal spine. Meetings and transcripts project into a queryable timeline, entities carry a bi-temporal ontology (sourced, confidence-weighted properties that supersede over time), and a low-friction diary captures interiority — so an agent can reconstruct "what happened the week of X", answer "when did I last interact with Y", and see how an entity's role or stance changed, instead of re-deriving chronology from scratch every session.** Built entirely on existing primitives (pages, the `facts` table, `timeline_entries`) — no new datastore. Auto-emission is off by default; opt in per below.
|
||||
|
||||
### Added
|
||||
- **Timeline events + reads.** Meetings/transcripts auto-emit `type:event` atoms (when·where·who·what) that project into a date index and backlink to the depth page. Query them with `gbrain day <date> [--week] [--narrative]`, `gbrain since <date> [--kind]`, `gbrain last-seen <entity>`, and `gbrain on-this-day`. Intra-day ordering, read-time hiding of deleted events, and source isolation throughout.
|
||||
- **Bi-temporal per-entity ontology.** An open-world, sourced, confidence-weighted property bag rides the existing `facts` table (new `dimension`/`value` columns). A new value supersedes the prior across a validity window, so `gbrain ontology <entity> [--asof <date>]` can time-travel; genuine two-source disagreement surfaces via `gbrain ontology-contradictions`; `gbrain ontology-dimensions` shows what the brain tracks about entities. Novel LLM-proposed dimensions quarantine until confirmed.
|
||||
- **Diary capture + agent orientation.** `gbrain capture --type diary` (and `--type event` with `--who/--what/--where/--kind`) for low-friction entries; `gbrain orient` hands an agent the recent timeline plus resolved-entity ontology in one zero-LLM payload. `gbrain chronicle-backfill` sweeps existing meetings into the timeline.
|
||||
- **Ambient temporal recall + proactive surfacing.** Temporal queries lift chronicle pages in search; `gbrain advisor` flags unresolved ontology conflicts and recent meetings missing from the timeline. A deterministic `gbrain eval chronicle` gates the feature (day-order, last-seen, supersession, contradiction, source isolation).
|
||||
- **Migrations v121 (event-projection column) + v122 (facts ontology columns).** Additive; legacy rows unchanged.
|
||||
|
||||
### Security
|
||||
- **Interiority stays local.** Diary content and diary-sourced ontology are redacted from untrusted (remote/MCP) readers across reads, search, advisor, and orientation. Auto-emission runs only for trusted local writes.
|
||||
|
||||
### To take advantage of v0.42.56.0
|
||||
`gbrain upgrade`, then `gbrain apply-migrations --yes` (or any command that opens the brain) to pick up v121/v122. Auto-emission is OFF by default: turn it on with `gbrain config set auto_chronicle true`, then `gbrain chronicle-backfill` to populate the timeline from existing meetings. Manual capture (`gbrain capture --type event/diary`) and every read surface work immediately. Closes #2390 (duplicate #2388).
|
||||
|
||||
## [0.42.55.0] - 2026-06-24
|
||||
|
||||
**A security-hardening pass: routing dotfiles, the skills directory, page slugs, and large-file transcription are confined against multi-user-host and untrusted-input edge cases; dynamic OAuth client registration defaults to a consent-bearing grant; and a schema-lint migration brings existing brains to the fresh-install posture.** Several of these close community-reported gaps. Fresh installs were already covered; existing brains are brought to the same bar automatically on upgrade. If `gbrain doctor` flags anything afterward, its message names the object and the exact fix.
|
||||
|
||||
@@ -16,6 +16,51 @@ Deferred from the BrainBench wave (eng-reviewed; plan + GSTACK REVIEW REPORT at
|
||||
- [ ] **Periodic re-baselining (the ratchet doesn't auto-tighten).** Improvements aren't banked into master's baseline until a PR updates it, so a regression back to a stale baseline level passes. Documented as an accepted residual in `docs/eval/BRAINBENCH.md`; the fix is an operator habit or a scheduled job that re-runs `--update-baseline` after metric-improving merges. Priority: P3.
|
||||
|
||||
- [ ] **Hermetic-ize the 7 env-sensitive LLM-availability tests.** `test/think-gateway-adapter.test.ts`, `test/conversation-parser/llm-base.test.ts`/`llm-fallback.test.ts`, `test/doctor-ze-checks.test.ts` assert behavior "when ANTHROPIC_API_KEY is unset" by reading the live process env — they fail on any dev shell that exports provider keys (verified failing on clean master in such a shell; green in keyless CI). Stub/save-restore the env per test so local runs match CI. Priority: P2.
|
||||
## Life Chronicle follow-ups (filed v0.42.56.0, #2390)
|
||||
|
||||
Deferred from the Life Chronicle wave (CEO Scope-Expansion + eng review CLEARED,
|
||||
3 codex rounds absorbed, PR #2533). Every item was an explicit review decision,
|
||||
not an oversight; each names its decision provenance.
|
||||
|
||||
- [ ] **P1 — Eval-gated auto-emit default-flip (D5.5 fast-follow).** Auto-emission
|
||||
ships OFF (`auto_chronicle=false`) per spend/consent posture. The headline
|
||||
fast-follow: run `gbrain eval chronicle` + a live-LLM OFF-vs-ON agent arm on a
|
||||
real brain, and if the lift holds, flip the default ON in the next minor with
|
||||
an upgrade notice. Where: `src/core/chronicle/config.ts`, upgrade banner in
|
||||
`src/commands/upgrade.ts`.
|
||||
- [ ] **P2 — Live-LLM OFF-vs-ON eval arm + LongMemEval temporal slice.** The
|
||||
shipped `gbrain eval chronicle` is the deterministic CI bar (6 gold tasks).
|
||||
The full North-Star proof adds (a) a live agent reconstructing a day with the
|
||||
chronicle ops ON vs OFF, and (b) the LongMemEval `question_type:
|
||||
temporal-reasoning` slice as secondary corroboration — verify the adapter can
|
||||
filter by question type first. Where: `src/eval/chronicle/harness.ts`,
|
||||
`src/commands/eval-longmemeval.ts`.
|
||||
- [ ] **P2 — Passive diary capture + consent model (D3.5/E5).** Active-only in v1
|
||||
by explicit decision (highest consent-risk surface). Passive detection of
|
||||
first-person interiority in transcripts requires a dedicated consent design:
|
||||
an explicit `chronicle.diary.passive` opt-in, a consent prompt, and
|
||||
provenance-aware redaction (the facts `visibility` lane is already in place).
|
||||
- [ ] **P2 — Ontology interval-splitting for backdated conflicts (G4).** A
|
||||
backdated observation whose validity window overlaps an existing row is
|
||||
flagged (not rewritten) in v1. Real interval algebra (split the prior window
|
||||
around the backdated fact) is deliberate follow-up scope; the conflict lane
|
||||
(`findOntologyConflicts`) is the holding surface. Where: both engines'
|
||||
`mergeOntologyFact`.
|
||||
- [ ] **P3 — Cross-brain federated timeline (D3.6/E6).** v1 holds source
|
||||
isolation (scoped-default, `--all-sources` opt-in within the host brain).
|
||||
Unifying across mounted team brains is its own epic with an access-policy
|
||||
surface.
|
||||
- [ ] **P3 — Place-as-entity (`gbrain where <venue>`).** `event.where` is
|
||||
captured as free text; resolving venues to entity pages + geo-adjacency
|
||||
queries is a follow-up.
|
||||
- [ ] **P3 — Richer meta-ontology dashboard.** `gbrain ontology-dimensions` is
|
||||
the v1 surface; a full dashboard (per-dimension drill-down, quarantine review
|
||||
queue for novel dimensions) is deferred until usage shows demand.
|
||||
- [ ] **P3 — Materialized daily timeline pages / emotional-arc view.** The
|
||||
query-time aggregator won D5.6; embeddable `life/timeline/YYYY/MM/DD.md`
|
||||
narrative pages (a single `materialize_timeline` cycle phase) revisit after
|
||||
the eval shows `reflect`-style recall needs them.
|
||||
|
||||
## reliability fix-wave follow-ups (filed v0.42.52.0)
|
||||
|
||||
Deferred from the autopilot/supervisor + sync/status/minion reliability wave
|
||||
|
||||
@@ -28,7 +28,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
|
||||
- `src/commands/doctor.ts` extension — `graph_signals_coverage` check wired into both `runDoctor` (local) and `doctorReportRemote` (HTTP/JSON thin-client path). Reads `search.graph_signals` config first, falls back to mode default; silent `ok` when disabled. Computes inbound link coverage on the page set; warns at <10% with `gbrain extract all` fix hint; `ok` at ≥30% ("fire on most queries") and 10-29% ("fire occasionally"), each with the percentage embedded. Pinned by cases in `test/doctor.test.ts`.
|
||||
- `src/core/engine-factory.ts` — Engine factory with dynamic imports (`'pglite'` | `'postgres'`).
|
||||
- `src/core/pglite-engine.ts` — PGLite (embedded Postgres 17.5 via WASM) implementation, all BrainEngine methods. `listLinkSources({sourceId?, sourceIds?})` returns distinct `link_source` provenances + counts (`ORDER BY count DESC, link_source ASC NULLS LAST`; scalar + federated scoped; parity with postgres-engine.ts) powering `gbrain link-sources`. `addLinksBatch`/`addTimelineEntriesBatch`/`addTakesBatch` pass the whole batch as one JSONB document via `jsonb_to_recordset(($1::jsonb)->'rows')` (bound through `executeRawJsonb` with a `{ rows }` wrapper; rows built by the shared `src/core/batch-rows.ts` helpers, NUL-stripped), and are `batchRetry`-wrapped. `connect()` wraps `PGlite.create()` in a try/catch that emits an actionable error (macOS 26.3 WASM bug #223, points at `gbrain doctor`); the lock is released on failure so the next process can retry cleanly. `searchKeyword`/`searchKeywordChunks` multiply `ts_rank` by the source-factor CASE at chunk grain; `searchVector` is a two-stage CTE — inner CTE keeps `ORDER BY cc.embedding <=> vec` so HNSW stays usable, outer SELECT re-ranks by `raw_score * source_factor`, inner LIMIT scales with offset to preserve pagination. `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL — probes for forward-referenced state the embedded blob needs (`pages.source_id`, `links.link_source`, `links.origin_page_id`, `content_chunks.symbol_name`, `content_chunks.language`, `sources` FK target, plus `files.source_id`, `files.page_id`, `oauth_clients.source_id`, `oauth_clients.federated_read`, `sources.archived`, `sources.archived_at`, `sources.archive_expires_at`) and adds only what's missing; threads the DDL connection from `initSchema` so probes run inside the advisory-lock scope; no-op on fresh installs and modern brains (closes the upgrade-wedge bug class #239/#243/#266/#357/#366/#374/#375/#378/#395/#396/#1018/#974/#820). `getBrainScore` returns 100/100 with full breakdown (35/25/15/15/10) when `pageCount === 0` (vacuous truth — empty brain has no coverage problem); Pinned by `test/brain-score-breakdown.test.ts` empty-brain assertion + `test/doctor-report-remote.serial.test.ts`. `disconnect()` uses snapshot+early-null (snapshot `_db`/`_lock`, null instance fields BEFORE any `await` so a concurrent `connect()` can't see a partial mid-close state) wrapped in try/finally guaranteeing lock-release even if `db.close()` throws; KEEPS close-then-release order (release-then-close was rejected: it would widen the window where a sibling process connects to a still-closing brain); Pinned by `test/pglite-engine-disconnect.serial.test.ts`. `PGlite.create()` runs inside `preservingProcessExitCode` (#2084): PGLite's Emscripten runtime writes its own status into `process.exitCode` (99 at create; in-memory brains run initdb whose status lands on a later tick; the exit status at close — and assigning `undefined` cannot clear a prior value), which would otherwise silently clobber an errored op's exit 1 back to 0. The wrapper keeps the global tidy for external readers; `db.close()` stays unwrapped (its 0-write is baseline behavior test runners depend on). The CLI's exit verdict is immune either way — it lives in the gbrain-owned channel in `cli-force-exit.ts` and never reads `process.exitCode` back. Exports `classifyPgliteInitError(message): 'bunfs' | 'macos-26-3' | 'unknown'` + `buildPgliteInitErrorMessage(verdict, original)` routing the catch-block hint by failure shape (`bunfs` matches literal `$$bunfs` OR `ENOENT[\s\S]*pglite\.data` co-occurrence, surfaces a paste-ready `bun upgrade` + Node fallback; `macos-26-3` keeps the #223 link; `unknown` falls through); Pinned by `test/pglite-init-classifier.test.ts`. Implements `deletePages(slugs, {sourceId})` + `resolveSlugsByPaths(paths, {sourceId})` via `slug = ANY($1::text[])` array-param binding, caller-chunking primitive throwing when input exceeds `DELETE_BATCH_SIZE`, `deletePages` returns `RETURNING slug` rows so callers filter `pagesAffected` to confirmed deletes. Implements the embedding-signature stale-detection quartet — `sumStaleChunkChars({sourceId?, signature?})`, `setPageEmbeddingSignature(slug, {sourceId?, signature})`, `invalidateStaleSignatureEmbeddings({signature, sourceId?})`, widened `countStaleChunks({sourceId?, signature?})` (the `signature` opt widens via `JOIN pages p ... WHERE cc.embedding IS NULL OR (p.embedding_signature IS NOT NULL AND p.embedding_signature <> $signature)`, NULL grandfathered); parity SQL with postgres-engine.ts. PGLite-specific DDL (pgvector, pg_trgm, triggers).
|
||||
- `src/core/pglite-lock.ts` (#2058) — advisory data-dir lock so two processes never open the same embedded PGLite (single-connection WASM Postgres) at once. Atomic `mkdir` of `.gbrain-lock/` + a `lock` file carrying `{pid, acquired_at, refreshed_at}`. A held lock HEARTBEATS its `refreshed_at` every 30s (`.unref()`ed timer) so a long-running but LIVE holder (an `embed` job can run for minutes) is never mistaken for stale. A waiting acquirer reaps the holder only when it is dead (PID gone) OR has stopped refreshing past the steal grace (`GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS`, default 600s) — pairing PID liveness with heartbeat freshness defeats BOTH the WAL-corruption bug (stealing a live writer) and the PID-reuse false positive (a recycled PID reading as alive). Each holder carries an ownership token (`<pid>:<acquired_at>`); the heartbeat and `releaseLock` verify the on-disk lock is STILL theirs before touching it, so a stalled-then-resumed holder that was already reaped + replaced can't clobber the new owner. In-memory engines take no lock (no file, no concurrent access). Pinned by `test/pglite-lock.test.ts`.
|
||||
- `src/core/pglite-lock.ts` — advisory data-dir lock so two processes never open the same embedded PGLite (single-connection WASM Postgres) at once. Atomic `mkdir` of `.gbrain-lock/` + a `lock` file carrying `{pid, acquired_at, refreshed_at}`. A held lock HEARTBEATS its `refreshed_at` every 30s (`.unref()`ed timer; informational). A waiting acquirer reaps a holder ONLY when its PID is dead — a LIVE holder is NEVER stolen, regardless of how stale its heartbeat is (#2348). The heartbeat runs on the JS event loop, which is blocked during long synchronous WASM imports/CHECKPOINTs, so a working `dream`/`embed` holder can look stale while alive; the old steal-on-stale-heartbeat grace let a second OS process open the same data dir and corrupt the catalog + pgvector extension (58P01 / `internal_load_library` / `type "vector" does not exist`), recoverable only by wipe+restore. A wedged-but-alive or PID-reused holder now makes the acquire TIME OUT with a message naming the PID (the user removes the lock explicitly) rather than risk corruption. Each holder carries an ownership token (`<pid>:<acquired_at>`); the heartbeat and `releaseLock` verify the on-disk lock is STILL theirs before touching it. In-memory engines take no lock. Pinned by `test/pglite-lock.test.ts`. A corrupted store surfaces a `reinit-pglite` recovery hint via `classifyPgliteInitError`'s `corrupt` verdict in `pglite-engine.ts`.
|
||||
- `src/core/postgres-engine.ts` — Postgres + pgvector implementation (Supabase / self-hosted). `addLinksBatch`/`addTimelineEntriesBatch`/`addTakesBatch` pass the batch as one JSONB document — `INSERT ... SELECT FROM jsonb_to_recordset(($1::jsonb)->'rows') AS v(...) JOIN pages ...` bound through `executeRawJsonb({ rows })` — which encodes arbitrary free text safely (the old `unnest(${arr}::text[])` array-literal path crashed Postgres with "malformed array literal" on calendar/Zoom context, gbrain#1861) and sidesteps the 65535-parameter cap; takes declares native recordset column types (`page_id int, weight real, active boolean, …`) so no per-element casts; all three are `batchRetry`-wrapped. `searchKeyword`/`searchVector` scope `statement_timeout` via `sql.begin` + `SET LOCAL` so the GUC dies with the transaction instead of leaking across the pooled postgres.js connection. `getEmbeddingsByChunkIds` uses `tryParseEmbedding` so one corrupt row skips+warns instead of killing the query. `searchKeyword`/`searchKeywordChunks`/`searchVector` apply source-aware ranking by inlining the source-factor CASE and `NOT (col LIKE …)` hard-exclude from `src/core/search/sql-ranking.ts`; `searchVector` is a two-stage CTE (HNSW-safe inner ORDER BY, source-boost re-rank in outer SELECT) carrying `p.source_id` inner→outer. `_savedConfig` retains the connect config; `reconnect()` tears down + recreates the pool from saved config (called by supervisor watchdog after 3 consecutive health-check failures, and by `batchRetry` on a retryable connection error). Concurrent callers share one in-flight `_reconnectPromise` (they await the single reconnect rather than racing a half-rebuilt pool); ownership re-samples through the atomic `db.connect()` token on the connect leg. `reconnect(ctx?)` accepts the triggering error and records a pool-recovery audit event (`reap_detected`/`reconnect_other`/`reconnect_succeeded`/`reconnect_failed`) for the `pool_reap_health` doctor check. `executeRaw` is a single-statement passthrough — no per-call retry (unsound for non-idempotent statements; recovery is supervisor-driven). `connect()` applies `resolveSessionTimeouts()` from `db.ts` as connection-time startup parameters (`statement_timeout`, `idle_in_transaction_session_timeout`) so orphan pgbouncer backends can't hold locks for hours. `countStaleChunks()`+`listStaleChunks()` server-side-filter on `embedding IS NULL` for `embed --stale` (eliminates ~76 MB/call client-side pull); `upsertChunks()` resets both `embedding` AND `embedded_at` to NULL when chunk_text changes without a new embedding. `initSchema()` calls `applyForwardReferenceBootstrap()` BEFORE replaying SCHEMA_SQL on the same probe set as PGLite (extended for column-only forward-reference cases: `files.source_id`, `files.page_id`, `oauth_clients.source_id`, `oauth_clients.federated_read`, `sources.archived`/`archived_at`/`archive_expires_at`); the entire probe path runs on the DDL connection threaded from `initSchema` (closing a concurrent-bootstrap race for Supabase pooler users); closes #1018/#974/#820. `disconnect()` is idempotent — `_connectionStyle` tracks whether the engine owns its pool (worker engines) or shares the module-level singleton; second call on an instance-pool engine is a no-op rather than clobbering the singleton; and a module-style engine only calls `db.disconnect()` when it owns the singleton (`_ownsModuleSingleton`, set from the `db.connect()` creation token), so a borrower probe engine's teardown leaves the cycle owner's connection intact. Pinned by `test/e2e/postgres-engine-disconnect-idempotency.test.ts` + `test/postgres-engine-singleton-ownership.test.ts`. `getBrainScore` empty-brain parity with PGLite — 100/100 with breakdown 35/25/15/15/10 when `pageCount === 0` (both engines must agree to keep `doctor-report-remote.serial.test.ts` deterministic). Implements `deletePages(slugs, {sourceId}): Promise<string[]>` via `DELETE FROM pages WHERE slug = ANY($1::text[]) AND source_id = $2 RETURNING slug` (single round-trip; caller chunks); `resolveSlugsByPaths` does `SELECT slug, source_path FROM pages WHERE source_path = ANY($1::text[]) AND source_id = $2`; FK cascades through `content_chunks`/`links`/`tags`/`raw_data`/`timeline_entries`/`page_versions`, `files.page_id`+`links.origin_page_id` go SET NULL; throws when input exceeds `DELETE_BATCH_SIZE` (from `src/core/engine-constants.ts`); both short-circuit on empty input. Implements the embedding-signature stale-detection quartet (`sumStaleChunkChars`, `setPageEmbeddingSignature`, `invalidateStaleSignatureEmbeddings`, widened `countStaleChunks`, all accept optional `signature` extending "stale" to model/dims-swap drift via the `pages.embedding_signature` JOIN, NULL grandfathered; the `embedding IS NULL` server-side filter is preserved as the no-signature fast path); Pinned by `test/e2e/engine-parity.test.ts`.
|
||||
- `src/core/cjk.ts` — Single source of truth for CJK detection. Exports `CJK_RANGES_REGEX`, `CJK_SLUG_CHARS` (character-class fragment for embedding inside other regexes), `CJK_SENTENCE_DELIMITERS` (`。！？`), `CJK_CLAUSE_DELIMITERS` (`；：，、`), `CJK_DENSITY_THRESHOLD = 0.30`, `hasCJK(s)`, `countCJKAwareWords(s)` (30% density threshold — English docs with one Japanese term stay whitespace-tokenized; Chinese-dominant docs get char-counted), and `escapeLikePattern(s)` (escapes `%`, `_`, `\\` for `ILIKE ... ESCAPE '\\'`). BMP-only ranges (Han / Hiragana / Katakana / Hangul Syllables). Consumers: `expansion.ts`, `sync.ts:slugifySegment`, `operations.ts:validatePageSlug + validateFilename`, `chunkers/recursive.ts:countWords + DELIMITERS`, `pglite-engine.ts:searchKeyword + searchKeywordChunks`.
|
||||
- `src/core/audit-slug-fallback.ts` — Weekly ISO-week-rotated audit JSONL at `~/.gbrain/audit/slug-fallback-YYYY-Www.jsonl`. `logSlugFallback(slug, sourcePath)` fires when `importFromFile` falls back to a frontmatter slug because `slugifyPath` returned empty (emoji / Thai / Arabic / non-CJK exotic-script filenames). `readRecentSlugFallbacks(days)` reads the last N days for `gbrain doctor`'s `slug_fallback_audit` check. Honors `GBRAIN_AUDIT_DIR` via the shared `resolveAuditDir()`. Separate surface from `sync-failures.jsonl` — that file carries bookmark-gating semantics that info events shouldn't trigger.
|
||||
@@ -153,6 +153,7 @@ per-release `**vX.Y.Z:**` narration — CI enforces this
|
||||
- `test/helpers/cli-pty-runner.ts` — generic real-PTY harness (~470 lines) using pure `Bun.spawn({terminal:})` (Bun 1.3.10+; engines.bun pin in package.json). Generic primitives only, no plan-mode orchestrators. Exports `launchPty`, `resolveBinary`, `stripAnsi`, `parseNumberedOptions`, `optionsSignature`, `isNumberedOptionListVisible`, `isTrustDialogVisible`. Self-tests in `test/cli-pty-runner.test.ts` (24 cases).
|
||||
- `src/core/skillpack/{init-brain-pack,brain-pack-advisory,brain-pack-lint,brain-resident-locate,nag-state}.ts` (#2180) — brain-resident skillpacks. `manifest-v1.ts` gains optional `brain_resident` + `schema_pack` (additive). `runInitBrainPack` scaffolds a pack beside brain content (`brain_resident:true`, exact `gbrain_min_version`, 5-section machine-parseable README); `applyWritePlan` is factored out of `init-scaffold.ts` for the shared refuse-overwrite loop. `brain-pack-lint.lintBrainPackTools` validates each skill's `tools:` against the serving op set (E6 version-skew). Topology A: `src/commands/sources.ts` `runAdd` prints `brain-pack-advisory` to stderr after `opsAddSource`, fail-open; `nag-state.ts` (`~/.gbrain/skillpack-nag-state.json`) keys declines by `(source-repo brain_id, source_id, pack_name)` with pure `decideNagAction` (first/reminder/version-bump/ceiling) — declines count ONLY on CLI-interactive displays. Topology B: `brain-resident-locate.loadResidentPacksForServer` (source-scoped via `sourceScopeOpts`) backs the `list_brain_skillpack` op; `getResidentSkillDetail` backs `get_skill` `source_id`; `scaffold_spec` is the git source, never a server FS path. Tests: `test/skillpack-{init-brain-pack,nag-state,brain-resident-locate}.test.ts` + the brain-resident cases in `test/skillpack-manifest-v1.test.ts`.
|
||||
- `src/core/advisor/{types,run,render,recommended-set,history,apply,collect-*}.ts` + `src/commands/advisor.ts` (#2180) — `gbrain advisor`: read-only ranked actions from brain state. `run.runAdvisor` executes 8 hardcoded collectors (version [cache-only], migration, schema-pack, stalled-jobs [absent-table tolerant], usage-shape, setup-smells, uninstalled-brain-pack, uninstalled-bundled), each in its own try/catch; `rankFindings` orders critical>warn>info then collector order, caps the info tail, and drops `workspace_dependent` findings when `remote` (A1). `render.ts` is the shared `=`-bar renderer used by the advisor AND `post-install-advisory.ts` (generalized to a single current-state `recommended-set.RECOMMENDED`, `install`→`scaffold`). `history.ts` appends bounded `~/.gbrain/advisor-history.jsonl` (no DB migration) for since-last-run deltas; local-only. `apply.resolveApplyTarget` is the allowlist+injection guard for `commands/advisor.ts --apply <id>` (structured argv, never a shell; local-only). The `advisor` op (`operations.ts`) is read-scoped, NOT localOnly, gated by `mcp.publish_advisor` (config.ts; default off) and strictly read-only on remote. CLI wired in `cli.ts` (`CLI_ONLY` + dispatch). Bundled skill `skills/gbrain-advisor/` + weekly cron recipe. Tests: `test/advisor-{core,apply,op-gate,ranking-eval}.test.ts`.
|
||||
- `src/core/chronicle/{eligibility,config,backstop,extract-events,ontology,narrative}.ts` + `src/eval/chronicle/harness.ts` + `src/commands/eval-chronicle.ts` (#2390) — Life Chronicle: the temporal spine. `eligibility.isChronicleEligible` decides which pages auto-emit events (meeting/conversation/calendar-event + directory rescue; diary and event pages NEVER eligible — privacy + anti-loop). `backstop.runChronicleBackstop` is the put_page hook body (fires ONLY on `status==='imported'` + the auto-link trust gate + the default-OFF `auto_chronicle` flag; enqueues a `chronicle_extract` minion job — LLM never runs on the write path). `extract-events.runChronicleExtract` is the job body: deterministic when/who, injectable judge (default = chat gateway), an ALL-or-nothing parse barrier (`isValidProposal` requires a real parseable date — a malformed batch writes NOTHING), then content-addressed `life/events/` pages + a `timeline_entries` projection via `engine.upsertEventProjection` (dedup `(event_page_id, date)`; idempotent re-runs). `ontology.ts` carries the deterministic pieces of the bi-temporal per-entity ontology that RIDES THE `facts` TABLE (migration v122 adds `dimension`/`value`/`value_hash`/`dim_status`): `valueHash` (normalized, timestamp-free → crash-retry idempotent), `normalizeDimension` (seed alias lexicon), `isNovelDimension` (novel → quarantined, excluded from resolution/context until confirmed). The engine methods (`mergeOntologyFact` — corroborate on same value, forward-supersede via `valid_until`+`superseded_by` on a new value, backdated conflicts kept + flagged; `getOntology` with `--asof` valid-time travel; `discoverOntologyDimensions`; `findOntologyConflicts` — currently-open rows only) live in BOTH engines; both engines are on the R8 `valid_until` write allow-list (engine-layer, `dimension IS NOT NULL` rows only). Chronicle reads (`getTimelineForDate`/`getSince`/`getLastSeen`/`getOnThisDay`) JOIN the depth page (`deleted_at IS NULL`), hide soft-deleted event projections at READ time, and order by event `effective_date` for intra-day sequence. Ops: `chronicle_day`/`chronicle_since`/`chronicle_last_seen`/`chronicle_on_this_day`/`ontology_*`/`volunteer_chronicle` (agent orientation via `src/core/context/chronicle-context.ts`)/`chronicle_backfill` (admin, localOnly). Diary privacy: diary-sourced ontology + conflict values redacted for `ctx.remote !== false` callers. Search: `applyChronicleTypeBoost` in `search/hybrid.ts` (bounded [1.0,1.25], fires only inside the `recency !== 'off'` post-fusion branch → non-temporal search bit-for-bit unchanged). Advisor collector `collect-chronicle.ts` (conflicts + coverage gap); doctor `chronicle_projection_health` (BRAIN category). Eval: `gbrain eval chronicle` — deterministic, own in-memory PGLite, 6 gold tasks (day order, last-seen, supersession, asof, conflict, isolation), exit 0 iff 6/6. Tests: `test/chronicle-*.test.ts`, `test/eval-chronicle.test.ts`.
|
||||
- `src/core/skill-manifest.ts` — parser for `skill-manifest.json` records. Used by skillpack installer to detect drift between the shipped bundle and the user's local edits, so updates merge instead of overwriting.
|
||||
- `src/commands/routing-eval.ts` + `src/core/routing-eval.ts` — `gbrain routing-eval` catches user phrasings that route to the wrong skill. Reads `skills/<name>/routing-eval.jsonl` fixtures (`{intent, expected_skill, ambiguous_with?}`). Structural layer runs in `check-resolvable` by default (zero API cost). `--llm` is a placeholder for a future LLM tie-break layer; today it emits a stderr notice and runs structural only. Uses `autoDetectSkillsDirReadOnly` and the same multi-file resolver merge as `check-resolvable`, so on OpenClaw layouts (`skills/RESOLVER.md` + `../AGENTS.md`) all three commands see the same trigger index. RESOLVER.md rows carry the full frontmatter `triggers:` arrays so the structural matcher sees realistic phrasings; ambiguous-fixture annotations cover deliberate skill chains like `enrich → article-enrichment`.
|
||||
- `src/core/filing-audit.ts` + `skills/_brain-filing-rules.json` — Check 6 of `check-resolvable`. Parses `writes_pages:` / `writes_to:` frontmatter on skills and audits their filing claims against the filing-rules JSON (error severity). Internal `parseFrontmatter` is a thin wrapper over the shared `src/core/skill-frontmatter.ts` parser so both filing-audit and skill-brain-first read the same shape (`tools?`, `triggers?`, `brain_first?: 'exempt'`, typed `brain_first_typo`) from one source of truth.
|
||||
|
||||
@@ -1417,6 +1417,14 @@ async function handleCliOnly(command: string, args: string[]) {
|
||||
return;
|
||||
}
|
||||
|
||||
// v0.42.x (#2390): `gbrain eval chronicle` is deterministic — brings its own
|
||||
// in-memory PGLite, no DB/gateway. CI fixture gate runs anywhere.
|
||||
if (command === 'eval' && args[0] === 'chronicle') {
|
||||
const { runEvalChronicle } = await import('./commands/eval-chronicle.ts');
|
||||
setCliExitVerdict(await runEvalChronicle(args.slice(1)));
|
||||
return;
|
||||
}
|
||||
|
||||
// v0.41.13.0: `gbrain eval conversation-parser` is pure-function
|
||||
// (parses fixture JSONL, runs parseConversation, scores results).
|
||||
// No DB access; bypass connectEngine entirely so the CI fixture
|
||||
|
||||
+47
-3
@@ -49,6 +49,12 @@ interface RunOpts {
|
||||
source?: string;
|
||||
quiet?: boolean;
|
||||
json?: boolean;
|
||||
// v0.42.x — Life Chronicle (#2390): manual `--type event` frontmatter sugar.
|
||||
who?: string; // comma-separated entity slugs
|
||||
what?: string;
|
||||
where?: string;
|
||||
kind?: string;
|
||||
depth?: string; // the depth page this event backlinks
|
||||
}
|
||||
|
||||
function parseArgs(args: string[]): RunOpts | { help: true; positional: string | undefined } {
|
||||
@@ -80,6 +86,12 @@ function parseArgs(args: string[]): RunOpts | { help: true; positional: string |
|
||||
if (v) opts.source = v;
|
||||
continue;
|
||||
}
|
||||
// v0.42.x — Life Chronicle event sugar.
|
||||
if (a === '--who') { const v = args[++i]; if (v) opts.who = v; continue; }
|
||||
if (a === '--what') { const v = args[++i]; if (v) opts.what = v; continue; }
|
||||
if (a === '--where') { const v = args[++i]; if (v) opts.where = v; continue; }
|
||||
if (a === '--kind') { const v = args[++i]; if (v) opts.kind = v; continue; }
|
||||
if (a === '--depth') { const v = args[++i]; if (v) opts.depth = v; continue; }
|
||||
if (a.startsWith('--')) continue; // unknown flag, ignore
|
||||
positional.push(a);
|
||||
}
|
||||
@@ -132,12 +144,21 @@ Examples:
|
||||
JOB=$(gbrain capture "..." --quiet)
|
||||
`;
|
||||
|
||||
function defaultSlug(content: string, now: Date = new Date()): string {
|
||||
// v0.42.x — Life Chronicle (#2390): route the default slug prefix by type so
|
||||
// `gbrain capture --type diary` lands under life/diary/ and `--type event`
|
||||
// under life/events/ (matching the chronicle path-prefix inference). Everything
|
||||
// else keeps the inbox/ default.
|
||||
function slugPrefixForType(type?: string): string {
|
||||
if (type === 'diary') return 'life/diary';
|
||||
if (type === 'event') return 'life/events';
|
||||
return 'inbox';
|
||||
}
|
||||
function defaultSlug(content: string, now: Date = new Date(), type?: string): string {
|
||||
const y = now.getUTCFullYear();
|
||||
const m = String(now.getUTCMonth() + 1).padStart(2, '0');
|
||||
const d = String(now.getUTCDate()).padStart(2, '0');
|
||||
const hashPrefix = computeContentHash(content).slice(0, 8);
|
||||
return `inbox/${y}-${m}-${d}-${hashPrefix}`;
|
||||
return `${slugPrefixForType(type)}/${y}-${m}-${d}-${hashPrefix}`;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -245,6 +266,22 @@ function deriveTitle(rawBody: string): string {
|
||||
* stamps a fresh frontmatter block, and if the body doesn't already look
|
||||
* like markdown (no `#` heading), wraps it under a `# {title}` heading.
|
||||
*/
|
||||
// v0.42.x — Life Chronicle (#2390): assemble the `event:` frontmatter block
|
||||
// from the --who/--what/--where/--kind/--depth flags (only for --type event).
|
||||
// Returns undefined when no event flags are set so non-event captures are
|
||||
// untouched.
|
||||
function buildEventBlock(opts: RunOpts): Record<string, unknown> | undefined {
|
||||
if (opts.type !== 'event') return undefined;
|
||||
const who = opts.who ? opts.who.split(',').map((s) => s.trim()).filter(Boolean) : [];
|
||||
const block: Record<string, unknown> = {};
|
||||
if (opts.what) block.what = opts.what;
|
||||
if (who.length) block.who = who;
|
||||
if (opts.where) block.where = opts.where;
|
||||
if (opts.kind) block.kind = opts.kind;
|
||||
if (opts.depth) block.depth = opts.depth;
|
||||
return Object.keys(block).length ? block : undefined;
|
||||
}
|
||||
|
||||
export function mergeCaptureFrontmatter(rawBody: string, opts: RunOpts): string {
|
||||
const nowIso = new Date().toISOString();
|
||||
// Detect frontmatter: leading `---\n` or `---\r\n`, tolerating leading BOM/whitespace.
|
||||
@@ -263,6 +300,8 @@ export function mergeCaptureFrontmatter(rawBody: string, opts: RunOpts): string
|
||||
captured_via: opts.source ?? 'capture-cli',
|
||||
captured_at: nowIso,
|
||||
};
|
||||
const ev = buildEventBlock(opts);
|
||||
if (ev) fm.event = ev;
|
||||
const looksMarkdown = /^#{1,6}\s/.test(rawBody.trimStart());
|
||||
const body = looksMarkdown ? rawBody : `# ${title}\n\n${rawBody}`;
|
||||
return matter.stringify(body, fm);
|
||||
@@ -290,6 +329,11 @@ export function mergeCaptureFrontmatter(rawBody: string, opts: RunOpts): string
|
||||
captured_via: userFm.captured_via ?? opts.source ?? 'capture-cli',
|
||||
captured_at: userFm.captured_at ?? nowIso,
|
||||
};
|
||||
// v0.42.x — merge the event block (user-declared keys win per-key).
|
||||
const ev = buildEventBlock(opts);
|
||||
if (ev || userFm.event) {
|
||||
merged.event = { ...(ev ?? {}), ...((userFm.event as Record<string, unknown>) ?? {}) };
|
||||
}
|
||||
return matter.stringify(parsed.content, merged);
|
||||
}
|
||||
|
||||
@@ -439,7 +483,7 @@ export async function runCapture(engine: BrainEngine | null, args: string[]): Pr
|
||||
// The daemon's 24h LRU dedup keys on this hash; identical captures must
|
||||
// produce identical hashes. The DB content_hash (importFromContent at
|
||||
// src/core/import-file.ts) gets the same treatment in Phase 3d.
|
||||
const slug = parsed.slug ?? defaultSlug(normalizedBody);
|
||||
const slug = parsed.slug ?? defaultSlug(normalizedBody, new Date(), parsed.type);
|
||||
const fullContent = buildContent(rawBody, parsed);
|
||||
const capturedAt = new Date().toISOString();
|
||||
const contentHash = computeContentHash(normalizedBody);
|
||||
|
||||
@@ -538,6 +538,33 @@ export async function doctorReportRemote(engine: BrainEngine): Promise<DoctorRep
|
||||
checks.push({ name: 'timeline_dedup_index', status: 'warn', message: 'Could not check idx_timeline_dedup shape' });
|
||||
}
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390): orphaned event projections. Reads already
|
||||
// hide projections whose event page is soft-deleted (read-time correctness);
|
||||
// this always-run probe surfaces the cleanup backlog. Keyed off the real
|
||||
// schema (event_page_id), NOT a migration verify-hook, per
|
||||
// migration-verify-hook-never-runs-on-stamped-brains.
|
||||
try {
|
||||
const orphans = await engine.executeRaw<{ n: number }>(
|
||||
`SELECT count(*)::int AS n FROM timeline_entries te
|
||||
JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE te.event_page_id IS NOT NULL AND ep.deleted_at IS NOT NULL`,
|
||||
);
|
||||
const n = Number(orphans[0]?.n ?? 0);
|
||||
checks.push(
|
||||
n === 0
|
||||
? { name: 'chronicle_projection_health', status: 'ok', message: 'No orphaned event projections' }
|
||||
: {
|
||||
name: 'chronicle_projection_health',
|
||||
status: 'warn',
|
||||
message:
|
||||
`${n} timeline projection(s) point to soft-deleted event pages ` +
|
||||
'(hidden at read time; clean up with `gbrain integrity auto`).',
|
||||
},
|
||||
);
|
||||
} catch {
|
||||
checks.push({ name: 'chronicle_projection_health', status: 'ok', message: 'no event projections yet' });
|
||||
}
|
||||
|
||||
// 3. Brain score
|
||||
try {
|
||||
const health = await engine.getHealth();
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
// v0.42.x — Life Chronicle (#2390) `gbrain eval chronicle` (Phase A.9).
|
||||
// Deterministic, brings its own in-memory PGLite (no DB, no gateway), so the
|
||||
// CI fixture gate runs anywhere. Exit 0 only on a perfect score.
|
||||
import { PGLiteEngine } from '../core/pglite-engine.ts';
|
||||
import { runChronicleEval } from '../eval/chronicle/harness.ts';
|
||||
|
||||
const HELP = `Usage: gbrain eval chronicle [--json]
|
||||
|
||||
Deterministic Life Chronicle (#2390) feature eval. Builds a synthetic month
|
||||
corpus with a known gold chronology + a planted ontology supersession + a
|
||||
planted conflict, then scores the chronicle layer on: day reconstruction
|
||||
(intra-day order), last-seen exact date, ontology supersession + --asof
|
||||
time-travel, contradiction surfacing, and source isolation.
|
||||
|
||||
Exit code 0 iff every task passes.
|
||||
`;
|
||||
|
||||
export async function runEvalChronicle(args: string[]): Promise<number> {
|
||||
if (args.includes('--help') || args.includes('-h')) {
|
||||
process.stdout.write(HELP);
|
||||
return 0;
|
||||
}
|
||||
const json = args.includes('--json');
|
||||
const engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
try {
|
||||
const result = await runChronicleEval(engine);
|
||||
if (json) {
|
||||
process.stdout.write(`${JSON.stringify(result, null, 2)}\n`);
|
||||
} else {
|
||||
process.stderr.write(
|
||||
`[eval chronicle] ${result.passed}/${result.total} tasks passed ` +
|
||||
`(score ${(result.score * 100).toFixed(0)}%)\n`,
|
||||
);
|
||||
for (const t of result.tasks) {
|
||||
process.stderr.write(` ${t.passed ? 'PASS' : 'FAIL'} ${t.id} — ${t.detail}\n`);
|
||||
}
|
||||
}
|
||||
return result.score === 1 ? 0 : 1;
|
||||
} finally {
|
||||
await engine.disconnect();
|
||||
}
|
||||
}
|
||||
@@ -1504,6 +1504,25 @@ export async function registerBuiltinHandlers(
|
||||
return result;
|
||||
});
|
||||
|
||||
// v0.42.x (#2390) — Life Chronicle event extraction. NOT protected (bounded
|
||||
// LLM spend per page; no shell). Enqueued by the put_page chronicle backstop
|
||||
// and by `gbrain chronicle backfill`. Idempotent (content-addressed event
|
||||
// slugs + projection upsert), so a retry re-runs to the same state.
|
||||
worker.register('chronicle_extract', async (job) => {
|
||||
const slug = typeof job.data.slug === 'string' ? job.data.slug : undefined;
|
||||
if (!slug) throw new Error('chronicle_extract job requires data.slug');
|
||||
const sourceId = typeof job.data.sourceId === 'string' ? job.data.sourceId : undefined;
|
||||
const { runChronicleExtract } = await import('../core/chronicle/extract-events.ts');
|
||||
const { chronicleTz } = await import('../core/chronicle/config.ts');
|
||||
const tz = await chronicleTz(engine);
|
||||
return await runChronicleExtract(engine, {
|
||||
slug,
|
||||
sourceId,
|
||||
tz,
|
||||
signal: (job as { signal?: AbortSignal }).signal,
|
||||
});
|
||||
});
|
||||
|
||||
// v0.41.39 (#1700) — enrich. NOT in PROTECTED_JOB_NAMES: per-call cost is
|
||||
// bounded by data.maxCostUsd (default DEFAULT_MAX_COST_USD) and the handler
|
||||
// re-creates the BudgetTracker in its own process. BudgetExhausted is caught
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
// v0.42.x — Life Chronicle (#2390) advisor collector (Phase A.7).
|
||||
// Brain-state (not workspace-dependent), so it runs over MCP too. Two signals:
|
||||
// - unresolved ontology conflicts (genuine disagreement, not supersession)
|
||||
// - recent meetings not yet swept into the timeline (coverage gap)
|
||||
// Advisory display only (no dispatch_id) — the user runs the shown command.
|
||||
import type { AdvisorCollector, AdvisorContext, AdvisorFinding } from './types.ts';
|
||||
|
||||
export const collectChronicle: AdvisorCollector = {
|
||||
id: 'chronicle',
|
||||
collect: async (ctx: AdvisorContext): Promise<AdvisorFinding[]> => {
|
||||
const findings: AdvisorFinding[] = [];
|
||||
|
||||
// 1. Unresolved ontology conflicts.
|
||||
try {
|
||||
const conflicts = await ctx.engine.findOntologyConflicts({ minConfidence: 0.5 });
|
||||
if (conflicts.length > 0) {
|
||||
findings.push({
|
||||
id: 'ontology_conflicts',
|
||||
severity: 'warn',
|
||||
title: `${conflicts.length} entity dimension(s) have conflicting current values`,
|
||||
detail: conflicts.slice(0, 5).map((c) => `${c.entity_slug}.${c.dimension}`).join(', '),
|
||||
fix: { command_argv: ['gbrain', 'ontology-contradictions'] },
|
||||
collector: 'chronicle',
|
||||
ask_user: false,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Ontology columns may be absent on a brain that hasn't migrated; ignore.
|
||||
}
|
||||
|
||||
// 2. Recent meetings not yet in the timeline (coverage gap).
|
||||
try {
|
||||
const rows = await ctx.engine.executeRaw<{ n: number }>(
|
||||
`SELECT count(*)::int AS n FROM pages p
|
||||
WHERE p.type IN ('meeting','conversation','calendar-event') AND p.deleted_at IS NULL
|
||||
AND p.updated_at > now() - interval '30 days'
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM timeline_entries te
|
||||
WHERE te.page_id = p.id AND te.event_page_id IS NOT NULL
|
||||
)`,
|
||||
);
|
||||
const gap = Number(rows[0]?.n ?? 0);
|
||||
if (gap > 0) {
|
||||
findings.push({
|
||||
id: 'chronicle_coverage_gap',
|
||||
severity: 'info',
|
||||
title: `${gap} recent meeting(s) aren't in the timeline yet`,
|
||||
detail: 'Sweep them into events with `gbrain chronicle-backfill`, or enable auto_chronicle.',
|
||||
fix: { command_argv: ['gbrain', 'chronicle-backfill'] },
|
||||
collector: 'chronicle',
|
||||
ask_user: true,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// timeline_entries.event_page_id may be absent pre-migration; ignore.
|
||||
}
|
||||
|
||||
return findings;
|
||||
},
|
||||
};
|
||||
@@ -16,6 +16,7 @@ import { collectUsageShape } from './collect-usage-shape.ts';
|
||||
import { collectSetupSmells } from './collect-setup-smells.ts';
|
||||
import { collectUninstalledBrainPack } from './collect-uninstalled-brain-pack.ts';
|
||||
import { collectUninstalledBundled } from './collect-uninstalled-bundled.ts';
|
||||
import { collectChronicle } from './collect-chronicle.ts';
|
||||
|
||||
/** Deterministic v1 collector order (also the secondary sort key for ranking). */
|
||||
export const COLLECTORS: AdvisorCollector[] = [
|
||||
@@ -27,6 +28,7 @@ export const COLLECTORS: AdvisorCollector[] = [
|
||||
collectSetupSmells,
|
||||
collectUninstalledBrainPack,
|
||||
collectUninstalledBundled,
|
||||
collectChronicle,
|
||||
];
|
||||
|
||||
const SEV_RANK: Record<AdvisorSeverity, number> = { critical: 0, warn: 1, info: 2 };
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
// v0.42.x — Life Chronicle (#2390) put_page chronicle backstop (Phase A.3).
|
||||
// Deterministic + fire-and-forget: it ONLY checks eligibility + the auto_chronicle
|
||||
// flag and enqueues a `chronicle_extract` minion job. The LLM judgment runs off
|
||||
// the write path in the job handler. Mirrors facts/backstop.ts (queue mode).
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { isChronicleEligible } from './eligibility.ts';
|
||||
import { isAutoChronicleEnabled } from './config.ts';
|
||||
|
||||
export interface ChronicleBackstopResult {
|
||||
enqueued: boolean;
|
||||
skipped?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Eligibility + enqueue. The CALLER is responsible for the trust gate and for
|
||||
* only invoking on a real import (status==='imported') — see the put_page hook,
|
||||
* which skips on remote/untrusted and on skipped/unchanged rewrites so an edit
|
||||
* that changes nothing never re-enqueues.
|
||||
*/
|
||||
export async function runChronicleBackstop(
|
||||
page: { slug: string; type: string; compiled_truth?: string; frontmatter?: Record<string, unknown> },
|
||||
ctx: { engine: BrainEngine; sourceId: string },
|
||||
): Promise<ChronicleBackstopResult> {
|
||||
const dreamGenerated = page.frontmatter?.dream_generated === true;
|
||||
const elig = isChronicleEligible({
|
||||
type: page.type, slug: page.slug, body: page.compiled_truth, dreamGenerated,
|
||||
});
|
||||
if (!elig.ok) return { enqueued: false, skipped: elig.reason };
|
||||
if (!(await isAutoChronicleEnabled(ctx.engine))) return { enqueued: false, skipped: 'auto_chronicle_off' };
|
||||
try {
|
||||
const { MinionQueue } = await import('../minions/queue.ts');
|
||||
const queue = new MinionQueue(ctx.engine);
|
||||
await queue.add('chronicle_extract', { slug: page.slug, sourceId: ctx.sourceId });
|
||||
return { enqueued: true };
|
||||
} catch {
|
||||
return { enqueued: false, skipped: 'enqueue_error' };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
// v0.42.x — Life Chronicle (#2390) config flags.
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
|
||||
/**
|
||||
* Auto-emit is OFF by default (plan D5.5: spend posture — extraction spends LLM
|
||||
* tokens per eligible write). Enable with `gbrain config set auto_chronicle true`.
|
||||
* The eval-gated default-flip is the headline fast-follow (TODO T8).
|
||||
*/
|
||||
export async function isAutoChronicleEnabled(engine: BrainEngine): Promise<boolean> {
|
||||
const val = await engine.getConfig('auto_chronicle');
|
||||
if (val == null) return false; // default OFF
|
||||
return ['true', '1', 'yes', 'on'].includes(val.trim().toLowerCase());
|
||||
}
|
||||
|
||||
/** Pinned timezone for the when→date projection cast (plan: default UTC). */
|
||||
export async function chronicleTz(engine: BrainEngine): Promise<string> {
|
||||
const val = await engine.getConfig('chronicle.tz');
|
||||
return (val && val.trim()) || 'UTC';
|
||||
}
|
||||
@@ -0,0 +1,34 @@
|
||||
// v0.42.x — Life Chronicle (#2390) chronicle-extract eligibility.
|
||||
// Mirrors src/core/facts/eligibility.ts. A page is chronicle-eligible (auto-
|
||||
// emits timeline events) when it is conversation-shape, NOT dream-generated,
|
||||
// and NOT a diary/event page. Diary interiority is NEVER mined into events
|
||||
// (privacy/consent — plan D5.4); event pages are already the output (anti-loop).
|
||||
import type { PageType } from '../types.ts';
|
||||
|
||||
export type ChronicleEligibility = { ok: true } | { ok: false; reason: string };
|
||||
|
||||
const ELIGIBLE_TYPES: PageType[] = ['meeting', 'conversation', 'calendar-event'];
|
||||
// Directory rescue: a meetings/… page that frontmatter-typed itself 'note' is
|
||||
// still conversation-shape. life/diary excluded explicitly below.
|
||||
const RESCUE_SLUG_PREFIXES = ['meetings/', 'conversations/', 'cal/', 'calendar/'] as const;
|
||||
const MIN_BODY_CHARS = 80;
|
||||
|
||||
export function isChronicleEligible(input: {
|
||||
type: PageType;
|
||||
slug: string;
|
||||
body?: string;
|
||||
dreamGenerated?: boolean;
|
||||
}): ChronicleEligibility {
|
||||
const { type, slug } = input;
|
||||
if (input.dreamGenerated === true) return { ok: false, reason: 'dream_generated' };
|
||||
// Diary: never extract events from private interiority. Event: anti-loop.
|
||||
if (type === 'diary' || slug.startsWith('life/diary/')) return { ok: false, reason: 'diary_excluded' };
|
||||
if (type === 'event' || slug.startsWith('life/events/')) return { ok: false, reason: 'event_self' };
|
||||
if (slug.startsWith('wiki/agents/')) return { ok: false, reason: 'subagent_scratch' };
|
||||
const bodyOk = (input.body?.length ?? MIN_BODY_CHARS) >= MIN_BODY_CHARS;
|
||||
if (!bodyOk) return { ok: false, reason: 'too_short' };
|
||||
const typeOk = ELIGIBLE_TYPES.includes(type);
|
||||
const slugOk = RESCUE_SLUG_PREFIXES.some((p) => slug.startsWith(p));
|
||||
if (!typeOk && !slugOk) return { ok: false, reason: `kind:${type}` };
|
||||
return { ok: true };
|
||||
}
|
||||
@@ -0,0 +1,214 @@
|
||||
// v0.42.x — Life Chronicle (#2390) event extractor (Phase A.3).
|
||||
//
|
||||
// Pipeline (mirrors facts/extract + facts/backstop, but emits EVENT pages +
|
||||
// timeline projections instead of facts):
|
||||
// deterministic when/who → judge (LLM, injectable) → PARSE BARRIER
|
||||
// → write event pages (content-addressed, idempotent) → project to timeline
|
||||
//
|
||||
// The judge is injectable so the deterministic write path is testable without a
|
||||
// real gateway. The default judge calls the chat gateway; when no gateway is
|
||||
// configured it returns zero events (auto-emit is a no-op, never an error).
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { computeContentHash } from '../ingestion/types.ts';
|
||||
|
||||
export interface ChronicleEventProposal {
|
||||
when: string; // ISO datetime or YYYY-MM-DD
|
||||
who: string[]; // entity slugs / names
|
||||
what: string; // one-clause summary
|
||||
where?: string | null;
|
||||
kind: string; // meeting|call|commitment|decision|… (open vocab)
|
||||
}
|
||||
export interface ChronicleJudgeInput {
|
||||
slug: string;
|
||||
type: string;
|
||||
title: string;
|
||||
body: string;
|
||||
effectiveDate: string | null; // depth page effective_date (deterministic when)
|
||||
attendees: string[]; // deterministic who from frontmatter
|
||||
}
|
||||
export interface ChronicleJudgeResult { events: ChronicleEventProposal[] }
|
||||
export type ChronicleJudge = (input: ChronicleJudgeInput) => Promise<ChronicleJudgeResult>;
|
||||
|
||||
export interface ChronicleExtractResult {
|
||||
slug: string;
|
||||
status: 'extracted' | 'no_events' | 'skipped';
|
||||
events_written: number;
|
||||
reason?: string;
|
||||
}
|
||||
|
||||
const KIND_VOCAB = new Set([
|
||||
'meeting', 'call', 'meal', 'solo', 'travel', 'work',
|
||||
'commitment', 'decision', 'intro', 'conflict', 'milestone', 'event',
|
||||
]);
|
||||
|
||||
function normalizeKind(k: string): string {
|
||||
const n = (k || '').trim().toLowerCase();
|
||||
return KIND_VOCAB.has(n) ? n : 'event';
|
||||
}
|
||||
|
||||
/** Parse a when string to a Date, or null when unparseable (for effective_date). */
|
||||
function safeDate(s: string): Date | null {
|
||||
const d = new Date(s);
|
||||
return Number.isNaN(d.getTime()) ? null : d;
|
||||
}
|
||||
|
||||
/** Resolve a when value to a stable YYYY-MM-DD at the pinned timezone. */
|
||||
export function isoDay(when: string, tz: string): string {
|
||||
if (/^\d{4}-\d{2}-\d{2}$/.test(when)) return when;
|
||||
const d = new Date(when);
|
||||
if (Number.isNaN(d.getTime())) return when.slice(0, 10);
|
||||
if (tz === 'UTC') return d.toISOString().slice(0, 10);
|
||||
try {
|
||||
return new Intl.DateTimeFormat('en-CA', {
|
||||
timeZone: tz, year: 'numeric', month: '2-digit', day: '2-digit',
|
||||
}).format(d);
|
||||
} catch {
|
||||
return d.toISOString().slice(0, 10);
|
||||
}
|
||||
}
|
||||
|
||||
/** PARSE BARRIER: a proposal must fully validate before ANY DB write. */
|
||||
export function isValidProposal(e: unknown): e is ChronicleEventProposal {
|
||||
if (!e || typeof e !== 'object') return false;
|
||||
const o = e as Record<string, unknown>;
|
||||
return (
|
||||
typeof o.when === 'string' && o.when.length >= 4 &&
|
||||
// Must be a REAL parseable date — otherwise isoDay()/::date would write a
|
||||
// garbage event page and then throw on the projection cast (partial write).
|
||||
!Number.isNaN(new Date(o.when).getTime()) &&
|
||||
typeof o.what === 'string' && o.what.trim().length > 0 &&
|
||||
Array.isArray(o.who) && o.who.every((w) => typeof w === 'string') &&
|
||||
typeof o.kind === 'string'
|
||||
);
|
||||
}
|
||||
|
||||
function collectAttendees(fm: Record<string, unknown>): string[] {
|
||||
const out = new Set<string>();
|
||||
for (const key of ['attendees', 'people', 'who']) {
|
||||
const v = fm[key];
|
||||
if (Array.isArray(v)) for (const x of v) if (typeof x === 'string' && x.trim()) out.add(x.trim());
|
||||
}
|
||||
return [...out];
|
||||
}
|
||||
|
||||
/**
|
||||
* Run the chronicle extractor for one depth page. Idempotent: event slugs are
|
||||
* content-addressed (re-run upserts the same pages) and the projection upserts
|
||||
* on (event_page_id, date). A crash between writes re-runs to the same state.
|
||||
*/
|
||||
export async function runChronicleExtract(
|
||||
engine: BrainEngine,
|
||||
opts: { slug: string; sourceId?: string; judge?: ChronicleJudge; tz?: string; signal?: AbortSignal },
|
||||
): Promise<ChronicleExtractResult> {
|
||||
const sourceId = opts.sourceId ?? 'default';
|
||||
const tz = opts.tz ?? 'UTC';
|
||||
const page = await engine.getPage(opts.slug, { sourceId });
|
||||
if (!page) return { slug: opts.slug, status: 'skipped', events_written: 0, reason: 'page_not_found' };
|
||||
|
||||
const fm = (page.frontmatter ?? {}) as Record<string, unknown>;
|
||||
const edRaw = page.effective_date as unknown;
|
||||
const effectiveDate: string | null =
|
||||
edRaw instanceof Date ? edRaw.toISOString()
|
||||
: typeof edRaw === 'string' && edRaw ? edRaw
|
||||
: typeof fm.date === 'string' ? fm.date
|
||||
: null;
|
||||
const attendees = collectAttendees(fm);
|
||||
|
||||
const judge = opts.judge ?? defaultJudge(engine);
|
||||
let result: ChronicleJudgeResult;
|
||||
try {
|
||||
result = await judge({
|
||||
slug: opts.slug, type: page.type, title: page.title,
|
||||
body: page.compiled_truth ?? '', effectiveDate, attendees,
|
||||
});
|
||||
} catch (e) {
|
||||
if ((e as Error)?.name === 'AbortError') throw e;
|
||||
return { slug: opts.slug, status: 'skipped', events_written: 0, reason: 'judge_error' };
|
||||
}
|
||||
|
||||
const proposals = Array.isArray(result?.events) ? result.events : [];
|
||||
if (proposals.length === 0) return { slug: opts.slug, status: 'no_events', events_written: 0 };
|
||||
// PARSE BARRIER — reject the WHOLE batch on any malformed proposal; no partial writes.
|
||||
if (!proposals.every(isValidProposal)) {
|
||||
return { slug: opts.slug, status: 'skipped', events_written: 0, reason: 'malformed_proposal' };
|
||||
}
|
||||
|
||||
let written = 0;
|
||||
for (const ev of proposals) {
|
||||
if (opts.signal?.aborted) { const e = new Error('aborted'); e.name = 'AbortError'; throw e; }
|
||||
const who = ev.who.length ? ev.who : attendees;
|
||||
const when = ev.when || effectiveDate || isoDay(new Date(0).toISOString(), tz);
|
||||
const day = isoDay(when, tz);
|
||||
const hash = computeContentHash(`${who.join(',')}|${ev.what}|${opts.slug}`).slice(0, 8);
|
||||
const eventSlug = `life/events/${day}-${hash}`;
|
||||
await engine.putPage(eventSlug, {
|
||||
type: 'event',
|
||||
title: ev.what.slice(0, 120),
|
||||
compiled_truth: `${ev.what} — see [[${opts.slug}]].`,
|
||||
frontmatter: {
|
||||
type: 'event',
|
||||
event: {
|
||||
when, who, what: ev.what, where: ev.where ?? null,
|
||||
kind: normalizeKind(ev.kind), depth: opts.slug,
|
||||
},
|
||||
captured_via: 'life-chronicle:auto',
|
||||
},
|
||||
effective_date: safeDate(when),
|
||||
}, { sourceId });
|
||||
await engine.upsertEventProjection({
|
||||
depthSlug: opts.slug, eventSlug, date: day, summary: ev.what, sourceId,
|
||||
});
|
||||
written++;
|
||||
}
|
||||
return { slug: opts.slug, status: 'extracted', events_written: written };
|
||||
}
|
||||
|
||||
const JUDGE_SYSTEM = `You segment a meeting/transcript page into discrete timeline EVENTS.
|
||||
Return ONLY a JSON array. Each element: {"when": ISO datetime or YYYY-MM-DD, "who": [entity slugs/names], "what": one-clause summary, "where": optional string, "kind": one of meeting|call|meal|solo|travel|work|commitment|decision|intro|conflict|milestone|event}.
|
||||
Prefer the page's known date for "when" when the text gives no explicit time. Use the provided attendee slugs for "who" when the text does not name participants. No prose, no markdown — just the JSON array.`;
|
||||
|
||||
function defaultJudge(engine: BrainEngine): ChronicleJudge {
|
||||
return async (input) => {
|
||||
const { isAvailable, chat } = await import('../ai/gateway.ts');
|
||||
if (!isAvailable('chat')) return { events: [] };
|
||||
const body = (input.body || '').slice(0, 12_000);
|
||||
let text: string;
|
||||
try {
|
||||
const res = await chat({
|
||||
system: JUDGE_SYSTEM,
|
||||
messages: [{
|
||||
role: 'user',
|
||||
content:
|
||||
`<page slug="${input.slug}" type="${input.type}" date="${input.effectiveDate ?? ''}">\n` +
|
||||
`${input.title}\n\n${body}\n</page>\n\n` +
|
||||
`Known attendees: ${input.attendees.slice(0, 10).join(', ') || '(none)'}.\nExtract the events.`,
|
||||
}],
|
||||
maxTokens: 1500,
|
||||
});
|
||||
if (res.stopReason === 'refusal' || res.stopReason === 'content_filter') return { events: [] };
|
||||
text = res.text;
|
||||
} catch (err) {
|
||||
if ((err as Error)?.name === 'AbortError') throw err;
|
||||
return { events: [] };
|
||||
}
|
||||
const parsed = parseJudgeJson(text);
|
||||
return { events: parsed };
|
||||
};
|
||||
}
|
||||
|
||||
/** Tolerant JSON-array extraction from a model response (mirrors facts parser). */
|
||||
export function parseJudgeJson(text: string): ChronicleEventProposal[] {
|
||||
if (!text) return [];
|
||||
let s = text.trim();
|
||||
const fence = s.match(/```(?:json)?\s*([\s\S]*?)```/i);
|
||||
if (fence) s = fence[1].trim();
|
||||
const start = s.indexOf('[');
|
||||
const end = s.lastIndexOf(']');
|
||||
if (start === -1 || end === -1 || end < start) return [];
|
||||
try {
|
||||
const arr = JSON.parse(s.slice(start, end + 1));
|
||||
return Array.isArray(arr) ? arr : [];
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
// v0.42.x — Life Chronicle (#2390). Shared last-seen finalizer so both engines
|
||||
// compute `days_ago` identically (engine parity is on the result, not the SQL).
|
||||
import type { LastSeenResult } from '../types.ts';
|
||||
|
||||
/**
|
||||
* Build a LastSeenResult from a resolved last-seen date. `lastDate` is a bare
|
||||
* 'YYYY-MM-DD' (or null when the entity was never seen). `asof` (also
|
||||
* 'YYYY-MM-DD') pins the reference day for deterministic tests; defaults to now.
|
||||
* Both are interpreted at UTC midnight so the day-difference is timezone-stable.
|
||||
*/
|
||||
export function finalizeLastSeen(
|
||||
entitySlug: string,
|
||||
lastDate: string | null,
|
||||
lastEventSlug: string | null,
|
||||
asof?: string,
|
||||
): LastSeenResult {
|
||||
let days_ago: number | null = null;
|
||||
if (lastDate) {
|
||||
const base = asof ? new Date(`${asof}T00:00:00Z`) : new Date();
|
||||
const then = new Date(`${lastDate}T00:00:00Z`);
|
||||
days_ago = Math.max(0, Math.floor((base.getTime() - then.getTime()) / 86_400_000));
|
||||
}
|
||||
return {
|
||||
entity_slug: entitySlug,
|
||||
last_date: lastDate,
|
||||
last_event_slug: lastEventSlug,
|
||||
days_ago,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,22 @@
|
||||
// v0.42.x — Life Chronicle (#2390) narrative rendering (Phase A.6 delight).
|
||||
// Pure function: turn timeline projection rows into a readable prose day-by-day
|
||||
// summary (the `--narrative` flag on `gbrain day`). No SQL, no I/O.
|
||||
import type { ChronicleTimelineRow } from '../types.ts';
|
||||
|
||||
export function renderTimelineNarrative(rows: ChronicleTimelineRow[]): string {
|
||||
if (!rows.length) return 'No events in this window.';
|
||||
const byDate = new Map<string, ChronicleTimelineRow[]>();
|
||||
for (const r of rows) {
|
||||
const list = byDate.get(r.date) ?? [];
|
||||
list.push(r);
|
||||
byDate.set(r.date, list);
|
||||
}
|
||||
const lines: string[] = [];
|
||||
for (const [date, rs] of byDate) {
|
||||
const items = rs
|
||||
.map((r) => (r.kind ? `${r.summary} (${r.kind})` : r.summary))
|
||||
.join('; ');
|
||||
lines.push(`${date} — ${rs.length} event${rs.length === 1 ? '' : 's'}: ${items}`);
|
||||
}
|
||||
return lines.join('\n');
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
// v0.42.x — Life Chronicle (#2390) ontology helpers (Phase B.10).
|
||||
// The ontology rides the `facts` table; these are the deterministic bits the
|
||||
// engine methods (mergeOntologyFact / getOntology / …) lean on.
|
||||
import { computeContentHash } from '../ingestion/types.ts';
|
||||
|
||||
/**
|
||||
* Deterministic dedup key for an ontology value. Normalized (trim + lowercase +
|
||||
* NFKC via computeContentHash) so "Advisor" and "advisor " dedup, and a crash
|
||||
* retry produces the same key — NO timestamp, so retries are idempotent.
|
||||
*/
|
||||
export function valueHash(value: string): string {
|
||||
return computeContentHash(value.trim().toLowerCase()).slice(0, 16);
|
||||
}
|
||||
|
||||
// Seed lexicon: canonicalize common dimension-name drift at WRITE time so the
|
||||
// open world doesn't fragment into role/job_role/position/title for one concept.
|
||||
const DIMENSION_ALIASES: Record<string, string> = {
|
||||
job_role: 'role', position: 'role', title: 'role', job_title: 'role',
|
||||
relationship: 'relation', rel: 'relation',
|
||||
risk_appetite: 'risk_tolerance',
|
||||
comms_style: 'communication_style', communication: 'communication_style',
|
||||
employer_name: 'employer', company: 'employer', org: 'employer',
|
||||
};
|
||||
|
||||
const KNOWN_DIMENSIONS = new Set<string>([
|
||||
'role', 'relation', 'risk_tolerance', 'decision_style', 'communication_style',
|
||||
'reliability', 'expertise', 'affect', 'location', 'employer',
|
||||
]);
|
||||
|
||||
/** Normalize a dimension name (case/space/hyphen + alias lexicon). */
|
||||
export function normalizeDimension(name: string): string {
|
||||
const n = name.trim().toLowerCase().replace(/[\s-]+/g, '_');
|
||||
return DIMENSION_ALIASES[n] ?? n;
|
||||
}
|
||||
|
||||
/**
|
||||
* A dimension is "novel" (→ quarantine) when it's neither a seed dimension nor
|
||||
* an alias of one. Quarantined observations are stored but excluded from
|
||||
* current-value resolution + context loading until confirmed, so an LLM
|
||||
* hallucination can't silently shape agent context.
|
||||
*/
|
||||
export function isNovelDimension(normalized: string): boolean {
|
||||
return !KNOWN_DIMENSIONS.has(normalized);
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// v0.42.x — Life Chronicle (#2390) agent-context loader (Phase B.12).
|
||||
//
|
||||
// Zero-LLM composition over the chronicle reads: hand an agent the recent
|
||||
// timeline + the validity-resolved ontology for the entities in play, so it
|
||||
// orients before acting (the exact thing missing when chronology gets fumbled).
|
||||
// Pure composition of engine.getSince + engine.getOntology — no new SQL.
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import type { ChronicleTimelineRow, OntologyValue } from '../types.ts';
|
||||
|
||||
export interface ChronicleContextOpts {
|
||||
/** Lookback window in days for the recent timeline (default 7). */
|
||||
days?: number;
|
||||
/** Entity slugs to resolve current ontology for (e.g. the people in the session). */
|
||||
entities?: string[];
|
||||
/** Cap on timeline rows (default 50). */
|
||||
limit?: number;
|
||||
/** Untrusted caller → diary-sourced ontology is redacted. */
|
||||
remote?: boolean;
|
||||
sourceId?: string;
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
export interface ChronicleContext {
|
||||
since: string; // YYYY-MM-DD lower bound
|
||||
recent_timeline: ChronicleTimelineRow[];
|
||||
ontologies: Record<string, OntologyValue[]>; // entity slug → current resolved values
|
||||
}
|
||||
|
||||
function daysAgoIso(days: number): string {
|
||||
return new Date(Date.now() - Math.max(0, days) * 86_400_000).toISOString().slice(0, 10);
|
||||
}
|
||||
|
||||
export async function loadChronicleContext(
|
||||
engine: BrainEngine,
|
||||
opts: ChronicleContextOpts = {},
|
||||
): Promise<ChronicleContext> {
|
||||
const days = typeof opts.days === 'number' && opts.days > 0 ? opts.days : 7;
|
||||
const since = daysAgoIso(days);
|
||||
const scope = { sourceId: opts.sourceId, sourceIds: opts.sourceIds };
|
||||
|
||||
const recent_timeline = await engine.getSince(since, { ...scope, limit: opts.limit ?? 50 });
|
||||
|
||||
const ontologies: Record<string, OntologyValue[]> = {};
|
||||
for (const slug of opts.entities ?? []) {
|
||||
let vals = await engine.getOntology(slug, scope);
|
||||
if (opts.remote) vals = vals.filter((v) => !(v.source ?? '').startsWith('life/diary/'));
|
||||
if (vals.length) ontologies[slug] = vals;
|
||||
}
|
||||
return { since, recent_timeline, ontologies };
|
||||
}
|
||||
@@ -58,6 +58,7 @@ export const BRAIN_CHECK_NAMES: ReadonlySet<string> = new Set([
|
||||
'brain_score',
|
||||
'calibration_freshness',
|
||||
'child_table_orphans',
|
||||
'chronicle_projection_health',
|
||||
'content_sanity_audit_recent',
|
||||
'contextual_retrieval_coverage',
|
||||
'contradictions',
|
||||
|
||||
@@ -4,6 +4,9 @@ import type {
|
||||
SearchResult, SearchOpts,
|
||||
Link, GraphNode, GraphPath, RelationalFanoutRow, RelationalFanoutOpts,
|
||||
TimelineEntry, TimelineInput, TimelineOpts,
|
||||
ChronicleTimelineRow, ChronicleTimelineOpts, LastSeenResult,
|
||||
OntologyObservationInput, OntologyMergeResult, OntologyValue, OntologyDimensionStat,
|
||||
OntologyConflict, OntologyReadOpts,
|
||||
RawData,
|
||||
PageVersion,
|
||||
BrainStats, BrainHealth,
|
||||
@@ -1382,6 +1385,42 @@ export interface BrainEngine {
|
||||
addTimelineEntriesBatch(entries: TimelineBatchInput[], opts?: BatchOpts): Promise<number>;
|
||||
getTimeline(slug: string, opts?: TimelineOpts): Promise<TimelineEntry[]>;
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390) timeline reads. All filter the depth page
|
||||
// (and any event page) on deleted_at IS NULL, order by COALESCE(event
|
||||
// effective_date, date), and honor source scope (sourceIds[] > sourceId).
|
||||
/** Events/timeline rows on a given day (or its ISO week when opts.week). */
|
||||
getTimelineForDate(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]>;
|
||||
/** Events/timeline rows on or after `date`, optionally filtered by event.kind. */
|
||||
getSince(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]>;
|
||||
/** "On this day" — events from the same month-day in PRIOR years (default: today). */
|
||||
getOnThisDay(opts?: { date?: string; limit?: number; sourceId?: string; sourceIds?: string[] }): Promise<ChronicleTimelineRow[]>;
|
||||
/** Most recent date an entity appears (its own page or an event's `who`). */
|
||||
getLastSeen(entitySlug: string, opts?: { asof?: string; sourceId?: string; sourceIds?: string[] }): Promise<LastSeenResult>;
|
||||
/**
|
||||
* Upsert the date-index projection row for an event page: page_id = depth
|
||||
* page, event_page_id = event page, keyed (event_page_id, date). Re-extraction
|
||||
* with a changed summary UPDATEs (no duplicate). Returns projected=false when
|
||||
* either slug is missing in the source. Idempotent.
|
||||
*/
|
||||
upsertEventProjection(opts: { depthSlug: string; eventSlug: string; date: string; summary: string; detail?: string; sourceId?: string }): Promise<{ projected: boolean }>;
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390) per-entity ontology (rides `facts`).
|
||||
/**
|
||||
* Record one ontology observation (entity has dimension=value). Idempotent on
|
||||
* the deterministic dedup key (source_id, entity_slug, dimension, value_hash,
|
||||
* source_markdown_slug). A new value forward-supersedes the prior open row
|
||||
* (expired_at + superseded_by); the same value corroborates; a backdated
|
||||
* conflicting value is inserted WITHOUT rewriting the prior (surfaced by
|
||||
* findOntologyConflicts). Never throws on dup — returns action 'noop'.
|
||||
*/
|
||||
mergeOntologyFact(obs: OntologyObservationInput): Promise<OntologyMergeResult>;
|
||||
/** Current resolved ontology for an entity at `asof` (default now). */
|
||||
getOntology(entitySlug: string, opts?: OntologyReadOpts): Promise<OntologyValue[]>;
|
||||
/** Meta-ontology: which dimensions exist across the brain, and how widely. */
|
||||
discoverOntologyDimensions(opts?: { sourceId?: string; sourceIds?: string[] }): Promise<OntologyDimensionStat[]>;
|
||||
/** Dimensions with ≥2 distinct current-open values from ≥2 provenances. */
|
||||
findOntologyConflicts(opts?: { sourceId?: string; sourceIds?: string[]; minConfidence?: number }): Promise<OntologyConflict[]>;
|
||||
|
||||
// Raw data
|
||||
/**
|
||||
* v0.31.8 (D21): `opts.sourceId` source-scopes the page-id lookup. When
|
||||
|
||||
@@ -466,6 +466,9 @@ const GBRAIN_BASE_PATH_PREFIXES: ReadonlyArray<{ prefixes: string[]; type: PageT
|
||||
{ prefixes: ['/cal/', '/calendar/'], type: 'calendar-event' },
|
||||
{ prefixes: ['/notes/', '/note/'], type: 'note' },
|
||||
{ prefixes: ['/meetings/', '/meeting/'], type: 'meeting' },
|
||||
// v0.42.x — Life Chronicle (#2390): timeline events + thought diary.
|
||||
{ prefixes: ['/life/events/'], type: 'event' },
|
||||
{ prefixes: ['/life/diary/'], type: 'diary' },
|
||||
];
|
||||
|
||||
function inferType(filePath?: string): PageType {
|
||||
|
||||
@@ -5439,6 +5439,72 @@ export const MIGRATIONS: Migration[] = [
|
||||
`,
|
||||
},
|
||||
},
|
||||
{
|
||||
version: 121,
|
||||
name: 'timeline_entries_event_page_id',
|
||||
// v0.42.x — Life Chronicle (#2390): the event→timeline projection pointer.
|
||||
// A `type:event` page projects ONE date-index row into timeline_entries
|
||||
// keyed to the depth/meeting page (page_id), with event_page_id pointing at
|
||||
// the event page itself. Additive + idempotent: nullable FK + partial
|
||||
// indexes; legacy rows keep event_page_id NULL so existing behavior is
|
||||
// unchanged. The partial UNIQUE(event_page_id, date) makes re-extraction
|
||||
// with a changed summary an UPDATE (not a duplicate). FK added via a guarded
|
||||
// DO block (mirrors the facts_source_id_fkey pattern) so the ALTER is a
|
||||
// no-op on re-runs. Mirrored in src/schema.sql, src/core/pglite-schema.ts,
|
||||
// and the generated src/core/schema-embedded.ts for fresh installs.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE timeline_entries ADD COLUMN IF NOT EXISTS event_page_id INTEGER;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint
|
||||
WHERE conname = 'timeline_entries_event_page_id_fkey'
|
||||
AND conrelid = 'timeline_entries'::regclass
|
||||
) THEN
|
||||
ALTER TABLE timeline_entries
|
||||
ADD CONSTRAINT timeline_entries_event_page_id_fkey
|
||||
FOREIGN KEY (event_page_id) REFERENCES pages(id) ON DELETE CASCADE;
|
||||
END IF;
|
||||
END $$;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_timeline_event_page
|
||||
ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_event_dedup
|
||||
ON timeline_entries(event_page_id, date) WHERE event_page_id IS NOT NULL;
|
||||
`,
|
||||
},
|
||||
{
|
||||
version: 122,
|
||||
name: 'facts_ontology_dimension',
|
||||
// v0.42.x — Life Chronicle (#2390): the per-entity ontology rides the
|
||||
// existing `facts` table. facts already gives bi-temporal validity
|
||||
// (valid_from/valid_until/expired_at), supersession (superseded_by),
|
||||
// remote redaction (visibility), confidence, provenance (source_markdown_slug),
|
||||
// embedding, and corroboration (consolidated_into). The ONLY genuinely-new
|
||||
// concept is a typed `dimension` (e.g. role, risk_tolerance) carrying a
|
||||
// resolved `value` + a deterministic `value_hash` dedup key, plus a
|
||||
// `dim_status` for quarantining novel/LLM-proposed dimensions. Plain facts
|
||||
// keep dimension NULL → unchanged behavior. The partial UNIQUE is
|
||||
// deterministic (no timestamp) so a crash-retry is idempotent. Additive;
|
||||
// facts is migration-created (absent from static schema), so this migration
|
||||
// is the single source for fresh + migrated brains.
|
||||
idempotent: true,
|
||||
sql: `
|
||||
ALTER TABLE facts ADD COLUMN IF NOT EXISTS dimension TEXT;
|
||||
ALTER TABLE facts ADD COLUMN IF NOT EXISTS value TEXT;
|
||||
ALTER TABLE facts ADD COLUMN IF NOT EXISTS value_hash TEXT;
|
||||
ALTER TABLE facts ADD COLUMN IF NOT EXISTS dim_status TEXT;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_facts_dimension
|
||||
ON facts(source_id, entity_slug, dimension, valid_from DESC)
|
||||
WHERE expired_at IS NULL AND dimension IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_facts_ontology_dedup
|
||||
ON facts(source_id, entity_slug, dimension, value_hash, source_markdown_slug)
|
||||
WHERE dimension IS NOT NULL;
|
||||
`,
|
||||
},
|
||||
];
|
||||
|
||||
export const LATEST_VERSION = MIGRATIONS.length > 0
|
||||
|
||||
@@ -24,6 +24,7 @@
|
||||
*/
|
||||
|
||||
const THIRTY_MIN_MS = 30 * 60 * 1000;
|
||||
const TEN_MIN_MS = 10 * 60 * 1000;
|
||||
|
||||
/**
|
||||
* Default wall-clock budget (ms) for long-running handler types. A handler
|
||||
@@ -37,6 +38,10 @@ export const HANDLER_DEFAULT_TIMEOUT_MS: Readonly<Record<string, number>> = {
|
||||
// #2194 fix #3: brain-wide maintenance (embed-all/orphans/purge/…) can run
|
||||
// longer than a single source cycle; give it the same 30-min budget.
|
||||
'autopilot-global-maintenance': THIRTY_MIN_MS,
|
||||
// v0.42.x (#2390) — Life Chronicle: one page = one LLM extraction call + a
|
||||
// few writes. Generous 10-min budget (vs the tight null-default) covers a
|
||||
// slow gateway without the 30-min loop budget.
|
||||
chronicle_extract: TEN_MIN_MS,
|
||||
};
|
||||
|
||||
/**
|
||||
|
||||
@@ -1034,6 +1034,34 @@ const put_page: Operation = {
|
||||
factsQueued = { skipped: 'backstop_error' };
|
||||
}
|
||||
|
||||
// v0.42.x (#2390): Life Chronicle backstop. ONLY on a real import
|
||||
// (status==='imported' — a skipped/unchanged rewrite still carries
|
||||
// parsedPage, so gating on parsedPage alone would re-enqueue forever),
|
||||
// behind the SAME trust gate as auto-link/timeline + the auto_chronicle
|
||||
// flag. Enqueues a chronicle_extract job; never blocks the write.
|
||||
let chronicleQueued: { queued: boolean } | { skipped: string } | undefined;
|
||||
if (result.status !== 'imported') {
|
||||
chronicleQueued = { skipped: 'not_imported' };
|
||||
} else if (ctx.remote !== false && !trustedWorkspace) {
|
||||
chronicleQueued = { skipped: 'remote' };
|
||||
} else if (result.parsedPage) {
|
||||
try {
|
||||
const { runChronicleBackstop } = await import('./chronicle/backstop.ts');
|
||||
const r = await runChronicleBackstop(
|
||||
{
|
||||
slug,
|
||||
type: result.parsedPage.type,
|
||||
compiled_truth: result.parsedPage.compiled_truth,
|
||||
frontmatter: result.parsedPage.frontmatter,
|
||||
},
|
||||
{ engine: ctx.engine, sourceId: ctx.sourceId ?? 'default' },
|
||||
);
|
||||
chronicleQueued = r.enqueued ? { queued: true } : { skipped: r.skipped ?? 'skipped' };
|
||||
} catch {
|
||||
chronicleQueued = { skipped: 'backstop_error' };
|
||||
}
|
||||
}
|
||||
|
||||
// Post-write validator lint (PR 2.5): feature-flag-gated, non-blocking.
|
||||
// When `writer.lint_on_put_page` is enabled, runs the BrainWriter's
|
||||
// validators on the freshly-written page and logs findings to
|
||||
@@ -1063,6 +1091,7 @@ const put_page: Operation = {
|
||||
...(autoTimeline ? { auto_timeline: autoTimeline } : {}),
|
||||
...(writerLint ? { writer_lint: writerLint } : {}),
|
||||
...(factsQueued ? { facts_backstop: factsQueued } : {}),
|
||||
...(chronicleQueued ? { chronicle_backstop: chronicleQueued } : {}),
|
||||
...(writeThrough ? { write_through: writeThrough } : {}),
|
||||
};
|
||||
},
|
||||
@@ -5019,6 +5048,271 @@ const run_skillopt: Operation = {
|
||||
},
|
||||
};
|
||||
|
||||
// ── v0.42.x — Life Chronicle (#2390) timeline read ops ───────────────────
|
||||
// CLI names avoid the existing `timeline` (get_timeline, a page's own timeline):
|
||||
// `gbrain day <date>` / `gbrain since <date>` / `gbrain last-seen <entity>`.
|
||||
// All route through sourceScopeOpts(ctx) so reads honor source isolation.
|
||||
const chronicle_day: Operation = {
|
||||
name: 'chronicle_day',
|
||||
description:
|
||||
'Life Chronicle: events + timeline entries on a given day (or its ISO week when week=true), ' +
|
||||
"ordered chronologically; each row backlinks to its depth page. Distinct from `get_timeline`/" +
|
||||
"`gbrain timeline <slug>`, which shows ONE page's timeline. CLI: `gbrain day <date>`.",
|
||||
scope: 'read',
|
||||
params: {
|
||||
date: { type: 'string', required: true, description: 'Day as YYYY-MM-DD.' },
|
||||
week: { type: 'boolean', description: 'Expand to the ISO week (Mon–Sun) containing the date.' },
|
||||
limit: { type: 'number', description: 'Max rows (default 200).' },
|
||||
narrative: { type: 'boolean', description: 'Also return a prose day-by-day narrative.' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const rows = await ctx.engine.getTimelineForDate(String(p.date), {
|
||||
week: p.week === true,
|
||||
limit: typeof p.limit === 'number' ? p.limit : undefined,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
if (p.narrative === true) {
|
||||
const { renderTimelineNarrative } = await import('./chronicle/narrative.ts');
|
||||
return { date: String(p.date), narrative: renderTimelineNarrative(rows), events: rows };
|
||||
}
|
||||
return rows;
|
||||
},
|
||||
cliHints: { name: 'day', positional: ['date'] },
|
||||
};
|
||||
|
||||
const chronicle_on_this_day: Operation = {
|
||||
name: 'chronicle_on_this_day',
|
||||
description:
|
||||
'Life Chronicle: events from the same calendar day in PRIOR years ("on this day"). ' +
|
||||
'CLI: `gbrain on-this-day [--date YYYY-MM-DD]`.',
|
||||
scope: 'read',
|
||||
params: {
|
||||
date: { type: 'string', description: 'Anchor day YYYY-MM-DD (default today); matches its month-day in prior years.' },
|
||||
limit: { type: 'number', description: 'Max rows (default 50).' },
|
||||
},
|
||||
handler: async (ctx, p) => ctx.engine.getOnThisDay({
|
||||
date: typeof p.date === 'string' ? p.date : undefined,
|
||||
limit: typeof p.limit === 'number' ? p.limit : undefined,
|
||||
...sourceScopeOpts(ctx),
|
||||
}),
|
||||
cliHints: { name: 'on-this-day' },
|
||||
};
|
||||
|
||||
const chronicle_since: Operation = {
|
||||
name: 'chronicle_since',
|
||||
description:
|
||||
'Life Chronicle: events + timeline entries on or after a date, optionally filtered by event kind. ' +
|
||||
'CLI: `gbrain since <date> [--kind commitment]`.',
|
||||
scope: 'read',
|
||||
params: {
|
||||
date: { type: 'string', required: true, description: 'Lower-bound day as YYYY-MM-DD (inclusive).' },
|
||||
kind: { type: 'string', description: "Filter event projections by event.kind (e.g. 'commitment')." },
|
||||
limit: { type: 'number', description: 'Max rows (default 200).' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
return ctx.engine.getSince(String(p.date), {
|
||||
kind: typeof p.kind === 'string' ? p.kind : undefined,
|
||||
limit: typeof p.limit === 'number' ? p.limit : undefined,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
},
|
||||
cliHints: { name: 'since', positional: ['date'] },
|
||||
};
|
||||
|
||||
const chronicle_last_seen: Operation = {
|
||||
name: 'chronicle_last_seen',
|
||||
description:
|
||||
"Life Chronicle: when an entity was last seen — its own timeline rows OR an event's `who`. " +
|
||||
'Returns last_date, the event slug, and days_ago. CLI: `gbrain last-seen <entity-slug>`.',
|
||||
scope: 'read',
|
||||
params: {
|
||||
entity: { type: 'string', required: true, description: 'Entity page slug (e.g. people/sarah-chen).' },
|
||||
asof: { type: 'string', description: 'Reference day YYYY-MM-DD for days_ago (default today).' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
return ctx.engine.getLastSeen(String(p.entity), {
|
||||
asof: typeof p.asof === 'string' ? p.asof : undefined,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
},
|
||||
cliHints: { name: 'last-seen', positional: ['entity'] },
|
||||
};
|
||||
|
||||
const ontology_get: Operation = {
|
||||
name: 'ontology_get',
|
||||
description:
|
||||
"Life Chronicle: the current resolved per-entity ontology (dimension → value) at `asof` " +
|
||||
"(default now), with provenance + confidence + validity. CLI: `gbrain ontology <entity> [--asof YYYY-MM-DD]`.",
|
||||
scope: 'read',
|
||||
params: {
|
||||
entity: { type: 'string', required: true, description: 'Entity page slug (e.g. people/sarah-chen).' },
|
||||
asof: { type: 'string', description: 'Valid-time as-of day YYYY-MM-DD (time-travel; default now).' },
|
||||
min_confidence: { type: 'number', description: 'Only return observations at/above this confidence (0..1).' },
|
||||
include_quarantined: { type: 'boolean', description: 'Include quarantined novel dimensions (default false).' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const rows = await ctx.engine.getOntology(String(p.entity), {
|
||||
asof: typeof p.asof === 'string' ? p.asof : undefined,
|
||||
minConfidence: typeof p.min_confidence === 'number' ? p.min_confidence : undefined,
|
||||
includeQuarantined: p.include_quarantined === true,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
// Remote redaction: never surface diary-sourced ontology to untrusted callers.
|
||||
return ctx.remote !== false ? rows.filter((r) => !(r.source ?? '').startsWith('life/diary/')) : rows;
|
||||
},
|
||||
cliHints: { name: 'ontology', positional: ['entity'] },
|
||||
};
|
||||
|
||||
const ontology_propose: Operation = {
|
||||
name: 'ontology_propose',
|
||||
description:
|
||||
'Life Chronicle: record one ontology observation (entity has dimension=value), sourced + ' +
|
||||
'confidence-weighted + bi-temporal. Idempotent on (entity,dimension,value,source). A new value ' +
|
||||
'supersedes the prior; a backdated conflict is flagged not rewritten. CLI: `gbrain ontology-add <entity> <dimension> <value>`.',
|
||||
scope: 'write',
|
||||
mutating: true,
|
||||
params: {
|
||||
entity: { type: 'string', required: true, description: 'Entity page slug.' },
|
||||
dimension: { type: 'string', required: true, description: 'Dimension (e.g. role, risk_tolerance). Normalized at write.' },
|
||||
value: { type: 'string', required: true, description: 'The resolved value (e.g. advisor).' },
|
||||
confidence: { type: 'number', description: '0..1; default 0.7.' },
|
||||
source: { type: 'string', description: 'Provenance (page slug / uri); default "manual".' },
|
||||
valid_from: { type: 'string', description: 'ISO date the value became true (default: now).' },
|
||||
valid_to: { type: 'string', description: 'ISO date the value stopped being true (default: open).' },
|
||||
visibility: { type: 'string', enum: ['private', 'world'], description: 'Default private.' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
return ctx.engine.mergeOntologyFact({
|
||||
entitySlug: String(p.entity),
|
||||
dimension: String(p.dimension),
|
||||
value: String(p.value),
|
||||
confidence: typeof p.confidence === 'number' ? p.confidence : undefined,
|
||||
source: typeof p.source === 'string' && p.source ? p.source : 'manual',
|
||||
validFrom: typeof p.valid_from === 'string' ? p.valid_from : undefined,
|
||||
validTo: typeof p.valid_to === 'string' ? p.valid_to : undefined,
|
||||
visibility: p.visibility === 'world' ? 'world' : 'private',
|
||||
sourceId: ctx.sourceId,
|
||||
});
|
||||
},
|
||||
cliHints: { name: 'ontology-add', positional: ['entity', 'dimension', 'value'] },
|
||||
};
|
||||
|
||||
const ontology_dimensions: Operation = {
|
||||
name: 'ontology_dimensions',
|
||||
description:
|
||||
'Life Chronicle meta-ontology: which dimensions the brain tracks across entities, with ' +
|
||||
'entity + observation counts. CLI: `gbrain ontology-dimensions`.',
|
||||
scope: 'read',
|
||||
params: {},
|
||||
handler: async (ctx) => ctx.engine.discoverOntologyDimensions(sourceScopeOpts(ctx)),
|
||||
cliHints: { name: 'ontology-dimensions' },
|
||||
};
|
||||
|
||||
const ontology_conflicts: Operation = {
|
||||
name: 'ontology_conflicts',
|
||||
description:
|
||||
'Life Chronicle: dimensions with ≥2 distinct current values from ≥2 provenances (genuine ' +
|
||||
'disagreement, not temporal supersession). CLI: `gbrain ontology-contradictions`.',
|
||||
scope: 'read',
|
||||
params: {
|
||||
min_confidence: { type: 'number', description: 'Only consider observations at/above this confidence (0..1).' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const conflicts = await ctx.engine.findOntologyConflicts({
|
||||
minConfidence: typeof p.min_confidence === 'number' ? p.min_confidence : undefined,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
if (ctx.remote === false) return conflicts;
|
||||
// Remote: redact diary-sourced values; drop conflicts that no longer have
|
||||
// ≥2 distinct values once diary provenance is removed (no leak via conflicts).
|
||||
return conflicts
|
||||
.map((c) => ({ ...c, values: c.values.filter((v) => !(v.source ?? '').startsWith('life/diary/')) }))
|
||||
.filter((c) => new Set(c.values.map((v) => v.value)).size >= 2);
|
||||
},
|
||||
cliHints: { name: 'ontology-contradictions' },
|
||||
};
|
||||
|
||||
const volunteer_chronicle: Operation = {
|
||||
name: 'volunteer_chronicle',
|
||||
description:
|
||||
'Life Chronicle agent-orientation: the recent timeline (last N days) + the current ' +
|
||||
'validity-resolved ontology for the named entities, in one zero-LLM payload, so an agent ' +
|
||||
'orients before acting. Diary-sourced ontology is redacted for remote callers. ' +
|
||||
'CLI: `gbrain orient [--days 7] [--entities people/a,people/b]`.',
|
||||
scope: 'read',
|
||||
params: {
|
||||
days: { type: 'number', description: 'Recent-timeline lookback in days (default 7).' },
|
||||
entities: { type: 'string', description: 'Comma-separated entity slugs to resolve ontology for.' },
|
||||
limit: { type: 'number', description: 'Max timeline rows (default 50).' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { loadChronicleContext } = await import('./context/chronicle-context.ts');
|
||||
const entities = typeof p.entities === 'string'
|
||||
? p.entities.split(',').map((s) => s.trim()).filter(Boolean)
|
||||
: undefined;
|
||||
return loadChronicleContext(ctx.engine, {
|
||||
days: typeof p.days === 'number' ? p.days : undefined,
|
||||
entities,
|
||||
limit: typeof p.limit === 'number' ? p.limit : undefined,
|
||||
remote: ctx.remote !== false,
|
||||
...sourceScopeOpts(ctx),
|
||||
});
|
||||
},
|
||||
cliHints: { name: 'orient' },
|
||||
};
|
||||
|
||||
const chronicle_backfill: Operation = {
|
||||
name: 'chronicle_backfill',
|
||||
description:
|
||||
'Life Chronicle: sweep existing meeting/conversation/calendar pages into timeline events by ' +
|
||||
'enqueuing chronicle_extract jobs (one per eligible page). --dry-run counts without enqueuing. ' +
|
||||
'Local-only bulk op. CLI: `gbrain chronicle-backfill [--since YYYY-MM-DD] [--limit N] [--dry-run]`.',
|
||||
scope: 'admin',
|
||||
mutating: true,
|
||||
localOnly: true,
|
||||
params: {
|
||||
since: { type: 'string', description: 'Only pages updated on/after this date (YYYY-MM-DD).' },
|
||||
limit: { type: 'number', description: 'Max pages per type to sweep (default 1000).' },
|
||||
dry_run: { type: 'boolean', description: 'Count eligible pages without enqueuing.' },
|
||||
},
|
||||
handler: async (ctx, p) => {
|
||||
const { isChronicleEligible } = await import('./chronicle/eligibility.ts');
|
||||
const TYPES = ['meeting', 'conversation', 'calendar-event'] as const;
|
||||
const limit = typeof p.limit === 'number' ? p.limit : 1000;
|
||||
const updated_after = typeof p.since === 'string' ? p.since : undefined;
|
||||
const dryRun = p.dry_run === true;
|
||||
const scope = sourceScopeOpts(ctx);
|
||||
type QueueLike = { add: (n: string, d: Record<string, unknown>) => Promise<unknown> };
|
||||
let queue: QueueLike | null = null;
|
||||
if (!dryRun) {
|
||||
const { MinionQueue } = await import('./minions/queue.ts');
|
||||
queue = new MinionQueue(ctx.engine) as unknown as QueueLike;
|
||||
}
|
||||
let scanned = 0, eligible = 0, enqueued = 0;
|
||||
const errors: { slug: string; error: string }[] = [];
|
||||
for (const type of TYPES) {
|
||||
const pages = await ctx.engine.listPages({ type, updated_after, limit, ...scope });
|
||||
for (const page of pages) {
|
||||
scanned++;
|
||||
const dreamGenerated = (page.frontmatter as Record<string, unknown> | undefined)?.dream_generated === true;
|
||||
const elig = isChronicleEligible({ type: page.type, slug: page.slug, body: page.compiled_truth, dreamGenerated });
|
||||
if (!elig.ok) continue;
|
||||
eligible++;
|
||||
if (dryRun || !queue) continue;
|
||||
try {
|
||||
await queue.add('chronicle_extract', { slug: page.slug, sourceId: ctx.sourceId ?? 'default' });
|
||||
enqueued++;
|
||||
} catch (e) {
|
||||
// Never swallow — surface per-page failures (the #2057 no-swallow pattern).
|
||||
errors.push({ slug: page.slug, error: e instanceof Error ? e.message : String(e) });
|
||||
}
|
||||
}
|
||||
}
|
||||
return { scanned, eligible, enqueued, dry_run: dryRun, errors };
|
||||
},
|
||||
cliHints: { name: 'chronicle-backfill' },
|
||||
};
|
||||
|
||||
export const operations: Operation[] = [
|
||||
// Page CRUD
|
||||
get_page, put_page, delete_page, list_pages,
|
||||
@@ -5069,6 +5363,10 @@ export const operations: Operation[] = [
|
||||
whoami, sources_add, sources_list, sources_remove, sources_status,
|
||||
// v0.29: Salience + anomalies + recent transcripts
|
||||
get_recent_salience, find_anomalies, get_recent_transcripts,
|
||||
// v0.42.x (#2390): Life Chronicle timeline reads
|
||||
chronicle_day, chronicle_on_this_day, chronicle_since, chronicle_last_seen,
|
||||
ontology_get, ontology_propose, ontology_dimensions, ontology_conflicts,
|
||||
volunteer_chronicle, chronicle_backfill,
|
||||
// v0.43 (#2095): push-based context
|
||||
volunteer_context,
|
||||
// v0.31: hot memory (facts table)
|
||||
|
||||
+288
-1
@@ -30,6 +30,9 @@ import type {
|
||||
SearchResult, SearchOpts,
|
||||
Link, GraphNode, GraphPath,
|
||||
TimelineEntry, TimelineInput, TimelineOpts,
|
||||
ChronicleTimelineRow, ChronicleTimelineOpts, LastSeenResult,
|
||||
OntologyObservationInput, OntologyMergeResult, OntologyValue, OntologyDimensionStat,
|
||||
OntologyConflict, OntologyReadOpts,
|
||||
RawData,
|
||||
PageVersion,
|
||||
BrainStats, BrainHealth,
|
||||
@@ -48,6 +51,7 @@ import { normalizeWeightForStorage } from './takes-fence.ts';
|
||||
import { executeRawJsonb } from './sql-query.ts';
|
||||
import { sanitizeForJsonb, buildLinkRows, buildTimelineRows, buildTakeRows } from './batch-rows.ts';
|
||||
import { GBrainError, PAGE_SORT_SQL, ENRICH_ORDER_SQL } from './types.ts';
|
||||
import { finalizeLastSeen } from './chronicle/last-seen.ts';
|
||||
import { computeAnomaliesFromBuckets } from './cycle/anomaly.ts';
|
||||
import { resolveBoostMap, resolveHardExcludes } from './search/source-boost.ts';
|
||||
import { buildSourceFactorCase, buildHardExcludeClause, buildVisibilityClause, buildRecencyComponentSql, buildBestPerPagePoolCte } from './search/sql-ranking.ts';
|
||||
@@ -154,10 +158,18 @@ export function computeSnapshotSchemaHash(
|
||||
* errors). Match the literal `$$bunfs` marker OR ENOENT+pglite.data
|
||||
* co-occurrence.
|
||||
*/
|
||||
export type PgliteInitFailure = 'bunfs' | 'macos-26-3' | 'unknown';
|
||||
export type PgliteInitFailure = 'bunfs' | 'macos-26-3' | 'corrupt' | 'unknown';
|
||||
|
||||
export function classifyPgliteInitError(message: string): PgliteInitFailure {
|
||||
if (/\$\$bunfs|ENOENT[\s\S]*pglite\.data/i.test(message)) return 'bunfs';
|
||||
// #2348: a corrupted PGLite data dir (two OS processes opened it concurrently
|
||||
// and trashed the catalog/extension state) surfaces as a 58P01 internal error
|
||||
// loading the pgvector library, or the vector type / a core relation gone
|
||||
// missing. Distinct, actionable cause — must beat the generic wasm-runtime
|
||||
// match below so the user is pointed at recovery, not the macOS WASM bug.
|
||||
if (/58P01|internal_load_library|type "?vector"? does not exist|relation "?content_chunks"? does not exist/i.test(message)) {
|
||||
return 'corrupt';
|
||||
}
|
||||
if (/abort.*runtime|macos.*26\.3|wasm.*runtime/i.test(message)) {
|
||||
return 'macos-26-3';
|
||||
}
|
||||
@@ -184,6 +196,18 @@ export function buildPgliteInitErrorMessage(
|
||||
' This is most commonly the macOS 26.3 WASM bug:\n' +
|
||||
' https://github.com/garrytan/gbrain/issues/223';
|
||||
break;
|
||||
case 'corrupt':
|
||||
hint =
|
||||
' Your PGLite store looks corrupted (the catalog or the pgvector\n' +
|
||||
' extension cannot load). This happens when two processes opened the\n' +
|
||||
' same brain at once — now prevented (#2348), but an already-damaged\n' +
|
||||
' store cannot be repaired in place. Recover:\n' +
|
||||
' 1. Restore a backup of the brain.pglite directory if you have one, OR\n' +
|
||||
' 2. Rebuild from your brain repo:\n' +
|
||||
' gbrain reinit-pglite --embedding-model <id> --embedding-dimensions <N>\n' +
|
||||
' (wipes + re-inits + re-syncs; DB-only state is re-derived).\n' +
|
||||
' Deleting .gbrain-lock/ or postmaster.pid does NOT fix this.';
|
||||
break;
|
||||
case 'unknown':
|
||||
default:
|
||||
hint =
|
||||
@@ -3376,6 +3400,269 @@ export class PGLiteEngine implements BrainEngine {
|
||||
return result.rows as unknown as TimelineEntry[];
|
||||
}
|
||||
|
||||
// ── v0.42.x Life Chronicle (#2390) timeline reads ───────────────────────
|
||||
// Same result contract as the Postgres engine (parity is on results, not the
|
||||
// builder idiom): JOIN depth page (deleted_at IS NULL), LEFT JOIN event page,
|
||||
// hide soft-deleted event projections, order by COALESCE(event effective_date,
|
||||
// date). Source scope: federated sourceIds[] > scalar sourceId > unscoped.
|
||||
private pushChronicleSource(where: string[], params: unknown[], opts?: { sourceId?: string; sourceIds?: string[] }): void {
|
||||
if (opts?.sourceIds && opts.sourceIds.length > 0) {
|
||||
params.push(opts.sourceIds);
|
||||
where.push(`p.source_id = ANY($${params.length}::text[])`);
|
||||
} else if (opts?.sourceId) {
|
||||
params.push(opts.sourceId);
|
||||
where.push(`p.source_id = $${params.length}`);
|
||||
}
|
||||
}
|
||||
|
||||
private static CHRONICLE_SELECT = `
|
||||
SELECT te.date::text AS date, te.summary, te.detail, te.source,
|
||||
te.page_id, p.slug AS page_slug,
|
||||
te.event_page_id, ep.slug AS event_slug,
|
||||
ep.effective_date::text AS effective_date,
|
||||
ep.frontmatter->'event'->>'kind' AS kind
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id`;
|
||||
|
||||
async getTimelineForDate(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> {
|
||||
const limit = opts?.limit ?? 200;
|
||||
const params: unknown[] = [date];
|
||||
const lower = opts?.week ? `date_trunc('week', $1::date)::date` : `$1::date`;
|
||||
const upper = opts?.week ? `(date_trunc('week', $1::date) + interval '6 days')::date` : `$1::date`;
|
||||
const where: string[] = [
|
||||
`te.date >= ${lower}`,
|
||||
`te.date <= ${upper}`,
|
||||
`(te.event_page_id IS NULL OR ep.deleted_at IS NULL)`,
|
||||
];
|
||||
this.pushChronicleSource(where, params, opts);
|
||||
params.push(limit);
|
||||
const result = await this.db.query(
|
||||
`${PGLiteEngine.CHRONICLE_SELECT}
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) ASC, te.id ASC
|
||||
LIMIT $${params.length}`,
|
||||
params,
|
||||
);
|
||||
return result.rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getSince(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> {
|
||||
const limit = opts?.limit ?? 200;
|
||||
const params: unknown[] = [date];
|
||||
const where: string[] = [
|
||||
`te.date >= $1::date`,
|
||||
`(te.event_page_id IS NULL OR ep.deleted_at IS NULL)`,
|
||||
];
|
||||
if (opts?.kind) {
|
||||
params.push(opts.kind);
|
||||
where.push(`ep.frontmatter->'event'->>'kind' = $${params.length}`);
|
||||
}
|
||||
this.pushChronicleSource(where, params, opts);
|
||||
params.push(limit);
|
||||
const result = await this.db.query(
|
||||
`${PGLiteEngine.CHRONICLE_SELECT}
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) ASC, te.id ASC
|
||||
LIMIT $${params.length}`,
|
||||
params,
|
||||
);
|
||||
return result.rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getOnThisDay(opts?: { date?: string; limit?: number; sourceId?: string; sourceIds?: string[] }): Promise<ChronicleTimelineRow[]> {
|
||||
const limit = opts?.limit ?? 50;
|
||||
const params: unknown[] = [];
|
||||
let target: string;
|
||||
if (opts?.date) { params.push(opts.date); target = `$${params.length}::date`; }
|
||||
else { target = `current_date`; }
|
||||
const where: string[] = [
|
||||
`EXTRACT(MONTH FROM te.date) = EXTRACT(MONTH FROM ${target})`,
|
||||
`EXTRACT(DAY FROM te.date) = EXTRACT(DAY FROM ${target})`,
|
||||
`te.date < ${target}`,
|
||||
`(te.event_page_id IS NULL OR ep.deleted_at IS NULL)`,
|
||||
];
|
||||
this.pushChronicleSource(where, params, opts);
|
||||
params.push(limit);
|
||||
const result = await this.db.query(
|
||||
`${PGLiteEngine.CHRONICLE_SELECT}
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY te.date DESC, te.id ASC
|
||||
LIMIT $${params.length}`,
|
||||
params,
|
||||
);
|
||||
return result.rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getLastSeen(entitySlug: string, opts?: { asof?: string; sourceId?: string; sourceIds?: string[] }): Promise<LastSeenResult> {
|
||||
const params: unknown[] = [entitySlug, `%${entitySlug}%`];
|
||||
const where: string[] = [
|
||||
`(te.event_page_id IS NULL OR ep.deleted_at IS NULL)`,
|
||||
`(p.slug = $1 OR (ep.id IS NOT NULL AND EXISTS (
|
||||
SELECT 1 FROM jsonb_array_elements_text(
|
||||
CASE WHEN jsonb_typeof(ep.frontmatter->'event'->'who') = 'array'
|
||||
THEN ep.frontmatter->'event'->'who' ELSE '[]'::jsonb END
|
||||
) AS w(name) WHERE w.name = $1 OR w.name LIKE $2)))`,
|
||||
];
|
||||
this.pushChronicleSource(where, params, opts);
|
||||
const result = await this.db.query(
|
||||
`SELECT te.date::text AS last_date, ep.slug AS last_event_slug
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE ${where.join(' AND ')}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) DESC, te.id DESC
|
||||
LIMIT 1`,
|
||||
params,
|
||||
);
|
||||
const row = result.rows[0] as { last_date?: string; last_event_slug?: string } | undefined;
|
||||
return finalizeLastSeen(entitySlug, row?.last_date ?? null, row?.last_event_slug ?? null, opts?.asof);
|
||||
}
|
||||
|
||||
async upsertEventProjection(opts: { depthSlug: string; eventSlug: string; date: string; summary: string; detail?: string; sourceId?: string }): Promise<{ projected: boolean }> {
|
||||
const sourceId = opts.sourceId ?? 'default';
|
||||
const r = await this.db.query(
|
||||
`INSERT INTO timeline_entries (page_id, date, source, summary, detail, event_page_id)
|
||||
SELECT dp.id, $1::date, $2, $3, $4, ep.id
|
||||
FROM pages dp, pages ep
|
||||
WHERE dp.slug = $5 AND dp.source_id = $6 AND ep.slug = $7 AND ep.source_id = $6
|
||||
ON CONFLICT (event_page_id, date) WHERE event_page_id IS NOT NULL
|
||||
DO UPDATE SET summary = EXCLUDED.summary, detail = EXCLUDED.detail,
|
||||
page_id = EXCLUDED.page_id, source = EXCLUDED.source
|
||||
RETURNING id`,
|
||||
[opts.date, 'life-chronicle:event:' + opts.eventSlug, opts.summary, opts.detail ?? '', opts.depthSlug, sourceId, opts.eventSlug],
|
||||
);
|
||||
return { projected: r.rows.length > 0 };
|
||||
}
|
||||
|
||||
async mergeOntologyFact(obs: OntologyObservationInput): Promise<OntologyMergeResult> {
|
||||
const sourceId = obs.sourceId ?? 'default';
|
||||
const { valueHash, normalizeDimension, isNovelDimension } = await import('./chronicle/ontology.ts');
|
||||
const dimension = normalizeDimension(obs.dimension);
|
||||
const vh = valueHash(obs.value);
|
||||
const conf = obs.confidence ?? 0.7;
|
||||
const status = obs.status ?? (isNovelDimension(dimension) ? 'quarantined' : 'active');
|
||||
const visibility = obs.visibility ?? 'private';
|
||||
const validFrom = obs.validFrom ?? null;
|
||||
const validUntil = obs.validTo ?? null;
|
||||
const factText = `${dimension}: ${obs.value}`;
|
||||
|
||||
// "current open" = open-ended (valid_until IS NULL) + not retracted.
|
||||
const cur = await this.db.query(
|
||||
`SELECT id, value_hash, valid_from FROM facts
|
||||
WHERE source_id = $1 AND entity_slug = $2 AND dimension = $3 AND expired_at IS NULL AND valid_until IS NULL
|
||||
AND (dim_status IS NULL OR dim_status = 'active')
|
||||
ORDER BY valid_from DESC NULLS LAST, confidence DESC, id DESC LIMIT 1`,
|
||||
[sourceId, obs.entitySlug, dimension],
|
||||
);
|
||||
const current = cur.rows[0] as { id: number; value_hash: string; valid_from: string | null } | undefined;
|
||||
|
||||
if (current && current.value_hash === vh) {
|
||||
const ins = await this.db.query(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, dimension, value, value_hash, dim_status,
|
||||
confidence, source, source_markdown_slug, valid_from, valid_until, expired_at, consolidated_into)
|
||||
VALUES ($1,$2,$3,'fact',$4,$5,$6,$7,$8,$9,$10,$10,COALESCE($11::timestamptz, now()),$12, now(), $13)
|
||||
ON CONFLICT (source_id, entity_slug, dimension, value_hash, source_markdown_slug) WHERE dimension IS NOT NULL
|
||||
DO NOTHING RETURNING id`,
|
||||
[sourceId, obs.entitySlug, factText, visibility, dimension, obs.value, vh, status, conf, obs.source, validFrom, validUntil, current.id],
|
||||
);
|
||||
return ins.rows.length
|
||||
? { action: 'corroborated', factId: Number((ins.rows[0] as { id: number }).id), supersededId: null }
|
||||
: { action: 'noop', factId: null, supersededId: null };
|
||||
}
|
||||
|
||||
const ins = await this.db.query(
|
||||
`INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, dimension, value, value_hash, dim_status,
|
||||
confidence, source, source_markdown_slug, valid_from, valid_until)
|
||||
VALUES ($1,$2,$3,'fact',$4,$5,$6,$7,$8,$9,$10,$10,COALESCE($11::timestamptz, now()),$12)
|
||||
ON CONFLICT (source_id, entity_slug, dimension, value_hash, source_markdown_slug) WHERE dimension IS NOT NULL
|
||||
DO NOTHING RETURNING id`,
|
||||
[sourceId, obs.entitySlug, factText, visibility, dimension, obs.value, vh, status, conf, obs.source, validFrom, validUntil],
|
||||
);
|
||||
if (!ins.rows.length) return { action: 'noop', factId: null, supersededId: null };
|
||||
const newId = Number((ins.rows[0] as { id: number }).id);
|
||||
|
||||
let supersededId: number | null = null;
|
||||
if (current && status === 'active') {
|
||||
const forward = validFrom == null || current.valid_from == null
|
||||
|| new Date(validFrom).getTime() >= new Date(current.valid_from).getTime();
|
||||
if (forward) {
|
||||
await this.db.query(
|
||||
`UPDATE facts SET valid_until = COALESCE($1::timestamptz, now()), superseded_by = $2 WHERE id = $3 AND valid_until IS NULL`,
|
||||
[validFrom, newId, current.id],
|
||||
);
|
||||
supersededId = current.id;
|
||||
}
|
||||
}
|
||||
return { action: supersededId ? 'superseded_prior' : 'inserted', factId: newId, supersededId };
|
||||
}
|
||||
|
||||
async getOntology(entitySlug: string, opts?: OntologyReadOpts): Promise<OntologyValue[]> {
|
||||
const minConf = opts?.minConfidence ?? 0;
|
||||
const includeQ = opts?.includeQuarantined ?? false;
|
||||
const asof = opts?.asof ?? null;
|
||||
const params: unknown[] = [entitySlug, asof, minConf, includeQ];
|
||||
let scope: string;
|
||||
if (opts?.sourceIds && opts.sourceIds.length) { params.push(opts.sourceIds); scope = `AND source_id = ANY($${params.length})`; }
|
||||
else { params.push(opts?.sourceId ?? null); scope = `AND ($${params.length}::text IS NULL OR source_id = $${params.length})`; }
|
||||
const r = await this.db.query(
|
||||
`SELECT DISTINCT ON (dimension) dimension, value, confidence,
|
||||
source_markdown_slug AS source, valid_from, valid_until AS valid_to,
|
||||
COALESCE(dim_status,'active') AS status, id AS fact_id
|
||||
FROM facts
|
||||
WHERE entity_slug = $1 AND dimension IS NOT NULL AND expired_at IS NULL ${scope}
|
||||
AND COALESCE(valid_from,'-infinity'::timestamptz) <= COALESCE($2::timestamptz, now())
|
||||
AND COALESCE(valid_until,'infinity'::timestamptz) > COALESCE($2::timestamptz, now())
|
||||
AND confidence >= $3
|
||||
AND ($4::boolean OR dim_status IS NULL OR dim_status = 'active')
|
||||
ORDER BY dimension, valid_from DESC NULLS LAST, confidence DESC, id DESC`,
|
||||
params,
|
||||
);
|
||||
return r.rows.map((row) => ({ ...(row as OntologyValue), confidence: Number((row as { confidence: number }).confidence), fact_id: Number((row as { fact_id: number }).fact_id) }));
|
||||
}
|
||||
|
||||
async discoverOntologyDimensions(opts?: { sourceId?: string; sourceIds?: string[] }): Promise<OntologyDimensionStat[]> {
|
||||
const params: unknown[] = [];
|
||||
let scope: string;
|
||||
if (opts?.sourceIds && opts.sourceIds.length) { params.push(opts.sourceIds); scope = `AND source_id = ANY($${params.length})`; }
|
||||
else { params.push(opts?.sourceId ?? null); scope = `AND ($${params.length}::text IS NULL OR source_id = $${params.length})`; }
|
||||
const r = await this.db.query(
|
||||
`SELECT dimension, count(DISTINCT entity_slug)::int AS entities, count(*)::int AS observations
|
||||
FROM facts WHERE dimension IS NOT NULL AND expired_at IS NULL ${scope}
|
||||
GROUP BY dimension ORDER BY entities DESC, dimension`,
|
||||
params,
|
||||
);
|
||||
return r.rows.map((row) => {
|
||||
const x = row as { dimension: string; entities: number; observations: number };
|
||||
return { dimension: x.dimension, entities: Number(x.entities), observations: Number(x.observations) };
|
||||
});
|
||||
}
|
||||
|
||||
async findOntologyConflicts(opts?: { sourceId?: string; sourceIds?: string[]; minConfidence?: number }): Promise<OntologyConflict[]> {
|
||||
const minConf = opts?.minConfidence ?? 0;
|
||||
const params: unknown[] = [minConf];
|
||||
let scope: string;
|
||||
if (opts?.sourceIds && opts.sourceIds.length) { params.push(opts.sourceIds); scope = `AND source_id = ANY($${params.length})`; }
|
||||
else { params.push(opts?.sourceId ?? null); scope = `AND ($${params.length}::text IS NULL OR source_id = $${params.length})`; }
|
||||
const r = await this.db.query(
|
||||
`WITH cur AS (
|
||||
SELECT entity_slug, dimension, value, source_markdown_slug AS source, confidence, id AS fact_id
|
||||
FROM facts WHERE dimension IS NOT NULL AND expired_at IS NULL AND valid_until IS NULL
|
||||
AND (dim_status IS NULL OR dim_status = 'active') AND confidence >= $1 ${scope}
|
||||
)
|
||||
SELECT entity_slug, dimension,
|
||||
json_agg(json_build_object('value', value, 'source', source, 'confidence', confidence, 'fact_id', fact_id)) AS values
|
||||
FROM cur GROUP BY entity_slug, dimension
|
||||
HAVING count(DISTINCT value) >= 2 AND count(DISTINCT source) >= 2
|
||||
ORDER BY entity_slug, dimension`,
|
||||
params,
|
||||
);
|
||||
return r.rows.map((row) => {
|
||||
const x = row as { entity_slug: string; dimension: string; values: OntologyConflict['values'] };
|
||||
return { entity_slug: x.entity_slug, dimension: x.dimension, values: typeof x.values === 'string' ? JSON.parse(x.values) : x.values };
|
||||
});
|
||||
}
|
||||
|
||||
// Raw data
|
||||
async putRawData(
|
||||
slug: string,
|
||||
|
||||
+25
-30
@@ -24,17 +24,17 @@ const LOCK_FILE = 'lock';
|
||||
// LIVE holder (embed jobs run for many minutes) is never mistaken for stale.
|
||||
const HEARTBEAT_INTERVAL_MS = 30_000;
|
||||
|
||||
// #2058: a holder whose heartbeat refreshed within this window is ALIVE and is
|
||||
// NEVER stolen, regardless of how old the lock is. Only a holder that STOPPED
|
||||
// refreshing past this grace (hung, crashed without cleanup, or a PID since
|
||||
// reused by an unrelated process) is reaped. Pairing heartbeat-age with PID
|
||||
// liveness is what defeats both the WAL-corruption bug (stealing a live
|
||||
// writer) AND the PID-reuse false-positive (a recycled PID reading as "alive").
|
||||
// Env-overridable as an incident escape hatch, matching the sync-lock knobs.
|
||||
function stealGraceMs(): number {
|
||||
const env = parseInt(process.env.GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS ?? '', 10);
|
||||
return Number.isFinite(env) && env > 0 ? env * 1000 : 10 * 60 * 1000; // default 600s
|
||||
}
|
||||
// #2348: there is NO steal-on-stale-heartbeat anymore. A holder whose PID is
|
||||
// alive is NEVER reaped, regardless of how long its heartbeat has been stale.
|
||||
// PGLite/WASM is strictly single-writer; the heartbeat runs on the JS event
|
||||
// loop, which is BLOCKED during long synchronous imports/CHECKPOINTs, so a
|
||||
// genuinely working `gbrain dream`/embed holder can look stale while alive.
|
||||
// Reaping it (the old #2058 grace window) let a second OS process open the same
|
||||
// data dir and corrupt the catalog + pgvector extension state (58P01 /
|
||||
// internal_load_library / `type "vector" does not exist`), recoverable only by
|
||||
// wipe+restore. Only a DEAD PID is reaped now; a wedged-but-alive or PID-reused
|
||||
// holder makes the acquire time out with a message naming the PID (the user
|
||||
// removes the lock explicitly) rather than risk corruption.
|
||||
|
||||
export interface LockHandle {
|
||||
lockDir: string;
|
||||
@@ -47,11 +47,12 @@ export interface LockHandle {
|
||||
heartbeat?: ReturnType<typeof setInterval>;
|
||||
lockPath?: string;
|
||||
/**
|
||||
* #2058 (codex): our ownership token (`<pid>:<acquired_at>`). If we stall
|
||||
* past the steal grace, another process can reap + re-acquire. When we
|
||||
* resume, the heartbeat and release MUST verify the on-disk lock is STILL
|
||||
* ours before touching it — otherwise a resumed stale holder would refresh
|
||||
* or delete the NEW owner's live lock, re-opening the concurrent-writer hole.
|
||||
* Our ownership token (`<pid>:<acquired_at>`). Since #2348 a LIVE holder is
|
||||
* never reaped, so reap-then-reacquire happens only after the original holder
|
||||
* is dead — but the heartbeat and release STILL verify the on-disk lock is
|
||||
* ours before touching it (defense-in-depth: a crash-then-restart on a reused
|
||||
* PID, or a misclassification, must never let a stale handle refresh or delete
|
||||
* the NEW owner's live lock and re-open the concurrent-writer hole).
|
||||
*/
|
||||
ownerToken?: string;
|
||||
}
|
||||
@@ -134,25 +135,19 @@ export async function acquireLock(dataDir: string | undefined, opts?: { timeoutM
|
||||
try {
|
||||
const lockData = JSON.parse(readFileSync(lockPath, 'utf-8'));
|
||||
const lockPid = lockData.pid as number;
|
||||
const lockTime = lockData.acquired_at as number;
|
||||
|
||||
// #2058: classify by PID liveness AND heartbeat freshness. A holder
|
||||
// that is alive AND refreshed its heartbeat within the steal grace is
|
||||
// genuinely working (e.g. a multi-minute embed) and is NEVER reaped —
|
||||
// force-removing it here is what corrupted the single-writer WAL.
|
||||
// #2348: classify ONLY by PID liveness. A live holder is NEVER reaped
|
||||
// (stealing a live single-writer is what corrupted the catalog/extension
|
||||
// state). A long synchronous import blocks the heartbeat, so "stale
|
||||
// heartbeat" is NOT evidence of death — only a dead PID is.
|
||||
const alive = isProcessAlive(lockPid);
|
||||
const lastRefresh = (lockData.refreshed_at as number | undefined) ?? lockTime;
|
||||
const sinceRefresh = Date.now() - lastRefresh;
|
||||
if (!alive) {
|
||||
// Holder process is gone — reap.
|
||||
// Holder process is gone — reap and try to acquire.
|
||||
try { rmSync(lockDir, { recursive: true, force: true }); } catch { /* race condition, try again */ }
|
||||
} else if (sinceRefresh > stealGraceMs()) {
|
||||
// PID is alive but the heartbeat stopped past the grace window:
|
||||
// either the holder hung, or this PID was reused by an unrelated
|
||||
// process (the real holder died and stopped refreshing). Reap.
|
||||
try { rmSync(lockDir, { recursive: true, force: true }); } catch { /* race condition */ }
|
||||
} else {
|
||||
// Live holder refreshing within grace — wait and retry.
|
||||
// Live holder — wait and retry. If it is genuinely wedged (or its PID
|
||||
// was reused by an unrelated process), the acquire times out below
|
||||
// with a message naming the PID; we never force-steal a live holder.
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
continue;
|
||||
}
|
||||
|
||||
@@ -371,6 +371,9 @@ CREATE TABLE IF NOT EXISTS timeline_entries (
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
summary TEXT NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT '',
|
||||
-- v0.42.x (Life Chronicle #2390): event-projection pointer. NULL for
|
||||
-- ordinary rows. See src/schema.sql for the full rationale.
|
||||
event_page_id INTEGER REFERENCES pages(id) ON DELETE CASCADE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
@@ -380,6 +383,9 @@ CREATE INDEX IF NOT EXISTS idx_timeline_date ON timeline_entries(date);
|
||||
-- v0.41.18.0 (codex finding #11): widened to include source so distinct
|
||||
-- meeting provenance survives. Legacy rows have source='' (schema default).
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_dedup ON timeline_entries(page_id, date, summary, source);
|
||||
-- v0.42.x (Life Chronicle): event-projection lookup + dedup (partial).
|
||||
CREATE INDEX IF NOT EXISTS idx_timeline_event_page ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_event_dedup ON timeline_entries(event_page_id, date) WHERE event_page_id IS NOT NULL;
|
||||
|
||||
-- ============================================================
|
||||
-- page_versions: snapshot history
|
||||
|
||||
@@ -40,6 +40,9 @@ import type {
|
||||
SearchResult, SearchOpts,
|
||||
Link, GraphNode, GraphPath,
|
||||
TimelineEntry, TimelineInput, TimelineOpts,
|
||||
ChronicleTimelineRow, ChronicleTimelineOpts, LastSeenResult,
|
||||
OntologyObservationInput, OntologyMergeResult, OntologyValue, OntologyDimensionStat,
|
||||
OntologyConflict, OntologyReadOpts,
|
||||
RawData,
|
||||
PageVersion,
|
||||
BrainStats, BrainHealth,
|
||||
@@ -52,6 +55,7 @@ import type {
|
||||
EnrichCandidatesOpts, EnrichCandidate,
|
||||
} from './types.ts';
|
||||
import { GBrainError, PAGE_SORT_SQL, ENRICH_ORDER_SQL } from './types.ts';
|
||||
import { finalizeLastSeen } from './chronicle/last-seen.ts';
|
||||
import { computeAnomaliesFromBuckets } from './cycle/anomaly.ts';
|
||||
import * as db from './db.ts';
|
||||
import { ConnectionManager } from './connection-manager.ts';
|
||||
@@ -3440,6 +3444,256 @@ export class PostgresEngine implements BrainEngine {
|
||||
return rows as unknown as TimelineEntry[];
|
||||
}
|
||||
|
||||
// ── v0.42.x Life Chronicle (#2390) timeline reads ───────────────────────
|
||||
// Shared shape: timeline_entries JOIN depth page (deleted_at IS NULL) LEFT
|
||||
// JOIN event page; hide soft-deleted event projections (read-time, not just
|
||||
// doctor); order by COALESCE(event effective_date, date) for intra-day
|
||||
// sequence. Source scope: federated sourceIds[] > scalar sourceId > unscoped.
|
||||
private chronicleSourceCond(opts?: { sourceId?: string; sourceIds?: string[] }) {
|
||||
const sql = this.sql;
|
||||
return opts?.sourceIds && opts.sourceIds.length > 0
|
||||
? sql`AND p.source_id = ANY(${opts.sourceIds}::text[])`
|
||||
: opts?.sourceId
|
||||
? sql`AND p.source_id = ${opts.sourceId}`
|
||||
: sql``;
|
||||
}
|
||||
|
||||
async getTimelineForDate(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> {
|
||||
const sql = this.sql;
|
||||
const limit = opts?.limit ?? 200;
|
||||
// ISO week (date_trunc('week') → Monday) or the single day.
|
||||
const lower = opts?.week ? sql`date_trunc('week', ${date}::date)::date` : sql`${date}::date`;
|
||||
const upper = opts?.week ? sql`(date_trunc('week', ${date}::date) + interval '6 days')::date` : sql`${date}::date`;
|
||||
const rows = await sql`
|
||||
SELECT te.date::text AS date, te.summary, te.detail, te.source,
|
||||
te.page_id, p.slug AS page_slug,
|
||||
te.event_page_id, ep.slug AS event_slug,
|
||||
ep.effective_date::text AS effective_date,
|
||||
ep.frontmatter->'event'->>'kind' AS kind
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE te.date >= ${lower} AND te.date <= ${upper}
|
||||
AND (te.event_page_id IS NULL OR ep.deleted_at IS NULL)
|
||||
${this.chronicleSourceCond(opts)}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) ASC, te.id ASC
|
||||
LIMIT ${limit}`;
|
||||
return rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getSince(date: string, opts?: ChronicleTimelineOpts): Promise<ChronicleTimelineRow[]> {
|
||||
const sql = this.sql;
|
||||
const limit = opts?.limit ?? 200;
|
||||
const kindCond = opts?.kind ? sql`AND ep.frontmatter->'event'->>'kind' = ${opts.kind}` : sql``;
|
||||
const rows = await sql`
|
||||
SELECT te.date::text AS date, te.summary, te.detail, te.source,
|
||||
te.page_id, p.slug AS page_slug,
|
||||
te.event_page_id, ep.slug AS event_slug,
|
||||
ep.effective_date::text AS effective_date,
|
||||
ep.frontmatter->'event'->>'kind' AS kind
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE te.date >= ${date}::date
|
||||
AND (te.event_page_id IS NULL OR ep.deleted_at IS NULL)
|
||||
${kindCond}
|
||||
${this.chronicleSourceCond(opts)}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) ASC, te.id ASC
|
||||
LIMIT ${limit}`;
|
||||
return rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getOnThisDay(opts?: { date?: string; limit?: number; sourceId?: string; sourceIds?: string[] }): Promise<ChronicleTimelineRow[]> {
|
||||
const sql = this.sql;
|
||||
const limit = opts?.limit ?? 50;
|
||||
const target = opts?.date ? sql`${opts.date}::date` : sql`current_date`;
|
||||
const rows = await sql`
|
||||
SELECT te.date::text AS date, te.summary, te.detail, te.source,
|
||||
te.page_id, p.slug AS page_slug,
|
||||
te.event_page_id, ep.slug AS event_slug,
|
||||
ep.effective_date::text AS effective_date,
|
||||
ep.frontmatter->'event'->>'kind' AS kind
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE EXTRACT(MONTH FROM te.date) = EXTRACT(MONTH FROM ${target})
|
||||
AND EXTRACT(DAY FROM te.date) = EXTRACT(DAY FROM ${target})
|
||||
AND te.date < ${target}
|
||||
AND (te.event_page_id IS NULL OR ep.deleted_at IS NULL)
|
||||
${this.chronicleSourceCond(opts)}
|
||||
ORDER BY te.date DESC, te.id ASC
|
||||
LIMIT ${limit}`;
|
||||
return rows as unknown as ChronicleTimelineRow[];
|
||||
}
|
||||
|
||||
async getLastSeen(entitySlug: string, opts?: { asof?: string; sourceId?: string; sourceIds?: string[] }): Promise<LastSeenResult> {
|
||||
const sql = this.sql;
|
||||
// "Seen" = the entity's own page has a timeline row, OR an event's `who`
|
||||
// array references the entity (exact slug or wikilink-substring match).
|
||||
const rows = await sql`
|
||||
SELECT te.date::text AS last_date, ep.slug AS last_event_slug
|
||||
FROM timeline_entries te
|
||||
JOIN pages p ON p.id = te.page_id AND p.deleted_at IS NULL
|
||||
LEFT JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE (te.event_page_id IS NULL OR ep.deleted_at IS NULL)
|
||||
AND (
|
||||
p.slug = ${entitySlug}
|
||||
OR (ep.id IS NOT NULL AND EXISTS (
|
||||
SELECT 1 FROM jsonb_array_elements_text(
|
||||
CASE WHEN jsonb_typeof(ep.frontmatter->'event'->'who') = 'array'
|
||||
THEN ep.frontmatter->'event'->'who' ELSE '[]'::jsonb END
|
||||
) AS w(name)
|
||||
WHERE w.name = ${entitySlug} OR w.name LIKE ${'%' + entitySlug + '%'}
|
||||
))
|
||||
)
|
||||
${this.chronicleSourceCond(opts)}
|
||||
ORDER BY COALESCE(ep.effective_date, te.date::timestamptz) DESC, te.id DESC
|
||||
LIMIT 1`;
|
||||
const row = rows[0] as { last_date?: string; last_event_slug?: string } | undefined;
|
||||
return finalizeLastSeen(entitySlug, row?.last_date ?? null, row?.last_event_slug ?? null, opts?.asof);
|
||||
}
|
||||
|
||||
async upsertEventProjection(opts: { depthSlug: string; eventSlug: string; date: string; summary: string; detail?: string; sourceId?: string }): Promise<{ projected: boolean }> {
|
||||
const sql = this.sql;
|
||||
const sourceId = opts.sourceId ?? 'default';
|
||||
const rows = await sql`
|
||||
INSERT INTO timeline_entries (page_id, date, source, summary, detail, event_page_id)
|
||||
SELECT dp.id, ${opts.date}::date, ${'life-chronicle:event:' + opts.eventSlug}, ${opts.summary}, ${opts.detail ?? ''}, ep.id
|
||||
FROM pages dp, pages ep
|
||||
WHERE dp.slug = ${opts.depthSlug} AND dp.source_id = ${sourceId}
|
||||
AND ep.slug = ${opts.eventSlug} AND ep.source_id = ${sourceId}
|
||||
ON CONFLICT (event_page_id, date) WHERE event_page_id IS NOT NULL
|
||||
DO UPDATE SET summary = EXCLUDED.summary, detail = EXCLUDED.detail,
|
||||
page_id = EXCLUDED.page_id, source = EXCLUDED.source
|
||||
RETURNING id`;
|
||||
return { projected: rows.length > 0 };
|
||||
}
|
||||
|
||||
async mergeOntologyFact(obs: OntologyObservationInput): Promise<OntologyMergeResult> {
|
||||
const sql = this.sql;
|
||||
const sourceId = obs.sourceId ?? 'default';
|
||||
const { valueHash, normalizeDimension, isNovelDimension } = await import('./chronicle/ontology.ts');
|
||||
const dimension = normalizeDimension(obs.dimension);
|
||||
const vh = valueHash(obs.value);
|
||||
const conf = obs.confidence ?? 0.7;
|
||||
const status = obs.status ?? (isNovelDimension(dimension) ? 'quarantined' : 'active');
|
||||
const visibility = obs.visibility ?? 'private';
|
||||
const validFrom = obs.validFrom ?? null;
|
||||
const validUntil = obs.validTo ?? null;
|
||||
const factText = `${dimension}: ${obs.value}`;
|
||||
|
||||
// The "current open" row is the open-ended one (valid_until IS NULL) that
|
||||
// hasn't been retracted (expired_at IS NULL). Supersession closes its
|
||||
// valid_until rather than expiring it, so --asof time-travel still sees it.
|
||||
const cur = await sql<{ id: number; value_hash: string; valid_from: string | null }[]>`
|
||||
SELECT id, value_hash, valid_from FROM facts
|
||||
WHERE source_id = ${sourceId} AND entity_slug = ${obs.entitySlug}
|
||||
AND dimension = ${dimension} AND expired_at IS NULL AND valid_until IS NULL
|
||||
AND (dim_status IS NULL OR dim_status = 'active')
|
||||
ORDER BY valid_from DESC NULLS LAST, confidence DESC, id DESC
|
||||
LIMIT 1`;
|
||||
const current = cur[0];
|
||||
|
||||
if (current && current.value_hash === vh) {
|
||||
// Same value → corroboration (or exact dup → noop via the dedup unique).
|
||||
const ins = await sql<{ id: number }[]>`
|
||||
INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, dimension, value, value_hash, dim_status,
|
||||
confidence, source, source_markdown_slug, valid_from, valid_until, expired_at, consolidated_into)
|
||||
VALUES (${sourceId}, ${obs.entitySlug}, ${factText}, 'fact', ${visibility}, ${dimension}, ${obs.value}, ${vh}, ${status},
|
||||
${conf}, ${obs.source}, ${obs.source}, COALESCE(${validFrom}::timestamptz, now()), ${validUntil}, now(), ${current.id})
|
||||
ON CONFLICT (source_id, entity_slug, dimension, value_hash, source_markdown_slug) WHERE dimension IS NOT NULL
|
||||
DO NOTHING
|
||||
RETURNING id`;
|
||||
return ins.length
|
||||
? { action: 'corroborated', factId: Number(ins[0].id), supersededId: null }
|
||||
: { action: 'noop', factId: null, supersededId: null };
|
||||
}
|
||||
|
||||
const ins = await sql<{ id: number }[]>`
|
||||
INSERT INTO facts (source_id, entity_slug, fact, kind, visibility, dimension, value, value_hash, dim_status,
|
||||
confidence, source, source_markdown_slug, valid_from, valid_until)
|
||||
VALUES (${sourceId}, ${obs.entitySlug}, ${factText}, 'fact', ${visibility}, ${dimension}, ${obs.value}, ${vh}, ${status},
|
||||
${conf}, ${obs.source}, ${obs.source}, COALESCE(${validFrom}::timestamptz, now()), ${validUntil})
|
||||
ON CONFLICT (source_id, entity_slug, dimension, value_hash, source_markdown_slug) WHERE dimension IS NOT NULL
|
||||
DO NOTHING
|
||||
RETURNING id`;
|
||||
if (!ins.length) return { action: 'noop', factId: null, supersededId: null };
|
||||
const newId = Number(ins[0].id);
|
||||
|
||||
let supersededId: number | null = null;
|
||||
if (current && status === 'active') {
|
||||
const forward = validFrom == null || current.valid_from == null
|
||||
|| new Date(validFrom).getTime() >= new Date(current.valid_from).getTime();
|
||||
if (forward) {
|
||||
// Close the prior row's valid window at the new fact's valid_from (or now()).
|
||||
await sql`UPDATE facts SET valid_until = COALESCE(${validFrom}::timestamptz, now()), superseded_by = ${newId}
|
||||
WHERE id = ${current.id} AND valid_until IS NULL`;
|
||||
supersededId = current.id;
|
||||
}
|
||||
}
|
||||
return { action: supersededId ? 'superseded_prior' : 'inserted', factId: newId, supersededId };
|
||||
}
|
||||
|
||||
async getOntology(entitySlug: string, opts?: OntologyReadOpts): Promise<OntologyValue[]> {
|
||||
const sql = this.sql;
|
||||
const minConf = opts?.minConfidence ?? 0;
|
||||
const includeQ = opts?.includeQuarantined ?? false;
|
||||
const asof = opts?.asof ?? null;
|
||||
const scope = opts?.sourceIds && opts.sourceIds.length
|
||||
? sql`AND source_id = ANY(${opts.sourceIds})`
|
||||
: sql`AND (${opts?.sourceId ?? null}::text IS NULL OR source_id = ${opts?.sourceId ?? null})`;
|
||||
const rows = await sql<OntologyValue[]>`
|
||||
SELECT DISTINCT ON (dimension)
|
||||
dimension, value, confidence,
|
||||
source_markdown_slug AS source, valid_from, valid_until AS valid_to,
|
||||
COALESCE(dim_status, 'active') AS status, id AS fact_id
|
||||
FROM facts
|
||||
WHERE entity_slug = ${entitySlug} AND dimension IS NOT NULL AND expired_at IS NULL
|
||||
${scope}
|
||||
AND COALESCE(valid_from, '-infinity'::timestamptz) <= COALESCE(${asof}::timestamptz, now())
|
||||
AND COALESCE(valid_until, 'infinity'::timestamptz) > COALESCE(${asof}::timestamptz, now())
|
||||
AND confidence >= ${minConf}
|
||||
AND (${includeQ}::boolean OR dim_status IS NULL OR dim_status = 'active')
|
||||
ORDER BY dimension, valid_from DESC NULLS LAST, confidence DESC, id DESC`;
|
||||
return rows.map((r) => ({ ...r, confidence: Number(r.confidence), fact_id: Number(r.fact_id) }));
|
||||
}
|
||||
|
||||
async discoverOntologyDimensions(opts?: { sourceId?: string; sourceIds?: string[] }): Promise<OntologyDimensionStat[]> {
|
||||
const sql = this.sql;
|
||||
const scope = opts?.sourceIds && opts.sourceIds.length
|
||||
? sql`AND source_id = ANY(${opts.sourceIds})`
|
||||
: sql`AND (${opts?.sourceId ?? null}::text IS NULL OR source_id = ${opts?.sourceId ?? null})`;
|
||||
const rows = await sql<{ dimension: string; entities: number; observations: number }[]>`
|
||||
SELECT dimension, count(DISTINCT entity_slug)::int AS entities, count(*)::int AS observations
|
||||
FROM facts
|
||||
WHERE dimension IS NOT NULL AND expired_at IS NULL ${scope}
|
||||
GROUP BY dimension ORDER BY entities DESC, dimension`;
|
||||
return rows.map((r) => ({ dimension: r.dimension, entities: Number(r.entities), observations: Number(r.observations) }));
|
||||
}
|
||||
|
||||
async findOntologyConflicts(opts?: { sourceId?: string; sourceIds?: string[]; minConfidence?: number }): Promise<OntologyConflict[]> {
|
||||
const sql = this.sql;
|
||||
const minConf = opts?.minConfidence ?? 0;
|
||||
const scope = opts?.sourceIds && opts.sourceIds.length
|
||||
? sql`AND source_id = ANY(${opts.sourceIds})`
|
||||
: sql`AND (${opts?.sourceId ?? null}::text IS NULL OR source_id = ${opts?.sourceId ?? null})`;
|
||||
const rows = await sql<{ entity_slug: string; dimension: string; values: OntologyConflict['values'] }[]>`
|
||||
WITH cur AS (
|
||||
SELECT entity_slug, dimension, value, source_markdown_slug AS source, confidence, id AS fact_id
|
||||
FROM facts
|
||||
WHERE dimension IS NOT NULL AND expired_at IS NULL AND valid_until IS NULL
|
||||
AND (dim_status IS NULL OR dim_status = 'active')
|
||||
AND confidence >= ${minConf} ${scope}
|
||||
)
|
||||
SELECT entity_slug, dimension,
|
||||
json_agg(json_build_object('value', value, 'source', source, 'confidence', confidence, 'fact_id', fact_id)) AS values
|
||||
FROM cur
|
||||
GROUP BY entity_slug, dimension
|
||||
HAVING count(DISTINCT value) >= 2 AND count(DISTINCT source) >= 2
|
||||
ORDER BY entity_slug, dimension`;
|
||||
return rows.map((r) => ({ entity_slug: r.entity_slug, dimension: r.dimension, values: r.values }));
|
||||
}
|
||||
|
||||
// Raw data
|
||||
async putRawData(
|
||||
slug: string,
|
||||
|
||||
@@ -540,6 +540,13 @@ CREATE TABLE IF NOT EXISTS timeline_entries (
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
summary TEXT NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT '',
|
||||
-- v0.42.x (Life Chronicle #2390): when this row is the date-index projection
|
||||
-- of a \`type:event\` page, event_page_id points at that event page; page_id
|
||||
-- stays the depth/meeting page. NULL for ordinary timeline entries. Reads
|
||||
-- hide rows whose event page is soft-deleted; the partial unique index below
|
||||
-- keys dedup on (event_page_id, date) so re-extraction with a changed summary
|
||||
-- updates the row instead of double-inserting.
|
||||
event_page_id INTEGER REFERENCES pages(id) ON DELETE CASCADE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
@@ -549,6 +556,10 @@ CREATE INDEX IF NOT EXISTS idx_timeline_date ON timeline_entries(date);
|
||||
-- include \`source\` so distinct meeting provenance survives. Legacy rows
|
||||
-- have source='' (schema default) so legacy dedup behavior is preserved.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_dedup ON timeline_entries(page_id, date, summary, source);
|
||||
-- v0.42.x (Life Chronicle): event-projection lookup + dedup. Partial
|
||||
-- (event_page_id IS NOT NULL) so ordinary timeline rows are unaffected.
|
||||
CREATE INDEX IF NOT EXISTS idx_timeline_event_page ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_event_dedup ON timeline_entries(event_page_id, date) WHERE event_page_id IS NOT NULL;
|
||||
|
||||
-- ============================================================
|
||||
-- page_versions: snapshot history for compiled_truth
|
||||
|
||||
@@ -302,6 +302,23 @@ page_types:
|
||||
extractable: true
|
||||
expert_routing: false
|
||||
|
||||
# v0.42.x — Life Chronicle (#2390): timeline events + thought diary.
|
||||
- name: event
|
||||
primitive: temporal
|
||||
path_prefixes:
|
||||
- life/events/
|
||||
aliases: []
|
||||
extractable: false
|
||||
expert_routing: false
|
||||
|
||||
- name: diary
|
||||
primitive: temporal
|
||||
path_prefixes:
|
||||
- life/diary/
|
||||
aliases: []
|
||||
extractable: false
|
||||
expert_routing: false
|
||||
|
||||
link_types:
|
||||
- name: partner_of
|
||||
inverse: partner_of
|
||||
|
||||
@@ -262,6 +262,27 @@ page_types:
|
||||
extractable: false
|
||||
expert_routing: false
|
||||
|
||||
# v0.42.x — Life Chronicle (#2390). `event` = timeline atom
|
||||
# (when·where·who·what) backlinking to a depth page; `diary` = timestamped
|
||||
# first-person interiority. Both `temporal` (events through time, not entity
|
||||
# surfaces). `extractable: false`: events are already the atomic form, and
|
||||
# diary interiority is NEVER mined into the facts table (privacy/consent).
|
||||
- name: event
|
||||
primitive: temporal
|
||||
path_prefixes:
|
||||
- life/events/
|
||||
aliases: []
|
||||
extractable: false
|
||||
expert_routing: false
|
||||
|
||||
- name: diary
|
||||
primitive: temporal
|
||||
path_prefixes:
|
||||
- life/diary/
|
||||
aliases: []
|
||||
extractable: false
|
||||
expert_routing: false
|
||||
|
||||
# Types with no path prefix (set explicitly via frontmatter).
|
||||
- name: code
|
||||
primitive: media
|
||||
|
||||
@@ -334,6 +334,32 @@ export function applyTitleBoost(
|
||||
/** Default title-phrase boost multiplier (mode-overridable via `title_boost`). */
|
||||
export const DEFAULT_TITLE_BOOST = 1.25;
|
||||
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) E1 temporal recall arm. On temporal queries
|
||||
* (the caller gates this on recency !== 'off'), give chronicle `event`/`diary`
|
||||
* pages a bounded boost so the timeline surfaces for "what happened…" / "when
|
||||
* did…" queries — ambient temporality without a separate recall arm. Bounded
|
||||
* ([1.0, 1.25]) + floor-gated like the other metadata stages, so it can't
|
||||
* leapfrog a strong primary hit. Mutate-in-place; caller re-sorts. Pure no-op
|
||||
* for non-chronicle results. NOT called on non-temporal queries (recency='off'),
|
||||
* so ordinary search is bit-for-bit unchanged.
|
||||
*/
|
||||
export function applyChronicleTypeBoost(
|
||||
results: SearchResult[],
|
||||
strength: 'on' | 'strong',
|
||||
floorThreshold?: number,
|
||||
): void {
|
||||
const factor = strength === 'strong' ? 1.25 : 1.15;
|
||||
for (const r of results) {
|
||||
if (!Number.isFinite(r.score)) continue;
|
||||
if (floorThreshold !== undefined && r.score < floorThreshold) continue;
|
||||
if (r.type === 'event' || r.type === 'diary') {
|
||||
r.score *= factor;
|
||||
r.chronicle_boost = factor;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* v0.29.1 — runPostFusionStages: wrap backlink + salience + recency in a
|
||||
* single stage that fires from EVERY hybridSearch return path (codex
|
||||
@@ -472,6 +498,12 @@ export async function runPostFusionStages(
|
||||
} catch {
|
||||
// Non-fatal.
|
||||
}
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390) E1: chronicle event/diary type boost.
|
||||
// Gated INSIDE the recency!=off branch so it fires ONLY on temporal queries;
|
||||
// non-temporal search never reaches here → bit-for-bit unchanged. Shares the
|
||||
// floor threshold so it can't leapfrog a strong primary hit.
|
||||
applyChronicleTypeBoost(results, opts.recency, floorThreshold);
|
||||
}
|
||||
|
||||
// T2 — title-phrase boost. Runs after the metadata stages, before graph
|
||||
|
||||
@@ -58,6 +58,12 @@ export const ALL_PAGE_TYPES: readonly string[] = [
|
||||
// loops via the dream_generated:true + type:extract_receipt belt-and-
|
||||
// suspenders pattern per plan D-EXTRACT-19.
|
||||
'extract_receipt',
|
||||
// v0.42.x — Life Chronicle (#2390). `event` = timeline atom
|
||||
// (when·where·who·what), lives under life/events/; `diary` = first-person
|
||||
// interiority, lives under life/diary/. Both temporal-primitive,
|
||||
// extractable:false (events are one-line atoms; diary is private interiority
|
||||
// never mined into the facts table). Pack entries in gbrain-base.yaml.
|
||||
'event', 'diary',
|
||||
] as const;
|
||||
|
||||
/**
|
||||
@@ -755,6 +761,8 @@ export interface SearchResult {
|
||||
salience_boost?: number;
|
||||
/** Multiplier applied by applyRecencyBoost. */
|
||||
recency_boost?: number;
|
||||
/** v0.42.x (#2390) — multiplier applied by applyChronicleTypeBoost (event/diary on temporal queries). */
|
||||
chronicle_boost?: number;
|
||||
/** Multiplier applied by applyExactMatchBoost. */
|
||||
exact_match_boost?: number;
|
||||
/** Multiplier applied by applyGraphSignals (adjacency hit). */
|
||||
@@ -1279,6 +1287,92 @@ export interface TimelineOpts {
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390): timeline read surfaces.
|
||||
// A ChronicleTimelineRow is a timeline_entries projection JOINed to its depth
|
||||
// page (always) and, when it is an event projection, to the event page (for
|
||||
// intra-day order via effective_date, the backlink slug, and the event kind).
|
||||
export interface ChronicleTimelineRow {
|
||||
date: string; // YYYY-MM-DD (the projection date, pinned tz)
|
||||
summary: string;
|
||||
detail: string;
|
||||
source: string;
|
||||
page_id: number; // depth page id (the rich page the row belongs to)
|
||||
page_slug: string; // depth page slug (backlink target)
|
||||
event_page_id: number | null;
|
||||
event_slug: string | null; // the type:event page, when this is an event projection
|
||||
effective_date: string | null; // event page effective_date — drives intra-day order
|
||||
kind: string | null; // event.kind from the event page frontmatter
|
||||
}
|
||||
|
||||
export interface ChronicleTimelineOpts {
|
||||
/** getTimelineForDate: expand to the ISO week (Mon–Sun) containing `date`. */
|
||||
week?: boolean;
|
||||
/** getSince: filter event projections by `event.kind`. */
|
||||
kind?: string;
|
||||
limit?: number;
|
||||
/** Source scope (scalar). Federated `sourceIds` takes precedence when set. */
|
||||
sourceId?: string;
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
export interface LastSeenResult {
|
||||
entity_slug: string;
|
||||
/** Most recent timeline date the entity appears in (own page or an event's who). NULL if never. */
|
||||
last_date: string | null;
|
||||
/** The most recent event page slug involving the entity, if the last hit was an event. */
|
||||
last_event_slug: string | null;
|
||||
/** Whole days between last_date and `asof` (default today). NULL when last_date is NULL. */
|
||||
days_ago: number | null;
|
||||
}
|
||||
|
||||
// v0.42.x — Life Chronicle (#2390) per-entity ontology (rides the `facts` table).
|
||||
// An observation is a sourced, confidence-weighted, bi-temporal claim that an
|
||||
// entity has dimension=value (e.g. role=advisor). Supersession/validity/visibility
|
||||
// are inherited from facts columns.
|
||||
export interface OntologyObservationInput {
|
||||
entitySlug: string;
|
||||
dimension: string;
|
||||
value: string;
|
||||
/** 0..1; default 0.7. */
|
||||
confidence?: number;
|
||||
/** Provenance — written to facts.source_markdown_slug (the dedup key + retraction key). */
|
||||
source: string;
|
||||
validFrom?: string | null; // ISO; null = -infinity
|
||||
validTo?: string | null; // ISO; null = open/current
|
||||
visibility?: 'private' | 'world';
|
||||
/** Novel/LLM-proposed dimensions land 'quarantined' (excluded from current resolution). */
|
||||
status?: 'active' | 'quarantined';
|
||||
sourceId?: string;
|
||||
}
|
||||
export interface OntologyMergeResult {
|
||||
action: 'inserted' | 'corroborated' | 'superseded_prior' | 'noop';
|
||||
factId: number | null;
|
||||
supersededId: number | null;
|
||||
}
|
||||
export interface OntologyValue {
|
||||
dimension: string;
|
||||
value: string;
|
||||
confidence: number;
|
||||
source: string | null;
|
||||
valid_from: string | null;
|
||||
valid_to: string | null;
|
||||
status: string; // 'active' | 'quarantined'
|
||||
fact_id: number;
|
||||
}
|
||||
export interface OntologyDimensionStat { dimension: string; entities: number; observations: number }
|
||||
export interface OntologyConflict {
|
||||
entity_slug: string;
|
||||
dimension: string;
|
||||
values: { value: string; source: string | null; confidence: number; fact_id: number }[];
|
||||
}
|
||||
export interface OntologyReadOpts {
|
||||
asof?: string;
|
||||
minConfidence?: number;
|
||||
includeQuarantined?: boolean;
|
||||
sourceId?: string;
|
||||
sourceIds?: string[];
|
||||
}
|
||||
|
||||
// Raw data
|
||||
export interface RawData {
|
||||
source: string;
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
// v0.42.x — Life Chronicle (#2390) feature eval (Phase A.9), the PRIMARY proof.
|
||||
//
|
||||
// Deterministic + CI-safe (no LLM): builds a small synthetic corpus with a KNOWN
|
||||
// gold chronology + a planted ontology supersession + a planted conflict, then
|
||||
// asserts the chronicle layer answers the temporal/longitudinal questions
|
||||
// correctly. This is the "does the feature deliver value" bar from the North
|
||||
// Star — chronology reconstruction, last-seen, supersession time-travel,
|
||||
// contradiction surfacing, source isolation — measured against gold, not vibes.
|
||||
//
|
||||
// The OFF baseline (raw meeting pages) structurally CANNOT order intra-day
|
||||
// events or time-travel ontology; the ON path (chronicle ops) does. We score
|
||||
// the ON path against gold; a perfect score is the proof the ops are correct.
|
||||
import type { BrainEngine } from '../../core/engine.ts';
|
||||
import { runChronicleExtract, type ChronicleJudge } from '../../core/chronicle/extract-events.ts';
|
||||
|
||||
export interface ChronicleEvalTask { id: string; passed: boolean; detail: string }
|
||||
export interface ChronicleEvalResult {
|
||||
tasks: ChronicleEvalTask[];
|
||||
passed: number;
|
||||
total: number;
|
||||
score: number; // passed / total, [0,1]
|
||||
}
|
||||
|
||||
const ALICE = 'people/alice-example';
|
||||
const BOB = 'people/bob-example';
|
||||
|
||||
/** Seed the synthetic corpus into a fresh engine (schema already initialized). */
|
||||
export async function seedChronicleEvalCorpus(engine: BrainEngine): Promise<void> {
|
||||
// Two same-day meetings → two events, gold intra-day order AM then PM.
|
||||
await engine.putPage('meetings/2026-03-02-am', {
|
||||
type: 'meeting', title: 'Morning standup', compiled_truth: 'x'.repeat(120),
|
||||
frontmatter: { attendees: [ALICE] }, effective_date: new Date('2026-03-02T09:00:00Z'),
|
||||
});
|
||||
await engine.putPage('meetings/2026-03-02-pm', {
|
||||
type: 'meeting', title: 'Afternoon review', compiled_truth: 'y'.repeat(120),
|
||||
frontmatter: { attendees: [ALICE] }, effective_date: new Date('2026-03-02T15:00:00Z'),
|
||||
});
|
||||
const judge = (when: string, what: string): ChronicleJudge => async () => ({
|
||||
events: [{ when, who: [ALICE], what, kind: 'meeting' }],
|
||||
});
|
||||
await runChronicleExtract(engine, { slug: 'meetings/2026-03-02-am', judge: judge('2026-03-02T09:00:00Z', 'Morning standup') });
|
||||
await runChronicleExtract(engine, { slug: 'meetings/2026-03-02-pm', judge: judge('2026-03-02T15:00:00Z', 'Afternoon review') });
|
||||
|
||||
// Ontology: alice was a founder, became an advisor (forward supersession).
|
||||
await engine.mergeOntologyFact({ entitySlug: ALICE, dimension: 'role', value: 'founder', source: 'meetings/2024-01-10', validFrom: '2024-01-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: ALICE, dimension: 'role', value: 'advisor', source: 'meetings/2026-03-02-pm', validFrom: '2026-03-01' });
|
||||
|
||||
// Planted conflict: bob has two backdated-vs-forward open values from 2 sources.
|
||||
await engine.mergeOntologyFact({ entitySlug: BOB, dimension: 'role', value: 'advisor', source: 'meetings/a', validFrom: '2026-05-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: BOB, dimension: 'role', value: 'founder', source: 'meetings/b', validFrom: '2026-01-01' });
|
||||
}
|
||||
|
||||
export async function runChronicleEval(engine: BrainEngine): Promise<ChronicleEvalResult> {
|
||||
await seedChronicleEvalCorpus(engine);
|
||||
const tasks: ChronicleEvalTask[] = [];
|
||||
const check = (id: string, ok: boolean, detail: string) => tasks.push({ id, passed: ok, detail });
|
||||
|
||||
// 1. Day reconstruction in correct intra-day order.
|
||||
const day = await engine.getTimelineForDate('2026-03-02', { sourceId: 'default' });
|
||||
const order = day.map((r) => r.summary);
|
||||
check('day_order', JSON.stringify(order) === JSON.stringify(['Morning standup', 'Afternoon review']),
|
||||
`order=${JSON.stringify(order)}`);
|
||||
|
||||
// 2. Last-seen exact date.
|
||||
const ls = await engine.getLastSeen(ALICE, { sourceId: 'default' });
|
||||
check('last_seen', ls.last_date === '2026-03-02', `last_date=${ls.last_date}`);
|
||||
|
||||
// 3. Supersession: current value is the new one.
|
||||
const now = await engine.getOntology(ALICE, { sourceId: 'default' });
|
||||
const roleNow = now.find((v) => v.dimension === 'role')?.value;
|
||||
check('supersession_now', roleNow === 'advisor', `role_now=${roleNow}`);
|
||||
|
||||
// 4. Valid-time travel: as-of before the change returns the prior value.
|
||||
const past = await engine.getOntology(ALICE, { sourceId: 'default', asof: '2025-01-01' });
|
||||
const rolePast = past.find((v) => v.dimension === 'role')?.value;
|
||||
check('supersession_asof', rolePast === 'founder', `role_asof=${rolePast}`);
|
||||
|
||||
// 5. Contradiction surfaced (genuine disagreement, not supersession).
|
||||
const conflicts = await engine.findOntologyConflicts({ sourceId: 'default' });
|
||||
check('conflict_surfaced', conflicts.some((c) => c.entity_slug === BOB && c.dimension === 'role'),
|
||||
`conflicts=${conflicts.length}`);
|
||||
|
||||
// 6. Source isolation: querying another source leaks nothing.
|
||||
const otherSource = await engine.getOntology(ALICE, { sourceId: 'nonexistent-source' });
|
||||
check('source_isolation', otherSource.length === 0, `leaked=${otherSource.length}`);
|
||||
|
||||
const passed = tasks.filter((t) => t.passed).length;
|
||||
return { tasks, passed, total: tasks.length, score: tasks.length ? passed / tasks.length : 0 };
|
||||
}
|
||||
@@ -536,6 +536,13 @@ CREATE TABLE IF NOT EXISTS timeline_entries (
|
||||
source TEXT NOT NULL DEFAULT '',
|
||||
summary TEXT NOT NULL,
|
||||
detail TEXT NOT NULL DEFAULT '',
|
||||
-- v0.42.x (Life Chronicle #2390): when this row is the date-index projection
|
||||
-- of a `type:event` page, event_page_id points at that event page; page_id
|
||||
-- stays the depth/meeting page. NULL for ordinary timeline entries. Reads
|
||||
-- hide rows whose event page is soft-deleted; the partial unique index below
|
||||
-- keys dedup on (event_page_id, date) so re-extraction with a changed summary
|
||||
-- updates the row instead of double-inserting.
|
||||
event_page_id INTEGER REFERENCES pages(id) ON DELETE CASCADE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
@@ -545,6 +552,10 @@ CREATE INDEX IF NOT EXISTS idx_timeline_date ON timeline_entries(date);
|
||||
-- include `source` so distinct meeting provenance survives. Legacy rows
|
||||
-- have source='' (schema default) so legacy dedup behavior is preserved.
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_dedup ON timeline_entries(page_id, date, summary, source);
|
||||
-- v0.42.x (Life Chronicle): event-projection lookup + dedup. Partial
|
||||
-- (event_page_id IS NOT NULL) so ordinary timeline rows are unaffected.
|
||||
CREATE INDEX IF NOT EXISTS idx_timeline_event_page ON timeline_entries(event_page_id) WHERE event_page_id IS NOT NULL;
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_timeline_event_dedup ON timeline_entries(event_page_id, date) WHERE event_page_id IS NOT NULL;
|
||||
|
||||
-- ============================================================
|
||||
-- page_versions: snapshot history for compiled_truth
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) advisor collector (Phase A.7).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { collectChronicle } from '../src/core/advisor/collect-chronicle.ts';
|
||||
import type { AdvisorContext } from '../src/core/advisor/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const ctx = (): AdvisorContext => ({ engine, remote: false } as unknown as AdvisorContext);
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw('DELETE FROM timeline_entries');
|
||||
await engine.executeRaw('DELETE FROM facts');
|
||||
await engine.executeRaw(`DELETE FROM pages WHERE type = 'meeting'`);
|
||||
});
|
||||
|
||||
describe('collectChronicle', () => {
|
||||
test('flags recent meetings with no timeline coverage', async () => {
|
||||
await engine.putPage('meetings/recent', { type: 'meeting', title: 'recent', compiled_truth: 'x'.repeat(120) });
|
||||
const findings = await collectChronicle.collect(ctx());
|
||||
const gap = findings.find((f) => f.id === 'chronicle_coverage_gap');
|
||||
expect(gap).toBeTruthy();
|
||||
expect(gap!.severity).toBe('info');
|
||||
expect(gap!.fix.command_argv).toEqual(['gbrain', 'chronicle-backfill']);
|
||||
});
|
||||
|
||||
test('flags unresolved ontology conflicts', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: 'people/x', dimension: 'role', value: 'advisor', source: 'm/a', validFrom: '2026-05-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: 'people/x', dimension: 'role', value: 'founder', source: 'm/b', validFrom: '2026-01-01' });
|
||||
const findings = await collectChronicle.collect(ctx());
|
||||
const conflict = findings.find((f) => f.id === 'ontology_conflicts');
|
||||
expect(conflict).toBeTruthy();
|
||||
expect(conflict!.severity).toBe('warn');
|
||||
});
|
||||
|
||||
test('no findings on a clean brain', async () => {
|
||||
const findings = await collectChronicle.collect(ctx());
|
||||
expect(findings).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,46 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) backfill op (Phase A.8).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { operationsByName } from '../src/core/operations.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const mkCtx = (): OperationContext => ({ engine, remote: false, sourceId: 'default' } as unknown as OperationContext);
|
||||
const LONG = 'B'.repeat(120);
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw(`DELETE FROM minion_jobs WHERE name = 'chronicle_extract'`);
|
||||
await engine.executeRaw(`DELETE FROM pages WHERE type IN ('meeting','diary')`);
|
||||
});
|
||||
|
||||
describe('chronicle_backfill op', () => {
|
||||
test('dry-run counts eligible meetings without enqueuing', async () => {
|
||||
await engine.putPage('meetings/m1', { type: 'meeting', title: 'm1', compiled_truth: LONG });
|
||||
await engine.putPage('meetings/m2', { type: 'meeting', title: 'm2', compiled_truth: LONG });
|
||||
await engine.putPage('life/diary/d1', { type: 'diary', title: 'd1', compiled_truth: LONG }); // excluded
|
||||
const r = await operationsByName.chronicle_backfill.handler(mkCtx(), { dry_run: true }) as { eligible: number; enqueued: number };
|
||||
expect(r.eligible).toBe(2);
|
||||
expect(r.enqueued).toBe(0);
|
||||
const jobs = await engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM minion_jobs WHERE name='chronicle_extract'`);
|
||||
expect(Number(jobs[0].n)).toBe(0);
|
||||
});
|
||||
|
||||
test('enqueues one chronicle_extract per eligible meeting', async () => {
|
||||
await engine.putPage('meetings/m1', { type: 'meeting', title: 'm1', compiled_truth: LONG });
|
||||
await engine.putPage('meetings/m2', { type: 'meeting', title: 'm2', compiled_truth: LONG });
|
||||
const r = await operationsByName.chronicle_backfill.handler(mkCtx(), {}) as { eligible: number; enqueued: number; errors: unknown[] };
|
||||
expect(r.eligible).toBe(2);
|
||||
expect(r.enqueued).toBe(2);
|
||||
expect(r.errors).toHaveLength(0);
|
||||
const jobs = await engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM minion_jobs WHERE name='chronicle_extract'`);
|
||||
expect(Number(jobs[0].n)).toBe(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,57 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) quick-capture (Phase A.5).
|
||||
* Pure-function tests for the type-routed default slug and the --type event
|
||||
* frontmatter sugar. No engine needed.
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import matter from 'gray-matter';
|
||||
import { __testing } from '../src/commands/capture.ts';
|
||||
const { defaultSlug, mergeCaptureFrontmatter } = __testing;
|
||||
|
||||
const FIXED = new Date('2026-06-18T00:00:00Z');
|
||||
|
||||
describe('defaultSlug type routing', () => {
|
||||
test('diary → life/diary/', () => {
|
||||
expect(defaultSlug('a private thought', FIXED, 'diary')).toMatch(/^life\/diary\/2026-06-18-[0-9a-f]{8}$/);
|
||||
});
|
||||
test('event → life/events/', () => {
|
||||
expect(defaultSlug('an event', FIXED, 'event')).toMatch(/^life\/events\/2026-06-18-[0-9a-f]{8}$/);
|
||||
});
|
||||
test('note / default → inbox/', () => {
|
||||
expect(defaultSlug('a note', FIXED, 'note')).toMatch(/^inbox\/2026-06-18-[0-9a-f]{8}$/);
|
||||
expect(defaultSlug('a note', FIXED)).toMatch(/^inbox\//);
|
||||
});
|
||||
});
|
||||
|
||||
describe('--type event frontmatter sugar', () => {
|
||||
test('builds the event block from flags', () => {
|
||||
const out = mergeCaptureFrontmatter('Sarah committed to Q3', {
|
||||
type: 'event', who: 'people/sarah-chen,people/bob', what: 'committed to Q3',
|
||||
where: 'Zoom', kind: 'commitment', depth: 'meetings/2026-06-18-sync',
|
||||
});
|
||||
const fm = matter(out).data as Record<string, any>;
|
||||
expect(fm.type).toBe('event');
|
||||
expect(fm.event.who).toEqual(['people/sarah-chen', 'people/bob']);
|
||||
expect(fm.event.what).toBe('committed to Q3');
|
||||
expect(fm.event.kind).toBe('commitment');
|
||||
expect(fm.event.depth).toBe('meetings/2026-06-18-sync');
|
||||
});
|
||||
|
||||
test('non-event capture gets no event block', () => {
|
||||
const fm = matter(mergeCaptureFrontmatter('just a note', { type: 'note' })).data as Record<string, any>;
|
||||
expect(fm.event).toBeUndefined();
|
||||
});
|
||||
|
||||
test('diary capture is typed diary, no event block', () => {
|
||||
const fm = matter(mergeCaptureFrontmatter('felt uncertain today', { type: 'diary' })).data as Record<string, any>;
|
||||
expect(fm.type).toBe('diary');
|
||||
expect(fm.event).toBeUndefined();
|
||||
});
|
||||
|
||||
test('user-declared event keys win over flags (per-key merge)', () => {
|
||||
const body = `---\ntype: event\nevent:\n kind: decision\n---\nbody`;
|
||||
const fm = matter(mergeCaptureFrontmatter(body, { type: 'event', kind: 'commitment', what: 'x' })).data as Record<string, any>;
|
||||
expect(fm.event.kind).toBe('decision'); // user wins
|
||||
expect(fm.event.what).toBe('x'); // flag fills the gap
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,35 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) agent-context loader (Phase B.12).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { loadChronicleContext } from '../src/core/context/chronicle-context.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const SARAH = 'people/sarah-chen';
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
// An entity with a public role + a diary-sourced affect.
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'advisor', source: 'meetings/a' });
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'affect', value: 'anxious', source: 'life/diary/2026-06-18', status: 'active' });
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('loadChronicleContext', () => {
|
||||
test('bundles recent timeline + resolved ontology for named entities', async () => {
|
||||
const ctx = await loadChronicleContext(engine, { days: 30, entities: [SARAH], remote: false });
|
||||
expect(ctx.since).toMatch(/^\d{4}-\d{2}-\d{2}$/);
|
||||
expect(Array.isArray(ctx.recent_timeline)).toBe(true);
|
||||
expect(ctx.ontologies[SARAH].map((v) => v.dimension).sort()).toEqual(['affect', 'role']);
|
||||
});
|
||||
|
||||
test('redacts diary-sourced ontology for remote callers', async () => {
|
||||
const ctx = await loadChronicleContext(engine, { entities: [SARAH], remote: true });
|
||||
const dims = (ctx.ontologies[SARAH] ?? []).map((v) => v.dimension);
|
||||
expect(dims).toContain('role');
|
||||
expect(dims).not.toContain('affect'); // diary-sourced → redacted remotely
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,65 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) delight bundle (Phase A.6): on-this-day + narrative.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { renderTimelineNarrative } from '../src/core/chronicle/narrative.ts';
|
||||
import { operationsByName } from '../src/core/operations.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
import type { ChronicleTimelineRow } from '../src/core/types.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const mkCtx = (): OperationContext => ({ engine, remote: false, sourceId: 'default' } as unknown as OperationContext);
|
||||
|
||||
async function seedEntry(slug: string, date: string, summary: string) {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO timeline_entries (page_id, date, source, summary)
|
||||
SELECT id, $1::date, 'test', $2 FROM pages WHERE slug = $3 AND source_id = 'default'`,
|
||||
[date, summary, slug],
|
||||
);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
await engine.putPage('meetings/m', { type: 'meeting', title: 'm', compiled_truth: 'x'.repeat(120) });
|
||||
await seedEntry('meetings/m', '2024-06-15', 'June 15 two years ago');
|
||||
await seedEntry('meetings/m', '2025-06-15', 'June 15 last year');
|
||||
await seedEntry('meetings/m', '2023-01-01', 'unrelated day');
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('renderTimelineNarrative', () => {
|
||||
test('renders prose grouped by date', () => {
|
||||
const rows = [
|
||||
{ date: '2026-06-18', summary: 'Project sync', kind: 'meeting' },
|
||||
{ date: '2026-06-18', summary: 'Vendor call', kind: 'call' },
|
||||
] as ChronicleTimelineRow[];
|
||||
const out = renderTimelineNarrative(rows);
|
||||
expect(out).toContain('2026-06-18 — 2 events');
|
||||
expect(out).toContain('Project sync (meeting)');
|
||||
});
|
||||
test('empty window', () => {
|
||||
expect(renderTimelineNarrative([])).toBe('No events in this window.');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getOnThisDay', () => {
|
||||
test('returns same month-day in prior years, excludes other days', async () => {
|
||||
const rows = await engine.getOnThisDay({ date: '2026-06-15' });
|
||||
const summaries = rows.map((r) => r.summary).sort();
|
||||
expect(summaries).toEqual(['June 15 last year', 'June 15 two years ago']);
|
||||
});
|
||||
|
||||
test('op surfaces on-this-day', async () => {
|
||||
const rows = await operationsByName.chronicle_on_this_day.handler(mkCtx(), { date: '2026-06-15' }) as ChronicleTimelineRow[];
|
||||
expect(rows).toHaveLength(2);
|
||||
});
|
||||
|
||||
test('chronicle_day --narrative returns prose + events', async () => {
|
||||
const r = await operationsByName.chronicle_day.handler(mkCtx(), { date: '2024-06-15', narrative: true }) as { narrative: string; events: ChronicleTimelineRow[] };
|
||||
expect(r.events).toHaveLength(1);
|
||||
expect(r.narrative).toContain('2024-06-15');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,38 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) doctor chronicle_projection_health (Phase B.13).
|
||||
* Verifies the orphan-detection query the doctor check runs: a timeline
|
||||
* projection whose event page is soft-deleted is counted (hidden at read time,
|
||||
* flagged for cleanup).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runChronicleExtract, type ChronicleJudge } from '../src/core/chronicle/extract-events.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
const orphanQuery = `SELECT count(*)::int AS n FROM timeline_entries te
|
||||
JOIN pages ep ON ep.id = te.event_page_id
|
||||
WHERE te.event_page_id IS NOT NULL AND ep.deleted_at IS NOT NULL`;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('chronicle_projection_health detection', () => {
|
||||
test('counts projections whose event page is soft-deleted; zero otherwise', async () => {
|
||||
await engine.putPage('meetings/d', { type: 'meeting', title: 'd', compiled_truth: 'x'.repeat(120), effective_date: new Date('2026-06-18T12:00:00Z') });
|
||||
const judge: ChronicleJudge = async () => ({ events: [{ when: '2026-06-18T12:00:00Z', who: [], what: 'an event', kind: 'meeting' }] });
|
||||
await runChronicleExtract(engine, { slug: 'meetings/d', judge });
|
||||
|
||||
let r = await engine.executeRaw<{ n: number }>(orphanQuery);
|
||||
expect(Number(r[0].n)).toBe(0); // healthy: event page live
|
||||
|
||||
// Soft-delete the event page → its projection becomes an orphan.
|
||||
await engine.executeRaw(`UPDATE pages SET deleted_at = now() WHERE type = 'event'`);
|
||||
r = await engine.executeRaw<{ n: number }>(orphanQuery);
|
||||
expect(Number(r[0].n)).toBe(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,139 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) auto-emit extractor (Phase A.3).
|
||||
* PGLite in-memory. Covers eligibility, the extractor's parse barrier +
|
||||
* idempotent writes (event pages + timeline projection), and the backstop's
|
||||
* auto_chronicle gating + enqueue. The LLM judge is stubbed so the deterministic
|
||||
* write path is tested without a gateway.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { isChronicleEligible } from '../src/core/chronicle/eligibility.ts';
|
||||
import { runChronicleExtract, type ChronicleJudge } from '../src/core/chronicle/extract-events.ts';
|
||||
import { runChronicleBackstop } from '../src/core/chronicle/backstop.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const LONG_BODY = 'A'.repeat(120);
|
||||
|
||||
async function countEvents(): Promise<number> {
|
||||
const r = await engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM pages WHERE type = 'event'`);
|
||||
return Number(r[0].n);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('isChronicleEligible', () => {
|
||||
const body = LONG_BODY;
|
||||
test('meeting is eligible', () => {
|
||||
expect(isChronicleEligible({ type: 'meeting', slug: 'meetings/x', body }).ok).toBe(true);
|
||||
});
|
||||
test('meetings/ slug rescues a note-typed page', () => {
|
||||
expect(isChronicleEligible({ type: 'note', slug: 'meetings/x', body }).ok).toBe(true);
|
||||
});
|
||||
test('diary is excluded (privacy)', () => {
|
||||
expect(isChronicleEligible({ type: 'diary', slug: 'life/diary/x', body })).toEqual({ ok: false, reason: 'diary_excluded' });
|
||||
});
|
||||
test('event is excluded (anti-loop)', () => {
|
||||
expect(isChronicleEligible({ type: 'event', slug: 'life/events/x', body })).toEqual({ ok: false, reason: 'event_self' });
|
||||
});
|
||||
test('dream-generated is excluded', () => {
|
||||
expect(isChronicleEligible({ type: 'meeting', slug: 'meetings/x', body, dreamGenerated: true })).toEqual({ ok: false, reason: 'dream_generated' });
|
||||
});
|
||||
test('too-short body is excluded', () => {
|
||||
expect(isChronicleEligible({ type: 'meeting', slug: 'meetings/x', body: 'hi' })).toEqual({ ok: false, reason: 'too_short' });
|
||||
});
|
||||
test('unrelated type is excluded', () => {
|
||||
expect(isChronicleEligible({ type: 'concept', slug: 'wiki/concepts/x', body })).toEqual({ ok: false, reason: 'kind:concept' });
|
||||
});
|
||||
});
|
||||
|
||||
describe('runChronicleExtract', () => {
|
||||
const oneEvent: ChronicleJudge = async () => ({
|
||||
events: [{ when: '2026-06-18T15:30:00Z', who: ['people/sarah-chen'], what: 'Sarah committed to Q3', kind: 'commitment' }],
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await engine.executeRaw('DELETE FROM timeline_entries');
|
||||
await engine.executeRaw(`DELETE FROM pages WHERE type = 'event' OR slug = 'meetings/2026-06-18-sync'`);
|
||||
await engine.putPage('meetings/2026-06-18-sync', {
|
||||
type: 'meeting', title: 'Weekly sync',
|
||||
compiled_truth: LONG_BODY,
|
||||
frontmatter: { attendees: ['people/sarah-chen'] },
|
||||
effective_date: new Date('2026-06-18T15:00:00Z'),
|
||||
});
|
||||
});
|
||||
|
||||
test('writes an event page + timeline projection', async () => {
|
||||
const r = await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: oneEvent });
|
||||
expect(r.status).toBe('extracted');
|
||||
expect(r.events_written).toBe(1);
|
||||
expect(await countEvents()).toBe(1);
|
||||
const day = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
|
||||
expect(day.length).toBe(1);
|
||||
expect(day[0].summary).toBe('Sarah committed to Q3');
|
||||
expect(day[0].page_slug).toBe('meetings/2026-06-18-sync'); // projection keyed to depth
|
||||
expect(day[0].event_slug?.startsWith('life/events/2026-06-18-')).toBe(true);
|
||||
expect(day[0].kind).toBe('commitment');
|
||||
});
|
||||
|
||||
test('is idempotent: running twice yields one event + one projection', async () => {
|
||||
await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: oneEvent });
|
||||
await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: oneEvent });
|
||||
expect(await countEvents()).toBe(1);
|
||||
const day = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
|
||||
expect(day.length).toBe(1);
|
||||
});
|
||||
|
||||
test('parse barrier: a malformed proposal writes NOTHING', async () => {
|
||||
const before = await countEvents();
|
||||
const bad: ChronicleJudge = async () => ({ events: [{ when: '2026-06-18', who: [], kind: 'x' } as never] });
|
||||
const r = await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: bad });
|
||||
expect(r.status).toBe('skipped');
|
||||
expect(r.reason).toBe('malformed_proposal');
|
||||
expect(await countEvents()).toBe(before); // no partial write
|
||||
});
|
||||
|
||||
test('parse barrier: a non-date `when` writes NOTHING (codex fix #2)', async () => {
|
||||
const before = await countEvents();
|
||||
const badDate: ChronicleJudge = async () => ({ events: [{ when: 'not-a-date', who: [], what: 'x', kind: 'meeting' }] });
|
||||
const r = await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: badDate });
|
||||
expect(r.status).toBe('skipped');
|
||||
expect(r.reason).toBe('malformed_proposal');
|
||||
expect(await countEvents()).toBe(before);
|
||||
});
|
||||
|
||||
test('no events → no_events status', async () => {
|
||||
const none: ChronicleJudge = async () => ({ events: [] });
|
||||
const r = await runChronicleExtract(engine, { slug: 'meetings/2026-06-18-sync', judge: none });
|
||||
expect(r.status).toBe('no_events');
|
||||
});
|
||||
});
|
||||
|
||||
describe('runChronicleBackstop gating', () => {
|
||||
beforeEach(async () => {
|
||||
await engine.unsetConfig('auto_chronicle');
|
||||
await engine.putPage('meetings/bs', { type: 'meeting', title: 'bs', compiled_truth: LONG_BODY });
|
||||
});
|
||||
|
||||
test('skips when auto_chronicle is off (default)', async () => {
|
||||
const r = await runChronicleBackstop({ slug: 'meetings/bs', type: 'meeting', compiled_truth: LONG_BODY }, { engine, sourceId: 'default' });
|
||||
expect(r).toEqual({ enqueued: false, skipped: 'auto_chronicle_off' });
|
||||
});
|
||||
|
||||
test('skips a diary page before consulting the flag', async () => {
|
||||
const r = await runChronicleBackstop({ slug: 'life/diary/x', type: 'diary', compiled_truth: LONG_BODY }, { engine, sourceId: 'default' });
|
||||
expect(r).toEqual({ enqueued: false, skipped: 'diary_excluded' });
|
||||
});
|
||||
|
||||
test('enqueues a chronicle_extract job when enabled + eligible', async () => {
|
||||
await engine.setConfig('auto_chronicle', 'true');
|
||||
const r = await runChronicleBackstop({ slug: 'meetings/bs', type: 'meeting', compiled_truth: LONG_BODY }, { engine, sourceId: 'default' });
|
||||
expect(r.enqueued).toBe(true);
|
||||
const jobs = await engine.executeRaw<{ n: number }>(`SELECT count(*)::int AS n FROM minion_jobs WHERE name = 'chronicle_extract'`);
|
||||
expect(Number(jobs[0].n)).toBeGreaterThanOrEqual(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) ontology ops (Phase B.11).
|
||||
* Exercises the op handlers (not just the engine), with focus on the
|
||||
* privacy-critical remote diary-source redaction in ontology_get.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { operationsByName } from '../src/core/operations.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const SARAH = 'people/sarah-chen';
|
||||
const ctx = (remote: boolean): OperationContext => ({ engine, remote, sourceId: 'default' } as unknown as OperationContext);
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => { await engine.executeRaw('DELETE FROM facts'); });
|
||||
|
||||
describe('ontology ops', () => {
|
||||
test('ontology_propose writes; ontology_get reads back', async () => {
|
||||
const r = await operationsByName.ontology_propose.handler(ctx(false), {
|
||||
entity: SARAH, dimension: 'role', value: 'advisor', source: 'meetings/a',
|
||||
});
|
||||
expect((r as { action: string }).action).toBe('inserted');
|
||||
const got = await operationsByName.ontology_get.handler(ctx(false), { entity: SARAH }) as { dimension: string; value: string }[];
|
||||
expect(got).toHaveLength(1);
|
||||
expect(got[0]).toMatchObject({ dimension: 'role', value: 'advisor' });
|
||||
});
|
||||
|
||||
test('remote callers never see diary-sourced ontology', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'advisor', source: 'meetings/a' });
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'affect', value: 'anxious', source: 'life/diary/2026-06-18', status: 'active' });
|
||||
const local = await operationsByName.ontology_get.handler(ctx(false), { entity: SARAH, include_quarantined: true }) as { source: string }[];
|
||||
const remote = await operationsByName.ontology_get.handler(ctx(true), { entity: SARAH, include_quarantined: true }) as { source: string }[];
|
||||
expect(local.some((r) => (r.source ?? '').startsWith('life/diary/'))).toBe(true);
|
||||
expect(remote.some((r) => (r.source ?? '').startsWith('life/diary/'))).toBe(false);
|
||||
});
|
||||
|
||||
test('remote ontology_conflicts redacts diary-sourced disagreement (codex fix #3)', async () => {
|
||||
// A conflict where one side is diary-sourced: advisor (meeting) vs founder (diary).
|
||||
await engine.mergeOntologyFact({ entitySlug: 'people/y', dimension: 'role', value: 'advisor', source: 'meetings/a', validFrom: '2026-05-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: 'people/y', dimension: 'role', value: 'founder', source: 'life/diary/2026-06-18', validFrom: '2026-01-01' });
|
||||
const local = await operationsByName.ontology_conflicts.handler(ctx(false), {}) as { entity_slug: string }[];
|
||||
expect(local.some((c) => c.entity_slug === 'people/y')).toBe(true);
|
||||
const remote = await operationsByName.ontology_conflicts.handler(ctx(true), {}) as { entity_slug: string }[];
|
||||
expect(remote.some((c) => c.entity_slug === 'people/y')).toBe(false); // diary side redacted → no longer a disagreement
|
||||
});
|
||||
|
||||
test('ontology_dimensions + ontology_conflicts surface via ops', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'advisor', source: 'meetings/a', validFrom: '2026-05-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/b', validFrom: '2026-01-01' });
|
||||
const dims = await operationsByName.ontology_dimensions.handler(ctx(false), {}) as { dimension: string }[];
|
||||
expect(dims.some((d) => d.dimension === 'role')).toBe(true);
|
||||
const conflicts = await operationsByName.ontology_conflicts.handler(ctx(false), {}) as { dimension: string }[];
|
||||
expect(conflicts.some((c) => c.dimension === 'role')).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,108 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) bi-temporal ontology on `facts` (Phase B.10).
|
||||
* PGLite in-memory. Covers insert / idempotent retry / corroboration / forward
|
||||
* supersession / --asof valid-time travel / quarantine / backdated-conflict
|
||||
* detection. The Postgres engine runs the identical SQL (see engine-parity).
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll, beforeEach } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const SARAH = 'people/sarah-chen';
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
beforeEach(async () => { await engine.executeRaw('DELETE FROM facts'); });
|
||||
|
||||
describe('mergeOntologyFact + getOntology', () => {
|
||||
test('inserts a known dimension as active', async () => {
|
||||
const r = await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a', validFrom: '2024-01-01' });
|
||||
expect(r.action).toBe('inserted');
|
||||
const o = await engine.getOntology(SARAH);
|
||||
expect(o).toHaveLength(1);
|
||||
expect(o[0]).toMatchObject({ dimension: 'role', value: 'founder', status: 'active' });
|
||||
});
|
||||
|
||||
test('idempotent retry (same value + source) is a noop', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a' });
|
||||
const r = await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a' });
|
||||
expect(r.action).toBe('noop');
|
||||
expect(await engine.getOntology(SARAH)).toHaveLength(1);
|
||||
});
|
||||
|
||||
test('same value from a new source corroborates (still one current value)', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a' });
|
||||
const r = await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/b' });
|
||||
expect(r.action).toBe('corroborated');
|
||||
const o = await engine.getOntology(SARAH);
|
||||
expect(o).toHaveLength(1);
|
||||
expect(o[0].value).toBe('founder');
|
||||
});
|
||||
|
||||
test('normalizes dimension aliases (job_role → role)', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'job_role', value: 'advisor', source: 'meetings/a' });
|
||||
const o = await engine.getOntology(SARAH);
|
||||
expect(o[0].dimension).toBe('role');
|
||||
});
|
||||
|
||||
test('forward supersession + --asof valid-time travel', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a', validFrom: '2024-01-01' });
|
||||
const r = await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'advisor', source: 'meetings/b', validFrom: '2026-05-01' });
|
||||
expect(r.action).toBe('superseded_prior');
|
||||
// Current value is the new one…
|
||||
const now = await engine.getOntology(SARAH);
|
||||
expect(now[0].value).toBe('advisor');
|
||||
// …but as-of before the change, time-travel still sees the old one.
|
||||
const past = await engine.getOntology(SARAH, { asof: '2025-01-01' });
|
||||
expect(past[0].value).toBe('founder');
|
||||
});
|
||||
|
||||
test('novel dimensions quarantine (excluded from current unless asked)', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'vibe', value: 'chaotic', source: 'meetings/a' });
|
||||
expect(await engine.getOntology(SARAH)).toHaveLength(0); // quarantined → hidden
|
||||
const incl = await engine.getOntology(SARAH, { includeQuarantined: true });
|
||||
expect(incl).toHaveLength(1);
|
||||
expect(incl[0].status).toBe('quarantined');
|
||||
});
|
||||
});
|
||||
|
||||
describe('findOntologyConflicts + discoverOntologyDimensions', () => {
|
||||
test('backdated conflicting value is NOT rewritten and surfaces as a conflict', async () => {
|
||||
const BOB = 'people/bob';
|
||||
await engine.mergeOntologyFact({ entitySlug: BOB, dimension: 'role', value: 'advisor', source: 'meetings/a', validFrom: '2026-05-01' });
|
||||
const r = await engine.mergeOntologyFact({ entitySlug: BOB, dimension: 'role', value: 'founder', source: 'meetings/b', validFrom: '2026-01-01' });
|
||||
expect(r.action).toBe('inserted'); // backdated → not a supersession
|
||||
const conflicts = await engine.findOntologyConflicts();
|
||||
const bobRole = conflicts.find((c) => c.entity_slug === BOB && c.dimension === 'role');
|
||||
expect(bobRole).toBeTruthy();
|
||||
expect(bobRole!.values.map((v) => v.value).sort()).toEqual(['advisor', 'founder']);
|
||||
});
|
||||
|
||||
test('forward supersession is NOT reported as a conflict (only live disagreement is)', async () => {
|
||||
// founder (2024) → advisor (2026): the founder row is closed via valid_until,
|
||||
// so it must NOT count as a current conflict (codex pre-landing fix #1).
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'm/a', validFrom: '2024-01-01' });
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'advisor', source: 'm/b', validFrom: '2026-05-01' });
|
||||
const conflicts = await engine.findOntologyConflicts();
|
||||
expect(conflicts.some((c) => c.entity_slug === SARAH && c.dimension === 'role')).toBe(false);
|
||||
});
|
||||
|
||||
test('discoverOntologyDimensions rolls up by dimension', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a' });
|
||||
await engine.mergeOntologyFact({ entitySlug: 'people/bob', dimension: 'role', value: 'advisor', source: 'meetings/b' });
|
||||
const dims = await engine.discoverOntologyDimensions();
|
||||
const role = dims.find((d) => d.dimension === 'role');
|
||||
expect(role).toBeTruthy();
|
||||
expect(role!.entities).toBe(2);
|
||||
});
|
||||
|
||||
test('source isolation: ontology is scoped by source_id', async () => {
|
||||
await engine.mergeOntologyFact({ entitySlug: SARAH, dimension: 'role', value: 'founder', source: 'meetings/a', sourceId: 'default' });
|
||||
expect(await engine.getOntology(SARAH, { sourceId: 'default' })).toHaveLength(1);
|
||||
expect(await engine.getOntology(SARAH, { sourceId: 'other' })).toHaveLength(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,43 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) E1 temporal recall boost (Phase A.4).
|
||||
* Pure-function test of applyChronicleTypeBoost: boosts event/diary on temporal
|
||||
* queries, leaves everything else untouched (the no-regression guarantee — the
|
||||
* function is only CALLED when recency !== 'off', and even then only moves
|
||||
* chronicle types).
|
||||
*/
|
||||
import { describe, test, expect } from 'bun:test';
|
||||
import { applyChronicleTypeBoost } from '../src/core/search/hybrid.ts';
|
||||
import type { SearchResult } from '../src/core/types.ts';
|
||||
|
||||
function r(slug: string, type: string, score = 1.0): SearchResult {
|
||||
return {
|
||||
slug, page_id: 1, title: slug, type, chunk_text: '', chunk_source: 'compiled_truth',
|
||||
chunk_id: 1, chunk_index: 0, score, stale: false,
|
||||
} as SearchResult;
|
||||
}
|
||||
|
||||
describe('applyChronicleTypeBoost', () => {
|
||||
test('boosts event + diary, leaves other types untouched', () => {
|
||||
const rows = [r('life/events/a', 'event'), r('life/diary/b', 'diary'), r('wiki/note', 'note'), r('people/x', 'person')];
|
||||
applyChronicleTypeBoost(rows, 'on');
|
||||
expect(rows[0].score).toBeCloseTo(1.15);
|
||||
expect(rows[0].chronicle_boost).toBeCloseTo(1.15);
|
||||
expect(rows[1].score).toBeCloseTo(1.15);
|
||||
expect(rows[2].score).toBe(1.0); // note unchanged
|
||||
expect(rows[2].chronicle_boost).toBeUndefined();
|
||||
expect(rows[3].score).toBe(1.0); // person unchanged
|
||||
});
|
||||
|
||||
test('strong uses a larger factor', () => {
|
||||
const rows = [r('life/events/a', 'event')];
|
||||
applyChronicleTypeBoost(rows, 'strong');
|
||||
expect(rows[0].score).toBeCloseTo(1.25);
|
||||
});
|
||||
|
||||
test('floor gate skips low-score results', () => {
|
||||
const rows = [r('life/events/a', 'event', 0.1)];
|
||||
applyChronicleTypeBoost(rows, 'on', 0.5); // 0.1 < floor 0.5 → skipped
|
||||
expect(rows[0].score).toBe(0.1);
|
||||
expect(rows[0].chronicle_boost).toBeUndefined();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,118 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) timeline read methods (Phase A.2).
|
||||
* Runs against PGLite in-memory. Covers getTimelineForDate (day + ISO week),
|
||||
* getSince (+ kind filter), getLastSeen (own page + via event `who`),
|
||||
* intra-day ordering by event effective_date, source isolation, read-time
|
||||
* hiding of soft-deleted event projections, and the (event_page_id, date)
|
||||
* dedup index.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
const ids: Record<string, number> = {};
|
||||
|
||||
async function insertPage(opts: {
|
||||
slug: string; type: string; sourceId?: string;
|
||||
effectiveDate?: string | null; frontmatter?: string;
|
||||
}): Promise<number> {
|
||||
const rows = await engine.executeRaw<{ id: number }>(
|
||||
`INSERT INTO pages (source_id, slug, type, title, effective_date, frontmatter)
|
||||
VALUES ($1, $2, $3, $4, $5::timestamptz, $6::text::jsonb)
|
||||
RETURNING id`,
|
||||
[opts.sourceId ?? 'default', opts.slug, opts.type, opts.slug,
|
||||
opts.effectiveDate ?? null, opts.frontmatter ?? '{}'],
|
||||
);
|
||||
return rows[0].id;
|
||||
}
|
||||
|
||||
async function insertProjection(depthId: number, eventId: number, date: string, summary: string): Promise<void> {
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO timeline_entries (page_id, date, source, summary, detail, event_page_id)
|
||||
VALUES ($1, $2::date, $3, $4, '', $5)`,
|
||||
[depthId, date, `life-chronicle:event:${eventId}`, summary, eventId],
|
||||
);
|
||||
}
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
|
||||
ids.meeting = await insertPage({ slug: 'meetings/2026-06-18-sync', type: 'meeting' });
|
||||
await insertPage({ slug: 'people/sarah-chen', type: 'person' });
|
||||
// 06-18: event2 at 09:00 (Bob), event1 at 15:30 (Sarah, commitment).
|
||||
ids.e1 = await insertPage({ slug: 'life/events/2026-06-18-001', type: 'event', effectiveDate: '2026-06-18T15:30:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"commitment"}}' });
|
||||
ids.e2 = await insertPage({ slug: 'life/events/2026-06-18-002', type: 'event', effectiveDate: '2026-06-18T09:00:00Z', frontmatter: '{"event":{"who":["people/bob"],"kind":"meeting"}}' });
|
||||
// 06-20 (same ISO week as 06-18): event3 (Sarah, decision).
|
||||
ids.e3 = await insertPage({ slug: 'life/events/2026-06-20-001', type: 'event', effectiveDate: '2026-06-20T10:00:00Z', frontmatter: '{"event":{"who":["people/sarah-chen"],"kind":"decision"}}' });
|
||||
await insertProjection(ids.meeting, ids.e1, '2026-06-18', 'Sarah committed to Q3');
|
||||
await insertProjection(ids.meeting, ids.e2, '2026-06-18', 'Bob standup');
|
||||
await insertProjection(ids.meeting, ids.e3, '2026-06-20', 'Decision on launch');
|
||||
|
||||
// Other-source event (isolation fixture).
|
||||
await engine.executeRaw(`INSERT INTO sources (id, name) VALUES ('other', 'Other') ON CONFLICT (id) DO NOTHING`);
|
||||
const otherMeeting = await insertPage({ slug: 'meetings/o', type: 'meeting', sourceId: 'other' });
|
||||
ids.eOther = await insertPage({ slug: 'life/events/2026-06-18-099', type: 'event', sourceId: 'other', effectiveDate: '2026-06-18T12:00:00Z', frontmatter: '{"event":{"who":["people/zed"],"kind":"meeting"}}' });
|
||||
await insertProjection(otherMeeting, ids.eOther, '2026-06-18', 'Other-source event');
|
||||
});
|
||||
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('Life Chronicle timeline reads', () => {
|
||||
test('getTimelineForDate orders intra-day by event effective_date', async () => {
|
||||
const rows = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
|
||||
expect(rows.map(r => r.event_slug)).toEqual([
|
||||
'life/events/2026-06-18-002', // 09:00 first
|
||||
'life/events/2026-06-18-001', // 15:30 second
|
||||
]);
|
||||
expect(rows[0].page_slug).toBe('meetings/2026-06-18-sync'); // backlink = depth page
|
||||
expect(rows[1].kind).toBe('commitment');
|
||||
});
|
||||
|
||||
test('week expansion pulls the whole ISO week', async () => {
|
||||
const day = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
|
||||
const week = await engine.getTimelineForDate('2026-06-18', { week: true, sourceId: 'default' });
|
||||
expect(day.length).toBe(2);
|
||||
expect(week.length).toBe(3); // adds 06-20 (same Mon–Sun week)
|
||||
expect(week[week.length - 1].event_slug).toBe('life/events/2026-06-20-001');
|
||||
});
|
||||
|
||||
test('getSince filters by lower-bound date and event kind', async () => {
|
||||
const since19 = await engine.getSince('2026-06-19', { sourceId: 'default' });
|
||||
expect(since19.map(r => r.event_slug)).toEqual(['life/events/2026-06-20-001']);
|
||||
const commitments = await engine.getSince('2026-06-18', { kind: 'commitment', sourceId: 'default' });
|
||||
expect(commitments.map(r => r.event_slug)).toEqual(['life/events/2026-06-18-001']);
|
||||
});
|
||||
|
||||
test('getLastSeen finds the entity via event who, with days_ago', async () => {
|
||||
const seen = await engine.getLastSeen('people/sarah-chen', { asof: '2026-06-25', sourceId: 'default' });
|
||||
expect(seen.last_date).toBe('2026-06-20');
|
||||
expect(seen.last_event_slug).toBe('life/events/2026-06-20-001');
|
||||
expect(seen.days_ago).toBe(5);
|
||||
const never = await engine.getLastSeen('people/nobody', { asof: '2026-06-25', sourceId: 'default' });
|
||||
expect(never.last_date).toBeNull();
|
||||
expect(never.days_ago).toBeNull();
|
||||
});
|
||||
|
||||
test('source isolation: default scope excludes other-source events', async () => {
|
||||
const def = await engine.getTimelineForDate('2026-06-18', { sourceId: 'default' });
|
||||
expect(def.some(r => r.event_slug === 'life/events/2026-06-18-099')).toBe(false);
|
||||
const other = await engine.getTimelineForDate('2026-06-18', { sourceId: 'other' });
|
||||
expect(other.map(r => r.event_slug)).toEqual(['life/events/2026-06-18-099']);
|
||||
});
|
||||
|
||||
test('(event_page_id, date) dedup index rejects a duplicate projection', async () => {
|
||||
await expect(
|
||||
insertProjection(ids.meeting, ids.e1, '2026-06-18', 'dup'),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
test('soft-deleting an event page hides it from reads (read-time, not doctor)', async () => {
|
||||
await engine.executeRaw('UPDATE pages SET deleted_at = now() WHERE id = $1', [ids.e3]);
|
||||
const since19 = await engine.getSince('2026-06-19', { sourceId: 'default' });
|
||||
expect(since19.length).toBe(0); // event3 hidden
|
||||
const seen = await engine.getLastSeen('people/sarah-chen', { asof: '2026-06-25', sourceId: 'default' });
|
||||
expect(seen.last_date).toBe('2026-06-18'); // falls back to event1
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,25 @@
|
||||
/**
|
||||
* v0.42.x — Life Chronicle (#2390) feature eval harness (Phase A.9, PRIMARY proof).
|
||||
* The chronicle layer must answer EVERY gold task on the synthetic corpus.
|
||||
*/
|
||||
import { describe, test, expect, beforeAll, afterAll } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { runChronicleEval } from '../src/eval/chronicle/harness.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({ database_url: '' });
|
||||
await engine.initSchema();
|
||||
});
|
||||
afterAll(async () => { await engine.disconnect(); });
|
||||
|
||||
describe('runChronicleEval', () => {
|
||||
test('scores a perfect 6/6 on the synthetic corpus (the value proof)', async () => {
|
||||
const result = await runChronicleEval(engine);
|
||||
const failed = result.tasks.filter((t) => !t.passed).map((t) => `${t.id}: ${t.detail}`);
|
||||
expect(failed).toEqual([]); // surfaces which gold task failed, if any
|
||||
expect(result.score).toBe(1);
|
||||
expect(result.total).toBe(6);
|
||||
});
|
||||
});
|
||||
@@ -31,9 +31,19 @@ import { join } from 'node:path';
|
||||
// - consolidate.ts (v0.35.4 — chronological writeback)
|
||||
// - facts/forget.ts (v0.32.2 — user-initiated `gbrain forget`; user is
|
||||
// the supersession authority, not the probe)
|
||||
// - postgres-engine.ts + pglite-engine.ts (v0.42.56.0, #2390 — Life
|
||||
// Chronicle ontology: `mergeOntologyFact` forward-supersession closes
|
||||
// the prior OPEN row's valid_until when a NEW value arrives for the
|
||||
// same (entity, dimension). Engine-layer, caller-requested, scoped to
|
||||
// `dimension IS NOT NULL` ontology rows only — plain facts untouched,
|
||||
// and the contradiction probe still never mutates, so the
|
||||
// auto-supersession.ts:4 invariant is preserved. Deliberate design
|
||||
// change per the #2390 eng review (G1: ontology extends facts).
|
||||
const VALID_UNTIL_WRITE_ALLOWLIST: ReadonlySet<string> = new Set([
|
||||
'src/core/cycle/phases/consolidate.ts',
|
||||
'src/core/facts/forget.ts',
|
||||
'src/core/postgres-engine.ts',
|
||||
'src/core/pglite-engine.ts',
|
||||
]);
|
||||
|
||||
function walkTs(dir: string, acc: string[] = []): string[] {
|
||||
|
||||
@@ -39,6 +39,9 @@ const PARITY_FIXTURES: ReadonlyArray<{ path: string; expected: string; reason: s
|
||||
{ path: 'emails/em-0001.md', expected: 'email', reason: 'emails/ prefix' },
|
||||
{ path: 'slack/sl-0037.md', expected: 'slack', reason: 'slack/ prefix' },
|
||||
{ path: 'cal/2026-05-20.md', expected: 'calendar-event', reason: 'cal/ prefix' },
|
||||
// v0.42.x — Life Chronicle (#2390): timeline events + thought diary.
|
||||
{ path: 'life/events/2026-06-18-002-sync.md', expected: 'event', reason: 'life/events/ prefix' },
|
||||
{ path: 'life/diary/2026-06-18-am.md', expected: 'diary', reason: 'life/diary/ prefix' },
|
||||
// Stronger-signal wins: writing/ inside projects/
|
||||
{ path: 'projects/blog/writing/essay.md', expected: 'writing', reason: 'writing/ wins over projects/' },
|
||||
// Fallback: paths not matching any prefix
|
||||
|
||||
@@ -82,13 +82,21 @@ describe('checkTypeProliferation (D16 pack-aware ratio)', () => {
|
||||
});
|
||||
|
||||
it('warns when distinct types exceed declared+5', async () => {
|
||||
// gbrain-base declares 24 types. warn threshold = 29.
|
||||
// Threshold-relative (v0.42.56.0): compute `declared` from the active pack
|
||||
// the same way checkTypeProliferation does, then seed declared+6 so the
|
||||
// test keeps passing when the base pack grows (e.g. #2390 added
|
||||
// event + diary and silently moved the fixed threshold).
|
||||
const { loadActivePack } = await import('../src/core/schema-pack/load-active.ts');
|
||||
const dbConfig = (await engine.getConfig('schema_pack')) ?? undefined;
|
||||
const active = await loadActivePack({ cfg: null, remote: false, dbConfig }).catch(() => null);
|
||||
const declared = active ? active.manifest.page_types.length : 15;
|
||||
const seedCount = declared + 6; // one past the warn threshold (declared+5)
|
||||
const types: string[] = [];
|
||||
for (let i = 0; i < 32; i++) types.push(`custom-type-${i}`);
|
||||
for (let i = 0; i < seedCount; i++) types.push(`custom-type-${i}`);
|
||||
await seedPages(types);
|
||||
const result = await checkTypeProliferation(engine);
|
||||
expect(result.check.status).toBe('warn');
|
||||
expect(result.check.message).toMatch(/32 distinct/);
|
||||
expect(result.check.message).toMatch(new RegExp(`${seedCount} distinct`));
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -51,6 +51,22 @@ describe('classifyPgliteInitError', () => {
|
||||
test('case-insensitive matching on bunfs marker', () => {
|
||||
expect(classifyPgliteInitError('SYSCALL ENOENT on /$$BUNFS/root')).toBe('bunfs');
|
||||
});
|
||||
|
||||
// #2348 — corrupted PGLite data dir (concurrent open trashed catalog/extension).
|
||||
test('corrupt verdict for the 58P01 internal_load_library signature', () => {
|
||||
const msg = 'error: relation "content_chunks" does not exist\n code: 58P01\n file: "dfmgr.c"\n routine: "internal_load_library"';
|
||||
expect(classifyPgliteInitError(msg)).toBe('corrupt');
|
||||
});
|
||||
|
||||
test('corrupt verdict when the vector type can no longer load', () => {
|
||||
expect(classifyPgliteInitError('type "vector" does not exist')).toBe('corrupt');
|
||||
});
|
||||
|
||||
test('corrupt verdict beats the wasm-runtime match (58P01 wins over "wasm runtime")', () => {
|
||||
// A message mentioning both must classify as corrupt, not macos-26-3 —
|
||||
// recovery guidance, not the wrong macOS-WASM hint.
|
||||
expect(classifyPgliteInitError('wasm runtime: 58P01 internal_load_library')).toBe('corrupt');
|
||||
});
|
||||
});
|
||||
|
||||
describe('buildPgliteInitErrorMessage — hint routing', () => {
|
||||
@@ -80,8 +96,16 @@ describe('buildPgliteInitErrorMessage — hint routing', () => {
|
||||
expect(msg).toContain(original);
|
||||
});
|
||||
|
||||
test('corrupt verdict surfaces the reinit-pglite recovery, NOT the macOS hint', () => {
|
||||
const msg = buildPgliteInitErrorMessage('corrupt', original);
|
||||
expect(msg).toContain('gbrain reinit-pglite');
|
||||
expect(msg).toContain('corrupted');
|
||||
expect(msg).toContain(original);
|
||||
expect(msg).not.toContain('issues/223');
|
||||
});
|
||||
|
||||
test('all verdicts produce the canonical header line', () => {
|
||||
for (const v of ['bunfs', 'macos-26-3', 'unknown'] as const) {
|
||||
for (const v of ['bunfs', 'macos-26-3', 'corrupt', 'unknown'] as const) {
|
||||
const msg = buildPgliteInitErrorMessage(v, original);
|
||||
expect(msg.startsWith('PGLite failed to initialize its WASM runtime.')).toBe(true);
|
||||
}
|
||||
|
||||
+10
-18
@@ -3,7 +3,6 @@ import { mkdirSync, rmSync, existsSync, readFileSync, writeFileSync } from 'fs';
|
||||
import { join } from 'path';
|
||||
import { tmpdir } from 'os';
|
||||
import { acquireLock, releaseLock, type LockHandle } from '../src/core/pglite-lock';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
const TEST_DIR = join(tmpdir(), 'gbrain-lock-test-' + process.pid);
|
||||
|
||||
@@ -133,25 +132,18 @@ describe('pglite-lock #2058 heartbeat + steal-grace', () => {
|
||||
expect(existsSync(join(TEST_DIR, '.gbrain-lock'))).toBe(true);
|
||||
});
|
||||
|
||||
test('a LIVE PID whose heartbeat went stale past the grace window IS reaped', async () => {
|
||||
// PID is alive (our own) but hasn't refreshed in 20min (> 600s grace):
|
||||
// hung holder, or a reused PID whose real holder is gone. Reap + acquire.
|
||||
test('[REGRESSION #2348] a LIVE PID with a STALE heartbeat is NOT stolen', async () => {
|
||||
// The #2348 corruption: a live `gbrain dream`/embed holder whose heartbeat
|
||||
// lapsed (the JS event loop is blocked during a long synchronous WASM
|
||||
// import) used to get its lock reaped past the grace window — letting a
|
||||
// second OS process open the same data dir and corrupt the catalog +
|
||||
// pgvector extension state. A live PID is now NEVER stolen, regardless of
|
||||
// how stale its heartbeat is. Acquire must time out, not steal.
|
||||
writeHolder({ pid: process.pid, acquiredAgoMs: 25 * 60_000, refreshedAgoMs: 20 * 60_000 });
|
||||
|
||||
const lock = await acquireLock(TEST_DIR, { timeoutMs: 2000 });
|
||||
expect(lock.acquired).toBe(true);
|
||||
await releaseLock(lock);
|
||||
});
|
||||
|
||||
test('GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS tunes the grace window', async () => {
|
||||
// withEnv keeps the process-global mutation isolated across shard files.
|
||||
await withEnv({ GBRAIN_PGLITE_LOCK_STEAL_GRACE_SECONDS: '5' }, async () => {
|
||||
// Refreshed 30s ago — fresh under the 600s default, STALE under 5s.
|
||||
writeHolder({ pid: process.pid, acquiredAgoMs: 60_000, refreshedAgoMs: 30_000 });
|
||||
const lock = await acquireLock(TEST_DIR, { timeoutMs: 2000 });
|
||||
expect(lock.acquired).toBe(true);
|
||||
await releaseLock(lock);
|
||||
});
|
||||
await expect(acquireLock(TEST_DIR, { timeoutMs: 1200 })).rejects.toThrow(/Timed out/);
|
||||
// The live holder's lock is still present — never force-removed.
|
||||
expect(existsSync(join(TEST_DIR, '.gbrain-lock'))).toBe(true);
|
||||
});
|
||||
|
||||
test('[REGRESSION] releaseLock does NOT remove a lock that was stolen + re-acquired by another process', async () => {
|
||||
|
||||
@@ -730,6 +730,17 @@ const COLUMN_EXEMPTIONS = new Set<string>([
|
||||
// brains is invisible to them). Migration is column-only, no FK,
|
||||
// no index — bootstrap probe would be pure overhead.
|
||||
'facts.event_type',
|
||||
// v0.42.56.0 (migration v122, #2390) — Life Chronicle ontology columns.
|
||||
// Same precedent as facts.claim_metric et al: the `facts` table itself is
|
||||
// migration-created (absent from PGLITE_SCHEMA_SQL), so no schema-blob
|
||||
// forward reference can exist; the partial indexes (idx_facts_dimension,
|
||||
// idx_facts_ontology_dedup) live INSIDE the same v122 migration. Every
|
||||
// reader filters `dimension IS NOT NULL`, so NULL on old brains is
|
||||
// invisible. Column-only, no bootstrap probe needed.
|
||||
'facts.dimension',
|
||||
'facts.value',
|
||||
'facts.value_hash',
|
||||
'facts.dim_status',
|
||||
// v0.39.1.0 (migration v88) — schema-pack provenance per-source captured as
|
||||
// inline canonical closure snapshot on every eval_candidates row. NULL by
|
||||
// default; no index in PGLITE_SCHEMA_SQL references it. Migration handles
|
||||
|
||||
@@ -73,7 +73,11 @@ describe('gbrain schema CLI (Phase C)', () => {
|
||||
// `conversation` and `atom` into gbrain-base.
|
||||
// v0.41.23.0: extended to 25 by adding `extract_receipt` for the
|
||||
// unified extract receipt-writer surface (D-EXTRACT-19 belt+suspenders).
|
||||
expect(r.stdout).toContain('Page types (25)');
|
||||
// v0.42.56.0 (#2390): extended to 27 by adding the Life Chronicle
|
||||
// `event` + `diary` temporal types (life/events/, life/diary/).
|
||||
expect(r.stdout).toContain('Page types (27)');
|
||||
expect(r.stdout).toContain('event :: temporal');
|
||||
expect(r.stdout).toContain('diary :: temporal');
|
||||
expect(r.stdout).toContain('Link verbs (12)');
|
||||
expect(r.stdout).toContain('Takes kinds: fact, take, bet, hunch');
|
||||
expect(r.stdout).toContain('person :: entity');
|
||||
|
||||
Reference in New Issue
Block a user