mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-31 04:07:52 +00:00
* feat(chronicle): register event + diary page types (#2390) Life Chronicle Phase A.1. Adds `event` (timeline atom) and `diary` (first-person interiority) as temporal-primitive page types under life/events/ and life/diary/, extractable:false — registered in ALL_PAGE_TYPES, both base schema packs, and the inferType prefix table, with parity fixtures. Also lands the chronicle read result types (ChronicleTimelineRow/ChronicleTimelineOpts/LastSeenResult) consumed by Phase A.2. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): event_page_id timeline projection + day/since/last-seen reads (#2390) Life Chronicle Phase A.2. Adds a nullable event_page_id FK to timeline_entries (migration v120; mirrored in schema.sql, pglite-schema, schema-embedded) so a type:event page projects ONE date-index row keyed to its depth page; a partial UNIQUE(event_page_id, date) makes re-extraction with a changed summary an update, not a duplicate. Dual-engine getTimelineForDate / getSince / getLastSeen filter the depth page on deleted_at, hide soft-deleted event projections at READ time (not just doctor), order by event effective_date for intra-day sequence, and honor source scope (sourceIds[] > sourceId). Ops surface as `gbrain day <date> [--week]`, `gbrain since <date> [--kind]`, `gbrain last-seen <entity>`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): auto-emit extractor — backstop + chronicle_extract job (#2390) Life Chronicle Phase A.3. A put_page backstop (gated on status==='imported', the auto-link/timeline trust gate, and the default-OFF auto_chronicle flag; diary + event pages never eligible) enqueues a chronicle_extract minion job. The job runs the extractor off the write path: deterministic when/who, an injectable LLM judge (default = chat gateway; no-op when no gateway), an all-or-nothing parse barrier (a malformed proposal writes NOTHING), then content-addressed event pages + a timeline projection via the new dual-engine upsertEventProjection (idempotent — re-run yields one event + one projection). New: src/core/chronicle/{eligibility,config,extract-events,backstop}.ts, engine.upsertEventProjection (both engines), the jobs.ts handler + a 10-min timeout. 14 unit tests (eligibility, idempotency, parse barrier, backstop gating + enqueue) green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): quick-capture for diary + manual events (#2390) Life Chronicle Phase A.5. `gbrain capture` now routes the default slug by type (diary → life/diary/, event → life/events/, else inbox/) and accepts --who/--what/--where/--kind/--depth to assemble the `event:` frontmatter block for `--type event`. User-declared event keys win per-key over the flags. Goes through the existing put_page → write-through → embed path. 6 new unit tests; existing capture tests stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): bi-temporal per-entity ontology on the facts table (#2390) Life Chronicle Phase B.10 — the feature's differentiator. Rather than a parallel store, the open-world per-entity ontology EXTENDS the existing `facts` table (eng-review G1): migration v121 adds dimension/value/value_hash /dim_status columns + a deterministic partial-UNIQUE dedup key + an asof read index. facts already supplies bi-temporal validity, supersession (superseded_by), visibility (remote redaction), confidence, provenance, and embedding — all inherited. Dual-engine methods: mergeOntologyFact (deterministic value_hash dedup → idempotent retry; same value corroborates; a different value forward-closes the prior row's valid_until + superseded_by; a BACKDATED conflicting value is recorded WITHOUT rewriting, surfaced by findOntologyConflicts), getOntology with `--asof` valid-time travel (expired_at + status + validity-window in the predicate so quarantined/superseded never leak), discoverOntologyDimensions, findOntologyConflicts. Novel/LLM-proposed dimensions quarantine; a seed lexicon canonicalizes name drift (job_role → role). 9 unit tests cover the full lifecycle; typecheck pins both engines to the interface. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): ontology ops — get/add/dimensions/contradictions (#2390) Life Chronicle Phase B.11. Exposes the bi-temporal ontology over CLI + MCP (contract-first, auto-generated): `gbrain ontology <entity> [--asof]`, `gbrain ontology-add <entity> <dim> <value>`, `gbrain ontology-dimensions` (meta-ontology rollup), `gbrain ontology-contradictions`. All reads route through sourceScopeOpts. Privacy: ontology_get redacts diary-sourced observations (source under life/diary/) for untrusted (remote) callers. 3 op tests (incl. the remote-redaction path); 47 op-registry/tool-def/description tests stay green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): agent-context loader — volunteer_chronicle (#2390) Life Chronicle Phase B.12. `loadChronicleContext` hands an agent the recent timeline (last N days) + the validity-resolved current ontology for the entities in play, in one zero-LLM payload, so it orients before acting — the exact gap behind fumbled chronology. Pure composition over getSince + getOntology (no new SQL). Exposed as the `volunteer_chronicle` read op (`gbrain orient [--days] [--entities a,b]`); diary-sourced ontology is redacted for remote callers. 2 loader tests + op-registry green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): backfill op — sweep existing meetings into events (#2390) Life Chronicle Phase A.8. `chronicle_backfill` (local-only admin op; `gbrain chronicle-backfill [--since] [--limit] [--dry-run]`) lists existing meeting/conversation/calendar pages (source-scoped via listPages), filters through the chronicle eligibility predicate, and enqueues one chronicle_extract job per eligible page so existing brains populate the timeline. --dry-run counts only; per-page enqueue failures are surfaced in `errors`, never swallowed. 2 op tests (dry-run count + enqueue). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): delight — on-this-day + narrative rendering (#2390) Life Chronicle Phase A.6 (delight). Dual-engine getOnThisDay (events from the same month-day in prior years; `gbrain on-this-day [--date]`) reusing the chronicle JOIN shape (deleted-event hiding + source scope). A pure renderTimelineNarrative turns timeline rows into prose; `gbrain day --narrative` returns it alongside the events. 5 tests. (Coverage gap-detection ships with the advisor collectors next.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): proactive advisor collector (#2390) Life Chronicle Phase A.7. A brain-state advisor collector surfaces two proactive signals in `gbrain advisor`: unresolved ontology conflicts (warn, → `gbrain ontology-contradictions`) and recent meetings with no timeline coverage (info, → `gbrain chronicle-backfill`). Advisory-only (no dispatch_id); runs over MCP too (not workspace-dependent); tolerant of pre-migration brains. 3 tests + advisor-op-gate green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): doctor chronicle_projection_health probe (#2390) Life Chronicle Phase B.13. An always-run doctor check (keyed off the event_page_id schema, NOT a migration verify-hook) counts timeline projections whose event page is soft-deleted — hidden at read time, surfaced here as a cleanup backlog (`gbrain integrity auto`). Tolerant of pre-migration brains. 1 detection test. (auto_chronicle / chronicle.tz flags already work via getConfig defaults; their docs land with document-release at ship.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): E1 temporal recall — chronicle-type boost on temporal queries (#2390) Life Chronicle Phase A.4 (E1, ambient temporality). Rather than a separate RRF arm (which needs chunk hydration + risks the fusion path), E1 is a bounded post-fusion boost: applyChronicleTypeBoost lifts `event`/`diary` results on temporal queries, wired INSIDE runPostFusionStages' `recency !== 'off'` branch so it fires ONLY on temporal intent — non-temporal search is bit-for-bit unchanged (proven by 110 passing search-path tests). Bounded ([1.0,1.25]) + floor-gated like the other metadata stages; attribution via `chronicle_boost`. 3 unit tests. (Empirical precision/negative measurement lands with the eval.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(chronicle): feature eval — gbrain eval chronicle (PRIMARY proof) (#2390) Life Chronicle Phase A.9, the North-Star proof. A deterministic, CI-safe eval (brings its own in-memory PGLite; no LLM, no gateway) builds a synthetic month corpus with a known gold chronology + a planted ontology supersession + a planted conflict, then scores the chronicle layer on six gold tasks: day reconstruction (intra-day order), last-seen exact date, ontology supersession, --asof valid-time travel, contradiction surfacing, and source isolation. `gbrain eval chronicle [--json]` exits 0 iff all pass — currently 6/6. The OFF baseline (raw meeting pages) structurally can't order intra-day events or time-travel ontology; the ON path does. (The live-LLM OFF-vs-ON agent arm + LongMemEval temporal slice are a follow-up; this deterministic bar gates CI.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(chronicle): pre-landing review — conflict validity, parse-barrier date, remote conflict redaction (#2390) Three bugs caught by the codex pre-landing review on the diff: 1. findOntologyConflicts ignored valid_until, so a normal forward supersession (founder→advisor from two sources) falsely reported as a live conflict. Now restricted to currently-open rows (valid_until IS NULL) in both engines. 2. The extractor parse barrier accepted any when-string >= 4 chars; a non-date value slipped past, wrote the event page, then threw on the projection's ::date cast (partial write). isValidProposal now requires a real parseable date. 3. The ontology_conflicts op had no remote diary redaction (ontology_get did); remote callers now get diary-sourced values filtered, and conflicts that lose their disagreement after redaction are dropped. Three regression tests added; 29 chronicle tests + eval 6/6 green; typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.56.0 feat(chronicle): Life Chronicle — temporal timeline + diary + bi-temporal ontology (#2390) Bump VERSION + package.json to 0.42.56.0 and add the CHANGELOG entry for the Life Chronicle feature (#2390, closes duplicate #2388). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(chronicle): register #2390 surfaces with the five CI guard suites (#2390) CI caught five guard tests that pin registration invariants my diff tripped: - schema-bootstrap-coverage: the four v122 facts ontology columns join COLUMN_EXEMPTIONS (facts is migration-created; the partial indexes live inside v122; every reader filters dimension IS NOT NULL — same precedent as facts.claim_metric et al). - no-valid-until-write (R8): both engines' mergeOntologyFact forward- supersession is a deliberate, documented valid_until write authority (engine-layer, dimension IS NOT NULL only; the contradiction probe still never mutates). - doctor-categories: chronicle_projection_health registered under BRAIN_CHECK_NAMES (same class as child_table_orphans). - checkTypeProliferation: the test is now threshold-relative (computes declared from the active pack, seeds declared+6) so base-pack growth can't silently move the fixed threshold again. - schema-cli: gbrain-base page-type count 25 → 27 (event + diary), with assertions on both new types. All 41 guard tests green; typecheck clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: file Life Chronicle follow-up TODOs (v0.42.56.0, #2390) Eight deferred items from the CEO/eng review decisions (auto-emit default-flip fast-follow, live eval arm + LongMemEval slice, passive diary consent, interval-splitting, federation, place-as-entity, meta-ontology dashboard, materialized daily pages), each with decision provenance. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(architecture): KEY_FILES entry for the Life Chronicle module (#2390) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
169 lines
7.5 KiB
TypeScript
169 lines
7.5 KiB
TypeScript
/**
|
|
* v0.35.4 (R1 + R8 — D-CDX-4 + D-CDX-7) — IRON-RULE: the contradiction
|
|
* probe NEVER writes `valid_until` on the facts table.
|
|
*
|
|
* The temporal trajectory wave (v0.35.4) gives `consolidate` the
|
|
* authority to write `valid_until` on chronologically-superseded facts.
|
|
* That authority is exclusive: the contradiction probe surfaces
|
|
* `temporal_supersession` verdicts via paste-ready commands, but it
|
|
* must NEVER auto-mutate. This preserves the
|
|
* `src/core/eval-contradictions/auto-supersession.ts:4` invariant.
|
|
*
|
|
* Two layered guards:
|
|
* R1 — grep guard over the entire `src/core/eval-contradictions/`
|
|
* subtree and the `src/commands/eval-suspected-contradictions*.ts`
|
|
* files: no code path may UPDATE facts.valid_until.
|
|
* R8 — broader guard over all of `src/`: the only file that writes
|
|
* valid_until is `src/core/cycle/phases/consolidate.ts`. Any new
|
|
* write site fails this guard; the human adding it must explicitly
|
|
* amend the allow-list AND document the deliberate design change.
|
|
*/
|
|
|
|
import { test, expect, describe } from 'bun:test';
|
|
import { readdirSync, readFileSync, statSync } from 'node:fs';
|
|
import { join } from 'node:path';
|
|
|
|
// Allow-listed files that legitimately write `valid_until`. Adding a new
|
|
// file here means you've thought carefully about the
|
|
// auto-supersession.ts:4 invariant and decided the new write site
|
|
// preserves it.
|
|
//
|
|
// - consolidate.ts (v0.35.4 — chronological writeback)
|
|
// - facts/forget.ts (v0.32.2 — user-initiated `gbrain forget`; user is
|
|
// the supersession authority, not the probe)
|
|
// - postgres-engine.ts + pglite-engine.ts (v0.42.56.0, #2390 — Life
|
|
// Chronicle ontology: `mergeOntologyFact` forward-supersession closes
|
|
// the prior OPEN row's valid_until when a NEW value arrives for the
|
|
// same (entity, dimension). Engine-layer, caller-requested, scoped to
|
|
// `dimension IS NOT NULL` ontology rows only — plain facts untouched,
|
|
// and the contradiction probe still never mutates, so the
|
|
// auto-supersession.ts:4 invariant is preserved. Deliberate design
|
|
// change per the #2390 eng review (G1: ontology extends facts).
|
|
const VALID_UNTIL_WRITE_ALLOWLIST: ReadonlySet<string> = new Set([
|
|
'src/core/cycle/phases/consolidate.ts',
|
|
'src/core/facts/forget.ts',
|
|
'src/core/postgres-engine.ts',
|
|
'src/core/pglite-engine.ts',
|
|
]);
|
|
|
|
function walkTs(dir: string, acc: string[] = []): string[] {
|
|
for (const name of readdirSync(dir)) {
|
|
const full = join(dir, name);
|
|
const st = statSync(full);
|
|
if (st.isDirectory()) {
|
|
// Skip generated / vendored / test directories.
|
|
if (name === 'node_modules' || name === '.git' || name === 'dist') continue;
|
|
walkTs(full, acc);
|
|
} else if (name.endsWith('.ts') && !name.endsWith('.test.ts')) {
|
|
acc.push(full);
|
|
}
|
|
}
|
|
return acc;
|
|
}
|
|
|
|
/**
|
|
* Detect lines that look like they are UPDATEing facts.valid_until.
|
|
* Permissive on SQL formatting (the same UPDATE can be split across lines,
|
|
* use template strings, or use parameterized SQL via postgres.js's
|
|
* tagged-template syntax). We look for the pair of substrings near
|
|
* each other in the same file: `UPDATE facts` and `valid_until`.
|
|
*
|
|
* Tolerated: a reference to `valid_until` in a SELECT projection list
|
|
* (which is fine — reading the column is not writing it) is filtered out
|
|
* by also requiring a write verb (SET) nearby OR an INSERT INTO facts
|
|
* with valid_until in the column list (INSERT path is OK from
|
|
* src/core/postgres-engine.ts + src/core/pglite-engine.ts because those
|
|
* are the engine layer, intentionally writing on caller request via
|
|
* insertFact/insertFacts; the issue is whether the contradiction probe
|
|
* path triggers those writes).
|
|
*/
|
|
function findValidUntilWrites(source: string): string[] {
|
|
// Quick-fail: no mention of valid_until at all.
|
|
if (!source.includes('valid_until')) return [];
|
|
const lines = source.split('\n');
|
|
const hits: string[] = [];
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i];
|
|
// Detect actual SQL writes. The narrow pattern `UPDATE facts SET ...
|
|
// valid_until` is unambiguous — UPDATE is a SQL verb, not text
|
|
// anyone writes in a description string. Tolerates same-line and
|
|
// multi-line UPDATEs (look at the next 4 lines after `UPDATE facts`
|
|
// for `valid_until`).
|
|
if (/\bUPDATE\s+facts\b/i.test(line)) {
|
|
const window = lines.slice(i, i + 5).join('\n');
|
|
if (/\bSET\b[\s\S]*\bvalid_until\b/i.test(window)) {
|
|
hits.push(`${i + 1}: ${line.trim()}`);
|
|
}
|
|
}
|
|
}
|
|
return hits;
|
|
}
|
|
|
|
describe('R1 — contradiction probe never writes valid_until', () => {
|
|
test('no file under src/core/eval-contradictions/ writes facts.valid_until', () => {
|
|
const dir = 'src/core/eval-contradictions';
|
|
const files = walkTs(dir);
|
|
expect(files.length).toBeGreaterThan(0);
|
|
for (const f of files) {
|
|
const src = readFileSync(f, 'utf-8');
|
|
const hits = findValidUntilWrites(src);
|
|
expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]);
|
|
}
|
|
});
|
|
|
|
test('no src/commands/eval-suspected-contradictions* file writes facts.valid_until', () => {
|
|
const dir = 'src/commands';
|
|
const files = readdirSync(dir)
|
|
.filter(n => n.startsWith('eval-suspected-contradictions') && n.endsWith('.ts'))
|
|
.map(n => join(dir, n));
|
|
expect(files.length).toBeGreaterThanOrEqual(1);
|
|
for (const f of files) {
|
|
const src = readFileSync(f, 'utf-8');
|
|
const hits = findValidUntilWrites(src);
|
|
expect(hits, `${f} contains valid_until write: ${hits.join(' | ')}`).toEqual([]);
|
|
}
|
|
});
|
|
});
|
|
|
|
describe('R8 — only the consolidate phase + engine insert layer may write valid_until', () => {
|
|
test('every src/ TypeScript file that writes valid_until is on the allow-list', () => {
|
|
const files = walkTs('src');
|
|
const offenders: Array<{ file: string; hits: string[] }> = [];
|
|
|
|
for (const f of files) {
|
|
// Normalize path separator for cross-platform matching of the
|
|
// allow-list keys.
|
|
const relForCheck = f.replace(/\\/g, '/');
|
|
if (VALID_UNTIL_WRITE_ALLOWLIST.has(relForCheck)) continue;
|
|
|
|
// Engine implementation files (postgres-engine.ts, pglite-engine.ts)
|
|
// legitimately write valid_until inside insertFact/insertFacts —
|
|
// those are caller-driven INSERTs. Pattern is `INSERT INTO facts (
|
|
// ... valid_until ...) VALUES`. Detect and exempt that pattern.
|
|
// The R8 guard is about UPDATE; INSERT carrying valid_until as a
|
|
// column value is not the failure mode auto-supersession.ts:4 cares
|
|
// about.
|
|
const src = readFileSync(f, 'utf-8');
|
|
const hits = findValidUntilWrites(src);
|
|
if (hits.length > 0) offenders.push({ file: relForCheck, hits });
|
|
}
|
|
|
|
expect(
|
|
offenders,
|
|
`Unexpected valid_until UPDATE sites:\n` +
|
|
offenders.map(o => ` ${o.file}:\n ${o.hits.join('\n ')}`).join('\n') +
|
|
`\n\nIf you added a deliberate write, append the path to ` +
|
|
`VALID_UNTIL_WRITE_ALLOWLIST in this test AND review the ` +
|
|
`\`auto-supersession.ts:4\` invariant first.`,
|
|
).toEqual([]);
|
|
});
|
|
|
|
test('VALID_UNTIL_WRITE_ALLOWLIST is non-empty (consolidate is on it)', () => {
|
|
// Self-check: if the test file ever ships with an empty allow-list,
|
|
// the R8 guard collapses to "no code writes valid_until anywhere" and
|
|
// becomes a tautology. Keep the consolidate phase on the list as the
|
|
// explicit positive control.
|
|
expect(VALID_UNTIL_WRITE_ALLOWLIST.has('src/core/cycle/phases/consolidate.ts')).toBe(true);
|
|
});
|
|
});
|