mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
docs: KEY_FILES + push-context guide + TODOS for the #2084/#2095 wave
- docs/architecture/KEY_FILES.md (current-state): context entries gain the window extractor, arm provenance/confidence, suppression modes, volunteer + volunteer-events modules; background-work entry now lists FIVE sinks and the drainThenDisconnect owner-disconnect contract; new entries for src/core/cli-force-exit.ts (the exit contract) and src/commands/watch.ts. - docs/guides/push-context.md (new): the three channels (reflex/op/watch), the confidence model, CLI usage, config keys, and the approximate-stats caveat. Linked from CLAUDE.md's reference map. - CLAUDE.md: ops line mentions volunteer_context + the guide link; bun run build:llms regenerated in the same commit (freshness test green). - TODOS.md: #2095 deferrals filed (SSE push channel, policy skill + doctor check, structured messages[] param); the #1981 entity-detection TODO narrowed (window extraction covered assistant-introduced entities + named-antecedent follow-ups); the drain-hoist P3 marked DONE by the #2084 wave. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
parent
f77740599e
commit
6f9d8d4241
@@ -38,7 +38,7 @@ mount, CEO-class with multiple team brains) and
|
||||
|
||||
## Architecture
|
||||
|
||||
Contract-first: `src/core/operations.ts` defines ~47 shared operations (v0.29 adds `get_recent_salience`, `find_anomalies`, `get_recent_transcripts`). CLI and MCP
|
||||
Contract-first: `src/core/operations.ts` defines ~47 shared operations (v0.29 adds `get_recent_salience`, `find_anomalies`, `get_recent_transcripts`; v0.43 adds `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
|
||||
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
|
||||
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
|
||||
markdown files (tool-agnostic, work with both CLI and plugin contexts).
|
||||
@@ -97,6 +97,7 @@ detail on demand.)
|
||||
| any file in `src/` (what it does + its invariants) | `docs/architecture/KEY_FILES.md` — find the file's entry |
|
||||
| search / ranking / hybrid / retrieval | `docs/architecture/RETRIEVAL.md` + the `search/*` entries in `KEY_FILES.md` |
|
||||
| search modes / cost knobs | `docs/guides/search-modes.md` |
|
||||
| push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` |
|
||||
| schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` |
|
||||
| thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` |
|
||||
| the CLI surface (commands + flags) | `gbrain --help` / `gbrain --tools-json`, plus the relevant `KEY_FILES.md` entry |
|
||||
|
||||
@@ -1,16 +1,42 @@
|
||||
# TODOS
|
||||
|
||||
## gbrain#2095 push-based context follow-ups (v0.43+)
|
||||
|
||||
Filed from the #2095 wave (volunteer_context op + reflex window + `gbrain watch`).
|
||||
Deliberately scoped OUT of v1 per the eng-review scope decision (success criteria
|
||||
are the bar). Plan + GSTACK REVIEW REPORT at
|
||||
`~/.claude/plans/system-instruction-you-are-working-cheerful-elephant.md`.
|
||||
|
||||
- [ ] **P3 — SSE/HTTP push channel via serve-http.** The op + `gbrain watch` cover
|
||||
pull-per-turn and stdin streaming; a serve-http SSE feed would push volunteered
|
||||
pages to remote agents without a local CLI. **Why:** thin-client/remote-MCP
|
||||
deployments get push too. **Cons:** async plumbing + auth scoping; no consumer
|
||||
wired today. **Where:** `src/commands/serve-http.ts` + `src/core/context/volunteer.ts`.
|
||||
**Blocked by:** a real consumer (revisit when one exists).
|
||||
- [ ] **P3 — policy skill + doctor check for push-context.** The ambient reflex
|
||||
needed doctor visibility because silent failure was invisible; volunteer is
|
||||
invoked-on-demand so v1 skipped it. If `volunteer-context --stats` adoption shows
|
||||
agents not discovering the surface, ship a `push-context` recipe (mirror
|
||||
`recipes/retrieval-reflex/`) + a doctor check reading the events table.
|
||||
**Where:** `recipes/`, `src/commands/doctor.ts`.
|
||||
- [ ] **P3 — structured `messages[]` param for volunteer_context.** v1 takes a
|
||||
string window (`user:`/`assistant:` prefixes) to avoid a dual-shape contract.
|
||||
If MCP callers accumulate parsing bugs, add a structured array param beside it.
|
||||
**Where:** `src/core/operations.ts:volunteer_context` + `src/core/context/volunteer.ts:parseWindow`.
|
||||
|
||||
## gbrain#1981 Retrieval Reflex follow-ups (v0.43+)
|
||||
|
||||
Filed from the #1981 ship (v0.42.39.0). Deliberately scoped OUT — the v1 extractor
|
||||
is deterministic + precision-biased. See plan + GSTACK REVIEW REPORT at
|
||||
`~/.claude/plans/system-instruction-you-are-working-wild-yeti.md`.
|
||||
|
||||
- [ ] **P3 — broaden entity detection beyond proper-case ASCII.** The v1 extractor
|
||||
(`src/core/context/entity-salience.ts`) misses lowercase names, many non-Latin
|
||||
scripts, pronoun follow-ups ("what about her?"), and assistant-introduced entities.
|
||||
These need conversation state or an LLM pass. **Why:** higher recall on the read
|
||||
side. **Where:** `entity-salience.ts` + the orchestrator's `priorContextText`.
|
||||
- [ ] **P3 — broaden entity detection beyond proper-case ASCII.** The extractor
|
||||
(`src/core/context/entity-salience.ts`) misses lowercase names and many non-Latin
|
||||
scripts; these need an LLM pass or script-aware heuristics. **Why:** higher recall
|
||||
on the read side. **Where:** `entity-salience.ts`. *(Partially done by the #2095
|
||||
wave: `extractCandidatesFromWindow` now covers assistant-introduced entities and
|
||||
pronoun follow-ups whose antecedent was NAMED in the rolling window; true pronoun
|
||||
coreference for never-named antecedents remains with the LLM-pass idea.)*
|
||||
- [ ] **P3 — recall knob: optional fuzzy/prefix-expansion resolution.** The resolver
|
||||
(`src/core/context/retrieval-reflex.ts`) is exact-only (alias + title + slug-suffix)
|
||||
for precision. Revisit adding `resolveEntitySlug`'s trgm-fuzzy / prefix-expansion
|
||||
@@ -213,17 +239,12 @@ GSTACK REVIEW REPORT at
|
||||
can't desync per-engine, bounded against CLI-hang by a top-level forced
|
||||
cleanup. Do this BEFORE introducing any concurrent module-engine connect path.
|
||||
|
||||
- [ ] **P3 — `dream` + CLI_ONLY fall-through paths don't drain the facts /
|
||||
last-retrieved queues before the owner disconnect.** The op-dispatch path
|
||||
(`cli.ts:~282-314`) drains `getFactsQueue().drainPending()` +
|
||||
`awaitPendingLastRetrievedWrites()` before `engine.disconnect()`; the `dream`
|
||||
owner-disconnect (`cli.ts:~1164`) and the fall-through owner-disconnect
|
||||
(`cli.ts:~1785`) do not. If the dream cycle ever enqueues a facts:absorb /
|
||||
last-retrieved write that's still in flight at disconnect, the owner nulls the
|
||||
singleton and the write throws "No database connection". Pre-existing (not
|
||||
introduced by the #1471 ownership fix), surfaced by the Claude adversarial
|
||||
review (F5). Fix: hoist the same drain-before-disconnect block the op-dispatch
|
||||
path uses into a shared helper and call it on all three owner-disconnect sites.
|
||||
- [x] **P3 — `dream` + CLI_ONLY fall-through paths don't drain the facts /
|
||||
last-retrieved queues before the owner disconnect.** DONE in the #2084 wave:
|
||||
`drainThenDisconnect(engine)` in `src/cli.ts` is the shared helper, applied at
|
||||
ALL eight owner-disconnect sites (op-dispatch, CLI_ONLY fall-through, search
|
||||
dashboard, doctor remediation x3, ze-switch, dream, read-only-timeout path),
|
||||
with its own bounded hard-deadline. Pinned by `test/cli-drain-then-disconnect.test.ts`.
|
||||
## v0.42.x AI SDK v6 tool-schema fix follow-ups (#1782/#1764)
|
||||
|
||||
Surfaced by the codex outside-voice pass during `/plan-eng-review` and
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,65 @@
|
||||
# Push-based context (#2095, v0.43)
|
||||
|
||||
Retrieval used to be pull-only: the agent had to *know to ask* before the brain
|
||||
contributed anything. Push-based context inverts that — the brain volunteers
|
||||
relevant pages from the recent conversation, confidence-gated so push noise
|
||||
never becomes worse than pull silence.
|
||||
|
||||
Three channels share one zero-LLM core (`src/core/context/volunteer.ts`):
|
||||
|
||||
| Channel | Surface | When to use |
|
||||
|---|---|---|
|
||||
| `reflex` | automatic, inside the context engine | default-on for plugin hosts; nothing to call |
|
||||
| `op` | `gbrain volunteer-context` / MCP `volunteer_context` | agents without the plugin; one call per turn |
|
||||
| `watch` | `gbrain watch` | stream a transcript in, volunteered pages stream out |
|
||||
|
||||
## How it decides
|
||||
|
||||
1. **Extract** entities across the last N turns (capitalized runs, `@handles`),
|
||||
merged with recency / frequency / user-role salience. Assistant-introduced
|
||||
entities and "what did she invest in?" follow-ups whose antecedent was named
|
||||
in the window now resolve.
|
||||
2. **Resolve** through the alias table, exact titles, and slug suffixes — each
|
||||
arm carries an honest confidence: alias 0.9, exact title 0.8, slug-suffix 0.6,
|
||||
+0.05 when mentioned in ≥2 turns or the newest turn.
|
||||
3. **Gate** at `min_confidence` (default 0.7 — slug-suffix matches need an
|
||||
explicit lower gate), suppress pages already surfaced (slug-presence only),
|
||||
cap at 3 pages (hard cap 5).
|
||||
|
||||
## CLI
|
||||
|
||||
```bash
|
||||
# one-shot: pipe recent turns (oldest → newest)
|
||||
printf 'user: ask alice-example about the deal\nassistant: noted\nuser: what did she say?\n' \
|
||||
| gbrain volunteer-context
|
||||
|
||||
# streaming: volunteered pages print as the transcript flows
|
||||
some-transcript-feed | gbrain watch --json
|
||||
|
||||
# the feedback loop: how often were volunteered pages actually opened?
|
||||
gbrain volunteer-context --stats
|
||||
```
|
||||
|
||||
Stats are **approximate** by design: "used" means `pages.last_retrieved_at >
|
||||
volunteered_at` — the 5-minute last-retrieved throttle causes false negatives
|
||||
and unrelated reads of the same page cause false positives. Use the per-arm
|
||||
precision to tune `min_confidence`, not as an exact metric.
|
||||
|
||||
## Config
|
||||
|
||||
| Key | Default | What it does |
|
||||
|---|---|---|
|
||||
| `retrieval_reflex_window_turns` | 4 | turns the ambient reflex extracts from; 1 = legacy current-turn-only (file/env plane: `GBRAIN_RETRIEVAL_REFLEX_WINDOW_TURNS`) |
|
||||
| `retrieval_reflex` | true | the ambient channel's master switch |
|
||||
| `retrieval_reflex_max_pointers` | 3 | pointer cap per turn |
|
||||
|
||||
Per-call knobs on the op/watch: `max_pages`, `min_confidence`, `session_id`,
|
||||
`turn` (attribution columns in the feedback log).
|
||||
|
||||
## Storage + privacy
|
||||
|
||||
Volunteered pages log to `context_volunteer_events` (migration v116): slug,
|
||||
arm, confidence, channel, optional session/turn — the rationale is a
|
||||
deterministic template string, never raw conversation text. Rows are pruned
|
||||
after 90 days by the dream cycle's purge phase. Synopses pass through the same
|
||||
takes/facts-fence privacy boundary as `get_page`.
|
||||
+2
-1
@@ -186,7 +186,7 @@ mount, CEO-class with multiple team brains) and
|
||||
|
||||
## Architecture
|
||||
|
||||
Contract-first: `src/core/operations.ts` defines ~47 shared operations (v0.29 adds `get_recent_salience`, `find_anomalies`, `get_recent_transcripts`). CLI and MCP
|
||||
Contract-first: `src/core/operations.ts` defines ~47 shared operations (v0.29 adds `get_recent_salience`, `find_anomalies`, `get_recent_transcripts`; v0.43 adds `volunteer_context` — push-based context, see `docs/guides/push-context.md`). CLI and MCP
|
||||
server are both generated from this single source. Engine factory (`src/core/engine-factory.ts`)
|
||||
dynamically imports the configured engine (`'pglite'` or `'postgres'`). Skills are fat
|
||||
markdown files (tool-agnostic, work with both CLI and plugin contexts).
|
||||
@@ -245,6 +245,7 @@ detail on demand.)
|
||||
| any file in `src/` (what it does + its invariants) | `docs/architecture/KEY_FILES.md` — find the file's entry |
|
||||
| search / ranking / hybrid / retrieval | `docs/architecture/RETRIEVAL.md` + the `search/*` entries in `KEY_FILES.md` |
|
||||
| search modes / cost knobs | `docs/guides/search-modes.md` |
|
||||
| push-based context (volunteer/watch/reflex window) | `docs/guides/push-context.md` |
|
||||
| schema packs / page types / extraction | `docs/architecture/schema-packs.md`, `type-taxonomy.md`, `lens-packs.md` |
|
||||
| thin-client / remote MCP / cross-modal | `docs/architecture/thin-client.md` |
|
||||
| the CLI surface (commands + flags) | `gbrain --help` / `gbrain --tools-json`, plus the relevant `KEY_FILES.md` entry |
|
||||
|
||||
Reference in New Issue
Block a user